HK1059133B

HK1059133B - Musical composition recognition method and system

Info

Publication number: HK1059133B
Application number: HK04101878.1A
Authority: HK
Inventors: 伊能美和子; 小林竜浩
Original assignee: Rakuten, Inc.
Priority date: 2000-10-23
Filing date: 2001-10-22
Publication date: 2009-07-10

Description

Music recognition method and system

Technical Field

The present invention relates to a music recognition method and system and a storage medium storing a music recognition program, and more particularly, to a music recognition method and system for recognizing and storing music broadcasted in image or voice information broadcasted in TV (television) or FM (frequency modulation broadcast) in real time, and a storage medium storing a music recognition program.

The present invention also relates to a CM (Commercial advertisement) recognition method and system, and a storage medium storing a CM recognition program, and more particularly, to a CM recognition method and system for recognizing and storing a CM of a broadcast such as a television or an FM in real time, and a storage medium storing a CM recognition program.

Background

Conventionally, there is no system that recognizes in real time a song title used in contents (contents) such as image information and music information broadcast in real time and stores the recognized song title in a storage device.

Further, there is no device for recognizing and storing the CM broadcasted in real time. And a system for recognizing CM data by comparing only CM information divided by a given length is more absent.

As described above, the conventional system does not have a technique for monitoring a music to be played, and cannot store a music name in a storage device together with play time information. When time information is added to a certain music title, it is not possible to add the name of the music being broadcast and the time information at which the music is broadcast in real time, only by manual processing.

Similarly, there is no device for recognizing and storing the CM broadcasted in the image or voice information to be played in real time, and there is no method other than manual processing, so that there is a limit in real time performance and scale-up.

Disclosure of Invention

In view of the above circumstances, an object of the present invention is to provide a playlist generation technique capable of recognizing and storing a musical tune name of a musical tune broadcasted in an image or voice information broadcasted by a television, an FM, or the like in real time.

It is another object of the present invention to provide a play list generation technique that can compare and recognize not only radio broadcasts but also broadcast data transmitted via a communication network such as the internet, for example, by a dump download method, and also target music data in real time without flag information or transparent information.

It is another object of the present invention to provide a CM recognition technique that can recognize and store a CM broadcasted in an image or voice information broadcast by a television, a TM, or the like in real time.

In order to achieve the above purpose, the present invention may adopt the following technical solutions.

The invention relates to a method for generating a play list, which generates a play list of contents from received data, wherein the method comprises the following steps:

extracting the content characteristics in advance, storing the content characteristics in a content characteristic file, and storing the content-related information in a content information database;

extracting the characteristics of the received data and storing the characteristics in a data characteristic file;

searching for contents by comparing the contents profile data with the data profile data;

if the content feature file data is consistent with the data feature file data, storing the data corresponding to the data as a retrieval result file;

and generating a broadcast list of the content from the search result file and the content information database.

According to the present invention, a time-series playlist can be automatically generated from broadcast data or the like to be played. The data profile is, for example, a broadcast profile.

In the above technical solution, the method may further include the steps of: when any data in the data profile is not included in the content profile in the search step, the data is registered in the content profile.

According to the present invention, data can be automatically registered with a content profile.

In the above technical solution, the method may further include the steps of:

enabling the contents corresponding to the data not contained in the content feature file to be viewed by people;

the content-related information is registered in the content information database.

According to the present invention, information related to data of a content profile that has already been registered can be automatically registered, and a time-series play list can be automatically generated using the registered information.

In the above technical solution, the method may further include the steps of:

if a certain data is not contained in the content profile in the data profile in the searching step, making the content corresponding to the data available for viewing by a human;

the contents are added to the play list together with the contents-related information.

According to the invention, the time-series playlist can be supplemented, and a higher-precision playlist can be generated.

In order to achieve the above object, the present invention may also be embodied as follows.

The invention is a music recognizing method for recognizing music from received data, comprising the steps of:

extracting characteristics of the content of the music in advance and storing the characteristics in a content characteristic file;

extracting the characteristics of the received data and storing the extracted characteristics in a broadcast characteristic file;

searching for music by comparing the data of the content profile and the data of the broadcast profile;

if the music name is known, the music name is taken as a retrieval result file to be stored;

a time-series playlist of musical compositions is generated from the search result file.

According to the present invention, a musical-piece-related time-series playlist can be automatically generated.

In the above technical solution, the method may further include the steps of:

determining whether the received data is music;

if the music is music, storing the item indicating that the data is music and the data receiving time in a music extraction file;

storing data in a music name unextracted file when the music name of certain data in the broadcast feature file is not known in the music searching step;

and generating a music undetected file from the broadcast profile file, the music extracted file, and the music name unextracted file.

According to the present invention, even though the music is a music, the user can grasp the music without being described in the time-series playlist.

In the above technical solution, the method may further include the steps of:

the music recorded in the file without the music detected can be heard by people;

the title and time of the music recorded in the file from which the music was not detected are added to the playlist.

In the above-described aspect, the time-series broadcast list may be generated using the search result file and a content information database having information associated with the same music title.

The time-series playlist may include a time of day, a name of a musical composition broadcast at the time of day, and information associated with the name of the musical composition.

According to the present invention, a time-series playlist having a music title and various pieces of information related to the music title can be automatically generated.

In the above-described aspect, the broadcast data may be received in a plurality of regions, and the data received in each region may be transmitted to the center system, and the time-series play list may be generated by the center device by the music recognition method.

According to the present invention, time-series broadcast lists relating to contents broadcast by each local broadcast station can be generated.

Further, the present invention may be a music recognizing method for recognizing a music from received data, including the steps of:

receiving broadcast data in a plurality of regions;

extracting the characteristics of the received data in each region and transmitting the extracted characteristics as broadcast characteristic file data to a central system;

searching, at the center system, music by referring to the data of the content profile and the broadcast profile;

According to the present invention, since the broadcast profile can be generated in each place and transmitted to the center, the amount of transmission to the center can be reduced.

In the above technical solution, the following features can also be provided:

the content information database and the information related to the music title include information related to CM;

the CM related information in the content information database is registered in the content information database in advance by a CM recognition method;

the CM recognition method includes the steps of:

detecting CM data from the received data;

extracting the CM data characteristics and storing the extracted CM data characteristics in a broadcast characteristic file;

comparing the broadcast feature file with data of a main CM content feature file in which CM content features are prestored;

when the data in the broadcast profile does not exist in the master CM content profile, the data is registered as a new CM in the master CM content profile included in the content information database.

According to the present invention, a time-series play-out ticket containing CM information can be generated.

The present invention is a CM recognition method for recognizing a CM from received data and storing the recognized CM data, including the steps of:

detecting CM data from the received data;

when the data in the broadcast profile does not exist in the master CM content profile, the data is registered as a new CM in the master CM content profile.

In this way, the CM monitoring job by the human can be automated, and CM data that can be used to generate a time-series play list can be provided.

In the above technical solution, the step of detecting the CM data from the received data may include a step of detecting a start point and an end point of the CM data.

When extracting the CM data feature, a certain length portion, which is a portion cut by extending equally from the center of the start point and the end point of the detected CM data to the start point and the end point, may be characterized.

According to the present invention, a CM data input error can be absorbed.

In the above technical solution, the method may further include the steps of:

displaying the CM data which is not in the main CM content feature file according to the data comparison result;

the CM data-related information is registered in each database of a CM management database group including a master CM content profile.

According to the present invention, it is possible to register CM data-related information that has been automatically registered in the master CM content profile, with which it is possible to generate a time-series play-out ticket having CM-related information.

Other objects, features, functions and advantages of the present invention will become apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.

Drawings

Fig. 1 is a schematic diagram of the principle of the present invention.

Fig. 2 is a schematic structural view of the present invention.

Fig. 3 is a block diagram of the music recognition system of the present invention.

Fig. 4 is a flow chart showing the outline of the operation of the music recognition system of the present invention.

Fig. 5 is a block diagram of the CM recognition system of the present invention.

Fig. 6 is (a) a flowchart of an outline operation of the CM recognition system of the present invention.

Fig. 7 is a flowchart of the outline operation of the CM recognition system according to the present invention (second embodiment).

Fig. 8 is a block diagram of a music recognition system according to example 1 of the present invention.

Fig. 9 is a flowchart showing the operation of the music recognition system according to example 1 of the present invention.

Fig. 10 is a schematic view of an application example of embodiment 1 of the present invention.

Fig. 11 is a schematic diagram showing the respective files and their relationship used when creating a time-series playlist.

Fig. 12 is a diagram of time-series playlist attribute information.

Fig. 13 is a diagram illustrating time-series playlist attribute information.

Fig. 14 is a block diagram of a CM recognition system according to specific example 2 of the present invention.

Fig. 15 is a schematic diagram of a clipping method for CM feature data.

FIG. 16 is a system configuration diagram showing a specific example 3 of the present invention.

Fig. 17 is a flowchart showing an outline of the operation of the system shown in fig. 16.

Fig. 18 is a detailed operation flowchart of the system according to example 3 of the present invention.

Fig. 19 is a schematic diagram of a music data storage method.

FIG. 20 is a diagram illustrating a CM data storage method.

Fig. 21 is a schematic diagram of a process of generating a content profile from a music CM unextracted file.

Detailed Description

First embodiment

Fig. 1 and 2 show the principle of the present embodiment.

As shown in fig. 1, according to the present embodiment, the content (contents) characteristics are extracted in advance and stored in the content characteristic file (step 1); extracting the received data characteristics and storing the extracted data characteristics in a broadcast characteristic file (step 2); retrieving a musical composition by comparing the data of the content profile and the broadcast profile (step 3); if the music is known, the information about the music is saved as a search result file (step 4); a time-series playlist of musical compositions is generated from the search result file (step 5).

As shown in fig. 2, the principle structure of the music recognition system according to the present embodiment includes: a content generation unit 300 for generating content in advance, extracting music content characteristics, and storing the extracted content characteristics in a content characteristic file 320; and a feature extraction unit 105 that extracts the received data features and saves them in the broadcast feature file 140; and a music retrieval unit 120 that retrieves music by referring to the data of the content profile 320 and the broadcast profile 140 and saves the retrieval result in the retrieval result file 150; and a playlist generating unit 200 that generates a time-series playlist of time-series musical compositions from the retrieval result file 150.

Fig. 3 shows an outline of the music recognition system of the present invention. The system shown in this figure comprises: a broadcast capture search device 100, a music confirmation registration device 200, and a content creation device 300. These devices are all constituted by a PC (personal computer) or the like. Note that the present system can be realized by one PC having the functions of all these devices, but is divided into 3 devices in consideration of the processing load of the PC, the cost performance ratio, the current hardware specification, and the like.

The play capture search device 100 includes: a broadcast capturing unit 110, a search unit 120, a music extraction file 130, a broadcast feature file 140, a search result file 150, a music name non-extraction file 160, a music non-detection file 170, and a time data providing unit 180.

The broadcast capturing section 110 continuously monitors broadcast data for broadcast by a television station or a radio station, and determines whether the monitored broadcast data is a music piece or a non-music piece; if it is a musical composition, an event indicating that the musical composition is broadcast during the monitoring time is recorded in the musical composition extraction file 130, and broadcast data characteristics are extracted from the musical composition at intervals of 7.5 seconds in real time and stored in the broadcast characteristics file 140. The music pieces are stored as digital data in files not shown in the figure at 7.5 second intervals. Here, the broadcast capturing unit 110 performs the above-described processing based on the time information from the time data providing unit 180. In addition, a time stamp is given to the captured information at the time of recording.

Here, the reason why the interval of 7.5 seconds is used is that generally when it is desired to know the music inserted in the CM, the CM playing time is currently 15 seconds at the shortest, and by taking half of the time as the search time, it is possible to reliably search for music data.

The judgment of whether the music piece or the non-music piece is performed by the broadcast capturing unit 110 may be performed by using a conventional technique (for example, "music show" (music non-music piece detection technique): japanese patent application 8-340293, japanese application 10-68158, etc.) for judging whether the music piece or the non-music piece is a music piece or a non-music piece (not a music piece, but a speech, etc.). The broadcast capture unit 110 uses this technique to register an item indicating that it is determined to be a music piece in the music piece extraction file 130.

The search unit 120 reads the file storing the content characteristics generated by the content generation device 300 in the memory, further reads the broadcast characteristics file 140, performs matching (matching) between the two files, and stores the matching result in the search result file 150. The data not collated is stored in the music title unextracted file 160.

As the comparison search, a learning activity (learning active) search method described in "high-speed signal search method and apparatus and storage medium thereof" of japanese patent No. 3065314 can be used. According to this method, the similarity value between the content profile data and the broadcast profile data is obtained, and the obtained similarity value is compared with a threshold value to perform a search. For a detailed description thereof, see the above-mentioned Japanese patent No. 3065314.

The music extraction file 130 is composed of data to which a time stamp is assigned as information indicating that the music is music, and the broadcast profile generated by the broadcast capture search unit 100 is a TAZ file (binary file). Note that the TAZ file is a file that can perform high-speed comparison processing in learning activity search.

The broadcast content is stored in a WAV file or the like, which is not shown in the figure.

In the broadcast feature file 140, the broadcast data to be played, which is automatically captured by the broadcast capture unit 110, and feature data of the broadcast data from which the features are extracted are stored.

In the music unextracted file 170, data extracted from the music extracted file 130, the music name unextracted file 160, and the broadcast profile 140 as music but not collated is stored.

The search result file 150 stores the comparison result between the content profile (described later) generated by the content generation device 300 and the broadcast profile 140 by the search unit 120. That is, information (such as a music title) corresponding to data matching the matching result is stored in the search result file, and data that does not exist in the content feature file among the broadcast feature files 140 is stored in a music title unextracted file (described later).

The music confirmation registration device 200 includes a music confirmation unit 210 and a registration unit 220, and outputs a time-series play list 230.

The music verification unit 210 extracts the time, music name, composer, program (CM) name, client, product, actor, and CD information corresponding to the recognized music using the search result file 150 and the content information database 330, and supplies these to the registration unit 220.

The registration unit 220 registers the content extracted by the music confirmation unit 210 in the time-series manifest 230 in chronological order.

In the music confirmation registration device 200, the operator confirms the broadcast content stored in the music non-detection file to recognize the music, and adds the corresponding data to the time-series playlist. Further, the operator can listen to the music captured by the broadcast capture unit 110 as a WAV file or the like based on the music undetected file time information to confirm the broadcast content.

The content creation device 300 includes a content creation unit 310, a content profile 320, and a content information database 330.

The content creation unit 310 acquires the content from the medium in which the music is recorded, stores the music name and the attribute data that can be commonly used for the content profile 320 in the content information database 330, and further extracts the music characteristics and stores the music name and the music characteristics in the content profile 320.

The content profile 320 is a file generated by the content generation unit 310, and stores a music name and the music profile information.

The content generation unit 310 stores all the attribute data of the music in the content information database 330.

The following describes the operation of the above-described music recognition system.

Fig. 4 shows an outline of the operation of the music recognition system of the present invention.

Step 101: in the content creation device 300, music from a music content storage medium or the like and attribute data of the music are registered in the content information database 330. The attribute data is information such as a name of a music tune, a composer, a program name, a client, a trade name, and a name of an actor.

Step 102: the content creation device 300 extracts music characteristics and stores the music characteristics in the content characteristics file 320.

The treatment up to this point is a pretreatment of the following treatment.

Step 103: the broadcast image, voice, and the like are captured by the broadcast capture search device 100. Here, "played image, voice, or the like" includes an image, voice, or the like broadcast via the internet.

Step 104: the broadcast capture retrieval device 100 detects whether the broadcast data is a musical piece or a non-musical piece using any technique of detecting whether the captured content is a musical piece or a non-musical piece.

Step 105: when the piece of music is captured in step 103, an item indicating that the piece of music is broadcast at that time is recorded in the music extraction file 130. The recording of an event indicating a piece of music means, for example, creating a mark representing the piece of music at that time.

Step 106: in addition to the above steps 104 and 105, music characteristics are extracted at intervals of 7.5 seconds in real time and stored in the broadcast characteristics file 140. The broadcast data of music is also stored in a file not shown in the figure.

Step 107: next, the playback/capture/search device 100 starts a search application to search for music, and reads the contents of the content profile 320 generated by the content generation device 300 in step 102 into the memory.

Step 108: the search unit 120 of the broadcast capture search device 100 also reads the broadcast feature file 140 and performs matching between the two files. In this case, the learning activity search technique is adopted.

Step 109: the comparison result is stored in the search result file 150. The feature data not matched is stored in the music title unextracted file 160.

Step 110: next, the music verification registration device 200 reads each content of the content information database 330 from the content creation device 300.

Step 111: the music confirmation registration apparatus 200 further reads the search result file 150 from the play capture search apparatus 100.

Step 112: accordingly, the music verification and registration device 200 extracts information such as a music title, a composer, a captured program, a client, a product, and an actor from the data in the search result file 150 and the data in the content information database 330, performs processing for arranging these data in time series, generates a time-series play list, and stores the time-series play list as the time-series play list database 230.

Step 113: the operator also views the music to check the music name corresponding to the feature data recorded in the music and detect the play list, thereby complementing the time-series play list.

Second embodiment

Next, a CM recognition system for recognizing and storing a CM (commercial advertisement) among image or voice information played by a television and an FM/AM will be described as embodiment 2 of the present invention. Note that by referring to the CM data generated by the CM recognition system, a play sheet containing CM data can be generated from the musical composition recognized in the first embodiment.

Fig. 5 is a schematic diagram of a CM recognition system of the present invention.

The CM recognition system shown in the figure includes a broadcast capture search device 400, a CM confirmation update unit 500, and a CM management database group 600.

The broadcast capture search device 400 includes a capture unit 410, a search unit 420, an FM/AM/CM file 430, a broadcast feature file 440, a CM data file 450, a TV/CM data file 460, and a time data providing unit 470.

The capturing unit 410 continuously monitors broadcast data for a television station or a radio station, and determines whether the monitored broadcast data is a CM by determining the end and the start of the CM; if CM, the monitored CM data is stored in TV-CM data file 460 or FM/AM-CM file 430. The capture unit 410 also assigns a time stamp to the CM data based on the time information supplied from the time data supply unit 470.

Further, the capture unit 410 generates feature data of the CM from the captured data and stores the feature data in the broadcast feature file 440. As will be described later, in order to absorb the cutting point error of the data divided from the start point and the end point of the CM when the broadcast profile 440 is generated, the data is further processed into data of a predetermined length (8.0 seconds), that is, data which is cut so as to be equally extended from the center of the CM data to both sides. Then, the re-processed data is subjected to characteristic data conversion by the technique disclosed in the learning activity search technique and stored in the TAZ format. The TAZ file is a file that allows the search unit 420 to perform a high-speed comparison process for the learning activity search.

The search unit 420 reads the broadcast feature file 440 and the file in which the CM feature is stored in the CM management database group 600 into the memory, performs the comparison between the two files, and stores the comparison result in the CM data file 450. It is assumed that the search unit 420 performs a learning activity search (japanese patent No. 3065314, etc.). At this time, the matching result indicates that no CM has been searched, and the CM data is stored in the CM data file 450 with the playback time as the name.

The FM/AM broadcast CM data captured by the capturing unit 410 is stored as a WAV format (voice only format) file in the FM/AM CM file 430.

The broadcast profile 440 stores CM profile data extracted from the CM data captured at 410. The broadcast profile 440 is a TAZ file (binary file).

The TV broadcast CM data captured by the capture unit 410 is stored as an AVI format file in the TV CM data file 460.

The CM confirmation update unit 500 reads the CM data file 450 storing the CM data whose CM name is not known. Since the CM in the CM data file 450 is highly likely to be a new CM, the operator confirms the newly registered CM using software such as Windows Media Player 7, for example, and extracts an advertiser (client), a product name, a music title, and an actor name, and stores them in the file of the CM management database group 600.

The CM management database cluster 600 includes a CM master file 610, a commodity master file 620, an actor master file 630, a tune name master file 640, an advertiser master file 650, and a master CM content profile 660. The CM confirmation update unit 500 extracts data in the product main file 620, the actor main file 630, the music title main file 640, the advertiser main file 650, and the main CM content profile 660. These master files are master files generated according to the attribute of the CM data stored in the CM master file 610.

The following describes actions according to the above-described structure.

Fig. 6 and 7 are schematic operation flowcharts of the CM recognition system according to the present invention.

Step 301: the capturing unit 410 of the broadcast capturing and retrieving device 400 captures broadcast data to be played.

Step 302: the search unit 410 detects CM data from captured broadcast data, and extracts broadcast characteristics from the CM data in the manner described above.

Step 303: the extracted broadcast characteristics are stored in the broadcast characteristics file 440, and the process proceeds to step 307.

Steps 304, 305: at the same time as the above processing, the TV CM is stored in the TV-CM data file 460.

Step 306: if the extracted CM is a CM that has already been FM/AM played, the extracted CM is also stored in an FM/AM/CM file 430.

Step 307: after the step 303, the search unit 420 reads the broadcast profile 440 and the CM content profile 660 of the CM management database group 600 into the memory, compares the two profiles, and performs a learning activity search.

Step 308: when the CM is found by the search, the process proceeds to step 307, and the next broadcast profile 440 data and CM content profile 660 are searched. And when the CM is not known, go to step 309.

Step 309: if the CM is not known, the playback time is registered as a name in the CM data file 450.

Step 310: the CM confirmation update unit 500 confirms the CM that has been registered in the CM data file 450 by the operator using existing software or the like.

Step 311: the CM master file 610 is maintained by an operator, and further, maintenance of each attribute master file is also performed from the CM master file 610.

In this way, a new CM may be registered in the database.

Third embodiment

It is also possible to generate a time-series play-out sheet from the CM recognized in the second embodiment, as in the case of generating a time-series play-out sheet from the recognized musical composition shown in the first embodiment.

Further, the content profile and the content information database of the music may be updated as in the CM example of the second embodiment.

Further, the content profile and the content information database relating to the CM or the music may be updated as in the second embodiment while generating the time-series playlist as in the first embodiment. Specific examples thereof will be described later.

Specific examples corresponding to the above-described embodiments are described below with reference to the drawings.

Specific examples corresponding to the first embodiment (specific example 1)

The same reference numerals are given to the respective devices shown in the figure as in the configuration shown in fig. 3, and the description thereof is omitted.

The following describes the music recognition system according to the present invention with reference to fig. 8 and 9.

As shown in fig. 8, the music recognition system is connected with: broadcast capture retrieval apparatus 100, which is formed by connecting PC110 capturing broadcast content in real time and PC120 retrieving broadcast profile 140; a content creation device 300 including a PC310 for managing a content profile 320 and a content information database 330 each of which retains a feature of a content of a musical composition; and a music confirmation registration device 200 that registers the time-series playlist in the database.

In the series a of processing shown in fig. 9, the PC110 captures the broadcast content (step 201), and outputs the broadcast profile 140 at 7.5-second intervals from the captured data (step 202); next, the PC120 searches for music by a learning activity search method using the broadcast profile 140 and the content of the content profile 320 acquired from the content generating device 300 and opened in the memory in advance (step 203), and outputs the search result to the search result file 150 (step 205). In this case, if the music is not found by the search, the feature data is stored in the music name non-extracted file (step 207).

Upon completion of the above processing, the music verification registration device 200 generates a time-series playlist from the search result and content information database in the search result file 150 and stores the generated time-series playlist in the database 230.

In the processing series B shown in fig. 9, the PC110 of the broadcast capture retrieval device 100 determines whether or not the broadcast data is a musical composition at a next time (step 301); if it is a music piece, an event indicating that it is a music piece and a time stamp are output to the music piece extraction file 130 (step 302). The music itself is also given a time stamp and stored in a WAV file or the like, which is not shown in the figure.

In this way, the music name unextracted file 160 (of the processing series a) and the music extracted file 130 (of the processing series B) are combined to output the music undetected file 170 by time division (steps 208 and 209), and the operator can feed back to the search result file 150. The time-series play-out list can be supplemented accordingly.

Note that, as the operation by the operator: the search result file 150 is supplemented with necessary data items while confirming the contents of the music undetected file on the PC120 that performs the search.

Note that, the content feature file 320 and the search result file 150 are referred to as TAZ files (binary files for learning activity search).

In order to generate a play list, a search result file and a content information database including a CM master file and the like are combined. Specific examples thereof will be described below.

Note that, in the example shown in fig. 8, the PC110 can capture broadcast content nationwide by receiving data from the corresponding area, and further perform broadcast profile generation and the like. The PCs 110 may be arranged all over the country, and each PC110 may capture local broadcast content to generate a broadcast profile and transmit the broadcast profile to the PC120 arranged at the center.

Application examples of the present invention are described below.

Fig. 10 is a schematic view showing an application example of the specific example of the present invention.

In the figure, the content management center corresponds to the content creation device 300 shown in fig. 8. The data center corresponds to the broadcast capture search device 100 shown in fig. 8.

First, at the content management center, a trial listening CD or the like is obtained from a disc company, a DVD, a new CD, or the like is purchased from a CD sales store or the like, the content is stored in the content information database 330 together with the content attribute based on these storage media, and the content characteristic is extracted and stored in the content characteristic file 320 (in the example of fig. 10, a music database).

The data center acquires broadcast contents such as television (VHS) and FM or satellite broadcast contents acquired via parabolic antennas by using a channel selector, for example, and extracts data characteristics and stores the data characteristics in the broadcast characteristics file 140, extracts a musical composition or a non-musical composition such as speech at 7.5-second intervals, and stores the result of the judgment in the musical composition extraction file 130.

In a search PC of a data center, a search engine (learning activity search technology) searches for music from a content profile and a broadcast profile acquired in advance from a content management center, and stores the music in a search result file 150.

Accordingly, in the PC serving as the music confirmation registration device 200 in the data center, a time-series playlist is generated using the search result file and the content information database 330. In the example of fig. 10, the music title, composer, program (CM), client, product, actor, and CD information are registered as time-series play lists in the database for the website (Web Site) in order of time (time stamp assigned to the search result file). Further, the operator adds music to music that cannot be searched out.

Fig. 11 shows respective files used when making a time-series playlist and their relationships.

As shown in the figure, a search result file and a music title unextracted file are generated from a broadcast profile and a content profile. And then a time-series playlist is generated from the search result file and the content information database.

On the other hand, a time-series playlist can be supplemented by generating a music unexpected file from a music extracted file and a music name unexpected file, and confirming a music name corresponding to a music recorded in the file by an operator. Further, the music pieces whose names of music pieces and the like are confirmed may be added to the content profile by performing a characterization process as necessary. Thus, when a musical piece corresponding to TAZ4 is captured, the musical piece can be recognized.

A method of generating the time-series playlist shown in fig. 10 using the contents information database is described below with reference to fig. 12 and 13.

Fig. 12 is a diagram illustrating items at the time point of 9 determined with respect to a play list. As shown in the figure, the content information database includes databases (master databases) serially connected from the TAZ data. Therefore, if the music title is found from the TAZ data, the information at time 9 can be acquired by searching each master file one by one from the sound source master file. In addition, the program name can be known from the broadcast station name and time. In this way, a time-series playlist containing various associated information can be generated.

Fig. 13 shows the time sharing at the time 11 and 46, and various information corresponding to the time can be acquired from the TAZ data by using the master files, as described above.

The following explains the simulation result of the music recognition system of the present invention. The simulation was broadcast sampling of an actual FM broadcast recording (about 35 cents) containing 7 music pieces; the CD music 193 was first used as a database sample, of which 6 were included in the broadcast samples.

The simulation conditions were set as: broadcast samples were compared to all 193 tunes at 7.5 second intervals (193 equals 20 hours); specification of PC Server-1 CPU (Pentium Xeon933MHZ), 2GB memory, Linux gcc 291.

The results of simulations carried out under the above conditions with the method of the invention show: all 6 heads to be detected are correctly detected; the time error is approximately within 7.5; in terms of speed, about 45 seconds completes a 35 minute (2100 seconds) long search, i.e., 50 times the speed in real time (2100/45), that is, about 9000 times the speed can be processed if the real time is occupied.

Although the above embodiment has been described with reference to fig. 8 and 9, the operation shown in fig. 9 may be programmed and stored in a removable storage medium such as a hard disk, a flexible disk, and a CD-ROM connected to computers used in a data center and a content management center, and the computer may be installed in the computers used in the data center and the content management center when the present invention is implemented. The invention can hereby be easily implemented.

Specific examples corresponding to the second embodiment (specific example 2)

Described herein is an example of recognizing and storing a CM detected in real time from broadcast data being broadcast. By using the CM data thus saved as the above-described master file, a time-series playlist including CM-related information can be generated.

Fig. 14 is a block diagram of a CM recognition system of embodiment 2 of the present invention.

The same reference numerals are given to the constituent elements shown in the figure as in the configuration shown in fig. 5, and the description thereof is omitted.

The CM recognition system of the present embodiment includes: broadcast capture retrieval means 400-retrieval PC420 by capture PC410 capturing broadcast content and learning activities comparing broadcast profile 440 with CM content profile 660; and a CM confirmation updating part 500 for performing CM confirmation and updating processing with reference to the CM data 450 outputted from the learning activity search PC 420; and a CM management database cluster 600, which manages the master files of the CM.

The file format in fig. 14 is explained here.

The broadcast profile 440 is a TAZ format file storing both TV · CM and FM/AM · CM profiles extracted in real time.

The TV · CM data file 460 is an AVI format file storing TV · CM, and is managed in pair with the content of the broadcast profile 440.

The FM/AM CM file 430 is a WAV format file storing FM/AM CM, and is managed in pair with the content of the broadcast profile 440.

The CM data file 450 is a file storing a CM to be newly registered, and stores data in, for example, an AVI format in the case of a TV · CM and stores data in, for example, a WAV format in the case of an FM/AM · CM.

The master CM content profile 660 of the CM management database group 600 is stored in TAZ format.

The operation of the above structure will be described with reference to fig. 6 and 7 described above.

The capturing unit 410 captures the broadcast content (step 301). Then, the capture PC410 acquires the start and end points of the CM by the CM detection module. The CM detection internal module is disclosed in Japanese patent application No. 6-312976, "image cutting point detection method and apparatus" (Cumura. village).

Next, in order to absorb the cutting point error of the cut data by the CM detection module, as shown in fig. 15, the cut data is further processed into data of a predetermined length (8.0 seconds), that is, data which is cut by being extended equally from the CM data center to both sides, and extracted as a broadcast feature (step 302) and stored in the broadcast feature file 440 (step 303).

The capture PC410 stores the broadcast data cut by the CM detection module in AVI format when the data is TV data, and stores the broadcast data in WAV format when the data is FM/AM data (steps 304, 305, and 306).

Next, the learning activity search PC420 reads the broadcast profile 440 and the master CM content profile 660 of the CM management database group 600 into a memory, and performs a learning activity search (step 307). Here, if the CM is not registered in the CM content profile 660, the CM is registered in the CM data file 450 (step 309), and if the CM content profile 660 is not registered, the CM is also unconditionally registered in the main CM content profile 660.

Next, the CM confirmation update unit 500 confirms the CM already registered in the CM data file 450 (step 310). The CM confirmation process is performed using existing software. The CM is registered in the CM master file 610 by adding various final additional information by the operator. Further, the product main file 620, the actor main file 630, the tune name main file 640, the advertiser main file 650, and the main CM content profile 660 are also updated with correspondence data based on the attributes of the CM that has been registered in the CM main file 610 (step 311).

In the above embodiment, the operations shown in fig. 6 and 7 are described with reference to the flow shown in fig. 6, but the operations shown in fig. 6 may be programmed and stored in a removable storage medium such as a hard disk, a floppy disk, or a CD-ROM connected to a computer used in the broadcast capture search device, and may be installed in a computer (capture PC, learning activity search PC) used in the broadcast capture search device when the present invention is implemented. The invention can hereby be easily implemented.

In the above-described embodiment, the process of capturing and recognizing and storing broadcast data broadcast by a broadcast station such as a TV or FM/AM is shown, but the present invention is not limited to this example, and the same process as described above may be applied to data transmitted via a communication network such as the internet.

Note that the example in which music is characterized at intervals of 7.5 seconds and CM at intervals of 8 seconds shown in the above embodiment is merely an example, and is not limited thereto.

Specific examples corresponding to the third embodiment (specific example 3)

Described here is an example of generating a play list as in the first embodiment by capturing a tune, CM, and registering data in the content generation section as in the second embodiment.

Fig. 16 shows a system configuration of example 3 of the present invention.

As shown in the figure, the music CM recognition system according to example 3 includes: broadcast capture retrieval device 700-capture broadcast content, perform music and CM retrieval; and a content generation device 800 that generates a content information database; and a CM confirmation registration device 900 for generating a time-series playlist from the search result file and the content information database, and registering attribute information of the newly confirmed music or CM in the content generation device 800.

The broadcast capture search device 700 includes a broadcast capture unit 710, a search unit 720, a music CM extraction file 730, a broadcast profile 740, a search result file 750, a music CM non-extraction file 760, a music CM non-extraction file 770, and a content profile generation unit 780. The content creation device includes a content creation unit 810, a content profile 820, and a content information database 830.

Fig. 17 is a flowchart showing an outline of the operation of the system shown in fig. 16. An outline of the action is described with reference to fig. 17.

First, the content creation device 800 extracts the characteristics of music and CM and stores the extracted characteristics as a content characteristic file (step 401). In the broadcast capture search device 700, the broadcast capture unit 710 captures broadcast contents including music and CM, stores the contents as an AVI file, a WAV file, or the like, extracts the characteristics of the contents, and stores the contents in the broadcast characteristics file 740 (step 402). Next, the music and CM are searched for using the content profile 820 and the broadcast profile 740 (step 403). When the music or CM is found, the information on the music or CM is stored in the search result file (step 404), and a time-series playlist is generated (step 405). If the music or CM is not found, the feature data of the unknown music or CM is stored in the music/CM unextracted file (step 406), and a content feature file corresponding to the music or CM is generated and automatically registered in the content feature file 820 of the content generation device 800 (step 407).

The above process is further described in the following with reference to the process flow diagram of the play capture retrieval apparatus 700 shown in fig. 18.

As shown in fig. 8, the processing of the broadcast capture search device 700 in this embodiment can be divided into: a processing sequence 1 for performing music and CM search; and a processing series 2 for searching, judging and storing music; and a processing sequence 3 for detecting and storing the CM.

In the processing sequence 2, it is determined whether or not the broadcast content is a music, and a music extraction file is generated and music data is stored as a WAV file or the like (steps 501 to 503), as in the first embodiment. The music data saving method is shown in fig. 9. Thus, the portion determined as music is stored every 7.5 seconds.

In the processing series 3, as in the second embodiment, a CM extraction file is generated by detecting the broadcast cutting point, and the CM data is also saved as an AVI file or the like (steps 511 to 513). The processing situation is shown in fig. 20. Thus, the portion of the CM between the cut points is preserved.

Note that the CM extraction file is a file like a music extraction file for music, and records events indicating CM and time thereof.

In the processing series 1, the feature extraction, the music, or the CM search is performed as in the first embodiment and as in the second embodiment for the CM (steps 521 and 522).

When the music or the CM name is known, a search result file 750 is generated (step 523). If the music or the name of the CM is not known, the data is stored in the CM unextracted file (step 524), and the content profile is automatically generated using the data (step 525) and is temporarily registered in the content profile 520.

Fig. 21 shows a process of generating a content profile from a music CM unextracted file. In this case, a content profile is generated from a data file (AVI or WAV) of music and CM corresponding to the data in the music/CM unextracted file.

Further, as in the first embodiment, a music CM undetected file is generated from the music CM unextracted file and the music CM extracted file (step 526), and the operator confirms the music and CM and performs time-series play list addition with the confirmed music and CM (step 527). Further, the database in the content creation device is updated (the TAZ file is associated with the music title, the composer, and the like) using the confirmation result, and the pseudo-registered content profile is converted into the formal registration (step 528).

By performing such processing, it is possible to add data of a content profile based on a time-series play list and register music or CM-related information while generating the time-series play list.

As described above, according to the present invention for recognizing music in real time, broadcast data (music used for CM, etc.) to be played is digitized and characterized at intervals of 7.5 seconds, and is compared with a content profile of a music prepared in advance, and broadcast time information and a music name can be stored as a time-series play list in a storage device. Accordingly, a time-division playlist of music titles, composers, programs (CMs), clients, products, actors, and CD information can be obtained, and meaningful information results can be obtained. These can be used by the selling object for marketing information and the like.

Note that the reason why the digital feature is performed at intervals of 7.5 seconds is that the shortest broadcast time of CM is generally 15 seconds at present, and half of the broadcast time is taken for reliable search. Therefore, it is effective to change the digitization interval time to another value depending on the kind of contents, and it is not necessarily 7.5 seconds.

Further, according to the present invention, it is possible to automatically recognize, in real time, the CM monitoring work which has been performed manually, and to register, in real time, CM data of TV and FM/AM broadcasts even without flag information, transparent information, or the like. Further, this CM data may also be used to generate a time-series playlist.

Further, according to the present invention, in the process of generating a time-series play list, since data for which the musical tune name is not known can be automatically registered in the content generating device, database data in the content generating device can be enriched, and a more accurate time-series play list can be generated.

The present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the scope of the present invention.

Claims

1. A music recognizing method for recognizing a music from received data, comprising the steps of:

2. A music recognition method according to claim 1, further comprising the steps of:

determining whether the received data is music;

3. A music recognition method according to claim 2, further comprising the steps of:

the title and time of the music recorded in the music undetected file are added to the broadcast list.

4. A music recognition method according to claim 1, wherein the time-series play list is generated using the search result file and a content information database having information associated with the same music title;

the time-series playlist includes a time, a musical composition name broadcasted at the time, and information related to the musical composition name.

5. A music recognition method according to claim 1, wherein broadcast data is received in a plurality of regions, the data received in each region is transmitted to a central system, and the time-series play list is generated by the music recognition method in the central system.

6. A music recognition method according to claim 1, comprising the steps of:

receiving broadcast data in a plurality of regions; and

extracting the characteristics of the received data in each region and transmitting the extracted characteristics as data of the broadcast characteristic file to a central system,

the following steps are executed in the central system:

7. A music recognizing method according to claim 4,

the CM recognition method includes the steps of:

detecting CM data from the received data;

8. A music recognizing system for recognizing a music from received data, comprising:

a unit for acquiring data in a content profile storing characteristics of the content of the music; and

a unit for extracting the characteristics of the received data and storing the extracted characteristics in a broadcast characteristic file; and

a unit for searching music by comparing the content profile and the data of the broadcast profile; and

a unit for saving the music name as a search result file if the music name is known; and

and a unit for generating a time-series playlist of music pieces from the search result file.

9. A musical composition understanding system according to claim 8, further comprising:

a unit for determining whether the received data is a music piece; and

a unit for storing the item indicating that the data is music and the data receiving time in the music extraction file if the music is music; and

a unit for storing certain data in the music name unextracted file when the music name of the certain data in the broadcast characteristic file is not known in the music searching unit; and

and a unit for generating a music undetected file from the broadcast feature file, the music extracted file, and the music name unextracted file.

10. A musical composition learning system as claimed in claim 9, further comprising:

a unit for making the music recorded in the music undetected file audible to human; and

the unit in the playlist adds the name and time of the music piece recorded in the music piece undetected file.

11. A musical composition learning system as set forth in claim 8,

a unit for acquiring data from a content information database having information associated with the same musical tune name, and a unit for generating the time-series broadcast list using the search result file and the data in the content information database;

12. A music recognizing system for recognizing a music from received data,

the multi-region broadcast data receiving system comprises a device for receiving broadcast data in a multi-region and a central system for receiving data received in each region from each region device;

the central system comprises the following units:

13. A music recognizing system for recognizing a music from received data,

a central system for receiving the feature data of the broadcast data received in each region from each region device as a broadcast feature file;

the central system comprises the following units: