JP2006033562A - Device for receiving onomatopoeia - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an onomatopoeia receiving device which is easily constituted for an auditorily disabled person to view a TV, generates a caption text with a high hit rate, and performs multiplexing on a reception image. <P>SOLUTION: The device comprises an onomatopoeia-transmitting part 16 for generating the database of onomatopoeia corresponding to the background sound of a preceding series among the series of programs to be broadcasted a plurality of times; an onomatopoeia-receiving part 31 for receiving the database of the preceding series from the onomatopoeia-transmitting part 16, based on the series to be broadcasted; a retrieving part 34 for retrieving a proximity background sound parameter which is most approximate to the background sound of the series to be broadcasted among the background sound parameters of the database, which is received by the onomatopoeia receiving part 31 and retrieving a proximity onomatopoeia corresponding to the proximity background sound parameter; and an image-compositing part 35 for compositing the proximity onomatopoeia retrieved by the retrieving part 34 with image information of the series of program. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention obtains an onomatopoeia related to the background sound from the series program onomatopoeia database based on the background sound included in the audio signal of the series program broadcast weekly or daily, and an image obtained by receiving the acquired onomatopoeia The present invention relates to an onomatopoeia receiving device that multiplexes and outputs a signal.

Recently, an increasing number of broadcast programs use voice recognition technology to convert the content of a performer's story into text characters and multiplex display them. It is effective for translation display of foreign language conversation and hearing aid for hearing impaired people. In the US, descriptive information other than conversation may be multiplexed as a closed caption for the hearing impaired, but the current broadcast is not a friendly broadcast for the hearing impaired.
Patent Document 1 discloses a more practical semi-automatic subtitle program production system that efficiently supports manual subtitle production by subtitle program producers for subtitles for speech based on video, audio, and time code of TV material programs. A subtitle text transcription unit that inputs additional information data such as text transcription and background sound, an automatic subtitle program data production unit that creates a subtitle screen and assigns timing based on the transcripted subtitle text, and A method is disclosed that includes a closed caption program editing / preview section for editing and previewing closed caption program data.
JP 2003-224774 A

  However, text-to-text conversion of background sounds is not as easy as text conversion of speech content, and even in the case of Patent Document 1, only work support is provided. Even in the field where subtitle production is carried out as a business, an apparatus for automatically generating subtitle text has not yet been realized. The onomatopoeia receiving device necessary for hearing-impaired persons to watch TV has not been realized.

  Therefore, the present invention has been made to solve the above-described problems, and can be used for a hearing-impaired person to view a television, is easy to configure, and generates caption text with a high hit rate. Another object of the present invention is to provide an onomatopoeia receiving device that multiplexes the received video.

  The present invention is based on the background sound used in the program in advance, in a system different from the program, when receiving the program to which the onomatopoeia created based on the background sound related by the frequency distribution characteristic is not added. Storing and transmitting the onomatopoeia database created in this manner, searching and receiving the onomatopoeia closest to the background sound used in the program from the database, adding the onomatopoeia corresponding to the background sound and adding the onomatopoeia An onomatopoeia receiving device used in an onomatopoeia transmission / reception system that outputs a video together with image information of a program, corresponding to the approximate background sound parameter closest to the background sound of the program and the approximate background sound parameter among the background sound parameters of the database A search unit for searching for an approximate onomatopoeia, and the approximate onomatopoeia searched by the search unit and image information of the program. Providing onomatopoeic word receiving apparatus characterized by comprising an image combining unit for forming a.

  According to the present invention, when receiving a program to which an onomatopoeia created based on a background sound associated with a frequency distribution characteristic is not added, a background sound that is used in the program in advance in a separate system from the program. Storing and transmitting a database of onomatopoeia created based on the above, searching and receiving the onomatopoeia closest to the background sound used in the program from the database, and adding the onomatopoeia corresponding to the background sound The onomatopoeia receiving device used in the onomatopoeia transmission / reception system that outputs video together with the image information of the program corresponds to the approximate background sound parameter closest to the background sound of the program and the approximate background sound parameter among the background sound parameters of the database A search unit that searches for the approximate onomatopoeia to be combined with the approximate onomatopoeia searched by the search unit and the image information of the program. And an image synthesizing unit that can be used for a hearing-impaired person to watch TV, and generates a subtitle text with a high hit rate and multiplexes it on the received video. Can be provided.

The onomatopoeia receiving device according to each embodiment of the present invention will be described below with reference to FIGS.
FIG. 1 is a diagram showing a schematic configuration example of an onomatopoeia generation / addition system according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a configuration example of the onomatopoeia information transmission device according to the embodiment of the present invention.
FIG. 3 is a diagram showing a configuration example of the onomatopoeia generation and reception system according to the embodiment of the present invention.
FIG. 4 is a diagram showing a description example of the onomatopoeia database according to the embodiment of the present invention.

The configuration of the onomatopoeia generation and reception system will be described.
The onomatopoeia generation / reception system shown in FIG. 1 includes onomatopoeia ID (Identification) input section 11, threshold determination section 12, background sound input section 13, feature parameter extraction section 14, table creation section 15, and onomatopoeia transmission section 16. An onomatopoeia generating / adding device 3 including a transmitting device 1, the Internet 2, an onomatopoeia receiving unit 31, a background sound input unit 32, a feature parameter extracting unit 33, an onomatopoeia searching unit 34, and an image synthesizing unit 35; Consists of
The onomatopoeic transmitter 16 which is a main part of the onomatopoeia information transmitting apparatus 1 shown in FIG. 2 includes an adjustment value holding circuit 161, an onomatopoeia ID storage circuit 162, a correlation table storage circuit 163, and a transmission control circuit 164.
The onomatopoeia receiving unit 31 that is a main part of the onomatopoeia generation / addition device 3 shown in FIG. 3 includes a communication control circuit 311, an adjustment value holding circuit 312, a correlation table storage circuit 313, and an onomatopoeia storage circuit 314.

The operation of the onomatopoeia generation and reception system will be described.
First, a viewing reservation is made for a program to be reproduced with an onomatopoeia added. The onomatopoeia generation / addition device 3 searches the Internet 2 for public onomatopoeia information related to the program, and obtains the onomatopoeia ID related to the background sound used in the program, the onomatopoeia feature parameters, and the threshold level information of the background sound. get. A multiplexed video signal is generated by searching for an onomatopoeia related to a feature parameter similar to the feature parameter acquired from the background sound received based on the onomatopoeia feature parameter upon reception of a program reserved for viewing.

Details will be described below.
First, the owner of the onomatopoeia information transmission device 1 determines a category of a program related to the onomatopoeia information providing service performed using the onomatopoeia information transmission device 1. As a program category suitable for the onomatopoeia providing service, for example, there is a so-called series program. The owner of the onomatopoeia information transmitting device 1 records the determined series program. The background sound obtained by reproducing the recorded program is input to the background sound input unit 13. A playback program is viewed by a viewing device (not shown). The background sound level is monitored with a level meter (not shown). When the user (viewer) determines that the background sound level is equal to or higher than a predetermined level and that it is effective to display the onomatopoeia, the viewer operates the onomatopoeia ID input unit 11 to determine the ID related to the onomatopoeia. Enter (Onomatopoeia ID). The input onomatopoeia ID is stored in the onomatopoeia ID storage circuit 162 of the onomatopoeia transmitter 16. At the same time, the threshold value determination unit 12 determines the volume level value input to the background sound input unit 13 as the threshold level. The threshold level is input to the adjustment value holding circuit 161 of the onomatopoeia transmission unit 16 and held (stored) there.

  At the timing when the threshold is determined, the adjustment value holding circuit 161 outputs a command for causing the feature parameter extraction unit 14 to extract the feature parameter of the background sound. The feature parameter extraction unit 14 extracts a feature parameter related to the center frequency component of the signal input from the background sound input unit 13 and the average sound pressure change of the signal of the component. The characteristic parameters are input to the table creation unit 15. The table creation unit 15 organizes and creates the input feature parameters as a feature parameter table. Background sounds expressed in the same onomatopoeia are collected as background sounds having the same characteristic parameters. The collected feature parameters are input to the correlation table storage circuit 163 and temporarily stored therein. In other words, the onomatopoeia information composed of the threshold level and the characteristic parameters input according to the onomatopoeia ID is stored in each storage circuit. The onomatopoeic information such as the onomatopoeia ID, threshold level, and feature parameters is grouped (sorted) for each program and stored. Onomatopoeia information grouped for each program is disclosed to users of the onomatopoeia generation and reception system via the Internet, for example, for a fee.

  When an operation unit (not shown) of the broadcast receiving apparatus 4 is operated and a viewing reservation is made for a program received weekly or daily, the viewing reservation information is transmitted to the communication control circuit 311 of the onomatopoeia receiving section 31 of the onomatopoeia generation / addition apparatus 3. Is done. Onomatopoeia information published on the Internet 2 is searched by the communication control circuit 311. The onomatopoeia information transmitting device 1 obtains information on which onomatopoeia information related to a program reserved for viewing is disclosed. When the public information is obtained, the communication control circuit 311 requests acquisition of the onomatopoeia information related to the viewing reservation program published on the Internet 2. Upon receiving the request, the transmission control circuit 164 of the onomatopoeia transmission unit 16 of the onomatopoeia information transmission device 1 transmits the requested onomatopoeia information after authenticating the communication control circuit 311 of the onomatopoeia generation / addition device 3 as necessary. The transmitted onomatopoeia information is transmitted via the communication control circuit 311, the onomatopoeia ID is input to the onomatopoeia storage circuit 314, the threshold level is input to the adjustment value holding circuit 312, and the characteristic parameter is input to the correlation table storage circuit 313. And memorized.

  Eventually, the viewing reservation time arrives, and the program reserved by the broadcast receiving device 4 is received. The received video signal is input to the image synthesis unit 35 and the audio signal is input to the background sound input unit 32. When the audio signal is a 2-channel stereo signal, a difference signal between the left channel audio signal and the right channel audio signal is input to the background sound input unit 32. The sum signal of the left and right channels contains many performer voice components, and the difference signal contains many background sound components. When the audio signal is a five-channel signal, the channel signal excluding the center channel is input to the background sound input unit 32 because many channel sound components are included in the channel signal excluding the center channel.

  The background sound component of the audio signal input to the background sound input unit 32 is input to the feature parameter extraction unit 33. There, it is detected whether or not the component of the input background sound signal is larger than the threshold level stored in the adjustment value holding circuit 312. When it is detected that it is greater than the threshold level, the center frequency component of the input background sound, the average sound pressure level of the frequency component, and the characteristic parameters related to the time change characteristic are extracted. The extracted feature parameters are input to the onomatopoeia search unit 34. The onomatopoeia search unit 34 searches for a feature parameter of the input speech signal similar to the feature parameter stored in the correlation table storage circuit 313. When a similar feature parameter is retrieved, the onomatopoeia ID related to the feature parameter and stored in the onomatopoeia storage circuit 314 is acquired from the onomatopoeia storage circuit 314. The acquired onomatopoeia ID is input to the image composition unit 35. The onomatopoeia ID is multiplexed with the video signal input to the image synthesizer 35, and a multiplexed video signal is output.

Here, similar word information such as the onomatopoeia ID stored in the onomatopoeia storage circuit 314 and the characteristic parameter stored in the correlation table storage circuit 313 in the series program is similar to the onomatopoeia information related to the program received previously or in the past. Sacrifice words with a high hit rate are obtained in many cases.
The relationship between the background sound feature parameter and the onomatopoeia will be further described with reference to FIG.
The onomatopoeia information shown in the figure shows the onomatopoeia characteristic parameters when the series program is a baseball broadcast.
The characteristic parameters are such that the center value of the frequency component is Pfc1, Pfc2, Pfc3 (Pfc1>Pfc2> Pfc3), and the temporal change of the average sound pressure level of the frequency component is attenuated, repeated at a slow cycle, And what is repeated at an early cycle is illustrated. Numbers 1, 4 and 6 are cases where the center frequency is Pfc1, and the time variation of the average sound pressure is different. Onomatopoeia are assigned to can, can, and cuckoo, respectively. Similarly, Goon, Gongon, and Gogging are assigned to those having a center frequency of Pfc2.

This example is an onomatopoeia based on the central frequency component of the background sound having a large level in the case of baseball broadcasting and the temporal change of the average sound pressure. In addition, it is possible to define more onomatopoeia correlation tables by using frequency component distribution, frequency component temporal change characteristics, and the like as characteristic parameters.
Similarly, by creating a similar onomatopoeia correlation table for each broadcast content of a series program, a common onomatopoeia can be defined and used for background sounds that appear multiple times in the series program. For example, for background sounds such as white noise with a wide frequency distribution, it will be “Zar” if it is a rainy landscape, “War” if it is the completion of people, “Crap” if it is applause, etc. It is necessary to display different onomatopoeia. In series programs, it is possible to obtain onomatopoeia that is highly relevant to the video, such as onomatopoeia of the same scene that was broadcast in the past.
In addition, the timing when the onomatopoeia is multiplexed and displayed on the video signal is when the level of the background sound reaches a predetermined level. The characteristic parameter preferably includes a level value related to display timing.

  As described above, according to the onomatopoeia generation and addition system described as the embodiment of the present invention, when receiving a program to which the onomatopoeia created based on the background sound related by the frequency distribution characteristic is not added, Stores and transmits a database of onomatopoeia previously created on the basis of background sounds used in the program in a separate system from the program, searches for onomatopoeia closest to the background sounds used in the program from the database, and The onomatopoeia receiving device 3 used in the onomatopoeia transmission / reception system for receiving and adding the onomatopoeia corresponding to the background sound and outputting the video together with the image information of the program is used as the background of the program from the background sound parameters of the database. A search unit for searching for an approximate background sound parameter closest to the sound and an approximate onomatopoeia corresponding to the approximate background sound parameter; Since the image synthesizing unit 35 is configured to synthesize the approximate onomatopoeia searched by the search unit and the image information of the program, a configuration that can be used by a hearing-impaired person to view TV is easy. Therefore, it is possible to realize an onomatopoeia generation / addition system that generates subtitle text with a high hit rate and multiplexes it on the received video.

  The present invention can be applied to an onomatopoeia-added reception system for the hearing impaired person that multiplexes and displays the onomatopoeia related to the background sound included in the received audio signal on the received video.

It is a figure which shows the schematic structural example of the onomatopoeia production | generation receiving system which concerns on implementation of this invention. It is a figure which shows the structural example of the onomatopoeia information transmitter which concerns on implementation of this invention. It is a figure which shows the structural example of the onomatopoeia production | generation receiving system which concerns on implementation of this invention. It is a figure which shows the example of a description of the onomatopoeia database based on implementation of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Onomatopoeia information transmission apparatus 2 Internet 3 Onomatopoeia generation addition apparatus 4 Broadcast receiving apparatus 11 Onomatopoeia ID input part 12 Threshold determination part 13 Background sound input part 14 Feature parameter extraction part 15 Table creation part 16 Onomatopoeia transmission part 31 Onomatopoeia reception part 32 Background sound Input unit 33 Feature parameter extraction unit 34 Onomatopoeia search unit 35 Image synthesis unit 161 Adjustment value holding circuit 162 Onomatopoeia ID storage circuit 163 Correlation table storage circuit 164 Transmission control circuit 311 Communication control circuit 312 Adjustment value holding circuit 313 Correlation table storage circuit 314 Onomatopoeia Memory circuit

Claims (1)

When receiving a program to which the onomatopoeia created based on the background sound related by the frequency distribution characteristic is not added, it was created based on the background sound used in the program in a system different from the program. Stores and transmits an onomatopoeic database, searches and receives an onomatopoeia closest to the background sound used in the program from the database, adds the onomatopoeia corresponding to the background sound, and adds image information of the program An onomatopoeia receiving device used in an onomatopoeia transmission / reception system that outputs video together with
A search unit for searching for an approximate background sound parameter closest to the background sound of the program and an approximate onomatopoeia corresponding to the approximate background sound parameter from the background sound parameters of the database;
An image synthesizing unit that synthesizes the approximate pseudonym searched by the search unit and image information of the program;
An onomatopoeia receiving device comprising:

Images