JP2012049748A - Sound volume controller, sound volume control method and program - Google Patents

Abstract

The present invention relates to a technique for controlling the volume of streaming broadcasting, a telephone conference, a TV conference, and the like, and in particular, to appropriately control the volume of sound corresponding to a plurality of objects.
A volume control device for controlling the volume of a sound corresponding to acquired data, comprising:
Acquisition means for acquiring the volume control information corresponding to the object specified by the specifying means from the specifying means for specifying the object corresponding to the sound and the storage means storing the volume control information corresponding to the object. And control means for controlling the volume of the sound according to the volume control information acquired by the acquisition means.
[Selection] Figure 3

Description

  The present invention relates to a volume control device, a volume control method, and a program, and more particularly, to an apparatus, method, and program for controlling the volume of a sound corresponding to an object.

  Streaming broadcast technology that distributes video captured by a camera installed at a base to another base via a network has been put into practical use. There is also known a technique in which terminals at a plurality of locations are connected via a network to perform a voice conference call. Furthermore, video conferencing using not only audio but also video has been put into practical use by transmitting and receiving both videos.

  In addition, there are the following prior art documents regarding volume control when there are a plurality of connected terminals in the above-described telephone conference or TV conference. The prior art document is a communication device capable of simultaneous communication with a plurality of terminals, and calculates the overall average value of the volumes of the received plurality of terminals so that the volume of each terminal becomes the average value. A communication device comprising means for adjusting is disclosed.

JP 2004-32296 A

  The technique disclosed in the above prior art document is a technique for adjusting the volume for each terminal, and there is no problem if there is only one speaker for each terminal, but a plurality of people speak on one terminal. The following problems occur. The technique disclosed in the above prior art document calculates the overall average value based on the volume received from each terminal. Here, when two people having a large difference in sound volume speak using one terminal, an intermediate sound volume between them is used as the sound volume of the terminal. Then, the volume of the terminal is adjusted based on the adjustment value of the terminal determined in view of the calculated overall average value, but the terminal has two speakers as described above. Even if the volume of the entire terminal is adjusted, the volume of a person with a loud voice remains high and the volume of a person with a low voice remains low.

  Note that it may be possible to simply equalize the volume of the sound received from the terminal, but then no matter how much the speaker is speaking, it will be adjusted to the same volume regardless of how much inflection the speaker is talking to. There is a problem that the nuances of the speaker are not communicated.

  The present invention relates to a technique for controlling the volume of streaming broadcasting, a telephone conference, a TV conference, and the like, and particularly aims to appropriately control the volume of sound corresponding to a plurality of objects.

  One aspect of an apparatus for solving the above-described problem is a volume control apparatus that controls the volume of a sound corresponding to acquired data, the specifying means for specifying an object corresponding to the sound, and an object In accordance with the volume control information acquired in the acquisition means, the acquisition means for acquiring the volume control information corresponding to the object specified in the specifying means, from the storage means that stores the corresponding volume control information, Control means for controlling the volume of the sound.

  As described above, according to the disclosed technique, the volume of the period during which the object is outputting sound is adjusted to the predetermined level set for the object, and therefore the adjustment level in the period is constant. Therefore, since the inflection in the output is maintained as it is, the difference in the volume level between the objects can be reduced, and thus the technique for controlling the volume of streaming broadcast, telephone conference, TV conference, etc. It is possible to appropriately control the volume of the sound corresponding to the object.

It is explanatory drawing which shows the volume relationship by the comparison of a face, and the magnitude | size comparison of a lip motion. It is an example of the hardware block diagram of the communication terminal used by this invention. It is an example of the block block diagram of the communication terminal used by this invention. It is a flowchart of the volume increase value setting of the specific speaker in one Embodiment of this invention. It is a conference volume adjustment value table (Example 1) for every speaker by this invention. It is explanatory drawing of the volume increase level value setting of the meeting audio | voice by this invention. 6 is a flowchart (first embodiment) illustrating a procedure of a conference volume adjustment method during a conference based on FIG. 5. It is a sequence diagram (the 1) which shows the flow of the signal between each functional block of the communication terminal by this invention. It is a sequence diagram (the 2) which shows the flow of the signal between each functional block of the communication terminal by this invention. It is a sequence diagram (the 3) which shows the flow of the signal between each functional block of the communication terminal by this invention. It is a meeting volume adjustment value table (Example 2) for every speaker by this invention. 12 is a flowchart (second embodiment) illustrating a procedure of a conference audio adjustment method during a conference based on FIG.

  Hereinafter, an embodiment in which the technology of the present invention is applied to a communication terminal such as a camera-equipped mobile phone will be described as an example of a volume control device to which the technology of the present invention is applied. In the present embodiment, in a video conference using video captured by the camera of the communication terminal, an example of adjusting the volume for each object based on the volume adjustment value set for each object. explain. Note that the object in this case refers to a speaker who speaks in a conference call.

FIG. 1 is an explanatory diagram showing the volume relationship between the comparison of the facial and the magnitude of the lip motion, and shows the facial and lip motion magnitudes of A1 and B2.
(A) Face and size of lip movement A-1 is a round face and the width of the lip movement is large, and Mr. B2 is a face-long face and the width of the lip movement is small. In order to recognize the width of the lip movement, the communication terminal monitors the movement of the lip of the person who is speaking.
(B) Volume of Mr. A and Mr. B It is assumed that Mr. B's voice is lower in volume than Mr. A's voice.
(C) Adjustment of Mr. B's volume In this embodiment, Mr. B's volume is increased by the set value for Mr. B (+ AdB) only while Mr. B is speaking a small voice. An example of making it easier to hear a voice will be described. Contrary to the embodiment described below, the same effect can be obtained by reducing the volume of the entire voice data by the set value for Mr. A (for example, ΔBdB) only while Mr. A with a loud voice is speaking. It is also possible to play.

  FIG. 2 is an example of a hardware configuration diagram of a communication terminal used in the present invention. The communication terminal has a function of collecting voices of conference participants and sending them to a remote location via a public network or the like, and also includes a display unit 5 and a camera unit 10 that detects the face and lip movement width of the speaker. have.

  The wireless communication unit 4 includes an RF (Radio Frequency) 41 and a BB (Broadband) 42, and the audio communication information from the audio input / output unit 8 and the image information from the camera unit 10 are transmitted to the antenna 3 under the control of the processor 11. It connects to a wireless communication network via and communicates with a communication terminal installed in a remote place.

  The display unit 5 and the input unit 9 are composed of an image display unit arranged on the operation surface of the communication terminal and a key string for performing an input operation.

  The voice input / output unit 8 inputs and outputs voice to and from the microphone 6 that collects the voice of the conference participant and the speaker 7 that transmits the voice of the speaker to the conference participant.

  The camera unit 10 captures content such as a conference person's face, materials and images, and the contents described on the blackboard under the control of the processor 11.

  The storage unit 12 includes a RAM 121 and a ROM 122, and is used in the operation of the communication terminal under the control of the processor 11.

  FIG. 3 is an example of a functional block configuration diagram of a communication terminal used in the present invention. As described in FIG. 2, the communication terminal has a function of collecting voices of conference participants and sending them to a remote location via a public network, and a display control unit 21 assuming a projector; The camera control unit 22 detects the face and lip movement width of the speaker.

  2 executes the program stored in the program storage unit 121, the object specifying unit 13, volume control information storage unit, volume increase level acquisition unit 15, conference audio adjustment described below The unit 17, the call recording unit 19, and each control unit operate. The program can be distributed by being recorded on a recording medium such as a CD-ROM. Furthermore, it is also conceivable that the program is stored in a server device on the network, and the terminal device is downloaded and installed.

  The object specifying unit 13 is obtained by adding an object specifying function to the conference voice adjustment unit communication terminal, and specifically identifies and specifies the speaker.

  The volume control information storage unit 14 is a storage unit that stores volume control information corresponding to an object, and stores volume control information corresponding to a conference volume adjustment value table.

  The volume increase / decrease level acquisition unit 15 acquires volume control information from the volume control information storage unit 14 stored in the form of a conference volume adjustment value table.

  The conference audio adjustment unit 17 is obtained by adding a conference audio adjustment function to the communication terminal. When the communication terminal is turned on, the conference audio adjustment unit 17 is activated, and the camera control unit 22, the microphone control unit 18, and the call The face data of the specific speaker is authenticated and registered by image analysis in cooperation with the recording unit 19, and the lip movement width of the speaker is measured and registered. Further, the volume increase level value is determined and registered by analyzing the voice recording of the specific speaker. Specifically, even when a person with a loud voice and a person with a low voice speak using the same communication terminal, the speaker is identified and registered by detecting the face and lip movement width of the speaker. Read out the volume increase level value that has already been adjusted.

  The speaker control unit 16 increases the output volume of a person with a low voice according to an instruction from the conference audio adjustment unit 17 by an amount corresponding to the volume increase level registered in the volume control information storage unit 14 based on the conference volume adjustment value table. To do.

  The input control unit 20 and the display control unit 21 control input / output keys and operation of the image display unit provided on the operation surface of the communication terminal.

  FIG. 4 is a flowchart for setting a volume increase value for a specific speaker in an embodiment of the present invention. The procedure for setting the conference volume adjustment value for a specific speaker will be described below.

  S1. The conference audio adjustment unit 17 determines whether to adjust the volume of audio while a specific conference participant is speaking. This determination is determined as YES when the user performs a predetermined operation on the communication terminal (for example, an instruction to execute the process from the menu).

  S2. When it is determined in S1 described above that the volume adjustment value is set, the conference audio adjustment unit 17 determines the number of specific speakers from the camera image.

  S3. The conference voice adjustment unit 17 determines whether or not the utterance voice adjustment values of the specific speakers for the number of persons are registered.

  S4. When the registration for the number of persons is not completed in S3 described above, the conference audio adjustment unit 17 determines the number of speakers in the conference.

  S5. When there is only one speaker, the conference audio adjustment unit 17 determines whether or not the speaker has been registered.

  S6. When the speaker is not registered in S5 described above, the conference voice adjustment unit 17 registers the face recognition data of the specific speaker in the conference volume adjustment value table.

  S7. The conference audio adjustment unit 17 records the utterance of the specific speaker and measures the volume.

  S8. The conference voice adjustment unit 17 extracts the lip motion width of the specific speaker, and registers the extracted lip motion width in the conference volume adjustment value table in association with the face recognition data of the specific speaker.

  S9. The conference audio adjustment unit 17 determines whether or not the loudness of the voice is greater than or equal to a threshold value.

  S10. When the loudness of the voice is less than the threshold value in S9 described above, the conference audio adjustment unit 17 determines the volume increase level.

  S11. When the voice volume is equal to or greater than the threshold value in S9 described above, the conference voice adjustment unit 17 sets the volume increase level to zero.

  S12. The conference voice adjustment unit 17 sets, as the volume control information, the adjustment value of the utterance voice of the specific speaker in the conference volume adjustment value table in association with the face recognition data of the specific speaker. As volume control information, in addition to setting an adjustment value, setting the volume itself may be considered. In this case, control is performed so that the volume of the sound corresponding to the acquired data is set to the set volume.

  FIG. 5 is a conference volume adjustment value table (first embodiment) for each speaker according to the present invention. In the face recognition data of the speaker A, A is set as a data value corresponding to the round face or the long face. Further, the lip movement width of the speaker A is measured as 4 cm as a result of the measurement. The loudness of the voice of the speaker B is determined from the face recognition data and the lip movement width.

  The determination of the volume increase level value of the speaker A is compared with the volume of the voices of all the conference participants. In this example, the volume of the speaker A is sufficiently large, and it is not necessary to increase the volume. The volume increase level is 0 dB.

  In the face recognition data of the speaker B, B is set as a data value corresponding to a round face or a long face. Further, the lip movement width of the speaker B is measured as 1 cm as a result of the measurement. The loudness of the voice of the speaker B is determined from the face recognition data and the lip movement width.

  The determination of the volume increase level value of the speaker B is compared with the volume of the voices of all the conference participants. In this example, the volume of the speaker B is small and the volume needs to be increased considerably. The volume increase level is +20 dB.

  FIG. 6 is an explanatory diagram of the volume increase level value setting of the conference recording voice according to the present invention. The waveform diagram of the lip motion width of Mr. A and the waveform diagram of the lip motion width of Mr. B shown in FIG. 6 are waveform diagrams of the lip motion width over time based on the speaker lip motion width shown in FIG.

  In the utterance recording voice, the utterance recording voice of the part where only Mr. B is speaking is cut out from the lip movement width and the volume is measured. Next, since the volume of Mr. B is very small compared to the reference volume (less than a predetermined threshold), plus 20 dB is added with the insufficient volume as the volume increase level. FIG. 6 is a waveform diagram before the volume increase level is added. When the volume increase level is added, only the voice of Mr. B is obtained as shown in the waveform diagram of the volume increase adjustment of Mr. B in FIG. The volume is increased. The reference volume is a standard volume at which individual utterance volumes can be heard clearly.

  FIG. 7 is a flowchart (Example 1) which shows the procedure of the meeting volume adjustment method during the meeting based on FIG. This flowchart is a flowchart of the conference volume adjustment method after the volume increase value is set, following the flowchart for setting the volume increase value for the specific speaker shown in FIG.

  The procedure of the conference volume adjustment method during the conference will be described below with reference to FIG.

  S13. The conference audio adjustment unit 17 determines whether the video conference is in progress.

  S14. The object specifying unit 13 determines whether or not the lip movement width of the speaker has been detected from the image on the camera monitor, and proceeds to the next step if it is determined that it has been detected.

  S15. The object specifying unit 13 recognizes the face of the speaker (such as a round face or a long face).

  S16. When the lip motion cannot be detected in S14 described above, the conference audio adjustment unit 17 determines that there is no speaker and returns the volume level to the initial value.

  S17. The object specifying unit 13 determines whether a plurality of speakers are speaking at the same time.

  S18. When it is detected in S17 that a plurality of speakers are speaking at the same time, the volume increase / decrease level acquisition unit 17 displays a record in which the facial recognition recognized in S15 is recorded as face recognition data. Extracted from the volume control information storage unit 14 based on the table. Then, the value set for the volume increase level of the record is extracted. When the volume increase level for all registered multiple speakers is obtained, the volume level is increased by the maximum increase level among the volume increase levels.

  S19. When it is detected that one person is speaking in S17 described above, the volume increase / decrease level acquisition unit 15 displays a record in which the face recognized in S15 is recorded as face recognition data, as a conference volume adjustment value. Extracted from the volume control information storage unit 14 based on the table. Then, the value set for the volume increase level of the record is extracted. Thereafter, the volume level is increased by the extracted volume increase level.

  FIG. 8 is a sequence diagram (part 1) showing a signal flow between the functional blocks of the communication terminal according to the present invention. The procedure for setting the conference volume adjustment value will be described below.

  S20. When the conference audio adjustment unit 17 is activated and an increase value setting for increasing the volume of a small voice is started, the conference audio adjustment unit 17 requests the camera control unit 22 to take a camera image.

  S21. The camera control unit 22 instructs the camera unit 10 to start capturing a camera image, and notifies the conference volume adjustment unit 17 of the camera image.

  S22. The conference volume adjustment unit 17 analyzes the image and determines the number of conference participants.

  S23. The conference audio adjustment unit 17 instructs the camera control unit 22 to start the camera monitor in order to start the image monitor.

  S24. The camera control unit 22 notifies the conference volume adjustment unit 17 of the camera monitor image.

  S25. The conference volume adjustment unit 17 detects the lip motion of one speaker shown in the camera monitor image and authenticates the face of the speaker.

  S26. When the face of the specific speaker has not been registered yet, the conference volume adjustment unit 17 registers the face data of the authenticated specific speaker.

  FIG. 9 is a sequence diagram (part 2) illustrating a signal flow between the functional blocks of the communication terminal according to the present invention. The procedure for setting the conference volume adjustment value will be described below.

  S27. The conference audio adjustment unit 17 requests the call recording unit 19 to start recording and instructs the microphone control unit 18 to turn on the microphone.

  S28. The call recording unit 19 starts voice recording.

  S29. The conference voice adjustment unit 17 requests the call recording unit 19 to stop recording after waiting for 10 seconds from the start of recording, for example, and instructs the microphone control unit 18 to turn off the microphone.

  S30. The call recording unit 19 notifies the conference audio adjustment unit 17 of the audio recorded by the specific speaker.

  S31. The conference voice adjustment unit 17 analyzes the volume of the recorded voice of the specific speaker received from the call recording unit 19.

  S32. When it is determined that the volume level of the specific speaker is less than the threshold, the conference audio adjustment unit 17 determines the volume increase level of the specific speaker. If the volume level of the specific speaker is found to be equal to or higher than the threshold value, the volume of the specific speaker does not increase.

  S33. The conference voice adjustment unit 17 sets, as volume control information, the adjustment value of the utterance voice of the specific speaker in the conference volume adjustment value table in association with the facial recognition data of the specific speaker.

  S34. The volume control information storage unit 14 stores volume control information for each speaker based on the conference volume adjustment value table.

  FIG. 10 is a sequence diagram (No. 3) showing a signal flow between the functional blocks of the communication terminal according to the present invention. The procedure for setting the conference volume adjustment value will be described below.

  S35. The conference audio adjustment unit 17 confirms that the video conference is in progress and requests the camera control unit 22 to capture a camera image.

  S36. The camera control unit 22 starts capturing a camera image and notifies the object specifying unit 13 of the camera image.

  S37. The object specifying unit 13 detects the lip movement width of the speaker from the camera image.

  S38. The object specifying unit 13 recognizes the face of the speaker (such as a round face or a long face).

  S39. The object specifying unit 13 performs the above-described S37. In S38, the specific speaker is specified, and the volume increase / decrease level acquisition unit 15 is requested to search the recorded data of the specific speaker.

  S40. The volume increase / decrease level acquisition unit 15 extracts a corresponding volume increase level from the conference volume adjustment value table of the volume control information storage unit 14 in which the volume control information of all speakers is recorded.

  S41. The volume control information storage unit 14 notifies the volume increase / decrease level acquisition unit 15 of the corresponding increase level value and also notifies the conference audio adjustment unit 17.

  S42. The conference audio adjustment unit 17 increases the volume level by the volume increase level extracted for the speaker control unit.

  In the embodiments described so far, the example using the lip motion detection technology has been described as the technology for specifying the speaker. However, the speaker may be specified by detecting other motions. Good. For example, the movement of a part other than the lips (chin, cheeks, etc.), the movement of the head, etc. may be detected.

  In addition, as an example of using a face recognition technique as a technique for uniquely identifying a speaker when associating a speaker with a volume increase level, an example using a part other than the face has been described. Also good. In implementing the present invention, it is not always necessary to use an image analysis technique. Other embodiments will be described below.

  As an example of a voice-only telephone conference that does not use images, the voice of the speaker is acquired using a communication terminal such as a mobile phone, and the acquired voice is transmitted to another communication via a network such as a public line. Assume that the message is sent to the terminal. The hardware configuration and the functional block diagram are the same as those in FIGS. 2 and 3, but the camera unit 10 in FIG. 2 and the camera control unit 22 in FIG. 3 are unnecessary.

  In this embodiment, it is assumed that the voice print data and volume increase level of each speaker are recorded in advance in the conference volume adjustment value table. The voiceprint data referred to here is a value obtained by analyzing a person's voice and extracting features.

  FIG. 11 is a conference volume adjustment value table (second embodiment) for each speaker according to the present invention. In the table, each voiceprint data is set in association with the speaker identifiers of the speaker A and the speaker B.

  The determination of the volume increase level value of the speaker A is compared with the volume of the voices of all the conference participants. In this example, the volume of the speaker A is sufficiently large, and it is not necessary to increase the volume. The volume increase level is 0 dB.

  The determination of the volume increase level value of the speaker B is compared with the volume of the voices of all the conference participants. In this example, the volume of the speaker B is small and the volume needs to be increased considerably. The volume increase level is +20 dB.

  FIG. 12 is a flowchart (Example 2) which shows the procedure of the meeting volume adjustment method during the meeting based on FIG. The procedure for setting the conference volume adjustment value will be described below with reference to FIG.

  S43. The conference audio adjustment unit 17 determines whether the video conference is in progress.

  S44. The object specifying unit 13 determines whether or not sound is input. If it is determined that the sound is input, the process proceeds to the next step.

  S45. The object specifying unit 13 analyzes the voiceprint of the speaker.

  S49. When voice input cannot be detected in S44 described above, the conference voice adjustment unit 17 determines that there is no speaker and returns the volume level to the initial value.

  S46. The object specifying unit 13 specifies the speaker based on the result of the voiceprint recognition in S45.

  S47. The object specifying unit 13 determines whether a plurality of speakers are speaking at the same time.

  S48. When it is detected in S47 that a plurality of speakers are speaking at the same time, the volume increase / decrease level acquisition unit 15 uses the volume based on the conference volume adjustment value table as a record in which the voiceprint data recognized in S46 is recorded. Extracted from the control information storage unit 14. Then, the value set for the volume increase level of the record is extracted. When the volume increase level for all registered multiple speakers is obtained, the volume level is increased by the maximum increase level among the volume increase levels.

  S50. When it is detected that one person is speaking in S47 described above, the volume increase / decrease level acquisition unit 15 records a record in which the voiceprint data recognized in S46 is recorded based on the conference volume adjustment value table. Extracted from the control information storage unit 14. Then, the value set for the volume increase level of the record is extracted. Thereafter, the volume level is increased by the extracted volume increase level.

  In the above-described embodiment, an example in which a speaker is specified by a voiceprint has been described. However, the present invention is not limited to this, and can use the characteristics of a speaker's voice to specify the speaker. I just need it. Furthermore, since it is only necessary to identify the speaker, it is not always necessary to use voice characteristics. For example, a speaker may always make a statement after pressing a predetermined key of a mobile phone assigned to each speaker, and the speaker may be specified based on the pressed key.

  In the above-described embodiment, an example of adjusting the volume of a person's voice has been described. However, the application field of the present invention is not limited to this. For example, an object that emits sound, such as a musical instrument or machine, may be considered, and a communication terminal may acquire the sound that is emitted and transmit it to another communication terminal. Even in this case, by detecting the motion and sound characteristics of these objects (or part of the objects), it is possible to determine which object corresponds to the sound and It is possible to control the volume based on the set volume increase level.

1 Facial appearance of Mr. A 2 Facial appearance of Mr. B 3 Antenna 4 Radio communication device 41 RF
42 BB
5 Display Unit 6 Microphone 7 Speaker 8 Audio Input / Output Unit 9 Input Unit 10 Camera Unit 11 Processor 12 Storage Unit 121 RAM
122 ROM
DESCRIPTION OF SYMBOLS 13 Object identification part 14 Volume control information storage part 15 Volume increase / decrease level acquisition part 16 Speaker control part 17 Conference audio | voice adjustment part 18 Microphone control part 19 Call recording part 20 Input control part 21 Display control part 22 Camera control part

Claims (3)

A volume control device for controlling the volume of sound corresponding to acquired data,
Identifying means for identifying an object corresponding to the sound;
Obtaining means for obtaining volume control information corresponding to the object specified by the specifying means, from storage means storing volume control information corresponding to the object;
Control means for controlling the volume of the sound according to the volume control information acquired by the acquisition means;
A volume control apparatus comprising: A volume control method for controlling the volume of sound corresponding to acquired data,
Identify the object corresponding to the sound,
Obtaining volume control information corresponding to the identified object from the storage means storing the volume control information corresponding to the object;
Controlling the volume of the sound according to the acquired volume control information;
A volume control method characterized by the above. Computer
Identifying means for identifying the object corresponding to the sound;
Obtaining means for obtaining volume control information corresponding to the object specified by the specifying means, from storage means storing volume control information corresponding to the object;
The program for functioning as a control means which controls the volume of the sound according to the volume control information acquired in the acquisition means.