JP2008160755A - Movie recording / playback device - Google Patents

Abstract

【Task】
The characters of the post-recording input speech are placed at an appropriate display position.
[Solution]
The voice direction detection device 41 detects the voice arrival direction from the outputs of the microphones 34 to 40. The monitor direction detection device 26 detects the direction in which the LCD monitor 24 faces. The system control circuit 30 refers to the detection result of the monitor direction detection device 26 during the post-recording of the voice characters, and corrects the detection result of the voice direction detection device 41 to the voice direction considering the monitor direction. The character conversion device 46 recognizes the input speech and converts it into a character code. The character code by the character conversion device 46 is applied to the character display device 52 and the related information adding device 48. The character display device 52 generates an image signal indicating a sentence according to the character code from the character conversion device 52, and the image signal at a timing at which the image signal is superimposed on the screen position corresponding to the voice direction from the system control circuit 30. Is output to the mixer 22.
[Selection] Figure 1

Description

  The present invention relates to a moving image recording / playback apparatus.

  Video cameras that can be photographed while a photographer confirms a subject image being photographed on a liquid crystal display monitor (LCD monitor) have been commercialized. A video camera has a built-in microphone and can record video and audio in real time.

  The LCD monitor is attached to the housing via a biaxial rotation mechanism so that the LCD monitor can be directed toward the photographer with respect to various shooting directions. Thereby, it is possible to check an image being shot in real time.

  The LCD monitor is also used as a reproduction monitor for reproducing a recorded moving image after photographing. The user can appreciate the recorded moving image by reproducing the moving image with the display screen directed toward the viewer.

By the way, in recent years, with the development of voice recognition technology, as a still image recording device such as a digital camera, the input voice is voice-recognized and converted into characters, and images and characters are superimposed on the liquid crystal display to give impressions and facts at the time of shooting An imaging apparatus with a voice recognition function that can be recorded and output as character information together with an image has been proposed (see, for example, Patent Document 1). In addition, it has been proposed to detect the voice arrival direction (voice direction) and to superimpose characters on a display position corresponding to the voice direction.
JP 11-055614 A

  The technique described in Japanese Patent Laid-Open Publication No. 2003-259542 is a technique for characterizing sound at the time of shooting in a still image recording apparatus such as a digital camera. Regarding the detection of the voice direction, the voice direction is always detected at a relative angle with the shooting direction (the direction in which the shooting lens faces) as the front.

  Simply by detecting the voice direction at a relative angle with the shooting direction (direction facing the shooting lens) as the front, and placing the character information of the input voice at the screen position corresponding to the detected voice direction, It may not fit with intuition.

  When a recorded moving image is reproduced and voice is recorded as characters, the post-recording speaker is not always at an appropriate position with respect to the reproduced image. Therefore, when using a voice processing device that detects a voice direction relative to the shooting direction and inserts a recognized character at a screen position corresponding to the voice direction, the post-recording character is placed at an inappropriate screen position. Sometimes.

  An object of this invention is to show the moving image recording / reproducing apparatus which eliminates such a malfunction.

  In order to achieve the above object, a moving picture recording / reproducing apparatus according to the present invention includes a voice input unit, a voice direction detection unit that detects a direction of arrival of input voice by the voice input unit, and an input voice by the voice input unit. A character conversion device for converting a character into a character, a voice character display means for displaying a voice character at a position corresponding to the voice arrival direction, an image display means, a monitor direction detection means for detecting the orientation of the image display means, and a moving image At the time of recording, the display position of the output character of the character conversion device on the screen is determined according to the detection result of the voice direction detection unit, and at the time of moving image reproduction, according to the detection result of the monitor direction detection unit and the voice direction detection unit. And a control means for determining the display position of the output character of the character conversion device on the screen.

  The moving image recording / reproducing apparatus according to the present invention includes a moving image reproducing unit that reproduces a moving image from a recording medium, an audio input unit, an audio direction detecting unit that detects an arrival direction of input audio by the audio input unit, and the audio input unit. A character conversion device that converts input speech by a character into a character, a voice character display unit that displays a voice character at a position corresponding to the direction of voice arrival, an image display unit, and a monitor direction detection unit that detects the orientation of the image display unit And, according to the detection results of the monitor direction detection means and the voice direction detection means, the control means for determining the display position of the output character of the character conversion device on the screen, and the display position determined by the control means In order to display the output character of the character conversion device, the image display unit superimposes the information of the output character of the character conversion device on the reproduction image by the moving image reproduction unit. Characterized in that it comprises: a mixer to be supplied; information on output characters of the character conversion device; and recording means for recording information on a display position determined by the control means on the recording medium as video related information. To do.

  The moving image recording / reproducing apparatus according to the present invention further comprises means for detecting time information of a moving image being reproduced by the moving image reproducing means, and the recording means comprises information on the output characters of the character conversion device, the display The position information and the time information are recorded on the recording medium as related information of the moving image.

  According to the present invention, when it is desired to superimpose sound on a moving image with characters, the sound characters can be arranged at an appropriate display position corresponding to the position of the speaker during recording and during reproduction (after-recording).

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an image pickup apparatus 10 that is an embodiment of the present invention. The imaging device 10 can record and reproduce both moving images and still images. FIG. 2 shows a recording format of the recording medium 12. The recording medium 12 is composed of a recordable optical disc, for example. In the present embodiment, one or more still images and one or more voice character information can be added to one moving image. FIG. 2 shows a configuration example of a moving image, a still image, and voice character information managed together on the recording medium 12. On the recording medium 12, a still image recording area 72 for recording JPEG still image data and a character data recording area 74 for recording character data are added to a moving image / audio recording area 70 for recording MPEG moving image data. . One or more still images can be stored in the still image recording area 72. The character recording area 74 is an area for storing characterized voice recorded by the voice character after-recording, and is composed of a time code area 76, a display position area 78, a character code area 80, and a link information area 82. Details of the data stored in the character recording area 74 will be described later.

  When recording a moving image, the imaging unit 62 converts the optical image of the subject by the photographing lens 60 into an image signal, and outputs the image signal at a constant rate. The microphone 64 takes in surrounding sounds and outputs a sound signal. The MPEG encoder 66 compresses and encodes the image signal from the imaging unit 62 and the audio signal from the microphone 64 in accordance with the MPEG standard. The disk recording device 50 records MPEG compressed data (MPEG moving image data) from the MPEG encoder 66 on the recording medium 12. The area where the MPEG compressed data is recorded on the recording medium 12 corresponds to the moving image / audio recording area 70 of FIG.

  When recording a still image, the imaging unit 62 converts the optical image of the subject by the photographing lens 60 into a single still image signal in response to a release operation (not shown). The JPEG encoder 68 compresses and encodes the still image signal from the imaging unit 62 using the JPEG method. The disk recording device 50 records the JPEG compressed data (JPEG still image data) from the JPEG encoder 68 on the recording medium 12. When the still image recorded here is associated with the moving image recorded in the moving image audio recording area 70, the JPEG compressed data is stored in the still image recording area 72 of FIG.

  FIG. 3 shows an image diagram during the voice character post-recording process. At the time of voice character post-recording, the moving image to be post-recorded is reproduced from the recording medium 12 and displayed on the screen of the LCD monitor 24. In the screen of the LCD monitor 24, the still images 92 and 94 of the speaker captured in advance are displayed superimposed on the moving image, and the character information 96 and 98 captured and recognized by the method described later is superimposed on the moving image. Is displayed. In the example shown in FIG. 3, it is assumed that the post-recording speakers are person A and person B, and person A is on the left side toward LCD monitor 24, and person B is on the right side toward LCD monitor 24. In the voice character post-recording mode, a character 94 indicating the sound “good” uttered while watching the playback video displayed on the LCD monitor 24 is displayed on the right side of the screen of the LCD monitor 24, and the person B A character 96 indicating the uttered voice “Stop it” is displayed on the left side of the screen of the LCD monitor 24.

  FIG. 4 shows a flow of the operation procedure of the voice character post-recording process. First, the still image of the person A is recorded (S1), and the still image of the person B is recorded (S2). These still images are stored in the still image recording area 72 for the moving image to be dubbed. The still image 92 of the person A is set to be displayed on the left side of the screen of the LCD monitor 24 (S3). The still image 94 of the person B is set to be displayed on the right side of the screen of the LCD monitor 24 (S4). Then, the voice character after-recording is started (S5).

  FIG. 5 shows a schematic block diagram of a processing system for the voice character after-recording of the imaging apparatus 10. Note that the audio reproduction processing system is omitted.

  When playback of a specific moving image is instructed by the operation device 28, the system control circuit 30 instructs the disc playback device 14 to play back the MPEG moving image data recorded in the moving image / audio recording area 70 of the recording medium 12. The disc playback device 14 reads MPEG moving image data of the designated moving image from the optical disc 12. The moving image reproducing device 20 decompresses the MPEG moving image data from the disk reproducing device 14 and converts it into a moving image signal. The moving image signal reproduced in this way is supplied to the LCD monitor 24 via the mixer 22, and the reproduced image is displayed on the screen of the LCD monitor 24.

  When the operation device 28 instructs the playback of a specific still image, the system control circuit 30 instructs the disc playback device 14 to play back the JPEG still image data recorded in the still image recording area 72 of the recording medium 12. The disc playback device 14 reads JPEG still image data of the designated still image from the optical disc 12. The still image reproduction device 16 decompresses the JPEG still image data from the disk reproduction device 14 and converts it into a still image signal. The size conversion device 18 converts the still image reproduced by the still image reproduction device 16 into a size that matches the screen size of the still image sub-screen of the LCD monitor 24, and converts the converted still image to the same rate as the moving image. To output repeatedly. The still image signal output from the size converter 18 is supplied to the LCD monitor 24 via the mixer 22. As a result, the reproduced still image is displayed on the screen of the LCD monitor 24.

  The voice input device 32 includes 4-channel microphones 34, 36, 38, and 40. The microphones 34, 36, 38, and 40 are arranged on the front, rear, left and right as shown in FIG. The voice direction detection device 41 detects the phase difference between the input voices of the microphones 34 and 36, thereby detecting the voice arrival direction in the front-rear direction and the phase difference between the input voices of the microphones 38 and 40. The direction of voice arrival is detected. The voice direction detection device 41 supplies the detection result to the system control circuit 30. By detecting the front-rear and left-right directions, it is possible to determine from which direction the sound is 360 degrees.

  A method for determining the voice direction by phase difference detection will be described. For example, as shown in FIG. 6, when sound comes from the diagonally right direction of the microphones 38, 40, the time difference between the sounds arriving at the microphones 38, 40 is ΔT, the distance between the microphones 38, 40 is d, and the arrival direction is θ Then, ΔT = d × sin θ is established. The voice arrival direction θ can be determined from this relational expression.

FIG. 7A shows a state of voice input from the microphone 40, and FIG. 7B shows a state of voice input from the microphone 38. The horizontal axis indicates time, and the vertical axis indicates frequency. The microphone 40 detects the voice group A and the voice group B at the time t2, the microphone 36 detects the voice group B at the time t1, and the voice group A is detected at the time t2. In this case, the arrival direction of the voice group A is the front direction of the microphones 38 and 40 (the direction perpendicular to the line connecting the microphones 38 and 40), and the arrival direction of the voice group B is
(T2−t1) = d × sin θ
It can be detected as the diagonally right direction θ represented by

  The system control circuit 30 controls the direction filter 42 so as to separate the sound from the output sound signal of the microphones 34, 36, 38, 40 according to the sound arrival direction according to the detection result of the sound direction detecting device 41. The sound extraction device 44 removes noise from the output sound signal of the directional filter 42 and extracts a human sounding portion. The character conversion device 46 recognizes characters of the voice signal extracted by the voice extraction device 44 and converts it into a character code. The character code by the character conversion device 46 is applied to the character display device 52 and the related information adding device 48.

  The monitor direction detection device 26 detects the direction in which the LCD monitor 24 is facing and supplies the detection result to the system control circuit 30. The monitor direction detector 26 includes, for example, a rotation angle detection device or the like equipped on a rotation shaft of a rotation mechanism that rotatably attaches the LCD monitor 24 to the housing.

  The system control circuit 30 refers to the detection result of the monitor direction detection device 26 and corrects the detection result of the voice direction detection device 41 to a voice direction considering the monitor direction. For example, when the LCD monitor 24 is directed outward to the side of the imaging device 10, the post-recording speakers A and B speak while looking at the screen of the monitor 24. Positioned on the rear side of the device 10 (for example, the right side), the speaker B is positioned on the front side of the same side of the imaging device 10. In the same voice direction, the voice of the speaker A is the voice from the right rear with respect to the imaging device 10, and the voice of the speaker B is the voice from the right front with respect to the imaging device 10. When it is superimposed on the screen position according to the above, it becomes strange.

  Therefore, in this embodiment, the system control circuit 30 supplies the sound direction corrected with reference to the detection result of the monitor direction detection device 26 to the related information adding device 48 and the character display device 52 as a display position. In the example described with reference to FIG. 8, the system control circuit 30 causes the voice of the speaker A to come from the front left direction of the imaging device 10 and the voice of the speaker B comes from the front right direction of the imaging device 10. Thus, the voice direction is corrected, and the display position corresponding to the corrected voice direction is notified to the related information adding device 48 and the character display device 52.

  The character display device 52 generates an image signal indicating a sentence according to the character code from the character conversion device 52, and the image signal is displayed at a timing at which the image signal is superimposed on a display position corresponding to the voice direction from the system control circuit 30. Is output to the mixer 22. The mixer 22 superimposes the still image from the size conversion circuit 18 and the character image from the character display device 52 on the moving image from the moving image reproduction device 20 and supplies them to the LCD monitor 24. Thus, for example, for the example shown in FIG. 8, as shown in FIG. 3, the voice of speaker A is superimposed on the left side of the screen, and the voice of speaker B is superimposed on the right side of the screen.

  The related information adding device 48 is further supplied with a time code attached to the moving image reproduced from the moving image reproducing device 20, and is supplied with link information of a still image from the size converting device 18. The related information adding device 48 actually reproduces the time code from the moving image reproducing device 20, the link information from the size converting device 18, the display position from the system control circuit 30, and the character code from the character converting device 46. The information is supplied to the disk recording device 50 as the related information of the moving image. The link information from the size conversion device 18 is information indicating a link to the corresponding still image.

  The disc recording device 50 records the related information from the related information adding device 48 in the character recording area 74 of the recording medium 12 in the format as shown in FIG. The time code from the moving image playback device 20 is stored in the region 76 of the character recording region 74, the display position information from the system control circuit 30 is stored in the region 78, and the character code from the character conversion device 46 is stored in the region 80. The link information from the size converter 18 is stored in the area 82. These pieces of related information in the character recording area 74 are generated in units of voice extracted by the voice extraction device 44 and additionally recorded.

  As described above, when a moving image in which voice characters are additionally recorded is reproduced, the characters recorded in the character code area 80 recorded in the character recording area 74 are displayed at the timing of the time code recorded in the time code area 76. The image is displayed at a screen position corresponding to the display position recorded in the position area 78. If necessary, the still image of the speaker is also reproduced and displayed by the link information recorded in the link information area 82.

  Although the embodiment has been described in which recorded moving image data is reproduced and audio characters are additionally recorded, the input sound during moving image shooting may be converted into text and recorded as related information of the moving image. In this case, the system control circuit 30 determines the display position of the output character of the character conversion device 46 on the screen according to the detection result of the voice direction detection device 41 during moving image recording, and plays back the moving image (during voice character dubbing). In accordance with the detection results of the monitor direction detection device 26 and the voice direction detection device 41, the display position of the output character of the character conversion device 46 on the screen is determined.

  FIG. 9 is a flowchart showing which direction is set to the front depending on the recording mode or the reproduction mode. As shown in FIG. 9, the system control circuit 30 first determines whether the operation mode is a moving image playback (after-recording) mode or a moving image recording mode (S10). In the moving image reproduction (after-recording) mode (S10), the voice direction and the character display areas 92 and 94 are associated with each other with the display direction of the LCD monitor 24 as the front (S11). In the moving image recording mode (S10), the voice direction is made to correspond to the character display areas 92 and 94 with the lens direction as the front (S12).

  The character display area 92 may be expressed in different colors such as blue and the character display area 94 may be expressed in red so that they can be easily distinguished. In addition, if silence continues for a long time, a character string such as “...” Indicating silence may be displayed.

  Although the embodiment in which the input voice is converted into characters and recorded on the recording medium 12 has been described, the present invention is also applicable to the case where the characters of the input voice are superimposed and displayed on the monitor screen in real time. . In this case, the related information adding device 48 and the disk recording device 50 are unnecessary.

It is a schematic block diagram of the recording system of the present embodiment. It is a figure which shows the recording format of a recording medium. It is an image figure of the voice character post-recording process of a present Example. It is a basic flow of a present Example. 2 is a block diagram schematically illustrating the configuration of a processing system for voice character post-recording in the imaging apparatus 10. FIG. It is a figure explaining the determination method of the audio | voice direction by a phase difference detection. It is a schematic diagram which shows the audio | voice input example. It is a schematic diagram which shows the example of arrangement | positioning of the speakers A and B at the time of voice input. It is a flowchart which shows the example of mode transfer of a present Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image pick-up device 12 Recording medium 14 Disc reproducing device 16 Still image reproducing device 18 Size converting device 20 Movie reproducing device 22 Mixer 24 LCD monitor 26 Direction detecting device 28 Operating device 30 System control circuit 32 Voice input devices 34, 36, 38, 40 Microphone 41: Voice direction detection device 42 Direction filter 44 Voice extraction device 46 Character conversion device 48 Related information addition device 50 Disk recording device 52 Character display device 60 Optical lens 62 Imaging unit 64 Microphone 66 MPEG encoder 68 JPEG encoder

Claims (3)

Voice input means (32);
Voice direction detection means (41) for detecting the direction of arrival of the input voice by the voice input means;
A character conversion device (46) for converting a voice input by the voice input means into a character;
Voice character display means (52) for displaying a voice character at a position corresponding to the voice arrival direction;
Image display means (24);
Monitor direction detection means (26) for detecting the orientation of the image display means;
At the time of moving image recording, the display position of the output character of the character conversion device on the screen is determined according to the detection result of the voice direction detection unit, and at the time of moving image reproduction, the detection result of the monitor direction detection unit and the voice direction detection unit In accordance with the control means (30) for determining the display position on the screen of the output character of the character conversion device
A moving picture recording / reproducing apparatus comprising: A moving image reproducing means (20) for reproducing a moving image from the recording medium (12);
Voice input means (32);
Voice direction detection means (41) for detecting the direction of arrival of the input voice by the voice input means;
A character conversion device (46) for converting a voice input by the voice input means into a character;
Voice character display means (52) for displaying a voice character at a position corresponding to the voice arrival direction;
Image display means (24);
Monitor direction detection means (26) for detecting the orientation of the image display means;
Control means (30) for determining the display position of the output character of the character conversion device on the screen according to the detection results of the monitor direction detection means and the voice direction detection means;
The image display means superimposes the output character information of the character conversion device on the reproduction image by the moving image reproduction means so that the output character of the character conversion device is displayed at the display position determined by the control means. A mixer (22) for feeding to
Recording means (48, 50) for recording the output character information of the character conversion device and the display position information determined by the control means on the recording medium (12) as related information of the moving image.
A moving picture recording / reproducing apparatus comprising: Furthermore, it comprises means for detecting time information of the moving image being played back by the moving picture playback means,
The recording means records the output character information, the display position information, and the time information of the character conversion device on the recording medium (12) as related information of the moving image. The moving picture recording / reproducing apparatus described.

Images