JP2015070455A - Imaging display device and control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging display device capable of displaying with a sense of presence, and to provide a control method of the same.SOLUTION: The imaging display device comprises: a projector 18 for displaying a display image; a second imaging unit 15 for imaging the face of a user viewing the display image displayed by the projector; a face recognition unit 401 which performs face recognition of an image imaged by the imaging unit 15; and a display control unit 404 which, when the face image of a person corresponding to the face recognized by the face recognition unit 401 is included in the display image, emphasizes and displays the face image in the display image.

Description

  The present invention relates to an imaging display device and a control method thereof.

  Patent Document 1 discloses a moving projector system. The projector system disclosed in Patent Document 1 includes a recording device that records drive signals such as a pan signal and a tilt signal of a camera during imaging. Then, the moving projector is rotated based on the pan signal and the tilt signal.

JP-A-9-149296

  In Patent Document 1, the moving projector rotates according to the angle at the time of imaging. Therefore, an effective presentation can be performed. However, it is desirable to display more realistically when displaying an image.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an imaging display device capable of performing a more realistic display and a control method thereof.

  An imaging display device according to an aspect of the present invention includes a display unit that displays a display image in a moving image format acquired in advance, and an imaging unit that captures the face of a user who is viewing the display image displayed on the display unit A face recognition unit that performs face recognition on a captured image captured by the imaging unit, and a face image of a person corresponding to the face recognized by the face recognition unit is included in the display image, A display control unit that highlights the face image in the display image.

  An imaging display device control method according to an aspect of the present invention includes: a display unit that displays a display image in a moving image format acquired in advance; and a face of a user who is viewing the display image displayed on the display unit A step of performing face recognition on a captured image captured by the imaging unit, and a face image of a person corresponding to the face that has been face-recognized. Is included in the display image, the step of highlighting the face image in the display image is provided.

  According to the present invention, it is an object to provide an imaging display device capable of performing a realistic display and a control method thereof.

It is a side view which shows typically the whole structure of the imaging device which concerns on this embodiment. It is a side view which shows typically the whole structure of the imaging device which concerns on this embodiment. It is a figure which shows typically the structure of the monitor part of an imaging device. It is a block diagram which shows the control system of an imaging device. It is a table for demonstrating control data. It is a figure for demonstrating the control operation | movement of an emphasis display. It is a figure for demonstrating the control operation | movement of an emphasis display. It is a flowchart which shows the control method of a highlight display. It is a figure for demonstrating the control action of the 1st angle control method. It is a figure for demonstrating the control action of the 1st angle control method. It is a figure for demonstrating the control action of the 1st angle control method. It is a figure for demonstrating the control action of the 1st angle control method. It is a flowchart which shows the recording operation of the 1st angle control method. It is a flowchart which shows the reproduction | regeneration operation | movement of the 1st angle control method. It is a figure for demonstrating the control action of the 2nd angle control method. It is a figure for demonstrating the control action of the 2nd angle control method. It is a flowchart which shows the control action of the 2nd angle control method.

(overall structure)
Hereinafter, an imaging device and a control method thereof according to an embodiment of the present invention will be described with reference to the drawings. First, the overall configuration of the imaging apparatus will be described with reference to FIGS. 1 and 2. 1 and 2 are side views showing the overall configuration of the imaging apparatus according to the present embodiment. The imaging device 1 includes a main body unit 11, a monitor unit 12, a first driving mechanism 13, a second imaging unit 15, a projector 18, and a second driving mechanism 19. 1 and 2 show an XYZ orthogonal coordinate system. 1 and 2, the direction parallel to the optical axis of the lens of the first imaging unit (not shown in FIGS. 1 and 2) serving as the main camera is defined as the Z direction. A plane perpendicular to the Z direction is defined as an XY plane. The Y direction is the vertical direction (up and down direction), and the X direction is the horizontal direction (left and right direction). Of course, the above-described directions are relative and change according to the orientation of the imaging device 1.

  The main body 11 includes a housing that houses a first imaging unit (not shown in FIG. 1) that includes a CMOS imaging device and a lens. The first imaging unit, which is the main camera, receives light from the subject and captures moving images and still images. Further, the main body 11 includes a battery, a built-in memory, a memory card slot, a CPU (Central Processing Unit), and the like, not shown. Image data acquired by the first imaging unit is subjected to predetermined processing and stored in a memory. The first imaging unit is arranged facing the front of the imaging device 1. In other words, the first imaging unit is arranged facing the side opposite to the user 21. Therefore, the first imaging unit images the projection plane P side. Note that shooting can be started or stopped by a photographer pressing an operation button or the like.

  A monitor unit 12 is attached to the side surface of the main body 11 so as to be openable and closable around the Y axis. For example, when the monitor unit 12 is in a closed state, the monitor unit 12 is parallel to the side surface of the main body unit 11. The monitor unit 12 has a built-in display such as a liquid crystal monitor, and displays a moving image or a still image acquired by the first imaging unit. The monitor unit 12 includes a display screen 12a on the user 21 side who is an observer. Therefore, the user 21 holding the imaging device 1 can take an image with the first imaging unit while visually recognizing the display screen 12 a of the monitor unit 12. As a result, it is possible to shoot while directing the first imaging unit toward a desired subject. In FIG. 1, the display screen 12a of the monitor unit 12 is parallel to the XY plane.

  Furthermore, the monitor unit 12 is attached to the main body unit 11 via the first drive mechanism 13. The main body 11 supports the monitor unit 12 via the first drive mechanism 13 so as to be rotatable. The first drive mechanism 13 includes an actuator such as a motor. Then, the first drive mechanism 13 rotationally drives the monitor unit 12 around the X axis. The first drive mechanism 13 changes the angle of the monitor unit 12 with respect to the main body unit 11. As a result, the monitor unit 12 rotates in the direction of the arrow with respect to the main body unit 11, and the monitor unit 12 is tilted up and down. Further, the photographer may directly operate the monitor unit 12 to rotate the monitor unit 12. FIG. 2 shows a state where the monitor unit 12 is tilted downward from the state shown in FIG. Thus, by driving the first drive mechanism 13, the angle of the display screen 12a changes.

  Further, the monitor unit 12 is provided with a projector 18. The projector 18 is a small pico projector, for example, and is installed on the back side of the monitor unit 12 opposite to the display screen 12a. That is, the projector 18 is disposed on the opposite side of the monitor unit 12 from the user 21. The projector 18 projects the display image displayed on the monitor unit 12 onto the projection plane P. Therefore, the user 21 can visually recognize the projection image projected on the projection plane P. The projection plane P is an indoor wall surface or screen.

  Note that the monitor unit 12 and the projector 18 can simultaneously display the same display image. For example, the monitor unit 12 and the projector 18 can display the moving image acquired by the first imaging unit. Alternatively, the monitor unit 12 and the projector 18 can display a moving image acquired by another imaging device. In this case, a moving image file acquired by another imaging device is stored in the memory or the like of the imaging device 1. Alternatively, the imaging measure 1 may be provided with a communication function to acquire moving image data captured by another imaging device 1 via a network.

  As described above, the imaging apparatus 1 includes the monitor unit 12 and the projector 18 that can simultaneously display the same image. In other words, the imaging device 1 includes two display units. Of course, the projector 18 may be turned off and display may be performed only by the monitor unit 12. Furthermore, the monitor unit 12 may be turned off and display may be performed only by the projector 18. For example, the projector 18 can be turned ON / OFF by the user pressing an operation button or the like. Thereby, it is possible to select whether the liquid crystal monitor of the monitor unit 12 alone is displayed or the liquid crystal monitor of the monitor unit 12 and the projector 18 are simultaneously displayed.

  The projector 18 is attached to the monitor unit 12 via the second drive mechanism 19. In other words, the monitor unit 12 supports the projector 18 via the second drive mechanism 19 so as to be rotatable. The second drive mechanism 19 rotates the projector 18 around the optical axis of the projector 18. The second drive mechanism 19 rotates the entire projector 19 including the projection lens and the display element. Thereby, the inclination of the projection image projected on the projection plane P can be changed. For example, in the state shown in FIG. 1, the optical axis of the projection lens of the projector 18 and the optical axis of the first imaging unit are parallel. Therefore, in the state shown in FIG. 1, the second drive mechanism 19 rotates the projector 18 around the Z axis. In other words, the second drive mechanism 19 rotates the projector 18 around an axis perpendicular to the projection plane P. Thereby, the projection image projected on the projection plane P can be tilted.

  Further, by driving the first drive mechanism 13, the projector 18 together with the monitor unit 12 rotates around the X axis. Thereby, the projection direction of the projector 18 changes up and down. Therefore, the projection position on the projection plane P can be changed. In FIG. 2, the height of the projected image on the projection plane P changes from the state shown in FIG. Thus, the first drive mechanism 13 changes the orientation of the monitor unit 12 and the projector 18. Further, the first drive mechanism 13 changes the projection direction of the projector 18. A method for controlling the first drive mechanism 13 and the second drive mechanism 19 will be described later.

  The configuration of the monitor unit 12 will be described with reference to FIG. FIG. 3 is a diagram schematically illustrating the configuration of the monitor unit 12. FIG. 3 shows a view of the monitor unit 12 as viewed from the display side. On the display side of the monitor unit 12, a rectangular display screen 12a is provided. The display screen 12a displays an image captured by the first imaging unit. A first drive mechanism 13 is provided on the side of the monitor unit 12. When the first drive mechanism 13 rotates the monitor unit 12, the display direction of the display screen 12a changes. For example, the first drive mechanism 13 tilts the display direction of the display screen 12a up and down.

  Further, the monitor unit 12 is provided with a second imaging unit 15 serving as a sub camera. The second imaging unit 15 is disposed outside the display screen 12a. Here, the second imaging unit 15 is fixed on the display screen 12a. The second imaging unit 15 images the display side of the monitor unit 12. The second imaging unit 15 is provided to capture an image of a photographer who holds the imaging device 1. When the photographer takes a picture while looking at the display screen 12a of the monitor unit 12, the second imaging unit 15 takes an image of the photographer. Note that the optical axis of the lens of the second imaging unit 15 is perpendicular to the display screen 12a. The imaging direction of the second imaging unit 15 changes according to the rotation angle of the monitor unit 12.

  The projector 18 is provided on the surface of the monitor unit 12 opposite to the display screen 12a. Therefore, when the first drive mechanism 13 drives the monitor unit 12, the angle of the projector 18 also changes. In addition, the optical axis of the second imaging unit 15 is parallel to the projector 18.

  A light 16 is provided in the vicinity of the second imaging unit 15. The light 16 includes a light source such as an LED (Light Emitting Diode) and emits illumination light. The illumination light illuminates the subject imaged by the second imaging unit 15.

(Control system)
Next, the control system of the imaging apparatus 1 will be described with reference to FIG. FIG. 4 is a block diagram illustrating a control system of the imaging apparatus 1.

  The central control unit 400 is configured by a semiconductor integrated circuit including a CPU (Central Processing Unit), a ROM (Read Only Memory) in which various programs are stored, a RAM (Random Access Memory) as a work area, and the like. The central control unit 400 can be constituted by a microcomputer. The central control unit 400 comprehensively controls processing of the entire imaging apparatus 1 related to imaging, display of various images, or image processing and image display described below.

  The first imaging unit 14 serving as a main camera includes a zoom lens 101, a focus lens 102, a diaphragm 103, and an image sensor 104. The zoom lens 101 is moved along the optical axis LA by a zoom actuator (not shown). Similarly, the focus lens 102 moves along the optical axis LA by a focus actuator (not shown). The diaphragm 103 operates by being driven by a diaphragm actuator (not shown). The imaging element 104 is configured by a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like.

  The image sensor 104 photoelectrically converts light that has passed through the zoom lens 101, the focus lens 102, and the aperture 103 to generate an analog image signal of the subject. After the analog image signal processing unit 105 amplifies the analog image signal, the image A / D conversion unit 106 converts the amplified signal into digital image data. Then, the image A / D conversion unit 106 outputs the digital image data to the image input controller 107 as a video signal. The image input controller 107 takes in digital image data output from the image A / D conversion unit 106 as captured image data and stores it in the main memory 206 via the bus 200.

  The second imaging unit 15 serving as a sub camera includes a zoom lens 501, a focus lens 502, a diaphragm 503, and an image sensor 504. The zoom lens 501 is moved along the optical axis LB by a zoom actuator (not shown). Similarly, the focus lens 502 is moved along the optical axis LB by a focus actuator (not shown). The diaphragm 503 operates by being driven by a diaphragm actuator (not shown). The image sensor 504 is configured by a CCD, a CMOS, or the like.

  The image sensor 504 photoelectrically converts light that has passed through the zoom lens 501, the focus lens 502, and the stop 503 to generate an analog image signal of the subject. After the analog image signal processing unit 505 amplifies the analog image signal, the image A / D conversion unit 506 converts the amplified signal into digital image data. Then, the image A / D conversion unit 506 outputs the digital image data to the image input controller 507 as a video signal. The image input controller 507 takes in digital image data output from the image A / D conversion unit 506 as captured image data and stores it in the main memory 206 via the bus 200.

  The first imaging unit 14 is a main camera, and the second imaging unit 15 is a sub camera. Therefore, the second imaging unit 15 can have imaging performance lower than that of the first imaging unit 14. For example, the image sensor 504 may have a lower number of pixels than the image sensor 104. Alternatively, the zoom lens 501 and the focus lens 502 may be lenses having lower lens performance than the zoom lens 101 and the focus lens 102.

  Based on a command from the central control unit 400 via the bus 200, the digital signal processing unit 108 takes captured image data stored in the main memory 206, performs predetermined signal processing, and includes a luminance signal and a color difference signal. Generate data. The digital signal processing unit 108 also performs various digital corrections such as offset processing, white balance adjustment processing, gamma correction processing, RGB interpolation processing, noise reduction processing, contour correction processing, color tone correction processing, and light source type determination processing.

  The audio processing unit 202 performs predetermined processing on audio data acquired by a microphone (not shown) or the like according to an instruction from the central control unit 400 via the bus 200. For example, voice data is converted into digital data, or noise removal processing is performed.

  The compression / decompression processing unit 201 performs predetermined compression processing on the captured image data and digital audio data acquired by the first imaging unit 14 according to an instruction from the central control unit 400 via the bus 200 to generate compressed data. To do. Further, the compression / decompression processing unit 201 generates uncompressed data by performing decompression processing of a predetermined format on the compressed data stored in the card type recording medium 302 which is an example of the recording unit in accordance with an instruction from the central control unit 400 To do. As the predetermined compression processing, a compression method based on the JPEG standard can be adopted for still images, and MPEG2 standard or AVC / H. A compression method compliant with the H.264 standard can be employed. The compression / decompression processing unit 201 performs processing on the captured image data acquired by the first imaging unit 14.

  The ROM 203 is connected to the central control unit 400 via the bus 200 and stores a control program executed by the central control unit 400 and various data necessary for control. The flash ROM 204 stores various setting information related to the operation of the imaging apparatus 1, such as user setting information. The imaging apparatus 1 stores in advance in the flash ROM 204 several imaging modes and imaging condition settings according to each mode in accordance with various imaging situations. And before starting imaging, the user can perform imaging under optimal imaging conditions by selecting an optimal mode from each mode. For example, a “portrait mode” suitable for capturing a specific person, a “sport mode” suitable for capturing an athletic meet, or a “night view mode” suitable for capturing a dark place such as a night view. The VRAM 205 is used as a temporary storage area for image data for display.

  The media control unit 207 controls to write data to the card type recording medium 302 via the card I / F 301 and to read data recorded on the card type recording medium 302 in accordance with an instruction from the central control unit 400. The card-type recording medium 302 is an external memory such as an SD card or a compact flash (registered trademark), and is provided so as to be removable from the imaging apparatus 1. The card-type recording medium 302 stores the captured image data of the moving image captured by the first imaging unit 14 in a compressed manner.

  The liquid crystal monitor 304, the speaker 305, the operation unit 306, and the input / output terminal 307 are connected to the input / output I / F 303. The liquid crystal monitor 304 is built in the monitor unit 12. The liquid crystal monitor 304 displays images generated from various image data such as captured image data and various menu image data temporarily recorded in the VRAM 205 or the main memory 206, for example. The liquid crystal monitor 304 displays the moving image acquired by the first imaging unit 14. Alternatively, the liquid crystal monitor 304 displays a moving image recorded on the card type recording medium 302.

  The speaker 305 outputs sound temporarily recorded in the main memory 206, for example. The operation unit 306 includes an operation key including a release switch and a power switch (not shown), a cross key, a joystick, a touch panel superimposed on the liquid crystal monitor 304, and the like. Accept. The input / output terminal 307 is connected to a television monitor (not shown), a PC (Personal Computer), or the like.

  Further, the projector 18 is connected to the input / output I / F 303. The projector 18 displays images generated from various image data such as captured image data and various menu image data temporarily recorded in the VRAM 205 or the main memory 206, for example. The projector 18 displays the moving image acquired by the first imaging unit 14. Alternatively, the projector 18 displays a moving image recorded on the card type recording medium 302.

  The projector 18 projects the same image as the display image displayed on the liquid crystal monitor 304. Therefore, the projector 18 projects a projection image on the projection plane P. The projector 18 adjusts the zoom and focus of the projection image.

  Further, the first drive mechanism 13 is connected to the input / output I / F 303. The first drive mechanism 13 includes a motor, a motor driver, an encoder, and the like. Then, the first drive mechanism 13 rotationally drives the monitor unit 12 in which the liquid crystal monitor 304 is provided. For example, the first drive mechanism 13 rotates the monitor unit 12 based on a command value from the central control unit 400.

  Similarly, the second drive mechanism 19 is connected to the input / output I / F 303. The second drive mechanism 19 includes a motor, a motor driver, an encoder, and the like. The second drive mechanism 19 rotates the projector 18. For example, the second drive mechanism 19 rotates the projector 18 based on a command value from the central control unit 400.

  A light 16 is connected to the input / output I / F 303. The light 16 illuminates a region imaged by the second imaging unit 15. The light 16 is controlled based on a control signal from the central control unit 400. For example, the light 16 is lit when the area imaged by the second imaging unit 15 is dark. Of course, the user may directly set ON / OFF of the light 16 or the like.

  The gyro sensor 208 detects changes in triaxial acceleration and angular velocity of the imaging device 1. By integrating the angular velocity detected by the gyro sensor 208, the angle of the main body 11 can be obtained.

  The illuminance sensor 209 detects the illuminance of the usage environment of the imaging device 1. The illuminance sensor 209 detects the brightness of the room including the projection plane P. Then, the central control unit 400 controls the light 16 according to the detection result of the illuminance sensor 209. For example, when the illuminance detected by the illuminance sensor 209 is lower than the threshold, the central control unit 400 turns on the light 16. Of course, the user may directly set ON / OFF of the light 16 or the like.

  The central control unit 400 includes a face recognition unit 401, an angle data acquisition unit 402, a control data generation unit 403, a display control unit 404, a rotation angle setting unit 405, and a drive control unit 406. The face recognition unit 401 performs face recognition processing on the image acquired by the second imaging unit 15. More specifically, the face recognition unit 401 performs face recognition on a frame image of a moving image.

  Further, the face recognition unit 401 extracts a facial feature amount extracted by face recognition. The face recognition unit 401 identifies a person whose face is recognized based on the feature amount of the face. That is, the face recognizing unit 401 recognizes that the person with the same facial feature amount is the same person. The face recognition unit 401 detects which person is included in the image. The face recognition unit 401 identifies a user included in an image for each frame of a moving image.

  The angle data acquisition unit 402 acquires the angle of the main body 11 during imaging based on the detection result of the gyro sensor 208. For example, the angle data acquisition unit 402 can obtain the angle of the main body 11 by integrating the angular velocity detected by the gyro sensor 208. The angle data acquisition unit 402 acquires a rotation angle of three axes for each frame.

  The control data generation unit 403 generates control data based on the face recognition data and the angle data. For example, the control data generation unit 403 stores face recognition data and angle data for each frame as a table. The control data generation unit 403 stores the control data file in the main memory 206 or the card type recording medium 302, for example. The control data file is stored in association with the acquired moving image file.

  FIG. 5 shows an example of a control data table generated by the control data generation unit 403. Here, control data for a moving image from 1 frame to n frames is stored. Note that the first frame at the start of recording is one frame, and the last frame at the end of recording is n frames. The frame number corresponds to the elapsed time of the moving image.

  In the control data table, face recognition data A, face recognition data B, and angle data are associated with each frame. The face recognition data A is data indicating whether or not the user A is imaged. That is, in the frame indicated as user A in the vertical column of face recognition data A, the face of user A is included in the captured image, and in the frame that is blank, the face of user A is included in the captured image. Not included. In FIG. 5, the face of the user A is captured from the third frame to the 101st frame. The face of the user A is not captured from the 102nd frame to the 200th frame. Further, the face of the user A is captured from the 201st frame to the nth frame. Thus, the face recognition data A is generated depending on whether or not the face of the user A has been detected.

  Similarly, the face recognition data B is data indicating whether or not the user B is imaged. That is, in the frame indicated as user B in the face recognition data B, the face of user B is included in the captured image, and in the frame that is blank, the face of user B is included in the captured image. Not. In FIG. 5, the face of user B is detected from the sixth frame to the 102nd frame. Of course, when the faces of users other than the users A and B are imaged, the face recognition data is also stored for the users. For example, when the faces of other users C and D are imaged, face recognition data is generated for each user.

  As described above, the face recognition unit 401 identifies a user corresponding to a face based on the facial feature amount obtained from the face recognition result. Then, the control data generation unit 403 acquires face recognition data in time series for each user appearing in the captured image. Note that the face recognition data is not limited to the presence or absence of the user in the captured image, and may include the position and inclination of the face image in the captured image.

  The angle data includes, for example, a rotation angle φ around the X axis, a rotation angle θ around the Y axis, and a rotation angle Ψ around the Z axis. The control data table stores (φ, θ, Ψ) for each frame. For example, the angle data of the nth frame is (φn, θn, Ψn). As described above, the control data generation unit 403 generates control data based on the face recognition data and the angle data at the time of imaging.

  Control data is stored as a metafile. Alternatively, the control data may be stored as stream data. Further, the control data may be generated in parallel with imaging, or may be generated after imaging is completed. Moreover, you may make it designate whether a user produces | generates control data.

  Then, the display control unit 404 controls display based on the control data. The display control unit compares the control data with the face recognition data of the captured image being captured by the second imaging unit 15. Then, highlighting is performed according to the comparison result. For example, the display control unit 404 highlights the projection image projected by the projector 18. As described above, the display control unit 404 emphasizes and displays the face of the display image in the moving image format based on the face recognition data of the control data corresponding to the display image. This highlighting will be described later.

  The rotation angle setting unit 405 sets the rotation angle based on the control data. The rotation angle setting unit 405 sets the rotation angle of the first drive mechanism 13 and the second drive mechanism 19 based on, for example, angle data. Then, the drive control unit 406 outputs a command value corresponding to the rotation angle to the first drive mechanism 13 and the second drive mechanism 19. Thereby, the 1st drive mechanism 13 and the 2nd drive mechanism 19 rotate so that it may become the set rotation angle. The control of the first drive mechanism 13 and the second drive mechanism 19 will be described later.

  In the above description, the control data is described as being generated for each frame. However, the control data may be generated for each of a plurality of frames. Or you may produce | generate control data for every predetermined time interval. That is, the time or frame at which the control data is acquired only needs to be associated with the control data. In other words, control data is stored along with the elapsed time of the moving image. Therefore, the angle data and the face recognition data are stored in association with the elapsed time of the moving image. Furthermore, the angle data and the face recognition data may not be acquired at the same time interval. For example, the angle data may be acquired every frame, and the face recognition data may be acquired every two frames.

(Explanation of highlight operation)
Hereinafter, the highlighting operation will be described with reference to FIG. FIG. 6 illustrates a state before highlighting, and FIG. 7 is a diagram schematically illustrating a state where highlighting is performed. In FIGS. 6 and 7, a part of the configuration of the imaging device 1 is omitted. In highlighting, face recognition data of control data is used. 6 and 7 schematically show a state where a moving image file associated with the face recognition data is being reproduced.

  As shown in FIG. 6, the projector 18 projects a projection image onto the projection plane P. Here, the moving image file already recorded on the card type recording medium 302 is reproduced. Hereinafter, a projected image projected by the projector 18 is referred to as a reproduced image 30. The reproduced image 30 includes a user 21a and a user 21b. That is, the projector 18 reproduces a moving image captured including the user 21a and the user 21b as subjects. In this case, as shown in FIG. 5, the control data includes face recognition data A of the user 21a and face recognition data B of the user 21b. Here, the face recognition data included in the control data is set as reference data.

  Further, during display by the projector 18, the second imaging unit 15 images in the direction opposite to the projection direction of the projector 18. That is, the second imaging unit 15 captures an observer (user) who is viewing the reproduced image 30. In FIG. 6, the second imaging unit 15 images the user 21 a and the user 21 c who are viewing the reproduced image 30 behind the imaging device 1. The face recognition unit 401 performs face recognition on the captured image captured by the second imaging unit 15. The face recognition unit 401 recognizes the faces of the user 21a and the user 21c, respectively. The face recognition unit 401 performs face recognition on each frame of the moving image captured by the second imaging unit 15. Thereby, face recognition data of a captured image of the second imaging unit 15 is generated. Here, the face recognition data of the captured image of the second imaging unit 15 is used as comparison data.

  Then, the display control unit 404 compares the reference data with the current comparison data. That is, the face recognition data included in the control data is compared with the face recognition data of the captured image being captured, and a matching face is detected. For example, in the example illustrated in FIG. 6, the face of the user 21 a is included in both the reproduced image 30 and the captured image. Therefore, it is determined that the comparison data of the user 21a being imaged matches the reference data.

  On the other hand, the comparison data of the user 21c being imaged does not match the reference data of the user 21b. That is, the face image of the user 21b is not included in the captured image. Further, the face image of the user 21 c is not included in the reproduced image 30. Therefore, the face recognition data of the user 21b being imaged does not match the face recognition data of the user 21c included in the reference data. Thus, in FIG. 6, only the user 21a matches the reference data with the comparison data.

  Then, the display control unit 404 generates highlighted display data according to the comparison result between the reference data and the comparison data. The projector 18 performs display so as to emphasize the face image of the user whose reference data and comparison data match. Thereby, as shown in FIG. 7, the face of the user 21a is highlighted. For example, in FIG. 7, the graphic 31 for emphasis is superimposed on the face of the user 21a and displayed. The display control unit 404 generates display data by superimposing the graphic 31 on the moving image being reproduced. Then, the projector 18 displays a highlighted display image in which the graphic 31 is superimposed on the original display image. Note that graphics such as frames, balloons, characters, symbols, and the like can be used for the reproduced image 30 as data to be superimposed for strong table display. Alternatively, it can be highlighted by the display color.

  Thus, the projector 18 highlights and displays only the user 21a who is viewing the reproduced image 30. It is easily recognized that the user 21a is included in the reproduced image 30. A graphic 31 is added and displayed to an observer who is currently viewing the reproduced image 30. By doing so, it is possible to display the reproduced image 30 with a more realistic feeling to the observer. Therefore, entertainment property can be enhanced.

  Next, a control method for performing highlight display will be described with reference to FIG. FIG. 8 is a flowchart illustrating a highlighting control method.

  First, when the user selects a moving image file acquired in advance, reproduction by the projector 18 is started. Then, it is determined whether or not the room is bright (step S101). For example, based on the illuminance detected by the illuminance sensor 209, it can be determined whether it is bright or dark. If the room is bright (YES in step S101), the process proceeds to step S106.

  When the room is dark (NO in step S101), the central control unit 400 turns on the light 16 (step S102). When the room is dark, the accuracy of face recognition in the captured image being captured decreases. Therefore, the light 16 is turned on to illuminate the area imaged by the second imaging unit 15. Thereby, the user who visually recognizes the projected image is illuminated, and face recognition can be performed with high accuracy.

  If the light 16 is turned on, it is determined whether or not to pre-register a face (step S103). For example, the user is inquired whether or not to pre-register a face by displaying on the liquid crystal monitor 304 or the projector 18. The user operates the operation unit 306 to determine whether or not to pre-register a face.

  When the operation unit 306 receives an input indicating that a face is pre-registered, a pre-face registration process is performed (step S104). That is, in the state where the light 16 is turned on, the second imaging unit 15 images the face of the user who is an observer. Here, the second imaging unit 15 may capture a still image. Furthermore, you may register the some user who visually recognizes a reproduced image. Then, the face recognition unit 401 performs face recognition on the captured still image. Thereby, face recognition data of the user who is an observer is acquired. Of course, when a plurality of users are registered in advance, each face recognition data is acquired. When a plurality of users are pre-registered, a single still image may be pre-registered, or a face still image may be captured for each user. Note that the image for performing preliminary face registration may be a moving image. Further, the first imaging unit 14 may capture an image for pre-face registration.

  When the user's preliminary face registration is completed, the light 16 is turned off (step S105). Then, the process proceeds to step S106. On the other hand, when the operation unit 306 receives an input indicating that the face is not pre-registered (NO in step S103), the process proceeds to step S106 while the light 16 remains on.

  In step S <b> 106, simultaneously with the start of moving image reproduction by the projector 18, face recognition processing is started on the captured image captured by the second imaging unit 15. Thereby, face recognition data is acquired in real time for the captured image of the second imaging unit 15. If face registration is performed in advance in step S104, real-time face recognition processing can be omitted.

  When face registration is not performed in advance, the face recognition unit 401 performs face recognition on the captured image as needed. In this case, face recognition is performed on a moving image captured in a bright room. Alternatively, face recognition is performed on a moving image captured with the light 16 turned on. Therefore, in any case, face recognition is performed on the captured image captured by the second imaging unit 15 in a bright environment. Thereby, the accuracy of face recognition can be improved.

  Next, face recognition data is compared (step S107). That is, the display control unit 404 compares the comparison data recognized by the face recognition unit 401 with the reference data. Here, when face registration is performed in advance in step S104, the face recognition data acquired in step S104 is used. That is, the face recognition data (comparison data) acquired in step S104 is compared with the reference data. In other words, face recognition is not performed on a moving image acquired in real time while the reproduced image 30 is displayed.

  When face pre-registration is not performed, face recognition data of a captured image captured by the second imaging unit 15 in real time during reproduction of the reproduced image 30 is used. That is, the display control unit 404 compares the comparison data acquired in real time with the reference data. Thereby, a comparison process can be performed according to the observation situation of the user 21 in real time.

  Then, it is determined whether or not the face recognition data match (step S108). That is, it is determined whether or not the reference data matches the user (observer) face comparison data acquired by the second imaging unit 15. If the face recognition data match (YES in step S108), the face image corresponding to the matching face recognition data is highlighted (step S109). For example, as shown in FIG. 7, a graphic 31 or the like is superimposed on the face image of the user 21a and displayed. Thereby, in the reproduced image 30, the face of the user 21 who is viewing the reproduced image 30 is highlighted. Of course, when two or more face recognition data match, each is highlighted. Then, the process proceeds to step S110.

  If the face recognition data do not match (NO in step S108), step S109 is skipped and the process proceeds to step S110. That is, without highlighting any user, the process proceeds to step S110.

  Then, it is determined whether or not the moving image reproduction has ended (step S110). That is, it is determined whether the user has pressed the stop button for moving image reproduction or whether the moving image has been reproduced to the last frame. Then, when the moving image reproduction is finished (YES in step S110), the process is finished. On the other hand, if the moving image reproduction has not ended (NO in step S110), the process returns to step S107 and the process is repeated. That is, the reference data is compared with the comparison data until the moving image reproduction is completed, and highlighting is performed based on the comparison result. Thereby, highlighting is performed for each frame.

  As described above, the reproduced image is highlighted based on the captured image of the second imaging unit 15. That is, the face recognition unit 401 performs face recognition on the captured image of the second imaging unit 15. Then, the face recognition unit 401 compares the comparison data based on the captured image with the reference data based on the reproduced image. When there is a matching face image in the comparison data and the reference data, the display control unit 404 performs highlighting on the matching face image. This makes it possible to reproduce a moving image with a more realistic feeling.

  In the above embodiment, the face recognition data serving as the control data is generated when the moving image is acquired. Then, the face recognition data is stored in association with the reproduced image 30, that is, the moving image file. Thus, by preparing face recognition data (reference data) in advance, the processing can be simplified. Therefore, it is possible to perform processing quickly, and it is possible to promptly emphasize and display the face. Of course, face recognition data may be generated when the display is actually performed. That is, the face recognition unit 401 may generate the reference data during reproduction of the moving image file.

  Further, when the reproduction environment is dark, face recognition is performed on the captured image with the light turned on. Thereby, the accuracy of face recognition can be improved. In particular, indoor lighting is often turned off in a room where the projector 18 is used. Even in this case, the accuracy of face recognition can be improved. Moreover, power consumption can be proposed by turning on the light 16 only when pre-registering. Furthermore, since the room can be darkened, visibility can be improved.

  Furthermore, it is possible to appropriately determine whether or not face information matches by using face recognition data that has been registered in advance. That is, even when the room being reproduced is dark, it can be appropriately processed.

  In addition, during moving image reproduction, it is possible to determine whether or not the faces appropriately match by performing face recognition in real time on the moving image captured by the second imaging unit 15. For example, even when the number of users serving as observers increases or decreases during reproduction, it is possible to appropriately perform highlighting. Therefore, it is possible to display a moving image with a more lasting feeling.

(First angle control method)
Next, a first angle control method using angle data included in the control data will be described with reference to FIGS. 9 to 12 are diagrams for explaining the operation of the first angle control method. 9 and 10 are schematic diagrams illustrating an operation during capturing a moving image, and FIGS. 11 and 12 are schematic diagrams illustrating an operation during playback of the moving image. 9 to 12, a part of the configuration of the imaging device 1 is omitted. In the first angle adjustment method, the first drive mechanism 13 changes the angle of the monitor unit 12.

  First, the operation during imaging of a moving image by the first imaging unit 14 will be described. Here, a case where the angle of the main body 11 is changed during moving image capturing will be described. FIGS. 9 and 10 are diagrams illustrating the imaging device 1 that is capturing images of the users 21 a and 21 b and the display image 40 displayed on the monitor unit 12. 9 and 10 differ in the angle of the main body 11.

  In FIG. 9, the optical axis LA of the first imaging unit 11 is horizontal. That is, the horizontal direction H and the optical axis LA are parallel. At this time, the faces of the users 21 a and 21 b are near the center of the angle of view of the first imaging unit 11. Therefore, in the display image 40 on the liquid crystal monitor of the monitor unit 12, the faces of the users 21a and 21b are near the center.

  FIG. 10 shows a state in which the main body 11 is tilted upward. That is, the optical axis LA of the first imaging unit 11 is inclined. The horizontal direction H and the optical axis LA are not parallel. The angle formed by the horizontal direction H and the optical axis LA is α. The 1st imaging part 11 images diagonally upward. Here, the positions of the users 21a and 21b have hardly changed. Accordingly, the faces of the users 21 a and 21 b move to the lower side of the angle of view of the first imaging unit 11. Therefore, also in the display image 40, the faces of the users 21a and 21b are on the lower side.

  Assume that the user changes the angle of the main body 11 while the first imaging unit 14 is imaging. For example, it is assumed that the user (photographer) changes the state shown in FIG. 10 by turning the main body 11 upward by an angle α from the state shown in FIG. Here, the gyro sensor 208 detects the angular velocity of the main body 11. Based on the angular velocity, the angle data acquisition unit 402 acquires the angle of the main body unit 11. Therefore, the angle data acquisition unit 402 can acquire the angle α of the main body unit 11. Note that the angle α is a rotation angle around the X axis.

  Here, in the moving image file, the elapsed time at the timing when the optical axis LA becomes horizontal as shown in FIG. 9 is T1, and the frame is F1. In the moving image file, the elapsed time at the timing when the optical axis LA is inclined by the angle α as shown in FIG. 10 is T2, and the frame is F2.

  Changes in the rotation angle φ acquired by the angle data acquisition unit 402 are stored as control data. The control data includes angle data for each frame or each elapsed time. Therefore, the elapsed time T1 or the frame F1 is stored in association with φ = 0. Further, the elapsed time T2 or the frame F2 is stored in association with φ = α. Here, φ in the case where the horizontal direction H and the optical axis LA are parallel is set to 0, but the reference direction where φ = 0 is not limited to the horizontal direction H. The angle data acquisition unit 402 stores the time-series angle data in a memory or the like in association with the moving image file of the captured image.

  When reproducing the moving image file associated with the angle data, the first drive mechanism 13 controls the angle of the monitor unit 12 according to the angle data. Thereby, the projection direction of the projector 18 changes. For example, at the timing of displaying the elapsed time T1 (frame F1) during reproduction, the projector 18 projects the reproduced image 30 with the projection direction of the projector 18 being horizontal as shown in FIG. That is, the optical axis LC of the projection lens of the projector 18 is parallel to the horizontal direction H.

Next, at the timing of displaying the elapsed time T2 (frame F2) during reproduction, as shown in FIG. 12, the reproduced image 30 is projected with the optical axis LC of the projector 18 inclined by an angle α from the horizontal direction. That is, the projection direction of the projector 18 is controlled according to the angle data included in the control data. Here, based on the angle data phi _F2 around the X-axis, the first drive mechanism 13 is controlling the angle of the monitor 12. The first drive mechanism 13 rotates the monitor unit 12 around the X axis as described in FIG. Thereby, the height of the reproduced image 30 can be changed. In FIG. 10, since the image is taken with the main body 11 facing upward, the position of the reproduced image 30 is high as shown in FIG.

  As described above, the first drive mechanism 13 rotates the monitor unit 12 based on the angle data acquired at the time of imaging. Thereby, since a projection direction changes according to the angle data at the time of imaging, a sense of reality can be made higher. In FIGS. 11 and 12, the optical axis of the first imaging unit 14 is parallel to the horizontal direction H.

  In the above description, the angle data at the time of imaging and the tilt angle in the projection direction are set to the same value, but may be set to different values. That is, the angle of the main body 11 at the time of imaging and the angle of the monitor unit 12 at the time of reproduction may not match. For example, the angle of the monitor unit 12 may be changed by an angle larger or smaller than the change of the angle data. Furthermore, the direction serving as the reference direction may not be parallel to the horizontal direction H, and may be set according to the installation angle of the main body 11. For example, when the main body 11 is installed inclined from the horizontal direction, the inclination angle may be φ = 0. In addition, it is preferable that the rotation axis serving as the reference for the angle data coincides with the rotation axis that tilts the projection direction. For example, it is preferable to rotate the monitor unit 12 or the projector 18 around the X axis based on the angle data φ around the X axis.

  In this way, the projection position of the reproduced image 30 can be moved on the projection plane P in accordance with the movement in the imaging direction at the time of imaging. Thereby, a wide space can be used effectively and a display with a high sense of reality can be performed.

  Next, processing in the first angle adjustment method will be described with reference to FIGS. 13 and 14. 13 and 14 are flowcharts for explaining the first angle adjustment method. FIG. 13 shows processing during imaging (recording), and FIG. 14 shows processing during reproduction.

  First, the processing during recording will be described with reference to FIG. Recording of the first imaging unit 14 is started by the user pressing a recording start button or the like. Then, the angle data acquisition unit 402 tries to acquire data from the gyro sensor 208 (step S201). Then, the angle data acquisition unit 402 determines whether data can be acquired (step S202). When data cannot be acquired from the gyro sensor 208 (NO in step S202), the process returns to step S201 and tries to acquire data again.

  When the angle data acquisition unit 402 can acquire data from the gyro sensor 208 (YES in step S202), the elapsed time and angle data are stored (step S203). Specifically, the angle data acquisition unit 402 calculates the angle of the main body 11 from the angular velocity data detected by the gyro sensor 208. Then, the control data generation unit 403 stores the angle data in association with the elapsed time (frame number).

  Then, after saving the elapsed time and the angle data, it is determined whether or not the recording of the first imaging unit 14 is stopped (step S204). If the recording is stopped (YES in step S204), the process is terminated. That is, when the user stops recording by pressing the recording stop button or the like, the recording process ends.

  If the recording is not stopped (NO in step S204), the process returns to step S201 and the process is repeated. That is, data is acquired from the gyro sensor 208 and angle data is stored. By repeating this process until the recording is stopped, the angle data from the recording start to the recording stop is stored. The angle data is associated with the elapsed time, that is, the frame. Therefore, angle data in each frame is stored. In this manner, the control data generation unit 403 generates angle data along the time series as control data. For example, the control data generation unit 403 generates a metafile in which the angle data is associated with the elapsed time. The metafile is recorded in the main memory 206 or the card type recording medium 302 together with the moving image file.

  Next, processing during reproduction will be described with reference to FIG. First, the user designates the moving image file acquired as described above and presses the play button. Then, the compression / decompression processing unit 201 decodes the moving image file and starts moving image reproduction (step S301). Thereby, the liquid crystal monitor 304 of the monitor unit 12 starts displaying the moving image file. Next, it is determined whether the output of the projector 18 is ON (step S302). For example, it is determined whether simultaneous display of the liquid crystal monitor 304 and the projector 18 is performed.

  When the output of the projector 18 is not ON (NO in step S302), the liquid crystal monitor 304 performs normal reproduction (step S303). That is, when the output of the projector 18 is OFF, the moving image file is reproduced only by the liquid crystal monitor 304. In normal reproduction, the first drive mechanism 13 does not change the angle.

  When the normal reproduction is performed, it is determined whether or not the reproduction is stopped (step S304). If reproduction has not stopped (NO in step S304), the process returns to step S303 to continue normal reproduction. When the reproduction is stopped (YES in step S304), the liquid crystal monitor 304 ends the reproduction. When the user presses the playback stop button or the like or plays back to the last frame of the moving image file, playback stops. As described above, when the projector 18 is OFF, the normal reproduction is continued until the reproduction is stopped. That is, when the projector 18 is not displaying, normal playback is performed with the angle of the liquid crystal monitor 304 fixed.

  On the other hand, when the output of the projector 18 is ON (YES in step S302), video output is performed from the projector 18 (step S305). As a result, the reproduced image is projected onto the projection plane P. Note that the user can set ON / OFF of the projector 18. Then, it is determined whether or not there is angle data for the current reproduction time (elapsed time) (step S306). For example, the rotation angle setting unit 405 refers to angle data included in the control data.

  If there is angle data for the current playback time (YES in step S306), angle data corresponding to the current playback time is acquired (step S307). That is, the rotation angle setting unit 405 acquires angle data corresponding to the elapsed time. Then, based on this angle data, the first drive mechanism 13 changes the angle of the monitor unit 12 (step S308). That is, the rotation angle setting unit 405 calculates the rotation angle of the first drive mechanism 13 based on the elapsed time angle data. Then, the drive control unit 406 calculates a command value corresponding to the rotation angle and outputs the command value to the first drive mechanism 13. Thereby, the first drive mechanism 13 changes the angle of the monitor unit 12. The orientation of the projector 18 attached to the back side of the monitor unit 12 also changes. Thereby, the projection direction of the projector 18 changes and the projection position changes. And if the angle of the monitor part 12 is changed, it will transfer to step S309.

  If there is no angle data for the current playback time (NO in step S306), steps S307 and S308 are skipped and the process proceeds to step S309. That is, the process proceeds to step S309 without changing the angle of the monitor unit 12.

  When the angle control of the monitor unit 12 is finished, it is determined whether or not the reproduction is stopped (step S309). When the user stops the reproduction by pressing the reproduction button or the like (YES in step S309), the reproduction process is terminated. When the reproduction is not stopped (NO in step S309), the process returns to step S307 and the process is repeated. As a result, the angle of the monitor unit 12 changes according to the time-series angle data. That is, the angle of the monitor unit 12 changes according to the reproduction time (elapsed time).

  The first drive mechanism 13 changes the tilt angle of the projector 18 according to the tilt angle of the main body 11. By doing so, the projection position of the reproduced image 30 on the projection plane P changes. Therefore, it is possible to display a more realistic image. That is, the reproduced image 30 is displayed reflecting the angle of the imaging device 1 at the time of imaging. For this reason, an observer (user) can experience a sense of reality at the time of imaging. Furthermore, the angle of the monitor unit 12 is changed according to the elapsed time. That is, the first drive mechanism 13 adjusts the angle according to the frame. Therefore, the space can be fully used for reproduction, and a more realistic image can be displayed. When the projection position is changed, image settings such as keystone correction may be automatically performed.

(Second angle control method)
Next, a second angle control method using the second drive mechanism 19 will be described. Here, the second drive mechanism 19 rotates the projector 18 in accordance with the angle data included in the control data. FIG. 15 is a diagram schematically showing a state before the angle control by the second drive mechanism 19, and FIG. 16 is a diagram schematically showing a state after the angle control. 15 and 16, the vertical direction (longitudinal direction) in the projection plane P is the Y direction, and the horizontal direction is the X direction. Further, the vertical direction of the reproduced image 30 is indicated as Y ′, and the horizontal direction is indicated as X ′. The horizontal direction in the reproduced image 30 is H ′.

  Here, control when the main body 11 rotates around the optical axis LA during imaging will be described. When the main body 11 rotates around the optical axis LA, as shown in FIG. 15, the users 21a and 21b included in the reproduced image 30 are displayed tilted. That is, when the user 21a and 21b standing in the vertical direction is imaged, the user 21a and 21b in the reproduced image 30 is captured in the vertical direction of the reproduced image 30 when the main body 11 is rotated. It will tilt from Y '. As described above, when the main body 11 rotates unintentionally due to an operation mistake of the photographer, the level of the reproduced image 30 cannot be maintained. The horizontal direction H ′ in the reproduced image 30 is inclined from the horizontal direction H in the real space. In this case, it becomes difficult for the observer to see the reproduced image 30.

  Therefore, in the present embodiment, the second drive mechanism 19 rotates the projector 18 based on the control data. For example, the second driving mechanism 19 rotates the projector 18 around the optical axis LC based on the angle data Ψ around the Z axis included in the control data. As a result, as shown in FIG. 16, the reproduced image 30 projected by the projector 18 is displayed in an inclined manner. The vertical direction Y ′ of the reproduced image 30 is inclined from the vertical direction Y of the projection plane P. Then, the users 21 a and 21 b in the reproduced image 30 are displayed in a state of standing along the vertical direction Y of the projection plane P. That is, the vertical direction of the real space at the time of imaging coincides with the vertical direction of the real space at the time of reproduction. In other words, the horizontal direction H ′ in the reproduced image 30 and the horizontal direction H in the real space are parallel to each other.

  In this way, the subject in the reproduced image 30 can be kept level at all times. Therefore, even a moving image file that has been imaged at an angle during imaging can be displayed easily. Thus, in the present embodiment, the angle of the reproduced image 30 is rotated within the projection plane P with the center of the reproduced image 30 as the rotation axis.

  FIG. 17 shows a flowchart of the second angle control method. First, the user designates a moving image file and presses the play button. Then, the compression / decompression processing unit 201 decodes the moving image file and starts moving image reproduction (step S401). Thereby, the liquid crystal monitor 304 of the monitor unit 12 starts displaying the moving image file. Next, it is determined whether or not there is angle data (step S402). That is, it is determined whether or not angle data corresponding to the moving image file exists.

  If there is no angle data associated with the moving image file (NO in step S402), the process proceeds to step S404. If angle data associated with the moving image file exists (YES in step S402), it is determined whether the output of the projector 18 is ON (step S403). For example, it is determined whether simultaneous display of the liquid crystal monitor 304 and the projector 18 is performed. When the output of the projector 18 is not ON (NO in step S403), the process proceeds to step S404.

  If there is no angle data (NO in step S402), or if the projector 18 is not ON (NO in step S403), the liquid crystal monitor 304 performs normal reproduction (step S404). Since the reproduced image 30 cannot be rotated in a moving image file having no angle data, the projector 18 is not used. When the output of the projector 18 is OFF, the moving image file is reproduced only by the liquid crystal monitor 304.

  If normal playback is performed in step S404, it is determined whether or not playback has been stopped (step S405). If reproduction has not stopped (NO in step S405), the process returns to step S404 to continue normal reproduction. When the reproduction is stopped by the user pressing the reproduction stop button or the like (YES in step S405), the liquid crystal monitor 304 ends the reproduction. As described above, when there is no angle data, or when the projector 18 is OFF, the normal reproduction is continued until the reproduction is stopped. That is, when the projector 18 is not displaying, normal playback is performed with the angle of the liquid crystal monitor 304 fixed.

  On the other hand, when the output of the projector 18 is ON (YES in step S403), angle data is acquired (step S406). For example, the rotation angle setting unit 405 refers to angle data included in the control data.

  And according to the acquired angle data, the 2nd drive mechanism 19 inclines the projector 18 (step S407). That is, the rotation angle setting unit 405 calculates the rotation angle of the projector 18 based on the elapsed time angle data. Then, the drive control unit 406 calculates a command value corresponding to the rotation angle and outputs the command value to the second drive mechanism 19. Thereby, the angle of the projector 18 changes. The projector 18 rotates around its optical axis LC, and the projection image by the projector 18 is inclined on the projection plane P. When the projector 18 is tilted, the process proceeds to step S408.

  When the angle control of the projector 18 is finished, it is determined whether or not the reproduction is stopped (step S408). If the reproduction is stopped by the user pressing the reproduction button or the like (YES in step S408), the reproduction process is terminated. When the reproduction is not stopped (NO in step S408), the process returns to step S406 and the process is repeated. Thereby, the angle of the projector 18 changes according to the angle data. Therefore, the angle of the projector 18 changes according to the elapsed time.

  The second drive mechanism 19 changes the tilt angle of the projector 18 according to the tilt angle of the main body 11. By doing so, the reproduced image 30 on the projection plane P is inclined. Therefore, easy-to-see display is possible. That is, since the reproduced image 30 is displayed reflecting the angle at the time of imaging, the level of the subject in the reproduced image 30 is kept constant. Further, the angle of the projector 18 is changed according to the elapsed time. That is, the second drive mechanism 19 adjusts the angle according to the frame. Therefore, the subject in the reproduced image 30 is always kept horizontal. When the projection position is changed, image settings such as keystone correction may be automatically performed.

  In the second angle control method, the angle data is calculated based on the detection result of the gyro sensor 208, but the present embodiment is not limited to this. For example, angle data may be calculated using face recognition data. That is, the face tilt data calculated when face recognition is performed is used as angle data. That is, the inclination of the projector 18 may be changed according to the inclination of the face in the reproduced image 30. Further, when there is both angle data based on the face recognition data and angle data based on the gyro sensor 208, the angle data of the face recognition data may be used with priority, or the angle data based on the gyro sensor 208 may be used with priority. May be.

  Further, although the highlighting, the first angle control, and the second angle control have been described, it is sufficient that at least one of them is performed. For example, only highlighting is performed, and the first and second angle controls may not be performed. In this case, only the face recognition data is necessary control data. Therefore, only data necessary for control may be generated as control data during imaging.

  Furthermore, two or more processes of highlighting, first angle control, and second angle control may be performed in combination. For example, the first angle control or the second angle control may be performed simultaneously with the highlight display. Further, the highlighting and the first and second angle control may be performed simultaneously.

  The place where the first drive mechanism 13 and the second drive mechanism 19 are arranged is not particularly limited. For example, the second drive mechanism 19 may rotate the monitor unit 12 around the Z axis. Further, not only the rotation angle around the X axis or the Z axis but also the rotation angle around the Y axis may be controlled. In this case, a drive mechanism for rotating the projector 18 around the Y axis is prepared. Then, by rotating the projector 18 around the Y axis, the reproduced image 30 changes along the horizontal direction, that is, the X direction, within the projection plane P.

  In the above description, the first imaging unit 14 has been described as capturing a reproduced image. However, an imaging device that captures a reproduced image may be another imaging device. That is, the moving image file associated with the control data may be acquired by another imaging device. In this case, control data is stored in the imaging device 1 together with the acquired moving image file. Then, the imaging device 1 reproduces the moving image file. In this way, control may be performed on a display image in a moving image format acquired in advance.

  Part or all of the above processing may be executed by a computer program. The above-described program can be stored and supplied to a computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)) are included. The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

  As mentioned above, the invention made by the present inventor has been specifically described based on the embodiment. However, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the scope of the invention. Needless to say.

DESCRIPTION OF SYMBOLS 1 imaging device, 11 main-body part, 12 monitor part, 13 1st drive mechanism 14 1st imaging part, 15 2nd imaging part, 18 projector,
19 Second drive mechanism

Claims (6)

A display unit for displaying a display image in a moving image format acquired in advance;
An imaging unit that images the face of the user who is viewing the display image displayed by the display unit;
A face recognition unit that performs face recognition on a captured image captured by the imaging unit;
An imaging display apparatus comprising: a display control unit that highlights the face image in the display image when a face image of a person corresponding to the face recognized by the face recognition unit is included in the display image.   The imaging display apparatus according to claim 1, wherein face recognition data of a face included in the display image is stored in association with an elapsed time of the display image. The display unit includes a projector that projects the display image onto a projection plane;
A drive mechanism for changing the tilt angle of the projector is provided,
The imaging display device according to claim 1, wherein the drive mechanism is controlled based on angle data associated with the moving image file of the display image.   The imaging display device according to claim 3, wherein the display image on the projection plane is rotated by the drive mechanism changing the tilt angle of the projector based on angle data detected when the display image is captured.   5. The projection position of the display image on the projection plane is changed by the drive mechanism changing the tilt angle of the projector based on angle data detected when the display image is captured. The imaging display device described. A display unit for displaying a display image in a moving image format acquired in advance;
An imaging unit comprising: an imaging unit that captures a face of a user who is viewing a display image displayed on the display unit;
Performing face recognition on a captured image captured by the imaging unit;
And a step of highlighting the face image in the display image when a face image of a person corresponding to the face recognized is included in the display image.

Images