JP2006339710A - IMAGING DEVICE, IMAGING DEVICE CONTROL METHOD, IMAGING DEVICE CONTROL METHOD PROGRAM, AND RECORDING MEDIUM CONTAINING IMAGING DEVICE CONTROL METHOD PROGRAM - Google Patents

Abstract

The present invention relates to an image pickup apparatus, an image pickup apparatus control method, an image pickup apparatus control method program, and a recording medium on which an image pickup apparatus control method program is recorded. Thus, it is possible to provide an easy-to-use user interface with a simpler operation.
According to the present invention, when a process related to recording of an imaging result on a recording medium is not executed, a motion vector of an imaging result by an imaging element is detected by a data compression unit, and the motion of the apparatus is detected based on the motion vector. Is detected and the operation is switched.
[Selection] Figure 1

Description

  The present invention relates to an imaging apparatus, an imaging apparatus control method, an imaging apparatus control method program, and a recording medium on which an imaging apparatus control method program is recorded, and can be applied to, for example, an electronic still camera. In the present invention, when a process related to recording of an imaging result to a recording medium is not performed, a data compression unit detects a motion vector of an imaging result by the imaging device, and detects a motion of the apparatus based on the motion vector. By switching the operation, it is possible to provide an easy-to-use user interface with a simpler operation than before.

  2. Description of the Related Art Conventionally, an imaging apparatus such as an electronic still camera is provided with various functions for user convenience, and various user interfaces have been devised so that these functions can be used easily and reliably.

  Here, as this type of function, there are optical zoom, electronic zoom, reproduction zoom, index display by thumbnail images, and the like. The reproduction zoom is electronic zoom processing at the time of reproducing the imaging result recorded on the recording medium. The index display by the thumbnail image is a function of separately recording and holding the imaging result as a thumbnail image and displaying the thumbnail image for the index of the imaging result. For example, Japanese Patent Laid-Open No. 2004-88352 discloses this thumbnail image. Various acquisition methods have been proposed.

  For such functions, the imaging apparatus is provided with various controls such as zoom-up and zoom-down, and further, a touch panel is provided on the liquid crystal display panel. Is configured to accept.

However, in recent years, imaging devices have been further reduced in size and functionality, and thus a user interface that is easier to use than ever is required. Of the conventional user interfaces, a user interface using a touch panel can improve operability by increasing the size of the liquid crystal display panel and the touch panel, but has drawbacks such as damage to the liquid crystal display panel and deterioration of visibility.
JP 2004-88352 A

  The present invention has been made in consideration of the above points, and is capable of providing an easy-to-use user interface with a simpler operation than conventional ones, an imaging device control method, and an imaging device control method The present invention intends to propose a recording medium on which the program and the program for controlling the imaging apparatus are recorded.

  In order to solve this problem, the invention of claim 1 is applied to an imaging apparatus, and outputs an imaging result of an optical image formed on an imaging surface, and a lens that creates the optical image on the imaging surface. Detecting a motion vector of an imaging result output from the imaging device, and data compressing the imaging result by the motion vector, and recording the imaging result compressed by the data compression unit on a recording medium A recording / reproducing unit that reproduces the imaging result recorded on the recording medium, a display unit that displays the imaging result reproduced by the recording / reproducing unit, and a control unit that controls the overall operation. Is configured to operate the image sensor and the data compression unit when the processing related to the recording of the imaging result output from the image sensor to the recording medium is not performed. Acquired by detecting more motion vectors, the motion vector, detects the movement of the device, based on the motion of the device, it switches the operation.

  According to a fourteenth aspect of the present invention, there is provided an imaging device that outputs an imaging result of an optical image formed on the imaging surface, a lens that creates the optical image on the imaging surface, and a movement of the imaging result output from the imaging device. A data compression unit that detects a vector and compresses the imaging result using the motion vector, records the imaging result compressed by the data compression unit on a recording medium, and reproduces the imaging result recorded on the recording medium Applied to a control method of an image pickup apparatus comprising a recording / playback unit, a display unit for displaying the imaging result reproduced by the recording / playback unit, and a control unit for controlling the overall operation, and output from the image pickup device When the processing related to the recording of the imaging result to be recorded on the recording medium is not executed, the motion vector is detected from the imaging result of the imaging device by operating the imaging device and the data compression unit. A detection step of motion vector obtained by, the motion vector, a motion detecting step of detecting a movement of the device, based on the motion of the device, to have a control step of switching the operation.

  According to a fifteenth aspect of the present invention, there is provided an imaging device that outputs an imaging result of an optical image formed on the imaging surface, a lens that creates the optical image on the imaging surface, and a movement of the imaging result output from the imaging device. A data compression unit that detects a vector and compresses the imaging result using the motion vector, records the imaging result compressed by the data compression unit on a recording medium, and reproduces the imaging result recorded on the recording medium The imaging device is applied to a program for a control method of an imaging apparatus, comprising: a recording / reproducing unit; a display unit that displays the imaging result reproduced by the recording / reproducing unit; and a control unit that controls the overall operation. When the processing related to recording of the imaging result output from the recording medium to the recording medium is not executed, the imaging element and the data compression unit are operated to move from the imaging result of the imaging element. A motion vector detecting step for detecting and obtaining a vector, a motion detecting step for detecting the motion of the device based on the motion vector, and a control step for switching the operation based on the motion of the device. .

  The invention of claim 16 is applied to a recording medium on which a program for controlling the imaging apparatus is recorded, and the program for controlling the imaging apparatus according to claim 15 is recorded.

  According to the configuration of claim 1, an imaging device that is applied to an imaging device and outputs an imaging result of an optical image formed on an imaging surface, a lens that creates the optical image on the imaging surface, and the imaging device A data compression unit that detects a motion vector of an imaging result output from the data and compresses the imaging result using the motion vector; and records the imaging result compressed by the data compression unit on a recording medium; and A recording / reproducing unit that reproduces the imaging result recorded in the recording unit, a display unit that displays the imaging result reproduced by the recording / reproducing unit, and a control unit that controls the overall operation. When the processing related to the recording of the imaging result output from the element to the recording medium is not executed, the image sensor and the data compression unit are operated, and the motion vector is obtained from the imaging result of the imaging element. Detected and acquired by the motion vector, detects the movement of the device, based on the motion of the device, by switching the operation, it is possible to switch the operation by the movement of the device. As a result, the display of a cursor or the like can be switched so as to move the device and respond to the movement of the device, and a user interface that is easier to use than the conventional one can be provided.

  Thus, according to the configurations of claims 14, 15, and 16, an imaging apparatus control method and an imaging apparatus control that can provide a user interface that is easier to use than conventional ones can be provided. It is possible to provide a recording medium in which a program for a method and a program for a method for controlling an imaging apparatus are recorded.

  According to the present invention, it is possible to provide a user interface that is easier to use than ever with a simpler operation.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.

(1) Configuration of Embodiment FIG. 2 is a block diagram showing an electronic still camera according to an embodiment of the present invention. The electronic still camera 1 acquires the imaging results of still images and moving images and records them on the recording medium 19, and reproduces the imaging results recorded on the recording medium 19 to display a monitor image.

  That is, in this electronic still camera 1, the lens block (lens) 2 collects incident light on the image pickup surface of the image pickup device 4 that changes the zoom lens, the focus, and the diaphragm under the control of a camera DSP (Digital Signal Processor) unit 3. As a result, an optical image is formed on the imaging surface of the imaging device 4.

  The image pickup device 4 is a CCD (Charge Coupled Device) solid-state image pickup device, a CMOS (Complementary Metal-Oxide Semiconductor) solid-state image pickup device or the like, and photoelectrically converts an optical image formed on the image pickup surface by driving the preprocessing unit 5. An imaging signal based on the imaging result is output.

  The preprocessing unit 5 drives the image sensor 4 and also performs signal processing on an image signal output from the image sensor 4 to output image data.

  That is, in the pre-processing unit 5, the timing generator (TG) 6 generates and outputs various timing signals that are driving references for the image sensor 4, and the V driver 7 uses the timing signal generated by the timing generator 6. The image sensor 4 is driven.

  The CDS / AGC / ADC unit 8 performs correlated double sampling (CDS: Correlated Double Sampling) on the imaging signal output from the imaging element 4, thereby generating, for example, a primary color system color signal. Further, the CDS / AGC / ADC unit 8 corrects the signal level of the color signal by automatic gain control (AGC), and then generates image data by performing analog-to-digital conversion processing (ADC: Analog to Digital Converter). Then, this image data is output to the camera DSP unit 3.

  The camera DSP unit 3 switches its operation under the control of a central processing unit (CPU) 20, and uses the memory 10 to perform processing related to recording / reproduction of image data output from the preprocessing unit 5, monitoring, and the like. Execute. Thus, the memory 10 forms a working area for the camera DSP unit 3.

  That is, in the camera DSP unit 3, the camera signal processing unit 11 performs white balance adjustment processing, gamma correction processing, and the like on the image data output from the preprocessing unit 5, and processes the processed image data via the memory control unit 12. And output to the resolution conversion unit 14. The camera signal processing unit 11 detects information necessary for aperture correction and autofocus adjustment from the image data, and outputs the detected information to the lens control unit 13.

  Here, the memory control unit 12 is a control unit that controls the memory 10 external to the camera DSP unit 3, and records image data output from each unit of the camera DSP unit 3 in the memory 10. Are read out and output to each part of the camera DSP unit 3.

  The resolution conversion unit 14 converts the resolution of the image data stored in the memory 10 or the image data output from the camera signal processing unit 11 and stores it in the memory 10. As a result, the camera DSP unit 3 converts the imaging result obtained from the imaging device 4 into a resolution for display on the liquid crystal display panel 16 and also converts it into a resolution designated by the user on the recording medium 19. In addition, a part of the imaging result is cut out and processing relating to the resolution conversion processing is executed, thereby executing processing of electronic zoom and reproduction zoom.

  The display control unit 15 drives a liquid crystal display panel (LCD: Liquid Crystal Display) 16 by image data stored in the memory 10, thereby displaying a monitor image of the imaging result and imaging result recorded on a recording medium. Is displayed.

  The image codec 17 compresses the image data stored in the memory 10 and outputs the compressed data to each part of the camera DSP unit 3. On the contrary, the image codec 17 decompresses the image data stored in the memory 10 and performs the camera DSP unit. 3 to each part. This data compression / decompression uses JPEG (Joint Photographic Coding Experts Group) for still images, and ISO / IEC JTC1 / SC29WG11 MPEG1 (Motion Picture Experts Group 1) for moving images. Data compression methods using motion vectors such as MPEG2 (Motion Picture Experts Group 2), MPEG4 (Motion Picture Experts Group 4), ITU-T H.263, H.264 / MPEG4-AVC (Advanced Video Coding) Is done.

  That is, FIG. 3 is a functional block diagram showing the configuration of the data compression unit related to the moving image processing of the image codec 17. The image codec 17 inputs image data D1 based on the imaging result to the motion detection unit 17A and inputs to the orthogonal transform unit 17C via the subtraction unit 17B when moving image data is compressed. Here, the motion detection unit 17A uses the image data of the reference image recorded in the frame memory 17D in the P picture and the B picture, and calculates the motion vector V of the image data D1 for each macroblock that is a motion vector detection unit. To detect. The motion compensation unit 17E performs motion correction on the image data of the reference image using the detected motion vector V, thereby generating predicted image data using the motion vector V and outputs the predicted image data to the subtraction unit 17B.

  The subtraction unit 17B outputs the input image data D1 as it is to the orthogonal transformation unit 17C for the I picture, while the P picture and the B picture are compared with the predicted image data output from the motion compensation unit 17E. The difference data is output to the orthogonal transform unit 17C.

  The orthogonal transform unit 17C generates coefficient data by performing orthogonal transform processing on the image data and difference data output from the subtraction unit 17B, and the quantization unit 17F quantizes and outputs the coefficient data. The variable length encoding unit 17G performs variable length encoding processing on the output data of the quantization unit 17F, and then multiplexes the output data with the encoding control information based on the motion vector V detected by the motion detection unit 17A to perform streaming data D2. Is output.

  The inverse quantization unit 17H performs an inverse quantization process on the output data of the quantization unit 17F and decodes the input data of the orthogonal transform unit 17C. The inverse orthogonal transform unit 17I performs inverse orthogonal transform processing on the output data of the inverse quantization unit 17H, and decodes the output data of the subtraction unit 17B. The adder 17J outputs the output data of the inverse orthogonal transform unit 17I as it is for the I picture, and the predicted image data output from the motion compensation unit 17E and the output of the inverse orthogonal transform unit 17I for the P picture and the B picture. Add the data and output. Thereby, the adder 17J decodes the input data of the subtractor 17B to generate image data of the reference image, and the frame memory 17D holds and outputs the image data of the reference image.

  Thus, the image codec 17 performs data compression processing on the image data D1 based on the moving image by motion prediction using the motion vector V. When the image codec 17 switches the operation of the data compression unit under the control of the central processing unit 20 and does not execute processing related to recording of the imaging result output from the imaging device 4 to the recording medium 19, the imaging device 4 For the image data D1 obtained, the output of the streaming data D2 is stopped, and the motion vector V is detected by setting the immediately preceding frame as the reference frame for each macro block in all the pictures. The detected motion vector V Is sent to the central processing unit 20. In AVC and the like, when data is compressed, motion vectors are detected by macroblocks of different sizes and encoded in the optimum mode, and in the case of AVC and the like, only a macroblock of a specific size is moved. Detect and notify the vector. In this motion vector detection process, the motion vector may be detected by directly setting the image data D1 as a reference frame instead of the reference frame decoded by the adder 17J.

  Note that the case where the processing relating to the recording of the imaging result output from the imaging device 4 to the recording medium 19 is not executed is the case where the image codec 17 does not necessarily process the image data. In the example, the case where the imaging result recorded on the recording medium 19 is monitored is applied. Thus, the image codec 17 can detect the motion of the electronic still camera 1 using the motion vector V used for data compression processing when monitoring the imaging result recorded on the recording medium 19.

  The media control unit 18 records the image data stored in the memory 10 on the recording medium 19, reads out the image data recorded on the recording medium 19, and outputs it to the memory 10. Here, as the recording medium 19, various recording media such as a so-called memory card using a semiconductor memory, a recordable DVD (Digital Versatile Disk), an optical disc using a recordable CD (Compact Disc), and the like can be widely applied. it can.

  The lens control unit 13 controls the aperture of the lens block 2 based on information obtained from the camera signal processing unit 11, and performs autofocus control of the lens block 2 using a so-called hill climbing method. Further, the zoom of the lens block 2 is varied in response to an operation by the user under the control of the central processing unit 20.

  A BIU (Bus Interface Unit) 21 is an interface between the central processing unit 20 and the camera DSP unit 3, and inputs and outputs commands related to the control of the camera DSP unit 3 by the central processing unit 20.

  The operation unit 22 is a user interface using an operator provided on the electronic still camera 1 and a touch panel provided on the display screen of the liquid crystal display panel 16, and notifies the central processing unit 20 of an operation by the user.

  A USB (Universal Serial Bus) 23 is an interface corresponding to a storage class, which connects an external device such as a personal computer and inputs / outputs various data to / from the external device.

  An EEPROM (Electrically Erasable Programmable Read-Only Memory) 24 is a recording medium that records a processing program of the central processing unit 20, and a RAM 25 is a work memory of the central processing unit 20. In this electronic still camera 1, the processing program of the central processing unit 20 is provided in advance by being installed in the electronic still camera 1, but instead of provision by such prior installation. It may be provided by installation via a network such as the Internet, or may be provided by various recording media such as a memory card, an optical disk, and a magnetic disk.

  The central processing unit 20 is a control unit that controls the operation of the electronic still camera 1, and controls the operation of each unit in response to the operation of the operation unit 22 by the user by executing the processing program recorded in the EEPROM 24.

  That is, when photographing is instructed by operating the operation unit 22, the central processing unit 20 acquires the imaging result of the moving image from the image sensor 4 by controlling the operations of the preprocessing unit 5 and the camera DSP unit 3, and the preprocessing unit 20 5 and sequentially stores the imaging results in the memory 10 via the camera signal processing unit 11 and the memory control unit 12. Further, the resolution of the image data based on the imaging results sequentially stored in the memory is converted by the resolution conversion unit 14, and the liquid crystal display panel 16 is driven by the display control unit 15 using the image data. Thereby, the central processing unit 20 displays the monitor image of the imaging result on the liquid crystal display panel 16 while recording the imaging result of the moving image in the memory 10.

  In this state, when the user instructs to record the imaging result of the still image, the storage of the output data of the camera signal processing unit 11 in the memory 10 is stopped, and when the acquisition of the imaging result is instructed, the memory 10 is instructed. The stored image data is converted by the resolution conversion unit 14 to the resolution designated by the user. The image data obtained by the resolution conversion process is compressed by the image codec 17 and then recorded on the recording medium 19. In this series of processing, the central processing unit 20 instructs the resolution conversion unit 14 to create a thumbnail image, thereby creating a thumbnail image for indexing. The thumbnail image is recorded on the recording medium 19 as an index image of the imaging result recorded on the recording medium 19.

  Similarly, when the user gives an instruction to start recording the imaging result of a moving image, storage of the output data of the camera signal processing unit 11 in the memory 10 is stopped, and the resolution conversion unit 14 outputs the data from the camera signal processing unit 11. The resolution conversion processing of image data to be started is started. Further, the image codec 17 of the image data processed by the resolution conversion unit 14 is instructed to compress data, and the image codec 17 is used to instruct the media control unit 18 to record the processing result. Also in this process, the central processing unit 20 instructs the resolution converter 14 to create a thumbnail image for the first frame at the start of recording, and for the index of the imaging result obtained by recording this thumbnail image on the recording medium 19. It records on the recording medium 19 as an image.

  The central processing unit 20 also acquires the imaging result by controlling the operation of each unit in the same manner in accordance with a control command input via the USB 23, and the acquired imaging result or the imaging result recorded on the recording medium 19. Is output to an external device via the USB 23.

  On the other hand, when the user instructs to monitor the imaging result recorded on the recording medium 19 by the operation of the operation unit 22, the central processing unit 20 displays the thumbnail recorded on the recording medium 19 as shown in FIG. A list of the imaging results recorded on the recording medium 19 by the image is displayed.

  That is, in this case, the central processing unit 20 sequentially reads out the thumbnail images of the imaging results recorded on the recording medium 19 by controlling the operation of the camera DSP unit 3, and sequentially records the read thumbnail images in the memory 10 to display a list. Is recorded in the memory 10. Further, the display control unit 15 drives the liquid crystal display panel 16 with the image data held in the memory 10, thereby displaying a list of imaging results based on thumbnail images as shown in FIG. 4. The central processing unit 20 displays a cursor on the thumbnail image displayed at the center of the list display when the list display is started.

  Here, the example in FIG. 4 shows a state in which the list display is started by 20 thumbnail images indicated by reference numerals AA to TT among a large number of thumbnail images that cannot be displayed by this list display. In this case, a frame surrounding the display of the thumbnail image is set to the cursor K, and the cursor K is set to the central thumbnail image indicated by the symbol HH.

  When this list display is started, the central processing unit 20 further acquires the imaging result from the imaging device 4 and detects the motion vector V by the image codec 17 by controlling the operations of the preprocessing unit 5 and the camera DSP unit 3. As a result, the central processing unit 20 sets the overall operation so as to acquire the imaging result of the moving image and detect the motion vector V even when the imaging result recorded on the recording medium 19 is monitored. Note that the central processing unit 20 accepts a normal imaging result when acquiring a motion vector V by acquiring a video imaging result by monitoring the imaging result recorded in the recording medium 19 in this way. In the operation related to the operation of the lens block 2 such as zooming, the operation is not accepted.

  Further, the central processing unit 20 detects the motion of the electronic still camera 1 based on the motion vector V detected from the moving image, and further switches the overall operation based on the motion detection result. In this embodiment, this operation switching is set to list display switching on the liquid crystal display panel 16.

  That is, FIG. 1 is a flowchart showing a processing procedure of the central processing unit 20 relating to the switching of the list display. When starting the list display, the central processing unit 20 moves from step SP1 to step SP2, and acquires the motion vector V of the imaging result acquired by the imaging device 4 from the image codec 17.

  Subsequently, the central processing unit 20 detects the motion of the electronic still camera 1 by statistical processing for one frame of the acquired motion vector V. That is, as shown in FIG. 5A, in a state where a relatively close object is imaged on the entire screen, all the Y-direction components of the motion vector V detected by each macroblock are 0, and the vector V If all the X-direction components are substantially equal in size, it can be determined that the image frame of the electronic still camera 1 has moved in the horizontal direction with respect to the subject being shot. In this case, it can be determined that the electronic still camera 1 is moved in the horizontal direction by parallel movement or the like.

  On the other hand, as shown in FIG. 5B, all X-direction components of the motion vectors V detected in each macroblock are 0 in a state where a relatively close object is imaged on the entire screen. If the Y direction components are approximately equal in size, it can be determined that the image frame of the electronic still camera 1 has moved in the vertical direction with respect to the subject being shot. In this case, it can be determined that the electronic still camera 1 is moved in the vertical direction by parallel movement, tilt, or the like.

  On the other hand, as shown in FIG. 6A, when the motion vector V is detected radially from the center of the screen toward the periphery, and the magnitude of the motion vector increases toward the periphery, the image is taken. It can be determined that the electronic still camera 1 is approaching the subject.

  On the other hand, as shown in FIG. 6B, when a motion vector V is detected from the periphery toward the center of the screen, and the magnitude of the motion vector V increases toward the periphery, It can be determined that the electronic still camera 1 is away from the subject being photographed.

  Thereby, the central processing unit 20 detects the same direction component of the motion vector V of each macroblock by integration processing by averaging the motion vector V detected in each macroblock, and the X direction of the electronic still camera 1 is detected. And motion in the Y direction is detected. Further, the movement of the electronic still camera 1 in the Z direction is detected by averaging inner product values with reference vectors set in accordance with the positions of the macroblocks. In addition, in such a reference vector, for example, as shown in FIG. 6 (A) or (B), it is a vector that goes from the center of the screen to the periphery, or a vector that goes from the periphery to the center of the screen and goes toward the center. A vector of increasing magnitude can be applied. Note that the motion detection in the Z direction is detected only by a motion vector V at a specific location based on the center of the screen, for example, when detecting by a concentric area centered on the center of the screen. It may be. Note that the X, Y, and Z directions in these processes are directions based on the imaging screen of the electronic still camera 1. In the following description, the direction approaching the subject is the positive direction of the Z direction.

  When the movement of the electronic still camera 1 is detected in each direction of the electronic still camera 1 in this way, the central processing unit 20 moves to step SP3 and normalizes the detected movement. Here, the normalization process is a process of dividing the motion amount in each direction by the total motion amount.

  In other words, the central processing unit 20 switches the currently set cursor to the adjacent thumbnail image according to the movement in the X direction and the Y direction, and captures the thumbnail image in which the cursor is set according to the movement in the Z direction. The result is displayed on the entire screen of the liquid crystal display panel 16. Further, when the movement of the electronic still camera 1 in the Z direction is detected in the opposite direction to the display of the thumbnail image in the state where the imaging result is displayed on the entire screen of the liquid crystal display panel 16 as described above, the imaging result on the full screen is detected. Return to the list display with thumbnail images.

  As shown in FIG. 7, the central processing unit 20 moves the electronic still camera 1 to any position among the grid points P0 to P7 related to the switching of the cursor and the switching of the full screen display in the normalization process. A motion vector indicating whether or not it has been detected is detected. Note that FIG. 7 shows only the positive direction of each direction, and the actual lattice points are also set in the negative direction of each direction. Here, the grid point P0 whose direction component is 0 corresponds to the thumbnail image where the cursor is currently set, and the grid point at a position displaced in the X and Y directions with respect to the grid point P0 at the current position. Corresponds to the thumbnail images adjacent in the direction corresponding to the thumbnail image at which the cursor is currently set. The Z direction component is a grid point corresponding to switching between list display and full screen.

  Therefore, if the central processing unit 20 determines that the electronic still camera 1 has moved from the grid point P0 at the current position to the grid point P3 in the X direction and the positive direction, the central processing unit 20 determines the position of the cursor K described above with reference to FIG. , Switch to the thumbnail image with the code II adjacent in the positive direction. If it is determined that the electronic still camera 1 has moved from the grid point P0 at the current position to the grid point P1 in the Y direction and the positive direction, the position of the cursor K described above with reference to FIG. Switch to a thumbnail image by CC. If it is determined that the electronic still camera 1 has moved from the grid point P0 at the current position to the grid point P4 in the Z direction and the positive direction, the list display by thumbnail images is switched to the display by full screen.

  When it is determined that the electronic still camera 1 has moved from the grid point P0 at the current position to the grid point P6 in the X, Y, Z, and positive directions, the position of the cursor K described above with reference to FIG. Then, the thumbnail image with the code DD adjacent in the Y direction and the positive direction is switched, and the imaged result of the thumbnail image with the code DD is displayed on the full screen.

  The normalization process in step SP3 is a process for detecting to which lattice point the movement of the electronic still camera 1 detected in step SP2 is, and the central processing unit 20 is represented by the following equation. Then, the detected motion amounts α, β, γ in each direction of the electronic still camera 1 are divided by the total motion amounts according to the motion amounts in these directions, whereby the motion amounts α, β, γ in each direction are divided as a whole. Normalize by the amount of movement.

Further, in this way, the total motion amounts Kx, Ky, Kz are respectively determined by a predetermined threshold value, and the motion amount in each direction is set to a value −1, a value 0, or a value 1, and is expressed by the following equation: Thus, the normalized motion P is detected. Here, Vx, Vy, and Vz are unit vectors in the X direction, the Y direction, and the Z direction, respectively. The threshold, for example ^1/2 1/2, is set to -1 / 2 ^1/2. Note that this threshold defines the sensitivity due to the operation of the electronic still camera 1, and may be set variously as necessary, and may be switched adaptively according to the operation by the user. May be.

Thus, when P = (0,0,0), the grid point P0 at the current position is indicated, and when P = (1,0,0), the grid point P3 in the X direction and the positive direction is indicated. It will be. If the overall movement (α ² + β ² + γ ² ) ^1/2 is equal to or less than a certain value, the central processing unit 20 returns to step SP2 without executing these processes, thereby preventing erroneous operation due to camera shake. To do.

  When the motion is normalized in this way, the central processing unit 20 moves to step SP5, determines the normalized motion P, and determines whether there is motion in the X direction and / or the Y direction. If a positive result is obtained here, the central processing unit 20 moves from step SP5 to step SP6, and moves the cursor K in the movement direction. Accordingly, when the electronic still camera 1 is moved in the upper right direction in the state shown in FIG. 4, as shown in FIG. 8, the cursor K is moved to the thumbnail image indicated by the code DD adjacent in the upper right direction.

  The central processing unit 20 moves the cursor K to the rightmost end or the leftmost end of this list display when there are more imaging results recorded on the recording medium 19 than can be displayed together by this list display. When the cursor K is further moved to the right or left side, the thumbnail image designated by the cursor K is switched by scrolling the list display. In this case, the display of thumbnail images may be switched by sequentially scrolling in the horizontal direction.

  Further, when the movement of the electronic still camera 1 is detected further upward or downward with the cursor K being moved to the top or bottom thumbnail image, the upward or downward movement is detected. Is ignored. Further, when all the imaging results recorded in the recording medium 19 are collectively displayed by this list display, the electronic still is further moved with the cursor K moved to the rightmost end or the leftmost end. Even when the movement of the camera 1 in the right direction or the left direction is detected, the movement in this direction is ignored. Thus, in these cases, the cursor K is moved only in the direction in which the cursor K can be moved and the thumbnail image can be scrolled, and the thumbnail image is further scrolled.

  When the cursor is moved in this way, the central processing unit 20 moves from step SP6 to step SP7. If a negative result is obtained in step SP5, the central processing unit 20 directly moves from step SP5 to step SP7. Here, the central processing unit 20 determines the normalized movement P and determines whether or not there is movement in the Z direction and the positive side. If a negative result is obtained here, the central processing unit 20 returns from step SP7 to step SP2. As a result, the central processing unit 20 repeats the processing loop of step SP2-SP3-SP4-SP5-SP6-SP7-SP2 or the processing loop of step SP2-SP3-SP4-SP5-SP7-SP2, thereby enabling the electronic processing by the user. In accordance with the movement of the still camera 1 in the X and Y directions, the cursor K is moved in the list display using thumbnail images.

  The central processing unit 20 repeats these processing loops at a predetermined frame period of the imaging result acquired by the imaging device 4. Instead of intermittent execution with such a predetermined frame period, the movement of the electronic still camera 1 detected with a predetermined frame is averaged and processed so that the display is switched with the predetermined frame period. Good. Thereby, the central processing unit 20 prevents the cursor K from moving at a significantly high speed.

  On the other hand, if an affirmative result is obtained in step SP7, the central processing unit 20 moves from step SP7 to step SP8, and for the thumbnail image at the cursor position, as shown in FIG. The image is displayed on the liquid crystal display panel 16 on the screen. The example shown in FIG. 9 shows a display in which movement in the Z direction and the positive direction is detected in the state shown in FIG.

  Specifically, the central processing unit 20 reads out the imaging result relating to the thumbnail image at the cursor position from the recording medium 19 and stores it in the memory 10, and then decompresses the data by the image codec 17 and stores it again in the memory 10. The resolution of the image data re-stored in the memory 10 is converted by the resolution conversion unit 14, and the liquid crystal display panel 16 is driven by the display control unit 15 based on the image data, whereby the imaging result is displayed by the liquid crystal display panel 16. indicate. The central processing unit 20 displays a frame W related to the reproduction zoom in this full screen display.

  When the central processing unit 20 switches the display on the liquid crystal display panel 16 to the full screen display in this way, the process proceeds from step SP8 to step SP9, and this processing procedure is terminated.

  FIG. 10 is a flowchart showing the processing procedure of the central processing unit 20 when switching to full screen display by executing the processing procedure of FIG. When switching the display on the liquid crystal display panel 16 to the full screen display, the central processing unit 20 starts this processing procedure and proceeds from step SP11 to step SP12. Here, the central processing unit 20 acquires the motion vector V in the same manner as described above for step SP3. In the subsequent step SP12, the movement of the electronic still camera 1 is detected in the same manner as described above for step SP3.

  Subsequently, the central processing unit 20 moves to step SP14, and processes the movement of the electronic still camera 1 detected in step SP13 in the same manner as described above for step SP4. In the processing in step SP4 described above with reference to FIG. 1, the display position of the cursor K, and the grid points in the horizontal and vertical directions correspond to the grid points described above with reference to FIG. In the processing at step SP14, the lattice points correspond to the movement unit of the frame W related to the playback zoom processing.

  Subsequently, the central processing unit 20 moves to step SP15, and determines whether or not the electronic still camera 1 has been moved in the X direction and the Y direction by determining the normalized motion P detected in step SP14. If an affirmative result is obtained here, the central processing unit 20 moves from step SP15 to step SP16, and as shown by the arrows in FIG. 9, the movement unit related to the grid point in the direction related to the motion P of the electronic still camera 1 The frame W is moved by a predetermined distance corresponding to. In this case, the central processing unit 20 ignores the movement of the electronic still camera 1 in the direction of popping out the display screen with the frame W moved to the peripheral area of the display screen.

  When the central processing unit 20 completes the process of step SP16, the process proceeds from step SP16 to step SP17. If a negative result is obtained in step SP15, the process directly proceeds from step SP15 to step SP17. In step SP17, the central processing unit 20 determines the normalized movement P and determines whether there is movement in the Z direction or the negative side. If a negative result is obtained here, the central processing unit 20 moves from step SP17 to step SP18, determines the normalized movement P here, and determines whether there is movement in the Z direction or the positive side.

  If a negative result is obtained here, the central processing unit 20 returns from step SP18 to step SP12. Thereby, the central processing unit 20 repeats the processing loop of step SP12-SP13-SP14-SP15-SP16-SP17-SP18-SP12 or the processing loop of step SP12-SP13-SP14-SP15-SP17-SP18-SP12 to the user. The frame W related to the reproduction zoom is moved according to the movement of the electronic still camera 1 in the X direction and the Y direction. Also in this case, the central processing unit 20 repeats these processing loops at a predetermined frame period of the imaging result acquired by the imaging device 4 or averages the motion of the electronic still camera 1 detected at the predetermined frame. The frame W is moved to prevent the frame W from moving suddenly.

  On the other hand, if an affirmative result is obtained in step SP18, the central processing unit 20 proceeds from step SP18 to step SP19, and executes a reproduction zoom process. That is, as shown in FIG. 11A, the central processing unit 20 notifies the resolution conversion unit 14 of the position information of the frame W related to the current display position and instructs the reproduction zoom, and is thereby surrounded by the frame W. The image of the selected part is enlarged by electronic zoom and displayed on the entire screen of the liquid crystal display panel 16. Thus, when the central processing unit 20 executes the process of step SP19, the process proceeds to step SP20 and ends the processing procedure.

  On the other hand, if a positive result is obtained in step SP17, the central processing unit 20 moves from step SP17 to step SP21, and returns the full-screen display of the imaging result on the liquid crystal display panel 16 to the list display by the original thumbnail image. Thereafter, the process proceeds to step SP20 and the processing procedure is terminated.

  On the other hand, the central processing unit 20 reproduces according to the movement of the electronic still camera 1 in the X direction and the Y direction in the same manner as described above with respect to the movement of the frame W while displaying the image enlarged by the reproduction zoom. Move the enlarged display area by zooming. When the movement to the negative side in the Z direction is detected, the enlarged display by the reproduction zoom is stopped and the display on the liquid crystal display panel 16 is switched to the original full screen display.

(2) Operation of Embodiment In the above configuration, in this electronic still camera 1 (FIG. 2), when the start of shooting is instructed by the user, an optical image by the lens block 2 is photoelectrically converted by the imaging device 4 and captured. A signal is generated, and this imaging signal is processed by the CDS / AGC / ADC unit 8 to generate image data. The image data is sequentially stored in the memory 10, and the resolution is converted by the resolution conversion unit 14 and displayed on the liquid crystal display panel 16 by the display control unit 15. Thus, the user can confirm the subject by displaying on the liquid crystal display panel 16.

  Thereby, in the electronic still camera 1, when acquisition of the imaging result by the still image is instructed by the user, acquisition of the imaging result by the imaging element 4 is stopped, and the image data stored in the memory 10 is processed by the resolution conversion unit 14. Or after being directly compressed by the image codec 17 and stored in the recording medium 19. When the user gives an instruction to acquire the imaging result by moving image, the imaging result obtained from the imaging element 4 is sequentially subjected to resolution conversion processing by the resolution conversion unit 14, and data is compressed by the image codec 17 and recorded on the recording medium 19. The At the time of recording, an index thumbnail image is generated by the resolution conversion unit 14, and this thumbnail image is recorded on the recording medium 19.

  On the other hand, when the user instructs the reproduction of the imaging result recorded on the recording medium 19, the image data recorded on the recording medium 19 is decompressed by the image codec 17 and is displayed on the liquid crystal display panel 16 by the display control unit 15. Is displayed.

  In the reproduction of the imaging result recorded on the recording medium 19, when the index display by the thumbnail image is instructed by the user, the electronic still camera 1 reproduces the index thumbnail image recorded on the recording medium 19. A list is displayed on the liquid crystal display panel 16. In the reproduction of the imaging result recorded on the recording medium 19, which is one of the cases where the processing related to the recording of the imaging result output from the imaging element to the recording medium 19 is not performed, the electronic still camera 1 The imaging element 4 sequentially acquires the imaging result of the moving image, and the motion vector V of the imaging result is detected by the image codec 17 (FIGS. 5 to 7). The motion of the electronic still camera 1 is detected by the central processing unit 20 based on the detected motion vector V, and the operation is switched by this motion.

  As a result, in the electronic still camera 1, the configuration relating to imaging is effectively used, the electronic still camera 1 is moved in a desired direction, and various operations are switched, and the electronic still camera 1 is operated like a mouse. Accordingly, it is possible to provide an easy-to-use user interface with a simpler operation than in the past.

  More specifically, in the horizontal direction and the vertical direction with reference to the imaging surface of the image sensor 4, the motion direction of the apparatus is detected and the amount of motion is further detected by integrating motion vectors. Further, the movement of the electronic still camera 1 is switched by an operation like a mouse by the movement detected in this way.

  At this time, in the central processing unit 20, the amount of movement in each direction is normalized by the total amount of movement, and the movement of the electronic still camera 1 is detected, thereby eliminating the difference in usability for the user. That is, when the operation is switched according to the movement of the electronic still camera 1 in this way, the amount by which the electronic still camera 1 is moved differs depending on the user. However, when switching the operation by normalizing the amount of movement in each direction, it is possible to switch the operation corresponding to different movement methods depending on the user, and the user's usability can be improved accordingly. it can.

  In the electronic still camera 1, the movement in each direction normalized in this way is further determined by a threshold value, and the movement in each direction is detected. As a result, the electronic still camera 1 can prevent, for example, switching of operations unintended by the user, such as switching of operations related to movement in the vertical direction with respect to user operations intended to move in the horizontal direction. This also makes it possible to provide an easy-to-use user interface with a simpler operation.

  That is, in the electronic still camera 1, in the list display by thumbnail images as described above, the cursor K moves in accordance with the movement of the electronic still camera 1 (FIGS. 1, 4, and 8). The display related to the cursor K can be switched by operating the electronic still camera 1 as if operating a mouse on the computer.

  Further, the movement in the Z direction switches the display to a full screen display based only on the imaging result in which the cursor K is set. In this case, the display can be switched by an operation as if the mouse is clicked.

  Further, by moving the frame indicating the playback zoom area in the state where the imaging result is displayed on the full screen in this way (FIG. 9), in this case, it is as if the mouse is operated on the computer. The playback zoom area can be switched.

  Further, by switching the display to the list display and the reproduction zoom by the movement in the Z direction while displaying the imaging result on the full screen, the display can be switched by an operation as if the mouse is clicked.

(3) Effects of the embodiment According to the above configuration, when the processing related to the recording of the imaging result to the recording medium is not executed, the motion vector of the imaging result by the imaging device is detected by the data compression unit. By detecting the movement of the apparatus based on the motion vector and switching the operation, it is possible to provide an easy-to-use user interface with a simpler operation than in the past.

  Further, since the movement of the device is at least the amount of movement in the horizontal direction and the vertical direction with reference to the imaging surface of the imaging device, a user interface with a feeling of operation can be provided as if it were a mouse.

  In addition, by normalizing and detecting the amount of movement in the horizontal and vertical directions according to the total amount of movement, the operation can be switched according to different movement methods depending on the user, thereby improving the user's convenience. Can do.

  In addition, after normalizing the amount of movement in each direction based on the total amount of movement, it is possible to prevent unintended operations by the user by detecting the movement of the device by determining each with a threshold value. It is possible to provide an easy-to-use user interface with simpler operations.

  In addition, by detecting such movement by integrating motion vectors, it is possible to prevent erroneous detection of apparatus movement due to movement of the subject.

  Specifically, the case where the process related to the recording of the imaging result output from the imaging element to the recording medium is not performed is the process related to the display of the imaging result recorded on the recording medium, and thus this imaging With regard to switching of operations related to the display of results, it is possible to provide an easy-to-use user interface with a simpler operation than in the past.

  That is, since the switching of the operation is switching of the display related to the movement of the cursor in the list display of the imaging results recorded on the recording medium by the display unit, the cursor can be moved by an operation like a mouse.

  In addition, since the switching of the operation is switching from the list display of the imaging results recorded on the recording medium in the display unit to the display of only the imaging results for which the cursor is set, the display is performed like a click operation on the mouse. Can be switched.

  In addition, when the switching of the operation is switching of the display related to the movement of the reproduction zoom area in the display of the imaging result, the reproduction zoom area can be moved by an operation like a mouse.

  In addition, since the switching of the operation is switching from the display of the imaging result to the enlarged display of the reproduction zoom area, the display can be switched like a click operation with the mouse.

  By the way, in the case of detecting the motion of the apparatus based on the motion vector of the imaging result described above with respect to the first embodiment, the detection accuracy of the motion detection in the Z direction changes depending on the distance to the subject. Accordingly, in this embodiment, the movement amount of the electronic still camera 1 is detected in the Z direction by focus control in the lens block 2. The electronic still camera according to this embodiment is configured in the same manner as the electronic still camera 1 described above with respect to the first embodiment except for the configuration related to the processing in the Z direction. This will be explained.

  In other words, in the second embodiment, when the display related to the imaging result described above with reference to FIG. 4 and the like is executed, the lens control unit 13 performs autofocus control on the lens block 2 and also detects the subject detected by the focus control. Is sent to the central processing unit 20.

  The central processing unit 20 detects the movement of the electronic still camera 1 in the Z direction based on the movement to the subject notified from the lens control unit 13 instead of the movement in the Z direction based on the motion vector described above. As a result, the central processing unit 20 executes only the X direction and the Y direction in a series of processes related to motion detection.

  According to this embodiment, when the processing related to the recording of the imaging result to the recording medium is not executed, the movement of the apparatus is detected by the focus adjustment in the lens control unit in addition to the vector by the data compression unit. By switching the, it is possible to provide a user interface that is easier to use than ever with a simpler operation. In addition, since the movement in the X direction is detected by focus control instead of the motion vector, the detection accuracy can be improved and user convenience can be improved.

  This embodiment is an electronic still camera that is detected by motion control and focus control of the electronic still camera detected by the motion vector V when the user switches the operation mode in the electronic still camera described above with reference to the first or second embodiment. Is notified to an external device such as a computer via the USB 23 as a pointing device operation. The electronic still camera according to this embodiment is configured in the same manner as the electronic still camera 1 described above with respect to the first or second embodiment except for the configuration related to the external output.

  According to this embodiment, by notifying the movement of the electronic still camera detected by the motion vector V and the movement of the electronic still camera detected by the focus control to an external device such as a computer as a pointing device operation, This electronic still camera can also be used as a user interface for these external devices.

  In the above-described embodiment, a case has been described in which a series of operations related to display of imaging results recorded on a recording medium is executed by the movement of an electronic still camera. It can be widely applied to the switching of the cursor in the menu.

  In the above-described embodiment, the case of detecting the movement in the Z direction by focus adjustment has been described. However, the present invention is not limited to this, and the movement in the Z direction may be detected by using it together with a motion vector.

  In the above-described embodiments, the case where the present invention is applied to an electronic still camera has been described. However, the present invention is not limited thereto, and can be widely applied to various imaging devices such as a video camera and a camera-equipped mobile phone. it can.

  The present invention relates to an imaging apparatus, an imaging apparatus control method, an imaging apparatus control method program, and a recording medium on which an imaging apparatus control method program is recorded, and can be applied to, for example, an electronic still camera.

It is a flowchart which shows the process sequence of the central processing unit in the electronic still camera which concerns on Example 1 of this invention. It is a block diagram which shows the electronic still camera which concerns on Example 1 of this invention. It is a block diagram which shows the structure of the image codec of the electronic still camera of FIG. It is a top view which shows the list display by the thumbnail image in the electronic still camera of FIG. It is a basic diagram with which it uses for description of the motion detection of an electronic still camera by a motion vector. It is a basic diagram with which it uses for description of the motion detection of the electronic still camera of a Z direction by a motion vector. It is an approximate line figure used for explanation of processing of motion normalization. It is a top view with which it uses for description of the movement of a cursor. It is a top view with which it uses for description of the area | region movement of a reproduction zoom. It is a flowchart which shows the process sequence of the central processing unit which concerns on the area | region movement of reproduction | regeneration zoom. It is an approximate line figure used for explanation of processing of reproduction zoom.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Electronic still camera, 2 ... Lens block, 3 ... Camera DSP part, 4 ... Image sensor, 11 ... Camera signal processing part, 13 ... Lens control part, 16 ... Liquid crystal display panel, 17 ... ... Image codec, 20 ... Central processing unit

Claims (16)

An image sensor that outputs an imaging result of an optical image formed on the imaging surface;
A lens for creating the optical image on the imaging surface;
A data compression unit that detects a motion vector of an imaging result output from the imaging element and compresses the imaging result using the motion vector;
A recording / reproducing unit that records the imaging result compressed by the data compression unit on a recording medium and reproduces the imaging result recorded on the recording medium;
A display unit for displaying the imaging result reproduced by the recording / reproducing unit;
A control unit for controlling the overall operation,
The controller is
When the processing related to the recording of the imaging result output from the imaging element to the recording medium is not executed, the image sensor and the data compression unit are operated to obtain a motion vector from the imaging result of the imaging element. Detect and get,
By detecting the motion of the device based on the motion vector,
An image pickup apparatus that switches operations based on movement of the apparatus. A lens control unit that adjusts the focus of the lens based on an imaging result output from the imaging element;
The controller is
The imaging apparatus according to claim 1, wherein the movement of the apparatus is detected by focus adjustment in the lens control unit in addition to the motion vector. The movement of the device is
The imaging apparatus according to claim 1, wherein the amount of movement is at least in a horizontal direction and a vertical direction with reference to an imaging surface of the imaging element. The controller is
The imaging apparatus according to claim 3, wherein the amount of motion in the horizontal direction and the vertical direction is detected by normalizing the amount of motion based on the total amount of motion. The controller is
The imaging apparatus according to claim 3, wherein the amount of motion in the horizontal direction and the vertical direction is normalized by the total amount of motion, and then determined and detected based on threshold values. The controller is
The imaging apparatus according to claim 1, wherein the movement of the apparatus is detected based on a movement direction of the apparatus based on the motion vector and a motion amount obtained by integrating the motion vector. The imaging apparatus according to claim 1, further comprising an interface that notifies an external apparatus of movement of the apparatus and can be used as coordinate input means of the external apparatus. When the processing related to the recording of the imaging result output from the imaging device to the recording medium is not executed,
The imaging apparatus according to claim 1, wherein the imaging apparatus is processing related to display on the display unit of the imaging result recorded on the recording medium. The switching of the operation is
The imaging apparatus according to claim 1, wherein in the list display of the imaging results recorded on the recording medium by the display unit, the display is switched according to movement of a cursor. The switching of the operation is
The imaging apparatus according to claim 1, wherein the display device is switched from a list display of the imaging results recorded on the recording medium to a display of only the imaging results for which a cursor is set. The switching of the operation is
The imaging apparatus according to claim 1, wherein in the display of the imaging result recorded on the recording medium in the display unit, the display is switched according to movement of a reproduction zoom area. The switching of the operation is
The imaging apparatus according to claim 1, wherein the display unit is switched from display of the imaging result recorded on the recording medium to enlarged display of a reproduction zoom area. The switching of the operation is
The imaging apparatus according to claim 1, wherein in the display of the menu in the display unit, the display is switched according to the movement of the cursor. An image sensor that outputs an imaging result of an optical image formed on the imaging surface;
A lens for creating the optical image on the imaging surface;
A data compression unit that detects a motion vector of an imaging result output from the imaging element and compresses the imaging result using the motion vector;
A recording / reproducing unit that records the imaging result compressed by the data compression unit on a recording medium and reproduces the imaging result recorded on the recording medium;
A display unit for displaying the imaging result reproduced by the recording / reproducing unit;
In a control method of an imaging apparatus including a control unit that controls the overall operation,
When the processing related to the recording of the imaging result output from the imaging element to the recording medium is not executed, the image sensor and the data compression unit are operated to obtain a motion vector from the imaging result of the imaging element. A motion vector detection step to detect and obtain;
A motion detection step of detecting the motion of the device based on the motion vector;
And a control step of switching the operation based on the movement of the apparatus. An image sensor that outputs an imaging result of an optical image formed on the imaging surface;
A lens for creating the optical image on the imaging surface;
A data compression unit that detects a motion vector of an imaging result output from the imaging element and compresses the imaging result using the motion vector;
A recording / reproducing unit that records the imaging result compressed by the data compression unit on a recording medium and reproduces the imaging result recorded on the recording medium;
A display unit for displaying the imaging result reproduced by the recording / reproducing unit;
In a program of a control method of an imaging device provided with a control unit that controls the overall operation,
When the processing related to the recording of the imaging result output from the imaging element to the recording medium is not executed, the image sensor and the data compression unit are operated to obtain a motion vector from the imaging result of the imaging element. A motion vector detection step to detect and obtain;
A motion detection step of detecting the motion of the device based on the motion vector;
And a control step of switching the operation based on the movement of the apparatus. 16. A recording medium recording a program of a control method for an imaging apparatus according to claim 15, wherein the program for the control method of the imaging apparatus is recorded.

Images