JP2013030939A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of optimizing an image sensor reading method in accordance with a required moving image specification.
According to a shooting mode, as a signal readout operation of pixels from an image sensor, (a) a first readout operation for reading out signals of all pixels of the image sensor, and (b) a row direction and a column of the image sensor. Any one of a second readout operation for reading out pixel signals by thinning out in both directions and (c) a third readout operation for reading out signals of all pixels in the row direction only in a specific row that has been thinned out is selected. Is done. In the photographing process, a still image or a moving image is generated using a pixel signal obtained by executing the selected signal reading operation.
[Selection] Figure 1

Description

  The present invention relates to an imaging apparatus having an imaging element, and more particularly to an imaging apparatus capable of performing still image shooting and moving image shooting.

  A solid-state imaging device mounted on a recent imaging apparatus such as a digital still camera has a significantly increased number of pixels, and can capture a high-definition still image. Furthermore, not only still images but also electronic cameras capable of displaying and recording moving images have been realized.

  When displaying and recording a moving image, it is necessary to secure a frame rate of at least about 24 to 30 frames / second in order to obtain a higher resolution in the time direction.

  In recent years, technological development of an imaging device that records a frame rate of 60 frames / second or higher and a playback device that reproduces moving image data at this frame rate has been advanced.

  In an imaging device that can shoot high-definition still images and also record moving images, both high-pixel-rate image data generation processing during still image shooting and high-frame-rate readout processing during moving image shooting are compatible. Therefore, the number of output pixels of the solid-state imaging device is reduced during moving image shooting.

  For example, when reading out all pixel regions of a solid-state image sensor at high speed, the number of output pixels is reduced by performing so-called thinning-out readout that reads out every predetermined number of pixels in the horizontal and vertical directions of the solid-state image sensor. be able to.

  As another method for reducing the number of output pixels, pixel addition thinning readout by so-called pixel mixture is known. In this pixel addition decimation readout, simply performing decimation readout reduces the sampling frequency spatially and increases spatial aliasing noise. Therefore, on the solid-state image sensor, pixel output that is not actually read out is output. This is added to the pixel output to be actually read and read.

  As for such pixel addition thinning readout, for example, Patent Document 1 discloses a method of thinning out outputs from either one of the horizontal and vertical imaging devices or both to 1 / (2n + 1), or thinning after pixel addition. Proposed.

  Further, Patent Document 2 discloses a method for performing pixel interpolation processing so that the image size is maintained in the subsequent stage when the thinning rate in either the horizontal or vertical direction is changed.

  Also, efforts are being made to improve the readout speed of the image sensor itself, including during still image shooting. In the CMOS type image pickup device, a method for improving the reading speed is employed for both still image shooting and moving image shooting by a method such as multi-channel reading that simultaneously reads signals of a plurality of pixels.

JP 2002-135793 A JP 2003-234964 A

  Using the prior art disclosed in the above-mentioned patent document, the number of pixels to be read out is, for example, 1/9 of the total number of pixels when thinned out to 1/3 in the horizontal and vertical directions, respectively, and the reading time can be greatly reduced. This makes it possible to easily achieve a desired frame rate.

  Further, if the actual readout time is short with respect to the frame rate, there is an effect of suppressing the average power consumption by appropriately turning off or reducing the power consumption of the image sensor and the peripheral circuit block during the unnecessary period.

  However, for example, when thinning out to 1/3 in the horizontal and vertical directions, the 3 × 3 pixel area of the image sensor is compressed to one pixel output. Here, considering a Bayer array image sensor composed of R, G, G, and B colors, when R is at the center of 3 × 3 pixels, no other R pixel exists in the 3 × 3 pixels. The same applies when the B pixel is in the center. On the other hand, when the G pixel is at the center, there are four other G pixels in the 3 × 3 pixel.

  Therefore, considering the pixel addition in the unit pixel after this 3 × 3 thinning, G can be added to the output of other G pixels in this unit pixel, but R and B are not. It turns out that it is possible.

  Since the addition effect cannot be obtained for R and B pixels, there is a problem that aliasing noise is generated for R and B as in the simple thinning operation.

  Furthermore, it is also possible to perform addition processing of the same color by using the outputs of the R and B pixels in the periphery of 3 × 3 pixels. However, when this is done, the aliasing noise is reduced, but this is equivalent to performing an averaging process with the output of adjacent pixels substantially, so that there is a problem that the resolution is lowered.

  Further, in the prior art disclosed in the above-mentioned patent document, the horizontal and vertical directions are thinned to 1 / (2n + 1), so that the number of pixels is 1/9 when the horizontal and vertical directions are thinned by 1/3. Further, when the next thinning rate is 1/5 in both the horizontal and vertical directions, the number of pixels is 1/25, and the thinning change rate is 2.8 times. This rate of change is slightly larger than the ratio of 2.24 times between full HD (1920 × 1080, approximately 2.70 million pixels) and HD (1280 × 720, approximately 920,000 pixels), which is common in moving image formats. Therefore, the change rate of this thinning is too large considering that it corresponds to two formats of full HD and HD.

  In order to perform the thinning process by the pixel addition operation in the image sensor, it is necessary to incorporate a switching circuit at the time of addition. However, it is difficult to incorporate a complicated circuit for that purpose because of an increase in circuit scale and restrictions on wiring layout. Therefore, in the circuit in the image sensor, in addition to dealing with high resolution and aliasing noise, it has been a very big problem to achieve both a desired thinning rate and a reading speed.

  On the other hand, the horizontal and vertical operations are highly circuit independent. For this reason, it is possible to relatively easily realize horizontal / vertical thinning at 1 / (2n + 1) instead of horizontal / vertical thinning at the same time in terms of circuit scale.

  However, in the case of such a thinning process, since the aspect ratio of the image read from the image sensor is different, a digital process after A / D conversion at the subsequent stage performs a process of multiplying different image scaling factors in the horizontal direction and the vertical direction. There is a need.

  In the above-mentioned Patent Document 2, the thinning rate is not limited to 1 / (2n + 1), but a process of multiplying different image scaling factors in the horizontal and vertical directions according to the thinning rate is disclosed.

  However, if the image signal once thinned out is interpolated in the subsequent stage, the resolution is lowered and the image quality is lowered.

  Therefore, the present invention provides an imaging apparatus capable of optimizing the imaging element readout method in accordance with the required moving image specifications.

  According to one aspect of the present invention, an image sensor in which a plurality of pixels including a photoelectric conversion element that photoelectrically converts a subject image is arranged in a row direction and a column direction, a setting unit that sets a shooting mode, and the setting Depending on the shooting mode, as a signal readout operation of pixels from the image sensor, (a) a first readout operation for reading out signals of all pixels of the image sensor, and (b) a row direction among the plurality of pixels. And a second readout operation for reading out pixel signals in the column direction, and (c) a third readout for reading out signals of all pixels in the row direction only in the thinned specific column among the plurality of pixels. Selection means for selecting one of the operations, and processing means for generating a still image or a moving image using the pixel signal obtained by executing the selected signal reading operation in the photographing process. Imaging apparatus is provided, wherein Rukoto.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can optimize an image pick-up element reading system according to the specification of the moving image requested | required can be provided.

1 is a block diagram illustrating a configuration example of an imaging device according to an embodiment. 6 is a flowchart illustrating an operation example of the imaging apparatus according to the embodiment. 6 is a flowchart illustrating an operation example at the time of still image shooting of the imaging apparatus according to the embodiment. 6 is a flowchart illustrating a shooting operation at the time of still image shooting of the imaging apparatus according to the embodiment. FIG. 2 is a block diagram showing a configuration of a DSP of the imaging apparatus in the embodiment. 6 is a flowchart illustrating an operation example at the time of moving image shooting of the imaging apparatus according to the embodiment. FIG. 4 is a diagram illustrating a still image readout operation of the imaging apparatus according to the embodiment. FIG. 6 is a diagram illustrating horizontal pixel addition and vertical thinning-out reading operations during moving image shooting of the imaging apparatus according to the embodiment. FIG. 6 is a diagram illustrating horizontal all-pixel reading and vertical thinning-out reading operations during moving image shooting of the imaging apparatus according to the embodiment.

  Hereinafter, embodiments of the present invention will be specifically described with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. As shown in the figure, the imaging apparatus of the present embodiment includes an imaging device 101, an AFE (analog front end) 102, a DSP (digital signal processor) 103, a timing generation circuit 104, and a CPU 105.

The image sensor 101 includes a plurality of pixels including a photoelectric conversion element that photoelectrically converts a subject image in a row direction and a column direction. In this specification, “row direction” and “column direction” are also referred to as “horizontal direction” and “vertical direction”, respectively. The image sensor 101 is, for example, a CMOS sensor, and an amplifier circuit (not shown) that switches the gain according to the ISO sensitivity is built in the CMOS sensor. Further, this CMOS sensor has the following modes as described above as pixel signal readout operation modes.
(1) All pixel reading mode for reading all pixels.
(2) Thinning readout mode for thinning out and reading out pixels.
(3) A pixel addition thinning readout mode in which the outputs of a plurality of pixels on the CMOS sensor are averaged and output in the thinning readout mode.
(4) A cut-out read mode in which the read start position and read end position in the horizontal and vertical directions can be selected and the cut position can be switched.

  These readout methods will be described with reference to FIG. This figure shows normal all-pixel readout, and the physical pixel arrangement of the image sensor 101 is shown on the left side of the figure. R, Gr, Gb, and B described in each pixel indicate colors of the color filters arranged in the primary color Bayer.

  An image obtained by sequentially scanning the image sensor in the horizontal direction and the vertical direction and obtaining outputs of all pixels in time series is shown on the right side of the figure. In this case, since all pixels are read out, the obtained image is naturally the same as the physical arrangement.

  The AFE 102 incorporates an A / D converter that performs analog-digital conversion on a signal from the image sensor 101 and has a function of clamping a dark offset level. The DSP 103 performs various correction processes, development, and compression processes on the data from the AFE 102. The DSP 103 controls various memories such as the ROM 106 and the RAM 107, performs image data writing processing on the recording medium 108, and performs various data display processing on the LCD 114. The DSP 103 has a flaw pixel correction circuit inside, and has a function of correcting output data of a designated pixel using output data of peripheral pixels.

  In response to an instruction from the CPU 105, the timing generation circuit 104 supplies a clock signal and a control signal to the image sensor 101, the AFE 102, and the DSP 103, and cooperates with the DSP 103 to generate timing signals corresponding to various reading modes of the CMOS sensor.

  The CPU 105 controls the DSP 103 and the timing generation circuit 104, and controls camera functions using each unit (not shown) such as photometry and distance measurement. For example, a power switch 109, a first-stage shutter switch SW1 110, a second-stage shutter switch SW2 111, a mode dial 112, and an ISO sensitivity setting switch 113 are connected to the CPU 105. The CPU 105 executes processing according to the setting state of these switches and dials.

  The ROM 106 stores a control program (that is, a program executed by the CPU 105) of the imaging apparatus, various correction data, and the like, and generally uses a flash memory. The RAM 107 temporarily stores image data and correction data processed by the DSP 103. Various correction data used by the CPU 105 at the time of processing is stored in the ROM 106, and is developed and used in the RAM 107 at the time of photographing. The RAM 107 is configured to be accessible at a higher speed than the ROM 106.

  As the recording medium 108, for example, a memory card or the like for storing captured images is used. The recording medium 108 is connected to the DSP 103 via a connector (not shown), for example.

  The power switch 109 is operated by the user when starting the imaging apparatus.

  The shutter switch SW1 110 is operated by the user when instructing the start of operations such as photometry processing and distance measurement processing. The shutter switch SW2 111 is operated by the user when instructing the start of a series of imaging operations for driving a mirror and a shutter (not shown) and writing a signal read from the imaging device 101 to the recording medium 108 via the AFE 102 and the DSP 103. The The mode dial 112 is operated by the user when selecting a shooting mode of the imaging apparatus. The ISO sensitivity setting switch 113 is operated by the user when setting the shooting ISO sensitivity of the imaging apparatus. The LCD 114 displays camera information, reproduces and displays captured images, or displays moving image data.

  Next, a method for controlling the imaging apparatus configured as described above will be described.

  FIG. 2 is a flowchart illustrating a method for controlling the imaging apparatus according to the embodiment. This control is mainly executed by the CPU 105.

  First, in S201, it is determined whether or not the power switch 109 is turned on, and this determination is repeated until it is detected that the power switch 109 is turned on. When the power switch 109 is turned on, the process proceeds to S202. In S202, the battery is checked. If it is determined that sufficient power cannot be supplied for shooting, a warning is displayed on the LCD 114, for example, in S211, and the power switch is awaited again. If it is determined that sufficient power can be supplied, the recording medium is checked in S203. If the storage medium is not set or the recordable capacity is insufficient in S203, the process proceeds to S211 and a warning is displayed on the LCD 114.

  If it is confirmed that a recording medium having a sufficient capacity is set, it is determined in S204 whether the mode dial 112 is set to the still image shooting mode or the moving image shooting mode. If the still image shooting mode is set, the process proceeds to the still image shooting of S205, and if the moving image shooting mode is set, the process proceeds to the moving image shooting of S206.

  Hereinafter, the still image shooting operation of S205 will be described with reference to the flowchart of FIG. In S301, it is determined whether or not the shutter switch SW1 110 is ON, and this determination is repeated until it is detected that the shutter switch SW1 110 is ON. When the shutter switch SW1 110 is turned on, the process proceeds to S302.

  In S302, using a photometry control unit and a distance measurement control unit (not shown), photometry processing for determining the aperture value and shutter speed and distance measurement processing for adjusting the photographing lens focus to the subject are performed.

  Next, in S303, it is determined whether or not the shutter switch SW2 111 is ON, and this determination is repeated until it is detected that the shutter switch SW2 111 is ON. If it is detected in S304 that the shutter switch SW1 has not been pressed, the process waits for the shutter switch SW1 to be pressed again in S301. When the shutter switch SW2 111 is turned on, the process proceeds to S305.

  In S305, shooting processing is executed. Details of this photographing process will be described later. In step S <b> 306, the DSP 103 is caused to perform image data development processing. In step S <b> 307, the DSP 103 performs compression processing on the image data for which development processing has been completed, and stores the compressed image data in an empty area of the RAM 107.

  In step S308, the image data stored in the RAM 107 is read out by the DSP 103, and recording processing on the recording medium 108 is executed. After the recording process is completed, the power switch is checked in S309. If the power switch is not turned off, the process returns to S301 again. If it is detected that the power switch is turned off in preparation for the next shooting, the process returns to S201 in FIG. 2 and the power switch is turned on again. wait.

  Next, details of the photographing process in S305 will be described with reference to the flowchart of FIG. First, in S400, the amplifier gain in the image sensor 101, the gain in the AFE 102, and the like are set so as to achieve the sensitivity set by the ISO sensitivity setting SW. In step S402, the mirror is moved to the mirror up position. Next, in step S403, the aperture is driven to a predetermined aperture value based on the photometric data acquired in the photometric processing in step S302. Next, in S404, the charge of the image sensor 101 is erased (cleared). Thereafter, in S405, charge accumulation in the image sensor 101 is started.

  Subsequently, in S406, the shutter is opened, and exposure of the image sensor 101 is started in S407. In step S408, the process waits until the exposure end time elapses according to the photometric data. When the exposure end time elapses, the shutter is closed in step S409.

  Next, in S410, the aperture is driven to the open aperture value. Next, in S411, the mirror is driven to the mirror down position. Thereafter, in S412, the process waits until the set charge accumulation time elapses. When the charge accumulation time elapses, the charge accumulation of the image sensor 101 is terminated in S413.

  Subsequently, in S414, the signal of the image sensor 101 is read out. The AFE 102 having a function of clamping the dark offset level performs a clamping operation using an output from an optical black portion (not shown) of the image sensor 101. Data input from the AFE 102 is subjected to various processing operations such as development, compression, and recording in S306, S307, and S308 by the DSP 103.

  The operation will be described using the configuration diagram of the DSP 103 in FIG. As illustrated, the DSP 103 includes a horizontal scaling unit 1032, an image generation unit 1033, a scaling unit 1034, a moving image generation unit 1035, a memory control unit 1036, a recording medium control unit 1037, and a bus 1031 that connects the above units. Composed.

  Since the horizontal magnification and the vertical magnification at the time of still image shooting are equal, the DSP 103 does not need to perform horizontal scaling processing for reducing the image size in the horizontal direction. The image generation unit 1033 performs arithmetic processing such as development and compression on the input RGB data to generate image data, and controls the recording medium control unit 1037 to record it on the recording medium.

Next, the moving image shooting operation of S206 will be described with reference to the flowchart of FIG. In the present embodiment, the following specifications are assumed for the moving image shooting mode. That is, (1) when the moving image shooting mode is entered, a monitor operation is performed in which a mirror (not shown) is raised, the shutter is opened, image data read from the image sensor is developed and displayed on the LCD.
(2) The moving image data is recorded on the recording medium 108 while the shutter switch SW2 111 is being pressed.
(3) When the setting of the mode dial 112 is changed or when the power switch 109 is turned off, the moving image shooting mode is ended.

  In addition, the size, position, frame rate, and the like of the moving image to be recorded can be set separately by a setting unit (not shown).

  In the present embodiment, the following setting is assumed as an example of a combination of a reading method of an image sensor at the time of moving image shooting, an image size, a position, and a frame rate.

1: High-speed moving image mode First, FIG. 7 shows correspondence of pixels of the image sensor at the time of still image shooting. The pixels on the image sensor are sequentially scanned in both the horizontal and vertical directions, and the signals of all the pixels are read (first reading operation).

  On the other hand, in the high-speed shooting mode in which a high frame rate moving image is shot, as shown in FIG. 8, before the reading operation of the reading target row, between the outputs of the same color pixels on both sides separated by two pixels in the horizontal direction. Addition averaging processing is performed.

  The image pickup device has means for mixing the pixel outputs in an analog manner, and performs a pixel mixing operation with the same color pixels separated by two pixels in the horizontal direction before reading. As a result, the output of the pixel read out is an average value output with the surrounding pixels.

  Next, pixel signals are read out in the row direction (horizontal direction) and the column direction (vertical direction) (second readout operation). For example, at the time of horizontal transfer, reading is performed by thinning out three pixels in the horizontal direction. As a result, the number of pixels read out in the horizontal direction is 1/3 that when reading a still image. Further, even during vertical readout, readout is performed by thinning out 3 pixels at a time. As a result, the number of readout pixels in the vertical direction is 1/3 that when reading a still image.

  As a result, as shown in the lower part of FIG. 8, the size of the output image is 1/3 both horizontally and vertically when the still image size is used as a reference. The total number of readout pixels is 1/9, and the readout time can be greatly shortened.

  The video output size is HD size and the frame rate is 60fps.

2: High-resolution mode In the high-resolution mode, signals of all pixels in the row direction (horizontal direction) are read only in the thinned-out specific rows (third read operation). FIG. 9 shows an image of 1/3 decimation readout in which all pixels are read out in the horizontal direction and 1 out of 3 pixels are read out at regular intervals in the vertical direction. As shown in the example of FIG. 9, all pixels are read out without thinning out the horizontal direction of the read target row. On the other hand, at the time of vertical reading, reading is performed by thinning out three pixels at a time. As a result, the number of readout pixels in the vertical direction is 1/3 that when reading a still image.

  As a result, as shown in the lower part of FIG. 9, the size of the output image is the same as the still image in the horizontal direction and 1/3 in the vertical direction when the still image size is used as a reference.

  The video output size is HD size and the frame rate is 30fps.

3: Low power consumption mode By adding three pixels of the same color in the row direction (horizontal direction) of the image sensor, only one pixel out of the three pixels is read out, and 1/3 addition decimation readout is performed. Also in the column direction (vertical direction), 1/3 thinning-out reading is performed, in which only one of the three pixels is read. The reading method may be the same as the above-described high-speed moving image mode.

  The video output size is HD size and the frame rate is 30fps.

  In any case, although not described here, in an image sensor that can mix pixels in the vertical direction, by performing pixel mixing in the vertical direction as well, the signal at the time of spatial sampling in both the horizontal and vertical directions can be obtained. The aliasing component can be removed.

  The above readout method assumes a CMOS image sensor, and can cope with various readout methods by switching the shift operation of the horizontal and vertical shift registers for each readout method.

  Hereinafter, based on this, the moving image shooting process of S206 will be described. If it is determined in S204 that the movie shooting mode is selected, the movie shooting mode shown in FIG. 6 is entered. First, in step S600, the moving image mode is set. For this setting, the mode dial 112 can select a high-speed moving image mode, a high-resolution mode, and a low power consumption mode.

  In S602, a mirror up (not shown) and a shutter opening (not shown) are performed. As a result, the subject image always enters the image sensor 101.

  In step S603, it is determined whether the shutter switch SW2 111 is pressed. If SW2 111 is not pressed, a monitor operation is repeated to repeat only the monitor display of image data on LCD 114. If SW2 111 is pressed, a recording operation for writing moving image data to the recording medium is started.

  The monitor operation will be described below. If it is determined in S603 that the shutter switch SW2 is OFF, first, in S604, a process for stopping the recording operation is performed if the recording operation is currently in progress. If the recording operation is not currently performed, nothing is done. Thereafter, exposure adjustment is performed in S606.

  The exposure adjustment is executed by determining the exposure amount from the image taken immediately before and adjusting the aperture of the lens and setting the gain so as to obtain an appropriate exposure amount. Since there is no previous data at the beginning of moving image shooting, it is controlled by the initial setting value.

  Thereafter, the moving image shooting process is performed in S607. At the time of moving image shooting, the image sensor 101 continues the operation of repeating charge clearing-accumulating-reading according to the drive signal from the timing generation circuit 104. Here, the image sensor 101 is driven in a moving image shooting mode of a reading method corresponding to the size of the moving image set by the timing generation circuit 104, and repeats the reading operation at a predetermined frame rate. The DSP 103 performs development processing, moving image generation, and compression processing for monitor display, and displays the result on the LCD 114 of the camera. At the time of moving image shooting, this series of operations is repeated at a necessary frame rate in S607.

  If it is determined in S603 that the SW2 111 is pressed, the recording operation is started in S605. When the recording operation is set, the DSP 103 performs development processing, and in S607, performs compression processing for moving image generation and monitor display, and displays the result on the LCD 114 of the camera. At the time of moving image shooting, this series of operations is repeated at a necessary frame rate in S607. This result is written and stored in the recording medium 108. Thereafter, the recording operation is continued until the SW2 111 is turned off.

  Of course, for the sake of safety, even if the SW2 111 is not turned off, the recording operation may be stopped when a predetermined time has elapsed or when the remaining capacity of the recording medium 108 is insufficient. If the on state of the power switch is detected in S612, the mode dial state is detected in S613.

  When the power switch is turned off in S612, or when the still image shooting mode is selected in S613, the process proceeds to the moving image end process in S614 and S615, respectively. Specifically, in S614 and S615, if the recording operation is being performed, the recording operation is terminated, the driving of the image sensor 101 is stopped, the reading process of the DSP 103 is stopped, the shutter (not illustrated) is closed, and the mirror (not illustrated) is performed. Down to S201 and S204, respectively.

  Next, the processing in the DSP 103 in S607 for each operation mode will be described with reference to FIG.

1: High-speed moving image mode In the high-speed moving image mode, data of the number of pixels in the horizontal direction 1/3 and the vertical direction 1/3 of a still image is processed. The state of the pixels read from the image sensor at this time and the overall pixel size will be described. The image data entered into the DSP 103 is first subjected to a clipping process in which the ratio of the number of pixels in the vertical and horizontal directions of the image sensor is matched to the aspect ratio of the moving image. This is realized by the memory control unit 1036 writing out only the necessary area data for the input image data to the memory.

  Thereafter, the image generation unit 1033 performs arithmetic processing such as development on the RGB data of the image data that has been cut out. Next, the scaling unit 1034 performs horizontal and vertical scaling processing to a predetermined moving image size, and then the moving image generation unit 1035 compresses and encodes image data generated at a predetermined frame rate. Thus, final moving image data is generated.

  In this mode, since the number of readout pixels is reduced to 1/9, the readout speed can be improved and 60 fps can be realized. Therefore, the DSP 103 performs processing corresponding to 60 fps.

2: High resolution mode In the high resolution mode, data having the same number of pixels as the still image in the horizontal direction and 1/3 of the number of pixels in the vertical direction is processed. Therefore, since it takes approximately three times as long as the readout time compared with the high-speed moving image mode, the operation at 60 fps is difficult. Further, since the magnification in the horizontal direction is different from that in the vertical direction, the DSP 103 needs to perform horizontal scaling processing for reducing the image size in the horizontal direction. If the horizontal pixel addition is performed on the image sensor, the process is limited to a simple process such as an averaging process. On the other hand, in the DSP 103, the horizontal scaling unit 1032 can perform calculation based on a reduction algorithm that applies various filter processes as an algorithm of the horizontal scaling process.

  As a result, it is possible to obtain a high-quality reduced image that is higher in definition and not affected by aliasing noise, as compared with the case of calculating the pixel addition average in the image sensor.

  Although the above-described processing is described as being processed by a dedicated processing circuit as shown in FIG. 5, it is also possible to execute the above-described processing by a DSP program.

  Next, as in the high-speed moving image mode, a clipping process is performed in which the ratio of the number of vertical and horizontal pixels of the image sensor is aligned with the aspect ratio of the number of pixels of the moving image. This is realized by the memory control unit 1036 writing out only the necessary area data for the input image data to the memory.

  Thereafter, the image generation unit 1033 performs arithmetic processing such as development on the RGB data of the image data that has been cut out. Next, the scaling unit 1034 performs horizontal and vertical scaling processing to a desired moving image size, and then the moving image generation unit 1035 compresses and encodes image data generated at a predetermined frame rate. Thus, final moving image data is generated.

  In this mode, the number of readout pixels is reduced to only one third of that of a still image, so the readout speed is not improved as compared with the high-speed moving image mode, and is processed as a frame rate of 30 fps. Therefore, the DSP 103 performs processing corresponding to 30 fps.

3: Low power consumption mode In the low power consumption mode, the operation sequence itself is the same as the high-speed moving image mode. However, the frame rate is 30 fps. That is, since the time required for reading out the image sensor during the time of 1 frame (1/30 sec) is the same as that in the case of 60 fps, a vertical blanking time of at least 1/60 sec occurs at least every frame. Power control per unit time can be reduced by performing power control of the image pickup device 101, the AFE 102, and the circuit block of the image pickup system in the DSP 103 in accordance with an instruction from the timing generation circuit 104 or the CPU 105 during this period. For this reason, an image pickup apparatus that operates on a battery has an effect of greatly extending the recording time per battery.

  As described above, in the present embodiment, even when moving image outputs of the same size are obtained, the readout method of the image sensor is switched according to the operation mode, and scaling processing corresponding to each readout method is appropriately performed. . As a result, even in a system using the same image sensor, it is possible to select any of frame rate priority, resolution priority, and low power consumption priority according to conditions.

  In the above-described embodiment, the horizontal scaling unit 1032 is configured to perform scaling processing only in the horizontal direction. However, even if the circuit scale is large, the horizontal scaling unit 1032 may be configured to scale in the horizontal and vertical directions. Good. In this case, if the scaling factor can be set independently in the horizontal and vertical directions, and the vertical scaling factor is set to 1, a configuration equivalent to the above-described embodiment can be obtained.

  Further, in the above-described embodiment, in the case of an operation example in which all pixels are read in a still image and horizontal 1/3 thinning in the moving image mode and the vertical direction is 1/3 thinning, the image data input to the scaling unit 1034 Since the aspect ratio is the same as for still images, both horizontal and vertical can be handled with the same scaling factor. However, for example, when the scaling factor of the horizontal scaling unit 1032 cannot be set to exactly 1/3, the horizontal scaling factor may be set to a value different from the vertical scaling factor on the scaling unit 1034 side. By doing so, it becomes possible to make the aspect ratio of the image data the same as that in the still image.

Claims (4)

An image sensor in which a plurality of pixels including photoelectric conversion elements for photoelectrically converting a subject image are arranged in a row direction and a column direction;
Setting means for setting the shooting mode;
Depending on the shooting mode set by the setting means, the pixel signal readout operation from the image sensor,
(A) a first read operation for reading signals of all pixels of the image sensor;
(B) a second reading operation of reading out a pixel signal by thinning out both the row direction and the column direction among the plurality of pixels;
(C) a third readout operation of reading out signals of all pixels in the row direction only in a specific row that is thinned out of the plurality of pixels;
A selection means for selecting any one of
In photographing processing, processing means for generating a still image or a moving image using a signal of a pixel obtained by executing the selected signal reading operation;
An imaging device comprising: When the still image shooting mode is set by the setting unit, the selection unit selects the first reading operation,
When the setting means selects a high-speed moving image mode in which high frame rate moving image shooting is performed or a low power consumption mode in which low power consumption priority moving image shooting is selected, the selection unit selects the second readout operation. ,
When the high resolution mode for performing high resolution video shooting is set by the setting means, the selection means selects the third readout operation.
The imaging apparatus according to claim 1.   The image processing apparatus further includes a scaling unit that performs scaling processing of the moving image so that the aspect ratio of the moving image generated by the processing unit is the same as the aspect ratio of the still image generated in the still image shooting mode. The imaging device according to claim 2.   The imaging apparatus according to claim 2 or 3, wherein a frame rate of a moving image generated in the low power consumption mode is lower than a frame rate of a moving image generated in the high-speed moving image mode.

Images