JP2019092219A - Imaging apparatus, control method of the same, and control program - Google Patents

Abstract

An object of the present invention is to reduce power consumption and processing load in an imaging apparatus. Kind Code: A1 A camera-shake amount calculation unit 32 for calculating a camera-shake amount; A control unit 72 is provided. [Selection drawing] Fig. 7

Description

  The present invention relates to an imaging device, a control method of an imaging device, and a control program.

  There has been proposed an electronic device provided with an imaging device (hereinafter, this imaging device is referred to as a laminated imaging device) in which a back side illumination type imaging chip and a signal processing chip are stacked (see, for example, Patent Document 1). The stacked imaging device is stacked such that the back side illumination type imaging chip and the signal processing chip are connected via the microbumps in units of blocks each including a plurality of pixels.

JP, 2006-49361, A

  However, in the imaging device provided with the conventional laminated imaging device, there are not many proposals for acquiring an image by imaging in units of a plurality of blocks, and the usability of the imaging device provided with the laminated imaging device has not been sufficient.

  An aspect of the present invention is to reduce power consumption and processing load in an imaging device.

  According to the first aspect of the present invention, a camera shake amount calculation unit for calculating a camera shake amount, drive control of the imaging device, and an area for driving control of the imaging device according to the camera shake amount calculated by the camera shake amount calculation unit And a drive control unit.

  According to a second aspect of the present invention, an imaging unit having an imaging element, and an image processing unit that performs image processing on an image signal output from the imaging unit, the imaging unit calculating a shake amount An imaging apparatus comprising: an amount calculation unit; and a signal output unit that outputs an image signal of an area according to the shake amount calculated by the shake amount calculation unit among the image signals read from the imaging device to the image processing unit Is provided.

  According to a third aspect of the present invention, there is provided a control method of an image pickup apparatus having an image pickup element, which comprises: calculating a shake amount; and changing an area for driving and controlling the image pickup element according to the calculated shake amount. And a control method of the imaging device.

  According to the fourth aspect of the present invention, the control device of the imaging device having the imaging element controls the driving of the imaging element according to the camera shake amount calculation processing for calculating the camera shake amount and the camera shake amount calculated in the camera shake amount calculation processing. A control program is provided to execute area change processing for changing an area to be processed.

  According to a fifth aspect of the present invention, there is provided a control method of an image pickup apparatus having an image pickup element, which comprises: calculating a shake amount; and an image signal read out from the image pickup element according to the calculated shake amount. There is provided a control method of an imaging device, including outputting an image signal of a region to an image processing unit.

  According to the sixth aspect of the present invention, in the control device of the imaging device having the imaging element, the camera shake amount calculation processing for calculating the amount of camera shake and the image stabilization amount calculation processing among the image signals read from the imaging element A control program is provided that executes signal output processing of outputting an image signal of an area according to the calculated amount of camera shake to an image processing unit.

  According to the aspect of the present invention, it is possible to reduce the power consumption and the processing load in the imaging device.

It is a sectional view of a lamination type image sensor. It is a figure explaining the pixel array and unit group of an imaging chip. It is a circuit diagram corresponding to the unit group of an imaging chip. It is a block diagram showing functional composition of an image sensor. It is a block diagram showing composition of an imaging device concerning a 1st embodiment. It is a block diagram showing the concrete composition of the signal processing chip concerning a 1st embodiment. FIG. 6 is a functional block diagram of an image processing unit and a system control unit shown in FIG. 5; It is a flowchart which shows an example of the control method of the imaging device which concerns on 1st Embodiment. FIG. 6 is a diagram showing the moving direction and the amount of movement of the subject during camera shake. FIG. 5 is a diagram showing the size of image data in an imaging unit, an image processing unit, and a recording unit according to the first embodiment. It is a block diagram showing composition of an imaging device concerning a 2nd embodiment. FIG. 12 is a functional block diagram of an image processing unit and a system control unit shown in FIG. It is a flow chart which shows an example of the control method of the imaging device concerning a 2nd embodiment. It is a block diagram which shows the specific structure of the signal processing chip which concerns on 3rd Embodiment. It is a flow chart which shows an example of the control method of the imaging device concerning a 3rd embodiment. It is a figure which shows the size of the image data in the imaging part which concerns on 3rd Embodiment, and an image processing part. It is a figure which shows the size of the image data in the imaging part which concerns on 4th Embodiment, and an image processing part. It is a block diagram which shows the structure of the imaging device and electronic device which concern on 5th Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to this. Further, in the drawings, in order to explain the embodiment, the scale is appropriately changed and expressed such that a part is described in a large or emphasized manner.

First Embodiment
FIG. 1 is a cross-sectional view of a stacked imaging device. The stacked imaging device 100 is described in Japanese Patent Application No. 2012-139026 filed by the applicant of the present application. The imaging device 100 includes an imaging chip 113 which outputs a pixel signal corresponding to incident light, a signal processing chip 111 which processes the pixel signal, and a memory chip 112 which stores the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by the bump 109 having conductivity such as Cu.

  As illustrated, incident light is mainly incident in the Z-axis plus direction indicated by the outlined arrow. In the present embodiment, in the imaging chip 113, the surface on which incident light is incident is referred to as the back surface. Further, as indicated by the coordinate axes, the left direction in the drawing, which is orthogonal to the Z axis, is the plus direction of the X axis, and the near direction in the drawing, which is orthogonal to the Z axis and the X axis is the plus direction of the Y axis. In the following several figures, coordinate axes are displayed so that the orientation of each figure can be known with reference to the coordinate axes in FIG.

  An example of the imaging chip 113 is a backside illumination type MOS image sensor. The PD layer 106 is disposed on the back side of the wiring layer 108. The PD layer 106 includes a plurality of photodiodes (hereinafter, referred to as PD) 104 which are two-dimensionally arranged and which accumulate charges corresponding to incident light, and a transistor 105 provided corresponding to the PD 104. .

  A color filter 102 is provided on the incident side of incident light in the PD layer 106 via a passivation film 103. The color filter 102 is a filter that passes a specific wavelength region of visible light. The color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filters 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  A microlens 101 is provided on the color filter 102 on the incident side of the incident light corresponding to each pixel. The microlenses 101 condense incident light toward the corresponding PDs 104.

  The wiring layer 108 has a wiring 107 for transmitting the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be a multilayer, and passive elements and active elements may be provided. A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposite surface of the signal processing chip 111. Then, by pressing the imaging chip 113 and the signal processing chip 111, the aligned bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are disposed on the surfaces facing each other of the signal processing chip 111 and the memory chip 112. These bumps 109 are aligned with one another. Then, the signal processing chip 111 and the memory chip 112 are pressurized or the like, whereby the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Also, about one bump 109 may be provided, for example, for one unit group described later. Therefore, the size of the bumps 109 may be larger than the pitch of the PDs 104. Further, in the peripheral area other than the pixel area (pixel area 113A shown in FIG. 2) in which the pixels are arranged, bumps larger than the bumps 109 corresponding to the pixel area may be provided.

  The signal processing chip 111 has TSVs (Through-Silicon Vias) 110 which connect circuits respectively provided on the front and back surfaces. The TSVs 110 are provided in the peripheral area. Also, the TSV 110 may be provided in the peripheral region of the imaging chip 113 or in the memory chip 112.

  FIG. 2 is a diagram for explaining a pixel array and a unit group of the imaging chip. FIG. 2 particularly shows a state in which the imaging chip 113 is observed from the back side. An area in which pixels are arranged in the imaging chip 113 is referred to as a pixel area 113A. More than 20 million pixels are arranged in a matrix in the pixel area 113A. In the example shown in FIG. 2, 16 adjacent 4 × 4 pixels form one unit group 131. The grid lines in FIG. 2 show the concept that adjacent pixels are grouped to form a unit group 131. The number of pixels forming the unit group 131 is not limited to this, and may be about 1000, for example, 32 pixels × 64 pixels, or more or less.

  As shown in the partial enlarged view of the pixel area 113A, the unit group 131 includes four so-called Bayer arrays consisting of four pixels of green pixels Gb and Gr, blue pixels B, and red pixels R in the top, bottom, left, and right. The green pixel is a pixel having a green filter as the color filter 102, and receives light in the green wavelength band of incident light. Similarly, the blue pixel is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band. The red pixel is a pixel having a red filter as the color filter 102 and receives light in the red wavelength band.

  FIG. 3 is a circuit diagram corresponding to a unit group of imaging chips. In FIG. 3, a rectangle surrounded by a dotted line representatively represents a circuit corresponding to one pixel. Note that at least a part of each of the transistors described below corresponds to the transistor 105 in FIG.

  As described above, the unit group 131 is formed of 16 pixels. The 16 PDs 104 corresponding to each pixel are connected to the transfer transistor 302, respectively. The gate of each transfer transistor 302 is connected to a TX wiring 307 to which a transfer pulse is supplied. In the present embodiment, the TX wiring 307 is commonly connected to the sixteen transfer transistors 302.

  The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and a so-called floating diffusion FD (charge detection unit) between the drain of the transfer transistor 302 and the source of each reset transistor 303 is an amplifier transistor 304. Connected to the gate. The drain of each reset transistor 303 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. The gate of each reset transistor 303 is connected to a reset wiring 306 to which a reset pulse is supplied. In the present embodiment, the reset wiring 306 is commonly connected to the sixteen reset transistors 303.

  The drain of each amplification transistor 304 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. Also, the source of each amplification transistor 304 is connected to the drain of each corresponding selection transistor 305. The gate of each selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. In the present embodiment, the decoder wiring 308 is provided independently for each of the 16 selection transistors 305. The source of each selection transistor 305 is connected to the common output wiring 309. The load current source 311 supplies a current to the output wiring 309. That is, the output wiring 309 for the selection transistor 305 is formed by a source follower. The load current source 311 may be provided on the imaging chip 113 side or may be provided on the signal processing chip 111 side.

  Here, the flow from the charge accumulation start to the pixel output after the charge end will be described. A reset pulse is applied to the reset transistor 303 through the reset wiring 306. At the same time, a transfer pulse is applied to the transfer transistor 302 through the TX wiring 307. Thereby, the potentials of the PD 104 and the floating diffusion FD are reset.

  When the application of the transfer pulse is released, the PD 104 converts incident light to be received into charge and accumulates it. Thereafter, when the transfer pulse is applied again while the reset pulse is not applied, the charges accumulated in the PD 104 are transferred to the floating diffusion FD. Thereby, the potential of the floating diffusion FD changes from the reset potential to the signal potential after charge accumulation. Then, when a selection pulse is applied to the selection transistor 305 through the decoder wiring 308, the fluctuation of the signal potential of the floating diffusion FD is transmitted to the output wiring 309 through the amplification transistor 304 and the selection transistor 305. By such operation of the circuit, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

  As shown in FIG. 3, in the present embodiment, the reset wiring 306 and the TX wiring 307 are common to 16 pixels forming the unit group 131. That is, the reset pulse and the transfer pulse are simultaneously applied to all 16 pixels. Therefore, all the pixels forming the unit group 131 start charge accumulation at the same timing and end charge accumulation at the same timing. However, pixel signals corresponding to the accumulated charges are selectively output to the output wiring 309 by sequentially applying selection pulses to the respective selection transistors 305. The reset wiring 306, the TX wiring 307, and the output wiring 309 are separately provided for each unit group 131.

  By thus configuring the circuit based on the unit group 131, the charge accumulation time can be controlled for each unit group 131. In other words, pixel signals with different charge accumulation times can be output between unit groups 131, respectively. Furthermore, while causing one unit group 131 to perform one charge accumulation, the other unit group 131 repeats the charge accumulation many times and outputs the pixel signal each time. It is also possible to output each frame for moving image at a different frame rate between unit groups 131.

  FIG. 4 is a block diagram showing a functional configuration of the imaging device. The analog multiplexer 411 selects 16 PDs 104 forming the unit group 131 in order. Then, the multiplexer 411 outputs the pixel signal of each of the 16 PDs 104 to the output wiring 309 provided corresponding to the unit group 131. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

  The pixel signal of the analog signal output via the multiplexer 411 is amplified by the amplifier 412 formed in the signal processing chip 111. Then, the pixel signal amplified by the amplifier 412 is processed by the signal processing circuit 413 which is formed in the signal processing chip 111 and performs correlated double sampling (CDS; Correlated Double Sampling) and analog / digital (Analog / Digital) conversion. Signal processing of correlated double sampling is performed, and A / D conversion (conversion from analog signal to digital signal) is performed. The signal processing of the correlated double sampling in the signal processing circuit 413 of the pixel signal reduces noise of the pixel signal. The A / D converted pixel signals are delivered to the demultiplexer 414 and stored in the pixel memory 415 corresponding to each pixel. The demultiplexer 414 and the pixel memory 415 are formed in the memory chip 112.

  The arithmetic circuit (camera shake amount calculation unit) 416 processes the pixel signal stored in the pixel memory 415 and delivers it to the image processing unit in the subsequent stage. In the present embodiment, the arithmetic circuit 416 is provided in the signal processing chip 111. However, the arithmetic circuit 416 may be provided in the memory chip 112. Although FIG. 4 shows the connection of one unit group 131, in actuality, these units exist for each unit group 131 and operate in parallel. However, the arithmetic circuit 416 may not be present for each unit group 131. For example, one arithmetic circuit 416 may sequentially process the values while referring to the values of the pixel memory 415 corresponding to each unit group 131 in order.

  As described above, the output wirings 309 are provided corresponding to each of the unit groups 131. The imaging element 100 is formed by stacking an imaging chip 113, a signal processing chip 111, and a memory chip 112. Therefore, by using the electrical connection between the chips using the bumps 109 for the output wirings 309, the wirings can be routed without enlarging each chip in the surface direction.

  Next, the blocks set in the pixel area 113A (see FIG. 2) of the imaging device 100 will be described. In the present embodiment, the pixel area 113A of the imaging device 100 is divided into a plurality of blocks. The plurality of blocks are defined to include at least one unit group 131 per block. In each block, pixels included in each block are controlled with different control parameters. That is, pixel signals having different control parameters are acquired between the pixel group included in a certain block and the pixel group included in another block. The control parameters include, for example, charge accumulation time or accumulation number, frame rate, gain, thinning rate, number of added rows or number of added columns for adding pixel signals, number of bits for digitization, and the like. Furthermore, the control parameter may be a parameter in image processing after acquisition of an image signal from a pixel.

  Here, the charge accumulation time refers to the time from when the PD 104 starts charge accumulation until it ends. Further, the number of times of charge accumulation refers to the number of times that the PD 104 accumulates charges per unit time. The frame rate is a value representing the number of frames processed (displayed or recorded) per unit time in a moving image. The unit of frame rate is represented by fps (Frames Per Second). The higher the frame rate, the smoother the motion of the subject in the video (ie the subject being imaged).

  Further, the gain refers to the gain factor (amplification factor) of the amplifier 412. By changing this gain, the ISO sensitivity can be changed. The ISO sensitivity is a standard of photographic film formulated by ISO, and indicates how weak the photographic film can record light. However, in general, the ISO sensitivity is also used when expressing the sensitivity of the imaging device 100. In this case, the ISO sensitivity is a value representing the ability of the imaging device 100 to capture light. Increasing the gain also improves the ISO sensitivity. For example, when the gain is doubled, the electric signal (pixel signal) is also doubled, and the appropriate brightness is obtained even if the amount of incident light is half. However, when the gain is increased, the noise contained in the electrical signal is also amplified, resulting in an increase in noise.

  Further, the thinning rate refers to the ratio of the number of pixels for which readout of pixel signals is not performed to the total number of pixels in a predetermined area. For example, when the thinning rate of the predetermined area is 0, it means that the reading of the pixel signal is performed from all the pixels in the predetermined area. Further, when the thinning rate of the predetermined area is 0.5, it means that the pixel signal is read out from half of the pixels in the predetermined area. Specifically, when the unit group 131 has a Bayer arrangement, every other unit of the Bayer arrangement in the vertical direction, that is, pixels in which pixel signals are read out alternately by two pixels (two rows) in pixel units. Pixels not to be output are set. Note that the resolution of the image is reduced when thinning out of reading out the pixel signals is performed. However, since 20 million or more pixels are disposed in the imaging device 100, an image can be displayed with 10 million or more pixels even if thinning is performed with a thinning rate of 0.5, for example. For this reason, it is considered that the user (photographer) does not mind the reduction in resolution.

  Further, the number of addition rows refers to the number of pixels in the vertical direction (the number of rows) to be added when pixel signals of pixels adjacent in the vertical direction are added. Further, the number of addition columns means the number of pixels in the horizontal direction (the number of columns) to be added when pixel signals of pixels adjacent in the horizontal direction are added. Such addition processing is performed, for example, in the arithmetic circuit 416. The arithmetic circuit 416 performs processing of adding pixel signals of a predetermined number of pixels adjacent in the vertical direction or horizontal direction, thereby achieving an effect similar to the processing of reading out pixel signals by thinning at a predetermined thinning rate. In the addition processing described above, the average value may be calculated by dividing the addition value by the number of rows or the number of columns added by the arithmetic circuit 416.

  Further, the number of bits for digitizing refers to the number of bits when the signal processing circuit 413 converts an analog signal into a digital signal in A / D conversion. As the number of bits of the digital signal increases, the brightness, color change, etc. are expressed in more detail.

  In the present embodiment, the storage condition refers to a condition regarding charge storage in the imaging device 100. Specifically, the storage conditions refer to the charge storage time or number, the frame rate, and the gain among the control parameters described above. The frame rate is included in the accumulation condition because the frame rate may change according to the charge accumulation time and the number of times of accumulation. Further, the amount of light of the proper exposure changes according to the gain, and the accumulation time or the number of accumulations of the charge may also change according to the amount of light of the proper exposure. Therefore, the gain is included in the accumulation condition.

  Further, the imaging condition refers to a condition regarding imaging of a subject. Specifically, the imaging condition refers to a control parameter including the storage condition described above. The imaging conditions are control parameters for controlling readout of a signal from the imaging element 100 in addition to control parameters for controlling the imaging element 100 (for example, charge accumulation time or number of accumulation, frame rate, gain). For example, thinning rate), control parameters for processing a signal from the image sensor 100 (for example, the number of addition rows or addition columns for adding pixel signals, the number of bits for digitization, image processing to be described later Control parameters) to execute.

  FIG. 5 is a block diagram showing the configuration of the imaging device according to the first embodiment. The imaging device 1 illustrated in FIG. 5 is configured by, for example, devices such as a digital camera provided with an imaging function, a smartphone, a mobile phone, and a personal computer. As shown in FIG. 5, the imaging device 1 includes a lens unit 10, an imaging unit 20, an image processing unit 30, a work memory 40, a display unit 50, an operation unit 55, a recording unit 60, and a system control unit 70. The lens unit 10 is an imaging optical system configured of a plurality of lens groups. The lens unit 10 guides the light flux from the subject to the imaging unit 20. The lens unit 10 may be integrated with the imaging device 1, or may be a replaceable lens that is detachable from the imaging device 1. Further, the lens unit 10 may incorporate a focus lens or may incorporate a zoom lens.

  The imaging unit 20 includes an imaging element 100 and a drive unit 21. The drive unit 21 is a control circuit that controls the drive of the imaging device 100 according to an instruction from the system control unit 70. Here, the drive unit 21 controls the timing (or the cycle of timing) of applying the reset pulse and the transfer pulse to the reset transistor 303 and the transfer transistor 302, respectively, to thereby store the charge accumulation time or number of accumulations as a control parameter. Control. In addition, the drive unit 21 controls the frame rate by controlling the timing (or the cycle of timing) of applying the reset pulse, the transfer pulse, and the selection pulse to the reset transistor 303, the transfer transistor 302, and the selection transistor 305, respectively. Do. Further, the drive unit 21 controls the thinning rate by setting pixels to which a reset pulse, a transfer pulse, and a selection pulse are applied.

  In addition, the drive unit 21 controls the ISO sensitivity of the imaging element 100 by controlling the gain (also referred to as a gain factor or amplification factor) of the amplifier 412. Further, the drive unit 21 sends an instruction to the arithmetic circuit 416 to set the number of addition rows or the number of addition columns to which the pixel signal is added. Further, the drive unit 21 sets the number of bits for digitization by sending an instruction to the signal processing circuit 413. Further, the drive unit 21 performs setting of a block in the pixel area 113A of the imaging device 100. As described above, the driving unit 21 has a function of an imaging device control unit that causes the imaging device 100 to capture an image under different imaging conditions for each of a plurality of blocks and output a pixel signal. The system control unit 70 instructs the drive unit 21 on the position, shape, range, and the like of the block.

  The imaging element 100 delivers the pixel signal from the imaging element 100 to the image processing unit 30. The image processing unit 30, using the work memory 40 as a work space, performs various types of image processing on RAW data consisting of pixel signals of respective pixels, and generates image data of a predetermined file format (for example, JPEG format etc.) . In the present specification, “image data” is data that configures an image (still image, moving image, live view image) captured by the imaging unit 20, and before image processing is performed in the image processing unit 30. It includes data (i.e., RAW data) and data after image processing has been performed. Further, in the present specification, “RAW data” refers to image data before image processing is performed in the image processing unit 30. Further, in the present specification, "image data" may be referred to as "image signal".

  The image processing unit 30 executes various image processing. For example, the image processing unit 30 generates an RGB image signal by performing color signal processing (tone correction) on the signal obtained by the Bayer arrangement. The image processing unit 30 also performs image processing such as white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the RGB image signal. In addition, the image processing unit 30 performs processing of compressing in a predetermined compression format (JPEG format, MPEG format, etc.) as necessary. The image processing unit 30 outputs the generated image data to the recording unit 60. Further, the image processing unit 30 outputs the generated image data to the display unit 50.

  Parameters referred to when the image processing unit 30 performs image processing are also included in the control parameters (imaging conditions). For example, parameters such as color signal processing (tone correction), white balance adjustment, tone adjustment, and compression ratio are included in the control parameters. A signal read from the imaging device 100 changes according to the charge accumulation time and the like, and a parameter referred to when performing image processing also changes according to the change of the signal. The image processing unit 30 sets different control parameters for each block and executes image processing such as color signal processing based on these control parameters.

  The image processing unit 30 extracts frames at predetermined timings among a plurality of frames obtained in chronological order from the imaging unit 20. Alternatively, the image processing unit 30 discards frames at predetermined timings among a plurality of frames obtained chronologically from the imaging unit 20. As a result, the amount of data can be reduced, and the load on subsequent processing can be reduced. Further, the image processing unit 30 calculates one or a plurality of frames to be interpolated between the frames based on a plurality of frames obtained in chronological order from the imaging unit 20. Then, the image processing unit 30 adds the calculated one or more frames between the frames. This makes it possible to reproduce a moving image with smoother motion when reproducing the moving image. Moreover, although the drive part 21 is comprised so that a thinning-out rate may be controlled, it is not restricted to such a structure. For example, although the drive unit 21 reads pixel signals from all pixels, the image processing unit 30 or the arithmetic circuit 416 may control the thinning rate by discarding predetermined pixel signals among the read pixel signals. Good.

  In the present embodiment, in addition to the above-described processing, the image processing unit 30 performs processing for calculating a camera shake amount (movement amount due to camera shake) by analysis processing of image data read from the imaging device 100. That is, the image processing unit 30 calculates the shake amount based on the movement direction and movement amount of the subject in a plurality of frames (at least the current frame and a frame before the current frame) obtained chronologically from the imaging unit 20 (See, for example, FIG. 9). Then, the image processing unit 30 outputs camera shake amount information indicating the calculated camera shake amount to the system control unit 70.

  The work memory 40 temporarily stores image data and the like when image processing by the image processing unit 30 is performed. The display unit 50 is configured of, for example, a liquid crystal display panel. The display unit 50 displays an image (still image, moving image, live view image) captured by the imaging unit 20 and various information. The display unit 50 has a touch panel 51 as shown in FIG. The touch panel 51 is formed on the display screen of the display unit 50. The touch panel 51 outputs a signal indicating a position touched by the user to the system control unit 70 when the user performs an image selection or the like.

  The operation unit 55 is a release switch operated by the user, a moving image switch, various operation switches, and the like. The operation unit 55 outputs a signal corresponding to the operation by the user to the system control unit 70. The recording unit 60 has a card slot in which a storage medium such as a memory card can be attached. The recording unit 60 stores the image data and various data generated by the image processing unit 30 in a recording medium loaded in the card slot. The recording unit 60 also has an internal memory. The recording unit 60 can also record the image data and various data generated by the image processing unit 30 in the internal memory.

  The system control unit 70 controls the overall processing and operation of the imaging device 1. The system control unit 70 has a CPU (Central Processing Unit). In the present embodiment, the system control unit 70 divides the imaging surface (pixel area 113A) of the imaging element 100 (imaging chip 113) into a plurality of blocks, and the charge accumulation time (or charge accumulation number) different between blocks, the frame rate , Get an image with gain. Therefore, the system control unit 70 instructs the drive unit 21 on the position, shape, range of the block, and storage conditions for each block. In addition, the system control unit 70 obtains an image with a thinning rate different between blocks, the number of addition rows or number of addition columns for adding pixel signals, and the number of bits of digitization. Therefore, the system control unit 70 instructs the drive unit 21 of imaging conditions for each block (decimation ratio, number of addition rows or number of addition columns for adding pixel signals, and number of bits for digitization). Further, the image processing unit 30 executes image processing under imaging conditions (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression ratio) which differ between blocks. Therefore, the system control unit 70 instructs the image processing unit 30 on imaging conditions for each block (color signal processing, white balance adjustment, gradation adjustment, control parameters such as compression ratio, etc.).

  The system control unit 70 also causes the recording unit 60 to record the image data generated by the image processing unit 30. The system control unit 70 causes the display unit 50 to display an image by causing the display unit 50 to output the image data generated by the image processing unit 30. The system control unit 70 causes the display unit 50 to display an image by reading out the image data recorded in the recording unit 60 and causing the display unit 50 to output the image data. Images displayed on the display unit 50 include still images, moving images, and live view images. Here, the live view image is an image displayed on the display unit 50 by sequentially outputting the image data generated by the image processing unit 30 to the display unit 50. The live view image is used by the user to confirm the image of the subject captured by the imaging unit 20. Live view images are also referred to as through images and preview images.

  In the present embodiment, in addition to the above-described processing, the system control unit 70 performs processing of selecting an area for driving and controlling the imaging element 100 according to the shake amount indicated by the shake amount information from the image processing unit 30. Then, the system control unit 70 outputs an instruction signal specifying the selected area to the drive unit 21. Based on an instruction signal from the system control unit 70, the drive unit 21 reads out pixel signals only in the area within the pixel area 113A, and processes image data (RAW data) composed of pixel signals of each pixel in the area Transfer to 30 The area in the pixel area 113A selected by the system control unit 70 according to the shake amount is referred to as a “transfer area”.

  FIG. 6 is a block diagram showing a specific configuration of the signal processing chip according to the first embodiment. In addition to the signal processing chip 111, FIG. 6 also shows the imaging chip 113, the memory chip 112, the image processing unit 30, and the system control unit 70.

  The signal processing chip 111 bears the function of the drive unit 21. The signal processing chip 111 includes a sensor control unit 441 as a shared control function, a block control unit 442, a synchronization control unit 443 and a signal control unit 444. Further, the signal processing chip 111 includes a control unit (drive control unit) 420 that centrally controls the sensor control unit 441, the block control unit 442, the synchronization control unit 443, and the signal control unit 444. The control unit 420 converts an instruction from the system control unit 70 into a control signal executable by each of the control units 441 to 444 and delivers the control signal to each.

  The sensor control unit 441 controls to send control pulses (reset pulse, transfer pulse, selection pulse) related to charge accumulation and charge readout of each pixel to the imaging chip 113 based on the control signal from the control unit 420. Do. Specifically, the sensor control unit 441 controls the start and end of charge accumulation by sending a reset pulse and a transfer pulse to the target pixel. Further, the sensor control unit 441 outputs the pixel signal to the output wiring 309 by transmitting a selection pulse to the readout pixel of the pixel signal.

  The block control unit 442 performs control to send a specific pulse specifying the unit group 131 to be drive-controlled to the imaging chip 113 based on the control signal from the control unit 420. As described above, the pixel area 113A of the imaging device 100 is divided into a plurality of blocks. The plurality of blocks include at least one unit group 131 per block. The pixels included in the same block start charge accumulation at the same timing and end charge accumulation at the same timing. Therefore, the block control unit 442 blocks one or more unit groups 131 by transmitting a specific pulse to the target unit group 131 based on a control signal from the control unit 420. The transfer pulse and the reset pulse that each pixel receives through the TX wiring 307 and the reset wiring 306 are the logical product of each pulse sent out by the sensor control unit 441 and the specific pulse sent out by the block control unit 442. As described above, the block control unit 442 controls the respective areas as blocks independent of each other to realize charge accumulation control for each area.

  The synchronization control unit 443 performs control to send out the synchronization signal to the imaging chip 113 based on the control signal from the control unit 420. Each pulse (reset pulse, transfer pulse, selection pulse) becomes active in the imaging chip 113 in synchronization with the synchronization signal. For example, the synchronization control unit 443 adjusts the synchronization signal to thereby realize random control, thinning control, and the like in which only specific pixels of pixels belonging to the same unit group 131 are targeted for drive control.

  The signal control unit 444 controls the gain (gain factor, amplification factor) of the amplifier 412 based on the control signal from the control unit 420. The signal control unit 444 also performs timing control on the A / D converter 413 b based on the control signal from the control unit 420. The pixel signal output via the output wiring 309 is input to the A / D converter 413 b via the amplifier 412 and the CDS circuit 413 a. The A / D converter 413 b is controlled by the signal control unit 444 to convert the input pixel signal into a digital signal. The pixel signal converted into the digital signal is delivered to the demultiplexer 414. Then, the pixel signal is stored by the demultiplexer 414 in the pixel memory 415 corresponding to each pixel as a pixel value of digital data.

  The signal processing chip 111 has a timing memory 430. Based on an instruction from system control unit 70, control unit 420 repeats block accumulation information on which unit groups 131 are combined to form a block and how many times each of the formed blocks accumulates charge. The information on the number of times of accumulation (that is, the timing of charge accumulation) is stored in the timing memory 430. The timing memory 430 is constituted by, for example, a flash RAM. The control unit 420 executes charge accumulation control in units of blocks by referring to the block division information and the number-of-accumulations information stored in the timing memory 430.

  As described above, the system control unit 70 selects the transfer area for driving and controlling the imaging device 100 according to the amount of camera shake. Then, system control unit 70 outputs an instruction signal specifying the selected transfer area to control unit 420. Based on an instruction signal from system control unit 70, control unit 420 performs control to combine a plurality of unit groups to form a block corresponding to a transfer area.

  FIG. 7 is a functional block diagram of the image processing unit and the system control unit shown in FIG. As shown in FIG. 7, the image processing unit 30 includes an image generation unit 31 and a camera shake detection unit (camera shake amount calculation unit) 32. The image generation unit 31 generates image data by performing various types of image processing on RAW data composed of pixel signals of respective pixels output from the imaging unit 20. The camera shake detection unit 32 detects a motion vector based on the movement direction and the movement amount of the subject due to camera shake in a plurality of frames obtained chronologically from the imaging unit 20. The motion vector is a representation of a reference frame and a motion from the frame as a vector. Then, the camera shake detection unit 32 predicts the position of the subject in the next frame from the detected motion vector. The camera shake detection unit 32 calculates (detects) the shift amount of the subject between the current frame and the next frame as the camera shake amount. Then, the shake detection unit 32 outputs shake amount information indicating the calculated shake amount to the system control unit 70.

  The system control unit 70 also includes a display control unit 71 and an imaging control unit (drive control unit) 72. The display control unit 71 causes the display unit 50 to output the image data generated by the image generation unit 31, and performs control to display an image (still image, moving image, live view image) on the display screen of the display unit 50. The imaging control unit 72 executes drive control of the imaging element 100. That is, the imaging control unit 72 outputs, to the driving unit 21, an instruction signal that instructs an imaging condition such that, for example, appropriate exposure is obtained. Further, the imaging control unit 72 selects a transfer area according to the shake amount indicated by the shake amount information from the shake detection unit 32, and outputs an instruction signal for specifying the selected transfer area to the drive unit 21. The drive unit 21 sets a transfer area in the imaging device 100 based on an instruction signal from the imaging control unit 72. Further, the drive unit 21 drives and controls the transfer area of the imaging element 100 in accordance with the imaging condition instructed by the instruction signal from the imaging control unit 72.

  In the system control unit 70, the display control unit 71 and the imaging control unit 72 are realized by the CPU executing processing based on the control program.

  Next, the photographing operation according to the first embodiment will be described. FIG. 8 is a flowchart illustrating an example of a control method of the imaging device according to the first embodiment. FIG. 9 is a diagram showing the movement direction and movement amount of the subject during camera shake, where (A) shows the position of the subject during normal shooting, and (B) shows the position of the subject during camera shake shooting ing.

  In the process shown in FIG. 8, when the user turns on the imaging apparatus 1, the imaging control unit 72 starts capturing a live view image (step S <b> 1). Further, the display control unit 71 displays the live view image captured by the imaging unit 20 on the display screen of the display unit 50 (step S2). At this time, since the live view image does not have to be a smooth image of the movement of the subject, the imaging control unit 72 drives and controls the imaging element 100 so as to capture an image at a low frame rate, for example. In addition, when the power of the imaging device 1 is turned on, the imaging control unit 72 performs drive control on all pixel regions 113A of the imaging element 100, for example.

  The image processing unit 30 receives image data of a live view image captured by the imaging unit 20. Then, the image generation unit 31 performs various image processing on the received image data. Further, the image generation unit 31 generates motion vector evaluation values in a plurality of frames obtained in time series. The camera shake detection unit 32 detects a motion vector indicating the movement direction and the movement amount of the subject due to the camera shake, based on the evaluation value of the motion vector.

  Here, the shake detection unit 32 detects a motion vector using a subject in the background among a plurality of subjects. In the case of a movable subject such as a person, it is difficult for the camera shake detection unit 32 to determine whether the subject is moving or the subject is moving due to camera shake. Therefore, the camera shake detection unit 32 detects a moving subject by comparing a plurality of frames obtained in time series. Further, the camera shake detection unit 32 detects a subject of a person by using a face detection function as described in, for example, Japanese Patent Application Laid-Open No. 2010-16621 (US 2010/0002940). Further, in addition to face detection, the camera shake detection unit 32 detects a human subject by detecting a human body included in image data as described in, for example, Japanese Unexamined Patent Application Publication No. 2010-16621 (US2010 / 0002940). To detect The camera shake detection unit 32 recognizes subjects other than moving subjects detected as described above as subjects in the background.

  In the example shown in FIG. 9, a subject O1 of a person and a subject of a background including a subject O2 of a building are imaged as live view images. The camera shake detection unit 32 detects a subject in the background including the subject O1 of a person and the subject O2 of a building in a frame (frame of an image shown in FIG. 9A) before the current frame. The camera shake detection unit 32 also detects a subject in the background including the subject O1 of a person and the subject O2 of a building in the current frame (frame of the image shown in FIG. 9B).

  Then, the camera shake detection unit 32 detects a motion vector by specifying the movement (change in position) of the subject in the background between the plurality of frames. In the example shown in FIG. 9, the area 201 in the live view image in (A) is moved to the position of the area 202 in the live view image in (B) due to the shake of the user. The camera shake detection unit 32 extracts an edge portion of the object O2 of the building in the previous frame, and extracts an edge portion of the object O2 of the building in the current frame. The camera shake detection unit 32 detects the motion vector 203 by specifying the movement direction and movement amount of the edge portion of the object O2 of the building.

  The camera shake detection unit 32 predicts the position of the subject in the next frame from the detected motion vector. The camera shake detection unit 32 calculates (detects) the shift amount of the subject between the current frame and the next frame as the camera shake amount. Then, the shake detection unit 32 outputs shake amount information indicating the calculated shake amount to the system control unit 70. The process of calculating the shake amount by the shake detection unit 32 is performed for each frame. However, in order to reduce the processing load and the power consumption of the shake detection unit 32, a process of calculating the shake amount every several frames instead of every one frame may be executed.

  The imaging control unit 72 acquires camera shake amount information from the camera shake detection unit 32 (step S3). Then, the imaging control unit 72 selects a transfer area for each frame according to the shake amount indicated by the shake amount information, and outputs an instruction signal for specifying the selected transfer area to the drive unit 21 (step S4). The drive unit 21 sets a transfer area in the imaging device 100 for each frame based on an instruction signal from the imaging control unit 72. In addition, the drive unit 21 drives and controls the transfer area of the image sensor 100. The process of selecting the area for driving and controlling the transfer area as described above is performed for each frame. However, in order to reduce the processing load and power consumption of the imaging control unit 72, the drive unit 21 and the like, a process of selecting an area to drive and control the transfer area is performed every several frames instead of every one frame. It is also good.

  As described above, the imaging control unit 72 sets a transfer area according to the amount of camera shake calculated by the camera shake detection unit 32, and performs control so as to capture an object using only the transfer area. FIG. 10 is a diagram illustrating the size of image data in the imaging unit, the image processing unit, and the recording unit according to the first embodiment. As shown in FIG. 10A 1, generally, in the electronic image stabilization, the system control unit 70 performs control so as to capture an image in the pixel area 113 A of the image sensor 100. Then, in the image processing unit 30, the shake detection unit 32 calculates the shake amount based on the image data. Further, the image generation unit 31 extracts (trimms) only the image data of the area 200 of the recording image size (field angle for recording on the recording unit 60) among the image data of the pixel area 113A based on the camera shake amount. . Thereafter, as shown in FIG. 10A2, the image data of the area 200 of the recording image size is output from the image processing unit 30 to the recording unit 60, and is recorded in the recording unit 60. Further, the image data of the area 200 of the recording image size is output from the image processing unit 30 to the display unit 50 and displayed on the display unit 50.

  On the other hand, in the present embodiment, as shown in FIG. 10 (b 1), the imaging control unit 72 sets a transfer area (area of the recording image size in FIG. 10) 200 according to the shake amount calculated by the shake detection unit 32. Then, control is performed to capture an object using only the transfer area 200. Therefore, in the image processing unit 30, the image generation unit 31 performs image processing only on the image data of the transfer area 200. Then, as shown in FIG. 10 (b 2), the image data in the transfer area 200 is output from the image processing unit 30 to the recording unit 60 and recorded in the recording unit 60. Further, the image data of the transfer area 200 is output from the image processing unit 30 to the display unit 50 and displayed on the display unit 50.

  Thereafter, the imaging control unit 72 determines whether or not recording of a still image is started by confirming whether the user has pressed the release switch (step S5). When it is determined that the recording of the still image is started, the imaging control unit 72 performs processing of photographing of the still image by the imaging unit 20 and recording of the still image in the recording unit 60 (step S6). The imaging control unit 72 prepares for imaging such as AF (Automatic Focusing) or AE (Automatic Exposure) in response to the half-press operation of the release switch. Then, following the half-pressing operation of the release switch, in response to the full-pressing operation of the release switch being performed, the imaging control unit 72 determines an imaging condition that results in a proper exposure, and Control is performed to cause the drive unit 21 to perform imaging. Thereafter, the imaging control unit 72 determines whether or not the user has performed an operation to turn off the power (step S8). When it is determined that the operation to turn off the power is not performed, the imaging control unit 72 repeatedly executes the processing of steps S1 to S4. As a result, as shown in FIGS. 10B1 and 10B2, the drive unit 21 controls the drive of only the transfer area of the imaging device 100, and the image processing unit 30 controls only the image data of the transfer area 200. Processing is performed. When the imaging control unit 72 determines that the user has performed an operation to turn off the power, the process ends.

  When it is determined that the recording of the still image is not started, the imaging control unit 72 determines whether the recording of the moving image is started by confirming whether the moving image switch is pressed (Step S7). ). When it is determined that the recording of the moving image is started, the imaging control unit 72 starts shooting of the moving image by the imaging unit 20 and recording of the moving image in the recording unit 60 (step S11). The display control unit 71 also displays the moving image captured by the imaging unit 20 on the display screen of the display unit 50 (step S12). At this time, it is preferable that the moving image be a smooth image of the movement of the subject than the live view image. Therefore, the imaging control unit 72 drives and controls the imaging element 100 so as to capture an image at a frame rate higher than that at the time of capturing a live view image.

  Even during shooting of a moving image, the shake detection unit 32 calculates a shake amount based on each frame of the moving image. Then, the shake detection unit 32 outputs shake amount information indicating the calculated shake amount to the system control unit 70. The process of calculating the shake amount by the shake detection unit 32 is performed for each frame. However, in order to reduce the processing load and the power consumption of the shake detection unit 32, a process of calculating the shake amount every several frames instead of every one frame may be executed.

  The imaging control unit 72 acquires camera shake amount information from the camera shake detection unit 32 (step S13). Then, the imaging control unit 72 selects a transfer area for each frame according to the shake amount indicated by the shake amount information, and outputs an instruction signal for specifying the selected transfer area to the drive unit 21 (step S14). The drive unit 21 sets a transfer area in the imaging device 100 for each frame based on an instruction signal from the imaging control unit 72. In addition, the drive unit 21 drives and controls the transfer area of the image sensor 100. The process of selecting the area for driving and controlling the transfer area as described above is performed for each frame. However, in order to reduce the processing load and power consumption of the imaging control unit 72, the drive unit 21 and the like, a process of selecting an area to drive and control the transfer area is performed every several frames instead of every one frame. It is also good.

  Thereafter, the imaging control unit 72 determines whether or not the recording of the moving image is ended (step S15). When it is determined that the recording of the moving image is not ended, the imaging control unit 72 repeatedly executes the processing of steps S11 to S14. As a result, as shown in FIGS. 10B1 and 10B2, the drive unit 21 controls the drive of only the transfer area of the imaging device 100, and the image processing unit 30 controls only the image data of the transfer area 200 by the image processing unit 30. Processing is performed. Then, when it is determined that the recording of the moving image is ended, the imaging control unit 72 determines that the operation of turning off the power by the user has been performed (step S8). If the imaging control unit 72 determines that the user does not perform an operation to turn off the power, the process returns to the process of step S1 and repeatedly executes the processes of steps S1 to S4. When the imaging control unit 72 determines that the user has performed an operation to turn off the power, the process ends.

  As described above, according to the first embodiment, the camera shake amount calculation unit 32 that calculates the camera shake amount and the image sensor 100 are drive-controlled, and imaging is performed according to the camera shake amount calculated by the camera shake amount calculation unit 32. And an imaging control unit 72 which selects a region (transfer region 200 shown in FIG. 10) for driving and controlling the element 100. According to such a configuration, it is possible to reduce the power consumption and the processing load in the imaging device 1. That is, in the electronic shake correction that is generally performed, imaging is performed in the pixel area 113A of the imaging device 100, and various image processing is performed on the image data of the pixel area 113A in the image processing unit 30. Thereafter, the image data of the pixel area 113A subjected to the image processing in the image processing unit 30 is trimmed to the image data of the area of the recording image size. Accordingly, the image processing unit 30 performs image processing also on extra image data other than the transfer area (area of the recording image size) 200 which is actually recorded in the recording unit 60. Specifically, image data of an area of about 10% in the upper, lower, left, and right of the transfer area 200 shown in FIG. 10 is subjected to image processing as extra image data. For this reason, power consumption in the imaging unit 20 and the image processing unit 30 increases. In addition, the processing load on the image processing unit 30 also increases, making high-speed image processing difficult. On the other hand, in the first embodiment, the image processing unit 30 performs image processing only on the transfer area (area of the recording image size) 200. Therefore, camera shake correction can be performed, power consumption in the imaging unit 20 and the image processing unit 30 can be reduced, and the processing load on the image processing unit 30 can be reduced.

  Further, in the first embodiment described above, the imaging control unit 72 outputs the image signal of the area of the selected imaging element 100, so that the power consumption and processing burden on the imaging unit 20 can be reduced. Further, since the imaging control unit 72 selects an area for driving and controlling the imaging device 100 for each frame, it is possible to select an area for driving and controlling the imaging device 100 with high accuracy in accordance with the movement of the subject.

  Further, in the first embodiment described above, the shake amount calculation unit 32 calculates the shake amount by analyzing the image signal read from the image pickup device 100, and thus calculates the shake amount without using a special mechanism. be able to. Therefore, the cost of the imaging device 1 does not increase. Further, in the first embodiment described above, since the camera shake amount calculation unit 32 calculates the amount of camera shake based on the movement direction and the movement amount of the subject in the image signal obtained in time series, the camera shake amount can be calculated accurately. Can. Further, in the first embodiment described above, the camera shake amount calculation unit 32 calculates the amount of camera shake based on the movement direction and movement amount of the background subject (for example, the object O2 of the building shown in FIG. 9). It is possible to avoid calculating an incorrect amount of camera shake based on the movement of. Further, in the first embodiment described above, since the camera shake amount calculation unit 32 is provided in the image processing unit 30 that executes image processing of the image signal, the camera shake amount is calculated by the same or substantially the same process as electronic camera shake correction. Processing can be realized.

Second Embodiment
In the first embodiment described above, the camera shake detection unit 32 is configured to calculate the camera shake amount based on a plurality of frames. On the other hand, in the second embodiment, the system control unit 70 calculates the shake amount based on the acceleration information from the acceleration sensor. Then, the system control unit 70 selects a transfer area based on the calculated shake amount.

  FIG. 11 is a block diagram showing the configuration of an imaging device according to the second embodiment. As shown in FIG. 11, an acceleration sensor 65 is provided in the imaging device 1A in the second embodiment. The acceleration sensor 65 is a sensor that detects the acceleration of the sensor itself (here, angular acceleration). As this acceleration sensor 65, a three-axis acceleration sensor capable of detecting acceleration in three directions of XYZ axes is used. The acceleration sensor 65 outputs acceleration information representing the detected acceleration to the system control unit 70. Further, the image processing unit 30A performs various types of image processing in the same manner as the image processing unit 30 shown in FIG. 5, but does not perform processing for calculating the amount of camera shake. Further, in addition to the processing executed by the system control unit 70 shown in FIG. 5, the system control unit 70A performs processing to calculate the shake amount based on the acceleration indicated by the acceleration information from the acceleration sensor 65. The configuration of the lens unit 10, the imaging unit 20, the image processing unit 30, the work memory 40, the display unit 50, the operation unit 55, and the recording unit 60 in the imaging device 1A shown in FIG. 11 is shown in FIG. It is similar to the configuration. Therefore, the same reference numerals are given to the same configuration, and redundant description will be omitted.

  FIG. 12 is a functional block diagram of the image processing unit and the system control unit shown in FIG. As shown in FIG. 12, the image processing unit 30A in the second embodiment includes only the image generation unit 31. That is, unlike the image processing unit 30 shown in FIG. 7, the image processing unit 30A does not have the camera shake detection unit 32. Further, unlike the system control unit 70 shown in FIG. 7, the system control unit 70A in the second embodiment includes a camera shake correction unit (camera shake amount calculation unit) 73 in addition to the display control unit 71 and the imaging control unit 72. It is. The camera shake correction unit 73 performs a process of calculating a camera shake amount based on the acceleration indicated by the acceleration information from the acceleration sensor 65. Specifically, the shake correction unit 73 specifies the direction of shake based on the direction of acceleration indicated by the acceleration information. The camera shake correction unit 73 also calculates a camera shake amount corresponding to the value of the acceleration indicated by the acceleration information. In the present embodiment, the imaging control unit 72 outputs, to the driving unit 21, an instruction signal for specifying a transfer area based on the shake amount calculated by the shake correction unit 73. The drive unit 21 sets a transfer area based on an instruction signal from the imaging control unit 72. Then, the drive unit 21 drives and controls the transfer area.

  In the configuration shown in FIG. 11, the configurations of the image generation unit 31, the display control unit 71, and the imaging control unit 72 are the same as those shown in FIG. Therefore, the same reference numerals are given to the same configuration, and redundant description will be omitted.

  FIG. 13 is a flowchart illustrating an example of a control method of the imaging device according to the second embodiment. In the process shown in FIG. 13, when the user turns on the power of the imaging apparatus 1, the camera shake correction unit 73 acquires acceleration information from the acceleration sensor 65 (step S <b> 21). The imaging control unit 72 also starts capturing a live view image (step S22). The display control unit 71 also displays the live view image captured by the imaging unit 20 on the display screen of the display unit 50 (step S23). The processes of steps S22 and S23 correspond to the processes of steps S1 and S2 shown in FIG.

  The camera shake correction unit 73 calculates the amount of camera shake based on the acceleration information from the acceleration sensor 65 (step S24). That is, the shake correction unit 73 specifies the direction of shake based on the direction of acceleration indicated by the acceleration information. The camera shake correction unit 73 also predicts the position of the subject in the next frame based on the value of the acceleration indicated by the acceleration information. Then, the shake correction unit 73 calculates a shake amount corresponding to the predicted position of the subject. Then, the imaging control unit 72 selects a transfer area for each frame according to the shake amount calculated by the shake correction unit 73, and outputs an instruction signal for specifying the selected transfer area to the drive unit 21 (step S25). The drive unit 21 sets a transfer area in the imaging device 100 for each frame based on an instruction signal from the imaging control unit 72. In addition, the drive unit 21 drives and controls the transfer area of the image sensor 100. The process of selecting the area for driving and controlling the transfer area as described above is performed for each frame. However, in order to reduce the processing load and power consumption of the imaging control unit 72, the drive unit 21 and the like, a process of selecting an area to drive and control the transfer area is performed every several frames instead of every one frame. It is also good.

  Thereafter, the imaging control unit 72 determines whether or not recording of a still image is started by confirming whether the user has pressed the release switch (step S26). When it is determined that the recording of the still image is started, the imaging control unit 72 performs processing of photographing of the still image by the imaging unit 20 and recording of the still image in the recording unit 60 (step S27). The processes of steps S26 and S27 correspond to the processes of steps S5 and S6 shown in FIG. Thereafter, the imaging control unit 72 determines whether or not the user has performed an operation to turn off the power (step S29). When it is determined that the operation to turn off the power is not performed, the imaging control unit 72 repeatedly executes the process of steps S21 to S25. Thus, the drive unit 21 drives and controls only the transfer area of the imaging device 100, and the image processing unit 30 performs image processing on only the image data of the transfer area 200 in the image processing unit 30 (see FIG. 10 in the first embodiment). . When the imaging control unit 72 determines that the user has performed an operation to turn off the power, the process ends.

  When it is determined that the recording of the still image is not started, the imaging control unit 72 determines whether the recording of the moving image is started by confirming whether the moving image switch is pressed (Step S28). ). When it is determined that the recording of the moving image is started, the camera shake correction unit 73 acquires acceleration information from the acceleration sensor 65 (step S30). Further, the imaging control unit 72 starts shooting of a moving image by the imaging unit 20 and recording of the moving image on the recording unit 60 (step S31). The display control unit 71 also displays the moving image captured by the imaging unit 20 on the display screen of the display unit 50 (step S32). The processes of steps S28, S31, and S32 correspond to the processes of steps S7, S11, and S12 shown in FIG.

  The camera shake correction unit 73 calculates the amount of camera shake based on the acceleration information from the acceleration sensor 65 (step S33). Then, the imaging control unit 72 selects a transfer area for each frame according to the camera shake amount calculated by the camera shake correction unit 73, and outputs an instruction signal specifying the selected transfer area to the drive unit 21 (step S34). The drive unit 21 sets a transfer area in the imaging device 100 for each frame based on an instruction signal from the imaging control unit 72. In addition, the drive unit 21 drives and controls the transfer area of the image sensor 100. The process of selecting the area for driving and controlling the transfer area as described above is performed for each frame. However, in order to reduce the processing load and power consumption of the imaging control unit 72, the drive unit 21 and the like, a process of selecting an area to drive and control the transfer area is performed every several frames instead of every one frame. It is also good.

  Thereafter, the imaging control unit 72 determines whether or not the recording of the moving image is ended (step S35). When it is determined that the recording of the moving image is not ended, the imaging control unit 72 repeatedly executes the processing of steps S30 to S34. Thus, the drive unit 21 controls the drive of only the transfer area of the imaging device 100, and the image processing unit 30 performs image processing on only the image data of the transfer area 200 in the image processing unit 30 (see FIG. 10). Then, when it is determined that the recording of the moving image is ended, the imaging control unit 72 determines that the operation of turning off the power by the user has been performed (step S29). When the imaging control unit 72 determines that the user does not perform an operation to turn off the power, the process returns to the process of step S21, and repeatedly executes the processes of steps S21 to S25. When the imaging control unit 72 determines that the user has performed an operation to turn off the power, the process ends.

  As described above, according to the second embodiment, it is possible to reduce the power consumption and the processing load in the imaging device 1 as in the first embodiment described above. Further, in the second embodiment described above, since the acceleration sensor 65 for detecting the acceleration is provided, and the shake amount calculation unit 32 calculates the shake amount based on the acceleration from the acceleration sensor 65, accurate using the acceleration sensor 65 Shake amount can be calculated.

  In the second embodiment described above, the system control unit 70 uses the shake amount calculated based on the acceleration information only for the selection of the transfer area. However, the system control unit 70 may use the shake amount calculated based on the acceleration information for optical shake correction. For example, the system control unit 70 calculates a shake amount based on the acceleration information from the acceleration sensor 65. Then, the system control unit 70 corrects the image shift due to camera shake by moving a part or all of the lenses of the lens unit 10 based on the camera shake amount. Alternatively, the system control unit 70 corrects the image shift due to camera shake by moving the imaging device 100 itself based on the camera shake amount. Alternatively, the system control unit 70 corrects the image shift due to camera shake by moving a part or all of the lenses of the lens unit 10 and the imaging device 100 itself.

  When moving the lens of the lens unit 10, the system control unit 70 outputs a control signal to the lens unit 10 according to the amount of camera shake. The lens unit 10 drives the drive device (not shown) based on the control signal from the system control unit 70 to move the lens partially or entirely by the direction and the movement amount indicated by the control signal. Do. Further, when moving the imaging element 100, the system control unit 70 outputs a control signal according to the amount of camera shake to the imaging unit 20. The imaging unit 20 controls the movement of the imaging element 100 by the direction and the movement amount indicated by the control signal by driving a driving device (not shown) based on the control signal from the system control unit 70.

  In the case of such a configuration, the lens of the lens unit 10 and the imaging device 100 themselves move in accordance with the amount of camera shake, so that image deviation due to camera shake is corrected. However, there are cases where it is not possible to compensate for the image shift due to camera shake only with optical camera shake correction. For example, there are cases where the speed of camera shake is high and it is difficult to move the lens or the like following the speed of camera shake. In that case, the system control unit 70 selects a transfer area according to the image shift that can not be compensated for by the optical shake correction.

  Specifically, the shake correction unit 73 calculates the shake amount based on the acceleration information from the acceleration sensor 65. Then, the shake correction unit 73 determines whether the calculated shake amount is equal to or more than a predetermined threshold. When it is determined that the camera shake amount is equal to or more than the predetermined threshold, the camera shake correction unit 73 determines a value within the predetermined threshold as the first camera shake amount. Then, the shake correction unit 73 outputs a control signal corresponding to the first shake amount to the lens unit 10 and the imaging unit 20. The lens unit 10 and the imaging unit 20 move the lens and the imaging device 100 itself based on the control signal from the camera shake correction unit 73.

  Further, the shake correction unit 73 determines a value obtained by subtracting the first shake amount from the shake amount calculated based on the acceleration information as a second shake amount. Then, the imaging control unit 72 selects a transfer area according to the second shake amount. The imaging control unit 72 outputs an instruction signal specifying the selected transfer area to the drive unit 21. The drive unit 21 performs drive control in the transfer area based on an instruction signal from the camera shake correction unit 73. The transfer area selected by the imaging control unit 72 based on the second shake amount is preferably an area larger than the area of the recording image size. This is because it is difficult for the imaging control unit 72 to predict the transfer area of the recording image size according to the image shift, in consideration of the movement amount of the lens and the image sensor 100 by the optical shake correction. Also with such a configuration, the power consumption and the processing load in the imaging device 1 can be reduced.

  If the imaging control unit 72 can not compensate for the image shift due to camera shake only by selecting the transfer area based on the camera shake amount, the image shift may be corrected by optical camera shake correction.

Third Embodiment
In the first embodiment described above, the camera shake detection unit 32 is configured to calculate the camera shake amount based on a plurality of frames. Further, in the above-described second embodiment, the system control unit 70 calculates the shake amount based on the acceleration information from the acceleration sensor. On the other hand, in the third embodiment, the calculation circuit 416 calculates the shake amount based on the plurality of frames, or the calculation circuit 416 calculates the shake amount based on the acceleration information from the acceleration sensor 65. Then, the arithmetic circuit 416 selects the transfer area based on the calculated shake amount.

  FIG. 14 is a block diagram showing a specific configuration of a signal processing chip according to the third embodiment. In the signal processing chip 111 shown in FIG. 14, the arithmetic circuit 416 reads the image data of each pixel (RAW data before image processing by the image processing unit 30) from the pixel memory 415. Then, the arithmetic circuit 416 calculates a shake amount based on a plurality of image data (frames). Alternatively, the arithmetic circuit 416 is connected to the acceleration sensor 65 and acquires acceleration information from the acceleration sensor 65. The arithmetic circuit 416 then calculates the amount of camera shake based on the acceleration information from the acceleration sensor 65. The arithmetic circuit 416 also selects a transfer area based on the calculated shake amount. Then, the arithmetic circuit 416 outputs a control signal specifying the selected transfer area to the control unit 420. The configuration shown in FIG. 14 is similar to that shown in FIG. 6 except that the acceleration sensor 65 is provided. Therefore, the same reference numerals are given to the same configuration, and redundant description will be omitted.

  FIG. 15 is a flowchart illustrating an example of a control method of the imaging device according to the third embodiment. FIG. 15 (M) shows a case where the arithmetic circuit 416 calculates the amount of camera shake based on image data, and FIG. 15 (N) shows a case where the arithmetic circuit 416 calculates the amount of camera shake based on acceleration information. The imaging control unit 72 outputs an instruction signal to the control unit 420 before the arithmetic circuit 416 executes the processes of FIGS. 15M and 15N, whereby imaging by the imaging element 100 is started.

  The process shown in FIG. 15 (M) will be described. As shown in FIG. 15M, the arithmetic circuit 416 reads the image data (RAW data) of each pixel stored in the pixel memory 415 for each frame (step S41M). Then, the arithmetic circuit 416 calculates the shake amount based on the plurality of frames read out in time series from the pixel memory 415 (step S42M). As the process (calculation process of the shake amount) of step S42M, it is possible to use the same process as the calculation process of the shake amount (the calculation process of the shake amount described in the first embodiment) by the shake detection unit 32 described above is there. That is, the arithmetic circuit 416 detects a motion vector based on the movement direction and movement amount of the subject (edge part of the subject in the background) due to camera shake in a plurality of frames obtained in time series. Then, the arithmetic circuit 416 predicts the position of the subject in the next frame from the detected motion vector. The arithmetic circuit 416 calculates (detects) the shift amount of the subject between the current frame and the next frame as a shake amount. Then, the arithmetic circuit 416 outputs a control signal indicating the calculated shake amount to the control unit 420. The process of calculating the shake amount by the arithmetic circuit 416 as described above is performed for each frame. However, in order to reduce the processing load and the power consumption of the arithmetic circuit 416, processing for calculating the shake amount may be performed not every one frame but every several frames.

  In the first embodiment described above, the camera shake detection unit 32 calculates the motion vector based on the image data subjected to the image processing by the image generation unit 31. On the other hand, in the third embodiment, the arithmetic circuit 416 calculates a motion vector based on image data (that is, RAW data) before being image-processed by the image generation unit 31. RAW data is monochrome image data in a state in which image processing such as color signal processing and gamma correction is not performed. As described above, the arithmetic circuit 416 extracts only the edge portion of the subject in the background of the image data, and determines how much the edge portion has moved to calculate the motion vector of the subject. Therefore, the motion vector can be calculated regardless of the color component of the image data. Therefore, the arithmetic circuit 416 can calculate a motion vector based on image data (that is, RAW data) before being image-processed by the image generation unit 31.

  The arithmetic circuit 416 outputs, to the control unit 420, a control signal specifying a transfer area corresponding to the shake amount calculated in step S42M (step S43M). The control unit 420 converts control signals from the arithmetic circuit 416 into control signals that can be executed by the sensor control unit 441, the block control unit 442, and the synchronization control unit 443. The sensor control unit 441, the block control unit 442, And output to the synchronization control unit 443.

  As described above, the sensor control unit 441 controls the start and end of charge accumulation by transmitting the reset pulse and the transfer pulse to each pixel of the transfer area based on the control signal from the control unit 420. . Further, the sensor control unit 441 outputs a pixel signal to the output wiring 309 by sending out a selection pulse to each pixel in the transfer area. In addition, the block control unit 442 sends a specific pulse specifying the unit group 131 of the transfer area to the imaging chip 113 based on the control signal from the control unit 420. As a result, charge accumulation control is performed for each pixel in the transfer area of the pixel area 113A, and pixel signals are read out from each pixel in the transfer area.

  Next, the process shown in FIG. 15 (N) will be described. As shown in FIG. 15 (N), the arithmetic circuit 416 acquires acceleration information from the acceleration sensor 65 (step S41N). Then, the arithmetic circuit 416 calculates the amount of camera shake based on the acceleration information from the acceleration sensor 65 (step S42N). As the process (calculation process of the shake amount) of step S42N, it is possible to use the same process as the calculation process of the shake amount (the calculation process of the shake amount described in the second embodiment) by the shake correction unit 73 described above is there. That is, the arithmetic circuit 416 specifies the direction of camera shake based on the direction of acceleration indicated by the acceleration information. The camera shake correction unit 73 also predicts the position of the subject in the next frame based on the value of the acceleration indicated by the acceleration information. Then, the shake correction unit 73 calculates a shake amount corresponding to the predicted position of the subject. The process of calculating the shake amount by the arithmetic circuit 416 as described above is performed for each frame. However, in order to reduce the processing load and the power consumption of the arithmetic circuit 416, processing for calculating the shake amount may be performed not every one frame but every several frames.

  The arithmetic circuit 416 outputs, to the control unit 420, a control signal for specifying a transfer area corresponding to the shake amount calculated in step S42N (step S43N). The control unit 420 outputs a control signal to the sensor control unit 441 and the block control unit 442 based on the control signal from the arithmetic circuit 416. The sensor control unit 441 and the block control unit 442 execute charge accumulation control for each pixel in the transfer area of the pixel area 113A, and perform control to read out a pixel signal from each pixel in the transfer area.

  FIG. 16 is a diagram showing the size of image data in the imaging unit and the image processing unit according to the third embodiment. As shown in FIG. 16 (c1), the signal processing chip 111 (that is, the control unit 420, the sensor control unit 441, and the block control unit 442) responds to the amount of camera shake calculated by the signal processing chip 111 (that is, the arithmetic circuit 416). The transfer area (area of the recording image size in FIG. 16) 200 is set in the pixel area 113A of the imaging chip 113. Then, the imaging chip 113 performs control so as to capture an object using only the transfer area 200. When the signal processing chip 111 receives the image data of the transfer area 200 from the imaging chip 113, predetermined signal processing (amplification by the amplifier 412, CDS processing by the CDS 413a, A / D by the A / D 413b) is performed on the image data. Perform conversion processing). Then, the signal processing chip 111 stores the image data in the pixel memory 415 of the memory chip 112.

  Thereafter, as shown in (c 2) of FIG. 16, the image data (RAW data) of each pixel of the transfer area stored in the pixel memory 415 is transferred to the image processing unit 30 under the control of the control unit 420. In the image processing unit 30, the image generation unit 31 performs various types of image processing on the image data in the transfer area. The image processing unit 30 outputs the image data subjected to the image processing to the recording unit 60. Thereby, the image data of the transfer area (area of the recording image size) is recorded in the recording unit 60. Further, the image processing unit 30 outputs the image data subjected to the image processing to the display unit 50. Thereby, the image data of the transfer area is displayed on the display screen of the display unit 50.

  As described above, according to the third embodiment, it is possible to reduce the power consumption and the processing load in the imaging device 1 as in the above-described first and second embodiments. Further, since the image processing unit 30 and the system control unit 70 do not need to calculate the shake amount, the processing of the image processing unit 30 and the system control unit 70 can be reduced.

  Further, in the third embodiment described above, the imaging device 100, the drive control unit (the control unit 420, the sensor control unit 441, the block control unit 442, the synchronization control unit 443), and the camera shake amount calculation unit (the arithmetic circuit 416) are integrated. It is configured. That is, the imaging chip 113, the signal processing chip 111, and the imaging chip 113 are stacked, and the amount of camera shake is calculated in the arithmetic circuit 416 of the signal processing chip 111. Therefore, it is possible to realize a configuration in which the transfer area is selected according to the amount of camera shake only by incorporating in the imaging device 1 a chip in which the imaging element 100 and the like are integrally formed.

  In the third embodiment described above, the signal processing chip 111 uses the camera shake amount calculated based on the acceleration information only for the selection of the transfer area. However, the signal processing chip 111 may use the shake amount calculated based on the acceleration information for optical shake correction. For example, the control unit 420 of the signal processing chip 111 corrects the image shift due to camera shake by moving a part or all of the lenses of the lens unit 10 based on the camera shake amount calculated based on the acceleration information by the arithmetic circuit 416. Do. Alternatively, the control unit 420 corrects the image shift due to camera shake by moving the imaging device 100 itself based on the camera shake amount. Alternatively, the control unit 420 corrects an image shift due to camera shake by moving a part or all of the lenses of the lens unit 10 and the imaging device 100 itself.

  Even in the case of such a configuration, as described in the second embodiment described above, there may be cases where image deviation due to camera shake can not be compensated for only by optical camera shake correction. In that case, the control unit 420 selects a transfer area according to the image shift that can not be compensated for by the optical shake correction.

  Specifically, the control unit 420 determines whether the shake amount calculated by the calculation circuit 416 is equal to or more than a predetermined threshold. When it is determined that the shake amount is equal to or more than the predetermined threshold, the control unit 420 determines a value within the predetermined threshold as the first shake amount. Then, the control unit 420 outputs a control signal corresponding to the first shake amount to the lens unit 10 or the imaging unit 20. The lens unit 10 and the imaging unit 20 move the lens and the imaging device 100 itself based on the control signal from the camera shake correction unit 73.

  Further, the control unit 420 determines a value obtained by subtracting the first shake amount from the shake amount calculated based on the acceleration information as a second shake amount. Then, control unit 420 selects a transfer area according to the second shake amount. The control unit 420 outputs a control signal specifying the selected transfer area to the sensor control unit 441 or the block control unit 442. The sensor control unit 441 and the block control unit 442 execute drive control in the transfer area based on the control signal from the control unit 420. The transfer area selected by the control unit 420 based on the second shake amount is preferably an area larger than the area of the recording image size. This is because it is difficult for the control unit 420 to predict the transfer area of the recording image size according to the image shift, in consideration of the movement amount of the lens and the image sensor 100 due to the optical shake correction. Also with such a configuration, the power consumption and the processing load in the imaging device 1 can be reduced.

  If the control unit 420 can not compensate for the image shift due to camera shake only by selecting the transfer area based on the camera shake amount, the image shift may be corrected by optical camera shake correction.

Fourth Embodiment
In the first embodiment, the second embodiment, and the third embodiment described above, the transfer area for driving and controlling the imaging device 100 is changed according to the amount of camera shake. On the other hand, in the fourth embodiment, the drive unit 21 performs drive control on all pixel areas 113A of the imaging device 100. Further, the drive unit 21 outputs, to the image processing unit 30, image data of a region corresponding to the amount of camera shake, out of the image data read from the imaging device 100.

  FIG. 17 is a diagram showing the size of image data in the imaging unit and the image processing unit according to the fourth embodiment. As shown in FIG. 17 (d 1), the drive unit 21 (the control unit 420, the sensor control unit 441, the block control unit 442, and the synchronization control unit 443) does not divide the pixel area 113 A of the imaging device 100. Drive control is performed on the pixel region 113A. Then, the drive unit 21 reads a pixel signal from each pixel of the pixel area 113A of the imaging device 100. On the other hand, as in the third embodiment described above, the arithmetic circuit 416 calculates the shake amount based on a plurality of frames. Alternatively, as in the above-described third embodiment, the arithmetic circuit 416 calculates the shake amount based on the acceleration information from the acceleration sensor 65. Then, the arithmetic circuit 416 outputs a control signal indicating the calculated shake amount to the control unit 420.

  As shown in FIG. 17 (d 2), the control unit (signal output unit) 420 is a transfer area (area of the recording image size) corresponding to the shake amount among the image data (RAW data) stored in the pixel memory 415. The image processing unit 30 outputs the image data of 200 for each frame. In the image processing unit 30, the image generation unit 31 performs various types of image processing on the image data in the transfer area. The image processing unit 30 outputs the image data subjected to the image processing to the recording unit 60. Thereby, the image data of the transfer area (area of the recording image size) is recorded in the recording unit 60. Further, the image processing unit 30 outputs the image data subjected to the image processing to the display unit 50. Thereby, the image data of the transfer area is displayed on the display screen of the display unit 50.

  As described above, according to the fourth embodiment, the imaging unit 21 having the imaging element 100 and the image processing unit 30 that performs image processing on the image signal output from the imaging unit 21 are provided. The unit 21 includes a shake amount calculation unit (calculation circuit 416) for calculating a shake amount, and an image signal of the area 200 corresponding to the shake amount calculated by the shake amount calculation unit 416 among the image signals read from the imaging device 100. And a signal output unit (control unit 420) that outputs an image signal to the image processing unit 30. According to such a configuration, since the image processing unit 30 may perform the image processing only on the transfer area 200, it is possible to reduce the power consumption and the processing load on the image processing unit 30.

  Further, in the fourth embodiment described above, the signal output unit 420 outputs the image signal of the area 200 according to the amount of camera shake for each frame to the image processing unit. Area 200 can be selected. In the fourth embodiment described above, the acceleration sensor 65 for detecting the acceleration is provided, and the shake amount calculation unit 416 calculates the shake amount based on the acceleration from the acceleration sensor 65. According to such a configuration, an accurate shake amount can be calculated using the acceleration sensor 65. Further, according to the fourth embodiment described above, the camera shake amount calculation unit 416 calculates the camera shake amount by analyzing the image signal read from the imaging device 100, so the camera shake amount can be calculated without using a special mechanism. It can be calculated. Therefore, the cost of the imaging device 1 does not increase.

Fifth Embodiment
In the fifth embodiment, the imaging device 1A in the above-described second embodiment is separated into an imaging device 2A and an electronic device 2B.

  FIG. 18 is a block diagram showing configurations of an imaging device and an electronic device according to the fifth embodiment. In the configuration shown in FIG. 18, the imaging device 2A is a device that captures an image of a subject. The imaging device 2A includes a lens unit 10, an imaging unit 20, an image processing unit 30A, a work memory 40, an operation unit 55, a recording unit 60, an acceleration sensor 65, and a first system control unit 75. The configuration of the lens unit 10, the imaging unit 20, the image processing unit 30A, the work memory 40, the operation unit 55, the recording unit 60, and the acceleration sensor 65 in the imaging device 2A is the same as that shown in FIG. is there. Therefore, the same reference numerals are given to the same configuration, and redundant description will be omitted.

  The electronic device 2B is a device that displays an image (still image, moving image, live view image). The electronic device 2B includes a display unit 50 and a second system control unit 75. The configuration of the display unit 50 in the electronic device 2B is the same as the configuration shown in FIG. Therefore, the same reference numerals are given to the same configuration, and redundant description will be omitted.

  The first system control unit 75 includes a first communication unit 75A. The second system control unit 76 also includes a second communication unit 76B. The first communication unit 75A and the second communication unit 76B mutually transmit and receive signals by wire or wirelessly. Further, the first system control unit 75 has a configuration corresponding to, for example, the imaging control unit 72 and the camera shake correction unit 73 in the configuration shown in FIG. Further, the second system control unit 76 has only a configuration corresponding to, for example, the display control unit 71 among the configurations shown in FIG.

  The configuration (the display control unit 71, the control unit 72, and the selection unit 73) illustrated in FIG. 11 may be provided in any of the first system control unit 75 and the second system control unit 76. 11 may be provided in the first system control unit 75 or the second system control unit 76, or a part of the configuration shown in FIG. 11 may be provided in the first system control unit 75. A configuration other than a part of the configuration shown in 11 may be provided in the second system control unit 76.

  The imaging device 2A is configured by, for example, a digital camera, a smartphone, a mobile phone, a personal computer, etc. provided with an imaging function and a communication function, and the electronic device 2B is, for example, a smartphone equipped with a display function and a communication function, a mobile phone, It comprises a portable terminal such as a portable personal computer.

  The first system control unit 75 shown in FIG. 18 is realized by the CPU (not shown) executing processing based on a control program. The second system control unit 76 shown in FIG. 18 is realized by the CPU (not shown) executing processing based on a control program.

  In the configuration shown in FIG. 18, the image processing unit 30 and the first system control unit 75 may be integrally configured. In this case, the system control unit having one or more CPUs performs processing based on the control program, thereby taking on the function of the image processing unit 30 and the function of the first system control unit 75.

  Further, the imaging device 1 in the first embodiment described above may be separated into the imaging device and the electronic device in the same manner as or substantially the same as the separation method shown in FIG.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in said embodiment. Various changes or modifications can be added to the above embodiment without departing from the spirit of the present invention. Also, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements and omissions are also included in the technical scope of the present invention. Moreover, it is also possible to apply combining the structure of above-mentioned embodiment or a modification suitably.

  In the first embodiment described above (see FIG. 8), the camera shake detection unit 32 may execute the process of calculating the camera shake amount only at the time of shooting a live view image or only at the time of shooting a moving image. In addition, the imaging control unit 72 executes the process of acquiring camera shake amount information (steps S3 and S13) and the process of specifying a transfer area (steps S4 and S14) only when shooting a live view image or only when shooting a moving image. You may do it. That is, the imaging control unit 72 may execute only the processes of steps S3 and S4, or may execute only the processes of steps S13 and S14.

  In the second embodiment described above (see FIG. 13), the camera shake correction unit 73 and the imaging control unit 72 calculate the amount of camera shake (steps S24 and S33) and specify the transfer area (steps S25 and S34). ) May be performed only when shooting a live view image or only when shooting a moving image. That is, the imaging control unit 72 may execute only the processes of steps S24 and S25, or may execute only the processes of steps S33 and S34.

  In the first embodiment described above, the camera shake detection unit 32 executes the process of calculating the camera shake amount for each frame only while shooting a moving image, and calculates the camera shake amount while shooting a live view image. The process may be performed every few frames. In the second embodiment described above, the camera shake correction unit 73 executes the process of calculating the camera shake amount for each frame only while shooting a moving image, and calculates the camera shake amount while shooting a live view image. The process may be performed every few frames. According to such a configuration, it is possible to reduce the processing load and power consumption of the camera shake detection unit 32 and the camera shake correction unit 73 during shooting of a live view image.

  In the first and second embodiments described above, the imaging control unit 72 outputs an instruction signal for specifying the transfer area to the drive unit 21 depending on the frame rate of the live view image and the frame rate of the moving image. After that, several frames worth of signal readout may be performed in the imaging device 100 until the drive unit 21 drives and controls the transfer region. In such a case, the shake detection unit 32 and the shake correction unit 73 predict the position of the subject in a frame several frames after the current frame. The camera shake detection unit 32 and the camera shake correction unit 73 calculate (detect) an amount of displacement of the subject between the current frame and a frame several frames after the current frame as the amount of camera shake. Then, the shake detection unit 32 and the shake correction unit 73 output shake amount information indicating the calculated shake amount to the system control unit 70.

  Further, in the third embodiment described above, the arithmetic circuit 416 calculates a shake amount from a plurality of frames (see FIG. 15M) and calculates a shake amount from acceleration information (FIG. 15N). And reference may be performed. In this case, the arithmetic circuit 416 can obtain the camera shake amount with higher accuracy by calculating the camera shake amount based on the two processes.

  In each embodiment described above, as a method of specifying the transfer area, for example, the imaging control unit 72, etc., coordinates of the start position of the transfer area, the size in the horizontal direction of the transfer area, and the size in the vertical direction of the transfer area. Specify Further, although the acceleration sensor 65 for detecting the acceleration (angular acceleration) is provided in the second embodiment, the third embodiment, and the fourth embodiment described above, a gyro for detecting an angular velocity may be provided. Also in this case, the shake correction unit 73 and the arithmetic circuit 416 can calculate the shake amount based on the angular velocity information indicating the angular velocity detected by the gyro. Further, although a 3-axis acceleration sensor is used as the acceleration sensor 65, a 5-axis acceleration sensor or the like may be used.

  Further, in the first embodiment described above, the camera shake detection unit 32 calculates a motion vector based on data (such as a luminance value) of each component of R, G, and B in a plurality of image data obtained in time series. The motion vector may be calculated based on only one or two data of R, G and B.

  In each of the above-described embodiments, the transfer area is the area of the recording image size (that is, the angle of view for recording to the recording unit 60), but even if the area is a size larger than the recording image size Good.

  In each of the above-described embodiments, the arrangement of the color filters 102 is a Bayer arrangement, but may be an arrangement other than this arrangement. Also, the number of pixels forming the unit group 131 may include at least one pixel. Also, the block may include at least one pixel. Therefore, it is also possible to perform imaging under different imaging conditions for each pixel.

  In each of the above-described embodiments, a part or all of the drive unit 21 may be mounted on the imaging chip 113, or a part or all of the drive unit 21 may be mounted on the signal processing chip 111. Further, part of the configuration of the image processing unit 30 may be mounted on the imaging chip 113 or the signal processing chip 111. Further, part of the configuration of the system control unit 70 may be mounted on the imaging chip 113 or the signal processing chip 111.

  Moreover, although the case where the magnitude | size of the area | region of a block was preset was demonstrated in each above-mentioned embodiment, you may comprise so that a user may set the magnitude | size of the area | region of a block.

  1, 1A, 2A: imaging device, 2B: electronic device, 20: imaging unit, 30, 30A: image processing unit, 31: image generation unit, 32: camera shake detection unit (camera shake amount calculation unit), 50: display unit, 65 acceleration sensor 70 70A system control unit 75 first system control unit 76 second system control unit 71 display control unit 72 imaging control unit (drive control unit) 73 image stabilization Unit (camera shake amount calculation unit) 100 Image pickup device 200 Transfer area 415 Pixel memory 416 Operation circuit (camera shake amount calculation unit) 420 Control unit (drive control unit) 441 Sensor control unit 442 ... block control unit, 443 ... synchronization control unit

Claims (1)

A shake amount calculation unit that calculates a shake amount;
A drive control unit configured to drive and control an imaging element and select an area to drive and control the imaging element according to the camera shake amount calculated by the camera shake amount calculation unit;
An imaging device comprising: