JP2018029404A - Imaging device and imaging apparatus - Google Patents

Abstract

A pixel signal from a pixel is subjected to image processing by an image processing chip which is a separate chip from an imaging unit. Therefore, there is a problem that the processing speed decreases unless the signal output band of the signal line between the imaging unit and the image processing chip is increased. An image pickup unit having a plurality of pixels provided on a first substrate and a signal compression unit for compressing a signal from the image pickup unit provided on a second substrate stacked on the first substrate. And an imaging device comprising: A plurality of connection electrodes are formed on the surface of the first substrate in contact with the second substrate, a plurality of connection electrodes are formed on the surface of the second substrate in contact with the first substrate, and the plurality of connection electrodes are joined together. Thus, the imaging unit and the signal compression unit may be electrically connected. [Selection] Figure 8

Description

  The present invention relates to an imaging element and an imaging apparatus.

There is known an imaging unit in which a back-illuminated imaging chip and a signal processing chip are connected via a micro bump for each cell unit in which a plurality of pixels are combined.
[Prior art documents]
[Patent Literature]
[Patent Document 1] Japanese Patent Application Laid-Open No. 2006-49361

  However, pixel signals from the pixels are subjected to image processing by an image processing chip that is a separate chip from the imaging unit. Therefore, there is a problem that the processing speed decreases unless the signal output band of the signal line between the imaging unit and the image processing chip is increased.

  In the first aspect of the present invention, a signal from the imaging unit provided on the first substrate provided on the first substrate and provided on the second substrate stacked on the first substrate is compressed. And an image sensor including a signal compression unit.

  In the second aspect of the present invention, an image pickup unit having a plurality of pixels, and a plurality of signals compressed for signals from the image pickup unit and provided for each pixel or a signal for each of the plurality of pixels. An imaging device including a compression unit is provided.

  According to a third aspect of the present invention, an imaging unit having a plurality of pixels and a plurality of signal compression units for compressing signals from the imaging unit are provided, and the signal compression ratios are respectively set in the plurality of signal compression units. An imaging device is provided that is possible.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 is a cross-sectional view of a backside illuminating type MOS imaging device according to the present embodiment. It is a figure explaining the pixel arrangement | sequence and unit block of an imaging chip. It is a circuit diagram corresponding to the unit block of an imaging chip. It is a block diagram which shows the structure of the imaging device which concerns on this embodiment. It is a block diagram which shows the specific structure as an example of a signal processing chip. 2 shows functional blocks of the image processing unit 470. The functional block of the arithmetic circuit 415 is shown. An example of the operation flow of the image sensor 100 is shown. It is an example of the reduced image 170. FIG. An example of the operation | movement flow of the system control part 501 is shown. This is another example of specifying a region of interest.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a cross-sectional view of a back-illuminated image sensor 100 according to this embodiment. The imaging device 100 includes an imaging chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that stores the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  As shown in the figure, incident light is incident mainly in the positive direction of the Z-axis indicated by a white arrow. In the present embodiment, in the imaging chip 113, the surface on the side where incident light is incident is referred to as a back surface. Further, as shown in the coordinate axes, the left direction of the paper orthogonal to the Z axis is the X axis plus direction, and the front side of the paper orthogonal to the Z axis and X axis is the Y axis plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

  An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is disposed on the back side of the wiring layer 108. The PD layer 106 includes a plurality of PDs (photodiodes) 104 that are two-dimensionally arranged and store charges corresponding to incident light, and transistors 105 that are provided corresponding to the PDs 104.

  A color filter 102 is provided on the incident side of incident light in the PD layer 106 via a passivation film 103. The color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filter 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident light incident side of the color filter 102, a microlens 101 is provided corresponding to each pixel. The microlens 101 condenses incident light toward the corresponding PD 104.

  The wiring layer 108 includes a wiring 107 that transmits the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multilayer, and a passive element and an active element may be provided.

  A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surfaces of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are pressed and aligned. The bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are disposed on the mutually facing surfaces of the signal processing chip 111 and the memory chip 112. The bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized, so that the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one unit block described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. Further, a bump larger than the bump 109 corresponding to the pixel region may be provided in a peripheral region other than the pixel region where the pixels are arranged.

  The signal processing chip 111 has a TSV (silicon through electrode) 110 that connects circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral area. The TSV 110 may also be provided in the peripheral area of the imaging chip 113 and the memory chip 112.

  FIG. 2 is a diagram for explaining the pixel array and the unit block 131 of the imaging chip 113. In particular, a state where the imaging chip 113 is observed from the back side is shown. In the imaging region 120 of the imaging chip 113, 20 million or more pixels are arranged in a matrix. In the example of FIG. 2, adjacent 16 pixels of 4 × 4 pixels form one unit block 131. The grid lines in the figure indicate the concept that adjacent pixels are grouped to form a unit block 131. The number of pixels forming the unit block 131 is not limited to this, and may be about 1000, for example, 32 pixels × 64 pixels, or more or less.

  As shown in the partially enlarged view of the imaging region 120, the unit block 131 includes four so-called Bayer arrays including four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R vertically and horizontally. The green pixel is a pixel having a green filter as the color filter 102, and receives light in the green wavelength band of incident light. Similarly, a blue pixel is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band, and a red pixel is a pixel having a red filter as the color filter 102 and receiving light in the red wavelength band. Receive light.

  FIG. 3 is a circuit diagram corresponding to the unit block 131 of the imaging chip 113. In the figure, a rectangle surrounded by a dotted line typically represents a circuit corresponding to one pixel. Note that at least some of the transistors described below correspond to the transistor 105 in FIG.

  As described above, the unit block 131 is formed of 16 pixels. The 16 PDs 104 corresponding to the respective pixels are respectively connected to the transfer transistors 302, and the gates of the transfer transistors 302 are connected to the TX wiring 307 to which transfer pulses are supplied. In the present embodiment, the TX wiring 307 is commonly connected to the 16 transfer transistors 302.

  The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and a so-called floating diffusion FD between the drain of the transfer transistor 302 and the source of the reset transistor 303 is connected to the gate of the amplification transistor 304. The drain of the reset transistor 303 is connected to a Vdd wiring 310 to which a power supply voltage is supplied, and the gate thereof is connected to a reset wiring 306 to which a reset pulse is supplied. In the present embodiment, the reset wiring 306 is commonly connected to the 16 reset transistors 303.

  The drain of each amplification transistor 304 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. The source of each amplification transistor 304 is connected to the drain of each corresponding selection transistor 305. Each gate of the selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. In the present embodiment, the decoder wiring 308 is provided independently for each of the 16 selection transistors 305. The source of each selection transistor 305 is connected to a common output wiring 309. The load current source 311 supplies current to the output wiring 309. That is, the output wiring 309 for the selection transistor 305 is formed by a source follower. Note that the load current source 311 may be provided on the imaging chip 113 side or on the signal processing chip 111 side.

  Here, the flow from the start of charge accumulation to pixel output after the end of accumulation will be described. When a reset pulse is applied to the reset transistor 303 through the reset wiring 306 and simultaneously a transfer pulse is applied to the transfer transistor 302 through the TX wiring 307, the potentials of the PD 104 and the floating diffusion FD are reset.

  When the application of the transfer pulse is canceled, the PD 104 converts the incident light to be received into electric charge and accumulates it. Thereafter, when the transfer pulse is applied again without the reset pulse being applied, the accumulated charge is transferred to the floating diffusion FD, and the potential of the floating diffusion FD changes from the reset potential to the signal potential after the charge accumulation. . When a selection pulse is applied to the selection transistor 305 through the decoder wiring 308, a change in the signal potential of the floating diffusion FD is transmitted to the output wiring 309 through the amplification transistor 304 and the selection transistor 305. Thereby, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

  As shown in the drawing, in the present embodiment, the reset wiring 306 and the TX wiring 307 are common to the 16 pixels forming the unit block 131. That is, the reset pulse and the transfer pulse are simultaneously applied to all 16 pixels. Therefore, all the pixels forming the unit block 131 start charge accumulation at the same timing and end charge accumulation at the same timing. However, the pixel signal corresponding to the accumulated electric charge is sequentially applied to each selection transistor 305 by a selection pulse, and is selectively output to the output wiring 309. In addition, the reset wiring 306, the TX wiring 307, and the output wiring 309 are provided separately for each unit block 131.

  In this way, by configuring the circuit with the unit block 131 as a reference, the imaging condition can be varied for each unit block 131. For example, pixel signals with different charge accumulation times can be output between adjacent unit blocks 131, respectively. More specifically, while one unit block 131 performs charge accumulation once, the other unit block 131 repeats charge accumulation several times and outputs a pixel signal each time. Each frame for moving images can be output at a different frame rate between the unit blocks 131. In addition, the photographing sensitivity can be varied.

  FIG. 4 is a block diagram illustrating a configuration of the imaging apparatus according to the present embodiment. The imaging apparatus 500 includes a photographic lens 520 as a photographic optical system, and the photographic lens 520 guides a subject luminous flux incident along the optical axis OA to the imaging element 100. The photographing lens 520 may be an interchangeable lens that can be attached to and detached from the imaging apparatus 500. The imaging apparatus 500 mainly includes an imaging device 100, a system control unit 501, a drive unit 502, a photometry unit 503, a work memory 504, a recording unit 505, and a display unit 506.

  The photographing lens 520 is composed of a plurality of optical lens groups, and forms an image of a subject light flux from the scene in the vicinity of its focal plane. In FIG. 4, the photographic lens 520 is representatively represented by a single virtual lens arranged in the vicinity of the pupil. The drive unit 502 is a control circuit that executes charge accumulation control such as timing control and area control of the image sensor 100 in accordance with instructions from the system control unit 501.

  The image sensor 100 delivers the pixel signal to the image processing unit 511 of the system control unit 501. The image processing unit 511 performs various image processes using the work memory 504 as a work space, and generates image data in a predetermined format. The generated image data is recorded in the recording unit 505, converted into a display signal, and displayed on the display unit 506 for a preset time.

  The photometric unit 503 detects the luminance distribution of the scene prior to a series of shooting sequences for generating image data. The photometry unit 503 includes, for example, an AE sensor having about 1 million pixels. The calculation unit 512 of the system control unit 501 receives the output of the photometry unit 503 and calculates the luminance for each area of the scene. The calculation unit 512 determines the shutter speed, aperture value, and ISO sensitivity according to the calculated luminance distribution. The light metering unit 503 may be shared by the image sensor 100. Note that the arithmetic unit 512 also executes various arithmetic operations for operating the imaging device 500.

  A part or all of the drive unit 502 may be mounted on the imaging chip 113, or part or all of the drive unit 502 may be mounted on the signal processing chip 111. A part of the system control unit 501 may be mounted on the imaging chip 113 or the signal processing chip 111.

  FIG. 5 is a block diagram showing a specific configuration as an example of the signal processing chip 111. The signal processing chip 111 has a function of the driving unit 502.

  The signal processing chip 111 performs overall control of the sensor control unit 441, the block control unit 442, the synchronization control unit 443, the signal control unit 444, the individual circuit unit 450A, and the like as shared control functions. Drive controller 420. The signal processing chip 111 further includes an image processing unit 470 disposed on the signal processing chip 111 side, and an I / F circuit 418 between the drive control unit 420 and the system control unit 501 of the imaging apparatus 500 main body. These sensor control unit 441, block control unit 442, synchronization control unit 443, signal control unit 444, drive control unit 420, and image processing unit 470 are provided one by one for the signal processing chip 111.

  On the other hand, the individual circuit units 450A, 450B, 450C, 450D, and 450E are provided for each unit block 131A, 131B, 131C, 131D, and 131E. Since the individual circuit units 450A, 450B, 450C, 450D, and 450E have the same configuration, the individual circuit unit 450A will be described below. The individual circuit unit 450A includes a CDS circuit 410, a multiplexer 411, an A / D conversion circuit 412, a demultiplexer 413, a memory 414, and an arithmetic circuit 415.

  The arithmetic circuit 415 transmits and receives signals to and from the system control unit 501 via the I / F circuit 418. In the present embodiment, the signal processing chip 111 and the image processing unit 511 on the system control unit 501 side are separate packages and are electrically connected to each other through a signal line 490. Accordingly, the arithmetic circuit 415 transmits and receives signals to and from the image processing unit 511 via the I / F circuit 418 and the signal line 490.

  The individual circuit unit 450A is preferably arranged in a region that overlaps the region in which the pixels of the corresponding unit block 131A are arranged. Accordingly, the individual circuit unit 450A can be provided in each of the plurality of unit blocks 131A without increasing the size of each chip in the surface direction.

  The drive control unit 420 refers to the timing memory 430, converts an instruction from the system control unit 501 into a control signal that can be executed by each control unit, and delivers the control signal to each. In particular, when the drive control unit 420 controls each of the unit blocks 131A and the like with separate control parameters, the drive control unit 420 hands over the control parameters to each control unit together with information specifying the unit block 131A. The drive control unit 420 once receives a shooting instruction signal from the system control unit 501 in one image acquisition control, and thereafter performs accumulation control without receiving an instruction from the system control unit 501 for each pixel control. Can be completed.

  The sensor control unit 441 performs transmission control of control pulses that are transmitted to the imaging chip 113 and are related to charge accumulation and charge readout of each pixel. Specifically, the sensor control unit 441 controls the start and end of charge accumulation by sending a reset pulse and a transfer pulse to the target pixel, and sends a selection pulse to the readout pixel. A pixel signal is output to the output wiring 309.

  The block control unit 442 executes transmission of a specific pulse that specifies the unit block 131 to be controlled and is transmitted to the imaging chip 113. The transfer pulse and the reset pulse received by each pixel via the TX wiring 307 and the reset wiring 306 are the logical product of each pulse sent by the sensor control unit 441 and a specific pulse sent by the block control unit 442. In this way, each area can be controlled as an independent block.

  The synchronization control unit 443 sends a synchronization signal to the imaging chip 113. Each pulse becomes active in the imaging chip 113 in synchronization with the synchronization signal. For example, by adjusting the synchronization signal, random control, thinning control, and the like that control only specific pixels of pixels belonging to the same unit block 131A or the like are realized.

  The signal control unit 444 mainly performs timing control for the A / D conversion circuit 412. The pixel signal output via the output wiring 309 is input to the A / D conversion circuit 412 via the CDS circuit 410 and the multiplexer 411. The CDS circuit 410 removes noise from the pixel signal.

  The A / D conversion circuit 412 is controlled by the signal control unit 444 to convert the input pixel signal into a digital signal. The pixel signal converted into the digital signal is transferred to the demultiplexer 413 and stored as a pixel value of digital data in the memory 414 corresponding to each pixel.

  The memory 414 is provided with a data transfer interface that transmits pixel signals in accordance with a delivery request. The data transfer interface is connected to a data transfer line connected to the image processing unit 511. The data transfer line is constituted by a data bus of the bus lines, for example. In this case, the delivery request from the system control unit 501 to the drive control unit 420 is executed by address designation using the address bus.

  Transmission of pixel signals by the data transfer interface is not limited to the addressing method, and various methods can be adopted. For example, when performing data transfer, a double data rate method in which processing is performed using both rising and falling edges of a clock signal used for synchronization of each circuit may be employed. Further, it is possible to adopt a burst transfer method in which data is transferred all at once by omitting a part of the procedure such as addressing and the like, and the speed is increased. Further, a bus system using a line in which a control unit, a memory unit, and an input / output unit are connected in parallel, or a serial system that transfers data one bit at a time can be combined.

  With this configuration, the image processing unit 511 can receive only necessary pixel signals, so that image processing can be completed at high speed, particularly when a low-resolution image is formed. In addition, when the arithmetic circuit 415 executes the integration process, the image processing unit 511 does not have to execute the integration process, so that the image processing can be speeded up by the function sharing and the parallel processing.

  The signal processing chip 111 has a timing memory 430 formed by a flash RAM or the like. The timing memory 430 stores control parameters such as the number of times of charge accumulation for which unit block 131A and the like is repeated in association with information specifying the unit block 131A and the like. Any one of the control parameters is calculated by the arithmetic circuit 415 such as the individual circuit unit 450A and stored in the timing memory 430.

  The drive control unit 420 refers not only to the charge accumulation control for the imaging chip 113 but also to the timing memory 430 in the execution of the read control. For example, the drive control unit 420 refers to the accumulation count information of each unit block 131 and stores the pixel signal output from the demultiplexer 413 at the corresponding address of the memory 414.

  The drive control unit 420 reads the target pixel signal from the memory 414 in accordance with a delivery request from the system control unit 501 and delivers it to the image processing unit 511. The memory 414 has a memory space that can store a pixel signal corresponding to each pixel.

  The image processing unit 470 acquires pixel signals from the arithmetic circuits 415 such as the individual circuit units 450A and 450B, and generates a reduced image. The image processing unit 470 specifies the compression rate of the pixel signal of the unit block 131 corresponding to the individual circuit units 450A and 450B based on the reduced image, and transmits the compression rate to each arithmetic circuit 415.

  As described above, the output wiring 309 is provided corresponding to each of the unit blocks 131. Since the image pickup device 100 has the image pickup chip 113, the signal processing chip 111, and the memory chip 112 laminated, by using electrical connection between the chips using the bump 109 for the output wiring 309, each chip is arranged in the surface direction. Wiring can be routed without increasing the size. Similarly, for the signal lines from each control unit to the unit block, by using the electrical connection between the chips using the bumps 109, the wiring can be routed without enlarging each chip in the surface direction.

  FIG. 6 shows functional blocks of the image processing unit 470. The image processing unit 470 includes a reduced image generation unit 472, an area determination unit 474, and a compression rate setting unit 478.

  The reduced image generation unit 472 acquires the pixel signal stored in the memory 414 such as each individual circuit unit 450A, and generates a reduced image having a smaller number of pixels than the pixels included in the imaging region. In this case, for example, the reduced image generation unit 472 generates a reduced image by thinning out pixels or using an average value as a representative value between adjacent pixels.

  The region determination unit 474 determines the saliency of the reduced image, and identifies the unit block 131 corresponding to the region of interest with high saliency in the reduced image and the unit block 131 corresponding to the other peripheral regions. For example, the region determination unit 474 hierarchically applies a Gaussian filter to the reduced image and takes a difference between the layers, thereby specifying a region having a property different from that of the peripheral region as the region of interest.

  The compression rate setting unit 478 sets a compression rate applied to the unit block 131 corresponding to the target area and a compression rate applied to the unit block 131 corresponding to the peripheral area. In this case, it is preferable that the compression rate applied to the unit block 131 corresponding to the peripheral region is set higher than the compression rate applied to the unit block 131 corresponding to the region of interest.

  FIG. 7 shows functional blocks of the arithmetic circuit 415. The arithmetic circuit 415 illustrated in FIG. 7 includes a compression circuit conforming to JPEG lossy compression as an example. That is, the arithmetic circuit 415 includes a DCT (discrete cosine transform) unit, a quantization unit 454, a quantization table 456, an encoding unit 458, and an encoding table 460 in addition to the functions described above.

  The DCT unit 452 calculates a DCT coefficient for each frequency component when the image signal represented by the pixel value at each pixel position is represented by a frequency component. In this case, the DCT unit 452 calculates a DCT coefficient in units of a predetermined number of pixels such as 4 × 4 or 8 × 8. The number of pixels of the unit block 131 is preferably an integer multiple of the number of pixels for which the DCT unit 452 calculates the DCT coefficient. Further, it is more preferable that the number of pixels in the X direction and the Y direction of the unit block 131 is an integral multiple of the number of pixels in the X direction and the Y direction, respectively, of the number of pixels for calculating the DCT coefficient.

  The quantization table 456 is a table in which a quantization step for quantizing a DCT coefficient is designated for each frequency component. In the quantization table 456, the quantization step is specified so that the low frequency components are omitted and the high frequency components are omitted. The quantization table 456 is generated based on the compression rate set by the compression rate setting unit 478. The quantization unit 454 quantizes the DCT coefficient calculated by the DCT unit 452 based on each quantization step of the quantization table 456.

  The encoding table 460 is a table indicating encoding rules for assigning a code having a smaller number of bits as the appearance frequency is higher. The encoding unit 458 encodes the DCT coefficient quantized by the quantization unit 454 according to the encoding rule specified by the encoding table 460. Further, the encoding unit 458 outputs the encoded pixel signal. In this case, the encoding unit 458 outputs the quantization table 456, the encoding table 460, etc. themselves or identification information for identifying them to the memory 414 as header information of the pixel signal.

  FIG. 8 shows an example of the operation flow of the image sensor 100. The operation flow starts when a live view is displayed (or sometimes referred to as a live view) or when the release button is pressed halfway. FIG. 9 is an example of the reduced image 170.

  The drive control unit 420 drives the sensor control unit 441 and the like for the through image display or the compression rate setting, acquires the pixel signal from the pixel, and stores it in the memory 414 (S100). In this case, the drive control unit 420 controls the accumulation and readout of pixel signals for each unit block 131.

  The reduced image generation unit 472 acquires the pixel signal stored in the memory 414 such as each individual circuit unit 450A, and generates a reduced image (S102). In this case, the reduced image generation unit 472 may acquire pixel signals included in all unit blocks 131 included in the image area, or may omit acquisition of pixel signals from several unit blocks 131. Good.

  The region determination unit 474 determines the saliency of the reduced image (S104), and specifies the unit block 131 corresponding to the region of interest with high saliency and the unit block 131 corresponding to other peripheral regions (S106). . For example, when the reduced image 170 shown in FIG. 9 is obtained, the thick frame including the subject 171 is set as the region of interest 173, and the others are specified as the peripheral region 175.

  The compression rate setting unit 478 sets different compression rates for the unit block 131 corresponding to the region of interest 173 and the unit block 131 corresponding to the peripheral region 175 (S108). In this case, the compression rate setting unit 478 sets a low compression rate quantization table 456 in the arithmetic circuit 415 of the unit block 131 corresponding to the region of interest 173. On the other hand, the compression rate setting unit 478 sets a high compression rate quantization table 456 in the arithmetic circuit 415 of the unit block 131 corresponding to the peripheral region 175. Instead, each arithmetic circuit 415 has the same quantization table 456, and the compression rate setting unit 478 applies coefficients to the entire quantization table 456 to the attention area 173 and the peripheral area 175, respectively. It may be set.

  Wait until the release button is pressed down (S110: No), and when the release button is pressed down, the drive control unit 420 drives the sensor control unit 441 and the like to acquire the pixel signal from the pixel for the main photographing. And stored in the memory 414 (S112). Further, the arithmetic circuit 415 of each unit block 131 compresses the pixel signal of the unit block 131 at the set compression rate (S114).

  In the example illustrated in FIG. 7, the compression rate is reflected in the quantization table 456. That is, since the quantization table 456 having a low compression rate is set in the arithmetic circuit 415 of the unit block 131 corresponding to the region of interest 173, the pixel signal of the unit block 131 is compressed at a low compression rate. On the other hand, since a high compression rate quantization table 456 is set in the arithmetic circuit 415 of the unit block 131 corresponding to the peripheral region 175, the pixel signal of the unit block 131 is compressed at a high compression rate.

  Each arithmetic circuit 415 outputs header information including the quantization table 456 used for compression together with the compressed pixel signal of the corresponding unit block 131 to the memory 414 (S116). Thereby, the flow of FIG. 8 is completed.

  FIG. 10 shows an example of the operation flow of the system control unit 501. The operation flow starts when a pixel signal is input from the image sensor 100 when the release button is pressed down.

  The image processing unit 511 acquires the pixel signal corresponding to each unit block 131 from the memory 414 via the I / F circuit 418 and the signal line 490 (S120). In this case, the image processing unit 511 also acquires header information associated with the pixel signal of the unit block 131.

  The image processing unit 511 expands the pixel signal corresponding to each unit block 131 corresponding to the compression rate when compressed (S124). In this case, the image processing unit 511 specifies a quantization table based on the header information associated with the pixel signal of each unit block 131, and expands the pixel signal using the quantization table. The entire decompression process is equivalent to tracing the operation of each part in FIG. As a result, the image processing unit 511 obtains a pixel signal of one image.

  The image processing unit 511 performs predetermined image processing on the pixel signals of one image (S124), and records the image obtained after the image processing in the recording unit 505 or the like. The predetermined image processing includes processing such as pixel interpolation and processing for changing the image format. Thereby, the flow of FIG. 10 is completed.

  As described above, according to the present embodiment, since the imaging chip 113 provided with the imaging region 120 and the signal processing chip 111 provided with the individual circuit unit 450A for compressing the pixel signal are stacked, the compression processing is performed at high speed. It can be performed. In particular, since the imaging region 120 and the individual circuit unit 450A and the like are physically close to each other, signal attenuation is small and deterioration of the S / N ratio can be prevented.

  Furthermore, since the individual circuit unit 450A is provided for each unit block 131, the compression process can be performed at a higher speed. By setting the compression rate for each unit block 131, the balance between the processing speed and the image quality can be set. In addition, since the pixel signal from the imaging region 120 is compressed and then transferred to the external image processing unit 511, high image quality can be achieved without increasing the signal output band between the imaging device 100 and the image processing unit 511. Pixel signals can be passed.

  When the header information includes information for specifying the table instead of the quantization table itself, the image processing unit 511 determines the quantization table specified by the information from a plurality of built-in quantization tables. May be read out. Further, instead of or in addition to the quantization table, the compression rate may be set by the encoding table. Furthermore, the present invention may be applied to the case where a compression / decompression method other than JPEG is used in the transfer of pixel signals between the signal processing chip 111 and the image processing unit 511.

  The reduced image may be composed of any one color of RGB or may be composed of luminance values in each pixel of RGB. Further, compression may be performed for each color. Furthermore, the area determination unit 474 may determine the saliency based on the pixel signals from the entire pixels of the imaging region 120 without providing the reduced image generation unit 472 in the image processing unit 470.

  The determination of the saliency may be made by using the magnitude of the high frequency component, the magnitude of the luminance value, the degree of hue, the presence / absence of motion, etc. instead of using the Gaussian filter. Instead of this, or in addition to this, it is also possible to determine a face and use a region including the face as a region of interest. Instead of or in addition to this, an area including the central portion of the imaging area 120 may be set as the area of interest.

  One arithmetic circuit 415 may be provided for the entire imaging region. Alternatively, the arithmetic circuit 415 may be provided for each pixel. In this case, the individual circuit unit 450A or the like may be provided for each pixel.

  FIG. 11 is another example of specifying the region of interest. 11, the same operations as those in FIG. 8 are denoted by the same reference numerals, and the description thereof is omitted.

  The reduced image generation unit 472 outputs the reduced image generated in step S102 to the image processing unit 511 via the signal line 490 (S130). The image processing unit 511 determines saliency by the same method as in step S104 (S132), and specifies the region of interest 173 (S134). The image processing unit 511 outputs information specifying the region of interest 173 to the compression rate setting unit 478 via the signal line 490 (S136). In this case, for example, the image processing unit 511 outputs information for identifying the unit block 131 included in the region of interest 173 to the compression rate setting unit 478.

  As described above, as illustrated in FIG. 11, the compression rate setting may be performed on the system control unit 501 side. In this case, the region determination unit 474 and the compression rate setting unit 478 in FIG. 6 may not be provided in the image processing unit 470 on the signal processing chip 111 side.

  The region of interest may be specified by the photographer using the display unit 506. In this case, the reduced image generated by the reduced image generation unit 472 is displayed on the display unit 506, and the region of interest 173 is specified by operating the touch panel, the cross key, and the like. The area determination unit 474 specifies the unit block 131 included in the specified attention area 173. Instead of displaying a reduced image on the display unit 506, a captured image generated by the image processing unit 511 may be displayed.

  Further, the region of interest 173 may be specified by the photographer using a display device other than the display unit 506. In this case, the display device is provided in an electronic device such as a smartphone that is connected to the imaging device 500 in communication. The electronic device receives the reduced image generated by the reduced image generation unit 472 and displays it on the display device. Furthermore, the region of interest 173 is specified by the operation of the photographer on the touch panel, the cross key, and the like of the electronic device. The electronic device transmits information specifying the region of interest to the imaging apparatus 500. The area determination unit 474 of the imaging apparatus 500 specifies the unit block 131 included in the specified attention area 173. Note that the captured image generated by the image processing unit 511 may be displayed instead of displaying the reduced image on the display device.

  In at least one of the steps S100 and S112, the imaging condition may be different between the unit block 131 of the region of interest 173 and the unit block 131 of the peripheral region 175. The imaging conditions to be varied include the frame rate, charge accumulation time, imaging sensitivity, and the like. When the imaging condition is the frame rate, it is preferable that the frame rate of the region of interest 173 be larger than the frame rate of the peripheral region 175. When the imaging condition is imaging sensitivity, the imaging sensitivity of the region of interest 173 is preferably higher than the imaging sensitivity of the peripheral region 175.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  100 imaging device, 101 microlens, 102 color filter, 103 passivation film, 104 PD, 105 transistor, 106 PD layer, 107 wiring, 108 wiring layer, 109 bump, 110 TSV, 111 signal processing chip, 112 memory chip, 113 imaging Chip, 120 imaging area, 131 unit block, 131A unit block, 131B unit block, 131C unit block, 131D unit block, 131E unit block, 170 images, 171 subject, 173 area of interest, 175 peripheral area, 302 transfer transistor, 303 reset Transistor, 304 amplification transistor, 305 selection transistor, 306 reset wiring, 307 TX wiring, 308 decoder wiring, 309 output wiring, 10 Vdd wiring, 311 load current source, 410 CDS circuit, 411 multiplexer, 412 A / D conversion circuit, 413 demultiplexer, 414 memory, 415 arithmetic circuit, 418 I / F circuit, 420 drive control unit, 430 timing memory, 441 Sensor control unit, 442 Block control unit, 443 Synchronization control unit, 444 Signal control unit, 450A Individual circuit unit, 450B Individual circuit unit, 450C Individual circuit unit, 450D Individual circuit unit, 450E Individual circuit unit, 452 DCT unit, 454 Quantum Conversion unit, 456 quantization table, 458 encoding unit, 460 encoding table, 470 image processing unit, 472 reduced image generation unit, 474 region determination unit, 478 compression rate setting unit, 490 signal line, 500 imaging device, 501 system Control unit, 502 drive , 503 photometric unit, 504 a working memory, 505 a recording unit, 506 display unit, 511 image processing unit, 512 operation unit, 520 imaging lens

Claims (1)

An imaging unit having a plurality of pixels provided on the first substrate;
A signal compression unit for compressing a signal from the imaging unit, provided on a second substrate laminated with the first substrate;
An imaging device comprising:

Images