TW201228382A - Capturing and processing of images using monolithic camera array with heterogeneous imagers - Google Patents

Abstract

Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. Lens stack arrays in accordance with many embodiments of the invention lens elements formed on substrates separated by spacers, where the lens elements, substrates and spacers are configured to form plurality of optical channels, at least one aperture located within each optical channel, at least one spectral filter located within each optical channel, where each spectral filter is configured to pass a specific spectral band of light, and light blocking materials located within the lens stack array to optically isolate the optical channels.

Description

201228382 VI. Description of the Invention: [Technical Field] The present invention relates to an image sensor comprising a plurality of heterogeneous imagers, and more particularly to a custom filter having a different architecture, sensing Image sensor for a plurality of wafer level imagers of optical instruments and optical instruments. [Prior Art] Image sensors are used in cameras and other imaging devices to capture images. In a typical imaging device, light enters through an opening (aperture) at one end of the imaging device and is guided to an image sensor by an optical member such as a lens. In most imaging devices, one or more layers of optical components are placed between the aperture and the image sensor to focus light onto the image sensor. The image sensor is composed of pixels that receive light through the optical member to generate a signal. A commonly used image sensor includes a CCD (Charge Coupled Device) image sensor and a CMOS (Complementary Metal Oxide Semiconductor) sensing. Device. Filters are often used in the image sensor to selectively deliver light of certain wavelengths to the pixel. A Bell filter mosaic is often formed in the image sensor. The Hebel filter is a color filter array to arrange one of the red, green and blue three-color wipes for each of the color pixels. The Bell filter pattern contains fifty percent green filter, twenty-five percent red patch, and one hundred and fifteen blue filter. Since each pixel produces a signal to represent the intensity of the color component of the light t, but not the full color range, the demosaicing technique is performed within (four) the red, green, and blue values of each image pixel. 201228382 These image sensors are subject to various execution efficiency limitations. The performance efficiencies of these image sensors include, inter alia, dynamic range, signal-to-noise ratio (SNR), and low light sensitivity. The dynamic range is defined as the ratio of the maximum possible signal that the pixel can capture to the total noise. Typically, the image sensor's well capacity limits the maximum possible signal that the image sensor can capture. The maximum possible signal is then dependent on the intensity of the incident illumination and the duration of the exposure (e.g., integration time and shutter width). The dynamic range can be expressed in decibels W... as a dimensionless number: DR = full well capacity rms noise formula (1) Typically, the noise level in the captured image affects the dynamic range floor. Therefore, for an eight-bit ‘ ‘ 像 像 像 像 s, assuming that the average root noise is aligned as a bit, the best example would be 48 decibels. However, in practice, the f-averaged noise level is higher than the -bit, and this step-by-step reduces the signal-to-noise ratio (SNR) of the slave image over a large range - the image port (four) magnitude . In general, the signal-to-noise ratio is higher when the pixel captures more light. The signal-to-noise ratio of the captured image is usually related to the light collecting capability of the pixel. In general, the Muir Chopper sensor has low light sensitivity. In the low light level, the low signal: = Γ light collecting capability is received by each pixel. In addition, the color data above the pixel enters the signal of the second. The IR (infrared) film simultaneously reduces the photoresponse from the near-infrared signal, which can carry useful information. 5 201228382 Due to the limited nature of the material, the efficiency limits of these image sensors are greatly magnified in the camera designed for the line (4). The pixels of a mobile camera are typically much smaller than the hub, the pixels of a digital camera (DSC). Due to the limitation of the light collecting capability, the reduced signal & 叼 (4) is less effective than the 'dynamic range limitation and reduced low-light scene sensitivity, and the camera in the action phase stack exhibits poor execution efficiency. SUMMARY OF THE INVENTION A camera array, an imaging device including a camera array, and/or a method of capturing images using a plurality of imagers are disclosed, wherein each imager includes a plurality of sensor components and can be used according to the present invention. Inventive embodiments are used for lens stack arrays in camera arrays. The plurality of imagers can include at least a first-imager and a second imager, wherein the first imager and the second imager can have the same imaging features or different imaging features. In an embodiment, the first imager and the second imager have different imaging features, and the imaging features may include, in particular, the imager size, a pixel type contained in the imager, the imager shape, and the imaging Filter associated with the filter, exposure time of the imager, aperture size associated with the imager, optical component architecture associated with the imager, gain of the imager, resolution of the imager, and operation of the imager Timing. In an embodiment the first imager comprises a filter for transmitting a spectrum. The second imager also includes the same type of filter for transmitting the same spectrum as the first imager, but captures images resulting from sub-pixel phase shifts in the image captured by the first imager. Images from the first imager and the 201228382 second imager are combined using a super-resolution method to obtain a higher resolution image. In one embodiment, the first imager includes a first sheet for transmitting a first light ", and the second imager includes a first filter for transmitting a second spectrum. Images from the first and second imagers are then processed to obtain a higher quality image. In a preferred embodiment, 'lens members are provided to direct and condense on the imagers." The lens members form a lens stack to create optical channels, and each lens stack focuses light onto an imager. Because each lens member is associated with an imager' each lens member can be designed and framed to provide a narrow spectrum. Further, the thickness of the lens member can be reduced to reduce the overall thickness of the camera array. In such embodiments, the lens members can be fabricated using any suitable fabrication technique, for example, using wafer level optical (WLO) technology, injection molding, and/or glass molding. In an embodiment, the plurality of imagers comprise at least one near infrared imager dedicated to receiving a near IR (infrared) spectrum. The image produced by the near infrared imager can be blended with images produced by other imagers having color filters to reduce noise and increase the quality of the images. Imagers that encompass other spectral ranges including far infrared and ultraviolet spectra may also be incorporated in another such embodiment. In an embodiment, the plurality of imagers can incorporate a lens member that provides zoom capability. In one such embodiment, different imagers can combine lenses of different focal lengths to have different fields of view and provide varying degrees of zoom capability. Imagers with different sensor sizes/formats can also be used for different fields of view, which are derived from different pixel sizes or different pixels/photosensitive members by g 7 201228382. - A mechanism can be provided to provide a smooth transition from one zoom level to another zoom level. In one or more embodiments, the plurality of imagers are coordinated to obtain at least one of a high dynamic range image, a panoramic image, a hyperspectral image, a distance to the object, and a high aspect rate video command. . In accordance with an embodiment of the invention, an imaging device includes at least an imaging array, and each of the imagers includes a plurality of photosensitive members and a lens stack including at least one lens surface, wherein the lens stack is structured to Forming an image on the photosensitive member; a control circuit configured to capture an image formed on the photosensitive member of each of the imagers; and an ultra-resolution processing module configured to use a plurality of The image is captured to produce at least one higher resolution super resolution image. In accordance with another embodiment of the present invention, a lens stack array includes lens members formed on a substrate separated by spacers, wherein the lens members, the substrate, and the spacers are structured to form a plurality of optical channels, each at each At least one aperture in an optical channel, at least one spectral filter positioned in each optical channel, and a light blocking material positioned within the lens stacking array to optically isolate three optical channels, wherein each • A spectral subtraction is clamped to pass a specific spectral band. The features and advantages of the present invention are not intended to be exhaustive, and many additional features and advantages will be apparent to those skilled in the art. It should be noted that the language used in the specification is in principle selected based on the ease of reading and teaching purposes, and is not intended to describe or limit the invention. 201228382 [Embodiment] Now, the embodiment of the present invention is described above, wherein the reference number indicates the same or similar function. In the figure, the leftmost digit of each reference number is in the number of the number. The first discussed embodiment is a distributed method for capturing images produced by a plurality of imagers using different imaging features. Each of the imagers can be constructed in such a way that each image captures an image that is offset by a single pixel amount, like images similar to those captured by other imagers. The mother-imager can also include separate optical instruments with different filters and operate with different operating parameters (e.g., 'exposure time). The different images produced by the imagers are processed to obtain a enhanced image. In many embodiments, the individual optical instruments integrated into each of the imagers are configured using a mirror stack array. The array of lens stacks can include one or more optical components fabricated using wafer level optical (WLO) technology. A sensor member or pixel is referenced to an individual photosensitive member within an imager. The photosensitive member may be a conventional CIS (Complementary Metal Oxide Semiconductor Image Sensing), a CCD (Charge Coupled Device), a high dynamic range pixel, a multi-edge pixel, and various alternative members thereof, but is not limited thereto. A sensor is referenced to a two-dimensional array of pixels for capturing images formed by the optical instrument of the imager on the sensor. The sensor components of each sensor have similar physical properties and receive light through the same optical components. Further, the sensor components within each sensor can incorporate the same color filter. 201228382 A camera array reference to the apricot. ... 1 And I used a lot of imaging smashing in the field. The camera array can be fabricated from a single wafer #%# on a daily basis or installed in or installed in various devices. The array of camera arrays is referenced to the aggregation of two or more camera arrays. Second, more camera arrays can operate together to provide an extended function of the single-camera array, for example, imaging features of a beta-imager such as stereo resolution refer to any feature or parameter of the imager associated with the image capture. The imaging feature may include, in particular, the imager scale: a type of pixel included in the image n, a shape of the imager, a cymbal associated with the imaging benefit, and a wide and a slanted slanting umbrella of the imaging test Exposure time, aperture size associated with the imager, optical component architecture associated with the imager (eg, number of components, shape, contour, and size of the lens surfaces, including radius of curvature 'non-spherical coefficients, The focal length and field of view of the objective lens, color correction, aperture ratio / 2 distance, etc.), the gain of the imager, the resolution of the imager, and the operational timing of the imager. Camera Array Structure Figure 1 is a plan view of a camera array 1 具有 having imagers i Α to 根据 according to a consistent embodiment. The sinusoidal camera array 100 is fabricated on a semiconductor wafer to include a plurality of imagers 1A through NM. Each of the imagers 1 to ^^^ may comprise a plurality of pixels (eg, 0. 32 megapixels). In one embodiment, the imagers 1 to ΝΜ are arranged in the mesh format shown in Fig. 1. In other embodiments, the imagers are arranged in a non-mesh format. For example, the imagers can be arranged in an annular pattern, a zigzag pattern or a scattering pattern or an irregular pattern containing sub-pixel shifts. The 201228382 5 camera array can contain two or more heterogeneous imager types, each of which includes two or more sensor components or pixels. Each of the imagers can have different imaging features. Alternatively, there may be two or more different imaging "types" in which the same imager type shares the same imaging characteristics. In the case of Bayes, each imager has its own filter and/or optical member (e.g., a lens). In particular, the imagers may incorporate certain spectral wavelengths in conjunction with a spectral color filter for each or a group of imagers. The demonstration filter contains the traditional cymbal, infrared cymbal, near-infrared, (four), polarized film and high suitable for use in the Bell pattern (red, green, blue or their complementary colors (cyan, magenta, yellow). Custom imaging filters for money imaging. Some imagers may have no filter to allow reception of both the visible and near infrared rays, which increases the signal to noise ratio of the imager. The number of different images can be correlated with the camera array. The number of imagers is as much as it is advanced. Each of the imagers 1 to 1 or a group of imagers can receive light through lenses having different optical characteristics (eg, focal length) or different aperture sizes. The camera array includes other associated circuitry. The circuitry may include, inter alia, circuitry for controlling imaging parameters and sensors that sense physical parameters. The control circuitry may control imaging parameters such as exposure time, increased black-order offset. The sensor can include dark pixels to estimate dark current at the operating temperature. The dark current can be measured to damage the substrate that may be subject to any thermal creep. Motion compensation. Alternatively, W, such as the thermal effect compensation associated with the optical instrument due to changes in the refractive index of the lens material, may be obtained by weighting the point spread function at different temperatures. In one embodiment, 'for The circuitry that controls the imaging parameters can trigger the parent imager individually or in a synchronized manner. The activation of the exposure periods of various imagers in the camera array (similar to opening the shutter) can be staggered in an overlapping manner such that the scenes are Simultaneous sampling allows some imagers to be exposed at the same time. When a conventional camera samples a scene with N exposures per second, the exposure time of each sample is limited to 1/N second, using multiple imagers because multiple The imager can be operated to capture images in a staggered arrangement, so there is no such exposure time limitation. The parent imager can be operated independently. The overall or majority of operations associated with each individual imager can be processed individually. In an embodiment, a main set parameter is programmed and the error of such a main set parameter of each imager (ie, offset or increase) Is configured to reflect, for example, a high dynamic range, a gain setting parameter, an integrated time setting parameter, a digital processing setting parameter, or a combination thereof. These errors may indicate that the particular camera array is at a low level (eg, the gain Error) or a high level (eg, a difference in the open system link number, which is then automatically converted to a delta integral for gain, integration time, or other aspects as shown in the context/master control register). Setting the master values and the errors of the master values, higher level control abstractions can be obtained, facilitating simpler program modules for many operations. In one embodiment, the imagers are The parameters are arbitrarily fixed for a target application. In another embodiment, the parameters are architected to allow for flexibility and programmability. In one embodiment, the camera array is designed as a direct replacement component 12 201228382 Instead of using it on mobile phones and straightforward, it’s overwhelming. The heart is too far from the existing camera image in the ', 匕 装置 。 。. Based on this w, although in many photographic situations, the resolution of the resulting camera may exceed that of a conventional image sensor, the camera array can still be set

The conventional image sensor that is calculated to be substantially identical to the resolution of the sound is a physical compatibility with the conventional image sensor of the & a, X degree. Obtaining the increased execution efficiency is superior, and the camera array according to the embodiment of the present invention may have the same or better than the conventional image sensor. Quality image. Alternatively, the pixel size in the imaging benefit of Jude can be reduced compared to the pixels in the conventional image sensor to achieve considerable results. In order to reduce the total number of original pixels of the conventional image sensor without increasing the area, the logic of the individual imagers is preferably limited to the area of the stone. __################################################################################################# Bo.产士 — J 王σ丨 or 夕数. In this embodiment, the conventional external interface for the imager can be used because the data of the imagers is not increased. In her example, a camera array containing these imagers replaces the traditional image device of M million pixels. The camera array includes a keyer, Μ each sensor contains iV pixels. Each imager in the camera array also has the same aspect ratio as the /, replaced by the traditional scene sensor. Table 1 lists an exemplary architecture of a camera array of the present invention that replaces conventional image sensors. 13 201228382 Table 1

The super-resolution coefficients in Table 1 are estimates I, and the effective resolution values may vary depending on the actual super-resolution factor obtained by the process. The number of imagers in the camera array can be determined in particular by (1) resolution, (ii) parallax, (iii) sensitivity, and (iv) dynamic range factor. The factor-factor resolution for the size of the imager. From a resolution point of view, the preferred number of imagers ranges from 2x2 to 6x6, which is more likely to corrupt the frequency information than the 6x6 array size and cannot be regenerated by the hyper-resolution program. For example, an 8 megapixel resolution with a 2x2 imager would require 2 megapixels per imaging state. Similarly, the 8 megapixel resolution of a 5 χ 5 imager would require 成像 32 megapixels per imager. In many embodiments, the number of imagers in the array is determined by a particular application requirement. The second factor that limits the number of imagers is parallax and shadowing issues. Viewing the object captured in the image, the imager is shaded by the background portion 14 201228382 ^ can be called the A shadow group when the two imagers capture the object from two different positions of the shadow group of each imager The system is different. Therefore, there is only one % of the scene pixels β A captured by the imaging benefit to solve the problem of shielding. The imaging state type is to include a minimum of imager groups to a certain extent, and the imagers are symmetrically distributed to the camera array. Around the center axis. The third factor that sets the lower limit on the number of imagers is the sensitivity 4 under low lighting conditions. In order to improve the low light sensitivity, an imager for detecting near-infrared light can be required. The number of imagers in the camera array may need to be increased to accommodate such near infrared imagers. The fourth factor that determines the size of the imager is the dynamic range. In order to provide dynamic range in the eyepiece array, it is advantageous to provide an imager of (4) cymbal type (chroma or party). Each of the same filter type imagers can then be operated simultaneously with different exposure times. ^The image captured by the exposure time can be processed to produce a high dynamic range image. Based on these factors, the number of preferred imagers is 2χ2 to 6χ6. The 4χ4 and 5x5 architectures are better than the 2χ2Λ3χ3 architecture, which is likely to provide sufficient number of imagers to solve the shadowing problem, increase sensitivity and increase the dynamic range. In addition, a rectangular array is also preferred. At the same time, the amount of computational load required to restore the resolution of these array sizes would be modest compared to the (four) array: the amount of computational load required. However, large 9W arrays may be used to provide additional features such as optical zoom and multi-spectral imaging. Although only square imagers are described herein, such imagers may have different X and y dimensions as will be explained in more detail later. Another consideration is the number of imagers dedicated to luminance sampling. By ensuring that the imager dedicated to near-infrared sampling in the array does not reduce the resolution of the acquisition, the information from the near-infrared imagers is added to the resolution captured by the luminance imagers. At least 50 percent of the imager based on the present purpose can be used to sample the brightness and/or near infrared spectrum. In an embodiment of a 4x4 imager, four imagers sample the brightness, four imagers sample the near infrared, and the remaining eight imagers sample the two chrominance (red and blue). In another 5x5 imager embodiment, 9 imagers sample brightness, 8 imagers sample near infrared, and the remaining 8 imagers sample dichroism (red and blue further, imagers with these filters) Can be symmetrically arranged within the camera array to account for shadowing due to parallax. In a further 5x5 imager embodiment, 17 imagers sample brightness, 4 imagers sample red' and 4 imager samples blue In an embodiment, the imagers in the camera array are spatially separated from each other by a predetermined distance. By increasing the spatial spacing, the parallax between the images captured by the imagers is increased. Parallax is advantageous where the more precise distance information is important. The spacing between the two imagers can also be increased to approximate the spacing between a pair of human eyes. By approaching the interval between human eyes, a realistic stereoscopic 3D image can be provided. The resulting image is rendered on a suitable stereoscopic display device. In one embodiment, multiple camera arrays are provided at different locations on a device to overcome spatial limitations. The array can be designed to be mounted in a limited workspace while another camera array can be placed in another limited space of the device. For example, if a total of 2 imagers are required but the available space is only allowed for each device A camera array with a 1x10 imager is provided on the side, and each of the 16 201228382 arrays containing 10 idiots and amps can be placed on the two available spaces of the device. Each of the secrets, m cameras The array can be fabricated on a substrate and secured to a host 4 ε; 4, board or other component of a device. Furthermore, such imagers do not have a homogenous size and may have a non-dimensional and y-dimensional. The images collected by the array can be processed to produce images of desired resolution and efficiency of execution. The design for the single-imager can be applied to different camera arrays each containing other imager types. Other variables, such as inter-set distances, color filters, and combinations of the same or different sensors, can be modified to produce a camera array with different imaging features. In this manner, the camera array A wide variety of columns can be created and preserved on an economic scale. Wafer-level optical integration In a common example, the camera array utilizes wafer level optics (WL〇) technology. In many embodiments, Similar optical channels can be constructed using any of a variety of techniques including injection molding, glass molding, and/or these techniques and junctions containing wafer level optical technology: but not limited to this. Wafer-level optical technology itself Included are some process techniques, including, for example, molding optical instruments on a glass wafer (eg, an array of lens modules and arrays of those lens arrays), stacking those wafers with appropriate spacers (including having replicated on the substrate) The wafer of the lens on either side, either at the wafer level or at the grain level, then directly encapsulates the optical instrument with the imager into a monolithic integrated module. Among other procedures, the wafer level optical process may involve the use of a diamond turning module to create each polymer lens structure on a glass substrate.

S 17 201228382 pieces. More particularly, the process chain in wafer level optics generally involves producing a diamond turning lens substrate (both at the individual and array levels), and then creating a negative mode to replicate that substrate (also known as a stamp or Tool), and finally a polymer replica is formed on a glass substrate that has been constructed using suitable supporting optical members such as apertures, light blocking materials, filters, and the like. 2A is a perspective view of a camera array assembly 200 having a wafer level optical instrument 2 and a sensor array 230, in accordance with an embodiment. The wafer level optical dry instrument 210 includes a plurality of lens members 220, each lens member 220 containing one of twenty-five imagers 24 of the sensor array 230. / In conclusion, the camera array assembly 200 has a smaller array of lens members occupying very little space compared to a single large lens containing the entire sensor array 2 3 〇. It should also be noted that each of the lenses may be of a different type. For example, each substrate level can include a lens that is diffractive, refractive, Fresnel, or a combination thereof. It should be further noted that within the camera array, a lens member 220 can include one or more separate optical lens members that are axially arranged relative to one another. Finally, it should be noted that for most lens materials, it will be a thermally induced change in the refractive index of the material that must be corrected for good image quality. - The temperature normalization procedure will be described in more detail later. 2B is a cross-sectional view of a camera array assembly 25A in accordance with an embodiment. The camera assembly 250 includes a top lens wafer 262, a bottom lens wafer 268, a plurality of sensors and associated photosensitive member substrates 278 formed thereon, and spacers 258, 264, and 270. The camera array assembly (10) is packaged in a seal 254... an optical top spacer & 258 can be placed between the seal w and the top lens wafer 262 1 and it is constructed for the camera assembly 25〇18 201228382 not important. An optical member 288 is formed on the top lens wafer 262. Although the optical members 288 shown in Figure 2B are identical, it should be understood that different component types, sizes, and shapes can still be used. An intermediate spacer 264 is placed between the top lens wafer 262 and a bottom lens wafer 268. Another set of optical members 286 are formed on the bottom lens wafer 268. A bottom spacer 27 is placed between the bottom lens wafer 268 and the substrate 278. Straight through twin vias 274 are also provided to the path to transmit signals from the imagers. The top lens wafer 262 can be partially coated with a light blocking material 284 (discussed below) to block light. The portion of the top lens wafer 262 that is not coated with the light blocking material 284 acts as an aperture stop for light to pass through to the sinusoidal lens wafer 268 and the photosensitive members. Although only a single aperture bar is shown in the embodiment provided in FIG. 2B, it should be understood that the additional aperture bar may be formed by an opaque layer disposed on any or all of the substrate faces of the camera assembly to improve stray light execution. A complete discussion of efficiency and reduced optical string a ° optical crosstalk suppression is provided below. In addition, although the above embodiments show the spacers 2 58 , 2 6 4 and 2 70 , the spacer function can also be modified by directly modifying the lens structures (or substrates) so that the lenses can be directly connected to each other. It was implemented directly. In such embodiments, the lens height can be extended' and the lens is directly bonded to the upper substrate, thereby eliminating the need for a spacer layer. In the embodiment of Figure 2B, a filter 282 is formed on the bottom lens wafer 268. Light carrying material 280 can also be applied to the bottom lens 268 to act as an optical isolator. A light-carrying material 280 can also be applied to the substrate 278 to protect the sensor electronics from incident light. Spacer 2 8 3 also 19 201228382 can be placed between the bottom lens wafer 268 and the substrate 278 and between the lens circles 262, 268. In many embodiments, the spacer 283 is similar to the spacers 264 and 270. In some embodiments, each spacer layer is configured using a - single board. Although not shown in FIG. 2B, many embodiments of the present invention also include spacers between each of the optical channels on top of the top lens wafer 262, which are similar or arranged in a single layer at the lens stack. A spacer 258 is shown at the edge of the array. As discussed further below, the spacers may be constructed of a light blocking material and/or coated with a light blocking material to isolate the optical channels formed by the wafer level optical instrument. Suitable light-blocking materials may comprise any opaque material, for example, metal materials such as titanium and chromium, or oxides of such materials as black chromium (chromium or chromium oxide) or black enamel, or like a black, for the purposes of this application. The matrix polymer (Purror Technologies' pSK2〇〇〇) is filled with a black particle-filled photoresist or the like. The bottom surface of the substrate covers a back redistribution layer ("RDL") and solder balls 276. In an embodiment, the camera array assembly 250 includes a 5x5 imager array. The camera array 25 has a width W of 7.2 mm and a length of 8.6 mm. Each imager in the camera array can have a width of 3 K4 mm. The total height t1 of the optical elements is approximately 1.26 mm and the total height t2 of the camera array assembly is less than 2 mm. Different lens designs may have different heights tl & t2 Optical crosstalk suppression As described above, the camera array assembly 250 is constructed of a plurality of imagers, as shown in Figures 2A and 2B, each And a corresponding optical path or channel to direct light from the scene through the top lens wafer 262, the intermediate spacer 264 'the bottom lens wafer 268, the bottom spacer 27 to 20 201228382 A plurality of photosensitive members of the sensor 24A disposed on the substrate 278 are formed. Light striking any particular imager comes only from the optical path or channel it specifies, which is important for the final image quality. When the light incident on the top of the imager is simultaneously received by the photosensitive member of another imager within the array: it can be considered to have an optical crosstalk. Any crosstalk from, for example, the diffracted optical channels and/or light scattering from the internal components of the camera can smash the 瑕疵 on the = image. In particular, crosstalk between optical channels means that the imager senses the source flux from the imager that is inconsistent with the reconstructed position of the image of the detector and the image location. This leads to imagery and the introduction of overlapping noise that cannot be distinguished from real imagery. Accordingly, all of the optical channels of the camera array should be optically isolated so that light from the lens or optical pass cannot pass from the optical channel to the other optical channel. In the embodiment illustrated in Figure 2C, an opaque spacer 28i or an opaque vertical wall 282 is disposed between each optical channel 284. Although the opaque spacer provides an optical crosstalk suppression stage, the opaque vertical wall system is preferred because the space between the substrate and the associated segments of the substrate itself are opaque. . The optical crosstalk suppression opaque vertical wall can be fabricated using any suitable technique that provides for the introduction of an opaque surface or material between the optical channels 284 of the camera array assembly 286. In an embodiment, the opaque vertical wall is formed by introducing all or a portion of the trench into the lens stack 288 of the camera array assembly 286. Preferably, the grooves that are completely through the stack of lenses are not cut to maintain the mechanical integrity of the camera array assembly. Such trenches may be derived, for example, by cutting into the front or back of the lens 21 201228382 array stack 286 using a wafer cutter (disc/blade), or by any suitable technique such as laser cutting techniques or water jet cutting techniques. people. Once the grooves are formed, they are filled with a light blocking material. Alternatively, the inner walls of the grooves may be coated with a -blocking material' and the remainder of the grooves may be coated with another material that exhibits low shrinkage characteristics. As described above, the light blocking material is optional; for example, a metal material, a metal oxide, black enamel or black particles like a black matrix polymer is filled with a photoresist or the like. In another embodiment, schematically illustrated in Figure 2D, optical crosstalk suppression is achieved by creating a virtual opaque wall formed by a series of stacked apertures. In this embodiment, the string apertures are formed on different substrate levels 290 of the camera array assembly 292 by coating the substrates with an opaque layer 294 provided with a narrow opening or aperture 296. If sufficient of these apertures are formed, the optical isolation provided by an opaque vertical wall can be simulated. In such systems, a vertical wall would be a mathematical limitation of stacking apertures on top of each other. Preferably, a plurality of apertures are provided as much as possible to create such virtual opaque walls in a manner that separates each other from ample space. For any camera array assembly, the number and configuration of opaque layers required to form such a virtual vertical wall can be determined by a ray tracing analysis. In a further embodiment as outlined in Figure 2E, optical crosstalk suppression is achieved using spacers 295 constructed of opaque materials. In the re-consistency example outlined in Figure 2F, the optical crosstalk suppression is obtained using a spacer 296 coated with an opaque coating 297. The embodiment shown in Figures 2E and 2F is shown in Figure 2D. Stack aperture 294 is similar to stacked aperture 294. In some embodiments, the optical crosstalk suppression is obtained without using a stacked aperture. In many embodiments, 22 201228382 any of a variety of light blocking materials can be used in the construction or coating of the spacer for optical isolation. Lens Characteristics Figs. 3A and 3B are diagrams showing a change in the xy plane as a function of the xy plane dimension. The ratio of the lens member 320 of Fig. 3B to the lens configuration of Fig. 3A is l/η. Note that it is important to keep the same aperture ratio during the measurement without changing the system. In the diameter of the lens member [Μ: the height of the lens member 32q is also smaller than the height of the lens member 32q by an factor of n. Therefore, by using a smaller mirror member (four) column, the height of the camera array assembly can be significantly reduced by the reduced resolution of the 6-camera array assembly. The low degree of difficulty can be broken to design with, for example, improvement. A smoothing lens with a dominant ray angle and reduced variability. "The preferred optics of the improved chromatic aberration Figure 3C illustrates the improvement of a main light, = CRA by reducing the thickness of the camera array assembly. The chief ray angle 1 covers the chief ray angle of a single lens of the entire camera array. The chief ray angle can be reduced by the distance between the early lens lenses, but this: : The camera array and the system are large, thus reducing the imaging in the 1st column of the chief ray angle of the optical execution array - it is pressed by two::::: : Design. The chief ray angle 2 is still the same as the main light of the conventional camera array: and the chief ray angle is not improved. However, by the distance between the imager and the lens, as shown in Fig. 3: The hall does not modify the angle 3 compared to the main ray xiao 1 or the main: 主 array component in the main ray first line angle 2 can be reduced, resulting in

23 S 201228382 Best optical performance. As described above, the camera array according to the present invention has a reduced thickness requirement, and therefore, the distance between the lens member and the camera array can be increased to improve the chief ray angle. Next, the reduced chief ray angle produces a lower aperture ratio and an improved modulation transfer function (MTF). In particular, one of the issues raised by camera design is how to correct field curvature. The image projected through a lens is not flat but has an inherently curved surface. One way to correct the curvature of the field is to position a thick negative lens member 312 proximally or directly on the imager surface 314. The negative lens member flattens the various angular beams 316 from the image, thereby coping with the field curvature problem. Such field flat images provide excellent image execution efficiency, can produce array cameras with reduced transistor-transistor logic circuit requirements, and deliver very homogeneous modulation transfer functions. However, one problem with this method is the nature of the field flat method. A high main ray angle is required. This makes this technique unsuitable for most cameras; however, the camera array of the present invention can use back-illuminated imaging technology (BSI). The position of the chief ray angle is eliminated after positioning the image sensor on the substrate, whereby the negative lens member field flatning method shown in Fig. 3D can be used. Another advantage of the s-ray array camera is the chromatic aberration. In particular, in a conventional multicolor lens, since the different wavelengths of light to the focal length of the lens are different, the lens must correct the chromatic aberration. Therefore, it is desirable to coordinate lens execution efficiencies of some of the color wavelengths to achieve acceptable overall color performance. By fabricating a narrow spectral band for each optical channel, the chromatic aberration is reduced and/or prevented, a and each lens can be optimized to a particular color wavelength. For example, an imager that receives a visible or near-infrared spectrum can have a lens member that is optimized for the specific band of the band 201222382. For imagers that detect other spectra, the lens members are constructed to have different characteristics such as radius of curvature, such that a fixed focal length across all wavelengths of light is taken to then make the focal planes of the different spectral bands the same. Pairs of focal planes across different wavelengths of light increase the accuracy of the image captured by the imager and reduce longitudinal chromatic aberration. Since each lens member can be designed to guide a narrow spectral band, the lack of chromatic aberration is accompanied by the less stringent design constraints of the lens members, but the conventional lens components that cover a vast spectrum are Produce better or equivalent enforcement efficiency. In particular, there is no need to perform expensive color difference balance correction. Even simple lenses generally have a better modulation transfer function and a lower aperture ratio (relative sensitivity). It should be noted that although the lenses used in these array cameras have much smaller chromatic aberrations than conventional multicolor lenses, each lens is designed to focus on a certain wavelength bandwidth. Accordingly, in each of the embodiments, each of these "monochrome lenses can be optimally color corrected by using a combination of high and low Abbe number materials (different light 4 color). The light is in the multi-color optics. The refractive index of a lens is visible through the lens. The lens will give different wavelengths. The color wavelength band can have a greener slightly different focal length (longitudinal chromatic aberration). Depending on the wavelength of the light, therefore, the color is different in magnification. For example, the red is small, and then the green color may have a slightly smaller magnification than blue. If the images taken from these different wavelengths of light are then overlapped If the correction is made, the image will lose its resolution because the different colors will not overlap correctly. According to the material characteristics, different lateral deformations of the color magnification can be determined and corrected. The correction can be achieved by limiting the lens contours. So that each of the colors of 5 25 201228382 has the same magnification, but this reduces the maximum degree of freedom that can be used in lens manufacturing, and reduces the best The ability to modulate the transfer function. Accordingly, in a camera array embodiment, the lateral deformation is optically allowed, and then corrected after computational imaging. The lateral color of the lens is automatically provided by the system. The improvement in execution efficiency goes beyond the simple correction of the original deformation, which directly improves the resolution of the system in terms of the multi-tone transfer function. In particular, the lateral chromatic aberration in a lens can be regarded as the color correlation of the lens. Deformation ^ by mapping all differently deformed monochromatic images of an object back to the same rectangle, can be produced with the monochromator (not just because of the individual color channel color blur correction, but also because of the correct overlay of different colors) A perfect overlap in the full-color image of the same multi-tone transfer function. Another advantage of using many lenses that are optimized to fit a narrow spectrum is that no lens type is used. Limitations. In particular, the array camera can be used for diffraction, refraction, Fresnel lenses or a combination of these lens types. Lenses are attractive because they can produce a composite wavefront using a substantially flat optical member, and they are relatively simple to manufacture. In conventional cameras, because of the single imager mechanism and the lens must be able to transmit efficiently When a large spectrum is used, it is not feasible to use a diffractive lens system, and the diffraction lens is very efficient when transmitting a narrow optical wavelength band. The execution efficiency of the optical wavelength outside the optimized range has a steep drop. At present, each array of cameras can be focused on a narrow wavelength of light, so the narrow optimized wavelength band of these diffractive lenses is not a limiting factor. Other advantages of smaller lens components include, inter alia, cost reduction, material volume 26 201228382 The reduction and manufacturing steps are reduced. By providing an n2 lens of size l/η on χ and y dimensions (hence the Ik thickness), the wafer size used to fabricate the lens member can also be reduced. This reduces the considerable cost and quantity of materials. Progressively, the number of lens substrates is reduced, resulting in a reduction in the number of manufacturing steps and a concomitant reduction in production costs. The configuration accuracy required to provide a lens array to such imagers is typically no more urgent than the example of a conventional imager, since the pixel size of the camera array according to the present invention can be substantially the same as that of a conventional image sensor. Therefore. In addition, the monochromatic chromatic aberration depends on the lens diameter. Since the array camera can use a smaller lens, any existing color difference system is small, and thus it is feasible to use a lens system having a simpler profile. This produces a system that is both manufactured at a higher quality and less costly. Smaller sized lenses also have a smaller volume that produces lower sag or shrinkage during manufacturing. Shrinkage is detrimental to the replication system because it deforms the desired lens profile and results in the need for the manufacturer to pre-compensate the predicted sink level in order to correct the final lens profile. This pre-compensation is difficult to control. Lower sag/shrinkage eliminates the need for these rigorous manufacturing controls and reduces the overall manufacturing cost of the lenses. In one example, the wafer level optical process includes (1) integrating the lens member columns onto the substrate by electroplating lens members prior to lens formation, and (u) etching the holes in the substrate. And the two-sided lens forming technology is performed throughout the substrate. Because the index mismatch does not occur between the plastic and the substrate, the hole etching in the substrate is advantageous. In this manner, the natural light absorbing substrate forming all the lens members (similar to the black lens edge) can be used. In an embodiment, the filter is part of the imager. 27 201228382 In another embodiment, the filter is part of a wafer level optical subsystem. In an embodiment comprising a filter, because small defects in those filter layers are evenly distributed over all incident pupil positions when positioned at a distance from the imaging sensor surface, they are less visible, so The filter (whether a color sputum array, an infrared filter, and/or a visible light filter) is placed in or near the surface of the aperture bar without being preferred on the imaging sensor surface. Imaging System and Processing Procedure Figure 4 is a functional block diagram illustrating an imaging system 4A in accordance with an embodiment. The imaging system 400 can include, among other components, the camera array 410, an image processing program module 42 and a controller. The camera array 410 includes the above reference to the figure! Two or more imagers detailed in item 2. Shadow I 412 is captured by two or more imagers in the camera array 41A. The controller 440 is a hard human limb, an ankle hip or a combination thereof, and is recorded in the camera array 41. The controller 44 receives an input 446 from a user or other external component and sends an operation signal 4 to control the camera array 41. The controller 440 also sends information 444 to the image processing program module 42G to assist in the processing of the (4) image 412. The image processing program module 42 is a hardware, a primary body, a software body or a video image received by the camera array 41G. The image is processed by the tnr, for example, as described below with reference to FIG. Progressive processing. ❹ is sent for display, storage, 彳 FIG. 5 illustrates the image processing program module 42 according to an embodiment. 28 201228382 Capability block diagram. The image processing program module 42 may include an upstream program processing module 5, an image pixel correlation module 5丨4, a parallax confirmation and measurement module 5丨8, a parallax compensation module 522, and other components. The super-resolution module 526, the address conversion module 53A, the address and phase shift calibration module 554, and the downstream color processing module 564. The address and phase shift calibration module 554 is a storage device for storing calibration data generated during characterization of the camera array in the process or the next recalibration procedure. In some embodiments, the calibration data may indicate a mapping between the address of the physical pixel 572 in the imager and the logical address 546, 548 of the image. In other embodiments, a variety of calibration data suitable for a particular application can be applied to the address and phase shift calibration module. The address translation module 530 is based on the address and phase shift calibration module 554. The stored calibration data is used to perform normalization. In particular, the address translation mode, and 530 converts the "entities of the individual pixels in the image, the address becomes the "logic," address 548 of the individual pixels of the imager t, or vice versa. For super-resolution processing to produce an image with increased resolution, the phase difference between the corresponding pixels in the individual images 7 needs to be resolved. The super-resolution program σ false for each pixel in the image generated at 5 Hz, the input pixel groups from each imager are systematically mapped, and the phase shift of the image captured by each imager is generated The pixel position of the image t is known. These phase shifts can be estimated before the hyper-resolution program. The address translation module 53G resolves the g 29 201228382 type phase difference for subsequent processing by converting the physical address in the images 412 into the logical bit M 548 in the generated image. The images 412 captured by the imagers 540 are provided to the upstream program processing module 51A. The upstream program processing module 51 can perform one of color plane normalization, black level calculation and adjustment, fixed miscellaneous compensation, optical chat (point spread function) solution cyclotron, noise reduction, lateral chromatic aberration correction, and crosstalk reduction. Or more. In an embodiment, the upstream program processing module also performs temperature normalization and two-temperature normalization to correct refractive index changes of the optical elements. The image states are received through the optical components of the optical device due to temperature changes during use of the camera. The light produced. In some embodiments, the temperature normalization procedure involves determining the camera array temperature by measuring the dark current of one of the imagers of the camera barrier or the dark current of its heart. Using this measurement method, the reflectance normalization is performed by selecting the correct point spread function from the temperature calibration data. Different point spread functions can be taken during the temperature-dependent index characterization of the camera at the time of manufacture and stored in the imaging system for use by the temperature normalization procedure. After the upstream program processing module 51 processes the image, the image image is read and the parallax calculation is performed, which becomes more apparent as the captured object approaches the array. In particular, the image pixel correlation module 51 captures portions of the image captured by different imagers to perform the parallax compensation. ^, in the embodiment, the image pixel correlation module 514 compares the difference between the adjacent pixel count:: value and the critical value, and when the difference exceeds the critical value, the 疋-hai parallax may exist. Flag. The threshold can be dynamically changed as a function of the camera array's stay condition. Further, these neighborhood calculations are also suitable for 2012 28382 and may reflect the specific operating conditions of the selected imager. The image is then processed by the parallax confirmation and measurement module 518 to detect and measure the parallax. In one embodiment, the parallax detection is performed by a running pixel correlation monitor. This operation occurs in a logical pixel space throughout the imager having similar aggregate time conditions. When the scene is in the actual infinite space, the data from the imagers is highly correlated and depends only on noise-based changes. However, when the object system is close enough to the camera, the parallax effect is introduced to change the correlation between the imagers. Because of the spatial layout of the imagers, the parallax-induced changes are inherent in all imagers. Consistent. The difference in correlation between any pair of imagers within the measurement accuracy limit specifies the difference between any other pair of imager and the ubiquity of it. The same or similar calculations are performed on other imager pairs. Highly accurate parallax confirmation and measurement are available. If the parallax exists in other imager frames, the parallax should occur at the same physical position of the scene as the position of the imager considered. In the case of the visual difference, the tracking of the various modes can be maintained and the sample data can be matched. The least squares value (or similar statistical value) to calculate "actual, parallax difference, other methods of H-heterodyne can include detecting and tracking vertical and horizontal high-frequency image components from each side." Group 522 processes an image containing an object that is sufficiently close to the camera array image to cause a difference greater than the difference in super-resolution differences. The phase shift information accuracy required by the program is used in the parallax detection and measurement mode. Before the super-resolution program, the advanced parallax information step based on the scan line adjusts the physical pixel address and the mapping between the logical blocks of the 201228382 series. There are two cases in the case of the super-resolution program. This process occurs. In the more general case, 'when the positions of the input pixels are compared with other imagers, the corresponding image t on the image has been biased. It is necessary to displace the whole week. In this fiscal case, there is no need to process the parallax before performing the super-resolution program. In the less common example, the pixel or pixel group is exposed to the masking group. Class-like offset. In this example, the disparity compensation program generates interlaced pixel data to (d) the pixels of the shadow group should not be considered in the hyper-resolution program. The parallax change in the special imager has been accurately After the decision, the disparity information 524 is sent to the address conversion module 53. The address conversion module 53 uses the disparity information 524 and the quasi-before material 558 from the address and phase shift calibration module 554. Determining the appropriate X Υ Υ offset value applied to the logical pixel address calculation. The HI address conversion module 5 3 〇 simultaneously determines a particular imager pixel relative to the pixel in the image 428 produced by the super-resolution sequence The associated sub-pixel offset. The address translation module 53 considers the disparity information 524 and provides a logical address 546 that describes the disparity. After performing the disparity compensation, the image is used by the super-resolution module $2 6 processing to self-low solution A high resolution composite image 422 is obtained in the image, as detailed below. The composite image 422 is then fed into the downstream color processing group 564 to perform one or more of the following operations: focus recovery 'white balance, color correction , gamma correction, RGB to γυν correction, edge auto-sharpening, contrast enhancement, and suppression. The image processing program module 420 can include components for additionally processing the image. For example, the image processing program module 420 can include For correcting 32 201228382 image anomalies caused by single pixel defects or - pixel defect groups. The correction module can be implemented on the same wafer as the camera array, become a separate component from the camera array, or become the A portion of the super-resolution module. Super-resolution processing In one embodiment, the super-resolution mode group 526 II generates a higher resolution composite image by processing the low-resolution images captured by the imagers. The overall image quality of the /5 percent shirt image is higher than the images captured by any of the imagers. In other words, the individual imagers work in concert, each using our 6 forces to capture the narrow-wave portion of the spectrum to contribute to the quality image without taking 4t into the human-sample. The image formation associated with these super-resolution techniques can be expressed as follows: • especially +, = formula (2) /, medium Wk represents the contribution of the high-resolution scene (χ) (through blur, motion, and human sampling) Each of the low resolution images (10) captured by each of the k imagers, and "" the noise contribution. Imager architecture Figures 6A-6F illustrate the use of super resolution in accordance with an embodiment of the present invention. Various imager architectures for high-resolution images, as shown in Figure 6A to Figure 6F, R represents an imager with a red cross, "G" represents an image with green flakes, and B" represents a blue filter. The imager, "p," represents a multicolor 33 with a sensitivity across m visible and near infrared spectra. The 201228382 imager and "i" represent an imager with a near infrared filter. The multicolor imager can Sampling images from all and near infrared regions of the visible spectrum (ie, from 650 nm to 800 nm) in the embodiment of Figure 6A, the middle rows of the imagers containing the multicolor imager "The rest of the camera array Regional department is full of The imager of the green filter, the blue filter and the red filter. Figure 6 实施 The embodiment does not include any imager that only detects the near-infrared spectrum. Figure 6 is an embodiment with an architecture similar to the traditional Bell filter mapping. Does not include any multi-color imager or near-infrared imager. As detailed above with reference to Figure 1, the Figure 6 embodiment differs from the conventional Bell filter architecture in that each color filter is mapped to each imager. The substitutions are mapped to a different pixel. Figure 6C illustrates an embodiment in which the multi-color imagers form a symmetric checkerboard pattern. Figure 6D illustrates an embodiment providing four near-infrared imagers. Figure 6A illustrates an imager with a non-specification map Embodiments Figure 6F illustrates an embodiment in which a 5χ5 sensor array is organized into 17 imagers with green filters, four imagers with red filters, and four imagers with blue filters. The sensors are symmetrically distributed around the central axis of the imaging array. As further described below, distributing the imagers in this manner prevents the pixels imaged by the sensor from being captured The light wavelength of the sensor is obscured. The embodiment of Figure 6A to Figure 6F is only illustrative, various other imager layouts can also be used. Because these sensors can capture high-quality images with low illumination conditions, multi-color imaging β and The use of a near-infrared imager is advantageous. The image captured by the multi-color imaging or the near-infrared imager is used to remove noise from the image obtained by a typical color imager. However, as described above, this 34 201228382 : The eclipse lens needs to use the associated color correction technique to counteract the inherent lens that is trying to capture the wavelength and transmit it to the same focal plane. Any conventional color correction technique can be applied to the proposed array camera. Layout # The desire to increase resolution by concentrating multiple low-resolution images represents different low-resolution images of slightly different perspectives in the same scene. If the low-resolution images are all shifted by an integer unit of one pixel, each of the images mainly contains the same information. Therefore, there is no new information that can be used to generate a high resolution image in these low resolution images. In a camera array according to an embodiment of the invention, the imager layout in the array can be pre-ported to capture an image of each imager in a column or row, which is adjacent to the image. The image captured by the device is offset by a fixed sub-pixel distance. Ideally, the image captured by each imager is spatially offset from the imager in such a manner as to provide a uniform sampling of the scene or the light field, and the sampling is such that the images are H The low resolution image captured by each of f produces non-redundant information about the sampled scene (light field). Such non-redundant information about the scene can be applied to subsequent signal processing programs to synthesize a single high resolution. image. However, the sub-pixel offset between the images captured by the second imager is not sufficient to ensure sample uniformity. The uniformity of sampling or sampling diversity of the two imagers is the object distance function. The sampling space of the pixels of the image pair is shown in Figure 6G. The first set of rays (61〇) maps to the pixels of imager a, while the second set of rays (620) maps to the pixels of imager B. Concept Above, two adjacent rays from an imager are defined in the object space by one of the imagers

S 35 201228382 Part of the sample taken by a particular pixel. At a distance z1 from the camera plane, there is sufficient sampling diversity because the light of the pixels of imager A is spatially offset from the light of the pixels of imager B. There are some specific distances (22, 23, 24) as the distance decreases, with no sample diversity between imager eight and imager B. The lack of sampling diversity between the two imagers is quite simple to imply that there is no additional information in the scene captured by Imager B compared to the scene captured by Imager B. As further described below, an increase in the number of imagers in an array camera mitigates the effect of the imager's sample space on the distance of completely overlapping objects. When an imager pair lacks sampling diversity, other imagers in the array provide the required sampling diversity to achieve increased resolution. Thus, the ability of an imager system to utilize a 2X2 imager array to achieve super-resolution is typically more limited than a camera system that uses a larger camera array in accordance with embodiments of the present invention. Referring back to the camera array structure illustrated in Figures 2A-2D, the wafer level optical instrument includes a plurality of lens members, wherein each lens member encompasses one of the sensors of the array. The pixel physical layout in a single imager of a camera array in accordance with an embodiment of the present invention is shown in Figure 6H. The imager is covered by an array of pixels 650 of color filter 652 and microlens 654. Microlenses positioned on top of the color filters are used to focus the light on the active area of each of the pixels below. The microlenses can be thought of as sampling a continuous light field within the object space taken by the main lens. In view of the fact that the main lens samples the scene illuminating the light field, the microlenses sample the sensor illuminating light field. The main lens associated with each imager maps points of the object space to points in the image space such that the mapping is a mapping function (a one-to-one function and a function). Each microlens samples a limited range of the irradiated light field of the sensor. The sensor illuminates the light field continuously and maps the results from the object space to a mapping function. Thus, a limited range of microlens sampling of the radiant field of the 5 sensible sensor is also a corresponding limited range of sampling of the scene radiant field within the object space. Moving the microlens approximately δ along the imager pixel plains can change the sampling object space of a certain distance 约 by a corresponding appropriate factor δ. Using an ηχη(η>2) array camera, we can choose a bottom line microlens offset that can be determined by the main lens profile of a bottom line imager (e.g., 'the chief ray angle'). For each of the other imagers that sample the same wavelength as the bottom line imager, the microlens of each pixel in the imager is offset by a single pixel amount to sample different portions of the scene radiation field. Therefore, for an imager group arranged in a ηχη lattice, the sub-pixel offset of the imager for imaging at the same wavelength as the bottom line imager (1, 丨) at the lattice position η) is subjected to ( χ χ, ^) is dominated, where (, -1) η (7-1) η pixel size < b 士 ><pixel size pixel size pixel size Many camera arrays according to embodiments of the present invention significantly contain More green imagers for red and blue imagers. For example, the array camera shown in Figure 6f includes a solid green imager, four red imagers, and four blue images. The green imager can be viewed as a leg-leg grid. However, based on the calculation of the sub-pixel shifting purposes, the red imagers and the blue imagers can be regarded as -2χ2 grids.

The shifting system is evenly distributed to obtain the largest sampling diversity. Limitations on the microlens sub-pixel offset defined above in terms of diversity

The biggest increase is achieved and the maximum increase can be handled by super resolution. Although it is not enough to be in the eye, the h phase h - the team is now rotten, and in many cases a variety of different microlens offset architectures are used to provide at least some increase in sampling diversity and satisfy one. The needs of a particular application. The imager configuration symmetry in the camera array. . The problem of dividing the photosensitive members into different imagers is the parallax caused by the isolation of the image-trapped bodies. By (4) the imagers are symmetrically placed with at least one imager to capture pixels around the edge of a foreground object. In this manner, pixels around the edge of a foreground object can be concentrated to increase resolution and avoid any shadowing. In the absence of a symmetrical distribution, a pixel visible by a first imager, such as a red camera, around a periphery of a foreground object, for a person such as a blue imager that captures a second imager of a different wavelength, accordingly, The color information of this pixel cannot be accurately weighted by 201228382. By symmetrically distributing the sensors, the likelihood that a foreground object will obscure the pixels is significantly reduced. The pixel masking caused by the asymmetric distribution of the red and blue imagers in a simple array is shown in FIG. A pair of red imagers 672 are tied to the left hand side of the camera array 670 and a pair of blue imagers 674 are tied to the right hand side of the camera array. A foreground object 676 is present and the red imagers 672 are capable of imaging an area of the foreground object on the left hand side of the foreground object. However, the foreground object obscures the red imagers and cannot image these areas. Therefore, the array camera cannot reconstruct the color information of these areas. An array comprising a symmetric distribution of red and blue imagers in accordance with an embodiment of the invention is shown in Figure 6J. The camera array 780 includes a pair of red imagers 782 symmetrically distributed about the central axis of the camera array and a pair of blue imagers 784 symmetrically distributed about the central axis of the camera array. Because of the uniform distribution, both a red imager and a blue imager can image foreground objects 786 on the left hand side of the foreground object, and both a red imager and a blue imager can be imaged more than The foreground object on the right hand side of the foreground object. The symmetrical configuration of the simple embodiment shown in Figure 6J can be generalized to an array camera comprising a red, green, blue camera and/or an additional multi-color or near-infrared camera. The parallax effect introduced by the foreground object by symmetrically distributing each of the different types of imagers around the central axis of the camera array can be significantly reduced' and the color ridges introduced in other respects are avoided. The parallax effect on color sampling can also be achieved by using a multi-color imager

S 39 201228382 Parallax information is used to improve the color sampling accuracy from this color filter imager. Using a near infrared ray imager to obtain an improved high resolution image In one embodiment, a near infrared ray imager is used to determine the relative brightness difference compared to the visible spectrum imager. The ability of an object to have different material reflections results in differences in the images captured by the S-visible spectrum and the near-infrared spectrum. The near-infrared imager exhibits a higher signal-to-noise ratio under low lighting conditions. Therefore, a signal from the near-infrared sensor can be used to enhance the luminance image. The transfer of the detail from the near-infrared image to the luminance image can be performed before the spectral image of the different imagers is concentrated by the super-resolution program. In this mode, the edge information about the scene can be improved to construct effectively. The image is used to maintain the image at the edge of the super-resolution program. The advantage of using a near-infrared imager is evident in equation (2), where any improvement in the evaluation of the noise (i.e., n) results in a better assessment of the original high-resolution scene (X). High Resolution Image Generation Figure 7 is a flow diagram illustrating a method for generating a high resolution image from a low resolution image captured by a plurality of imagers, in accordance with an embodiment. First, the luminance image 'near-infrared image and chrominance image are captured by the imager in the camera array. The normalization of the captured images is then performed in step 714. The images may be normalized in a variety of ways, including normalizing the color planes of the images, performing temperature compensation, and mapping physical addresses of the images to logical addresses of the enhanced image, but are not limited thereto. In other embodiments, a variety of normalization procedures are available for these particular imagers and imaging 201228382 applications. The disparity compensation is then performed at step 720 to account for any differences in the field of view of the imager due to spatial isolation between the inter-imagers. The super-resolution processing is then performed in step 724 to obtain a super-resolution luminance image, a super-resolution near-infrared image, and a super-resolution chrominance image. Next step 728 determines if the lighting condition is superior to a predetermined parameter. If the illumination condition is superior to the parameter' then the method continues to normalize the super-resolution near-infrared image associated with a super-resolution luminance image. A focus recovery is then performed in step 742. In one embodiment, the focus restoration performed in step 742 uses P S F (point spread function) to remove blurring on each color channel. Next, the super-resolution is processed in step 746 based on the near-infrared image and the brightness images. A composite image is then constructed in step 705. If step 728 determines that the illumination condition is not superior to the preset parameter, then the super-resolution near-infrared image and the luminance image are aligned to step 734. Then, the super-resolution luminance images are used in step 738 to remove the noise using the near-outline super-resolution images. The method continues to perform the focus restoration step 742 and repeats when the illumination condition is superior to the preset parameters. The same steps. Color plane normalization The relative response of each of the red, green, and blue imagers throughout the imaging plane is not @. The variation can be the result of the optical alignment of the lens and the optical path of the asymmetric sensor and the multiple factors of the six noon. For the lens and imager, the 丄 丄 丄 , , , , , , , , , , , , , , , , 可 可 可 可 可 可 校准 校准 校准 校准 校准 校准 校准 校准 校准 校准 校准 校准丨4 The difference a causes, for example, a darkened color. A method for normalizing an imager associated with a bottom line imager of a green imager typically located at the center of the camera 201228382 array is referenced to a red associated with a bottom line green imager in accordance with the present invention. The imager is normalized to illustrate the following. A similar approach can be used to normalize the blue imager associated with the bottom line green imager. In many embodiments, the method is applied to each of the red and blue imagers within the normalized camera array. The normalized surface can be calibrated by first capturing a scene with the same reflection coefficient and calculating a color ratio surface to serve as a normalization reference. The ideal normalized surface is uniform and can be described as: Color ratio G/R=G(i,j)/R(i,j) = K=Gcenter/Rcenter where (i,j) describes the pixel position, K A constant is specified, and Gcen..., Reenter describes the pixel value at the center position. The output pixel value of the calibration scene contains the ideal pixel values plus noise plus black level offset, and can be described as follows: SR(i'j)=R(i,j) +no noise R(i , j) + black level offset SG(i, j) = G(i, j) + noise G(i, j) + black level offset where SR and SG are the output pixel values of each imager. A method for calibrating the sensor in accordance with the present invention is shown in FIG. 7A, and includes removing (step 762) the black-order offset from the sensor pixel values and low-passing _ ( Step 764) the image planes to reduce noise. The normalization plane is calculated (step brown), and some embodiments are calculated as 42 201228382. The normalization is R=G(i,j)/(R(i,j)x(Gcenter/Rcenter)) 〆, In 'Gcenter and Rcenter are the pixel values at the center position. The normalization plane is then calculated, an average filter can be applied (step 768), and the value of the normalized R plane can be stored (77 〇). The cost of carrying all of the normalized data for each sensor in a sensor array can be quite high. Thus, many embodiments use spatial fill curves to scan the δ-hai normalized R-plane to form a one-dimensional array. The resulting-dimensional array can be modeled in a variety of different ways, including building a polynomial model with the appropriate hierarchy. In some embodiments, a plurality of < of the appropriate polynomials are stored (810) as parameters to reconstruct the two-dimensional normalization plane during use during calibration. Spatial fill curve construction in accordance with some embodiments of the present invention is further described below. In some embodiments, a space fill curve is used to form a -dimensional array that describes a normalized plane. The space filling curve constructed by the ❹ spiral sweep is shown in Fig. 7B. The space fill curve can be started from the center of the normalization plane 78ι and traverses the four squares outward. Each side of the square is enlarged by one pixel compared to the square, such that each pixel is correctly crossed. In each of the illustrated embodiments, each location 782 labeled "X" corresponds to an effective pixel location. The imager may not have a square geometry such that the broom path may traverse unoccupied space (as indicated by the dashed lines). For each position traversed and m is the effective pixel position, a new data item is added to the one

S 43 201228382 Dimensional data array. Otherwise, the traversal action continues without adding new values to the data array. In many embodiments, the one-dimensional data array can use a sixth-order polynomial to effectively estimate 'seven coefficients that can be used to represent the polynomial. Each red and blue imager typically needs to give that calibration data, and representing these normalized planes as polynomial coefficients represents a significant reduction in storage requirements. In many embodiments, higher or lower order polynomials, their 匕 functions, and/or other compressed representations are utilized to represent the normalization plane as needed for a particular application. A fixed geometric relationship is presented along the data values on each side. The optical path to the focus of the lens is short for cells near the centerline. This sensitivity can be considered as a one-dimensional center in the calibration surface and is estimated by the low-level formula. The sensitivity polynomial is not stored as a machine constant or is common to all devices having the same design, and is stored along with the sweeping polynomial to provide additional flexibility. Accordingly, many of the present invention adjust the pixel value in accordance with the distance factor as described below. For the = side sweep, the ones in the coordinates will be constant, that is, the constant "y" is the horizontal scan and the constant "x" is the vertical sweep. For each pixel in the side scan, the sensitivity factor is adjusted towards the constant "X" or "y" distance, for example, for a horizontal broom, the base value can be based on the center The distance is evaluated by evaluating the sensitivity polynomial ^ In many implementations ^ Polynomial is a fourth-order polynomial. However, other polynomials and/or their 匕 functions can be applied according to the needs of the particular application. For each pixel in the sweeping cat path, the distance from the origin of the surface is used in the same way as 201228382 to determine the corresponding sensitivity from the equation. The pixel value is multiplied by the adjustment factor and then stored in the scan data array. This adjustment factor is calculated by dividing the sensitivity value of the target by the basic value. For this vertical scan, a similar method can be applied. Although the present example uses polynomial-based sensitivity σ, other sensitivity functions and/or adjustments may be utilized in accordance with various embodiments of the present invention depending on the needs of the particular application. Once the calibration data for the imager is obtained, the calibration data can be used in the normalization of pixel information captured by the imager. The method typically involves taking the stored calibration data, removing the black level offset from the captured image, and multiplying the resulting value having the normalized plane. When the normalized plane is represented as a polynomial in the manner outlined above, the Xiao polynomial is used to generate a one-dimensional array, and a reverse scan of the one-dimensional array is used to form the two-dimensional normalized plane. Where the sensitivity adjustment is applied during calibration, the 5-week factor is counted as the reciprocal of the applied adjustment factor during the calibration (iv) and the adjustment factor is applied to # within the array during the reverse broom. When other spatial fill curves, representations and/or sensitivity adjustments of the resulting one-dimensional data array are performed on the calibration program, the normalization procedure is adjusted accordingly. As can be easily understood, the program can be applied to the camera one. In many embodiments, the device is used in performing image calibration and/or multiple green imaging and blue imager calibrations. Green imaging of each of the red and blue imagers of the calibration and regular arrays in accordance with embodiments of the present invention at the center of the camera array is not beneficial in other embodiments, and can be applied to the red color of the camera array 45 201228382 color Image fusion with near-infrared image The spectral response of a complementary metal oxide semiconductor imager is typically very good in the near-infrared region covering 650 nm to 800 nm and is between 800 nm and 丨000 nm. Quite good. Because near-infrared imagers are relatively free of noise, these near-infrared imagers have no chromaticity constraints, but the information in this spectral region is useful in low lighting conditions. Thus, some near-infrared images can be used to remove color image noise in the low illumination conditions. In one embodiment, the image from the near infrared imager is fused to another image from the visible light imager. Before the fusion, the registration is performed between the near-infrared image and the visible image to resolve the difference in viewing angle. The login procedure can be performed in an offline one-time processing step. After the registration is performed, the brightness information of the near-infrared image is interpolated into the grid points corresponding to each of the grid points on the visible light image. After the pixel correlation between the near-infrared image and the visible light image is established, the noise removal and detail transfer procedure can be performed. The noise removal program allows signal information to be transferred from the near infrared image to the visible light image to improve the overall signal to noise ratio of the fused image. The detail transfer ensures that the near infrared image and the edges of the visible image are preserved and emphasized to improve the overall visibility of the object in the blended image. In one embodiment, the near-infrared flash can act as a near-infrared source during the capture of an image by the near-infrared imager. The use of the near-infrared line flash is advantageous, among other reasons, because (1) prevents poor illumination of objects of interest, (ii) preserves the background color of the object, and 46 201228382 (iii) prevents red-eye effects. In an embodiment, visible light passages that only allow near infrared rays to pass are used to further optimize optical instruments for near infrared imaging. Since the optical filter and the sheet produce a more precise detail in the near-infrared image, the visible optical filter improves the transfer function of the near-infrared optical instrument. Then, the details can be transferred to the visible light images using a pair of two-sided sheets, as in the example

Female is described by Eric P. Bennett et al. in the Computer Graphics Society (ACM Bulletin) (July 25, 2G, 6th), "Multi-spectral video fusion (Shi (1) 〒. (4) Vldeo Fus -"), which is here The whole - and integrated reference. The dynamic range auto-exposure (AE) algorithm is determined by the different exposure of the imager to obtain the appropriate exposure (4) of the scene to be captured. The design of the automatic exposure algorithm affects the dynamic range of the captured image. The automatic exposure algorithm determines that the captured image "the camera array j photosensitive region (4) light value. Because a good signal-to-noise ratio is obtained in the linear region, the linear region is preferred. If the exposure is too small, the image becomes insufficiently full, and if the exposure is too much, the image becomes excessively full. In a conventional camera, the repeating program is executed to measure the brightness of the image and the previously defined brightness. The difference between the two decreases below the critical value. This repetitive procedure requires a lot of time to converge and sometimes produces an unacceptable shutter delay. > In one embodiment, by a plurality of imagers The image intensity of the captured image is measured separately. In particular, a plurality of imagers are set to different exposure images to reduce the time to calculate the appropriate exposure. For example, in a camera array having a 5x5 imager with g 47 201228382 8 brightness imagers and 9 near-infrared imagers are provided, each of the imagers being set to have different exposures. The near-infrared imagers are used to capture the low-light appearance of the scene' and These brightness imagers are used to capture the high illumination of the scene. This produces a total of 17 possible exposures. If the exposure of each imager is offset from an adjacent imager by, for example, 2 factors, the maximum dynamic range that can be captured It is 217 or 102 decibels. The maximum dynamic range is much larger than the typical 48 decibels available in conventional cameras with 8-bit image output. At each instant, the response from each of the multiple imagers (underexposed, overexposed, or optimal exposure) is based on how much exposure is needed in the next instant. Compared to an example where only exposure is tested The ability to simultaneously ask for multiple exposures within the possible range of light accelerates the search. By reducing the processing time that determines the appropriate exposure, the shutter delay and

In an embodiment, by combining the imager response to each exposure

The high dynamic range shadow is logged in to account for the difference in viewing angles of the imagers.

'At least—the imager contains high dynamic range pixels to capture high dynamic range with a high dynamic range pixel system. The high dynamic range pixels are compared to other pixel resolutions. But compared to near infrared imagers, high dynamic months Poor execution efficiency is shown under conditions. To improve poor execution efficiency, signals from these near-infrared imaging 48 201228382 can be used in conjunction with images from the high dynamic range imager to achieve better quality images under different lighting conditions. In one embodiment, a high dynamic range image is obtained by processing images captured by a plurality of imagers, as for example by Paul Debevec et al. in the Computer Graphics Society (ACM SIGGRAPH Gazette) (August 16, 1997) Recover high dynamic range radiation map from photos (Recovering High

Dynamic Range Radiance Maps from Photographs)) "disclosed here in the article", all of which are incorporated herein by reference. Because the artifacts caused by the movement of objects in this scene can be reduced or eliminated, the imager is simultaneously used. The ability to capture multiple exposures is advantageous. Hyperspectral Imaging of Multiple Imagers In one embodiment, multispectral images are generated by multiple imagers to assist in segmentation or recognition of objects in the scene because of these spectral reflections. The coefficients change smoothly in most real world objects, so the spectral reflectance coefficients can be captured in multiple spectral ranges using imagers with different color filters and analyzed using principal component analysis (PCA). Capturing the image is estimated. In one embodiment, half of the imager of the camera array is dedicated to sampling of the fundamental spectral range (red, green, and blue), and another of the imagers Half is dedicated to the sampling of the offset basic spectral range (R, G, and B,). The offset fundamental spectral range is offset from the fundamental spectral range by a certain Long (eg, 10 nm). In an embodiment, pixel correlation and nonlinear interpolation are performed to account for the sub-pixel offset field of view of the scene. Next, the spectral reflectance of the scene is 5 49 201228382. Group positive parent light 4 base function to synthesize, as for example s

Parkkinen, J. Hallikainen and T. Jaashelainen at "j. 0pt s〇c

Am., A 6: 318 (August 1989) "The characteristic spectrum of the Munsell chromaticity, as disclosed in the article, is hereby incorporated by reference in its entirety. The basis functions are feature vectors derived from principal component analysis of the correlation matrix and the correlation matrix is composed of Munsell chromaticity wafers storing, for example, spectral distributions representing a wide range of real world materials (total i 257) The spectrum of the measured spectral reflectance of 3: derived from the database to reconstruct each point in the scene. In the first scan, the scene is captured by different imagers in the camera array. The different spectral images show a higher spectral size sample in exchange for resolution. However, some of this loss resolution can be recovered. The plurality of imagers sample scenes of different spectral ranges, wherein each sample grid of each imager is offset by one pixel offset from others. In one embodiment, there are no two sample grid overlaps in the imager. That is, the stacking of all of the sample grids from all of the imagers constitutes a dense and possibly uneven point puzzle. Scattered data interpolation can be used to determine the spectral density at each sample point in each non-uniform puzzle of each spectral image, as for example, SPIA of Newport Beach, California (February 1996) by Xiaofan Fang et al. In the Spie Inti Symposium, in the 1996 SPIE 2710 Bulletin, "Medical Imaging" on pages 404-415, "Formation Deformation for Landmark-Based Stereoscopic Image Deformation" The method described in the article (Volume Morphing Methods for Landmark Based 3D Image Deformation) is hereby incorporated by reference in its entirety. In this mode, a certain analytical metric lost during the sampling of the scene using different spectral filters can be recovered. As described above, image segmentation and object recognition are assisted by determining the spectral reflectance of the object. This situation typically occurs in security applications where a camera network is used to track an object as it moves from one camera operating area to another. Each zone can have its own unique lighting conditions (fluorescent, incandescent, D65, etc.) to cause the object to have a different color appearance in each image captured by a different camera. If these cameras capture the images in hyperspectral mode, all images can be converted to the same source to enhance the efficiency of object recognition. In an embodiment, a camera array having a plurality of imagers is used to provide medical diagnostic images. Because doctors and medical staff have a high level of confidence in the diagnosis produced, the full-spectrum digital image of the sample contributes to accurate diagnosis. Images in these camera arrays can be equipped with color filters to provide full spectrum data. Such camera arrays can be installed on mobile phones to capture and transmit diagnostic information to, for example, by Andres W Martinez et al., Analytical Chemistry (American Chemical Society) (April 11, 2008). Simple telemedicine in the area: camera phones for immediate, off-site diagnosis and report-based k-fluid devices (Simple Telemedicine for Developing Regions:

Camera Phones and Paper-Based Microfluidic Devices for Real-Time, Off-Site Diagnosis) is described in the article, and is hereby incorporated by reference in its entirety. Further, a camera array comprising multiple imagers can provide images with large depth of field to enhance the reliability of image capture of wounds, rashes and other symptoms. 51 201228382 In one embodiment, a small imager with a narrow spectral bandpass filter (including, for example, 20-500 pixels) is used to produce features of the background and local light sources in the scene. By using this small imager, the exposure and white balance features allow for more precise decisions at faster speeds. The spectral bandpass filter can be a generally colored filter or diffractive member' having a bandpass width sufficient to allow the number of camera arrays to encompass a visible spectrum of about 400 nm. These imagers can operate at very high face rates and obtain data (which may or may not be used in its image) to be processed to control exposure and white balance information for other larger imagers in the same camera array. Optical Zoom Configured Using Multiple Imagers In one embodiment, the subset of imagers in the camera array includes a telephoto lens. The HM imager subset can have the same other imaging features as an imager with a non-telephoto lens. Images from a subset of the present imager are combined and subjected to super-resolution processing to form a super-resolution long-range image. In another embodiment, the camera array includes two or more subsets of imagers equipped with lenses greater than two magnifications to provide different zoom magnifications. The camera array embodiments can focus the image through super-resolution to obtain its final resolution. Select a 5x5 imager example with 3x optical zoom feature. 'If 17 imagers are used to sample the brightness (G) and 8 images are used to sample the chrominance (R and B), then 17 brightness imaging The device allows four times higher resolution than any of the 17 imager groups. If the number of imagers is increased from 5x5 to 6x6, then 11 additional imagers are added. Compared to the 8 megapixel conventional image sensor with 3x zoom lens, 8 of the additional imagers are 52 201228382 Z brightness (9) and the remaining 3 imagers are dedicated to color Degree (R and b) ^ 3: The conventional image sensing % resolution is obtained when sampling near infrared rays under zoom. This considerably reduces the ratio of the chroma samples (or near-infrared, =-sampled brightness samples. The reduced chroma-to-brightness sampling ratio uses a 3x zoom super-resolution brightness image as the previous chroma (and Near-infrared) image recognition for slightly compensated for higher acne. Re-sampling the chrominance image at resolution and using a 6x6 imager's resolution equivalent to traditional image sensor resolution with 1x zoom Acquired. With 3x zoom, the imager can get about 6〇% resolution and τ phase 专 with traditional image sensor with 3x zoom for traditional image sensing with 3x zoom resolution. The resolution of the 3x zoom brightness is reduced. However, the reduced brightness resolution is compensated for the fact that the optical instrument of the conventional image sensor due to crosstalk and optical chromatic aberration is reduced in efficiency at 3x zoom. The zoom operation achieved by the imager has the following advantages. First, because some of the lens members can be tailored to suit each focal length, the resulting zoom quality is considerably higher than that. Traditional Like the quality achieved by the sensor. In the same image sensor wiper, the optical chromatic aberration and field curvature across the entire operating range of the lens must be corrected, which is only used in zoom lenses with moving components. The fixed focal length chromatic aberration needs to be considerably more difficult to be fixed by the fixed lens member. Furthermore, the fixed lens in the imagers has a fixed chief ray angle that gives a height, which is not a conventional image sensing with a moving zoom lens. An example of the second 'the imager allows the zoom lens to be simulated without significantly increasing the light 53 201228382 thousand track trajectory. 3 Hai lowered height configurable thin module, even for camera arrays with zoom force According to the second embodiment, the series of recurring expenses required to support an optical zoom level in the camera array is shown in Table 2 ^ Table 2 - Chroma Imager with different zoom levels for different brightness levels of different zoom stages Quantity Quantity - 1x 2x3x1x2x3x17 0 0 8 0 〇16 0 8 Lj__ 0 4 Number of imagers in the camera array 25 b. 36 in a real In the embodiment, the pixels in the images are mapped onto the output image having the size and resolution corresponding to the desired amount of f, to provide a smooth zoom capability from the widest field of view to the maximum magnification field of view. The higher magnification lenses have the same field of view as the lower magnification lenses, and the available image information allows the central region of the image to have a higher effective resolution than the outer region. In three or more different magnifications The nested regions of different resolutions may be provided as the resolution increases toward the center. The image with the largest telephoto effect has a resolution determined by the super-resolution capability of the imager equipped with the telephoto lenses. The image of the widest view can be formatted in at least one of the following two ways. First, the wide-angle view image can be formatted into an image with a uniform resolution, where the resolution is determined by the super-resolution energy of the imager group with a wider-angle lens. The first-'6 Hz wide-angle view image is formatted into a higher resolution 54 201228382 image, where 'the resolution of the center portion of the image is determined by the super-resolution capability of the imager set equipped with telescopes. In the lower resolution sound regions, 'the reduction in the number of pixels per image area is flat: interpolated more, the amount "digits," everywhere. In this type of image, the pixel information

It can be processed and interpolated so that the 5 handles from the age increase the transition from the lower to the lower resolution region. In a practical example, the zoom is induced into some or all of the array of lenses by a barrel-like deformation such that the number of unbalanced pixels is dedicated to the central portion of each image. In this embodiment, each image must be processed to remove the barrel deformation. In the case of producing a wide-angle image, the pixel adjacent to the center is subsampled with respect to the external pixel system oversampling. When the zoom is applied, the pixels around the imagers are gradually discarded and the pixel samples closer to the center of the imager are added. In the embodiment, the texture hard shot filter is created to allow the image to be pressed. The zoom scale between the specific zoom range of the optical component (eg, 1 侪 and 3x zoom scale of the camera array) is expressed. The texture hard-frame is accompanied by a pre-computed optimal image set of the bottom line image. The image group with the 3x zoom and light can be generated by a 3x bottom line size down to 丨. Each image in this group is the version of the bottom line 3x zoom image at a reduced detail level. Use this texture hard shot to represent—the image that you want to zoom in is the image that is zoomed in by i and the scene coverage for that desired zoom level is calculated (that is, those pixels in the bottom line image need to be required Scale performance to produce the output image), (ii) for each pixel in the coverage group, determine whether the pixel is at the 3x zoom brightness 55 201228382 == image, (1Η) if the pixel The 3x zoom brightness slider can be selected - the closest texture hard shot image and interpolated (using a flat film) from the corresponding pixels of the two texture hard shot image to produce (4) if the pixel cannot be "3 times In the zoom brightness image, the scale = the pixel of the bottom line 丨 chromaticity image is enlarged and enlarged to the desired 4 output pixel. By using the texture hard shot, the smooth 1 zoom can give the discontinuous series in two ( That is

Simulate at any point between zooms). In the U 3L, the zoom system is achieved by electronically switching between different optical channels having different sensations but having a fixed effective focal length (EFL) to achieve different fields of view (F 〇 V). In such an embodiment as schematically illustrated in Figure 8a, the variable field of view is obtained by creating an optical channel on the same substrate having the same imaging s... with the effective focal length mT. With this configuration, it is possible to generate an arbitrary zoom magnification value by including an image sensor having a larger or smaller number of pixels. The technology can be integrated into a wafer level optical array camera when these zoom sensor arrays are directly fabricated on the bottom camera array substrate without any further modification to the design of the array camera assembly itself. It is especially simple. In another embodiment, illustrated in Figure 8B, different fields of view are obtained by constructing different effective focal lengths 805 into a particular optical channel of the camera array 8〇6 and maintaining a fixed imager size of 8G8. Different effective focal lengths are disposed on the same substrate stack, that is, the substrate stack having a fixed thickness and space is more complicated due to the main plane and its entrance pupil and the aperture associated with the image sensor 814. The distance of column 81〇 needs to be changed to change the focal length of the optical channel by 56 201228382. In the current embodiment, this can be accomplished by introducing "virtual substrates" 816, 818, and 820 into the stack 806, such that each zoom channel 822, 824, and 826 has a different surface disposed on a different substrate or substrate. The associated apertures are 8 2 8, 8 3 0 and 8 3 2 so that different effective focal lengths can be obtained. As shown, the lenses (834, 836, and 838) and the aperture stops (828, 830, and 832) are distributed and positioned on the particular substrate or substrate surface at exactly the desired effective focal length, at all In the example, the substrate thickness and distance are still fixed. Alternatively, in such embodiments, each of the substrates may be equipped with a lens, but distributed differently to allow for different effective focal lengths. Such structures can have higher image quality but higher cost. In still another embodiment illustrated in FIG. 8C, different fields of view may also be created in a similar manner as shown in FIG. 8B except that there are exceptions to the lens members on all of the substrates within each optical channel, using a "virtual" substrate to establish a different The effective focal length is 805. However, these lens members have different regulations to provide different effective focal lengths. Accordingly, any of a wide variety of optical instrument and sensor sizes and/or photosensitive member sizes can be utilized with array cameras in accordance with embodiments of the present invention to achieve different effective focal lengths. Capturing Video Images In one embodiment, the camera array produces a high picture sequence. The imager in the 2-machine array can operate independently to capture images. Compared to the wire 2 sensor, the camera array can be used up to N times (where the image captures the image at a phantom speed. Further, the circumference of each imager can be heavy: a: to improve the low Operation under light conditions. To increase the resolution 57 i 201228382 degrees, the imager subset can be operated in a synchronous manner to produce a higher resolution image. In this example, the maximum face rate is operated in a synchronous manner. The number of imagers is reduced. The high-speed video rate allows slow-motion video to be played at normal video rates. In one example, a 'two-brightness imager (green imager or near-infrared imager), two blue imagers and Two green imagers are used to obtain high resolution l〇8〇p images. Use a four-brightness imager (two green imagers and two near-infrared imagers or three green imagers and one near-infrared imager) to connect the same blue An arrangement of an imager and a red imager that can be sampled to obtain 12 frames per second for 1〇8〇p video. For higher face rate imaging devices In other words, the face rate value can be linearly increased. For standard resolution (480p) operation, a facet rate of 240昼/sec can be obtained using the same camera array. With a high resolution image sensor (for example, A conventional imaging device of 8 megapixels uses a drop method or a skip method to capture lower resolution images (eg, 1〇80ρ30, 720p30, and 480p30). In the put method, the columns in the captured images And the row is interpolated to the charge, voltage or pixel domain to obtain the 5th target video resolution and reduce the noise. In the skip method, the columns and rows are skipped 'to reduce the sensor's Power consumption. These two techniques produce a result of image quality degradation. In one embodiment, the imagers in the camera arrays are selectively activated to capture a video image. For example, nine imagers (including a near Infrared imager can be used to get 1〇8〇ρ (192〇χ1〇8〇 pixels) image, and 6 imagers (including a near-infrared imager) can be used to get 58 201228382 720p (1280x720 pixels) Image or 4 imagers ( A near-infrared imager can be used to obtain a 480p (720x480 pixels) image. Because of the precise one-to-one pixel relationship between the imager and the target video images, the resulting resolution is higher than conventional methods. Further, since only a subset of the imager is activated to capture the images, significant power savings can be achieved. For example, a power consumption of 1080p can achieve a reduction of 6〇% and a power consumption of 48〇p can achieve 80%. The reduction from the near-infrared imager can be used to remove the noise of each of the video images, so it is advantageous to use the near-infrared imager to capture the video image. In this manner, the camera array of the embodiment Shows excellent low light sensitivity and can be operated in extremely low light conditions. In one embodiment, super-resolution processing is performed on images from a plurality of imagers to obtain a higher resolution video image. The noise reduction feature of the super-resolution program, along with image fusion from the near-infrared imager, produces very low noise images. In one embodiment, high dynamic range (HDR) video capture is achieved by activating more imagers. For example, in a 5X5 camera array operating in the 1〇8〇p video capture mode, only 9 cameras are active. The 16 cameras can be overexposed or underexposed by two or four sets of blocks to achieve a very high dynamic range of video output. Other Applications of Multiple Imagers In various embodiments, the plurality of imagers are used to estimate the distance to an object in the scene. The distance information of each of the points in the image can be used for the camera array. The size of the image component can be used in the range of the X and y coordinates of the image member. Gu, the decree, & Step-by-step, the absolute rule of the physical project 59 201228382 Inch and shape can be measured without other reference information. For example, a foot image can be taken and the generated information can be used to accurately estimate the size of a suitable shoe. In one embodiment, the depth of field reduction uses distance information to be modeled in the image captured by the camera array. The camera array according to the present invention produces an image with a large increase in the hall and ice. However, in some applications it may not be possible to expect a long depth of field. In such an example, a certain distance or some distance may be selected as the "best focus, 'distance, and based on the distance (Ζ) information from the disparity information' can be made using, for example, simple Gaussian blurring techniques The pixels are blurred one by one. In one embodiment, the depth map taken from the camera array is utilized such that a: hue mapping algorithm uses the depth information to perform the mapping to direct the level, thereby emphasizing or Exaggerating the stereoscopic effect. In a consistent embodiment, apertures of different sizes are provided to obtain aperture diversity. The aperture size has a direct relationship with the depth of field. However, in a compact camera, the aperture control is generally As large as possible to allow more light to reach the camera array. Different imagers can receive light through different sized apertures. For an imager that produces a large depth of field, the aperture can be reduced' however other imaging Can have a large aperture to maximize the received light. By blending images from sensor images of different aperture sizes, a large depth of field can be obtained Image without sacrificing the image quality. In one embodiment, the camera array in accordance with the present invention refocuses based on images captured from a viewing angle shift. Unlike a conventional all-optical camera, images from the camera array of the present invention do not. Suffering from a large loss of resolution. However, compared to the all-optical camera, the camera array according to the present invention generates a rare amount of resources for refocusing. To overcome these rare data points, interpolation can be performed. To refocus the data from the rare data points. In the real yoke case, each imager in the shai camera array has a different distance centroid. That is, the optical instrument of each imager is designed and The women's volleyball makes the field of view of each imager slightly overlap but for the most part, constructs a different brick with a larger field of view. The images from each of the bricks are stitched together to perform Single high resolution image. In an embodiment, the camera array can be formed on a separate substrate and mounted on the same motherboard with spatial isolation. In each imager The lens member can be arranged such that the field of view corners slightly include a line perpendicular to the substrate. Thus, if four imagers are mounted on the motherboard, each imager is rotated 90 degrees relative to the other imager, The fields of view will be four slightly overlapping bricks. This allows a single wafer level optical lens and imager wafer design to be used to capture different monuments of a panoramic image. In one embodiment, one or more imagers The set is arranged to capture images stitched to produce a panoramic image with overlapping fields of view, while the other imaging benefit or imager set has a field of view including the resulting tile image. This embodiment provides different effective resolutions for Different features of the imager, for example, can be expected to have more brightness resolution than the chroma resolution. Therefore, some phaser groups can use the panoramic view of the door to detect the brightness. Less imager can be The chromaticity is detected to utilize a field of view of the stitched field of view including the brightness imagers. In one embodiment, a camera array having a plurality of imagers is mounted on a flexible motherboard in 201228382 to allow the motherboard to be manually bent to change the aspect ratio of the image. For example, an imager group can be mounted on a flexible motherboard in a horizontal line such that in the stationary state of the motherboard, the field of view of all of the imagers is approximately the same. If there are four imagers, then each has The imager of the individual imager has twice the resolution, so that the details in the main image are half of the range of detail that can be resolved. If the motherboard is bent such that it forms part of a vertical cylinder, the imagers are pointed outward. As the portion is curved, since each point of the main image t is within the field of view of the two imagers instead of the four imagers, the width of the main image is doubled and the resolution of the image is reduced. At maximum bending, the main image is four times wider and the detail that is solvable in the main image is further reduced. Offline Reconstruction and Processing 4 The image processed by imaging system 400 can be previewed before or at the same time as the image data is stored on a flash memory device or a storage medium of a hard disk. In one embodiment, the image or video material includes a bright field data set and other useful image information originally captured by the camera array. Other traditional file formats can also be used. The stored image or video can be played or transmitted to other devices via various wired or wireless communication methods. In one embodiment, the tool is provided to the user via a remote server. The remote server can act as both a repository of these images or video and an off-line processing engine. In addition, applet applets that are part of a photo-sharing site such as Flikr, picasaweb, Facebook, etc. can allow images to be interactively controlled, either individually or collectively. Further, the soft-to-software-to-image editing program can be provided to process images generated by a computer such as the desktop computer and laptop computer of Table 62 201228382. The various modules produced by the imaging device 400 can include the computer selectively activated or re-architected by a computer program stored in a general purpose computer. Such computer programs can be stored in a computer readable storage medium, for example, any disc type including a floppy disk, a CD-ROM, a magnetic disk, a read-only memory (ROM), and a random access memory. Body (RAM), erasable and programmable; read memory, electrical can be programmed to read-only memory, magnetic or optical card, special i# integrated circuit (ASIC) or suitable for storing electronic instructions Any media type, but not limited to, and each one is connected to a computer system bus. Further, the computer referenced in this specification may include a single processor or an architecture that can be designed to increase the power of a computer using multiple processors. Although specific embodiments and applications of the present invention have been shown and described herein, it is understood that the invention is not limited to the specific structures and elements disclosed herein, and the various arrangements, operations and details of the methods and apparatus of the present invention. The modifications, changes and variations can be made without departing from the spirit and scope of the invention as defined in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a plan view of a camera array having a plurality of imagers in accordance with an embodiment. 2 is a perspective view of a camera array having a lens member in accordance with an embodiment. 2 is a cross-sectional view of a camera array in accordance with an embodiment. 63 201228382 FIG. 2C is a cross-sectional view of a camera array with (4) sound suppression according to an embodiment. Fig. 2D is a cross-sectional view of a camera array having a contention suppression function according to a second embodiment. Figure 2E is a cross-sectional view of a camera array incorporating an opaque spacer to provide optical crosstalk suppression in accordance with a further embodiment. 2F is a cross-sectional view of a camera array incorporating a spacer coated with an opaque material to provide crosstalk suppression, in accordance with another embodiment. 3A and 3B are cross-sectional views illustrating changes in the height of a lens member in accordance with changes in size of an imager, in accordance with an embodiment. Fig. 3C illustrates a graph in which the main ray angle is changed depending on the size of the lens members. Figure 3D is a cross-sectional view of a camera array having field flatness in accordance with an embodiment. 4 is a functional block diagram of an imaging device in accordance with an embodiment. Figure 5 is a functional block diagram of an image processing program module in accordance with an embodiment. 6A-6F are plan views of a camera array having different heterogeneous imager layouts, in accordance with an embodiment. Figure 6G is a diagram conceptually illustrating the manner in which the diversity of visible objects is sampled. Figure 6H is a cross-sectional view of a pixel of an imager in accordance with an embodiment of the present invention. Figure 61 is a conceptual illustration of the 64 201228382 graphic of the shadowed area produced when the red and basket color imagers are not symmetrically divided; - around the center of the camera array. Figure 6J conceptually illustrates a pattern of the manner in which the masking regions shown in Figure 61 are eliminated by symmetrically distributing the red and blue imagers around the center access of a camera array. 7 is a flow chart illustrating a method of generating a enhanced image from a lower resolution image captured by a plurality of imagers, in accordance with an embodiment of the present invention. 7A is a flow chart illustrating a method for constructing a normalized plane during calibration, in accordance with an embodiment of the present invention. Figure 7B conceptually illustrates a method of constructing a normalized plane during calibration in accordance with an embodiment of the present invention illustrated in Figure 7A. Figure 8A is a cross-sectional view of a camera array with optical zoom according to an embodiment. Figure 8B is a cross-sectional view of a camera array with optical zoom in accordance with a second embodiment. Figure 8C is a cross-sectional view of a camera array incorporating an imager having different fields of view, according to a further embodiment. [Main 100 2〇〇21〇221 231 Element Symbol Description] Camera Array Camera Array Component Wafer Level Optical Instrument Lens Member Sensor Array Imager 65 240 201228382 250 Camera Array Assembly 254 Seal 258 Top Spacer 262 Top Lens Wafer 264 Intermediate spacer 268 Bottom lens wafer 270 Bottom spacer 274 Straight through perforated 276 Tin ball 278 Substrate 280 Light blocking material 281 Opaque spacer 282 Septum 283 Spacer 284 Light blocking material 286 Optical member 288 Optical member 290 Substrate level 292 Camera array assembly 294 opaque layer 295 spacer 296 aperture 297 opaque coating 310 lens member 66 201228382 3 12 negative lens member 314 imager surface 316 beam 320 lens member 400 imaging system 410 camera array 412 image 420 image processing program Module 422 Composite Image 428 Generate Image 422 Processed Image 440 Controller 442 Operation Signal 444 Information 446 Input 510 Upstream Program Processing Module 514 Image Pixel Dependency Module 518 Parallax Confirmation and Measurement Module 522 Parallax Compensation Module 524 Parallax Information 526 Super Resolution Module 530 Address Conversion Module 540 Imager 546 Logic Address 67 201228382 548 554 558 564 572 610 620 650 652 654 670 672 674 676 780 782 784 786 800 802 805 806 808 810 logical address and phase shift calibration module calibration data downstream color processing module entity pixel light group light group pixel array color ferrite micro lens camera array red imager blue imager background object camera array red imager blue Color imager background object zoom sensor array zoom sensor array effective focus camera array imager size aperture bar 68 201228382 814 image sensor 816 virtual substrate 818 virtual substrate 820 virtual substrate 8 22 zoom channel 824 zoom channel 826 Zoom Channel 828 Aperture Bar 830 Aperture Bar 832 Aperture Bar 834 Lens 836 Lens 838 Lens 710-750 Steps 760-768 Step B Imager with Blue Pulsator G Imager with Green Filter L Diameter R with Red Sepals Imager S width W width t Tl t2 total of twice the overall height 5 69 201228382 CRAl CRA2 principal ray angle principal ray angle of a principal ray angle CRA3 z 1 -z4, Zk from the imager 1A-NM

Claims (1)

201228382 VII. Patent application scope: 1. An imaging device comprising: at least one imager array, each imager in the array comprising a plurality of sensing members and a lens stack comprising at least one lens surface, wherein the lens The stack is structured to form an image on the photosensitive member; the control circuit is configured to capture an image formed on the photosensitive member of each of the imagers; and the super-resolution module is configured to use the plurality Capture images to produce at least one higher resolution super resolution image. 2. The imaging device of claim 1, wherein the optical filter is provided in each of the imagers and is configured to pass through a specific spectral band; and spectral filters through different spectral bands are provided At least two of the imagers. 3. The image forming apparatus of claim 2, wherein the spectral filter is selected from the group consisting of an organic color filter, an absorbent material, a dielectric coating, an interference filter, a multilayer coating, and combinations thereof. in. 4. The image forming apparatus of claim 2, wherein at least one of the imagers further comprises a polarizing filter. 5. The imaging device of claim 2, wherein the lens stacking structure within the imager differs depending on the specific spectral band through which the spectral filter in the imager passes, thereby reducing chromatic aberration. '疋6· The imaging device of claim 5, wherein the at least one surface of the lens stack within the image is the specific light passed by the IJ in the imager 71 £ 201228382 The imaging stack of the sixth aspect of the patent, wherein the surface of the lens stack is structured such that the back focal length of the image, lens stack is the same, and is independent of the spectral band. In the image forming apparatus of item 2, the combination of the high and low Abbe number materials is applied to each of the lens stacks to reduce the chromatic aberration. 9. The image forming apparatus of the second item of the Shenqing patent scope, In one aspect, each of the lens stacks includes at least one aperture stop positioned in front of the photosensitive member, and the spectral filter is positioned adjacent to the aperture column. 10. The imaging device of claim 9, wherein At least the aperture stop is formed of a light blocking material selected from the group consisting of a metal material, an oxide material, a black particle-filled photoresist, and a combination thereof, and an image forming apparatus according to claim 2, wherein , At least one lens surface in the optical stack of the imager differs depending on the particular spectral band through which the spectral filter within the imaging benefit passes; and each lens surface is selected from the group consisting of diffraction, Fresnel, refraction, and 12. The imaging device of claim U. The imaging device of claim U, wherein the radius of curvature of the lens surfaces varies according to a particular spectral band through which the spectral filter within the imager passes. 13. The imaging device of claim 5, wherein the lens stack of each of the imagers has the same back focus. 14. The imaging device of claim 1, wherein Sense 72 201228382 Light member is placed to detect an image in a back-illuminated image. 15. The imaging device of claim 14 wherein the independent negative lens member is placed in proximity to the photosensitive member of the imager And optically aligning the lens stack of the imager to correct the field curvature of the lens stack. 16. The image forming apparatus of claim 1, wherein A separate negative lens member is placed at a photosensitive member that is in close proximity to the imager and optically aligned with the lens stack of the imager such that the field curvature of the lens stack is corrected. 1 7. Imaging as in claim 1 The apparatus, wherein at least two of the imagers of the imager array have different fields of view such that different scene sizes are captured by the imagers. The lens stack of the at least imager has a different focal length. 19. The image forming apparatus of claim 17, wherein the photosensitive member of each of the at least one imager has a different size. The imaging device of item 17, wherein the at least imager comprises a different number of photosensitive members. 21. The imaging process of claim 1, wherein the step further comprises a mechanical zoom mechanism positioned within the lens stack of the imager for smooth transition between different fields of view. For example, the application of the patent range imager imaging, at least the winter clothing, Dan τ, % Jiancheng is designed to perform in the near-infrared spectrum of light 201273 201228382 23. Imaging as claimed in item i The monitor imager is configured to perform the visible light misdetection: the towel, to >, the path is configured to use the measurement to control the imaging device = the control features of the control electrical. Other imagers in the line 24. As claimed in the patent scope! The imaging device of the item, ... the device comprises at least two camera arrays. Eight, the imaging 25. As in the imaging device of claim 24, the camera array is simultaneously operated during imaging to provide a stereoscopic image. 〆 to / -26. For example, the image mount 2 of claim 24 is applied. The two camera arrays are spatially isolated from each other in at least one plane. The image forming apparatus of claim 24, wherein each of the machine arrays includes a different imager, the plurality of which have different imaging characteristics. The imaging device of claim 1, wherein the camera array comprises an NxM imager array, wherein the H2 in the MM is greater than 2, and each of the imagers An imaging apparatus according to claim 1, wherein the embossing circuit is structured to control imaging features of each of the imagers. The imaging device of claim 29, wherein the controllable imaging feature comprises exposure time, gain, and black-order offset. . 3 1 . The imaging device of claim 1, wherein the control circuit is configured to trigger each of the imagers according to a desired sequence. 201228382 32. The imaging device of claim 31 Wherein the control circuit operates the imagers in an interlaced sequence. 33. The imaging device of claim 1, wherein initial primary setting parameters for imaging features of each imager are set by the control circuit; pre-defined errors from the initial primary setting parameters are also Set by the control circuit; and the errors include a group selected from the group consisting of an open system connection setting parameter, a high dynamic range 'gain setting parameter, an integrated time setting parameter, an exposure setting parameter, a digital processing setting parameter, and a combination thereof. function. 34. The image forming apparatus of claim 1, wherein the 竣 phase machine array is fabricated using techniques in a group consisting of wafer level optical technology, injection molding, glass molding, and combinations thereof. 35. The imaging device of claim 1, wherein each of the bristle imagers is optically isolated from each other. 36. The imaging device of claim 35, wherein each of the imagers is positioned in front of the photosensitive members and has a lens arranged to be axially aligned with the imager. How many opaque surfaces of the stacked openings are optically isolated. 37. The image forming apparatus of claim 35, wherein the image forming devices are optically isolated from each other by an opaque wall of the interface between the plurality of imagers. 38. The imaging device of claim 37, wherein the opaque wall is a cavity between imagers filling the light blocking material. 75 201228382 wherein 39. The imaging device of claim 3 " the imaging device comprises a spacer; and the spacers are constructed of an opaque material. Wherein the imaging device of the patent application range of 帛35$ includes a spacer; and the spacers are coated with an opaque material. Among them, the control 41. The circuit of the third paragraph of the patent application is structured to use the dimension space to fill in the secret plane to form a normalized plane for capturing low-resolution images. For example, the imaging skirt of claim 41, wherein the one-dimensional inter-fill curve is expressed by a coefficient of a polynomial. 43. The imaging device of claim φ ^ M ^ ± wherein the control circuit is configured to use a plurality of low resolution images having different exposures. The imaging device of claim 43, wherein the imaging device of the range of the image captures at least one photosensitive member of the imaging benefit used for the image. The photosensitive member has a high dynamic range. 45. For example, the patent application is the 44th is ^ ^ m ^ _ 'imaging device, in which an imager used for capturing a low-resolution shirt image is captured, - an infrared low-resolution image. The person is architected to capture nearly 46. The resolution module as described in the scope of the patent application is structured to be used for the super-decoding device, wherein the super-solution 47. A depth map of the resolution image. 47. As claimed in claim 46, the resolution module is structured to select at least one device', wherein the super-resolution distance is used as the focus plane and the modulo is applied to pixels that are not close to the focal plane. 48. If you apply for a patent scope! The imaging device of the item is configured, , and in the super-decomposition. (4) Some catches (4) Like (4) Temperatures are received as in the imaging device of the 48th patent application, the point spread function of the lens stacks in the dragon varies with temperature; and, the 忒 super-resolution processing module is structured Temperature normalization is performed by a one-point spread function between the temperature calibrations of the point spread functions corresponding to the lens stacks. 50. Selecting the imaging device of the i-th item of the patent scope of the patent application, wherein the resolution processing module is broadcasted and called _Zeng & "The leaf adjusts the color between the captured images in an anisotropic manner. 51. The imaging device of the item, wherein the resolution processing module is configured to computationally correct the difference in geometric distortion between the captured images. 4 5 2 · A lens stack array comprising: a lens member formed on a substrate separated by a spacer, wherein the lens member 'substrate and spacer are structured to form a plurality of optical channels; At least one aperture in the optical channel; at least one spectral filter positioned in each optical channel, wherein each spectral filter is configured to pass a particular spectral band; and is positioned within the array of lens stacks to optically isolate the A fluorescent material for the optical channel. 77 201228382 5 3. The lens stack array of claim 5, wherein the light blocking material is selected from the group consisting of opaque materials, reflective materials, and combinations thereof. 54. The lens stack array of claim 52, wherein spectral filters passing through different spectral bands are provided within at least one of the imagers. 5 5. The lens stacking array according to claim 5, wherein each spectral filter is selected from the group consisting of an organic color filter, an absorbent material, a dielectric coating, an interference filter, a multilayer coating, and It is combined with the group of people formed. 5 6. The lens stack array of claim 5, wherein the step further comprises at least one polarizing filter positioned in each of the optical channels. 5 A lens stack array as in claim 54 wherein each optical channel architecture differs depending on the particular spectral band through which the spectral filter in the imager passes, thereby reducing chromatic aberration. 58. The lens stack array of claim 54, wherein the at least one surface of the lens within each optical channel is a function of a particular spectral band through which the spectral filter within the optical channel passes. 59. The lens stacking array of claim 58, wherein each of the optical channels in the array of lens stacks has the same back focus, independent of a particular spectral band through which the spectral filter in the optical channel passes. . 6. A lens stack array as claimed in claim 54 wherein a combination of high and low Abbe number materials is used in each of the lens stacks within the optical channel to reduce chromatic aberration. '6h, as claimed in claim 54 of the lens stack array, further package 78 201228382 includes: at least one aperture resting in each optical channel; wherein a parent spectrum/sense is positioned in each optical channel The spectral filter is brought close to the aperture column. 62. The lens stacking array of claim 6 wherein each aperture stop is a light blocking material selected from the group consisting of a metal material, an oxide material, a black particle-filled photoresist, and combinations thereof. Formed. 63. The lens stacking array of claim 54, wherein at least one surface of the lens in each optical channel differs depending on a particular spectral band through which the spectral filter in the optical channel passes; A lens surface is selected from the group consisting of diffraction, Fresnel, refraction, and combinations thereof. 64. The lens stacking array of claim 63, wherein the radius of curvature of the surface of the lens varies depending on a particular spectral band through which the spectral filter in the optical channel passes. The lens stack array of claim 63, wherein each of the optical channels has the same back focus. 66. The lens stack array of claim 52, wherein at least one of the lens members in the channel is a negative lens member, the negative lens member being in proximity to an image formed by the optical channel. 67. The lens stacking array of claim 52, wherein at least two of the optical channels have different focal lengths. 68. The lens stacking array of claim 52, which incorporates a mechanical zoom mechanism located within the optical channel, is configured to smoothly rotate between different fields of view. A lens stacking array of 52, wherein the lens stack array comprises an NxM optical channel array, wherein at least one of n&m is greater than 2. 70. The lens stack array of claim 52, wherein The lens stack array is fabricated using techniques in the group consisting of wafer level optical technology, injection molding, glass molding, and combinations thereof. 71. The lens stack array of claim 52, wherein the lens is positioned in the lens A light blocking material within the stacked array for optically isolating the optical channels includes at least two opaque surfaces on a substrate positioned within the optical channel, wherein the two opaque surfaces have openings arranged to be axially aligned with the optical channel. 72. The lens stack array of claim 52, wherein the lens stack is optically isolated within the lens stack array The light blocking material of the optical channels includes an opaque wall disposed at an interface between the optical channels. 73. The lens stacking array of claim 72, wherein the opaque walls are filled The lens stack between the optical channels of the optical material. 74. The lens stack array of claim 52, wherein the plurality of spacers are constructed by a light blocking material and positioned within the lens stack array to optically isolate the 5. The lens stack of claim 5, wherein the plurality of spacers are coated with a light blocking material and positioned within the array of lens stacks to optically isolate the optical channels.