TWI888827B - One kind of pixel shifter and the method for the optical precision stitching. - Google Patents

Abstract

One kind of pixel shifter and the method for the optical precision sitiching, this pixel shifter can decompose, move or combine light rays of an image, thus can be applied to the image device (CMOS image capture chip or projection display chip) or large telescope system, to do the precision optical stitching. It can increase the pixels by 4 times, 9 times, or even 25 times. So, the limitation of chip size by the semiconductor technology will not become the bottleneck of the system pixels number, and it can also make the huge lenses change to the combination of many small lenses becomes possible.

Description

Pixel shifter and method for optical precision splicing using the device

The present invention relates to a pixel shifter and a method for optical precision stitching using the device, particularly an optical image stitching method applicable to camera equipment, projection equipment and telescope systems, which can greatly increase the number of system pixels and improve resolution without using giant lenses.

In recent years, due to the rapid development of computer digital devices and the advancement of semiconductor technology, the capture and playback of digital images are increasingly required to achieve high resolution; for example, the resolution of the CMOS sensor inside the mobile phone camera has reached 64M (9248*6944); the resolution of the projector can reach True 4K (3840*2160); however, human beings' pursuit of resolution is unlimited, and many industrial, national defense, and scientific applications still require cameras and projectors with higher resolutions; for cameras, traditionally, if you want to obtain a very high-pixel image, you must divide the area and take photos with many low-pixel cameras at the same time, and then stitch these photos together using software. This method has fatal flaws, because the speed is too slow, and if different cameras have shaking or focus problems, there are often obvious boundaries or unreasonable situations in the splicing. Similarly, for projectors, if you want to obtain a very high-pixel projection image, you must divide the area and use many low-pixel projectors to project the image at the same time, and then synthesize them. This method also has fatal flaws, the splicing boundaries cause obvious black shadows, or important information may be lost due to cutting black shadows (for example: the loss of star observation caused by cutting of cosmic nebula photos), this situation often happens.

For large reflective space telescopes, the size of the main mirror is closely related to the resolution. The current demand is that the diameter of the mirror is several meters, but the polishing accuracy is required to be at the nanometer level. Such a large and ultra-precise optical reflector becomes a super difficult problem in manufacturing; the solution is to use many small main mirrors for splicing; for example, the small main mirror is designed as a hexagonal lens, and seven, nineteen, or even more lenses are used for splicing. However, this splicing method requires a gap between the main mirrors in the arrangement. Because of the thermal expansion and contraction of the material and the mechanical support problem, this gap is not small. Therefore, the image we finally get will cause defects due to this gap. For example, the light spot caused by the diffraction effect shows six rays of light, and this light area also blocks the original observable area. Therefore, the application must do post-processing software here, which makes the processing complicated and the labor cost, which is one of the many shortcomings caused by the application.

The main purpose of this invention is to propose a pixel shifter that can perform precise optical segmentation and splicing on image imaging devices (CMOS image acquisition chips or projection display chips) or large precision lenses, which can increase the number of pixels by 4 times, 9 times, or even 25 times; so that the general semiconductor chip size restrictions will not become the bottleneck of system pixels, and it can also make it possible to turn a super large lens into a combination of many small lenses, achieving precise positioning of light and reducing costs, which is the feature of this invention.

To achieve the above-mentioned purpose, the pixel displacement device of the present invention can be realized by the following embodiments: a light-entering layer is provided, which is a wedge-shaped structure formed by a light-transmitting material, the outer side is a plane, and the inner side is an angled inclined surface; a light-exiting layer is provided, which is a wedge-shaped structure formed by a light-transmitting material, the outer side is a plane, and the inner side is an angled inclined surface; the light-entering layer and the light-exiting layer have the same structure and are placed opposite to each other. A middle layer is arranged, which is made of a light-transmitting material and is arranged in the middle of the inclined surfaces of the light-entering layer and the light-exiting layer; the refractive index of the light-entering layer and the light-exiting layer is the same, and the refractive index is greater than that of the middle layer; accordingly, when the image enters from the light-entering layer and exits from the light-exiting layer, it can be deflected outward by a displacement, so that the corresponding image sensor can be arranged at the displacement to receive the deflected image.

Another purpose of the present invention is that the light-entry layer, the middle layer, and the light-exit layer of the present invention can be square, and can be combined with the determinant method and the formula guidance generated by the present invention to form a seamless pixel shifter for images (such as those taken by cameras, projectors, etc.).

Another purpose of the present invention is that the light-entry layer, the middle layer, and the light-exit layer of the present invention can be in a hexagonal shape and arranged in a ring shape, and guided by the formula generated by the present invention, to form a seamlessly joined pixel shifter for use in astronomical telescopes.

10,100,100A,1000: light entrance layer

10A: Wedge-shaped structure

10B: First central plane block

100C: Small pixel shifter

100D: Small pixel shifter

11,110,1100: plane

12,120,12A,1200: Inclined surface

121,221: Micro bevel

122,222: Micro vertical surface

20,200,200A,2000: light emission layer

20A: Wedge-shaped structure

20B: Second central plane block

200C: Small pixel shifter

200D: Small pixel shifter

21,210,2100: plane

22,220,22A,2200: Inclined surface

30,300,300A,3000: Middle layer

40,41,42: Image sensor

50:Sensor

51: Valid pixel area

60:Mask

61: Light-transmitting hole

70: Light entrance layer

71: Plane

72: Incline

80:Light-emitting layer

81: Plane

82: Incline

90: Main reflector

91: Secondary reflector

92: Image sensor

E, B: light group

D _N : vector displacement

d : displacement

h _B ,h _E : thickness

L1,L2,L3,L4,L5,L6: light

Lx: vertical line

P:Pixel shifter

S: Gap

ω : angle

n1,n2: refractive index

β : angle of incidence

w ₁ ,w ₂ ,w ₃ ,w ₄ ,w ₅ ,w ₆ ,w ₇ ,w ₈ ,w ₉ : partial image

w _all : full image

Figure 1 is an illustration of the basic structure of the pixel shifter of the present invention and a single light ray incident vertically.

Figure 2 is an illustration of the basic structure of the pixel shifter of the present invention and a single light ray incident from the side at an angle of ω.

FIG3 is a schematic side view illustrating the application of two pixel shifters in the present invention.

FIG4A is a schematic diagram of the application of the present invention using an array 2X2 pixel shifter.

FIG4B is a decomposition diagram of the 2X2 pixel shifter array used in the present invention.

Figure 4C is a top-down structural diagram of the light-incident layer of the 2X2 pixel shifter array used in the present invention.

FIG4D is a side view of the 2X2 pixel shifter array applied in the present invention.

Figure 5A is a diagram showing the sensor corresponding to the 2X2 pixel shifter of the present invention, and the effect after combination.

Figure 5B is a diagram showing the actual position of the sensor corresponding to the 2x2 pixel shifter of the present invention.

Figure 6 is a schematic diagram of the application of the present invention using an array 3X3 pixel shifter.

Figure 7A is a diagram showing the sensor corresponding to the 3X3 pixel shifter of the present invention, and the effect after combination.

Figure 7B is a diagram showing the actual position of the sensor corresponding to the 3x3 pixel shifter of the present invention.

Figure 8 is a structural decomposition diagram of the present invention when a 3x3 pixel shifter array is applied and the middle layer is air.

Figure 9 is a structural diagram of the 2X2 pixel shifter after micro-prismization in the present invention.

Figure 10 is a three-dimensional view of a 2X2 pixel shifter with micro-prism and air in the middle layer in the present invention.

Figure 11 is a side view of the one-dimensional pixel shifter after microprismization in the present invention.

Figure 12 is a diagram illustrating the corresponding sensor positions when the pixel shifter of the present invention is arranged in an even number nxn, with n=4 as an example.

Figure 13 is a diagram illustrating the corresponding sensor positions when the pixel shifter of the present invention is arranged in an odd number nxn, with n=5 as an example.

FIG. 14A is a diagram illustrating the arrangement of conventional image sensors used when implementing the present invention.

FIG14B is a schematic diagram showing the package and actual pixel range of a conventional image sensor when implementing the present invention.

FIG15 is a diagram illustrating the splicing sequence of six hexagonal lenses to form a ring structure, using the pixel shifter of the present invention as an example of a hexagonal embodiment.

Figure 16 shows the present invention using 18 hexagonal pixel shifters to form an inner and outer two-ring structure, and illustrates the splicing sequence of the second ring.

Figure 17A is a schematic diagram of the three-dimensional structure of Figure 15.

Figure 17B is a side view of Figure 17A.

Figure 18A is the light path diagram 1 of the present invention through which the two light groups E and B pass.

Figure 18B is the light path diagram 2 of the present invention through which the two light groups E and B pass.

Figure 18C is a stereoscopic view of a modified embodiment of the present invention used in a 2X2 vector pixel shifter.

FIG. 18D is a side view of the structure of a 2X2 vector pixel shifter used in a modified embodiment of the present invention.

Figure 19 is a schematic diagram of the pixel shifter of the present invention placed in a camera system.

Figure 20 is a schematic diagram of the pixel shifter of the present invention placed in a projector system.

Figure 21 is a schematic diagram illustrating the pixel shifter of the present invention implemented in a large telescope system.

Figure 22 is a partial enlarged view of Figure 21.

Please refer to Figures 1 and 2, which are the first basic structure diagrams of the pixel shifter P of the present invention, which at least include:

A light incident layer 10 is a wedge-shaped structure made of a light-transmitting material, with a flat surface 11 on the outside and an inclined surface 12 on the inside that forms an angle ω with the horizontal.

A light output layer 20 is a wedge-shaped structure made of a light-transmitting material, with a flat surface 21 on the outside and an inclined surface 22 on the inside with an angle ω to the horizontal; the light incident layer 10 and the light output layer 20 have the same structure and are placed opposite to each other, and the material refractive index n1 of the light incident layer 10 and the light output layer 20 is the same.

An intermediate layer 30 is located between the inclined surfaces 12 and 22 of the light incident layer 10 and the light emitting layer 20 and fills them. The refractive index n2 of the intermediate layer 30 is ≧1 and is smaller than the refractive index n1 of the light incident layer 10 and the light emitting layer 20. Therefore, in actual application, the intermediate layer 30 can be an air layer or a soft transparent glue. When the intermediate layer 30 is air, several positioning support points can be set between the light incident layer 10 and the light emitting layer 20. However, this is a related existing manufacturing technology and will not be described or drawn.

1 , when (for example, image) light L1 enters the light incident layer 10, passes through the intermediate layer 30, and exits the light exit layer 20, the vertical line LX from which the light L1 is incident on the light incident layer 10 is used as a reference, and can be offset outward by a displacement d , so that a paired image sensor 40 can receive the deflected image.

其中，d為位移量，h為中間層30材質厚度，n ₁為入光層10及出光層20(例如：玻璃)的折射率，n ₂為中間層30(例如：空氣或透明塑膠)的折射率，ω為入光層內側傾斜角度。 The displacement d of the light ray L entering from the light incident layer 10, passing through the intermediate layer 30, and exiting from the light exit layer 20 must meet the following formula:

Wherein, d is the displacement, h is the material thickness _{of the intermediate layer 30, n1} is the refractive index of the light incident layer 10 and the light output layer 20 (eg, glass), n2 is the refractive index of the intermediate layer 30 (eg, air or transparent plastic), and ω _is the inner tilt angle of the light incident layer.

The above-mentioned light rays are displaced after passing through the present invention. The displacement formula is the result of the careful design of the present invention. First, a schematic diagram of a single light ray vertically incident on the pixel shifter of the present invention is described. Please refer to FIG1 . The symbols are as follows: X: The node where the vertical light ray L just enters the light layer 10

n ₁ : Refractive index of the light-entering layer and the light-exiting layer (e.g. glass)

n ₂ : The refractive index of the intermediate layer (e.g. air or transparent plastic)

ω : Inner tilt angle of the light incident layer 10 and the light output layer 20

A: The light enters the intersection of the light-entering layer 10 and the middle layer 30

θ : The angle between the normal line and the refracted light at point A at the junction of the incident layer 10 and the intermediate layer 30

h : middle layer 30 material thickness

d : The displacement d between the vertical light ray L and the position of the node vertical line L _x when the vertical light ray L enters the light-entering layer 10 from the node x position, passes through the intermediate layer 30, and then exits the light-exiting layer 20.

Consider a vertical ray of light, and the situation when the ray is refracted at point A: (Note: This is an approximation of the Fresnel principle)

n ₁ ω = n ₂ θ (1)

h cos ² ω tan θ = d + h cos ² ω tan ω (9)

approximate

approximate

Finally, it is found that when the vertical light ray L passes through the intermediate layer 30 and then exits the light emitting layer 20 from the base point of the light emitting layer 10, the displacement d between the base point is:

As shown in FIG2 , the present invention considers that when the incident angle β of the light ray L2 that is not perpendicular to the incident layer 10 corresponds to a certain inclined angle, total reflection may occur at point A (i.e., the intersection of the light ray L2 incident on the incident layer 10 and the intermediate layer 30 ). This is a situation that should be considered and avoided in order to successfully implement the present invention. Therefore, the Fresnel formula is written here in a non-approximate way as follows:

由(14)(15)式

假設近似的狀況：當θ比較小的時候，n ₁ sinθ

θ(16)(17)重寫如下

sin θ <1 (15) At point F: β is the angle of incidence

From (14) (15)

Assume the approximate situation: when θ is small, n ₁ sin θ

θ (16)(17) can be rewritten as follows

Therefore, the angle range of the inner side inclined surfaces 12 and 22 of the light incident layer 10 and the light output layer 30 (i.e., the angle range between the inclined surface of the wedge-shaped structure and the horizontal plane) must meet the following formula:

以此到最佳的像素位移器中入光層10及出光層30的內側面斜面12,22與水平綫的角度ω，使結構更為精確。 The β value here corresponds exactly to the numerical aperture (NA) of the optical system, so formula (20) can be rewritten as the following formula:

In this way, the angle ω between the inner side inclined surfaces 12, 22 of the light incident layer 10 and the light output layer 30 and the horizontal line in the pixel shifter is optimized, making the structure more precise.

假設光學系統中的數據如下：Fno=1.5，可得NA=0.33，n ₂=1(空氣折射率)，n ₁=1.713(n ₁採用日本小原公司生產的SLAL8光學玻璃材質)從前面的(21)式可得ω<0.391徑度將徑度轉換成角度

由以上的分析，可以證明如圖2所示，本發明入光層10及出光層30的內側斜面12,22角度ω，以設定在22.42度以下為最佳，如此則可以避免全反射的情況發生，以使本發明像素位移器符合產業上的利用價值。 Therefore, when using the pixel shifter P of the present invention, according to the aperture value and the numerical aperture value conversion formula:

Assume that the data in the optical system are as follows: Fno = 1.5, we can get NA = 0.33, n ₂ = 1 (refractive index of air), n ₁ = 1.713 ( n ₁ is made of SLAL8 optical glass produced by Japan's Ohara Corporation). From the previous formula (21), we can get ω < 0.391 diameter. Convert the diameter to angle

From the above analysis, it can be proved that as shown in FIG. 2 , the angle ω of the inner side slopes 12 , 22 of the light incident layer 10 and the light output layer 30 of the present invention is best set to be below 22.42 degrees, so that total reflection can be avoided, so that the pixel shifter of the present invention meets the industrial utilization value.

Please refer to FIG. 3 , which is an embodiment of using two pixel shifters P of the present invention. That is, when the light L3 (e.g., image) directly hits the two parallel pixel shifters of the present invention, it will be diverted to different directions and respectively received by two image sensors 40. As described above, the image sensors 40 can be substituted into formula (13) to calculate the deflection displacement d , so that the two image sensors 40 can perfectly receive the light and form a clear and complete image.

Please refer to Figures 4A, 4B, 4C, and 4D, which are embodiments of the pixel shifter P of the present invention in a 2X2 matrix, including a light-entering layer 100, which is composed of four wedge-shaped structures made of transparent materials, with flat surfaces 110 on the outside and angled slopes 120 on the inside. However, the slopes 120 surround the light-entering layer 100, which can deviate the incident light L4 to the outside after entering.

A light-emitting layer 200 is composed of four wedge-shaped structures made of light-transmitting materials. The outer sides are all flat surfaces 210, and the inner sides are all angled slopes 220. However, the slopes 220 surround the light-emitting layer 200, and the slopes 220 of the light-emitting layer 200 are exactly aligned with the slopes 120 of the light-entering layer 100, and are arranged opposite to each other.

An intermediate layer 300 is located between the inclined surfaces 120, 220 of the light incident layer 100 and the light emitting layer 200 and fills them. The refractive index n2 of the intermediate layer 300 is ≧1 and is smaller than the refractive index n1 of the light incident layer 10 and the light emitting layer 20. Therefore, in actual application, the intermediate layer 300 can be an air layer or a soft transparent glue. This is used as an example when drawing the series of Figure 4. The intermediate layer 300 is a soft transparent layer sealed between the light incident layer 100 and the light emitting layer 200.

Accordingly, when the image light L4 enters the light incident layer 100, passes through the middle layer 300, and exits from the light exit layer 200, based on the vertical line Lx of the image light L4 incident on the light incident layer 100, it can be offset outward by a vector displacement D _N (described in detail later) according to the vector summation, so that multiple (four in this embodiment) matched image sensors 41 can receive the deflected image, and the images of the sensors 41 can be subsequently reassembled into a complete full image through software.

When the pixel shifters P of the present invention are arranged in a matrix, when the light L4 exits the light-entering layer 100, the intermediate layer 300, and the light-exiting layer 200, the displacement of the light L4 has a vector factor. The following deduction is obtained from the above formula (13): The one-dimensional displacement distance

Therefore, we have reached a very important conclusion. Please refer to Figure 1. In one-dimensional space, the original position of the light is x . Assume that -ω is the angle from the slope to the horizontal plane. At the same time, because it is clockwise, the coordinate is a negative value. After pixel displacement, the position of the light is x - d .

)<n ₂該公式宣告了材料的選擇，決定了斜面角度的極限值從該公式(17)的最小值n ₂=1(材質是空氣)，n ₁是高折率玻璃，如果我們分別設定n ₁=1.8，光線最大變化角度

=10°，可得ω<23.76°。惟該ω值越大，整個向量位移裝置體積越大，因此我們在ω的選擇上，會小於20度，從而避免了全反射的問題。 As shown in Figures 5A and 5B, in the two-dimensional space, the original position of the light is ( x , y ). Assuming that ωx , ωy are _the x and _y components of the two-dimensional angle ω from the inclined plane to the horizontal plane, the following formulas can be listed according to their positive and negative values: After the pixel displacement in the X direction, the position of the light is ( x ± d , y ) After the pixel displacement in the Y direction, the position of the light is ( x , y ± d ) After the pixel displacement in the X and Y directions, the position of the light is ( x ± d , y ± d ) The above describes the vector displacement of the light, which represents the various vector displacements of a single module. From the above formula (17), n ₁ sin( ω +

)< n ₂ This formula declares the choice of material and determines the limit of the bevel angle. From the minimum value of formula (17) n ₂ = 1 (the material is air), n ₁ is high refractive index glass. If we set n ₁ = 1.8, the maximum angle of light change

=10 ° , we get ω <23.76 ° . However, the larger the ω value is, the larger the volume of the entire vector displacement device is. Therefore, we choose ω to be less than 20 degrees, thus avoiding the problem of total reflection.

其中N=1,2,3,4是如圖5A，5B中，每個模塊對應位置的順序。其中d為其中兩個影像感測器在一維狀況下得到的偏移值，可參考前揭公式第(13)式。 As shown in FIGS. 5A and 5B, when the pixel shifter P _of the present invention is used and the number of rows and columns is 2×2 to form a module, the vector displacement DN corresponding to the corresponding sensor can be expressed as follows: The vector of the second quadrant is expressed as (-d, d); the vector of the first quadrant is expressed as (d, d); the vector of the third quadrant is expressed as (-d, -d); the vector of the second quadrant is expressed as (d, -d). The vector displacement can be uniformly expressed by formula (23):

Where N=1,2,3,4 is the order of the corresponding position of each module as shown in Figures 5A and 5B. Where d is the offset value obtained by two image sensors in one dimension, which can be referred to in the above formula (13).

The pixel shifter P of the present invention can be further refined. Originally, as shown in FIG. 4A and FIG. 4B, the wedge-shaped inclined surfaces 120 and 220 on the inner side of the light-entering layer 100 and the light-emitting layer 200 can be processed into micro-prisms, as shown in FIG. 9, FIG. 10, and FIG. 11, the inner inclined surfaces 12A and 22A of the light-entering layer 100A and the light-emitting layer 200A are formed into micro-prisms, including a plurality of micro-inclined surfaces 121, 221 and micro-vertical surfaces 122, 222. , wherein the micro vertical surfaces 122, 222 should be treated to be opaque, such as being painted black, while the micro inclined surfaces 121, 221 can allow light to pass through; when the image light L5 enters the light incident layer 100A, passes through the middle layer 300A (the figure takes the air layer as an example), and exits from the light exit layer 200A, the image light L5 is deflected by the micro inclined surfaces 121, 221, and is offset outward by a _vector displacement amount DN according to the vector sum, so that a plurality of (four in this embodiment) matched image sensors 40 receive the deflected images, and are reassembled into a complete full image through software. This design can greatly reduce the thickness of the pixel shifter of the present invention, reduce the weight and volume, and is more conducive to storage and transportation, and saves costs.

Another embodiment of the present invention, please refer to FIG. 6 and FIG. 8, is a nine-square grid embodiment of the pixel shifter P of the present invention in a 3X3 matrix, including a light-entering layer 1000, a nine-square grid composed of eight wedge-shaped structures 10A made of transparent materials in the outer ring and a first central plane block 10B. The outer sides of the wedge-shaped structures 10A made of eight transparent materials in the outer ring and the first central plane block 10B are planes 1100, but the inner side of the wedge-shaped structures made of eight transparent materials in the outer ring is an angled slope 1200, but the slope 1200 surrounds the light-entering layer 1000 to deviate the light to the outside; and the inner side of the first central plane block 10A is also a plane.

A light-emitting layer 2000 is a nine-square grid composed of eight wedge-shaped structures 20A made of a transparent material in an outer ring and a second central plane block 20B. The outer sides of the wedge-shaped structures 20A made of the eight transparent materials in the outer ring and the second central plane block 20B are planes 2100, but the inner sides of the wedge-shaped structures made of the eight transparent materials in the outer ring are flat. It is an angled inclined surface 2200, but the inclined surface 2200 surrounds the light-emitting layer 2000, and the inclined surface 2200 of the light-emitting layer 2000 just matches the inclined surface 1200 of the light-entering layer 1000, and the two are arranged oppositely; and the inner side of the second central plane block 20A is also a plane, and is opposite to the inner side of the first central plane block 10A.

An intermediate layer 3000 is located between the inclined surfaces 1200, 2200 of the light incident layer 1000 and the light emitting layer 2000 and fills them. The refractive index of the intermediate layer 3000 is ≧1 and is smaller than the refractive index of the light incident layer 1000 and the light emitting layer 2000. Therefore, in actual application, the intermediate layer 3000 can be an air layer or a soft transparent glue, but this embodiment is described by taking the air layer as an example.

Accordingly, as shown in FIG6 , when the (image) light L5 enters the light incident layer 1000, passes through the middle layer 3000, and exits from the light exit layer 2000, based on the vertical line LX of the image light L5 incident on the light incident layer 1000, a displacement D _N can be offset outward according to the vector sum, as shown in FIG7B , so that multiple (8 in this embodiment) matched image sensors 42 can receive the deflected partial images w ₁ , w ₂ , w ₃ , w ₄ , w ₅ , w ₆ , w ₇ , w ₈ , w ₉ , so that the images of the sensors 42 can be subsequently reassembled into a complete full image w _all (as shown in FIG7A ) by software. In this way, there is no need to use large lenses the size of a nine-square grid as is customary, which can save costs and has the added value of being easy to transport, not taking up space in storage, and being easy to manufacture.

The above-mentioned pixel shifter P of the present invention has a module with 3×3 rows and columns, and the corresponding arrangement of sensors is a non-uniform vector displacement device, which is expressed as follows: FIG. 7A shows the combined effect, and FIG. 7B shows the positions of the 9 sensors 42 to be combined: the 3rd sensor: (-d, d), the 2nd sensor: (0, d), the 1st sensor: (d, d) the 4th sensor: (-d, 0), the 9th sensor: (0, 0), the 8th sensor: (d, 0) the 5th sensor: (-d, -d), the 6th sensor: (0, -d), the 7th sensor: (d, -d) can be uniformly expressed by two formulas (24) and (25):

N=1,2,3,4,5,6,7,8. Here N is the order of position, and the direction is expressed in counterclockwise direction.

D _N =[0,0]-------------(25)

(26)式是在(24)式中之係數函數。其中d為其中兩個影像感測器在一維狀況下得到的偏移值，可參考前揭公式第(13)式。 N=9 [0,0] represents the central position, and the central position is represented by N=9.

Formula (26) is the coefficient function in formula (24). Where d is the offset value obtained by two image sensors in one dimension, which can be referred to in the above formula (13).

個內環：假設N=4，整個結構分為2個內環：第一內環的位移量是(±d,±d) 第二內環的位移量是(±2d,±2d)如圖13是奇數N×N位移向量裝置；整個結構分為

個內環：假設N=5，整個結構分為3個內環；第一內環的位移量是(0,0)第一內環的位移量是(±d,±d)第一內環的位移量是(±2d,±2d)以本發明之方法類推，可將本發明像素位移器依需求而執行更高的行列數的組合，但因使用場合較少，以下不再贅述。 Therefore, if the present invention is applied to a larger number of rows and columns, such as FIG. 12 which is an even number N × N displacement vector device, it can be seen that the entire structure is divided into

Inner rings: Assuming N = 4, the entire structure is divided into 2 inner rings: The displacement of the first inner ring is (±d, ±d) The displacement of the second inner ring is (±2d, ±2d) As shown in Figure 13, it is an odd number N × N displacement vector device; the entire structure is divided into

Inner rings: Assume N = 5, the entire structure is divided into 3 inner rings; the displacement of the first inner ring is (0,0) The displacement of the first inner ring is (±d, ±d) The displacement of the first inner ring is (±2d, ±2d) By analogy with the method of the present invention, the pixel shifter of the present invention can be used to perform a combination of a higher number of rows and columns as required, but since it is rarely used, it will not be elaborated below.

The following is to explain that the present invention is indeed an invention with great industrial utilization value. Now, the present invention is applied to a specific embodiment of a camera, and the explanation is as follows: the original very common image sensor is used to be spliced into a 4K-level advanced camera sensor; suppose a commonly used OV2735 cmos image sensor produced by OmniVision, USA, is 2 million pixels and is often used in dashcams and home monitoring devices. The present invention can be used to splice 4 2 million pixel images to form an 8 million pixel sensor module: the specifications of a single sensor are: number of pixels: 1920X1080pixels

Pixel size: 3umX3um

Image range size: 5808um*3288um

Package size: CSP 6956um*4765um

Image equivalent size: 1/2.7inch

Chief light angle: less than 12 degrees (linear)

該值為封裝後的影像感測器邊緣到x方向畫素及y方向畫素的距離。這裡我們假設最後採用的d向量為d _x =

=810，該選擇810um的原因是：810>738.5>574；而且810um是像素大小3um的整數。以下計算像素位移器的參數的案例：假設我們採用日本小原公司生產的S-LAH55玻璃材質n ₁=1.835，及中間層是空氣，n ₂=1； ω=20°=0.349 When the pixel shifter P of the present invention is used with a row and column number of 2×2, the arrangement of the corresponding sensor 50 is shown in FIG. 14A , and FIG. 14B is a front view of the arrangement of the sensor. The actual receiving area of the sensor 50 is the effective pixel area 51 which is slightly reduced in volume and shape. Therefore, in order to make the effective pixel area 51 of the sensor 50 fully and accurately align with the image to be received, the minimum required displacement vector calculated from the above specification data is:

This value is the distance from the edge of the packaged image sensor to the x-direction pixel and the y-direction pixel. Here we assume that the final d vector used is d _x =

=810. The reason for choosing 810um is: 810>738.5>574; and 810um is an integer of the pixel size 3um. The following is an example of calculating the parameters of the pixel shifter: Assuming that we use S-LAH55 glass material produced by Japan's Ohara Company, n ₁ =1.835, and the middle layer is air, n ₂ =1; ω =20 ° =0.349

從前揭公式(13)，可以求得h=8212.3um，可近似為h=8.21mm請參考圖4A對應於圖14A的2X2像素位移器，入光層與出光層的中心間距h為8.22mm。而感測器的具體排列方式可以參考圖14B，經過拼接之後，即可得到：四個感測器整合相加之後變成原先4倍的像素畫質，參數如下：像素數目：3840X2160pixels

From the above formula (13), we can get h = 8212.3 um , which can be approximated as h = 8.21 mm. Please refer to Figure 4A. For the 2X2 pixel shifter corresponding to Figure 14A, the center distance h between the light-entering layer and the light-exiting layer is 8.22 mm. The specific arrangement of the sensors can be referred to Figure 14B. After splicing, we can get: After the four sensors are integrated and added, the pixel quality becomes 4 times the original. The parameters are as follows: Number of pixels: 3840X2160pixels

Pixel size: 3umX3um

Image range size: 13342um*8196um

Package size: 14384um*9673um

Image equivalent size: 1 inch (13.2*8.8 mm). This proves that the use of this invention can achieve seamless splicing and increase the number of pixels by 4 times. This is the bottleneck in the industry and has been solved by this invention.

The present invention can also be used in the splicing of the main mirror of an astronomical telescope; because the larger the area of the main mirror of an astronomical telescope, the higher the resolution; the main mirror is generally composed of many mirrors, but the cost of manufacturing ultra-large and high-precision mirrors is very high; therefore, the best method is to use nearly circular hexagonal mirrors 60 for splicing; and if the pixel shifter of the present invention is used in an astronomical telescope, it is used as a "mirror", so the present invention should be called a mirror when applied thereto, so as to be easier to understand; as shown in FIG15, six mirrors are arranged around Mirror 60 (i.e., the pixel shifter of the present invention) can expand the overall receiving area by 6 times, but a light-transmitting hole 61 must be left in the center area; in contrast, as shown in Figure 16, eighteen mirrors 60 can be assembled to expand the overall receiving area by 18 times, and a light-transmitting hole 61 must be left in the center area. However, a gap S is still required between all adjacent mirrors 60 to prevent collision or friction between each other; the gap S usually causes the defect of image reception leakage; but the present invention can be applied here to help improve this defect.

When the pixel shifter P of the present invention is applied to an astronomical telescope, it can be used as a face mirror. Due to the limitation of the application occasion, the appearance must be hexagonal. As shown in FIG. 17A and FIG. 17B , the pixel shifter of the present invention is applied to a space telescope as an embodiment, and at least comprises: a hexagonal light incident layer 70, which is made of a light-transmitting material, the outer side is a plane 71, and the inner side is a plane 71 with an angle to the horizontal. The light incident layer 70 and the light emitting layer 80 have the same structure and are placed opposite to each other; and the refractive index of the materials of the light incident layer 70 and the light emitting layer 80 is the same. An intermediate layer 30 is located between the inclined surfaces 72, 82 of the light incident layer 70 and the light emitting layer 80 and fills them. The refractive index of the intermediate layer 30 is ≧1 and is smaller than the refractive index of the light incident layer 70 and the light emitting layer 80. Therefore, in actual application, the intermediate layer 30 can be an air layer, or preferably a soft transparent glue. When the intermediate layer 30 is air, a few positioning fulcrums can be provided between the light incident layer 70 and the light emitting layer 80. However, this is a related existing manufacturing technology and will not be described or drawn here.

As shown in FIG15 , the present invention is used as a mirror 60 on the main mirror of an astronomical telescope. The six mirrors 60 are spliced in a ring shape. The arrangement order of the matched pixel shifters P is as shown in the figure. The center point of the ring is taken as an X axis as the starting point, and the pixels are arranged in a ring in a counterclockwise direction in the order of N=1,2,3,4,5,6. When each position is in order, its displacement vector DN is _:

Where N is substituted into 1, 2, 3, 4, 5, 6 according to the position. Where d is the offset value of two mirrors in one dimension, which can be defined in formula (13) mentioned above.

Please refer to FIG. 17A, which is a three-dimensional view of FIG. 15, that is, six hexagonal pixel shifters P are used as an embodiment in the form of a mirror combination of an astronomical telescope, and FIG. 17B is a side view of FIG. 17A. It can be seen from this that the present invention can be applied to an astronomical telescope and is also very practical and excellent.

M=1~12，此為外環面鏡60位置的標示；其中公式(28)中之係數函數G(M)為

M=1~12，此為外環面鏡60位置的標示；其中d為其中兩個面鏡在一維狀況下得到的偏移值，可參考前揭公式第(13)式。 The present invention can also be used in a higher-end astronomical telescope. For example, FIG. 16 shows the main mirror used in a larger astronomical telescope, which is composed of eighteen inner rings and outer rings of the present invention as a mirror 60. The arrangement order of the mirror 60 is based on the X axis of the ring center point as the starting point, and is arranged in a counterclockwise direction. The inner ring is numbered N, ranging from 1 to 6, and the outer ring is numbered M, ranging from 1 to 12. The first is the displacement _vector DN of the inner ring as mentioned above, formula (27) where N=1 to 6, which is the mark of the inner ring position; the second is the displacement vector DM of the outer _ring as follows

M = 1~12, which is the mark of the position of the outer ring mirror 60; the coefficient function G ( M ) in formula (28) is

M=1~12, which is the mark of the position of the outer ring mirror 60; where d is the offset value obtained by two of the mirrors in a one-dimensional state, which can be referred to in the above formula (13).

Accordingly, when the present invention is used as the faceplate 60 of an astronomical telescope, the displacement of the image is calculated, whether it is _the displacement vector DN of the inner ring or the displacement vector DM _of the outer ring, so that the faceplate 60 can be placed in the best position to receive the full image. Therefore, when taking pictures, the present invention as the faceplate 60 will not have the defect of missing images, especially the defect of missing cosmic nebulae, thus solving the defect of conventional use.

Therefore, please refer to FIG. 21, which is the implementation method of the present invention used in an astronomical telescope, and FIG. 22 is a partial enlarged view of FIG. 21. The astronomical telescope includes a main reflector 90 with a central hole, a secondary reflector 91 is arranged in front of the main reflector 90, and an image sensor 92 is arranged behind the main reflector 90 facing the central hole; and the pixel shifter P of the present invention is placed in front of the main reflector 90. When the light L6 is incident on the main reflector 90, it is first deflected by the light incident layer 70, the middle layer 30, and the light exit layer 80 of the pixel shifter P, and then reflected by the main reflector 90 to the secondary reflector 91, and then received by the image sensor 92, so that a complete and clear image of the cosmic nebula can be obtained.

When the present invention is used, as shown in FIG. 4B , when the volume of the pixel shifter P is larger, as shown in FIG. 18A , when two light groups E and B are incident from two ends of the pixel shifter P, the light groups E and B respectively enter the light output layer 200 and pass through the thickness h _B and h _E is different, so there may be uneven aberration and slight blur. In detail, the inventors of the present invention calculated the aberration when considering a group of focused light rays entering the pixel shifter P. For example, in the one-dimensional pixel shifter P in FIG18A , both light groups E and B encounter a bend caused by an inclined surface, but the positions of the bends are different. Generally speaking, the path difference is related to the aberration value. Here, the difference between the two areas DFF'D' and ACC'A' is calculated, and then the impact of this difference is discussed.

由(30)(31)

From (30)(31)

假設

Assumptions

公式(44)是重要結論，從該公式發現，(h _B -h _E )是關鍵因子，此值越接近0，像差越小。

Formula (44) is an important conclusion. From this formula, we can see that ( h _B - h _E ) is the key factor. The closer this value is to 0, the smaller the aberration.

To solve this concern, please refer to FIG. 4B . As shown by the dotted line, each pixel shifter P is cut in half at the diagonal of each pixel shifter, so that the light-entering layer 100 and the light-exiting layer 200 are both composed of two small pixel shifters 100C, 100D, 200C, 200D. Then, one of the cut small pixel shifters 100D, 200D is reversed to the state of FIG. 18B , and the middle of the two reversed small pixel shifters 100C, 100D; 200C, 200D forms a sawtooth shape, that is, the original continuous inclined surface as shown in FIG. 18A becomes a sawtooth-shaped inclined surface (or prism array) as shown in FIG. 18B , so ( h _B - h _E ) value can be 0; and the detailed structure after the change is shown in FIG. 18B, FIG. 18C, and FIG. 18D, which constitutes a 2X2 pixel shifter, and the middle layer 300 is set as an air layer, so that the inclined surface between the small pixel shifters 100C, 100D; 200C, 200D is changed to a sawtooth shape; not only can the situation of uneven aberration and blurring be avoided, but also the overall thickness can be reduced, achieving the advantage of reducing weight, which is one of the application changes of the present invention when used in a larger volume.

The present invention can also be applied to cameras. As shown in FIG19 , a pixel shifter P is applied to a camera. The purpose is to increase the number of pixels of the camera. By using the present invention, four 2M pixel image sensors can be combined to obtain an 8M module. Four 8M pixel image sensors can be combined to obtain a 32M module. Four 32M pixel image sensors can be combined to obtain a 128M module. Traditionally, 128M pixel image sensors are expensive and difficult to manufacture. Therefore, the pixel shifter of the present invention is used; it can not only obtain perfect splicing, but also form a parallel processing method, which is more advantageous in image processing speed; the pixel shifter of the present invention is generally placed between the rear of the lens group and the sensor, and the main light angle of the sensor is close to 0 degrees, which is ideal, because the pixel shifter has a light angle limitation; if the main light angle is too large, the light will be totally reflected in the pixel shifter.

The present invention can also be applied to projectors, as shown in FIG20 , where a pixel shifter P is applied to a projector; the purpose is to increase the number of pixels of the projector; (taking LCOS image sensors or DMD image sensors as an example), combining 4 image sensors with 1K pixels can obtain a 4K module; combining 4 image sensors with 2K pixels can obtain an 8K module; traditionally, image sensors with 8K pixels are expensive and difficult to manufacture, but using the pixel shifter P of the present invention; not only can perfect splicing be obtained, but also a parallel processing method can be formed, which is more advantageous in the speed of processing the projected image; here, the pixel shifter is generally placed between the rear of the lens group and the sensor, and should be close to the lens, because the light source device does not need to pass through the pixel shifter to avoid loss of light energy. For this application, the system needs to be telecentric, that is, the main light is close to 0 degrees.

Due to the ingenious design of the invention, it has the following advantages in use and manufacturing:

1. The pixel shifter of the present invention can perform precise optical splicing, which can increase the resolution by 4 times, 9 times, or even 25 times. In addition, the image processing can also use parallel processing, which will not increase the processing time, which is the main advantage of the present invention.

2. Since the present invention starts from the physical effect of light displacement caused by the tilted interlayer in a glass sandwich structure, a vectorized, arrayed, and polygonal pixel displacement array can be made to separate and synthesize the incident light, thereby avoiding the gap problem between the imaging device and the small main mirror, which is another advantage of the present invention.

However, the structure described above is only a preferred embodiment of the present invention and is not intended to limit the scope of implementation of the present invention. Therefore, equivalent or easy changes made by those who are familiar with this technology, equal changes and modifications made without departing from the spirit and scope of the present invention, such as: roughly changing the shape or size of the components, or increasing or decreasing the array arrangement, or using different materials, or applying to different projection occasions, etc., but using the characteristics of the present invention, should all be included in the characteristics of the present invention.

100A: light incident layer

200A:Light-emitting layer

300A: Middle layer

41: Image sensor

L5: Light

P:Pixel shifter

Claims (13)

A pixel shifter at least comprises: a light-entry layer, which is a wedge-shaped structure made of a light-transmitting material, with a flat surface on the outside and a slope on the inside at an angle to the horizontal; a light-exit layer, which is another wedge-shaped structure made of a light-transmitting material, with a flat surface on the outside and a slope on the inside at the angle to the horizontal; the light-entry layer and the light-exit layer have the same structure and are placed opposite to each other, and the refractive index of the material of the light-entry layer and the light-exit layer is the same ; An intermediate layer is located between the inclined surfaces of the light-entering layer and the light-exiting layer and fills the inclined surfaces therein. The refractive index of the intermediate layer is ≧1 and is smaller than the refractive index of the light-entering layer and the light-exiting layer. Accordingly, when the image light enters the light-entering layer, passes through the intermediate layer, and exits the light-exiting layer, it can be deflected outward by a displacement based on the vertical line from the image light incident to the light-entering layer, so that the corresponding image sensor can receive the deflected image. As for the pixel shifter described in Item 1 of the patent application, the middle layer is one of an air layer and a soft transparent glue. For the pixel shifter described in item 1 of the patent application, the displacement d of the image light entering from the light-input layer, passing through the intermediate layer, and exiting from the light-output layer must meet the following formula:

; and d shown in the formula is the displacement, h is the thickness of the middle layer material, n1 is the refractive _index of the light incident layer and the light output layer, n2 is the refractive index _of the middle layer, and ω is the inner tilt angle of the light incident layer and the light output layer. For the pixel shifter described in item 1 of the patent application, the inner side surfaces of the light-entering layer and the light-exiting layer have angled slopes, and the angle range must comply with the formula angle

, where n1 is the refractive index of the light incident layer and the light exit layer, n2 is the refractive index _of the intermediate layer, ω is the inner tilt angle of the light incident layer and the light exit layer, _and NA is the numerical aperture of the optical system. As described in item 1 of the patent application scope, the pixel shifter can be arranged in an array to form a pixel shifter group, and the number of rows and columns of the array arrangement group is greater than or equal to 2, so that the image sensor displacement group can receive the deflected image. For example _, when the pixel shifter described in Item 5 of the patent application is composed of 2×2 rows and columns, the vector displacement DN corresponding to the image sensor must comply with the following formula: the vector of the second quadrant is represented by (-d, d); the vector of the first quadrant is represented by (d, d); the vector of the third quadrant is represented by (-d, -d); the vector of the second quadrant is represented by (d, -d). The vector displacement

Where N = 1, 2, 3, 4 and d is the one-dimensional displacement

; h is the thickness _of the middle layer material, n1 is _the refractive index of the light incident layer and the light output layer, n2 is the refractive index of the middle layer, ω is the inner tilt angle of the light incident layer and the light output layer. For example, when the pixel shifter described in item 5 of the patent application scope is composed of 3×3 rows and columns, the arrangement of the corresponding sensors must comply with the following figure: the third sensor: (-d, d), the second sensor: (0, d), the first sensor: (d, d) the fourth sensor: (-d, 0), the ninth sensor: (0, 0), the eighth sensor: (d, 0) the fifth sensor: (-d, -d), the sixth sensor: (0, -d), the seventh sensor: (d, -d) The vector displacement corresponding to the corresponding image sensor must comply with the following formula:

N=1,2,3,4,5,6,7,8 where d is the one-dimensional displacement

; h is the thickness _of the middle layer material, n1 is the refractive index _of the light-entering layer and the light-exiting layer, n2 is the refractive index of the middle layer, ω is the inner tilt angle of the light-entering layer and the light-exiting layer; N is the order of the position, and the azimuth order is represented in a counterclockwise direction; D _N = [0,0] N = 9 [0,0] represents the central position, and the central position is represented by N = 9

. As described in Item 5 of the patent application scope, the pixel shifter group is formed by array arrangement, the number of rows and columns of the array arrangement group is an odd number, and the center position is a flat optical glass. As described in the first item of the patent application, the pixel shifter has a micro-prism structure formed on the inner slope of the light-entering layer and the light-exiting layer, which includes multiple micro-slopes and micro-vertical surfaces. The angles of the micro-slopes of the light-entering layer and the light-exiting layer must be consistent, and the micro-vertical surfaces are blackened, while the micro-slopes allow light to pass through. As described in Item 1 of the patent application, the pixel shifter is composed of two small pixel shifters, and the junction of the small pixel shifters is exactly the diagonal line of the pixel shifter. As described in the first item of the patent application, the light-entering layer, the middle layer, and the light-emitting layer together form the pixel shifter in a square or hexagonal shape, so that the image sensor can be displaced to receive the deflected image. As described in item 11 of the patent application, the pixel shifter is formed in a hexagonal shape, and is composed of six rings spliced together to form a mirror of an astronomical telescope to form a main mirror. The arrangement order of the matching pixel shifting device is to use the center point of the ring as an X-axis as the starting point, and to arrange a ring in a counterclockwise direction in sequence of N=1,2,3,4,5,6, so that when each position is in sequence, its displacement vector DN is _:

Where d is the one-dimensional displacement

; h is the thickness _of the middle layer material, n1 is the refractive index of the light incident layer and the light output layer, n2 is the refractive index of the middle layer, _and ω is the inner tilt angle of the light incident layer. As described in the 12th item of the patent application, the pixel shifter is formed in a hexagonal shape, and six pixel shifters are firstly spliced into an inner ring in a ring shape, and then twelve pixel shifters are spliced into an outer ring in a ring shape to form sixteen mirrors of an astronomical telescope to form a main mirror of an astronomical telescope; wherein, the arrangement order of the pixel shifters of the outer ring is from the center point of the ring as an X-axis as the starting point, in a counterclockwise direction, in order of M=1~12, and the displacement vector DM _of the outer ring is as follows: the inner ring is numbered N, and the outer ring is numbered M; then

The coefficient function

And the displacement vector DM _is provided to the image sensor to displace and receive the image.

Images