CN115334258B - Image motion compensation device and method based on rolling shutter electronic shutter area array detector - Google Patents

Abstract

The invention discloses an image motion compensation device and method based on a rolling shutter electronic shutter area array detector, wherein the image motion compensation device comprises a mechanical image motion compensation mechanism, the rolling shutter electronic shutter area array detector in a working mode and a focal plane slit window; the mechanical image motion compensation mechanism can realize an image stabilizing function, so that the imaging position of the moving target and the area array detector are relatively static; the area array detector is used for converting the target optical signal into an electric signal to generate image information; the focal plane slit window is arranged close to the detector photosurface and parallel to the detector photosurface, the focal plane slit window at least comprises 1 light-transmitting area and 1 light-shielding area, the boundary line between the light-shielding area and the light-transmitting area is parallel to the pixel row direction of the area array detector, and the parallel detector surface is driven by a motor to precisely reciprocate according to a certain frequency during the progressive exposure of the area array detector. The invention solves the problem of high-speed image motion compensation, fully utilizes the advantages of the rolling shutter electronic shutter, and has high signal-to-noise ratio, large dynamic range and no deformation of images.

Description

An image motion compensation device and method based on rolling shutter electronic shutter area array detector

Technical field

The invention belongs to the technical field of optical imaging and measurement. Specifically, it is an image motion compensation device and method based on a rolling shutter electronic shutter area array detector.

Background technique

When the camera is imaging a moving target, due to the relative motion between the camera and the target during the exposure time, the image of the target on the focal plane is not static, and a moving image shift will occur. The moving image shift causes the images of different targets to overlap with each other, resulting in an image Blurry, contrast and resolution loss.

Commonly used image motion compensation methods include: 1. Electronic image motion compensation method, which mainly includes linear array TDI image motion compensation and compensation based on full frame/frame transfer CCD; 2. Mechanical compensation method, which mainly includes mirror compensation and moving focal plane compensation. , moving lens or lens compensation, rotating double wedge compensation, etc.; 3. Image compensation, etc.

Regardless of the image motion compensation method, when traditional area array detectors perform image motion compensation, they either use full frame/frame transfer CCD and use mechanical/electronic shutters (similar to global electronic shutters, all pixels in the image plane are exposed simultaneously) to achieve Electronic TDI compensation function; or choose an inter-line transfer detector with a global electronic shutter and use mechanical compensation method for image motion compensation.

Area array image detectors are generally divided into full frame transfer CCD, frame transfer CCD, and interline transfer CCD/CMOS according to their working principles. Full frame transfer CCD requires an additional mechanical shutter to work, and the other two types are electronic shutters. Electronic shutter is divided into two types: global shutter (Global Shutter) and rolling shutter (Rolling Shutter). A global shutter detector exposes all pixels simultaneously, while a rolling shutter detector exposes each pixel row by row. When a rolling shutter images a high-speed moving target, the target image is distorted and deformed due to the progressive exposure characteristics. Therefore, all current methods of image motion compensation using area array detectors use the global electronic shutter exposure mode (mechanical shutters also similar).

Compared with the global electronic shutter, in the rolling shutter electronic shutter operating mode, the area array detector has very little noise and a larger dynamic range. It has been widely used in shooting situations of stationary or slow-moving scene targets. However, the rolling shutter electronic shutter detector gradually Sequential exposure of rows will cause serious deformation of high-speed moving target images, and is rarely used in high-speed motion situations. Precisely because all pixels are not exposed at the same time in the rolling shutter working mode, it is rarely used in high-speed image motion compensation.

Contents of the invention

The purpose of the present invention is to provide an image motion compensation device and method based entirely on a rolling shutter electronic shutter area array detector, which overcomes the problem that the rolling shutter electronic shutter area array detector cannot be used for high-speed motion image motion compensation, and not only solves High-speed image motion compensation problem, and full use of the advantages of the rolling electronic shutter, the camera has a high signal-to-noise ratio, a large dynamic range, and no image deformation.

In order to achieve the above objects, the technical solutions adopted by the present invention are as follows:

An image motion compensation device based on a rolling shutter electronic shutter area array detector, including a mechanical image motion compensation mechanism, an area array detector in a rolling shutter electronic shutter working mode, and a focal plane slit window;

The mechanical image motion compensation mechanism can realize the image stabilization function, so that the imaging position of the moving target and the area array detector are relatively stationary;

The area array detector is used to convert target optical signals into electrical signals to generate image information;

The focal plane slit window is set close to the photosensitive surface of the detector and is parallel to the photosensitive surface of the detector. The focal plane slit window includes at least one light-passing area and one light-shielding area, and the boundary between the light-shielding area and the light-passing area is Parallel to the row direction of the pixels of the area array detector, during the row-by-row exposure of the area array detector, driven by the motor, the parallel detector surface moves precisely back and forth at a certain frequency.

Furthermore, the area array detector does not have an electronic image motion compensation function.

Further, the area array detector is a black-and-white detector or a color detector.

Furthermore, the surface of the focal plane slit window should be dyed black.

According to the above image motion compensation method of the image motion compensation device based on the rolling shutter electronic shutter area array detector, the image motion compensation method includes the following steps:

Step 1: Before starting work, the focal plane slit window is at position 1. At position 1, the light area of the focal plane slit window ensures that the first j rows of pixels can be exposed;

Step 2: During image motion compensation, the mechanical image motion compensation mechanism is driven by the motor to keep the image of the moving target relatively stationary with the target surface of the area array detector;

Step 3: The rolling electronic shutter area array detector starts to expose line by line from the 1st line until the jth line is completed.

The light-passing area of the focal plane slit window is located at position 1. The detector row direction performs imaging exposure according to the exposure time Δt. When the exposure of the jth row of N in the previous frame of the detector is completed, the first row of N+1 in the next frame also starts to be synchronized. exposure;

Step 4: When the detector completes the exposure of the first j rows of the current frame, the focal plane slit window quickly moves to position 2, and the detector continues to expose the j+1 to Nth rows of the current frame.

The light-passing area of the focal plane slit window quickly and precisely shifts to position 2, blocking the pixels from the 1st to the jth row of the detector while ensuring that the j+1 to Nth rows of the detector can continue to be exposed. At this time, the detector is lowered The 1st line of N+1 of a frame also begins to be exposed. Due to light occlusion, there is no valid data;

Step 5: When all rows of the current frame N of the detector are exposed, if the i-th row of the detector's next frame N+1 is being exposed, i<j, then wait for the j-th row to be exposed. Otherwise, the focal plane slit window will be opened immediately. Returning to position 1, the image motion compensation mechanism also returns to the initial position before image motion compensation, preparing for image motion compensation and exposure of the next frame of image;

Step 6: Repeat steps 1 to 5. The focal plane slit window performs reciprocating translational motion at a certain frequency, thereby achieving image motion compensation for a rolling electronic shutter camera with a certain shooting frame rate.

Compared with the prior art, the significant advantages of the present invention are:

1. The present invention provides an image motion compensation device and method based on a rolling shutter electronic shutter area array detector, which overcomes the problem that the rolling shutter electronic shutter area array detector cannot be used for high-speed motion image motion compensation, and not only solves the problem of high-speed motion compensation Image motion compensation problem, and full use of the advantages of the rolling electronic shutter, the camera has a high signal-to-noise ratio, a large dynamic range, and no image deformation.

2. The image motion compensation device of the present invention has a simple structure, is easy to implement, and has good applicability. The invention expands the selection of array detectors in the field of image motion compensation, especially for ultra-large area array detection that only has a rolling electronic shutter exposure mode. It is of great significance to reduce the system volume and improve system performance.

3. In view of the detector exposure characteristics in the rolling electronic shutter mode, the present invention innovatively proposes a strategy of using the focal plane slit window to match the detector row exposure time, providing a new idea for image motion compensation.

4. The application of the focal plane slit window provides great convenience for later stitching of image motion compensation images.

Description of the drawings

Figure 1 is a working principle diagram of the exposure of a rolling shutter electronic shutter area array detector.

Figure 2 is a schematic diagram of the image motion compensation device based on the rolling shutter electronic shutter area array detector of the present invention.

Figure 3 is a working schematic of the focal plane slit window and a schematic diagram of the slit window of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with examples. It should be understood that the specific embodiments described here are only used to explain the present invention and are not intended to limit the present invention.

As shown in Figure 2, an image motion compensation device based on a rolling electronic shutter area array detector consists of a mechanical image motion compensation mechanism, an area array detector in rolling electronic shutter operating mode, a focal plane slit window and servo control System composition.

The mechanical image shift compensation mechanism uses mirrors, moving focal planes, etc. to achieve image stabilization, so that the imaging position of the moving target and the detector are relatively stationary.

The area array detector is used to convert the target optical signal into an electrical signal to generate image information.

The focal plane slit window is close to the photosensitive surface of the detector and is parallel to the photosensitive surface of the detector, and the boundary line between the light-shielding area and the light-passing area is parallel to the row direction of the detector pixels. During the line-by-line exposure of the detector, driven by the motor, the parallel detector surface moves precisely back and forth at a certain frequency.

Motion image shift compensation can be completed by combining the image stabilization of the mechanical image shift compensation mechanism, the progressive exposure of the rolling electronic shutter, and the exposure area control of the focal plane slit.

The mechanical image motion compensation mechanism can be implemented using any mechanical image motion compensation method introduced in the background art.

Further, the area array detector is an area array detector with a rolling electronic shutter operating mode;

Further, the area array detector is an ordinary imaging detector and does not have an electronic image motion compensation function;

Further, the area array detector is not limited to black and white detectors, but can be color detectors;

The focal plane slit window contains at least two areas, namely a light-passing area and a light-shielding area. The light-shielding area is only used for light blocking and does not have the function of a shutter;

Furthermore, when the first j rows of the detector are exposed, the size of the focal plane slit window should not block the first j rows of pixels; when the detector continues to expose the j+1 to Nth rows, the focal plane slit window should pass The light area is translated to ensure that it does not block the j+1 to Nth row of the detector. At the same time, the light blocking area of the focal plane slit should block the pixels in the 1st to jth row of the detector.

Further, the focal plane slit is located above the photosensitive surface of the detector, close to the detector, and parallel to the detector;

Further, the precise movement time of the focal plane slit is related to the frame rate of the selected rolling shutter electronic shutter area array detector;

Further, the moving direction of the focal plane slit moves horizontally along the detector surface and is parallel to the row direction of the detector's line-by-line exposure, and the moving direction is perpendicular to the row direction;

Furthermore, the focal plane slit window mainly controls the exposure area of the detector target surface.

Furthermore, the surface of the focal plane slit window should be dyed black.

The principle of rolling electronic shutter exposure is shown in Figure 1. Assume that the size of the detector target surface is N (rows) After the row starts to be exposed for a period of time, the second row starts to be exposed..., and finally the exposure of N rows of pixels is completed to form a frame of image, and the exposure time of each row of pixels is the same.

Due to the progressive exposure characteristics of the rolling electronic shutter, the exposure time of the current Nth frame image and the N+1th frame image of the next frame overlap, that is, the exposure of the jth row of the current Nth frame image (j<N) is completed When, the next N+1 frame image starts to be exposed again from the 1st line. Since the next frame image is different from the current frame image during image motion compensation, the rolling electronic shutter operating mode cannot be applied.

Combined with Figure 3, the image motion compensation principle of the rolling shutter electronic shutter area array detector of the present invention is:

Assume that when the exposure of the jth line of N in the current frame of the detector ends, the exposure of the 1st line of N+1 frame in the next frame of the detector begins. When the first j rows of the detector are exposed, the size of the light-passing area of the focal plane slit window should not block the first j rows of pixels; when the j+1 to N-th rows of the detector continue to be exposed, the size of the light-passing area of the focal plane slit window should not block the first j rows of pixels. Translate to ensure that the detector does not block the j+1 to Nth rows. At the same time, the focal plane slit light blocking area should block the 1st to jth rows of the detector pixels.

The length L of the light-passing area of the focal plane slit window should be greater than M×a, and the width H should be greater than Max(j, N-j), that is, j×a and (N-j)×a take the maximum value. The length L of the light-blocking area should be greater than M×a, and the width H should be greater than j×a.

During image motion compensation, the mechanical image motion compensation mechanism is driven by the motor to keep the image of the moving target relatively static with the detector target surface. The light-passing area of the focal plane slit window is located at position 1 to ensure that the front j rows of pixels can be exposed. , then the detector row direction performs imaging exposure according to the exposure time Δt in sequence. When the exposure of the jth row of N in the current frame is completed (the exposure of the 1st row of N+1 in the next frame begins), the light area of the focal plane slit window quickly and accurately translates Go to position 2, while blocking the pixels in the 1st to jth rows of the detector, while ensuring that the j+1 to Nth rows of the detector can continue to be exposed. When the exposure of all rows is completed, the focal plane slit returns to the initial position, and the image motion compensation mechanism returns to the initial position to prepare for the exposure of the next frame of image. The focal plane slit window performs reciprocating translational motion at a certain frequency, thereby achieving image motion compensation for a camera with a certain shooting frame rate.

The image motion compensation frame rate is smaller than the detector working frame rate, and the position of the jth line is related to the detector working frame rate.

The steps of the image motion compensation method based on the rolling shutter electronic shutter area array detector of the present invention are as follows:

Step 1: Before starting work, the focal plane slit window is at position 1;

The focal plane slit window is located at position 1, and its light-passing area ensures that the first j rows of pixels can be exposed.

Step 2: During image motion compensation, the mechanical image motion compensation mechanism is driven by the motor to keep the image of the moving target relatively stationary to the detector target surface.

Step 3: The rolling electronic shutter detector starts to expose row by row from row 1 until the exposure of row j is completed;

The light-passing area of the focal plane slit window is located at position 1. The detector row direction performs imaging exposure according to the exposure time Δt. When the exposure of the jth row of N in the previous frame of the detector is completed, the synchronization of the 1st row of N+1 in the next frame also begins. exposure.

Step 4: When the detector completes the exposure of the first j rows of the current frame, the focal plane slit window quickly moves to position 2, and the detector continues to expose the j+1 to Nth rows of the current frame;

The light-passing area of the focal plane slit window quickly and precisely shifts to position 2, blocking the pixels from the 1st to the jth row of the detector while ensuring that the j+1 to Nth rows of the detector can continue to be exposed. At this time, although the detector also begins to expose the first line of N+1 in the next frame, there is no valid data due to light obstruction.

Step 5: When all rows of the current frame N of the detector are exposed, if the i-th row (i<j) of the next frame N+1 of the detector is being exposed, wait for the j-th row to be exposed. Otherwise, the focal plane slit will be opened immediately. Quickly return to the initial position, the image motion compensation mechanism also returns to the initial position, preparing for image motion compensation and exposure of the next frame of image.

Step 6: Repeat steps 1 to 5. The focal plane slit window can perform reciprocating translational motion at a certain frequency, thereby achieving image motion compensation for a rolling electronic shutter camera with a certain shooting frame rate.

The image motion compensation device and method proposed by the present invention based on the rolling shutter electronic shutter area array detector make full use of the advantages of low detector noise and large dynamic range in the rolling shutter exposure mode, and apply it to high-speed motion image motion compensation. occasion. The invention uses a mechanical image shift compensation mechanism, an area array detector with a rolling electronic shutter, a focal plane slit window and other devices to combine the image stabilization of the mechanical image shift compensation mechanism, the progressive exposure of the rolling electronic shutter with the focal plane narrowing. The camera is combined with seam exposure area control to complete motion image motion compensation, which not only solves the problem of high-speed motion image motion compensation, but also makes full use of the advantages of the rolling electronic shutter. The camera has a high signal-to-noise ratio, a large dynamic range, and no image deformation.

The basic principles, main features and advantages of the present invention have been shown and described above. Those skilled in the industry should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions only illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will also have other aspects. Various changes and modifications are possible, which fall within the scope of the claimed invention. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims (4)

1. An image motion compensation method of an image motion compensation device based on a rolling shutter electronic shutter area array detector, wherein the device comprises a mechanical image motion compensation mechanism, an area array detector and a focal plane slit window;

the mechanical image motion compensation mechanism can realize an image stabilizing function, so that the imaging position of the moving target and the area array detector are relatively static;

the area array detector is used for converting a target optical signal into an electric signal to generate image information;

the focal plane slit window is arranged close to the photosensitive surface of the area array detector and is parallel to the photosensitive surface of the area array detector, the focal plane slit window at least comprises 1 light-transmitting area and 1 light-shielding area, the boundary line between the light-shielding area and the light-transmitting area is parallel to the pixel row direction of the area array detector, and during the progressive exposure of the area array detector, the focal plane slit window is driven by a motor to precisely reciprocate parallel to the photosensitive surface of the area array detector according to a certain frequency;

characterized in that the method comprises the steps of:

step 1: before starting to work, the light transmission area of the focal plane slit window is positioned at the position 1, and the light transmission area of the focal plane slit window ensures that the front j rows of pixels can be exposed;

step 2: when in image motion compensation, the mechanical image motion compensation mechanism is driven by the motor to keep the image of the moving target relatively static to the photosensitive surface of the area array detector;

step 3: the area array detector starts to expose from the 1 st row line by line until the j th row exposure is completed,

the light transmitting area of the focal plane slit window is positioned at the position 1, imaging exposure is sequentially carried out on the pixel row direction of the area array detector according to the exposure time delta t, and when the j row exposure of the current frame of the area array detector is completed, the 1 st row of the next frame is synchronously exposed;

step 4: when the j rows before the current frame of the area array detector are exposed, the light-passing area of the slit window of the focal plane is rapidly moved to the position 2, the area array detector continues to expose the j+1 rows to the M rows of the current frame,

the light-passing area of the focal plane slit window rapidly and precisely translates to a position 2, and when the 1 st row to the j th row of pixels of the area array detector are shielded, the j+1st to the M th rows of the area array detector can be ensured to be continuously exposed, and at the moment, the 1 st row of the next frame of the area array detector is also exposed, and no effective data exists due to light shielding;

step 5: when the exposure of all rows of the current frame of the area array detector is completed, if the i row of the next frame of the area array detector is being exposed, i is less than j, waiting for the completion of the j row exposure, otherwise, immediately returning the light passing area of the focal plane slit window to the position 1, and returning the image motion compensation mechanism to the initial position before the image motion compensation to prepare the image motion compensation and exposure of the next frame of image;

step 6: repeating the steps 1-5, and making reciprocating translational motion of the light-passing area of the focal plane slit window according to a certain frequency, thereby realizing the image motion compensation of the rolling shutter electronic shutter camera with a certain shooting frame frequency.

2. The image motion compensation method of an image motion compensation apparatus based on a rolling shutter electronic shutter area array detector according to claim 1, wherein the area array detector does not have an electronic image motion compensation function.

3. The image motion compensation method of an image motion compensation device based on a rolling shutter electronic shutter area array detector according to claim 1, wherein the area array detector is a black-and-white detector or a color detector.

4. The image motion compensation method of an image motion compensation device based on a rolling shutter area array detector according to claim 1, wherein the focal plane slit window surface is subjected to black dyeing treatment.