TWI863206B - Device for image sensing and the sensing method thereof - Google Patents

Abstract

The invention provides an image-sensing device and a sensing method thereof. The image sensing device includes an optical sensor capable of sensing changes in light intensity and an adjustable optical path medium; through the displacement of the optical path medium, for a static scene, the projection position of the image thereof on the sensor can be shifted so that the optical sensor could display the image at the place where the light intensity changes. Therefore, the present invention can overcome the problem that the static scene cannot be detected without light intensity diversity.

Description

Image sensing device and sensing method thereof

The present invention relates to an image sensing device and a sensing method thereof, and in particular to an image sensing device and a sensing method thereof that can compare the light intensity changes of different images after capturing them to display the images at the light intensity changes.

The general image sensing system performs relevant calculations and processing on the pixels of the captured screen image. The captured image can be color pixels or pure black and white pixels. However, especially for the capture and recording of color pixels, the amount of recorded data is very large, which has a great impact on the efficiency of subsequent image processing. In addition to the front-end sensing components, the performance of the back-end processing components must be higher than usual, so the cost is also higher.

Compared to general image sensing, which records images in units of frames, the so-called dynamic vision sensor (DVS) records and compares in units of events. After comparing the images captured before and after, if there is a change in light intensity, it will be recorded, and if there is no change, it will not be recorded. Because it only processes pixels with changes in light intensity (light intensity), the overall amount of data is much less than that of traditional image sensors, and therefore when transmitted to the back-end processor, the response speed is fast and the delay is low. In addition, because the DVS sensor only responds to changes in light intensity, it will not be affected in environments with different brightness, and the dynamic range is large. Because fewer pixels need to be processed, the amount of data processed is smaller, and it can achieve a low-power working mode.

However, the DVS sensor can only process pixels with changing light intensity in the image. If the scene on the screen is fixed, the light intensity will not change. At this time, the DVS sensor will not record and generate the image of the static scene. The screen it generates will be black, and the static scene cannot be seen. In addition, if it is necessary to detect dynamic scenes at the same time, it can only display the image of the moving scene, and it cannot know the relative position of the static scene. Therefore, the use of DVS sensors to sense images alone still has considerable limitations.

Although there is a method of vibrating the DVS sensor in the previous technology, so that the DVS sensor can receive the image of relative displacement due to the vibration displacement and display the difference pixel, this vibration method will generate noise on the one hand, affecting the quietness of the environment, and may also record this noise, and on the other hand, the device will be damaged after long-term vibration, thereby increasing the maintenance cost of the equipment or the precision of the equipment itself. In addition, there is also a technology for adjusting the amount of light entering the DVS sensor, hoping to reproduce the image by changing the light intensity. However, this method makes the DVS imaging of dynamic scenes and static scenes very different in nature, increasing the difficulty of image recognition and the size of the required database, and is not good for image processing performance.

Therefore, if a device can be developed that allows static image projection to change easily and be captured and displayed under the existing DVS sensor equipment, the aforementioned problems of the prior art regarding the degradation of the DVS sensor's suitability, or the difficulty and poor performance in image processing can be avoided.

In view of the above-mentioned problems faced by the prior art, such as poor durability of the mechanism resulting in increased maintenance costs, and increased difficulty in image comparison and analysis resulting in poor performance, one of the purposes of the embodiments of the present invention is to provide a better image sensing device and a sensing method thereof, which can easily sense and display static scenes; in addition, one of the other purposes of the embodiments of the present invention is to provide an image sensing device and a sensing method thereof with better mechanism stability and higher performance in image recognition and comparison.

In order to achieve the above-mentioned purpose, in one embodiment of the present invention, an image sensing device is provided for simultaneously sensing static or moving scenes in an area. The image sensing device comprises: an optical sensor, which can sense the scene image projected in the area and analyze the light intensity change of the scene image. When there is a light intensity change, the image at the light intensity change can be recorded and displayed; an optical diameter adjustment mechanism, which is provided with an optical diameter medium. Before the optical diameter adjustment mechanism is not actuated, the optical diameter medium, the optical sensor and the area are located on a reference axis, wherein the optical diameter medium is arranged on the reference axis and is parallel to the reference axis. The optical sensor is located on a base with a first distance between it, the optical medium and the scene are located at a second distance between them, and a first plane is formed at a position perpendicular to the base and the reference axis; wherein, at a first time point, the optical medium is located on the base, and the optical sensor receives and analyzes a first image; and at a second time point, the optical medium is moved from the base by an offset distance, so that the projection position of the scene on the optical sensor is displaced, and the optical sensor receives and analyzes a second image whose light intensity changes after the displacement of the first image.

In one embodiment of the present invention, the image sensing device, wherein the optical medium can cause the projection position of the scene image on the optical sensor to shift by refraction or reflection. In one embodiment of the present invention, the optical medium can be a prism, a concave-convex lens, a lens set, a reflector, or any combination thereof, but is not limited thereto, and other mechanisms that can change the path of light can be used.

In one embodiment of the present invention, the image sensing device, wherein the optical medium can be translated on the first plane by the offset distance, and can be translated upward, downward, leftward, rightward, or in other directions from the base position on the first plane.

In another embodiment of the present invention, the image sensing device, wherein the optical path medium can be translated on the reference axis close to or away from the optical sensor by the offset distance, for example, it can be translated to the second plane close to the optical sensor, or translated to the third plane away from the optical sensor.

In another embodiment of the present invention, the image sensing device, wherein the offset distance can be the displacement distance of the optical path medium on the base at a predetermined angle toward the optical sensor.

In another embodiment of the present invention, the optical diameter adjustment mechanism may be further provided with a brake, by which the optical diameter medium can be moved as described above. In one embodiment, the brake may be a piezoelectric actuator or a moving mechanism with a motor, or other mechanisms that can provide moving motion, but is not limited thereto.

In another embodiment of the present invention, an automatic zoom lens may be further provided between the optical sensor and the optical lens medium.

In another embodiment of the present invention, a method for image sensing is provided for sensing a static or moving scene in an area, comprising: providing an optical sensor that can sense the scene image projected in the area and analyze the light intensity change of the scene image. When there is a light intensity change, the image at the light intensity change can be recorded and displayed; an optical path medium is set between the scene and the optical sensor, and the optical path medium is set at a At a first time point, the scene is projected onto the optical sensor through the optical medium, and the optical sensor receives and analyzes a first image; and at a second time point, the optical medium is moved from the base by an offset distance, so that the projection position of the scene on the optical sensor is displaced, and the optical sensor receives and analyzes a second image that is a change in image light intensity after the first image is displaced.

In another embodiment of the present invention, the image sensing method, wherein the optical medium can cause the projection position of the scene image on the optical sensor to shift by refraction or reflection. In one embodiment of the present invention, the optical medium can be a prism, a concave-convex lens, a lens set, a reflector, or any combination thereof, but is not limited thereto, and other mechanisms that can change the path of light can be used.

Through the embodiment of the present invention, a movable optical medium is added to the existing image optical sensor. By moving the optical medium, the projected image is displaced, thereby achieving the purpose of sensing and displaying static scenes. In addition, through the embodiment of the present invention, the optical sensor is not affected by physical forces, the device is more stable, and because the amount of light entering the sensed image is consistent, when performing image comparison and processing, there is no need for a large amount of data to compensate, and better processing performance can be achieved, thereby solving and improving the shortcomings of the known technology.

10: Optical sensor

20: Aperture adjustment mechanism

21: Optical Path Medium

22: Brake

30: Image processing system

40: Auto zoom lens

A1: Datum axis

B: Scenery

G1: First projection position

G2: Second projection position

J1, J2, J3, J4: offset

L1: First distance

L2: Second distance

S0: base position

S1: First plane

S2: Second plane

S3: The third plane

Figure 1 is a schematic diagram of the configuration of an embodiment of the image sensing device of the present invention.

Figure 2 is a schematic diagram of the configuration and operation of an embodiment of the image sensing device of the present invention.

Figure 3 is a schematic diagram of another embodiment of the image sensing device of the present invention.

Figure 4 is a schematic diagram of the image sensing device in one embodiment of the present invention.

The following will further illustrate the implementation of the present invention. The following examples and drawings are used to illustrate the present invention and are not used to limit the scope of the present invention.

Please refer to FIG. 1, which is a schematic diagram of the configuration of the image sensing device of the present invention. Since FIG. 1 is only a schematic diagram, the configuration relationship and structure of each component are only for illustration, and the actual mechanism size and configuration are not limited to the diagram. The image sensing device of the present invention includes an optical sensor 10, an optical diameter adjustment mechanism 20 and an image processing system 30. The optical sensor 10 can sense the change in light intensity of a scene B in a preset area, that is, the image of the scene B is projected onto the optical sensor 10 after passing through the optical medium 21 provided by the optical adjustment mechanism 20. When the optical medium 21 is displaced by an offset distance (e.g., offset distance J1-J3), the projection position of the scene B on the optical sensor 10 is also displaced, so that the optical sensor 10 can capture pixels with changed light intensity in the previous and next images, and display the analyzed image after calculation and analysis by the image processing system 30.

Please continue to refer to Figure 1. The optical sensor 10, the optical medium 21, and the scene B in the embodiment of the present invention are located on the reference axis A1 before the optical diameter adjustment mechanism 20 is actuated. The optical medium 21 is located on the reference axis A1 at a base position S0 with a first distance L1 from the optical sensor 10, and the optical medium 21 and the scene B are separated by a second distance L2. When sensing, the position of the optical sensor 10 is fixed, and only the optical diameter adjustment mechanism 20 drives the optical medium 21 to move, that is, the sum of the first distance L1 and the second distance L2 is fixed. In addition, a first plane S1 is formed at the base position S0 perpendicular to the reference axis A1.

The "optical sensor" described herein refers to a device that can sense the projected scene image in a certain area and analyze and compare the light intensity changes before and after the scene image. When there is a light intensity change, the image at the light intensity change can be recorded and displayed. In this embodiment, the optical sensor 10 can be a dynamic vision sensor (DVS) or an optical sensor of similar principle, but is not limited to this. The optical path adjustment mechanism 20 includes an optical path medium 21, and is provided with a brake 22 for controlling and adjusting the displacement of the optical path medium 21. In this embodiment, the displacement of the optical path medium 21 causes the optical path of the scene B image passing therethrough to change, so that the B image is projected at different positions on the optical sensor 10 at different incident angles. Therefore, any mechanism or device that can change the optical diameter can be used, so refraction or reflection can be used. If the refraction principle is used, a prism, a concave-convex lens, or a combination of multiple lenses can be used; if the reflection principle is used, a reflector can be used. The aforementioned arrangement of the optical medium 21 is only an example and is not limited to this. In addition, in order to match the overall arrangement of the sensor, the optical medium 21 can also be a combination of the aforementioned optical lenses.

In addition, the brake 22 can be a piezoelectric actuator, which can use the characteristics of piezoelectric ceramics that they will stretch or contract when voltage is applied to pull or push the optical medium 21 connected to it to move. The brake 22 can also be a general stepping motor or a similar mechanism to pull or push the displacement of the optical medium 21, but the mechanism of the brake is not limited to this. However, the use of piezoelectric brakes can perform micro-distance and more precise movements.

Please refer to FIG. 1 and FIG. 2 together, and the following describes the sensing method of the image sensing device of the embodiment of the present invention. At a first time point, for example, when the optical diameter adjustment mechanism 20 is not actuated, or when it is reset after actuation, the optical diameter medium 21 provided on the optical diameter adjustment mechanism 20 is located at the base position S0 on the reference axis A1. At this time, the image of the scene B is projected through the optical diameter medium 21 to the first projection position G1 on the optical sensor 10 (please refer to FIG. 4). The optical sensor 10 receives and displays the first image after being analyzed by the image processing system 30 (not shown in the figure). Then at the second time point, the optical diameter adjustment mechanism 20 is activated, and the optical diameter medium 21 is moved at the same time. When moving, it can be displaced by a preset offset distance, and its displacement direction can be on the first plane S1, from the base position S0 to the top, bottom, left or right, or other directions by an offset distance J1, so that the projection position of the scene B on the optical sensor 10 is displaced to the second projection position G2 (please refer to Figure 4), and at this time, the optical sensor 10 can analyze the second image (not shown in the figure) after receiving the first image due to the change in image light intensity after the displacement. At a subsequent time point, the optical diameter medium 21 is reset to the original base position S0, and at this time, the image with the changed light intensity after the projection position displacement can be displayed. Thus, the purpose of displaying static scenes can be achieved through repeated movements of the optical medium 21 such as displacement and reset.

In addition to the aforementioned displacement method, the optical medium 21 can also be displaced on the reference axis A1 by an offset distance J2 in the direction close to the optical sensor 10 (i.e., displaced to the second plane S2), or by an offset distance J3 in the direction of the scene B (i.e., displaced to the third plane S3), which can also make the first projection position G1 and the second projection position G2 fall at different positions. However, in order to sense the target scene, or to consider the subsequent image processing and image compensation, the preset distance of the offset distance needs to consider whether the projection of the optical medium 21 still falls within the visible range of the optical sensor 10 after this displacement.

In another embodiment of the present invention, please refer to FIG. 3. The optical sensor 10 can also be tilted or rotated a predetermined angle toward the optical sensor 10 on the reference axis A1, and displaced by an offset distance J4, so that the first projection position G1 and the second projection position G2 can fall at different positions, thereby achieving the purpose of the present invention. In addition, when the optical path medium 21 is displaced, its projection position on the optical sensor 10 will change, which sometimes affects the focus clarity of the original image. Therefore, an automatic zoom lens 40 can be further provided between the optical sensor 10 and the optical path medium 21 to automatically adjust the clarity of the image. In this embodiment, the automatic zoom lens can be a lens/lens group provided with a voice coil motor (VCM), but is not limited thereto.

The action of the optical diameter adjustment mechanism 20/optical diameter medium 21 in the embodiment of the present invention can be operated all the time, that is, while the optical sensor 10 is working, the optical diameter adjustment mechanism 20 continuously drives the optical diameter medium 21 to move from the base position S0 by an offset distance, and then returns to the base position S0. This action is repeated, and at this time, the static and dynamic images can be recognized and displayed in full view. In addition, the movement of the optical diameter adjustment mechanism 20/optical diameter medium 21 can also be paused for a predetermined interval before moving again, operating in an intermittent manner, that is, after it is displaced from the base position S0 and then reset, it waits for an interval before moving and resetting again. At this time, it still has no effect on dynamic images, but for static images, the image processing system 30 can perform image compensation for the interval time, that is, it can continuously display static scenes. In addition, when it is particularly applied to the monitoring of dynamic scenes relative to static scenes, the image processing system 30 can make a judgment (based on the data transmission amount of the optical sensor) and activate the optical diameter adjustment mechanism 20/optical diameter medium 21. That is, if the image processing system 30 determines that the image has not changed, the optical diameter adjustment mechanism 20/optical diameter medium 21 can be kept inactive for a period of time. When a new event is generated by the movement of the dynamic scene, the optical diameter adjustment mechanism 20/optical diameter medium 21 is activated to perform image recognition and make compensation. After the new event movement ends, the optical diameter adjustment mechanism 20/optical diameter medium 21 can be allowed to operate for a period of time to identify and compensate for subsequent images.

By using the embodiment of the present invention, a movable optical medium is added to the existing image optical sensor mechanism, and the optical path of the scene projection can be changed by moving the optical medium, so that the projection position of the scene on the optical sensor changes, and the purpose of sensing and displaying static scenes can be achieved.

10: Optical sensor

20: Aperture adjustment mechanism

21: Optical Path Medium

22: Brake

30: Image processing system

40: Auto zoom lens

A1: Datum axis

B: Scenery

J1, J2, J3: offset

L1: First distance

L2: Second distance

S0: base position

S1: First plane

S2: Second plane

S3: The third plane

Claims (10)

An image sensing device for sensing static or moving objects in an area, comprising: An optical sensor can sense the scene image projected in the area and analyze the light intensity change of the scene image. When there is a light intensity change, it can record and display the image at the light intensity change location; An optical diameter adjustment mechanism is provided with an optical diameter medium. Before the optical diameter adjustment mechanism is not actuated, the optical diameter medium, the optical sensor and the region are located on a reference axis, wherein the optical diameter medium is located on a base position on the reference axis at a first distance from the optical sensor, and a first plane is formed at a position perpendicular to the reference axis; Wherein, at the first time point, the optical path medium is located on the base, and at this time the optical sensor receives and analyzes the first image; and At the second time point, the optical medium is moved from the base by an offset distance, so that the projection position of the scene on the optical sensor is displaced. At this time, the optical sensor receives and analyzes a second image that is a change in image light intensity after the first image is displaced. An image sensing device as described in claim 1, wherein the optical path medium causes a displacement of the projection position of the scene image on the optical sensor by refraction or reflection. An image sensing device as described in claim 1, wherein the optical path medium is translated on the first plane by the offset distance. An image sensing device as described in claim 1, wherein the optical path medium is translated on the reference axis toward or away from the optical sensor by an offset distance. An image sensing device as described in claim 1, wherein the offset distance is the displacement distance of the optical path medium on the base at a predetermined angle toward the optical sensor. An image sensing device as described in any one of claims 1 to 5, wherein the optical path medium is a prism, a concave-convex lens, a lens set, a reflector, or any combination thereof. An image sensing device as described in any one of claims 1 to 5, wherein the optical diameter adjustment mechanism is further provided with a brake, and the brake is capable of moving the optical diameter medium. An image sensing device as described in any one of claims 1 to 5, wherein an automatic zoom lens is further provided between the optical sensor and the optical lens medium. A method of image sensing for sensing static or moving objects in an area, including: Provide an optical sensor that can sense the scene image projected in the area and analyze the light intensity change of the scene image. When there is a light intensity change, the image at the light intensity change can be recorded and displayed; An optical path medium is arranged between the scene and the optical sensor, and the optical path medium is arranged on a base; At a first time point, the scene is projected onto the optical sensor through the optical path medium, and the optical sensor receives and analyzes the first image; and At the second time point, the optical medium is moved from the base by an offset distance, so that the projection position of the scene on the optical sensor is displaced. At this time, the optical sensor receives and analyzes a second image that is a change in image light intensity after the first image is displaced. The method of image sensing as described in claim 9, wherein the optical path medium causes the projection position of the scene image on the optical sensor to shift by refraction or reflection.

Images