CN106303155B - Image processing method and image processing device - Google Patents

Abstract

The invention discloses an image processing method, which is applied to an image processing device, wherein the image processing device comprises a light source and an image sensor, and comprises the following steps: capturing a first image by an image sensor when the light source operates in a first mode; capturing a second image by the image sensor when the light source operates in a second mode; capturing a third image by the image sensor when the light source operates in the first mode; forming a mixed image by the first image and the third image; the mixed image is subtracted from the second image to obtain a target image without background noise. By this method, the target image with background noise removed can be obtained more accurately.

Description

Image processing method and image processing device

technical field

The present invention relates to an image processing method and an image processing device, in particular to an image processing method and an image processing device capable of obtaining a target image after removing background noise.

Background technique

In recent years, automatic cleaning devices (such as floor sweeping robots) have become more and more popular. Even if the user is not present, this type of device can still perform cleaning work by itself. This type of device can be connected to a base for charging when not cleaning, and will leave the base for cleaning when the scheduled cleaning time is up or when the environment is dirty. After cleaning, it will automatically return to the base for charging. Therefore, the automatic cleaning device must have a distance measuring function to measure the distance between the automatic cleaning device and peripheral objects. Otherwise, during the automatic cleaning work, objects may be hit and the automatic cleaning device may be damaged, or the objects may be damaged.

The automatic cleaning device usually includes a distance measuring device for measuring the distance. The distance measuring device can use various mechanisms to measure the distance. One such mechanism is the use of images to measure distances. Under this mechanism, the distance measurement device will include an image sensor to capture multiple images of the target object (such as a wall), and calculate the distance based on these images. For example, the distance can be calculated based on the distance, angle or deformation of objects in multiple images.

However, the captured image may be disturbed by the surrounding environment (such as ambient light), causing errors in calculating the distance. In order to improve this situation, the captured image will be corrected by performing a "removal of background noise" step, and the distance will be calculated using the corrected image. The usual practice is to first illuminate the target with a light source and capture image A, then capture image B in a state where no light is emitted, and subtract image B from image A to obtain the target image after removing background noise , and then use the target image after removing background noise to calculate the distance. However, this type of background noise removal mechanism may cause some problems when the automatic cleaning device is moving and the frame rate is low.

FIG. 1A is a schematic diagram of an automatic cleaning device moving away from a target W gradually in the prior art. In the example shown in FIG. 1A , the automatic cleaning device R gradually moves away from the target object W (such as a wall). Therefore, as shown in FIG. 1A , the image ranges captured by it will vary. When the automatic cleaning device R is at the positions P1, P2, and P3, the captured images are f1, f2, and f3 respectively, and when the automatic cleaning device R is at the positions P1, P3, the light source inside is turned on, while at the position At P2, the light source inside is off. Therefore, the image f2 is subtracted from the image f3 to obtain the target image after removing the background noise. However, because the image ranges captured by the automatic cleaning device R are different at positions P2 and P3, the image f2 will have less information than the image f3 (the oblique line in FIG. 1B ), and The objects Ob1, Ob2 may have different sizes. Therefore, wrong target images may be calculated when removing background noise.

The same problem may also occur when the automatic cleaning device moves or rotates relative to the object W (parallel or at an angle). 2A is a schematic diagram of an automatic cleaning device moving relative to an object W in the prior art, and FIG. 2B is a schematic diagram of how to obtain an object image after removing background noise in the example of FIG. 2A . As shown in FIG. 2B , the automatic cleaning device R moves relative to the target object W, and when it is at the positions P1 , P2 , and P3 , the captured images are f1 , f2 , and f3 , respectively. And when the automatic cleaning device R is at the positions P1 and P3, the light source inside is turned on, and when the automatic cleaning device R is at the position P2, the light source inside is turned off. Therefore, the image f2 is subtracted from the image f3 to obtain the target image after removing the background noise. However, because the images f2 and f3 contain different contents, as shown in Figure 2B, the image f2 contains the objects ob1 and ob2, but the image f3 only contains the object ob2. Therefore, if the images f2 and f3 are subtracted, a false background decluttering result will be obtained. The image of the target after information will affect the calculation of the distance. When the automatic cleaning device rotates, the situation as shown in FIG. 2A and FIG. 2B may also occur.

To sum up, if the background noise removal calculation method of the prior art is used, the movement of the automatic cleaning device will result in wrong target image after background noise removal, which will affect the distance calculation. Such problems will be more obvious when the automatic cleaning device moves fast or the frame rate (ie image capture frequency) is low.

Contents of the invention

Therefore, an object of the present invention is to provide an image processing method capable of calculating a target image after background noise has been correctly removed.

Another object of the present invention is to provide an image processing device capable of calculating a target image after background noise has been correctly removed.

An embodiment of the present invention discloses an image processing method, which is implemented on an image processing device, and the image processing device includes a light source and an image sensor. The image processing method includes: capturing a first image with an image sensor when the light source is operating in a first mode; capturing a second image with the image sensor when the light source is operating in a second mode; In the first mode, a third image is captured by the image sensor; a mixed image is formed by the first image and the third image; the mixed image is subtracted from the second image to obtain a target image after background noise is removed.

Another embodiment of the present invention discloses an image processing device, which includes a light source, an image sensor, and an image computing unit. When the light source operates in a first mode, the image sensor captures a first image; when the light source operates in a second mode, the image sensor captures a second image; when the light source operates in the first mode, The image sensor captures a third image. The image calculation unit forms a mixed image from the first image and the third image, and subtracts the mixed image from the second image to obtain a target image after removing background noise.

The background noise removal calculation method provided by the present invention can avoid calculating the wrong target image after background noise removal due to the movement of the automatic cleaning device R in the prior art, and then calculate the correct distance.

Description of drawings

FIG. 1A is a schematic diagram of an automatic cleaning device gradually moving away from a target object in the prior art;

FIG. 1B is a schematic diagram of how to obtain a target image after removing background noise in the example of FIG. 1A;

FIG. 2A is a schematic diagram of an automatic cleaning device moving relative to an object in the prior art;

FIG. 2B is a schematic diagram of how to obtain a target image after removing background noise in the example of FIG. 2A;

3 and 4 illustrate schematic diagrams of an image processing method according to an embodiment of the present invention;

5 and 6 illustrate schematic diagrams of an image processing method according to an embodiment of the present invention;

FIG. 7 illustrates a flowchart of an image processing method according to an embodiment of the present invention;

FIG. 8 is a block diagram of an image processing device according to an embodiment of the invention.

Explanation of reference numbers:

R Automatic cleaning device

f1, f2, f3 images

ob1, ob2 objects

I1, Ia, Ib, Ic, Id Image area

fm mixed video

701-709 steps

801 Image processing device

803 image sensor

805 light source

807 light source controller

809 image computing unit

811 distance calculation unit

The realization of the purpose of the present invention, functional characteristics and advantages will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

The content of the present invention will be described below with different embodiments, but please note that the following embodiments are only for illustration, and do not limit the scope of the present application only to the following embodiments.

3 and 4 illustrate schematic diagrams of an image processing method according to an embodiment of the present invention. Please note that the embodiment shown in FIG. 3 and FIG. 4 corresponds to the movement method shown in FIG. 1A , so please refer to FIG. 1A , FIG. 3 , and FIG. 4 to better understand the present invention. FIG. 3 shows schematic diagrams of images f1 , f2 , f3 captured by the automatic cleaning device R at different positions P1 , P2 , P3 . And when the images f1 and f3 are captured, the light source in the automatic cleaning device R is in a first mode, and when the image f2 is captured, the light source in the automatic cleaning device R is in a second mode. In one embodiment, the light source does not emit light to illuminate the target W in the first mode, while the light source emits light in the second mode. In another embodiment, the light source emits light in the first mode, and the light source does not emit light to illuminate the target W in the second mode.

As shown in FIG. 3 , compared to the image f2 , the image f1 also has the content of the image area I1 , but lacks the content of the image areas Ia and Ib. Compared with the image f2, the image f3 has the contents of the image regions I1, Ia and Ib, but has more contents of the image regions Ic and Id. Regardless of whether the image f2 is directly subtracted from the image f1 or the image f3, an erroneous target image after background noise removal will be obtained.

Therefore, the image f3 and the image f1 are first synthesized to form a mixed image, and the mixed image is subtracted from the image f2 to obtain the target image after removing background noise. FIG. 4 illustrates an exemplary embodiment of the blended image fm. In this embodiment, the mixed image fm includes the content of the image area I1 of the image f1, and includes the contents of the image areas Ia and Ib of the image f3. That is to say, the mixed image fm includes all the contents of the image f1 and only a part of the image f3, and the size of the image f2 is equal to the size of the mixed image fm. The corresponding position of the image f2 and the object W is the same as the corresponding position of the mixed image fm and the object. Since the image f2 differs from the images f1 and f3 in different directions, for example, it has a negative difference with the image f1 and a positive difference with the image f3, so if the content of the image f1 is replaced by part of the content of the image f3 to form a mixture Image, the differences can be offset each other to obtain a more accurate target image after removing background noise. However, please note that the embodiment in FIG. 4 is only for illustration, and all relevant changes according to the teaching of the embodiment in FIG. 4 should fall within the scope of the present invention.

5 and 6 are schematic diagrams of an image processing method according to an embodiment of the present invention, which correspond to the moving manner of FIG. 2A in this case, and can also correspond to the situation when the automatic cleaning device R rotates. FIG. 5 shows schematic diagrams of images f1 , f2 , f3 captured by the automatic cleaning device R at different positions P1 , P2 , P3 . And when the images f1 and f3 are captured, the light source in the automatic cleaning device R is in a first mode, and when the image f2 is captured, the light source in the automatic cleaning device R is in a second mode. In one embodiment, the light source does not emit light to illuminate the target W in the first mode, while the light source emits light in the second mode. In another embodiment, the light source emits light in the first mode, and the light source does not emit light to illuminate the target W in the second mode.

As shown in FIG. 5 , due to the movement of the automatic cleaning device R, the images f1 , f2 , and f3 contain different contents. In detail, the image f1 only contains the object ob1, the image f2 contains the objects ob1 and ob2, and the image f3 only contains the object ob2, no matter whether the image f2 is subtracted from the image f1 or the image f3, the false background noise will be obtained the target image after. Therefore, in this embodiment, a part of the image f1 and a part of the image f3 are used to form a mixed image. As shown in Figure 6, the mixed image fm includes the right half of the image f1 and the left half of the image f3, so the mixed image fm includes the objects ob1 and ob2, so after subtracting the mixed image fm from the image f2, a more correct The target image after removing background noise. Since the part of image f1 and image f3 to be taken to form a mixed image is related to the moving direction of the automatic cleaning device R, in one embodiment, it is determined to use the front and rear images according to the moving direction of the automatic cleaning device R That part of it to generate a blended image.

In one embodiment, the aforementioned steps of generating images f1, f2, f3, the step of generating mixed images, and the step of calculating the target image after removing background noise will be performed when the automatic cleaning device R is moving, or as shown in FIG. 1A , movement as shown in Figure 2A, or when the distance between the automatic cleaning device R and the target changes, or when it rotates, it will be executed. That is to say, the automatic cleaning device R does not generate mixed images as described in the foregoing embodiments when it is stationary, thereby saving power.

Please note that the background noise-removed target image obtained by the above method is not limited to be used for distance measurement, and can also be used for other purposes. Moreover, this method is not limited to using three consecutive images. Therefore, according to the aforementioned embodiments, an image processing method for obtaining a target image after removing background noise can be obtained. The method is implemented on an image processing device, and the image processing device includes a light source and an image sensor. This image processing method includes the steps shown in Figure 7:

Step 701

When the light source is operating in a first mode, a first image (such as f1 ) is captured by the image sensor. In one embodiment, the first image includes an image of at least a part of the object (such as a wall).

Step 703

When the light source is operating in a second mode, a second image (for example, f2 ) is captured by the image sensor. In one embodiment, the second image includes an image of at least a part of the target object.

Step 705

When the light source operates in the first mode, capture a third image (eg f3 ) with the image sensor. In one embodiment, the third image includes an image of at least a part of the target object.

Step 707

A mixed image (eg fm) is formed with the first image and the third image.

Step 709

The blended image is subtracted from the second image to obtain a target image with background noise removed.

FIG. 8 is a block diagram of an image processing device according to an embodiment of the invention. In this embodiment, the image processing device 801 is disposed in the automatic cleaning device R, but it is not limited thereto. As shown in FIG. 8 , the image processing device 801 includes an image sensor 803 , a light source 805 , a light source controller 807 and an image computing unit 809 . The light source 805 is controlled by the light source controller 807 to emit or not emit light. The image sensor 803 is used to capture the images of the light source 805 in different modes as described above, and the image calculation unit 809 calculates the target image after the background noise is removed according to the images captured by the image sensor 803 as in the previous embodiment, and then the target After the image is corrected, the corrected image CF is sent to the distance calculation unit 811 . The distance calculation unit 811 calculates the distance between the automatic cleaning device R and the target object according to the corrected images CF, and the distance calculation unit 811 can also be located in the image processing device 801 . However, please note that the embodiment in FIG. 8 is only used as an example, and is not intended to limit the present invention. Each component can be integrated with each other or divided into multiple components.

To sum up, the background noise removal calculation method provided by the present invention can avoid calculating the wrong target image after background noise removal due to the movement of the automatic cleaning device R in the prior art, and then calculate the correct distance.

The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the present invention.

Claims (23)

1. a kind of image treatment method, which is characterized in that be implemented on an image processor, which includes one Light source and an image sensor, which includes:

When the light source operates under a first mode, one first image is captured with the image sensor；

When the light source operates under a second mode, one second image is captured with the image sensor；

When the light source operates under the first mode, a third image is captured with the image sensor；

A composite image is formed with first image and the third image, which contains all first images Content and the third image only a part of content, alternatively, to contain first image only a part of interior for the composite image Hold and the third image only a part of content；And

The target image after background noise is removed with second image and the composite image to obtain one.

2. image treatment method as described in claim 1, which is characterized in that second image is after capturing first image It is subtracted, and the third image is subtracted after capturing second image.

3. image treatment method as described in claim 1, which is characterized in that the light source operates on first mode Shi Huifa Light, the light source do not shine when operating on the second mode.

4. image treatment method as described in claim 1, which is characterized in that the light source operates on first mode Shi Bufa Light can shine when the light source operates on the second mode.

5. image treatment method as described in claim 1, which is characterized in that further include:

The distance of an automatic cleaning apparatus and an object is calculated according to the target image gone after background noise.

6. image treatment method as claimed in claim 5, which is characterized in that first image, second image, the third shadow As, the composite image and this to remove the target image after background noise be the generation when the automatic cleaning apparatus is mobile.

7. image treatment method as claimed in claim 6, which is characterized in that first image, second image, the third shadow As, the composite image and this to remove the target image after background noise changed at a distance from the object in the automatic cleaning apparatus It is generated when change.

8. image treatment method as claimed in claim 6, which is characterized in that first image, second image, the third shadow As, the composite image and this to remove the target image after background noise be mobile relative to the object in the automatic cleaning apparatus When generate.

9. image treatment method as claimed in claim 6, which is characterized in that first image, second image, the third shadow As, the composite image and this to remove the target image after background noise be the production when the automatic cleaning apparatus executes spinning movement It is raw.

10. image treatment method as described in claim 1, which is characterized in that the size of second image is equal to the mixing shadow The size of picture, and second image and the corresponding position of an object and the corresponding position phase of the composite image and the object Together.

11. image treatment method as described in claim 1, which is characterized in that with first image and the third image shape It is that the composite image is formed according to the moving direction of automatic cleaning apparatus at a composite image.

12. image treatment method as described in claim 1, which is characterized in that when the composite image contains first image Only a part of content and the third image only a part of content when, which contains the right one side of something of the first image Content and the left one side of something of the third image content.

13. a kind of image processor, the image processor is set in automatic cleaning apparatus, which is characterized in that packet Contain:

One light source；

One image sensor；

When the light source operates under a first mode, which captures one first image；One is operated in the light source When under second mode, which captures one second image；When the light source operates under the first mode, which is passed Sensor captures a third image；And

One image computing unit forms a composite image with first image and the third image, which contains Only a part of content or the composite image contains first shadow for the content of all first images and the third image As only a part of content and the third image only a part of content；And the composite image is subtracted to obtain with second image The target image after background noise is removed to one.

14. image processor as claimed in claim 13, which is characterized in that second image is to capture first image After be subtracted, and the third image is subtracted after capturing second image.

15. image processor as claimed in claim 13, which is characterized in that the light source operates on first mode Shi Huifa Light, the light source do not shine when operating on the second mode.

16. image processor as claimed in claim 13, which is characterized in that the light source operates on first mode Shi Bufa Light can shine when the light source operates on the second mode.

17. image processor as claimed in claim 13, which is characterized in that a metrics calculation unit is further included, according to this Target image after removing background noise calculates the distance of an automatic cleaning apparatus and an object.

18. image processor as claimed in claim 17, which is characterized in that first image, second image, the third Image, the composite image and the target image gone after background noise are the generations when the automatic cleaning apparatus is mobile.

19. image processor as claimed in claim 18, which is characterized in that first image, second image, the third Image, the composite image and the target image gone after background noise be in the automatic cleaning apparatus at a distance from the object It is generated when change.

20. image processor as claimed in claim 18, which is characterized in that first image, second image, the third Image, the composite image and the target image gone after background noise are to move in the automatic cleaning apparatus relative to the object It is generated when dynamic.

21. image processor as claimed in claim 18, which is characterized in that first image, second image, the third Image, the composite image and the target image gone after background noise are the productions when the automatic cleaning apparatus executes spinning movement It is raw.

22. image processor as claimed in claim 13, which is characterized in that the size of second image is equal to the mixing shadow The size of picture, and second image and the corresponding position of an object and the corresponding position phase of the composite image and the object Together.

23. image processor as claimed in claim 13, which is characterized in that the image computing unit is filled according to automated cleaning The moving direction set forms the composite image.