CN107169930A - Image processing method and device - Google Patents

Abstract

The invention discloses an image processing method and device to improve the accuracy of image processing. In some feasible implementations of the present invention, the method includes: acquiring the first color value of the reference background object in the reference image captured under the first lighting condition; acquiring the reference background object in the image to be processed captured under the second lighting condition The second color value of the background object and the third color value of the foreground object; according to the first color value, the second color value and the third color value, estimate the fourth color value of the foreground object photographed under the first lighting condition ; Use the fourth color value to perform processing on the foreground object, such as retrieval or query.

Description

Image processing method and device

technical field

The present invention relates to the technical field of image processing, in particular to an image processing method and device.

Background technique

In many applications, such as face recognition, iris recognition, and video surveillance systems, image processing is required, but the impact of illumination can significantly change the color values of image objects, which is critical for systems that rely on color values for retrieval or recognition of images. Validity is a very challenging problem.

In order to effectively remove the influence of illumination on image color values, many schemes for image illumination processing to restore image color have been proposed, including schemes based on retinal cortex theory (Retinex), dark channel processing technology, high dynamic range compression, low dynamic Range compression technology, schemes based on gradient theory, partial differential equations, and variational extremum theory, etc.

The principle of these schemes is generally to first estimate the illumination, and then estimate the intrinsic reflection coefficient of the image object by stripping the illumination value on the basis of obtaining the estimated illumination value, so as to obtain the intrinsic color of the object that is not affected by illumination. value. For example, the basic idea can be simply understood as: by estimating the illumination value L, according to the illumination mathematical model S=R*L or a more complex model S=R*L+N, to solve the reflection image R, where N represents noise , S represents the color value.

Practice has found that the above-mentioned various schemes need to be calculated on the basis of obtaining the estimated illumination value. However, the method for estimating illumination in the prior art is not ideal, and the obtained estimated illumination value is not accurate enough, which leads to the adoption of the above-mentioned various schemes. Accuracy is not good enough for image processing.

Contents of the invention

Embodiments of the present invention provide an image processing method and device, so as to improve the accuracy of image processing.

The first aspect of the embodiments of the present invention provides an image processing method, which can unify the object colors under different lighting conditions to the "same" reference lighting condition. After obtaining the first color value of the reference background object in the reference image, and obtaining the second color value of the reference background object and the third color value of the foreground object in the image to be processed captured under the second lighting condition, according to the The first color value, the second color value and the third color value estimate the fourth color value of the foreground object captured under the first lighting condition, so as to unify the color values of the foreground object under different lighting conditions to the first lighting condition Conditions, the fourth color value of the foreground object under the first lighting condition is subsequently used to process the foreground object, such as retrieval or query and image editing. Compared with the existing technology, this scheme does not need to strip the illumination value of the image, avoids the estimation of the illumination value, thereby avoids the error generated in the illumination estimation process, and solves the problem that it is difficult to obtain an accurate illumination estimation value in the prior art , which improves the accuracy of image processing.

The second aspect of the embodiments of the present invention provides an image processing device, which can unify the object colors under different lighting conditions under the "same" reference lighting conditions. In specific implementation, the device can include: an acquisition module, a color value A calculation module, and a processing module, wherein the acquisition module acquires the first color value of the reference background object in the reference image taken under the first lighting condition, and acquires the reference background object in the image to be processed taken under the second lighting condition The second color value of the second color value and the third color value of the foreground object; the color value calculation module calculates the fourth color value of shooting the foreground object under the first lighting condition according to the first, second and third; the processing module utilizes The fourth color value performs processing on the foreground object, such as retrieval or query and image editing. Compared with the existing technology, this scheme does not need to strip the illumination value of the image, avoids the estimation of the illumination value, thereby avoids the error generated in the illumination estimation process, and solves the problem that it is difficult to obtain an accurate illumination estimation value in the prior art , which improves the accuracy of image processing.

Based on the above method or device, some other feasible implementation modes may also be provided, such as:

Optionally, record the first, second, third and fourth color values as S ₁ , S ₂ , S ₃ , S ₄ respectively, then the formula S ₄ =S ₃ -(S ₂ -S ₁ ) to calculate the fourth color value. It should be noted that S ₁ , S ₂ , S ₃ , and S ₄ can be the logarithmic values of the original observed values, and the original observed values can be RGB values, that is, S ₁ , S ₂ , S ₃ , S ₄ can be the RGB value on the logarithmic domain.

Optionally, a reference image can be preset, the lighting condition of the reference image is defined as the first lighting condition, and the lighting condition of the image to be processed is defined as the second lighting condition, then the first lighting condition can be obtained from the reference image. The first color value of the reference background object under the lighting condition, and the second color value of the reference background object and the third color value of the foreground object under the second lighting condition are obtained from the image to be processed.

Optionally, acquire a plurality of images; acquire an image in the plurality of images as the reference image; acquire any image in the plurality of images except the reference image as the image to be processed ; The reference image includes the reference background object, and the image to be processed includes the reference background object and the foreground object.

Optionally, an image may be selected from the plurality of images as the reference image according to the brightness values and/or shooting time of the plurality of images, for example, the brightness value is within a set threshold range and/or Or an image whose shooting time is within a preset time range is set as a reference image.

Optionally, when the multiple images come from multiple cameras, the color calibration of the multiple cameras may be performed in advance; Refer to the illumination difference of the reference background object in the images captured by the respective cameras; remove the corresponding illumination difference from the fourth color value of the foreground object, and use the fourth color value after the illumination difference to process the foreground object. Thereby reducing the impact of differences between cameras on image processing.

The third aspect of the embodiments of the present invention also provides a computer device, which is characterized in that the computer device includes a processor, a memory, a bus, and a communication interface; the memory is used to store programs, and the processor and the memory Through the bus connection, when the computer device is running, the processor executes the program stored in the memory, so that the computer device executes the image processing method as described above.

A fourth aspect of the embodiments of the present invention also provides a computer-readable storage medium storing one or more programs, the one or more programs including instructions, and the instructions are executed by a computer device including one or more processors When executed, the computer device is made to execute the above-mentioned image processing method.

It can be seen from the above that in some feasible implementations of the embodiments of the present invention, the foreground is calculated according to the color values of the reference background object under the first and second lighting conditions, and the color value of the foreground object under the second lighting condition The color value of the object under the first lighting condition, so as to unify the foreground object in the image to be processed under the first lighting condition, and then use the color value of the foreground object under the first lighting condition to perform retrieval or query processing. Compared with the prior art, it is not necessary to strip the illumination value of the image, which solves the problem that it is difficult to obtain an accurate illumination estimation value in the prior art, and improves the accuracy of image processing.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that are required in the description of the embodiments and prior art. Obviously, the accompanying drawings in the following description are only some implementations of the present invention For example, those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a schematic flow chart of an image processing method provided by an embodiment of the present invention;

Fig. 2 is a schematic flowchart of an image processing method provided by another embodiment of the present invention;

Figures 3a and 3b are schematic diagrams of images before and after processing, respectively;

Fig. 4 is a schematic flowchart of an image processing method provided by another embodiment of the present invention;

Fig. 5 is a schematic structural diagram of an image processing device provided by an embodiment of the present invention;

Fig. 6 is a schematic structural diagram of an image processing device provided by another embodiment of the present invention;

Fig. 7 is a schematic structural diagram of a computer device provided by an embodiment of the present invention.

detailed description

In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only It is an embodiment of a part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

The terms "first", "second", "third" and the like in the description and claims of the present invention and the above drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "include" and "have", as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally further includes For other steps or units inherent in these processes, methods, products or apparatuses.

The technical solution of the embodiment of the present invention applies to various computer systems related to image processing such as face recognition, iris recognition, and video surveillance. Group, etc.; storage unit for storing images, such as read-only memory (full English name: Read-Only Memory, English abbreviation: ROM), random access memory (English full name: Random Access Memory, English abbreviation: RAM), mobile hard disk , magnetic disk or optical disc, etc.; a processing unit for image processing, such as a central processing unit (English full name: Central Processing Unit, English abbreviation: CPU), or a specific integrated circuit (English full name: Application Specific Integrated Circuit, English abbreviation: ASIC) etc.; a display unit for displaying images, such as a cathode ray tube display (English full name: Cathode Ray Tube, English abbreviation: CRT), plasma display (English full name: Plasma Display Panel, English abbreviation: PDP), liquid crystal display (English full name: : Liquid Crystal Display, English abbreviation: LCD), etc.; and a bus that connects the above-mentioned units and is used for mutual communication between the above-mentioned units, such as an Industry Standard Architecture (Industry Standard Architecture, referred to as ISA) bus or external device interconnection (Peripheral Component, referred to as PCI) bus or Extended Industry Standard Architecture (Extended Industry Standard Architecture, referred to as EISA) bus, etc.; and a communication interface, etc.

In images taken under different lighting conditions, the color value of the same physical object will change, which will affect the results of query and retrieval with color as the main feature. The present invention addresses this problem.

The technical solution of the embodiment of the present invention can be used in such a scene: in the given multiple images, given an image, the foreground object in the given image is used as the query target, and the query target is compared with other images Foreground objects are matched to determine whether they are the same object, so as to query or retrieve all images containing the query target.

Please refer to Figure 3a. The person in the first image in the upper left corner is used as the query target, and the query is directly performed from multiple images to find the query results of the image containing the query target. The query results show that the first row of the first row A total of 7 images from the seventh to the seventh all contain the query target, or in other words, the portraits in these 7 images are the same person. However, since the color value of each image is affected by the light value, it is difficult to find all the images containing the query target in the query result. Assuming that there are 15 images containing the query target in the multiple images, the query hit rate is only 7/15= 47%.

The image processing method provided by an embodiment of the present invention may include: acquiring the first color value of the reference background object in the reference image captured under the first lighting condition; acquiring the reference background object in the image to be processed captured under the second lighting condition The second color value of the background object and the third color value of the foreground object; according to the first color value, the second color value and the third color value, estimate the fourth color value of the foreground object photographed under the first lighting condition ; Using the fourth color value to process the foreground object in the image to be processed, for example, perform retrieval or query.

From the above, in order to avoid estimating the illumination value and re-estimating the reflection coefficient (intrinsic color) of the image based on the estimated illumination value, so as to avoid errors in the illumination estimation process, in the embodiment of the present invention, different illumination conditions The color of the object under the same lighting condition is unified to the "same" reference lighting condition, that is, the first lighting condition, so as to obtain the color value of the object under approximately the same lighting condition.

For this reason, the embodiment of the present invention assumes that the color value of the reference background object under the condition of better illumination can be obtained, which is usually possible. For example, for video surveillance applications, the color value of a reference background object under better lighting conditions can be obtained at a certain time during the day. The color value of the reference background object under better lighting is used as prior information, and the lighting condition under better lighting is called the first lighting condition in this paper. In the case of uniform lighting, the color value of the foreground object under the first lighting condition can be calculated by using the difference of the color value of the same or the same reference background object under different lighting conditions, and the foreground object can be unified under the first lighting condition .

Please refer to Fig. 3b, after using the example method of the present invention to unify the foreground objects of multiple images under the first lighting condition, the portrait in the first image in the upper left corner is used as the query target, and the query is performed from multiple images to find Query results for images of the same person. The query results show that a total of 15 images containing the query target are found, such as the first 15 images in the first row in Figure 3b, and the query hit rate is 15/15=100%.

Please refer to Fig. 1, the specific process of the image processing method provided by one embodiment of the present invention may include:

110. Acquire a first color value of a reference background object in a reference image captured under a first lighting condition.

In the embodiment of the present invention, the first color value of the reference background object under the first lighting condition may be acquired from the reference image including the reference background object. The so-called first lighting condition may be understood as the lighting condition for shooting the reference image. In the embodiment of the present invention, one of the acquired images that need to be processed may be selected as a reference image, and the lighting condition of the reference image is defined as the first lighting condition. Among them, the multiple acquired images should include the same or the same kind of reference background objects, such as roads, trees, walls, etc. This article does not limit the specific category of the reference background objects.

Herein, the images in the plurality of images except the reference image are referred to as images to be processed. Optionally, before step 110, it may also include: acquiring a plurality of images; acquiring an image in the plurality of images as the reference image; acquiring any one of the plurality of images except the reference image image as the image to be processed.

Said first color value, and the second, third or fourth color value mentioned later, etc., can be referred to as color value or color value for short, specifically can refer to RGB value, and RGB expresses in English and is exactly R ( red), G (green), B (blue). The RGB color mode is a color standard in the industry. It obtains a variety of colors by changing the three color channels of red (R), green (G), and blue (B) and superimposing them with each other. Yes, RGB is the color that represents the three channels of red, green, and blue. This standard includes almost all the colors that human vision can perceive. It is one of the most widely used color systems at present. RGB refers to brightness and is expressed using integers. Normally, R, G, and B each have 256 levels of brightness, which are represented by numbers from 0, 1, 2... until 255, a total of 256 levels.

The methods of encoding a color are collectively referred to as "color spaces" or "gamuts". The "color space" of any color can be defined as a fixed number or variable. With the RGB encoding method, each color can be represented by three variables—the brightness of red, green, and blue. A variety of RGB formats can be used to represent a color, for example, a format called RGB555, RGB555 is a 16-bit RGB format, the three components of RGB are represented by 5 bits, and the remaining 1 bit is not used; thus, The color of each pixel in an image can be represented by a 16-bit binary number. In practical applications, the format is not limited to RGB555, and formats such as RGB565, RGB24, and RGB32 can also be used, which will not be repeated here.

In order to improve the accuracy of image processing, the image taken under better lighting conditions can be selected as the reference image in this paper. Wherein, the lighting condition can be better understood as the image brightness value is within a set threshold range, and/or, the shooting time is within a set time range. In some embodiments, one image may be selected from the multiple images as The reference image. Wherein, the unit of the luminance value may be candela per square meter (cd/m2).

120. Acquire the second color value of the reference background object and the third color value of the foreground object in the image to be processed captured under the second lighting condition.

In this step, specifically, the second color value of the reference background object and the third color value of the foreground object under the second lighting condition may be acquired from the image to be processed that includes both the reference background object and the foreground object. The so-called second lighting condition may be understood as the lighting condition for capturing the image to be processed. The image to be processed may refer to any image except the selected reference image among the multiple images acquired in the previous step.

130. Estimate a fourth color value for shooting the foreground object under the first lighting condition according to the first color value, the second color value and the third color value.

Record the first color value, the second color value, the third color value and the fourth color value as S ₁ , S ₂ , S ₃ , S ₄ respectively, then S ₄ =S ₃ -(S ₂ -S ₁ ); It should be noted that S ₁ , S ₂ , S ₃ , and S ₄ here are the values after taking the logarithm of the original observed value. The original observed values may be RGB values. That is to say, S ₁ , S ₂ , S ₃ , and S ₄ are RGB values in the logarithmic domain, and the RGB values in the logarithmic domain specifically refer to values obtained by taking logarithms of the RGB values. By taking the logarithm, it not only conforms to the visual characteristics of the human eye, but also plays a role in data compression.

The calculation principle is briefly explained as follows. According to the color values S ₁ and S ₂ of the reference background object under the first and second lighting conditions respectively, the difference between the lighting values under the first and second lighting conditions can be calculated as ΔS=(S ₂ -S ₁ ); similarly, ΔS=(S ₄ -S ₃ ); then ΔS=(S ₂ -S ₁ )=(S ₄ -S ₃ ), and S ₄ =S ₃ - (S ₂ -S ₁ ). Thus, the color value of the foreground object under the first lighting condition is obtained, and the color of the foreground object under the second lighting condition is unified to that under the first lighting condition.

Based on this algorithm, the color value of the foreground object in any image to be processed under the first lighting condition can be calculated, and the color of the foreground object in multiple images under different lighting conditions can be unified to the first lighting condition Down.

140. Process the foreground object by using the fourth color value.

In this step, the color value of the foreground object unified under the first lighting condition can be used for image processing, including retrieval or query. For example, as shown in Figure 3b, an image with higher definition can be selected from the images shown in Figure 3b as a reference image, and the color values of the foreground objects (specifically, portraits) in other images can be unified to the color value of the reference image. Under the first lighting condition, the color value under the first lighting condition estimated by using the foreground object of each image is used for retrieval or query, wherein, the portrait in the first image in the upper left corner can be used as the query target, and any other The portraits in the images are matched with the query target to determine whether they are the same person to find all images containing the search target, and finally 15 images containing the query target are obtained from the query, as shown in the first 15 images in the first row in Figure 3b image.

The following is a further introduction to the technical principles of this solution:

An illumination model widely used at present is the Retinex mathematical model, and the technical principle of the present invention is based on this model, and under the assumption that the color value of the reference background object under the first illumination condition L0 can be obtained, deduce that in the second illumination condition L0 The color value of the foreground object obtained under the condition L1 under the illumination of L0, where the specific values of L0 and L1 do not need to be known.

The Retinex model is as follows:

s(x,y)=l(x,y)*r(x,y) (1.1)

Among them, s(x,y) is the color value, l(x,y) is the light value, and r(x,y) is the reflection coefficient of the object.

By taking the logarithm of both sides of formula (1) and simplifying the sign, formula (1.1) is equivalent to

S(x,y)=L(x,y)+R(x,y) (1.2)

In formula (2), use uppercase S(x, y), L(x, y), R(x, y) respectively to represent s(x, y), l(x, y), r(x, y ) takes the value after the logarithm.

In the case of illumination L0, the following formula holds for the reference background object

S _BG,0 ＝L ₀ (x,y)+R _BG (x,y) (1.2.1)

Among them, the subscript BG represents the background object, and the subscript 0 represents the light L0.

In the new lighting L1 case, for background objects there are

S _BG,1 =L ₁ (x,y)+R _BG (x,y) (1.2.2)

Among them, the subscript BG represents the background object, and the subscript 1 represents the light L1.

At this point, for the foreground object there are

S _FG,1 =L ₁ (x,y)+R _FG (x,y) (1.2.3)

Among them, the subscript FG represents the foreground object, and the subscript 1 represents the light L1.

Subtract (1.2.1) from (1.2.2) to get

S _BG,1 -S _BG,0 ＝L ₁ (x,y)-L ₀ (x,y)

And then get

L ₁ (x,y)=L ₀ (x,y)+(S _BG,1 -S _BG,0 ),

Substituting the obtained L ₁ (x,y) into formula (1.2.3), then we have

S _FG,1 ＝R _FG (x,y)+L ₀ (x,y)+(S _BG,1 -S _BG,0 )

Further, one can get

R _FG (x,y)+L ₀ (x,y)＝S _FG,1 -(S _BG,1 -S _BG,0 ) (1.2.4)

The right end of the formula (1.2.4) is the observed color value, which is known. The left end is exactly the color value of the foreground object under the illumination L0, which can be recorded as S _FG,0 .

Formula (1.2.4) can also be written as S _FG,0 ＝S _FG,1 -(S _BG,1 -S _BG,0 )

It can be understood that the above solutions of the embodiments of the present invention may be specifically implemented in computer equipment, for example. This scheme calculates the color value of the foreground object under the first lighting condition according to the color value of the reference background object under the first and second lighting conditions, and the color value of the foreground object under the second lighting condition, so that the to-be-processed The foreground objects in the image are unified under the first lighting condition, and the color value of the foreground object under the first lighting condition can be used for subsequent processing such as retrieval or query. Compared with the existing technology, it is not necessary to strip the lighting value of the image, which solves the problem There is a problem that it is difficult to obtain accurate lighting estimates in the technology, and the accuracy of image processing is improved.

In order to facilitate a better understanding of the technical solutions provided by the embodiments of the present invention, an implementation manner in a specific scenario is used as an example to introduce below.

Please refer to FIG. 2 , another image processing method according to the embodiment of the present invention. May include:

210. Acquire multiple images, where the multiple images all include a reference background object.

The object to be processed by the scheme of this embodiment is a group of image sets with the same background object, including multiple images, and the multiple images all include the same background object, and the same background object is defined as a reference background object. Multiple images in an image set can be taken by the same camera and have the same or similar background objects; they can also be taken by different cameras, and the images taken by different cameras have overlapping areas, or no overlapping areas, but have the same background Objects, such as ground of the same material (such as asphalt road).

220. Select one of the multiple images as the reference image according to the brightness values and/or shooting time of the multiple images.

According to the luminance values and/or shooting time of the multiple images, an image shot under the condition of good light intensity may be selected as a reference image. For example, it can be considered that the brightness value is between 4.5 and 5.5, and the shooting time is in the morning, the light intensity is better, and the captured image is relatively clear, then such an image can be selected as a reference image.

A specific background object in the reference image is selected as the reference background object (SBG,0), and the selection requirement of the reference background object is to appear in multiple images of the aforementioned image set to be processed. The light intensity corresponding to the reference background object is used as a reference light level, that is, the first light condition L0 mentioned above.

230. Acquire a reference image and an image to be processed.

Define images other than the reference image in the image set as images to be processed, and any image to be processed can be processed according to the reference image, including the following steps:

240. Acquire the first color value of the reference background object under the first lighting condition from the reference image; acquire the second color value of the reference background object and the foreground object under the second lighting condition from the image to be processed Tertiary color value.

Extract the color value S _BG,0 of the reference background object from the reference image, and for each image to be processed in the aforementioned image set to be processed, extract the same background object S _BG,1 as the reference background object and hope to return to the reference illumination Horizontal foreground object S _FG,1 . Note that the first color value, the second color value and the third color value are respectively S _BG,0 , S _BG,1 , and S _FG,1 .

250. Calculate the fourth color value S _FG,0 of the foreground object under the first lighting condition;

Using the formula (1.2.4), we can get S _FG,0 ＝S _FG,1 -(S _BG,1 -S _BG,0 )

Or, denote the first, second, third and fourth color values as S ₁ , S ₂ , S ₃ , S ₄ respectively. Then there is S ₄ =S ₃ -(S ₂ -S ₁ ).

260. Use the fourth color value to process the foreground object in the image to be processed.

The input calculation result—the fourth color value S _FG,0 , can be used for subsequent index creation for retrieval or query processing. For example, when the foreground object is a person and it is desired to find an image of a certain person from the image set, the color values of the foreground object in each image in the image set can be unified to the reference light level, and then use the color value under the reference light level to perform portrait search or query. Please refer to Figure 3a, which is the query result obtained by using the first image in the upper left corner as the query object without the above image processing, as shown in the wireframe in the figure, find 7 images containing the query target, as shown in the figure The first 7 images of the first row in 3a, but in fact, the image collection includes 15 images of the query object, so the query hit rate is 7/15=47%; please refer to Figure 3b, it is first for all images After performing the above image processing and unifying all the images under the same lighting condition, the first image in the upper left corner is still used as the query object to obtain the query result, as shown in the wireframe in the figure, find 15 images containing the query object images, such as the first 15 images in the first row in Figure 3b, the hit rate is 15/15=100%.

It should be noted that, in some embodiments, color correction can be performed on the foreground object in the image to be processed according to the fourth color value, and the corrected image to be processed can be displayed; in other embodiments, it is also possible to only use the fourth color value It is used for retrieval and query without color correction of the foreground object, and does not display the corrected image to be processed; whether to display it can be determined according to actual needs.

Above, the embodiment shown in FIG. 2 has further described the image processing method of the embodiment of the present invention, wherein the multiple images in the processed image set may be taken by the same camera or by different cameras. The images captured by the cameras can have overlapping areas; or have no overlapping areas, but have the same reference background object, such as the ground of the same material, such as an asphalt road. That is, the technical solutions of the embodiments of the present invention can be directly applied to cross-camera applications.

For each image captured by a camera, an image of the first light condition with better illumination, such as L0, can be selected as a reference image, and the foreground objects of each image captured by the camera are mapped to L0, that is, the estimated The color value under the first lighting condition L0. For the convenience of description below, this process, that is, the process described in steps 110-130 above, or the process described in steps 210-250, is called illumination preprocessing. Although different cameras may choose different reference lighting levels, because they are all relatively good lighting, for many applications, images after such lighting preprocessing can basically meet the requirements of query or retrieval.

However, in some cases, there are large differences between cameras. After performing the above-mentioned lighting preprocessing on multiple images taken by multiple cameras, if the situation still cannot meet the application, some technical means can be used to analyze the images taken by different cameras. Images that have been pre-lighted are processed again to map them to approximately the same reference lighting level. The following example describes a technical approach to this end.

Please refer to FIG. 4, in some embodiments, the image processing method of the embodiment of the present invention may include:

410. Color calibration between cameras.

Color calibration for multiple cameras can be pre-calibrated when the cameras are installed.

420. Calibrate the corresponding reference light levels between different reference background objects between cameras.

In this step, the illumination difference of the reference background object in the images captured by each camera except the reference camera among the plurality of cameras and the reference camera is respectively calculated. In the absence of a publicly referenced background object, each camera can be used to take pictures of multiple colors of the same palette while taking pictures of the background environment. The resulting palette color image is consistent with the light level of the background object. The palette color image captured by each camera can be used to calculate and estimate the illumination difference ΔL between reference background objects captured by different cameras. In order to avoid a large amount of computational burden, a certain camera can be selected as a comparison standard, here called the reference standard camera, and the illumination difference ΔSL between the reference background objects captured by all other cameras and the reference background objects captured by the reference standard camera can be calculated. The illumination difference ΔSL of the reference background object in the images captured by each of the plurality of cameras and the reference camera is respectively calculated.

430. Perform lighting preprocessing on the foreground object captured by each camera.

In this step, an illumination preprocessing operation is performed on images captured by each camera. The specific process of illumination preprocessing corresponds to the process described in steps 110-130 above, or the process described in steps 210-250, and will not be described in detail here.

440. Perform illumination difference removal processing on foreground objects after illumination preprocessing captured by different cameras.

In this step, the corresponding illumination difference is removed from the fourth color value of the foreground object in each image to be processed to obtain a corrected fourth color value. That is to say, for the obtained foreground object sequence, the illumination difference ΔSL is removed. Note that for the image sequence captured by the reference standard camera, after the illumination preprocessing step, the step of removing the illumination difference is not required, because there is no difference among themselves. The image sequence obtained after removing the illumination difference can be called the illumination indifferent estimation image sequence. The obtained illumination invariant estimation image sequence can be used for query retrieval in applications.

450. Use the color value of the foreground object obtained after the above steps to perform retrieval or query processing.

In this step, the image query and retrieval is performed using the output result image of the above steps, that is, the retrieval or query is performed using the corrected fourth color value of the image sequence of the illumination indifference estimation image sequence.

It should be noted that the above step 440 may also be performed before step 430 .

As can be seen from the above, in some feasible implementations of the present invention, an image processing method is provided, according to the color values of the reference background object under the first and second lighting conditions, and the color value of the foreground object under the second lighting condition Color value to calculate the color value of the foreground object under the first lighting condition, so that the foreground object in the image to be processed is unified under the first lighting condition, and the color value of the foreground object under the first lighting condition can be used for subsequent retrieval Compared with the prior art, it is not necessary to peel off the illumination value of the image, which solves the problem that it is difficult to obtain an accurate illumination estimation value in the prior art, and improves the accuracy of image processing.

In order to better implement the above solutions of the embodiments of the present invention, related devices for coordinating the implementation of the above solutions are also provided below.

Please refer to FIG. 5, an embodiment of the present invention provides an image processing device 500, which may include:

An acquisition module 510, configured to acquire a first color value of the reference background object in the reference image captured under the first lighting condition; acquire a second color value of the reference background object in the image to be processed captured under the second lighting condition and the third color value of the foreground object;

A color value calculation module 520, configured to estimate a fourth color value of the foreground object photographed under the first lighting condition according to the first color value, the second color value and the third color value;

A processing module 530, configured to process the foreground object by using the fourth color value.

Wherein, the color value may be referred to as a color value for short.

In some embodiments, the first color value, the second color value, the third color value and the fourth color value are RGB values or RGB values in logarithmic domain.

In some embodiments, the color value calculation module 520 is specifically configured to calculate the fourth color value using the following formula: S ₄ =S ₃ -(S ₂ -S ₁ ); wherein, S ₁ , S ₂ , S ₃ , S ₄ are the first color value, the second color value, the third color value and the fourth color value respectively, and S ₁ , S ₂ , S ₃ , S ₄ are RGB values in the logarithmic domain.

Please refer to FIG. 6 , in some embodiments, the acquiring module 510 is further configured to: acquire a plurality of images; acquire an image in the plurality of images as the reference image; acquire all but one of the plurality of images Any image other than the reference image is used as the image to be processed; the reference image includes the reference background object, and the image to be processed includes the reference background object and the foreground object.

Please refer to FIG. 6, in some embodiments, the image processing device 500 further includes: a setting module 540;

The setting module 540 is configured to select an image from the multiple images as the reference image according to the brightness values and/or shooting time of the multiple images.

Please refer to FIG. 6, in some embodiments, the image processing device 500 further includes:

A calibration module 550, configured to calibrate the colors of the plurality of cameras in advance;

A difference calculation module 560, configured to separately calculate the illumination difference of the reference background object in the images captured by each camera except the reference camera among the plurality of cameras and the reference camera;

The processing module 530 is further configured to remove the corresponding illumination difference from the fourth color value, and use the fourth color value after the illumination difference to process the foreground object.

In some embodiments, the processing module 530 is specifically configured to retrieve or query the foreground object.

The image processing apparatus in the embodiment of the present invention may be, for example, computer equipment. The above functional modules can be realized by the processor of the computer device executing the programs stored in the memory.

It can be understood that the functions of each functional module of the image processing device in the embodiment of the present invention can be specifically implemented according to the method in the above method embodiment, and the specific implementation process can refer to the relevant description in the above method embodiment, and will not be repeated here.

As can be seen from the above, in some feasible implementation manners of the present invention, an image processing device is provided, according to the color values of the reference background object under the first and second lighting conditions, and the color value of the foreground object under the second lighting condition Color value to calculate the color value of the foreground object under the first lighting condition, so that the foreground object in the image to be processed is unified under the first lighting condition, and the color value of the foreground object under the first lighting condition can be used for subsequent retrieval Compared with the prior art, it is not necessary to peel off the illumination value of the image, which solves the problem that it is difficult to obtain an accurate illumination estimation value in the prior art, and improves the accuracy of image processing.

Please refer to FIG. 7, the embodiment of the present invention also provides a computer device 700, which may include:

Processor 710, memory 720, communication interface 730, bus 740;

The processor 710, memory 720, and communication interface 730 are connected and communicate with each other through the bus 740; the communication interface 730 is used to receive and send data; the memory 720 is used to store a program 750; the processor 710 is used to execute the program 750 in the memory; when the computer device 700 is running, the processor 710 executes the program 750 stored in the memory 720, so that the computer device 700 performs as above The image processing method described in the method embodiment.

Wherein, the processor 710 may perform the following steps: acquire the first color value of the reference background object in the reference image taken under the first lighting condition; acquire the first color value of the reference background object in the image to be processed taken under the second lighting condition The second color value and the third color value of the foreground object; according to the first color value, the second color value and the third color value, estimate the fourth color value of the foreground object photographed under the first lighting condition; use the The fourth color value is used to process the foreground object.

The bus 740 may be an Industry Standard Architecture (Industry Standard Architecture, referred to as ISA) bus or a Peripheral Component Interconnect (Pipheral Component, referred to as PCI) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, referred to as EISA) bus etc. The bus can be classified into one or more of address bus, data bus and control bus. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

The memory 720 may include a high-speed RAM (Ramdom Access Memory) memory. Optionally, the memory 720 may further include a non-volatile memory (non-volatile memory). For example, the memory 720 may include disk storage.

The processor 710 may be a central processing unit (Central Processing Unit, referred to as CPU), or the processor 710 may be a specific integrated circuit (Application Specific Integrated Circuit, referred to as ASIC), or the processor 710 may be One or more integrated circuits configured to implement embodiments of the invention.

It can be understood that the functions of the computer device in the embodiments of the present invention can be specifically implemented according to the methods in the above method embodiments, and the specific implementation process can refer to the relevant descriptions in the above method embodiments, which will not be repeated here. By implementing the methods in the above method embodiments, the computer equipment can obtain the technical effects obtained by the above method embodiments.

(Embodiment 3) The embodiment of the present invention also provides a computer-readable storage medium that stores one or more programs, and the one or more programs include instructions, and when the instructions are executed by one or more processors When the computer device executes, the computer device is made to execute the image processing method described in the above method embodiments.

In the foregoing embodiments, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

It should be noted that for the foregoing method embodiments, for the sake of simple description, they are expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because Certain steps may be performed in other orders or simultaneously in accordance with the present invention. Secondly, those skilled in the art should also know that the embodiments described in the specification belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

A unit described as a separate component may or may not be physically separated, and a component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .

The image processing method and device provided by the embodiments of the present invention have been introduced in detail above. In this paper, specific examples are used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only used to help understand the method of the present invention. and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and scope of application. limits.

Claims (15)

1. An image processing method, characterized in that, comprising: Acquiring a first color value of a reference background object in a reference image captured under a first lighting condition; Acquiring the second color value of the reference background object and the third color value of the foreground object in the image to be processed captured under the second lighting condition; Estimating a fourth color value of the foreground object photographed under the first lighting condition according to the first color value, the second color value and the third color value; The foreground object is processed using the fourth color value. 2. The method according to claim 1, wherein the first color value, the second color value, the third color value and the fourth color value are RGB values or RGB values in a logarithmic domain. 3. The method according to claim 2, wherein, according to the first color value, the second color value and the third color value, estimating the fourth color value of shooting the foreground object under the first lighting condition comprises : The fourth color value is calculated using the following formula: S ₄ =S ₃ -(S ₂ -S ₁ ); Wherein, S ₁ , S ₂ , S ₃ , S ₄ are respectively the first color value, the second color value, the third color value and the fourth color value, and S ₁ , S ₂ , S ₃ , S ₄ are RGB value on the logarithmic domain. 4. The method according to claim 1, wherein, before the acquisition of the first color value of the reference background object in the reference image taken under the first lighting condition, further comprising: get multiple images; Acquiring one of the plurality of images as the reference image; Acquiring any one of the plurality of images except the reference image as the image to be processed; The reference image includes the reference background object, and the image to be processed includes the reference background object and the foreground object. 5. The method according to claim 4, wherein said acquiring an image in said plurality of images as said reference image comprises: An image is selected from the multiple images as the reference image according to the brightness values and/or shooting time of the multiple images. 6. The method according to claim 4, wherein when the multiple images are from multiple cameras, the acquisition of the first color value of the reference background object in the reference image taken under the first lighting condition Previously, also included: Calibrating the colors of the plurality of cameras in advance; separately calculating the illumination difference of the reference background object in the images captured by each camera except the reference camera among the plurality of cameras and the reference camera; The processing the foreground object by using the fourth color value includes: The corresponding illumination difference is removed from the fourth color value, and the foreground object is processed by using the fourth color value after the illumination difference is removed. 7. The method according to any one of claims 1-6, wherein said processing said foreground object comprises: The foreground object is retrieved or queried. 8. An image processing device, comprising: An acquisition module, configured to acquire the first color value of the reference background object in the reference image taken under the first lighting condition; acquire the second color value and the value of the reference background object in the image to be processed taken under the second lighting condition the third color value of the foreground object; A color value calculation module, configured to estimate a fourth color value of the foreground object photographed under the first lighting condition according to the first color value, the second color value and the third color value; A processing module, configured to process the foreground object by using the fourth color value. 9. The method according to claim 8, wherein the first color value, the second color value, the third color value and the fourth color value are RGB values or RGB values in a logarithmic domain. 10. The apparatus of claim 9, wherein: The color value calculation module is specifically used to calculate the fourth color value using the following formula: S ₄ =S ₃ -(S ₂ -S ₁ ); Wherein, S ₁ , S ₂ , S ₃ , S ₄ are respectively the first color value, the second color value, the third color value and the fourth color value, and S ₁ , S ₂ , S ₃ , S ₄ are RGB value on the logarithmic domain. 11. The apparatus of claim 8, wherein: The acquisition module is also used to: acquire a plurality of images; acquire an image in the plurality of images as the reference image; acquire any image in the plurality of images except the reference image as the reference image The image to be processed; the reference image includes the reference background object, and the image to be processed includes the reference background object and the foreground object. 12. The device of claim 11, further comprising: A setting module, configured to select an image from the plurality of images as the reference image according to brightness values and/or shooting time of the plurality of images. 13. The device of claim 11, further comprising: A calibration module, configured to calibrate the colors of the plurality of cameras in advance; a difference calculation module, configured to separately calculate the illumination difference of the reference background object in the images captured by each of the plurality of cameras except the reference camera and the reference camera; The processing module is further configured to remove the corresponding illumination difference from the fourth color value, and process the foreground object by using the fourth color value after the illumination difference is removed. 14. The device according to any one of claims 8-13, characterized in that, The processing module is specifically configured to retrieve or query the foreground object. 15. A computer device, characterized in that the computer device comprises a processor, a memory, a bus and a communication interface; The memory is used to store programs, and the processor is connected to the memory through the bus. When the computer device is running, the processor executes the program stored in the memory, so that the computer device Executing the image processing method described in any one of claims 1-7.