TWI661393B - Image segmentation method, computer program, storage medium, and electronic device - Google Patents

Abstract

An image segmentation method, the method includes: obtaining an image and a depth image of the image; obtaining a bar graph of the depth image, the bar graph including an abscissa and an ordinate; using a clustering algorithm to The bar graph data is clustered to obtain the two categories and the centers of the two categories; determine that the abscissa corresponding to the smallest vertical coordinate value between the centers of the two categories in the bar graph is the segmentation threshold; determine the segmentation threshold Whether a preset condition is satisfied; when the segmentation threshold value satisfies the preset condition, the image is segmented into a foreground and a background area according to the segmentation threshold value. The invention also provides a computer program, a storage medium and an electronic device, which can avoid front-background segmentation at positions where the image depth distribution is continuous.

Description

Image division method, computer program, storage medium and electronic device

The invention relates to the technical field of digital image processing, in particular to an image segmentation method, a computer program, a storage medium and an electronic device.

At present, post-processing of images, such as background virtual processing, requires pre-background segmentation processing of the image in advance. However, the existing background segmentation processing of existing images often involves segmentation at positions where the image depth distribution is continuous, resulting in improper foreground and background segmentation of the image, which will affect the post-processing of the image.

In view of the above, it is necessary to provide an image segmentation method, a computer program, a storage medium, and an electronic device, which can avoid front-background segmentation at locations where the image depth distribution is continuous.

A first aspect of the present invention provides an image segmentation method. The method includes:

Acquiring an image and a depth image of the image;

Acquiring a bar graph of the depth image, where the bar graph includes an abscissa and an ordinate;

The clustering algorithm is used to cluster the bar graph data to obtain two categories and the centers of the two categories;

Determining that the abscissa corresponding to the smallest ordinate value between the centers of the two categories in the bar graph is the segmentation threshold;

Determining whether the segmentation threshold value satisfies a preset condition;

When the segmentation threshold value satisfies a preset condition, the image is segmented into a foreground and a background region according to the segmentation threshold value.

Preferably, the obtaining a depth image of the image includes:

A depth image of the image is acquired by a binocular matching algorithm or a depth image acquisition device.

Preferably, the preset condition is that an average value of the ordinate values in a preset range centered on the segmentation threshold in the bar graph is smaller than a preset value. Alternatively, the preset condition is that an ordinate value corresponding to the segmentation threshold in the bar graph is smaller than a preset value.

Preferably, the method further includes:

When the segmentation threshold does not satisfy a preset condition, it is determined that the image is not suitable for segmentation into the foreground and the background regions.

Preferably, the abscissa in the bar graph represents a depth value of a primitive of the depth image;

The ordinate in the bar graph represents the number of the primitives corresponding to each of the depth values.

Preferably, the clustering algorithm is a K-means algorithm.

A second aspect of the present invention provides a computer program. The computer program includes at least one computer program instruction, and the at least one computer program instruction is executed by a processor to implement the image segmentation method according to any one of the above.

The third-party vendor of the present invention provides a storage medium that stores at least one computer program instruction, and the at least one computer program instruction is used by a processor to implement the image segmentation method according to any one of the above.

According to a fourth aspect of the present invention, an electronic device is provided. The electronic device includes a processor and a memory, and the processor is configured to execute at least one computer program instruction stored in the memory to implement the image according to any one of the foregoing Segmentation method.

In this solution, an image and a depth image of the image are obtained; a bar graph of the depth image is obtained, and the bar graph includes an abscissa and an ordinate; a clustering algorithm is used to aggregate the bar graph data; Class to obtain two categories and the centers of the two categories; determine that the abscissa value corresponding to the smallest ordinate value between the centers of the two categories in the bar graph is the segmentation threshold value; determine whether the segmentation threshold value meets a preset condition; When the segmentation threshold value satisfies a preset condition, the image is segmented into foreground and background regions according to the segmentation threshold value, so as to avoid front background segmentation at positions where image depth distribution is continuous.

In the following, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by a person skilled in the art without paying any creative labor belong to the protection scope of the present invention.

Example one

FIG. 1 is a flowchart of an image segmentation method according to a first embodiment of the present invention. According to different requirements, the order of steps in this flowchart can be changed, and some steps can be omitted. The method is applied to an electronic device, and the electronic device may be any kind of electronic product, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), and the like. As shown in FIG. 1, the image segmentation method may include the following steps:

Step S11: Obtain an image and a depth image of the image.

The acquiring an image and a depth image of the image may include: acquiring an image, and acquiring a depth image of the image by using a binocular matching algorithm. The acquiring the image and the depth image of the image may further include: acquiring the image and the depth image of the image by a depth image acquisition device. The depth image acquisition device can directly acquire an image and a depth image of the image. In this embodiment, the depth image acquisition device may be a Kinect camera. In other embodiments, the depth image acquisition device may be a sensor such as PrimeSense.

The depth image is a monochrome image, and its size is the same as the image. A gray value of the depth image is a depth value of a primitive of the image. The depth value is a distance between a scene captured by the image and the depth image acquisition device.

Step S12: Obtain a bar graph of the depth image, where the bar graph includes an abscissa and an ordinate.

In this embodiment, before obtaining a bar graph of the depth image, the bar graph includes an abscissa and an ordinate, the method further processes the depth image to eliminate noise. The processing may be an image smoothing process. In this embodiment, the image smoothing process may be a neighborhood average method, a median filter method, or the like.

In this embodiment, the bar graph is a deep bar graph. In this embodiment, the bar graph reflects the number of primitives having the depth value in the depth image, as shown in FIG. 2. The abscissa of the bar graph represents the depth value of the primitives of the depth image, and the ordinate of the bar graph represents the number of the primitives corresponding to each depth value. In this embodiment, the range of the depth value is [0,255], and the range of the number of primitives is [0,3000].

In step S13, the bar graph data is clustered by using a clustering algorithm to obtain two categories and the centers of the two categories.

In this embodiment, the clustering algorithm is a K-means algorithm. The K-means algorithm may be any known K-means algorithm. One K-means algorithm is described in detail below, and other known K-means algorithms are not described in detail. The K-means algorithm includes:

a1: Select K points from the bar graph data as the center of the initial category, where K is an integer greater than one;

a2: scan all the data in the bar graph, calculate the distance between each point and the center of the category, and classify the points into the corresponding category according to the minimum distance;

a3: recalculate the center of each category;

a4: The algorithm ends when the preset number of iterations is repeated or the center of the new category is equal to the center of the original category or the distance is less than a preset threshold. In this embodiment, the K = 2, the preset number of times is 10, and the preset threshold value is 0.1.

In other embodiments, the clustering algorithm is a kernel density estimation algorithm and the like. The kernel density estimation algorithm is any known kernel density estimation algorithm. Since the kernel density estimation algorithm is known, it will not be repeated here.

In this embodiment, after the clustering algorithm ends, the two categories and the centers of the two categories are obtained in a bar graph. In this embodiment, the bar graph shown in FIG. 2 is taken as an example. The two categories are represented by two different shades of gray in FIG. 3, and the abscissa values of the centers of the two categories are 106.19, respectively. And 236.28 are shown by dotted lines in FIG. 3, respectively.

Step S14: Determine the abscissa value corresponding to the minimum ordinate value between the centers of the two categories in the bar graph as the division threshold.

For example: if the abscissa value with the smallest ordinate value between the abscissa value 106.19 and the abscissa value 236.28 in the bar graph is 206, then the abscissa value 206 is determined as the segmentation threshold, which is indicated by a dashed line in FIG. .

In this embodiment, the foreground is usually a scene closer to the image acquisition device, and the background is usually a scene farther away from the image acquisition device. The distance between the foreground and background from the image acquisition device is often different, so that in the depth image, the foreground and The depth distribution of the background is discontinuous, so that the position of the depth distribution discontinuity between the centers of the two classes can be set as the segmentation threshold to achieve effective segmentation of the foreground and background.

Step S15: Determine whether the segmentation threshold value satisfies a preset condition.

The preset condition may be that an average value of the ordinate values in a preset range centered on the segmentation threshold value in the bar graph is smaller than a preset value. In this embodiment, the preset range is a range in which the abscissa value is [T-N, T + N], T is the segmentation threshold, and N is a positive integer, for example, N may be 2. The preset value is 50 ~ 200, for example, the preset value is 100. Obviously, the N and the preset value are not limited to the above-mentioned values, and may be set to other values as required.

The preset condition may also be that an ordinate value corresponding to the segmentation threshold in the bar graph is smaller than a preset value. The preset value may be 50 ~ 200, for example, the preset value is 100. Obviously, the preset value is not limited to the above-mentioned value, and may be set to other values as needed.

In step S16, when the segmentation threshold value meets a preset condition, the image is segmented into a foreground and a background region according to the segmentation threshold value.

The preset condition may be that an average value of an ordinate value in a preset range centered on the segmentation threshold value in the bar graph is smaller than a preset value, or a corresponding value of the segmentation threshold in the bar graph. The ordinate value of is smaller than the preset value. In this embodiment, the segmentation threshold meets a preset condition to indicate that there is a clear boundary between the foreground and the background. At this time, the image may be segmented into two regions, the foreground and the background, according to the segmentation threshold. Specifically, a primitive having a depth value less than the segmentation threshold in the image may be divided into the foreground, and a primitive having a depth value greater than the segmentation threshold in the image may be divided into the background. The primitives with a depth value equal to the segmentation threshold in the image may be divided into the foreground or the background. In this embodiment, after the image is divided into the foreground and background regions according to the segmentation threshold, the image can be post-processed, such as background virtual processing, changing the background, etc., thereby achieving various functions.

The method provided in the first embodiment is to obtain an image and a depth image of the image; obtain a bar graph of the depth image, the bar graph including the horizontal coordinate and the vertical coordinate; The graph data is clustered to obtain the two categories and the centers of the two categories; determine that the abscissa corresponding to the smallest vertical coordinate value between the centers of the two categories in the bar graph is the segmentation threshold; determine whether the segmentation threshold meets A preset condition; when the segmentation threshold value meets the preset condition, the image is divided into two regions of foreground and background according to the segmentation threshold value, so as to determine the position where the segmentation threshold value is discontinuous in the image depth distribution, so that subsequent The front background is properly segmented, and the front background is segmented only when the boundary between the foreground and the background is clear, to further effectively segment the front background.

Example two

FIG. 5 is a flowchart of an image segmentation method provided by Embodiment 2 of the present invention. According to different requirements, the order of steps in this flowchart can be changed, and some steps can be omitted. The method is applied to an electronic device, and the electronic device may be any kind of electronic product, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), and the like. As shown in FIG. 5, the image segmentation method may include the following steps:

Step S71: Obtain an image and a depth image of the image.

Step S71 in this embodiment is consistent with step S11 in the first embodiment. For details, refer to the description of step S11 in the first embodiment, and details are not described herein.

Step S72: Obtain a bar graph of the depth image, where the bar graph includes an abscissa and an ordinate.

Step S72 in this embodiment is consistent with step S12 in the first embodiment. For details, refer to the description of step S12 in the first embodiment, and details are not described herein.

In step S73, the bar graph data is clustered by using a clustering algorithm to obtain two categories and the centers of the two categories.

Step S73 in this embodiment is consistent with step S13 in the first embodiment. For details, refer to the description of step S13 in the first embodiment, and details are not described herein.

Step S74: Determine the abscissa value corresponding to the minimum ordinate value between the centers of the two categories in the bar graph as the division threshold.

Step S74 in this embodiment is consistent with step S14 in the first embodiment. For details, refer to the description of step S14 in the first embodiment, and details are not described herein.

Step S75: Determine whether the segmentation threshold value satisfies a preset condition.

Step S75 in this embodiment is consistent with step S15 in the first embodiment. For details, refer to the description of step S15 in the first embodiment, and details are not described herein.

In step S76, when the segmentation threshold value meets a preset condition, the image is segmented into two regions, the foreground and the background, according to the segmentation threshold value.

Step S76 in this embodiment is consistent with step S16 in the first embodiment. For details, refer to the description of step S16 in the first embodiment, and details are not described herein.

In step S77, when the segmentation threshold does not satisfy a preset condition, it is determined that the image is not suitable for segmentation into foreground and background regions.

The segmentation threshold value not satisfying the preset condition may be that an average value of an ordinate value in the preset range centered on the segmentation threshold value in the bar graph is greater than or equal to a preset value, or is the length The ordinate value corresponding to the segmentation threshold in the bar graph is greater than or equal to a preset value.

In this embodiment, the segmentation threshold does not satisfy the preset condition, indicating that there is no clear boundary between the foreground and the background. At this time, the image is not suitable for being divided into the foreground and background regions, so the segmentation of the image is abandoned.

The method provided in the second embodiment is to obtain an image and a depth image of the image; obtain a bar graph of the depth image, where the bar graph includes an abscissa and an ordinate; and use a clustering algorithm for the bar The graph data is clustered to obtain the two categories and the centers of the two categories; determine that the abscissa corresponding to the smallest vertical coordinate value between the centers of the two categories in the bar graph is the segmentation threshold; determine whether the segmentation threshold meets A preset condition; when the segmentation threshold value meets the preset condition, the image is divided into two regions of foreground and background according to the segmentation threshold value; when the segmentation threshold value does not satisfy the preset condition, it is determined that the image is not suitable It is divided into two areas, foreground and background, so that the segmentation threshold is a position where the depth distribution of the image is discontinuous, so that the subsequent front background segmentation is appropriate, and the front background segmentation is performed only when the boundary between the current scene and the background is clear. The front background is effectively segmented, and the front background is not segmented when the boundary between the foreground and the background is not clear, which prevents the front background from being segmented inappropriately.

Example three

FIG. 6 is a schematic diagram of an electronic device according to a third embodiment of the present invention.

The electronic device 8 includes: a memory 81, at least one processor 82, and a computer program 83 stored in the memory 81 and executable on the at least one processor 82. The at least one processor 82 is configured to execute the computer program 83 to implement the steps in the foregoing image segmentation method embodiment.

Exemplarily, the computer program 83 may be divided into one or more modules / units, and the one or more modules / units are stored in the memory 81 and processed by the at least one processor 82 is executed to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 83 in the electronic device 8.

The electronic device 8 may be any type of electronic product, for example, a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), and the like. Those skilled in the art can understand that the schematic diagram 6 is only an example of the electronic device 8 and does not constitute a limitation on the electronic device 8. The electronic device 8 may include more or fewer components than shown in the figure, or Combining certain components or different components, for example, the electronic device 8 may further include an input-output device, a network access device, a display screen, and the like.

The at least one processor 82 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSPs), and application specific integrated circuits. ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 82 may be a microprocessor or the processor 82 may be any conventional processor, etc. The processor 82 is a control center of the electronic device 8, and uses various interfaces and lines to connect the entire electronic device 8. Various parts.

The memory 81 may be used to store the computer program 83 and / or module / unit, and the processor 82 runs or executes the computer program and / or module / unit stored in the memory 81, And calling data stored in the memory 81 to implement various functions of the electronic device 8. The memory 81 may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, at least one application required by a function (such as a sound playback function, an image playback function, etc.), and the like; storage data The area may store materials (such as audio materials, phone books, etc.) created according to the use of the electronic device 8. In addition, the memory 81 may include a high-speed random access memory and a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), and a Secure Digital (Secure Digital , SD) card, flash memory card (Flash Card), at least one disk memory device, flash memory device, or other volatile solid state memory device.

When the integrated module / unit of the electronic device 8 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the method of the above embodiment, and can also be completed by controlling related hardware with a computer program, which can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the foregoing method embodiments can be implemented. The computer program includes computer program code, and the computer program code may be in a form of an original code, a form of an object code, an executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read- Only Memory), Random Access Memory (RAM, Random Access Memory), electric carrier signals, telecommunication signals, and software distribution media. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdictions. For example, in some jurisdictions, the computer-readable medium Excludes electric carrier signals and telecommunication signals.

In the several embodiments provided by the present invention, it should be understood that the disclosed electronic device and method can be implemented in other ways. For example, the embodiments of the electronic device described above are merely schematic.

It is obvious to a person having ordinary skill in the art that the present invention is not limited to the details of the above-mentioned exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, regardless of the point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the scope of the attached patent application rather than the above description, so it is intended to fall into the application. The meaning of the equivalent scope of the patent scope and all changes within the scope are included in the present invention. Any reference sign in the scope of a patent application should not be considered as limiting the scope of the patent application involved. Furthermore, it is clear that the word "comprising" does not exclude other units or that the singular does not exclude the plural. Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limiting. Although the present invention is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be Modifications or equivalent substitutions can be made without departing from the spirit and scope of the technical solution of the present invention.

8‧‧‧ electronic device

81‧‧‧Memory

82‧‧‧ processor

83‧‧‧Computer Program

FIG. 1 is a flowchart of an image segmentation method according to a first embodiment of the present invention. FIG. 2 is a bar graph of a depth image according to a preferred embodiment of the present invention. FIG. 3 is a schematic diagram of a category center obtained according to a preferred embodiment of the present invention. FIG. 4 is a schematic diagram of determining a segmentation threshold according to a preferred embodiment of the present invention. FIG. 5 is a flowchart of an image segmentation method provided by Embodiment 2 of the present invention. FIG. 6 is a schematic diagram of an electronic device according to a third embodiment of the present invention.

no

Claims (10)

An image segmentation method, wherein the method includes: obtaining an image and a depth image of the image; obtaining a bar graph of the depth image, the bar graph including an abscissa and an ordinate; using a clustering algorithm Cluster the bar graph data to obtain two categories and the centers of the two categories; determine that the abscissa corresponding to the smallest vertical coordinate value between the centers of the two categories in the bar graph is the segmentation threshold; determine the Whether the segmentation threshold value satisfies a preset condition; and when the segmentation threshold value satisfies the preset condition, the image is segmented into two regions of a foreground and a background according to the segmentation threshold. The image segmentation method according to item 1 of the scope of patent application, wherein obtaining the depth image of the image comprises: obtaining a depth image of the image by using a binocular matching algorithm or a depth image acquisition device. The image segmentation method according to item 1 of the scope of patent application, wherein: the preset condition is that an average value of ordinate values in a preset range centered on the segmentation threshold in the bar graph is smaller than a preset value value. The image segmentation method according to item 1 of the scope of patent application, wherein: the preset condition is that an ordinate value corresponding to the segmentation threshold in the bar graph is smaller than a preset value. The image segmentation method according to item 1 of the scope of patent application, wherein the method further comprises: when the segmentation threshold does not satisfy the preset condition, determining that the image is not suitable for segmentation into the foreground and the background Two areas. The image segmentation method according to item 1 of the patent application scope, wherein: the abscissa in the bar graph represents a depth value of a primitive of the depth image; and the vertical bar in the bar graph The coordinates represent the number of the primitives corresponding to each of the depth values. The image segmentation method according to item 1 of the scope of patent application, wherein: the clustering algorithm is a K-means algorithm. A computer program, wherein the computer program includes at least one computer program instruction, and the at least one computer program instruction is executed by a processor to implement the method described in any one of claims 1 to 7 of the scope of patent application. Image segmentation method. A storage medium, wherein the storage medium stores at least one computer program instruction, and the at least one computer program instruction is used to be executed by a processor to implement the method according to any one of claims 1 to 7 of the scope of patent application. Image segmentation method. An electronic device, wherein the electronic device includes a processor and a memory, and the processor is configured to execute at least one computer program instruction stored in the memory to implement any one of items 1 to 7 of the scope of patent application. An image segmentation method as described in one item.