CN116228567A - Image blurring method, device and live video system - Google Patents
Image blurring method, device and live video system Download PDFInfo
- Publication number
- CN116228567A CN116228567A CN202211734708.0A CN202211734708A CN116228567A CN 116228567 A CN116228567 A CN 116228567A CN 202211734708 A CN202211734708 A CN 202211734708A CN 116228567 A CN116228567 A CN 116228567A
- Authority
- CN
- China
- Prior art keywords
- image
- blurring
- pixel
- semitransparent
- range
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Image Processing (AREA)
- Processing Of Color Television Signals (AREA)
Abstract
The application relates to an image blurring method, an image blurring device, a video live broadcast system, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: acquiring a semitransparent channel image of a foreground image; counting the transverse pixel proportion of the communication area in the semitransparent channel image row by row and counting the longitudinal pixel proportion of the communication area in the semitransparent channel image row by row; calculating a target fuzzy range matched with each pixel point of the semitransparent channel image according to the transverse pixel ratio and the longitudinal pixel ratio; respectively carrying out fuzzy processing on each pixel point of the semitransparent channel image according to the target fuzzy range; according to the technical scheme, the blurring range of the image can be adjusted in a self-adaptive mode, the defect that color toning effects are not matched due to the fact that the blurring ranges are globally consistent is avoided, and the color toning effects are improved.
Description
Technical Field
The application relates to the technical field of image processing, in particular to an image blurring method, an image blurring device, a video live broadcast system, electronic equipment and a computer readable storage medium.
Background
Image blurring is an image smoothing technique, which refers to a process of reducing or eliminating sharp edges, noise and other high-frequency information in an image; the image blurring is generally realized through low-pass filtering, and common image blurring methods include mean blurring, gaussian blurring, kawase blurring, double blurring and the like.
In the current common blurring method, the blurring ranges of different pixels on an image are consistent, while in some live video systems, different areas of the image require different ranges of image blurring for implementation under different scenes. For example, in the process of combining the foreground and the background of an image after green screen matting (assuming that the foreground contains a person), a blurring process is required for a semitransparent channel of the foreground to obtain a mask for further processing the foreground picture, so that the blurring range at a narrower part of a human body is smaller, and the blurring range at a wider part of the human body is larger.
However, after the common image blurring method uses the image blurring process with the globally consistent blurring range, the mask generated based on the image blurring process may cause image color imbalance, obviously not match with the image content, and affect the image blurring effect.
Disclosure of Invention
Based on this, it is necessary to provide an image blurring method, an apparatus, a video live broadcast system, an electronic device and a computer readable storage medium to improve the image blurring application effect in view of at least one of the above technical drawbacks.
An image blurring method comprising:
acquiring a semitransparent channel image of a foreground image;
counting the transverse pixel proportion of the communication area in the semitransparent channel image row by row and counting the longitudinal pixel proportion of the communication area in the semitransparent channel image row by row;
calculating a target fuzzy range matched with each pixel point of the semitransparent channel image according to the transverse pixel ratio and the longitudinal pixel ratio;
and respectively carrying out blurring processing on each pixel point of the semitransparent channel image according to the target blurring range.
In one embodiment, acquiring a semitransparent channel image of a foreground image includes:
carrying out image matting on an input image to obtain a portrait image, and generating a semitransparent channel image of the portrait image to obtain a portrait region image;
and dividing the input image by a face segmentation technology and generating a semitransparent channel image to obtain a face region image.
In one embodiment, counting the lateral pixel duty cycle of the connected region in the semitransparent channel image row by row includes:
counting the number of pixels contained in each connected region of the portrait region image line by line;
selecting a communication area with the largest pixel number to calculate the transverse pixel ratio; wherein the transverse pixel duty ratio is a proportion value of the number of pixels to the number of the whole row of pixels;
the counting the longitudinal pixel ratio of the connected region in the semitransparent channel image column by column comprises the following steps:
counting the number of pixels contained in each connected region of the portrait region image column by column;
selecting a communication area with the largest pixel number to calculate the longitudinal pixel ratio; the vertical pixel duty ratio is a proportion value of the number of pixels to the number of pixels in the whole column.
In one embodiment, the image blurring method further includes:
acquiring the number of pixels contained in a communication area belonging to the face area range in the line-by-line statistical face area image; calculating the transverse pixel ratio of the face communication area; wherein the transverse pixel duty ratio is a proportion value of the number of pixels to the number of the whole row of pixels;
acquiring the number of pixels contained in a connected region belonging to a face region range in a row-by-row statistical face region image; calculating the longitudinal pixel ratio of the face communication area; the vertical pixel duty ratio is a proportion value of the number of pixels to the number of pixels in the whole column.
In one embodiment, calculating the target blur range of each pixel point match of the semitransparent channel image according to the transverse pixel ratio and the longitudinal pixel ratio comprises:
for each pixel point of the semitransparent channel image, respectively acquiring the transverse pixel duty ratio of the row and the longitudinal pixel duty ratio of the column;
selecting the smaller of the horizontal pixel duty cycle and the vertical pixel duty cycle as a correlation coefficient;
and calculating the target fuzzy range matched with the pixel point according to the correlation coefficient and the basic fuzzy range.
In one embodiment, the image blurring method further includes:
setting a minimum blurring range;
and when the calculated target fuzzy range is smaller than the minimum fuzzy range, taking the minimum fuzzy range as the target fuzzy range of the pixel point.
In one embodiment, the blurring processing is performed on each pixel point of the semitransparent channel image according to the target blurring range, including:
setting the convolution kernel size of a convolution-based blurring method according to the target blurring range;
and carrying out blurring processing on each pixel point of the semitransparent channel image by using the blurring method to obtain a corresponding blurred image.
In one embodiment, the blurring processing is performed on each pixel point of the semitransparent channel image according to the target blurring range, including:
setting a convolution kernel with a fixed size based on a convolution fuzzy method;
calculating corresponding downsampling times and sampling step sizes according to the fuzzy range of each pixel point;
setting the global downsampling times of the double blurring method as the maximum downsampling times in all pixels;
and performing double blurring processing on each pixel point of the semitransparent channel image in parallel.
An image blurring apparatus comprising:
the acquisition module is used for acquiring a semitransparent channel image of the foreground image;
the statistics module is used for counting the transverse pixel proportion of the communication area in the semitransparent channel image row by row and counting the longitudinal pixel proportion of the communication area in the semitransparent channel image row by row;
the calculating module is used for calculating a target fuzzy range matched with each pixel point of the semitransparent channel image according to the transverse pixel ratio and the longitudinal pixel ratio;
and the blurring module is used for respectively blurring each pixel point of the semitransparent channel image according to the target blurring range.
A video live broadcast system, comprising: at least one anchor end and a live broadcast server; wherein, the live broadcast server is connected with each audience terminal through a network;
the anchor terminal is used for acquiring an anchor video image and carrying out fuzzy processing on the anchor video image by adopting the image fuzzy method;
the server is used for receiving the anchor video image uploaded by the anchor terminal and transmitting the anchor video image to each audience terminal for playing.
An electronic device, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the image blurring method described above.
A computer readable storage medium storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded by the processor and performing the image blurring method described above.
The image blurring method, the image blurring device, the video live broadcast system, the electronic equipment and the computer readable storage medium firstly acquire a semitransparent channel image of a foreground image, then count pixel occupation ratios of a communication area row by row/column, calculate a target blurring range matched with each pixel point according to the counted pixel occupation ratios, and adaptively carry out blurring processing on each pixel point of the semitransparent channel image; according to the technical scheme, the target fuzzy range matched with the pixels is generated by counting the pixel proportion of the communication area row by row/column, so that the fuzzy range of the image can be adjusted in a self-adaptive mode, the defect that the color toning effect is not matched due to the fact that the fuzzy ranges are in global consistency is avoided, and the color toning effect is improved.
Further, the size of the bright pixel point area in the effective mask in each row/column of pixels can be obtained by scanning the connected area of the image row/column, the foreground image containing a human body is represented by the largest connected area, the pixel occupancy rate is low at the narrower part of the human body, and the pixel occupancy rate is high at the wider part of the human body, so that when the mask for further processing the foreground image is obtained by the image blurring process, the blurring range can be adjusted according to the pixel occupancy rate, and different color schemes can be conveniently realized.
Furthermore, on the basis of double blurring, the blurring operation scheme with each pixel point having self-adaptive different blurring ranges is realized, the method can well operate on various terminal devices, and the defects of stripe problem or large calculated amount caused by large sampling step length are avoided.
Drawings
FIG. 1 is a flow chart of an image blurring method of an embodiment;
FIG. 2 is a schematic illustration of an exemplary semi-transparent channel image;
FIG. 3 is a schematic illustration of an exemplary segmentation of a region of interest image and a region of interest image;
FIG. 4 is a schematic diagram of an exemplary fuzzy operation;
FIG. 5 is a flow diagram of an image blurring operation of one embodiment;
FIG. 6 is a schematic diagram of an exemplary double fuzzy operation;
FIG. 7 is a schematic diagram of a conventional blurred image method for picture composition;
FIG. 8 is a schematic diagram of a frame composition of the blurred image method of the present application;
fig. 9 is a schematic structural view of an image blurring apparatus of an embodiment;
fig. 10 is a block diagram of an example electronic device.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
In the embodiments of the present application, reference to "first," "second," etc. is used to distinguish between identical items or similar items that have substantially the same function and function, "at least one" means one or more, "a plurality" means two or more, e.g., a plurality of objects means two or more. The words "comprise" or "comprising" and the like mean that information preceding the word "comprising" or "comprises" is meant to encompass the information listed thereafter and equivalents thereof as well as additional information not being excluded. Reference to "and/or" in the embodiments of the present application indicates that there may be three relationships, and the character "/" generally indicates that the associated object is an "or" relationship.
In the present application, image blurring refers to the input of a sharp image I ε R H×W×C Obtaining a corresponding blurred image I through an image processing scheme blur ∈R H×W×C Where W and H denote the width and height of a picture, respectively, the subscript blu denotes blurring processing, and C denotes the number of channels of an input picture. The technical scheme of the application is mainly applied to a semitransparent channel image I obtained by matting a foreground image of an input picture, and generally C=1, and color matching application is realized through blurring processing.
Referring to fig. 1, fig. 1 is a flowchart of an image blurring method according to an embodiment, including the steps of:
s10, acquiring a semitransparent channel image of the foreground image.
In this step, a semitransparent channel image may be generated according to a foreground image of an input picture, as shown in fig. 2, fig. 2 is a schematic diagram of an exemplary semitransparent channel image, and the left side is the input picture, and the foreground image is obtained after the portrait image is scratched, and a corresponding semitransparent channel image is generated.
In the image blurring process, in order to realize the color matching of different colors of the human face and the human image part, the human face image and the human image can be respectively segmented, so that the image blurring process can be respectively carried out.
Accordingly, in one embodiment, step S10 may include the following:
carrying out image matting on an input image to obtain a portrait image, and generating a semitransparent channel image of the portrait image to obtain a portrait region image; and dividing the input image by a face segmentation technology and generating a semitransparent channel image to obtain a face region image.
Specifically, as shown in fig. 3, fig. 3 is an exemplary schematic diagram of a face region image and a face region image obtained by segmentation, where the face region image F e R may be obtained by a face segmentation technique based on the user having obtained the face region image I by a matting technique H×W The human image area image and the human face area image are obtained, so that the human image area image and the human face area image can be respectively subjected to different-range color matching or no color matching in the blurring process.
According to the technical scheme, the human image region image and the human face region image can be obtained respectively, and on the basis, different blurring ranges can be set for the two semitransparent channel images respectively in the image blurring process, so that different blurring effects are achieved.
S20, counting the transverse pixel proportion of the communication area in the semitransparent channel image row by row and counting the longitudinal pixel proportion of the communication area in the semitransparent channel image row by row.
In the step, for the pixel points with brightness in the semitransparent channel image, the pixel ratio of the whole row/column occupied by the pixel number of the connected area containing the foreground image content is counted row by row/column, the pixel ratio is respectively marked as a horizontal pixel ratio and a vertical pixel ratio, and the connected area (Connected Component) refers to an image area formed by foreground pixel points which have the same pixel value and are adjacent in position in the image.
As an example, for the portrait area image, the statistical method of step S20 may be as follows:
counting the number of pixels contained in each connected region of the portrait region image line by line from left to right; selecting a communication area with the largest pixel number to calculate the transverse pixel ratio; wherein the transverse pixel duty ratio is a proportion value of the number of pixels to the number of the whole row of pixels;
counting the number of pixels contained in each connected region of the portrait region image from top to bottom; selecting a communication area with the largest pixel number to calculate the longitudinal pixel ratio; the vertical pixel duty ratio is a proportion value of the number of pixels to the number of pixels in the whole column.
Specifically, in the lateral proportion statistics, for the H (H is more than or equal to 1 and less than or equal to H) row of the portrait region image, all K is counted from left to right h A number n of pixels included in each of the communication regions k (1≤k≤K h ) Selecting the number of pixels
Maximum communication area->
For representing the connected area of the row, for calculating the transversal pixel duty of the portrait area image, provided that the finally calculated scale value +.>
For the horizontal pixel duty ratio of the row, recording the statistical result of the horizontal pixel duty ratio as S hor (x,y)∈R H×W . In the longitudinal directionIn the proportion statistics, the proportion value of each column of the image of the portrait region can be obtained similarly to the transverse direction
Recording the vertical pixel duty ratio statistical result as S ver (x,y)∈R H×W (x, y) represents pixel coordinates.
Further, the statistics of the pixel duty ratio of the face area image may further include the following steps when the face area image is counted:
acquiring the number of pixels contained in a communication area belonging to the face area range in the line-by-line statistical face area image; calculating the transverse pixel ratio of the face communication area; the transverse pixel duty ratio is a proportion value of the number of pixels to the number of pixels in the whole row.
Acquiring the number of pixels contained in a connected region belonging to a face region range in a row-by-row statistical face region image; calculating the longitudinal pixel ratio of the face communication area; the vertical pixel duty ratio is a proportion value of the number of pixels to the number of pixels in the whole column.
Specifically, when the face region image F is encountered during the statistics of the transverse pixel duty ratio, a connected region corresponding to the face region image is selected
To represent the connected region of the row of pixels, the transverse pixel duty cycle of the face region image is calculated. Correspondingly, when the vertical pixel duty ratio statistics is carried out, when the face region image F is encountered, the communication region corresponding to the face region image is selected>
For the connected region representing the column of pixels, the vertical pixel duty ratio of the face region image is calculated.
According to the technical scheme of the embodiment, the size of the bright pixel point area in the effective mask in each row/column of pixels can be obtained by scanning the connected areas of the images row by row/column, the pixels of the row are represented by the largest connected area for a foreground image containing a human body, the pixel occupation ratio is low at the narrower part of the human body, and the pixel occupation ratio is high at the wider part of the human body, so that when the mask for further processing the foreground image is obtained through image blurring processing, the blurring range can be adjusted according to the pixel occupation ratio, and different color schemes can be conveniently realized.
S30, calculating a target blurring range matched with each pixel point of the semitransparent channel image according to the transverse pixel ratio and the longitudinal pixel ratio.
In the step, according to the statistical results of the horizontal pixel duty ratio and the vertical pixel duty ratio, a corresponding blurring range is set for each pixel point of the image of the human image area and the image of the human face area, so that the blurring range can be adaptively adjusted according to the image content, and different color tuning effects are realized.
In one embodiment, for a method of matching a target fuzzy range, the following may be included:
for each pixel point of the semitransparent channel image, respectively acquiring the transverse pixel duty ratio of the row and the longitudinal pixel duty ratio of the column; selecting the smaller of the horizontal pixel duty cycle and the vertical pixel duty cycle as a correlation coefficient; and calculating the target fuzzy range matched with the pixel point according to the correlation coefficient and the basic fuzzy range.
Further, when calculating the target blur range, a minimum blur range may be set, and when the calculated target blur range is smaller than the minimum blur range, the minimum blur range is used as the target blur range of the pixel point.
Specifically, each pixel I when outputting a blurred image blur (x, y) fuzzy range and transverse pixel duty ratio statistical result S hor (x, y) and vertical pixel duty statistics S ver (x, y) correlation, in the present embodiment S is selected hor (x, y) and S ver The smaller of (x, y) is used as the correlation coefficient; in practical application, S can also be calculated hor (x, y) and S ver The mean of (x, y) to obtain the correlation coefficient. Then calculating a target fuzzy range k matched with the pixel points x,y ＝max[min(S hor (x,y),S ver (x,y))·k,k min ]Wherein k represents the basic blurring range, k min Representing the minimum blur range.
According to the technical scheme, when a blurred picture is generated by using an image blurring method, a target blurring range can be adaptively matched with any pixel point in the picture, and the blurring range is positively correlated with the pixel duty ratio of bright pixels in a row and a column where the blurring range is located, so that a high-performance color mixing scheme can be realized in a blurring process; by setting the minimum blurring range, the minimum blurring effect of the image can be ensured.
And S40, respectively carrying out blurring processing on each pixel point of the semitransparent channel image according to the target blurring range.
In the step, each pixel point of the portrait area image and the face area image is respectively subjected to blurring processing to obtain a blurred image I blur The self-adaptive change effect of the blurring range of each pixel point in the image blurring picture along with the picture content is realized.
In one embodiment, for the method of blurring processing, the following steps may be included:
setting the convolution kernel size of a convolution-based blurring method according to the target blurring range; and carrying out blurring processing on each pixel point of the semitransparent channel image by using the blurring method to obtain a corresponding blurred image.
Specifically, referring to fig. 4, fig. 4 is a schematic diagram illustrating an exemplary blurring operation, in which a portrait region image and a face region image are obtained by segmentation, and each pixel point I is obtained during the blurring process blur Matching of (x, y) by statistics S hor (x, y) and S ver (x, y) the calculated blur range k x,y Applying convolution-based blurring methods (e.g., gaussian, mean, median) and based on the range k of the blur x, The size of the corresponding convolution kernel (sampling range) is set, so that the blurring range of each pixel point of the human image region image and the human face region image is adaptive to follow the change of the pixel point.
According to the technical scheme of the embodiment, the size of the convolution kernel is continuously adjusted according to different blurring ranges of each pixel point in the blurring process based on a common convolution blurring method, and therefore the effect of adaptively changing the blurring range can be achieved.
In addition, in the above embodiment, when the required blurring range is larger, the number of pixels to be sampled is larger, resulting in larger calculation amount, when the calculation amount is too large, it is difficult to support real-time application on the middle-low end device, if the size of the convolution kernel is fixed and the step length of the pixel sampling is adjusted to realize blurring in a large range, there is a problem of selection of the size and effect of the convolution kernel (calculation amount), the problem of streaks caused by long-distance sampling occurs when the convolution kernel is small, and the problem of large calculation amount occurs when the convolution kernel is large.
In order to realize the blurring processing method of the adaptive change of the blurring range with high performance, the application further provides a blurring processing scheme based on double blurring, and referring to fig. 5, fig. 5 is a flowchart of an image blurring operation according to an embodiment, which may include the following steps:
s401, setting a convolution kernel with a fixed size based on a convolution-based blurring method.
The double blurring is a blurring method which adopts convolution kernel with fixed size, iterates blurring and down (up) sampling for a plurality of times to realize a larger range, and the blurring range is determined by the sampling step length lambda of the pixel point and the down (up) sampling frequency n; in general, the larger the number of downsampling times n, the larger the sampling step size, and the larger the blurring range.
S402, corresponding down (up) sampling times and sampling step sizes are calculated according to the fuzzy range of each pixel point.
Since the down (up) sampling times and the sampling steps together determine the blurring range of the double blurring, and the target blurring ranges of the respective pixels are different, a function lambda can be constructed in this embodiment x, ,n x, ＝f(k x, ) For passing through the target blur range k x, To calculate the number of downsampling times n of each pixel in real time x, Sampling step lambda x, 。
S403, setting the global downsampling frequency of the double blurring method as the maximum downsampling frequency in all pixels.
In a rendering engine running based on a GPU, each pixel point performs image blurring calculation in parallel, and different pixel points share the same processing logic. Here, for the target blur range with a small part of pixels, which is enlarged by the influence of downsampling (upsampling), the present embodiment uses the largest downsampling number in all pixels in the blur calculation process based on the case that the blur ranges of the double blur are globally uniform
Set to the global number of downsampling.
And S404, performing double blurring processing on each pixel point of the semitransparent channel image in parallel.
According to the downsampling times n x,y Sampling step lambda x,y Performing double blurring processing, and calculating an up-sampling layer U during up-sampling i At the time of (1), let pixel point U i The (x, y) is calculated as:
wherein D is i (x, y) means downsampling calculation pixel points, blur means blurring operator, and for pixel points with smaller blurring range, the upsampling times are also smaller than the maximum downsampling times
The pixel blurring operation result can be taken out in advance.
As shown in fig. 6, fig. 6 is a schematic diagram of an exemplary dual fuzzy operation, in which, when performing a fuzzy process of adaptively changing a fuzzy range, a rendering engine performs downsampling calculation on each pixel point in parallel, and then performs upsampling calculation in parallel, and since sampling times of a part of pixel points are small, a required fuzzy calculation result is obtained by performing calculation in advance, and in this case, a required result can be obtained in advance when the fuzzy calculation obtains a corresponding sampling time, as in the figure, a required result can be obtained in U 2 ，U 1 Or U 0 And (5) taking out the required fuzzy operation result in the layering process.
According to the technical scheme, on the basis of double blurring, the blurring operation scheme with self-adaptive different blurring ranges of each pixel point is realized, the technical scheme can well operate on various terminal equipment, and the defects of stripe problem or large calculated amount caused by large sampling step length are avoided, so that the method is a high-performance blurring operation method.
The image blurring method according to the above embodiment can be applied as a general image processing technology in various scenes, such as meta-universe, movie creation, short video, network video live broadcast, image editing and creation, etc.
Taking an application in a video live broadcast system as an example, the method acts on a picture synthesis process in the video live broadcast image matting. Referring to fig. 7, fig. 7 is a schematic diagram of picture composition by a common blurred image method, and when a live video image of a main cast is edited, a live video is often shot under a green screen background, and then a proper background image is added for picture composition, so as to obtain a required live video picture; when the video image is edited, firstly, a semitransparent channel image of the anchor portrait image is obtained through green screen matting, then a needed background image is added for synthesis, the portrait image and the background image are fused through fuzzy processing, and the brightness and the color of a foreground image are adjusted according to background brightness and color information in the synthesized image through fuzzy image processing. When the common image blurring method is adopted, the blurring range is globally consistent, so that all parts of the human body image in the figure have the same color mixing effect, and particularly the part of the human face image also has more background colors, so that the effect of the synthesized image is very mismatched.
After the technical scheme provided by the application is adopted, referring to fig. 8, fig. 8 is a picture synthesis schematic diagram of the blurred image method, and since the image blurring range can be adaptively adjusted, the blurring range is positively related to the size of a communication region in a picture, the adjusting range of the foreground can be intelligently adjusted during color mixing, so that the color of the synthesized picture at the edge of a figure image can be well adjusted, and the face region can be adaptively matched with a smaller blurring range, thereby avoiding the influence of large-range blurring on the color and brightness of the face region, and enabling the effect of the synthesized image to be very matched.
An embodiment of the image blurring apparatus is set forth below.
Referring to fig. 9, fig. 9 is a schematic structural diagram of an image blurring apparatus according to an embodiment, including:
an acquisition module 10 for acquiring a semitransparent channel image of a foreground image;
a statistics module 20, configured to count, row by row, a transversal pixel duty ratio of a connected region in the semitransparent channel image, and count, column by column, a longitudinal pixel duty ratio of the connected region in the semitransparent channel image;
a calculating module 30, configured to calculate a target blur range matched with each pixel point of the semitransparent channel image according to the transverse pixel ratio and the longitudinal pixel ratio;
and the blurring module 40 is used for respectively blurring each pixel point of the semitransparent channel image according to the target blurring range.
The image blurring apparatus of the present embodiment may perform an image blurring method provided in the embodiments of the present application, and its implementation principle is similar, and actions performed by each module in the image blurring apparatus of each embodiment of the present application correspond to steps in the image blurring method of each embodiment of the present application, and detailed functional descriptions of each module in the image blurring apparatus may be specifically referred to the descriptions in the corresponding image blurring method shown in the foregoing, which are not repeated herein.
Embodiments of video live systems, electronic devices, and computer readable storage media are set forth below.
The application also provides a video live broadcast system which mainly comprises at least one main broadcasting end and a live broadcast server in a structure; wherein, the live broadcast server is connected with each audience terminal through a network; each user can watch video images of the anchor through the audience terminal. The image blurring scheme provided by the application can be applied to a main broadcasting terminal, the main broadcasting terminal can acquire a foreground image of a portrait image after acquiring a video image of a main broadcasting through a camera, for example, the main broadcasting portrait image is obtained through a green curtain matting mode, then a semitransparent channel image is generated, in order to add a new background image to the portrait image of the main broadcasting, a mask is added to the semitransparent channel image of the portrait image, blurring processing is carried out, and a more fused matching effect is achieved between the finally synthesized main broadcasting video image and the background image. And uploading the processed anchor video image to a server by the anchor terminal, and carrying out related processing after the server receives the anchor video image uploaded by the anchor terminal and transmitting the anchor video image to each audience terminal for playing.
The application provides a technical scheme of electronic equipment, which is used for realizing related functions of an image blurring method.
In one embodiment, the present application provides an electronic device comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in memory and configured to be executed by the one or more processors, the one or more applications configured for use with the image blur method of any embodiment.
As shown in fig. 10, fig. 10 is a block diagram of an example electronic device. The electronic device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, or the like. The apparatus may include one or more of the following components: a processing component 1002, a memory 1004, a power component 1006, a multimedia component 1008, an audio component 1009, an input/output (I/O) interface 1012, a sensor component 1014, and a communication component 1016.
The processing component 1002 generally controls overall operation of the apparatus 1000, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.
The memory 1004 is configured to store various types of data to support operations at the device 1000. Such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
The power supply component 1006 provides power to the various components of the device 1000.
The multimedia component 1009 includes a screen between the device 1000 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). In some embodiments, the multimedia assembly 1008 includes a front-facing camera and/or a rear-facing camera.
The audio component 1009 is configured to output and/or input audio signals.
The I/O interface 1012 provides an interface between the processing assembly 1002 and peripheral interface modules, which may be a keyboard, click wheel, buttons, and the like. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.
The sensor assembly 1014 includes one or more sensors for providing status assessment of various aspects of the device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact.
The communication component 1016 is configured to facilitate communication between the apparatus 1000 and other devices, either wired or wireless. The device 1000 may access a wireless network based on a communication standard, such as WiFi, an operator network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof.
The application provides a technical scheme of a computer readable storage medium, which is used for realizing related functions of an image blurring method. The computer readable storage medium stores at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded by a processor and performing the image blurring method of any embodiment.
In an exemplary embodiment, the computer-readable storage medium may be a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, for example, the non-transitory computer-readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
The foregoing examples represent only a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (12)
1. An image blurring method, comprising:
acquiring a semitransparent channel image of a foreground image;
counting the transverse pixel proportion of the communication area in the semitransparent channel image row by row and counting the longitudinal pixel proportion of the communication area in the semitransparent channel image row by row;
calculating a target fuzzy range matched with each pixel point of the semitransparent channel image according to the transverse pixel ratio and the longitudinal pixel ratio;
and respectively carrying out blurring processing on each pixel point of the semitransparent channel image according to the target blurring range.
2. The image blurring method of claim 1 wherein acquiring a semitransparent channel image of a foreground image comprises:
carrying out image matting on an input image to obtain a portrait image, and generating a semitransparent channel image of the portrait image to obtain a portrait region image;
and dividing the input image by a face segmentation technology and generating a semitransparent channel image to obtain a face region image.
3. The image blurring method of claim 2 wherein counting the lateral pixel duty ratio of the connected region in the semitransparent channel image line by line comprises:
counting the number of pixels contained in each connected region of the portrait region image line by line;
selecting a communication area with the largest pixel number to calculate the transverse pixel ratio; wherein the transverse pixel duty ratio is a proportion value of the number of pixels to the number of the whole row of pixels;
the counting the longitudinal pixel ratio of the connected region in the semitransparent channel image column by column comprises the following steps:
counting the number of pixels contained in each connected region of the portrait region image column by column;
selecting a communication area with the largest pixel number to calculate the longitudinal pixel ratio; the vertical pixel duty ratio is a proportion value of the number of pixels to the number of pixels in the whole column.
4. The image blurring method of claim 3 further comprising:
acquiring the number of pixels contained in a communication area belonging to the face area range in the line-by-line statistical face area image; calculating the transverse pixel ratio of the face communication area; wherein the transverse pixel duty ratio is a proportion value of the number of pixels to the number of the whole row of pixels;
acquiring the number of pixels contained in a connected region belonging to a face region range in a row-by-row statistical face region image; calculating the longitudinal pixel ratio of the face communication area; the vertical pixel duty ratio is a proportion value of the number of pixels to the number of pixels in the whole column.
5. The image blurring method of claim 1 wherein calculating a target blurring range for each pixel point match of the semitransparent channel image based on the horizontal pixel ratio and vertical pixel ratio comprises:
for each pixel point of the semitransparent channel image, respectively acquiring the transverse pixel duty ratio of the row and the longitudinal pixel duty ratio of the column;
selecting the smaller of the horizontal pixel duty cycle and the vertical pixel duty cycle as a correlation coefficient;
and calculating the target fuzzy range matched with the pixel point according to the correlation coefficient and the basic fuzzy range.
6. The image blurring method of claim 1 further comprising:
setting a minimum blurring range;
and when the calculated target fuzzy range is smaller than the minimum fuzzy range, taking the minimum fuzzy range as the target fuzzy range of the pixel point.
7. The image blurring method according to any one of claims 1 to 6, wherein blurring processing is performed on each pixel point of the semitransparent channel image according to the target blurring range, respectively, comprising:
setting the convolution kernel size of a convolution-based blurring method according to the target blurring range;
and carrying out blurring processing on each pixel point of the semitransparent channel image by using the blurring method to obtain a corresponding blurred image.
8. The image blurring method according to any one of claims 1 to 6, wherein blurring processing is performed on each pixel point of the semitransparent channel image according to the target blurring range, respectively, comprising:
setting a convolution kernel with a fixed size based on a convolution fuzzy method;
calculating corresponding downsampling times and sampling step sizes according to the fuzzy range of each pixel point;
setting the global downsampling times of the double blurring method as the maximum downsampling times in all pixels;
and performing double blurring processing on each pixel point of the semitransparent channel image in parallel.
9. An image blurring apparatus, comprising:
the acquisition module is used for acquiring a semitransparent channel image of the foreground image;
the statistics module is used for counting the transverse pixel proportion of the communication area in the semitransparent channel image row by row and counting the longitudinal pixel proportion of the communication area in the semitransparent channel image row by row;
the calculating module is used for calculating a target fuzzy range matched with each pixel point of the semitransparent channel image according to the transverse pixel ratio and the longitudinal pixel ratio;
and the blurring module is used for respectively blurring each pixel point of the semitransparent channel image according to the target blurring range.
10. A video live broadcast system, comprising: at least one anchor end and a live broadcast server; wherein, the live broadcast server is connected with each audience terminal through a network;
the anchor terminal is used for acquiring anchor video images and carrying out fuzzy processing on the anchor video images by adopting the image fuzzy method as set forth in any one of claims 1-8;
the server is used for receiving the anchor video image uploaded by the anchor terminal and transmitting the anchor video image to each audience terminal for playing.
11. An electronic device, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the image blurring method of any of claims 1-8.
12. A computer readable storage medium, characterized in that the storage medium stores at least one instruction, at least one program, a set of codes or a set of instructions, the at least one instruction, the at least one program, the set of codes or the set of instructions being loaded by the processor and performing the image blurring method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211734708.0A CN116228567A (en) | 2022-12-30 | 2022-12-30 | Image blurring method, device and live video system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211734708.0A CN116228567A (en) | 2022-12-30 | 2022-12-30 | Image blurring method, device and live video system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116228567A true CN116228567A (en) | 2023-06-06 |
Family
ID=86577783
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211734708.0A Pending CN116228567A (en) | 2022-12-30 | 2022-12-30 | Image blurring method, device and live video system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116228567A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070286520A1 (en) * | 2006-06-07 | 2007-12-13 | Microsoft Corporation | Background blurring for video conferencing |
| CN105874506A (en) * | 2013-10-04 | 2016-08-17 | 谷歌公司 | Blurred images with detail preservation |
| CN107704798A (en) * | 2017-08-09 | 2018-02-16 | 广东欧珀移动通信有限公司 | Image blurring method, device, computer readable storage medium and computer equipment |
| CN113379623A (en) * | 2021-05-31 | 2021-09-10 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN113673474A (en) * | 2021-08-31 | 2021-11-19 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| CN113709365A (en) * | 2021-08-20 | 2021-11-26 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN115150606A (en) * | 2021-03-31 | 2022-10-04 | 武汉Tcl集团工业研究院有限公司 | Image blurring method and device, storage medium and terminal equipment |
-
2022
- 2022-12-30 CN CN202211734708.0A patent/CN116228567A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070286520A1 (en) * | 2006-06-07 | 2007-12-13 | Microsoft Corporation | Background blurring for video conferencing |
| CN105874506A (en) * | 2013-10-04 | 2016-08-17 | 谷歌公司 | Blurred images with detail preservation |
| CN107704798A (en) * | 2017-08-09 | 2018-02-16 | 广东欧珀移动通信有限公司 | Image blurring method, device, computer readable storage medium and computer equipment |
| CN115150606A (en) * | 2021-03-31 | 2022-10-04 | 武汉Tcl集团工业研究院有限公司 | Image blurring method and device, storage medium and terminal equipment |
| CN113379623A (en) * | 2021-05-31 | 2021-09-10 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN113709365A (en) * | 2021-08-20 | 2021-11-26 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN113673474A (en) * | 2021-08-31 | 2021-11-19 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 王振阳: "基于视频图像序列的背景虚化", 中国优秀硕士论文电子期刊网, 15 July 2016 (2016-07-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Learning to enhance low-light image via zero-reference deep curve estimation | |
| KR101743861B1 (en) | Methods of image fusion for image stabilization | |
| US20160065864A1 (en) | System and method for online processing of video images in real time | |
| CN110956219B (en) | Video data processing method, device and electronic system | |
| CN112767281B (en) | Image ghost elimination method, device, electronic device and storage medium | |
| US9449371B1 (en) | True motion based temporal-spatial IIR filter for video | |
| JP2003058894A (en) | Method and apparatus for segmenting a pixelated image | |
| CN112308797B (en) | Corner point detection method, device, electronic equipment and readable storage medium | |
| CN111401411A (en) | Method and device for acquiring sample image set | |
| WO2023273868A1 (en) | Image denoising method and apparatus, terminal, and storage medium | |
| US20250008047A1 (en) | Imaging system and method for high resolution imaging of a subject | |
| CN113439286B (en) | Processing image data in composite images | |
| US8983188B1 (en) | Edge-aware smoothing in images | |
| CN120997039A (en) | Directional scaling systems and methods | |
| CN111539895B (en) | Video denoising method and device, mobile terminal and storage medium | |
| CN115049572A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| US9215353B2 (en) | Image processing device, image processing method, image display device, and image display method | |
| CN113256785A (en) | Image processing method, apparatus, device and medium | |
| CN112950465B (en) | Video super processing method, video super processing device and storage medium | |
| CN116228567A (en) | Image blurring method, device and live video system | |
| CN113284063A (en) | Image processing method, image processing apparatus, electronic device, and readable storage medium | |
| US20250173877A1 (en) | Methods and systems motion vector calculation and processing | |
| CN112446848B (en) | Image processing method, device and electronic equipment | |
| CN112055255A (en) | Shooting image quality optimization method, device, smart TV and readable storage medium | |
| CN119299592B (en) | Image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |