HK1251751B - Method and device for blurring preview picture and equipment - Google Patents

Description

Method, device and equipment for processing blur of preview image

Technical Field

The present application relates to the field of image processing technology, and in particular to a method, device and equipment for processing blurring of a preview image.

Background Art

Typically, in order to highlight a subject being photographed and focus the captured image on the subject, the background area in the captured image may be instantly blurred.

Currently, when blurring the background area of a captured image, the depth of field information of the frame captured by the image sensor is typically calculated while the frame is being displayed. The background area is then blurred based on the calculated depth of field information. However, the inventors have discovered that blurring the captured image using this method not only takes a long time to calculate the depth of field information, but also, when the subject in the captured image shakes, the depth of field information is not updated in time, which can easily cause part of the subject to be blurred, thereby affecting the overall blur effect and reducing the user experience.

Summary of the Invention

The present application aims to solve at least one of the above technical deficiencies to a certain extent.

To this end, the first purpose of this application is to propose a preview screen blur processing method, which not only improves the blur processing speed of the preview screen and saves time, but also ensures that the shooting subject will not be blurred due to shaking, thereby improving the blur effect of the preview screen and improving user satisfaction.

The second objective of this application is to provide a preview image blur processing device.

The third objective of this application is to provide a terminal device.

The fourth object of this application is to provide a computer-readable storage medium.

In order to achieve the above-mentioned purpose, the preview screen blur processing method of the first aspect embodiment of the present application includes: performing portrait recognition on the current first preview screen, determining the first portrait information in the first preview screen, wherein the first portrait information includes the area of the first portrait; determining the target area to be protected based on the first portrait information; when the preview screen is switched, blurring the other areas in the second preview screen except the target area.

In order to achieve the above-mentioned purpose, the preview screen blur processing device of the second aspect embodiment of the present application includes: a first determination module, which is used to perform portrait recognition on the current first preview screen and determine the first portrait information in the first preview screen, wherein the first portrait information includes the area of the first portrait; a second determination module, which is used to determine the target area to be protected based on the first portrait information; and a blur processing module, which is used to blur other areas in the second preview screen except the target area when the preview screen is switched.

In order to achieve the above-mentioned purpose, the terminal device of the third aspect embodiment of the present application includes: a memory, a processor and a camera module; the camera module is used to capture images in the current scene; the memory is used to store executable program code; the processor is used to read the executable program code stored in the memory to run the program corresponding to the executable program code, so as to implement the preview screen blur processing method described in the first aspect embodiment.

In order to achieve the above-mentioned purpose, the computer-readable storage medium of the fourth embodiment of the present application stores a computer program thereon, which, when executed by a processor, implements the preview image blurring processing method described in the first embodiment.

The technical solution disclosed in this application has the following beneficial effects:

By performing portrait recognition on the current first preview image, the first person portrait in the first preview image is determined. Then, based on the determined first person portrait information, a target area to be protected is determined. When the preview image is switched, the remaining area in the second preview image, excluding the target area, is blurred. This achieves a faster blurring process for the preview image, saving time, while also ensuring that the subject is not blurred due to shaking, thereby improving the blurring effect of the preview image and enhancing user satisfaction.

Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

FIG1 is a flowchart of a method for processing blurring a preview image according to an embodiment of the present application;

FIG2 is a flowchart of performing portrait recognition on the current first preview image according to one embodiment of the present application;

FIG3 is a flowchart of a method for processing preview image blur according to another embodiment of the present application;

FIG4 is a schematic structural diagram of a preview image blur processing device according to an embodiment of the present application;

FIG5 is a schematic structural diagram of a terminal device according to an embodiment of the present application;

FIG6 is a schematic structural diagram of an image processing circuit according to an embodiment of the present application.

DETAILED DESCRIPTION

The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

In order to solve the problem in the related art that when blurring the background area of the preview screen, it not only takes a long time to calculate the depth of field information of the shooting screen, but also when the subject in the shooting screen shakes, the depth of field information of the shooting screen cannot be updated in time, so that part of the subject is easily blurred, which in turn affects the overall blurring effect of the shooting screen and reduces the user experience, a preview screen blurring processing method is proposed.

The preview image blurring method provided in this application performs portrait recognition on the current first preview image to determine the first person portrait information in the first preview image, where the first person portrait information includes the area of the first person portrait. Based on the determined first person portrait information, a target area to be protected is determined. Then, when the preview image switches, the remaining area in the second preview image, excluding the target area, is blurred. This method not only improves the blurring speed of the preview image, saving time, but also ensures that the subject is not blurred due to shaking, thereby improving the blurring effect of the preview image and enhancing user satisfaction.

The blurred image acquisition method according to an embodiment of the present application is described below with reference to the accompanying drawings.

FIG1 is a flowchart of a method for processing blurring a preview image according to an embodiment of the present application.

As shown in FIG1 , the preview image blur processing method of the present application may include the following steps:

Step 101, performing portrait recognition on the current first preview screen, determining the first portrait information in the first preview screen, wherein the first portrait information includes the area of the first portrait.

Specifically, the preview image blurring processing method provided in the embodiment of the present application can be executed by the preview image blurring processing device provided in the present application. The above device is configured in the terminal device to realize blurring control of the preview image.

In this embodiment, the terminal device can be any hardware device with a camera function, such as a smart phone, a camera, a personal computer (PC), etc., and this application does not make any specific restrictions on this.

Optionally, in this embodiment, the terminal device may also be a hardware device with dual cameras, where the dual cameras refer to a device with two rear cameras.

It should be noted that, in this embodiment, the two rear cameras may be arranged in the following manners, but are not limited to:

Method 1: Set in the horizontal direction.

Method 2: Set in the vertical direction.

The horizontal direction refers to the direction parallel to the short side of the terminal device, and the vertical direction refers to the direction parallel to the long side of the terminal device.

When specifically executing step 101 , this embodiment can perform portrait recognition on the current first preview image through the steps shown in FIG. 2 .

It's important to note that portrait recognition, also known as portrait focus technology, is a widely used technology in digital cameras. Portrait recognition technology works by identifying features such as eyes and mouths in the frame, locking onto the subject's position, and automatically focusing on the subject, setting the correct focus and exposure. When facial recognition is activated, the camera automatically adjusts the focus based on the subject's position and illumination, ensuring clarity and accurate exposure. Furthermore, facial recognition can accurately identify the primary subject even when multiple people are in the frame.

Specifically, performing portrait recognition on the preview image may include the following steps:

Step 201: Determine depth information corresponding to a face region in a first preview image.

In a specific implementation, the face in the first preview image can be identified using any form of facial recognition technology. For example, the eyes in the first preview image can be captured, and then the facial region can be determined based on the captured face. Then, different methods can be used based on the number of cameras on the terminal to obtain the depth information corresponding to the facial region.

For example, if the terminal is a single-camera device, the first preview image can be scanned by an ultrasonic radar to obtain depth information corresponding to the face area.

If the terminal is a dual-camera device, the depth information of the object, that is, the distance from the object to the plane where the dual cameras are located, can be calculated based on the principle of triangulation and the differences between the dual cameras regarding the same object.

Step 202: Determine the outline of the portrait included in the first preview image based on the depth information.

Specifically, in actual applications, the images captured by the terminal typically include a foreground area and a background area, and the depth of the foreground and background areas varies significantly. Therefore, after obtaining the depth information corresponding to the face area, this embodiment can determine the outline of the portrait included in the first preview image based on the depth information.

Step 203 , determining whether the depth transformation value corresponding to the portrait outline is greater than a threshold, if so, executing step 204 , otherwise executing step 205 .

The threshold value may be adaptively set according to the shooting conditions of the portrait in the first preview image, and this application does not impose any specific limitation on this.

Specifically, after obtaining the portrait outline in the first preview image, the depth change value of the facial outline can be compared with a threshold value to determine whether the facial outline is accurately obtained. If the depth change value of the facial outline is greater than the threshold value, it means that the portrait outline determined in step 202 meets the requirements. If it is less than the threshold value, it means that the portrait outline determined in step 202 does not meet the requirements, and the portrait outline needs to be corrected to make the final portrait outline more accurate.

Step 204: Use the determined portrait outline as a portrait recognition result.

Step 205 : Correcting the portrait outline according to the color information of each pixel and adjacent pixels in the portrait outline.

Specifically, the color information of the pixels in the portrait outline and the adjacent pixels is obtained. The color variation between the adjacent pixels in the portrait outline segmentation is then compared to see if it is significant. If not, the currently determined portrait outline is inaccurate. Therefore, the portrait outline can be expanded or contracted until the color difference between the adjacent pixels in the portrait outline segmentation is significant. The current portrait outline can then be used as the final portrait recognition result.

Then, information of the first portrait in the first preview image can be determined according to the determined portrait outline.

The first person portrait may be located in any area of the first preview screen, and this application does not impose any specific limitation on this.

In specific implementation, in this embodiment, the number of portraits in the first preview screen may be one or more, for example, two, three, five, etc., which is not specifically limited in this application.

Accordingly, when the preview image blurring processing device performs portrait recognition on the first preview image, the determined first portraits may be one or more.

In a possible implementation of this embodiment, when it is identified that the portraits in the first preview screen include a portrait that does not need to be protected, the user can manually select the first portrait, that is, the user can draw a trajectory in the terminal display interface to select the first portrait.

Then, the preview image blurring processing device determines the area information of the first portrait according to the user's drawing trajectory.

The trajectory drawn by the user on the display interface of the terminal may be a closed curve or a non-closed curve, and this application does not make any specific restrictions on this.

Alternatively, the first person portrait may be determined based on an image pre-captured by the terminal. That is, the first person is first photographed and stored using the terminal device. Then, when capturing an image with a blurred background, the terminal device automatically searches for a portrait similar to the first person from among the portraits recognized in the preview interface based on the pre-captured image of the first person, and uses the found portrait as the first person portrait.

Step 102: Determine a target area to be protected based on the first portrait information.

Specifically, when actually shooting an image with a blurred background, the subject may shake, or the photographer's hand may shake when pressing the capture button, resulting in the subject being blurred in the captured image. To avoid the above situation, this application can determine the area containing the first person portrait in the preview image based on the determined first person portrait information, and use the area containing the first person portrait as the target area to be protected.

In specific implementation, this embodiment can determine the target area to be protected by the following methods, as illustrated below:

The first implementation method:

The area of the first portrait is enlarged by a preset multiple to determine the area of the target area.

Among them, the preset magnification factor of the portrait area in this embodiment can be the terminal default, or can be adaptively set by the user according to actual needs, and this application does not limit this.

Specifically, the present application enlarges the area of the first portrait in the first preview screen so that the first portrait in the subsequently captured frame screen is always within the enlarged area, thereby ensuring that when the background of the preview screen is subsequently blurred, the portrait in the captured screen will not be blurred, thereby ensuring the integrity of the portrait.

The second implementation method:

The area of the target region is determined according to the matching degree between the first portrait information and the preset portrait template in the portrait template library.

In this embodiment, the portrait template library can be obtained by training based on a large number of portrait image samples, and this application does not limit this.

The portrait template library is used to define the corresponding relationship between different attributes, portrait templates and maximum areas.

In specific implementation, this embodiment can determine the attributes of the first portrait by matching the recognized first portrait with the portrait template library, and then determine the area of the target region according to the attributes of the first portrait.

The attribute information of the first person portrait may include, but is not limited to, age, gender, etc., and this application does not impose any specific limitation on this.

For example, if the first recognized portrait is matched with the portrait template library and the attribute of the first portrait is determined to be a child, the area information of the target area corresponding to the child can be found in the attribute and target area mapping table.

It can be understood that this application can save time in blurring the preview image and further improve the user experience by matching the recognized portrait with the portrait template library to determine the target area to be protected.

Step 103 : When the preview screen is switched, blurring is performed on the other areas except the target area in the second preview screen.

Specifically, when the first preview screen switches to the second preview screen, the preview screen blur processing device can blur the other areas in the second preview screen except the target area to be protected according to the determined target area to be protected, thereby ensuring that the portrait in the preview screen is not blurred and improving the blur processing effect of the preview screen.

It should be noted that in the actual shooting process, the user does not perform the blur processing operation on each preview image frame. If the preview image blur processing device in this embodiment performs the blur processing on each preview image frame in the above manner, it is easy to cause an excessive processing burden on the terminal device.

Therefore, in order to reduce the processing burden of the terminal device, the present application may further perform the following operations before blurring other areas of the preview image except the target area:

Operation 1:

Get portrait edge protection instructions;

Specifically, when a preview image in the preview image blurring device needs to be blurred, the user can input a portrait edge protection instruction through a variety of methods provided by the preview image blurring device.

Among them, the input method of the portrait edge protection instruction can be to input the portrait edge protection instruction by manual pressing; or, the portrait edge protection instruction can be input by voice input; or, the portrait edge protection instruction can be input by manual sliding, etc., which is not limited in this application.

Operation 2:

Determine whether the shaking amplitude of the shooting terminal is greater than a first threshold.

Among them, the first threshold can be adaptively set according to the actual shooting situation of the shooting terminal, and this application does not make any specific limitation.

Specifically, during the actual photo-taking process, the camera terminal will often experience jitter due to external factors, and this jitter will cause a relative displacement difference between the subject and the terminal in the captured image. Therefore, in order to ensure that a clear image is captured without blurring the subject when the camera terminal experiences jitter, and the jitter is greater than a first threshold, the present application can perform a blurring operation on the other areas of the second preview image, excluding the target area to be protected, based on the determined target area to be protected, to obtain an image in which the first person's portrait is prominent and the background is blurred.

The preview image blurring method of the embodiment of the present application performs portrait recognition on the current first preview image to determine the first person portrait in the first preview image. Then, based on the determined first person portrait information, a target area to be protected is determined. When the preview image is switched, the remaining area in the second preview image, excluding the target area, is blurred. This method not only saves blurring time when blurring the preview image, but also ensures that the subject is not mistakenly blurred due to shaking, thereby improving the blurring effect of the preview image and enhancing user satisfaction.

From the above analysis, it can be seen that by performing portrait recognition on the first preview screen, the first portrait information is determined, and the target area to be protected is determined. Then, when the preview screen is switched, the other areas in the second preview screen except the target area are blurred. In specific implementation, since the target area to be protected is determined based on the first portrait information in this application, it can be implemented in the following two ways. The first implementation method is to match the first portrait information with the portrait template library to determine the target area to be protected; the second implementation method is to expand the area of the first portrait by a preset multiple to determine the target area to be protected. Among them, if this embodiment adopts the first implementation method to determine the target area to be protected, it is necessary to first establish a portrait template library, and then match the first portrait information based on the established portrait template library to determine the target area to be protected. The process of establishing a portrait template library in this embodiment is described in detail below in conjunction with Figure 3.

FIG3 is a flowchart of another preview image blurring processing method of the present application.

As shown in FIG3 , the preview image blur processing method of the present application may include the following steps:

Step 301: Obtain a portrait image sample set, where the sample set includes various portrait pose graphs.

Among them, various portrait pose diagrams can be understood as different expressions of the portrait, or portrait postures, etc.

In specific implementation, the portrait image sample set in this embodiment may be pre-configured when the terminal leaves the factory, or may be obtained from the server side, which is not limited in this embodiment.

It should be noted that, in order to ensure that the portrait template library subsequently established is more persuasive, the portrait image sample set that can be obtained in this embodiment can be tens of thousands, hundreds of thousands, or even millions of images. This application does not make any specific restrictions on this.

Step 302 : training the portrait image sample set, determining the portrait templates and the maximum portrait area corresponding to users with different attributes, and establishing a portrait template library.

Specifically, after obtaining a sample set of portrait images, the present application may first analyze the obtained sample images to determine the attribute information of each portrait image in the sample set. Training may then be performed based on the attribute information of the sample set to obtain portrait templates and maximum portrait areas corresponding to portrait images with different attributes. Subsequently, based on the correspondence between the portrait image samples, the portrait templates, and the maximum portrait area, a portrait template library may be recommended.

Step 303 : Perform portrait recognition on the current first preview image to determine first portrait information in the first preview image, wherein the first portrait information includes the area of the first portrait.

Step 304 : Determine the attributes of the first portrait based on the matching degree between the first portrait information and each preset portrait template in the portrait template library.

Specifically, after determining the first portrait information in the first preview image, the determined first portrait information can be matched with each portrait template in the portrait template library, and the attribute information corresponding to the portrait template with the highest matching degree is used as the first portrait attribute.

In a specific implementation, a portrait template library can be obtained, wherein the portrait template library includes a correspondence between portrait attributes, portrait templates, and the maximum area of the portrait. Then, the first portrait is matched with each portrait template in the obtained portrait template library to determine the portrait template with the highest matching degree with the first portrait, and the portrait attributes corresponding to the portrait template are determined as the attributes of the first portrait.

Furthermore, in order to improve the accuracy of the portrait template library, in an embodiment of the present application, the portrait template library can also be periodically updated based on the images actually captured by the terminal.

For example, different users may frequently photograph different objects using terminals, and different target objects may have different portrait features. For example, some subjects may have small head movements when photographing, while some subjects may have large head movements when photographing; or, some subjects may tend to turn their heads to the left when photographing, while some subjects may tend to turn their heads to the right, and so on.

Therefore, when the terminal device is actually used, the portrait template library can be updated according to the portrait images collected within the preset time period, so that each terminal can use the updated portrait template library to determine the target area more accurately and reliably.

For example, if the portrait image captured by terminal device A within a preset time period shows that the portrait is accustomed to tilting to the left, and in the initial portrait template library, the maximum area of the portrait = portrait area + a + b, where a represents the area extending along the left side of the portrait, and b represents the area extending along the right side of the portrait; then after terminal device A updates the portrait template library based on the captured portrait image, the determined maximum area of the portrait may be: portrait area + a + b + c, where c is the area added along the left side of the portrait determined based on the portrait habit in the captured portrait image.

Among them, the preset time period can be set according to actual conditions, such as one week, one month, etc., and this application does not limit this.

In this embodiment, the initial portrait template library includes portrait templates corresponding to users with different attributes and a template library of portraits with the largest area.

That is to say, by updating the initial portrait template library from time to time, the updated portrait template can further increase the recognition accuracy and reduce the recognition failure rate when determining portrait attributes in the subsequent process, thereby further improving the user experience.

Step 305: Determine the area of the target region according to the attributes of the first portrait.

Specifically, since the portrait template library stores the correspondence between portrait attributes, portrait templates, and the maximum area of portraits, after determining the attributes of the first portrait, the present application can directly query and determine the target area of the first portrait based on the correspondence between the portrait attributes, portrait templates, and the maximum area of the portrait.

Step 306: When the preview screen is switched, obtain a portrait edge protection instruction.

Specifically, in order to reduce the burden of blurring the preview screen on the terminal device, this embodiment can first receive multiple input methods provided by the user through the preview image blurring processing device to input the portrait edge protection instruction before blurring the preview screen that needs to be blurred.

Among them, the input method of the portrait edge protection instruction can be to input the portrait edge protection instruction by manual pressing; or, the portrait edge protection instruction can be input by voice input; or, the portrait edge protection instruction can be input by manual sliding, etc., which is not limited in this application.

Step 307 : Based on the obtained portrait edge protection instruction, blurring is performed on other areas to varying degrees according to the distance between the other areas and the target area.

Specifically, after obtaining the portrait edge protection instruction input by the user, the preview image blur processing device can perform background blur processing for protecting the portrait area on the preview image according to the portrait edge protection instruction.

Furthermore, in order to make the preview image after blurring highlight the subject and make the transition with the background smoother and more natural, the present application can blur other areas of the preview screen in order of blurring degree from low to high.

Among them, the present application blurs the background area of the preview screen in order of blur degree from low to high, which can make the transition between the protection area and the blurred area smoother, thereby making the image presented on the shooting preview interface more natural.

The blurred image acquisition method of the embodiment of the present application first obtains different portrait image sample sets for training to establish a portrait template library, wherein the portrait template library includes a correspondence between portrait attributes, portrait templates, and the maximum area of the portrait. Then, after performing portrait recognition on the first preview screen and determining the first portrait information in the first preview screen, the first portrait information can be matched with each portrait template in the portrait template library to obtain the attributes of the first portrait. Then, based on the attributes of the first portrait, the area of the target area can be determined. Then, when the preview screen is switched, the areas other than the target area in the preview screen can be blurred to varying degrees based on the determined target area and the portrait edge protection instruction. In this way, when blurring the preview screen, not only the blurring processing speed of the preview screen is improved, saving time, but also ensuring that the subject will not be blurred due to shaking, thereby improving the blurring effect of the preview screen and enhancing user satisfaction.

In order to implement the above embodiment, the present invention further proposes a preview image blur processing device.

FIG4 is a schematic structural diagram of a preview image blurring processing apparatus according to an embodiment of the present invention.

As shown in FIG4 , the preview image blurring processing apparatus of the present application includes: a first determination module 11 , a second determination module 12 and a blurring processing module 13 .

The first determining module 11 is configured to perform portrait recognition on the current first preview image and determine first portrait information in the first preview image, wherein the first portrait information includes an area of the first portrait;

The second determining module 12 is used to determine the target area to be protected based on the first portrait information;

The blur processing module 13 is configured to perform blur processing on other areas except the target area in the second preview image when the preview image is switched.

It should be noted that the above explanation of the embodiment of the preview image blur processing method is also applicable to the preview image blur processing device of this embodiment. The implementation principles are similar and will not be repeated here.

In the preview image blurring device provided in this embodiment, portrait recognition is performed on the current first preview image to determine the first portrait in the first preview image. Based on the determined first portrait information, a target area to be protected is then determined. When the preview image is switched, the remaining area in the second preview image, excluding the target area, is blurred. This not only saves blurring time when blurring the preview image, but also ensures that the subject is not mistakenly blurred due to shaking, thereby improving the blurring effect of the preview image and enhancing user satisfaction.

In order to implement the above embodiment, the present invention also provides a terminal device.

FIG5 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.

5 , the terminal device of the present application includes a memory 21 , a processor 22 , and a camera module 23 ;

The camera module 23 is used to capture images of the current scene;

The memory 21 is used to store executable program codes;

The processor 22 is configured to read the executable program code stored in the memory 21 and run a program corresponding to the executable program code to implement the preview image blurring processing method described in the first embodiment. The preview image blurring processing method includes: performing portrait recognition on the current first preview image to determine first portrait information in the first preview image, wherein the first portrait information includes the area of the first portrait; determining a target area to be protected based on the first portrait information; and blurring the remaining areas in the second preview image except the target area when the preview image is switched.

Among them, in this embodiment, the terminal device can be any hardware device with a camera function, such as a smart phone, a camera, a personal computer (PC), etc., and this application does not make any specific restrictions on this.

Optionally, the terminal device in this embodiment may also be a hardware device with dual cameras, wherein the dual cameras refer to having two rear cameras and optionally also including a front camera.

It should be noted that the above explanation of the embodiment of the preview image blur processing method is also applicable to the terminal device of this embodiment. The implementation principle is similar and will not be repeated here.

Furthermore, the terminal device of the present application may also include an image processing circuit 24 .

The memory 21 is used to store executable instructions of the processor 22;

The processor 22 is used to call the program code in the memory 21 and implement the preview image blurring processing method of the first embodiment according to the blurred image output by the image processing circuit 24.

Specifically, the image processing circuit 24 may be implemented using hardware and/or software components, and may include various processing units that define an ISP (Image Signal Processing) pipeline.

Figure 6 is a schematic diagram of an image processing circuit according to an embodiment of the present invention. As shown in Figure 6, for ease of explanation, only various aspects of image processing technology related to the embodiment of the present invention are shown.

As shown in Figure 6, the image processing circuit 24 includes an imaging device 1140, an ISP processor 1150 and a control logic unit 1160. The imaging device 1140 may include a camera having one or more lenses 1141, an image sensor 1142 and a structured light projector 1143. The structured light projector 1143 projects structured light onto the object under test. The structured light pattern may be laser stripes, Gray code, sinusoidal stripes, or a randomly arranged speckle pattern. The image sensor 1142 captures the structured light image projected onto the object under test, and sends the structured light image to the ISP processor 1150, which demodulates the structured light image to obtain the depth information of the object under test. At the same time, the image sensor 1142 can also capture the color information of the object under test. Of course, two image sensors 1142 can also capture the structured light image and color information of the object under test respectively.

Taking speckle structured light as an example, the ISP processor 1150 demodulates the structured light image. Specifically, it collects a speckle image of the object under test from the structured light image, performs image data calculation on the speckle image of the object under test and a reference speckle image according to a predetermined algorithm, and obtains the movement distance of each speckle in the speckle image on the object under test relative to the reference speckle in the reference speckle image. It then uses trigonometric calculation to obtain the depth value of each speckle in the speckle image, and uses this depth value to obtain the depth information of the object under test.

Of course, the depth image information can also be obtained through binocular vision methods or time-of-flight TOF-based methods, etc., which are not limited here. As long as the method can obtain or calculate the depth information of the object being measured, it falls within the scope of this embodiment.

After the ISP processor 1150 receives the color information of the object under test captured by the image sensor 1142, it processes the image data corresponding to the color information of the object under test. The ISP processor 1150 analyzes the image data to obtain image statistics that can be used to determine and/or adjust one or more control parameters of the imaging device 1140. The image sensor 1142 may include a color filter array (e.g., a Bayer filter). The image sensor 1142 can obtain light intensity and wavelength information captured by each imaging pixel of the image sensor 1142 and provide a set of raw image data that can be processed by the ISP processor 1150.

The ISP processor 1150 processes raw image data on a pixel-by-pixel basis in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits. The ISP processor 1150 may perform one or more image processing operations on the raw image data and collect image statistics about the image data. The image processing operations may be performed at the same or different bit depths of precision.

The ISP processor 1150 may also receive pixel data from an image memory 1170. The image memory 1170 may be a portion of a memory device, a storage device, or a separate dedicated memory within the electronic device and may include a DMA (Direct Memory Access) feature.

Upon receiving raw image data, the ISP processor 1150 may perform one or more image processing operations.

After the ISP processor 1150 obtains the color information and depth information of the object to be measured, it can be fused to obtain a three-dimensional image. Among them, the features of the corresponding object to be measured can be extracted by at least one of the appearance contour extraction method or the contour feature extraction method. For example, the features of the object to be measured are extracted by active shape model method ASM, active appearance model method AAM, principal component analysis method PCA, discrete cosine transform method DCT and other methods, which are not limited here. Then, the features of the object to be measured extracted from the depth information and the features of the object to be measured extracted from the color information are aligned and feature fused. The fusion processing referred to here can be a direct combination of features extracted from the depth information and color information, or a combination of the same features in different images after weighting, or other fusion methods. Finally, a three-dimensional image is generated based on the fused features.

Image data for the 3D image may be sent to image memory 1170 for further processing before being displayed. The ISP processor 1150 receives processed data from image memory 1170 and processes the processed data in the raw domain and in RGB and YCbCr color spaces. The image data for the 3D image may be output to a display 1180 for viewing by a user and/or further processing by a graphics engine or GPU (Graphics Processing Unit). Furthermore, the output of the ISP processor 1150 may be sent to image memory 1170, and the display 1180 may read the image data from the image memory 1170. In one embodiment, the image memory 1170 may be configured to implement one or more frame buffers. Furthermore, the output of the ISP processor 1150 may be sent to an encoder/decoder 1190 for encoding/decoding the image data. The encoded image data may be stored and decompressed before being displayed on the display 1180 device. The encoder/decoder 1190 may be implemented by a CPU, a GPU, or a coprocessor.

The image statistics determined by the ISP processor 1150 may be sent to a control logic unit 1160. The control logic 1160 may include a processor and/or microcontroller that executes one or more routines (e.g., firmware) that may determine control parameters for the imaging device 1140 based on the received image statistics.

In the terminal device provided in this embodiment, portrait recognition is performed on the current first preview screen to determine the first portrait in the first preview screen. Based on the determined first portrait information, a target area to be protected is then determined. When the preview screen switches, the remaining area in the second preview screen, excluding the target area, is blurred. This not only saves blurring time when blurring the preview screen, but also ensures that the subject is not accidentally blurred due to shaking, thereby improving the blurring effect of the preview screen and increasing user satisfaction.

In order to implement the above embodiments, the present invention also provides a computer-readable storage medium.

The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the preview image blurring processing method of the first embodiment. The preview image blurring processing method includes: performing portrait recognition on a current first preview image to determine first portrait information in the first preview image, wherein the first portrait information includes the area of the first portrait; determining a target area to be protected based on the first portrait information; and, when the preview image switches, blurring the remaining area in the second preview image, excluding the target area.

In the present invention, unless otherwise specified or limited, terms such as "disposed" and "connected" should be understood in a broad sense. For example, they can refer to mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

In the description of this specification, reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention.

Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Thus, features defined as "first" or "second" may explicitly or implicitly include at least one of these features.

Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and are not to be construed as limiting the present invention. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims (10)

1. A method for blurring a preview image, characterized in that it includes: Perform facial recognition on the current first preview screen to determine the first facial information in the first preview screen, wherein the first facial information includes the area of the first facial image; Based on the first portrait information, a target area to be protected is determined, wherein the area of the target area is larger than the area of the first portrait. When the first preview screen switches to the second preview screen, the areas in the second preview screen other than the target area are blurred. The step of determining the target area to be protected based on the first portrait information includes: Based on the matching degree between the first portrait information and each preset portrait template in the portrait template library, the portrait attribute corresponding to the portrait template with the highest matching degree with the first portrait is determined as the attribute of the first portrait. The portrait template library includes the correspondence between portrait attributes, portrait templates, and the maximum area of the portrait. The maximum area of the portrait is greater than the area of the corresponding portrait template. The area of the target region is determined based on the attributes of the first portrait and the correspondence between the portrait attributes and the portrait template and the maximum area of the portrait. 2. The method as described in claim 1, characterized in that, before determining the portrait attribute corresponding to the portrait template with the highest matching degree to the first portrait as the attribute of the first portrait, it further includes: Obtain the portrait template library. 3. The method as described in claim 2, wherein obtaining the portrait template library includes: Obtain a set of human portrait image samples, the set of samples including various human portrait pose images; The set of portrait image samples is used for training to determine the portrait templates and maximum portrait area corresponding to users with different attributes. 4. The method as described in claim 2, wherein obtaining the portrait template library includes: Obtain an initial portrait template library, which includes portrait templates corresponding to users with different attributes and the maximum area of the portrait; The initial portrait template library is updated based on portrait images collected within a preset time period. 5. The method according to any one of claims 1-4, characterized in that, before blurring the areas other than the target area in the second preview image, it further includes: Obtain the portrait edge protection command; Alternatively, determine that the shaking amplitude of the shooting terminal is greater than the first threshold. 6. The method according to any one of claims 1-4, characterized in that, the step of blurring other areas in the second preview image besides the target area includes: Depending on the distance between the other regions and the target region, the other regions are blurred to varying degrees. 7. The method as described in any one of claims 1-4, characterized in that, the facial recognition of the current first preview image includes: Determine the depth information corresponding to the face region in the first preview image; Based on the depth information, the human silhouette included in the first preview image is determined; Determine whether the depth transformation value corresponding to the human portrait contour is greater than a threshold; If not, the portrait outline is corrected based on the color information of each pixel and its adjacent pixels in the portrait outline. 8. A preview screen blurring processing device, characterized in that it comprises: The first determining module is used to perform facial recognition on the current first preview screen and determine the first facial information in the first preview screen, wherein the first facial information includes the area of the first facial image; The second determining module is used to determine the target area to be protected based on the first portrait information, wherein the area of the target area is larger than the area of the portrait; The blurring module is used to blur other areas in the second preview screen except for the target area when the first preview screen is switched to the second preview screen. The second determining module is specifically used for: The step of determining the target area to be protected based on the first portrait information includes: Based on the matching degree between the first portrait information and each preset portrait template in the portrait template library, the portrait attribute corresponding to the portrait template with the highest matching degree with the first portrait is determined as the attribute of the first portrait. The portrait template library includes the correspondence between portrait attributes, portrait templates, and the maximum area of the portrait. The maximum area of the portrait is greater than the area of the corresponding portrait template. The area of the target region is determined based on the attributes of the first portrait and the correspondence between the portrait attributes and the portrait template and the maximum area of the portrait. 9. A terminal device, characterized in that it comprises: a memory, a processor, and a camera module; The camera module is used to capture images of the current scene; The memory is used to store executable program code; The processor is configured to read executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the preview screen blurring processing method as described in any one of claims 1-7. 10. A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the preview screen blurring processing method as described in any one of claims 1-7.