CN106203280A - A kind of augmented reality AR image processing method, device and intelligent terminal - Google Patents

Abstract

The embodiment of the present invention discloses an augmented reality (AR) image processing method, device and intelligent terminal. The method comprises: using face recognition technology to determine whether the image to be processed is a person image; if it is a person image, using image recognition technology to determine whether the image to be processed contains a characteristic posture; if it contains a characteristic posture, AR processing is performed on the characteristic posture contained in the image to be processed. In the embodiment of the present invention, the characteristic posture contained in the image to be processed is obtained, and the obtained characteristic posture is used as the AR target to be processed. The AR target in the image to be processed is determined by face recognition technology and image recognition technology. Compared with the prior art of manually determining the AR target, the AR processing operation is simplified and the efficiency of AR image processing is improved.

Description

A kind of augmented reality AR image processing method, device and intelligent terminal

technical field

The present invention relates to the technical field of image processing, in particular to an augmented reality AR image processing method, device and intelligent terminal.

Background technique

Augmented Reality (AR), the physical information that is difficult to experience in the real world within a certain time and space range, such as visual information, sound, taste or touch, etc., is simulated by science and technology and then superimposed on the real world Perceived by human senses to achieve a sensory experience beyond reality, this technology is called augmented reality technology, or AR technology for short.

The existing AR image processing method generally requires the user to manually determine the AR target to be processed, resulting in a reduction in the efficiency of the existing AR image processing method.

Contents of the invention

The purpose of the embodiments of the present invention is to provide an augmented reality AR image processing method, device, and smart terminal, so as to improve the efficiency of AR image processing and realize AR processing of character's characteristic poses.

In a first aspect, an embodiment of the present invention provides an augmented reality AR image processing method, including:

Use face recognition technology to determine whether the image to be processed is a human image;

If it is an image of a person, image recognition technology is used to determine whether the image to be processed contains a characteristic pose;

If the feature pose is included, AR processing is performed on the feature pose included in the image to be processed.

In a second aspect, an embodiment of the present invention provides an augmented reality AR image processing device, including:

The image recognition module is used to adopt face recognition technology to determine whether the image to be processed is a person image;

The gesture recognition module is used to determine whether the image to be processed contains a characteristic gesture by using image recognition technology if it is a person image;

The gesture processing module is configured to perform AR processing on the feature gestures contained in the image to be processed if the feature gestures are included.

In a third aspect, an embodiment of the present invention provides a smart terminal, including the augmented reality AR image processing device provided in any embodiment of the present invention.

The technical solution provided by the embodiment of the present invention uses face recognition technology to determine whether the image to be processed is a person image, if it is a person image, then uses image recognition technology to determine whether the image to be processed contains a characteristic posture, and if it contains a characteristic posture, then Acquire the feature poses contained in the image to be processed, and use the acquired feature poses as the AR target to be processed, and perform AR processing on the acquired feature poses to realize the AR processing of the feature poses in the person image. Face recognition technology and image recognition technology determine the AR target in the image to be processed. Compared with the prior art of manually determining the AR target, the AR processing operation is simplified and the efficiency of AR image processing is improved.

Description of drawings

FIG. 1a is a schematic flowchart of an AR image processing method provided by Embodiment 1 of the present invention;

FIG. 1b is a schematic diagram of the AR image processing effect provided by Embodiment 1 of the present invention;

FIG. 2 is a schematic flowchart of an AR image processing method provided in Embodiment 2 of the present invention;

FIG. 3 is a schematic flowchart of an AR image processing method provided in Embodiment 3 of the present invention;

FIG. 4 is a schematic structural diagram of an AR image processing device provided in Embodiment 4 of the present invention;

FIG. 5 is a schematic structural diagram of a smart terminal provided by Embodiment 5 of the present invention.

detailed description

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, only some structures related to the present invention are shown in the drawings but not all structures.

Embodiment one

Figure 1a is a schematic flow chart of an AR image processing method provided by Embodiment 1 of the present invention. The method can be executed by an AR image processing device, wherein the device can be implemented by software and/or hardware, and can be built into a smart terminal as a part of a smart terminal. inside the terminal. As shown in Figure 1a, the implementation process includes:

Step 11, using face recognition technology to determine whether the image to be processed is a person image.

Wherein, the image to be processed may be an image captured by a camera in the smart terminal, or an image received or downloaded by the smart terminal. For example, an image to process can be selected from the photo gallery application. Specifically, face recognition technology is used to determine whether the image to be processed contains a human face, and if so, it is determined that the image to be processed is a person image; otherwise, it is determined that the image to be processed is a non-person image. In this embodiment, the processing method of the non-person image is not specifically limited. For example, AR processing may be performed on the non-person image, or other processing other than AR such as whitening processing may be performed, and the image processing operation may also be stopped.

Step 12. If it is a person image, use image recognition technology to determine whether the image to be processed contains a characteristic pose.

In this embodiment, the characteristic pose refers to a preset character pose that is expected to be processed by AR, for example, the characteristic pose may include running, blowing, putting hands together, crying, or hugging. The characteristic postures can be set uniformly, or can be personalized according to user needs.

Specifically, image recognition technology is used to identify whether the image to be processed contains characteristic gestures such as running, shouting, blowing air, or folded hands, and if so, obtain the characteristic posture contained in the image to be processed. If it is recognized that the image to be processed does not contain a characteristic pose, then AR processing may not be performed on the image to be processed subsequently.

Step 13: If the feature pose is included, perform AR processing on the feature pose included in the image to be processed.

Specifically, if the image to be processed contains a feature pose, the feature pose included in the image to be processed is used as an AR target to be processed, and AR processing is performed on the feature pose. Referring to FIG. 1 b , a virtual picture of the moon, a virtual picture of stars, a virtual picture of sweat, etc. can be added to the character image including the running posture. It should be noted that, in this embodiment, the manner of performing AR processing on the characteristic poses is not specifically limited, and it is only necessary to adjust the display effect of the image to be processed by adding a virtual screen at the position corresponding to the characteristic poses.

The technical solution provided by this embodiment uses face recognition technology to determine whether the image to be processed is a person image, and if it is a person image, then uses image recognition technology to determine whether the image to be processed contains a characteristic posture, and if it contains a characteristic posture, obtains The feature pose contained in the image to be processed, and the acquired feature pose is used as the AR target to be processed, and AR processing is performed on the acquired feature pose to realize the AR processing of the feature pose in the person image. The face recognition technology and the image recognition technology determine the AR target in the image to be processed, which simplifies the AR processing operation and improves the efficiency of the AR image processing compared with the manual determination of the AR target in the prior art.

Exemplarily, before performing AR processing on the feature gesture contained in the image to be processed, the method may include: starting an AR processing mode in the smart terminal. In this embodiment, the smart terminal provides an AR processing mode different from the normal (real) picture processing mode, and is used to add a virtual picture to the AR target. Specifically, if the value of the status identifier under the specified path is True, the smart terminal is in the AR processing mode; if the value of the status identifier is False, the smart terminal is in the normal image processing mode.

Embodiment two

This embodiment provides a new AR image processing method on the basis of the first embodiment above. Fig. 2 is a schematic flow diagram of the AR image processing method provided in Embodiment 2 of the present invention. As shown in Fig. 2, the implementation flow includes:

Step 21, using face recognition technology to determine whether the image to be processed is a person image.

Step 22. If it is a person image, use image recognition technology to determine whether the image to be processed contains a characteristic pose.

Step 23, if the feature pose is included, then determine the priority of the feature pose included in the image to be processed.

Specifically, the priority can be preset according to the limb position corresponding to the feature pose. For example, the priority of the head pose is higher than that of the hand pose, and the priority of the hand pose is higher than that of the foot pose.

The priority of the feature poses can also be determined according to the image recognition result of the image to be processed. Exemplarily, determining the priority of the feature pose contained in the image to be processed may include: determining the priority of the character in the image to be processed according to the position, face area, face orientation or occlusion of the character to which the feature pose belongs. Priority of included feature poses.

Wherein, the priority of the characteristic posture can be determined according to the priority of the person to which the characteristic posture belongs. If the priority of the first person is higher than that of the second person, the priority of the characteristic posture of the first person is higher than that of the characteristic posture of the second person. priority; otherwise, the priority of the characteristic pose of the first character is lower than the priority of the characteristic pose of the second character. For example, the priority of a person in the center is higher than that of people on the sides, the priority of a person in the front is higher than that of people on the sides, and the priority of a person with a large face is higher than that of a person with a small face. For people, the priority of the person whose face is facing forward is higher than that of the person whose face is facing sideways, and the priority of the person with a small occlusion area is higher than that of a person with a large occlusion area, etc.

Step 24: Select a feature pose to be processed from feature poses included in the image to be processed according to the priority of the feature pose.

Exemplarily, in order to avoid the heavy workload of AR processing caused by AR processing for all feature poses, a preset number of poses to be processed with higher priority may be selected from the feature poses contained in the image to be processed, where the pre-set The value is not specifically limited, for example, it may be 3. A feature gesture with a priority higher than a preset priority may also be determined as a gesture to be processed.

Step 25, performing AR processing on the selected feature poses to be processed.

In the technical solution provided by this embodiment, it is determined whether the image to be processed contains a characteristic pose by using face recognition technology and image recognition technology. If the image to be processed contains a characteristic pose, then according to the priority of the feature pose, the Select the pose to be processed from the included feature poses, and perform AR processing on the selected pose to be processed. This method selectively performs AR processing on the feature poses based on the priority of the feature poses, that is, this method further improves the AR by reducing the AR target. Image processing efficiency.

Embodiment three

This embodiment provides a new AR image processing method on the basis of the first embodiment above. FIG. 3 is a schematic flow diagram of the AR image processing method provided by Embodiment 3 of the present invention. As shown in FIG. 3 , the implementation flow includes:

Step 31, using face recognition technology to determine whether the image to be processed is a person image.

Step 32. If it is a person image, use image recognition technology to determine whether the image to be processed contains a characteristic pose.

It should be noted that if it is recognized that the image to be processed does not contain a characteristic pose, the processing method of the image to be processed is not specifically limited. For example, other processing other than AR can be performed on the image to be processed, such as whitening processing, etc., and the image can also be stopped. Processing operations.

Step 33: If the feature pose is included, determine the AR processing method associated with the feature pose.

In this embodiment, the AR processing methods associated with different feature poses may be different or the same. Moreover, there may be more than one AR processing method associated with the same feature pose. Exemplarily, the AR processing manner associated with the feature gesture may include a virtual blowing effect image, a virtual expression effect image, or a virtual motion effect image. For example, the AR processing method associated with the running posture is a virtual sweat image, the AR processing method associated with the blowing posture is a virtual fire-breathing image, the AR processing method associated with the folded hands posture is a virtual Buddha light image, and the AR processing method associated with the hugging posture is a virtual image. The AR processing method associated with pet pictures or crying gestures is virtual tear pictures, etc.

Step 34: Perform AR processing on the feature pose contained in the image to be processed by using the determined AR processing method.

In the technical solution provided by this embodiment, by using image recognition technology to determine whether the image to be processed contains a characteristic pose, if the image to be processed contains a characteristic pose, then determine the AR processing method associated with the characteristic pose, and use the determined AR processing method to AR processing is performed on feature poses, which further improves the effect of AR image processing.

Embodiment Four

FIG. 4 is a schematic structural diagram of an AR image processing device provided in Embodiment 4 of the present invention, and the device can be configured in a smart terminal. As shown in Figure 4, the specific structure of the AR image processing device may include:

The image recognition module 41 is used to adopt face recognition technology to determine whether the image to be processed is a person image;

Gesture recognition module 42, for if it is a person image, then adopt image recognition technology to determine whether the image to be processed contains a characteristic posture;

The gesture processing module 43 is configured to perform AR processing on the feature gesture contained in the image to be processed if the feature gesture is included.

Exemplary, the gesture processing module 43 may include:

A priority determining unit, configured to determine the priority of the feature pose contained in the image to be processed if the feature pose is included;

A pose selection unit, configured to select a feature pose to be processed from feature poses contained in the image to be processed according to the priority of the feature pose;

The first gesture processing unit is configured to perform AR processing on the selected feature gestures to be processed.

Exemplarily, the priority determination unit may be specifically configured to:

The priority of the feature poses included in the image to be processed is determined according to the position, face area, face orientation or occlusion of the person to which the feature poses belong.

Exemplary, the device may include:

A mode starting module, configured to start an AR processing mode in the smart terminal before performing AR processing on the feature gesture contained in the image to be processed.

Exemplary, the gesture processing module 43 may include:

A processing mode determining unit, configured to determine the AR processing mode associated with the feature pose if the feature pose is included;

The second gesture processing unit is configured to perform AR processing on the feature gesture contained in the image to be processed by using a determined AR processing manner.

Exemplarily, the AR processing manner associated with the feature gesture may include a virtual blowing effect image, a virtual expression effect image, or a virtual motion effect image.

The AR image processing device provided in this embodiment belongs to the same inventive concept as the AR image processing method provided in any embodiment of the present invention, can execute the AR image processing method provided in any embodiment of the present invention, and is capable of executing the AR image processing method. functional modules and beneficial effects. For technical details not exhaustively described in this embodiment, refer to the AR image processing method provided in any embodiment of the present invention.

Embodiment five

This embodiment provides an intelligent terminal, and the intelligent terminal may include the AR image processing apparatus provided by any embodiment of the present invention. FIG. 5 is a schematic structural diagram of a smart terminal provided in Embodiment 5 of the present invention. As shown in FIG. 5 , a smart phone 500 shown in the figure may include: a memory 501, a central processing unit (Central Processing Unit, hereinafter referred to as CPU) 502, a peripheral interface 503, RF circuit 505, audio circuit 506, speaker 511, power management chip 508, input/output (I/O) subsystem 509, other input/control devices 510, and external port 504, these components communicate via one or more communication buses Or signal line 507 to communicate.

It should be understood that the illustrated smartphone 500 is only one example of a smart terminal, and that the smartphone 500 may have more or fewer components than shown, two or more components may be combined, or Different component configurations are possible. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.

The smart phone used for AR image processing provided by this embodiment will be described in detail below.

Memory 501: The memory 501 can be accessed by the CPU 502, the peripheral interface 503, etc. The memory 501 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more disk storage devices, flash memory devices , or other volatile solid-state storage devices.

Peripheral interface 503 , which can connect the input and output peripherals of the device to CPU 502 and memory 501 .

I/O subsystem 509: The I/O subsystem 509 can connect the input and output peripherals on the device, such as the touch screen 502 (equivalent to the display in the above-mentioned embodiment) and other input/control devices 510, to the peripherals Interface 503. I/O subsystem 509 may include a display controller 5091 and one or more input controllers 5092 for controlling other input/control devices 510 . Among them, one or more input controllers 5092 receive electrical signals from or send electrical signals to other input/control devices 510, which may include physical buttons (push buttons, rocker buttons, etc.) ), dials, slide switches, joysticks, click wheels. It should be noted that the input controller 5092 can be connected to any of the following: a keyboard, an infrared port, a USB interface, and a pointing device such as a mouse.

Touch screen 512: The touch screen 512 is an input interface and an output interface between the user terminal and the user, and displays visual output to the user. The visual output may include graphics, text, icons, videos, and the like.

The display controller 5091 in the I/O subsystem 509 receives electrical signals from the touch screen 512 or sends electrical signals to the touch screen 512 . The touch screen 512 detects the contact on the touch screen, and the display controller 5091 converts the detected contact into an interaction with the user interface object displayed on the touch screen 512, that is, realizes human-computer interaction, and the user interface object displayed on the touch screen 512 can be a running Icons for games, icons for networking to appropriate networks, etc. It is worth noting that the device may also include an optical mouse, which is a touch-sensitive surface that does not display visual output, or that is an extension of a touch-sensitive surface formed by a touch screen.

The RF circuit 505 is mainly used to establish communication between the mobile phone and the wireless network (that is, the network side), and realize data reception and transmission between the mobile phone and the wireless network. Such as sending and receiving short messages, e-mails, etc. Specifically, the RF circuit 505 receives and sends RF signals, which are also called electromagnetic signals, and the RF circuit 505 converts electrical signals into electromagnetic signals or converts electromagnetic signals into electrical signals, and communicates with communication networks and other devices through the electromagnetic signals to communicate. RF circuitry 505 may include known circuitry for performing these functions including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chip Group, Subscriber Identity Module (Subscriber Identity Module, SIM) and so on.

The audio circuit 506 is mainly used to receive audio data from the peripheral interface 503 , convert the audio data into electrical signals, and send the electrical signals to the speaker 511 .

The speaker 511 is used to restore the voice signal received by the mobile phone from the wireless network through the RF circuit 505 into sound and play the sound to the user.

The power management chip 508 is used for power supply and power management for the hardware connected to the CPU 502 , the I/O subsystem and the peripheral interface.

The CPU 502 provided by the embodiment of the present invention can perform the following operations:

Use face recognition technology to determine whether the image to be processed is a human image;

If it is an image of a person, image recognition technology is used to determine whether the image to be processed contains a characteristic pose;

If the feature pose is included, AR processing is performed on the feature pose included in the image to be processed.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention, and the present invention The scope is determined by the scope of the appended claims.

Claims (11)

1. A kind of augmented reality AR image processing method, it is characterized in that, comprising: Use face recognition technology to determine whether the image to be processed is a human image; If it is an image of a person, image recognition technology is used to determine whether the image to be processed contains a characteristic pose; If the feature pose is included, AR processing is performed on the feature pose included in the image to be processed. 2. The method according to claim 1, wherein performing AR processing on the feature pose contained in the image to be processed comprises: determining the priority of the feature poses contained in the image to be processed; Selecting the feature pose to be processed from the feature poses included in the image to be processed according to the priority of the feature pose; Perform AR processing on the selected feature poses to be processed. 3. The method according to claim 2, wherein determining the priority of the feature poses contained in the image to be processed comprises: The priority of the feature poses included in the image to be processed is determined according to the position, face area, face orientation or occlusion of the person to which the feature poses belong. 4. The method according to claim 1, wherein performing AR processing on the feature pose contained in the image to be processed comprises: Determine the AR processing method associated with the feature pose; AR processing is performed on the feature poses included in the image to be processed by using a determined AR processing manner. 5 . The method according to claim 4 , wherein the AR processing method associated with the characteristic gesture includes a virtual blowing effect picture, a virtual expression effect picture or a virtual action effect picture. 6. An augmented reality AR image processing device, characterized in that, comprising: The image recognition module is used to adopt face recognition technology to determine whether the image to be processed is a person image; The gesture recognition module is used to determine whether the image to be processed contains a characteristic gesture by using image recognition technology if it is a person image; The gesture processing module is configured to perform AR processing on the feature gestures contained in the image to be processed if the feature gestures are included. 7. The device according to claim 6, wherein the gesture processing module comprises: A priority determining unit, configured to determine the priority of the feature pose contained in the image to be processed if the feature pose is included; A pose selection unit, configured to select a feature pose to be processed from feature poses contained in the image to be processed according to the priority of the feature pose; The first gesture processing unit is configured to perform AR processing on the selected feature gestures to be processed. 8. The device according to claim 7, wherein the priority determination unit is specifically configured to: The priority of the feature poses included in the image to be processed is determined according to the position, face area, face orientation or occlusion of the person to which the feature poses belong. 9. The device according to claim 6, wherein the gesture processing module comprises: A processing mode determining unit, configured to determine the AR processing mode associated with the feature pose if the feature pose is included; The second gesture processing unit is configured to perform AR processing on the feature gesture contained in the image to be processed by using a determined AR processing manner. 10 . The device according to claim 9 , wherein the AR processing method associated with the feature gesture includes a virtual blowing effect image, a virtual expression effect image, or a virtual motion effect image. 11 . 11. An intelligent terminal, comprising the augmented reality (AR) image processing device according to any one of claims 6-10.