CN108765321B - Photo restoration method, device, storage medium and terminal device - Google Patents
Photo restoration method, device, storage medium and terminal device Download PDFInfo
- Publication number
- CN108765321B CN108765321B CN201810468017.8A CN201810468017A CN108765321B CN 108765321 B CN108765321 B CN 108765321B CN 201810468017 A CN201810468017 A CN 201810468017A CN 108765321 B CN108765321 B CN 108765321B
- Authority
- CN
- China
- Prior art keywords
- incomplete
- dimensional
- dimensional data
- data
- deformity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Processing Or Creating Images (AREA)
Abstract
本申请实施例中提供的一种拍照修复方法、装置、存储介质及终端设备，该方法通过在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据；通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；对所述残缺物体上的残缺位置进行特征修复。通过采用上述技术方案，可以根据残缺物体的物体三维数据对拍摄的残缺物体进行残缺修复，丰富终端设备与用户的交互功能，优化现有的拍照技术。
A method, device, storage medium, and terminal device for photographing restoration provided in the embodiments of the present application. The method obtains object three-dimensional data of the incomplete object through the identification camera when the incomplete object is photographed by the identification camera; The set defect recognition model identifies the three-dimensional data of the object to determine the defect position on the defect object; and performs feature restoration on the defect position on the defect object. By adopting the above technical solution, it is possible to perform incomplete repair on the photographed incomplete object according to the object three-dimensional data of the incomplete object, enrich the interaction function between the terminal device and the user, and optimize the existing photographing technology.
Description
Technical Field
The embodiment of the application relates to the technical field of terminal equipment, in particular to a photographing restoration method and device, a storage medium and terminal equipment.
Background
With the development of the photographing technology of the terminal device, most users have the terminal device capable of photographing. The terminal equipment is easy to photograph, and a user can complete the photographing operation only by pressing the photographing key after aligning a photographed object, so that the photographing can be conveniently carried out at any time and any place. However, the existing photographing technology is still imperfect, and needs to be optimized.
Disclosure of Invention
The photographing method, the photographing device, the storage medium and the terminal device provided by the embodiment of the application can optimize the existing photographing technology.
In a first aspect, an embodiment of the present application provides a photographing restoration method, including:
when the incomplete object is shot by the recognition camera, acquiring object three-dimensional data of the incomplete object through the recognition camera;
identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object;
and performing characteristic repair on the incomplete position on the incomplete object.
In a second aspect, an embodiment of the present application provides a photographing restoration apparatus, including:
the three-dimensional data acquisition module is used for acquiring object three-dimensional data of the incomplete object through the identification camera when the incomplete object is shot by the identification camera;
the position determining module is used for identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object;
and the repairing processing module is used for performing characteristic repairing on the incomplete position on the incomplete object.
In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements a photo-repairing method according to an embodiment of the present application.
In a fourth aspect, an embodiment of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the photo repair method according to the embodiment of the present application.
According to the photographing repair scheme provided by the embodiment of the application, when the incomplete object is photographed by the recognition camera, the object three-dimensional data of the incomplete object is acquired by the recognition camera; identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object; and performing characteristic repair on the incomplete position on the incomplete object. By adopting the technical scheme, the photographed incomplete object can be repaired according to the three-dimensional object data of the incomplete object, the interaction function of the terminal equipment and a user is enriched, and the existing photographing technology is optimized.
Drawings
Fig. 1 is a schematic flowchart of a photographing restoration method according to an embodiment of the present application;
fig. 2 is a scene schematic diagram of a photographing restoration method provided in an embodiment of the present application;
fig. 3 is a schematic flowchart of another photographing restoration method provided in the embodiment of the present application;
fig. 4 is a scene schematic diagram of another photographing restoration method provided in the embodiment of the present application;
fig. 5 is a schematic flowchart of another photographing restoration method provided in the embodiment of the present application;
fig. 6 is a schematic flowchart of another photographing restoration method provided in the embodiment of the present application;
fig. 7 is a schematic flowchart of another photographing restoration method provided in the embodiment of the present application;
fig. 8 is a schematic flowchart of another photographing restoration method provided in the embodiment of the present application;
FIG. 9 is a schematic diagram of initial three-dimensional data provided by an embodiment of the present application;
fig. 10 is a schematic flowchart of another photographing restoration method according to an embodiment of the present application;
fig. 11 is a block diagram illustrating a photographing restoration apparatus according to an embodiment of the present disclosure;
fig. 12 is a schematic structural diagram of a terminal device according to an embodiment of the present application;
fig. 13 is a schematic structural diagram of another terminal device according to an embodiment of the present application.
Detailed Description
The technical scheme of the application is further explained by the specific implementation mode in combination with the attached drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be further noted that, for the convenience of description, only some of the structures related to the present application are shown in the drawings, not all of the structures.
Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the steps as a sequential process, many of the steps can be performed in parallel, concurrently or simultaneously. In addition, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
The three-dimensional camera is gradually popularized and opened, more and more terminal equipment all adopt the three-dimensional camera, and the three-dimensional camera can gather more information than traditional camera, and the three-dimensional information that this application can gather according to the three-dimensional camera optimizes terminal equipment's the technique of shooing.
Fig. 1 is a schematic flowchart of a photo-repairing method provided in an embodiment of the present application, where the method may be executed by a photo-repairing apparatus, where the apparatus may be implemented by software and/or hardware, and may be generally integrated in a terminal device, or may be integrated in other devices installed with an operating system. As shown in fig. 1, the method includes:
s110, when the incomplete object is shot by the recognition camera, acquiring object three-dimensional data of the incomplete object through the recognition camera.
Wherein, the incomplete object is an object with part of parts being blocked or damaged. The incomplete object is shot through the recognition camera, and the shot picture comprises the incomplete object.
The identification camera can be a camera on the terminal equipment, a front camera of the terminal equipment and/or a rear camera. For example, at least one camera is generally disposed in the existing terminal device, and generally includes a front camera and a rear camera. If the shot user is the current user of the terminal equipment, the identification camera can be a front camera of the terminal equipment, so that the user can know the shot picture through a screen of the terminal equipment. For another example, if the shot object is not the current user of the terminal device but another user or another object, the recognition camera may be a rear camera of the terminal device; the current user of the current terminal equipment shoots other incomplete objects through the terminal equipment.
The images captured by a conventional camera are generally two-dimensional data, i.e., a set of color values (RGB values) or gray values of pixels arranged in a row-column matrix rule. Compared with the two-dimensional data, the three-dimensional data also comprises depth data of the shot incomplete object, namely the distance between different spatial points of the shot incomplete object and the camera, so that the three-dimensional data can represent the spatial information of the shot object. The identification camera can be a camera with a distance sensor, the distance sensor can acquire the distance between different space points on the shot object and the camera, and therefore three-dimensional data of the shot incomplete object can be acquired.
The object three-dimensional data includes: and the three-dimensional data of the incomplete object shot by the identification camera. The object three-dimensional data may be a set of three-dimensional data acquired when the incomplete object is stationary. The object three-dimensional data can also be a plurality of groups of three-dimensional data acquired when the incomplete object is dynamic.
For example, if the recognition camera shoots a portrait, the deformity object may be a partially blocked portrait, or a facial portrait with a hat carried by the user and blocked eyes by a brim. The three-dimensional data of the corresponding incomplete object includes three-dimensional data of other parts which are not occluded. If the recognition camera shoots other objects, as shown in fig. 2, the objects may be broken porcelain 113, and many museums have cultural relic research value and ornamental value, but are partially broken porcelain.
And S111, identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object.
The three-dimensional data of the object is actually a set of one or more sets of data, and further, the position of the face, which is actually corresponding to the three-dimensional data of the object and needs to be repaired, needs to be analyzed and identified according to the set of data.
The deformity recognition model may be a recognition system that has been trained for determining a position of a deformity based on object three-dimensional data of the deformity object. The deformity identification model can be pre-stored in the terminal equipment or a background server. When the three-dimensional data of the object needs to be identified, the pre-stored defect identification model is called to identify the three-dimensional data of the object so as to determine the defect position of the shot defect object.
The incomplete position can be a position where an inherent feature is missing on the incomplete object; and if the incomplete object is a face part with the eyes shielded, the incomplete position is the position of the eyes, and if the incomplete object is a damaged object, the incomplete position is the position where the damage occurs.
And S112, performing characteristic repair on the incomplete position on the incomplete object.
And performing characteristic repairing treatment on the incomplete position of the three-dimensional data of the object according to the incomplete recognition model. The inherent characteristics of the corresponding deletion can be determined according to the incomplete position, and the material of the corresponding inherent characteristics is added to the incomplete position. Exemplarily, if the incomplete position is a face image with the blocked eyes, the inherent characteristic of the incomplete position is the eyes, and the material of the corresponding eye characteristic is set at the incomplete position, so that the face image comprising the eye characteristic can be obtained; therefore, the user can acquire the face image with complete characteristics, the interestingness of photographing of the user can be increased, and the conventional photographing technology is optimized.
Optionally, as shown in fig. 3, the characteristic repairing of the incomplete position on the incomplete object may be implemented by:
and S1121, determining missing information according to the incomplete position, and determining a corresponding three-dimensional repair material according to the missing information.
The missing information includes the inherent characteristics missing from the incomplete object and the characteristic information of the incomplete object. The three-dimensional repair material is a material file which is matched with the missing information and is used for filling the missing information, and the three-dimensional repair material can be preset by a system. Illustratively, if the incomplete object is an incomplete cultural relic of a museum, the three-dimensional repair material can be a repair material restored by a researcher according to historical materials.
Illustratively, if the missing position is a hand part of a person image, the actual inherent feature is a hand feature, whereas if the person feature information in the person image is a woman, the missing information includes a hand of the woman. It can be confirmed that there are many different places for the hand characteristics of the female and the hand characteristics of the male, and more accurate three-dimensional repair materials can be further obtained according to the characteristic information of the deformity object.
And S1122, performing characteristic repairing on the incomplete position according to the three-dimensional repairing material.
The three-dimensional repairing material can be arranged at a position corresponding to the incomplete position, so that incomplete information of the incomplete position in the incomplete object is filled, and a complete image of the incomplete object can be generated.
As shown in fig. 2 and 4, the defective object is a porcelain 113, the defective position is a damaged area 114 occurring at an opening of the porcelain 113, and the porcelain 113 is a celadon porcelain, and the determining of the missing information from the defective position includes: broken blocks at the opening of the porcelain and green glaze characteristics; further, the three-dimensional repair material 115 which is a blue-glazed material and which matches the damaged area 114 is specified from the missing information, and as shown in fig. 4, the three-dimensional repair material 115 is set at a position corresponding to the missing position 114, whereby a complete image of the porcelain 116 can be obtained.
And determining missing information according to the incomplete position, and determining a corresponding three-dimensional repair material according to the missing information, so that the authenticity of the three-dimensional repair material can be further improved.
According to the photographing repair scheme provided by the embodiment of the application, when the incomplete object is photographed by the recognition camera, the object three-dimensional data of the incomplete object is acquired by the recognition camera; identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object; and performing characteristic repair on the incomplete position on the incomplete object. By adopting the technical scheme, the photographed incomplete object can be repaired according to the three-dimensional object data of the incomplete object, the interaction function of the terminal equipment and a user is enriched, and the existing photographing technology is optimized.
Fig. 5 is a schematic flow chart of another photographing repair method provided in an embodiment of the present application, where on the basis of the technical solution provided in the embodiment, an operation of performing feature repair on a defective position according to the three-dimensional repair material is optimized, and optionally, as shown in fig. 5, the method includes:
and S120, when the incomplete object is shot by the identification camera, acquiring the three-dimensional object data of the incomplete object by the identification camera.
And S121, identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object.
And S122, determining missing information according to the incomplete position, and determining a corresponding three-dimensional repair material according to the missing information.
For the above-mentioned specific implementation of the operations, reference may be made to the above-mentioned related description, and further description is omitted here.
And S123, displaying the shot incomplete object on a screen of the terminal equipment in real time.
And S124, determining the display angle of the three-dimensional repair material according to the shooting angle of the incomplete object, and determining the display position of the three-dimensional repair material according to the incomplete position on the incomplete object.
And S125, displaying the three-dimensional repairing material on a screen of the terminal equipment according to the display angle and the display position.
The real-time display is the image of the incomplete object acquired by the recognition camera in real time, and if a user adjusts the shooting angle of the recognition camera, the corresponding real-time display angle of the incomplete object can be changed. And determining the display angle of the three-dimensional repairing material according to the shooting angle of the incomplete object, so that the display angle of the three-dimensional repairing material is matched with the display angle of the incomplete object.
Meanwhile, the display angle of the incomplete object can also influence the specific position of the incomplete position, and the three-dimensional repair material needs to be matched with the incomplete position, so that the display position of the three-dimensional repair material is determined according to the incomplete position, the display position of the three-dimensional repair material can be matched with the incomplete position, and further the three-dimensional repair material can be matched with the whole incomplete object.
The display angle may be a relative angle with reference to the deformity object, and the display position may be a relative position with reference to the deformity position.
According to the embodiment of the application, the display angle and the display position of the three-dimensional repair material are determined, the incomplete object and the three-dimensional repair material are displayed on the screen of the terminal equipment in real time, the shooting angle of the identification camera is adjusted by a user, the three-dimensional repair material can also move along with the movement of the incomplete object in the picture, the user can see the complete incomplete object displayed in real time through the screen of the terminal equipment, the shooting operation of the user is more interesting, and the shooting function of the terminal equipment is optimized.
Fig. 6 is a schematic flow chart of another photographing repair method provided in an embodiment of the present application, where on the basis of the technical solution provided in any of the embodiments, an operation of identifying three-dimensional data of the object by using a preset defect identification model to determine a defect position on the defective object is optimized, and optionally, as shown in fig. 6, the method includes:
s130, when the incomplete object is shot by the recognition camera, acquiring object three-dimensional data of the incomplete object through the recognition camera.
Reference may be made to the above description for specific embodiments, which are not repeated herein.
S131, identifying the three-dimensional data of the object through a preset incomplete identification model so as to determine the object type of the incomplete object.
S132, determining the incomplete position on the incomplete object according to the object type.
The incomplete object may include various types, such as a photographed portrait, a porcelain, and a sculpture. Different types of objects have different inherent characteristics, so that the object type of the incomplete object needs to be determined firstly, and then the incomplete positions of different objects can be determined according to the types of the objects. After determining the object type of the incomplete object, the complete intrinsic characteristics of the object type can be further determined, and what the intrinsic characteristics of the incomplete object are can be determined.
And S133, performing characteristic repair on the incomplete position on the incomplete object.
Reference may be made to the above description for specific embodiments, which are not repeated herein.
Alternatively, as shown in fig. 7, the characteristic repairing of the incomplete position on the incomplete object can be implemented by the following steps:
and S1331, determining missing information according to the object type of the incomplete object and the incomplete position, and determining a corresponding three-dimensional repair material according to the missing information.
And S1332, performing feature repair on the incomplete position according to the three-dimensional repair material.
After the object type of the incomplete object is determined, the missing information can be determined according to the object type and the missing position, and then the three-dimensional repairing material of the corresponding object type is determined, so that the material with the similar shape is prevented from being mistakenly used on the incomplete objects of different types, and the accuracy of characteristic repairing can be further improved.
According to the embodiment of the application, the three-dimensional data of the object is identified through a preset incomplete identification model so as to determine the object type of the incomplete object; and determining the incomplete position on the incomplete object according to the type of the object, determining missing information according to different types of the object, and improving the identification accuracy of the missing information.
Fig. 8 is a schematic flow chart of another photographing repair method provided in an embodiment of the present application, where based on the technical solution provided in any of the embodiments, an operation of acquiring object three-dimensional data of the defective object by using the recognition camera is optimized, and optionally, as shown in fig. 8, the method includes:
s140, when the incomplete object is shot by the recognition camera, the position depth data and the position infrared data of the incomplete object are acquired through the recognition camera.
The identification camera is a Three-Dimensional (3D) camera, and the Three-Dimensional camera comprises a plurality of hardware structures and can comprise: infrared sensors, distance sensors, lenses, and the like.
The position depth data is a set of distance values of space points included by the incomplete object from the identification camera; the position depth data of the incomplete object can be acquired by identifying a distance sensor in the camera.
The position infrared data is a set of infrared data reflected by space points included by the incomplete object. Wherein, three-dimensional camera is when shooing, and wherein infrared sensor transmits infrared signal to incomplete object, and incomplete object can reflect infrared information, and infrared sensor can realize the formation of image of incomplete object according to the infrared data of received reflection.
And S141, determining initial three-dimensional data of the incomplete object according to the position depth data.
The position depth data includes the distance value of the space point included by the incomplete object, so that the initial three-dimensional data of the incomplete object can be determined according to the position depth data. Illustratively, as shown in fig. 9, points a, b, c, and d in the drawing are four spatial points, X, Y and the Z-axis represents space, where the Z-axis represents depth data of the spatial points, and the X-and Y-axes represent plane position coordinates of the spatial points. Wherein the depth data of the point a is the largest, that is, the point a is the farthest from the recognition camera, it can be seen from the figure that a three-dimensional vertebral body can be formed according to the plane coordinates and the depth data of four spatial points, so that the initial three-dimensional data can be determined according to the position depth data of a plurality of spatial points and the plane coordinates of the spatial points.
However, if some details of the incomplete object are blocked or data loss occurs, the corresponding detail position in the initial three-dimensional data may have a data loss problem, so the initial three-dimensional data further needs to be corrected according to the position infrared data.
And S142, correcting the initial three-dimensional data according to the position infrared data to obtain object three-dimensional data of the incomplete object.
And for the space points included by the incomplete object, the depth data and the infrared data of each space point are in one-to-one correspondence. For the depth data of the spatial points with data missing, the overall initial three-dimensional data can be measured and compared according to the corresponding infrared data, and then the missing spatial points are subjected to feature completion. The infrared signal is an electromagnetic wave, and people can not see the infrared signal, but if the infrared signal is at night or in a dark environment without visible light, infrared light can still be transmitted, so that in the dark environment, a clearer image can be generated according to the infrared data; the initial three-dimensional data can then be corrected based on the positional infrared data.
Optionally, a fitting relation function may be established according to the depth data and the infrared data of the adjacent points, and corresponding depth data may be calculated according to the fitting relation function and the position infrared data of the missing space point, so as to obtain corrected three-dimensional data of the object; the missing space point is a space point where the depth data is missing, and the adjacent space point is an adjacent space point of the missing space point.
S143, identifying the three-dimensional data of the object through a preset deformity identification model so as to determine the deformity position on the deformity object.
And S144, performing characteristic repair on the incomplete position on the incomplete object.
For the above-mentioned specific implementation of the operations, reference may be made to the above-mentioned related description, and further description is omitted here.
Although the image of the incomplete object, i.e. the acquired two-dimensional data of the incomplete object, is shot by the common camera, the shape of the incomplete object can also be recognized by the image processing recognition technology. However, the two-dimensional data only includes data of a planar image, which has a high requirement for light, and if a user photographs a defective object in a dark environment, an accurate defective position may not be identified in the acquired data of the planar image, so that the accuracy of the two-dimensional data is low.
According to the embodiment of the application, the initial three-dimensional data of the incomplete object is determined according to the position depth data, and the initial three-dimensional data is corrected according to the position infrared data to obtain the three-dimensional data of the incomplete object; the method has the advantages that the initial three-dimensional data can be corrected through the position infrared data when the position is identified in a dark position, accurate three-dimensional data of an object can be obtained, and accuracy of identification of incomplete positions can be improved.
Fig. 10 is a schematic flow chart of another photographing repair method provided in an embodiment of the present application, where on the basis of the technical solution provided in any of the embodiments, an operation of acquiring object three-dimensional data of the defective object by using the recognition camera is optimized, and optionally, as shown in fig. 10, the method includes:
s150, inputting preset sample data into a preset classifier for training to obtain a incomplete recognition model;
the incomplete recognition model is used for determining a corresponding incomplete position according to the shot object three-dimensional data of the incomplete object; the preset sample data comprises sample three-dimensional data of the incomplete object and a corresponding sample incomplete position.
The preset sample data can comprise a plurality of different sample data, wherein the different sample data correspond to different acquired incomplete objects, corresponding sample three-dimensional data and corresponding incomplete positions; for example, if the incomplete object is a portrait, the preset sample data may be sample three-dimensional data for shielding different portrait at different positions and labels of corresponding incomplete positions.
And if the three-dimensional data of the object is identified through a preset incomplete identification model so as to determine the object type of the incomplete object and determine the incomplete position on the incomplete object according to the object type, the corresponding preset sample data comprises sample three-dimensional data of the incomplete object, the object type of the incomplete object and the corresponding incomplete position. Because each incomplete object is different, the types of the objects are different, and the corresponding incomplete positions are also different, the sample three-dimensional data, the corresponding object types and the corresponding incomplete positions are used as preset sample data to be input into a preset classifier to be trained so as to obtain an incomplete recognition model, and the input three-dimensional data of the objects can be used for recognition to determine the corresponding object types and the corresponding incomplete positions.
The preset classifier can be a neural network and is trained by inputting preset sample data into the preset classifier; the preset classifier can extract the feature data of the sample three-dimensional data, and the corresponding incomplete position and/or object type of the incomplete object can be determined according to the extracted feature data because the sample three-dimensional data is marked with the corresponding incomplete position and/or object type.
After the preset classifier is trained through preset sample data, the obtained incomplete recognition model can determine the object types of different incomplete objects and determine the corresponding incomplete positions.
And S151, when the incomplete object is shot by the identification camera, acquiring the three-dimensional object data of the incomplete object through the identification camera.
S152, identifying the three-dimensional data of the object through a preset incomplete identification model so as to determine the incomplete position on the incomplete object.
And S153, performing characteristic repair on the incomplete position on the incomplete object.
For the above-mentioned specific implementation of the operations, reference may be made to the above-mentioned related description, and further description is omitted here.
After the preset classifier is trained through preset sample data, the obtained incomplete recognition model can determine the object types of different incomplete objects, and the accuracy of determining the incomplete position can be improved.
Fig. 11 is a block diagram of a photo-repairing apparatus according to an embodiment of the present disclosure, where the apparatus may perform a photo-repairing method, and as shown in fig. 11, the apparatus includes:
the three-dimensional data acquisition module 210 is configured to acquire object three-dimensional data of a defective object through an identification camera when the identification camera shoots the defective object;
the position determining module 211 is configured to identify the three-dimensional data of the object through a preset incomplete identification model, so as to determine an incomplete position on the incomplete object;
and the repairing processing module 212 is used for performing characteristic repairing on the incomplete position on the incomplete object.
According to the photographing repair scheme provided by the embodiment of the application, when the incomplete object is photographed by the recognition camera, the object three-dimensional data of the incomplete object is acquired by the recognition camera; identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object; and performing characteristic repair on the incomplete position on the incomplete object. By adopting the technical scheme, the photographed incomplete object can be repaired according to the three-dimensional object data of the incomplete object, the interaction function of the terminal equipment and a user is enriched, and the existing photographing technology is optimized.
Optionally, the repair processing module specifically includes:
the material determining module is used for determining missing information according to the incomplete position and determining a corresponding three-dimensional repairing material according to the missing information;
and the characteristic repairing module is used for performing characteristic repairing on the incomplete position according to the three-dimensional repairing material.
Optionally, the feature repair module is specifically configured to:
the incomplete object display module is used for displaying shot incomplete objects on a screen of the terminal equipment in real time;
the display determining module is used for determining the display angle of the three-dimensional repair material according to the shooting angle of the incomplete object and determining the display position of the three-dimensional repair material according to the incomplete position on the incomplete object;
and the repairing display module is used for displaying the three-dimensional repairing material on a screen of the terminal equipment according to the display angle and the display position.
Optionally, the position determining module is specifically configured to:
identifying the three-dimensional data of the object through a preset incomplete identification model so as to determine the object type of the incomplete object;
and determining the incomplete position on the incomplete object according to the object type.
Optionally, the repair processing module is specifically configured to:
determining missing information according to the object type of the incomplete object and the incomplete position, and determining a corresponding three-dimensional repair material according to the missing information;
and performing characteristic repair on the incomplete position according to the three-dimensional repair material.
Optionally, the recognition camera is a three-dimensional camera; correspondingly, the three-dimensional data acquisition module is specifically configured to:
acquiring position depth data and position infrared data of the incomplete object through the identification camera;
determining initial three-dimensional data of the incomplete object according to the position depth data;
and correcting the initial three-dimensional data according to the position infrared data to obtain object three-dimensional data of the incomplete object.
Optionally, the method further comprises:
the training module is used for inputting preset sample data into a preset classifier for training before the preset deformity recognition model is used for recognizing the three-dimensional data of the object, so as to obtain a deformity recognition model;
the incomplete recognition model is used for determining a corresponding incomplete position according to the shot object three-dimensional data of the incomplete object; the preset sample data comprises sample three-dimensional data of the incomplete object and a corresponding sample incomplete position.
The storage medium containing the computer-executable instructions provided in the embodiments of the present application is not limited to the above-described photo repair operation, and may also perform related operations in the photo repair method provided in any embodiments of the present application.
Embodiments of the present application also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform a photo remediation method, the method including:
when the incomplete object is shot by the recognition camera, acquiring object three-dimensional data of the incomplete object through the recognition camera;
identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object;
and performing characteristic repair on the incomplete position on the incomplete object.
Storage medium-any of various types of memory devices or storage devices. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk, or tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Lanbas (Rambus) RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system connected to the first computer system through a network (such as the internet). The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations, such as in different computer systems that are connected by a network. The storage medium may store program instructions (e.g., embodied as a computer program) that are executable by one or more processors.
The embodiment of the application provides a terminal device, and the photographing repair device provided by the embodiment of the application can be integrated in the terminal device.
Fig. 12 is a schematic structural diagram of a terminal device according to an embodiment of the present application, and the embodiment of the present application provides a terminal device 30, which includes a memory 31, a processor 32, and a computer program stored in the memory 31 and executable on the processor, where the processor executes the computer program to implement the photo repair method according to the above embodiment. The terminal equipment provided by the embodiment of the application can optimize the existing photographing technology.
Fig. 13 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in fig. 13, the terminal device may include: a casing (not shown), a touch screen (not shown), touch keys (not shown), a memory 301, a Central Processing Unit (CPU) 302 (also called a processor, hereinafter referred to as CPU), a circuit board (not shown), and a power circuit (not shown). The circuit board is arranged in a space enclosed by the shell; the CPU302 and the memory 301 are disposed on the circuit board; the power supply circuit is used for supplying power to each circuit or device of the terminal equipment; the memory 301 is used for storing executable program codes; the CPU302 executes a computer program corresponding to the executable program code by reading the executable program code stored in the memory 301 to implement the steps of:
when the incomplete object is shot by the recognition camera, acquiring object three-dimensional data of the incomplete object through the recognition camera;
identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object;
and performing characteristic repair on the incomplete position on the incomplete object.
The terminal device further includes: peripheral interface 303, RF (Radio Frequency) circuitry 305, audio circuitry 306, speakers 311, power management chip 308, input/output (I/O) subsystems 309, touch screen 312, other input/control devices 310, and external ports 304, which communicate via one or more communication buses or signal lines 307.
It should be understood that the illustrated terminal device 300 is only one example of a terminal device, and that the terminal device 300 may have more or fewer components than shown in the figures, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.
The following describes in detail the terminal device for implementing the photographing restoration provided in this embodiment, where the terminal device is a mobile phone as an example.
A memory 301, the memory 301 being accessible by a CPU302, a peripheral interface 303, or the like, the memory 301 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other volatile solid state storage devices.
A peripheral interface 303, said peripheral interface 303 being capable of connecting input and output peripherals of the device to the CPU302 and the memory 301.
I/O subsystem 309, the I/O subsystem 309 may connect input and output peripherals on the device, such as touch screen 312 and other input/control devices 310, to the peripheral interface 303. The I/O subsystem 309 may include a display controller 3091 and one or more input controllers 3092 for controlling other input/control devices 310. Where one or more input controllers 3092 receive electrical signals from or send electrical signals to other input/control devices 310, the other input/control devices 310 may include physical buttons (push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels. It is noted that the input controller 3092 may be connected to any of the following: a keyboard, an infrared port, a USB interface, and a pointing device such as a mouse.
A touch screen 312, which touch screen 312 is an input interface and an output interface between the user terminal device and the user, displays visual output to the user, which may include graphics, text, icons, video, and the like.
The display controller 3091 in the I/O subsystem 309 receives electrical signals from the touch screen 312 or transmits electrical signals to the touch screen 312. The touch screen 312 detects a contact on the touch screen, and the display controller 3091 converts the detected contact into an interaction with a user interface object displayed on the touch screen 312, i.e., implements a human-machine interaction, and the user interface object displayed on the touch screen 312 may be an icon for running a game, an icon networked to a corresponding network, or the like. It is worth mentioning that the device may also comprise a light mouse, which is a touch sensitive surface that does not show visual output, or an extension of the touch sensitive surface formed by the touch screen.
The RF circuit 305 is mainly used to establish communication between the mobile phone and the wireless network (i.e., the network side), and implement data reception and transmission between the mobile phone and the wireless network. Such as sending and receiving short messages, e-mails, etc. In particular, the RF circuitry 305 receives and transmits RF signals, also referred to as electromagnetic signals, through which the RF circuitry 305 converts electrical signals to or from electromagnetic signals and communicates with communication networks and other devices. RF circuitry 305 may include known circuitry for performing these functions including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC (CODEC) chipset, a Subscriber Identity Module (SIM), and so forth.
The audio circuit 306 is mainly used to receive audio data from the peripheral interface 303, convert the audio data into an electric signal, and transmit the electric signal to the speaker 311.
And a speaker 311 for converting the voice signal received by the handset from the wireless network through the RF circuit 305 into sound and playing the sound to the user.
And the power management chip 308 is used for supplying power and managing power to the hardware connected with the CPU302, the I/O subsystem, and the peripheral interface.
The terminal equipment provided by the embodiment of the application can optimize the existing photographing technology.
The photographing restoration device, the storage medium and the terminal device provided in the above embodiments may execute the photographing restoration method provided in any embodiment of the present application, and have corresponding functional modules and beneficial effects for executing the method. For technical details that are not described in detail in the above embodiments, reference may be made to the photo repair method provided in any of the embodiments of the present application.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present application and the technical principles employed. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, although the present application has been described in more detail with reference to the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the appended claims.
Claims (8)
1. A method for repairing a photographed image, comprising:
when the identification camera shoots a defective object, acquiring object three-dimensional data of the defective object through the identification camera, wherein the object three-dimensional data comprises a plurality of groups of three-dimensional data acquired when the defective object is dynamic, and the defective object is an object with partial parts being blocked or defective;
identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object;
performing characteristic repair on the incomplete position on the incomplete object;
the identifying the three-dimensional data of the object through a preset deformity identification model to determine the position of the deformity on the deformity object comprises:
identifying the three-dimensional data of the object through a preset incomplete identification model so as to determine the object type of the incomplete object;
determining the incomplete position on the incomplete object according to the object type;
the characteristic repairing of the incomplete position on the incomplete object comprises the following steps:
determining missing information according to the object type of the incomplete object and the incomplete position, and determining a corresponding three-dimensional repair material according to the missing information;
and performing characteristic repair on the incomplete position according to the three-dimensional repair material.
2. The method of claim 1, wherein characterizing the incomplete location on the incomplete object comprises:
determining missing information according to the incomplete position, and determining a corresponding three-dimensional repairing material according to the missing information;
and performing characteristic repair on the incomplete position according to the three-dimensional repair material.
3. The method of claim 2, wherein performing feature repair on the incomplete location according to the three-dimensional repair material comprises:
displaying shot defective objects on a screen of the terminal equipment in real time;
determining a display angle of a three-dimensional repair material according to the shooting angle of the incomplete object, and determining a display position of the three-dimensional repair material according to the incomplete position on the incomplete object;
and displaying the three-dimensional repairing material on a screen of the terminal equipment according to the display angle and the display position.
4. The method of any of claims 1 to 3, wherein the recognition camera is a three-dimensional camera; correspondingly, the acquiring of the three-dimensional object data of the incomplete object by the recognition camera comprises:
acquiring position depth data and position infrared data of the incomplete object through the identification camera;
determining initial three-dimensional data of the incomplete object according to the position depth data;
and correcting the initial three-dimensional data according to the position infrared data to obtain object three-dimensional data of the incomplete object.
5. The method according to any one of claims 1 to 3, further comprising, before the identifying the object three-dimensional data by a preset deformity identification model:
inputting preset sample data into a preset classifier for training to obtain a deformity identification model;
the incomplete recognition model is used for determining a corresponding incomplete position according to the shot object three-dimensional data of the incomplete object;
the preset sample data comprises sample three-dimensional data of the incomplete object and a corresponding sample incomplete position.
6. A photographic rehabilitation device, comprising:
the three-dimensional data acquisition module is used for acquiring object three-dimensional data of the incomplete object through the identification camera when the incomplete object is shot by the identification camera, wherein the object three-dimensional data comprises a plurality of groups of three-dimensional data acquired when the incomplete object is dynamic, and the incomplete object is an object with partial parts being blocked or defective;
the position determining module is used for identifying the three-dimensional data of the object through a preset deformity identification model so as to determine a deformity position on the deformity object;
the repairing processing module is used for performing characteristic repairing on the incomplete position on the incomplete object;
the position determination module is specifically configured to:
identifying the three-dimensional data of the object through a preset incomplete identification model so as to determine the object type of the incomplete object;
determining the incomplete position on the incomplete object according to the object type;
the repair processing module is specifically configured to:
determining missing information according to the object type of the incomplete object and the incomplete position, and determining a corresponding three-dimensional repair material according to the missing information;
and performing characteristic repair on the incomplete position according to the three-dimensional repair material.
7. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the photo restoration method according to any one of claims 1 to 5.
8. A terminal device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the photo repair method according to any one of claims 1 to 5 when executing the computer program.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810468017.8A CN108765321B (en) | 2018-05-16 | 2018-05-16 | Photo restoration method, device, storage medium and terminal device |
| PCT/CN2019/085452 WO2019218878A1 (en) | 2018-05-16 | 2019-05-05 | Photography restoration method and apparatus, storage medium and terminal device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810468017.8A CN108765321B (en) | 2018-05-16 | 2018-05-16 | Photo restoration method, device, storage medium and terminal device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108765321A CN108765321A (en) | 2018-11-06 |
| CN108765321B true CN108765321B (en) | 2021-09-07 |
Family
ID=64008166
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810468017.8A Active CN108765321B (en) | 2018-05-16 | 2018-05-16 | Photo restoration method, device, storage medium and terminal device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108765321B (en) |
| WO (1) | WO2019218878A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108765321B (en) * | 2018-05-16 | 2021-09-07 | Oppo广东移动通信有限公司 | Photo restoration method, device, storage medium and terminal device |
| CN112785633A (en) * | 2020-12-30 | 2021-05-11 | 深兰人工智能芯片研究院（江苏）有限公司 | Cultural relic repair strategy acquisition method and device, electronic equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107343148A (en) * | 2017-07-31 | 2017-11-10 | 广东欧珀移动通信有限公司 | Image completion method, device and terminal |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101142739B1 (en) * | 2009-10-29 | 2012-05-03 | 삼성전자주식회사 | Apparatus and method for inpainting image by restricting reference image region |
| TWI419078B (en) * | 2011-03-25 | 2013-12-11 | Univ Chung Hua | Instant stereo image generating device and method |
| CN105069833A (en) * | 2015-07-29 | 2015-11-18 | 苏州华漫信息服务有限公司 | Three-dimensional model repair method capable of retaining texture information data |
| CN105598450B (en) * | 2016-02-02 | 2017-11-10 | 陕西天元智能再制造股份有限公司 | A kind of laser three-dimensional profiling restorative procedure of parts damage |
| CN105763812B (en) * | 2016-03-31 | 2019-02-19 | 北京小米移动软件有限公司 | Intelligent photographing method and device |
| CN106127147B (en) * | 2016-06-23 | 2019-07-26 | 深圳市唯特视科技有限公司 | A kind of face depth texture restorative procedure based on three-dimensional data |
| CN106600701A (en) * | 2016-10-28 | 2017-04-26 | 黑龙江省科学院自动化研究所 | Digital museum cultural relics display method based on virtual reality technology |
| CN107392873A (en) * | 2017-07-28 | 2017-11-24 | 上海鋆创信息技术有限公司 | The virtual restorative procedure of object, object restorative procedure and object virtual display system |
| CN107437268A (en) * | 2017-07-31 | 2017-12-05 | 广东欧珀移动通信有限公司 | Photographing method, device, mobile terminal and computer storage medium |
| CN107452034B (en) * | 2017-07-31 | 2020-06-05 | Oppo广东移动通信有限公司 | Image processing method and device thereof |
| CN107948499A (en) * | 2017-10-31 | 2018-04-20 | 维沃移动通信有限公司 | A kind of image capturing method and mobile terminal |
| CN108765321B (en) * | 2018-05-16 | 2021-09-07 | Oppo广东移动通信有限公司 | Photo restoration method, device, storage medium and terminal device |
-
2018
- 2018-05-16 CN CN201810468017.8A patent/CN108765321B/en active Active
-
2019
- 2019-05-05 WO PCT/CN2019/085452 patent/WO2019218878A1/en active Application Filing
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107343148A (en) * | 2017-07-31 | 2017-11-10 | 广东欧珀移动通信有限公司 | Image completion method, device and terminal |
Non-Patent Citations (1)
| Title |
|---|
| 基于RBF神经网络的三维残缺数据修补算法;熊邦书 等;《计算机工程与应用》;20060719(第16期);第2节 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019218878A1 (en) | 2019-11-21 |
| CN108765321A (en) | 2018-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12073502B2 (en) | Electronic device for displaying avatar corresponding to external object according to change in position of external object | |
| US12190576B2 (en) | Electronic device for providing shooting mode based on virtual character and operation method thereof | |
| CN109523485B (en) | Image color correction method, device, storage medium and mobile terminal | |
| WO2019101021A1 (en) | Image recognition method, apparatus, and electronic device | |
| CN110059652B (en) | Face image processing method, device and storage medium | |
| CN110412765B (en) | Augmented reality image capturing method, device, storage medium and augmented reality device | |
| US20190354662A1 (en) | Apparatus and method for recognizing an object in electronic device | |
| CN108681402A (en) | Identification interaction method and device, storage medium and terminal equipment | |
| CN108491780B (en) | Image beautification processing method and device, storage medium and terminal equipment | |
| CN109522863B (en) | Ear key point detection method and device and storage medium | |
| CN108646920A (en) | Identification interaction method and device, storage medium and terminal equipment | |
| US20200104581A1 (en) | Electronic device and gesture recognition method thereof | |
| KR102661596B1 (en) | Electronic device for providing recognition result about external object using recognition information for image, similarity recognition information related to recognition information, and layer information, and operating method thereof | |
| US11144197B2 (en) | Electronic device performing function according to gesture input and operation method thereof | |
| KR20190036168A (en) | Method for correcting image based on category and recognition rate of objects included image and electronic device for the same | |
| WO2019218879A1 (en) | Photographing interaction method and apparatus, storage medium and terminal device | |
| CN108765321B (en) | Photo restoration method, device, storage medium and terminal device | |
| WO2024139261A1 (en) | Method and apparatus for determining depth image | |
| CN115702443A (en) | Apply stored digital makeup enhancements to recognized faces in digital images | |
| CN112153300A (en) | Multi-view camera exposure method, device, equipment and medium | |
| CN110213205B (en) | Verification method, device and equipment | |
| US11164388B2 (en) | Electronic device and method for providing augmented reality object therefor | |
| CN109389561B (en) | An imaging method and device | |
| CN112767453B (en) | Face tracking method and device, electronic equipment and storage medium | |
| CN111982293B (en) | Body temperature measuring method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |