CN112419179B - Method, apparatus, device and computer readable medium for repairing image - Google Patents

Abstract

Embodiments of the present disclosure disclose methods, apparatuses, electronic devices, and computer-readable media for repairing images. One embodiment of the method comprises the following steps: preprocessing an image to be repaired to obtain a first image, wherein the image to be repaired is an image with missing or unobvious image characteristics; determining at least one target region in the first image; processing the target area of the at least one target area to obtain a target modification area; constructing a second image based on at least one target modification region corresponding to the at least one target region; and carrying out image enhancement on the second image to obtain a target image. According to the embodiment, the image with missing or unobvious image characteristics is repaired, the repaired image content is more complete and the image quality is better by reasonably applying the repair technology.

Description

Method, device, apparatus and computer-readable medium for repairing images

Technical Field

Embodiments of the present disclosure relate to the field of image processing, and in particular to a method, apparatus, device, and computer-readable medium for repairing an image.

Background technique

In the process of image acquisition, transmission and storage, due to various factors (such as electromagnetic interference), image features may be lost or unclear. In the process of global image restoration, some image details may be ignored. However, related image restoration technologies cannot solve this problem well.

Summary of the invention

The content of this disclosure is used to introduce concepts in a brief form, and these concepts will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection. Some embodiments of the present disclosure propose methods, devices, equipment and computer-readable media for repairing images to solve the technical problems mentioned in the above background technology section.

In a first aspect, some embodiments of the present disclosure provide a method for repairing an image, the method comprising: preprocessing an image to be repaired to obtain a first image, wherein the image to be repaired is an image in which image features are missing or not obvious; determining at least one target area in the first image; for a target area of at least one target area, processing the target area to obtain a target modification area; constructing a second image based on at least one target modification area corresponding to the at least one target area; and performing image enhancement on the second image to obtain a target image.

In a second aspect, some embodiments of the present disclosure provide a device for repairing an image, the device comprising: a preprocessing unit, configured to preprocess an image to be repaired to obtain a first image, wherein the image to be repaired is an image in which image features are missing or not obvious; a determination unit, configured to determine at least one target area in the first image; a processing unit, configured to process the target area of the at least one target area to obtain a target modification area; a construction unit, configured to construct a second image based on at least one target modification area corresponding to the at least one target area; and an enhancement unit, configured to perform image enhancement on the second image to obtain a target image.

In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method for repairing an image as in the first aspect.

In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for repairing an image according to the first aspect is implemented.

One of the above-mentioned embodiments of the present disclosure has the following beneficial effects: first, the image to be repaired is preprocessed to obtain a first image. This step is to repair the entire image, and the image repair effect can be grasped as a whole. Since the repair of the entire image may cause the loss of local details, it is necessary to repair the local details. Then, at least one target area in the first image is determined, and a target modification area of each target area is obtained, so as to further repair the local image, which can enhance the specific details of the image. Finally, the second image constructed by the target modification area is enhanced to obtain the target image, so as to supplement and enhance the image features of the image to be repaired. This implementation realizes the repair of images with missing or unclear image features, and through the reasonable use of repair technology, the content of the repaired image is more complete and the image quality is better.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

FIG1 is a schematic diagram of an application scenario of a method for repairing an image according to some embodiments of the present disclosure;

FIG2 is a flow chart of some embodiments of a method for repairing an image according to the present disclosure;

FIG3 is a flow chart of other embodiments of a method for repairing an image according to the present disclosure;

FIG4 is a flow chart of still further embodiments of a method for repairing an image according to the present disclosure;

FIG5 is a schematic structural diagram of some embodiments of an apparatus for repairing an image according to the present disclosure;

FIG. 6 is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure.

Detailed ways

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

FIG. 1 is a schematic diagram of an application scenario of a method for repairing an image according to some embodiments of the present disclosure.

In the application scenario of FIG. 1 , first, the electronic device 101 can receive the image to be repaired 102. Then, the electronic device 101 pre-processes the image to be repaired 102 to obtain a first image 103. The first image 103 is an image after global repair. Therefore, the first image 103 needs to be further processed. Next, a target area 104 in the first image 103 is determined. The target area 104 is processed to obtain a target modification area 105. A second image 106 is constructed based on the target modification area 105. This leads to a more detailed processing of the local details of the image. Finally, the second image 106 is enhanced to obtain a target image 107. Image enhancement is to further strengthen the image features.

It should be noted that the electronic device 101 can be hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the electronic device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

It should be understood that the number of electronic devices in FIG1 is only for illustration and any number of electronic devices may be provided according to implementation requirements.

Continuing to refer to FIG2 , a process 200 of some embodiments of the method for repairing an image according to the present disclosure is shown. The method for repairing an image comprises the following steps:

Step 201: pre-process the image to be repaired to obtain a first image.

In some embodiments, the execution subject of the method for repairing an image (e.g., the electronic device 101 shown in FIG. 1 ) may receive the image to be repaired via a wired connection or a wireless connection. The image to be repaired may be an image whose image features are missing or unclear. It should be noted that the wireless connection method may include but is not limited to 3G/4G/5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

In some embodiments, the image to be restored is generally an image that has lost some necessary image features or has unclear image features. The image to be restored may be an image acquired by an old photographing device or an image obtained by scanning an old photo. The image to be restored may be any image. As an example, the image to be restored may be an image showing a human face, a piglet, a sparrow, a house, etc.

In some embodiments, the above preprocessing process is to repair the entire image to be repaired. As an example, the image to be repaired can be repaired by existing image repair software.

Step 202: determine at least one target area in the first image.

In some embodiments, based on the first image in step 201, the execution subject (e.g., the electronic device shown in FIG. 1 ) may determine at least one target area in the first image. Typically, there is a target object image in the target area. The execution subject may determine the target area in the first image by target detection.

Step 203: for at least one target region, process the target region to obtain a target modification region.

In some embodiments, as an example, the execution subject may obtain the target modification area by performing image processing operations such as adjusting contrast and sharpening on the target area. In this way, the details of the target area are repaired again in a targeted manner, thereby realizing the repair of local image features.

Step 204: construct a second image based on at least one target modification region corresponding to the at least one target region.

In some embodiments, the first image of the target area that has been processed in detail again is the final second image.

Step 205: Perform image enhancement on the second image to obtain a target image.

In some embodiments, image enhancement can be achieved through the following algorithms: grayscale linear transformation, histogram equalization transformation, homomorphic filtering, etc. Image enhancement is to further strengthen image features and make image details clearer.

In the methods provided by some embodiments of the present disclosure, the purpose of the preprocessing operation of the image to be repaired is to perform image repair on the image to be repaired within the entire image range, and the first image obtained mainly takes into account the overall effect of the image. Then, the details of the target area are repaired again in a targeted manner. Finally, image enhancement is performed, the purpose of which is to improve the visual effect of the image, and to purposefully emphasize the overall or local characteristics of the image for the application of a given image. The originally unclear image becomes clear or certain specific image features are emphasized, the differences between the features of different objects in the image are enlarged, and uninteresting features are suppressed. In this way, the image quality can be improved, the amount of information can be enriched, the image interpretation and recognition effects can be enhanced, and the needs of certain special analyses can be met. The repair of images with missing or unclear image features is achieved in the above manner, and the content of the repaired image is more complete and the image quality is better through the reasonable use of repair technology.

Further referring to FIG3 , it shows a process 300 of another embodiment of the method for repairing an image. The process 300 of the method for repairing an image comprises the following steps:

Step 301: input the image to be repaired into a preprocessing model to obtain a first image.

In some embodiments, as examples, the algorithms of the preprocessing model may include: a diffusion-based method, a texture synthesis-based method, a data-driven image restoration method, and the like.

In an optional implementation of some embodiments, the preprocessing model is obtained by the following steps: obtaining multiple sample images and sample target images corresponding to each sample image in the multiple sample images, the sample images are images in which image features are missing or unclear, and the sample target images are images in which image features of the corresponding sample images are complete; taking each sample image in the multiple sample images as input, and taking the sample target image corresponding to each sample image in the multiple sample images as output, and training to obtain the preprocessing model.

In an optional implementation of some embodiments, the sample target image is obtained by the following steps: identifying the sample target object in the sample image; adding color based on the sample target object to obtain a sample color image; performing color balancing on the sample color image to obtain a sample color balanced image; performing defogging on the sample color balanced image to obtain a sample defogging image; adjusting the clarity of the sample defogging image to obtain a sample clear image; increasing the pixels of the sample clear image to obtain a sample target image corresponding to the sample image. As an example, adding color to the sample image can be done through online software; color balancing of the color image can also be done through image processing software; the sample defogging operation can be done through an end-to-end neural network model, or it can be done according to an atmospheric degradation model; the algorithm for making the image clear can be Wiener filtering, Fourier transform, etc.; increasing the pixels of the sample clear image is the super-resolution of the image, and the image super-resolution methods include interpolation methods, methods based on sparse representation (dictionary learning), etc. The above-mentioned image restoration process is to first colorize the image to obtain a sample color image. Then, the sample color image is color balanced to obtain a sample color balanced image. This step is to make the color transition in the image more natural. Then, the sample color-balanced image is defogged to obtain the sample defogged image. This step makes the foggy image clearer. For the image without fog, this step does not affect the image quality. After defogging, the clarity of the sample defogged image needs to be adjusted to obtain the sample clear image. This step makes the image quality better. Finally, the pixels of the sample clear image are increased to obtain the sample target image corresponding to the sample image. This step is the super-resolution operation of the image, which supplements the pixels of the image details.

Step 302: Input the first image into a target detection model to obtain at least one target region of the first image.

In some embodiments, the target detection model is used to identify at least one target object image in the first image, and to set a corresponding target area for each target object image in the at least one target object image. As an example, the target detection model can determine the target area through a target detection network. As an example, the target detection network can be an R-CNN network, an SPPNet network, a Fast R-CNN network, etc.

Step 303: for a target region in at least one target region, input the target region into a region processing model to obtain a target modification region corresponding to the target region.

In some embodiments, the region processing model is used to repair the image features of the target object image in the target region. The target modification region is obtained by further processing the target region in the image. As an example, the region processing model can be an image restoration model based on convolutional autoencoder, an image restoration model based on generative adversarial network, or an image restoration model based on recurrent neural network.

Step 304: construct a second image based on at least one target modification region corresponding to the at least one target region.

In some embodiments, the second image is an image obtained by further repairing the details of the target area based on the first image.

Step 305: Perform image enhancement on the second image to obtain a target image.

In some embodiments, the specific implementation of steps 304 and 305 and the technical effects brought about by them can refer to steps 204 and 205 in the embodiment corresponding to Figure 2, and will not be repeated here.

As can be seen from FIG. 3, compared with the description of some embodiments corresponding to FIG. 2, the process 300 of the method for repairing an image in some embodiments corresponding to FIG. 3 embodies how to obtain a first image, how to determine a target area, and how to perform operations on a target modification area. The first image is obtained by preprocessing the model. The image features can be controlled as a whole. In addition, the order of image repair in the pre-model is also very important, and different operation orders may bring different results. By training the pre-model, the operation steps for repairing the image can be reasonably allocated. The target area is obtained by the target detection model. Using the target detection model, the area that needs special repair can be found well, so that the details of the repaired image can be more fully reflected. The target modification area is obtained by the regional processing model. Image repair is performed on the region, so that the repair is more targeted and the repair effect is better.

Further referring to FIG4 , it shows a process 400 of another embodiment of the method for repairing an image. The process 400 of the method for repairing an image comprises the following steps:

Step 401: pre-process the image to be repaired to obtain a first image.

Step 402: determine at least one target area in the first image.

Step 403, adjusting the clarity of the target area to obtain a target clear area.

In some embodiments, the definition of the target area can be adjusted by non-neighborhood filtering, neighborhood filtering, least squares filtering and other methods.

Step 404, increasing the pixels of the target clear area to obtain the target modified area.

In some embodiments, as an example, increasing the pixels of the target area can be achieved by a method based on neighbor embedding, an example-based super-resolution reconstruction method, etc.

Step 405: construct a second image based on at least one target modification region corresponding to at least one target region.

Step 406: Perform image enhancement on the second image to obtain a target image.

In some embodiments, the specific implementation of steps 401, 402, 405, and 406 and the technical effects brought about by them can refer to steps 201, 202, 204, and 205 in the embodiment corresponding to Figure 2, and will not be repeated here.

As can be seen from FIG. 4 , compared with the description of some embodiments corresponding to FIG. 3 , the process 400 of the method for repairing an image in some embodiments corresponding to FIG. 4 embodies an implementation method for repairing a target area. The detail adjustment of the target area is completed by adjusting the clarity of the target area and increasing the pixels of the target area. This can well supplement the image details and improve the overall image repair, but may cause the problem of loss of local image details.

Further referring to FIG. 5 , as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a device for repairing an image, which device embodiments correspond to the method embodiments shown in FIG. 2 , and the device can be specifically applied to various electronic devices.

As shown in FIG5 , an apparatus 500 for repairing an image in some embodiments includes: a preprocessing unit 501, configured to preprocess an image to be repaired to obtain a first image, wherein the image to be repaired is an image in which image features are missing or not obvious; a determination unit 502, configured to determine at least one target area in the first image; a processing unit 503, configured to process the target area of at least one target area to obtain a target modification area; a construction unit 504, configured to construct a second image based on at least one target modification area corresponding to the at least one target area; and an enhancement unit 505, configured to perform image enhancement on the second image to obtain a target image.

In an optional implementation of some embodiments, the preprocessing unit 501 is further configured to: input the image to be repaired into a preprocessing model to obtain a first image.

In an optional implementation of some embodiments, the preprocessing model is obtained by the following steps: obtaining multiple sample images and sample target images corresponding to each sample image in the multiple sample images, the sample images are images in which image features are missing or unclear, and the sample target images are images in which image features of the corresponding sample images are complete; taking each sample image in the multiple sample images as input, and taking the sample target image corresponding to each sample image in the multiple sample images as output, and training to obtain the preprocessing model.

In an optional implementation of some embodiments, the sample target image is obtained by the following steps: identifying a sample target object within the sample image; adding color based on the sample target object to obtain a sample color image; performing color balancing on the sample color image to obtain a sample color balanced image; performing dehazing on the sample color balanced image to obtain a sample dehazed image; adjusting the clarity of the sample dehazed image to obtain a sample clear image; increasing the pixels of the sample clear image to obtain a sample target image corresponding to the sample image.

In an optional implementation of some embodiments, the determination unit 502 is further configured to: input the first image into a target detection model to obtain at least one target area of the first image, the target detection model is used to identify at least one target object image in the first image, and set a corresponding target area for each target object image in the at least one target object image.

In an optional implementation of some embodiments, the processing unit 503 is further configured to: for a target area in at least one target area, input the target area into a region processing model to obtain a target modification area corresponding to the target area, and the region processing model is used to repair image features of the target object image in the target area.

In an optional implementation of some embodiments, the processing unit 503 is further configured to: adjust the clarity of the target area to obtain a target clear area; and increase the pixels of the target clear area to obtain a target modified area.

It is understandable that the storage units recorded in the device 500 correspond to the steps in the method described with reference to Figure 2. Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 500 and the units contained therein, and will not be repeated here.

Referring to FIG. 6 below, it shows a schematic diagram of the structure of an electronic device (e.g., the server or terminal device in FIG. 1 ) 600 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG. 6 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG6 , the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to the bus 604.

Typically, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the electronic device 600 to communicate wirelessly or wired with other devices to exchange data. Although FIG. 6 shows an electronic device 600 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively. Each box shown in FIG. 6 may represent one device, or may represent multiple devices as needed.

In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: pre-processes the image to be repaired to obtain a first image, where the image to be repaired is an image with missing or unclear image features; determines at least one target area in the first image; processes the target area for the target area of the at least one target area to obtain a target modification area; constructs a second image based on at least one target modification area corresponding to the at least one target area; and performs image enhancement on the second image to obtain a target image.

Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a preprocessing unit, a determination unit, a processing unit, a construction unit and an enhancement unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the preprocessing unit may also be described as a "unit for preprocessing the image to be repaired".

The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

According to one or more embodiments of the present disclosure, a method for repairing an image is provided, comprising: preprocessing an image to be repaired to obtain a first image, wherein the image to be repaired is an image in which image features are missing or unclear; determining at least one target area in the first image; for a target area of the at least one target area, processing the target area to obtain a target modification area; constructing a second image based on at least one target modification area corresponding to the at least one target area; and performing image enhancement on the second image to obtain a target image.

According to one or more embodiments of the present disclosure, preprocessing the image to be repaired to obtain a first image includes: inputting the image to be repaired into a preprocessing model to obtain the first image.

According to one or more embodiments of the present disclosure, the preprocessing model is obtained by the following steps: obtaining multiple sample images and sample target images corresponding to each sample image in the multiple sample images, the sample images are images in which image features are missing or unclear, and the sample target images are images in which image features of the corresponding sample images are complete; taking each sample image in the multiple sample images as input, and taking the sample target image corresponding to each sample image in the multiple sample images as output, and training to obtain the preprocessing model.

According to one or more embodiments of the present disclosure, a sample target image is obtained by the following steps: identifying a sample target object within a sample image; adding color based on the sample target object to obtain a sample color image; performing color balancing on the sample color image to obtain a sample color balanced image; performing dehazing on the sample color balanced image to obtain a sample dehazed image; adjusting the clarity of the sample dehazed image to obtain a sample clear image; and increasing the pixels of the sample clear image to obtain a sample target image corresponding to the sample image.

According to one or more embodiments of the present disclosure, determining at least one target area in a first image includes: inputting the first image into a target detection model to obtain at least one target area of the first image, the target detection model is used to identify at least one target object image in the first image, and setting a corresponding target area for each target object image in the at least one target object image.

According to one or more embodiments of the present disclosure, for a target area of at least one target area, the target area is processed to obtain a target modification area, including: for the target area of at least one target area, the target area is input into a region processing model to obtain a target modification area corresponding to the target area, and the above-mentioned region processing model is used to repair image features of the target object image in the target area.

According to one or more embodiments of the present disclosure, the target area is input into the area processing model to obtain a target modification area corresponding to the target area, including: adjusting the clarity of the target area to obtain a target clear area; increasing the pixels of the target clear area to obtain a target modification area.

According to one or more embodiments of the present disclosure, there is provided a device for repairing an image, comprising: a preprocessing unit, configured to preprocess an image to be repaired to obtain a first image, wherein the image to be repaired is an image in which image features are missing or unclear; a determining unit, configured to determine at least one target area in the first image; a processing unit, configured to process the target area of the at least one target area to obtain a target modification area; a constructing unit, configured to construct a second image based on at least one target modification area corresponding to the at least one target area; and an enhancing unit, configured to perform image enhancement on the second image to obtain a target image.

According to one or more embodiments of the present disclosure, the preprocessing unit is further configured to: input the image to be repaired into a preprocessing model to obtain a first image.

According to one or more embodiments of the present disclosure, the preprocessing model is obtained by the following steps: obtaining multiple sample images and sample target images corresponding to each sample image in the multiple sample images, the sample images are images in which image features are missing or unclear, and the above-mentioned sample target images are images with complete image features corresponding to the above-mentioned sample images; taking each sample image in the multiple sample images as input, and taking the sample target image corresponding to each sample image in the multiple sample images as output, and training to obtain the preprocessing model.

According to one or more embodiments of the present disclosure, a sample target image is obtained by the following steps: identifying a sample target object within a sample image; adding color based on the sample target object to obtain a sample color image; performing color balancing on the sample color image to obtain a sample color balanced image; performing dehazing on the sample color balanced image to obtain a sample dehazed image; adjusting the clarity of the sample dehazed image to obtain a sample clear image; and increasing the pixels of the sample clear image to obtain a sample target image corresponding to the above-mentioned sample image.

According to one or more embodiments of the present disclosure, the determination unit is further configured to: input the first image into a target detection model to obtain at least one target area of the first image, the target detection model is used to identify at least one target object image in the first image, and set a corresponding target area for each target object image in the at least one target object image.

According to one or more embodiments of the present disclosure, the processing unit is further configured to: for a target area in at least one target area, input the target area into a region processing model to obtain a target modification area corresponding to the target area, and the region processing model is used to repair image features of the target object image in the target area.

According to one or more embodiments of the present disclosure, the processing unit is further configured to: adjust the clarity of the target area to obtain a target clear area; increase the pixels of the target clear area to obtain a target modified area.

The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims (8)

1. A method of repairing an image, comprising:

Inputting an image to be repaired into a preprocessing model to obtain a first image, wherein the image to be repaired is an image with missing or unobvious image characteristics; the preprocessing model is obtained based on a sample image and a sample target image;

determining at least one target region in the first image;

Processing the target area of the at least one target area to obtain a target modification area;

constructing a second image based on at least one target modification region corresponding to the at least one target region;

Performing image enhancement on the second image to obtain a target image;

the sample target image is obtained through the following steps:

identifying a sample target object within the sample image;

Adding color based on the sample target object to obtain a sample color image;

Performing color balance processing on the sample color image to obtain a sample color balance image;

Defogging the sample color balance image to obtain a sample defogging image;

adjusting the definition of the defogging image of the sample to obtain a clear image of the sample;

and adding pixels of the sample clear image to obtain a sample target image corresponding to the sample image.

2. The method of claim 1, wherein the pre-processing model is obtained by:

Acquiring a plurality of sample images and sample target images corresponding to each sample image in the plurality of sample images, wherein the sample images are images with missing or unobvious image features, and the sample target images are images with complete image features corresponding to the sample images;

And taking each sample image in the plurality of sample images as input, taking a sample target image corresponding to each sample image in the plurality of sample images as output, and training to obtain the preprocessing model.

3. The method of claim 1, wherein the determining at least one target region in the first image comprises:

And inputting the first image into a target detection model to obtain at least one target area of the first image, wherein the target detection model is used for identifying at least one target object image in the first image and setting a corresponding target area for each target object image in the at least one target object image.

4. A method according to claim 3, wherein said processing said target area for said at least one target area to obtain a target modified area comprises:

and inputting the target region into a region processing model for the target region in the at least one target region to obtain a target modification region corresponding to the target region, wherein the region processing model is used for repairing the image characteristics of the target object image in the target region.

5. The method of claim 4, wherein the inputting the target region into a region processing model results in a target modified region corresponding to the target region, comprising:

adjusting the definition of the target area to obtain a target clear area;

and adding pixels of the target clear region to obtain the target modification region.

6. An apparatus for acquiring an image, comprising:

The preprocessing unit is configured to input an image to be repaired into the preprocessing model to obtain a first image, wherein the image to be repaired is an image with missing or unobvious image characteristics; the preprocessing model is obtained based on a sample image and a sample target image;

A determining unit configured to determine at least one target area in the first image;

The processing unit is configured to process the target area of the at least one target area to obtain a target modification area;

a construction unit configured to construct a second image based on at least one target modification region corresponding to the at least one target region;

an enhancement unit configured to perform image enhancement on the second image to obtain a target image;

the sample target image is obtained through the following steps:

identifying a sample target object within the sample image;

Adding color based on the sample target object to obtain a sample color image;

Performing color balance processing on the sample color image to obtain a sample color balance image;

Defogging the sample color balance image to obtain a sample defogging image;

adjusting the definition of the defogging image of the sample to obtain a clear image of the sample;

and adding pixels of the sample clear image to obtain a sample target image corresponding to the sample image.

7. An electronic device, comprising:

one or more processors;

a storage device having one or more programs stored thereon,

When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1 to 5.

8. A computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements the method of any of claims 1 to 5.