WO2026007784A1 - Loss information calculation method and apparatus, and related device - Google Patents

Abstract

The present application relates to the technical field of computers, and discloses a loss information calculation method and apparatus, and a related device. The loss information calculation method in embodiments of the present application comprises: acquiring a target frequency band component of an output image of a neural network; acquiring a target frequency band component of a target image corresponding to the output image; and calculating loss information of the neural network, the loss information comprising loss information between the target frequency band component of the output image and the target frequency band component of the target image.

Description

Loss information calculation methods, devices and related equipment

Cross-references to related applications

This application claims priority to Chinese Patent Application No. 202410886308.4, filed in China on July 3, 2024, the entire contents of which are incorporated herein by reference.

Technical Field

This application belongs to the field of computer technology, specifically relating to a method, apparatus and related equipment for calculating loss information.

Background Technology

Neural networks primarily learn based on loss information. In some related technologies, the loss information of image-related neural networks is calculated equally for all pixels in the entire image, resulting in poor targeting of the loss information and thus poor learning performance.

Summary of the Invention

This application provides a method, apparatus, and related equipment for calculating loss information, which can solve the problem that the poor learning effect of neural networks is caused by the poor targeting of loss information.

Firstly, a method for calculating loss information is provided, including:

Obtain the target frequency band components of the output image of the neural network;

Obtain the target frequency band component of the target image corresponding to the output image;

Calculate the loss information of the neural network, which includes the loss information between the target frequency band component of the output image and the target frequency band component of the target image.

Secondly, a loss information calculation device is provided, comprising:

The first acquisition module is used to acquire the target frequency band components of the output image of the neural network;

The second acquisition module is used to acquire the target frequency band component of the target image corresponding to the output image;

The calculation module is used to calculate the loss information of the neural network, the loss information including the loss information between the target frequency band component of the output image and the target frequency band component of the target image.

Thirdly, a loss information calculation apparatus is provided, the apparatus being configured to perform the steps of the loss information calculation method as provided in the embodiments of this application.

Fourthly, an electronic device is provided, the terminal including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions being executed by the processor to implement the steps of the loss information calculation method provided in the embodiments of this application.

Fifthly, an electronic device is provided, including a processor and a communication interface, wherein the processor is configured to acquire target frequency band components of an output image of a neural network; acquire target frequency band components of a target image corresponding to the output image; and calculate loss information of the neural network, the loss information including loss information between the target frequency band components of the output image and the target frequency band components of the target image.

In a sixth aspect, an electronic device is provided, comprising: a memory configured to store video data, and processing circuitry configured to implement the steps of the loss information calculation method provided in the embodiments of this application.

In a seventh aspect, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the loss information calculation method provided in the embodiments of this application.

Eighthly, a chip is provided, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the steps of the loss information calculation method provided in the embodiments of this application.

In a ninth aspect, a computer program/program product is provided, which is stored in a storage medium and is executed by at least one processor to implement the steps of the loss information calculation method provided in the embodiments of this application.

In this embodiment, the target frequency band component of the output image of the neural network is obtained; the target frequency band component of the target image corresponding to the output image is obtained; and the loss information of the neural network is calculated, wherein the loss information includes the loss information between the target frequency band component of the output image and the target frequency band component of the target image. Since the loss information includes the loss information between the target frequency band component of the output image and the target frequency band component of the target image, it is possible to obtain loss information specific to the target frequency band, making the loss information of the neural network more targeted and beneficial to improving the learning effect of the neural network.

Attached Figure Description

Figure 1 is a schematic diagram of the encoding and decoding system provided in an embodiment of this application;

Figure 2 is a schematic diagram of the encoder provided in an embodiment of this application;

Figure 3 is a schematic diagram of the decoder provided in an embodiment of this application;

Figure 4 is a flowchart of a loss information calculation method provided in an embodiment of this application;

Figure 5 is a schematic diagram of a frequency domain transformation provided in an embodiment of this application;

Figure 6 is a schematic diagram of a loss information provided in an embodiment of this application;

Figure 7 is a structural diagram of a loss information calculation device provided in an embodiment of this application;

Figure 8 is a structural diagram of an electronic device provided in an embodiment of this application;

Figure 9 is a structural diagram of a terminal provided in an embodiment of this application.

Detailed Implementation

The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, the scope of protection for "A or B" covers at least three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. In addition, the terms "A and/or B," "at least one of A and B," and "at least one of A or B" also cover at least the above three scenarios. The character "/" generally indicates that the preceding and following objects are in an "or" relationship.

Figure 1 is a schematic diagram of the encoding/decoding system 10 provided in an embodiment of this application. The technical solution of this application embodiment relates to encoding and decoding (CODEC) video data (including encoding or decoding). The video data includes original unencoded video, encoded video, decoded (e.g., reconstructed) video, or syntax elements, etc.

As shown in Figure 1, the encoding/decoding system 10 includes a source device 100, which provides encoded video data to be decoded and displayed by the destination device 110. Specifically, the source device 100 provides video data to the destination device 110 via a communication medium 120. The source device 100 and the destination device 110 may include any one or more of the following: desktop computer, laptop computer, tablet computer, set-top box, mobile phone, wearable device (e.g., smartwatch or wearable camera), television, camera, display device, in-vehicle device, virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, digital media player, video game console, video conferencing equipment, video streaming equipment, broadcast receiver equipment, broadcast transmitter equipment, spacecraft, aircraft, robot, satellite, etc.

In the example of Figure 1, source device 100 includes a data source 101, memory 102, encoder 200, and output interface 104. Destination device 110 includes an input interface 111, decoder 300, memory 113, and display device 114. Source device 100 represents an example of a video encoding device, while destination device 110 represents an example of a video decoding device. In other examples, source device 100 and destination device 110 may not include some of the components shown in Figure 1, or they may include components other than those shown in Figure 1. For example, source device 100 may receive video data from an external data source (such as an external camera). Similarly, destination device 110 may interface with an external display device instead of including an integrated display device. As another example, memory 102 and memory 113 may be external memories.

Although Figure 1 illustrates the source device 100 and the destination device 110 as separate devices, in some examples, they may be integrated into a single device. In such embodiments, the same hardware or software, separate hardware or software, or any combination thereof may be used to implement the functionality corresponding to the source device 100 and the functionality corresponding to the destination device 110.

In some examples, source device 100 and destination device 110 can perform unidirectional or bidirectional video transmission. If it is bidirectional video transmission, source device 100 and destination device 110 can operate in a substantially symmetrical manner, that is, each of source device 100 and destination device 110 includes an encoder and a decoder.

Data source 101 represents the source of video data (i.e., raw, unencoded video data) and provides encoder 200 with a series of images containing video data, which encoder 200 encodes. Data source 101 of source device 100 may include video acquisition devices (such as video cameras), video archives containing previously acquired raw video, or video feed interfaces for receiving video from video content providers. Alternatively, data source 101 may generate computer graphics-based data as source video, or combine live video, archived video, and computer-generated video. In these cases, encoder 200 encodes the acquired, pre-acquired, or computer-generated video data. Encoder 200 may rearrange the images from the received order (sometimes referred to as the "display order") according to the encoded order. Encoder 200 may generate a bitstream including the encoded video data. Source device 100 may then output the encoded video data to communication medium 120 via output interface 104 for reception or retrieval, for example, by input interface 111 of destination device 110.

The memory 102 of the source device 100 and the memory 113 of the destination device 110 represent general-purpose memory. In some examples, memory 102 may store raw video data from data source 101, and memory 113 may store decoded video data from decoder 300. Additionally or alternatively, memories 102 and 113 may respectively store software instructions executable by, for example, encoder 200 and decoder 300. Although memories 102 and 113 are shown separately from encoder 200 and decoder 300 in this example, it should be understood that encoder 200 and decoder 300 may also include internal memory for functionally similar or equivalent purposes. If encoder 200 and decoder 300 are deployed on the same hardware device, memories 102 and 113 may be the same memory. Furthermore, memories 102 and 113 may store, for example, encoded video data output from encoder 200 and input to decoder 300. In some examples, portions of memories 102 and 113 may be allocated as one or more video buffers, for example, to store raw, decoded, or encoded video data.

In some examples, source device 100 can output encoded data from output interface 104 to memory 113. Similarly, destination device 110 can access encoded data from memory 113 via input interface 111. Memory 113 or memory 102 can include any of a variety of distributed or locally accessed data storage media, such as hard drives, Blu-ray discs, digital versatile discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

Output interface 104 may include any type of medium or device capable of transmitting encoded video data from source device 100 to destination device 110. For example, output interface 104 may include a transmitter or transceiver, such as an antenna, configured to transmit encoded video data directly from source device 100 to destination device 110 in real time. The encoded video data may be modulated according to the communication standards of a wireless communication protocol and transmitted to destination device 110.

Communication medium 120 may include transient media, such as wireless broadcasting or wired network transmission. For example, communication medium 120 may include radio frequency (RF) spectrum or one or more physical transmission lines (e.g., cables). Communication medium 120 may form part of a packet-based network (such as a local area network, a wide area network, or a global network such as the Internet). Communication medium 120 may also take the form of a storage medium (e.g., a non-transitory storage medium), such as a hard disk, flash drive, compact disc, digital video disc, Blu-ray disc, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

In some implementations, the communication medium 120 may include a router, switch, base station, or any other device that can be used to facilitate communication from source device 100 to destination device 110. For example, a server (not shown) may receive encoded video from source device 100 and provide the encoded video data to destination device 110, for example, via network transmission. The server may include (for example, a web server for a website), a server configured to provide file transfer protocol services (such as File Transfer Protocol (FTP) or File Delivery Over Unidirectional Transport (FLUTE) protocol), a content delivery network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Services (MBMS) or Evolved Multimedia Broadcast Multicast Service (eMBMS) server, or a Network-attached storage (NAS) device, etc. The server can implement one or more HTTP streaming protocols, such as Moving Picture Experts Group (MPEG) Media Transport (MMT), Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), or Real Time Streaming Protocol (RTSP).

Destination device 110 can access encoded video data from a server, for example, via a wireless channel (e.g., Wi-Fi connection) or a wired connection (e.g., Digital Subscriber Line (DSL), Cable Modem, etc.) for accessing encoded video data stored on the server.

Output interface 104 and input interface 111 may represent a wireless transmitter/receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard or IEEE 802.15 standard (e.g., ZigBee™), Bluetooth standard, or other physical components. In the example where output interface 104 and input interface 111 include wireless components, output interface 104 and input interface 111 may be configured to transmit data, such as encoded video data, according to Wi-Fi, Ethernet, cellular networks (such as 4th Generation Mobile Communication Technology (4G), LTE (Long Term Evolution), Advanced LTE, 5th Generation Mobile Communication Technology (5G), 6th Generation Mobile Communication Technology (6G), etc.).

The technology provided in this application can be applied to support video encoding and decoding in one or more multimedia applications such as video conferencing, over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission, digital video encoded onto a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

The input interface 111 of the destination device 110 receives an encoded video bitstream from the communication medium 120. The encoded video bitstream may include syntax elements and encoded data units (e.g., sequences, image groups, images, slices, blocks, etc.), where the syntax elements are used to decode the encoded data units to obtain decoded video data. The display device 114 displays the decoded video data to the user. The display device 114 may include a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

The encoder 200 and decoder 300 can be implemented as one or more of various processing circuits, which may include microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. When the technology is implemented wholly or partially in software, the device may store instructions for the software in a suitable non-transitory computer-readable storage medium and use one or more processors to execute the instructions in hardware to perform the technology provided in the embodiments of this application.

The encoder 200 and decoder 300 can process based on the following video codec standards: H.263, H.264, H.265 (also known as High Efficiency Video Coding (HEVC)), H.266 (also known as Versatile Video Coding (VVC), Moving Picture Experts Group 2 (MPEG-2), MPEG-4, VP8, VP9, Alliance for Open Media Video 1 (AV1), Audio Video Coding Standard 1 (AVS1), AVS2, AVS3, or next-generation video standard protocols. This application does not specifically limit the implementation of these protocols.

Typically, encoder 200 and decoder 300 can perform block-based encoding and decoding of images. The term "block" generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used during encoding or decoding). For example, a block can include a two-dimensional matrix of samples of luminance or chrominance data. For example, encoder 200 and decoder 300 can encode and decode video data represented in YUV format.

Referring to Figure 2, which is a schematic diagram of the encoder 200 provided in an embodiment of this application, the encoder 200 can be the encoder 200 in Figure 1. In the example of Figure 2, the encoder 200 includes a memory 201, an encoding parameter determination unit 210, a residual generation unit 202, a transform processing unit 203, a quantization unit 204, an inverse quantization unit 205, an inverse transform processing unit 206, a reconstruction unit 207, a filter unit 208, a decoded picture buffer (DPB) 209, and an entropy encoding unit 220.

The memory 201 can store video data to be encoded. For example, the encoder 200 can receive and store video data from the data source 101 shown in Figure 1. In some examples, the memory 201 can be on the same chip as other components of the encoder 200 (as shown in Figure 2), or it can be on a separate chip from those components.

The coding parameter determination unit 210 includes a mode selection unit 211, an inter-frame prediction unit 212, and an intra-frame prediction unit 213. The inter-frame prediction unit 212 is used to obtain a first prediction block for the current block using an inter-frame prediction mode. The intra-frame prediction unit 213 is used to obtain a second prediction block for the current block using an intra-frame prediction mode. The mode selection unit 211 is used to obtain a target prediction block based on the first and second prediction blocks and determine the final prediction mode. Furthermore, the coding parameter determination unit 210 may also include other functional units, such as functional units for determining the partitioning method of coding units (CUs), functional units for determining the transformation type of the residual data of the CUs, or functional units for determining the quantization parameters of the residual data of the CUs.

For ease of description and understanding, in the embodiments of this application, the CU to be processed in the current image is referred to as the current CU, and the image block to be processed in the current CU is referred to as the current block or the image block to be processed. For example, in encoding, it refers to the block currently being encoded; in decoding, it refers to the block currently being decoded.

Inter-frame prediction unit 212 may include a motion estimation unit and a motion compensation unit. For inter-frame prediction of the current block, the motion estimation unit may perform a motion search to identify one or more matching reference blocks in one or more reference pictures (e.g., one or more previously encoded/decoded pictures stored in DPB 209).

The motion estimation unit can generate one or more motion vectors (MVs) representing the position of a reference block in a reference image relative to the position of the current block in the current image. The motion compensation unit can then use interpolation to obtain a predicted value with the precision indicated by the motion vectors.

The encoding parameter determination unit 210 can provide the target prediction block to the residual generation unit 202. The residual generation unit 202 receives the raw uncoded video data of the current block from the memory 201 and calculates the residual between the current block and the target prediction block to obtain the residual block. In some examples, the function of the residual generation unit 202 can be implemented using one or more subtractor circuits that perform binary subtraction.

As an example, the encoding parameter determination unit 210 can provide the entropy encoding unit 220 with syntax elements representing encoding parameters for encoding. The encoding parameters include one or more of the following: the partitioning method of the CU, the final prediction mode, the transformation type of the residual data of the CU, or the quantization parameters of the residual data of the CU.

The transform processing unit 203 transforms the residual block output by the residual generation unit 202 to obtain a transform coefficient block. This transformation may include Discrete Cosine Transform (DCT), integer transform, direction transform, or Karhunen-Loeve transform, etc. In some examples, the encoder 200 may not include the transform processing unit 203.

Quantization unit 204 can quantize the transform coefficients in the transform coefficient block according to the quantization parameter (QP) value associated with the current block to produce a quantized transform coefficient block.

The inverse quantization unit 205 and the inverse transform processing unit 206 can perform inverse quantization and inverse transform on the transform coefficient block, respectively, to obtain the reconstructed residual block. The reconstruction unit 207 can generate a reconstructed block corresponding to the current block based on the reconstructed residual block and the target prediction block generated by the coding parameter determination unit 210.

Filter unit 208 can perform one or more filter operations on the reconstructed block. For example, filter unit 208 can be a deblocking filter (DBF), an adaptive loop filter (ALF), a sample adaptive offset (SAO) filter, etc. In some examples, encoder 200 may not include filter unit 208.

Encoder 200 stores the reconstructed image obtained from the reconstructed blocks in DPB 209. For example, in an example where the operation of filter unit 208 is not required, reconstruction unit 207 can store the reconstructed blocks in DPB 209. In an example where the operation of filter unit 208 is required, filter unit 208 can store the filtered reconstructed blocks in DPB 209. Inter-frame prediction unit 212 retrieves the reconstructed image from DPB 209 to perform inter-frame prediction on blocks of subsequent images to be encoded. In some examples, DPB 209 can be replaced with other types of memory.

Entropy coding unit 220 can entropy code the syntax elements of other components in encoder 200 to output encoded video data. For example, entropy coding unit 220 can entropy code the quantized transform coefficient block from quantization unit 204. As another example, entropy coding unit 220 can entropy code the syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from coding parameter determination unit 210.

It is understood that the composition of the encoder 200 shown in Figure 2 is only illustrative and does not constitute a limitation on the embodiments of this application.

Figure 3 is a schematic diagram of the structure of the decoder 300 provided in an embodiment of this application. The decoder 300 can be the decoder 300 described in Figure 1. In the example of Figure 3, the decoder 300 includes a coded picture buffer (CPB) 301, an entropy decoding unit 302, a prediction processing unit 310, an inverse quantization unit 303, an inverse transform processing unit 304, a reconstruction unit 305, a filter unit 306, and a DPB 307.

The entropy decoding unit 302 can receive encoded video data from the CPB 301 and perform entropy decoding on the video data to obtain syntax elements. The syntax elements indicate encoding parameters, including one or more of the following: CU partitioning method, final prediction mode, transformation type of CU residual data, or quantization parameters of CU residual data.

When the syntax element includes the final prediction mode, the prediction processing unit 310 obtains the final prediction mode. If the final prediction mode is an inter-frame prediction mode, the prediction block of the current CU can be obtained through the inter-frame prediction unit 311 of the prediction processing unit 310; if the final prediction mode is an intra-frame prediction mode, the prediction block of the current CU can be obtained through the intra-frame prediction unit 312 of the prediction processing unit 310. In some examples, the prediction processing unit 310 may also include a unit for performing prediction functions according to other prediction modes.

CPB 301 can acquire and store encoded video data from the communication medium 120 shown in Figure 1. DPB 307 is used to store decoded images. Optionally, CPB 301 and DPB 307 can be replaced with other types of memory, which are not specifically limited in this application. In some examples, CPB 301 can be on the same chip as other components of decoder 300 (as shown in the figure), or it can be on a separate chip from the chip where those components are located.

Decoder 300 can perform reconstruction operations on each block individually. Entropy decoding unit 302 can entropy decode the syntax elements and transform information (e.g., QP or transform mode indication) of the quantized transform coefficients to obtain the quantized transform coefficients. Dequantization unit 303 dequantizes the quantized transform coefficients to obtain a transform coefficient block including the transform coefficients. Inverse transform processing unit 304 performs an inverse transform on the transform coefficient block to generate a residual block corresponding to the current block; this inverse transform is the reverse operation of the above transform.

Reconstruction unit 305 can reconstruct the current block based on the prediction block and the residual block. For example, reconstruction unit 305 can add samples from the residual block to the corresponding samples from the prediction block to reconstruct the current block.

Filter unit 306 can perform one or more filter operations on the reconstructed block. For example, the type of filter unit 306 can be referenced to the type of filter unit 208, and will not be described again here. In some examples, the operations of filter unit 306 can be skipped.

Decoder 300 can store the reconstructed image obtained from the reconstructed blocks in DPB 307. For example, in an example where filter unit 306 is not operated, reconstruction unit 305 can store the reconstructed blocks in DPB 307. In an example where filter unit 306 is operated, filter unit 306 can store the filtered reconstructed blocks in DPB 307. Decoder 300 can output the decoded image (e.g., decoded video) from DPB 307 for subsequent rendering on a display device (such as display device 114 of FIG. 1).

The loss information calculation method provided in the embodiments of this application is described below with reference to the accompanying drawings. The loss information calculation method provided in the embodiments of this application can be executed by an encoding end, such as the encoder 200 shown in Figure 1 or Figure 2. The loss information calculation method provided in the embodiments of this application can be executed by a decoding end, such as the decoder 300 shown in Figure 1 or Figure 3. The encoding end and decoding end can be implemented by software, hardware, or a combination thereof. When implemented by hardware, the encoding end can be referred to as an encoding end device or a video encoding device, and the decoding end can be referred to as a decoding end device or a video decoding device.

Please refer to Figure 4, which is a flowchart of a loss information calculation method provided in an embodiment of this application. As shown in Figure 4, it includes the following steps:

Step 401: Obtain the target frequency band component of the output image of the neural network.

The aforementioned neural network can be a convolutional neural network, a deep convolutional neural network, a feedforward neural network, or a recurrent neural network, etc.

The aforementioned neural network is an image-related neural network, such as a neural network used to process image-related tasks in the field of video encoding and decoding, such as a neural network used for super-resolution (SR), a neural network used for image enhancement, or a neural network used for loop filtering, etc.

The above output image is the image output by the neural network, that is, the image obtained after processing by the neural network.

It should be noted that the type or structure of the neural network is not limited in the embodiments of this application, nor is the function of the neural network limited. Specifically, it can be a neural network related to the field of image or video encoding and decoding.

In this embodiment of the application, the target frequency band component is a component of the target frequency band, which may specifically be a low-frequency component, a high-frequency component, or a mid-frequency component.

Step 402: Obtain the target frequency band component of the target image corresponding to the output image.

The target image corresponding to the output image can be the original image, specifically an image that approximates the target of the neural network, meaning the neural network learns using this target image as its target. Alternatively, the target image can be referred to as a positive image sample or a real image sample during the neural network learning process. Furthermore, the target image and the output image have the same size.

It should be noted that the execution order of steps 401 and 402 is not limited in this embodiment. Step 401 can be executed first, followed by step 402, or step 402 can be executed first, followed by step 401, or steps 401 and 402 can be executed simultaneously.

Step 403: Calculate the loss information of the neural network, the loss information including the loss information between the target frequency band component of the output image and the target frequency band component of the target image.

The loss information of the aforementioned computational neural network can be the loss information between the target frequency band component of the output image and the target frequency band component of the target image.

In some implementations, the loss function used to calculate the loss information can be loss function L1 (i.e., absolute error loss function), loss function L2 (i.e., mean squared error loss function), or structural similarity index (SSIM) loss function, etc.

It should be noted that the loss function used to calculate the loss information is not limited in the embodiments of this application.

In this embodiment of the application, since the loss information includes the loss information between the target frequency band component of the output image and the target frequency band component of the target image, it is possible to obtain loss information for the target frequency band, making the loss information of the neural network more targeted and beneficial to improving the learning effect of the neural network.

In this embodiment of the application, the above-mentioned loss information calculation method can be executed by the encoding end, that is, the encoding end executes the above steps. In some embodiments, the above-mentioned loss information calculation method can also be executed by other electronic devices other than the encoding end, such as by an electronic device that performs neural network learning to execute the steps in the above method. The electronic device can specifically be a computer, server, mobile phone or other electronic devices.

In the embodiments of this application, steps 401 to 403 may be configured to be executed in the aforementioned neural network, or may be executed using an algorithm or program other than the aforementioned neural network, and there is no limitation on this.

As an optional implementation, the above method further includes the following steps:

The neural network is then trained based on the aforementioned loss information.

In this embodiment of the application, the learning of the neural network can also be referred to as the training of the neural network.

The above learning can improve the neural network's attention to the target frequency band, thereby improving the neural network's processing performance for tasks related to the target frequency band. If the target frequency band is high frequency, the neural network can pay more attention to high frequency information and focus more directly on the high frequency components in the image. This can improve the conversion effect from low resolution to high resolution for super-resolution tasks.

It should be noted that the embodiments of this application are not limited to performing the above steps. For example, in some implementations, after obtaining the above loss information, the loss information may be sent to other devices, and the other devices may learn the above neural network based on the loss information.

In some implementations, the above method may further include using the learned neural network to perform video encoding/decoding related image tasks such as SR or image enhancement to improve video encoding/decoding performance.

As an optional implementation, the acquisition of the target frequency band components of the output image of the neural network includes:

The output image of the neural network is transformed in the frequency domain to obtain the frequency domain information of the output image, and the target frequency band component is obtained from the frequency domain information of the output image.

The frequency domain transformation of the neural network output image described above can be achieved by using a spatial-to-frequency domain transformation technique to obtain the frequency domain information of the output image. This frequency domain information includes the components of each frequency band in the output image.

The above-mentioned method of obtaining the target frequency band component from the frequency domain information of the output image can be achieved by directly extracting the target frequency band component from the frequency domain information of the output image.

The aforementioned target frequency band component can be understood as the target frequency band information in the frequency domain information of the image.

In this embodiment, frequency domain transformation can be implemented to obtain the target frequency band component of the output image. This allows neural networks that do not include frequency domain information in the output image to obtain the loss information corresponding to the target frequency band, thereby improving the performance of these neural networks.

It should be noted that the embodiments of this application are not limited to obtaining the target frequency band component through the above frequency domain transformation. For example, in some embodiments, the output image of the above neural network itself carries frequency domain information, so the target frequency band component can be directly extracted from the output image.

As an optional implementation, obtaining the target frequency band component of the target image corresponding to the output image includes:

The target image corresponding to the output image is subjected to frequency domain transformation to obtain the frequency domain information of the target image, and the target frequency band component is obtained from the frequency domain information of the target image.

The aforementioned frequency domain transformation of the target image can be performed using a spatial-to-frequency domain transformation method to obtain the frequency domain information of the target image. This frequency domain information includes the components of each frequency band in the target image.

The above-mentioned method of obtaining the target frequency band component from the frequency domain information of the target image can be achieved by directly extracting the target frequency band component from the frequency domain information of the target image.

In this embodiment, frequency domain transformation can be used to obtain the target frequency band component of the target image. This allows loss information corresponding to the target frequency band to be obtained even for target images that do not include frequency domain information, thereby reducing the limitations of loss information calculation.

It should be noted that the embodiments of this application are not limited to obtaining the target frequency band components of the target image through the above frequency domain transformation. For example, in some embodiments, the frequency information of the target image can be obtained directly from other devices.

Optionally, the frequency domain transformation includes the following:

Discrete Wavelet Transform (DWT), Discrete Cosine Transform (DCT), and Discrete Fourier Transform (DFT).

In this implementation, multiple frequency domain transformations are supported to improve the flexibility of loss calculation.

In some implementations, the order of the frequency domain transform can be set according to the complexity requirements. Taking the first-order transform of DWT as an example, as shown in Figure 5, the digital image represents the output image or target image. The high-frequency components and low-frequency components are obtained through row decomposition, column decomposition, column reconstruction, and row reconstruction, specifically H and L in Figure 5.

As an optional implementation, the target frequency band component includes at least one of the following:

High-frequency components, mid-frequency components, and low-frequency components.

In this embodiment, loss information associated with at least one of the high-frequency component, mid-frequency component, and low-frequency component can be calculated, so that the neural network can better focus on the high-frequency component, mid-frequency component, or low-frequency component, thereby improving the processing details of the high-frequency component, mid-frequency component, or low-frequency component in the image processing process and thus improving the processing performance of the neural network.

For example, taking high-frequency components as an example, by calculating the loss information of high-frequency components, the neural network can focus more directly on the information of high-frequency parts, thereby improving the neural network's ability to learn high-frequency information from data. This enables the network to supplement more high-frequency detail information during high-resolution reconstruction, thereby improving the image's ability to recover high-frequency information.

In some implementations, when the target frequency band component includes one of high-frequency components, mid-frequency components, and low-frequency components, more targeted loss information can be obtained. Taking high-frequency components and loss function LI as an example, the loss information can include loss information calculated by the following formula:

Wherein, L1_DWT_loss represents the loss information between the high-frequency components of the output image and the high-frequency components of the target image, n is the number of high-frequency component sample points of the output image and the target image, _Yi is the high-frequency component of the target image, and f( _xi ) is the high-frequency component of the output image.

A specific schematic diagram can be shown in Figure 6. In Figure 6, the neural network is SR network as an example. SR in Figure 6 represents the above output image, and GT represents the above target image.

Optionally, calculating the loss information of the neural network includes at least one of the following:

Calculate a first loss value between the high-frequency components of the output image and the high-frequency components of the target image;

Calculate a second loss value between the mid-frequency components of the output image and the mid-frequency components of the target image;

Calculate a third loss value between the low-frequency components of the output image and the low-frequency components of the target image.

The first loss value, the second loss value, and the third loss value mentioned above can be calculated using the same or different loss functions.

In this implementation, a loss value can be obtained for any one of the high-frequency, mid-frequency, and low-frequency components, or multiple loss values. Calculating a single loss value allows the neural network to better focus on the high-frequency, mid-frequency, or low-frequency components, improving the processing detail of these components in image processing and thus enhancing the network's performance. Calculating multiple loss values allows the neural network to selectively focus on multiple frequency bands, improving its ability to learn information from multiple frequency bands in the data and further enhancing its performance.

Optionally, when the target frequency band component includes at least two of high-frequency components, mid-frequency components, and low-frequency components, the loss information between the target frequency band component of the output image and the target frequency band component of the target image includes: a weighted average loss value of at least two of the first loss value, the second loss value, and the third loss value.

The weighted average loss value of at least two of the aforementioned first, second, and third loss values can be a pre-configured or protocol-defined weight value for each frequency band. The loss value of each frequency band is multiplied by its corresponding weight, and then the average is calculated to obtain the aforementioned weighted average loss value. For example, if the weights corresponding to the high-frequency component, mid-frequency component, and low-frequency component are weight 1, weight 2, and weight 3, respectively, the aforementioned weighted average loss value can be equal to one of the following:

(First loss value x weight 1 + Second loss value x weight 2) / 2;

(First loss value x weight 1 + Third loss value x weight 3) / 2;

(Second loss value x weight 2 + Third loss value x weight 3) / 2;

(First loss value x weight 1 + second loss value x weight 2 + third loss value x weight 3)/2.

In this implementation, loss information of the neural network can be obtained based on loss values from multiple frequency bands. This allows the neural network to selectively focus on multiple frequency bands, improving its ability to learn information from data across multiple frequency bands and thus enhancing its performance. Furthermore, since it uses a weighted average loss value, the weights of each frequency band are considered, making the neural network's focus on multiple frequency bands more targeted, thereby further improving its ability to learn information from data across multiple frequency bands and enhancing its performance.

It should be noted that when the target frequency band components include at least two of the high-frequency components, mid-frequency components, and low-frequency components, the embodiments of this application are not limited to obtaining the loss information by weighted averaging. For example, the loss average of at least two components can also be calculated directly.

As one optional implementation of the above, the output image includes:

The upsampled image output by the neural network.

In this embodiment, since the output image includes an upsampled image output by the neural network, when the neural network is deployed at the decoding end, it can support the encoding end to compress the downsampled image, saving transmission bandwidth. Furthermore, for SR, the decoding end decodes the reconstructed low-resolution image, and then outputs a high-quality upsampled image (i.e., the original resolution image) via the neural network to improve image processing performance.

It should be noted that the output image in this application embodiment is not limited to an upsampled image. For example, in some implementations, an unsampled or downsampled image may also be used.

In this embodiment, the target frequency band component of the output image of the neural network is obtained; the target frequency band component of the target image corresponding to the output image is obtained; and the loss information of the neural network is calculated, wherein the loss information includes the loss information between the target frequency band component of the output image and the target frequency band component of the target image. Since the loss information includes the loss information between the target frequency band component of the output image and the target frequency band component of the target image, it is possible to obtain loss information specific to the target frequency band, making the loss information of the neural network more targeted and beneficial to improving the learning effect of the neural network.

The loss information calculation method provided in this application can be executed by a loss information calculation device. As an example, the device can be an electronic device or a component within an electronic device, such as a chip or circuit. This application uses the execution of the loss information calculation method by a loss information calculation device as an example to illustrate the loss information calculation device provided in this application.

Please refer to Figure 7, which is a structural diagram of a loss information calculation device provided in an embodiment of this application. As shown in Figure 7, the loss information calculation device 700 includes:

The first acquisition module 701 is used to acquire the target frequency band component of the output image of the neural network.

The second acquisition module 702 is used to acquire the target frequency band component of the target image corresponding to the output image;

The calculation module 703 is used to calculate the loss information of the neural network, the loss information including the loss information between the target frequency band component of the output image and the target frequency band component of the target image.

Optionally, the first acquisition module 701 is used to perform frequency domain transformation on the output image of the neural network to obtain the frequency domain information of the output image, and to obtain the target frequency band component from the frequency domain information of the output image.

Optionally, the second acquisition module 702 is used to perform frequency domain transformation on the target image corresponding to the output image to obtain the frequency domain information of the target image, and to obtain the target frequency band component from the frequency domain information of the target image.

Optionally, the frequency domain transformation includes the following:

Discrete Wavelet Transform (DWT), Discrete Cosine Transform (DCT), and Fourier Transform (DFT).

Optionally, the target frequency band component includes at least one of the following:

High-frequency components, mid-frequency components, and low-frequency components.

Optionally, the computing module 703 is used for at least one of the following:

Calculate a first loss value between the high-frequency components of the output image and the high-frequency components of the target image;

Calculate a second loss value between the mid-frequency components of the output image and the mid-frequency components of the target image;

Calculate a third loss value between the low-frequency components of the output image and the low-frequency components of the target image;

Optionally, when the target frequency band component includes at least two of high-frequency components, mid-frequency components, and low-frequency components, the loss information between the target frequency band component of the output image and the target frequency band component of the target image includes: a weighted average loss value of at least two of the first loss value, the second loss value, and the third loss value.

Optionally, the output image includes:

The upsampled image output by the neural network.

The aforementioned loss information calculation device is beneficial for improving the learning effect of neural networks.

The loss information calculation device 700 provided in this application embodiment can implement the various processes implemented in the method embodiment of FIG4 and achieve the same technical effect. To avoid repetition, it will not be described again here.

As shown in Figure 8, this application embodiment also provides an electronic device 800, including a processor 801 and a memory 802. The memory 802 stores programs or instructions that can run on the processor 801. For example, when the electronic device 800 is an encoding device, the program or instructions executed by the processor 801 implement the various steps of the above-described loss information calculation method embodiment and achieve the same technical effect. When the electronic device 800 is a decoding device, the program or instructions executed by the processor 801 implement the various steps of the above-described loss information calculation method embodiment and achieve the same technical effect. To avoid repetition, this will not be described again here. Optionally, the memory 802 can be the memory 102 or memory 113 in the embodiment shown in Figure 1, and the processor 801 can implement the functions of the encoder 200 or decoder 300 in the embodiments shown in Figures 1-3.

This application also provides an electronic device, including: a memory configured to store video data; and a processing circuit configured to implement the various steps of the loss information calculation method embodiments described above. Optionally, the memory may be memory 102 or memory 113 in the embodiment shown in FIG1, and the processing circuit may implement the functions of encoder 200 or decoder 300 in the embodiments shown in FIG1-3.

This application also provides an electronic device, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps in the method embodiment shown in FIG4. This device embodiment corresponds to the above method embodiment, and all implementation processes and methods of the above method embodiments can be applied to this terminal embodiment and achieve the same technical effect.

The processor or processing circuit in this application embodiment may include general-purpose processors, special-purpose processors, etc., such as central processing units (CPUs), microprocessors, digital signal processors (DSPs), artificial intelligence (AI) processors, graphics processing units (GPUs), application-specific integrated circuits (ASICs), network processors (NPs), field-programmable gate arrays (FPGAs), or other programmable logic devices, gate circuits, transistors, discrete hardware components, etc. The communication interface in this application embodiment may include transceivers, pins, circuits, buses, etc.

The aforementioned electronic devices can be terminals or other devices besides terminals, such as servers, network attached storage (NAS), etc.

The terminal can be a mobile phone, tablet computer, laptop computer, notebook computer, personal digital assistant (PDA), handheld computer, netbook, ultra-mobile personal computer (UMPC), mobile internet device (MID), augmented reality (AR), virtual reality (VR) device, mixed reality (MR) device, robot, wearable device, flight vehicle, vehicle user equipment (VUE), shipborne equipment, pedestrian user equipment (PUE), smart home (home devices with wireless communication capabilities, such as refrigerators, televisions, washing machines, or furniture), game console, personal computer (PC), ATM or self-service machine, etc. Wearable devices include: smartwatches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart chains, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among these, in-vehicle devices can also be referred to as in-vehicle terminals, in-vehicle controllers, in-vehicle modules, in-vehicle components, in-vehicle chips, or in-vehicle units, etc. It should be noted that the embodiments in this application do not limit the specific type of terminal.

A server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server. A cloud server can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), or cloud computing services based on big data and artificial intelligence platforms.

For example, the aforementioned electronic device may include, but is not limited to, the type of source device 100 or destination device 110 shown in FIG1.

Taking an electronic device as an example, Figure 9 is a schematic diagram of the hardware structure of a terminal that implements an embodiment of this application.

The terminal 900 includes, but is not limited to, at least some of the following components: radio frequency unit 901, network module 902, audio output unit 903, input unit 904, sensor 905, display unit 906, user input unit 907, interface unit 908, memory 909, and processor 910.

Those skilled in the art will understand that the terminal 900 may also include a power supply (such as a battery) for powering various components. The power supply can be logically connected to the processor 910 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The terminal structure shown in Figure 9 does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

It should be understood that, in this embodiment, the input unit 904 may include a graphics processor 9041 and a microphone 9042. The graphics processor 9041 processes image data of still images or videos obtained by an image acquisition device (such as a camera) in video acquisition mode or image acquisition mode, or it may process the obtained point cloud data. The display unit 906 may include a display panel 9061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include two parts: a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

In this embodiment, after receiving downlink data from the network-side device, the radio frequency unit 901 can transmit it to the processor 910 for processing; in addition, the radio frequency unit 901 can send uplink data to the network-side device. Typically, the radio frequency unit 901 includes, but is not limited to, antennas, amplifiers, transceivers, couplers, low-noise amplifiers, duplexers, etc.

The memory 909 can be used to store software programs or instructions, as well as various data. The memory 909 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 909 may include volatile memory or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 909 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

Processor 910 may include one or more processing units; optionally, processor 910 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 910.

The processor 910 is configured to acquire the target frequency band component of the output image of the neural network; acquire the target frequency band component of the target image corresponding to the output image; and calculate the loss information of the neural network, wherein the loss information includes the loss information between the target frequency band component of the output image and the target frequency band component of the target image.

Optionally, obtaining the components of the target frequency band of the output image of the neural network includes:

The output image of the neural network is transformed in the frequency domain to obtain the frequency domain information of the output image, and the target frequency band component is obtained from the frequency domain information of the output image.

Optionally, obtaining the target frequency band component of the target image corresponding to the output image includes:

The target image corresponding to the output image is subjected to frequency domain transformation to obtain the frequency domain information of the target image, and the target frequency band component is obtained from the frequency domain information of the target image.

Optionally, the frequency domain transformation includes the following:

Discrete Wavelet Transform (DWT), Discrete Cosine Transform (DCT), and Fourier Transform (DFT).

Optionally, the target frequency band component includes at least one of the following:

High-frequency components, mid-frequency components, and low-frequency components.

Optionally, calculating the loss information of the neural network includes at least one of the following:

Calculate a first loss value between the high-frequency components of the output image and the high-frequency components of the target image;

Calculate a second loss value between the mid-frequency components of the output image and the mid-frequency components of the target image;

Calculate a third loss value between the low-frequency components of the output image and the low-frequency components of the target image;

Optionally, when the target frequency band component includes at least two of high-frequency components, mid-frequency components, and low-frequency components, the loss information between the target frequency band component of the output image and the target frequency band component of the target image includes: a weighted average loss value of at least two of the first loss value, the second loss value, and the third loss value.

Optionally, the output image includes:

The upsampled image output by the neural network.

The aforementioned terminals are beneficial for improving the learning performance of neural networks.

It is understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description of the loss information calculation method in the method embodiment, and achieve the same or corresponding technical effect. To avoid repetition, it will not be described again here.

This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described loss information calculation method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

The processor mentioned above is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as ROM, RAM, magnetic disk, or optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.

This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described loss information calculation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

It should be understood that the chips mentioned in the embodiments of this application may include system-on-a-chip (also known as system chip, chip system, or system-on-a-chip) or discrete display chips, etc.

This application also provides a computer program/program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described loss information calculation method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.

The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.

Claims (17)

A method for calculating loss information, comprising: Obtain the target frequency band components of the output image of the neural network; Obtain the target frequency band component of the target image corresponding to the output image; Calculate the loss information of the neural network, which includes the loss information between the target frequency band component of the output image and the target frequency band component of the target image. According to the method of claim 1, wherein obtaining the components of the target frequency band of the output image of the neural network includes: The output image of the neural network is transformed in the frequency domain to obtain the frequency domain information of the output image, and the target frequency band component is obtained from the frequency domain information of the output image. According to the method of claim 1 or 2, wherein obtaining the target frequency band component of the target image corresponding to the output image includes: The target image corresponding to the output image is subjected to frequency domain transformation to obtain the frequency domain information of the target image, and the target frequency band component is obtained from the frequency domain information of the target image. According to claim 2 or 3, the method wherein the frequency domain transformation includes one of the following: Discrete Wavelet Transform (DWT), Discrete Cosine Transform (DCT), and Fourier Transform (DFT). The method according to any one of claims 1 to 4, characterized in that the target frequency band component includes at least one of the following: High-frequency components, mid-frequency components, and low-frequency components. According to the method of claim 5, the calculation of the loss information of the neural network includes at least one of the following: Calculate a first loss value between the high-frequency components of the output image and the high-frequency components of the target image; Calculate a second loss value between the mid-frequency components of the output image and the mid-frequency components of the target image; Calculate a third loss value between the low-frequency components of the output image and the low-frequency components of the target image. According to the method of claim 6, wherein, when the target frequency band component includes at least two of a high-frequency component, a mid-frequency component, and a low-frequency component, the loss information between the target frequency band component of the output image and the target frequency band component of the target image includes: a weighted average loss value of at least two of the first loss value, the second loss value, and the third loss value. The method according to any one of claims 1 to 7, wherein the output image comprises: The upsampled image output by the neural network. A loss information calculation device, comprising: The first acquisition module is used to acquire the target frequency band components of the output image of the neural network; The second acquisition module is used to acquire the target frequency band component of the target image corresponding to the output image; The calculation module is used to calculate the loss information of the neural network, the loss information including the loss information between the target frequency band component of the output image and the target frequency band component of the target image. According to the apparatus of claim 9, the first acquisition module is used to perform frequency domain transformation on the output image of the neural network to obtain frequency domain information of the output image, and to obtain the target frequency band component from the frequency domain information of the output image. According to the apparatus of claim 9 or 10, the second acquisition module is used to perform frequency domain transformation on the target image corresponding to the output image to obtain the frequency domain information of the target image, and to obtain the target frequency band component from the frequency domain information of the target image. The apparatus according to any one of claims 9 to 11, wherein the target frequency band component comprises at least one of the following: High-frequency components, mid-frequency components, and low-frequency components. The apparatus according to claim 12, wherein the computing module is used for at least one of the following: Calculate a first loss value between the high-frequency components of the output image and the high-frequency components of the target image; Calculate a second loss value between the mid-frequency components of the output image and the mid-frequency components of the target image; Calculate a third loss value between the low-frequency components of the output image and the low-frequency components of the target image. An electronic device includes a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the loss information calculation method as described in any one of claims 1 to 8. A readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the loss information calculation method as described in any one of claims 1 to 8. A computer program product, the computer program product being stored in a storage medium, the computer program product being executed by at least one processor to implement the steps of the loss information calculation method as described in any one of claims 1 to 8. A chip includes a processor and a communication interface coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the loss information calculation method as described in any one of claims 1 to 8.