CN121309817A

CN121309817A - Image processing method, device, system and related equipment

Info

Publication number: CN121309817A
Application number: CN202410910756.3A
Authority: CN
Inventors: 周兴扬; 张希舟; 亚历山大·拉图什尼亚克
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2024-07-08
Filing date: 2024-07-08
Publication date: 2026-01-09
Also published as: WO2026011715A1

Abstract

An image processing method, apparatus, system, and related equipment are disclosed, relating to the field of data processing technology. The method involves acquiring multiple images and grouping them according to their pixel features, resulting in multiple groups. The similarity between different images within the same group is higher than the similarity between images in different groups. Using a first pixel library corresponding to a first group, the images within the first group are encoded to obtain an image encoding result corresponding to the first group. The first pixel library includes pixels from some images within the first group. The image encoding result corresponding to the first group is used to reconstruct the images within the first group using the first pixel library. Thus, by utilizing the similarity between different images within the same group, the data volume of each group can be significantly compressed, resulting in a much smaller amount of encoded image data than the original image data, thereby reducing the storage space required for storing multiple images.

Description

Image processing method, device, system and related equipment

Technical Field

The present application relates to the field of data processing technologies, and in particular, to an image processing method, apparatus, system, and related device.

Background

The image is already important data in application scenes such as the Internet, scientific research, industrial production and the like.

As the resolution of an image increases, the amount of data of the image increases. For this reason, the image may be encoded into image data in a specified format, for example, the image is encoded by using an entropy encoding technique, and the amount of the encoded image data is generally smaller than the amount of the image before encoding, so as to reduce the storage space required for storing the image. The process of encoding an image is a process of representing as much image information as possible using as little data as possible.

In practical application scenarios, after encoding an image, the reduction rate of the image data is not high, for example, the reduction rate may only reach 2% to 10%, so that the data volume of the image obtained by encoding is still large, and thus the image storage still occupies a large storage space.

Disclosure of Invention

The application provides an image processing method for reducing the storage space occupied by image storage. Furthermore, the application also provides an image processing device, an audio processing method, an audio processing device, a computing device, a computer readable storage medium and a computer program product.

In a first aspect, the present application provides an image processing method, specifically, a plurality of images to be processed are acquired, the plurality of images may be images in the same format or different formats, and the plurality of images may be sent by a client on a user side, for example. Then, the plurality of images are grouped according to the pixel characteristics of the plurality of images, so that a plurality of groups are obtained, wherein the similarity degree between different images in the same group is higher than that between the images in different groups, namely, similar images in the plurality of images are classified into the same group, and dissimilar images are classified into different groups. And finally, coding the image in the first group by using a first pixel library corresponding to the first group to obtain an image coding result corresponding to the first group, wherein the first group is one group of a plurality of groups obtained by dividing, the first pixel library comprises pixels in partial images in the first group, and the image coding result corresponding to the first group is used for restoring the images in the first group with the first pixel library. For example, when the subsequent application requests the stored image in the first group, the image in the first group may be restored according to the image encoding result corresponding to the first group and the first pixel library. Similarly, for the other packets than the first packet among the plurality of packets, the encoding results corresponding to the respective packets may also be generated with reference to the above manner.

In this way, since the similarity degree between the plurality of images in the same group is high, the plurality of images in the group are encoded based on the pixels in the partial images in the group, and the similarity between different images can be fully utilized to improve the encoding effect on the plurality of images in the group. That is, the data amount of the images within each packet can be sufficiently compressed using the similarity between the images so that the number of image data after encoding is much smaller than the number of image data before encoding. Therefore, in the subsequent process of storing a plurality of images, only the encoding result with smaller data quantity is needed to be stored, so that the storage space occupied by storing a plurality of images can be reduced, namely the reduction rate of the images is improved. Wherein the reduction rate is the ratio between the reduced data amount relative to the data amount before reduction.

In a possible implementation manner, the plurality of images to be processed comprise images with different formats, and then the method further comprises performing format conversion on the plurality of images to obtain a plurality of images with standard formats, wherein the standard formats can be, for example, HUIF formats or other formats, and when the plurality of images are grouped according to pixel characteristics of the plurality of images, the plurality of images with standard formats can be grouped according to pixel characteristics of the plurality of images with standard formats. In this way, by format conversion, a plurality of images of different formats can be converted into an image of a unified format, so that the encoding processing of the plurality of images can be performed later, and the encoding performance can be improved (there is no need to execute different encoding logic for the images of different formats).

In a possible implementation manner, the pixel library corresponding to the first packet further comprises reference metadata, wherein the reference metadata refers to metadata used for reference in the process of encoding metadata of the image, and then the method further comprises the step of encoding the metadata of the image in the standard format in the first packet by using the reference metadata to obtain metadata encoding results corresponding to the first packet, and the metadata encoding results corresponding to the first packet are used for restoring the metadata of the image in the standard format in the first packet with the reference metadata. In this way, metadata of the images in the group are encoded by using the reference metadata in the pixel library, so that the data volume of the metadata can be reduced, and the storage space required for storing the metadata later can be reduced.

In one possible implementation manner, when the first pixel library corresponding to the first packet is utilized to encode the image in the first packet, specifically, the pixels in the image in the first packet and the pixels in the first pixel library may be compared to obtain pixel difference information, where the pixel difference information is used to describe the difference between the pixels in the image in the first packet and the pixels in the first pixel library, so that the pixel difference information is encoded, and an image encoding result corresponding to the first packet may be obtained, where the image encoding result includes the encoding result of each image in the first packet. Because the pixels of the image in the first group are similar to the pixels in the first pixel library, the difference information obtained by comparing the pixels of the image with the pixels in the pixel library generally has certain similarity and regularity, so that after the difference information with the regularity and the similarity is encoded, the data volume of the obtained encoding result is effectively smaller than the number of the difference information and is also effectively smaller than the data volume of the image in the first group, thereby effectively compressing the image data and reducing the storage space required for storing the image.

In a possible implementation manner, when the pixels in the image in the first group and the pixels in the first pixel library are compared, the image in the first group can be divided to obtain a plurality of original pixel blocks, and the reference pixel blocks which respectively meet the matching condition with each of the plurality of original pixel blocks are determined according to the first pixel library, wherein the matching condition comprises that the pixel similarity between the two pixel blocks is greater than a threshold value, so that the residual between the original pixel block meeting the matching condition and the reference pixel block is calculated, and the pixel difference information is obtained. In this way, the image in the first group is encoded with the granularity of the pixel blocks, so that in the process of recovering each pixel block in the image, only the reference pixel block corresponding to the pixel block in the pixel library is needed, and all the reference pixel blocks in the pixel library (and the reference pixel blocks in the pixel libraries corresponding to other groups) are not needed to be read, thereby avoiding the problem of reading amplification.

In one possible implementation, each of the plurality of reference pixel blocks is configured with a priority level, and the priority level is used for indicating whether the reference pixel block is used for judging whether the reference pixel block meets the priority level of the matching condition with the original pixel blocks, and when determining the reference pixel blocks respectively meeting the matching condition with the original pixel blocks in the plurality of original pixel blocks, the reference pixel blocks respectively meeting the matching condition with the original pixel blocks in the plurality of original pixel blocks can be determined from the first pixel library according to the priority level of each reference pixel block. In this way, for each original pixel block, a reference pixel block with a higher priority can be selected for matching preferentially, which can quickly traverse the reference pixel block meeting the matching condition with the original pixel block, so that all the reference pixel blocks in the pixel library do not need to be traversed for the original pixel block and calculated, thereby reducing the resource consumption required for determining the reference pixel block and improving the efficiency of determining the reference pixel block.

In one possible implementation manner, when the plurality of original pixel blocks include the first original pixel block, in determining, according to the first pixel library, a reference pixel block that satisfies a matching condition with each of the plurality of original pixel blocks, specifically, whether a reference pixel block that satisfies a matching condition with the first original pixel block exists in the first pixel library first may be searched, and when a reference pixel block that satisfies a matching condition with the first original pixel block does not exist in the first pixel library, the first original pixel block is added to the first pixel library as the reference pixel block. In this way, in the process of encoding the image in the first group, the reference pixel blocks in the new pixel library are added, so that the new reference pixel blocks can be utilized to realize the effective compression of other original pixel blocks in the first group, thereby improving the encoding effect on the image in the first group.

In one possible implementation manner, when the pixel difference information is encoded, the data characteristic of the pixel difference information is extracted first, and at least one encoding algorithm matched with the data characteristic is determined from a plurality of encoding algorithms according to the data characteristic, so that the encoding operation for the pixel difference information is sequentially performed by using the at least one encoding algorithm. Therefore, a plurality of proper coding algorithms are selected to code the pixel difference information, so that the coding effect of the pixel difference information can be ensured, namely, the data volume of the coding result of a plurality of images can be reduced as much as possible, and the storage space required for storing the images can be effectively reduced.

The application provides an image processing device, which comprises an acquisition module, a grouping module and an encoding module, wherein the acquisition module is used for acquiring a plurality of images to be processed, the grouping module is used for grouping the plurality of images according to the pixel characteristics of the plurality of images to obtain a plurality of groups, the similarity degree between different images in the same group is higher than that between the images in different groups, the plurality of groups comprise a first group, the encoding module is used for encoding the images in the first group by utilizing a first pixel library corresponding to the first group to obtain an image encoding result corresponding to the first group, the first pixel library comprises pixels in partial images in the first group, and the image encoding result corresponding to the first group is used for restoring the images in the first group with the first pixel library.

In one possible implementation, the plurality of images to be processed comprise images in different formats, and the device further comprises a conversion module for performing format conversion on the plurality of images to obtain a plurality of images in a standard format, and a grouping module for grouping the plurality of images in the standard format according to pixel characteristics of the plurality of images in the standard format.

In one possible implementation, the pixel library corresponding to the first packet further comprises reference metadata, and the encoding module is further configured to encode metadata of the standard format image in the first packet by using the reference metadata to obtain a metadata encoding result corresponding to the first packet, where the metadata encoding result corresponding to the first packet is used to restore metadata of the standard format image in the first packet with the reference metadata.

In one possible implementation, the encoding module is configured to compare a pixel in an image in the first group with a pixel in the first pixel library to obtain pixel difference information, where the pixel difference information is used to describe a difference between the pixel in the image in the first group and the pixel in the first pixel library, and encode the pixel difference information to obtain an image encoding result corresponding to the first group.

In one possible implementation, the first pixel library includes pixels in a plurality of reference pixel blocks, the reference pixel blocks are pixel blocks in an image, pixel difference information is a residual error, the encoding module is used for dividing the image in the first group to obtain a plurality of original pixel blocks, determining, according to the first pixel library, the reference pixel blocks which respectively meet a matching condition with each of the plurality of original pixel blocks, wherein the matching condition includes that pixel similarity between two pixel blocks is greater than a threshold value, and calculating the residual error between the original pixel blocks which meet the matching condition and the reference pixel blocks.

In one possible implementation, each of the plurality of reference pixel blocks is configured with a priority indicating a degree of priority to which the reference pixel block is used to determine whether the reference pixel block meets a matching condition with an original pixel block, and an encoding module for determining, from the first pixel library, a reference pixel block that meets a matching condition with each of the plurality of original pixel blocks, respectively, according to the priority of each reference pixel block.

In one possible implementation, the plurality of original pixel blocks include a first original pixel block, an encoding module configured to find whether a reference pixel block satisfying a matching condition with the first original pixel block exists in the first pixel library, and add the first original pixel block as the reference pixel block to the first pixel library when the reference pixel block satisfying the matching condition with the first original pixel block does not exist in the first pixel library.

In one possible implementation, the encoding module is configured to extract data features of the pixel difference information, determine at least one encoding algorithm matching the data features from a plurality of encoding algorithms according to the data features, and sequentially perform encoding operations for the pixel difference information using the at least one encoding algorithm.

The image processing apparatus provided in the second aspect corresponds to the image processing method provided in the first aspect, so the technical effects of any implementation manner in the second aspect may be described with reference to the relevant points of the technical effects of the corresponding implementation manner in the first aspect, which are not described herein.

The application provides an image processing method, which comprises the steps of obtaining a first image to be processed, determining a first pixel library from a plurality of pixel libraries, wherein similar conditions are met between pixels in the first pixel library and pixels in the first image, the similar conditions are not met between pixels in different pixel libraries, comparing the first image with at least one reference pixel set in the first pixel library to obtain difference information, wherein the difference information is used for describing pixel differences between the first image relative to the at least one reference pixel set, encoding the difference information to obtain an encoding result corresponding to the first image, and the encoding result is used for recovering the first image with the at least one reference pixel set.

Since the pixels in the first image and the pixels in the reference pixel set in the first pixel library satisfy the similarity condition, that is, the pixels in the first image and the pixels in one or more reference pixel sets in the first pixel library have a certain similarity, the difference information obtained by comparing the first image and the reference pixel set generally has a certain similarity and rule based on the similarity. Thus, after the difference information with regularity and similarity is encoded, the data size of the obtained encoding result is effectively smaller than the number of the difference information and is also effectively smaller than the data size of the first image. In this way, when the first image is stored later, only the encoding result with smaller data quantity is needed to be stored, so that the storage space required by image storage can be reduced. Meanwhile, the first image can be restored by the coding result and the reference pixel set, so that the coding result with smaller data size is stored later, namely the first image is stored. Further, when the difference information is encoded in a lossless encoding mode, the complete first image can be recovered according to the encoding result and the reference pixel set, so that the lossless storage of the first image can be realized while the storage space required by the image storage is reduced.

In a possible implementation, the pixels in the reference pixel set belong to at least one image, the difference information is used for describing pixel differences between the first image and one image in the first pixel library, or the reference pixel set comprises pixels in reference pixel blocks, the reference pixel blocks are pixel blocks in one image, and the difference information is used for describing pixel differences between each pixel block in the first image and one reference pixel block in the first pixel library respectively.

In one possible implementation, the reference pixel set includes pixels in a reference pixel block, the difference information is a residual error, the comparing the first image with at least one reference pixel set in the first pixel library to obtain the difference information includes dividing the first image to obtain a plurality of original pixel blocks, determining a reference pixel block from the first pixel library, which satisfies a matching condition with each of the plurality of original pixel blocks, the matching condition includes that a similarity between pixels in two pixel blocks is greater than a threshold value, and calculating a residual error between pixels in the original pixel block and pixels in the reference pixel block, which satisfies the matching condition.

In one possible implementation, each reference pixel block in the first pixel library is configured with a priority indicating whether the reference pixel block is used to determine a priority degree of meeting a matching condition with an original pixel block, and the determining the reference pixel block meeting the matching condition with each original pixel block in the plurality of original pixel blocks from the first pixel library includes determining the reference pixel block meeting the matching condition with each original pixel block in the plurality of original pixel blocks from the first pixel library according to the priority of each reference pixel block.

In one possible implementation, the determining the first pixel library from the plurality of pixel libraries includes extracting pixel features of the first image, and determining the first pixel library from the plurality of pixel libraries according to the pixel features of the first image, wherein the similarity between the reference pixel features of the first pixel library and the pixel features of the first image is the largest.

In one possible implementation, the encoding of the difference information includes extracting a data characteristic of the difference information, determining at least one encoding algorithm matching the data characteristic from a plurality of encoding algorithms based on the data characteristic, and encoding the difference information using the at least one encoding algorithm.

In one possible implementation, the determining the first pixel library from the plurality of pixel libraries includes performing format conversion on the first image to obtain a first image in a standard format, and determining the first pixel library from the plurality of pixel libraries for the first image in the standard format.

In a possible implementation manner, the difference information comprises metadata difference information and pixel difference information, the first pixel library further comprises reference metadata, and the comparing the image in the standard format with at least one reference pixel set in the first pixel library comprises comparing the metadata of the image in the standard format with the reference metadata in the first pixel library to obtain the metadata difference information, and comparing the image in the standard format with at least one reference pixel set in the first pixel library to obtain the pixel difference information.

The application provides an image processing device which is characterized by comprising an acquisition module, a determination module and a comparison module, wherein the acquisition module is used for acquiring a first image to be processed, the determination module is used for determining a first pixel library from a plurality of pixel libraries, the pixels in the first pixel library and the pixels in the first image meet similar conditions, the pixels in different pixel libraries do not meet the similar conditions, the comparison module is used for comparing the first image with at least one reference pixel set in the first pixel library to obtain difference information, the difference information is used for describing pixel differences between the first image and the at least one reference pixel set, the encoding module is used for encoding the difference information to obtain an encoding result corresponding to the first image, and the encoding result is used for recovering the first image with the at least one reference pixel set.

In one possible implementation, the reference pixel set includes pixels in a reference pixel block, the difference information is a residual error, the comparison module is used for dividing the first image to obtain a plurality of original pixel blocks, determining a reference pixel block meeting a matching condition with each of the plurality of original pixel blocks from the first pixel library, wherein the matching condition includes that the similarity between pixels in two pixel blocks is greater than a threshold value, and calculating the residual error between the pixels in the original pixel block and the pixels in the reference pixel block meeting the matching condition.

In one possible implementation, each reference pixel block in the first pixel library is configured with a priority, the priority is used for indicating whether the reference pixel block is used for judging whether the reference pixel block meets the priority degree of a matching condition with the original pixel block, and the determining module is used for determining the reference pixel block meeting the matching condition with each original pixel block in the plurality of original pixel blocks from the first pixel library according to the priority of each reference pixel block.

In one possible implementation, the determining module is configured to extract a pixel feature of the first image, and determine a first pixel library from the plurality of pixel libraries according to the pixel feature of the first image, where a similarity between a reference pixel feature of the first pixel library and a pixel feature of the first image is the largest.

In one possible implementation, the encoding module is configured to extract a data feature of the difference information, determine at least one encoding algorithm matching the data feature from a plurality of encoding algorithms according to the data feature, and encode the difference information using the at least one encoding algorithm.

In one possible implementation manner, the determining module is configured to perform format conversion on the first image to obtain a first image in a standard format, and determine the first pixel library from the plurality of pixel libraries for the first image in the standard format.

In one possible implementation, the difference information includes metadata difference information and pixel difference information, the first pixel library further includes reference metadata, and the comparison module is configured to compare the metadata of the first image in the standard format with the reference metadata in the first pixel library to obtain the metadata difference information, and compare the first image in the standard format with at least one reference pixel set in the first pixel library to obtain the pixel difference information.

The application further provides an audio processing method which is characterized by comprising the steps of obtaining audio to be processed, dividing the audio to obtain a plurality of original audio segments, comparing each audio segment in the plurality of original audio segments with at least one reference audio segment in an audio library to obtain difference information corresponding to the plurality of original audio segments, wherein at least one reference audio segment in the audio library comprises part of original audio segments in the plurality of original audio segments, the difference information is used for describing audio differences between the audio relative to the at least one reference audio segment, encoding the difference information to obtain an encoding result corresponding to the audio, and the encoding result is used for recovering the audio with the at least one reference audio segment.

Since the reference audio segments in the audio library comprise part of the original audio segments in the audio, there is a certain similarity between the original audio segments in the audio and the reference audio segments in the audio library, which makes the difference information itself obtained by comparing the original audio segments with the reference audio segments, usually also have a certain similarity and regularity, based on this similarity. Thus, after the difference information with regularity and similarity is encoded, the data size of the obtained encoding result is effectively smaller than the number of the difference information and is also effectively smaller than the data size of the audio. Therefore, when the audio image is stored later, only the encoding result with smaller data quantity is needed to be stored, so that the storage space occupied by the audio data storage can be reduced. Meanwhile, the audio can be recovered by the coding result and the audio library, so that the coding result with smaller data size is stored later, namely the audio is stored.

The application further provides an audio processing device, which is characterized by comprising an acquisition module, a dividing module and a comparison module, wherein the acquisition module is used for acquiring audio to be processed, the dividing module is used for dividing the audio to obtain a plurality of original audio segments, the comparison module is used for comparing each audio segment in the plurality of original audio segments with at least one reference audio segment in an audio library to obtain difference information corresponding to the plurality of original audio segments, the at least one reference audio segment in the audio library comprises part of original audio segments in the plurality of original audio segments, the difference information is used for describing audio differences between the audio relative to the at least one reference audio segment, and the encoding module is used for encoding the difference information to obtain an encoding result corresponding to the audio and the encoding result is used for recovering the audio with the at least one reference audio segment.

In a seventh aspect, the present application provides a computing device comprising a processor, a memory. The processor and the memory communicate with each other. The processor is configured to execute instructions stored in the memory, to cause the computing device to perform the operation steps of the image processing method described in the above first aspect or any implementation manner of the first aspect, or to cause the computing device to perform the operation steps of the image processing method described in the above third aspect or any implementation manner of the third aspect. It should be noted that the memory may be integrated into the processor or may be independent of the processor. The computing device may also include a bus. The processor is connected with the memory through a bus. The memory may include a readable memory and a random access memory, among others.

In an eighth aspect, the present application provides a storage system, the storage system including a processor, at least one storage medium, the processor being configured to perform the operation steps of the image processing method described in any implementation manner of the first aspect or the first aspect, or to perform the operation steps of the image processing method described in any implementation manner of the third aspect or the third aspect, and the at least one storage medium being configured to persistently store an image encoding result.

In a ninth aspect, the present application provides a computer readable storage medium having instructions stored therein, which when run on a computing device, cause the computing device to perform or be operative to perform the steps of the image processing method described in any one of the implementations of the first aspect or the third aspect.

In a tenth aspect, the present application provides a computer program product comprising instructions which, when run on a computing device, cause the computing device to perform or be operative in performing the steps of the image processing method as described in any of the implementations of the first aspect or the third aspect.

Further combinations of the present application may be made to provide further implementations based on the implementations provided in the above aspects.

Drawings

FIG. 1 is a schematic diagram of an exemplary storage system;

FIG. 2 is a schematic flow chart of an image processing method according to the present application;

FIG. 3 is a diagram showing the unified conversion of various formats of images by the processor 100 into standard format images;

FIG. 4 is a schematic diagram of a portion of pixel regions in different images;

fig. 5 is a flowchart illustrating the process of reducing the data amount of the pixel data by the processor 100;

FIG. 6 is a schematic diagram of a difference sequence obtained by comparing an original pixel block and a reference pixel block;

FIG. 7 is a flow chart of the processor 100 for reducing the data amount of metadata;

FIG. 8 is a flowchart of another image processing method according to the present application;

FIG. 9 is a schematic diagram of another exemplary storage system;

Fig. 10 is a schematic flow chart of an audio processing method according to the present application;

FIG. 11 is a schematic diagram of an image processing apparatus according to the present application;

FIG. 12 is a schematic view of another image processing apparatus according to the present application;

Fig. 13 is a schematic structural diagram of an audio processing device according to the present application;

fig. 14 is a schematic hardware structure of a computing device according to the present application.

Detailed Description

In order that the above objects, features and advantages of the present application will be more readily understood, a more particular description of various non-limiting embodiments of the application will be rendered by reference to the appended drawings. It will be apparent that the described embodiments are some, but not all, embodiments of the application. All other embodiments obtained based on the above are within the scope of protection of the present application based on the embodiments of the present application.

Referring to FIG. 1, a schematic diagram of an exemplary storage system is shown. As shown in fig. 1, the storage system 10 includes a processor 100 and at least one storage medium, and in fig. 1, N storage media are illustrated as an example, and are respectively a storage medium 1 to a storage medium N. Each storage medium may be, for example, a Solid State Disk (SSD) STATE DRIVE, a mechanical hard disk (HARD DISK DRIVE, HDD), a magnetic disk, or the like, and can be used to persistently store data. Also, the storage system 10 may include different types of storage media.

Illustratively, the processor 100 may be a general-purpose processor, such as a central processing unit (central processing unit, CPU), or the like, or the processor 100 may be a controller that may be used to encode an image and write it to a storage medium, or the processor 100 may be an accelerator card, such as a graphics processor (graphics processing unit, GPU), or a tensor processor (tensor processing unit, TPU), or the like, or the processor 100 may be any one of or any combination of an application-specific integrated circuit (ASIC), a programmable logic device (programmable logic device, PLD), a complex program logic device (complex programmable logical device, CPLD), a field-programmable gate array (field-programmable GATE ARRAY, FPGA), a general-purpose array logic (GENERIC ARRAY logic, GAL), a system-on-chip (SoC), a software defined architecture (software-defined infrastructure, SDI) chip, an artificial intelligence (ARTIFICIAL INTELLIGENCE, dpai) chip, a data processing unit (Data processing unit, dpai), or the like.

In an actual application scenario, the processor 100 may be configured with a memory 301, as shown in fig. 1, so that the processor 100 may use the memory 301 to store data and perform corresponding processing operations based on the data in the memory 301. Or storage system 10 may be configured with a separate memory 302 as shown in fig. 1, such that data required for processor 100 to perform data processing operations or generated data may be stored by memory 302.

Also, the storage system 10 may be in communication with the application 200, such as receiving images provided by the application 200, feeding back notification messages to the application 200 (e.g., notifying the application 200 that the image was stored successfully, etc.). The application 200 may be a client on the user side, or may be a process running on a server, or the like, for example. During operation, the application 200 may send a plurality of images to the storage system 10 so that the storage system 10 persists the images in a corresponding storage medium.

The processor 100 may receive images sent by the application 200. Typically, if the processor 100 directly stores the image to the storage medium, this causes the image to occupy a larger storage space on the storage medium due to the larger amount of data of the image. Even if the processor 100 encodes multiple images one by one, the amount of data for each encoded image is still large because there may be a large difference between the different pixels in each image, and thus the storage system 10 still needs to consume a large amount of storage resources to store the multiple images.

Based on this, in the storage system 10 shown in fig. 1, the processor 100 may encode a plurality of images according to the similarity between the images to reduce the data amount of the encoded plurality of images as much as possible.

In particular, after receiving a plurality of images sent by the application 200, the processor 100 may group the plurality of images according to pixel characteristics of the plurality of images, specifically, group similar images into the same group, so as to obtain a plurality of groups. Wherein the degree of similarity between different images within the same group is higher than the degree of similarity between images of different groups. For example, assuming that the plurality of images transmitted by the application include image A, B, C, D, processor 100 may group similar image a and image B into group 1 and similar image C and image D into group 2, the degree of similarity between image a and image B within the same group may be higher than the degree of similarity between image a and image C within group 2 within group 1.

Then, for one of the packets, hereinafter referred to as the first packet, the processor 100 encodes the image in the first packet by using the pixel library corresponding to the first packet, to obtain the image encoding result corresponding to the first packet. The pixel library includes pixels in the partial image in the first group, that is, the pixel library may be created by using pixels in the partial image in the first group in the process of encoding the image in the first group by the processor, and the created pixel library may be cached in the memory 301 or the memory 302. For example, for group 1 described above, processor 100 creates a pixel library or the like corresponding to group 1 using some or all of the pixels in image a. Similarly, for each of the remaining packets, the processor 100 may encode the images within each of the remaining packets in a similar manner, resulting in respective corresponding image encoding results for each of the packets.

Finally, the processor 100 may send the image encoding results corresponding to each packet to one or more storage media for persistent storage. Further, the method comprises the steps of. Processor 100 may also send the pixel library corresponding to each packet to the storage medium for persistent storage.

Accordingly, when the subsequent application 200 requests the plurality of images from the storage system 10, the processor 100 may restore each image in the first packet according to the pixel library corresponding to the first packet and the image encoding result corresponding to the first packet. Referring to a similar manner, processor 100 may recover images within each group so that processor 100 may feed back the recovered images to application 200 to satisfy the query requirements of application 200 for the images. In this way, the storage system 10 stores the image encoding results corresponding to the respective packets, that is, stores the plurality of images transmitted by the application 200.

Since the similarity between the multiple images in the same group is high, the processor 100 encodes the multiple images in the group based on the pixels in the partial images in the group, so that the similarity between different images can be fully utilized to improve the encoding effect on the multiple images in the group. That is, the processor 100 can sufficiently compress the data amount of the images within each packet using the similarity between the images so that the number of image data after encoding is much smaller than the number of image data before encoding. Therefore, in the subsequent process of storing a plurality of images, only the encoding result with smaller data quantity is needed to be stored, so that the storage space occupied by storing a plurality of images can be reduced, namely the reduction rate of the images is improved. Wherein the reduction rate is the ratio between the reduced data amount relative to the data amount before reduction.

For example, assuming that one image a in the first group includes 172800 pixels (resolution 480×360) and the value of each pixel is expressed by 1 byte (B) (e.g., the image may be a gray-scale image), the amount of pixel data in the entire image a is 172800B. Further, assuming that there is a high similarity between the image B and the image a in the pixel library, specifically, the values of 170000 consecutive pixels in the image a may be the same as the values of the pixels in the image B, and the values of the remaining 2800 consecutive pixels may be different from the values of the pixels in the image B. Then, the processor 100 may encode the image a by using the image B in the pixel library, specifically, may calculate a difference between the value of the pixel in the image a and the value of the pixel in the image B, to obtain a corresponding difference sequence, where the first 170000 values in the difference sequence are all 0, and the remaining 2800 differences may be any value between-255 and 255. The processor 100 may then encode the sequence of differences into "(0,170000)," a.i. "(i.e. the image encoding result corresponding to image a). Thus, the processor 100 may encode the value of the first 170000 pixels in the image a into "(0,170000)", and assuming that "(0,170000)" occupies 10 bytes (each value occupies 4 bytes), the processor 100 may save at least the number of 169990 bytes after encoding the image a, so that at least the storage space of 169990B may be saved, and the reduction rate of the image a may reach 98.37% (i.e., 169990/172800). Similarly, for the plurality of images in the first group, the encoding can be performed in the above-described manner, whereby the entire data amount of the plurality of images in the first group can be reduced.

Further, when the processor 100 encodes the image in each group by using the lossless encoding algorithm, the processor 100 can restore the complete image data in the group according to the encoding result corresponding to each group and the pixel library corresponding to the group, which can reduce the storage space occupied by the image storage, and can realize lossless storage of the image, i.e. can retain the complete image information.

In practice, the N storage media in the storage system 10 shown in FIG. 1 may be deployed centrally or may be deployed in a distributed manner.

Where centralized deployment is employed, the storage system 10 may be deployed in a storage server configured with N storage media, e.g., multiple SSDs (one storage medium as shown in fig. 1) may be inserted on the storage server. The processor 100 in the storage system 10 may store the encoded result obtained by encoding a plurality of images in a storage medium configured with the processor.

When distributed deployment is employed, the N storage media in storage system 10 may be distributed among a plurality of different storage servers, each of which may be configured with one or more storage media, e.g., multiple SSDs (one storage media as shown in FIG. 1) may be inserted on each storage server. Also, each storage server may be configured with a processor 100 to encode and store a plurality of images received by the storage server. Or the processor 100 may be deployed independently of multiple storage servers, so that the processor 100 may distribute the encoding results to different storage servers after encoding multiple images. Or the processor 100 may be deployed to one of the storage servers, and after encoding a plurality of images received by the storage server, the generated encoding results are stored in a distributed manner to a plurality of different storage servers. In a distributed storage system, the different storage servers may communicate based on a communication protocol, which may be, for example, a computing quick link (compute express link, CXL) protocol, a unified bus (unified bus, UB, or Ubus) protocol, or a remote direct data access (remote direct memory access, RDMA) protocol, or may be other types of communication protocols.

It should be noted that the storage system 10 shown in fig. 1 is described above as an exemplary embodiment only and is not intended to be limiting. For example, a general purpose processor or a heterogeneous processor, among other possible storage systems, may also be included, or other hardware may also be included. Or in other possible storage systems, the processor 100 may be disposed integrally with a storage medium, where the storage system 10 may specifically be a storage device capable of storing received image data after encoding, where the processor 100 may be a controller in the storage device.

For ease of understanding, embodiments of the image processing method provided by the present application are described below with reference to the accompanying drawings.

Referring to fig. 2, fig. 2 is a flowchart of an image processing method according to an embodiment of the present application, and the method may be applied to the storage system 10 shown in fig. 1, or may be applied to other applicable storage systems. For convenience of description, the present embodiment is applied to the storage system 10 shown in fig. 1 as an example for illustration.

The image processing method shown in fig. 2 may specifically include the following steps.

S201, the application 200 sends the plurality of images to be processed to the storage system 10.

The image transmitted by the application 200 may be an encoded image in any format. The image sent by the application 200 may be, for example, but not limited to, a joint photographic experts group (joint photographic experts group, JPEG) format, a portable network graphics (portable network graphics, PNG) format, a network picture (web picture, webP) format, a scalable vector graphics (scalable vector graphics, SVG) format, a Bitmap (BMP) format, an original image file (RAW IMAGE FILE, RAW) format, a run-length coding (run length coding, RLE) format, or a label image file format (TAGGED IMAGE FILE format, TIFF) format.

In one possible implementation, the application 200 may generate multiple images during runtime, such as generating multiple rendered images based on a rendering indication of a user, and so on. The application 200 may then send the image 1 to the storage system 10. For example, when the application 200 is specifically a client on the user side, the client may generate a data storage request according to a storage operation instruction of the user for the generated plurality of images, and transmit the data storage request to the storage system 10, the data storage request including the plurality of images. Accordingly, the storage system 10 may parse the plurality of images specified to be saved by the user from the received data storage request. In practice, the storage system 10 may also acquire a plurality of images generated by the application 200 in other manners, which is not limited thereto.

In practice, the multiple images sent by the application 200 to the storage system 10 may include images in multiple formats, and there are often large differences in coding algorithms, image data organization, and the like used for the images in different formats. Thus, the processor 100 may perform format conversion on the received plurality of images to obtain an image in a standard format, that is, converting the plurality of images transmitted by the application 200 into an image in a uniform format, as shown in fig. 2. Thus, the subsequent processor 100 can perform subsequent processing operations on the image in the standard format. For example, as shown in fig. 3, the processor 100 may convert a plurality of images in a format such as JPEG, PNG, webP, SVG into a unified image in a universal picture format (huawei universal image format, HUIF), and the HUIF format is a standard format. Or the standard format may be in other types of formats, not limited in this regard.

In particular, the processor 100 may decode each received image separately to obtain complete pixel data in each image, where the pixel data may be, for example, a value of each pixel point in the image, and metadata of the image, where the metadata may be, for example, data describing a format of the image, an encoding algorithm used, a color type of the image (such as a gray scale image or a color image), and so on. The pixel data obtained by decoding can be image data in a standard format. Or after the pixel data corresponding to each image is obtained by decoding, the processor 100 may encode the pixel data of each image by using an encoding algorithm corresponding to the identification format, so as to obtain an image in the standard format. Further, when the processor 100 encodes the pixel data of each image based on the encoding algorithm corresponding to the standard format, the image may be further divided into a plurality of pixel blocks, so that the processor 100 may learn the division information of the plurality of pixel blocks in each image. The size of each pixel block may be, for example, 8 (pixels) ×8 (pixels), 16×16, or 32×32, or the like, as an example.

Further, the processor 100 may also extract metadata of each image from the image file of the image during format conversion of the plurality of images. Because metadata in images with different formats have differences in terms of data organization, description and the like, the processor 100 can also convert metadata corresponding to each image into metadata with a uniform format, so as to facilitate subsequent processing and management of metadata. For example, as shown in fig. 3, the processor 100 may convert metadata corresponding to a plurality of images in a format of JPEG, PNG, webP, SVG or the like into mixed metadata (hybrid meta) in a uniform format.

In this way, the image file obtained after the format conversion may include pixel data and mixed metadata (and pixel block division information). In this way, it is possible to convert a plurality of images of different formats into an image of a uniform format. It should be noted that, in practical application, the application 200 may also send multiple images in the same format to the storage system 10, for example, send multiple images in a standard format, where the processor 100 may not need to perform a format conversion operation on the received multiple images. For convenience of explanation, the following does not distinguish whether the plurality of images processed by the processor 100 from the step S202 are the plurality of images obtained after the format conversion or the plurality of images received by the processor 100.

In this embodiment, after the storage system 10 acquires a plurality of images (or after converting to a plurality of images in a unified format), the processor 100 may perform data reduction on the plurality of images to increase the reduction rate of the plurality of images, so as to reduce the storage space required for storing the plurality of images. Next, a process of processing a plurality of images will be described in detail with reference to fig. 2.

S202, the processor 100 extracts pixel characteristics of each image.

The pixel characteristics of the image may be, for example, at least one of brightness characteristics and color characteristics of pixels in the image, or may be other types of characteristics, such as pixel characteristics generated by weighting brightness characteristics and color characteristics, and the like, which are not limited thereto. Various non-limiting implementations of extracting luminance features and color features from an image are described below.

As some examples, when the extracted pixel features are specifically luminance features of pixels, the processor 100 may extract the luminance features from the image by various implementations.

In one implementation, the image may be a color image, where the values of the pixels in the image include values of three channels of red (red, R), green (G), and blue (B). The processor 100 may calculate the brightness value of each pixel according to the pixel value of each pixel in the image, for example, the value obtained by weighted averaging the values of three channels of each pixel at R, G, B is used as the brightness value of the pixel. The processor 100 may then extract the brightness characteristic of the image according to the brightness value of each pixel in the image, for example, an average value, a median value, a maximum value, or a minimum value of the brightness values of all the pixels included in the image may be used as the brightness characteristic of the image, or may extract the brightness characteristic of the image according to the brightness value of each pixel in the image in other manners.

When the image is a gray image, the processor 100 may use the gray value of each pixel point as the brightness value of the pixel point, and extract the brightness feature of the image according to the brightness value of each pixel point in the image.

In a second implementation manner, the processor 100 may calculate, according to the pixels in the image, the optical flow directions (or gradient descent directions) of the pixels in the image by using an optical flow estimation algorithm (or a gradient calculation method), so as to use the optical flow directions (or gradient descent directions) of the pixels in the image as the brightness features of the image.

In a third implementation manner, the processor 100 may perform edge detection on the image through an edge detection operator according to the brightness value of each pixel point in the image, and use the detected boundary information as the brightness feature of the image. The edge detection operator may be, for example, a Roberts Cross operator, a Prewitt operator, a Sobel operator, a Kirsch operator, a Canny operator, or a compass operator, or may be other operators that can be used to detect boundaries in an image.

In a fourth implementation, the processor 100 may input the image into an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) model that has been trained in advance, and evaluate the brightness of the image using the AI model, so that the processor 100 may use the reasoning result (e.g., the evaluated brightness value, etc.) output by the AI model as the brightness feature of the image. The AI model may be a model constructed based on a deep learning algorithm or other algorithms, for example.

In practice, the processor 100 may extract the brightness features from the image by other methods, which are not limited thereto.

As some examples, when the extracted pixel features are specifically color features of a pixel, the processor 100 may extract the color features from the image by various implementations.

In one implementation, the image is a color image, i.e., each pixel in the image may include R, G, B three-channel values. Then, the processor 100 may calculate an average value R _average of the R channel values of all the pixels in the image, an average value G _average of the G channel values of all the pixels in the image, and an average value B _average of the B channel values of all the pixels in the image, and use the calculated R _average、G_average、B_average as a color feature of the image. Alternatively, the processor 100 may perform corresponding weighting calculation on the values of the pixels in the image, and then use the weighted calculation result as the color feature of the image.

In a second implementation, the processor 100 may input an image into an AI model that has been trained in advance, and output color information of the image using the AI model, where the color information may be, for example, color saturation of the image, and the color information is used as a color feature of the image. The AI model may be a model constructed based on a deep learning algorithm or other algorithms, for example.

In practice, the processor 100 may extract the color features from the image by other methods, which are not limited thereto.

In the above manner, the processor 100 may extract a color feature or a luminance feature of each image, and use the color feature or the luminance feature as a pixel feature of the image, or use a feature obtained by performing corresponding calculation based on the color feature and the luminance feature as a pixel feature of the image, or the like.

And S203, the processor 100 groups the images according to the pixel characteristics of the images to obtain a plurality of groups, wherein the similarity degree between the images in the same group is higher than that between the images in different groups.

In an actual application scene, certain similarity may exist between different images. For example, assume that the plurality of images transmitted by the application 200 to the storage system 10 may be specifically 4 images as shown in fig. 4, in which pixels in a broken line are similar between the image a and the image B, and pixels in a background area (an area other than a person) are similar between the image C and the image D. Thus, the processor 100 may group the images according to the similarity between the images, such as dividing image a and image B into a group and dividing image C and image D into a group.

In particular, the processor 100 may calculate the similarity between the pixel features of different images according to the pixel features of each image, and divide the images with the similarity between the pixel features greater than the threshold value into the same group, so as to obtain multiple groups, which are respectively a group 1 to a group N, where N is a positive integer greater than 1. Illustratively, the processor 100 may group the multiple images by a K-means algorithm or AI model, grouping the multiple images according to their pixel characteristics, or the like.

Typically, multiple images may be included in each packet. When there is an image that is dissimilar to other images, the processor 100 may divide the image into a single group. In addition, when the pixel characteristics of the image include the color characteristics and the brightness characteristics, or are determined based on the color characteristics and the brightness characteristics, the processor 100 determines the similarity between different images according to the color information and the illumination information of the image, so that the accuracy of judging the similarity between different images can be improved, and the misjudgment can be reduced.

After dividing into a plurality of packets, the processor 100 may encode the image within the packet for each packet to reduce the data amount of the image within the packet. For convenience of explanation, description will be made below taking encoding for an image in one of the packets, which is packet 1 among the plurality of packets.

S204, the processor 100 encodes the image in the group 1 by using the pixel library 1 corresponding to the group 1 to obtain an image encoding result corresponding to the group 1, wherein the pixel library 1 comprises pixels in a part of the images in the group 1.

In encoding the multiple images within group 1, processor 100 may create a pixel library 1 for group 1 for subsequent compression of the multiple images in group 1 with pixels in pixel library 1. Moreover, the processor 100 may compare the pixels in the image in the group 1 with the pixels in the pixel library 1 to obtain pixel difference information, where the pixel difference information is used to describe the difference between the pixels in the image in the group 1 and the pixels in the pixel library 1, so that the processor 100 may encode the pixel difference information to obtain an image encoding result corresponding to the group 1, that is, an encoding result for multiple images in the group 1.

In the present embodiment, the following several implementation examples of generating the image encoding result corresponding to the packet 1 are provided.

As a first implementation example, the processor 100 may add all pixels in the partial images in the group 1 to the pixel library 1, so that the processor 100 may calculate differences between each image in the group 1 and the reference image in the pixel library 1 with respect to the pixels using the pixels of the partial images as references (hereinafter referred to as reference images), resulting in pixel difference information corresponding to each image in the group 1, which is used to describe differences between the pixels in the image and the pixels in the reference image in the pixel library 1, for example, the pixel difference information may be a residual (difference sequence) between the values of all pixels in the image 1 and the values of all pixels in one image in the pixel library 1. Then, the processor 100 may encode the pixel difference information corresponding to each image to obtain an encoding result corresponding to the image, so that the encoding results corresponding to the plurality of images respectively are the image encoding results corresponding to the group 1.

For example, assuming that the group 1 includes the image 1, the image 2 and the image 3, the processor 100 may add all pixels in the image 1 to the pixel library 1, and at this time, the reference image in the pixel library 1 is the image 1, so that the processor 100 may calculate pixel differences between the image 1, the image 2 and the image 3 relative to the reference pixels in the pixel library 1 respectively, to obtain pixel difference information corresponding to the image 1, the image 2 and the image 3 respectively. Then, the processor 100 encodes the pixel difference information of the image 1, the image 2, and the image 3, respectively, to obtain the image encoding result corresponding to the group 1. The implementation of selecting the image to be added to the pixel library 1 by the processor 100 is not limited, and may be, for example, a part of the images may be selected randomly from the plurality of images to be added to the pixel library 1, or the images to be processed may be added to the pixel library 1 according to the image processing sequence, etc., which is not limited.

In practical application, since the pixel library 1 may include multiple reference images, the processor 100 may record the mapping relationship between the encoding results of the images in the group 1 and the reference images in the pixel library 1 in the process of generating the image encoding results corresponding to the group 1, so as to select the correct reference image and the encoding result to recover the images in the group 1 according to the mapping relationship.

As a second implementation example, the processor 100 encodes at the granularity of pixel blocks. Specifically, the processor 100 may sequentially process each image in the group 1, and assuming that the currently processed image is the image 1, the processor 100 may divide the image 1 into a plurality of pixel blocks, and as shown in fig. 5, the size of each pixel block may be 8 (pixels) ×8 (pixels), 16×16, or 32×32, or may be other sizes smaller than the resolution of the image 1. For ease of distinction, the pixel block in image 1 will be referred to as the original pixel block hereinafter. Processor 100 may traverse the plurality of original pixel blocks in image 1. For the first original pixel block traversed by the processor 100, since no pixels are included in the pixel bank 1, the processor 100 may directly add pixels in the first original pixel block to the pixel bank 1. For ease of distinction, the original pixel block added to the pixel library 1 is referred to as a reference pixel block.

Then, the processor 100 traverses to the second original pixel block, and determines whether there is a reference pixel block in the pixel library 1 that satisfies a matching condition, specifically, a similarity of pixels between two pixel blocks is greater than a threshold.

Specifically, the processor 100 may calculate the similarity between the pixels in the second original pixel block and the pixels in the reference pixel block in the pixel library 1. For example, the processor 100 may respectively calculate the pixels in the second pixel block and the reference pixel block by using discrete cosine transform (discrete cosine transform, DCT) to obtain a calculation result corresponding to the second pixel block and a calculation result corresponding to the reference pixel block, and the processor 100 calculates a similarity between the two calculation results, where the similarity is a similarity between a value of each pixel in the second pixel block and a value of each pixel in the reference pixel block.

When the similarity is greater than the threshold value, the processor 100 may calculate pixel difference information according to the second original pixel block and the reference pixel block, where the pixel difference information can describe a deviation between a value of a pixel point in the second original pixel block and a value of a pixel point in the reference pixel block, as shown in fig. 5. For example, the processor 100 may calculate a residual between the value of the pixel in the second original pixel block and the value of the pixel in the reference pixel block to obtain a difference sequence, and determine the difference sequence as pixel difference information corresponding to the second original pixel block.

For example, assume that image 1 is a gray scale image, and the second original pixel block and the reference pixel block are each 8×8 pixel blocks, wherein the values of 64 pixel points in the second original pixel block and the reference pixel block are shown in fig. 6. Then, the processor 100 may calculate a residual between the value of the pixel point in the second original pixel block and the value of the pixel point in the reference pixel block, and may obtain a difference sequence as shown in fig. 6.

In general, if the degree of similarity between a pixel in an original pixel block and a pixel in a reference pixel block is higher, pixel difference information for describing a difference between the pixels in the two pixel blocks is more regular. For example, when the pixel difference information is a difference sequence, the difference sequence includes more identical or repeated differences, such as the difference sequence in fig. 6 has more values of 0 and 5. At this time, the smaller the data amount of the encoding result obtained by encoding the pixel difference information, that is, the better the effect of reducing the memory space required for image storage.

And when the similarity is less than the threshold, the similarity between the second original pixel block and the reference pixel block is characterized as being lower, the processor 100 may add the second original pixel block as the reference pixel block to the pixel library 1, as shown in fig. 6. At this time, 2 reference pixel blocks are included in the pixel bank 1.

Next, the processor 100 traverses to the third original pixel block, and calculates the similarity between the pixels from the third original pixel block and each reference pixel block in the pixel library 1. And, when there is a similarity between pixels of a reference pixel block and the third original pixel block greater than the threshold, the processor 100 may calculate pixel difference information corresponding to the third original pixel block using the reference pixel block. If the similarity between the pixels of the third original pixel block and the reference pixel block is greater than the threshold value, the processor 100 may add the third original pixel block as a new reference pixel block to the pixel library 1, where the pixel library 1 includes 3 reference pixel blocks.

By analogy, for each original pixel block in image 1, processor 100 may calculate pixel difference information corresponding to each original pixel block in the manner described above.

For the original pixel block added to the pixel library 1, the pixel difference information may describe that the difference between the pixel in the original pixel block and the reference pixel block in the pixel library 1 is 0 (e.g., the residual is 0), that is, the original pixel block is the same as one reference pixel block in the pixel library 1, so that the processor 100 may process each original pixel block in the image 1 based on unified logic in the above manner. Thus, after calculating the pixel difference information corresponding to each original pixel block in the image 1 based on the above manner, the processor 100 can obtain the pixel difference information of the whole image 1 with respect to the pixel library 1. Or the processor 100 may not need to encode the original pixel blocks added to the pixel library 1, and the image encoding result corresponding to the first packet finally generated by the processor 100 is the encoding result for a plurality of original pixel blocks not added to the pixel library 1, so that the processor 100 may store the image encoding result together with the pixel library 1 in a storage medium. Or for an original block of pixels added to the pixel library 1, the processor 100 may adjust or set the value of the original block of pixels (e.g., all set to 0, etc.) so that the amount of data of the subsequent encoding results for the original block of pixels is as minimal as possible. In the present embodiment, whether or not the original pixel block added to the pixel library 1 is encoded and the specific encoding method are not limited.

Similarly, for each image within group 1, processor 100 may calculate pixel difference information corresponding to each image in the manner described above. Accordingly, in calculating the pixel difference information, the processor 100 completes the construction for the pixel library 1.

Finally, the processor 100 may encode the pixel difference information corresponding to each original pixel block, so that after encoding the pixel difference information corresponding to all original pixel blocks in all images in the group 1, an image encoding result corresponding to the group 1 can be obtained, as shown in fig. 5. Since the similarity between the original pixel block in each image and the pixels of the reference pixel block in the pixel library 1 is high, the pixel difference information for describing the difference of pixels between the original pixel block and the reference pixel block is more regular. At this time, the smaller the data amount of the encoding result obtained by the processor 100 for the difference information, that is, the better the effect of reducing the memory space required for storing the image.

For example, taking the pixel difference information corresponding to a certain original pixel block in the image 1 as an example, the difference sequence shown in fig. 6 is specifically taken as an example, the data amount of the difference sequence is 64B, and a plurality of consecutive values in the difference sequence are the same, for example, a plurality of consecutive values 0 and 5 are present. Processor 100 may then encode the sequence of differences using RLE algorithm to obtain an encoded result (0, 24), 1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4, (5, 24). At this time, the data amount of the encoding result is 24B, which is smaller than the data amount 64B of the original pixel block itself, so as to achieve the enhancement of the reduction rate of the original pixel block to 62.5% (i.e., (64-24)/64). Similarly, the reduction rate of image 1 can be effectively improved when the encoding is performed in a similar manner for the remaining pixel blocks in image 1.

In practical application, since the pixel library 1 may include a plurality of reference pixel blocks, the processor 100 may record a mapping relationship between the encoding result of each original pixel block and the reference pixel blocks in the pixel library 1 in the process of generating the image encoding result corresponding to the group 1, so as to select a correct reference pixel block and restore the original pixel block from the encoding result according to the mapping relationship.

In practical application, in the process of calculating pixel difference information corresponding to each of the plurality of images in the group 1, the number of reference pixel blocks included in the pixel block 1 is gradually increased, for example, more than 1000 reference pixel blocks may be included in the pixel library 1, so that more calculation resources are required to be consumed when determining a reference pixel block satisfying a matching condition for one original pixel block in the images.

For this reason, in a further possible implementation example, when the number of reference pixel blocks included in the pixel library 1 is greater than a preset value (e.g., 1000 or the like), each of the reference pixel blocks in the pixel library 1 may be configured with a priority indicating a degree of priority to which the reference pixel block is used to determine whether or not the matching condition is satisfied with the original pixel block. In this way, when determining a reference pixel block that satisfies the matching condition for each original pixel block, the processor 100 can preferentially calculate the similarity between the original pixel block and the reference pixel block with higher priority in the pixel library 1 according to the priority of each reference pixel block in the pixel library 1. And, when the similarity between the reference pixel block and the original pixel block is greater than the threshold, the processor 100 may calculate pixel difference information corresponding to the original pixel block using the reference pixel block, and continue to determine a reference pixel block with higher similarity for the next original pixel block. When the similarity between the reference pixel block and the original pixel block is smaller than the threshold, the processor 100 may continue to calculate the similarity between the original pixel block and the next reference pixel block in the pixel library 1 according to the priority until the reference pixel block satisfying the matching condition with the original pixel block is determined. In this way, the processor 100 can quickly traverse the reference pixel block that meets the matching condition with the original pixel block for each original pixel block, so that it is not necessary to traverse all the reference pixel blocks in the pixel library 1 for the original pixel block and perform calculation, so that the resource consumption required for determining the reference pixel block can be reduced, and the efficiency for determining the reference pixel block can be improved.

Further, the processor 100 may be configured with various encoding algorithms, and in the process of encoding the pixel difference information corresponding to the packet 1, the processor 100 may sequentially encode the pixel difference information using the various encoding algorithms to achieve a reduction rate as high as possible.

For example, for encoding the difference sequence shown in fig. 6, the processor 100 may first encode the difference sequence into (0, 24), 1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4, (5, 24) using RLE algorithm. Because there are consecutive "1,2,3,4" occurrences of repetition in the result encoded by the RLE algorithm, the processor 100 may also continue to encode the current encoding result using an LZ (Lempel-Ziv) encoding algorithm. The LZ coding algorithm is a dictionary-based coding algorithm, and it is assumed that the dictionary includes character strings "1,2,3,4", and that the character strings may be replaced with a. Then the processor 100 encodes (0, 24), a, (5, 24) using the LZ encoding algorithm. Finally, the processor 100 may also continue to encode the current encoding results into (0, 24), (a, 4), (5, 24) using the RLE algorithm. In this way, the data amount of the finally obtained encoding result can be reduced to 6B, so that the reduction rate of the original pixel block is increased to 90.6% (i.e., (64-6)/64).

In practical applications, after the processor 100 acquires the pixel difference information, a data feature may be extracted from the pixel difference information, where the data feature may be, for example, a feature of the same value that is continuously repeated, a feature of a value that is circularly repeated, and the like as shown in fig. 6. Since different coding algorithms are suitable for coding information having different data characteristics (i.e., the information can achieve a higher reduction rate after being coded), the processor 100 can determine one or more coding algorithms matching the data characteristics from a plurality of configured coding algorithms according to the data characteristics, and sequentially code the difference information using the determined one or more coding algorithms. The method comprises the steps of utilizing a plurality of coding algorithms to code difference information in sequence, namely utilizing the current coding algorithm to code the difference information to obtain a coding result, continuing to code the coding result by the next coding algorithm, and the like until a final coding result is obtained.

In a further possible embodiment, after encoding the difference information using the determined at least one encoding algorithm, the processor 100 may further extract data features of the encoding result, and determine whether to further encode the encoding result according to the data features, so as to further reduce the data amount of the encoding result.

The coding algorithm used for coding the pixel difference information may be a lossless coding algorithm, i.e., no information loss is generated after coding the pixel difference information, such as RLE algorithm. Or the coding algorithm adopted for coding the pixel difference information can also be a lossy coding algorithm, i.e. a small amount of information loss can be generated after the pixel difference information is coded, such as a predictive coding algorithm and the like.

The foregoing describes, taking the encoding of the image in the group 1 as an example, in practical application, for each of the multiple groups, the processor 100 may encode the image in the group in the manner described above to obtain the image encoding result corresponding to the group, so as to obtain the image encoding results respectively corresponding to the multiple groups, that is, obtain the image encoding results corresponding to the multiple images. For example, for group 2, processor 100 may create pixel library 2 from pixels in a portion of the images within the group, and encode the images within group 2 using a reference image or reference pixel block in pixel library 2 to obtain the image encoding results corresponding to group 2.

In addition, when the processor 100 converts a plurality of received images into an image of a standard format, each image also carries metadata. At this time, the processor 100 may also save metadata of each image.

Wherein the processor 100 may directly store the metadata of each image in the storage medium when recording the metadata of each image.

Alternatively, the processor 100 may store the encoding results corresponding to the metadata of the plurality of images in the storage medium, which will be referred to as metadata encoding results hereinafter, with reference to the above-described manner of storing the image encoding results corresponding to the plurality of images, thereby reducing the data amount of the metadata of the plurality of images and further reducing the data amount of the entire plurality of images.

In particular, the pixel library corresponding to each group may further include reference metadata, and as shown in fig. 7, the processor 100 may calculate a similarity between the metadata of the image in the group and each reference metadata in the pixel library corresponding to the group. And, when there is a similarity between the reference metadata and the metadata of the image in the pixel library that is greater than the threshold, the processor 100 may compare the differences between the metadata of the image and the reference metadata, and obtain metadata difference information describing the differences between the two metadata. The metadata difference information may be, for example, a residual between two metadata. And when there is no reference metadata having a similarity with the metadata of the image greater than the threshold value in the pixel library, the processor 100 may add the metadata of the image as the reference metadata to the pixel library, as shown in fig. 7, so as to reduce the data amount of the metadata of other images by using the newly added reference metadata later.

Then, the processor 100 may encode the metadata difference information corresponding to each image in each group to obtain an encoding result of the metadata of each image, so as to obtain metadata encoding results corresponding to each group (including encoding results corresponding to metadata of a plurality of images in the group).

Further, in encoding metadata, when there is a similarity between the reference metadata and the metadata of the image in the pixel library that is greater than a threshold, the processor 100 may calculate profit 1 for encoding metadata of the image based on a preset formula, and calculate profit 2 for encoding metadata difference information of the image. When benefit 1 is less than benefit 2, the processor 100 may encode the metadata difference information to reduce the storage space required for storing metadata later, and when benefit 1 is greater than or equal to benefit 2, the processor 100 may directly encode the metadata of the image so that the encoding result of the metadata of the image can be as few as possible to reduce the storage space required for storing metadata later as possible.

Further, the processor 100 may store the image encoding results (and metadata encoding results) corresponding to the respective packets for persistence after generating them. For this purpose, the embodiment shown in fig. 2 may further comprise the following step S205.

The processor 100 transmits the image encoding results corresponding to the plurality of images to the storage medium S205.

After generating the image encoding results corresponding to the respective packets, the processor 100 may output the image encoding results corresponding to the respective packets to one or more storage media for storage. Thus, the image coding results corresponding to the groups are stored, namely, the images are stored.

In practical applications, in the process of storing the image encoding result, the processor 100 may specifically store the image encoding result, the identifier of the group to which the image encoding result belongs, the identifier of the plurality of images included in each group, and the mapping relationship, where the mapping relationship may be, for example, a correspondence relationship between the encoding result of each image in each group and a reference image in a pixel library corresponding to the group, or may be a correspondence relationship between the encoding result of each original pixel block of each image in each group and a reference pixel block in a pixel library corresponding to the group.

Further, when the processor 100 performs format conversion on the received multiple images, the processor 100 may also send the metadata encoding results corresponding to each packet and the mapping relationship between the metadata encoding results and the reference metadata in the pixel library to the storage medium for persistent storage. In addition, the processor 100 may also send the pixel library corresponding to each packet to the storage medium for persistent storage.

Accordingly, when the application 200 requests multiple stored images from the storage system 10, the processor 100 may restore the multiple images according to the image encoding results corresponding to the multiple images and the pixel library, and feed back the multiple images to the application 200.

Next, the process of the storage system 10 feeding back a plurality of images to the application 200 will be described.

S206, the application 200 sends a data read request to the storage system 10 for requesting a plurality of images from the storage system 10.

In response to the data reading request, the processor 100 restores the plurality of images based on the image encoding results corresponding to the plurality of images stored in the storage medium S207.

In one possible implementation manner, after receiving the data reading request, the processor 100 may parse the identifiers of the multiple images to be read from the data reading request, and read, according to the identifiers of the multiple images, the image encoding results corresponding to the multiple images in the storage medium, the group identifier to which the image encoding results belong, the identifiers of the multiple images included in each group, and the mapping relationship. Then, the processor 100 may decode the image encoding result according to the group identifier to which the image encoding result belongs and the identifiers of the plurality of images included in each group, to obtain image difference information corresponding to the images in each group. Then, for each group, the processor 100 may access the pixel library corresponding to the group according to the image difference information and the mapping relation corresponding to the image in the group, so as to obtain the reference image/reference pixel block corresponding to the image in the group, so that the processor 100 may calculate, according to the image difference information and the reference image/reference pixel block, the value of the pixel point in each image in the group, so as to recover the complete pixel data in the multiple images in the group. Similarly, for each remaining packet, the processor 100 may also recover the complete pixel data for the multiple images within the packet in the manner described above.

Further, when the processor 100 further records metadata of the plurality of images in the process of storing the plurality of images, the processor 100 may further read the metadata encoding results corresponding to the plurality of images and the mapping relationship between the metadata encoding results and the reference metadata from the storage medium in the process of decoding the image encoding results, and decode the metadata decoding results by using the reference metadata in the pixel library according to the mapping relationship, so as to obtain metadata corresponding to each image. The processor 100 may decode the metadata encoding result according to the reference metadata in the pixel library, and the specific implementation manner thereof may be described in the above-mentioned related places for decoding the image encoding result, which is not described herein. In this way, after recovering the complete pixel data in the multiple images, the processor 100 may encode the pixel data of the recovered multiple images according to the metadata of the decoded multiple images, so as to recover the multiple images sent to the storage system 10 by the application 200, where the format of the recovered multiple images is consistent with the format of the multiple images sent to the storage system 10 for storage before the application 200.

It should be noted that, in this embodiment, the reference metadata and the reference image/reference metadata are recorded in the pixel library at the same time, and in practical application, the processor 100 may be configured with two independent databases, one database is used for recording the reference metadata and the other database is used for recording the reference image/reference metadata.

The processor 100 feeds back the restored plurality of images to the application 200S 208.

Illustratively, after recovering the plurality of images, the processor 100 may generate a response message including the plurality of images and send the response message to the application 200 to satisfy the query requirement of the application 200 for the plurality of images.

In this way, when the pixel library is configured in the storage system 10, when the storage system 10 stores the plurality of images 1, it is sufficient to store the image encoding results (and the metadata encoding results) corresponding to the plurality of images having a smaller data amount, and there is no need to store the data of the original plurality of images, which can effectively reduce the storage space required for storing the plurality of images, that is, increase the reduction rate of the plurality of images. Moreover, when the application 200 requests multiple images from the storage system 10, the storage system 10 can restore the complete multiple images according to the pixel library and the saved image coding result (and metadata coding result), so as to meet the query requirement of the application 200 on the multiple images.

In addition, when the encoding algorithm adopted by the processor 100 is a lossless encoding algorithm, the storage system 10 can restore a plurality of complete images according to the image encoding result and the pixel library, which can ensure the information integrity during image storage and realize lossless storage of the plurality of images.

Furthermore, when the processor 100 encodes the image in each group with the granularity of the pixel block, the processor 100 only needs to read the reference pixel blocks adopted in the pixel library corresponding to the group for recovering the original pixel blocks in the image in the process of recovering the image in each group, and does not need to read all the reference pixel blocks in the pixel library (and does not need to read the reference pixel blocks in the pixel libraries corresponding to other groups), so that the problem of reading and amplifying can be avoided.

In the embodiment shown in fig. 2, the application 200 transmits a plurality of images to the storage system 10 and stores the images therein. In other embodiments, the application 200 may also send an image to the storage system 10 for storage. The process by which the processor 100 encodes and stores a received image is described below.

Referring to fig. 8, a flowchart of another image processing method according to an embodiment of the present application is shown, and the method may be applied to the storage system 10 shown in fig. 1, or may be applied to other applicable storage systems. For convenience of description, the present embodiment is applied to the storage system 10 shown in fig. 1 as an example for illustration.

The image processing method shown in fig. 8 may specifically include the following steps.

S801 the application 200 sends the image 1 to be processed to the storage system 10.

The image 1 transmitted by the application 200 may be an encoded image in any format. The format of the image 1 may be, for example, a JPEG format, a PNG format, webP format, an SVG format, a BMP format, a RAW format, an RLE format, or a TIFF format, or may be other formats, which are not limited thereto.

In one possible implementation, the application 200 may generate the image 1 during operation and send the image 1 to the storage system 10. For example, when the application 200 is specifically a client on the user side, the client may generate a data storage request according to a storage operation instruction of the user for the image 1, and send the data storage request to the storage system 10, where the data storage request includes the image 1. Accordingly, the storage system 10 may parse the user-specified saved image 1 from the received data storage request. In practice, the storage system 10 may acquire the image 1 by other methods, which is not limited thereto.

In this embodiment, after the storage system 10 acquires the image 1, the processor 100 may perform data reduction on the image 1 to increase the reduction rate corresponding to the image 1 and reduce the storage space required for storing the image 1.

In particular, the processor 100 may compare the image 1 with the reference pixel set in the pixel library to obtain difference information for describing the pixel difference between the image 1 and the reference pixel set, so that the processor 100 may encode the difference information and send the obtained encoding result to the storage medium for storage. Subsequently, when the application 200 needs to read the stored image 1, the processor 100 may restore the image 1 according to the encoding result and the reference pixel set. In this way, the storage system 10 stores the encoding result corresponding to the image 1, that is, stores the image 1. Next, a detailed description will be given of a process of processing the image 1 with reference to fig. 8.

The processor 100 extracts pixel characteristics of the image 1S 802.

In practical applications, the image sent by the application 200 to the storage system 10 may be any format of image, and there are generally large differences in coding algorithms, image data organization manners, and the like used for different formats of images. Thus, the processor 100 may perform format conversion on the received image to obtain an image in a standard format, i.e., an image in an arbitrary format is converted into an image in a uniform format. Thus, the subsequent processor 100 can perform subsequent processing operations on the image in the standard format.

In particular, the processor 100 may decode the received image 1 to obtain the complete pixel data in the image 1 and the metadata of the image 1, where the pixel data may include, for example, the values of the pixels in the image 1. The decoded image data may be image data in a standard format. Or after decoding to obtain the pixel data of the image 1, the processor 100 may use a preset encoding algorithm to encode the pixel data to obtain an image in a standard format.

Processor 100 may then extract the corresponding pixel features from image 1 in the standard format.

The pixel characteristics of the image 1 may be, for example, at least one of brightness characteristics and color characteristics of pixels in the image 1, or may be other types of characteristics, such as pixel characteristics generated by weighting calculation of brightness characteristics and color characteristics, and the like, which are not limited.

As some examples, when the extracted pixel features are specifically luminance features of pixels, the processor 100 may extract the luminance features from the image 1 by various implementations as follows.

In one implementation, the image 1 may be a color image, where the values of the pixels in the image 1 include the values of three channels R (red), G (green), and B (blue). The processor 100 may calculate the brightness value of each pixel in the image 1 according to the pixel value of each pixel, for example, the value obtained by weighted averaging the values of three channels of each pixel at R, G, B is used as the brightness value of the pixel. Then, the processor 100 may extract the luminance characteristics of the image 1 according to the luminance values of the pixels in the image 1, for example, the average value, the median value, the maximum value or the minimum value of the luminance values of all the pixels included in the image 1 may be used as the luminance characteristics of the image 1, or may extract the luminance characteristics of the image 1 according to the luminance values of the pixels in the image 1 in other manners.

When the image 1 is a gray scale image, the processor 100 may use the gray scale value of each pixel point as the brightness value of the pixel point, and extract the brightness feature of the image 1 according to the brightness value of each pixel point in the image 1.

In the second implementation manner, the processor 100 may calculate, according to the pixels in the image 1, the optical flow directions (or gradient descent directions) of the plurality of pixels in the image 1 by using an optical flow estimation algorithm (or a gradient calculation method), so that the optical flow directions (or gradient descent directions) of the pixels in the image 1 are taken as the brightness characteristics of the image 1.

In a third implementation manner, the processor 100 may perform edge detection on the image 1 through an edge detection operator according to the brightness value of each pixel point in the image 1, and use the detected boundary information as the brightness feature of the image 1. The edge detection operator may be, for example, a Roberts Cross operator, a Prewitt operator, a Sobel operator, a Kirsch operator, a Canny operator, or a compass operator, or may be other operators that can be used to detect boundaries in an image.

In a fourth implementation, the processor 100 may input the image 1 into an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) model that has been trained in advance, and evaluate the brightness of the image 1 using the AI model, so that the processor 100 may use the reasoning result (e.g., the evaluated brightness value, etc.) output by the AI model as the brightness feature of the image 1. The AI model may be a model constructed based on a deep learning algorithm or other algorithms, for example.

In practice, the processor 100 may extract the brightness features from the image 1 by other methods, which are not limited thereto.

As some examples, when the extracted pixel features are specifically color features of pixels, the processor 100 may extract the color features from the image 1 by various implementations.

In one implementation, image 1 is a color image, i.e., each pixel in image 1 may include R, G, B of the three-channel values. Then, the image processing apparatus 100 may calculate an average value R _average of the R channel values of all the pixels in the image 1, an average value G _average of the G channel values of all the pixels in the image 1, and an average value B _average of the B channel values of all the pixels in the image 1, and use the calculated R _average、G_average、B_average as the color feature of the image 1. Alternatively, the image processing apparatus 100 may perform the corresponding weighting calculation on the values of the respective pixels in the image 1, and then use the result of the weighting calculation as the color feature of the image 1.

In implementation two, the processor 100 may input the image 1 into a pre-trained AI model, and output color information of the image 1 using the AI model, where the color information may be, for example, color saturation of the image 1, and the color information is used as a color feature of the image 1. The AI model may be a model constructed based on a deep learning algorithm or other algorithms, for example.

In practice, the processor 100 may extract the color features from the image 1 by other methods, which are not limited thereto.

S803 processor 100 determines a pixel pool 1 from a plurality of pixel pools according to the pixel characteristics of image 1, wherein the pixels in the pixel pool 1 and the pixels in the image 1 satisfy a similar condition, and the pixels in different pixel pools do not satisfy the similar condition.

In this embodiment, the processor 100 may be preconfigured with a plurality of pixel libraries, for example, the pixel libraries may be preconfigured in the processor 100 by a technician, or the processor 100 may obtain a plurality of images in advance before obtaining the image 1, and construct the pixel libraries according to pixels in the plurality of images, and so on. For example, processor 100 may add pixels in a partial image to one pixel library, with different pixel libraries including pixels in different images. At this time, each pixel bank may include pixels in at least one image of the plurality of images, and the similarity condition is not satisfied between pixels in different pixel banks, for example, the similarity between pixels in different pixel banks is smaller than a threshold value, or the like, that is, the pixels in different pixel banks are greatly different. And, when the pixel library includes a plurality of pixel sets, the pixels in different pixel sets included in the pixel library are similar, such as the similarity between the pixel features in different pixel sets is greater than a threshold value, etc.

The pixel sets may be all pixels in one image, and in this case, each pixel set in the pixel library may be specifically one image. Or the pixel sets can be part of pixels in one image, and each pixel set in the pixel library can be specifically one pixel block in one image at the moment, and a plurality of pixel blocks (as a plurality of pixel sets) in one image can be added into the pixel library in the process of constructing the pixel library.

In general, if the similarity between a pixel feature in an image and a pixel feature of a pixel set in a pixel library is higher, difference information describing a pixel difference between the image and the pixel set is more regular, for example, when the difference information is a difference sequence, the difference sequence includes more identical or repeated differences. At this time, the smaller the data amount of the encoding result obtained by encoding the difference information, that is, the better the effect of reducing the memory space required for image storage. Based on this, for the image 1 to be stored, the processor 100 may determine an appropriate pixel library for that image 1 in order to achieve as low a storage space as possible that is required when storing the image 1. For ease of understanding, the present embodiment refers to a set of pixels in the pixel library 1 having a high degree of similarity between the pixel characteristics and the pixel characteristics of the image 1 as a reference pixel set.

In one possible implementation, each pixel library in processor 100 may be configured with a reference pixel feature, which may be, for example, an average of pixel features of individual sets of pixels in the pixel library, or the like.

The processor 100 may then perform similarity calculation on the pixel characteristics of the image 1 and reference pixel characteristics of each pixel library in turn, and determine the pixel library having the reference pixel characteristics as the above-mentioned pixel library 1 when there is a reference pixel characteristic having a similarity with the pixel characteristics of the image 1 greater than a threshold value.

The processor 100 compares the image 1 with at least one reference pixel set in the pixel library 1 to obtain difference information describing the pixel difference between the image 1 with respect to the at least one reference pixel set in the pixel library 1S 804.

In a first possible embodiment, the pixels in the pixel sets belong to at least one image, i.e. each pixel set may include one or more images, and the processor 100 may sequentially perform similarity calculation on the image 1 and each image in the pixel library 1, specifically may calculate similarity between the value of each pixel point in the image 1 and the value of each pixel point in each image included in the pixel library 1. For example, the processor 100 may respectively calculate the image 1 and the image in the pixel library 1 by using DCT to obtain a calculation result corresponding to the image 1 and a calculation result corresponding to the image in the pixel library 1. And, the processor 100 may further calculate a similarity between the two calculation results, that is, a similarity between the value of each pixel point in the image 1 and the value of the pixel point in the image included in the pixel library 1.

When it is determined that there is one image in the pixel library 1 and the similarity between the value of the pixel point of the image and the value of the pixel point in the image 1 is greater than the threshold, the processor 100 may determine the image as the reference pixel set, and at this time, the number of the reference pixel sets participating in the comparison of the pixel differences is one. Then, the processor 100 may calculate a residual error between the value of the pixel in the image 1 and the value of the pixel in the image, to obtain a difference sequence corresponding to a plurality of pixels in the image 1, where the difference sequence is the difference information, and can describe a deviation between the value of each pixel in the image 1 and the value of the corresponding pixel in the reference pixel set.

In a second possible embodiment, the set of pixels comprises pixels in at least one block of pixels in one image, and the image 1 may be divided into blocks of pixels, each block of pixels having the same size as the set of pixels in the pixel library 1. For example, the size of each pixel block may be 8 (pixels) ×8 (pixels), 16×16, or 32×32, or may be other sizes smaller than the resolution of the image 1. For ease of distinction, the pixel block in image 1 will be referred to as the original pixel block hereinafter. For each original pixel block included in the image 1, the processor 100 may perform similarity calculation on the original pixel block and each pixel block in the pixel library 1, specifically, calculate a similarity between a value of a pixel point included in the original pixel block and a value of a pixel point included in the pixel block in the pixel library 1. When there is one pixel block in the pixel library 1, and the similarity between the value of the pixel point included in the pixel block and the value of the pixel point included in the original pixel block is greater than the threshold value, the processor 100 may determine that the pixel block and the original pixel block satisfy the matching condition, and determine the pixel block as the reference pixel block (i.e., the above-mentioned reference pixel set). As such, for each original pixel block in the image 1, a reference pixel block satisfying the matching condition with the original pixel block can be determined from the pixel library 1 in the above-described manner.

Then, for each original pixel block in the image 1, the processor 100 may calculate a residual error between the values of the pixels in the original pixel block and the values of the pixels in the reference pixel block corresponding to the original pixel block, to obtain a difference sequence corresponding to the original pixel block, so that the difference sequences respectively corresponding to the plurality of original pixel blocks are the difference information, where the difference information can describe a deviation between the values of the pixels in each original pixel block in the image 1 and the values of the corresponding pixels in the plurality of reference pixel sets.

For example, assume that image 1 is a gray scale image, and the original pixel block and the reference pixel block are each 8×8 pixel blocks, wherein the values of 64 pixel points in the original pixel block and the reference pixel block are shown in fig. 6, respectively. Then, the processor 100 may calculate the value of the pixel point in the original pixel block, and the residual between the values of the pixel point in the reference pixel block corresponding to the original pixel block may obtain the difference sequence as shown in fig. 6. Since the original pixel block and the reference pixel block have a certain similarity, there are more repeated values in the obtained difference sequence, such as more values 0 and 5 in the difference sequence in fig. 6.

In this embodiment, taking an example that each original pixel block in the image 1 has a reference pixel block corresponding to the original pixel block, when a part of the original pixel blocks in the image 1 are not matched with the reference pixel block, the processor 100 may not determine the difference information for the original pixel block, that is, the original pixel block may be directly stored in the storage medium. Further, the processor 100 may add the original pixel block to the pixel library 1 as a new reference pixel block (i.e., a new reference pixel set) in the pixel library 1, so as to subsequently encode other original pixel blocks or pixel blocks in other images to be stored with the new reference pixel block.

It will be appreciated that the above manner of generating the difference information is merely a few exemplary illustrations, and that in other embodiments, the processor 100 may generate the difference information in other manners. For example, the difference information may be other types of information besides the residual.

In this way, the at least one reference pixel set determined for image 1 based on the above manner has a high similarity in pixel characteristics with image 1, i.e. satisfies the similarity condition.

Further, when a greater number of pixel sets (e.g., more than 1000 pixel sets) are included in the pixel library 1, taking a pixel set, specifically a pixel block as an example, each pixel block in the pixel library 1 may be configured with a priority level for indicating a priority level to which the pixel block is used to determine whether the matching condition is satisfied with the original pixel block. In this way, when determining a reference pixel block matching each original pixel block in image 1, processor 100 may preferentially calculate the similarity between the original pixel block and the pixel block with the higher priority in pixel library 1 according to the priority of each pixel block in pixel library 1. And when the similarity between the pixel block and the original pixel block is larger than a threshold value, determining the pixel block as a reference pixel block, and continuously determining a reference pixel block with higher similarity for the next original pixel block. When the similarity between the pixel block and the original pixel block is smaller than the threshold, the processor 100 may continue to calculate the similarity between the original pixel block and the next pixel block (higher priority) in the pixel library 1 according to the priority until the reference pixel block satisfying the matching condition with the original pixel block is determined. In this manner, the processor 100 may not need to traverse all pixel blocks in the computing pixel library 1 for each original pixel block, so that the resource consumption for determining the reference pixel block may be reduced, and the efficiency of determining the reference resource block may be improved.

And S805, the processor 100 encodes the difference information to obtain an encoding result corresponding to the image 1.

As an implementation example, the processor 100 may be configured with an encoding algorithm, and the processor 100 may encode the difference information using the configured encoding algorithm to obtain a corresponding encoding result, that is, the encoding result of the image 1. In practical application, when the pixels in the image 1 and the pixels in the reference pixel set meet the similar condition, the generated difference information has a certain rule, so that the data size of the result of the encoding generated by the processor 100 for encoding the difference information is smaller than the data size of the image 1, thereby improving the reduction rate of the image 1.

Taking the difference information corresponding to the image 1 as an example, the difference sequence shown in fig. 6 is specifically taken as an example, the data size of the difference sequence is 64B, and a plurality of continuous values are the same in the difference sequence, for example, a plurality of continuous values 0 and 5 exist. Processor 100 may then encode the sequence of differences using RLE algorithm to obtain an encoded result (0, 24), 1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4, (5, 24). At this time, the data amount of the encoding result is 24B, which is smaller than the data amount 64B of the image 1 itself, thereby achieving an improvement in the reduction rate of the image 1 to 62.5% (i.e., (64-24)/64).

Further, the processor 100 may be configured with a plurality of encoding algorithms, and the processor 100 may sequentially encode using the plurality of encoding algorithms to achieve a reduction rate as high as possible.

For example, for the difference sequence shown in fig. 6, the processor 100 may first encode the difference sequence into (0, 24), 1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4, (5, 24) using the RLE algorithm. Because there are consecutive "1,2,3,4" occurrences of repetition in the result encoded by the RLE algorithm, the processor 100 may also continue to encode the current encoding result using an LZ (Lempel-Ziv) encoding algorithm. The LZ coding algorithm is a dictionary-based coding algorithm, and it is assumed that the dictionary includes character strings "1,2,3,4", and that the character strings may be replaced with a. Then the processor 100 encodes (0, 24), a, (5, 24) using the LZ encoding algorithm. Finally, the processor 100 may also continue to encode the current encoding results into (0, 24), (a, 4), (5, 24) using the RLE algorithm. In this way, the data amount of the finally obtained encoding result can be reduced to 6B, and the reduction rate of the image 1 is increased to 90.6% (i.e., (64-6)/64).

In practice, after the processor 100 acquires the difference information, it may extract a data feature from the difference information, where the data feature may be, for example, a feature of the same value that is continuously repeated, a feature of a value that is circularly repeated, and the like as shown in fig. 6. Since different coding algorithms are suitable for coding information having different data characteristics (i.e., the information can achieve a higher reduction rate after being coded), the processor 100 can determine one or more coding algorithms matching the data characteristics from a plurality of configured coding algorithms according to the data characteristics, and sequentially code the difference information using the determined one or more coding algorithms. The method comprises the steps of utilizing a plurality of coding algorithms to code difference information in sequence, namely utilizing the current coding algorithm to code the difference information to obtain a coding result, continuing to code the coding result by the next coding algorithm, and the like until a final coding result is obtained.

The coding algorithm used for coding the difference information may be a lossless coding algorithm, i.e. no information loss is generated after coding the difference information, such as RLE algorithm. Or the coding algorithm adopted for coding the difference information can also be a lossy coding algorithm, i.e. a small amount of information loss can be generated after the difference information is coded, such as a predictive coding algorithm and the like.

The processor 100 saves the encoding result to the storage medium S806.

After generating the encoding results corresponding to image 1, processor 100 may output the encoding results to one or more storage media for storage. Thus, the result of the encoding is stored, that is, the stored image 1.

In practical applications, the processor 100 may specifically store the encoding result, the identifier of the image 1, and the mapping between the encoding result and the reference pixel set in the pixel library 1 in the storage medium, where the mapping may be, for example, a correspondence between the identifier of the encoding result and the identifier of the reference pixel set in the pixel library 1. Accordingly, when the application 200 requests the image 1 from the storage system 10, the processor 100 may restore the image 1 according to the encoding result corresponding to the image 1, and feed back the image 1 to the application 200.

Next, the process of the storage system 10 feeding back the image 1 to the application 200 will be described.

S807 the application 200 sends a data read request to the storage system 10 for requesting the image 1 from the storage system 10.

In response to the data read request, processor 100 restores image 1 based on the encoding result corresponding to image 1 stored in the storage medium S808.

In one possible implementation, after receiving the data reading request, the processor 100 may parse the identifier of the image 1 from the data reading request, and read, according to the identifier of the image 1, a mapping between the encoding result corresponding to the image 1 in the storage medium, and the reference pixel set in the pixel library 1. The processor 100 may then decode the encoded result to obtain the difference information. Meanwhile, the processor 100 may access the pixel library 1 according to the mapping between the encoding result and the reference pixel set to obtain a corresponding reference pixel set in the pixel library 1, and calculate the value of each pixel point in the image 1 according to the difference information obtained by decoding and the reference pixel set, so as to recover the complete pixel data in the image 1.

Further, in the process of saving the encoding result corresponding to the image 1, the processor 100 may record metadata of the image 1, which may be, for example, data for describing the format of the image 1, the encoding algorithm employed, the image color type (gray-scale image or color image, etc.), and the like. Thus, after restoring the complete pixel data in image 1, processor 100 may encode the restored pixel data of image 1 according to the recorded metadata of image 1 to restore image 1 that is sent by application 200 to storage system 10.

Wherein the processor 100 may directly store the metadata of the image 1 in the storage medium when recording the metadata of the image 1. Alternatively, the processor 100 may refer to the above-described method of storing the encoding result corresponding to the pixel data in the image 1, and store the encoding result corresponding to the metadata in the storage medium.

In particular implementation, the processor 100 may be configured with a metadata base including one or more reference metadata, and the processor 100 may compare differences between the metadata of the image 1 and the reference metadata in the metadata base to generate difference information for the metadata, hereinafter referred to as metadata difference information, where the metadata difference information is used to describe differences between the metadata of the image 1 and the reference metadata, and the metadata difference information may be, for example, a residual error or the like. The specific implementation process of determining the reference metadata in the metadata base for the metadata of the image 1 by the processor 100 may be described in the above-mentioned related parts for determining the reference pixel set, which is not described herein. The processor 100 may then encode the metadata difference information and map the encoding result, and the identification of the reference metadata in the metadata database. Further, the processor 100 may also send the metadata base to the storage medium for persistent storage. At this time, the processor 100 may also compare the metadata of the image 1 with the reference metadata in the metadata base while comparing the pixels in the image 1 with the pixels in the reference pixel set in the pixel base 1, so that the difference information obtained by the processor 100 may also be used to describe the difference between the metadata of the image 1 and the reference metadata. Illustratively, the difference information may specifically include pixel difference information for describing a difference between a pixel in the image 1 and a pixel in the reference pixel set in the pixel library 1, and metadata difference information for describing a difference between metadata of the image 1 and reference metadata.

Accordingly, in the process of recovering the image 1, the processor 100 may read the encoding result and the mapping relation corresponding to the metadata difference information from the storage medium, and decode the encoding result according to the mapping relation to obtain the metadata difference information, and determine the reference metadata in the metadata database according to the mapping relation. In this way, the processor 100 can restore the metadata of the image 1 based on the metadata difference information and the reference metadata in the metadata base.

In practical application, the metadata base and the pixel base can be two independent databases, or the metadata base and the pixel base can be integrated into one database, for example, the pixel base can simultaneously store a reference pixel set, reference metadata and the like.

The processor 100 feeds back the restored image 1 to the application 200S 809.

Illustratively, after restoring image 1, the processor 100 may generate a response message including image 1 and send the response message to the application 200 to satisfy the query requirement of the application 200 for image 1.

In this way, in the case where the pixel library is configured in the storage system 10, when the storage system 10 stores the image 1, it is only necessary to store the encoding result corresponding to the image 1 with a smaller data amount, and the original image data does not need to be stored, which can effectively reduce the storage space required for storing the image 1, that is, increase the reduction rate of the image. Moreover, when the application 200 requests the image 1 from the storage system 10, the storage system 10 can restore the complete image 1 according to the pixel library and the saved encoding result, so as to meet the requirement of the application 200 on the image 1.

In addition, when the encoding algorithm used for encoding the difference information is a lossless encoding algorithm, the storage system 10 can recover the pixel data of the complete image 1 according to the encoding result and the reference pixel result, which can ensure the information integrity during image storage and realize lossless storage of the image.

Furthermore, when the difference information is obtained by comparing the pixel block granularity, the processor 100 only needs to read the reference pixel blocks used for recovering the image 1 in the pixel library 1 in the process of recovering the image 1, and does not need to read all the pixel blocks in the pixel library 1 (and does not need to read the pixel blocks in other pixel libraries), so that the problem of reading and amplifying can be avoided.

In this embodiment, a process of processing one image by using the pixel library 1 by the processor 100 is taken as an example, and in practical application, for other images provided by the application 200, the processor 100 may also use the pixel library 1 or other pixel libraries to process other images accordingly, so as to improve the reduction rate of the images and reduce the storage space required for storing the images. For example, when the application 200 sends the image 2 to the storage system 10, the processor 100 may determine the pixel library 2 for the image 2 from a plurality of pixel libraries, where the pixels in the pixel library 2 and the pixels in the image 2 satisfy a similar condition, so that the processor 100 may compare the image 2 with at least one reference pixel set in the pixel library 2 to obtain difference information, and encode the difference information to obtain an encoding result corresponding to the image 2, so as to save the encoding result corresponding to the image 2 in a storage medium, thereby implementing saving of the image 2. The plurality of images sequentially stored in the storage system 10 may be images having no association relationship, such as different images photographed by the user at different times and different places.

The processing of images during image storage is described above in connection with fig. 1 to 8. In practical applications, there may be a scenario where audio is stored, and in this case, the storage space required for storing audio may be reduced by data reduction of audio. An exemplary description is provided below in connection with fig. 9 and 10.

Referring to fig. 9, a schematic diagram of another storage system is shown. As shown in fig. 9, the storage system 90 includes a processor 900 and at least one storage medium, where each storage medium, which may be, for example, an SSD, HDD, disk, etc., can be used to persist storage data. Storage system 90 may also be in communication with application 200, such as receiving audio provided by application 200, feeding back notification messages to application 200 (e.g., notifying application 200 that audio was stored successfully, etc.).

The processor 900 is configured to reduce the audio provided by the application 200, and then store the reduced audio data to a storage medium, so as to reduce the storage space required for storing the audio. The specific implementation of the processor 900 may be described with reference to the above-mentioned related points of the implementation of the processor 100 in fig. 1, which is not described herein.

Also, a memory 1001 may be configured in the processor 900, as shown in fig. 9, so that the processor 900 may store data using the memory 1001 and perform corresponding processing operations and the like based on the data in the memory 1001. Alternatively, the storage system 90 may also configure the processor 900 with a separate memory 1002, as shown in fig. 9, so that data required for the processor 900 to perform data processing operations or generated data may be stored by the memory 1002.

Referring to fig. 10, a flow chart of an audio processing method is shown, which can be applied to the storage system 90 shown in fig. 9. As shown in fig. 10, the method may specifically include the following steps.

S1001, the application 200 sends the audio to be processed to the storage system 90.

The audio transmitted by the application 200 may be any format of audio, such as Windows media audio (Windows media audio, WMA) format, waveform audio file format (waveform audio file format, WAV), advanced audio coding (advanced audio coding, AAC) format, and so on.

The processor 900 divides the audio to obtain a plurality of original audio segments S1002.

In one possible implementation, since the audio transmitted by the application 200 may be in any format, there is typically a large difference in coding algorithms, audio data organization, etc. employed for different formats of audio. Thus, the processor 900 may format convert the received audio to a unified standard format audio. Thus, the subsequent processor 900 may perform subsequent processing operations on the standard format audio. In particular, the processor 900 may decode the received audio to obtain audio data, and encode the audio data into audio in a standard format according to a standard format using an encoding algorithm corresponding to the standard format.

The processor 900 may then divide the audio (in standard format) into a plurality of segments according to a preset size, hereinafter referred to as the original audio segments for ease of distinction and description. For example, the processor 900 may divide audio into a plurality of original audio segments including the same number of audio frames by frame number or time length, or divide audio into a plurality of original audio segments corresponding to the same play time length. In the present embodiment, the manner of dividing the audio is not limited.

And S1003, the processor 900 compares each original audio segment in the plurality of original audio segments with at least one reference audio segment in the audio library to obtain difference information respectively corresponding to the plurality of original audio segments, wherein the difference information corresponding to each original audio segment is used for describing the audio difference between the original audio segment and one reference audio segment.

Accordingly, difference information corresponding to the plurality of original audio segments is used to describe an audio difference between the entire audio with respect to the at least one reference audio segment.

In this embodiment, the processor 900 may be configured with an audio library and one or more audio segments may be included in the audio library. For example, the audio library may be pre-configured in the processor 900 by a technician, or may be created by the processor 900 based on audio received over a past period of time. After the division into the plurality of original audio segments, for each original audio segment, the processor 900 may perform similarity calculation on the original audio segment in turn with each audio segment in the audio library, e.g., may calculate a similarity between two audio segments based on a discrete cosine transform (discrete cosine transform, DCT) technique, etc. When there is similarity between an audio segment in the audio library and the original audio segment (e.g., the similarity is greater than a threshold), the processor 900 may determine the audio segment as a reference audio segment for subsequent use to reduce the amount of data of the audio to be stored. As such, the audio similarity condition is satisfied between the reference audio segment and the original audio segment as determined by the processor 900, such as a similarity between the two audio segments being greater than a threshold. And when there is no similarity between an audio segment in the audio library and the original audio segment, the processor 900 may add the original audio segment to the audio library for subsequent data retrieval of other audio or other original audio segments using the added original audio segment. In this embodiment, it is assumed that a reference audio segment corresponding to each original audio segment exists in the audio library.

Further, when a greater number of audio segments are included in the audio library, each audio segment in the audio library may be configured with a priority indicating how prioritized the audio segment is to be used to participate in the similarity calculation. Thus, when looking up an audio segment (i.e., a reference audio segment) from the audio library that is similar to the original audio segment, the processor 900 may preferentially calculate the similarity between the original audio segment and the audio segment with the higher priority according to the priority of each audio segment. Thus, by configuring priorities for audio segments in the audio library, the efficiency with which the processor 900 determines the reference audio segments for the original audio segments can be increased, saving computing resource overhead.

After determining the reference audio segment corresponding to the original audio segment, the processor 900 may compare the audio difference between the original audio segment and the reference audio segment to obtain difference information, where the difference information may be, for example, a residual error between audio data included in the original audio segment and audio data included in the reference audio segment. As such, for each original audio segment, the processor 900 may calculate the difference information corresponding to each original audio segment in a similar manner as described above.

And S1004, the processor 900 codes the difference information corresponding to each original audio segment to obtain the coding results corresponding to the plurality of original audio segments respectively.

As an implementation example, the processor 900 may be configured with an encoding algorithm, and the processor 900 may encode the difference information corresponding to each original audio segment using the configured encoding algorithm, to obtain a corresponding encoding result. The encoding results corresponding to the original audio segments are the encoding results of the audio.

In practical application, since the original audio segments are similar to the reference audio segments, the difference information generated according to the original audio segments and the reference audio segments has a certain rule, so that the data size of the encoding result generated by the processor 100 for encoding the difference information corresponding to each original audio segment is smaller than that of the original audio segments, thereby improving the audio reduction rate and reducing the storage space required by audio storage.

Further, the processor 900 may be configured with a plurality of encoding algorithms, and in the process of encoding the difference information corresponding to the original audio segment, the processor 900 may first extract a data feature in the difference information, determine one or more encoding algorithms matching the data feature from the configured plurality of encoding algorithms according to the data feature, and sequentially encode the difference information using the determined one or more encoding algorithms. The method comprises the steps of utilizing a plurality of coding algorithms to code difference information in sequence, namely utilizing the current coding algorithm to code the difference information to obtain a coding result, continuing to code the coding result by the next coding algorithm, and the like until a final coding result is obtained. In this way, by encoding the difference information by matching an appropriate encoding algorithm, it is possible to contribute to an improvement in the reduction rate of audio.

And S1005, the processor 900 sends the coding results corresponding to the original audio segments to a storage medium for storage.

In practical applications, the processor 900 may store the encoding result, the identification of the audio corresponding to the encoding result, and the mapping between the encoding result and the reference audio segment in the audio library in the storage medium during the process of storing the encoding result. Further, the processor 900 may also send the audio library to a storage medium for persistent storage. In this way, during subsequent requests of the audio from the application 200 to the storage system 100, the processor 900 may determine, based on the identification of the audio corresponding to the encoding results, the encoding results corresponding to all original audio segments included in the audio, and determine, based on the mapping, the reference audio segments in the audio library corresponding to each original audio segment in the audio. Thus, the processor 900 may restore the original audio segments included in the audio according to the encoding result corresponding to the original audio segments and the reference audio segment corresponding to each original audio segment, so as to restore the complete audio data. Further, the processor 900 may also store metadata for the audio when storing the encoded data, which may include, for example, the encoding format of the audio transmitted by the application 200. Thus, after recovering the complete audio data, the processor 900 may encode the audio data into an audio file in a format required by the application 200 according to the saved metadata, so as to meet the requirement of the application 200 for audio.

In this embodiment, the processor 900 is described as an example of processing one audio. When the application 200 provides a greater amount of other audio to the storage system 100, the processor 900 may utilize the audio library to reduce the amount of data of the other audio for storage in a similar manner as described above.

Further, the processor 900 may also be configured with multiple audio libraries, and for different audio transmitted by the application 200, the processor 900 may first determine an appropriate audio library for the audio and utilize the determined audio library to data reduce the audio. In particular implementations, the processor 900 may extract audio features from the audio, such as, but not limited to, average energy, power spectrum, etc. of the audio. Meanwhile, each audio library is configured with a reference audio feature, which may be, for example, an average value of audio features of individual audio segments included in the audio library, or the like. The processor 900 may then calculate the similarity between the audio features of the audio and the corresponding reference audio features of the respective audio libraries and determine the audio library with the greatest similarity as the audio library used for data reduction of the audio to be processed.

In this embodiment, taking the example that the processor 900 uses a pre-created audio library to perform data reduction on the currently received audio, in other embodiments, the processor 900 may also create an audio library for the audio during the process of processing the audio. Specifically, when the playing duration corresponding to the audio is longer (for example, the playing duration reaches 1 hour, etc.), the processor 900 may divide the audio first to obtain a plurality of original audio segments (for example, 3600 original audio segments are obtained by dividing), and construct an audio library according to a part of the original audio segments (for example, 600 original audio segments) in the plurality of original audio segments, so that the processor 900 may utilize the original audio segments in the audio library to perform data reduction on the remaining original audio segments (for example, the remaining 3000 original audio segments) in the audio, thereby improving the overall reduction rate for the audio and reducing the storage space required for storing the audio.

It is noted that other reasonable combinations of steps can be envisaged by the person skilled in the art from the above description, and are also within the scope of the application. Furthermore, those skilled in the art will be familiar with the preferred embodiments, and the description of the preferred embodiments does not necessarily require the application.

The storage system, the image processing method and the audio processing method according to the embodiments of the present application are described above with reference to fig. 1 to 10, and the structures of the apparatus and the computing device according to the embodiments of the present application are described below with reference to the accompanying drawings.

Referring to fig. 11, there is shown a schematic configuration diagram of an image processing apparatus 1100 including:

An acquisition module 1101, configured to acquire a plurality of images to be processed;

a grouping module 1102, configured to group a plurality of images according to pixel characteristics of the plurality of images, to obtain a plurality of groups, where a similarity degree between different images in a same group is higher than a similarity degree between images in different groups, and the plurality of groups includes a first group;

The encoding module 1103 is configured to encode the image in the first packet by using a first pixel library corresponding to the first packet, so as to obtain an image encoding result corresponding to the first packet, where the first pixel library includes pixels in a part of the images in the first packet, and the image encoding result corresponding to the first packet is used to restore the images in the first packet with the first pixel library.

In one possible implementation, the plurality of images to be processed include images of different formats, and the image processing apparatus 1100 further includes:

a conversion module 1104, configured to perform format conversion on the multiple images to obtain multiple images in a standard format;

The grouping module 1102 is specifically configured to group the plurality of images in the standard format according to the pixel characteristics of the plurality of images in the standard format.

In one possible implementation, the pixel library corresponding to the first packet further includes reference metadata;

The encoding module 1103 is further configured to encode metadata of the standard format image in the first packet by using the reference metadata, so as to obtain a metadata encoding result corresponding to the first packet, where the metadata encoding result corresponding to the first packet is used to restore metadata of the standard format image in the first packet with the reference metadata.

In one possible implementation, the encoding module 1103 is configured to:

Comparing pixels in the image in the first group with pixels in the first pixel library to obtain pixel difference information, wherein the pixel difference information is used for describing differences between the pixels in the image in the first group and the pixels in the first pixel library;

and encoding the pixel difference information to obtain an image encoding result corresponding to the first group.

In one possible implementation, the first pixel library includes pixels in a plurality of reference pixel blocks, the reference pixel blocks being pixel blocks in one image, and the pixel difference information being a residual;

An encoding module 1103 for:

dividing images in a first group to obtain a plurality of original pixel blocks;

according to the first pixel library, determining reference pixel blocks which respectively meet matching conditions with each original pixel block in the plurality of original pixel blocks, wherein the matching conditions comprise pixel similarity between two pixel blocks being larger than a threshold value;

a residual between the original pixel block and the reference pixel block satisfying the matching condition is calculated.

In one possible implementation, each of the plurality of reference pixel blocks is configured with a priority for indicating a degree of priority with which the reference pixel block is used to determine whether a matching condition is satisfied with the original pixel block;

The encoding module 1103 is configured to determine, from the first pixel library, a reference pixel block that meets a matching condition with each of the plurality of original pixel blocks according to the priority of each reference pixel block.

In one possible implementation, the plurality of original pixel blocks includes a first original pixel block;

An encoding module 1103 for:

searching whether a reference pixel block meeting a matching condition with a first original pixel block exists in the first pixel library;

And when the first pixel library does not have the reference pixel block which meets the matching condition with the first original pixel block, adding the first original pixel block into the first pixel library as the reference pixel block.

In one possible implementation, the encoding module 1103 is configured to:

Extracting data characteristics of pixel difference information;

determining at least one coding algorithm matching the data characteristic from a plurality of coding algorithms according to the data characteristic;

The encoding operations for the pixel difference information are sequentially performed using at least one encoding algorithm.

Since the image processing apparatus 1100 shown in fig. 11 corresponds to the processor 100 in the embodiment shown in fig. 2 and is used for executing the method executed by the processor 100 in the embodiment shown in fig. 2, the specific implementation of the image processing apparatus 1100 shown in fig. 11 and its technical effects are described with reference to the relevant points in the embodiment shown in fig. 2, and will not be described herein.

Referring to fig. 12, there is shown a schematic configuration of another image processing apparatus 1200 including:

an acquisition module 1201, configured to acquire a first image to be processed;

a determining module 1202, configured to determine a first pixel library from a plurality of pixel libraries, where pixels in the first pixel library and pixels in the first image satisfy a similar condition, and pixels in different pixel libraries do not satisfy the similar condition;

A comparison module 1203, configured to compare the first image with at least one reference pixel set in the first pixel library to obtain difference information, where the difference information is used to describe a pixel difference between the first image and the at least one reference pixel set;

The encoding module 1204 is configured to encode the difference information to obtain an encoding result corresponding to the first image, where the encoding result is used to restore the first image with at least one reference pixel set.

In one possible implementation, the pixels in the reference pixel set belong to at least one image, and the difference information is used to describe a pixel difference between the first image and one image in the first pixel library;

Or the reference pixel set comprises pixels in reference pixel blocks, the reference pixel blocks are pixel blocks in one image, and the difference information is used for describing pixel differences between each pixel block in the first image and one reference pixel block in the first pixel library respectively.

In one possible implementation, the reference pixel set includes pixels in a reference pixel block, and the difference information is a residual;

an alignment module 1203 for:

dividing the first image to obtain a plurality of original pixel blocks;

Determining a reference pixel block from the first pixel library that satisfies a matching condition with each of the plurality of original pixel blocks, the matching condition including a similarity between pixels in the two pixel blocks being greater than a threshold;

Residual errors between pixels in the original pixel block and pixels in the reference pixel block satisfying the matching condition are calculated.

In one possible implementation, each reference pixel block in the first pixel library is configured with a priority for indicating a priority to which the reference pixel block is used to determine whether a matching condition is satisfied with the original pixel block;

A determining module 1202 is configured to determine, from the first pixel library, a reference pixel block that meets a matching condition with each of the plurality of original pixel blocks according to priorities of the respective reference pixel blocks.

In one possible implementation, the determining module 1202 is configured to:

Extracting pixel characteristics of a first image;

And determining the first pixel library from a plurality of pixel libraries according to the pixel characteristics of the first image, wherein the similarity between the reference pixel characteristics of the first pixel library in the plurality of pixel libraries and the pixel characteristics of the first image is the largest.

In one possible implementation, the encoding module 1204 is configured to:

Extracting data characteristics of the difference information;

the difference information is encoded using at least one encoding algorithm.

In one possible implementation, the determining module 1202 is configured to:

Performing format conversion on the first image to obtain a first image in a standard format;

For a first image in a standard format, a first pixel library is determined from a plurality of pixel libraries.

In one possible implementation, the difference information includes metadata difference information and pixel difference information, and the first pixel library further includes reference metadata;

an alignment module 1203 for:

comparing the metadata of the first image in the standard format with the reference metadata in the first pixel library to obtain metadata difference information;

and comparing the first image in the standard format with at least one reference pixel set in the first pixel library to obtain pixel difference information.

Since the image processing apparatus 1200 shown in fig. 12 corresponds to the processor 100 in the embodiment shown in fig. 8 and is used for executing the method executed by the processor 100 in the embodiment shown in fig. 8, the specific implementation of the image processing apparatus 1200 shown in fig. 12 and its technical effects are described with reference to the relevant points in the embodiment shown in fig. 8, and will not be described herein.

Referring to fig. 13, there is shown a schematic structural diagram of another audio processing apparatus 1300 including:

An acquiring module 1301, configured to acquire audio to be processed;

a dividing module 1302, configured to divide audio to obtain a plurality of original audio segments;

A comparison module 1303, configured to compare each of the plurality of original audio segments with at least one reference audio segment in an audio library to obtain difference information corresponding to the plurality of original audio segments, where the at least one reference audio segment in the audio library includes a portion of the original audio segments in the plurality of original audio segments, and the difference information is used to describe an audio difference between the audio with respect to the at least one reference audio segment;

the encoding module 1304 is configured to encode the difference information to obtain an encoding result corresponding to the audio, where the encoding result is used to restore the audio with at least one reference audio segment.

Since the audio processing apparatus 1300 shown in fig. 13 corresponds to the processor 900 in the embodiment shown in fig. 10 and is configured to execute the method executed by the processor 900 in the embodiment shown in fig. 10, the specific implementation of the audio processing apparatus 1300 shown in fig. 13 and its technical effects will be described with reference to the relevant points in the embodiment shown in fig. 10, and will not be described herein.

Fig. 14 is a schematic hardware structure of a computing device 1400 provided in the present application, where the computing device 1400 may implement, for example, the processor 100 in the embodiment shown in fig. 2 or fig. 8, or the processor 900 in the embodiment shown in fig. 10, etc.

As shown in fig. 14, computing device 1400 includes a processor 1401, a memory 1402, and a communication interface 1403. The processor 1401, the memory 1402, and the communication interface 1403 communicate with each other via the bus 1404, or can communicate with each other via other means such as wireless transmission. The memory 1402 is for storing instructions, and the processor 1401 is for executing the instructions stored by the memory 1402. Further, computing device 1400 may also include a memory unit 1405, where memory unit 1405 may be coupled to processor 1401, storage medium 1402, and communication interface 1403 via bus 1404. Wherein the memory 1402 stores program code, and the processor 1401 may call the program code stored in the memory 1402 to:

acquiring a plurality of images to be processed;

grouping the images according to the pixel characteristics of the images to obtain a plurality of groups, wherein the similarity between different images in the same group is higher than that between the images in different groups, and the plurality of groups comprise a first group;

and encoding the image in the first group by using a first pixel library corresponding to the first group to obtain an image encoding result corresponding to the first group, wherein the first pixel library comprises pixels in partial images in the first group, and the image encoding result corresponding to the first group is used for recovering the images in the first group with the first pixel library.

Or processor 1401 may invoke program code stored in memory 1402 to:

Acquiring a first image to be processed;

determining a first pixel library from a plurality of pixel libraries, wherein the pixels in the first pixel library and the pixels in the first image meet similar conditions, and the pixels in different pixel libraries do not meet similar conditions;

Comparing the first image with at least one reference pixel set in a first pixel library to obtain difference information, wherein the difference information is used for describing pixel differences between the first image and the at least one reference pixel set;

and encoding the difference information to obtain an encoding result corresponding to the first image, wherein the encoding result is used for recovering the first image with at least one reference pixel set.

Or processor 1401 may invoke program code stored in memory 1402 to:

Acquiring audio to be processed;

dividing the audio to obtain a plurality of original audio segments;

Comparing each of the plurality of original audio segments with at least one reference audio segment in an audio library to obtain difference information corresponding to the plurality of original audio segments, wherein the at least one reference audio segment in the audio library comprises part of the original audio segments in the plurality of original audio segments, and the difference information is used for describing audio differences between the audio relative to the at least one reference audio segment;

And encoding the difference information to obtain an encoding result corresponding to the audio, wherein the encoding result is used for recovering the audio with at least one reference audio segment.

It is to be appreciated that in this embodiment, the processor 1401 may be a CPU, and the processor 1401 may also be other general purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete device components, or the like. A general purpose processor may be a microprocessor or any conventional processor or the like.

The memory 1402 may include read only memory and random access memory, and provides instructions and data to the processor 1401. Memory 1402 may also include nonvolatile random access memory.

The memory 1402 may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an erasable programmable ROM (erasable PROM), an electrically erasable programmable EPROM (EEPROM), or a flash memory. The volatile memory may be random access memory (random access memory, RAM) which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (STATIC RAM, SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (doubledata DATE SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCHLINK DRAM, SLDRAM), and direct memory bus random access memory (direct rambus RAM, DR RAM).

The communication interface 1403 is used to communicate with other devices connected to the computing device 1400. The bus 1404 may include a power bus, a control bus, a status signal bus, and the like in addition to the data bus. But for clarity of illustration, the various buses are labeled as bus 1404 in the figures.

It should be appreciated that the computing device 1400 according to the embodiments of the present application may correspond to the processor 100 or the processor 900 in the embodiments of the present application, and may correspond to a method executed by the processor 100 or the processor 900 in the embodiments described above, and that the foregoing and other operations and/or functions implemented by the computing device 1400 are, respectively, for implementing the flow of the corresponding method executed by the processor 100 or the processor 900 in the embodiments of the methods described above, which are not repeated herein for brevity.

The embodiment of the application also provides a computer readable storage medium. The computer readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), etc. The computer-readable storage medium includes instructions that instruct a computing device to perform the image processing method or the audio processing method described above.

The embodiment of the application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the processes or functions in accordance with embodiments of the present application are fully or partially developed.

The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.).

The computer program product may be a software installation package which may be downloaded and executed on a computing device in case any one of the aforementioned image processing methods or audio processing methods is required.

The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded or executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.

The terminology used in the above-described embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include, for example, "one or more" such forms of expression, unless the context clearly indicates to the contrary. It should also be understood that in embodiments of the present application, "one or more" means one, two or more than two, and that the character "/" generally indicates that the associated object is an "or" relationship. In the embodiment of the application. "simultaneous" refers to the case where the same time period, including the same time, is within the same time period. The terms first, second and the like in the description and in the claims and drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are merely illustrative of the manner in which embodiments of the application have been described in connection with the description of the objects having the same attributes.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

While the application has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the application. Therefore, the protection scope of the application is subject to the protection scope of the claims.

Claims

1. An image processing method, the method comprising:

acquiring a plurality of images to be processed;

Grouping the images according to the pixel characteristics of the images to obtain a plurality of groups, wherein the similarity between different images in the same group is higher than that between the images in different groups, and the groups comprise a first group;

And coding the image in the first group by using a first pixel library corresponding to the first group to obtain an image coding result corresponding to the first group, wherein the first pixel library comprises pixels in partial images in the first group, and the image coding result corresponding to the first group is used for restoring the image in the first group with the first pixel library.

2. The method of claim 1, wherein the plurality of images to be processed comprise images of different formats, the method further comprising:

Performing format conversion on the plurality of images to obtain a plurality of images in a standard format;

the grouping the plurality of images according to the pixel characteristics of the plurality of images includes:

and grouping the plurality of images in the standard format according to the pixel characteristics of the plurality of images in the standard format.

3. The method of claim 2, wherein the first grouping of corresponding pixel bins further comprises reference metadata, the method further comprising:

and encoding the metadata of the standard format image in the first packet by utilizing the reference metadata to obtain a metadata encoding result corresponding to the first packet, wherein the metadata encoding result corresponding to the first packet is used for restoring the metadata of the standard format image in the first packet with the reference metadata.

4. A method according to any one of claims 1 to 3, wherein said encoding the image within said first packet with a first library of pixels corresponding to said first packet comprises:

5. The method of claim 4, wherein the first pixel library comprises pixels in a plurality of reference pixel blocks, the reference pixel blocks being pixel blocks in an image, the pixel difference information being residuals;

comparing the pixels in the image in the first group with the pixels in the first pixel library to obtain pixel difference information, including:

Dividing the image in the first group to obtain a plurality of original pixel blocks;

Determining a reference pixel block which respectively meets a matching condition with each original pixel block in the plurality of original pixel blocks according to the first pixel library, wherein the matching condition comprises that the pixel similarity between two pixel blocks is larger than a threshold value;

6. The method of claim 5, wherein each of the plurality of reference pixel blocks is configured with a priority indicating a degree of priority with which the reference pixel block is used to determine whether a matching condition is satisfied with the original pixel block;

The determining a reference pixel block that satisfies a matching condition with each of the plurality of original pixel blocks, respectively, includes:

And determining the reference pixel blocks which respectively meet the matching conditions with the original pixel blocks in the plurality of original pixel blocks from the first pixel library according to the priority of the reference pixel blocks.

7. The method of claim 5 or 6, wherein the plurality of original pixel blocks comprises a first original pixel block;

The determining, according to the first pixel library, a reference pixel block that satisfies a matching condition with each of the plurality of original pixel blocks, respectively, includes:

searching whether a reference pixel block meeting the matching condition with the first original pixel block exists in the first pixel library or not;

8. The method according to any one of claims 4 to 7, wherein said encoding said pixel difference information comprises:

extracting data characteristics of the pixel difference information;

Determining at least one coding algorithm matching the data feature from a plurality of coding algorithms according to the data feature;

And sequentially performing encoding operations for the pixel difference information by using the at least one encoding algorithm.

9. An image processing apparatus, characterized in that the apparatus comprises:

The acquisition module is used for acquiring a plurality of images to be processed;

The grouping module is used for grouping the images according to the pixel characteristics of the images to obtain a plurality of groups, the similarity degree between different images in the same group is higher than that between the images in different groups, and the plurality of groups comprise a first group;

The encoding module is configured to encode the image in the first packet by using a first pixel library corresponding to the first packet, so as to obtain an image encoding result corresponding to the first packet, where the first pixel library includes pixels in a part of the images in the first packet, and the image encoding result corresponding to the first packet is used for restoring the image in the first packet with the first pixel library.

10. The apparatus of claim 9, wherein the plurality of images to be processed comprise images of different formats, the apparatus further comprising:

the conversion module is used for carrying out format conversion on the plurality of images to obtain a plurality of images in a standard format;

The grouping module is specifically configured to group the plurality of images in the standard format according to the pixel characteristics of the plurality of images in the standard format.

11. The apparatus of claim 10, wherein the first grouping of corresponding pixel bins further comprises reference metadata;

The encoding module is further configured to encode metadata of the standard format image in the first packet by using the reference metadata, so as to obtain a metadata encoding result corresponding to the first packet, where the metadata encoding result corresponding to the first packet is used for restoring metadata of the standard format image in the first packet with the reference metadata.

12. The apparatus according to any one of claims 9 to 11, wherein the encoding module is configured to:

13. The apparatus of claim 12, wherein the first pixel library comprises pixels in a plurality of reference pixel blocks, the reference pixel blocks being pixel blocks in an image, the pixel difference information being residuals;

The coding module is used for:

14. The apparatus of claim 13, wherein each of the plurality of reference pixel blocks is configured with a priority indicating a degree of priority with which the reference pixel block is used to determine whether a matching condition is satisfied with an original pixel block;

The coding module is configured to determine, from the first pixel library, a reference pixel block that satisfies a matching condition with each of the plurality of original pixel blocks, respectively, according to priorities of each reference pixel block.

15. The apparatus of claim 13 or 14, wherein the plurality of original pixel blocks comprises a first original pixel block;

The coding module is used for:

16. The apparatus according to any one of claims 12 to 15, wherein the encoding module is configured to:

extracting data characteristics of the pixel difference information;

17. A computing device comprising a processor and a memory;

The processor is configured to execute instructions stored in the memory to cause the computing device to perform the steps of the method of any one of claims 1 to 8.

18. A storage system comprising a processor for performing the method of any of claims 1 to 8, at least one storage medium for persistently storing image encoding results.

19. A computer readable storage medium comprising instructions which, when run on a computing device, cause the computing device to perform the steps of the method of any of claims 1 to 8.

20. A computer program product containing instructions which, when run on at least one computing device, cause the at least one computing device to perform the steps of the method of any of claims 1 to 8.