CN116828200B

CN116828200B - Image processing method, processing device, equipment and medium

Info

Publication number: CN116828200B
Application number: CN202311093408.3A
Authority: CN
Inventors: 孙旭; 周玉龙; 刘刚; 李拓
Original assignee: Suzhou Inspur Intelligent Technology Co Ltd
Current assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Priority date: 2023-08-29
Filing date: 2023-08-29
Publication date: 2024-01-23
Anticipated expiration: 2043-08-29
Also published as: CN116828200A

Abstract

The invention relates to an image processing method, a processing device, equipment and a medium, wherein the image processing method comprises a line cache compression storage method, which comprises the following steps: acquiring bare data and data to be compressed corresponding to each component in video image line data; generating quantized difference values and sign bits corresponding to the components according to the data to be compressed corresponding to the components; storing the bare data corresponding to each component into original data; and generating combined record data corresponding to each component according to the quantized difference value and the sign bit corresponding to each component, and taking the combined record data corresponding to each component as line cache compression data of data to be compressed corresponding to each component for compression storage. By the technical scheme, the problem of large consumption of storage resources in the current image data line caching process can be solved.

Description

Image processing method, processing device, equipment and medium

Technical Field

The present invention relates to the field of image data technologies, and in particular, to an image processing method, an image processing device, an apparatus, and a medium.

Background

In the field of video image processing, partial line images are buffered (e.g., written into an internal RAM or FIFO memory) and then read out, a method called line buffering or line buffer processing. The line buffering method has wide application, for example, in the field of JPEG image coding, the unit of data coding processing is generally 8 x 8 image blocks, so that 8 lines of image data are generally required to be buffered first, then are sequentially read out to form the 8 x 8 image blocks, and then are subjected to subsequent processing; in the image ISP processing, when noise reduction filtering is performed, the matrix blocks 3*3 around the central pixels need to be processed, and 3 lines of images are generally cached; at least 3 lines of images need to be cached according to different algorithms also in the image scaling process.

Although the conventional line cache processing method can complete the calculation process of the algorithm, the conventional line cache processing method also has inherent disadvantages:

the space/capacity of the line buffer is linearly related to the increase of the image specification, for example, 1080P-specification image, 1920 pixels per line, each line buffer unit needs to buffer 1920 pixels of data, while 4K image needs to buffer 3840 pixels of data, which leads to the rapid increase of the RAM hardware resource, and the increase of the RAM resource leads to the rapid increase of the area and cost of the chip, which is the largest part of the chip area. This has a negative impact on the timing, area, and power consumption of the entire chip.

Therefore, in the prior art, the method of buffering the original image data in the line buffer consumes a large amount of memory resources (such as RAM or FIFO memory), resulting in an increase in the area of the chip or cost, power consumption, and time sequence degradation.

Disclosure of Invention

In order to solve the technical problems, the invention provides an image processing method, an image processing device and a medium, wherein the image processing method is used for solving the problem that the memory resource is consumed when the current image data line is cached.

To achieve the above object, the present invention provides an image processing method, including a line cache compression storage method, comprising:

Acquiring bare data and data to be compressed corresponding to each component in video image line data; generating quantized difference values and sign bits corresponding to the components according to the data to be compressed corresponding to the components; storing the bare data corresponding to each component into original data; and generating combined record data corresponding to each component according to the quantized difference value and the sign bit corresponding to each component, and taking the combined record data corresponding to each component as line cache compression data of data to be compressed corresponding to each component for compression storage.

Further, the obtaining of the bare data and the data to be compressed corresponding to each component in the video image line data specifically includes:

acquiring bare data and data to be compressed corresponding to each component in JPEG video image line data; wherein each component includes a Y component, a Cb component, and a Cr component.

Further, generating quantized difference values and sign bits corresponding to the components according to the data to be compressed corresponding to the components specifically includes:

respectively performing difference calculation on each component of adjacent pixels in the JPEG video image line data to generate a difference value corresponding to each component;

taking the highest bit of the differential value corresponding to each component as the sign bit corresponding to each component, and taking the rest bit of the differential value corresponding to each component as the differential data bit corresponding to each component;

And respectively carrying out grouping quantization on the differential data bits corresponding to each component, and generating quantized difference values corresponding to each component.

Further, the method respectively performs grouping quantization on the differential data bits corresponding to each component, and generates a quantized difference value corresponding to each component, and specifically includes:

according to a first preset data bit is removed, data compression is carried out on differential data bits corresponding to Y components in the JPEG video image line data, and quantized difference values corresponding to the Y components are generated;

according to a second preset data bit is removed, data compression is carried out on differential data bits corresponding to Cb components in the JPEG video image line data, and quantized difference values corresponding to the Cb components are generated;

and according to the second preset data bit is removed, data compression is carried out on the differential data bit corresponding to the Cr component in the JPEG video image line data, and a quantized difference value corresponding to the Cr component is generated.

Further, the acquiring the bare data and the data to be compressed corresponding to each component in the JPEG video image line data specifically includes:

for a single-path computing unit system, taking first column data of each row in the JPEG video image row data as bare data and the rest all data as data to be compressed;

And for the multi-path computing unit system, grouping the JPEG video image line data according to a preset image processing basic unit, and taking the first data of each group of data after grouping as bare data and the rest all data as data to be compressed.

Further, the image processing method further includes a first decompression method, which includes:

performing inverse quantization processing on line cache compressed data of the data to be compressed corresponding to each component to generate the data to be compressed corresponding to each component;

generating original data corresponding to each component according to the bare data corresponding to each component and the data to be compressed corresponding to each component after decompression;

the image processing method further comprises a first matrix point-by-point quantization calculation method, which comprises the following steps:

and carrying out point-by-point quantization calculation on the preset image processing basic unit according to a preset original quantization matrix table corresponding to each component.

Further, the image processing method further includes a second decompression method, which includes:

performing partial reduction processing on line cache compressed data of data to be compressed corresponding to each component to generate partial decompressed data corresponding to each component;

the image processing method further comprises a second matrix point-by-point quantization calculation method, which comprises the following steps:

Generating a complete decompression quantization matrix table corresponding to each component according to a preset original quantization matrix table corresponding to each component, and performing point-by-point quantization calculation on the preset image processing basic unit through the complete decompression quantization matrix table corresponding to each component.

Further, the image processing method further includes:

and before carrying out point-by-point quantization calculation on the preset image processing basic unit, carrying out secondary matrix calculation on a data matrix corresponding to the preset image processing basic unit.

Further, the image processing method further includes:

after the decompressed data corresponding to each component are obtained, format conversion is carried out on the decompressed data corresponding to each component according to the data matrix corresponding to the preset image processing basic unit; wherein the decompressed data corresponding to each component includes one of: standard decompressed data, partial decompressed data.

Further, format conversion is performed on decompressed data corresponding to each component according to a data matrix corresponding to the preset image processing basic unit, and specifically includes:

setting the preset image processing basic unit as 8 x 8 image blocks; the format of the decompressed data corresponding to each component is converted into a format of 8 x 8block units.

Further, the image processing method further includes:

before bare data and data to be compressed corresponding to each component in the JPEG video image line data are acquired, the original RGB format of the JPEG video image line data is converted into a Y/Cb/Cr brightness and chromaticity separation format.

Further, the image processing method further includes:

and after carrying out point-by-point quantization calculation on the preset image processing basic unit, encoding and outputting data corresponding to each component.

Further, the method for encoding the data corresponding to each component specifically includes:

and performing coding processing on the data corresponding to each component, wherein the coding processing comprises at least one of the following steps: DPCM/RLE encoding and Huffman encoding.

Further, the image processing method further includes:

before data corresponding to each component is subjected to data stream output, header file information is added to the data corresponding to each component according to a JPEG standard format.

setting the first preset truncated data bit as 2 bits, and performing data compression on differential data bits corresponding to Y components in the JPEG video image line data to generate quantized difference values corresponding to the Y components;

Setting the second preset truncated data bit to be 3 bits, and performing data compression on differential data bits corresponding to Cb components in the JPEG video image line data to generate quantized difference values corresponding to the Cb components;

Further, generating a complete decompression quantization matrix table corresponding to each component according to a preset original quantization matrix table corresponding to each component specifically includes:

dividing the value of each position in the preset original quantization matrix table corresponding to the Y component by 4 to form a whole, and taking the whole value as the value of the corresponding position in the complete decompression quantization matrix table corresponding to the Y component;

dividing the value of each position in a preset original quantization matrix table corresponding to the Cb component by 8 to form a whole, and taking the whole value as the value of the corresponding position in a complete decompression quantization matrix table corresponding to the Cb component;

dividing the value of each position in the preset original quantization matrix table corresponding to the Cr component by 8 to form a whole, and taking the whole value as the value of the corresponding position in the complete decompression quantization matrix table corresponding to the Cr component.

Further, performing inverse quantization processing on line cache compressed data of data to be compressed corresponding to each component, which specifically includes:

Multiplying the quantized difference value corresponding to the Y component by 4 to serve as an inverse quantization intermediate parameter corresponding to the Y component;

multiplying the quantized difference value corresponding to the Cb component by 8 to serve as an inverse quantization intermediate parameter corresponding to the Cb component;

the quantized difference value corresponding to the Cr component is multiplied by 8 as an inverse quantized intermediate parameter corresponding to the Cr component.

The invention also provides an image processing device for implementing the image processing method, the image processing device comprises a line cache compression storage unit, which comprises:

a line cache compression unit, configured to: acquiring bare data and data to be compressed corresponding to each component in video image line data; generating quantized difference values and sign bits corresponding to the components according to the data to be compressed corresponding to the components; a line cache storage unit configured to: storing the bare data corresponding to each component into original data; and generating combined record data corresponding to each component according to the quantized difference value and the sign bit corresponding to each component, and taking the combined record data corresponding to each component as line cache compression data of data to be compressed corresponding to each component for compression storage.

The present invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the image processing method described above when executing the computer program.

The present invention further provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the image processing method described above.

Compared with the prior art, the technical scheme provided by the invention has the following technical effects:

in the invention, the original image data is compressed first; namely, separating data to be compressed corresponding to each component in video image line data, and generating quantized difference values and sign bits corresponding to each component according to the data;

then, caching the compressed image; namely, the bare data corresponding to each component is kept unchanged, and the original data is stored; combining the quantized difference value corresponding to each component with the sign bit to generate combined record data corresponding to each component, and taking the combined record data as line cache compression data of data to be compressed corresponding to each component for compression storage;

therefore, the original image data is compressed and encoded and then stored in the line cache, so that the mode of later reading out from the line cache and decompressing and restoring the image data into the image data is facilitated, the storage resources required by the line cache are reduced, the chip area and the power consumption are further reduced, and the time sequence is optimized.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a prior art JPEG image data compression process;

FIG. 2 is a flow chart of an image processing method according to a first embodiment of the invention;

FIG. 3 is a flow chart of a hardware implementation system of an image processing method in a practical embodiment of the invention;

FIG. 4 is a schematic diagram of a two-way parallel computing unit system in accordance with an exemplary embodiment of the present invention;

FIG. 5 is a schematic diagram of data classification of a single-pass computing unit according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of data classification of a multi-path computation unit according to an embodiment of the present invention;

FIG. 7 is a diagram of a matrix of original quantized coefficients of a luminance component according to an embodiment of the invention;

FIG. 8 is a schematic diagram of an original quantization coefficient matrix of a chrominance component in accordance with an embodiment of the present invention;

FIG. 9 is a flowchart of a hardware implementation system of an image processing method in the second embodiment of the present invention;

FIG. 10 is a flowchart of a hardware implementation system of an image processing method in the second embodiment of the present invention;

fig. 11 is a block diagram showing the structure of an image processing apparatus in the second embodiment of the present invention;

fig. 12 is an internal structure diagram of a computer device in the second embodiment of the present invention.

Detailed Description

In the prior art, JPEG video image compression is a common international standard for video image compression for compressing continuous tone still images (including gray scale images and color images). JPEG is an abbreviation for joint photographic experts group (Joint Photographic Experts Group), and this standard aims to support various continuous tone still image compression based applications. The image to be compressed can be any color space, and a user can enable the compression effect to reach or approach the compression performance of the center in the industry field by adjusting the compression ratio, and the image to be compressed has good analysis and reduction quality. Therefore, JPEG image compression is very necessary when a large number of images need to be stored and transmitted.

As shown in fig. 1, a classical JPEG image compression flow chart mainly comprises color mode conversion, DCT transformation, quantization, zig-zag arrangement, run-length encoding, huffman encoding, and data packing. The color mode conversion is to convert an RGB color mode into a YCbCr color mode. DCT transform, known as discrete cosine transform (DCT for Discrete Cosine Transform), is a common processing method in the field of video images, where the whole image is processed according to 8X8 small blocks before transformation, and 3 color channels of each small block are separately DCT transformed.

Furthermore, the essence of the quantization process is a process of optimizing the generation of an 8×8 coefficient matrix after discrete cosine transform. Since the human eye is insensitive to high frequency components, the goal of reducing the amount of data can be achieved by preserving a small amount or removing the high frequency portion of the 8 x 8 coefficient matrix. The actual purpose of the quantization process is to reduce the amplitude value of the non-zero frequency component coefficients and to increase the number of zero-valued frequency component coefficients. The actual implementation of the quantization procedure is to divide each coefficient value of the discrete cosine transform to generate an 8×8 coefficient matrix by a constant corresponding to the coefficient value (these coefficients are combined together to form a quantization parameter table), and round the obtained quotient value to obtain the quantized coefficient value. The quantization process is a lossy process, and the high frequency component parts of the quantized coefficient matrix are substantially near 0. The human eye is more sensitive to luminance than to chromatic aberration, so two quantization tables are often used in the quantization process: a luminance quantization table and a color difference quantization table.

Huffman coding, zig-zag arrangement, run length coding, data packing processes and modes are relatively fixed and will not be further described herein.

Note that FIFO, first Input First Output, means a first-in first-out technique;

SOC, system on Chip, refers to a System on Chip;

RAM, random Access Memory, refers to random access memory;

IC, integrated Circuit, refers to an integrated circuit;

IP, intellectual Property, refers to intellectual property;

DDR, double data Rate, is a Double Rate synchronous dynamic random Access memory;

JPEG, joint Photographic Experts Group, refers to the joint photographic experts group;

ISP, image Signal Processing, refers to image signal processing technology.

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Embodiment one:

as shown in fig. 2, an embodiment of the present invention provides an image processing method, where the image processing method includes a line cache compression storage method, which includes:

In a preferred embodiment, the acquiring bare data and data to be compressed corresponding to each component in video image line data specifically includes:

In a specific embodiment, the original image data is compressed first; separating data to be compressed corresponding to each component in JPEG video image line data, and generating quantized difference values and sign bits corresponding to each component according to the data;

In a practical embodiment, the line buffer compression unit is used for compressing original image data; the line cache storage unit is used for caching the compressed image, wherein the emphasis is on the compressed data storage mode.

In a preferred embodiment, generating the quantized difference value and the sign bit corresponding to each component according to the data to be compressed corresponding to each component specifically includes:

In a specific embodiment, during compression, a data classification, differential calculation, packet quantization and classification storage mode may be used, where the differential calculation is performed by using a difference between components corresponding to adjacent pixels.

The image processing method reduces the required storage space, reduces the resource consumption or the chip area, and simultaneously reduces the static power consumption which is reduced because the chip area is smaller;

meanwhile, in an actual video image, adjacent image values have small phase difference, and when the difference value is used for storage, the difference value only changes in a small range, so that the dynamic power consumption of the chip can be reduced.

In a preferred embodiment, the method separately performs packet quantization on the differential data bits corresponding to each component and generates a quantized difference value corresponding to each component, specifically includes:

according to the first preset data bit is removed, data compression is carried out on differential data bits corresponding to Y components in JPEG video image line data, and quantized difference values corresponding to the Y components are generated;

according to the second preset data bit is removed, data compression is carried out on differential data bits corresponding to Cb components in the JPEG video image line data, and quantized difference values corresponding to the Cb components are generated;

And according to the second preset data bit, carrying out data compression on the differential data bit corresponding to the Cr component in the JPEG video image line data, and generating a quantization difference value corresponding to the Cr component.

setting the first preset truncated data bit as 2 bits, and carrying out data compression on differential data bits corresponding to Y components in JPEG video image line data to generate quantized difference values corresponding to the Y components;

setting the second preset truncated data bit as 3 bits, and performing data compression on differential data bits corresponding to Cb components in JPEG video image line data to generate quantized difference values corresponding to the Cb components;

In particular embodiments, data classification + differential computation + packet quantization + classification storage may be used at compression.

At packet quantization, the sign bit is left unprocessed.

The differential data of the Y component and the Cb and Cr components are quantized in this step, respectively, because the sensitivity of the human eye to the Y chrominance component is much higher than to the chrominance component, and thus the accuracy (e.g., data bit width) of the chrominance component is properly reduced with little effect on the display.

In an actual embodiment, for the quantized difference y_diff_data_x corresponding to the Y component, the data bit with 2 bits lower is truncated, and the original 8-bit data is compressed into 6-bit data, which is effectively the original data/4 and the quotient.

For the quantized difference values corresponding to Cb and Cr components, the data bit with 3 bits lower is respectively removed, and the original 8-bit data is compressed into 5-bit data, which effectively is original data/8 and quotient taking.

The difference value and sign bit quantized for each pixel have been obtained by the foregoing calculation so as to be combined and recorded.

In a preferred embodiment, the acquiring bare data and data to be compressed corresponding to each component in the JPEG video image line data specifically includes:

for a single-path computing unit system, taking first column data of each row in JPEG video image row data as bare data and the rest all data as data to be compressed;

and for the multi-path computing unit system, grouping JPEG video image line data according to a preset image processing basic unit, wherein the first data of each group of data after grouping is used as bare data, and all the remaining data are used as data to be compressed.

In a specific embodiment, the image processing method is not only applicable to one-path computing unit system, but also applicable to multiple-path computing unit systems; for different computing unit systems, different image line compression caching methods are used.

In an actual embodiment, for a single-path computing unit, classifying first column data of each line of original image data into bare data, and all subsequent data are data to be compressed;

for a multi-path computing unit, firstly grouping each row of image data, wherein the grouping basis is a basic unit for subsequent image processing; in jpeg processing, the basic unit of processing is a matrix of 8×8, so each line of images is grouped into groups of 8 pixels, and then the first pixel of each group is recorded as bare data, and the other is data to be compressed.

And for bare data, directly storing the original data without subsequent processing, and entering the processing of subsequent steps for data to be compressed.

In a preferred embodiment, the image processing method further includes a first decompression method including:

the image processing method further includes a first matrix point-by-point quantization calculation method, which includes:

And carrying out point-by-point quantization calculation on the preset image processing basic unit according to the preset original quantization matrix table corresponding to each component.

In a preferred embodiment, performing inverse quantization processing on line cache compressed data of data to be compressed corresponding to each component specifically includes:

In a practical embodiment, the line buffer decompression unit is configured to restore the compressed image data to the original image data completely or partially.

For standard decompression, the method restores the compressed data to the original data.

For the data of the bare data position, the original data is stored, so the data is not processed after being read.

For data of other positions, firstly performing inverse quantization processing, a specific calculation method and a compression method, and multiplying 4 by 2 bits to obtain an intermediate parameter x_diff_temp; for compressing 3 bits, multiplying 8 to obtain an intermediate parameter x_diff_temp;

Y_diff_temp = Y_diff_Q_x * 4;

Cb_diff_temp = Cb_diff_Q_x * 8;

Cr_diff_temp = Cr_diff_Q_x * 8。

Furthermore, for jpeg algorithms, there is a matrix point-wise quantization calculation process itself after the DCT calculation. The quantization calculation process of the matrix point by point is different from the quantization process, wherein the quantization refers to the point by point quantization of the 8 x 8 matrix after DCT calculation, namely, the data of each position in the matrix has a corresponding quantization coefficient;

since the matrix point-by-point quantization step exists in the subsequent step, the quantization table in the subsequent matrix point-by-point quantization step may be changed without completely decompressing the previously compressed data in the line buffer decompression step.

Of course, if a standard decompression method is used in line cache decompression, the original quantization table is used here.

In a preferred embodiment, the image processing method further includes a second decompression method, which includes:

the image processing method further includes a second matrix point-by-point quantization calculation method, which includes:

generating a complete decompression quantization matrix table corresponding to each component according to a preset original quantization matrix table corresponding to each component, and performing point-by-point quantization calculation on a preset image processing basic unit through the complete decompression quantization matrix table corresponding to each component.

In a preferred embodiment, the generating a complete decompressed quantization matrix table corresponding to each component according to a preset original quantization matrix table corresponding to each component specifically includes:

In a practical embodiment, when the partial decompression method is used, the original image data has already undergone partial quantization calculation, so that when calculation is performed at this step, the original quantization tables y_qn, c_qn need to be processed, and quantization calculation is performed using the processed quantization tables. The method comprises the following steps:

Y_QN_D = Y_QN/4

C_QN_D = C_QN/8

the Y_QN_ D, Y _QN is a matrix of 8 x 8, and the value of each position of the Y_QN is divided by 4 to form a whole, and the whole is taken as the value of the corresponding position of the Y_QN_D; the c_qn_ D, C _qn is a matrix of 8×8, and the value of each position of c_qn is divided by 8 to form a whole, which is the value of the position corresponding to c_qn_d.

Therefore, the image processing method is applied to a JPEG system, and on the basis of reducing the resources required by line buffering, the calculation flow of a JPEG algorithm is optimized, the calculated amount is reduced, and the coding efficiency is improved.

In a preferred embodiment, the image processing method further includes:

after the decompressed data corresponding to each component is obtained, format conversion is carried out on the decompressed data corresponding to each component according to a data matrix corresponding to a preset image processing basic unit; wherein the decompressed data corresponding to each component includes one of: standard decompressed data, partial decompressed data.

In a preferred embodiment, format conversion is performed on decompressed data corresponding to each component according to a data matrix corresponding to a preset image processing basic unit, and specifically includes:

setting a preset image processing basic unit as 8 x 8 image blocks; the format of the decompressed data corresponding to each component is converted into a format of 8 x 8block units.

In an actual embodiment, after the decompressed data is read out from the line cache compression system, a data matrix in an 8×8 format is formed, and the data matrix enters subsequent DCT computation, where the DCT computation can be regarded as performing matrix computation for 2 times.

Wherein, 8 x 8 image blocks refer to: the image data format is converted into a format of 8×8block as a unit.

In a preferred embodiment, the image processing method further includes:

before bare data and data to be compressed corresponding to each component in JPEG video image line data are acquired, the original RGB format of the JPEG video image line data is converted into a Y/Cb/Cr brightness and chromaticity separation format.

In practical embodiments, the data format is typically RGB after the original image is input to the system, but the image is typically processed by calculating the Y/Cb/Cr format, so that it needs to be converted into the Y/Cb/Cr format, i.e. the format with luminance and chrominance separated.

In a preferred embodiment, the image processing method further includes:

after the point-by-point quantization calculation is carried out on the preset image processing basic unit, the data corresponding to each component is encoded and outputted in a data stream.

In a preferred embodiment, the data corresponding to each component is encoded, specifically including:

And carrying out coding processing on the data corresponding to each component, wherein the coding processing comprises at least one of the following steps: DPCM/RLE encoding and Huffman encoding.

In a preferred embodiment, the image processing method further includes:

In a practical embodiment, the subsequent DPCM/RLE encoding, huffman encoding and header file addition are performed after the point-wise quantization, and finally the code stream is outputted.

Adding header files refers to: header information is added according to the JPEG standard format.

As shown in fig. 3, in a flowchart of a hardware implementation system of an image processing method according to an actual embodiment of the present invention, an emphasis module functions as follows:

1. line cache compression system-line cache compression unit: for compressing the original image data;

2. line cache compression system-line cache storage unit: the method is used for caching the compressed image, and the emphasis is on the compressed data storage mode;

3. line cache compression system-line cache decompression unit: for fully or partially restoring the compressed image data to the original image data;

4. quantification calculation after optimization: the method is used for carrying out optimization processing based on the quantization processing process of line cache compression and JPEG.

In addition, the remaining modules function as follows:

Y/Cb/Cr calculation: the original data format is generally an RGB format, and when an image is encoded, the original data format needs to be converted into a format with separated brightness and chromaticity;

2.8 x 8 tiles: converting the image data format into a format taking 8 x 8block as a unit;

3. a calculation unit: including DCT (discrete cosine transform), DPCM (differential pulse coding), RLE (run length coding), huffman coding;

4. adding a header file: header information is added according to the JPEG standard format.

Accordingly, in the image processing method provided by the practical embodiment of the present invention, the specific implementation steps are as follows:

the image line cache compression system method is not only suitable for one-path computing unit system, but also suitable for multiple-path computing unit systems, and different image line compression cache methods are used for different systems.

Here, 1920×1080 resolution, rgb888 format image, one-way and two-way calculation units are taken as examples for illustration. This method is applicable to all standard resolutions and can be extended to more computational units according to this method.

As shown in fig. 4, when the 2-way calculation unit is used, the image jpeg calculation of block-1 and block-2 can be performed simultaneously, and the efficiency can be improved by parallel calculation.

Step 1: image format conversion

After the original image is input into the system, the data format is usually RGB format, but the Y/Cb/Cr format is usually calculated during image processing, so that the Y/Cb/Cr format needs to be converted into a Y/Cb/Cr format, namely a format with separated brightness and chromaticity.

Step 2: line cache compression

The data classification, differential calculation, grouping quantization and classification storage modes are used in compression, and different processing methods are used for the single-path and multi-path calculation units.

Data classification:

as shown in fig. 5, for the single-path calculation unit, the first column data of each line of the original image data is classified as bare data, and all the subsequent data are data to be compressed;

as shown in fig. 6, for the multi-path calculation unit, each line of image data is first grouped according to the basic unit of the subsequent image processing; in jpeg processing, the basic unit of processing is a matrix of 8×8, so each line of images is grouped into groups of 8 pixels, and then the first pixel of each group is recorded as bare data, and the other is data to be compressed.

And a differential calculation step:

the differential calculation is carried out by using the mode of carrying out the difference on the components corresponding to the adjacent pixels, and the processing modes of the Y, cb and Cr components are the same; taking the Y component as an example, the following description will be made:

Y_diff_1 = Y1,

Y_diff_2 = Y2-Y1,

Y_diff_3 = Y3-Y2,

Y_diff_4 = Y4-Y3,

Y_diff_5 = Y5-Y4,

……

the calculation of each row is completed with such a push. In the same way, the differential values of Cb and Cr are also calculated and denoted as cb_diff_x and cr_diff_x.

When differential calculation is used, the calculated result is 1bit more sign bit than the original data bit, and the 8bit component is 9bit after calculation, namely the result range is between-255 and 255.

For Y_diff_x, cb_diff_x and Cr_diff_x, the highest bit is a sign bit, and the highest bit is respectively marked as Y_flag_x, cb_flag_x and Cr_flag_x;

the data bit is denoted y_diff_data_ xCb _diff_data_xcr_diff_data_x; namely:

Y_flag_x = Y_diff_x[8],

Y_diff_data_x = Y_diff_x[7:0]；

Cb_flag_x = Cb_diff_x[8],

Cb_diff_data_x = Cb_diff_x[7:0]；

Cr_flag_x = Cr_diff_x[8],

Cr_diff_data_x = Cr_diff_x[7:0]

and a grouping quantization step:

in the packet quantization, the processing method is the same for the differential data in the single-path and multi-path calculation units, and the symbol bit is reserved and not processed.

In the example, the data bit width of the image difference data used is 8 bits, i.e., in the range of 0 to 255. The differential data of the Y component and the Cb and Cr components are quantized in this step, respectively, because the sensitivity of the human eye to the Y chrominance component is much higher than to the chrominance component, and thus the accuracy (e.g., data bit width) of the chrominance component is properly reduced with little effect on the display.

For Y_diff_data_x, the data bit with 2 bits lower is omitted, the original 8-bit data is compressed into 6-bit data, and the original data is effectively original data/4 and quotient taking.

For example: the original data value is 203 and the compressed value is 50. For the bit number of data compression, 1, 2 or 3 bits can be generally taken, and the bit number can be selected according to the actual application scene and the subsequent processing flow. In the additional example, 2 bits are taken:

Y_diff_Q_x = Y_diff_data_x[7:2]

for Cb and Cr components, the lower 3bit data bit is removed, the original 8bit data is compressed into 5bit data, and the original data/8 and quotient are obtained effectively.

For example: the original data value is 203 and the compressed value is 25. For the bit number of data compression, 2 or 3 or 4 bits can be generally taken, and the bit number can be selected according to the actual application scene and the subsequent processing flow. In the additional example, take 3 bits:

Cr_diff_Q_x = Cr_diff_data_x[7:3]

step 3: line cache storage

And a classification storage step:

through the foregoing calculation, the difference value and sign bit quantized for each pixel have been obtained, and the combination thereof is recorded as:

Y_com_x={Y_flag_x，Y_diff_Q_x}

Cb_com_x={ Cb_flag_x，Cb_diff_Q_x}

Cr_com_x={ Cr_flag_x，Cr_diff_Q_x}

for the bare data locations described above (i.e., blue location data in fig. 5 and 6), storing the original data; for the pixels of the data location to be compressed (i.e. white location data in fig. 5, 6), y_com_x, cb_com_x, cr_com_x are stored.

Through the data compression of the steps, the data quantity required to be stored in each row is reduced; therefore, in the process of hardware design, the corresponding line cache storage resource capacity is correspondingly reduced, so that the area of a chip is reduced, and meanwhile, the power consumption and the time sequence are optimized. The specific memory cell space calculation mode is as follows:

in the prior art, for the existing common technical scheme, each row stores the required space: 24bit x 1920= 46080 bits;

after being compressed by the method, the technology is improved as follows:

when the unit is calculated in a single way, the space required by each row is as follows: 19×1919+24= 36485bit

The space required for each row when computing the units in multiple ways: 19×1680+24×240= 37680 bits

It can be seen that by this method the required memory space is reduced, the resource consumption or chip area is reduced, and the static power consumption, which is reduced because of the smaller chip area, is also reduced.

Meanwhile, in an actual video image, the adjacent image values have small phase difference, and when the difference value is used for storage, the difference value only changes in a small range, so that the dynamic power consumption of the chip can be reduced.

Step 4: line cache decompression

After the data is read out from the line cache unit, line cache data decompression is needed, and the line cache data decompression can be divided into standard decompression and partial decompression based on a subsequent algorithm flow.

Standard decompression step:

the method restores the compressed data to the original data.

Y_diff_temp = Y_diff_Q_x * 4;

Cb_diff_temp = Cb_diff_Q_x * 8;

Cr_diff_temp = Cr_diff_Q_x * 8;

the sign bit is then combined with the intermediate parameter combination value:

Y_diff_temp_f = {Y_flag_x ,Y_diff_temp}

Cb_diff_temp_f = {Cb_flag_x , Cb_diff_temp}

Cr_diff_temp_f = { Cr_flag_x , Cr_diff_temp}

and then restored to the original pixel by the method of adding to the previous pixel value, for example:

Y1= Y1,

Y2= Y_diff_temp_f_2 +Y1,

Y3= Y_diff_temp_f_3 +Y2,

and so on to complete decompression of all data. For a single-pass computing system, only the first pixel of each row is the original value; for a multi-way computing system, the first pixel of each group is the original value, and no computation is required.

Partial decompression step based on the following algorithm flow:

it should be noted that, the decompression method is related to the subsequent algorithm flow;

that is, for the jpeg algorithm, there is a "matrix point-wise quantization calculation" process itself after the DCT calculation. The process of the 'matrix point-by-point quantization calculation' is different from the quantization process, wherein the quantization refers to the point-by-point quantization of the 8 x 8 matrix after DCT calculation, namely, the data of each position in the matrix has a corresponding quantization coefficient;

As shown in fig. 7, for the luminance component, the original quantization coefficient matrix is denoted as y_qn, and the matrix data is detailed in the figure;

as shown in fig. 8, for the chrominance components (including Cr, cb), the original quantization coefficient matrix is denoted as c_qn, and the matrix data is detailed in the figure.

Since the "matrix point-by-point quantization calculation" step exists in the subsequent step, the quantization table in the subsequent "matrix point-by-point quantization calculation" step may be changed without completely decompressing the previously compressed data in the line buffer decompression step. The method comprises the following steps:

the method comprises the steps of carrying out quantization processing on raw data of a bare data position, wherein the specific method is the same as a processing method of data to be compressed in a line cache compression step; in the example, 2 bits are compressed for the Y component and 3 bits are compressed for the Cr and Cb components; after this processing, all the data becomes compressed data;

and then the data is restored by adding the front value pixel and the difference value, the restored data are marked as Y_com_tol, cb_com_tol and Cr_com_tol, and at the moment, the data are the data after 2bit quantization and 3bit quantization and are not the original data, and only partial decompression is completed.

Step 5: DCT computation

After the decompressed data is read out from the line cache compression system, a data matrix with an 8 x 8 format is formed, and the data matrix enters subsequent DCT (discrete cosine transform) calculation, wherein the DCT calculation can be regarded as 2 matrix calculations on the data matrix.

Step 6: optimized quantization calculation/matrix point-by-point quantization calculation

In the foregoing, when the line cache is decompressed, a partial decompression method based on a subsequent algorithm flow can be selected; when this method is used, the original image data has already undergone partial quantization calculation, and therefore, when calculation is performed at this step, it is necessary to process the original quantization tables y_qn, c_qn, and quantization calculation is performed using the processed quantization tables. The method comprises the following steps:

Y_QN_D = Y_QN/4

C_QN_D = C_QN/8

If a standard decompression method is used in line cache decompression, the original quantization table is used here.

Step 7: encoding and data stream output

And after quantization, carrying out subsequent DPCM/RLE coding, huffman coding and header file addition, and finally outputting a code stream.

In summary, the implementation method of the video image line cache compression processing reduces the storage resources required by the line cache by the way of performing compression encoding on the original image data, storing the original image data into the line cache, reading the original image data from the line cache, decompressing the original image data into the image data, further reducing the chip area and the power consumption, and optimizing the time sequence.

Meanwhile, the application mode of the line cache compression method in the JPEG system optimizes the calculation flow of the JPEG algorithm on the basis of reducing the resources required by the line cache, reduces the calculated amount and improves the coding efficiency.

The implementation method of the video image line cache compression processing can reduce hardware storage resources of the image line cache, reduce chip area and power consumption, and can be applied to various fields of video images (such as image coding, ISP (Internet service provider), image scaling and the like).

It should be noted that, although the steps in the flowchart are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least a portion of the steps in the flowcharts may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order in which the sub-steps or stages are performed is not necessarily sequential, and may be performed in turn or alternately with at least a portion of the sub-steps or stages of other steps or other steps.

Embodiment two:

as shown in fig. 9, the image processing method provided by the practical embodiment of the invention can also be used for h.264/AVC and H265/HEVC coding standards.

H.264 is a highly compressed digital Video codec standard proposed by the Joint Video Team (JVT) consisting of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG) Joint. This standard is commonly referred to as H.264/AVC (either AVC/H.264 or H.264/MPEG-4 AVC or MPEG-4/H.264 AVC).

H.265 is a new video coding standard established by ITU-T VCEG followed by h.264. The h.265 standard surrounds the existing video coding standard h.264, retaining some of the original techniques, while improving some of the related techniques. The new technology uses advanced technology to improve the relation among code stream, coding quality, delay and algorithm complexity, so as to achieve the optimal setting. This standard is commonly referred to as h.265/HEVC.

For the H.264/AVC and H265/HEVC coding standards, FIG. 9 is a schematic flow chart of the image processing method; since the coding structure of H265 is substantially the same as H264, the same structure may be used, except that the specific processing manner of each sub-module is slightly different.

The line cache compression system is the same as that described in the JPEG scheme; in a line buffer decompression unit of the line buffer compression system, the compressed image data is completely restored to the original image data.

Other sub-modules in the figures are common techniques for h.264 and h.265, including inter-prediction compression (motion estimation, motion compensation), intra-prediction and mode selection, transformation, quantization, inverse transformation, inverse quantization, deblocking filtering, entropy coding, and so on.

Embodiment III:

as shown in fig. 10, the image processing method provided in the practical embodiment of the present invention can also be used in the JPEG-LS encoding standard.

The JPEG-LS standard is the ISO-14495-1 standard published by the International organization for standardization ISO at 12, 1999, and the ITU-T.87 standard published by the International telecommunication Union ITU at 6, 1998. JPEG-LS is a lossless and near-lossless compression algorithm based on prediction, and uses fixed prediction, golomb coding and run-length coding to realize lossless compression and near-lossless compression of images.

FIG. 10 is a flow chart of a line buffer compression system for the JPEG-LS encoding standard.

The JPEG-LS encoder firstly carries out context modeling on source image data, and then carries out coding mode selection according to local gradient values of pixels to be coded; if the local gradient values are all zero, performing run-length coding on the local gradient values; otherwise, normally encoding the code; and outputting the coded code stream after the coding is finished.

It should be noted that, when JPEG or H264/H265 coding, it is necessary to generate 8×8 or 16×16 image blocks for subsequent processing, so at least 8 or 16 line buffering systems are required; whereas the JPEG-LS standard only requires 2 lines of images to be correlated when computing local gradients, thus only requiring 2 line buffer systems.

The JPEG-LS standard is not a modification and supplement to the JPEG standard, but is a brand new standard which adopts a brand new architecture and a brand new algorithm.

Embodiment four:

as shown in fig. 11, an embodiment of the present invention further provides an image processing apparatus for implementing the foregoing image processing method, where the image processing apparatus includes a line cache compression storage unit, which includes:

In a preferred embodiment, the line cache compression unit comprises:

a differential value generation unit configured to: respectively performing difference calculation on each component of adjacent pixels in the JPEG video image line data to generate a difference value corresponding to each component;

a sign bit and difference data bit generation unit for: taking the highest bit of the differential value corresponding to each component as the sign bit corresponding to each component, and taking the rest bit of the differential value corresponding to each component as the differential data bit corresponding to each component;

a quantized difference generating unit configured to: and respectively carrying out grouping quantization on the differential data bits corresponding to each component, and generating quantized difference values corresponding to each component.

In a preferred embodiment, the quantized-difference-value generating unit is further configured to:

In a preferred embodiment, the line cache compression unit is further configured to:

In a preferred embodiment, the image processing apparatus further comprises a first decompression unit for:

the image processing apparatus further includes a first matrix point-by-point quantization calculation unit configured to:

In a preferred embodiment, the image processing apparatus further comprises a second decompression unit for:

the image processing apparatus further includes a second matrix point-by-point quantization calculation unit configured to:

In a preferred embodiment, the image processing apparatus further comprises a DCT computation unit for:

In a preferred embodiment, the image processing apparatus further comprises a decompressed data format conversion unit for:

In a preferred embodiment, the decompressed data format conversion unit is further adapted to:

In a preferred embodiment, the image processing apparatus further comprises an original RGB format converting unit for:

In a preferred embodiment, the image processing apparatus further comprises an encoding and data stream output unit for:

In a preferred embodiment, the encoding and data stream output unit is further adapted to:

In a preferred embodiment, the image processing apparatus further includes an add header file unit for:

In a preferred embodiment, the second matrix point-wise quantization calculation unit is further configured to:

In a preferred embodiment, the first decompression unit is further configured to:

For specific limitations of the above apparatus, reference may be made to the limitations of the method described above, which are not repeated here.

Each of the modules in the above apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware, or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

The computer device may be a terminal, as shown in fig. 12, which includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external terminal through a network connection. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It is to be understood that the structures shown in the above figures are merely block diagrams of some of the structures associated with the present invention and are not limiting of the computer devices to which the present invention may be applied, and that a particular computer device may include more or less components than those shown, or may combine some of the components, or have a different arrangement of components.

Implementation of all or part of the flow in the above-described embodiment methods may be accomplished by a computer program that instructs related hardware, and the computer program may be stored in a non-volatile computer readable storage medium, and the computer program may include the flow in the above-described embodiment methods when executed.

Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.

It should be noted that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.

Claims

1. An image processing method, characterized in that the image processing method comprises a line cache compression storage method, comprising:

acquiring bare data and data to be compressed corresponding to each component in video image line data; generating quantized difference values and sign bits corresponding to the components according to the data to be compressed corresponding to the components;

storing the bare data corresponding to each component into original data; generating combined record data corresponding to each component according to the quantized difference value and the sign bit corresponding to each component, and taking the combined record data corresponding to each component as line cache compression data of data to be compressed corresponding to each component for compression storage;

The method for generating the quantized difference value and the sign bit corresponding to each component according to the data to be compressed corresponding to each component specifically comprises the following steps:

and carrying out data compression on differential data bits corresponding to each component in the JPEG video image line data according to preset data bit discarding, and generating quantized difference values corresponding to each component.

2. The image processing method according to claim 1, wherein acquiring bare data corresponding to each component in video image line data and data to be compressed specifically comprises:

3. The image processing method according to claim 2, wherein the generating the quantized difference value corresponding to each component by performing data compression on the differential data bit corresponding to each component in the JPEG video line data according to the preset truncated data bit specifically includes:

4. The image processing method according to claim 3, wherein acquiring the bare data and the data to be compressed corresponding to each component in the JPEG video line data specifically comprises:

5. The image processing method according to claim 4, further comprising a first decompression method comprising:

6. The image processing method according to claim 5, further comprising a second decompression method comprising:

7. The image processing method according to claim 5 or 6, characterized in that the image processing method further comprises:

8. The image processing method according to claim 5 or 6, characterized in that the image processing method further comprises:

9. The image processing method according to claim 8, wherein format conversion is performed on decompressed data corresponding to each component according to a data matrix corresponding to the preset image processing basic unit, and specifically comprising:

10. The image processing method according to claim 9, characterized in that the image processing method further comprises:

11. The image processing method according to claim 10, characterized in that the image processing method further comprises:

12. The image processing method according to claim 11, wherein the encoding of the data corresponding to each component specifically includes:

13. The image processing method according to claim 11, characterized in that the image processing method further comprises:

14. The image processing method according to claim 3, wherein the step of respectively performing packet quantization on the differential data bits corresponding to the respective components and generating quantized difference values corresponding to the respective components, comprises:

15. The method of image processing according to claim 6, wherein generating the fully decompressed quantization matrix table corresponding to each component according to the preset original quantization matrix table corresponding to each component, specifically comprises:

16. The image processing method according to claim 5, wherein performing dequantization processing on line buffer compressed data of data to be compressed corresponding to each component specifically comprises:

17. An image processing apparatus for implementing the image processing method according to any one of claims 1 to 16, the image processing apparatus comprising a line cache compression storage unit comprising:

a line cache compression unit, configured to: acquiring bare data and data to be compressed corresponding to each component in video image line data; generating quantized difference values and sign bits corresponding to the components according to the data to be compressed corresponding to the components;

a line cache storage unit configured to: storing the bare data corresponding to each component into original data; and generating combined record data corresponding to each component according to the quantized difference value and the sign bit corresponding to each component, and taking the combined record data corresponding to each component as line cache compression data of data to be compressed corresponding to each component for compression storage.

18. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the image processing method according to any of claims 1-16 when the computer program is executed.

19. A computer-readable storage medium storing a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the image processing method according to any one of claims 1-16.