CN102200947A - Memory management method and related memory device - Google Patents

Abstract

The invention provides a memory management method and a related memory device, wherein the memory management method comprises the following steps: capturing data corresponding to a plurality of image blocks, wherein the image blocks comprise at least two image blocks with different block sizes; and storing data corresponding to the plurality of image blocks by using a memory with a plurality of memory groups. The technology provided by the invention can enable the memory to more efficiently buffer the data of the dynamic blocks with variable sizes.

Description

Memory management method and related memory device

technical field

The present invention relates to memory management, especially a memory management method and a related memory device, which are used to efficiently configure a buffer memory to store dynamic blocks in image compression technology.

Background technique

Motion compensation is a very important part of image compression and image decompression technology. Among various motion compensation technologies, motion compensation with variable block size is most commonly used in popular applications such as H.264 and MPEG-4. image compression technology. This motion compensation technique utilizes motion blocks to record relative motion vectors. In the process of restoring compressed image files, the data corresponding to the dynamic blocks must be loaded for subsequent calculation and processing. However, because the data corresponding to the dynamic block is loaded from the data source (such as: dynamic random access memory) at a speed different from the speed at which it is processed, the prior art fetches data corresponding to multiple dynamic blocks in advance. block data, and store the data in a buffer memory, and then perform a subsequent image processing operation (such as a filtering operation) on the data.

In the technology of dynamic compensation of variable block size, the data sizes corresponding to different dynamic blocks may be different. Therefore, the width of the buffer memory (bus width) will be limited by the data width of the largest dynamic block in the dynamic blocks. Therefore, when the buffer memory with a limited width stores small dynamic blocks, a part of the space will be wasted. Please refer to FIG. 1 , which illustrates the situation where the space of the buffer memory is wasted in the prior art. As shown in the figure, the dynamic blocks A, B, and C respectively include 21x13 pixels, 9x9 pixels, and 9x13 pixels, and are respectively stored in buffer memories 101, 102, and 103 with the same capacity. The data size of dynamic block A. As shown in the figure, except that the capacity of the buffer memory 101 for storing the dynamic block A is exactly the same as the data volume of the dynamic block A, considerable storage space in the other buffer memories 102 and 103 is wasted. Furthermore, speaking from the other side, each of the buffer memories 101, 102, and 103 must wait until the data stored in it has been read out for subsequent operations during data buffering. After the processing is completed, its space can be released, and then continue to store other dynamic blocks. Obviously, quite a lot of time is needlessly wasted in the data buffering operation of the dynamic block in the prior art, so there are still many deficiencies that need to be improved urgently in the management and configuration of the existing buffer memory.

Contents of the invention

In view of this, the present invention provides a memory management method and a related memory device, which can more efficiently configure a buffer memory for data buffering of image blocks (especially dynamic blocks with variable block sizes), Furthermore, the situation of wasting storage space of the buffer memory in the prior art is improved. In addition, the present invention can also improve the time wasting situation in the existing data buffering operation.

According to an embodiment of the present invention, a memory management method is provided. The memory management method includes: retrieving data corresponding to a plurality of image blocks, wherein at least two image blocks with different block sizes are included in the plurality of image blocks; ) to store data corresponding to the plurality of image blocks.

According to another embodiment of the present invention, a memory device is provided, and the memory device includes: a memory, a capture unit, and a configuration unit. The memory contains multiple memory banks. The capturing unit is used for capturing data corresponding to a plurality of image blocks, wherein the plurality of image blocks include at least two image blocks with different block sizes. The configuration unit is coupled to the capture unit and the memory, and is used for using the memory to store data corresponding to the plurality of image blocks.

Preferably, the plurality of image blocks are dynamic blocks, and data corresponding to the plurality of dynamic blocks is retrieved according to different motion vectors used in an image processing operation.

Preferably, data corresponding to different columns in an image block of the plurality of image blocks is stored in a space of the same column address in different memory groups of the plurality of memory groups.

Preferably, the number of different columns within the image block is greater than the number of different memory banks.

Preferably, the data corresponding to a column in an image block of the plurality of image blocks is stored in a space of the same column address in different memory groups of the plurality of memory groups.

Description of drawings

FIG. 1 is a schematic diagram of an existing memory configuration method.

FIG. 2 is a schematic diagram of a first embodiment of the memory management method of the present invention.

FIG. 3 is a schematic diagram of a second embodiment of the memory management method of the present invention.

FIG. 4 is a schematic diagram of a third embodiment of the memory management method of the present invention.

FIG. 5 is a schematic diagram of a fourth embodiment of the memory management method of the present invention.

FIG. 6 is a schematic diagram of the relationship between a dynamic block and its corresponding data.

FIG. 7 is a schematic diagram of an embodiment of a memory device of the present invention.

[Description of main component symbols]

200, 300, 400, 500, 712 memory

700 Image processing system

710 memory device

714 Extraction unit

716 hive

720 Image processing unit

730 cache device

740 data source

Detailed ways

According to an embodiment of the present invention, a memory management method used in an image processing operation includes: retrieving data corresponding to a plurality of image blocks (eg, dynamic blocks), wherein the plurality of image blocks including at least two image blocks with different block sizes; and using a memory with a plurality of memory banks to store data corresponding to the plurality of image blocks. The image processing operation may be a filtering operation of a variable block size motion compensation technique. Generally speaking, the filtering operation may utilize data including a dynamic block and part of data of neighboring dynamic blocks adjacent to the dynamic block for processing. Therefore, in the subsequent description, the so-called "data corresponding to a dynamic block" actually includes the data of the dynamic block itself and part of the data of other adjacent dynamic blocks. Since the data corresponding to the dynamic block is used in the dynamic compensation technique of variable block size, the dynamic block may have different sizes. In addition, the data corresponding to the dynamic block may be related to the motion vector to be processed in the filtering process, and, based on different image processing architectures, the data may be stored in a data source (such as DRAM) retrieved, or retrieved from a cache device. Both ways can be used in the memory management method of the present invention.

In order to use the memory with the memory banks to store data corresponding to the dynamic blocks, the present invention provides several different memory configurations. These memory configurations vary with the height and width of each dynamic block. Therefore, the following description will be combined with several different embodiments to explain the memory configuration method included in the present invention. However, it should be noted that these configuration methods may be used individually or in combination. These changes all belong to the category of the present invention.

Please refer to FIG. 2 , which shows how to store the data of the dynamic block when the data width of a row (ie, a pixel row) of the dynamic block is larger than the memory width corresponding to a column address in the memory bank. As shown in the figure, assuming that each column of the dynamic block A contains 21 horizontally arranged pixels, and each pixel contains 8 bits of data, at this time, the total data width of each column will be 168 bits. If the memory 200 is composed of memory bank 0 and memory bank 1, and one of them has a width of 80 bits, and the other has a width of 88 bits, then in the data captured in the previous step, corresponding to dynamic The data of one column in the block will be stored in the space of the same column address in different memory banks. For example, the data corresponding to the first column of the dynamic block A will be stored in the space of the column address 0 in the memory bank 0 and the space of the column address 0 in the memory bank 1 respectively. The data corresponding to the second column of the dynamic block A will be respectively stored in the memory bank 0 space with the column address 1, and the memory bank 1 space with the column address 1, and so on. Therefore, in this embodiment, the data corresponding to each column of the dynamic block A is just stored in the space of the same column address of the memory banks 0 and 1 . In addition, in this embodiment, when the dynamic block A is stored in the memory 200, no space will be wasted.

Please refer to FIG. 3 , which illustrates how to store the data of the dynamic block when the data width of each column in the dynamic block is close to the memory width corresponding to a column address in the memory bank. As shown in the figure, assuming that each column of the dynamic block B contains 11 pixels arranged horizontally, and each pixel contains 8 bits of data, at this time, the total data width of each column will be 88 bits. If the memory 300 is composed of memory bank 0 and memory bank 1, and one of them has a width of 80 bits and the other has a width of 88 bits, then the retrieved data corresponding to different columns in the dynamic block C will be are stored in the space of the same column address in different memory banks. For example, the data corresponding to the first row of the dynamic block B will be stored in the memory bank 0, and the data corresponding to the second row of the dynamic block B in the space whose column address is 0 will be stored In memory group 1, the data corresponding to the third column of dynamic block B in the space with column address 0 will be stored in memory group 0, in the space with column address 1, and corresponding to the dynamic block The data of the 4th column of B will be stored in memory group 1, in the space whose column address is 1, and so on. Wherein, in the memory bank 1, the space whose column address is 2 will be reserved, or used to store data corresponding to other dynamic blocks. In another example of FIG. 3, the data corresponding to each column in the dynamic block C will be symmetrically and separately stored in memory banks 0 and 1, and there will be no space remaining in memory bank 0 or memory in Group 1.

Please refer to FIG. 4 again, which illustrates how to store the data of the dynamic block when the data width of each column in the dynamic block is smaller than the memory width corresponding to a column address in the memory bank. As shown in the figure, assuming that each column of the dynamic block D contains 3 pixels arranged horizontally, and each pixel contains 8 bits of data, at this time, the total data width of each column will be 24 bits. If the memory 400 is composed of a memory bank 0 and a memory bank 1, and one of them has a width of 80 bits and the other has a width of 88 bits, then the retrieved data corresponding to different columns in the dynamic block D will be retrieved. Stored in the space of the same column address in the same memory bank. For example, the data corresponding to the 1st, 2nd, 3rd and 4th columns of the dynamic block D will all be stored in the space of the column address 0 in the memory bank 0, and some free space will be left in this space (ie, 8 bits). And if the data corresponding to the 1st, 2nd, 3rd and 4th column of the dynamic block D are all stored in the memory group 1, when the column address is 0 in the space, then at this time, there will be 16 bits remaining in the memory space of available space. Therefore, when the memory configurations shown in FIGS. 2, 3 and 4 are combined to store dynamic blocks of different sizes, the buffer memory can be used more efficiently and with less waste than the prior art Space.

Please refer to FIG. 5 , which illustrates how to store data corresponding to a dynamic block in a memory with more than two memory banks in a memory configuration different from the above-mentioned embodiments. As shown in the figure, assuming that each column of the dynamic block E contains 11 pixels arranged horizontally, and each pixel contains 8 bits of data, at this time, the total data width of each column will be 88 bits. If the memory 500 is composed of memory group 0, memory group 1, memory group 2, and memory group 3, and the width of each memory group is 88 bits, then the retrieved data corresponding to different columns in the dynamic block E will be stored in the space of the same column address in different memory banks. For example, the data corresponding to the 1st, 4th, 7th and 10th columns of the dynamic block E will be stored in memory banks 0, 1, 2 and 3 in the space with column address 0, and so on. It can be seen from this embodiment that the present invention has no limitation on how to configure how to store each row of data in the dynamic block and the number of memory banks included in the used memory. At the same time, the present invention is not limited to the width of each memory bank and the data length of each column of the dynamic block, so they may vary with different practices. Therefore, the specific numbers quoted in the foregoing paragraphs are only for explaining the concept of the present invention, rather than limiting the present invention.

In addition, because the filtering operation in the dynamic compensation of variable block size may simultaneously process the data of a dynamic block and part of the data of adjacent blocks around the dynamic block, so in the method of the present invention, the extraction corresponding to The step of data in the dynamic block may result in repeated retrieval of the same data. For further explanation, please refer to FIG. 6 . As shown, FIG. 6 shows 5 adjacent dynamic blocks. In fact, the so-called “retrieving data corresponding to the dynamic block A” in the present invention means retrieving data corresponding to the dynamic block surrounded by the dotted line area 600 . Wherein, in the step of retrieving the data corresponding to the dynamic block A, part of the data of the dynamic blocks B, C, D and E will also be retrieved together (that is, those surrounded by the dotted line area 600 place). Therefore, in order to avoid retrieving the same data from the data source repeatedly, the present invention uses a cache device to store previously retrieved data. Accordingly, before retrieving the data corresponding to a dynamic block, the present invention first checks whether at least a part of the data corresponding to the dynamic block has been stored in the cache device. And, when a part of the data corresponding to the dynamic block has been stored in the cache device, the part of the data corresponding to the image block will be retrieved in the cache device, and stored in the cache device Retrieving the remaining part of data corresponding to the dynamic block (that is, part of the data not stored in the cache device) from the data source, and storing the remaining part of the data corresponding to the dynamic block in the cache device. Furthermore, when there is no data corresponding to the dynamic block stored in the cache device, the present invention will retrieve the data corresponding to the dynamic block from the data source, and will correspond to the dynamic block The data is stored in the cache device.

In addition, since the frequency of sending addresses and instructions when capturing data will cause a burden on the image processing system (especially for the data source), in the present invention, the images corresponding to the plurality of image blocks are continuously captured. A time interval between two image blocks is adjustable. More specifically, the present invention dynamically adjusts the time interval according to the delay state of the data source. If the delay state of the data source is serious, the data of the dynamic block must be retrieved more frequently. Otherwise, the speed at which data is captured will lag behind the speed at which data is processed. Therefore, in the case of severe delay, the time interval for continuously capturing data corresponding to two dynamic blocks will be shortened, and in the case of slight delay, the time interval will be extended. The above operations can be achieved by sending a delay signal through a memory interface accessing the data source.

Based on the above memory management method, the present invention accordingly provides a memory device. Please refer to FIG. 7 , which is a functional block diagram of a memory device of the present invention applied to an image processing system. As shown in the figure, the image processing system 700 includes (but is not limited to) a memory device 710 , an image processing unit 720 , a cache device 730 and a data source 740 . Image processing system 700 may be part of a motion compensation system in an image decompression architecture that may be used to process H.263, H.264, MPEG-4 AVC or VC-1 formats multimedia files. In detail, the image processing unit 720 may be used to perform a filtering operation on dynamic blocks with variable block sizes, wherein the data of the dynamic blocks comes from the data source 740 (eg, DRAM). In other words, data of dynamic blocks with different sizes may be stored in the data source 740 and processed by the image processing unit 720 . Before the image processing unit 720 processes the data corresponding to a plurality of dynamic blocks, these data will be temporarily stored in the memory device 710 of the present invention, because the speed at which data is read from the data source 740 is usually not the same as the data being read. The processing speed of the image processing unit 720 varies.

The memory device 710 is used for storing data corresponding to a plurality of dynamic blocks. The memory device 710 includes a memory 712, a capture unit 714 and a configuration device. The memory 712 has multiple memory banks and is used to store data corresponding to dynamic blocks with different sizes. The retrieving unit 714 is used for retrieving data corresponding to the plurality of dynamic blocks. In detail, the retrieval unit 714 may transmit addresses and instructions to the data source 740 or the cache device 730 to obtain relevant data according to the dynamic blocks corresponding to different dynamic vectors. The configuration unit 716 is coupled to the capture unit 714 and the memory 712, and utilizes (or in other words: configures) the space of the memory 712 to store data corresponding to the plurality of dynamic blocks. Since the method of configuring the memory 712 by the configuration unit 716 has been described in detail previously, it will not be repeated here. In addition, the cache device 730 (which may be included in a memory interface) is used for caching data during the process of moving data from the data source 740 to the memory 712 .

In fact, the "data corresponding to a dynamic block" used by the image processing unit 720 usually exceeds the data content contained in the dynamic block itself, which is a common technique in motion compensation operations. Furthermore, in this technique, the data processed by the image processing unit 720 includes data of the dynamic block itself and partial data of adjacent dynamic blocks adjacent to the dynamic block. Accordingly, when the data corresponding to a dynamic block is retrieved, usually some data of adjacent dynamic blocks will also be retrieved together. Therefore, when the retrieval unit 714 retrieves data from the data source 740 , the previously retrieved data may be retrieved repeatedly, so the cache device 730 is designed to avoid repeated access to the data source 740 .

Before the retrieval unit 714 retrieves the data corresponding to a dynamic block from the data source 740 or the cache device 730, the retrieval unit 714 will first check whether at least a part of the data corresponds to the dynamic block has been stored in the cache device 730. According to this, if the retrieval unit 714 finds that part of the data corresponding to the dynamic block has been stored in the cache device 730, the retrieval unit 714 will retrieve the data corresponding to the dynamic block from the cache device 730 The partial data of the block and a remaining partial data corresponding to the dynamic block are retrieved from the data source 740 . Finally, the remaining data corresponding to the dynamic block will be stored in the cache device 730 together. Conversely, if the fetching unit 714 finds that there is no data corresponding to the dynamic block stored in the cache device 730, the fetching unit 714 will fetch the data corresponding to the dynamic block from the data source 740, and corresponding The data in the dynamic block will be stored in the cache device 730 together.

In addition, because the frequency of sending addresses and commands when the capture unit 714 captures data will cause a burden on the image processing system 700 (especially for the data source 740), therefore, in the present invention, continuous capture corresponds to the multiple A time interval between two image blocks in the image blocks is adjustable. More specifically, the retrieval unit 714 dynamically adjusts the time interval according to the delay state of the data source 740 . Wherein, the retrieval unit 714 determines the length of the time interval according to the delay status of the data source 740 . If the delay state of the data source 740 is serious, the data of the dynamic block must be retrieved more frequently. Otherwise, the speed at which data is captured will lag behind the speed at which data is processed. Therefore, in the case of severe delay, the time interval for continuously capturing data corresponding to two dynamic blocks will be shortened, and in the case of slight delay, the time interval will be extended. The above operations can be achieved through a delay signal sent by a memory interface for accessing the data source 740 .

In general, the present invention provides a memory management method and a related memory device. Compared with the prior art, the present invention can configure the buffer memory more efficiently. Furthermore, the concept of the present invention regards the buffer memory as a memory pool, so in the prior art, the time spent waiting for memory release will be reduced. In detail, since the memory management method of the present invention can efficiently find the available space in the buffer memory and allocate it to the dynamic block, therefore, in the present invention, there is no longer the waiting time required by the prior art, that is, the present invention The technology provided by the invention can store dynamic blocks without interruption. For dynamic compensation using variable block size, such as: H.264 image compression technology, the present invention significantly improves the performance of the buffer memory.

The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the claims of the present invention shall fall within the scope of the present invention.

Claims (18)

1. A memory management method, comprising: Retrieving data corresponding to a plurality of image blocks, wherein the plurality of image blocks include at least two image blocks with different block sizes; and A memory with multiple memory banks is used to store data corresponding to the multiple image blocks. 2. The memory management method as claimed in claim 1, wherein the plurality of image blocks are dynamic blocks, and the step of retrieving the data corresponding to the plurality of image blocks comprises: Data corresponding to the plurality of dynamic blocks are retrieved according to each motion vector used in an image processing operation. 3. The memory management method as claimed in claim 1, wherein the step of storing data corresponding to the plurality of image blocks comprises: The data corresponding to different columns in an image block among the plurality of image blocks is stored in the same column address space in different memory groups among the plurality of memory groups. 4. The memory management method as claimed in claim 3, wherein the number of the different columns in the image block is greater than the number of the different memory banks. 5. The memory management method as claimed in claim 1, wherein the step of storing data corresponding to the plurality of image blocks comprises: The data corresponding to a row in an image block of the plurality of image blocks is stored in a space of the same row address in different memory groups of the plurality of memory groups. 6. The memory management method according to claim 1, wherein the data corresponding to each image block in the plurality of image blocks includes data of the image block and at least one neighboring image adjacent to the image block Partial data of the block. 7. The memory management method as claimed in claim 6, wherein the step of retrieving data corresponding to the plurality of image blocks comprises: For each image block in the plurality of image blocks: Before retrieving the data corresponding to the image block, checking whether at least a part of the data corresponding to the image block has been stored in a cache device; When a part of the data corresponding to the image block has been stored in the cache device, the part of the data corresponding to the image block is retrieved from the cache device, and from a data source retrieving a remaining portion of data corresponding to the image block, and storing the remaining portion of data corresponding to the image block in the cache device; and When there is no data corresponding to the image block stored in the cache device, retrieve the data corresponding to the image block from the data source, and store the data corresponding to the image block in the high-speed in the cache device. 8. The memory management method as claimed in claim 1, wherein a time interval between two image blocks in the plurality of image blocks corresponding to the continuous capture is adjustable. 9. The memory management method as claimed in claim 8, wherein the time interval is dynamically adjusted according to a delay state corresponding to a data source providing data of the plurality of image blocks. 10. A memory device comprising: A memory with a plurality of memory banks (memory banks); An extracting unit, used to extract data corresponding to a plurality of image blocks, wherein the image blocks include at least two image blocks with different block sizes; and A configuration unit, coupled to the capture unit, is used for using the memory to store data corresponding to the plurality of image blocks. 11. The memory device as claimed in claim 10, wherein the plurality of image blocks are dynamic blocks, and the configuration unit retrieves corresponding to the plurality of dynamic vectors according to each motion vector used in an image processing operation. block data. 12. The memory device according to claim 10, wherein the configuration unit stores data corresponding to different columns in an image block among the plurality of image blocks in different memory banks among the plurality of memory banks space for the same column address. 13. The memory device of claim 12, wherein the number of different columns within the image block is greater than the number of different memory banks. 14. The memory device according to claim 10, wherein the configuration unit stores the data corresponding to a column in an image block in the plurality of image blocks in the same memory group in different memory groups of the plurality of memory groups Space for column addresses. 15. The memory device according to claim 10, wherein the data corresponding to each image block in the plurality of image blocks includes data of the image block and at least one neighboring image area adjacent to the image block Partial data of the block. 16. The memory device of claim 15, wherein for each image block in the plurality of image blocks: Before retrieving the data corresponding to the image block, the retrieval unit checks whether at least a part of the data corresponding to the image block has been stored in a cache device; When a part of the data corresponding to the image block has been stored in the cache device, the retrieval unit retrieves the part of data corresponding to the image block from the cache device, and from retrieving a remaining portion of data corresponding to the image block from a data source, and storing the remaining portion of data corresponding to the image block in the cache device; and When there is no data corresponding to the image block stored in the cache device, the retrieval unit retrieves the data corresponding to the image block from the data source, and stores the data corresponding to the image block stored in the cache device. 17. The memory device as claimed in claim 10, wherein successive captures correspond to a time interval between two image blocks in the plurality of image blocks being adjustable. 18. The memory device according to claim 17, wherein the fetching unit dynamically adjusts the time interval according to a delay state corresponding to a data source providing data of the plurality of image blocks.

Images