JP2007034555A - DYNAMIC MEMORY MANAGEMENT METHOD, MEMORY MANAGEMENT DEVICE, ELECTRONIC DEVICE, AND IMAGING DEVICE - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a new dynamic memory management capable of freely handling various sizes of data including two-dimensional data by using a simple and low-cost memory management method and suppressing generation of a useless memory area. Realize technology.
A memory area securing / releasing unit 122 of a memory management device 120 freely secures two-dimensional (including one-dimensional) memory areas 112a to 112c of various sizes in a two-dimensional shared memory 110, Release after using the memory area. In securing the memory area, the horizontal and vertical relations of the rectangular data to be stored can be maintained as they are, and the memory area of an appropriate size is secured in consideration of suppressing fragmentation of the memory. The usage status (unused status) of the memory space of the memory 110 is appropriately managed by the management information 124.
[Selection] Figure 2

Description

  The present invention relates to a dynamic memory management method, a memory management device, an electronic apparatus, and an imaging device, and more particularly to a technique for effectively using a memory.

  In an imaging apparatus (such as an electronic still camera or a video camera) equipped with a solid-state image sensor, analog signals obtained by photoelectric conversion by the solid-state image sensor are A / D converted, and the resulting image data is used as is or necessary. The image data is compressed and temporarily stored in the memory, and then the image data is read out and sent to the image processing circuit for predetermined image processing (for example, distortion correction, YC separation, reduction zoom, image rotation, post filter A series of processes such as processing, JPEG compression, and various types of image processing such as vertical enlargement are performed, and the image data after image processing is compressed as necessary and stored in the memory again.

  In the processing described above, a memory is frequently used for temporarily storing imaging data or temporarily storing processing data during various image processing (including compression / decompression processing).

  In imaging devices such as digital cameras, paying attention to the fact that line data is output for each row from an image sensor such as a CCD, there are many line memories (static memories with a predetermined size) for storing one row of data. It is used.

However, if the size of the line memory is fixed uniformly, the use efficiency of the memory may be reduced (memory is wasted). Therefore, in Patent Document 1, a shared memory is constructed by an assembly of a large number of line memories, and each time the use of the shared memory becomes necessary, the line memory to be used is changed to the memory use request source circuit. The line memory is used efficiently so that it can be used.
JP 2004-312072 A

  In the technique described in Patent Document 1, the shared memory used is an aggregate of line memories, and therefore, the data that can be handled is only the image data for one line output from the image sensor after all.

  However, in recent years, with the increase in functionality and multifunction of imaging devices (including portable devices such as electronic still cameras, video cameras, and camera-equipped portable terminals), in portable devices such as camera-equipped cellular phones, Processing such as combining the image downloaded from the above server with a photographed photograph is also possible, and even in a normal imaging device, not only data for one line (line data) output from the image sensor There is an increasing need to freely handle two-dimensional data of various sizes.

  Further, even when the processing is limited to processing one line of line data output from the image sensor, the technique of Patent Document 1 uses a large number of fixed-size line memories, so that a useless memory area is likely to be generated. Unavoidable. That is, even if it is referred to as line data, the size of the line data varies in accordance with the model of the imaging device or the resolution selected by the user. Thus, when using a line memory with a fixed size for one line, it is necessary to determine the capacity of the line memory assuming the largest size. If the actual data to be handled is a small size, the remaining part will be It will be useless.

  As the shared memory, a DRAM having the structure shown in FIG. 12 can be used. As shown in FIG. 12, the DRAM can write data to the memory cells (m1 to m12) selected by the word lines W1 to W4 and the bit lines (data lines) C1 to C4 at any time. It is a memory that can read data from memory cells at any time. A slight delay occurs when the word lines (W1 to W4) are switched.

  In addition, in an imaging apparatus (such as an electronic still camera) that generates image data using an image sensor such as a CCD, it is basically necessary to process an image in units of line data for one line.

  That is, as shown in FIG. 13A, image data of one image obtained by imaging with an image sensor such as a CCD is divided into line data (1) to (7) of each row. If the image data of FIG. 13A is stored in the memory space of the shared memory in the manner as shown in FIG. 13B, the line data (2) and (4) for one line is stored across two lines. Will be. Therefore, when reading out these line data, it is necessary to switch the word line, and at this time, there is a possibility that a delay occurs in reading out the image data. In this case, the subsequent processing may be adversely affected (for example, image disturbance or image processing delay due to a DSP or the like).

  As described above, when managing the two-dimensional data by the dynamic memory management method, it is not only necessary to secure an area of a necessary size in the memory space, and it is appropriate to consider the influence on the subsequent image processing. It is necessary to quickly secure a memory area having a proper shape. It is also important to avoid unnecessarily securing a large memory area and to prevent memory fragmentation.

  The present invention has been made based on such considerations, and can freely handle not only one-dimensional data but also two-dimensional data, and can freely handle data of various sizes. It is another object of the present invention to realize a novel memory management technique that can suppress the generation of useless memory areas.

The above object of the present invention is achieved by the following configuration.
(1) When it becomes necessary to store two-dimensional data in the memory space, an unused area in the memory space is searched with reference to memory management information for managing the memory usage of the memory space, and the two-dimensional data A first step of securing a two-dimensional memory area capable of storing data, and a second step of releasing the two-dimensional memory area when the secured two-dimensional memory area is no longer needed In the memory management method, the two-dimensional data is rectangular image data, and in the first step, a rectangular memory area that can be stored while maintaining the rectangular shape of the image data is secured. Dynamic memory management method.

  According to this dynamic memory management method, a used area or an unused area in the memory space is managed by the memory management information, and the memory information is referred to as appropriate so that two-dimensional of various sizes can be obtained in the common memory space. It is possible to realize a dynamic memory management method that can freely secure data (including one-dimensional data) and free a memory area that is no longer needed. The memory space prepared here is not a collection of line memories as found in the prior art, but is a single memory space that can be used without limitation for multiple purposes. Not only data but also any other type of data (whether 2D or 1D) can be stored in the memory space. A memory area of the required size can be secured when necessary, and the memory area that is no longer needed can be released and reused, thus eliminating memory waste and efficient memory usage. It becomes possible.

  In addition, image data such as a line data unit for one row is output from an image sensor such as a CCD. Here, if the size of the reserved memory area is insufficient and the line data for one line is stored across the two memory areas, the line switching (that is, the word (Line switching) causes a delay in data reading, which adversely affects subsequent image processing (for example, disturbance in a display image or delay in image signal processing by a DSP or the like). There is a case. Therefore, when dynamically securing a rectangular memory area for storing image data of a rectangular (rectangular) block in the memory space, the image is maintained while maintaining the state of the rectangular (rectangular) image data (block). A rectangular memory area is secured so that data can be stored as it is. In other words, even if the memory area can be secured freely, it does not mean that the area of any shape can be secured as long as the required size is satisfied, but it is conscious of the vertical and horizontal size of the image data. It is important to secure the area. As a result, the line data for one line can always be secured in the memory area of one line, a delay can be prevented during the reading of the line data for one line, and the shape of the rectangular image data is maintained in the memory. Since the image data is stored in the space, the image processing circuit or the like can read the image data as it is, and can immediately process and process the image data, and there is no additional burden associated with image processing and processing.

(2) The dynamic memory management method according to claim 1, wherein a table for determining a used area or an unused area in the memory space is used as the memory management information, and a start address is used in the table. And specifying a used area or an unused area in the memory space in units of a rectangular area defined by a size in a first direction and a second direction orthogonal to the first direction. Dynamic memory management method.

  According to this dynamic memory management method, a table representing a used area or an unused area in a memory space is prepared, and in the table, for each rectangular area (rectangular block), a start address and horizontal and vertical sizes are prepared. Are defined and used as memory management information. By referring to the table, it is possible to immediately search what size rectangular block is located in which position in the memory space, thereby contributing to promptly securing a necessary memory area. In addition, since the memory management information is updated appropriately as the memory area is secured or released, the memory space usage (unused) status can be managed in real time by the memory management information.

(3) The dynamic memory management method according to (2), wherein when the unused area is searched with reference to the memory management information and a necessary memory area is secured, the first scan is performed on the memory space. A first search step for searching in the direction, and if there is an unused portion of the required size in the first scanning direction, search in the second scanning direction to search for an unused portion of the required size When a rectangular unused area that is larger than the required size is detected by the second search step and the second search step, the first detected rectangular unused area is used as the storage destination of the two-dimensional data. And a storage location setting step for securing the dynamic memory management method.

  According to this dynamic memory management method, when a rectangular area of a required size is secured on the memory space, the memory space is searched in order from a predetermined location in a predetermined direction, and can be found first. A rectangular unused area having a necessary size is secured as a memory area for storing data. Thereby, a high-speed search becomes possible.

(4) The dynamic memory management method according to (2), wherein when the unused area is searched with reference to the memory management information and a necessary memory area is secured, the first scan is performed on the memory space. A first search step for searching in the direction, and if there is an unused portion of the required size in the first scanning direction, search in the second scanning direction to search for an unused portion of the required size The second search step, the first search step and the second search step are repeated to detect all unused areas having a rectangular shape larger than the required size, and from the detected unused areas And a storage destination selection setting step for selectively securing any one rectangular unused area as a storage destination for the two-dimensional data.

  According to this dynamic memory management method, when searching for unused areas by referring to memory management information and securing necessary memory areas, the memory space is searched in order from a predetermined location in a predetermined direction. Any rectangular unused area that is larger than the required size is extracted, and any of the extracted rectangular unused areas that satisfy the conditions such as being able to reduce unnecessary blank areas that are close to the required size, etc. One rectangular unused area is selected, and the selected rectangular unused area is secured as a storage destination of two-dimensional data. As a result, it is no longer necessary to secure an unnecessarily large size area, and the problem of memory fragmentation (fragmentation) caused by securing the area (the memory area is divided due to scattering of the fragmented areas in the memory space). The problem that the utilization efficiency is reduced) can be made difficult to occur.

(5) The dynamic memory management method according to any one of (1) to (4), wherein the two-dimensional memory area is secured between the first step and the second step. A third step of updating the memory management information; and a fourth step of updating the memory management information accompanying the release of the two-dimensional memory area after the second step. A dynamic memory management method characterized by the above.

  According to this dynamic memory management method, the memory management information is updated after securing the memory area and after releasing the memory area. Therefore, it is possible to grasp the usage status (unused status) of the memory space in real time by the memory management information.

(6) The dynamic memory management method according to (5), wherein the third step is newly performed on a remaining unused memory area after the two-dimensional memory area is secured. A dynamic memory management method, comprising: setting a rectangular area again, and registering information about a start address and horizontal and vertical sizes for each of the set rectangular areas in the table.

  According to this dynamic memory management method, in the used area or unused area in the memory space, an unused area (or used area) is defined for each rectangular (quadrangle) area. Therefore, as a result of newly securing one memory area, if the shape of the remaining unused area becomes an irregular polygon, it is difficult to manage as it is, so in this case, the remaining unused area A new rectangular area with the lowest flattening ratio is newly set for the memory area, definition information about each of the rectangular areas is added, and the contents of the table are thereby updated.

(7) The dynamic memory management method according to (5), wherein the fourth step is performed by combining a rectangular area of the released memory with an unused area around the rectangular area. This is a process for forming a rectangular area of a size and registering information on the start address and the size in the horizontal and vertical directions in the table for a large rectangular area newly generated by this combination. Dynamic memory management method.

  According to this dynamic memory management method, when a memory area is released, there is an unused area adjacent to the area to be released, and by combining the released area and the unused adjacent area. If a rectangular area of a larger size can be formed, such a combination is executed, and definition information about the combined rectangular area is added to the table that is the memory management information. . This makes it easy to grasp the size of consecutive unused areas the next time you search for unused areas (that is, if definition information is managed for each small unused area, they will be Thus, it is necessary to confirm at what time the size of the unused area can be ensured at the time of the search, and the search becomes troublesome), and an efficient search of the unused area becomes possible.

(8) The dynamic memory management method according to (1), wherein the memory space is divided in units of rectangular areas corresponding to m pixels × n pixels (m and n are natural numbers of 2 or more), and A dynamic memory management method characterized in that the usage status of each rectangular area is represented by a representative bit representing each rectangular area, and the representative bit is used as the memory management information.

  According to this dynamic memory management method, memory management is performed using bitmap information representing a real memory space instead of the above-described table. For example, if an 8 × 8 pixel block (total of 64 pixels) employed in a JPEG image is used as a unit and even one pixel in this block is in use, a bit representing that block is used. Set to “1” (in this case “0” indicates unused). In this way, by simply checking the array of “1” and “0” in the bitmap, what size of unused area exists, where is the unused area, and how much unused area exists. It becomes possible to easily acquire information such as. Therefore, it is possible to search for an unused area efficiently.

(9) The dynamic memory management method according to (8), wherein when the unused area is searched with reference to the memory management information and a necessary memory area is secured, the first scan is performed on the memory space. A first search step for searching in the direction, and if there is an unused portion of the required size in the first scanning direction, search in the second scanning direction to search for an unused portion of the required size When a rectangular unused area larger than the required size is detected by the second search step and the second search step, the detected rectangular unused area is secured as a storage destination for the two-dimensional data. And a storage destination setting step.

  According to this dynamic memory management method, when an unused area is searched using a bitmap having a plurality of pixels as a unit, the search is sequentially performed from a predetermined location on the bitmap in a predetermined direction. A rectangular unused area having a required size is secured as a memory area for storing data. Thereby, a high-speed search becomes possible.

(10) A memory management device that implements the dynamic memory management method according to any one of (1) to (9), comprising the memory management information, and performing the first and second steps. A memory management device comprising a memory area securing / releasing unit that performs the processing.

  According to this memory management device, two-dimensional data (including one-dimensional data) of various sizes can be freely secured in a common memory space, and a memory area that is no longer needed is released. A memory management device capable of performing a general management method is realized. According to this memory management device, it is possible to use a memory with little waste, and this contributes to solving a memory-related problem in an electronic device that is strictly required to be reduced in size, weight, and cost.

(11) An electronic apparatus comprising: the memory management device according to (10); and at least one memory that provides the memory space.

  According to this electronic device, it is possible to use a very efficient memory even in an electronic device that is strictly required to be small in size, light in weight, and low in cost.

(12) A solid-state imaging device, an optical system that forms a subject image on the solid-state imaging device, a signal processing unit that processes a signal output from the solid-state imaging device and generates image data, and the generated An image pickup apparatus comprising: a display unit that displays an image; and a memory used when generating the image data; and a memory management device according to (10) that manages the memory An imaging apparatus characterized by that.

  According to this imaging apparatus, even when miniaturization, weight reduction, and cost reduction are strictly demanded, it is possible to use the memory very efficiently. In addition, it is possible to efficiently process data of various sizes by effectively using a common memory space, and the performance and functionality of the imaging apparatus are promoted. In addition, line data for one line output from an image sensor such as a CCD is not stored across other lines in the memory space, and is stored while maintaining the vertical and horizontal relationship of the image data. Will be. Therefore, it is possible to prevent a delay from occurring during the reading of one line of line data, and the rectangular image data is stored in the memory space while maintaining the vertical and horizontal relationship as it is. The image data can be read as it is, and can be processed and processed immediately, and an efficient processing of the image data can be realized without causing an extra burden accompanying the processing and processing of the image.

  According to the present invention, the used / unused areas in the memory space are managed by the memory management information, and the memory information is referred to as appropriate, so that two-dimensional data (one-dimensional (Including data) can be freely secured, and a memory dynamic management method capable of releasing a memory area that has become unnecessary can be realized.

  Further, the memory management device of the present invention can freely secure two-dimensional data (including one-dimensional data) of various sizes in a common memory space, and free a memory area that is no longer needed. The dynamic management method can be implemented. This memory management device enables use of memory with little waste, and thus contributes to solving memory-related problems in electronic devices that are strictly required to be downsized, lightened, and reduced in cost.

  In addition, the present invention has an effect of enabling very efficient use of a memory even in an electronic apparatus and an imaging apparatus that are strictly required to be reduced in size, weight, and cost.

Embodiments of the dynamic memory management method of the present invention will be specifically described below with reference to the drawings. In the following embodiments, an electronic still camera will be described as an example of an electronic device, but the present invention is not limited to this.
(First embodiment)

FIG. 1 is a block diagram showing an example of the configuration of an imaging apparatus (electronic still camera) according to the present invention.
As shown in the figure, this imaging apparatus includes an optical system including a lens 10 and a lens driving unit 11, a CPU 12 that comprehensively controls the operation of the imaging apparatus, a system memory 13, a timing generator 26, and a solid-state imaging. A CCD 31, which is an element, an LCD display 32 as a display unit, an AF circuit 36, an AE / AWB circuit 37, an analog signal processing circuit (CDS / AGC / ADC) 38, a memory card 49, and a DSP (Digital Signal Processor). ) 100 (signal processing unit including image input interface 42, image processing circuit 43, compression / decompression processing circuit 44, media controller 46, video encoder 47, and system bus 55), memory 110, and memory management device 120 (memory area) Secure / release unit 122 and management information 12 4).

  An analog imaging signal captured from the CCD image sensor 31 is input to the analog signal processing circuit 38.

  The analog signal processing circuit 38 includes a correlated double sampling circuit (CDS), an auto gain controller (AGC), and an AD converter (ADC). The CDS removes noise from the analog signal, and the AGC automatically adjusts the gain of the analog signal. The ADC generates image data by digitally converting an analog signal.

  This image data is CCD-RAW data (image data for one row output from the CCD line sensor) having R, G, and B density values for each pixel, and this CCD-RAW data is input to the DSP 100. Is done. The analog signal processing circuit 38 is also supplied with the timing signal from the timing generator 26 and is synchronized with the timing at which charges are taken in from the CCD image sensor 31.

  The DSP 100 is an IC chip including an AF circuit 36, an AE / AWB circuit 37, an image input interface 42, an image processing circuit 43, a compression / decompression processing circuit 44, a media controller 46, and a video encoder 47. Various signal processing is performed on the input image data, and then the image is displayed on the LCD display 32 and the image data is written to the memory card.

  The AF circuit 36 detects the focal position based on the image data captured by the CCD image sensor 31 and performs focusing. As the focus detection method, for example, a passive method that detects a focus position using a feature that the contrast of an image is high in a focused state is employed.

  The AE circuit 37 measures subject brightness based on the image data, and determines an aperture value, a shutter speed, and the like. Similarly, the AWB circuit 37 is an auto white balance circuit, and automatically adjusts the white balance at the time of shooting.

  The image input interface 42 takes in the CCD-RAW data from the analog signal processing circuit 38 and writes it into the memory 110.

  In addition, the image processing circuit 43 performs various image processing such as distortion correction, YC separation, reduction zoom, image rotation, post filter processing, JPEG compression, and vertical enlargement, and appropriately performs the memory 110 for each processing. Is used.

  The compression / decompression processing circuit 44 appropriately compresses the image data from the image input interface 42, compresses the image data after various image processing by the image processing circuit 43 as necessary, and compresses the image data. The written data is written into the memory 110. In the decompression process, the memory 110 is used as in the case of compression.

  The memory 110 is an LSI such as a DRAM that provides a shared memory space. Although only one memory is shown in FIG. 1, a single memory space may be configured by combining a plurality of ICs.

  The memory space provided by the memory 110 is a shared memory space that is used for various purposes, can be used for temporary storage of images, and can be used as an intermediate work area for various processes, or can be displayed. It can also be used as a frame memory for storing image data.

  The memory management device 120 is a part that realizes efficient dynamic memory management with less waste with the simple and low-cost configuration of the present invention. The memory area securing / releasing unit 122 can freely store one-dimensional and two-dimensional image data (may be other than image data) in the shared memory space provided by the memory 110.

  The management information 124 always manages the usage status of the shared memory space of the memory 110, and requests memory usage (data) from the memory usage request source (image input interface 42, image processing circuit 43, compression / decompression circuit 44). (Including size notification), and based on the data size, a storage area of the required size is secured, and the start address of the secured memory area is returned to the request source. (This point will be described with reference to FIG. 2).

  FIG. 2 is a block diagram for explaining the function of the memory management unit of FIG. 1 and the procedure for exchanging information between the memory use request source circuit and the memory management unit. In FIG. 2, the same reference numerals are given to the portions common to FIG.

  As shown in FIG. 2, the memory 110 can freely store various-purpose data of various sizes regardless of one-dimensional data, two-dimensional data, and any purpose. Memory (for example, in the C language, a two-dimensional heap memory in which a memory area is secured by a malloc function and the memory area is released by a free function). As shown in the drawing, in the shared memory space provided by the memory 110, an area 112a storing one-dimensional data and an area 112b storing two-dimensional image data are secured. In the figure, an area 112 is a newly reserved area in response to a memory use request from a memory use request source.

  As described above, the memory management device 120 of the present invention is provided with the memory area securing / releasing unit 122, which monitors and manages the usage status of the memory 110. Information 124 is updated as needed.

  Further, the memory area securing / releasing unit 122 receives a memory 110 use request (data storage) from a memory use request source (the image input interface 42, the image processing circuit 43, and the compression / decompression processing circuit 44 shown on the left side of FIG. 2). (Including size information) (step P1), while referring to the management information 124, search the shared memory space of the memory 110 for an unused area larger than the required size and store the data. A memory area (here, area 112c) is secured.

  Then, the memory area securing / releasing unit 122 uses the memory storage request source circuit (in this case, the image input interface 42) as the storage destination address of the secured data (that is, information such as the start address and size of the memory area 112c). (Step P2).

  Next, the image input interface 42 accesses the memory 110 (step P3), performs a desired process using the reserved memory area 112c freely, and when the memory use is finished, instructs the release of the memory area 112c. Is issued to the memory area securing / releasing unit 122 (step P4).

  Next, a procedure for securing and releasing a memory area by the memory area securing / releasing unit 122 will be described with reference to FIGS. In the following example, an unused area table is used as the management information 124.

  FIG. 3 is a diagram illustrating an example of the contents of an unused area table as management information.

  As shown in the figure, the unused area table 125 is a table for managing unused areas in the memory 110 for each of a plurality of rectangular areas. Each rectangular area is numbered, and each rectangular area is It is defined by the start address, X size (horizontal size), and Y size (vertical size).

  FIGS. 4A and 4B are diagrams for explaining an outline of search (search) for securing a memory area of a necessary size by the memory area securing / releasing unit.

  As shown in FIG. 4A, first, the search is advanced in the vertical direction (first direction) in the memory 110 (procedure FS).

  When an unused area of the required size in the vertical direction is found, a search is performed in the horizontal direction (second direction) starting from the start address Px of the unused area as shown in FIG. 4B. (Procedure SS), it is confirmed whether a necessary size area can be secured in the horizontal direction. In the drawing, Py indicates the address of the end of the unused area obtained as a result of the horizontal search.

  There are two methods for selecting a discovered unused area as a data storage destination. One is to secure the first unused area that is found (high-speed search is possible), and the other one is close to the required size from among multiple candidates and minimizes memory fragmentation as much as possible. This is a method of selecting an appropriate region under the condition that it can be suppressed.

  FIGS. 5A and 5B are diagrams for explaining a specific example when a memory area of a necessary size is secured.

  As shown in FIG. 5B, the horizontal size of data to be stored is x, and the vertical size is y.

  In FIG. 5A, DT1 to DT3 and DT5 are used areas, and BL0 to BL3 are unused areas. DT4 indicated by a dotted line is an area newly secured as a result of the current search. The newly secured area DT4 is determined through the following search.

  First, a search is started in the vertical direction from the coordinate P0. Since the required vertical size can be secured in P1 and P2, next, the search is advanced in the horizontal direction starting from P1. Here, since the sizes from P1 to P3 do not reach the required size, it is found that the storage area memory area cannot be secured in the unused area BL2. Similarly, if the search is advanced, the vertical size of P4 to P5 satisfies the required size, but the horizontal size of P4 to P6 does not satisfy the required size. It can be seen that cannot be secured. As a result, it is found that the storage area (DT4) having a necessary size can be secured in the unused area BL3.

  Therefore, in the unused area BL3, the area DT4 having the minimum necessary size is secured in close contact with the area DT3 in use so that no gap is generated.

  Next, update of the memory management information after the memory area is secured (especially, resetting of unused blocks) will be described.

  FIGS. 6A to 6C are diagrams for explaining a procedure for updating memory management information (resetting unused blocks) after securing a memory area.

  FIG. 6A shows a state before a new memory area is secured. In the figure, DT6 to DT8 are rectangular blocks in use, and BL5 to BL6 are unused rectangular blocks. .

  In FIG. 6B, a new memory area DT9 is secured. As a result, the unused block BL6 becomes a distorted polygon.

  Therefore, as shown in FIG. 6C, the rectangular blocks (BL7 to BL9) are reset from the viewpoint of providing a rectangular block having a larger size. Then, the redefined definition information of (BL7 to BL9) is added to the unused area table 125 shown in FIG. 3, and the table (that is, the memory management information 124) is updated.

  Next, a series of processes for securing the memory area, resetting the rectangular block after securing, releasing the memory area, and updating the memory management information after releasing the memory area (including the process of joining adjacent unused blocks) Will be described.

  FIGS. 7A to 7E show how to secure a memory area, reset a rectangular block after securing, release the memory area, and update the memory management information after releasing the memory area (joining adjacent unused blocks together) A series of processing (including processing) will be described.

  As shown in FIG. 7A, in the initial state, the memory 110 (the memory space provided by the memory 110) is completely unused, and the entire surface is an unused area BL0.

  Next, as shown in FIG. 7B, the memory area DT1 is secured. As a result, the rectangular blocks BL2 and BL3 of the unused area are reset, and the memory management information 124 (unused area table) is set accordingly. 125) is updated.

  Next, as shown in FIG. 7C, the memory area DT2 is secured, and as a result, the rectangular blocks BL3 and BL4 of the unused area are reset, and accordingly, the memory management information 124 (unused area table) 125) is updated.

  Next, as shown in FIG. 7D, the memory area DT2 is released. Here, it should be noted that there are unused rectangular blocks BL3 and BL4 adjacent to the released area DT2, and if these are connected to each other, the flatness is low, and a larger unused rectangle. The block can be generated. If a larger unused rectangular block is generated and the unused status of the memory can be managed in units of the block, the size of the free area can be easily grasped, and the subsequent search for the unused area becomes efficient. .

  Therefore, as shown in FIG. 7E, the released area DT2 and the unused rectangular blocks BL3 and BL4 are combined to generate a larger unused rectangular block BL5. The management information 124 (unused area table 125) is updated.

  The procedure up to securing a new memory area as described above is summarized as shown in FIG. FIG. 8 is a flowchart showing an outline of a procedure up to securing a new memory area.

  That is, the memory area securing / releasing unit 122 creates an unused area table 124 and starts managing the memory space (step 10; hereinafter abbreviated as S10), and there is a memory securing request from the memory use request source. (S11), information on the size (x size, y size) of data to be stored is received from the request source (S12).

  Then, the memory area securing / releasing unit 122 searches the unused area table 124 to search whether there is an unused area that satisfies each size of x and y in the shared memory space provided by the memory 110 (search). (S13).

  As a result of the search, an area that is found first may be determined as the memory area of the storage destination, and a plurality of candidates are extracted (S14), from which the required size is close and memory fragmentation can be suppressed. Narrowing may be performed under the above conditions, and through these processes, a memory area of a data storage destination is secured (S15).

  Then, a rectangular block is reset for the unused area after the new memory area is secured (S16), the unused area table is updated (S17), and the start address of the secured memory area is notified to the request source. (S18).

  Also, the procedure for releasing the memory area is summarized as shown in FIG. FIG. 9 is a diagram showing an outline of a memory area release procedure.

  First, information (head address, x, y size) for specifying a memory area to be released is input together with a release instruction from a memory use request source (S20).

  The memory area securing / release unit 122 releases the specified memory area (S21).

  Next, if there is an empty area adjacent to the released area, it is examined whether the empty areas can be combined to form a rectangular block with a larger empty area (S22). If a simple rectangular block can be formed, the empty areas are combined (S23).

  Then, the memory area securing / releasing unit 122 updates the unused area table 125 (that is, the memory management information 124) (S24).

  As described above, in the memory management method and the memory management device according to the present embodiment, the used / unused areas in the memory space are managed by the memory management information, and the memory information is referred to as appropriate, so that the common memory space can be used. It is possible to realize a dynamic memory management method that can freely secure two-dimensional data (including one-dimensional data) of various sizes and free a memory area that is no longer needed. It becomes possible.

  The memory space prepared in the present invention is not a collection of line memories as found in the prior art, but is a single memory space that can be used without limitation for multiple purposes. Not only data, but all other types of data (whether two-dimensional or one-dimensional) can be stored in the memory space, and a memory area of the required size is secured when necessary. The memory area that has become available can be released and reused, and the memory is wasted and the memory can be used efficiently.

  In addition, by securing a rectangular memory area that can store the image data as it is while maintaining the rectangular shape of the rectangular image data block, one line of line data can always be secured in one memory area, It is possible to prevent a delay from occurring during the reading of one line of line data. Further, the image processing circuit or the like can read out the image data as it is and immediately process and process the image data, thereby preventing an extra burden associated with image processing and processing.

  In addition, an unused area table (which may be a used area table) is prepared, and by defining a start address, a horizontal size, and a vertical size for each rectangular area (rectangular block) in the table, the memory It becomes possible to immediately search what size rectangular block is in which position in the space. In addition, by appropriately updating the memory management information as the memory area is secured or released, it is possible to manage the use (unused) status of the memory space in real time by using the table.

  In addition, when securing a rectangular area of the required size in the memory space, the memory space is searched in order from a predetermined location in a predetermined direction, and the rectangular shape having the required size that was found first can be found. By adopting a search method that secures an unused area as a memory area for storing data, a high-speed search becomes possible.

  In addition, a plurality of unused area candidates having a size larger than the required size are extracted from the memory space, and the size is close to the required size and wasteful margins can be suppressed. By adopting a search method that selects and secures one rectangular area, it is possible to avoid unnecessarily securing an area of a large size and to cause the problem of memory fragmentation caused by securing the area. Can be difficult.

  Also, after the memory area is secured and after the memory area is released, the memory management information is updated so that the memory space usage status (unused status) can be grasped in real time by referring to the memory management information. It becomes possible to do.

  In addition, when a new memory area is newly allocated, a new rectangular area is newly set for the remaining unused memory area, and the definition information for each of the rectangular areas is added to change the contents of the table. By updating, the table as the memory management information (the table in the format for managing the memory area in units of rectangular blocks) can always be kept up-to-date.

In addition, when a memory area is released, there is an unused area adjacent to the area to be released, and a larger-sized rectangular area is formed by combining the released area and the unused adjacent area. If it can be formed, by executing such a join and reflecting the join in the table contents, it becomes possible to easily grasp the size of continuous unused areas, and an efficient search for unused areas can be performed. It becomes possible.
(Second Embodiment)

Next, a second embodiment of the dynamic memory management method according to the present invention will be described.
In the present embodiment, an example in which a bitmap is used as memory management information instead of a table will be described. Since the other points are the same as those in the first embodiment, description thereof is omitted.

  FIGS. 10A and 10B are diagrams for explaining a method of generating a bitmap as memory management information.

  That is, as shown in FIG. 10A, the memory 110 is composed of m pixels × n pixels (m and n are natural numbers of 2 or more, and the figure shows the case of 8 pixels × 8 pixels). Divide a rectangular area as a unit.

  Then, as shown in FIG. 10B, the use / unuse of each of the divided rectangular areas is represented by representative bits representing each rectangular area, and thereby a bitmap 126 for projecting the real memory space is represented. The bitmap 126 is used as the memory management information 124.

  That is, in FIGS. 10A and 10B, if a block of 8 × 8 pixels (64 pixels in total) is used and even one pixel in this block is in use, a bit representing that block is Set to “1”, indicating that it is in use, and “0” if it is not used. In this way, just by checking the arrangement of “1” and “0” in the bitmap, what size of unused area exists, where is the unused area, and how many unused areas exist continuously. It becomes possible to easily acquire information such as. Therefore, it is possible to search for an unused area efficiently.

  Note that the 8 × 8 (64 pixels in total) block is also standardly adopted in the JPEG standard for image compression, has a block size that matches the actual image, and facilitates pixel correspondence.

  FIGS. 11A, 11B, and 11C are diagrams for explaining specific examples in the case where an unused area is searched using a bitmap.

  As shown in FIG. 11A, there are areas in use of DT10 to DT13 in the memory 110. Under such circumstances, a storage location of data having a size as shown in FIG.

  That is, as shown in FIG. 11C, referring to the bitmap 126, first, a search is performed in the vertical direction (first direction), and an empty area of a predetermined size or more is confirmed in the column direction. Is searched in the horizontal direction (second direction) to check whether there is an empty area larger than the required size, and if the empty area can be confirmed in this way, the empty area DT14 first found is The memory area of the data storage location shown in FIG.

  By performing a search using a bitmap, a high-speed search is possible. In other words, it is possible to easily obtain information such as where the unused area of what size exists, how long the unused area exists, and efficient search of unused areas is possible. It becomes possible.

  Thus, according to the present invention, it is possible to perform dynamic memory management that can handle two-dimensional data of various sizes with a simple and low-cost configuration. The present invention can be widely used for all electronic devices that need to store various data of different sizes in a memory space and perform various processes.

  Further, the present invention can be widely used in portable devices such as electronic still cameras, video cameras, and camera-equipped portable terminals. In recent years, since the user can freely set the standard mode, the fine mode, and the like, the size of the image data handled inside the imaging apparatus is not uniform, and this makes it difficult to effectively use the memory. ing. However, a high-grade dynamic memory management method is difficult in the field of imaging devices where cost reduction and size reduction are progressing.

  However, according to the present invention, the procedure between the memory use request source such as the image input interface unit, the image processing unit, and the image compression / decompression unit and the memory management device is greatly simplified, and the dynamic memory Management is easy and quick. Therefore, according to the present invention, while satisfying the strict demands for small size, light weight, and low cost for the image pickup apparatus, the memory space is effectively suppressed while effectively preventing generation of unnecessary areas and fragmentation in the memory space. Use can be promoted.

  As described above, according to the present invention, the used / unused areas in the memory space are managed by the memory management information, and the memory information is referred to as appropriate so that various sizes can be obtained in the common memory space. It is possible to realize a dynamic memory management method capable of freely securing two-dimensional data (including one-dimensional data) and releasing a memory area that is no longer needed.

  In addition, by using the bitmap information that projects the real memory space instead of the table in the first embodiment, memory management is performed to determine what size of the unused area exists, where the unused area exists. It is possible to easily obtain information such as how long a continuity exists, and an efficient search for unused areas becomes possible.

  When searching for an unused area using a bitmap, for example, firstly, an empty area of a predetermined size or more is confirmed in the column direction, and then a search is performed in the row direction to make an empty area of a necessary size or more. If a vacancy is confirmed in this way, and a vacant area can be confirmed in this way, a method of securing the vacant area as a memory area of the data storage destination enables high-speed search.

  According to the present invention, not only one-dimensional data but also two-dimensional data can be handled freely, various sizes of data can be handled freely, and generation of useless memory areas can be suppressed. Memory management technology is realized.

It is a block diagram which shows an example of a structure of the imaging device (electronic still camera) of this invention. FIG. 2 is a block diagram for explaining functions of a memory management unit in FIG. 1 and a procedure for exchanging information between a memory use request source circuit and the memory management unit. It is a figure which shows an example of the content of the unused area | region table as management information. (A), (b) is a figure for demonstrating the outline | summary of the search (search) for memory area reservation of a required size by a memory area reservation / release part, respectively. (A), (b) is a figure for demonstrating the specific example in the case of ensuring the memory area of required size, respectively. (A)-(c) is a figure for demonstrating the procedure of the update (resetting of an unused block) of the memory management information after memory area ensuring, respectively. (A) to (e) are for securing a memory area, resetting a rectangular block after securing, releasing the memory area, and updating processing of memory management information after releasing the memory area (joining processing between adjacent unused blocks) It is a figure for demonstrating a series of processes (including these). FIG. 8 is a flowchart showing an outline of a procedure up to securing a new memory area. It is a figure which shows the outline | summary of the release procedure of a memory area. (A), (b) is a figure for demonstrating the production | generation method of the bitmap as memory management information, respectively. (A), (b), (c) is a figure for demonstrating the specific example in the case of searching an unused area | region using a bitmap, respectively. It is a block diagram which shows the structure of the memory cell part of DRAM. It is a figure for demonstrating the problem in the case of storing the line data for one line output from an image sensor in the memory area over two lines.

Explanation of symbols

10 Lens 11 Lens Driving Unit 12 CPU
13 System memory 26 Timing generator 31 CCD
32 LCD display 36 AF circuit 37 AE / AWB circuit 38 Analog signal processing circuit (CDS / AGC / ADC)
42 Image Input Interface 43 Image Processing Circuit 44 Compression / Expansion Processing Circuit 46 Media Controller 47 Video Encoder 49 Memory Card 55 System Bus 100 DSP
110 memory (memory with two-dimensional memory space such as synchronous DRAM)
120 Memory management device 122 Memory area securing / release unit 124 Memory management information 125 Unused area table 126 Bitmap

Claims (12)

When it becomes necessary to store two-dimensional data in the memory space, an unused area in the memory space is searched by referring to memory management information for managing the memory usage of the memory space, and the two-dimensional data is stored. A dynamic memory management method comprising: a first step of securing a possible two-dimensional memory area; and a second step of releasing the secured two-dimensional memory area when the secured two-dimensional memory area is no longer needed Because
The two-dimensional data is rectangular image data,
In the first step, a rectangular memory area that can be stored while maintaining a rectangular shape of the image data is secured. The dynamic memory management method according to claim 1, comprising:
As the memory management information, a table for determining a used area or an unused area in the memory space is used. In the table, a start address, a second direction orthogonal to the first direction and the first direction are used. A dynamic memory management method characterized in that a used area or an unused area in the memory space is specified in units of a rectangular area defined by a size in the direction of the direction. A dynamic memory management method according to claim 2, comprising:
When searching for an unused area with reference to the memory management information and securing a necessary memory area,
A first search step for searching in the first scanning direction on the memory space;
A second search step of searching for an unused portion of the required size by searching in the second scan direction when there is an unused portion of the required size in the first scan direction;
A storage destination setting step for securing a rectangular unused area detected first as a storage destination of the two-dimensional data when a rectangular unused area larger than a necessary size is detected by the second search step; ,
A dynamic memory management method comprising: A dynamic memory management method according to claim 2, comprising:
When searching for an unused area with reference to the memory management information and securing a necessary memory area,
A first search step for searching in the first scanning direction on the memory space;
A second search step of searching for an unused portion of the required size by searching in the second scan direction when there is an unused portion of the required size in the first scan direction;
Repeating the first and second search steps to detect all unused areas having a rectangular shape larger than the required size,
A storage destination selection setting step for selectively securing any one of the rectangular unused areas as the storage destination of the two-dimensional data from the detected unused areas;
A dynamic memory management method comprising: The dynamic memory management method according to claim 1, wherein:
A third step of updating the memory management information accompanying the securing of the two-dimensional memory area between the first step and the second step;
A fourth step of updating the memory management information following the release of the two-dimensional memory area after the second step;
A dynamic memory management method comprising: The dynamic memory management method according to claim 5, comprising:
In the third step, a new rectangular area is newly set for the remaining unused memory area after the two-dimensional memory area is secured, and each of the set rectangular areas is set. A dynamic memory management method, which is a process of registering information on a start address and horizontal and vertical sizes in the table. The dynamic memory management method according to claim 5, comprising:
The fourth step forms a larger-sized rectangular area by combining the rectangular area of the released memory with an unused area around the rectangular area, and a newly generated large-sized rectangular area. A dynamic memory management method, which is processing for registering information about a start address and horizontal and vertical sizes of a rectangular area in the table. The dynamic memory management method according to claim 1, comprising:
The memory space is divided in units of rectangular areas corresponding to m pixels × n pixels (m and n are natural numbers of 2 or more), and the usage status of each of the rectangular areas is representative of each rectangular area. A dynamic memory management method, characterized in that the representative bits are expressed as bits and used as the memory management information. 9. The dynamic memory management method according to claim 8, wherein
When searching for an unused area with reference to the memory management information and securing a necessary memory area,
A first search step for searching in the first scanning direction on the memory space;
A second search step of searching for an unused portion of the required size by searching in the second scan direction when there is an unused portion of the required size in the first scan direction;
A storage location setting step for securing the detected rectangular unused area as a storage destination of the two-dimensional data when a rectangular unused area of a size larger than a necessary size is detected by the second search step;
A dynamic memory management method comprising: A memory management device that implements the dynamic memory management method according to any one of claims 1 to 9,
A memory management apparatus comprising the memory management information and having a memory area securing / releasing unit for executing the first and second steps.   11. An electronic apparatus comprising: the memory management device according to claim 10; and at least one memory that provides the memory space. A solid-state image sensor, an optical system that forms a subject image on the solid-state image sensor, a signal processing unit that processes a signal output from the solid-state image sensor and generates image data, and displays the generated image An imaging device including a display unit,
A memory used when generating the image data;
The memory management device according to claim 10 for managing the memory;
An imaging apparatus comprising:

Images