WO2018061161A1 - Storage apparatus and method for controlling same - Google Patents

Abstract

[Problem] To propose a highly useful storage apparatus and a method for controlling same. [Solution] A storage apparatus provided with one or a plurality of memory devices and a storage controller for dynamically allocating memory areas of the memory devices to virtual volumes and writing the data to be written that is given from a host along with a write request into the memory areas allocated to the virtual volumes. The storage controller manages the memory areas of the memory devices separately for an area for overwrites that permits overwriting and an area for additional writes for storing postscripts, allocates the area for overwrites in units of a prescribed first size to areas in the virtual volumes, writes the data to be written into the allocated area for overwrites, compresses the data written into the area for overwrites in units of a second size smaller than the first size, and stores the compressed data to the area for additional writes.

Description

Storage apparatus and control method thereof

The present invention relates to a storage apparatus and a control method therefor, and is suitable for application to a storage apparatus equipped with both a data compression function and a deduplication function, for example.

The storage apparatus has a plurality of storage devices for storing data and a storage controller for controlling these storage devices, and aims to provide a large capacity data storage space to the host computer.

Storage devices are required to store large amounts of data at low cost. In order to satisfy such a requirement, a technique for recording data by lossless compression (hereinafter simply referred to as compression) is known. If the data size is reduced by compression and then recorded in the storage device, more data can be stored in the storage device than if the data is recorded in the storage device without being compressed. Therefore, it is possible to reduce the data holding host (such as the bit cost of the storage medium and the power consumption cost of the storage device).

Deduplication technology is another technology that reduces the amount of data to be stored. When the storage apparatus detects that a plurality of data having the same content exists in the storage apparatus, the deduplication technology leaves only one of them in the storage device in the storage apparatus and stores the remaining data in the storage device. This is a technique to prevent storage.

Compression technology and deduplication technology can be used together. For example, Patent Document 1 discloses a storage device that performs deduplication processing on data that is duplicated among data transferred from a host device and compresses data that has not been deduplicated.

Japanese Patent No. 521691

By the way, the data compression processing unit affects the overhead when decompressing the data. For example, when data is compressed in a large capacity unit, even if only a part of the data is to be read, the data must be decompressed in that capacity unit, and extra overhead is generated. descend. On the other hand, if the data compression processing unit is made too small, the amount of management information increases, and the utilization efficiency of the storage area for storing the management information deteriorates.

Also, when compressing data, the compression ratio changes depending on the data content. For this reason, after the storage system compresses the data and stores it in the storage device, when the update data of the data is received from the host device, the size of the update data after compression is smaller than the size of the compressed data stored in the storage device Can also be large. In this case, the compressed update data cannot be overwritten on the area where the compressed data before update is stored, and a different storage method is required.

The present invention has been made in view of the above points, and intends to propose a highly useful storage apparatus and control method thereof that can solve the above-mentioned various problems associated with data compression at once. .

In order to solve this problem, in one embodiment of the present invention, a virtual volume that is a virtual storage space is provided to a host, and is dynamically stored in response to a data write request to the virtual volume from the host. In the storage apparatus to which the area is allocated, one or a plurality of storage devices and the storage area of the storage device are dynamically allocated to the virtual volume, and the write target data given from the host together with the write request is stored in the virtual volume. A storage controller that writes to the storage area allocated to the storage device, and the storage controller manages the storage area of the storage device separately into an overwrite area that permits overwriting and an additional writing area that additionally stores data. , In a predetermined first capacity unit for the area on the virtual volume An overwrite area is allocated, the data to be written is written in the allocated overwrite area, and the data written in the overwrite area is compressed in a second capacity unit smaller than the first capacity and compressed. The data is stored in the additional writing area.

In another embodiment of the present invention, a virtual volume that is a virtual storage space is provided to a host, and a storage area is dynamically allocated according to a data write request from the host to the virtual volume. In the apparatus control method, the storage apparatus dynamically allocates one or a plurality of storage devices and a storage area of the storage device to the virtual volume, and writes write target data given from the host together with the write request. A storage controller that writes to the storage area allocated to the virtual volume, and the storage controller includes an overwrite area that allows overwriting, and an additional write area that additionally stores the storage area of the storage device. The storage controller is configured to manage the virtual volume. A first step of allocating the area for overwriting in a predetermined first capacity unit to an area on a storage area, and writing the data to be written into the allocated area for overwriting; And a second step of compressing the data written in the area for use in a second capacity unit smaller than the first capacity and storing the compressed data in the area for additional writing.

In the storage apparatus and control method thereof according to the present invention, the capacity unit for allocating the overwrite area to the virtual volume and the capacity unit for compressing the data written to the virtual volume are divided into the second capacity that is a unit for compressing the data. Is made smaller than the first capacity, which is a unit for allocating the overwrite area to the virtual volume, while reducing the amount of extra decompression processing (overhead) when reading compressed data, An increase in management information for managing the area can be prevented. Therefore, it is possible to prevent an increase in the amount of management information while preventing a decrease in access performance of the storage apparatus due to such overhead.

In the storage apparatus and the control method thereof according to the present invention, the uncompressed data is written in the overwrite area and the compressed data is stored in the additional write area. Can be stored in the storage device without any problem.

According to the present invention, a highly useful storage apparatus and its control method can be realized.

1 is a configuration example of a computer system including a storage device according to an embodiment. It is a figure showing the relationship between a virtual volume and a page. It is a list of management information. It is a structural example of a page management table. It is a structural example of a pool management table. It is a structural example of a virtual volume management table. It is a structural example of a write-once space mapping table. It is an example of a structure of LP conversion table. It is an example of composition of FPMQ and FPTD. It is a list of programs that the storage device has. It is a flowchart of a read process. It is a flowchart of a write process. It is a flowchart of a buffer update program. It is a flowchart (1) of a deduplication program. It is a flowchart (2) of a deduplication program. It is a flowchart of the process which adds to an additional writing space. It is a flowchart of a destage process. It is a flowchart of FPK calculation and data compression processing. (A)-(E) are diagrams for explaining FPK calculation / data compression processing. It is a flowchart of a data expansion process. (A)-(C) are diagrams for explaining data decompression processing.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

In the following embodiments, the processing executed in the storage apparatus may be described using “program” as the subject. Actually, the processor (CPU) included in the storage apparatus executes the program to perform the process described in the program. Therefore, the main subject of the process is the processor (CPU), but the explanation becomes redundant. In order to prevent this, the contents of processing may be explained using the program as the subject. Further, part or all of the program may be realized by dedicated hardware. Various programs described below may be provided by a storage medium that can be read by a program distribution server or a computer, and may be installed in each device that executes the program. The computer-readable storage medium is a non-transitory computer-readable medium such as a non-volatile storage medium such as an IC card, an SD card, or a DVD.

Prior to describing the embodiment, various terms used in the present embodiment will be described.

In this specification, “compression” means a process of reducing the data size while maintaining the meaning of the data using a reversible compression algorithm such as the LZW algorithm. In the storage apparatus according to the present embodiment, data written from the host to the storage apparatus may be compressed. Data that has been reduced in size as a result of compression processing performed by the storage device is referred to as “compressed data”, and data that has not been subjected to compression processing by the storage device is referred to as “uncompressed data”. The process of returning the compressed data to the original data size using the lossless compression algorithm is called “decompression”.

“Volume” means a storage space provided by a target device such as a storage device or storage device to an initiator device such as a host computer. When the initiator device issues a data write request to a certain position (address) in the storage space, the data is stored in the storage area assigned to that address. The storage apparatus according to this embodiment provides a virtual volume formed by so-called Thin Provisioning technology to a host as a volume. In a virtual volume in its initial state (immediately after the virtual volume is defined), no storage area is allocated to each address on the virtual volume. When the storage apparatus receives a data write request for an address in the storage space from the initiator device (host), the storage apparatus dynamically determines a storage area to be allocated to the address.

“Block” means an area on the volume. In the embodiment described below, a block is an area having a size equal to the minimum access unit when an initiator device accesses a volume, and has a fixed length. As an example, the block size is 512 bytes. When the initiator device accesses the area on the volume, the access is made by designating the address assigned to each block. This address is called a logical block address (hereinafter referred to as LBA (Logical Block Address)). The LBA of the first block of the volume is 0, and subsequent blocks are 1, 2,. . . The serial number is attached. In the following embodiments, the storage apparatus may use an address other than the LBA as an address for specifying an area on the volume. This will be described in the embodiment.

“Duplicate elimination process” is a process of leaving only one data in the storage apparatus and deleting other data from the storage apparatus when there are a plurality of data of the same content in the storage apparatus. The process of determining whether data with the same content exists in the storage device is referred to as a “duplication determination” process. Unless otherwise specified, the deduplication processing is processing including duplication determination processing.

In the storage apparatus according to the embodiment described below, duplication determination is performed for each data of a predetermined size called a chunk. In the following embodiment, an example in which the chunk size is 8 KB will be described. However, the chunk size may be other than 8 KB. Data (or chunks) with the same content is called “duplicate data (or duplicate chunks)”.

When comparing two data in bit units or byte units, it takes a long time for the determination process. Therefore, in general, an apparatus that performs duplication determination is characterized by a small size (for example, about 8 bytes) that is calculated based on data to be compared, such as a hash value that is calculated using a hash function. A check code consisting of a quantity is generated, and duplication is determined using the generated check code. In the following embodiment, a check code generated from data is referred to as “FingerPrint Key”. FingerPrint Key may be abbreviated as FPK.

In the embodiment described below, if the value of FPK calculated from data A is H, value H is referred to as FPK of data A. Conversely, data A may be referred to as “data having FPK H”.

「“ Duplicate exclusion range ”means the data search range when determining the presence or absence of duplicate data. For example, when the deduplication range is “storage device”, the storage device compares the data A with all data in the storage device when determining whether the same data as the data A exists. On the other hand, when the scope of deduplication is limited to a part of the storage devices in the storage apparatus (herein referred to as “storage groups”), the storage apparatus The presence or absence of duplicate data is searched from only the data inside.

“Updating” the storage area means rewriting (overwriting) the contents of the data stored in the storage area with new contents. Data that has been stored in a storage area before it is updated is referred to as “pre-update data”. On the other hand, data newly written in the storage area is called “update data” or “updated data”.

“Log-structured write” or “log-structured” means an operation of sequentially writing data to an unused area of a storage area. In the storage device in the following embodiment, a storage area for additional writing is provided. When the storage device performs additional writing processing, data is written in order from the beginning of the storage area for additional writing. The storage device stores the end address to which data has been written in the immediately preceding write process, and writes data from the address next to this end address when performing the write process.

(1) System Configuration FIG. 1 shows a configuration example of a computer system including a storage apparatus 1 according to this embodiment. The storage device 1 includes a storage controller 10 and a plurality of storage devices 20 connected to the storage controller 10.

The storage device 20 is used for the storage device 1 to store write data from a host device such as the host 2. As the storage device, for example, an HDD (Hard Disk Drive) 22 using a magnetic disk as a storage medium or an SSD (Solid State Drive) 21 adopting a nonvolatile semiconductor memory such as a flash memory as a storage medium is used. For example, the storage device 20 is connected to the storage controller 10 by a transmission line (SAS link) conforming to the SAS (Serial Attached SCSI) standard, a transmission line (PCI link) conforming to the PCI (Peripheral Component Interconnect) standard, or the like.

One or more hosts 2 are connected to the storage controller 10. A management host 5 is connected to the storage controller 10. The storage controller 10 and the host 2 are connected via a SAN (Storage Area Network) 3 formed using a fiber channel as an example. The storage controller 10 and the management host 5 are connected via a LAN (Local Area Network) 6 formed using Ethernet (registered trademark) as an example.

The storage controller 10 includes at least a processor (also referred to as a CPU) 11, a host interface (also referred to as “host I / F”) 12, a device interface (also referred to as “device I / F”) 13, a cache memory 14, a management I / O. F15 and local memory 16 are provided. The CPU 11, host I / F 12, device I / F 13, cache memory 14, and management I / F 15 are interconnected via an internal switch (also referred to as an internal SW) 17. Although only one of these components is shown in FIG. 1, a plurality of these components may be mounted in the storage controller 10 in order to ensure high performance and high availability. In addition, each component may be connected to each other via a common bus instead of the internal SW 17.

The device I / F 13 has at least an interface controller and a transfer circuit. The interface controller is a component for converting a protocol (SAS in one example) used in the storage device 20 into a communication protocol (PCI-Express as an example) used in the storage controller 10. The transfer circuit is used when the storage controller 10 transfers data (read, write) to the storage device 20.

The host I / F 12 has at least an interface controller and a transfer circuit, like the device I / F 13. The interface controller of the host I / F 12 converts the communication protocol (for example, fiber channel) used in the data transfer path between the host 2 and the storage controller 10 and the communication protocol used in the storage controller 10. Is to do.

The CPU 11 performs various controls of the storage device 1. The local memory 16 stores a program executed by the CPU 11 (referred to as a storage control program). Further, the CPU 11 uses a part of the storage area on the local memory 16 as a kind of work area when performing data compression processing. As the local memory 16, a volatile storage medium such as DRAM or SRAM is used. However, as another embodiment, a nonvolatile memory may be used.

The cache memory 14 is used for temporarily storing I / O target data for the storage device 20. Further, it is used for storing various management information of the storage apparatus 1 used by the CPU 11. As the cache memory 14, a volatile storage medium such as DRAM or SRAM is used. However, as another embodiment, the cache memory 14 may be configured using a nonvolatile memory. When a volatile storage medium is used for the cache memory 14, an auxiliary power source such as a battery may be mounted on the storage device 1 so that the stored contents of the cache memory 14 can be maintained during a power failure.

In the storage apparatus 1 according to the present embodiment, the CPU 11 is configured to be able to access both the local memory 16 and the cache memory 14. However, when the CPU 11 accesses the cache memory 14, it accesses via the internal switch 17. Therefore, the response time when the CPU 11 accesses the cache memory 14 is longer than the response time when the CPU 11 accesses the local memory 16.

The management host 5 is a computer for performing management operations of the storage apparatus 1. The management host 5 includes input / output devices (not shown) such as a keyboard and a display, and a user (administrator) can issue a setting instruction to the storage apparatus 1 using the input / output devices. The management host 5 can also display information such as the status of the storage apparatus 1 on an output device such as a display.

The storage apparatus 1 according to the present embodiment can compress data stored in the storage device 20. Data compression is performed by the CPU 11 (compression is performed by executing a program code for performing data compression by the CPU 11). However, as another embodiment, the storage controller 10 may be provided with dedicated hardware for performing data compression, and the storage controller 10 may be configured to perform data compression using this hardware.

(2) Storage Area Management Next, storage areas managed by the storage apparatus 1 according to the present embodiment will be described. The one or more storage devices 20 included in the storage apparatus 1 according to the present embodiment provide the storage controller 10 with a storage space of a predetermined size. However, the storage controller 10 does not directly provide the storage space provided by the storage device 20 to the host 2. The storage controller 10 provides the host 2 with one or more virtual storage spaces different from the storage space provided by the storage device 20. This virtual storage space is called a “virtual volume”. A virtual volume (a storage space provided to the host 2) may be referred to as an “overwrite space”.

The virtual volume is a volume that is formed using a known Thin Provisioning technology. When the storage apparatus 1 receives an access request for an area on the virtual volume from the host 2, the storage apparatus 1 allocates (maps) the storage area of the storage device 20 to the area.

In the present embodiment, among the plurality of storage devices 20 included in the storage apparatus 1, the storage device 20 that can be allocated to the virtual volume belongs to a management unit called “pool”. The storage device 1 manages the storage space of the storage device 20 belonging to the pool by dividing it into partial areas of a predetermined size (42 MB as an example). In the present embodiment, this partial area is referred to as “page” or “physical page”. A plurality of pools may be provided in the storage device 1. Each pool is assigned an identification number that is unique within the storage apparatus 1, and this identification number is called a pool number (pool #). A pool with pool #k is denoted as “pool #k”.

When the storage device 1 allocates the storage area of the storage device 20 to the virtual volume, it allocates for each page. Therefore, the storage apparatus 1 manages the virtual volume by dividing it into an area having the same size as the page. An area of the same size as a page on the virtual volume is called a “virtual page” or “overwrite page”. In this embodiment, an example in which the size of the virtual page is 42 MB will be described. However, the size of the virtual page may be other than this.

When the host 2 accesses the virtual volume, the host 2 issues an access request (read request, write request, etc.) to the storage device 1. The read request and write request include an identifier (Logical Unit Unit Number (LUN) or the like) for specifying the virtual volume to be accessed and information for specifying an area on the virtual volume. The information specifying the area on the virtual volume includes information on the LBA and the length of the area. When the storage apparatus 1 receives a write request from the host 2, the storage apparatus 1 specifies a virtual page to be accessed from information (LBA and area length) that specifies an area on the virtual volume included in the write request. . When a page is not yet allocated to the virtual page to be accessed, the storage apparatus 1 selects an unused page (a page that has not been allocated to the virtual page) among the pages belonging to the pool, and sets it as the virtual page to be accessed. Assign (map) the selected page. Write data from the host 2 is stored in a page mapped to this access target virtual page.

The storage device 1 also stores the correspondence (mapping) between the virtual page and the page assigned to the virtual page in a mapping table (such as a virtual volume management table 300 described later with reference to FIG. 6). When a read request for a virtual page is received, the storage apparatus 1 refers to the mapping table to identify a page assigned to the virtual page and reads data from the identified page.

FIG. 2 is a conceptual diagram showing the relationship between a virtual volume and a storage area (page) allocated to the virtual volume. A rectangular object 40 shown in FIG. 2 represents a virtual volume (overwriting space). On the other hand, the cylindrical object 50 represents a pool.

Furthermore, the storage device 1 has a storage space different from the virtual volume (overwrite space) 40. This storage space is called “write-once space”. A rectangular object 60 shown in FIG. 2 represents a write-once space. The write-once space 60 is not a storage space that can be accessed by the host 2 but a storage space that can be used only by the storage controller 10. The write-once space 60 is used for the CPU 11 of the storage controller 10 to store the compressed data in the storage device 20.

First, the relationship between the virtual page and the physical page in the overwrite space will be described. Each virtual page is assigned a unique identification number (a non-negative integer value is used for the identification number) called a virtual page number (sometimes referred to as VP #). Hereinafter, a virtual page with a virtual page number n (n is a non-negative integer value) is denoted as “VP # n”. Similarly, each physical page is given a unique identification number in the storage apparatus 1 called a physical page number (or page #) (this identification number is also a non-negative integer value). Hereinafter, a physical page whose physical page number is n (n is a non-negative integer value) is denoted as “page #n”. In the example of FIG. 2, an example in which page # 0 is mapped to VP # 0 is shown. VP # 1 and VP # 2 will be described later.

When the storage apparatus 1 receives a write request (and write data) for an area (one or a plurality of blocks) in the virtual page from the host 2, the write data is stored in the block in the physical page mapped to the virtual page. Is done. The storage apparatus 1 manages the correspondence (mapping) between virtual pages and physical pages by storing them in a virtual volume management table 300 described later with reference to FIG.

In principle, when the storage apparatus 1 according to the present embodiment receives a data write request from the head of the virtual page to the nth block (except for the case of compressed data storage described later), the data is mapped to the virtual page. Stored in the nth block from the top of the physical page. In order to avoid redundant description, in the following, the storage apparatus 1 stores data in a physical page allocated to a virtual page, and “stores data in a virtual page (or overwrite space)”. It expresses.

As with the overwrite space, the write-once space is a storage space formed by so-called Thin Provisioning technology. The storage controller 10 dynamically assigns a page (physical page) to an area on the write-once space, and data is written to the physical page assigned to the write-once space. Similar to the overwrite space, the storage device 1 manages the write-once space by dividing it into an area having the same size as the physical page. This area is called “additional page”. A unique identification number called an additional page number is also attached to the additional page. The additional page number may be written as an additional page #. The storage device 1 manages the correspondence (mapping) between the additional page and the physical page by storing it in an additional space mapping table 350 (FIG. 7) described later. Further, in order to avoid redundant description, in the following description, the storage apparatus 1 writes data to a physical page assigned to an additional page, and “writes data to the additional page (or additional space). ".

As described above, the data written in the write-once space is compressed data. Further, an additional write space is provided for each virtual volume. When there are n virtual volumes, there are n additional write spaces. As the physical page allocated to the write-once space, an unused physical page is selected from the pool, similarly to the physical page allocated to the virtual page. When a chunk of a virtual volume to which a physical page of pool #k is allocated is moved to the write-once space, the physical page allocated to the write-in space is a physical page of pool #k.

The storage device 1 performs compression processing on data (uncompressed data) written from the host 2 to the virtual page as necessary. The compressed data generated by the compression process is moved to a location different from the physical page storing the non-compressed data, specifically, to the write-once space. At this time, the presence / absence of duplicate data is also determined.

The storage apparatus 1 according to the present embodiment divides the area in the virtual page into 8 KB partial areas in order from the top of the virtual page, and performs data compression for each partial area. In the present embodiment, this 8 KB partial area (or data stored in this partial area) is called a “chunk”. Note that a size other than 8 KB may be adopted as the size of the chunk. Data generated by compressing a chunk is called a “compression chunk”. The size of the compressed chunk varies depending on the data contents, but is 512 bytes at the minimum and 8 KB at the maximum. Further, the storage device 1 according to the present embodiment generates a compressed chunk having an integer multiple of 512 bytes.

When the storage controller 10 (the CPU 11 thereof) generates a compressed chunk, the storage controller 10 adds the compressed chunk to the additional write space. When all the chunks of a certain virtual page are compressed and all the compressed chunks are written to the write-once space, the page assigned to the virtual page is discarded. Specifically, the page assigned to the virtual page is changed to an unused state (a state not assigned to the virtual page). As a result, the data written to the overwrite space is moved to the additional write space. Note that a plurality of virtual page chunks (compressed chunks) may be stored in the physical page allocated to the write-once space. This is because the chunk size is reduced by the compression, and data of a plurality of virtual pages may be accommodated in one physical page. Also, the storage apparatus 1 manages the correspondence (mapping) between the chunks in the overwrite space and the compressed chunks in the write-once space by storing them in an LP conversion table 500 described later with reference to FIG.

If there is an update data write request (that is, an update request) from the host 2 to the virtual page in which the data written from the host 2 has been moved to the write space, the update data is compressed and added to the write space. Is done. However, as another embodiment, the storage apparatus 1 again allocates a physical page to the virtual page in the overwrite space, expands the data moved to the write-once space, and expands the data expanded to the physical page allocated to the virtual page. You may make it write back and update (overwrite) the data on this physical page.

An address used by the storage apparatus 1 to identify a chunk on the overwrite space is called LA. LA is an address assigned to each chunk.
LA = LBA ÷ 16
Are in a relationship.

Therefore, the LA of the chunk located at the top of the overwrite space is 0, and the LA of the subsequent chunks are 1, 2,. . . It becomes the serial number.

On the other hand, since the compressed chunk on the write-once space has a variable length and a minimum size of 512 bytes, an address is assigned to each 512-byte area (block) on the write-once space. This address is called PA. The PA in the 512-byte area located at the head of the write-once space is 0, and the PA in the subsequent area is 1, 2,. . . The serial number is used.

(3) Management Information Next, the contents of management information used for managing the pool, virtual volume (overwrite space), and write-once space described above will be described. In this specification and drawings, there is a character string “0x” added to the beginning of a numerical value, which means that the numerical value is expressed in hexadecimal.

FIG. 3 shows the management information that the storage apparatus 1 has. The storage apparatus 1 has a page management table 200, a virtual volume management table 300, an additional write space mapping table 350, a logical-physical conversion table (LP conversion table) 500, an FPMQ (FingerPrint Match４００Queue) 400, and an FPTD (FingerPrint) on the cache memory 14. Table Directory) 450 and pool management information 600.

FIG. 4 shows an example of a page management table 200 that is information for managing pages belonging to the pool. The page management table 200 is a table provided for each pool. When n pools are provided, the storage apparatus 1 has n page management tables 200.

Each row (record) of the page management table 200 stores physical page information in the pool. Each record has columns of page # (201), DEV # (202), Addr (203), and use (204). Page number (201) stores the page number of the physical page. That is, the record of page # (201) n is a record for managing information of page #n.

As described above, a page (physical page) is an area on the storage device 20. In DEV # (202) and Addr (203), the identification number of the storage device 20 where the physical page exists and the address in the storage device are stored. In the case of FIG. 4, the record managing the information of page # 1 stores 0 in DEV # (202) and 0x15000 in Addr (203). Therefore, page # 1 represents that it corresponds to a 42 MB area (an area equal to the size of one physical page) of the storage device 20 with the identification number 0 starting from address 0x15000.

Use (204) stores information indicating the usage status of the physical page. When the physical page is mapped to the virtual volume or the write-once space, “1” is stored in the use (204). Conversely, when the physical page is not mapped to the virtual volume or the write-once space, “0” is stored in use (204). A page whose use (204) is “1” is called a page in use, and conversely, a page whose use (204) is “0” is called an unused page.

In the present embodiment, an example is described in which the identification number and address of the storage device 20 (for example, HDD) are registered in the page management table 200 and each page corresponds to a storage area in one storage device 20. . However, other pool (or page) configuration methods may be employed. For example, when the storage apparatus 1 forms a single logical storage device (referred to as a logical device) from a plurality of storage devices 20 using RAID (Redundant Arrays of Independent (or Inexpensive) Disks) technology, A storage area of a predetermined size (for example, 42 MB) may be handled as a page. In this case, the DEV # (202) and Addr (203) of the page management table 200 may store the logical device identification number and the address in the logical device.

FIG. 5 shows an example of the pool management information 600. The pool management information 600 is a table for storing information about virtual volumes belonging to each pool. When there are a plurality of pools, the storage apparatus 1 causes each virtual volume to belong to one of the pools. When allocating a physical page to a virtual page of a virtual volume, the storage apparatus 1 selects an unused physical page in the pool to which the virtual volume belongs and allocates the physical page to the virtual page.

In each record of the pool management information 600, information on the pool number (601), virtual volume # (603), and last write position (602) is stored. The pool number (601) stores the pool number. Virtual volume # (603) stores the identification number (referred to as a virtual volume number) of the virtual volume belonging to the pool. When there are a plurality of virtual volumes belonging to the pool, a plurality of virtual volume numbers are stored in the virtual volume # (603).

The last write position (602) corresponds to each virtual volume that belongs to the corresponding pool (each virtual volume whose virtual volume number is stored in virtual volume # (603) in the same row) and corresponds to that virtual volume. The additional page number of the additional page to which the compressed chunk is written at the end in the attached additional space is stored. When the storage controller 10 writes the compressed chunk on the additional write page, the last write position is specified by referring to the final write position (602).

FIG. 6 shows an example of the virtual volume management table 300. A virtual volume management table 300 is also stored in the cache memory 14. Each record of the virtual volume management table 300 stores information about virtual pages in the virtual volume. Specifically, information of virtual page # (301), last update time (302), page type (303), page # (304), and pool # (305) is stored in each record.

The virtual page number (301) stores the virtual page number of the virtual page. On the other hand, the page # (304) stores the page # of the physical page assigned to the virtual page. Pool # (305) is the pool number of the pool to which the physical page assigned to the virtual page belongs.

In the last update time (302), the time when the last write request is received for the virtual page is stored. Each time the storage apparatus 1 receives a write request for a certain virtual page, the storage apparatus 1 stores the time when the write request is received as the last update time (302) of the virtual page.

1 or 0 is stored in the page type (303). In the initial state, the page type (303) of all virtual pages is set to 1.

When 0 is stored in the page type (303), this indicates that the data written to the virtual page is compressed and moved to the write-once space. In this case, page # (304) and pool # (305) have invalid values (referred to as null, specifically, values such as “−1” that are not used for page # or pool #). Stored. The storage location of the moved data (compressed chunk) is recorded in an LP conversion table 500 (FIG. 8) to be described later.

On the other hand, when 1 is stored in the page type (303), the data written to the virtual page has not yet been moved to the write-once space, or the write request from the host 2 has not yet been sent to the virtual page. This means that has not occurred. If the page type (303) is 1 and a valid value (a non-null value) is stored in the page # (304) and the pool # (305), a physical page is allocated to the virtual page, This indicates that write data from the host 2 is stored in the physical page. Conversely, when the page type (303) is 1 and the page # (304) and the pool # (305) are null, a write request from the host 2 has not yet occurred for the virtual page, and the physical page is Indicates not assigned.

Since the virtual volume management table 300 is created for each virtual volume, when there are a plurality of virtual volumes, a plurality of virtual volume management tables 300 exist. When there are a plurality of virtual volumes, each record of the virtual volume management table 300 includes information for specifying a virtual volume (for example, a virtual volume number) in addition to the information such as the virtual page # (301) described above. ) May be included.

FIG. 7 shows an example of the write-once space mapping table 350. The write-once space mapping table 350 is management information for managing physical pages mapped to the write-once space. Since the additional write space is provided for each virtual volume, the additional write space mapping table 350 also exists for each virtual volume. In each line's additional page # (351), additional page numbers are stored in ascending order. Each record of the additional space mapping table 350 indicates that the page specified by the page # (352) and the pool # (353) is allocated to the additional page specified by the additional page # (351). Represents. When no page is assigned to the additional writing page, null is stored in page # (352) and pool # (353) of the additional writing page.

In the last write position (354) in the page, the relative address of the block in which the compressed chunk is written last among the blocks in the additional write page is stored. Specifically, the relative address is an address of a block when the address of the first block in the page is set to 0. In the example of FIG. 7, since the last write position (354) in the page of the record with the additional write page # (351) of “2” is “0x00005”, the additional write page # is changed to the additional write page with “2”. This indicates that a compressed chunk is written in the allocated page from the top of the page to the sixth block. When the storage apparatus 1 adds a compressed chunk to the additional write space, the storage device 1 refers to the final write position (354) in the page and the final write position (354) of the pool management information 600 to write the compressed chunk. Is identified.

Next, the logical / physical conversion table 500 will be described. The logical-physical conversion table 500 is a table for managing the movement destination of each chunk when the chunk on the overwrite space is compressed and moved to the write-once space. FIG. 8 shows an example of the logical-physical conversion table 500. The logical-physical conversion table 500 may be expressed as an LP conversion table 500.

The record of the LP conversion table 500 represents information on the correspondence (mapping) between the chunk on the virtual volume and the recording position of the compressed chunk. Each record is stored in an area of length (504) starting from the address specified by PA (503) after the 8 KB area (chunk) specified by LA (501) of the record is compressed. It represents that.

As described above, since the storage apparatus 1 according to the present embodiment performs the deduplication processing, in principle, control is performed so that a plurality of chunks having the same contents are not stored in the write-once space. The outline of the deduplication processing will be described again with reference to FIG. When the chunk A and chunk B of the VP # 1 of the virtual volume are stored in the write-once space as the compressed chunk a and the compressed chunk b, the storage apparatus 1 additionally writes the VP # 2 chunk. Assume that the process of moving to space is started. If the contents of the chunk D are the same as the chunk B, the compressed chunk of the chunk D is not written to the write-once space. Instead, the storage apparatus 1 uses the same value as the PA (503) of the record storing the chunk B information in the LP conversion table 500, and the PA of the record storing the chunk D information in the LP conversion table 500. (503).

In addition, when the storage apparatus 1 performs the deduplication processing, a check code having a value corresponding to the chunk data is calculated for each chunk. As described above, in this embodiment, this check code is called FPK (FingerPrint Key). In the present embodiment, FKP is information having a size of 8 bytes. However, the FKP may have a size other than 8 bytes. A specific calculation method of FKP in the present embodiment will be described later.

The storage device 1 records the correspondence between the FPK generated from the chunk and the LA of the chunk in the management information. This management information is called FingerPrint Match Queue (hereinafter referred to as “FPMQ”). FIG. 9 shows an example of the FPMQ 400. The FPMQ 400 is a table having columns of FPK (401) and LA (402), and an FPK generated from an 8 KB area (chunk) starting from LA (402) is stored in the FPK (401). Each row (record) stored in the FPMQ 400 is sorted in the ascending order of the FPK (401). Hereinafter, a record of the FPMQ 400, that is, a set of the FPK (401) and the LA (402) may be referred to as “chunk metadata” (or simply “metadata”).

FPMQ 400 is provided for each virtual volume. For example, when the storage device 1 moves a chunk of the virtual volume #n (overwrite space) to the write-once space (this chunk is called “chunk A”), by referring to the FPMQ 400 for the virtual volume #n, It is determined whether the same chunk (compressed chunk) as chunk A already exists in the write-once space. When the same FPK as the FPK of the chunk A does not exist in the FPMQ 400 for the virtual volume #n, it can be determined that the same chunk (compressed chunk) as the chunk A does not exist in the additional write space for the pool #n. In the storage apparatus 1 according to this embodiment, the deduplication range is a virtual volume. Therefore, the same chunk (compressed chunk) as chunk A does not exist in virtual volume #n, but if it exists in a virtual volume other than virtual volume #n, chunk A (compressed chunk) is written to virtual volume #n.

Further, in order to make it possible to efficiently search for information in the FPMQ 400, the storage apparatus 1 has management information that records addresses (addresses on the cache memory 14) in which some records in the FPMQ 400 are stored. . This management information is called FingerPrint Table Directory. Hereinafter, this may be referred to as “FPTD”.

FIG. 9 shows an example of FPTD450. As with the FPMQ 400, the FPTD 450 is also provided for each virtual volume. The FPMQ storage address (452) stores the record having the smallest FPK401 among the records of the FPMQ400, in which the upper 32 bits of the FPK stored in the FPK401 are equal to the first 32 bits (451) of the FPK. An address (address on the cache memory 14) is stored.

For example, in the FPMQ 400, it is assumed that the FPK 401 having the smallest value among the records having the upper 32 bits of the FPK 401 being “0x00000001” is “0x0000000100000001”. At this time, if the address storing the record “0x0000000100000001” in the FPK 401 is “ADR2”, the FPTD 450 has a record in which the first 32 bits (451) of the FPK is “0x00000001” and the FPMQ storage address 452 is “ADR2”. Is recorded.

In the deduplication program 1000 to be described later, after calculating the FPK of the chunk, a process of determining whether the same value as the FPK is stored in the FPK (401) of the FPMQ 400 is performed. Since the search takes time when the number of records stored in the FPMQ 400 is very large, the FPTD 450 is used.

For example, when searching for a record in which “0x0000000100000001” is stored, the deduplication program 1000 refers to the FPTD 450 so that the FPMQ storage address (452) with the FPK leading 32 bits (451) “0x00000001” is “ADR2”. Recognize that there is. Subsequently, the deduplication program 1000 searches the records in the FPMQ 400 for records in which “0x0000000100000001” is stored in order from the record at the address “ADR2” (this is the address on the cache memory 14). The FPTD 450 is information for improving the search speed of information in the FPQ 400, and thus the FPTD 450 is not essential information.

The FPMQ 400 stores chunk metadata for each chunk. If there are many chunks that have generated FPK, the size of the FPMQ 400 increases, and the storage area of the cache memory 14 is consumed. In order to avoid excessive consumption of the storage area of the cache memory 14, the storage apparatus 1 according to the present embodiment performs control not to store the chunk metadata in the FPMQ 400 for chunks that satisfy a predetermined condition. Details of this control will be described later.

FPK upper 32 bits (502) and FPMQ registration (505) in FIG. 8 will be described. The upper 32 bits of the FPK of the chunk are stored in the FPK upper 32 bits (502) of the LP conversion table 500. The FPMQ registration (505) is information indicating whether or not the chunk metadata is registered in the FPMQ 400. When the storage device 1 registers the metadata of the chunk specified by the LA (501) in the FPMQ 400, “1” is recorded in the FPMQ registration (505), and otherwise “0” in the FPMQ registration (505). Is recorded. In addition, after the FPK generated from the chunk (and the LA of the chunk) is registered in the FPMQ 400, the FPK and the LA of the chunk may be deleted from the FPMQ 400. Also in this case, the storage apparatus 1 records “0” in the FPMQ registration (505).

(4) Process Flow Next, a process flow executed by the storage apparatus 1 according to this embodiment will be described. As shown in FIG. 10, at least the I / O program 100, the buffer update program 150, and the deduplication program 1000 are stored in the local memory 16 of the storage apparatus 1 according to the present embodiment. Run the program. The local memory 16 is provided with operation information 300 ′ and a buffer 700. The buffer 700 is a memory area used when the CPU 11 performs compression processing or decompression processing.

The operation information 300 ′ is information in which a part of the information stored in the virtual volume management table 300 (FIG. 6) is stored. Specifically, the operation information 300 ′ has only the virtual page # (301) and the last update time (302) among the records of the virtual volume management table 300. Therefore, the operation information 300 ′ is not shown in this specification.

First, the flow of processing executed by the I / O program 100 will be described with reference to FIGS. In each drawing described below, the alphabet “S” attached before the reference number means “step”.

(4-1) Read Processing When the storage apparatus 1 receives an I / O request from the host 2, execution of the I / O program 100 is started. FIG. 11 shows the flow of read processing executed by the I / O program 100 when the storage apparatus 1 receives a read request as an I / O request from the host 2. FIG. 11 illustrates an example of a case where the data in the area specified by the read request does not exist in the cache memory 14 (case of cache miss).

Step 11: Upon receiving a read request from the host 2, the I / O program 100 converts the read destination address (LBA) included in the read request into a virtual page number. Hereinafter, an example in which the virtual page number of the read destination area is p will be described. In the following, an example in which the read destination area specified in the read request does not extend over a plurality of virtual pages will be described.

Step 12: The I / O program 100 refers to the virtual volume management table 300 and determines whether the page type (303) of the virtual page number (p) obtained in Step 11 is “1”. If the page type (303) is “1” (step 12: Yes), then step 13 is performed. If not (step 12: No), then step 15 is performed.

Step 13: The I / O program 100 refers to the virtual volume management table 300 and identifies the page number (304) of the physical page assigned to the virtual page (VP # p). Further, the I / O program 100 refers to the page management table 200 to identify the DEV # (202) and Addr (203) of the storage device 20 in which the physical page exists.

Step 14: The I / O program 100 reads data from the storage device 20 specified in step 13, returns the read data to the host 2, and ends the processing. Note that the data read here may be stored in the cache memory 14. Then, when the read request for the data is received again later, the I / O program 100 can return the data to the host 2 without accessing the storage device 20.

Step 15: When step 15 is executed, since the data in the area specified by the read request has been moved to the additional write space, the I / O program 100 stores the read target data (compressed chunk). It is necessary to calculate the address (PA) on the write-once space. Hereinafter, an example will be described in which the size of the area specified in the read request is the same as one chunk (8 KB) and this area matches the chunk boundary. In step 15, the chunk I / O program 100 converts the read destination address (LBA) included in the read request into LA.

Step 16: The I / O program 100 refers to the LP conversion table 500, identifies a record in which LA (501) is equal to the LA identified in step 15, and identifies PA (502) and Length (503) of the record. To do. Further, the I / O program 100 calculates the additional page number from the identified PA, and refers to the additional space mapping table 350 to thereby determine the page # of the physical page assigned to the identified additional page. (352) and specify the pool # (353). Further, the I / O program 100 refers to the page management table 200 to identify the DEV # (202) and Addr (203) of the storage device 20 in which the physical page exists.

Step 17: The I / O program 100 reads the data (compressed chunk) requested to be read from the storage device 20 specified in Step 16, stores the data in the buffer 700, and decompresses the compressed chunk stored in the buffer 700. The data decompression process to be returned to the host 2 is executed. Details of the data decompression process will be described later. Thereafter, the I / O program 100 ends the process. Note that the data read here may be stored in the cache memory 14.

(4-2) Write Processing With reference to FIG. 12, the flow of write processing executed by the I / O program 100 when the I / O request from the host 2 is a write request will be described.

Step 101: Upon receiving a write request from the host 2, the I / O program 100 secures an area on the cache memory 14 in order to receive data to be written by the write request. The I / O program 100 stores the write target data in the secured area. Since this is a process performed also in a known storage device, detailed description thereof is omitted. In step 101, the I / O program 100 converts the address (LBA) of the write destination area specified in the write request into a virtual page #. Hereinafter, an example in which the virtual page # in the write destination area is p will be described.

Step 102: The I / O program 100 responds to the host 2 that the write processing has been completed.

Step 103: The I / O program 100 updates the last update time (p) of the record whose virtual page # (301) is p among the records of the operation information 300 ′ (FIG. 10) held in the local memory 16 (FIG. 10). 302) stores the current time (the time when step 103 is executed).

After step 103, the I / O program 100 writes the data stored in the cache memory 14 to the storage device 20 at a predetermined opportunity (step 110). This process is called destage. When data written to a virtual page (assuming this is VP # p) is destaged, if a physical page is allocated to the virtual page, the data is de-stored in the storage device 29 where the allocated physical page exists. Stage. If no physical page is allocated, the physical page is allocated to the virtual page before destage. When the VP # p chunk has been moved to the write-once space, the I / O program 100 adds data to the write-once space. Details of this processing will be described later.

Also, step 110 need not be performed immediately after step 103, and may be performed at any time. For example, it may be executed when the load on the CPU 11 of the storage device 1 is low, or when the amount of data stored in the cache memory 14 exceeds a predetermined threshold.

(4-3) Buffer Update Processing Information recorded in the operation information 300 ′ in step 103 is reflected in the virtual volume management table 300 on the cache memory 14 by the buffer update program 150. The buffer update program 150 is a program that is periodically executed.

When the execution of the buffer update program 150 is started, the buffer update program 150 reflects the contents of the operation information 300 ′ in the virtual volume management table 300. The flow of processing will be described with reference to FIG.

Step 151: The buffer update program 150 selects one record of the operation information 300 '. Hereinafter, a case where the value of the virtual page # (101) of this record is p will be described as an example.

Step 152: The buffer update program 150 records the last update time of the record of the selected operation information 300 ′ at the last update time (302) of the record whose virtual page # (301) is p in the virtual volume management table 300. .

Step 153: The buffer update program 150 deletes the record of the operation information 300 'selected in Step 151 from the local memory 16.

Step 154: When the processing up to step 153 is completed for all the records of the operation information 300 '(step 154: Yes), the processing is terminated. When the record of the operation information 300 ′ still remains in the local memory 16 (Step 154: No), the buffer update program 150 repeats the processing from Step 151.

Since the storage apparatus 1 according to this embodiment performs access via the internal switch 17 when the CPU 11 accesses the cache memory 14, the access performance (response time) is the performance when the CPU 11 accesses the local memory 16. It is not good (becomes longer) than. Therefore, the storage apparatus 1 according to the present embodiment temporarily records the information of the last update time in the operation information 300 ′ on the local memory 16, and regularly reflects the contents on the virtual volume management table 300 on the cache memory 14. Take control.

However, as another embodiment, when the storage apparatus 1 has only the local memory 16 and the virtual volume management table 300 is stored in the local memory 16, it is not necessary to perform such control, and I / O The program 100 may directly update the last update time (302) of the virtual volume management table 300 in step 103. In the case where the storage apparatus 1 has only the cache memory 14 and all the information is stored in the cache memory 14, it is not necessary to perform the control described above, and the I / O program 100 is executed in step 103. The virtual volume management table 300 on the cache memory 14 may be updated.

(4-4) Deduplication Processing Next, the processing of the deduplication program 1000 will be described with reference to FIGS. The deduplication program 1000 performs a process of moving data written in the overwrite space from the host 2 (this data is stored in a physical page once assigned to the overwrite space) to a page assigned to the additional write space. It is a program. As described above, when data (chunk) is moved to a page allocated to the write-once space, the chunk is compressed. Deduplication processing may be performed.

Step 1010: The deduplication program 1000 refers to the virtual volume management table 300 and selects a virtual page for moving data to the write-once space. The virtual page selection criteria may be arbitrary. As an example, a virtual page with a page type (303) of “1” is selected such that the difference between the current time and the last update time (302) is equal to or greater than a predetermined threshold and the last update time (302) is the oldest. Good. Since the data (chunk) stored in the write-once space is compressed, the response time at the time of access becomes long. Therefore, it is preferable that data with low access frequency is arranged in the write-once space rather than data with high access frequency.

Step 1020: The deduplication program 1000 reads the data of the virtual page selected in Step 1010 from the storage device 20 onto the cache memory 14. If such virtual page data already exists on the cache memory 14, this step 1020 need not be executed.

Step 1030: The deduplication program 1000 selects a chunk from the data read onto the cache memory 14 in step 1020. The method for selecting chunks is arbitrary. For example, you may make it select in order from the chunk with small LA.

Step 1040: The deduplication program 1000 transfers the chunk selected in step 1030 (or step 1080 described later) to the buffer 700.

Step 1050: The deduplication program 1000 calculates the FPK of the chunk transferred to the buffer 700, stores it in the LP conversion table 300, and generates a compressed chunk by compressing the data of the chunk. Details of this processing (hereinafter referred to as FPK calculation / data compression processing) will be described later. The compressed chunk generated here is temporarily stored in the buffer 700.

Step 1060: In step 1050, it is determined whether or not data compression is successful. If the data compression is successful (step 1060: Yes), step 1070 is performed next. If not (step 1060: No), step 1080 is performed next.

Step 1070: In this step, the deduplication program 1000 stores the compressed chunk in the write-once space 60. In addition, deduplication processing is also performed as necessary. Details of this processing will be described later.

Step 1080: The deduplication program 1000 selects the next chunk (chunk that has not yet been processed in steps 1040 to 1070) from the data read onto the cache memory in step 1020.

Step 1090: If the next chunk cannot be selected in Step 1080, that is, if the processing of Step 1040 to Step 1070 for all the chunks is completed (Step 1090: Yes), Step 1100 is performed next. If the next chunk can be selected in Step 1080 (Step 1090: No), the deduplication program 1000 performs the processing from Step 1040 again.

Step 1100: The deduplication program 1000 discards the data in the overwrite space. Specifically, the deduplication program 1000 updates the contents of the virtual volume management table 300 so that the page allocated to the virtual page selected in step 1010 is not allocated to the virtual page (this virtual The page type (303) of the page is “0”, and “null” is stored in page # (304) and pool # (305). Further, the deduplication program 1000 sets the value of the use (204) of the allocated physical page of the records of the page management table 200 for this physical page to make the allocated physical page unused. Is updated to “0”.

Here, the details of the processing performed in step 1070 will be described with reference to FIG.

Step 6010: The deduplication program 1000 searches whether the same FPK as the FPK obtained in step 1050 is stored in the FPMQ 400. As described above, the deduplication program 1000 searches the FPMQ 400 record in which the same FPK as the FPK obtained in step 1050 is stored by referring to the FPTD 450 and the FPMQ 400.

Step 6020: If the same FPK as the FPK obtained in step 1050 is stored in the FPMQ 400 (step 6020: Yes), the deduplication program 1000 next performs step 6030. If the same FPK as the FPK obtained in step 1050 is not stored in the FPQM 400 (step 6020: No), step 6070 is performed next.

Step 6030: The deduplication program 1000 refers to the FPMQ 400, the LP conversion table 500, the write space mapping table 350, and the page management table 200, so that a chunk (compression chunk) having the same FPK as the FPK obtained in Step 1050 is obtained. Are read from the storage device 20.

Specifically, the deduplication program 1000 first refers to the FPMQ 400 and the LP conversion table 500 to determine the PA in which the chunk (compression chunk) having the same FPK as the FPK obtained in step 1050 is stored and its length. Identify. Hereinafter, this chunk is referred to as “overlapping candidate chunk”, and the PA of the overlapping candidate chunk is referred to as “PA1”. Subsequently, the deduplication program 1000 identifies the page number of the page (physical page) where the duplication candidate chunk is stored by referring to the write-once space mapping table 350, and further refers to the page management table 200. The storage device 20 and the address where the page storing the duplicate candidate chunk exists are specified. This is the same processing as step 16 performed by the I / O program 100. Then, the duplicate elimination program 1000 reads duplicate candidate chunks from the specified storage device 20.

Step 6040: The deduplication program 1000 compares the chunk read in step 6030 (compressed chunk) with the contents of the chunk compressed in step 1060, and determines whether or not they are the same. At the time of comparison, both are in a compressed state, but the deduplication program 1000 compares the compressed chunks in byte units. However, as another embodiment, the deduplication program 1000 may temporarily decompress the compressed chunk and compare the decompressed data. As a result of the comparison, if the two match (step 6040: Yes), step 6050 is performed next. When both are different (step 6040: No), step 6070 is performed next.

Step 6050: The deduplication program 1000 registers the chunk FPK and the chunk LA in the FPMQ 400. Further, the deduplication program 1000 stores “1” in the FPMQ registration (505) of the record whose LA (501) matches the LA of the processing target chunk among the records of the LP conversion table 500.

Step 6060: The duplicate elimination program 1000 stores the PA (503) of the duplicate candidate chunk in the PA (503) of the record whose LA (501) matches the LA of the processing target chunk among the records of the LP conversion table 500. Then, the length of the duplication candidate chunk is stored in Length (504). In this case, the compressed chunk is not stored in the write-once space. Thereafter, the deduplication program 1000 performs the processing after step 1080.

Step 6070: The processing content of this step is the same as step 6050.

Step 6080: The deduplication program 1000 writes the compressed chunk temporarily stored in the buffer 700 to the write-once space, and deletes the compressed chunk in the buffer 700. As described above, when the deduplication program 1000 writes a compressed chunk in the write-once space, it additionally writes it in the write-once space. Here, the flow of the process of adding the compressed chunk to the additional writing space will be described with reference to FIG.

First, the deduplication program 1000 determines the address (PA) on the write-once space where the compressed chunk is to be added (step 51). The storage apparatus 1 holds information on the final write position of the write space in the pool management information 600 and the write space mapping table 350. The deduplication program 1000 refers to the last write position (602) of the corresponding virtual volume in the pool management information 600, thereby adding the last page to be written in the write space corresponding to the virtual volume. Specify writing page #. Hereinafter, this is referred to as AP1. Subsequently, the deduplication program 1000 refers to the additional write space mapping table 350 to identify the final write position (354) in the page of the record of the additional write page # (351). Hereinafter, this is denoted as RA1. Then, the deduplication program 1000 calculates “AP1 × page size (42 MB) ÷ 512 + RA1 + 1” to obtain an address (PA) on the additional write space where the compressed chunk is to be written. Hereinafter, a case where the PA to which the compressed chunk is written is determined as “PA2” and the length of the compressed chunk is “L2” will be described.

Subsequently, the deduplication program 1000 determines whether a physical page is allocated to the additional page including the PA (PA2) determined in step 51 (step 52). For this purpose, the deduplication program 1000 obtains the additional page # of the additional page including PA2 by dividing PA2 by the page size (42 MB) (hereinafter, the additional page # obtained here is “ AP2 "). Further, the deduplication program 1000 refers to the write space mapping table 350, so that a valid value (a value other than null) is stored in the page # (352) of the record of AP2 for the write page # (351). judge. When a valid value is stored in page # (352), a physical page is assigned to the additional page including PA2.

When the physical page is not allocated to the additional page including PA2 (step 52: No), the deduplication program 1000 allocates the physical page to the additional page (step 53). Specifically, the deduplication program 1000 selects one unused physical page from the page management table 200, and each page # and the pool # of the pool to which the page belongs are added to the additional page # (351) as AP2. Are stored in page # (352) and pool # (353). The deduplication program 1000 stores “1” in the use (204) of the physical page selected from the page management table 200. Conversely, if a physical page is assigned to PA2 (step 52: Yes), step 53 is not performed.

In step 54, the deduplication program 1000 refers to the write space mapping table 350 and the page management table 200 to calculate the address of the storage device 20 to which the compressed chunk is written. Subsequently, the deduplication program 1000 writes the compressed chunk at the address calculated in step 54 (step 55). In step 56, the deduplication program 1000 substitutes the additional write page # of the additional write page in which the compressed chunk has been written this time for the final write position (602) of the pool management information 600, and in the last written write page. Is substituted into the last write position (354) in the page of the additional write space mapping table 350, and the process ends.

The above is the processing performed in step 6080. Returning to the description of FIG.

Step 6090: The deduplication program 1000 stores the compression chunk PA (that is, PA2) in the PA (502) of the record whose LA (501) matches the LA of the processing target chunk among the records of the LP conversion table 500. , Length (503) stores the length (L2) of the compressed chunk. Thereafter, the deduplication program 1000 performs the processing after step 1080.

(4-5) Destage Processing The flow of destage processing executed by the I / O program 100 at step 110 in FIG. 12 will be described with reference to FIG.

Step 5010: The I / O program 100 selects data to be processed from each data stored in the cache memory 14. There are various methods for selecting data. As an example, there is a method of selecting data with the oldest time written from the host 2. In this case, the storage controller 10 needs to store the time written from the host 2 for each data stored in the cache memory 14. Hereinafter, an example in which data for one chunk is selected will be described. However, as another embodiment, a plurality of chunks, for example, a plurality of chunks continuous on the virtual volume may be selected.

Step 5020: The I / O program 100 specifies the virtual page to which the data selected in Step 5010 is written. Hereinafter, an example in which VP # p is specified in step 5020 will be described. Then, the I / O program 100 refers to the virtual volume management table 300 and determines whether the page type (303) is “0” or “1” for the record whose virtual page # (301) is “p”. If the page type (303) is “1” (step 5020: No), the data write destination is the overwrite space, so step 5110 is executed next. If the page type (303) is “0” (step 5020: Yes), then step 5030 is performed.

Step 5030: When this step is executed, it means that the data written in VP # p has been moved to the write-once space. In this case, the I / O program 100 determines whether the data of VP # p that has been moved to the write-once space is returned to the overwrite space again.

For example, when almost all chunks in VP # p are updated, the I / O program 100 determines to return the data of VP # p that has been moved to the write-once space to the overwrite space again. In this case, since VP # p may be frequently updated in the future, if data is compressed and stored every time destage is performed, the overhead of write processing increases. However, this determination method is only an example, and it may be determined whether or not the data of VP # p that has been moved to the write-once space is returned to the overwrite space by another determination method. If the determination at step 5030 is negative, then step 5040 is performed, and if the determination at step 5030 is positive, then step 5210 is performed.

Step 5040: The I / O program 100 compresses the processing target data. This is the same as the processing performed in steps 1040 to 1060 in FIG. Similar to steps 1040 to 1060, the I / O program 100 also calculates FPK.

Step 5050: The I / O program 100 stores the processing target data (compressed chunk) in the write-once space. This process is the same as step 1080 in FIG. That is, the processing from step 6010 to step 6090 in FIG. 15 is performed. After step 5050, the processing target data is deleted from the cache memory 14, and the processing target data and its compressed chunk are deleted from the buffer 700, and the destage processing ends.

Step 5210: When it is determined that the data of VP # p that has been moved to the write-once space is returned to the overwrite space, the I / O program 100 stores the data of VP # p that has been moved to the write-once space. All are read from the storage device 20, decompressed data, and temporarily stored in the cache memory 14. For this, the same processing as that described with reference to FIG. 11 may be performed.

Step 5220: The I / O program 100 allocates a physical page to VP # p. Specifically, the I / O program 100 selects one unused physical page from the page management table 200, and each of the page # and the pool # of the pool to which the page belongs are stored in the virtual volume management table 300. The virtual page # (301) is stored in the page # (304) and the pool # (305) of the record of p. The I / O program 100 stores “1” in the use (204) of the physical page selected from the page management table 200.

Step 5230: The I / O program 100 stores the data read in Step 5210 in the physical page assigned to VP # p. Specifically, the I / O program 100 refers to the page management table 200 as in Step 13 of FIG. 11, and the DEV # (202) of the storage device 20 in which the physical page allocated to VP # p exists. , Addr (203) is specified. The I / O program 100 stores the data read in step 5210 in the specified storage device.

Step 5240: The I / O program 100 overwrites the processing target data on the physical page assigned to VP # p, and ends the destage processing.

Step 5110: The I / O program 100 allocates a physical page if no physical page is allocated to VP # p. This is the same processing as in step 5220. However, if a physical page has already been allocated to VP # p, it is not necessary to allocate a physical page.

Step 5120: Similar to step 5240, the I / O program 100 overwrites the processing target data on the physical page assigned to VP # p, and ends the destage processing.

Note that the destage processing described here is an example, and processing different from the processing described above may be performed. For example, when the data of VP # p has been moved to the write-once space, the write-once space may be necessarily added without performing the determination in step 5030.

Alternatively, when returning data from the write-once space to the overwrite space, control may be performed to reduce the amount of data to be returned. For example, when a request to update the entire page of VP # p is received from the host 2, data for one page of VP # p is stored in the cache memory 14. In this state, even if data is returned from the write-once space to the overwrite space, if the data stored in the cache memory 14 is destaged, the data written back from the write-once space to the overwrite space is overwritten. Therefore, in such a case, it is more efficient to prevent data from being returned from the write-once space to the overwrite space.

(4-6) FPK Calculation / Data Compression Processing Next, the flow of the FPK calculation / data compression processing executed by the deduplication program 1000 in step 1050 of FIG. 14 will be described with reference to FIGS. In the following, as shown in FIG. 19A, an 8-byte guarantee code 1200 called DIF (Data Integrity Field) is added to the 512-byte data 1100 by the host 2 as shown in FIG. The explanation will proceed as if The DIF 1200 is information composed of a 2-byte error correction code (hereinafter referred to as CRC (Cyclic Redundancy Code)) 1210 and a 6-byte LA 1220 indicating the address of the data write destination managed by the host 2 side. .

Step 7010: Each guarantee added to every 512 bytes of data 1100 of the chunk (hereinafter referred to as a target chunk) transferred to the buffer 700 (FIG. 10) of the local memory 16 (FIG. 10) in Step 1040 of FIG. As shown in FIG. 19B, zero padding is executed to fill the LA 1220 of the code 1200 with “0” (replace it with “0”).

The reason why the LA 1220 of the target chunk is zero-patented is that the value of the LA 1120 differs depending on the data writing position. Therefore, when the compression processing is performed on the data 1100 with the LA 1120 added to the write target data 1100, This is because even if the data has the same contents, different compression results are obtained, and the same data cannot be deduplicated. Further, when the LA 1220 is compressed without zero padding, the compression effect may be reduced. Therefore, in the present embodiment, when the write target data 1100 is compressed, first, a process of zero-padding the LA 1220 is executed. Thus, the compression effect is expected to be improved by zero-padding LA1220.

Step 7020: The deduplication program 1000 calculates the FPK of the data of the target chunk. Specifically, the deduplication program 1000 generates a CRC group in which the above-described CRC 1210 added every 512 bytes of the target chunk is combined. In the present embodiment, since 2 bytes of CRC 1210 are added per 512 bytes, a CRC group of 2 bytes × 16 sub-blocks = 32 bytes is obtained for each chunk (8 KB) of data. Since the amount of data is large with 32 bytes, a 20-byte hash value is calculated from a 32-byte CRC group using a predetermined hash function (for example, SHA-1), and the upper 8 bytes of the 20-byte hash value are further calculated. Is taken out, and this is used as the FPK of the target chunk. In this way, by calculating the FPK of the target chunk using the CRC added to every 512 bytes of the target chunk, it is more suitable for the content of the target chunk than when calculating the FPK using the data of the entire target chunk. The FPK can be calculated efficiently.

Step 7030: As shown in FIG. 19C, the deduplication program 1000 deletes all CRCs 1210 assigned to each 512 bytes of the data of the target chunk stored in the buffer 700 from the data of the target chunk. .

Step 7040: The deduplication program 1000 compresses the data of the target chunk stored in the buffer 700 (including the zero-padded LA 1220 represented by reference numeral 1220 ′ in FIG. 19C). Generate compressed chunks. The compressed chunk is temporarily stored in the buffer 700 together with the data before the target chunk is compressed.

Here, the reason for compressing the data of the target chunk while adding the zero-padded LA 1220 is to improve the performance. In practice, if the LA 1220 is deleted from the data of the target chunk and left-justified, it is necessary to create a gap for re-inserting the LA 1220 for every 512 bytes of the decompressed data after decompressing the data. On the other hand, when the LA 1220 is not deleted, it is only necessary to rewrite the LA 1220 after decompressing the read data, and the overhead required for memory copy can be reduced. For example, when LA1220 is padded with 0 as described above, data write of 6B × 16 subblocks = 96B is sufficient for 8 KB, but when LA1220 is deleted from the data of the target chunk and left-justified, 514 bytes × In addition to the memory copy of 16 sub-blocks = 8224 bytes, the same memory copy is generated after decompressing the data.

Step 7050: The deduplication program 1000 decompresses the compressed chunk obtained in step 7040.

Step 7060: The deduplication program 1000 uses the data obtained by decompressing the compressed chunk in Step 7050 (hereinafter referred to as decompressed data), the data before compression of the target chunk stored in the buffer 700, and Compare This comparison is performed by sequentially comparing the decompressed data and the data before compression of the target chunk in units of 1 bit or 1 byte. By such comparison, it is possible to confirm whether or not the original data can be restored when the data of the target chunk is compressed and decompressed, and it is possible to ensure reliability with respect to data compression and decompression.

Step 7070: As a result of the comparison in Step 7060, when the decompressed data matches the data before compression of the target chunk (Step 7070: Yes), Step 7080 is performed next, otherwise (Step 7070: No) Next, step 7100 is performed.

Step 7080: The deduplication program 1000 calculates a new CRC for the compression chunk of the target chunk stored in the buffer 700, and the calculated CRC and the logical address of the compression chunk on the buffer 700 (hereinafter referred to as LA ′). To the compressed chunk.

Specifically, the deduplication program 1000 divides the compressed chunk of the target chunk into 512 bytes, and as shown in FIG. 19D, the data of these 512 byte compressed chunks (hereinafter referred to as compressed chunk classified data). ) For each 1110, calculate the CRC 1230 of the compressed chunk classification data 1110. The deduplication program 1000 uses the compressed chunk partition data corresponding to the CRC 1230 for each compressed chunk partition data 1110 calculated in this way and the logical address (LA ′) 1240 of the compressed chunk partition data 1110 on the buffer 700. It is added to 1110. Note that LA ′ 1240 added to the compressed chunk partition data 1110 at this time, as shown in FIG. 19 (E), when the compressed chunk partition data 1110 of the target chunk is subsequently written to the additional write space (see FIG. 19). 16 step 55), the address (LA ″) 1250 in the write-once additional write space is rewritten.

Step 7090: The deduplication program 1000 registers the FPK of the target chunk calculated in step 7020 in the FPK upper 32 bits (502) of the corresponding record in the LP conversion table 500 (FIG. 8), and thereafter ends this processing. .

Step 7100: It is determined that the data of the target chunk is handled without being compressed, and thereafter, this processing is terminated.

(4-7) Data Expansion Processing Next, the flow of data expansion processing executed by the I / O program 100 at step 17 in FIG. 12 will be described using FIG. 20 and FIG.

Step 8010: The I / O program 100 compresses the read target data (based on the DEV # (202) and Addr (203) of the storage device 20 specified in step 16 of the read process described above with reference to FIG. (Compressed data) is read from the storage device 20 into the buffer 700.

Step 8020: The I / O program 100 determines the integrity of the compressed data read out to the buffer 700 based on the CRC included in every 512 bytes (corresponding to the new CRC 1230 in FIG. 19E). A CRC check to be checked is executed (502). Specifically, as shown in FIG. 21 (A), the I / O program 100, for every 512 bytes of each compressed chunk constituting the read compressed data (that is, for each compressed chunk division data 1110 described above) The CRC is calculated based on the compressed chunk partition data 1110, and the calculated CRC is compared with the CRC 1230 added to the compressed chunk partition data 1110.

Step 8030: As a result of the CRC check in Step 8020, the I / O program 100 adds one of the compressed chunk partition data to the CRC calculated based on the compressed chunk partition data 1110 and the compressed chunk partition data 1110. It is determined whether or not a CRC error that does not match the CRC 1230 has occurred. If the CRC calculated based on the compressed chunk partition data 1110 and the CRC 1230 added to the compressed chunk partition data 1110 match for all the compressed chunk partition data 1110 (step 8030: Yes). Next, Step 8100 is performed, and the CRC calculated based on the compressed chunk partition data 1110 does not match the CRC 1230 added to the compressed chunk partition data 1110 for any one of the compressed chunk partition data. If this is the case (step 8030: No), then step 8040 is performed.

Step 8040: As shown in FIG. 21B, the I / O program 100 adds a CRC 1230 that is added to every 512 bytes of the compressed data (that is, every compressed chunk division data 1110) from the compressed data read in Step 8010. And LA ″ are deleted, and the compressed data is decompressed.

Step 8050: The I / O program 100 generates an FPK for each chunk of decompressed data obtained by the decompression process in Step 8010. Specifically, the I / O program 100 calculates a CRC for each 512 bytes of decompressed data, and generates a CRC group that combines the calculated CRCs into one. In the present embodiment, since a CRC of 2 bytes is added per 512 bytes, a CRC group of 2 bytes × 16 subblocks = 32 bytes is obtained for each chunk (8 KB) of data. In addition, the I / O program 100 uses the hash function used when calculating the FPK in step 7020 of the FPK calculation / data compression processing from the CRC group for each chunk obtained in this way to obtain a 20-byte hash value. For each chunk, the upper 8 bytes of the calculated hash value are extracted as the FPK of the chunk.

Step 8060: For each chunk, the FPK generated in step 8050 is compared with the FPK stored in the FPK upper 32 bits (502) of the LP conversion table for the chunk.

Step 8070: For all chunks, if the FPK generated in step 8050 matches the FPK stored in the FPK upper 32 bits (502) of the LP conversion table 500 for that chunk (step 8070: Yes), Next, Step 8080 is performed. If not (Step 8070: No), Step 8120 is performed next.

Step 8080: The I / O program 100, as shown in FIG. 21C, provides an error correction code for each 512 bytes of the decompressed data calculated in the decompression process of Step 8040 (hereinafter referred to as CRC (Cyclic Redundancy Code)). 1260 is added to the corresponding 512-byte data (step 8080). In addition, the I / O program 100 also stores in the LA 1270 storing the data that the host 2 recognizes LA ″ (see FIG. 19E) added every 512 bytes of the data. Write back.

Step 8090: The I / O program 100 returns the decompressed data 1120 to which the CRC 1260 and LA 1270 are added every 512 bytes in step 8080 to the host 2 and ends the process.

Step 8100: The I / O program 100 executes an error response for notifying the host 2 that has transmitted the read request that a read error has occurred, and ends the process.

(5) Effects of this Embodiment As described above, in the storage apparatus 1 according to this embodiment, an overwrite space is allocated to a virtual volume in units of 42 MB virtual pages, and data written to the virtual volume is chunked in 8 KB. Since compression is performed in units, decompression processing of the compressed data is performed in units of 8 KB chunks. For example, compared to data compression in units of virtual pages, even unnecessary data at the time of decompression processing of compressed data It is possible to significantly reduce the amount of extra overhead that is extended. Even if the data compression unit is 8 KB, the virtual volume can be managed in units of 42 MB virtual pages. Thus, it is possible to prevent an increase in the amount of management information while preventing a decrease in access performance of the storage apparatus due to such overhead.

Further, in the storage device 1 according to the present embodiment, since the uncompressed data is written in the overwrite area and the compressed data is stored in the additional write area, the data size after compression of the updated data is larger than that before the update. Even when it becomes larger, it can be stored in the storage device 20 without any problem.

Therefore, according to the present embodiment, a highly useful storage apparatus can be realized.

The present invention can be applied to a storage apparatus equipped with a data compression function.

DESCRIPTION OF SYMBOLS 1 ... Storage device, 2 ... Host, 10 ... Storage controller, 11 ... CPU, 14 ... Cache memory, 16 ... Local memory, 20 ... Storage device, 40 ... Overwrite space, 60 ... Additional Write space, 100 ... I / O program, 700 ... Buffer, 500 ... LP conversion table, 1000 ... Deduplication program.

Claims (12)

In a storage apparatus that provides a virtual volume, which is a virtual storage space, to a host, and dynamically allocates a storage area in response to a data write request to the virtual volume from the host.
One or more storage devices;
A storage controller that dynamically allocates a storage area of the storage device to the virtual volume, and writes data to be written given from the host together with the write request to the storage area allocated to the virtual volume, and
The storage controller
The storage area of the storage device is divided into an overwrite area allowing overwriting and an additional writing area for additionally writing and managing,
The overwriting area is allocated to the area on the virtual volume in a predetermined first capacity unit, the data to be written is written to the allocated overwriting area, and the data written to the overwriting area is A storage apparatus, comprising: compressing in a second capacity unit smaller than one capacity, and storing the compressed data in the additional write area. The storage controller
The check code of the data before compression is calculated in the second capacity unit, the check code of the calculated data in the second capacity unit is stored, and the data in the second capacity unit is compressed. And store it in the area for additional writing,
In response to a read request from the host, the data stored in the additional writing area is read out from the additional writing area and decompressed, and a check code of the decompressed data is calculated for each second capacity unit, and is calculated. The check code is compared with the corresponding check code stored before compression, and when the two match, the decompressed data is returned to the host. Storage device. The storage controller
The storage apparatus according to claim 2, wherein the check code is calculated based on an error correction code added to data from the host in a predetermined unit. The storage controller
All of the error correction code added to the data from the host is deleted, and the address of the write destination of the data managed by the host added to the data in the predetermined unit together with the error correction code is zero. Padded, compressed with the zero padded address and written to the additional write area,
The address stored in the additional writing area is read out from the additional writing area in response to a read request from the host and expanded, and the zero-padded address added to the expanded data in the predetermined unit The storage apparatus according to claim 3, wherein the address is rewritten to the address of the data write destination managed by the host and returned to the host. The storage controller
Comparing the calculated check code of the second capacity unit data before compression with the check code before compression of other data stored in the additional writing area after being compressed;
If the check code of the second capacity unit data does not match the check code of any other data compressed and stored in the additional write area, the second capacity unit data While compressing the data and storing it in the additional write area,
When the check code of the second capacity unit data matches the check code of any other data compressed and stored in the additional write area, the second capacity unit The compressed data of the data is not stored in the additional write area, and the storage position of the data in the second capacity unit in the additional write area is the same as the other check code in the additional write area. The storage apparatus according to claim 2, wherein the storage apparatus is managed as a data storage location. The storage controller
The data written in the overwriting area is compressed after being compressed in the second capacity unit, the expanded data is compared with the compressed data, and if the two match, the compressed data is The storage device according to claim 1, wherein the storage device is stored in a write-once area. In a control method of a storage apparatus that provides a virtual volume that is a virtual storage space to a host and dynamically allocates a storage area in response to a data write request from the host to the virtual volume.
The storage device
One or more storage devices;
A storage controller that dynamically allocates a storage area of the storage device to the virtual volume, and writes data to be written given from the host together with the write request to the storage area allocated to the virtual volume, and
The storage controller
The storage area of the storage device is divided into an overwrite area allowing overwriting and an additional writing area for additionally writing and managing,
A first step in which the storage controller allocates the overwrite area in a predetermined first capacity unit to an area on the virtual volume, and writes the data to be written to the allocated overwrite area;
A second step in which the storage controller compresses data written in the overwrite area in a second capacity unit smaller than the first capacity, and stores the compressed data in the additional write area; And a storage apparatus control method. In the second step, the storage controller calculates a check code of the data before compression in the second capacity unit, and stores each check code of the calculated data in the second capacity unit. And compressing the data of the second capacity unit and storing it in the additional write area,
In response to a read request from the host, the storage controller reads and expands data stored in the additional write area from the additional write area, and outputs a check code for the expanded data in the second capacity unit. And a third step of comparing the calculated check code with the corresponding check code stored before compression and returning the decompressed data to the host if they match. The storage apparatus control method according to claim 7, wherein: The storage controller
The method of controlling a storage apparatus according to claim 8, wherein the check code is calculated based on an error correction code added to data from the host in a predetermined unit. In the second step, the storage controller
All of the error correction code added to the data from the host is deleted, and the address of the write destination of the data managed by the host added to the data in the predetermined unit together with the error correction code is zero. Padded, compressed with the zero padded address and written to the additional write area,
In the third step, the storage controller
The address stored in the additional writing area is read out from the additional writing area in response to a read request from the host and expanded, and the zero-padded address added to the expanded data in the predetermined unit The storage apparatus control method according to claim 8, wherein the data is rewritten to the address of the data write destination managed by the host and returned to the host. In the second step, the storage controller
Comparing the calculated check code of the second capacity unit data before compression with the check code before compression of other data stored in the additional writing area after being compressed;
If the check code of the second capacity unit data does not match the check code of any other data compressed and stored in the additional write area, the second capacity unit data While compressing the data and storing it in the additional write area,
When the check code of the second capacity unit data matches the check code of any other data compressed and stored in the additional write area, the second capacity unit The compressed data of the data is not stored in the additional write area, and the storage position of the data in the second capacity unit in the additional write area is the same as the other check code in the additional write area. 9. The storage apparatus control method according to claim 8, wherein the storage apparatus is managed as a data storage location. In the second step, the storage controller
The data written in the overwriting area is compressed after being compressed in the second capacity unit, the expanded data is compared with the compressed data, and if the two match, the compressed data is The storage device control method according to claim 7, wherein the storage device is stored in a write-once area.

Images