CN118377440B

CN118377440B - Buffer management method, device, solid state hard disk and storage medium

Info

Publication number: CN118377440B
Application number: CN202410823909.0A
Authority: CN
Inventors: 赵梓伸; 付凤之; 李瑞东; 高美洲; 王陈园; 王晓琦
Original assignee: Shandong Yunhai Guochuang Cloud Computing Equipment Industry Innovation Center Co Ltd
Current assignee: Shandong Yunhai Guochuang Cloud Computing Equipment Industry Innovation Center Co Ltd
Priority date: 2024-06-25
Filing date: 2024-06-25
Publication date: 2024-12-06
Anticipated expiration: 2044-06-25
Also published as: CN118377440A

Abstract

The invention relates to the technical field of storage equipment and discloses a buffer area management method, equipment, a solid state disk and a storage medium, wherein the method comprises the steps of setting a working mode of a buffer manager; and determining a single buffer space corresponding to the second read/write instruction in response to the second read/write instruction in the case that the buffer manager is in the discrete buffer mode, and performing discrete read/write operations according to the single buffer space. The invention can realize the management of the continuous buffer space and the single discrete buffer space, thereby optimizing the data read-write operation, improving the utilization efficiency and turnaround of the read-write buffer area in the storage system, reducing the waiting time of the read-write buffer area and improving the read-write performance of the solid-state disk.

Description

Buffer management method, equipment, solid state disk and storage medium

Technical Field

The invention relates to the technical field of storage equipment, in particular to a buffer area management method, equipment, a solid state disk and a storage medium.

Background

The storage device can store data in the storage space and execute corresponding read-write operation based on a read-write command issued by a Host (Host). Solid state disk (Solid STATE DRIVE, SSD) is increasingly used in various scenes due to good read-write performance.

In order to improve the read-write performance, a part of storage space in the SSD is reserved to be used as a Buffer, and how to efficiently manage allocation and release of the Buffer so as to ensure the read-write performance of the SSD is a technical problem to be solved urgently.

Disclosure of Invention

In view of the above, the present invention provides a method, an apparatus, a solid state disk and a storage medium for managing a buffer area, so as to solve the problem of low read-write performance of an SSD.

In a first aspect, the present invention provides a method for managing a buffer, applied to a buffer manager, including:

Setting the working mode of the buffer manager, wherein the working mode comprises a continuous buffer mode and a discrete buffer mode;

Determining a plurality of continuous buffer spaces corresponding to a first read-write instruction in response to the first read-write instruction under the condition that the buffer manager is in a continuous buffer mode, and executing continuous read-write operation according to the plurality of continuous buffer spaces;

in response to a second read/write instruction, determining a single buffer space corresponding to the second read/write instruction, and performing discrete read/write operations according to the single buffer space, with the buffer manager in a discrete buffer mode.

In some alternative embodiments, the method further comprises:

at least one buffer identification list is configured according to the set working mode, and the buffer identification list is used for recording buffer identifications corresponding to the buffer space;

under the condition that the set working mode is a continuous buffer mode, each bit in the buffer identification list corresponds to a corresponding buffer identification;

and under the condition that the set working mode is a discrete buffer mode, r bits in the buffer identification list correspond to corresponding buffer identifications, and r is more than 1.

In some alternative embodiments 2^r is not less than the number of buffer identities in the list of buffer identities.

In some optional embodiments, the configuring at least one buffer identification list according to the set operation mode includes:

setting the depth of the buffer identification list as a first depth under the condition that the set working mode is a continuous buffer mode, and setting the buffer identification list to be full of buffer identifications;

and setting the depth of the buffer identification list as a second depth and setting the buffer identification list as empty under the condition that the set working mode is a discrete buffer mode.

In some optional embodiments, the buffer identification list is provided with a head pointer and a tail pointer;

the configuring at least one buffer identification list according to the set working mode further comprises:

Under the condition that the set working mode is a continuous buffer mode, pointing the head pointer of the buffer identification list to the tail part of the buffer identification list, and pointing the tail pointer of the buffer identification list to the head part of the buffer identification list;

And under the condition that the set working mode is a discrete buffer mode, the head pointer and the tail pointer of the buffer identification list point to the same position.

In some optional embodiments, the configuring at least one buffer identification list according to the set operation mode further includes:

checking whether the highest bit of the buffer identification list is a preset value or not under the condition that the set working mode is a continuous buffer mode;

And under the condition that the highest bit of the buffer identification list is a preset value, determining that the configuration of the buffer identification list is completed.

In some optional embodiments, the performing continuous read-write operations according to the plurality of continuous buffer spaces includes determining a plurality of continuous buffer identifications corresponding to a plurality of currently required buffer spaces in the buffer identification list, and then releasing the plurality of continuous buffer identifications in the buffer identification list;

The performing discrete read-write operations according to the single buffer space includes writing a target buffer identification corresponding to the single buffer space into the buffer identification list after the flash memory controller or processor releases the target buffer identification.

In some optional embodiments, the buffer identification list is provided with a head pointer and a tail pointer, and the method further comprises:

triggering a tail pointer of the buffer identification list to be increased by one after the plurality of continuous buffer identifications in the buffer identification list are released;

and triggering a head pointer of the buffer identification list to be increased by one after the target buffer identification is written into the buffer identification list.

In some alternative embodiments, the method further comprises:

And responding to a mode switching instruction, switching from a current first mode to another second mode, wherein the first mode is a continuous buffer mode, the second mode is a discrete buffer mode, or the first mode is a discrete buffer mode, and the second mode is a continuous buffer mode.

In some alternative embodiments, the switching from the current first mode to the other second mode includes:

Releasing all buffer marks in the buffer mark list under the condition that the first mode is a continuous buffer mode, and then switching to the discrete buffer mode;

and when the first mode is a continuous buffer mode, adding all released buffer identifications into the buffer identification list, and then switching to the continuous buffer mode.

In some alternative embodiments, the mode switch instruction is generated based on:

when the current garbage data quantity exceeds a first preset threshold value under the condition that the first mode is a continuous buffer mode, generating a mode switching instruction for switching to a discrete buffer mode;

and when the current garbage data quantity is smaller than a second preset threshold value under the condition that the first mode is a discrete buffer mode, generating a mode switching instruction for switching to a continuous buffer mode.

In some alternative embodiments, the list of buffer identification is configured for exclusive use by one processor core, or the list of buffer identification is configured for shared use by multiple processor cores.

In some alternative embodiments, the buffer identification list is stored in a corresponding buffer register, and the bit width of the buffer register is consistent with the bit width of the buffer identification list.

In some alternative embodiments, the method further comprises:

Collecting a target data page in the single buffer space with the buffer manager in a discrete buffer mode;

Determining a data classification for the target data page, the data classification comprising a cold data page and a hot data page;

aggregating the cold data page into a cold data block if the target data page is a cold data page;

And aggregating the hot data page into a hot data block in the case that the target data page is the hot data page.

In some alternative embodiments, the determining the data classification of the target data page includes:

Extracting characteristics of the target data page, and determining characteristic information of the target data page, wherein the characteristic information comprises at least one of attribute information, time information, a logical block address and a physical block address of the target data page;

and inputting the characteristic information of the target data page into a pre-trained classification prediction model, and determining the data classification of the target data page according to the classification prediction result of the classification prediction model.

In some alternative embodiments, the classification prediction model is a decision tree model, and the t-th decision tree is generated based on the t-1 st decision tree; Wherein, the method comprises the steps of, wherein, Representing the t decision tree versus sampleIs provided with a prediction output of (c) for the prediction,A least square loss value of residual error corresponding to a J-th sample is represented, and J is the number of samples;

the step of inputting the characteristic information of the target data page into a pre-trained classification prediction model, and determining the data classification of the target data page according to the classification prediction result of the classification prediction model comprises the following steps:

Preprocessing the characteristic information of the target data page to generate preprocessing information of the target data page, wherein the preprocessing comprises one or more of missing value processing, outlier processing, characteristic engineering processing, category type characteristic processing and standardized processing format conversion processing;

inputting the preprocessing information into a pre-trained decision tree model, and determining the prediction output of at least part of decision trees;

And carrying out weighting processing on the prediction output of the at least part of decision trees, and determining the data classification of the target data page according to the weighting processing result.

In a second aspect, the present invention provides a computer device, including a memory and a processor, where the memory and the processor are communicatively connected to each other, and the memory stores computer instructions, and the processor executes the computer instructions, thereby executing the buffer management method according to the first aspect or any implementation manner corresponding to the first aspect.

The invention provides a solid state disk, which comprises a flash memory conversion layer and a flash memory chip, wherein the flash memory conversion layer comprises a processor, a flash memory controller and a buffer manager, and the buffer manager is used for executing the buffer area management method of the first aspect or any implementation mode corresponding to the first aspect.

In a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon computer instructions for causing a computer to execute the method for managing a buffer of the first aspect or any of the embodiments corresponding thereto.

In a fifth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to perform the method of managing a buffer of the first aspect or any of its corresponding embodiments.

The buffer manager in the embodiment of the invention supports two working modes of a continuous buffer mode and a discrete buffer mode, can realize the management of a continuous buffer space and a single discrete buffer space, can optimize data read-write operation, can be applied to more refined read-write operation, improves the utilization efficiency and turnaround of a read-write buffer area in a storage system, reduces the waiting time of the read-write buffer area in the storage system, improves the read-write performance of a solid disk, and is more beneficial to garbage recovery and can improve the garbage recovery efficiency.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the related art, the drawings that are required to be used in the description of the embodiments or the related art will be briefly described, and it is apparent that the drawings in the description below are some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.

FIG. 1 is a flow chart of a method of buffer management according to an embodiment of the invention;

FIG. 2 is a flow chart of another buffer management method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a flash translation layer according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a buffer identification list according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a plurality of buffer identification lists in a continuous buffer mode according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a plurality of buffer identification lists in a discrete buffer mode according to an embodiment of the present invention;

FIG. 7 is a flow chart of a method for managing a buffer according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of a solid state disk according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of a process for modeling according to an embodiment of the present invention;

FIG. 10 is a block diagram of a buffer management apparatus according to an embodiment of the present invention;

fig. 11 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The solid state disk is internally provided with a flash Memory chip, for example, a Nand flash Memory chip (NAND FLASH Memory), the read-write characteristics of the Nand flash Memory are greatly different from those of a magnetic disk, for example, the Nand flash Memory has the limit of erasing before writing, and the granularity of erasing operation in the flash Memory is much larger than that of writing operation, and the like. Because of the unique physical characteristics of Nand flash memory, LBA (Logical Block Address ) needs to be mapped to PBA (Physical Block Address ) inside the device by virtue of FTL (Flash Translation Layer ) layer, and HRW/HRR (Host Random Write/Host Random Read, host Random write/Host Random Read) performance or HSW/HSR (Host Sequential Write/Host Sequential Read ) performance on SSD is highly dependent on the performance of FTL layer algorithm.

Currently, a portion of the storage space in the SSD is set aside to be used as a Buffer (Buffer) to compensate for the physical characteristics of Nand flash memory. The Buffer space management technique is a technique for performing space management on data stored in a Buffer to optimize read-write operations of the data. When the data is needed to be accessed, the data is needed to be read from the flash memory chip into the Buffer, and when the data in the Buffer is modified or is no longer needed, the data is needed to be written back into the flash memory chip. In this process, it is necessary to reasonably allocate and release buffers.

According to an embodiment of the present invention, there is provided an embodiment of a method for managing a buffer, it being noted that the steps shown in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in an order different from that herein.

The method for managing the Buffer area can be applied to a Buffer Manager (Buffer Manager), wherein the Buffer Manager is located on a flash memory conversion layer (FTL) of a solid state disk, and cooperates with a flash memory Controller (NAND FLASH Controller, NFC) and a processor of the flash memory conversion layer to manage the Buffer area. Fig. 1 is a flowchart of a buffer management method according to an embodiment of the present invention, and as shown in fig. 1, the flowchart includes the following steps.

Step S101, setting the working mode of the buffer manager, wherein the working mode comprises a continuous buffer mode and a discrete buffer mode.

The Buffer manager manages a Buffer area (Buffer) in the solid state disk so as to realize allocation and release of the Buffer area. In this embodiment, the Buffer area (Buffer) is divided into a plurality of Buffer spaces, each Buffer space generally takes a Page (Page) as a unit to process a corresponding data Page, and the Buffer manager executes different management modes for the Buffer spaces in different working modes.

In particular, two modes of operation are provided for the buffer manager, namely a continuous buffer mode and a discrete buffer mode. In the continuous buffer mode, a plurality of continuous buffer spaces can be managed in a unified way, and in the discrete buffer mode, a buffer manager can manage a single buffer space, namely, a certain independent buffer space can be allocated and released.

In step S102, in the case that the buffer manager is in the continuous buffer mode, in response to the first read/write command, a plurality of continuous buffer spaces corresponding to the first read/write command are determined, and continuous read/write operations are performed according to the plurality of continuous buffer spaces.

In this embodiment, the buffer manager is in different working modes, and different processing modes are executed.

Specifically, if the current working mode of the buffer manager is a continuous buffer mode, that is, when the buffer manager is in the continuous buffer mode, the buffer manager may acquire a corresponding operation instruction, that is, a first read-write instruction, for example, the Host (Host) may issue a corresponding first read-write instruction to the solid state disk, where the first read-write instruction may specifically be a read instruction or a write instruction. The Buffer manager of the solid state disk may respond to the first read-write command, so as to determine a plurality of Buffer spaces corresponding to the first read-write command, where the Buffer spaces are continuous, for example, each Buffer space may be provided with a unique Buffer identifier (Buffer ID), and the Buffer identifiers corresponding to the Buffer spaces are continuous. And after determining the continuous buffer spaces, performing continuous read-write operation on the continuous buffer spaces to finish the allocation of a plurality of continuous buffer spaces.

For example, when the host issues a first read-write command for writing data, the buffer manager may determine a plurality of continuous buffer spaces for corresponding to the first read-write command based on the first read-write command, and then instruct the processor to write corresponding data into the plurality of continuous buffer spaces.

In step S103, in a case where the buffer manager is in the discrete buffer mode, in response to the second read/write instruction, a single buffer space corresponding to the second read/write instruction is determined, and a discrete read/write operation is performed according to the single buffer space.

If the current working mode of the buffer manager is a discrete buffer mode, that is, when the buffer manager is in the discrete buffer mode, the buffer manager can also acquire an operation instruction applicable to the discrete buffer mode, that is, a second read-write instruction, for example, the Host (Host) can issue the second read-write instruction to the solid state disk, where the second read-write instruction can be a read instruction or a write instruction specifically, or in the discrete buffer mode, the buffer manager can perform Garbage Collection (GC), and the second read-write instruction can also be a corresponding garbage collection instruction.

The second read/write instruction is an instruction using a buffer space. Specifically, the Buffer manager may respond to the second read/write instruction to determine a single Buffer space corresponding to the second read/write instruction, for example, each Buffer space may be provided with a unique Buffer identifier (Buffer ID), where the single Buffer space corresponds to one Buffer identifier. After the single buffer space is determined, continuous read-write operation can be executed on the single buffer space, and the allocation of the single buffer space is completed.

For example, when garbage collection is performed, a read instruction (i.e., a second read/write instruction) for reading a data page in a certain buffer space may be generated, and the buffer manager may determine a single buffer space corresponding to the read instruction and perform garbage collection on the single buffer space, thereby implementing fine garbage collection and facilitating the execution of an erase operation.

The buffer manager supports two working modes of a continuous buffer mode and a discrete buffer mode, can manage the continuous buffer space and a single discrete buffer space, can optimize data read-write operation, can be applied to more refined read-write operation, improves the utilization efficiency and turnaround of the read-write buffer in a storage system, reduces the waiting time of the read-write buffer in the storage system, improves the read-write performance of a solid disk, and is more beneficial to garbage recovery and can improve the garbage recovery efficiency.

The embodiment provides a buffer area management method which can be applied to a buffer manager, wherein the buffer manager is positioned on a flash memory conversion layer of a solid state disk. Fig. 2 is a flowchart of a buffer management method according to an embodiment of the present invention, and as shown in fig. 2, the flowchart includes the following steps.

Step S201, setting the working mode of the buffer manager, wherein the working mode comprises a continuous buffer mode and a discrete buffer mode.

Please refer to step S101 in the embodiment shown in fig. 1 in detail, which is not described herein.

Step S202, at least one buffer identification list is configured according to the set working mode, wherein the buffer identification list is used for recording buffer identifications corresponding to the buffer space.

In the embodiment, the Buffer manager realizes the management of the Buffer space based on the Buffer identifier (Buffer ID), specifically, the Buffer manager is configured with a Buffer identifier List (Buffer ID List) for recording the Buffer identifier, and the Buffer identifier is efficiently and reasonably distributed to the front-end command based on the Buffer identifier List, so that the full utilization of the Buffer space is realized, and the hardware acceleration is realized.

The buffer manager cooperates with a flash memory controller of the flash memory conversion layer and a processor, which is typically a microprocessor (Micro Controller Unit, MCU), to manage the buffer. Fig. 3 shows an architecture schematic diagram of the flash memory conversion layer, as shown in fig. 3, the flash memory conversion layer of the solid state disk can process a host command (for example, a first read-write command, etc.) issued by a host, wherein the flash memory controller can release corresponding buffer identifiers as required, the processor can acquire the required buffer identifiers and release certain buffer identifiers, the buffer manager is a key component responsible for receiving front-end commands, coordinating interaction with the processor and a hardware module of the flash memory controller, and can allocate the buffer identifiers so that the processor can execute read-write operations based on a buffer space corresponding to the allocated buffer identifiers.

Wherein the list of buffer identification is related to the currently set operating mode. And when the set working mode is a discrete buffer mode, r bits in the buffer identification list correspond to corresponding buffer identifications, and r is more than 1. Wherein, in order for the buffer identification list to represent all the buffer identifications, 2^r is not less than the number of buffer identifications in the buffer identification list.

In this embodiment, if the currently set operation mode is the continuous buffer mode, since a plurality of continuous buffer spaces are managed at this time, that is, a plurality of continuous buffer flags are managed at this time, it is not necessary to completely represent each buffer flag. Specifically, in the continuous buffer mode, each bit (bit) in the buffer identifier list corresponds to a buffer identifier, a plurality of buffer identifiers can be used as an identifier block, and a corresponding initial address can be returned when the identifier block is allocated, for example, when the plurality of buffer identifiers are reallocated, the corresponding plurality of buffer identifiers are represented by the initial address of the buffer block corresponding to the plurality of buffer identifiers.

If the currently set working mode is a discrete buffer mode, a single buffer space is managed at this time, and the managed buffer space is random, that is, a random buffer identifier is recorded in the buffer identifier list, so that a complete buffer identifier needs to be recorded in the buffer identifier list, and in this embodiment, each r bits represents a buffer identifier. In addition, in order to enable the buffer identification list to represent all the buffer identifications, 2^r is not smaller than the number of the buffer identifications in the buffer identification list, namely, r is larger than or equal to log ₂ n, wherein n is the number of the buffer identifications in the buffer identification list.

In some alternative embodiments, the step S202 "configuring at least one buffer identification list according to the set operation mode" may specifically include the following steps A1 to A2.

And step A1, setting the depth of the buffer identification list as a first depth and setting the buffer identification list to be full of buffer identifications under the condition that the set working mode is a continuous buffer mode.

In this embodiment, when entering the continuous buffer mode, the buffer manager needs to initialize the buffer identifier list to configure the buffer identifier list, so that the buffer identifier list can meet the requirement of the continuous buffer mode.

Specifically, the depth of the buffer identification list needs to be set. In the continuous buffer mode, each bit in the buffer identification list corresponds to one buffer identification, so that the depth of the buffer identification list can be determined based on the number of the buffer identifications required to be represented, for example, the depth of the buffer identification list is set to be consistent with the number of the buffer identifications.

In addition, for the continuous buffer mode, after the initialization of the buffer identification list is finished, the default buffer identification list is full of buffer identifications, and at the moment, the IDs in the buffer identification list can be 1. In this case, the buffer space corresponding to the buffer identifier in the buffer identifier list may be used to store data, specifically, when the read-write operation is performed on the buffer space based on the buffer identifier list, the corresponding buffer identifier may be read from the buffer identifier list, so that the corresponding buffer space may be used, the buffer space may be released after the buffer identifier is used up, the corresponding buffer identifier may be released (the buffer identifier is marked as 0 from 1), and when the buffer identifier list is empty, all the buffer identifiers are 0.

Optionally, the buffer identification list is provided with a head pointer and a tail pointer, and the step S202 "configuring at least one buffer identification list according to the set operation mode" may further include a step A3.

And step A3, under the condition that the set working mode is a continuous buffer mode, pointing the head pointer of the buffer identification list to the tail part of the buffer identification list, and pointing the tail pointer of the buffer identification list to the head part of the buffer identification list.

In this embodiment, the head pointer and the tail pointer of the buffer identifier list are used to indicate the use condition of the buffer identifier list, where the buffer identifier is recorded. For the continuous buffer mode, when the buffer identification list is set to be full of buffer identifications, the positions of a head pointer and a tail pointer of the buffer identification list are also adaptively set, namely, the head pointer of the buffer identification list points to the tail of the buffer identification list and corresponds to the end address of the buffer identification list, and the tail pointer of the buffer identification list points to the head of the buffer identification list and corresponds to the start address of the buffer identification list.

For example, the buffer identification list is stored in a corresponding buffer register, for which a corresponding head pointer and tail pointer can be set, and the bit width of the buffer register coincides with the bit width of the buffer identification list so that both can correspond.

Fig. 4 shows a schematic diagram of a buffer identity list after initializing the buffer identity list in continuous buffer mode. Each square corresponds to an identification block formed by a plurality of continuous buffer identifications, for example, if s continuous buffer identifications need to be allocated each time, the identification block corresponds to s buffer identifications, namely s bits in total.

Further alternatively, a plurality of buffer identification lists may be provided, for example, the number of buffer identification lists may be set based on the number of processor cores of the processor, wherein the base address of the buffer identification list may be determined upon initialization by the flash controller. Also, the buffer identification list may be configured to be exclusively used by one processor core, for example, one buffer identification list for each processor core, or the buffer identification list may be configured to be shared for use by a plurality of processor cores.

If the buffer identifier list may be configured to be exclusively used by one processor core, each processor core may read the buffer register corresponding to the buffer identifier list to obtain the corresponding buffer identifier, that is, the start address of multiple consecutive buffer identifiers. When multiple processor cores share a certain buffer identification list, at a certain point in time, the buffer identification list can only be used by one of the processor cores.

Fig. 5 shows a schematic diagram of a plurality of buffer identification lists set in the continuous buffer mode. As shown in FIG. 5, 8 buffer identification lists, namely list 0-list 7, are provided in total, and after the buffer identifications in the buffer identification list are read, the corresponding buffer identifications are released, namely, the bit corresponding to the buffer identification is set to 0.

In this embodiment, the solid state disk is generally provided with a buffer space of DRAM (dynamic random access memory) and a buffer space of SRAM (static random access memory), and the main responsibility of the buffer manager includes management of the DRAM buffer space and the SRAM buffer space. The buffer manager supports both random and continuous release when managing DRAM buffer space and SRAM buffer space. This means that the buffer manager can flexibly manage these buffers, freeing up a single or a series of consecutive buffer spaces as needed to meet the system requirements.

For example, taking the example shown in fig. 5, list 0 to list 3 represent the buffer identification list of the SRAM buffer space, list 4 to list 7 represent the buffer identification list of the DRAM buffer space, and their base addresses are all determined when the NFC is initialized.

And step A2, setting the depth of the buffer identification list as a second depth and setting the buffer identification list as empty under the condition that the set working mode is a discrete buffer mode.

In this embodiment, when entering the discrete buffer mode, the buffer manager also needs to initialize the buffer identifier list to configure the buffer identifier list, so that the buffer identifier list can meet the requirement of the discrete buffer mode.

Specifically, the depth of the buffer identification list needs to be set. And in the discrete buffer mode, each r bits in the buffer identification list correspond to one buffer identification, so that the depth of the buffer identification list can be determined based on the number of the buffer identifications required to be represented, for example, the ratio of the number of the buffer identifications to r is used as the depth of the buffer identification list.

In addition, for the discrete buffer mode, after the initialization of the buffer identification list is finished, the default buffer identification list is empty, that is, the buffer identification list does not have a full buffer identification, and in the embodiment, the IDs in the buffer identification list are set to be 1. At this time, when a certain buffer space is needed to be used, the flash memory controller or the processor releases the corresponding buffer space, and then the buffer controller acquires the released buffer identifier and records the buffer identifier into the buffer identifier list to indicate that the buffer space corresponding to the buffer identifier can be used.

In this embodiment, each buffer flag list may also be limited by a depth and a width, where the depth is defined as a maximum depth and the width is width. In continuous buffer mode, each bit represents one buffer identity, so that each buffer identity list can accommodate (width x depth) at most. In the discrete buffer mode, each r bits represents a buffer flag, so each buffer flag list can accommodate (width×depth)/r buffer flags at most, where (width×depth)/r is an integer. It will be appreciated that if the depths of the buffer identification list are the same in different modes of operation, i.e. the first depth is the same as the second depth, then the buffer identification list in the continuous buffer mode may represent more buffer identifications.

Optionally, the buffer identification list is provided with a head pointer and a tail pointer, and the step S202 "configuring at least one buffer identification list according to the set operation mode" may further include step A4.

And step A4, under the condition that the set working mode is a discrete buffer mode, the head pointer and the tail pointer of the buffer identification list point to the same position.

In this embodiment, for the discrete buffer mode, when the buffer flag list is set to be full of the buffer flag, the positions of the head pointer and the tail pointer of the buffer flag list are also adaptively set, that is, the head pointer and the tail pointer of the buffer flag list point to the same position, for example, the head pointer and the tail pointer of the buffer flag list point to the head of the buffer flag list. In the discrete buffer mode, the initialized buffer identifier list is similar to the structure shown in fig. 4, wherein the numerical values are 1, but the head pointer and the tail pointer of the buffer identifier list point to the same position, wherein each block can specifically represent r-bit buffer identifiers.

In the discrete buffer mode, if a certain buffer space is needed to be allocated at present, the buffer identifier corresponding to the buffer space needs to be released first, then the acquired buffer identifier is sequentially stored in a buffer identifier list, wherein the buffer identifier can be stored to the position pointed by the current head pointer, and the head pointer can be added with one after the buffer identifier is stored.

Further alternatively, a plurality of buffer identification lists may be provided, for example, the number of buffer identification lists may be set based on the number of processor cores of the processor. Also, the buffer identification list may be configured to be exclusively used by one processor core, for example, one buffer identification list for each processor core, or the buffer identification list may be configured to be shared for use by a plurality of processor cores. Wherein when a plurality of processor cores share a certain buffer identification list, at a certain point in time, the buffer identification list can only be used by one of the processor cores.

Fig. 6 shows a schematic diagram of a plurality of buffer identity lists set in a discrete buffer mode. As shown in FIG. 6, 8 buffer identification lists, namely list 0 to list 7, are provided in total, and after the buffer identification is obtained, the buffer identification is recorded in the corresponding buffer identification list. As shown in fig. 6, the positions where all are 1 (i.e., FFFF) indicate that no buffer flag is recorded, in other words, in the discrete buffer mode, if all data in the buffer flag list is 1, it indicates that the buffer flag list is empty.

Further optionally, the step S202 "configuring at least one buffer identification list according to the set operation mode" may further include checking whether the highest bit of the buffer identification list is a preset value in the case that the set operation mode is the continuous buffer mode, and determining that the configuration of the buffer identification list is completed in the case that the highest bit of the buffer identification list is the preset value.

In this embodiment, since all the buffer identifiers are recorded in the initialized buffer identifier list in the continuous buffer mode, in order to ensure that the configuration of the buffer identifier list can be accurately completed at this time, after the buffer identifier list is set to be full of the buffer identifiers, whether the highest bit of the buffer identifier list is a preset value or not may be further checked, for example, the buffer manager reads the buffer register and checks whether the highest bit is 1 or not. If the highest bit of the buffer identification list is a preset value, the configuration is correct, and the initialization is completed at this time, the completion of the configuration of the buffer identification list can be determined, and then the configured buffer identification list can be used for managing the buffer identification.

In the discrete buffer mode, after the buffer identification list is set to be empty, the configuration of the buffer identification list can be directly determined to be completed, namely, the buffer manager can be directly used according to the discrete buffer mode and the designated depth.

In step S203, in the case that the buffer manager is in the continuous buffer mode, in response to the first read/write command issued by the host, a plurality of continuous buffer spaces corresponding to the first read/write command are determined, and continuous read/write operations are performed according to the plurality of continuous buffer spaces.

Please refer to step S102 in the embodiment shown in fig. 1 in detail, which is not described herein.

Optionally, the step of performing continuous read-write operations according to the plurality of continuous buffer spaces specifically includes determining a plurality of continuous buffer identifications corresponding to the plurality of currently required buffer spaces in the buffer identification list, and then releasing the plurality of continuous buffer identifications in the buffer identification list.

In this embodiment, in the continuous buffer mode, after the initialization is finished, the default buffer identifier list is full of buffer identifiers, so when a plurality of continuous buffer spaces need to be operated, a plurality of corresponding continuous buffer identifiers need to be acquired first, and then released. The continuous buffer identifier release is mainly that the DMA (Direct Memory Access ) of the flash memory controller processes corresponding data, corresponding buffer identifiers are released one by one according to a buffer identifier list, a buffer manager releases according to bits of the buffer identifier, only after all bits of the whole identifier block are released, the buffer identifier can be provided for a processor to obtain, and the processor can obtain an initial buffer identifier by reading the buffer register, so that the initial addresses of a plurality of continuous buffer identifiers are distributed to the processor. It will be appreciated that in the continuous buffer mode, s buffer identifications can be acquired at a time. If the s buffer identifications acquired at the same time are not completely released, or the buffer identifications acquired before are not completely released, the s buffer identifications cannot be acquired.

In the continuous buffer mode, the continuous release buffer flag mainly accelerates the continuous writing performance, and can accelerate the release and acquisition of the buffer space, for example, the release and acquisition of the SRAM buffer space.

Optionally, if the buffer flag list is provided with a head pointer and a tail pointer, the method may further include triggering the tail pointer of the buffer flag list to be incremented by one after releasing a plurality of consecutive buffer flags in the buffer flag list.

In this embodiment, the continuous buffer mode release involves obtaining a certain amount of buffer flags each time, where the buffer flags are expressed according to bits, and s buffer flags are formed into one flag block, i.e. each flag block has a size of s bits. In continuous buffer mode, the main goal is to acquire a series of buffer identifications and return their starting addresses. After initialization, the buffer identification list head pointer is located at the head and the tail pointer is located at the tail. The tail pointer is triggered once every time one of the identification blocks is released, so that the tail pointer is incremented by 1, and furthermore, if the buffered identification list is not full, the head pointer of the buffered identification list is incremented by 1 when each of the identification blocks is occupied.

When the processor needs to occupy the buffer identification, a certain amount of buffer identification can be acquired through the bus and distributed to the flash memory controller. After the processor configures the buffer controller to occupy the corresponding buffer space, the buffer space is automatically released, and the flash memory controller releases the corresponding buffer mark, so that the position 0 corresponding to the buffer mark can be set. Thus, through the management of the continuous buffer mode, the system can effectively allocate and release the buffer identification, and the efficient utilization of resources is ensured.

In step S204, when the buffer manager is in the discrete buffer mode, in response to the second read/write command issued by the host, a single buffer space corresponding to the second read/write command is determined, and the discrete read/write operation is performed according to the single buffer space.

Please refer to step S103 in the embodiment shown in fig. 1 in detail, which is not described herein.

Alternatively, the step of performing discrete read and write operations based on a single buffer space may include writing the target buffer identification into a list of buffer identifications after the flash controller or processor releases the target buffer identification corresponding to the single buffer space to allocate the target buffer identification to the processor.

In this embodiment, in the discrete buffer mode, since the default buffer identifier list is empty after the initialization is completed, when the buffer space needs to be operated, the corresponding buffer identifier needs to be released first, then acquired, and the actual buffer identifier with r bits is acquired. In the discrete buffer mode, any buffer identification that has been previously released can be acquired. Further optionally, if the buffer identifier list is provided with a head pointer and a tail pointer, the method may further include triggering the head pointer of the buffer identifier list to be incremented by one after writing the target buffer identifier into the buffer identifier list, so as to ensure that the head pointer and the tail pointer in the buffer identifier list correspond to the recorded buffer identifier position.

In some alternative embodiments, the discrete buffer mode may be used primarily for garbage collection. When the solid state disk is used, the default working mode is a continuous buffer mode, namely, the buffer manager firstly enters the continuous buffer mode, and when garbage recovery is needed, the solid state disk is switched to a discrete buffer mode, and the continuous buffer mode can be restored after the garbage recovery is finished.

The method can further comprise switching from a current first mode to another second mode in response to the mode switching instruction, wherein the first mode is a continuous buffer mode, the second mode is a discrete buffer mode, or the first mode is a discrete buffer mode, and the second mode is a continuous buffer mode. I.e. the mode switch instruction may be used to switch from continuous buffer mode to discrete buffer mode or may also be used to switch from discrete buffer mode to continuous buffer mode.

In this embodiment, the mode switching instruction may be generated based on generating a mode switching instruction for switching to the discrete buffer mode when the current amount of garbage data exceeds a first preset threshold in the case where the first mode is the continuous buffer mode, and generating a mode switching instruction for switching to the continuous buffer mode when the current amount of garbage data is less than a second preset threshold in the case where the first mode is the discrete buffer mode.

The buffer mark released in the discrete buffer mode is mainly used for garbage collection and aims to quickly write data read during garbage collection into the Nand flash memory, wherein the speed of acquiring the buffer mark in the discrete buffer mode is generally the speed of a bus, namely, two clocks are read once.

For example, when configuring the Link of the flash memory controller, each buffer identifier occupies a Link unit, and the release of the random buffer identifiers is mainly that the flash memory controller processes corresponding data, the buffer identifiers are released one by one according to the Link of the flash memory controller, the buffer manager puts the released buffer identifiers into a corresponding buffer identifier list, and the buffer identifiers in the buffer identifier list are disordered. The processor obtains 1 corresponding buffer identification by reading the corresponding buffer register. In the discrete buffer mode, after initialization, the head and tail pointers may all be located at the head of the buffer identification list. When the flash controller or processor releases the buffer identification to the buffer manager, the buffer manager first stores the buffer identification in a buffer. When the processor needs to acquire the buffer identifier, if the buffer space is not empty, that is, the head pointer and the tail pointer are not in the same identifier block (at this time, one buffer identifier corresponds to one identifier block), the processor acquires the corresponding buffer identifier from the position of the tail pointer, and then the tail pointer is added with 1 to move to the next identifier block. If the buffer space is empty, but there is a buffer identification in the buffer, the processor may obtain the buffer identification from the buffer. If the buffer space is empty and the buffer area has no buffer mark, the position of all 1 is obtained, which indicates that the acquired buffer mark is invalid.

In this embodiment, compared with the conventional garbage collection for erasing the whole data block, in the discrete buffer mode, a single buffer identifier can be released, and the buffer identifier release accelerates the streaming of the buffer region, thereby improving the garbage collection efficiency.

Optionally, the step of switching from the current first mode to the other second mode specifically includes releasing all buffer identifications in the buffer identification list and then switching to the discrete buffer mode when the first mode is the continuous buffer mode, and adding all released buffer identifications to the buffer identification list and then switching to the continuous buffer mode when the first mode is the continuous buffer mode.

In this embodiment, if the buffer identifier is currently in the continuous buffer mode and needs to be switched to the discrete buffer mode, that is, before entering the discrete buffer mode (for example, before step A2), the buffer identifier stored before the buffer identifier is emptied, that is, all the buffer identifiers in the buffer identifier list need to be released, and then the buffer identifier list is switched to the discrete buffer mode, so that the initialization can be performed to reconfigure the depth of the buffer identifier list and enable the buffer identifier therein to be empty. Wherein, after the reconfiguration is completed, the status of each buffer identification list needs to be restarted.

If the buffer identifier is currently in the discrete buffer mode and needs to be switched to the continuous buffer mode, that is, before entering the continuous buffer mode (for example, before the step A1), all the released buffer identifiers need to be acquired into the buffer identifier list, and then the buffer identifier is switched to the discrete buffer mode, so that the initialization can be performed. In other cases, configuration of modes and depths is not allowed.

In this embodiment, when the buffer identifier list needs to be reconfigured, it is ensured that all the acquired buffer identifiers in the continuous buffer mode are released, or all the released buffer identifiers in the discrete buffer mode are acquired into the buffer identifier list, so that the problem that the buffer identifiers are repeatedly acquired and then reused can be avoided, and the robustness and the correctness of the buffer manager between different configurations can be ensured.

The embodiment provides a buffer area management method which can be applied to a buffer manager, wherein the buffer manager is positioned on a flash memory conversion layer of a solid state disk. Fig. 7 is a flowchart of a buffer management method according to an embodiment of the present invention, and as shown in fig. 7, the flowchart includes the following steps.

In step S701, the buffer manager is set to the continuous buffer mode.

In this embodiment, the default working mode of the buffer manager is a continuous buffer mode, and after the power-on, the buffer manager can automatically enter the continuous buffer mode. The process of setting the working mode of the buffer manager can be specifically referred to step S101 in the embodiment shown in fig. 1, and will not be described herein.

Step S702, a first read-write command issued by the host is received, and a continuous read-write operation is performed according to a plurality of continuous buffer spaces.

In this embodiment, the host may issue a write command to the solid state disk, and the solid state disk receives the host command and then executes the host write command. The interaction of write commands with data between the host and the SSD is initiated. The flash memory conversion layer of the solid state disk may further include a host write unit, and after the buffer manager starts the continuous buffer mode, the buffer area may be managed and the host write unit may be mobilized to process a write task issued by the host.

In step S703, when the garbage data is too much, the discrete buffer mode is turned on, and garbage collection is performed by using a single buffer space.

When the current garbage data quantity exceeds a first preset threshold value, switching to a discrete buffer mode, and managing a buffer space and mobilizing a garbage recycling processing unit by a buffer manager.

Fig. 8 shows a schematic structural diagram of a solid state disk. As shown in fig. 8, the solid state disk includes a flash conversion layer and a flash Memory chip (e.g., NAND FLASH Memory), and the flash conversion layer includes a host write processing unit and a garbage collection processing unit in addition to the above-mentioned processor, flash Memory controller, and buffer manager.

The solid state disk is communicated with the host through the host interface unit, so that corresponding read-write commands can be obtained, and a read-write result can be returned. The processor mainly obtains the buffer space by a user, and controls and processes the read-write task on the solid state disk. The host write processing unit is used for processing host write tasks, including continuous writing and random writing on the host. The garbage collection processing unit is used for garbage collection, wherein when the solid state disk writes new data into the flash memory chip, the solid state disk is required to be subjected to block erasure, and effective data in one or more data blocks (blocks) can be read and written onto the new data blocks by utilizing the garbage collection processing unit, the data in the original data blocks are all invalid data, and the original data blocks can be subjected to block erasure and then can be used for writing the new data.

In step S704, in the case where the buffer manager is in the discrete buffer mode, the target data page in the single buffer space is acquired.

In this embodiment, in the discrete buffer mode, since a single buffer identifier is managed, a data Page (Page) corresponding to a buffer space can be read and written, so that a data Page in the single buffer space, that is, a target data Page, can be acquired.

Step S705, determining a data classification of the target data page, the data classification comprising a cold data page and a hot data page.

Based on the frequency of use of the data, the data can be divided into cold data and hot data, and the corresponding data pages are the cold data page and the hot data page. In this embodiment, for example, the characteristic that the data page can be used as a unit for reading and writing in a discrete buffer mode can be used for collecting a corresponding target data page, and then the classification of the target data page, that is, whether the target data page is a cold data page or a hot data page, can be determined.

In step S706, in the case where the target data page is a cold data page, the cold data page is aggregated into a cold data block.

In step S707, in the case where the target data page is a hot data page, the hot data page is aggregated into a hot data block.

In this embodiment, for different types of data pages, classification aggregation is performed respectively, and the data pages are aggregated to different data blocks, so that the cold data blocks and the hot data blocks can be processed respectively, and related processing such as wear leveling on subsequent SSDs is facilitated.

Alternatively, a sort prediction model for sort prediction may be constructed in advance, and a sort prediction may be performed on the target data page based on the sort prediction model to determine whether it belongs to the cold data page or the hot data page.

Specifically, the above-described step S705 "determining the data classification of the target data page" may include the following steps B1 to B2.

And B1, extracting characteristics of the target data page, and determining characteristic information of the target data page, wherein the characteristic information comprises at least one of attribute information, time information, a logical block address and a physical block address of the target data page.

And step B2, inputting the characteristic information of the target data page into a pre-trained classification prediction model, and determining the data classification of the target data page according to the classification prediction result of the classification prediction model.

In this embodiment, a model for performing classification prediction may be trained in advance, and then classification prediction may be performed on the target data page on which the acquisition is performed based on the model. As shown in FIG. 8, the flash translation layer may further include a training data acquisition unit, a prediction data acquisition unit, a model training unit, a model prediction unit, and a data aggregation unit.

The training data acquisition unit is used for acquiring a data page corresponding to the discrete buffer identifier after the discrete buffer mode is started, extracting corresponding characteristic information by taking the data page as training data, and delivering the training data to the model training unit. The model training unit is used for learning and training the characteristic information of the data page corresponding to the acquired discrete buffer identification to train a convergent classification prediction model. The prediction data acquisition unit is used for implementing acquisition of characteristic information of a data page corresponding to the current discrete buffer identification (namely, a target data page) when the discrete buffer mode is started, and taking the characteristic information as prediction data, so that the characteristic information is handed to the model prediction unit. The model prediction unit analyzes the characteristic information of the collected target data page and predicts whether the target data page is cold data or hot data. After prediction, the data aggregation unit can be utilized to aggregate the finely-crushed cold and hot data pages after prediction on larger data blocks in a classified manner, so that the subsequent relevant processing such as wear balancing on SSD is facilitated.

Specifically, since one discrete buffer identifier corresponds to one data page, after the discrete buffer mode is started, the data page corresponding to the one buffer identifier is written or read, in this embodiment, the feature information extracted from the data page corresponding to the discrete buffer identifier is labeled, so as to implement feature extraction for each data page, where the feature information includes attribute information, time information, a Logical Block Address (LBA), a Physical Block Address (PBA), and the like of the data page.

In training or prediction, the acquired data page may be preprocessed to enable the data page to be used for training or prediction. FIG. 9 is a schematic diagram of a model building process, and as shown in FIG. 9, a required classification prediction model can be trained through data preprocessing and model building processing.

Specifically, the collected real-time operation data (current data page) can be used as training data, and the history operation data (history collected data page) is marked with actual labels (represented as cold data or hot data), and the information extracted from the data pages is subjected to a data preprocessing module. If necessary, the processing is finished and processed finally through multiple steps such as processing missing values, processing abnormal values, feature engineering, category type features, data standardization, category unbalance, dividing data sets, data format conversion and the like. If the information of the data preprocessing exceeds the threshold value required by training, a training set can be constructed, and model training is carried out through model construction processing.

In the model construction processing stage, model initialization can be performed, model training is performed based on the initialized model, the generalized learning capacity of the trained model is evaluated, and whether the model converges or not is judged. And if the model is converged, outputting a classification prediction model obtained by final training, and if the model is not converged, continuing model training based on training data obtained after data preprocessing until the model is converged, so as to obtain the classification prediction model which has convergence and can be used for predicting cold and hot data pages.

Specifically, the present embodiment employs a decision tree model to train learning, such as XGBoost model. The method comprises the steps of processing missing values, namely interpolating and filling the missing values, marking the missing values and the like, so that the missing values do not affect the training process of a model any more, processing the abnormal values, namely detecting and processing the abnormal values in data, namely processing the abnormal values through a statistical method, carrying out feature engineering, namely creating new features and converting original features, carrying out category type features, namely processing category type features, comprising methods of One-Hot Encoding (One-Hot Encoding) and the like, carrying out data standardization, namely carrying out standardization or normalization on the category type features to ensure that different features have similar scales, carrying out category imbalance, namely balancing the data by adopting methods of oversampling, undersampling or using weight adjustment and the like if the category imbalance exists in the data set, dividing the data set into a training set and a testing set, and carrying out data format conversion, namely converting the data into a special format of a decision tree model so as to improve the training efficiency of the model.

Because the decision tree model is adopted in the embodiment, the model mainly comprises a classification tree in the model construction processing process, the classification tree is constructed according to training characteristics and training data, and the prediction result of each piece of data is judged.

Wherein, if the training is started by entering the discrete buffer mode for the first time, the initial model is a simple tree with only one node, and the predicted value is the average value of the whole samples. Initializing the obtained initial treeCan be expressed as: Wherein N is the total number of samples, and y _i is the corresponding predicted value.

And traversing to calculate residual errors between the predicted value of the current model and each actual label, wherein the residual errors are differences between the actual labels and the predicted value of the current model. And constructing a first decision tree by taking the residual error as a target. And adding the prediction result of the first tree with the initial model to form an updated model. And (3) starting to construct more decision trees, taking the contribution of the first tree to the residual errors into consideration when updating the model, so as to iteratively train more weak learners, namely calculating new residual errors, calculating the residual errors of the current updated model, taking the calculated new residual errors as training targets of the next weak learners, and constructing more decision trees by taking the new residual errors as targets. Wherein, generate the firstThe model of the tree is: Wherein, the method comprises the steps of, wherein, Representing the t decision tree versus sampleIs provided with a prediction output of (c) for the prediction,I.e. t-1 decision tree pair sampleIs provided with a prediction output of (c) for the prediction,Is the least squares loss of the residual corresponding to the jth sample, J being the number of samples, e.g., the corresponding residual may be minimized to determine the least squares loss。

Each new tree generated is in an attempt to capture information that was not learned in the previous model. According to the frontPrediction result of treeAnd (d)Model of treeCan push out the firstAnd (5) predicting the result of the sample i after the iteration. And adding the prediction result of the newly built decision tree with the existing model to form a more powerful model. Wherein push out the firstThe prediction result of the sample i after the iteration is:。

The model is combined, and the weight of each tree is proportional to its contribution to the model performance by weighted summation of the outputs of all trees. If the model is found to be over-fitted, regularization parameters can be introduced to prevent the model from over-fitting, and the parameters can be adjusted to optimize the model performance. According to the methods of cross validation and the like, the depth of the tree and the number of nodes are adjusted to find the optimal model complexity. Using the trained model, an appropriate threshold is selected and the output of the model is mapped to a classification of the cold and hot data. The verification set is used for evaluating the performance of the model, so that the model can be converged in the new data, the model not only has good performance in the training set, but also has good convergence performance in the general generalized data.

After training to obtain the decision tree model, classification prediction can be performed based on the decision tree model. Specifically, the step B2 "inputting the feature information of the target data page into the classification prediction model trained in advance, and determining the data classification of the target data page according to the classification prediction result of the classification prediction model" may include steps B11 to B13.

And step B11, preprocessing the characteristic information of the target data page to generate preprocessing information of the target data page, wherein the preprocessing comprises one or more of missing value processing, outlier processing, characteristic engineering processing, category type characteristic processing and standardized processing format conversion processing.

And step B12, inputting the preprocessing information into a pre-trained decision tree model, and determining the prediction output of at least part of the decision tree.

And step B13, carrying out weighting processing on the prediction output of at least part of the decision tree, and determining the data classification of the target data page according to the weighting processing result.

In this embodiment, similar to the processing manner of the collected sample in the training process, the feature information of the target data page needs to be preprocessed after being obtained, wherein the feature information is irrelevant to the class imbalance and the division of the data set in the recognition stage, so that the feature information only needs to be subjected to missing value processing, outlier processing, feature engineering processing, class type feature processing, standardized processing format conversion processing and the like, and finally preprocessing information suitable for the decision tree model is generated, and then the preprocessing information can be input into the decision tree model.

At least part of the decision trees in the decision tree model may be used for classification prediction, wherein the reverse order selection of part of the decision trees for classification prediction, i.e. the last T-th decision tree must be selected (t=1, 2,..once, T), a certain number of decision trees may be selected based on the actual need, e.g. classification prediction using all decision trees in the decision tree model. For each decision tree, a classification prediction result corresponding to the preprocessing information can be output, and the data are weighted, wherein the weighted result can represent the probability that the target data page is a cold data page or a hot data page, so that the data classification of the target data page can be determined.

Wherein, during the weighting process, the weight of each decision tree is proportional to its contribution to model performance. Generally, the later decision trees are weighted more heavily, e.g., the T-th decision tree is weighted most heavily.

The buffer area management method provided by the embodiment uses the continuous buffer mode and the discrete buffer mode to manage the inside of the solid state disk, can be easily applied to the current solid state disk products, can improve the utilization efficiency and turnaround of the read-write buffer area in the storage system, simultaneously reduces the waiting time of the read-write buffer area in the storage system, improves the read-write speed of the solid state disk, and can greatly improve the HRW/HRR performance and the HSW/HSR performance. When in a discrete buffer mode, finely divided cold and hot data pages can be extracted, and classification of finely divided data is determined based on modes such as machine learning, so that finely divided data can be collected on the whole data block, finely divided cold and hot data aggregation is realized, and the overall performance of the solid state disk can be further improved. Compared with the prior cold and hot data identification which only aims at a data block with larger data volume, the embodiment can identify finely divided cold and hot data pages based on a buffer management mode, and the cold and hot data pages are aggregated on the data block, so that finely divided hot/cold data separation is realized, and the finely divided cold and hot data pages after prediction are classified and aggregated on the larger data block in a classified manner, thereby being convenient for the related processing such as wear balance on the subsequent SSD.

The embodiment of the invention also provides a solid state disk, which comprises a flash memory conversion layer and a flash memory chip, wherein the flash memory conversion layer comprises a processor, a flash memory controller and a buffer manager, and the buffer manager is used for executing the buffer area management method provided by any embodiment. The architecture diagram of the solid state disk may be shown in fig. 8, and will not be described herein.

The embodiment also provides a buffer management device, which is used for implementing the foregoing embodiments and preferred embodiments, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements the intended function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

The present embodiment provides a buffer management device, applied to a buffer manager, as shown in fig. 10, including:

A mode setting module 1001, configured to set an operation mode of the buffer manager, where the operation mode includes a continuous buffer mode and a discrete buffer mode;

A continuous buffer module 1002, configured to determine, in response to a first read/write instruction, a plurality of continuous buffer spaces corresponding to the first read/write instruction, and perform continuous read/write operations according to the plurality of continuous buffer spaces, when the buffer manager is in a continuous buffer mode;

a discrete buffer module 1003, configured to determine, in response to a second read/write instruction, a single buffer space corresponding to the second read/write instruction when the buffer manager is in a discrete buffer mode, and perform a discrete read/write operation according to the single buffer space.

In some alternative embodiments, the mode setting module 1001 is further configured to:

In some optional embodiments, the mode setting module 1001 configures at least one buffer identifier list according to the set operation mode, including:

The mode setting module 1001 configures at least one buffer identifier list according to the set operation mode, and further includes:

In some optional embodiments, the mode setting module 1001 configures at least one buffer identifier list according to the set working mode, and further includes:

In some optional embodiments, the continuous buffering module 1002 performs continuous read/write operations according to the plurality of continuous buffer spaces, including determining a plurality of continuous buffer identifications corresponding to a plurality of currently required buffer spaces in the buffer identification list, and then releasing the plurality of continuous buffer identifications in the buffer identification list;

the discrete buffer module 1003 performs discrete read and write operations according to the single buffer space, including writing the target buffer identifier into the buffer identifier list after the flash memory controller or processor releases the target buffer identifier corresponding to the single buffer space.

the device method pointer processing module is used for:

Triggering a tail pointer of the buffer identification list to be increased by one after the plurality of continuous buffer identifications in the buffer identification list are released; and triggering a head pointer of the buffer identification list to be increased by one after the target buffer identification is written into the buffer identification list.

In some alternative embodiments, the apparatus further comprises a mode switching module for:

In some alternative embodiments, the mode switching module switches from a current first mode to another second mode, including:

In some alternative embodiments, the discrete buffer module 1003 is further configured to:

In some alternative embodiments, the discrete buffer module 1003 determines the data classification of the target data page, including:

The discrete buffer module 1003 inputs the feature information of the target data page into a pre-trained classification prediction model, and determines the data classification of the target data page according to the classification prediction result of the classification prediction model, including:

Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.

The buffer management device in this embodiment is in the form of a functional unit, where the unit refers to an ASIC (Application SPECIFIC INTEGRATED Circuit) Circuit, including a processor and a memory that execute one or more software or firmware programs, and/or other devices that can provide the above functions.

The embodiment of the invention also provides computer equipment, which is provided with the buffer area management device shown in the figure 10.

Referring to fig. 11, fig. 11 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, and as shown in fig. 11, the computer device includes one or more processors 10, a memory 20, and interfaces for connecting components, including a high-speed interface and a low-speed interface. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 11.

The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.

Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods shown in implementing the above embodiments.

The memory 20 may include a storage program area that may store an operating system, application programs required for at least one function, and a storage data area that may store data created according to the use of the computer device, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The memory 20 may comprise volatile memory, such as random access memory, or nonvolatile memory, such as flash memory, hard disk or solid state disk, or the memory 20 may comprise a combination of the above types of memory.

The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.

The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium may be a magnetic disk, an optical disk, a read-only memory, a random-access memory, a flash memory, a hard disk, a solid state disk, or the like, and further, the storage medium may further include a combination of the above types of memories. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.

Portions of the present invention may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or aspects in accordance with the present invention by way of operation of the computer. Those skilled in the art will appreciate that the existence of computer program instructions in a computer-readable medium includes, but is not limited to, source files, executable files, installation package files, and the like, and accordingly, the manner in which computer program instructions are executed by a computer includes, but is not limited to, the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled programs, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed programs. Herein, a computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by a computer.

Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.

Claims

1. A method for managing a buffer, the method being applied to a buffer manager, the method comprising:

In the case that the buffer manager is in a discrete buffer mode, in response to a second read-write instruction, determining a single buffer space corresponding to the second read-write instruction, and performing discrete read-write operations according to the single buffer space;

The method further comprises the steps of:

And, the configuring at least one buffer identification list according to the set working mode includes:

setting the buffer mark list to be full of buffer marks under the condition that the set working mode is a continuous buffer mode;

and setting the buffer identification list to be empty under the condition that the set working mode is a discrete buffer mode.

2. The method of claim 1, wherein the step of determining the position of the substrate comprises,

3. The method of claim 2, wherein 2^r is not less than the number of buffer identities in the list of buffer identities.

4. The method of claim 2, wherein configuring at least one list of buffer identification according to the set operation mode further comprises:

Setting the depth of the buffer identification list as a first depth under the condition that the set working mode is a continuous buffer mode;

and setting the depth of the buffer identification list as a second depth under the condition that the set working mode is a discrete buffer mode.

5. The method of claim 4, wherein the buffer identification list is provided with a head pointer and a tail pointer;

6. The method of claim 4, wherein configuring at least one list of buffer identification according to the set operation mode further comprises:

7. The method of claim 4, wherein the step of determining the position of the first electrode is performed,

The continuous read-write operation is executed according to the plurality of continuous buffer spaces, and comprises the steps of determining a plurality of continuous buffer marks corresponding to the plurality of currently required buffer spaces in the buffer mark list, and then releasing the plurality of continuous buffer marks in the buffer mark list;

The performing discrete read-write operations according to the single buffer space includes:

After the flash memory controller or the processor releases the target buffer identification corresponding to the single buffer space, the target buffer identification is written into the buffer identification list.

8. The method of claim 7, wherein the buffer identification list is provided with a head pointer and a tail pointer;

The method further comprises the steps of:

9. The method as recited in claim 4, further comprising:

10. The method of claim 9, wherein the switching from the current first mode to the other second mode comprises:

11. The method of claim 10, wherein the mode switch instruction is generated based on:

12. The method of claim 2, wherein the list of buffer identification is configured for exclusive use by one processor core or the list of buffer identification is configured for shared use by multiple processor cores.

13. The method of claim 2, wherein the buffer identification list is stored in a corresponding buffer register, and wherein a bit width of the buffer register corresponds to a bit width of the buffer identification list.

14. The method as recited in claim 1, further comprising:

15. The method of claim 14, wherein said determining the data classification of the target data page comprises:

16. The method of claim 15, wherein the classification prediction model is a decision tree model and the t decision tree is generated based on the t-1 st decision tree; Wherein, the method comprises the steps of, wherein, Representing the t decision tree versus sampleIs provided with a prediction output of (c) for the prediction,A least square loss value of residual error corresponding to a J-th sample is represented, and J is the number of samples;

17. A computer device, comprising:

A memory and a processor, the memory and the processor being communicatively coupled to each other, the memory having stored therein computer instructions, the processor executing the computer instructions to perform the method of buffer management of any of claims 1 to 16.

18. The solid state disk is characterized by comprising a flash memory conversion layer and a flash memory chip;

the flash memory conversion layer comprises a processor, a flash memory controller and a buffer manager, wherein the buffer manager is used for executing the buffer management method of any one of claims 1 to 16.

19. A computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the method of managing a buffer as claimed in any one of claims 1 to 16.

20. A computer program product comprising computer instructions for causing a computer to perform the method of buffer management of any one of claims 1 to 16.