CN103037203B - A kind of index restoration methods based on block storage and device - Google Patents

Abstract

The invention provides a block storage-based index recovery method and device, belonging to the technical field of video surveillance. The method includes: when the index restoration condition is satisfied, obtaining the end time of the data from the super block of the storage resource corresponding to the video coding device to be index restored; obtaining the I corresponding to the end time according to the index of the storage resource frame group, and obtain the timestamp information TM_Last from the I frame group; obtain the subsequent I frame group of the I frame group from the storage resource, and obtain the time from the I frame group for each I frame group Stamp information TM, when 0<TM-TM_Last≤Tc, update the index of the storage resource according to the information of the I frame group, where Tc is the index cache time corresponding to the video encoding device. The present invention can restore the corresponding index data in time after the video encoding device is offline, thereby ensuring the integrity of the index data.

Description

Method and device for index restoration based on block storage

technical field

The present invention relates to the technical field of video monitoring, in particular to a block storage-based index recovery method and device.

Background technique

Currently, Internet Protocol Storage Area Network (IPSAN) is widely used as storage in video surveillance systems. The monitoring encoding device stores the obtained monitoring data in the IPSAN, and when the user needs to view the historical monitoring data, he can read the corresponding monitoring data from the IPSAN.

Figure 1 is a schematic diagram of a typical network structure of an existing video surveillance system, in which a video encoder (Encoder, EC) is connected to an IPSAN through an Internet Small Computer System Interface (iSCSI), and the video acquired by the surveillance camera The monitoring data is encoded and stored in IPSAN. When users need to view historical monitoring data, they can send a playback request to the video monitoring server (VideoManagement, VM) through the video monitoring client (VideoClient, VC) or video decoder (Decoder, DC). After the VM receives the playback request, Send an instruction message to the data management server (Data Management, DM), instruct the DM to read the corresponding monitoring data stored in the IPSAN, and return it to the VC or DC.

In the above networking structure, after IPSAN forms multiple disks into a redundant array of independent disks (RAID), the RAID is divided into a series of logical units, each logical unit has a logical unit number (LUN), and then the LUN is allocated To EC for storage. A kind of storage format of EC recording data can be shown in Figure 2, wherein, this storage format is based on block storage format, comprises super block, main index area and multiple data areas, and each data area corresponds to a data block, data The size of the block may be 256M, and each data block includes a secondary index area and a plurality of I frame groups. Optionally, the data block also includes a padding area.

EC stores video data into IPSAN according to the index structure shown in Figure 2, and the index area needs to be changed every time data is written, so the data in the index area is frequently read and written. When playing back the video, first read the main index area (first-level index), then find the specified data area, and then find the I frame group data at the specified time according to the sub-index (secondary index) from the data area, and then read The I frame group data is sent to the client for playback. Among them, the structure of the main index area and the sub-index area is shown in Figure 3, including index descriptions and index items. The content of the index items is time and/or space information. Location.

In order to avoid frequent reading and writing of the index area, an existing solution is that the EC stores the index area in its own cache, sets an index cache time (for example, 16 seconds), and performs an index when the index cache time arrives Area data is written, and I frame group data is continuously written into the data area.

In the above solutions, since the index data is not written to the physical hard disk immediately, when the EC fails or is artificially damaged, the video data has been written to the storage device, but the index data in the cache may not have been written to the storage device, resulting in some Recorded data cannot be viewed.

Contents of the invention

In view of this, the purpose of the present invention is to provide a method and device for index recovery based on block storage, which can recover index data in time after the video encoding device is offline, thereby ensuring the integrity of the index data.

To achieve the above object, the present invention provides technical solutions as follows:

A block storage-based index recovery method applied in a video surveillance system, the index recovery method comprising:

When the index recovery condition is satisfied, the end time of data is obtained from the super block of the storage resource corresponding to the video encoding device to be index recovered;

Obtain the I frame group corresponding to the end time according to the index of the storage resource, and obtain the time stamp information TM_Last from the I frame group;

Obtain the subsequent I-frame group of the I-frame group from the storage resource, and acquire time stamp information TM from the I-frame group for each I-frame group, and when 0<TM-TM_Last≤Tc, according to the I-frame group The frame group information updates the index of the storage resource, where Tc is the index cache time corresponding to the video encoding device.

An index recovery device based on block storage, applied in a video surveillance system, the index recovery device includes:

The end time acquisition module is used to obtain the end time of the index from the super block of the storage resource corresponding to the video coding device to be indexed when the index recovery condition is met;

An end frame group acquisition module, configured to acquire the I frame group corresponding to the end time according to the index of the storage resource, and obtain time stamp information TM_Last from the I frame group;

An index update module, configured to obtain the subsequent I frame group of the I frame group from the storage resource, and obtain time stamp information TM from the I frame group for each I frame group obtained, when 0<TM-TM_Last≤ At Tc, the index of the storage resource is updated according to the information of the I frame group, where Tc is the index cache time corresponding to the video encoding device.

According to the above technical solution of the present invention, when the video coding device needs to restore the index after being offline, the end time of the index is obtained from the super block of the storage resource corresponding to the video coding device, and the index is obtained according to the index of the storage resource. According to the I frame group corresponding to the end time, the subsequent I frame group can be obtained according to the I frame group, and the index of the storage resource is updated according to the information of the obtained I frame group, so that the index The timely recovery of data ensures the integrity of the index data, so that the client can view the video data at any time according to the complete index data.

Description of drawings

Fig. 1 is a schematic diagram of a typical networking structure of an existing video surveillance system;

Fig. 2 is a schematic diagram of a storage format of video data;

Figure 3 is a schematic structural diagram of the main index area and the sub-index area;

FIG. 4 is a flow chart of an index recovery method based on block storage according to an embodiment of the present invention;

FIG. 5 is a structural diagram of an index restoration device based on block storage according to an embodiment of the present invention.

FIG. 6 is a structural diagram of a super index block in an embodiment of the present invention.

FIG. 7 is a data structure diagram of an index description in a first-level index in an embodiment of the present invention.

FIG. 8 is a data structure diagram of index items in a primary index in an embodiment of the present invention.

FIG. 9 is a data structure diagram of an index description in a secondary index in an embodiment of the present invention.

FIG. 10 is a data structure diagram of index items in a secondary index in an embodiment of the present invention.

detailed description

In order to solve the problem in the prior art that the video data has been written into the storage device, but the index data in the cache may not be written into the storage device, resulting in a part of the video data cannot be viewed, a solution provided by the embodiment of the present invention is Obtain the end time of the index from the super block of the storage resource corresponding to the offline video encoding device, then obtain the I frame group corresponding to the end time according to the index of the storage resource, and obtain the I frame group corresponding to the end time according to the I frame group from the Subsequent I frame groups are acquired from the storage resource, and each I frame group is acquired, and the index of the storage resource is updated according to the information of the I frame group.

The present invention will be described in detail below in conjunction with the accompanying drawings.

In the following embodiments of the present invention, the storage format shown in Figure 2 is used as an example to describe the restoration of index data. It should be noted that the solutions in the embodiments of the present invention can also be applied to other storage based on block storage Format.

FIG. 4 is a flow chart of an index restoration method based on block storage according to an embodiment of the present invention. The index restoration method is applied to a video surveillance system, and the video surveillance system may include a video surveillance server, a video surveillance client, a video encoding device, IPSAN, etc., the video monitoring server includes a data management server DM and a video management server VM, etc., and the video encoding device can be a network camera IPC and a video encoder EC, etc. Wherein, the index restoration method may be executed by a video monitoring server such as DM, or may be executed by a video encoding device. In the following, the method for recovering the index is executed by the DM and the video encoding device is an EC as an example for illustration.

Referring to Figure 4, the index recovery method may include the following steps:

Step 401, when the index restoration condition is met, the end time of the data is obtained from the super block of the storage resource corresponding to the video encoder to be index restored;

The index recovery condition may be: after the video encoding device is offline, the inspection time interval of the data management server arrives; or, after the video encoding device is offline, the user manually queries the storage resources corresponding to the video encoding device; or, the video encoding device back online.

Specifically, after the EC's network is abnormal or artificially destroyed, the VM will detect that the EC is offline within a period of time, and then the VM can query whether the EC has a storage plan configured. If the storage plan has been configured, the EC's storage plan can be Make a "to be restored" mark, and query the DM corresponding to the EC, and notify the DM of the offline information of the EC. Among them, the VM records the index cache time of each EC (Tc, if it is unified globally, it will be recorded in the global parameter). When the DM arrives at the inspection time interval or when a user manually inquires the channel recording on the encoder, the DM obtains the index cache time Tc of the corresponding channel of the encoder from the VM, and then performs the index recovery process on the recording data.

In addition, when the EC goes online, after receiving the registration message from the EC, the VM detects the status of the storage plan corresponding to the EC. If it is in the "to be resumed" state, it immediately resumes the recording process and notifies the EC that the storage plan is suspended until the following After the recovery process ends, the EC is notified to start storage.

When the DM determines that index recovery is required, first obtain the read and write permissions for the storage resources of the EC, specifically, mount the storage resources corresponding to the EC channel in read and write mode (that is, set the Write, and reconnect to the storage disk through iSCSI at the same time); then, query the superblock information of the storage resource to obtain the end time of the index in the current superblock.

For example, for the storage format shown in FIG. 2 , the super block data structure therein may refer to that shown in FIG. 6 , and the end_time therein may be obtained as the end time of the index.

In the above data structure, units are statically configured, that is, their values are fixed, and the specific values are related to the size of the storage resources corresponding to ECs. For end_time, it corresponds to the time of the last I-frame group with index information.

Step 402, obtain the I frame group corresponding to the end time according to the index of the storage resource, and obtain the time stamp information TM_Last from the I frame group;

First, locate the data block corresponding to the end time according to the primary index of the storage resource; then, obtain the I frame group corresponding to the end time according to the secondary index in the located data block.

Specifically, after obtaining the end time, the DM finds the data block corresponding to the end time by querying the time information in the first-level index item (the time information of the first-level index item is the first I in each data block) The time of the frame group, the query process here can be realized by dichotomy).

For example, for the storage format shown in FIG. 2 , reference can be made to the data structure shown in FIG. 7 for the index description in the first-level index. The data structure of the index items in the first-level index can be referred to as shown in FIG. 8 .

The index item includes the following time information: year is the year, month is the month, day is the day, hour is the hour, minute is the minute, and second is the second. By comparing the end time with the time information in the index item, the data block corresponding to the end time can be determined.

After the data block corresponding to the end time is located, the I frame group corresponding to the end time can be obtained from the secondary index in the data block. For example, for the storage format shown in FIG. 2 , refer to FIG. 9 for the data structure of the index description in the secondary index; and refer to FIG. 10 for the data structure of the index items in the secondary index.

In the block storage scheme, for example, 16k may be used as a storage unit, and the storage unit may be used as a minimum unit to store received data. In order to distinguish it from the data blocks in the primary index, the storage units in the secondary index are called secondary data blocks.

In the above data structure, pos indicates the end position of the last secondary data block (16K) in a data block (256M), and items indicates the index number of the currently written secondary data block; year is year, month is month, day is the day, hour is the hour, minute is the minute, and second is the second; block_offset indicates the number of secondary data blocks offset by the start position of the I frame group corresponding to the index item relative to the start position of the 256M data block.

According to the secondary index item, the I frame group corresponding to the end time can be located, and the time stamp information TM_Last is obtained from the I frame group.

For example, for the storage format shown in Figure 2, the data structure of each I frame group is as follows:

Among them, the data structure of the 20-byte header is as follows:

It can be seen that the 20-byte header contains timestamp information. In the embodiment of the present invention, TM_Last is the timestamp information in the 20-byte header corresponding to the initial I-frame data packet in the I-frame group.

Step 403: Obtain the subsequent I frame group of the I frame group from the storage resource, and obtain time stamp information TM from the I frame group for each I frame group obtained. When 0<TM-TM_Last≤Tc, The index of the storage resource is updated according to the information of the I frame group; when TM-TM_Last>Tc, or TM-TM_Last≤0, the index recovery process ends.

For subsequent I frame groups, there is no information about these I frame groups in the index area, so these I frame groups cannot be queried according to the index. Since the I-frame group data is stored continuously in the storage resource, in the embodiment of the present invention, by obtaining the end position of the last I-frame group recorded with an index, the subsequent non-index information can be obtained according to the end position Specifically, according to the end position, the next I frame group can be obtained, according to the end position of the next I frame group, the next I frame group can be obtained, and so on, subsequent multiple I frames can be obtained Group. Afterwards, the index can be updated according to the information of these I frame groups, so as to ensure that the index data is complete.

In the embodiment of the present invention, an end condition for index recovery is also set. Specifically, the cycle for EC to write index data is Tc, then, even if the EC is offline for some reason, for the I frame group data of TM-TM_Last > Tc (after the EC goes online, the data has already started before the index recovery is completed In the case of storage, there will be a situation where TM-TM_Last>Tc), and there will be corresponding index data in the index area. Therefore, an end condition for index recovery is set to TM-TM_Last>Tc; since the storage resource is repeatedly written In, that is, when the storage space of the storage resource is full, the storage resource will be overwritten. Therefore, another end condition for index recovery is set to TM-TM_Last≤0.

The specific process of updating the index of the storage resource according to the information of the I frame group is:

(1) Update the secondary index according to the size, position and timestamp information of the I frame group;

Specifically, the time corresponding to the timestamp of the I frame group can be written into the time field of the secondary index item, and the position and size of the I frame group (that is, the number of secondary data blocks included) can be written into the secondary index block_offset and block_num of the item, and correspondingly update the items value and pos value in the secondary index description.

(2) If the I frame group is the first I frame group in the data block, also update the primary index;

Specifically, the time corresponding to the time stamp of the I frame group may be written into the primary index item, and at the same time, the value of items in the primary index description may be updated as the index number of the currently written data block.

(3) The end time of the index in the super block is updated to be the timestamp information corresponding to the I frame group.

Furthermore, after the DM completes the index recovery, it can also send a message to the VM. After receiving the message, the VM clears the "to be recovered" mark of the storage plan corresponding to the EC. Then, the VM checks whether the corresponding EC channel is online. If it is online, it can Notify it to resume storage.

Furthermore, after the DM completes the index recovery, it can also send the recording data of the predetermined time (for example, the last 1 second) before the failure of the EC to the VM, and the VM binds it to the offline alarm of the EC.

Corresponding to the above method for restoring an index based on block storage, an embodiment of the present invention further provides an apparatus for implementing the method.

FIG. 5 is a diagram of an index recovery device based on block storage according to an embodiment of the present invention. The index recovery device is applied in a video surveillance system. Specifically, the index recovery device can be set on a video surveillance server such as a data management server DM, It can also be set on a video encoding device such as an IP camera IPC or a video encoder EC.

Referring to FIG. 5 , the index recovery device may include an end time acquisition module 10 , an end frame group acquisition module 20 and an index update module 30 . in:

The end time acquisition module 10 is configured to acquire the end time of data from the super block of the storage resource corresponding to the video encoding device to be index restored when the index restoration condition is met. The index recovery condition may be: after the video encoding device is offline, the inspection time interval of the data management server arrives; or, after the video encoding device is offline, the user manually queries the storage resources corresponding to the video encoding device; or, the video encoding device back online.

The end frame group obtaining module 20 is configured to obtain the I frame group corresponding to the end time according to the index of the storage resource, and obtain time stamp information TM_Last from the I frame group. Specifically, the data block corresponding to the end time may be located according to the primary index of the storage resource, and then the I frame group corresponding to the end time may be acquired according to the secondary index in the located data block.

The index updating module 30 is used to obtain the subsequent I frame group of the I frame group from the storage resource, each time an I frame group is obtained, the time stamp information TM is obtained from the I frame group, when 0<TM-TM_Last ≤Tc, update the index of the storage resource according to the information of the I frame group, where Tc is the index cache time corresponding to the video encoding device; when TM-TM_Last>Tc, or, TM-TM_Last≤0 , end the index recovery process. Specifically, the index update can be performed in the following manner: update the secondary index according to the size, position and timestamp information of the I frame group; update the end time of the index in the super block to be the corresponding timestamp information of the I frame group; if The I frame group is the first I frame group in the data block, and the primary index is also updated.

To sum up, according to the technical solution of the embodiment of the present invention, after the video coding device is offline, the end time of the index is obtained from the super block of the storage resource corresponding to the video coding device, and the end time of the index is obtained from the index of the storage resource Obtain the I frame group corresponding to the end time, obtain the subsequent I frame group according to the I frame group, and update the index of the storage resource according to the information of the obtained I frame group, so that the The timely recovery of the index data ensures the integrity of the index data, so that the client can view the video data at any time according to the complete index data.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the present invention. within the scope of protection.

Claims (6)

1., based on an index restoration methods for block storage, it is applied in video monitoring system, it is characterised in that described index restoration methods includes:

When satisfied index recovery condition, from the pending superblock indexing storage resource corresponding to the video encoder recovered, obtain the end time of data;

Index according to described storage resource obtains I frame group corresponding to described end time, and obtains timestamp information TM_Last from described I frame group;

The follow-up I frame group of described I frame group is obtained from described storage resource, often obtain an I frame group, timestamp information TM is obtained from this I frame group, as 0 < TM-TM_Last��Tc, the index of described storage resource is updated by the information according to this I frame group, wherein, Tc is the indexed cache time that described video encoder is corresponding; As TM-TM_Last > Tc, or, during TM-TM_Last��0, terminate index recovery process;

Wherein, the index of described storage resource is updated by the described information according to this I frame group, including:

According to the size of this I frame group, position and timestamp information, update secondary index;

In renewal superblock, the end time of index is the timestamp information that this I frame group is corresponding;

If this I frame group is first I frame group in data block, update one-level index.

2. index restoration methods as claimed in claim 1, it is characterised in that described index recovers condition and is:

After video encoder off-line, the monitoring time interval of video monitoring server arrives; Or

After video encoder off-line, the storage resource that user's manual queries video encoder is corresponding; Or

Video encoder is reached the standard grade again.

3. index restoration methods as claimed in claim 1, it is characterised in that the described I frame group corresponding according to the index acquisition described end time of described storage resource, including:

One-level index according to described storage resource positions data block corresponding to described end time;

According to the I frame group that the secondary index acquisition described end time in the data block navigated to is corresponding.

4., based on an index recovery device for block storage, it is applied in video monitoring system, it is characterised in that described index recovery device includes:

End time acquisition module, for when satisfied index recovery condition, obtaining the end time of index from the pending superblock indexing storage resource corresponding to the video encoder recovered;

End frame group acquisition module, for the I frame group that the index acquisition described end time according to described storage resource is corresponding, and obtains timestamp information TM_Last from described I frame group;

Index upgrade module, for obtaining the follow-up I frame group of described I frame group from described storage resource, often obtain an I frame group, timestamp information TM is obtained from this I frame group, as 0 < TM-TM_Last��Tc, the index of described storage resource is updated by the information according to this I frame group, and wherein, Tc is the indexed cache time that described video encoder is corresponding; As TM-TM_Last > Tc, or, during TM-TM_Last��0, terminate index recovery process;

Wherein, described index upgrade module is further used for:

According to the size of this I frame group, position and timestamp information, update secondary index;

In renewal superblock, the end time of index is the timestamp information that this I frame group is corresponding;

If this I frame group is first I frame group in data block, update one-level index.

5. index recovery device as claimed in claim 4, it is characterised in that described index recovers condition and is:

After video encoder off-line, the monitoring time interval of video monitoring server arrives; Or

After video encoder off-line, the storage resource that user's manual queries video encoder is corresponding; Or

Video encoder is reached the standard grade again.

6. index recovery device as claimed in claim 4, it is characterised in that described end frame group acquisition module is further used for:

One-level index according to described storage resource positions data block corresponding to described end time;

According to the I frame group that the secondary index acquisition described end time in the data block navigated to is corresponding.