[go: up one dir, main page]

CN114443248A - Object life cycle management method, device, electronic device and storage medium - Google Patents

Object life cycle management method, device, electronic device and storage medium Download PDF

Info

Publication number
CN114443248A
CN114443248A CN202111680650.1A CN202111680650A CN114443248A CN 114443248 A CN114443248 A CN 114443248A CN 202111680650 A CN202111680650 A CN 202111680650A CN 114443248 A CN114443248 A CN 114443248A
Authority
CN
China
Prior art keywords
processed
priority
queue
processing
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111680650.1A
Other languages
Chinese (zh)
Inventor
黄鹄
林洁琬
钟龙山
颜文强
黄润怀
李旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Telecom Cloud Technology Co Ltd
Original Assignee
China Telecom Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Telecom Cloud Technology Co Ltd filed Critical China Telecom Cloud Technology Co Ltd
Priority to CN202111680650.1A priority Critical patent/CN114443248A/en
Publication of CN114443248A publication Critical patent/CN114443248A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/485Task life-cycle, e.g. stopping, restarting, resuming execution
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5018Thread allocation
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5021Priority

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application provides a method and a device for managing a life cycle of an object, electronic equipment and a storage medium, and relates to the technical field of data processing. For any processing queue, based on the work thread allocated to any processing queue, the object information of the object to be processed can be read from any processing queue, and the data of the object to be processed is processed according to the object information. The processing queue may include object information of at least one object to be processed, the object priority of the object to be processed in the processing queue matches the queue priority of the processing queue to which the object to be processed belongs, and the work thread allocated to each processing queue is related to the queue priority of the corresponding processing queue. Therefore, the data of the objects can be isolated and processed through the priority, the problem that the processing speed of all the objects is reduced due to the fact that the data of a certain object is processed at a low speed is avoided, and the efficiency of life cycle management of the objects is improved.

Description

对象的生命周期管理方法、装置、电子设备及存储介质Object life cycle management method, device, electronic device and storage medium

技术领域technical field

本申请涉及数据处理技术领域,尤其涉及一种对象的生命周期管理方法、装置、电子设备及存储介质。The present application relates to the technical field of data processing, and in particular, to a method, apparatus, electronic device and storage medium for life cycle management of an object.

背景技术Background technique

ceph对象存储提供了对象的生命周期管理功能,可以根据生命周期配置规则,对到期的对象进行删除或者将到期的对象迁移到指定类别的存储池中。ceph可以通过扫描桶的索引池分片数据,逐个读取对象的修改时间,跟生命周期配置规则进行比较,如果符合条件就将对象进行删除或者迁移。Ceph object storage provides the life cycle management function of objects. According to the life cycle configuration rules, expired objects can be deleted or expired objects can be migrated to a storage pool of a specified category. Ceph can scan the index pool sharded data of the bucket, read the modification time of the objects one by one, compare them with the life cycle configuration rules, and delete or migrate the objects if they meet the conditions.

然而,索引的分片是基于哈希规则的,不同业务、不同存储池的索引在索引池中是混布在一起的,如果某个业务的压力较大,或者某个存储池出现故障,就会导致对与该业务或该存储池相关的对象进行删除或者迁移的速度下降,进而会导致对全部对象进行生命周期管理的效率降低。However, the sharding of the index is based on hash rules. The indexes of different services and different storage pools are mixed together in the index pool. If a certain service is under great pressure or a certain storage pool fails, the As a result, the speed of deleting or migrating objects related to the business or the storage pool decreases, which in turn leads to a decrease in the efficiency of life cycle management for all objects.

发明内容SUMMARY OF THE INVENTION

为解决现有存在的技术问题,本申请实施例提供了一种对象的生命周期管理方法、装置、电子设备及存储介质,可以提高对全部对象进行生命周期管理的效率。In order to solve the existing technical problems, the embodiments of the present application provide an object life cycle management method, apparatus, electronic device and storage medium, which can improve the efficiency of life cycle management for all objects.

为达到上述目的,本申请实施例的技术方案是这样实现的:In order to achieve the above purpose, the technical solutions of the embodiments of the present application are implemented as follows:

第一方面,本申请实施例提供一种对象的生命周期管理方法,所述方法包括:In a first aspect, an embodiment of the present application provides a life cycle management method for an object, the method comprising:

针对任意一个处理队列,基于为所述任意一个处理队列分配的工作线程,从所述任意一个处理队列中读取待处理对象的对象信息,并根据读取的所述待处理对象的对象信息,对所述待处理对象的数据进行处理;For any processing queue, based on the worker thread allocated to the any processing queue, read the object information of the object to be processed from the any processing queue, and according to the read object information of the object to be processed, processing the data of the object to be processed;

其中,所述处理队列中包括至少一个待处理对象的对象信息,所述处理队列中待处理对象的对象优先级与所属处理队列的队列优先级相匹配,较高队列优先级的处理队列的工作线程数量多于较低队列优先级的处理队列的工作线程数量,或者,较高队列优先级的处理队列的工作线程启动时间早于较低队列优先级的处理队列的工作线程启动时间。The processing queue includes object information of at least one object to be processed, the object priority of the object to be processed in the processing queue matches the queue priority of the processing queue to which it belongs, and the work of the processing queue with a higher queue priority The number of threads is greater than the number of worker threads for the processing queue of the lower queue priority, or the worker thread start time of the processing queue of the higher queue priority is earlier than the start time of the worker threads of the processing queue of the lower queue priority.

本申请实施例提供的对象的生命周期管理方法,针对任意一个处理队列,基于为任意一个处理队列分配的工作线程,可以从任意一个处理队列中读取待处理对象的对象信息,并根据读取的待处理对象的对象信息,对待处理对象的数据进行处理。其中,处理队列中可以包括有至少一个待处理对象的对象信息,处理队列中待处理对象的对象优先级与所属处理队列的队列优先级相匹配,且较高队列优先级的处理队列的工作线程数量多于较低队列优先级的处理队列的工作线程数量,或者,较高队列优先级的处理队列的工作线程启动时间早于较低队列优先级的处理队列的工作线程启动时间。由于可以将不同对象优先级的待处理对象的对象信息,置于不同队列优先级的处理队列中,且采用工作线程根据从相应的处理队列中读取的待处理对象的对象信息,对待处理对象的数据进行处理,从而可以将具有不同优先级的对象通过队列来进行隔离,避免了对某个对象的数据进行处理的速度较慢,而导致的所有对象的处理速度降低,提高了对对象进行生命周期管理的效率。The object life cycle management method provided by the embodiment of the present application, for any processing queue, based on the worker thread allocated to any processing queue, can read the object information of the to-be-processed object from any processing queue, and according to the read The object information of the object to be processed is processed, and the data of the object to be processed is processed. The processing queue may include object information of at least one object to be processed, the object priority of the object to be processed in the processing queue matches the queue priority of the processing queue to which it belongs, and the worker thread of the processing queue with a higher queue priority There are more worker threads than the processing queue of the lower queue priority, or the processing queue of the higher queue priority has a worker thread start time that is earlier than the start time of the worker thread of the processing queue of the lower queue priority. Because the object information of the to-be-processed objects of different object priorities can be placed in the processing queues of different queue priorities, and the worker thread is used to read the object information of the to-be-processed objects from the corresponding processing queues. The data of the object is processed, so that objects with different priorities can be isolated through the queue, which avoids the slow processing speed of the data of a certain object, which reduces the processing speed of all objects and improves the processing speed of the object. Efficiency of Lifecycle Management.

在一种可选的实施例中,通过下列方式确定所述待处理对象的对象优先级:In an optional embodiment, the object priority of the object to be processed is determined in the following manner:

根据所述待处理对象对应的系统状况或业务需求,为所述待处理对象设置相应的对象优先级。A corresponding object priority is set for the to-be-processed object according to the system status or business requirement corresponding to the to-be-processed object.

在该实施例中,由于可以为具有不同系统状况或业务需求的待处理对象,设置不同的对象优先级,从而可以根据不同的对象优先级,来对待处理对象进行处理,提高了对对象进行处理的效率。In this embodiment, since different object priorities can be set for objects to be processed with different system conditions or business requirements, the objects to be processed can be processed according to different object priorities, and the processing of objects can be improved. s efficiency.

在一种可选的实施例中,所述根据所述待处理对象对应的系统状况或业务需求,为所述待处理对象设置相应的对象优先级,包括:In an optional embodiment, setting a corresponding object priority for the object to be processed according to the system status or business requirement corresponding to the object to be processed includes:

若所述待处理对象对应的系统状况运行正常,或所述待处理对象对应的业务需求为即时处理,则为所述待处理对象设置第一对象优先级;If the system status corresponding to the object to be processed is running normally, or the business requirement corresponding to the object to be processed is immediate processing, set a first object priority for the object to be processed;

若所述待处理对象的对应的系统状况出现故障,或所述待处理对象对应的业务需求为稍后处理,则为所述待处理对象设置第二对象优先级;所述第一对象优先级高于所述第二对象优先级。If the corresponding system status of the to-be-processed object fails, or the business requirement corresponding to the to-be-processed object is to be processed later, a second object priority is set for the to-be-processed object; the first object priority higher than the second object priority.

在该实施例中,若待处理对象对应的系统状况运行正常,或待处理对象对应的业务需求为即时处理,则为待处理对象设置第一对象优先级,若待处理对象的对应的系统状况出现故障,或待处理对象对应的业务需求为稍后处理,则为待处理对象设置第二对象优先级。其中,第一对象优先级高于第二对象优先级。从而可以根据系统状况的运行正常或出现故障,以及业务需求为即时处理还是稍后处理,分别为不同的待处理对象分别设置相对较高或相对较低的对象优先级,进而可以按照不同的对象优先级,来对不同的对象的数据分别进行处理,提高对全体对象进行处理的效率。In this embodiment, if the system status corresponding to the object to be processed is running normally, or the business requirement corresponding to the object to be processed is immediate processing, the first object priority is set for the object to be processed. If a fault occurs, or the business requirement corresponding to the object to be processed is to be processed later, a second object priority is set for the object to be processed. The priority of the first object is higher than the priority of the second object. Therefore, according to the normal operation or failure of the system status, and whether the business needs are processed immediately or later, a relatively high or relatively low object priority can be set for different objects to be processed respectively, and then different objects can be processed according to different objects. Priority, to process the data of different objects separately, and improve the efficiency of processing all objects.

在一种可选的实施例中,所述从所述任意一个处理队列中读取待处理对象的对象信息之前,所述方法还包括:In an optional embodiment, before the object information of the object to be processed is read from any one of the processing queues, the method further includes:

根据所述待处理对象的对象优先级,确定与所述对象优先级相同的队列优先级;According to the object priority of the object to be processed, determine the queue priority that is the same as the object priority;

将所述待处理对象的对象信息置于所述队列优先级对应的处理队列中。The object information of the object to be processed is placed in a processing queue corresponding to the queue priority.

在该实施例中,可以根据待处理对象的对象优先级,确定与对象优先级相同的队列优先级,将待处理对象的对象信息置于队列优先级对应的处理队列中。从而可以通过具有不同队列优先级的处理队列对所有待处理对象进行隔离处理,进而可以采用相应的工作线程,分别对位于不同队列优先级的处理队列中的待处理对象的数据进行处理,提高对对象进行处理的速度。In this embodiment, a queue priority that is the same as the object priority may be determined according to the object priority of the object to be processed, and the object information of the object to be processed is placed in the processing queue corresponding to the queue priority. Therefore, all the objects to be processed can be isolated and processed through the processing queues with different queue priorities, and then the corresponding worker threads can be used to process the data of the objects to be processed in the processing queues with different queue priorities respectively. The speed at which the object is processed.

在一种可选的实施例中,所述对象信息中包括所述待处理对象对应的标识和处理要求;所述根据读取的所述待处理对象的对象信息,对所述待处理对象的数据进行处理,包括:In an optional embodiment, the object information includes an identifier corresponding to the object to be processed and processing requirements; the object information of the object to be processed is read according to the object information of the object to be processed. Data is processed, including:

根据所述待处理对象对应的标识,从索引存储器中查找与所述标识相匹配的索引信息;According to the identifier corresponding to the object to be processed, look up the index information matching the identifier from the index memory;

根据所述索引信息,确定所述待处理对象的数据在数据存储器中的原始存储位置,并根据所述待处理对象对应的处理要求,对所述待处理对象的数据进行处理。According to the index information, the original storage location of the data of the object to be processed in the data memory is determined, and the data of the object to be processed is processed according to the processing requirements corresponding to the object to be processed.

在该实施例中,可以根据对象信息中包括的待处理对象对应的标识,从索引存储器中查找与该标识相匹配的索引信息,并根据查找到的索引信息,确出述待处理对象的数据在数据存储器中的原始存储位置,以及根据对象信息中包括的待处理对象对应的处理要求,对待处理对象的数据进行处理,从而可以更加高效地对待处理对象进行处理,提高对对象进行处理的效率。In this embodiment, according to the identifier corresponding to the object to be processed included in the object information, the index information matching the identifier can be searched from the index memory, and the data describing the object to be processed can be determined according to the found index information. The data of the object to be processed is processed at the original storage location in the data storage and according to the processing requirements corresponding to the object to be processed included in the object information, so that the object to be processed can be processed more efficiently and the efficiency of processing the object can be improved .

在一种可选的实施例中,所述根据所述待处理对象对应的处理要求,对所述待处理对象的数据进行处理,包括:In an optional embodiment, the processing of the data of the object to be processed according to the processing requirements corresponding to the object to be processed includes:

若所述处理要求指示对所述待处理对象的数据进行删除操作,则将所述待处理对象的数据从所述原始存储位置处删除,并将所述索引信息从所述索引存储器中删除;If the processing request instructs to delete the data of the object to be processed, delete the data of the object to be processed from the original storage location, and delete the index information from the index memory;

若所述处理要求指示对所述待处理对象的数据进行迁移操作,则将所述待处理对象的数据从所述原始存储位置处,迁移到所述对象信息包括的目标存储位置处,并对所述索引存储器中的所述索引信息进行更新。If the processing request indicates to perform a migration operation on the data of the object to be processed, the data of the object to be processed is migrated from the original storage location to the target storage location included in the object information, and the data of the object to be processed is migrated to the target storage location included in the object information. The index information in the index memory is updated.

在该实施例中,若待处理对象对应的处理要求指示要对待处理对象的数据进行删除操作,则可以将待处理对象的数据从数据存储器中的原始存储位置处删除,并将相应的索引信息从索引存储器中删除,若待处理对象对应的处理要求指示要对待处理对象的数据进行迁移操作,则可以将待处理对象的数据从数据存储器中的原始存储位置处,迁移到待处理对象的对象信息中包括的目标存储位置处,并对索引存储器中的相应的索引信息进行更新,从而可以根据对象对应的处理要求,分别执行对对象的删除或迁移操作。In this embodiment, if the processing requirement corresponding to the object to be processed indicates that the data of the object to be processed is to be deleted, the data of the object to be processed can be deleted from the original storage location in the data storage, and the corresponding index information can be deleted. Delete from the index storage. If the processing requirement corresponding to the object to be processed indicates that the data of the object to be processed is to be migrated, the data of the object to be processed can be migrated from the original storage location in the data storage to the object of the object to be processed. At the target storage location included in the information, the corresponding index information in the index memory is updated, so that the deletion or migration operation of the object can be performed respectively according to the processing requirements corresponding to the object.

第二方面,本申请实施例还提供了一种对象的生命周期管理装置,所述装置包括:In a second aspect, an embodiment of the present application further provides an apparatus for managing a life cycle of an object, the apparatus comprising:

对象信息读取单元,用于针对任意一个处理队列,基于为所述任意一个处理队列分配的工作线程,从所述任意一个处理队列中读取待处理对象的对象信息;其中,所述处理队列中包括至少一个待处理对象的对象信息,所述处理队列中待处理对象的对象优先级与所属处理队列的队列优先级相匹配,较高队列优先级的处理队列的工作线程数量多于较低队列优先级的处理队列的工作线程数量,或者,较高队列优先级的处理队列的工作线程启动时间早于较低队列优先级的处理队列的工作线程启动时间;an object information reading unit, configured to, for any processing queue, read the object information of the object to be processed from the any processing queue based on the worker thread allocated to the processing queue; wherein, the processing queue It includes object information of at least one object to be processed, the object priority of the object to be processed in the processing queue matches the queue priority of the processing queue to which it belongs, and the number of worker threads in the processing queue with a higher queue priority is more than that of a lower queue priority. The number of worker threads of the processing queue of the queue priority, or the start time of the worker thread of the processing queue of the higher queue priority is earlier than the start time of the worker thread of the processing queue of the lower queue priority;

数据处理单元,用于根据读取的所述待处理对象的对象信息,对所述待处理对象的数据进行处理。The data processing unit is configured to process the data of the to-be-processed object according to the read object information of the to-be-processed object.

在一种可选的实施例中,所述装置还包括优先级确定单元,用于:In an optional embodiment, the apparatus further includes a priority determination unit, configured to:

根据所述待处理对象对应的系统状况或业务需求,为所述待处理对象设置相应的对象优先级。A corresponding object priority is set for the to-be-processed object according to the system status or business requirement corresponding to the to-be-processed object.

在一种可选的实施例中,所述优先级确定单元,具体用于:In an optional embodiment, the priority determining unit is specifically configured to:

若所述待处理对象对应的系统状况运行正常,或所述待处理对象对应的业务需求为即时处理,则为所述待处理对象设置第一对象优先级;If the system status corresponding to the object to be processed is running normally, or the business requirement corresponding to the object to be processed is immediate processing, set a first object priority for the object to be processed;

若所述待处理对象的对应的系统状况出现故障,或所述待处理对象对应的业务需求为稍后处理,则为所述待处理对象设置第二对象优先级;所述第一对象优先级高于所述第二对象优先级。If the corresponding system status of the to-be-processed object fails, or the business requirement corresponding to the to-be-processed object is to be processed later, a second object priority is set for the to-be-processed object; the first object priority higher than the second object priority.

在一种可选的实施例中,所述装置还包括队列加入单元,用于:In an optional embodiment, the apparatus further includes a queue adding unit for:

根据所述待处理对象的对象优先级,确定与所述对象优先级相同的队列优先级;According to the object priority of the object to be processed, determine the queue priority that is the same as the object priority;

将所述待处理对象的对象信息置于所述队列优先级对应的处理队列中。The object information of the object to be processed is placed in a processing queue corresponding to the queue priority.

在一种可选的实施例中,所述对象信息中包括所述待处理对象对应的标识和处理要求;所述数据处理单元,具体用于:In an optional embodiment, the object information includes an identifier corresponding to the object to be processed and processing requirements; the data processing unit is specifically configured to:

根据所述待处理对象对应的标识,从索引存储器中查找与所述标识相匹配的索引信息;According to the identifier corresponding to the object to be processed, look up the index information matching the identifier from the index memory;

根据所述索引信息,确定所述待处理对象的数据在数据存储器中的原始存储位置,并根据所述待处理对象对应的处理要求,对所述待处理对象的数据进行处理。According to the index information, the original storage location of the data of the object to be processed in the data memory is determined, and the data of the object to be processed is processed according to the processing requirements corresponding to the object to be processed.

在一种可选的实施例中,所述数据处理单元,还用于:In an optional embodiment, the data processing unit is further configured to:

若所述处理要求指示对所述待处理对象的数据进行删除操作,则将所述待处理对象的数据从所述原始存储位置处删除,并将所述索引信息从所述索引存储器中删除;If the processing request instructs to delete the data of the object to be processed, delete the data of the object to be processed from the original storage location, and delete the index information from the index memory;

若所述处理要求指示对所述待处理对象的数据进行迁移操作,则将所述待处理对象的数据从所述原始存储位置处,迁移到所述对象信息包括的目标存储位置处,并对所述索引存储器中的所述索引信息进行更新。If the processing request indicates to perform a migration operation on the data of the object to be processed, the data of the object to be processed is migrated from the original storage location to the target storage location included in the object information, and the data of the object to be processed is migrated to the target storage location included in the object information. The index information in the index memory is updated.

第三方面,本申请实施例还提供了一种计算机可读存储介质,所述计算机可读存储介质内存储有计算机程序,所述计算机程序被处理器执行时,实现第一方面的对象的生命周期管理方法。In a third aspect, embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the life of the object of the first aspect is realized. cycle management approach.

第四方面,本申请实施例还提供了一种电子设备,包括存储器和处理器,所述存储器上存储有可在所述处理器上运行的计算机程序,当所述计算机程序被所述处理器执行时,使得所述处理器实现第一方面的对象的生命周期管理方法。In a fourth aspect, embodiments of the present application further provide an electronic device, including a memory and a processor, where the memory stores a computer program that can run on the processor, and when the computer program is executed by the processor When executed, the processor is caused to implement the object lifecycle management method of the first aspect.

第二方面至第四方面中任意一种实现方式所带来的技术效果可参见第一方面中对应的实现方式所带来的技术效果,此处不再赘述。For the technical effects brought by any one of the implementation manners of the second aspect to the fourth aspect, reference may be made to the technical effects brought by the corresponding implementation manners in the first aspect, which will not be repeated here.

附图说明Description of drawings

为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简要介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域的普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

图1为本申请实施例提供的一种对象的生命周期管理方法的流程图;FIG. 1 is a flowchart of a method for life cycle management of an object provided by an embodiment of the present application;

图2为本申请实施例提供的一种对象的生命周期管理方法的示意图;FIG. 2 is a schematic diagram of a life cycle management method of an object provided by an embodiment of the present application;

图3为本申请实施例提供的另一种对象的生命周期管理方法的流程图;3 is a flowchart of another object lifecycle management method provided by an embodiment of the present application;

图4为本申请实施例提供的一种对象的生命周期管理系统的处理流程图;FIG. 4 is a process flow diagram of an object life cycle management system provided by an embodiment of the present application;

图5为本申请实施例提供的一种对象的生命周期管理装置的结构示意图;FIG. 5 is a schematic structural diagram of a device for managing a life cycle of an object according to an embodiment of the present application;

图6为本申请实施例提供的另一种对象的生命周期管理装置的结构示意图;FIG. 6 is a schematic structural diagram of another object life cycle management apparatus provided by an embodiment of the present application;

图7为本申请实施例提供的一种电子设备的结构示意图。FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

具体实施方式Detailed ways

为了使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请作进一步地详细描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例,都属于本申请保护的范围。In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

需要说明的是,本申请的文件中涉及的术语“包括”和“具有”以及它们的变形,意图在于覆盖不排他的包含,例如,包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元,而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。It should be noted that the terms "comprising" and "having" and their variants mentioned in the documents of this application are intended to cover non-exclusive inclusion, for example, a process, method, system, product including a series of steps or units Or apparatus is not necessarily limited to those steps or units expressly listed, but may include other steps or units not expressly listed or inherent to the process, method, product or apparatus.

下面将结合附图,对本申请实施例提供的技术方案进行详细说明。The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

下文中所用的词语“示例性”的意思为“用作例子、实施例或说明性”。作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。在本申请实施例的描述中,除非另有说明,“多个”的含义是两个或两个以上。As used hereinafter, the word "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment described as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

本申请实施例提供了一种对象的生命周期管理方法,如图1所示,包括如下步骤:The embodiment of the present application provides a life cycle management method for an object, as shown in FIG. 1 , including the following steps:

步骤S101,针对任意一个处理队列,基于为任意一个处理队列分配的工作线程,从任意一个处理队列中读取待处理对象的对象信息。Step S101 , for any processing queue, based on the worker thread allocated to any processing queue, read the object information of the object to be processed from any processing queue.

其中,处理队列中包括至少一个待处理对象的对象信息,处理队列中待处理对象的对象优先级与所属处理队列的队列优先级相匹配,较高队列优先级的处理队列的工作线程数量多于较低队列优先级的处理队列的工作线程数量,或者,较高队列优先级的处理队列的工作线程启动时间早于较低队列优先级的处理队列的工作线程启动时间。The processing queue includes object information of at least one to-be-processed object, the object priority of the to-be-processed object in the processing queue matches the queue priority of the processing queue to which it belongs, and the processing queue with a higher queue priority has more worker threads than The number of worker threads for a processing queue with a lower queue priority, or the worker thread start time for a processing queue with a higher queue priority is earlier than that for a processing queue with a lower queue priority.

可选的,可以先根据待处理对象对应的系统状况或业务需求,为待处理对象设置相应的对象优先级。Optionally, a corresponding object priority may be set for the to-be-processed object according to the system status or business requirement corresponding to the to-be-processed object.

具体地,若待处理对象对应的系统状况运行正常,或待处理对象对应的业务需求为即时处理,则为待处理对象设置第一对象优先级。若待处理对象的对应的系统状况出现故障,或待处理对象对应的业务需求为稍后处理,则为待处理对象设置第二对象优先级。其中,第一对象优先级高于第二对象优先级。Specifically, if the system status corresponding to the object to be processed is running normally, or the business requirement corresponding to the object to be processed is immediate processing, a first object priority is set for the object to be processed. If the corresponding system status of the object to be processed fails, or the business requirement corresponding to the object to be processed is to be processed later, a second object priority is set for the object to be processed. The priority of the first object is higher than the priority of the second object.

在对待处理对象进行对象优先级设置后,可以根据待处理对象的对象优先级,确定与对象优先级相同的队列优先级,并将待处理对象的对象信息置于队列优先级对应的处理队列中。After the object priority is set for the object to be processed, the queue priority that is the same as the object priority can be determined according to the object priority of the object to be processed, and the object information of the object to be processed is placed in the processing queue corresponding to the queue priority .

例如,如图2所示,存在待处理对象A、待处理对象B和待处理对象C共3个待处理对象,且待处理对象A的对象优先级为1,待处理对象B的对象优先级为2,待处理对象C的对象优先级为3。其中,对象优先级为1大于对象优先级为2大于对象优先级为3。For example, as shown in FIG. 2 , there are 3 to-be-processed objects, namely, to-be-processed object A, to-be-processed object B, and to-be-processed object C, and the object priority of the to-be-processed object A is 1, and the object priority of the to-be-processed object B is If it is 2, the object priority of the object C to be processed is 3. Wherein, the object priority is 1 greater than the object priority 2 is greater than the object priority 3.

根据待处理对象分别对应的对象优先级,可以将对象优先级为1的待处理对象A的对象信息置于队列优先级为1对应的处理队列a中,将对象优先级为2的待处理对象B的对象信息置于队列优先级为2对应的处理队列b中,将对象优先级为3的待处理对象C的对象信息置于队列优先级为3对应的处理队列c中。According to the object priorities corresponding to the objects to be processed, the object information of the to-be-processed object A with the object priority of 1 can be placed in the processing queue a corresponding to the queue priority of 1, and the object to be processed with the object priority of 2. The object information of B is placed in the processing queue b corresponding to the queue priority of 2, and the object information of the to-be-processed object C with the object priority of 3 is placed in the processing queue c corresponding to the queue priority of 3.

根据队列优先级,可以为队列优先级为1的处理队列a分配工作线程a1、工作线程a2和工作线程a3共3个工作线程,为队列优先级为2的处理队列b分配工作线程b1和工作线程b2共2个工作线程,为队列优先级为3的处理队列c分配工作线程c1共1个工作线程。According to the queue priority, a total of 3 worker threads can be allocated to the processing queue a with queue priority 1, including worker thread a1, worker thread a2 and worker thread a3, and the processing queue b with queue priority 2 can be allocated worker thread b1 and work thread Thread b2 has a total of 2 worker threads, and assigns a total of 1 worker thread c1 to the processing queue c with a queue priority of 3.

且工作线程a1、工作线程a2和工作线程a3从处理队列a中读取待处理对象A的对象信息的启动时间要早于工作线程b1和工作线程b2从处理队列b中读取待处理对象B的对象信息的启动时间,工作线程b1和工作线程b2从处理队列b中读取待处理对象B的对象信息的启动时间要早于工作线程c1从处理队列c中读取待处理对象C的对象信息的启动时间。And the start time of worker thread a1, worker thread a2 and worker thread a3 reading object information of pending object A from processing queue a is earlier than that of worker thread b1 and worker thread b2 reading pending object B from processing queue b The start time of the object information, the start time of the worker thread b1 and the worker thread b2 to read the object information of the to-be-processed object B from the processing queue b is earlier than the start time of the worker thread c1 to read the object of the to-be-processed object C from the processing queue c The start time of the message.

步骤S102,根据读取的待处理对象的对象信息,对待处理对象的数据进行处理。Step S102, according to the read object information of the object to be processed, process the data of the object to be processed.

其中,待处理对象的对象信息中包括有待处理对象对应的标识和处理要求。Wherein, the object information of the object to be processed includes an identifier and a processing requirement corresponding to the object to be processed.

工作线程在读取出待处理对象的对象信息后,可以根据对象信息中的包括的待处理对象对应的标识,从索引存储器中查找与标识相匹配的索引信息,并根据查找到的索引信息,确定出待处理对象的数据在数据存储器中的原始存储位置。After reading the object information of the object to be processed, the worker thread can search for the index information matching the identification from the index memory according to the identifier corresponding to the object to be processed included in the object information, and according to the found index information, Determine the original storage location of the data of the object to be processed in the data storage.

例如,待处理对象可以是文件,则待处理对象对应的标识可以是文件名,工作线程就可以根据文件名,从索引存储器中查找与该文件名相匹配的索引信息,该索引信息可以是文件的存储信息,根据文件的存储信息,就可以确定出文件在数据存储器中的原始存储位置。For example, the object to be processed may be a file, and the identifier corresponding to the object to be processed may be a file name, and the worker thread may search the index memory for index information matching the file name according to the file name, and the index information may be a file According to the storage information of the file, the original storage location of the file in the data storage can be determined.

在确定出待处理对象的数据在数据存储器中的原始存储位置后,若待处理对象对应的处理要求指示对待处理对象的数据进行删除操作,则可以将待处理对象的数据从原始存储位置处删除,并将待处理对象对应的索引信息从索引存储器中删除。After determining the original storage location of the data of the object to be processed in the data storage, if the processing requirement corresponding to the object to be processed indicates that the data of the object to be processed is deleted, the data of the object to be processed can be deleted from the original storage location , and delete the index information corresponding to the object to be processed from the index memory.

若待处理对象对应的处理要求指示对待处理对象的数据进行迁移操作,则可以将待处理对象的数据从原始存储位置处,迁移到待处理对象的对象信息中包括的目标存储位置处,并对索引存储器中的待处理对象对应的索引信息进行更新。If the processing requirement corresponding to the object to be processed indicates that the data of the object to be processed is migrated, the data of the object to be processed can be migrated from the original storage location to the target storage location included in the object information of the object to be processed, and the The index information corresponding to the object to be processed in the index memory is updated.

在一种实施例中,本申请实施例中提供的对象的生命周期管理方法,还可以按照如图3所示的过程进行实现,如图3所示,包括如下步骤:In an embodiment, the object lifecycle management method provided in the embodiment of the present application can also be implemented according to the process shown in FIG. 3 , as shown in FIG. 3 , including the following steps:

步骤S301,确定待处理对象和待处理对象的对象优先级。Step S301, determining the object to be processed and the object priority of the object to be processed.

在对待处理对象的数据进行处理之前,需要先根据每个待处理对象对应的系统状况或业务需求,分别为每个待处理对象设置相应的对象优先级。Before processing the data of the object to be processed, it is necessary to set a corresponding object priority for each object to be processed according to the system status or business requirements corresponding to each object to be processed.

步骤S302,根据待处理对象的对象优先级,确定与对象优先级相同的队列优先级,并将待处理对象的对象信息置于队列优先级对应的处理队列中。Step S302, according to the object priority of the object to be processed, determine a queue priority that is the same as the object priority, and place the object information of the to-be-processed object in a processing queue corresponding to the queue priority.

在设置好每个待处理对象分别对应的对象优先级后,针对一个待处理对象,可以根据待处理对象的对象优先级确定与对象优先级相同的队列优先级,并将待处理对象的对象信息置于队列优先级对应的处理队列中。After the object priority corresponding to each object to be processed is set, for an object to be processed, the queue priority that is the same as the object priority can be determined according to the object priority of the object to be processed, and the object information of the object to be processed can be determined. Placed in the processing queue corresponding to the queue priority.

其中,队列优先级与对象优先级的数量相同,且一个对象优先级对应的待处理对象的对象信息,可以置于与该对象优先级相同的队列优先级对应的处理队列中。The number of queue priorities and object priorities is the same, and the object information of an object to be processed corresponding to an object priority can be placed in a processing queue corresponding to a queue priority that is the same as the object priority.

步骤S303,基于分配的工作线程,从处理队列中读取待处理对象的对象信息。Step S303, based on the assigned worker thread, read the object information of the object to be processed from the processing queue.

在将待处理队列置于处理队列中后,可以为处理对象分配相应的工作线程。其中,较高队列优先级的处理队列的工作线程数量多于较低队列优先级的处理队列的工作线程数量,或者,较高队列优先级的处理队列的工作线程启动时间早于较低队列优先级的处理队列的工作线程启动时间。After the pending queue is placed in the processing queue, the corresponding worker thread can be assigned to the processing object. Among them, the number of worker threads of the processing queue with higher queue priority is more than the number of worker threads of the processing queue with lower queue priority, or the start time of worker threads of the processing queue with higher queue priority is earlier than that of the lower queue priority The worker thread start time of the processing queue of the level.

基于分配的工作线程,可以从处理队列中读取待处理对象的对象信息。Based on the assigned worker thread, object information for pending objects can be read from the processing queue.

步骤S304,根据对象信息中包括的待处理对象对应的标识,从索引存储器中查找与待处理对象对应的标识相匹配的索引信息。Step S304, according to the identifier corresponding to the object to be processed included in the object information, search for index information matching the identifier corresponding to the object to be processed from the index memory.

待处理对象的对象信息中可以包括待处理对象对应的标识,根据待处理对象对应的标识,就可以从索引存储器中查找与该标识相匹配的索引信息。The object information of the object to be processed may include an identifier corresponding to the object to be processed, and according to the identifier corresponding to the object to be processed, index information matching the identifier can be searched from the index memory.

步骤S305,根据索引信息,确定待处理对象的数据在数据存储器中的原始存储位置。Step S305, according to the index information, determine the original storage location of the data of the object to be processed in the data storage.

根据查找到的索引信息,就可以确定待处理对象的数据在数据存储器中的原始存储位置。According to the found index information, the original storage location of the data of the object to be processed in the data storage can be determined.

步骤S306,若对象信息中包括的待处理对象对应的处理要求指示对待处理对象的数据进行删除操作,则将待处理对象的数据从原始存储位置处删除,并将索引信息从索引存储器中删除。Step S306, if the processing requirement corresponding to the object to be processed included in the object information indicates that the data of the object to be processed is deleted, the data of the object to be processed is deleted from the original storage location, and the index information is deleted from the index memory.

待处理对象的对象信息中还可以包括待处理对象对应的处理要求,若处理要求指示对待处理对象的数据进行删除操作,则可以将待处理对象的数据从数据存储器中的原始存储位置处删除,并将索引信息从索引存储器中删除。The object information of the object to be processed may also include the processing request corresponding to the object to be processed. If the processing request indicates that the data of the object to be processed is deleted, the data of the object to be processed can be deleted from the original storage location in the data storage, and delete the index information from the index memory.

步骤S307,若处理要求指示对待处理对象的数据进行迁移操作,则将待处理对象的数据从原始存储位置处,迁移到对象信息包括的目标存储位置处,并对索引存储器中的索引信息进行更新。Step S307, if the processing requirement indicates that the data of the object to be processed is migrated, the data of the object to be processed is migrated from the original storage location to the target storage location included in the object information, and the index information in the index memory is updated. .

若处理要求指示对待处理对象的数据进行迁移操作,则可以待处理对象的数据从数据存储器中的原始存储位置处,迁移到对象信息中包括的目标存储位置处,并对索引存储器中的索引信息进行更新。If the processing requirement indicates that the data of the object to be processed is migrated, the data of the object to be processed can be migrated from the original storage location in the data storage to the target storage location included in the object information, and the index information in the index storage can be migrated. to update.

在一些实施例中,本申请还对应提供了一个对象的生命周期管理系统,该系统主要包括:优先级策略管理器、索引存储器、数据存储器、索引扫描器、处理队列和任务调度管理器。In some embodiments, the present application also provides an object life cycle management system, which mainly includes: a priority policy manager, an index storage, a data storage, an index scanner, a processing queue, and a task scheduling manager.

其中,优先级策略管理器:接收来自系统管理员的优先级设置策略,为不同的业务(对应系统中的桶、数据池)设置优先级,管理业务和处理队列的对应关系。Among them, the priority policy manager: receives the priority setting policy from the system administrator, sets priorities for different services (corresponding to buckets and data pools in the system), and manages the correspondence between services and processing queues.

索引存储器:用来保存对象存储器的桶索引数据,桶的对象数很大时,可以进行哈希分片。Index storage: It is used to store the bucket index data of the object storage. When the number of objects in the bucket is large, hash sharding can be performed.

数据存储器:用来保存对象存储的数据,可以根据业务划分不同的池,不同的池有不同的硬件资源,互不影响。Data storage: It is used to store the data of object storage. It can be divided into different pools according to the business. Different pools have different hardware resources and do not affect each other.

索引扫描器:索引扫描器是后台线程,按照系统设定的策略(在什么时间段运行、多长时间进行一轮扫描等)扫描索引分片,逐个读取从索引分片里面得到的对象元数据,根据元数据里面的桶名、对象名、修改时间、桶的生命周期配置规则以及优先级配置规则,将需要处理的对象元数据及操作类型(删除或者迁移)保存到相应的处理队列中,且索引扫描器是处理队列的生产者。Index scanner: The index scanner is a background thread that scans the index shards according to the policy set by the system (in what time period, how long to perform a round of scanning, etc.), and reads the object elements obtained from the index shards one by one Data, according to the bucket name, object name, modification time, bucket life cycle configuration rules and priority configuration rules in the metadata, save the object metadata and operation type (deletion or migration) to be processed in the corresponding processing queue , and the index scanner is the producer of the processing queue.

处理队列:具有不同队列优先级的处理队列,且处理队列为消息队列,用来保存从索引扫描器扫描得到的需要处理的对象元数据。Processing queue: Processing queues with different queue priorities, and the processing queues are message queues, which are used to store the object metadata scanned by the index scanner to be processed.

任务调度管理器:任务调度管理器管理了一组工作线程,这些工作线程是处理队列的消费者,用来从处理队列中读取对象元数据信息,根据操作类型执行相应操作,且任务调度管理器可以根据优先级设置动态分配工作线程。Task scheduling manager: The task scheduling manager manages a set of worker threads, which are consumers of the processing queue, which are used to read object metadata information from the processing queue, perform corresponding operations according to the operation type, and manage the task scheduling. The processor can dynamically allocate worker threads based on priority settings.

具体地,本申请实施例中提供的对象的生命周期管理系统中的各个组件的处理流程可以如图4所示。如图4所示,系统管理员根据系统状况和业务需求设置生命周期任务优先级策略,优先级策略管理器可以接收系统管理员发送的优先级设置策略,并根据该优先级设置策略进行对象的优先级设置和处理队列的优先级设置,以及将优先级设置策略和优先级设置结果持久化保存。Specifically, the processing flow of each component in the object life cycle management system provided in the embodiment of the present application may be as shown in FIG. 4 . As shown in Figure 4, the system administrator sets the priority policy of life cycle tasks according to the system status and business requirements. The priority policy manager can receive the priority setting policy sent by the system administrator, and carry out object assignment according to the priority setting policy. Priority setting and priority setting of the processing queue, as well as persistence of the priority setting policy and the priority setting result.

索引扫描器可以加载优先级设置策略和生命周期规则,根据系统设置定期进行扫描,从索引存储器中获取索引分片数据,根据生命周期规则对分片中的每个对象进行分析,如果需要处理,则将待处理对象的对象信息写入到对应的处理队列中,如果不需要处理,则继续分析下一对象。其中,对象的对象信息中可以包括对象元数据和操作类型等。The index scanner can load priority setting policies and life cycle rules, scan periodically according to system settings, obtain index shard data from index storage, analyze each object in the shard according to the life cycle rules, and process if necessary. Then write the object information of the object to be processed into the corresponding processing queue, and continue to analyze the next object if no processing is required. The object information of the object may include object metadata, operation type, and the like.

任务调度管理器可以加载优先级设置策略,并根据该优先级设置策略为不同的处理队列分配工作线程,工作线程轮询分配到的处理队列,逐个对待处理对象的数据进行处理:如果是删除操作,则从数据存储器中删除对象的数据并更新索引;如果是迁移操作,则将对象的数据从数据存储器中的原始存储位置迁移到目标存储位置处,并更新索引。The task scheduling manager can load the priority setting policy and assign worker threads to different processing queues according to the priority setting policy. The worker threads poll the assigned processing queues and process the data of the objects to be processed one by one: if it is a delete operation , delete the data of the object from the data storage and update the index; if it is a migration operation, migrate the data of the object from the original storage location in the data storage to the target storage location, and update the index.

本申请提供的对象的生命周期管理系统,与相关技术相比,具有以下优点:Compared with the related art, the object life cycle management system provided by this application has the following advantages:

1、提高了系统的可运营性。可以根据业务和系统的具体情况进行任务调度,使系统的整体资源利用更加高效。1. Improve the operability of the system. Task scheduling can be performed according to the specific conditions of the business and the system to make the overall resource utilization of the system more efficient.

2、提高了系统的稳定性。不同的业务之间的任务可通过任务队列、工作线程的隔离来避免互相影响,避免了某个业务的局部动荡引发整体动荡。2. Improve the stability of the system. Tasks between different businesses can be isolated from each other through task queues and worker threads, so that local turbulence of a business can cause overall turbulence.

本申请提供一种对象的生命周期管理方法,可以解决ceph对象存储中原先存在的顺序处理对象,而如果某个业务压力大,或者某个存储池故障,就会导致对整体对象的处理速度下降,进而影响整体生命周期的管理效率的问题。本申请中提供的对象的生命周期管理方法,使得ceph对象存储可以根据业务特点进行生命周期管理任务的精确调度,提高了系统的可运营性,同时将索引的扫描和处理通过优先级队列进行解耦,将不同业务(或数据池)的扫描结果保存到不同的队列,任务管理器根据业务的优先级进行任务调度,某个队列的任务异常不会影响其他任务的执行,做到了业务之间任务的隔离,避免单个业务的局部震荡引发系统的整体震荡,提高了系统的稳定性。This application provides an object life cycle management method, which can solve the problem of sequential processing of objects in the ceph object storage. However, if a certain business pressure is high, or a certain storage pool fails, the processing speed of the whole object will decrease. , which in turn affects the management efficiency of the overall life cycle. The object life cycle management method provided in this application enables the ceph object storage to perform precise scheduling of life cycle management tasks according to business characteristics, improving the operability of the system, and at the same time, the scanning and processing of the index is resolved through the priority queue. Coupling, save the scan results of different services (or data pools) to different queues, the task manager schedules tasks according to the priority of the business, the task exception of a queue will not affect the execution of other tasks, so that the business Isolation of tasks prevents local shocks of a single business from causing overall shocks to the system, and improves system stability.

与图1所示的对象的生命周期管理方法基于同一发明构思,本申请实施例中还提供了一种对象的生命周期管理装置。由于该装置是本申请对象的生命周期管理方法对应的装置,并且该装置解决问题的原理与该方法相似,因此该装置的实施可以参见上述方法的实施,重复之处不再赘述。Based on the same inventive concept as the object life cycle management method shown in FIG. 1 , the embodiment of the present application further provides an object life cycle management apparatus. Since the device is a device corresponding to the life cycle management method of the object of the present application, and the principle of solving the problem of the device is similar to that of the method, the implementation of the device can refer to the implementation of the above method, and the repetition will not be repeated.

图5示出了本申请实施例提供的一种对象的生命周期管理装置的结构示意图,如图5所示,该对象的生命周期管理装置包括对象信息读取单元501和数据处理单元502。FIG. 5 shows a schematic structural diagram of an object life cycle management apparatus provided by an embodiment of the present application. As shown in FIG. 5 , the object life cycle management apparatus includes an object information reading unit 501 and a data processing unit 502 .

其中,对象信息读取单元501,用于针对任意一个处理队列,基于为任意一个处理队列分配的工作线程,从任意一个处理队列中读取待处理对象的对象信息;其中,处理队列中包括至少一个待处理对象的对象信息,处理队列中待处理对象的对象优先级与所属处理队列的队列优先级相匹配,较高队列优先级的处理队列的工作线程数量多于较低队列优先级的处理队列的工作线程数量,或者,较高队列优先级的处理队列的工作线程启动时间早于较低队列优先级的处理队列的工作线程启动时间;The object information reading unit 501 is configured to, for any processing queue, read the object information of the object to be processed from any processing queue based on the worker thread allocated to any processing queue; wherein, the processing queue includes at least Object information of a pending object. The object priority of the pending object in the processing queue matches the queue priority of the processing queue to which it belongs. The processing queue with a higher queue priority has more worker threads than those with a lower queue priority. The number of worker threads of the queue, or, the worker thread start time of the processing queue of the higher queue priority is earlier than the start time of the worker thread of the processing queue of the lower queue priority;

数据处理单元502,用于根据读取的待处理对象的对象信息,对待处理对象的数据进行处理。The data processing unit 502 is configured to process the data of the object to be processed according to the read object information of the object to be processed.

在一种可选的实施例中,如图6所示,上述装置还可以包括优先级确定单元601,用于:In an optional embodiment, as shown in FIG. 6 , the foregoing apparatus may further include a priority determination unit 601, configured to:

根据待处理对象对应的系统状况或业务需求,为待处理对象设置相应的对象优先级。Set the corresponding object priority for the to-be-processed object according to the system status or business requirement corresponding to the to-be-processed object.

在一种可选的实施例中,优先级确定单元601,具体用于:In an optional embodiment, the priority determination unit 601 is specifically configured to:

若待处理对象对应的系统状况运行正常,或待处理对象对应的业务需求为即时处理,则为待处理对象设置第一对象优先级;If the system status corresponding to the object to be processed is running normally, or the business requirement corresponding to the object to be processed is immediate processing, set the first object priority for the object to be processed;

若待处理对象的对应的系统状况出现故障,或待处理对象对应的业务需求为稍后处理,则为待处理对象设置第二对象优先级;第一对象优先级高于第二对象优先级。If the corresponding system status of the to-be-processed object fails, or the business requirement corresponding to the to-be-processed object is to be processed later, a second object priority is set for the to-be-processed object; the first object priority is higher than the second object priority.

在一种可选的实施例中,如图6所示,上述装置还可以包括队列加入单元602,用于:In an optional embodiment, as shown in FIG. 6 , the above-mentioned apparatus may further include a queue adding unit 602 for:

根据待处理对象的对象优先级,确定与对象优先级相同的队列优先级;According to the object priority of the object to be processed, determine the queue priority that is the same as the object priority;

将待处理对象的对象信息置于队列优先级对应的处理队列中。The object information of the object to be processed is placed in the processing queue corresponding to the queue priority.

在一种可选的实施例中,对象信息中包括待处理对象对应的标识和处理要求;数据处理单元502,具体用于:In an optional embodiment, the object information includes the identifier and processing requirements corresponding to the object to be processed; the data processing unit 502 is specifically used for:

根据待处理对象对应的标识,从索引存储器中查找与标识相匹配的索引信息;According to the identifier corresponding to the object to be processed, the index information matching the identifier is searched from the index memory;

根据索引信息,确定待处理对象的数据在数据存储器中的原始存储位置,并根据待处理对象对应的处理要求,对待处理对象的数据进行处理。According to the index information, the original storage location of the data of the object to be processed in the data storage is determined, and the data of the object to be processed is processed according to the processing requirements corresponding to the object to be processed.

在一种可选的实施例中,数据处理单元502,还用于:In an optional embodiment, the data processing unit 502 is further configured to:

若处理要求指示对待处理对象的数据进行删除操作,则将待处理对象的数据从原始存储位置处删除,并将索引信息从索引存储器中删除;If the processing request indicates that the data of the object to be processed is deleted, the data of the object to be processed is deleted from the original storage location, and the index information is deleted from the index memory;

若处理要求指示对待处理对象的数据进行迁移操作,则将待处理对象的数据从原始存储位置处,迁移到对象信息包括的目标存储位置处,并对索引存储器中的索引信息进行更新。If the processing requirement indicates that the data of the object to be processed is migrated, the data of the object to be processed is migrated from the original storage location to the target storage location included in the object information, and the index information in the index memory is updated.

与上述方法实施例基于同一发明构思,本申请实施例中还提供了一种电子设备。该电子设备可以用于执行对象的生命周期管理过程。在该实施例中,电子设备的结构可以如图7所示,包括存储器701以及一个或多个处理器702。Based on the same inventive concept as the above method embodiments, the embodiments of the present application further provide an electronic device. The electronic device may be used to perform a lifecycle management process of the object. In this embodiment, the structure of the electronic device may be as shown in FIG. 7 , including a memory 701 and one or more processors 702 .

存储器701,用于存储处理器702执行的计算机程序。存储器701可主要包括存储程序区和存储数据区,其中,存储程序区可存储操作系统,以及运行即时通讯功能所需的程序等;存储数据区可存储各种即时通讯信息和操作指令集等。The memory 701 is used for storing computer programs executed by the processor 702 . The memory 701 may mainly include a stored program area and a stored data area, wherein the stored program area may store the operating system and the programs required for running the instant messaging function, and the like; the storage data area may store various instant messaging information and operation instruction sets.

存储器701可以是易失性存储器(volatile memory),例如随机存取存储器(random-access memory,RAM);存储器701也可以是非易失性存储器(non-volatilememory),例如只读存储器,快闪存储器(flash memory),硬盘(hard disk drive,HDD)或固态硬盘(solid-state drive,SSD)、或者存储器701是能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质,但不限于此。存储器701可以是上述存储器的组合。The memory 701 may be a volatile memory (volatile memory), such as random-access memory (random-access memory, RAM); the memory 701 may also be a non-volatile memory (non-volatile memory), such as read-only memory, flash memory (flash memory), hard disk drive (HDD) or solid-state drive (SSD), or memory 701 is capable of carrying or storing desired program code in the form of instructions or data structures and capable of being used by a computer Access any other medium without limitation. The memory 701 may be a combination of the above-described memories.

处理器702,可以包括一个或多个中央处理单元(central processing unit,CPU)或者为数字处理单元等等。处理器702,用于调用存储器701中存储的计算机程序时实现上述对象的生命周期管理方法。The processor 702 may include one or more central processing units (central processing units, CPU), or be a digital processing unit or the like. The processor 702 is configured to implement the above object life cycle management method when invoking the computer program stored in the memory 701 .

本申请实施例中不限定上述存储器701和处理器702之间的具体连接介质。本公开实施例在图7中以存储器701和处理器702之间通过总线703连接,总线703在图7中以粗线表示,其它部件之间的连接方式,仅是进行示意性说明,并不引以为限。总线703可以分为地址总线、数据总线、控制总线等。为便于表示,图7中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。The specific connection medium between the above-mentioned memory 701 and the processor 702 is not limited in this embodiment of the present application. In the embodiment of the present disclosure, the memory 701 and the processor 702 are connected through a bus 703 in FIG. 7 , and the bus 703 is represented by a thick line in FIG. 7 . The connection between other components is only for schematic illustration, not for be limited. The bus 703 can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in FIG. 7, but it does not mean that there is only one bus or one type of bus.

根据本申请的一个方面,提供了一种计算机程序产品或计算机程序,该计算机程序产品或计算机程序包括计算机指令,该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令,处理器执行该计算机指令,使得该计算机设备执行上述实施例中的对象的生命周期管理方法。According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the object lifecycle management method in the foregoing embodiment.

程序产品可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件,或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括:具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本申请的保护范围之内。The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application.

Claims (10)

1.一种对象的生命周期管理方法,其特征在于,包括:1. A life cycle management method for an object, comprising: 针对任意一个处理队列,基于为所述任意一个处理队列分配的工作线程,从所述任意一个处理队列中读取待处理对象的对象信息,并根据读取的所述待处理对象的对象信息,对所述待处理对象的数据进行处理;For any processing queue, based on the worker thread allocated to the any processing queue, read the object information of the object to be processed from the any processing queue, and according to the read object information of the object to be processed, processing the data of the object to be processed; 其中,所述处理队列中包括至少一个待处理对象的对象信息,所述处理队列中待处理对象的对象优先级与所属处理队列的队列优先级相匹配,较高队列优先级的处理队列的工作线程数量多于较低队列优先级的处理队列的工作线程数量,或者,较高队列优先级的处理队列的工作线程启动时间早于较低队列优先级的处理队列的工作线程启动时间。The processing queue includes object information of at least one object to be processed, the object priority of the object to be processed in the processing queue matches the queue priority of the processing queue to which it belongs, and the work of the processing queue with a higher queue priority The number of threads is greater than the number of worker threads for the processing queue of the lower queue priority, or the worker thread start time of the processing queue of the higher queue priority is earlier than the start time of the worker threads of the processing queue of the lower queue priority. 2.如权利要求1所述的方法,其特征在于,通过下列方式确定所述待处理对象的对象优先级:2. The method of claim 1, wherein the object priority of the object to be processed is determined by: 根据所述待处理对象对应的系统状况或业务需求,为所述待处理对象设置相应的对象优先级。A corresponding object priority is set for the to-be-processed object according to the system status or business requirement corresponding to the to-be-processed object. 3.如权利要求2所述的方法,其特征在于,所述根据所述待处理对象对应的系统状况或业务需求,为所述待处理对象设置相应的对象优先级,包括:3. The method according to claim 2, wherein, according to the system status or business requirement corresponding to the to-be-processed object, setting a corresponding object priority for the to-be-processed object, comprising: 若所述待处理对象对应的系统状况运行正常,或所述待处理对象对应的业务需求为即时处理,则为所述待处理对象设置第一对象优先级;If the system status corresponding to the object to be processed is running normally, or the business requirement corresponding to the object to be processed is immediate processing, set a first object priority for the object to be processed; 若所述待处理对象的对应的系统状况出现故障,或所述待处理对象对应的业务需求为稍后处理,则为所述待处理对象设置第二对象优先级;所述第一对象优先级高于所述第二对象优先级。If the corresponding system status of the to-be-processed object fails, or the business requirement corresponding to the to-be-processed object is to be processed later, a second object priority is set for the to-be-processed object; the first object priority higher than the second object priority. 4.如权利要求1所述的方法,其特征在于,所述从所述任意一个处理队列中读取待处理对象的对象信息之前,所述方法还包括:4. The method according to claim 1, wherein before reading the object information of the object to be processed from the any one of the processing queues, the method further comprises: 根据所述待处理对象的对象优先级,确定与所述对象优先级相同的队列优先级;According to the object priority of the object to be processed, determine the queue priority that is the same as the object priority; 将所述待处理对象的对象信息置于所述队列优先级对应的处理队列中。The object information of the object to be processed is placed in a processing queue corresponding to the queue priority. 5.如权利要求1所述的方法,其特征在于,所述对象信息中包括所述待处理对象对应的标识和处理要求;所述根据读取的所述待处理对象的对象信息,对所述待处理对象的数据进行处理,包括:5. The method according to claim 1, wherein the object information includes a corresponding identifier and processing requirements of the object to be processed; The data of the object to be processed is processed, including: 根据所述待处理对象对应的标识,从索引存储器中查找与所述标识相匹配的索引信息;According to the identifier corresponding to the object to be processed, look up the index information matching the identifier from the index memory; 根据所述索引信息,确定所述待处理对象的数据在数据存储器中的原始存储位置,并根据所述待处理对象对应的处理要求,对所述待处理对象的数据进行处理。According to the index information, the original storage location of the data of the object to be processed in the data memory is determined, and the data of the object to be processed is processed according to the processing requirements corresponding to the object to be processed. 6.如权利要求5所述的方法,其特征在于,所述根据所述待处理对象对应的处理要求,对所述待处理对象的数据进行处理,包括:6. The method according to claim 5, wherein the processing of the data of the object to be processed according to the processing requirements corresponding to the object to be processed comprises: 若所述处理要求指示对所述待处理对象的数据进行删除操作,则将所述待处理对象的数据从所述原始存储位置处删除,并将所述索引信息从所述索引存储器中删除;If the processing request instructs to delete the data of the object to be processed, delete the data of the object to be processed from the original storage location, and delete the index information from the index memory; 若所述处理要求指示对所述待处理对象的数据进行迁移操作,则将所述待处理对象的数据从所述原始存储位置处,迁移到所述对象信息包括的目标存储位置处,并对所述索引存储器中的所述索引信息进行更新。If the processing request indicates to perform a migration operation on the data of the object to be processed, the data of the object to be processed is migrated from the original storage location to the target storage location included in the object information, and the data of the object to be processed is migrated to the target storage location included in the object information. The index information in the index memory is updated. 7.一种对象的生命周期管理装置,其特征在于,包括:7. A device for managing the life cycle of an object, comprising: 对象信息读取单元,用于针对任意一个处理队列,基于为所述任意一个处理队列分配的工作线程,从所述任意一个处理队列中读取待处理对象的对象信息;其中,所述处理队列中包括至少一个待处理对象的对象信息,所述处理队列中待处理对象的对象优先级与所属处理队列的队列优先级相匹配,较高队列优先级的处理队列的工作线程数量多于较低队列优先级的处理队列的工作线程数量,或者,较高队列优先级的处理队列的工作线程启动时间早于较低队列优先级的处理队列的工作线程启动时间;The object information reading unit is configured to, for any processing queue, read the object information of the object to be processed from the any processing queue based on the worker thread allocated to the processing queue; wherein, the processing queue It includes object information of at least one object to be processed, the object priority of the object to be processed in the processing queue matches the queue priority of the processing queue to which it belongs, and the number of worker threads in the processing queue with a higher queue priority is more than that of a lower queue priority. The number of worker threads of the processing queue of the queue priority, or the start time of the worker thread of the processing queue of the higher queue priority is earlier than the start time of the worker thread of the processing queue of the lower queue priority; 数据处理单元,用于根据读取的所述待处理对象的对象信息,对所述待处理对象的数据进行处理。The data processing unit is configured to process the data of the to-be-processed object according to the read object information of the to-be-processed object. 8.如权利要求7所述的装置,其特征在于,所述装置还包括优先级确定单元,用于:8. The apparatus according to claim 7, wherein the apparatus further comprises a priority determination unit for: 根据所述待处理对象对应的系统状况或业务需求,为所述待处理对象设置相应的对象优先级。A corresponding object priority is set for the to-be-processed object according to the system status or business requirement corresponding to the to-be-processed object. 9.一种电子设备,其特征在于,其包括处理器和存储器,其中,所述存储器存储有程序代码,当所述程序代码被所述处理器执行时,使得所述处理器执行权利要求1~6中任一所述方法的步骤。9. An electronic device, characterized in that it comprises a processor and a memory, wherein the memory stores a program code that, when executed by the processor, causes the processor to execute claim 1 The steps of any of the methods in ~6. 10.一种计算机可读存储介质,其特征在于,其包括程序代码,当所述程序代码在电子设备上运行时,所述程序代码用于使所述电子设备执行权利要求1~6中任一所述方法的步骤。10. A computer-readable storage medium, characterized in that it comprises program code, when the program code is run on an electronic device, the program code is used to cause the electronic device to execute any one of claims 1 to 6 a step of the method.
CN202111680650.1A 2021-12-27 2021-12-27 Object life cycle management method, device, electronic device and storage medium Pending CN114443248A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111680650.1A CN114443248A (en) 2021-12-27 2021-12-27 Object life cycle management method, device, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111680650.1A CN114443248A (en) 2021-12-27 2021-12-27 Object life cycle management method, device, electronic device and storage medium

Publications (1)

Publication Number Publication Date
CN114443248A true CN114443248A (en) 2022-05-06

Family

ID=81366245

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111680650.1A Pending CN114443248A (en) 2021-12-27 2021-12-27 Object life cycle management method, device, electronic device and storage medium

Country Status (1)

Country Link
CN (1) CN114443248A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116737400A (en) * 2023-08-15 2023-09-12 中移(苏州)软件技术有限公司 Queue data processing method and device and related equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105117284A (en) * 2015-09-09 2015-12-02 厦门雅迅网络股份有限公司 Scheduling method for worker thread based on priority proportion queue
CN109889373A (en) * 2019-01-22 2019-06-14 视联动力信息技术股份有限公司 A kind of transmission method of warning information, device and system
CN110674064A (en) * 2019-09-05 2020-01-10 苏州浪潮智能科技有限公司 Data transmission method, device, equipment and computer readable storage medium
CN111510395A (en) * 2020-06-16 2020-08-07 腾讯科技(深圳)有限公司 Service message reporting method, device, equipment and medium
CN112783659A (en) * 2021-02-01 2021-05-11 北京百度网讯科技有限公司 Resource allocation method and device, computer equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105117284A (en) * 2015-09-09 2015-12-02 厦门雅迅网络股份有限公司 Scheduling method for worker thread based on priority proportion queue
CN109889373A (en) * 2019-01-22 2019-06-14 视联动力信息技术股份有限公司 A kind of transmission method of warning information, device and system
CN110674064A (en) * 2019-09-05 2020-01-10 苏州浪潮智能科技有限公司 Data transmission method, device, equipment and computer readable storage medium
CN111510395A (en) * 2020-06-16 2020-08-07 腾讯科技(深圳)有限公司 Service message reporting method, device, equipment and medium
CN112783659A (en) * 2021-02-01 2021-05-11 北京百度网讯科技有限公司 Resource allocation method and device, computer equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116737400A (en) * 2023-08-15 2023-09-12 中移(苏州)软件技术有限公司 Queue data processing method and device and related equipment
CN116737400B (en) * 2023-08-15 2023-11-03 中移(苏州)软件技术有限公司 Queue data processing method and device and related equipment

Similar Documents

Publication Publication Date Title
US11340803B2 (en) Method for configuring resources, electronic device and computer program product
US11425194B1 (en) Dynamically modifying a cluster of computing nodes used for distributed execution of a program
EP3796150B1 (en) Storage volume creation method and apparatus, server, and storage medium
US8321558B1 (en) Dynamically monitoring and modifying distributed execution of programs
JP4677239B2 (en) Application-aware systems that dynamically partition and allocate resources on demand
CN105049268A (en) Distributed computing resource allocation system and task processing method
CN113986497B (en) Queue scheduling method, device and system based on multi-tenant technology
EP4052126A1 (en) Management of multiple physical function non-volatile memory devices
WO2021114848A1 (en) Data reading and writing method and device for database
CN114116158A (en) A task scheduling method and system based on SD-WAN system
US11900155B2 (en) Method, device, and computer program product for job processing
CN114443248A (en) Object life cycle management method, device, electronic device and storage medium
TW201709047A (en) Routine task allocating method and multicore computer using the same
US11650846B2 (en) Method, device, and computer program product for job processing
US11269525B2 (en) Co-processing a plurality of dependent systems with a finite number of processing threads
WO2025025589A1 (en) Cross-runtime instance hybrid deployment method and apparatus based on cloud computing technology
US8176105B2 (en) Automated file relocation
WO2024125114A1 (en) Resource allocation method and apparatus based on cloud service
US20200242084A1 (en) Dynamically increasing available storage space in a storage volume
CN109478151B (en) Network accessible data volume modification
CN116774939A (en) A file migration method, device, equipment and readable storage medium
US11348656B2 (en) Efficient resource sharing
US11768704B2 (en) Increase assignment effectiveness of kubernetes pods by reducing repetitive pod mis-scheduling
US11983570B2 (en) Conditionally deploying a reusable group of containers for a job based on available system resources
CN115934363A (en) Method, system and device for emptying bucket of distributed object storage system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination