CN113434551B - Data processing method, apparatus, device and computer storage medium - Google Patents

Abstract

The present disclosure discloses a data processing method, device, device and computer storage medium, and relates to the technical fields of artificial intelligence such as augmented reality, deep learning, and computer vision technology. The specific implementation scheme is as follows: the algorithm cache module obtains the algorithm data obtained by the first processing of the data to be processed by the algorithm module, and transmits the time stamp encapsulated with the data to be processed, the algorithm capability type and the handle of the algorithm data to the management module ; The management module determines the corresponding processing module according to the algorithm capability type, and transmits the timestamp and the algorithm data to the processing module; the processing module determines the corresponding data to be processed according to the timestamp, and using the algorithm data to perform second processing on the data to be processed. The present disclosure can ensure the synchronization of algorithm data and data to be processed, and improve the effect of data processing.

Description

Data processing method, apparatus, device and computer storage medium

technical field

The present disclosure relates to the technical field of data processing, and in particular, to the technical field of artificial intelligence such as augmented reality, deep learning, and computer vision technology.

Background technique

With the continuous development of computer vision technology, more and more applications realize complex visual processing effects. Such as image enhancement technology, image special effects, 3D rendering and so on. However, in the current data processing process, it is difficult to synchronize both the data output from the data source and the algorithm data, resulting in poor processing results. For example, when the data source outputs each frame of data, the original segmentation data such as face, hair and other regions (ie algorithm data) is for the first frame of data, but for some reasons the algorithm data is applied to the second frame of data or The third frame data, etc., cause deviations in the coloring performed for each area.

SUMMARY OF THE INVENTION

In view of this, the present disclosure provides a data processing method, apparatus, device, and computer storage medium, so as to improve the effect of data processing.

According to a first aspect of the present disclosure, there is provided a data processing method, comprising:

The algorithm cache module obtains the algorithm data obtained by the first processing of the data to be processed by the algorithm module, and transmits the time stamp encapsulated with the data to be processed, the algorithm capability type and the handle of the algorithm data to the management module;

The management module determines a corresponding processing module according to the algorithm capability type, and transmits the timestamp and the algorithm data to the processing module;

The processing module determines the corresponding data to be processed according to the time stamp, and performs second processing on the data to be processed by using the algorithm data.

According to a second aspect of the present disclosure, a data processing apparatus is provided, including an algorithm cache module, a management module, and one or more processing modules;

The algorithm cache module is used to obtain the algorithm data obtained by the algorithm module performing the first processing on the data to be processed, and pass the timestamp encapsulated with the data to be processed, the algorithm capability type and the handle of the algorithm data to the management module;

The management module is configured to determine a corresponding processing module according to the algorithm capability type, and transmit the time stamp and the algorithm data to the determined processing module;

The processing module is configured to determine the corresponding data to be processed according to the time stamp, and use the algorithm data to perform second processing on the data to be processed.

According to a third aspect of the present disclosure, there is provided an electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method as described above.

According to a fifth aspect of the present disclosure, a computer program product comprising a computer program, when executed by a processor, implements the method as described above.

It should be understood that what is described in this section is not intended to identify key or critical features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of drawings

The accompanying drawings are used for better understanding of the present solution, and do not constitute a limitation to the present disclosure. in:

1 is a schematic diagram of a system architecture involved in an embodiment of the present disclosure;

2 is a flowchart of a data processing method provided by an embodiment of the present disclosure;

3 is a flowchart of a detailed method provided by an embodiment of the present disclosure;

4 is a structural diagram of an apparatus provided by an embodiment of the present disclosure;

5 is a block diagram of an electronic device used to implement embodiments of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

In order to facilitate the understanding of the present disclosure, the system architecture involved in the present disclosure is first described. As shown in Figure 1, the system architecture mainly includes: a data acquisition device, a data source module, an algorithm module, an algorithm cache module, a management module and a processing module.

Wherein, the data acquisition device and the algorithm module are modules on the client. The data acquisition device is used for collecting data, and the data is the data to be processed in the present disclosure. The data to be processed can be text, images, video, audio, etc. As a typical application scenario, in subsequent embodiments of the present disclosure, the data to be processed is described by taking each image frame included in the video as an example.

The algorithm module is used to perform algorithm-related processing on the data to be processed to obtain algorithm data. This part will be described in detail in subsequent examples.

The data source module, management module and processing module are modules in the engine. The data source module is used to obtain the collected data from the data acquisition device, cache it, and distribute the data as a data source in the engine. The processing module is a module used to process the data to be processed, and performs processing to produce final effects, such as image enhancement, special effects, and 3D rendering. Usually, more than one processing module is included in an engine to complete different types of processing. The management module is a module for scheduling and managing each processing module.

The algorithm cache module is a newly added module in the present disclosure, and is used to connect the algorithm module and the management module in series. Under normal circumstances, data is transferred between the algorithm caching module and the management module through a virtual machine such as a LUA. The improvement in the present disclosure to improve the data processing effect lies in the above-mentioned algorithm cache module and the module in the engine, which do not affect the algorithm module on the end and can be perfectly adapted.

The data processing method provided by the present disclosure will be described in detail below with reference to the embodiments. FIG. 2 is a flowchart of a data processing method provided by an embodiment of the present disclosure. As shown in FIG. 2 , the method may include the following steps:

In 201, the algorithm cache module obtains algorithm data obtained by performing first processing on the data to be processed by the algorithm module.

In 202, the algorithm cache module transmits the timestamp encapsulated with the data to be processed, the algorithm capability type and the handle of the algorithm data to the management module.

In 203, the management module determines the corresponding processing module according to the algorithm capability type, and transmits the timestamp and the algorithm data to the processing module.

In 204, the processing module determines the corresponding data to be processed according to the timestamp, and performs second processing on the data to be processed by using the algorithm data.

In the above-mentioned embodiment, the processing of the data to be processed by the algorithm module and the processing module is distinguished, which are respectively expressed as "first processing" and "second processing". It should be noted that "first" and "second" here only distinguish the two processes in name, and do not have limiting meanings such as order, quantity, and size. Wherein, the first processing provided by the algorithm module in the present disclosure is ultimately for the second processing service, that is, as the basis of the second processing service. For example, if the second processing is image special effects processing such as beauty makeup, the first processing can be some preprocessing algorithms before image special effects processing, such as face recognition, region segmentation processing, and the algorithm data obtained can be human faces. key points, region segmentation results, etc. For another example, if the second processing is image enhancement processing such as adding filters, the first processing can be some algorithms before the filtering processing, and the algorithm data can be the pixels determined by the maximum value method. new color values, etc.

It can be seen from the above technical solutions that the algorithm cache module transmits the time stamp of the data to be processed, the algorithm type and the algorithm data to the management module through the handle. The data is transmitted to the processing module, so that the processing module can ensure the synchronization of the algorithm data and the data to be processed according to the time stamp, and improve the effect of data processing.

The above method will be described in detail below with reference to FIG. 3 . In this embodiment, the data to be processed takes each image frame in the video as an example.

First, in 301, the data acquisition device will provide the current image frame to the data source module, and the data source module will buffer the image frame sequence.

In the present disclosure, the data acquisition apparatus may acquire videos from a collection device such as a terminal camera, or acquire videos from a server, or acquire locally stored videos.

The data acquisition device will provide the image frame by frame to the data source module in the engine, and the data source module will cache the image, for example, into an image queue.

In 302, the data source module will provide the current image frame to the algorithm module in advance for first processing.

In order to minimize the time difference between the algorithm data and the image frame arriving at the processing module, the data source module provides the image frame to the algorithm module for processing in advance before sending the image frame to the processing module.

In 303, after the algorithm module performs the first processing on the image frame, the obtained algorithm data is sent to the synchronization algorithm queue.

The algorithm module is also a module of the client. For different clients and different operating systems, the algorithms or codes used may be different when the same first processing is performed. However, since the present disclosure does not improve or affect the algorithm module itself, various algorithm modules can be directly adapted. In addition, the present disclosure does not improve or affect the data acquisition module and the data source module, and can be directly adapted.

The first processing performed on the image frame may include, but is not limited to: SLAM (Simultaneous Localization And Mapping), face recognition, region segmentation, gesture recognition, object detection, gesture detection, and the like. The region segmentation may further include portrait segmentation, hair segmentation, sky segmentation and so on. The specific first processing performed reflects the type of algorithm capability, that is, the algorithm capability possessed by the algorithm module determines the content of the first processing. Various algorithm capability types can be distinguished by type identifiers, for example, the type identifier of SLAM is 5, the type identifier of face recognition is 10, and so on.

The result obtained after the first process is performed is the algorithm data. For example, after face recognition, the key point information of the face is obtained. For another example, after the region is divided, region information is obtained. If the client has synchronization requirements for these algorithm data, it can be sent to the synchronization algorithm queue, and the processing method described in the present disclosure can be executed to ensure synchronization. If the client does not have synchronization requirements for these algorithm data, it can be sent to the asynchronous algorithm queue to execute the existing asynchronous processing method.

The above synchronization algorithm queue is a storage space allocated in the memory for the algorithm data that needs to be synchronized.

In 304, the algorithm cache module detects that algorithm data exists in the synchronization algorithm queue and an image frame exists in the image frame sequence, and then acquires a timestamp corresponding to the image frame.

The existence of algorithm data in the synchronization algorithm queue indicates that the image frame corresponding to the time stamp needs to be synchronized with the algorithm data in time.

In the present disclosure, the detection of the sequence of image frames may be invoked by the management module, such as the camera update callback interface, to trigger the execution of the algorithm caching module. Among them, the camera update callback interface can be used such as:

void algo_handle_update()

When the algorithm cache module obtains algorithm data from the synchronous algorithm queue, it can be implemented by calling the obtain algorithm interface. Among them, the acquisition algorithm interface can be used such as:

long long get_algo_handle(int algo_type, long long pts)

Among them, algo_type represents the algorithm capability type, and pts represents the timestamp.

In 305, the algorithm cache module transmits the handle encapsulated with the timestamp, the algorithm capability type and the algorithm data to the management module through the LUA virtual machine.

In this step, as an achievable way, the algorithm cache module can set the algorithm capability type and time stamp in the memory storing the algorithm data, and put the handle identifying the memory resource into the handle queue corresponding to the algorithm capability type, to be passed to the management module through the handle queue.

Wherein, when creating resources (for example, each algorithm data in the synchronization algorithm queue of the present disclosure), the system allocates memory for them, and returns an identification number identifying these resources, that is, a handle. A handle refers to the unique index of a core object in a process. In the present disclosure, in addition to storing algorithm data, an algorithm capability type and a timestamp are further set in the memory. It can be understood that the algorithm data, algorithm capability type and time stamp are stored in the memory as "a piece of data", and the handle pointing to the "a piece of data" is equivalent to encapsulating the time stamp, algorithm capability type and algorithm data.

The algorithm cache module can implement by calling the set algorithm handle interface when placing the handle encapsulating the timestamp, the algorithm capability type and the algorithm data into the algorithm queue corresponding to the algorithm capability type. Among them, the setting algorithm handle interface can be used such as:

void set_algo_handle(long long algo_handle)

In addition, in this embodiment, the handle queues may be set separately for algorithm capability types, or may be set uniformly.

Since the algorithm data stored in the synchronization algorithm queue by the algorithm module can be deleted by the algorithm module. In order to prevent the algorithm module from deleting the algorithm data due to some reasons and causing the algorithm data to be missing when the subsequent processing module performs the second processing, as another achievable method, the algorithm cache module can copy the algorithm data, and store the copied algorithm data in the location where the copied algorithm data is stored. The algorithm capability type and time stamp are set in the memory, and the handle identifying the memory resource is put into the handle queue corresponding to the algorithm capability type, so as to be passed to the management module through the handle queue.

In the embodiment of the present disclosure, the LUA virtual machine only performs the transparent transmission process of the handle.

In the present disclosure, the client can add a new handle queue by calling the interface of the new algorithm type after the algorithm module is successfully initialized. The new algorithm type interface can be used such as:

bool add_algo_type(std::vector<int>&type_list,int sync_mode)

Among them, the type_list parameter indicates the list of new algorithm capability types that need to be added; sync_mode indicates whether the added new algorithm capability type needs to be synchronized. The return value of calling this interface indicates whether there is enough memory to create the corresponding cache space.

For handle queues, they can be deleted to free memory resources when appropriate. Among them, the deletion of the handle queue can be adopted but not limited to the following two ways:

The first way: if all the second processing corresponding to the algorithm capability type is completed, the handle queue corresponding to the algorithm capability type is deleted. For example, after the execution of the current image processing task is completed, the handle queue corresponding to the algorithm capability type under the image processing task may be deleted. The removal algorithm type interface implementation can be invoked by the client, where the removal algorithm type interface can be implemented using methods such as:

void remove_algo_type(std::vector<int>&type_list)

Among them, type_list represents the list of algorithm capability types to be deleted.

The second way: if all the second processing corresponding to all the algorithm capability types is completed, or it is detected that the application exits, the handle queue corresponding to all the algorithm capability types is deleted. For example, when the application exits, the client of the application calls the flushing algorithm type interface, where the flushing algorithm type interface can be used such as:

void clear_algo_cache()

In addition, it should be noted that, in some cases, the processing module may use multiple algorithm data to perform the second processing. For example, beauty processing may involve face recognition results, region segmentation results, etc. In this case, the algorithm cache module needs to pass the handles of multiple algorithm results corresponding to the same timestamp to the management module.

For algorithm data without synchronization requirements, such as algorithm data in the asynchronous algorithm queue, the earliest algorithm data can be passed to the management module.

In step 306, the management module determines the corresponding processing module according to the algorithm capability type, and transmits the timestamp and the algorithm data to the processing module.

There may be multiple processing modules in an engine with image processing functions, and different processing modules have different functions for performing different second processing. For example, the engine may contain filter processing modules, rendering modules, color fill modules, and so on.

Since the second processing performed by the processing module corresponds to the algorithm capability, in the management module, the correspondence relationship between the type of the algorithm capability and the processing module is pre-stored. The corresponding processing module, that is, which processing module performs the second processing of the corresponding image frame, can be determined according to the algorithm capability type encapsulated by the handle. Then the timestamp and algorithm data are obtained from the memory space corresponding to the handle and passed to the processing module.

In step 307, the processing module obtains the image frame corresponding to the timestamp from the data source module, and performs second processing on the image frame by using the algorithm data.

After the processing module receives the time stamp and the algorithm data sent by the management module, it can know which image frame needs to be synchronized with the algorithm data according to the time stamp. The image frame is then subjected to a second processing using the algorithm data.

In step 308, the processing module provides the second processed image frame to the display device.

For example, after the processing module uses algorithm data to filter the image frame, the filter effect can be displayed on the screen. For another example, after the processing module uses the algorithm data to perform 3D rendering, the 3D rendering effect can be displayed on the screen.

The above is the processing process for one frame of image frame, and the subsequent image frames with synchronization requirements can be processed frame by frame according to the above process.

It can be seen from the above embodiments that the above method provided by the present disclosure can have the following advantages:

1) It can ensure the synchronization of algorithm data and data to be processed, and improve the effect of data processing. For example, algorithm data generated for an image frame is applied to the processing of the image frame without bias.

2) The present disclosure has no improvement or impact on the client, can adapt to various clients, and avoids the impact of differences due to different client platforms (eg, IOS or Android), and provides a unified set of caching and synchronization Strategy.

The above is a detailed description of the method provided by the present disclosure, and the device provided by the present disclosure is described in detail below with reference to the embodiments.

FIG. 4 is a structural diagram of an apparatus provided in an embodiment of the present disclosure. The apparatus may be a functional unit, a plug-in, or a software development kit located in an engine, which is not particularly limited in this embodiment of the present disclosure. As shown in FIG. 4 , the apparatus 400 may include: an algorithm caching module 401 , a management module 402 , and one or more processing modules 403 , and may further include a queue maintenance module 404 . The main functions of each component module are as follows:

The algorithm cache module 401 is configured to obtain algorithm data obtained by the algorithm module performing the first processing on the data to be processed, and transmit the time stamp encapsulating the data to be processed, the algorithm capability type and the handle of the algorithm data to the management module 402 .

The management module 402 is configured to determine the corresponding processing module 403 according to the algorithm capability type, and transmit the timestamp and the algorithm data to the determined processing module 403 .

The processing module 403 is configured to determine the corresponding data to be processed according to the time stamp, and perform second processing on the data to be processed by using the algorithm data.

As one of the achievable ways, the algorithm cache module 401 can set the algorithm capability type and time stamp in the memory storing the algorithm data, and put the handle identifying the memory resource into the handle queue corresponding to the algorithm capability type, so as to pass the handle queue Passed to the management module 402 .

As another achievable way, the algorithm cache module 401 can copy the algorithm data, set the algorithm capability type and time stamp in the memory for storing the copied algorithm data, and put the handle identifying the memory resource into the corresponding algorithm capability type The handle queue is passed to the management module 402 through the handle queue.

Further, the algorithm cache module 401 can detect the time stamp of the data to be processed from the data source module before passing the time stamp encapsulating the data to be processed, the algorithm capability type and the handle of the algorithm data to the management module, and synchronize the algorithm. Algorithm data corresponding to this timestamp exists in the queue.

The data source module, after acquiring the pending data of a new frame from the data acquisition module, will send the pending data to the cache queue for caching, and the algorithm cache module 401 detects the timestamp of the pending data from the cache queue. . In addition, the above-mentioned synchronization algorithm queue is a queue into which the algorithm module sends the algorithm data that needs to be synchronized. For algorithm data that does not need to be synchronized, it can be sent to the asynchronous algorithm queue.

The processing module 403 acquires the data to be processed corresponding to the timestamp from the data source module.

As an achievable manner, if all the second processing corresponding to the algorithm capability type is completed, the queue maintenance module 404 deletes the handle queue corresponding to the algorithm capability type.

As another achievable manner, if all second processes corresponding to all algorithm capability types are completed, or if exiting the application is detected, the queue maintenance module 404 deletes the handle queues corresponding to all algorithm capability types.

As some typical application scenarios, the above-mentioned first processing may include but is not limited to: concurrent mapping and positioning, face recognition, region segmentation, gesture recognition, object detection or gesture detection; the second processing may include but not limited to: image enhancement processing, image effects processing or 3D rendering processing.

Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for related parts.

In the technical solution of the present disclosure, the acquisition, storage and application of the user's personal information involved are all in compliance with the provisions of relevant laws and regulations, and do not violate public order and good customs.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

As shown in FIG. 5 , it is a block diagram of an electronic device according to a data processing method according to an embodiment of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 5 , the device 500 includes a computing unit 501 that can be executed according to a computer program stored in a read only memory (ROM) 502 or loaded from a storage unit 508 into a random access memory (RAM) 503 Various appropriate actions and handling. In the RAM 503, various programs and data necessary for the operation of the device 500 can also be stored. The computing unit 501 , the ROM 502 , and the RAM 503 are connected to each other through a bus 504 . An input/output (I/O) interface 505 is also connected to bus 504 .

Various components in the device 500 are connected to the I/O interface 505, including: an input unit 506, such as a keyboard, mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc. ; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 509 allows the device 500 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

Computing unit 501 may be various general-purpose and/or special-purpose processing components with processing and computing capabilities. Some examples of computing units 501 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, a data processing method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 508 .

In some embodiments, part or all of the computer program may be loaded and/or installed on device 500 via ROM 802 and/or communication unit 509 . When a computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the data processing method by any other suitable means (eg, by means of firmware).

Various implementations of the systems and techniques described herein may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip System (SOC), Load Programmable Logic Device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, causes the execution of the flowcharts and/or block diagrams The function/operation is implemented. The program code may execute entirely on the machine, partly on the machine, partly on the machine and partly on a remote machine as a stand-alone software package or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or a cloud host. It is a host product in the cloud computing service system to solve the management difficulties in traditional physical host and virtual private server (VPs, VIirtual Private Server) services. Large, weak business expansion defects. The server can also be a server of a distributed system, or a server combined with a blockchain.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present application can be executed in parallel, sequentially or in different orders, and as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, no limitation is imposed herein.

The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure should be included within the protection scope of the present disclosure.

Claims (12)

1. A method of data processing, comprising:

the algorithm cache module acquires algorithm data obtained by the algorithm module performing first processing on data to be processed, transmits a timestamp encapsulating the data to be processed, an algorithm capability type and a handle encapsulating the algorithm data to the management module,

wherein, the transmitting the timestamp encapsulating the data to be processed, the algorithm capability type and the handle of the algorithm data to a management module comprises: setting an algorithm capability type and a timestamp in a memory for storing the algorithm data, and putting a handle for identifying the memory resource into a handle queue corresponding to the algorithm capability type so as to transmit the handle queue to a management module;

the management module determines a corresponding processing module according to the algorithm capability type, and transmits the timestamp and the algorithm data to the processing module;

and the processing module determines corresponding data to be processed according to the time stamp and performs second processing on the data to be processed by utilizing the algorithm data.

2. The method of claim 1, wherein communicating the timestamp encapsulating the data to be processed, the algorithm capability type, and the handle of the algorithm data to a management module further comprises:

copying the algorithm data, setting an algorithm capability type and a timestamp in a memory for storing the copied algorithm data, and putting a handle for identifying the memory resource into a handle queue corresponding to the algorithm capability type so as to transmit the handle queue to a management module.

3. The method of claim 1, prior to said passing a timestamp encapsulating the data to be processed, an algorithm capability type, and a handle to the algorithm data to a management module, further comprising: the algorithm cache module detects a time stamp of the data to be processed from the data source module, and algorithm data corresponding to the time stamp exist in the synchronous algorithm queue;

the processing module determines corresponding data to be processed according to the timestamp, and the determining includes: and the processing module acquires the data to be processed corresponding to the time stamp from the data source module.

4. The method of claim 1 or 2, further comprising:

if all second processing corresponding to the algorithm capability type is completed, deleting the handle queue corresponding to the algorithm capability type; or,

and if all second processing corresponding to all algorithm capability types is finished or quit of application is detected, deleting handle queues corresponding to all algorithm capability types.

5. The method of any of claims 1 to 3, wherein the first processing comprises: drawing and positioning are carried out concurrently, face recognition, region segmentation, gesture recognition, object detection or gesture detection is carried out;

the second processing includes: image enhancement processing, image special effect processing, or 3D rendering processing.

6. A data processing device comprises an algorithm cache module, a management module and more than one processing module;

the algorithm cache module is used for acquiring algorithm data obtained by the algorithm module performing first processing on the data to be processed, transmitting a timestamp encapsulating the data to be processed, an algorithm capability type and a handle encapsulating the algorithm data to the management module,

the algorithm cache module is specifically configured to set an algorithm capability type and a timestamp in a memory for storing the algorithm data, and place a handle for identifying the memory resource in a handle queue corresponding to the algorithm capability type, so as to transmit the handle queue to the management module;

the management module is used for determining a corresponding processing module according to the algorithm capability type and transmitting the timestamp and the algorithm data to the determined processing module;

and the processing module is used for determining corresponding data to be processed according to the time stamp and performing second processing on the data to be processed by utilizing the algorithm data.

7. The apparatus according to claim 6, wherein the algorithm cache module is further configured to copy the algorithm data, set an algorithm capability type and a timestamp in a memory storing the copied algorithm data, and place a handle identifying the memory resource in a handle queue corresponding to the algorithm capability type, so as to transmit the handle queue to the management module.

8. The apparatus according to claim 6, wherein the algorithm cache module, before the time stamp encapsulating the data to be processed, the algorithm capability type and the handle of the algorithm data are transferred to the management module, is further configured to detect the time stamp of the data to be processed from the data source module, and the algorithm data corresponding to the time stamp exists in the synchronization algorithm queue;

the processing module is specifically configured to obtain the to-be-processed data corresponding to the timestamp from the data source module.

9. The apparatus of claim 6 or 7, further comprising:

the queue maintenance module is used for deleting the handle queue corresponding to the algorithm capability type if all second processing corresponding to the algorithm capability type is finished; or,

and if all second processing corresponding to all algorithm capability types is finished or quitting the application is detected, deleting the handle queues corresponding to all algorithm capability types.

10. The apparatus of any of claims 6 to 8, wherein the first process comprises: drawing and positioning are carried out concurrently, face recognition, region segmentation, gesture recognition, object detection or gesture detection is carried out;

the second processing includes: image enhancement processing, image special effect processing, or 3D rendering processing.

11. A data processing electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-5.

Images