TWI775170B - Method for cpu to execute artificial intelligent related processes - Google Patents

Abstract

A method for CPU to execute artificial intelligent related processes is disclosed and includes following steps of: when running TensorFlow on an electronic device, calls a corresponding AI module of TensorFlow according to the content of program code; determining and obtaining one or more sparse matrix used by the AI module while the AI module runs calculations; performing a matrix simplified procedure to the one or more sparse matrix; performing an instruction transforming procedure to an instruction set applied by the AI module; applying the transformed instruction set by the AI module to issue instructions to a weighted CPU of the electronic device; and, the weighted CPU, after receiving the instructions, equally distributing multiple processes indicated by the AI module accompanying with the instructions to be respectively run on multiple threads of the weighted CPU according to weight values of the multiple processes.

Description

Execution method when CPU is applied to artificial intelligence related programs

The present invention relates to a central processing unit, in particular to an execution method of the central processing unit when executing a program related to artificial intelligence.

In recent years, artificial intelligence technology has developed vigorously, and major companies have invested in the research and development of artificial intelligence-related products.

Generally speaking, software engineers often use TensorFlow developed by Google (Google) to develop artificial intelligence related programs. Specifically, TensorFlow is an open source software library applied to machine learning, and TensorFlow provides a variety of artificial intelligence related models, which are helpful for software engineers to directly use for artificial intelligence development.

However, in the prior art, the matrix used by the AI-related models to perform operations is generally not optimized, so these models use many computing parameters when performing operations. Because of this, to perform artificial intelligence-related operations, it usually requires the use of higher-end hardware. For example, a high-level central processing unit (Central Processing Unit, CPU) needs to be used, or a specific image processing unit (Graphics Processing Unit, GPU) needs to be used. However, the use of the above-mentioned hardware greatly increases the development and use costs of artificial intelligence, and also causes the problem that the heat dissipation of the hardware is not easy.

In addition, in view of the powerful computing capabilities of the current CPU and GPU, the CPU does not and does not need to optimize the ordering of various programs in execution. Specifically, the CPU usually only executes the received multiple instructions in sequence, so it often happens that one thread (Thread) of the CPU cannot complete a program and needs to be handed over to the next thread to continue the calculation. When the above phenomenon occurs, the CPU increases the time cost of executing SWAP, resulting in a slower overall speed, and a higher-end CPU needs to be replaced, or an additional GPU must be set up. In this case, the possibility of developing and using artificial intelligence by using low-end hardware, such as the Industrial Personal Computer (IPC) with the X86 architecture, is undoubtedly blocked, which is quite a pity.

The main purpose of the present invention is to provide an execution method when the CPU is applied to artificial intelligence related programs, which simplifies the matrix used by the artificial intelligence model in the operation, and optimizes the execution time of the CPU, thereby enabling low-level execution time. Hardware devices can also be used to develop and use artificial intelligence.

In order to achieve the above-mentioned purpose, the execution method of the present invention mainly includes the following steps: when executing TensorFlow on an electronic device, calling one or more corresponding artificial intelligence models according to the content of the program; judging and extracting these artificial intelligence models during operation One or more sparse matrices used; perform a matrix simplification procedure on these sparse matrices; perform instruction conversion on an instruction set adopted by these artificial intelligence models; these artificial intelligence models pass the converted instruction set to an The weighted central processing unit issues the instruction; and after receiving the instruction, the weighted central processing unit distributes the multiple threads evenly according to the weighted value to execute the multiple programs indicated by the artificial intelligence model.

Compared with the prior art, the present invention first compresses the sparse matrix used by the artificial intelligence model in the calculation through the matrix simplification procedure, so as to reduce the calculation parameters in the calculation. Furthermore, the present invention first After setting the weight values of the multiple programs indicated by the artificial intelligence model through the converted instructions, and then issuing instructions to the weighted value central processing unit, so that the weighted value central processing unit can evenly distribute these programs to multiple executions. Threads are executed to optimize CPU execution time. In this way, the main purpose of enabling low-end hardware devices to be used to develop and use artificial intelligence is achieved.

1: TensorFlow

2: Prediction procedure

3: Matrix simplification procedure

31: Sparse Matrix

32: Merge Matrix

4: Weighted value central processing unit

41: Weighted CPU time

42: Weighted Program Time

43: Execution frequency

44: CPU time

45: Cycle time

46: Cycle count

47: Pause Cycle

5: Thread

S10~S24: Execution steps

FIG. 1 is a system architecture diagram of a first specific embodiment of the present invention.

FIG. 2 is a flow chart of the execution method of the first specific embodiment of the present invention.

FIG. 3 is a simplified schematic diagram of a matrix according to the first specific embodiment of the present invention.

FIG. 4 is a schematic diagram of a weighted value central processing unit according to the first specific embodiment of the present invention.

Hereinafter, a preferred embodiment of the present invention will be described in detail in conjunction with the drawings.

The present invention discloses an execution method when a CPU is applied to an artificial intelligence related program (hereinafter referred to as an execution method in the specification), and the execution method can be mainly applied to a central processing unit (Weighted Central Processing Unit, Weighted Central Processing Unit) using a weighted value. CPU) electronic equipment. More specifically, the electronic equipment referred to in the present invention mainly refers to the electronic equipment that adopts a lower-level CPU and does not regard a graphics processing unit (Graphics Processing Unit, GPU) as a necessary equipment, such as a personal computer ( Personal Computer, PC), industrial computer (Industrial PC, IPC), industrial server, etc., but not limited to this. Through the execution method of the present invention, the user can be assisted in developing and using artificial intelligence with relatively low-end hardware devices.

First, please refer to FIG. 1 , which is a system architecture diagram of a first specific embodiment of the present invention. As mentioned above, the implementation method of the present invention is mainly used to assist users to use low-end hardware devices to develop and use artificial intelligence that generally requires a lot of computing resources. In the development of TensorFlow, TensorFlow developed by Google is often used to perform various operations.

TensorFlow 1 is an open source software library applied to machine learning, and TensorFlow provides a variety of models that are mainly used to develop artificial intelligence related programs (hereinafter referred to as artificial intelligence models in the specification). When writing programs, software engineers can directly refer to multiple artificial intelligence models provided by TensorFlow to analyze and calculate data directly and quickly, and then realize artificial intelligence-related functions such as machine learning and prediction. program.

As shown in Figure 1, when a user writes a program on an electronic device, he can execute the above-mentioned TensorFlow 1, thereby calling one or more artificial intelligence models of TensorFlow 1, and using these artificial intelligence models to realize predictions in artificial intelligence Procedure (Inference Process) 2. In one embodiment, the user can mainly execute TensorFlow on the electronic device through an instruction such as import TensorFlow in the code, so as to call one or more artificial intelligence models in TensorFlow. In another embodiment, the artificial intelligence model may be, for example, an imageNet model, an inception model and other models related to image processing, but not limited thereto.

It is well known to those skilled in the art that the vast majority of artificial intelligence models (such as the above-mentioned imageNet model and inception model) use a matrix to perform data operations, and use an instruction set to instruct the CPU Execute (run) the program related to the operation. In view of the continuous improvement of the current CPU specifications, and some high-end hardware devices are also equipped with GPUs specially used to process images, the artificial intelligence models are generally not processed when performing artificial intelligence operations. Line optimization, and does not specifically allocate and order the program to be executed by the CPU. Therefore, it is difficult to implement artificial intelligence operations on low-end hardware devices (eg, industrial computers that use low-end CPUs and do not have GPUs).

One of the main technical features of the present invention is that when an application program executes TensorFlow 1 and calls the above-mentioned artificial intelligence models to perform operations in order to realize the prediction program 2, firstly, the matrices to be used by these artificial intelligence models in the operation are performed. Matrix simplification procedure 3 to reduce the number of computational parameters in the computation of these artificial intelligence models.

In addition to reducing the number of calculation parameters in the operation, the present invention can also query the instruction sets required by these artificial intelligence models in advance, and perform an instruction conversion program on the instructions in the instruction set. Thereby, after receiving the instruction issued by the artificial intelligence model, the weighted value central processing unit 4 of the present invention can sort the programs according to the weight value (w) of the multiple programs included in the operation, so as to These programs are evenly distributed to multiple threads 5 under them to be executed respectively, thereby realizing the prediction program 2 to be realized by the application program. In this way, the present invention can better reflect the average resource utilization of each program required by the artificial intelligence for the CPU, thereby optimizing the execution time of the CPU.

The execution method of the present invention combines the matrix simplification program 3 and the thread allocation technology of the weighted central processing unit 4, which greatly reduces the resources occupied by artificial intelligence operations, and makes general low-end hardware devices (such as X86 architecture industrial computer) can also perform training and recognition tasks under artificial intelligence. In this way, the user can effectively save the cost of developing and using artificial intelligence, and can improve the computing speed of the electronic device when implementing the prediction program 2 .

Please also refer to FIG. 2 , which is a flowchart of the execution method of the first specific embodiment of the present invention. The execution method of the present invention is mainly applied to the electronic equipment using the weighted value central processing unit 4 Above, in one embodiment, the execution method of the present invention is mainly used in low-end electronic equipment (eg, an industrial computer using the X86 architecture), and the electronic equipment adopts the weighted central processing unit 4 of the Intel architecture, but not be limited.

As shown in FIG. 2 , when the electronic device executes the code, it first executes TensorFlow 1 according to the content of the code (step S10 ), and calls at least one artificial intelligence model in TensorFlow according to the content of the code (step S12 ). In one embodiment, the code executes TensorFlow 1 through an instruction such as import TensorFlow, and calls artificial intelligence models such as imageNet model, inception model, etc., which are mainly used for image processing, but not limited thereto.

The imageNet model and the inception model are models that are often used in artificial intelligence related programs, and are commonly used by those skilled in the art, and will not be repeated here.

The execution method of the present invention mainly exists between TensorFlow and the weighted value central processing unit 4 in the form of a driver or middleware, so as to reduce the calculation parameters of the artificial intelligence model of TensorFlow during operation. , and the weighted central processing unit 4 can process the operation content on average through a plurality of threads 5 , thereby optimizing the execution time of the weighted central processing unit 4 .

As shown in FIG. 2 , the execution method of the present invention further determines and obtains one or more sparse matrices (Sparse Matrix) used in the operation of the artificial intelligence model (step S14 ), and executes the matrix operation on the one or more sparse matrices Simplify the procedure 3 (step S16 ), thereby reducing the number of calculation parameters of the one or more artificial intelligence models in the operation.

Please also refer to FIG. 3 , which is a simplified schematic diagram of a matrix according to the first specific embodiment of the present invention. As shown in Figure 3, the sparse matrix 31 refers to a matrix whose internal elements are mostly zero. Large sparse matrices 31 are often used when solving linear models, and therefore, models related to artificial intelligence often use sparse matrices 31 for operations.

In order to reduce the number of calculation parameters, the matrix reduction program 3 mainly stores only non-zero elements in the sparse matrix 31 to compress the sparse matrix 31 . Moreover, the matrix simplification program 3 further decomposes the compressed sparse matrix 31 into one or more consolidated matrices (Consolidate Matrix) 32 to complete the action of simplifying the matrix. In the present invention, when the called artificial intelligence model performs operations, it mainly uses the merge matrix 32 as shown in FIG. 3 to perform operations.

The compression of the above-mentioned sparse matrix 31 and the generation of the merge matrix 32 are well-known techniques in the technical field, and details are not described herein again. In addition, the above-mentioned method of decomposing the sparse matrix 31 into the merged matrix 32 is only one specific implementation example of the matrix simplification program 3 of the present invention. No longer.

Back to Figure 2. After step S16, the execution method of the present invention further queries and analyzes the instruction set used by the artificial intelligence model, and performs an instruction conversion procedure on the instructions in the instruction set to generate a converted instruction set (step S18). In one embodiment, the weighted central processing unit 4 is an Intel architecture central processing unit, and the instruction set is an Intel architecture instruction set, but not limited thereto.

As mentioned above, the prior art does not effectively sort and allocate the programs to be executed by the CPU when performing artificial intelligence operations, that is, the CPU simply executes multiple instructions in sequence according to the order in which the instructions are received , without optimal allocation of multiple threads under it for load balancing.

In the present invention, the instruction conversion program modifies the instruction set used by the artificial intelligence model, so that the artificial intelligence model can By first judging and setting the weight values of multiple programs to be executed, these programs can be sorted and assigned. In this way, when the weighted value central processing unit 4 receives the instructions issued by the artificial intelligence model, it can evenly distribute these programs to each of the threads 5 under it according to the weighted value, so that the weighted value can be effectively utilized. The limited resources of the central processing unit 4, thereby increasing the execution speed of these programs.

After step S18, the artificial intelligence model can issue instructions to the weighted value central processing unit 4 through the converted instruction set (step S20), and after receiving the instructions, the weighted value central processing unit 4 can use the artificial intelligence model. According to the weight value of the multiple programs indicated, the multiple programs are evenly distributed to multiple threads 5 under them to be executed respectively (step S22). Wherein, the multiple programs include operations on one or more merged matrices 32 after the matrix simplification program 3, but it is not limited thereto.

It is worth mentioning that the artificial intelligence model can perform an action of calling a resource library (call library) during operation, and then can execute (run) the instruction set of the CPU. The present invention modifies the instructions in the instruction set (preferably the instruction set of Intel architecture), so that the artificial intelligence model can first determine and set multiple programs to be executed before issuing instructions to the weighted central processing unit 4 The weight value of the weight value, and then these programs can be evenly distributed to a plurality of threads 5 under the weight value central processing unit 4 .

The action of calling the resource library is a common technical means in the technical field, and details are not described herein again.

Referring to FIG. 4 , it is a schematic diagram of a weighted value central processing unit according to the first specific embodiment of the present invention. As shown in FIG. 4 , a weighted central processing unit 4 includes complex weighted CPU time 41 based on its execution cycle during operation. In FIG. 4 , the first weighted CPU time to the nth weighted CPU time are used as examples. Among them, the weighted value of the central processing unit 4 under each weighted CPU time 41 can control multiple threads 5 under it (the first thread to the nth thread are taken as an example in FIG. 4 ) to operate at the same time, and the weighted CPU time 41 is the respective weighted program time 42 of all threads 5 Sum. Specifically, the weighted program time 42 refers to the time for a single thread 5 to execute a program.

More specifically, the weighted program time 42 is the product of the execution frequency (Program Frequency) of executing the assigned program for this thread 5 and the CPU time 44 occupied by the thread 5, and the CPU time 44 is the time of the thread 5. The product of Cycle time 45 and the sum of Cycle count 46 and Idle Cycles of this thread 5.

It can be seen from the above description that the weighted central processing unit 4 can use multiple threads 5 to execute multiple programs synchronously in each weighted CPU time 41. Therefore, if each thread 5 is used in each weighted program time All the programs that can be executed in 42 are effectively allocated, and the overall resource utilization of the weighted central processing unit 4 can be greatly improved.

Back to Figure 2. After step S22, the execution method of the present invention (ie, the driver or the intermediary software) determines whether the execution of the code is completed (step S24), and before the execution of the code is completed, steps S12 to S22 are repeatedly executed, So that the electronic device can implement the artificial intelligence related program (for example, the prediction program 2 ) through the weighted value central processing unit 4 according to the execution method of the present invention.

The execution method of the present invention combines the matrix simplification technology and the thread allocation technology of the weighted central processing unit, first reducing the number of calculation parameters in the operation, and then optimizing the multiple threads under the weighted central processing unit. Program management, thereby achieving the main purpose of enabling low-end hardware devices to be used to develop and use artificial intelligence. Compared with the prior art, the execution method of the present invention effectively reduces the development and use costs and threshold restrictions of artificial intelligence.

The above description is only a preferred specific example of the present invention, and therefore does not limit the scope of the patent of the present invention. Therefore, all equivalent changes made by using the content of the present invention are all included in the scope of the present invention. Bright.

1: TensorFlow

2: Prediction procedure

3: Matrix simplification procedure

4: Weighted value central processing unit

5: Thread

Claims (5)

An execution method when a CPU is applied to an artificial intelligence-related program, is applied to an electronic device having at least one weighted central processing unit (Weighted Central Processing Unit, Weighted CPU), and includes the following steps: a) executing on the electronic device TensorFlow; b) calling at least one artificial intelligence model in TensorFlow according to the content of the code; c) judging and extracting one or more sparse matrices (Sparse Matrix) used by the artificial intelligence model in the operation; d) for the one or more sparse matrices (Sparse Matrix); A sparse matrix executes a matrix simplification program; e) an instruction conversion program is performed on an instruction set adopted by the artificial intelligence model to generate a converted instruction set; f) the artificial intelligence model passes the converted instruction set to the The weighted value central processing unit issues instructions; and g) after the weighted value central processing unit receives the instruction, according to the weight values of the plurality of programs indicated by the artificial intelligence model, the plurality of programs are evenly distributed among the plurality of programs under it. Threads are executed separately, wherein a weighted CPU time of the weighted central processing unit is the sum of a weighted program time of the multiple threads, and the weighted program time of each thread is to execute the corresponding The product of a CPU time of a program and an execution frequency (Program Frequency), the CPU time is the sum of a cycle time (Cycle time), a cycle count (Cycle count) and an idle cycle (Idle cycles) of the thread product. The execution method when the CPU is applied to an artificial intelligence-related program according to claim 1, wherein the step a) is to execute TensorFlow in the program code by importing the TensorFlow instruction. The execution method when the CPU is applied to an artificial intelligence related program according to claim 1, wherein the artificial intelligence model at least includes an imageNet model and an inception model. The execution method when a CPU is applied to an artificial intelligence-related program according to claim 1, wherein the matrix reduction program comprises decomposing the one or more sparse matrices into one or more consolidated matrices (Consolidate Matrix), wherein the plurality of programs At least operations on the one or more merged matrices are included. The execution method when the CPU of claim 1 is applied to an artificial intelligence related program, wherein the weighted central processing unit is an Intel architecture central processing unit, and the instruction set used by the artificial intelligence model is an Intel architecture instruction set.

Images