CN104881320A - Scalable text generation method - Google Patents

Abstract

The invention provides a scalable text generation method, which can render text files faster and utilize CPU resources of a server more efficiently. The method includes: decomposing the entire template file into multiple sub-template files; generating a rendering task list, the rendering task list includes multiple rendering tasks, and the multiple rendering tasks are used to associate the multiple rendering tasks corresponding to the multiple sub-template files, and implement the logic of preparing data before rendering; perform multi-thread rendering on the multiple rendering tasks; and splicing the rendering results of the multiple rendering tasks to generate text. By adopting the technical solution of the invention, the development and maintenance cost of the template file is reduced, the data preparation is more efficient, and the CPU resource utilization of the server is more reasonable.

Description

A Scalable Approach to Generating Text

technical field

The invention relates to the technical field of computer networks, in particular to a scalable text generation method.

Background technique

Text files are an important class of computer files. Due to their simple structure, text files are widely used to record information. In order to improve the development efficiency and make the code reusable, a template engine (template engine) is introduced when generating text. After using the template engine, the function of logic separation can be achieved, so that the program application logic and page display logic are separated, allowing program developers Focus on the control of data or the achievement of functions; while visual designers can focus on page layout to make the page look more professional.

Take the Java language to generate a common text—HTML page as an example. When using the Java language to generate HTML pages, under normal circumstances, developers will not directly write the entire HTML page, but divide an HTML page into many HTML templates according to certain logic, and then use the Java-based template engine Eventually the entire HTML page is generated.

Commonly used Java-based template engines include Velocity, FreeMarker, etc. For example, Velocity, which allows anyone to use a simple template language (template language) to refer to objects defined by Java code, and render Java objects and templates themselves, and finally generate an HTML fragment. When Velocity is applied to Web page development, page designers can only focus on the display effect of the page, while Java program developers focus on business logic coding. Velocity separates Java code from web pages, which also facilitates long-term maintenance of web sites.

When it is necessary to render an HTML page with Velocity, the prior art usually includes the following steps:

1. Replace the variables in the entire HTML page with Java object references according to Velocity syntax to generate Velocity template files;

2. The data required by the Velocity template file is assembled sequentially through the Java background logic single thread;

3. Use the Velocity template file and the data assembled by the Java background logic as the input of the rendering method in Velocity, execute the rendering (render) method in Velocity, and finally obtain the output of the rendering method to generate a complete HTML page.

It can be seen that the existing Java-based method for generating text has the following disadvantages:

1. If the template file has a lot of content, the entire template file will be very long, which will increase the development and maintenance costs of developers.

2. The process of preparing data in the Java background is single-threaded. If the template file requires a lot of data, the logic of assembling data in the Java background will take a long time to prepare all the data, and then call the rendering method, so the page generation time is very long.

3. If the CPU utilization rate of the server is low, the process of rendering text must still follow the process of preparing data and then rendering in sequence, and the CPU resources of the server cannot be used more efficiently.

Contents of the invention

In view of this, the present invention provides a scalable text generation method. The specific solution is: decompose a large template file into multiple sub-template files, write the corresponding template file and data associated with the rendering task, and then send the rendering task to Multi-threaded rendering is performed for the thread pool, and after all rendering tasks are completed, the rendering results are spliced to generate a complete text. This method can render text files faster, can use as much CPU resources of the server as possible when the CPU utilization of the server is low, speeds up the rendering of text, and automatically protects the CPU resources of the server when the utilization of the server reaches a certain level, Extend the working time of the server when receiving a large number of rendering requests.

To achieve the above purpose, the present invention provides a scalable text generation method.

A scalable text generation method of the present invention includes: decomposing the entire template file into multiple sub-template files; generating a rendering task list, the rendering task list includes multiple rendering tasks, and the multiple rendering tasks are used for Associating the plurality of sub-template files corresponding to the plurality of rendering tasks, and implementing the logic of preparing data before rendering; performing multi-threaded rendering of the plurality of rendering tasks; and splicing the rendering results of the plurality of rendering tasks to generate text.

Optionally, decomposing the entire template file into multiple sub-template files includes: replacing variables in the text with Java object references according to template engine syntax to generate the entire template file; and decomposing into multiple sub-template files.

Optionally, decomposing the entire template file into multiple sub-template files includes: decomposing the variables in the text into multiple sub-template files; and replacing the variables of each sub-template file with Java object references according to the syntax of the template engine to generate sub-template files.

Optionally, decomposing the entire template file into multiple sub-template files includes: decomposing the entire template file into multiple sub-template files according to the page structure and estimated time required for data preparation.

Optionally, the multiple rendering tasks include location information corresponding to the multiple rendering tasks.

Optionally, before performing multi-thread rendering on the rendering task list, it is determined whether the thread pool is in a full load state.

Optionally, when the thread pool is not fully loaded, the thread currently submitting the rendering task submits the current rendering task to the thread pool for multi-thread rendering; and when the thread pool is fully loaded, the thread currently submitting the rendering task no longer requests Use multithreading to perform rendering tasks, but directly use the current thread for rendering.

Optionally, when a new rendering request is received, the thread pool starts a new thread to execute the rendering task.

Optionally, splicing the rendering results of the multiple rendering tasks to generate the text includes: sequentially splicing the rendering results of the multiple rendering tasks according to the order in which the multiple rendering tasks are submitted to generate the text.

According to the technical solution of the present invention, the entire template file is decomposed into multiple sub-template files, so that the content of each sub-template file is relatively small, thereby reducing development and maintenance costs; each sub-template file corresponds to a rendering task, and can be multi-threaded in parallel Obtaining data makes the data preparation of each module in the page more efficient, and the logic of organizing data is clearer, so that the development method is more efficient; multi-threaded rendering is performed on multiple rendering tasks, and the rendering method is adjusted according to the workload of the thread pool , can protect the server CPU resources while using the server CPU resources more efficiently, so as not to generate too many rendering threads in high-concurrency scenarios, thus intensifying the competition for CPU resources. It is a dynamically scalable The way the text is generated.

Description of drawings

The accompanying drawings are used to better understand the present invention, and do not constitute improper limitations to the present invention. in:

Fig. 1 is a schematic diagram of main steps of a scalable method for generating text according to an embodiment of the present invention;

Fig. 2 is a flow chart of detailed steps of a scalable method for generating text according to an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 is a schematic diagram of main steps of a scalable method for generating text according to an embodiment of the present invention. As shown in FIG. 1 , a scalable text generation method according to an embodiment of the present invention mainly includes the following steps S11 to S14.

Step S11: Decompose the entire template file into multiple sub-template files. This step can be performed through the following two options:

Solution 1, replace the variables in the text with references to Java objects according to the syntax of the template engine to generate the entire template file; and decompose it into multiple sub-template files.

The second solution is to decompose the variables in the text into multiple groups; and replace the variables of each group with Java object references according to the syntax of the template engine to generate sub-template files.

In practical applications, you can flexibly decide which solution to choose according to the page structure of the text to be rendered, the difficulty of program writing and maintenance, etc.

Decomposing the entire template file into multiple sub-template files is, in principle, decomposing the entire template file into multiple sub-template files according to the page structure and the estimated time required for data preparation. Usually, multiple time-consuming interfaces can be decomposed into different rendering tasks to ensure the time-consuming balance of multiple tasks.

Step S12: Generate a rendering task list, the rendering task list includes a plurality of rendering tasks, the plurality of rendering tasks are used to associate the plurality of sub-template files corresponding to the plurality of rendering tasks, and at the same time realize the preparation of data before rendering logic.

Each sub-template file generated according to step S11 is associated with a rendering task, and the rendering tasks associated with all sub-template files are sequentially submitted to the thread pool for rendering. At the same time, the rendering task can also implement the logic of preparing data before rendering, that is, a single thread in the Java background sequentially assembles the data required by the template file. Since the rendering task list contains multiple rendering tasks, the data required by multiple sub-template files can be assembled simultaneously in multiple threads, thereby greatly saving the time for data preparation.

The plurality of rendering tasks include location information corresponding to the plurality of rendering tasks. When the entire template file is decomposed into multiple sub-template files, it will be decomposed according to the page structure and other conditions. At this time, each sub-template file should contain its position information in the entire template file, so that the results can be spliced after rendering. corresponding text.

Step S13: performing multi-thread rendering on the plurality of rendering tasks. After step S12 is completed, the thread that submitted the rendering task submits the plurality of rendering tasks to the thread pool for multi-thread simultaneous rendering.

Before performing multi-thread rendering on the rendering task list, it is first determined whether the thread pool is in a full load state.

When the thread pool is not fully loaded, the thread that currently submits the rendering task submits the current rendering task to the thread pool for multi-thread rendering. When receiving a new rendering request, the thread pool starts a new thread to execute the rendering task.

When the thread pool is full, the thread that currently submits the rendering task no longer requests to execute the rendering task in a multi-threaded manner, but directly uses the current thread for rendering.

When the rendering task is completed, save the rendering result in the object specified by each thread.

Step S14: Merge the rendering results of the multiple rendering tasks to generate text. After all the rendering tasks in the rendering task list are completed, the rendering results of the multiple rendering tasks are sequentially spliced in the order in which the multiple rendering tasks are submitted to generate the entire text page.

It can be seen from steps S11 to S14 that the entire template file is decomposed into multiple sub-template files, so the content of each sub-template file is relatively small, thereby reducing development and maintenance costs; each sub-template file corresponds to a rendering task, which can be multi- Threads acquire data in parallel, which makes the data preparation of each module in the page more efficient, and the logic of organizing data is clearer, so that the development method is more efficient; multiple rendering tasks are multi-threaded, and the rendering is adjusted according to the workload of the thread pool The method can protect the server CPU resources while using the server CPU resources more efficiently, so as not to generate too many rendering threads in high concurrency scenarios, thereby intensifying the competition for CPU resources. It is a dynamically scalable way of generating text.

Fig. 2 is a flow chart of detailed steps of a scalable method for generating text according to an embodiment of the present invention.

As shown in Figure 2, it is illustrated by using the Java-based template engine Velocity to render a text page as an example. In order to decompose the entire template file into multiple sub-template files (step S21), the following two methods can be used:

1. First replace the variables in the entire text page with references to Java objects according to the Velocity syntax to generate a Velocity template file, and then decompose the Velocity template file into multiple parts according to the page structure of the text and the estimated time required for data preparation. A child Velocity template file.

2. First decompose the variables in the entire text page into multiple groups according to the page structure of the text and the estimated time required for data preparation, and then replace the variables in each group with references to Java objects according to the Velocity syntax. Velocity template files.

After step S21 is completed, according to each generated sub-Velocity template file, write a corresponding rendering task to associate the rendering task with its corresponding sub-Velocity template file, and at the same time realize the logic of preparing data before rendering. There are multiple rendering tasks, which are submitted to the rendering factory in the form of a rendering task list for rendering (step S22). The rendering factory is used to render rendering tasks and stitch the rendering results to generate text. The rendering factory contains a thread pool. Before receiving a rendering task, the rendering factory first performs an initialization operation. For example, the size of the thread pool, the minimum number of threads, and the size of the work queue can be set for the rendering factory. When the rendering factory receives the rendering task, it first traverses the received rendering task list, then initializes the rendering task, and passes information such as the total number of rendering tasks to the rendering task, and then injects public tools into it according to the characteristics of the rendering task, etc. Variables that can be shared, and then render the above-mentioned multiple sub-Velocity template files in a multi-threaded manner (step S23).

Before performing multi-thread rendering on the rendering task list, it is necessary to determine whether the thread pool corresponding to the rendering factory is in a full load state (step S24). When the thread pool is fully loaded, the thread currently submitting the rendering task no longer requests to execute the rendering task in a multi-threaded manner, but directly uses the current thread for rendering (step S25). When the thread pool is not fully loaded, the thread that currently submits the rendering task submits the current rendering task to the thread pool for multi-thread rendering. At this time, the thread pool of the rendering factory opens a new thread to perform the rendering task (step S26). After each thread rendering task is completed, save the rendering result in the designated object of each thread (step S27), then check whether the rendering task list has been executed (step S28), if not, turn to step S23, if executed After completion, the rendering results of the multiple rendering tasks are sequentially spliced according to the order in which the rendering tasks are submitted (step S29 ), so as to generate a complete text page.

It can be seen from the above preferred embodiments that the scalable text generation method proposed by the present invention can adjust the rendering mode according to the workload of the thread pool when rendering with multiple threads. When the number of threads occupied by the thread pool is small , you can use multi-thread to render text, change to single-thread rendering when the thread pool has a large number of occupied threads, and return to multi-thread rendering after the number of occupied threads decreases, which speeds up the generation of text pages and saves The time the text page was generated.

The present invention not only provides a method of efficiently utilizing the CPU by detecting the state of the thread pool before submitting the rendering task, but also proposes a method of decomposing the Velocity template data into multiple rendering tasks, and the rendering is performed by multiple threads in parallel. The method of obtaining data makes the data preparation of each module in the page more efficient, and at the same time, the logic of organizing data is also clearer, which is a more efficient development method.

The basic principles of the present invention have been described above in conjunction with specific embodiments. In the device and method of the present invention, obviously, each component or each step can be decomposed and/or reassembled. These decompositions and/or recombinations should be considered equivalents of the present invention. Also, the steps for performing the above series of processes may naturally be performed in chronological order in the order described, but need not necessarily be performed in chronological order. Certain steps may be performed in parallel or independently of each other.

The above specific implementation methods do not constitute a limitation to the protection scope of the present invention. It should be apparent to those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (9)

1. A scalable method for generating text, comprising: Decompose the entire template file into multiple sub-template files; Generate a rendering task list, the rendering task list includes multiple rendering tasks, the multiple rendering tasks are used to associate the multiple sub-template files corresponding to the multiple rendering tasks, and implement the logic of preparing data before rendering; performing multi-thread rendering on the plurality of rendering tasks; and The rendering results of the multiple rendering tasks are stitched together to generate text. 2. The method according to claim 1, wherein decomposing the entire template file into a plurality of sub-template files comprises: Replace the variables in the text with references to Java objects according to the syntax of the template engine to generate the entire template file; and Break down into multiple sub-template files. 3. The method according to claim 1, wherein decomposing the entire template file into a plurality of sub-template files comprises: break down the variables in the text into groups; and Replace the variables of each group with references to Java objects according to the syntax of the template engine to generate sub-template files. 4. The method according to claim 1, wherein decomposing the whole template file into a plurality of sub-template files comprises: decomposing the whole template file into Multiple child template files. 5. The method according to claim 1, wherein the plurality of rendering tasks include location information corresponding to the plurality of rendering tasks. 6. The method of claim 1, further comprising: Before performing multi-thread rendering on the rendering task list, it is determined whether the thread pool is in a full load state. 7. The method of claim 6, further comprising: When the thread pool is not fully loaded, the thread that currently submits the rendering task submits the current rendering task to the thread pool for multi-thread rendering; and When the thread pool is full, the thread that currently submits the rendering task no longer requests to execute the rendering task in a multi-threaded manner, but directly uses the current thread for rendering. 8. The method according to claim 7, further comprising: when a new rendering request is received, the thread pool starts a new thread to execute the rendering task. 9. The method according to claim 1, wherein splicing the rendering results of the multiple rendering tasks to generate the text comprises: combining the rendering results of the multiple rendering tasks in the order in which the multiple rendering tasks were submitted The rendered results are sequentially concatenated to generate text.