Disclosure of Invention
The embodiment of the invention provides a file generation method and equipment, which are used for solving the problems that the development cycle of a task of newly adding a type of interface data is long and the test difficulty is high in the conventional task of obtaining various types of data through the operation on a database entity table.
In a first aspect, an embodiment of the present invention provides a file generation method, including:
determining a service to be processed, and determining a source of data to be processed according to the service to be processed;
determining a business rule corresponding to the interface file, and loading the business rule;
analyzing the service rule to determine a task processing node;
and if the task processing node is used for generating the interface file, generating the interface file according to the source of the data to be processed and the business rule.
In one possible design, the method further includes:
determining task execution time, and sending a task initiating instruction according to the task execution time, wherein the task initiating instruction carries a task identifier;
the determining the service to be processed and determining the source of the data to be processed according to the service to be processed includes:
determining a service to be processed according to the task identifier, determining a source of data to be processed according to the service to be processed, and feeding back corresponding information;
the method further comprises the following steps:
and sending reminding information to the first preset personnel according to the feedback information.
In one possible design, the method further includes:
and if the task processing node is used for uploading the file, uploading the interface file.
In one possible design, the method further includes:
and if the task processing node is a public service, providing the public service according to the source of the data to be processed and the business rule.
In one possible design, the method further includes:
checking whether each data file corresponding to the task identifier exists or not;
and sending the inspection result to a second preset person.
In a second aspect, an embodiment of the present invention provides a file generating apparatus, including:
the data analysis processing module is used for determining a service to be processed and determining a source of data to be processed according to the service to be processed;
the business rule configuration management module is used for determining the business rules corresponding to the interface files;
the business rule loading module is used for loading the business rules;
the task analysis module is used for analyzing the service rule and determining a task processing node;
and the interface file generation module is used for generating an interface file according to the source of the data to be processed and the business rule if the task processing node generates the interface file.
In a possible design, the above device further includes a task timing module and a communication service module;
the task timing module is used for determining task execution time and sending a task initiating instruction to the data analysis processing module according to the task execution time, wherein the task initiating instruction carries a task identifier;
the data analysis processing module is used for determining a service to be processed according to the task identifier, determining a source of data to be processed according to the service to be processed, and feeding corresponding information back to the task timing module;
and the communication service module is used for sending reminding information to a first preset person according to the information fed back by the data analysis processing module.
In a possible design, the apparatus further includes a file uploading module;
and the file uploading module is used for uploading the interface file if the task processing node is a file uploading module.
In one possible design, the apparatus further includes a common service module;
and the public service module is used for providing public service according to the source of the data to be processed and the business rule if the task processing node is the public service.
In a possible design, the apparatus further includes a data checking module;
the data checking module is used for checking whether each data file corresponding to the task identifier exists or not;
and the communication service module is also used for sending the checking result of the data checking module to a second preset person.
In one possible design, the common services module includes:
the log service unit is used for providing log service according to the source of the data to be processed and the business rule;
the database connection service unit is used for providing database connection service according to the source of the data to be processed and the business rule;
the task monitoring service unit is used for providing task monitoring service according to the source of the data to be processed and the business rule;
the exception handling service unit is used for providing exception handling service according to the source of the data to be processed and the business rule;
and the task string analyzing and converting unit is used for processing the corresponding information into key value pairs according to the source of the data to be processed and the business rules.
In a third aspect, an embodiment of the present invention provides a file generating apparatus, including: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the file generation method as described above in the first aspect and various possible designs of the first aspect.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where computer-executable instructions are stored, and when a processor executes the computer-executable instructions, the file generation method according to the first aspect and various possible designs of the first aspect is implemented.
In the file generation method and device provided by the embodiment, a to-be-processed service is determined, and a source of to-be-processed data is determined according to the to-be-processed service; determining a business rule corresponding to the interface file, and loading the business rule; analyzing the service rule to determine a task processing node; if the task processing node is used for generating the interface file, the interface file is generated according to the source of the data to be processed and the service rule, the flows and the processing modes such as data determination, rule determination, file generation and the like can be configured in a configuration mode, the coupling degree of each task code is reduced, code change is reduced, code reusability is improved, and the problems that the task of acquiring various types of data through the operation on the database data entity table in the prior art is long in development period and high in testing difficulty of a newly added interface data task are solved.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
With the gradual development of information-based construction, more and more frequent data communication interaction occurs before a large number of application systems, and data interaction inside the systems is more and more. At present, in order to meet the transmission requirements of various types of data with specific requirements among multiple systems or realize data sharing in the systems so as to quickly and effectively adapt to the continuously changing data interaction requirements, a technology for performing file generation on database data entity table operation is developed. However, tasks for acquiring various types of data through the operation on the database data entity table are defined in a specific program, the program needs to be modified every time one type of interface data task is newly added, the coupling degree of each task code is high, the development period is long, and the test difficulty is large.
Therefore, in view of the above problems, the present invention provides a file generation method, which determines a service to be processed and determines a source of data to be processed according to the service to be processed; determining a business rule corresponding to the interface file, and loading the business rule; analyzing the service rule to determine a task processing node; if the task processing node is used for generating an interface file, the interface file is generated according to the source of the data to be processed and the service rule, and the flows and processing modes of data determination, rule determination, file generation and the like are configured in a configuration mode, so that the coupling degree of each task code is reduced, the code change is reduced, the code reusability is improved, and the problems that the existing task of acquiring various data through the operation on the database data entity table is long in development period and high in test difficulty of a newly added interface data task are solved.
Fig. 1 is an application scenario diagram of a file generation method provided by the present invention. As shown in fig. 1, a processor 101 may determine a service to be processed, determine a source of data to be processed according to the service to be processed, determine a service rule corresponding to an interface file, load the service rule, analyze the service rule, determine a task processing node, and generate an interface file 102 according to the source of the data to be processed and the service rule if the task processing node is generated for an interface file.
The processor 101 can organize complex and disordered work to form a standard job task through simple flow configuration, and analyzes required service data according to a node sequence by taking the job task as a core, so that the coupling of service requirement data and a software method is reduced, and required data information is quickly formed.
Fig. 2 is a first flowchart of a file generation method according to an embodiment of the present invention, where an execution main body of the embodiment may be the processor in the embodiment shown in fig. 1, or may also be other devices, such as a server, and the embodiment is not limited herein. As shown in fig. 2, the method may include:
s201, determining a service to be processed, and determining a source of data to be processed according to the service to be processed.
Here, the service to be processed can be determined according to actual conditions, and further information such as which database the corresponding data originates from, which province score originates from, and the like can be determined.
S202, determining a business rule corresponding to the interface file, and loading the business rule.
The business rules can be customized, and after the corresponding business rules are customized, customized rule information can be loaded to provide information for other services.
Specifically, the business rule may include a rule specified by a user and parameter information specified by the user, and the rule information is organically combined with the parameter information. After the business rules are loaded, the business rules can be displayed, the rules and parameters are visually displayed for a user, and the autonomous customization of user tasks and the visualization of user customized information are realized.
S203, analyzing the service rule and determining a task processing node.
Here, parsing the business rule may include: analyzing the business rules by using one or more of SERVICE, RSET-INDB, FUNC, RSHELL, and SQL-RSET, that is, analyzing the information according to a defined key value by task analysis, and linking to specific SERVICE information, where the SERVICE information is extensible, and a plurality of different analysis SERVICEs can be added according to the user's requirements, and specifically, analyzing the business rules includes: providing Structured Query Language (SQL) analysis to generate result set data; the warehousing can be directly realized; an external executable or shell, etc. may be invoked.
And S204, if the task processing node is generated for the interface file, generating the interface file according to the source of the data to be processed and the business rule.
After the interface file is generated, an INSERT type SQL statement can be formed and transmitted to a system internal database, so that the interface file can be provided for a peripheral system, and the file can be put in storage according to self-defined rule information, so that data persistence is realized.
In the file generation method provided by this embodiment, a service to be processed is determined, and a source of data to be processed is determined according to the service to be processed; determining a business rule corresponding to the interface file, and loading the business rule; analyzing the service rule to determine a task processing node; if the task processing node is used for generating an interface file, the interface file is generated according to the source of the data to be processed and the service rule, and the flows and processing modes of data determination, rule determination, file generation and the like are configured in a configuration mode, so that the coupling degree of each task code is reduced, the code change is reduced, the code reusability is improved, and the problems that the existing task of acquiring various data through the operation on the database data entity table is long in development period and high in test difficulty of a newly added interface data task are solved.
Fig. 3 is a second flowchart of a file generation method according to an embodiment of the present invention, and this embodiment describes details of a specific implementation process of this embodiment on the basis of the embodiment of fig. 2. As shown in fig. 3, the method includes:
s301, determining task execution time, and sending a task initiating instruction according to the task execution time, wherein the task initiating instruction carries a task identifier.
Here, the anontab can be used to scan python engineering at regular time, in which many task strings are defined according to time points, and the task strings are initiated when the tasks needing to be executed are scanned. Among them, the crontab command is commonly found in Unix and Unix-like operating systems and is used to set instructions that are executed periodically. Python is a computer programming language, a dynamic, object-oriented scripting language, originally designed for writing automated scripts (shells), and is increasingly being used for development of independent, large projects with the continued updating of versions and the addition of new functionality to the language.
S302, determining a service to be processed according to the task identifier, determining a source of data to be processed according to the service to be processed, and feeding back corresponding information.
And S303, sending reminding information to a first preset person according to the feedback information.
And when the task is initiated, a message is returned, and the corresponding message is sent to the corresponding personnel according to the configuration.
S304, checking whether each data file corresponding to the task identifier exists.
S305, sending the inspection result to a second preset person.
In addition, after the task is initiated, whether a corresponding data file exists is also checked, and a corresponding message is sent to a corresponding person according to the configuration.
S306, determining a business rule corresponding to the interface file, and loading the business rule.
The task rule loading can load custom rule information and provide information for other services. The business rules may include user-specified rules and user-specified parameter information. After the business rules are loaded, the business rules can also be presented.
S307, analyzing the service rule and determining a task processing node.
The task analysis analyzes information according to a defined key value, and links to specific service information, the service information can be expanded, a plurality of different analysis services can be added according to the requirements of users, and specifically, the analysis of the business rule can include: and analyzing the business rule by utilizing one or more of SERVICE, RSET-INDB, FUNC, RSHELL and SQL-RSET.
S308, if the task processing node is generated for the interface file, generating the interface file according to the source of the data to be processed and the business rule.
After the interface file is generated, an INSERT type SQL statement can be formed and transmitted to a database inside the system.
S309, if the task processing node is used for uploading files, uploading the interface files.
And S310, if the task processing node is a public service, providing the public service according to the source of the data to be processed and the business rule.
Wherein, the public service can include: log service, database connection service, task monitoring service, exception handling service, and task string parsing, conversion functions, etc. Specifically, the log service: the method can provide the user to print the relevant service processing information and provide the information record, the log file can cross linux and windows platforms, the process is unique, the log can be printed in a grading way, and the process and the thread mark are provided; database connection service: providing database related information connection string service, and returning corresponding database connection information according to information provided by the configuration management service; and task monitoring service: according to the task execution state, implementing processing task state information and providing a corresponding task processing information graph; exception handling service: providing exception handling services for database connection service, task rule analysis service, task analysis service and the like, and performing task string analysis and conversion service: the provided series of task information is processed into key value equivalent information.
The file generation method provided by the embodiment can reduce the coupling of service-related development and system function development in tasks, is efficient in task development and clear in scheduling execution process, solves the problems that the existing tasks for acquiring various data through the operation on the database data entity table are long in task development period and high in test difficulty of newly added interface data, can concentrate more efforts on service requirement development, realizes the self-definition of the tasks, realizes the quick realization of data transmission, provides convenient and quick experience and visual feeling, is more convenient for processing the tasks, and is efficient in data processing.
Fig. 4 is a first schematic structural diagram of a file generation device according to an embodiment of the present invention. As shown in fig. 4, the file generating apparatus 40 includes: the system comprises a data analysis processing module 401, a business rule configuration management module 402, a business rule loading module 403, a task analysis module 404 and an interface file generation module 405.
The data analysis processing module 401 is configured to determine a service to be processed, and determine a source of data to be processed according to the service to be processed.
And a business rule configuration management module 402, configured to determine a business rule corresponding to the interface file.
A business rule loading module 403, configured to load the business rule.
And the task analysis module 404 is configured to analyze the service rule and determine a task processing node.
An interface file generating module 405, configured to generate an interface file according to the source of the to-be-processed data and the service rule if the task processing node is an interface file.
The device provided in this embodiment may be used to implement the technical solution of the above method embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
Fig. 5 is a schematic structural diagram of a file generation device according to an embodiment of the present invention. As shown in fig. 5, this embodiment further includes, on the basis of the embodiment in fig. 4: a task timing module 406, a communication service module 407, a file uploading module 408, a public service module 409 and a data checking module 410.
In a possible design, the task timing module 406 is configured to determine a task execution time, and send a task initiation instruction to the data analysis processing module according to the task execution time, where the task initiation instruction carries a task identifier.
The data analysis processing module 401 is configured to determine a service to be processed according to the task identifier, determine a source of data to be processed according to the service to be processed, and feed back corresponding information to the task timing module.
And the communication service module 407 is configured to send a reminding message to a first preset person according to the information fed back by the data analysis processing module.
In a possible design, the file uploading module 408 is configured to upload the interface file if the task processing node is a file uploading module.
In a possible design, the public service module 409 is configured to provide a public service according to a source of the to-be-processed data and the business rule if the task processing node is a public service.
In one possible design, the data checking module 410 is configured to check whether each data file corresponding to the task identifier exists.
The communication service module 407 is further configured to send the inspection result of the data inspection module to a second preset person.
In one possible design, the common services module 409 includes:
and a log service unit 4091, configured to provide a log service according to the source of the to-be-processed data and the service rule.
And a database connection service unit 4092, configured to provide a database connection service according to the source of the to-be-processed data and the business rule.
And a task monitoring service unit 4093, configured to provide a task monitoring service according to the source of the to-be-processed data and the business rule.
And an exception handling service unit 4094, configured to provide an exception handling service according to the source of the to-be-processed data and the business rule.
And the task string analyzing and converting unit 4095 is used for processing the corresponding information into key value pairs according to the source of the data to be processed and the business rules.
The device provided in this embodiment may be used to implement the technical solution of the above method embodiment, and the implementation principle and technical effect are similar, which are not described herein again.
Fig. 6 is a schematic diagram of a hardware structure of a file generation device according to an embodiment of the present invention. As shown in fig. 6, the file generating apparatus 60 of the present embodiment includes: a processor 601 and a memory 602; wherein
A memory 602 for storing computer-executable instructions;
the processor 601 is configured to execute the computer execution instructions stored in the memory to implement the steps performed by the file generation method in the foregoing embodiments. Reference may be made in particular to the description relating to the method embodiments described above.
Alternatively, the memory 602 may be separate or integrated with the processor 601.
When the memory 602 is provided separately, the file generating apparatus further includes a bus 603 for connecting the memory 602 and the processor 601.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer execution instruction is stored in the computer-readable storage medium, and when a processor executes the computer execution instruction, the file generation method as described above is implemented.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit. The unit formed by the modules can be realized in a hardware form, and can also be realized in a form of hardware and a software functional unit.
The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application.
It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc.
The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (enhanced Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.
The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in an electronic device or host device.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.