HK1227131B - Source code translation - Google Patents

Description

Source code translation

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. application serial number 61/912,594, filed December 6, 2013.

Technical Field

This specification relates to source code translation, and more particularly, to the translation of source code specified in one or more original software programming languages into one or more other different software programming languages.

Background Art

In the world of software development, software engineers can choose to develop software using one or more of many different programming languages. At the time of writing, some examples of modern programming languages commonly used by developers are Java, C#, and C++. Generally speaking, each programming language has its own advantages and disadvantages, and it is the software engineer's job to consider these advantages and disadvantages when choosing the right programming language for a given application.

Over the years, programming language technology has advanced, causing some earlier programming languages to become less used, unsupported, and/or obsolete. Examples of these earlier programming languages include Basic and Fortran. Despite this, source code written in these earlier programming languages (often referred to as "legacy" code) has remained in the industry for many years due to its adequate performance. However, when this legacy code no longer performs adequately, code modifications become necessary, and finding software engineers with the necessary skills to update legacy code can be difficult.

For this reason, source-to-source compilers have been developed that receive as input a first software specification specified in a first programming language and generate as output a second software specification specified in a different second programming language. Such source-to-source compilers are used to translate legacy code into modern programming languages that are more easily edited by software engineers skilled in the use of modern programming languages.

Summary of the Invention

The technical problem addressed involves converting between a software specification containing source code in a procedural language and a software specification containing source code in another language that is not limited to procedural programming structures but rather operates using a different modality. For example, the other language can operate in a modality involving data flowing between different programming entities to drive execution alone or in conjunction with explicit control flow, rather than solely through control explicitly passed between different procedures. Converting source code between languages with such fundamental differences involves more than just literal translation between different styles of language. For systems with multilingual source code, another technical problem addressed involves providing the source code to a new system that merges these multiple languages into a single, distinct language.

In one aspect, in general, a method for software specification translation includes: receiving a first software specification specified in a first programming language (e.g., COBOL); receiving a second software specification specified in a second programming language (e.g., COBOL); receiving a third software specification specified in a third programming language (in some embodiments, the third programming language (e.g., JCL) is different from the first programming language and the second programming language), the third software specification defining one or more data relationships between the first software specification and the second software specification; forming a representation of the first software specification in a fourth programming language (e.g., a data flow diagram) that is different from the first programming language, the second programming language, and the third programming language; forming a representation of the second software specification in the fourth programming language; analyzing the third software specification to identify one or more data relationships; and forming a combined representation of the first software specification and the second software specification in the fourth programming language, including forming a connection between the representation of the first software specification in the fourth programming language and the representation of the second software specification in the fourth programming language based on the identified one or more data relationships.

Various aspects can include one or more of the following features.

The first programming language was a procedural programming language.

The fourth programming language enables parallelism between different parts of a software specification.

The fourth programming language enables multiple types of parallelism, including: a first type of parallelism that enables multiple instances of a portion of a software specification to operate on different portions of an input data stream; and a second type of parallelism that enables different portions of a software specification to execute simultaneously on different portions of an input data stream.

The second programming language is a procedural programming language.

The second programming language is the same as the first programming language.

The one or more data relationships between the first software specification and the second software specification include at least one data relationship corresponding to the first software specification receiving data from the first data set and the second software specification providing data to the first data set.

The fourth programming language is a data flow graph based programming language.

A connection in the fourth programming language corresponds to a directional link representing a flow of data.

The first software specification is configured to interact with one or more data sets, each data set having an associated data set type from a plurality of data set types of the first software specification, and the second software specification is configured to interact with one or more data sets, each data set having an associated type from a plurality of data set types of the second software specification, the method further comprising: processing the first software specification, the processing comprising: identifying one or more data sets of the first software specification, and determining, for each of the one or more identified data sets, an associated type of the data set in the first software specification; and forming a representation of the first software specification in a fourth programming language, comprising forming a specification of the data set in the fourth programming language for each of the one or more identified data sets, the specification of the data set in the fourth programming language having an associated type from the first programming language the associated type of the dataset in the first programming language; wherein at least one of the specifications of the one or more datasets in the fourth programming language has: an input dataset type or an output dataset type; processing the second software specification, the processing comprising: identifying one or more datasets of the second software specification, and determining the associated type of the dataset in the second software specification for each of the one or more datasets identified; and forming a representation of the second software specification in the fourth programming language, comprising forming a specification of the dataset in the fourth programming language for each of the one or more datasets identified, the specification of the dataset in the fourth programming language having a type corresponding to the associated type of the dataset in the first programming language; wherein at least one of the specifications of the one or more datasets in the fourth programming language enables an input function or an output function.

Forming a combined representation includes at least one of the following: forming one or more connections to replace the connection between the specification of one or more data sets of the second software specification of the fourth programming language enabling input functionality and the representation of the second software specification of the fourth programming language with the connection between the representation of the first software specification of the fourth programming language and the representation of the second software specification of the fourth programming language; or forming one or more connections to replace the connection between the specification of one or more data sets of the first software specification of the fourth programming language enabling input functionality and the representation of the first software specification of the fourth programming language with the connection between the representation of the second software specification of the fourth programming language and the representation of the first software specification of the fourth programming language.

The method also includes: saving one or more data sets of the first software specification in the fourth programming language that enable the output function in the representation of the first software specification in the fourth programming language, or saving one or more data sets of the second software specification in the fourth programming language that enable the output function in the representation of the second software specification in the fourth programming language.

The first software specification includes one or more data conversion operations, and analyzing the first software specification includes identifying at least some of the one or more data conversion operations and classifying the identified data conversion operations into corresponding data conversion types in a fourth programming language, and forming a representation of the first software specification in the fourth programming language includes forming a specification of the data conversion operation in the fourth programming language for each of the identified data conversion operations, the specification of the data conversion operation in the fourth programming language enabling data conversion operations corresponding to the data conversion types of the identified data conversion operations in the first programming language.

At least one of the specifications of the one or more data sets in the fourth programming language is of a read-only random access data set type.

Determining the associated type of the data set in the first software specification includes analyzing the data set definition and parameters of commands that access the data set.

These parameters include one or more of a file organization associated with the data set, an access mode associated with the data set, a mode used to open the data set, and input and output operations.

The method also includes storing a combined representation of the first software specification and the second software specification in a storage medium.

The first software specification defines one or more data processing operations that interact with the one or more data sets, and the second software specification defines one or more data processing operations that interact with the one or more data sets.

The third software specification defines one or more data relationships between the one or more data sets of the first software specification and the one or more data sets of the second software specification.

In another embodiment, generally, software is stored in a non-transitory form on a computer-readable medium for software specification translation. The software includes instructions that cause a computing system to: receive a first software specification specified in a first programming language; receive a second software specification specified in a second programming language; receive a third software specification specified in a third programming language different from the first and second programming languages, the third software specification defining one or more data relationships between the first and second software specifications; form a representation of the first software specification in a fourth programming language different from the first, second, and third programming languages; form a representation of the second software specification in the fourth programming language; analyze the third software specification to identify one or more data relationships; and form a combined representation of the first and second software specifications in the fourth programming language, including forming a connection in the fourth programming language between the representation of the first software specification in the fourth programming language and the representation of the second software specification in the fourth programming language based on the one or more identified data relationships.

In another embodiment, generally, a computing system for software specification translation includes: an input device or port configured to receive a software specification, the software specification including: a first software specification specified in a first programming language; a second software specification specified in a second programming language; a third software specification specified in a third programming language different from the first programming language and the second programming language, the third software specification defining one or more data relationships between the first software specification and the second software specification; and at least one processor configured to process the received software specification, the processing including: forming a representation of the first software specification in a fourth programming language different from the first programming language, the second programming language, and the third programming language; forming a representation of the second software specification in the fourth programming language; analyzing the third software specification to identify one or more data relationships; and forming a combined representation of the first software specification and the second software specification in the fourth programming language including forming a connection between the representation of the first software specification in the fourth programming language and the representation of the second software specification in the fourth programming language based on the identified one or more data relationships.

Various aspects may include one or more of the following advantages.

Converting a program based on certain data relations between different program specifications makes it possible to form a combined specification that can be more effectively executed in various environments (such as, data processing systems). For example, by converting a program written in one or more procedural programming languages into a data flow graph, component parallelism, data parallelism, and pipeline parallelism are achieved. For component parallelism, a data flow graph includes a plurality of components interconnected by directional links representing the flow (or "data flow") of the data between the components, and the components of the different parts of the data flow graph can be run simultaneously for each data flow. For data parallelism, the data flow graph is processed to the data divided into a plurality of fragments (or "partitions"), and a plurality of instances of a component can be operated simultaneously for each fragment. For pipeline parallelism, the components connected by the data flow links in the data flow graph can be run simultaneously, and the upstream component adds data to the data flow and the downstream component receives data from the data flow.

Converting a program written in a procedural programming language (or a specification of at least some parts of the program) into a data flow graph representation of the program can enable execution of different components of the data flow graph representation on different servers.

Programs written in procedural programming languages that may require intermediate data sets (due to their non-parallel nature) can be eliminated from the data flow graph by converting the program into a data flow graph representation and replacing the intermediate data sets with flows of data. In some examples, the intermediate data sets are taken from the path of the data flowing through the data flow graph and saved to ensure that any other program that uses the data set can still access the data included in the data set. In some examples, eliminating the intermediate data sets can reduce storage and I-O traffic requirements.

Converting a program written in one or more procedural programming languages into a data flow graph representation enables visualization of the lineage of data through the program.

Dataflow programming languages are database-agnostic. Therefore, converting a program written in a procedural programming language into a dataflow graph representation of the program can enable the use of the program with database types not originally supported by the procedural programming language. That is, various methods can abstract the inputs and outputs in code (e.g., JCL/COBOL code) into flows that can be connected to many different types of sources and sinks (e.g., queues, database tables, files, etc.).

The data flow diagram that the program written in procedural programming language is converted into program can realize the use of user-defined multiplexed data types. Compared with some procedural programming languages (such as COBOL) that do not clearly distinguish between data type (that is, metadata) and storage allocation but combine the two in data division (data division), this is advantageous. The method described herein extracts metadata from COBOL source code and creates multiplexed data types (for example, DML data types) and type definition files. Multiplexed data types and type definition files can be used for storage allocation in the first place of process conversion and for port and lookup file record definition. In some examples, the extracted data type (for example, data type metadata from COBOL) is combined with the extracted data set (for example, data set metadata from JCL) and can also be used to integrate the partial description (that is, partial metadata) of the data of multiple programs accessing the same data set into a comprehensive description of data.

Converting a program written in one or more procedural programming languages into a data flow graph representation enables simplified editing of the program through a graphical development environment based on data flow graphs.

Other features and advantages of the invention will be apparent from the following description and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a block diagram of a system including a software translation module.

FIG2 is a schematic example of a software specification.

Figure 3 is a block diagram of the top-level software translation module.

FIG4 is a block diagram of a software translation module.

FIG5 is a table of dataset functions and their associated combinations of dataset organization type, access mode, and open mode.

FIG6 is a first process transition.

FIG7 is a second process transition.

FIG8 is a third process transition.

Figure 9 is a data flow graph representation of the program.

Figure 10 illustrates the creation of a composite dataflow graph.

FIG. 11 is an operational example of the top-level software translation module of FIG. 3 .

FIG. 12 is an operational example of the software translation module of FIG. 4 .

FIG13 is a composite data flow diagram.

DETAILED DESCRIPTION

FIG1 illustrates an example of a data processing system 100 capable of translating a program using the source code translation techniques described herein. The translated program is executable to process data from a data source 102 of the data processing system 100. A translation module 120 receives as input a first software specification 122 in one or more procedural programming languages and processes the software specification 122 to generate a composite dataflow graph representation 332 of the first software specification 122 in a dataflow-based programming language. The dataflow graph representation 332 of the first software specification 122 is stored in the data storage system 116, where it can be visually presented within the development environment 118. Developers 120 can use the development environment 118 to verify and/or modify the dataflow graph representation 332 of the first software specification 122.

System 100 includes data source 102, which may include one or more data sources, such as storage devices or connections to online data streams, each of which can store or provide data in any of a variety of formats (e.g., database tables, spreadsheet files, plain text files, or native formats used by the host). Execution environment 104 includes load modules 106 and execution modules 112. Execution environment 104 can be hosted on one or more general-purpose computers under the control of a suitable operating system (e.g., a UNIX version of the operating system). For example, execution environment 104 can include a multi-node parallel computing environment, including a computer system configuration using multiple central processing units (CPUs) or processor cores, which can be local (e.g., a multiprocessor system such as a symmetric multiprocessing (SMP) computer) or locally distributed (e.g., a clustered multiprocessor or massively parallel processing (MPP) system), remote or remotely distributed (e.g., a multiprocessor coupled via a local area network (LAN) and/or a wide area network (WAN)), or any combination thereof.

Load module 106 loads data flow graph representation 332 into execution module 112, where it is executed to process data from data source 102. The storage device providing data source 102 can be local to execution environment 104, such as stored on a storage medium (e.g., hard drive 108) connected to the computer hosting execution environment 104, or can be remote to execution environment 104, such as hosted on a remote system (e.g., host computer 110) that communicates with the computer hosting execution environment 104 via a remote connection (e.g., provided by a cloud computing infrastructure). Data flow graph representation 332 executed by execution module 104 can receive data from a variety of systems, including different forms of database systems, that can embody data source 102. This data can be organized into records (also called "rows") with values for various fields (also called "attributes" or "columns"), including possible null values. When data is first read from a data source, data flow graph representation 332 typically begins with some initial formatting information about the records in that data source. In some cases, the record structure of the data source may not be known initially, but may be determined after analyzing the data source or the data. The initial information about the record can include, for example, the number of bits representing different values, the order of the fields within the record, and the type of value represented by the bits (e.g., string, signed integer/unsigned integer).

The data flow graph representation 332 can be configured to generate output data that can be stored back in a data storage system 116 accessible to the data source 102 or the execution environment 104, or used in other ways. The development environment 118 can also access the data storage system 116. In some implementations, the development environment 118 is a system for developing applications as data flow graphs, which include a plurality of vertices (representing data processing components or data sets) connected by directed links (also referred to as "flows") between the vertices, representing the flow of work elements (i.e., data). For example, such an environment is described in more detail in U.S. Publication No. 2007/0011668, entitled "Managing Parameters for Graph-Based Applications," which is incorporated herein by reference. A system for performing such graph-based computations is described in U.S. Patent No. 5,966,072, entitled "Executing Computations Expressed as Graphs," which is incorporated herein by reference. Dataflow graphs created according to the present system provide methods for getting information into and out of the various processes represented by the graph components, for moving information between processes, and for defining the order in which processes should run. The present system includes algorithms for selecting an inter-process communication method from any available methods (e.g., the communication paths according to the graph's links can use TCP/IP or UNIX domain sockets, or use shared memory to pass data between processes).

1. Software Specifications

In some examples, the first software specification 122 is specified using one or more procedural text-based programming languages, such as C, C++, Java, C#, IBM's Job Control Language (JCL), COBOL, Fortran, Assembly, and the like. For some examples below, the software specification 122 includes a batch script written in the JCL scripting language and a number of programs written in the COBOL programming language. The JCL scripts reference the COBOL programs and implement a controlled execution flow based on decisions. It should be understood that the first software specification 122 is not limited to a combination of the JCL and COBOL programming languages, and it should be understood that this combination of programming languages is merely used to illustrate one exemplary embodiment of the translation module 120.

2 , a schematic diagram of an example of the software specification 122 of FIG. 1 includes a JCL script 226 including a number of steps 230, some of which execute a COBOL program 228. To simplify this description, other possible steps of the JCL script 226 are omitted. Each step in the JCL script that executes a COBOL program specifies the name of the COBOL program (e.g., COBOL1) and the data sets on which the COBOL program operates. For example, step 3 of the JCL script executes a COBOL program called "COBOL1" on the "DS1.data" and "DS2.data" data sets. In the JCL script 226, each data set associated with a given COBOL program is assigned a file handle (also referred to as a "DD name"). For example, in FIG. 2 , "DS1.data" is assigned a file handle "A," and "DS2.data" is assigned a file handle "B." Each of the COBOL programs 228 includes source code (written in the COBOL programming language) that operates on the data sets specified by the JCL script 226. The file handle (ie, DD name) of a given data set is an identifier that the JCL script 226 and the code of the COBOL program use to identify the data set.

In operation, a conventional job scheduler, such as one running on an IBM mainframe computer, accesses the JCL script 226 and executes the script's steps 230 sequentially (i.e., one at a time) according to the control flow defined by the JCL script 226. Generally speaking, any COBOL program accesses input or output data sets by reading from or writing to a storage medium storing the data sets (e.g., a storage medium of the data source 102 or the data storage system 116, such as a hard disk drive ("disk")). Typically, each COBOL program executed by the JCL script 226 reads all of its input data from disk and writes all of its output data to disk before passing control back to the JCL script 226. Thus, any step that relies on the output of a previous step for input data must read the input data from disk.

2 Translation Module

3 , one example of the translation module 120 of FIG 1 receives as input a software specification 122 including a JCL script 226 and a COBOL program 228 referenced by the JCL script 226, and processes the software specification 122 to generate a composite data flow graph 332 that implements the same functionality as the first software specification 122 and is used by the execution environment 104 of FIG 1 . The translation module 120 includes a COBOL to data flow graph translator 334 and a composite graph synthesizer 336.

In general, the COBOL to data flow graph translator 334 receives COBOL programs 228 as input and converts each COBOL program into a separate data flow graph representation 338 of the COBOL program. As described in more detail below, each data flow graph representation 338 of a COBOL program includes a data flow graph component, referred to as a "process transformation" component, and zero or more data sets and/or other data flow graph components. The process transformation component includes ports, such as input ports and output ports, for connecting the process transformation component to the data sets and other components of the data flow graph representation 338 of the COBOL program and performing some or all of the functions of the COBOL program. In some examples, the data flow graph representation of a COBOL program includes data flow graph components that are similar to commands present in the COBOL program. In some examples, the data flow graph representation 338 of a COBOL program can be implemented as a "subgraph" having input ports and output ports for forming flows between instances of the data flow graph representation 338 of the COBOL program and other data flow graph components (e.g., other data flow graph components of the composite data flow graph 332 of FIG. 3 ).

The JCL script 226 and the data flow graph representation 338 of the COBOL program are provided to a composite graph synthesizer 336, which analyzes the JCL script 226 to determine the data flow interconnections between the COBOL program and any other components. The composite graph synthesizer 336 then synthesizes a composite data flow graph 332 by connecting the input/output ports of the data flow graph representation 338 of the COBOL program using flows based on the determined data flow interconnections. The composite graph synthesizer 336 determines the data flow interconnections between the COBOL programs by identifying "intermediate" data sets that are written at earlier steps in the JCL and read at later steps in the JCL. In some examples, the intermediate data sets can be eliminated and replaced by data flows between components in the composite data flow graph 332. Due to pipeline parallelism, significant performance improvements can be achieved by allowing data to flow directly between components without performing the intermediate steps of writing to and reading from disk. It should be noted that the term "elimination" as used above does not necessarily mean that intermediate data sets are deleted. In some cases, intermediate data sets taken from the path of data flowing through the data flow graph are still written to disk to ensure that other programs that depend on the intermediate data sets (for example, those executed by other JCL scripts) can still access their data. If the intermediate files can be eliminated completely (because the JCL deletes them after using them), the data flow graph representation will also reduce storage capacity requirements.

In some examples, the sequential nature of certain steps in the JCL code can be ignored, resulting in component parallelism in the composite dataflow graph 332. In other examples, pipeline parallelism is achieved by preserving the sequential nature of steps by connecting the components of the steps using flows, for steps that provide the output of one step as the input of another step.

2.1COBOL to Data Flow Graph Translator

4 , a detailed block diagram of an implementation of a COBOL to data flow graph translator 334 receives a plurality of COBOL programs 228 as input and processes the COBOL programs 228 to generate a plurality of data flow graph representations 338 of the COBOL programs. The COBOL to data flow graph translator 334 includes a COBOL parser 440, an internal component analyzer 444, a data set function analyzer 442, a metadata analyzer 441, an SQL analyzer 443, a procedure portion translator 445, and a subgraph synthesizer 446.

Each COBOL program 228 is first provided to a COBOL parser 440 that parses the COBOL program 228 to generate a parse tree. The parse tree generated by the COBOL parser 440 is then passed to an internal component analyzer 444, a data set function analyzer 442, a metadata analyzer 441, and an SQL analyzer 443.

The internal component analyzer 444 analyzes the parse tree to identify program processes in the data flow graph programming language that have similar data flow graph components (e.g., internal sort). Some examples of COBOL operations that can be converted into data flow graph components are "internal sort" and "internal recirculate" (temporary storage) operations. The internal sort operation corresponds to a component having an input port and an output port, the input port receiving a stream of unsorted data and the output port providing a stream of sorted data, the input port and the output port being linked to the main component, as described in more detail below. The internal recirculate operation corresponds to an intermediate file that is first written in its entirety in sequence and then read in its entirety within the COBOL program. The output of the data set function analyzer 444 is an internal component result 448 that includes a list of the identified operations along with their corresponding positions in the COBOL parse tree.

The above applies to any procedural language that can identify declarations or a sequence of declarations and/or operations that perform a particular transformation on a series of records in a stream, corresponding to a component or subgraph that receives the stream at an input port and provides the transformed records from an output port.

The data set function analyzer 442 analyzes the parse tree to identify all data sources and sinks (e.g., data sets) that the COBOL program 228 accesses (e.g., opens, creates, writes to, or reads from) and determines the types associated with the data sets of the COBOL program. To do this, the data set function analyzer 442 identifies and analyzes COBOL statements that access (e.g., open, read, write to, delete, etc.) data sets. In some examples, possible types that may be associated with a data set include: input, output, lookup, and updateable lookup. COBOL defines a handle or path to a specified data set, the file organization of the data set, and the access mode of the data set, with additional information such as the file open mode(s) determined by the Input-Output statement.

Possible dataset file organizations include: sequential, indexed, and relative. A dataset with sequential organization includes records that can only be accessed sequentially (i.e., in the order in which they were originally written to the dataset). A dataset with indexed organization includes each record associated with one or more index keys. The records of an indexed dataset can be accessed randomly using the key, or sequentially from any given position in the file. A dataset with relative organization has record slots numbered with positive integers, with each slot marked as either empty or containing a record. When a file with relative organization is read sequentially, empty slots are skipped. Records of the relative file can be directly accessed using the slot number as a key. The concept of 'file position' is common to all three file organizations.

Possible access modes include: sequential access mode (SEQUENTIAL), random access mode (RANDOM), and dynamic access mode (DYNAMIC). Sequential access mode means that records in the data set are accessed sequentially according to the order in which they are entered, in ascending or descending key order. Random access mode means that records in the data set are accessed using record identification keys. Dynamic access mode means that records in the data set can be accessed directly using record identification keys, or sequentially from any selected file location.

Possible open modes include: Input Open Mode (INPUT), Output Open Mode (OUTPUT), Extended Open Mode (EXTEND), and I-O Open Mode (I-O). Input Open Mode indicates that a dataset is opened as an input dataset. Output Open Mode indicates that an empty dataset is opened as an output dataset. Extended Open Mode indicates that a dataset containing pre-existing records is opened as an output dataset with new records appended. I-O Open Mode indicates that this dataset open mode supports input and output dataset operations (regardless of whether these operations exist in the program).

The data set functionality analyzer 442 applies the following set of rules to the file organization, access mode, and open mode of a COBOL data set access command to determine the functionality associated with a data set of a COBOL program:

An output (OUTPUT) data set is a data set with sequential organization, indexed organization, or associative organization, sequential access mode, random access mode, or dynamic access mode, and output open mode or extended open mode.

An input data set is a data set with index organization, associative organization, or sequential organization, sequential access mode, and input open mode.

A lookup data set is a data set with index organization or associative organization, random access mode or dynamic access mode, and input open mode.

An updateable lookup data set is a data set with index organization or correlation organization, random access mode or dynamic access mode, and I-O open mode.

In some examples, the "effective open mode" of a file can be determined by counting the actual input and output operations of the file. For example, if a file is opened in I-O mode, but has only write operations and no read or start operations, the "effective open mode" can be changed to extended open mode.

5 , table 501 lists different combinations of dataset organization, access mode, and open mode along with the dataset functions associated with each combination.

Referring again to FIG. 4 , the output of the data set function analyzer 442 is a data set function result 450 that includes a listing of all data sets accessed by the COBOL program along with their associated functions in the COBOL program.

Metadata analyzer 441 analyzes the parse tree to extract metadata and creates reusable data types (e.g., DML data types) and type definition files. Reusable data types are different from storage allocations in COBOL programs. The output of metadata analyzer 441 is data type results 447.

The SQL analyzer 443 analyzes the parse tree to identify embedded structured query language (SQL) code (or simply "embedded SQL") in the COBOL program. Any identified embedded SQL is processed into database interface information 449. The database application programming interface (API) for accessing the database can provide primitives that can be used within the database interface information 449. In some examples, the inclusion of these primitives avoids the need to compile portions of the embedded SQL using a specific solution to access a specific database (e.g., compiling into a binary form that is manipulated using binary operations). Instead, the flexibility of being able to interpret the embedded SQL at runtime using appropriate API primitives arranged within the database interface information 449, potentially using different databases and/or solutions as needed, can trade off some of the efficiencies provided by such compilation.

The parse tree of the COBOL program is then provided to a process translator 445, along with internal component results 448, data set function results 450, data type results 447, and database interface information results 449. The process translator 445 analyzes the parse tree to translate the COBOL logic into a "process transformation" data flow graph component 452. Typically, the process transformation data flow graph component 452 is a container-type component that contains some or all of the COBOL logic associated with the COBOL program and has input ports and output ports that receive input data from the component and provide output data, respectively. In the event that the COBOL code includes code from a different programming language (e.g., SQL code identified by the SQL analyzer 443 and provided in the database interface information result 449), the process translator 445 uses the database interface information result 449 to generate an appropriate representation of the embedded code within the process transformation data flow graph component 452. In some examples, the process translator 445 uses the database API to generate an appropriate representation of the embedded code. In other examples, embedded SQL tables and cursors are replaced with input table components, replacing FETCH operations with calls to read_record(port_number), as is done for files.

In some examples, the procedural translator 445 generates only a file containing Data Manipulation Language (DML) code representing the procedural logic of the COBOL program. The subgraph synthesizer 446 generates a process transformation data flow component using the file generated by the procedural translator 445.

4 and the above description relate to one possible order of operation of internal component analyzer 444, dataset functional analyzer 442, metadata analyzer 441, and SQL analyzer 443. However, the order of operation of these analyzers is not limited to the order described above, and other orders of operation of these analyzers are possible.

6 , a simple example of a process transformation component 554 named "COBOL2" (i.e., the result of translating the COBOL program executed in step 5 of the JCL script 226 of FIG. 2 ) has an input port 556 labeled "in0," an output port 560 labeled "out0," and a lookup port 562 labeled "lu0." Note that lookup data sets are not necessarily accessed via ports on the component; instead, they can be accessed using the lookup data set API. However, to simplify the description, the lookup data sets are described as being accessed via the lookup ports.

Each of these ports is configured to connect to their respective data sets (identified by the JCL script 226) via flows. In some examples, a developer can view and edit the DML translation of the COBOL code underlying the process transformation component 554 by, for example, shift-double-clicking on the component or hovering over the component until an information bubble appears and clicking a 'Transform' link in the information bubble.

7 , another example of a process transformation component 664 illustrates a case where a COBOL program named "COBOL1" (i.e., the COBOL program executed in step 3 of JCL script 226 of FIG. 2 ) includes a sort command in its code. In this case, internal component analyzer 448 identifies the sort command and passes information related to the sort command to procedure translator 445. Procedure translator 445 uses the information from internal component analyzer 448 to replace the sort command in the code associated with process transformation 664 with a specialized internal sort subgraph using an interface. Subgraph synthesizer 446 uses the sort information created by 448 and creates an output port out1 for providing to-be-sorted data from process transformation 664 to internal sort dataflow subgraph component 666, and an input port in1 for receiving sorted data from internal sort dataflow subgraph component 666.

Referring to Figure 8, another similar example of a process transformation including a sort command is shown. In this example, two process transformations are created rather than a single process transformation having an output for providing data to be sorted and an input for receiving the sorted data. A first process transformation 768 of the two process transformations has an output for providing data to be sorted, while a second process transformation 770 of the two process transformations has an input for receiving the sorted data. As shown, in some examples, a sort data flow component 766 can be automatically connected between the two process transformations 768, 770 by the subgraph synthesizer 446. In other examples, the sort data flow component 766 can be manually connected between the two process transformations 768, 770.

2.2 Sub-graph synthesizer

4 , the COBOL program's process transformations 452 are passed to a subgraph synthesizer 446 along with internal component results 448, data set function results 450, data type results 447, and database interface information results 449. The subgraph synthesizer 446 uses these inputs to generate a data flow graph representation 338 for the COBOL program 228. Generally speaking, the subgraph synthesizer 446 creates a data flow graph for each COBOL program 228 that includes the process transformations for the COBOL program 228, the data sets associated with the COBOL program 228, and any internal components identified by the internal component analyzer 444. The subgraph synthesizer 446 then uses the internal component results 448 and data set function results 450 to appropriately connect the flows between the data sets, internal components, and process transformations 452. The subgraph synthesizer 446 uses the data type results 447 to describe the data flowing through the component ports. 9 , an example of a data flow graph representation 838 of an exemplary COBOL program named COBOL1 includes a process transformation 864 having: an input port labeled in0, connected to an input file via a flow, file handle “A” associated with a data set DS1.data; an output port labeled out0, connected to an output file via a flow, file handle “B” associated with a data set DS2.data; and output port out1 and input port in1, connected to an internal sorting component 866 via flows.

2.3 Composite Graph Synthesizer

3 , the data flow graph representation 338 of the COBOL program is then passed along with the JCL script 226 to a composite graph synthesizer 336. By analyzing the execution order of the COBOL program in the JCL script 226 along with the functionality of the data sets associated with the COBOL program, the composite graph synthesizer 336 connects the data flow graph representations of the COBOL code into a single composite data flow graph 332.

For example, referring to Figure 10, the data flow graph of a COBOL program named COBOL2 shows reading from input file "C" associated with data set DS2.data at an input port labeled in0, supplementing data by accessing lookup file "D" associated with DS3.data at lookup port lu0, and writing to output file "E" associated with data set DS4.data at an output port labeled out0. The data flow graph of a COBOL program named COBOL3 shows reading from two input data sets: "F" associated with DS4.data at an input port labeled in0 and "G" associated with DS5.data at an input port labeled in1, and writing to output data set "H" associated with DS6.data at an output port labeled out0. Composite graph synthesizer 336 merges JCL script 226 information with information derived from the translation of the COBOL program to determine that COBOL2 executes before COBOL3 and that COBOL2 outputs DS4.data and COBOL3 inputs DS4.data. This allows COBOL2's output port, labeled out0, to be connected via a flow to COBOL3's input port, labeled in0, thereby eliminating the need for COBOL3 to read data set DS4.data from disk. FIG10 illustrates an exemplary composite data flow graph 932, which connects COBOL2's output port, labeled out0, to COBOL3's input port, labeled in0, via a copy component 933. Copy component 933 writes data to DS4.data on disk and also passes the data directly to COBOL3's input port, labeled in0, via a flow. In this way, COBOL3 can read data streamed from COBOL2 without having to wait for data set DS4.data to be written to disk, and the data stored in DS4.data, which was not deleted by JCL script 226, is available to other processes.

In some examples, if a JCL procedure does not delete an intermediate dataset (e.g., a file) after it is created, the dataset is likely to be used by some other process running in the execution environment. In examples where this is the case, the intermediate dataset is preserved in the data flow graph representation of the JCL procedure (e.g., by using a replication component as described above). In some examples, if the JCL procedure deletes the intermediate dataset after it is created, the intermediate dataset is completely eliminated from the data flow graph representation of the JCL procedure, and the replication component is no longer required.

In some examples, the metadata for ports connected by flows in the COBOL 2 and COBOL 3 dataflow graphs described above may differ because the first software specification uses alternative definitions for the same data set. Composite graph synthesizer 336 can then insert reformatted components on the connected flows. The presence of such reformatted components can be used later to integrate the data set metadata. Metadata analyzer 441 derives metadata information for each dataflow graph 338.

3. Example Operations

11 , a simple example of the operation of the translation module 120 receives the JCL script 226 and the four COBOL programs 228 of FIG. 2 as inputs and processes these inputs to generate a composite data flow graph 332 .

In a first phase of the translation process, the COBOL programs 228 are provided to a COBOL to data flow graph translator 334, which processes each of the COBOL programs to generate data flow graph representations 338a-d of the COBOL programs. In a second phase, the JCL scripts 226 and the data flow graph representations 338a-d of the COBOL programs are provided to a composite graph synthesizer 336, which processes the JCL scripts 226 and the data flow graph representations 338a-d of the COBOL programs to generate a composite data flow graph 332.

12 , the COBOL to data flow graph translator 334 processes each of the COBOL programs 228 using a COBOL parser 440, an internal component analyzer 444, a data set functional analyzer 442, a metadata analyzer 441, and an SQL analyzer 443. Output generated by the COBOL parser 440, the internal component analyzer 444, the data set functional analyzer 442, the metadata analyzer 441, and the SQL analyzer 443 is provided to a procedure translator 445 and, along with the output of the procedure translator 445, to a subgraph synthesizer 446 that generates a data flow graph representation 338 a-d for each of the COBOL programs.

For the COBOL 1 program executed in step 3 of JCL script 226, internal component analyzer 444 confirms that the program includes an internal sequencing component. Data set functionality analyzer 442 confirms that the COBOL 1 program accesses an input data set "A" and an output data set "B." The identified internal sequencing component, data sets, and their relationship to the process transformations of the COBOL 1 program are reflected in the data flow graph representation 338a of the COBOL 1 program.

For the COBOL 2 program executed in step 5 of JCL script 226, internal component analyzer 444 did not identify any internal components, and SQL analyzer 443 did not identify any embedded SQL code. Dataset function analyzer 442 determined that the COBOL 2 program accessed one data set "C" as an input data set, another data set "E" as an output data set, and another data set "D" as a lookup data set. The identified data sets and their relationship to the process transformations of the COBOL 2 program are reflected in the data flow graph representation 338b of the COBOL 2 program.

For the COBOL 3 program executed in step 6 of JCL script 226, internal component analyzer 444 did not identify any internal components, and SQL analyzer 443 did not identify any embedded SQL code. Dataset functionality analyzer 442 determined that the COBOL 3 program accessed two data sets "F" and "G" as input data sets and one data set "H" as output data set. The identified data sets and their relationship to the process transformations of the COBOL 3 program are reflected in the data flow graph representation 338c of the COBOL 3 program.

For the COBOL 4 program executed in step 10 of JCL script 226, internal component analyzer 444 did not identify any internal components, and SQL analyzer 443 did not identify any embedded SQL code. Dataset functionality analyzer 442 determined that the COBOL 4 program accessed one data set "I" as an input data set and another data set "J" as an output data set. The identified data sets and their relationship to the COBOL 4 program's process transformations are reflected in the data flow graph representation 338d of the COBOL 4 program.

11, the JCL script 226 and the data flow graph representations 338a-d of the four COBOL programs are provided to a composite graph synthesizer 336 that analyzes the JCL script 226 and the data flow graph representations 338a-d to connect the data flow graph representations 338a-d into a single composite graph 332. Referring to FIG13, the composite graph of the JCL script 226 and the four COBOL programs 228 of FIG2 includes four process transformations COBOL1 452a, COBOL2 452b, COBOL3 452c, and COBOL4 452d interconnected by flows. The copy component 933 is used to leave (i.e., write as output data sets) multiple intermediate data sets (i.e., DS2.data, DS4.data, and DS5.data) in the composite data flow graph 332 directly using the flow connection component.

4 options available

Although the above description describes only a limited number of operations and elements of a program written in a procedural programming language translated into dataflow graph components, in some examples, all source code of the original program (e.g., a COBOL program) is translated into a dataflow graph representation.

The above-described system can be used to translate a software specification comprising any combination of one or more procedural programming languages into a data flow graph representation of the software specification.

In some examples, the translation module may encounter translation tasks that it is not ready to process. In these examples, the translation module outputs a list of unfinished translation tasks that a developer can read and use to manually correct the translation.

While the above description describes certain modules of the COBOL to data flow graph translator 334 operating in parallel, this is not necessarily the case. In some examples, metadata analyzer 441 first receives the parse tree from COBOL parser 440. Metadata analyzer 441 supplements and/or simplifies the parse tree and provides it to dataset function analyzer 442. Dataset function analyzer 442 supplements and/or simplifies the parse tree and provides it to SQL analyzer 443. SQL analyzer 443 supplements and/or simplifies the parse tree and provides it to internal component analyzer 444. Internal component analyzer 444 supplements and/or simplifies the parse tree and provides it to procedure translator 445. This is a component that operates on the parse tree sequentially.

5 Implementation

The source code translation method described above can be implemented, for example, using a programmable computing system that executes appropriate software instructions, or it can be implemented in appropriate hardware (e.g., a field programmable gate array (FPGA)) or in some hybrid form. For example, in a programming method, the software can include multiple processes in one or more computer programs executed on one or more programmed or programmable computing systems (which can be of various architectures, such as distributed, client/server, or grid), each computing system including at least one processor, at least one data storage system (including volatile and/or non-volatile memory and/or storage elements), at least one user interface (for receiving input using at least one input device or port and for providing output using at least one output device or port). The software can include, for example, one or more modules of a larger program that provides services related to the design, configuration, and execution of data flow graphs. The modules of the program (e.g., elements of a data flow graph) can be implemented as data structures or other organized data that conform to a data model stored in a data repository.

The software may be provided on a tangible, non-transitory medium (such as a CD-ROM) or other computer-readable medium (e.g., readable by a general-purpose or special-purpose computing system or device), or the software may be delivered (e.g., in the form of a propagated signal) to a tangible, non-transitory medium of a computing system via a communication medium of a network when it is executed. Some or all of the processing may be performed on a dedicated computer, or using dedicated hardware, such as a coprocessor or field programmable gate array (FPGA) or application-specific integrated circuit (ASIC). The processing may be implemented in a distributed manner, where different parts of the calculations specified by the software are performed by different computing elements. Each such computer program is preferably stored or downloaded to a computer-readable storage medium (e.g., solid-state memory or medium, or magnetic or optical medium) of a storage device accessible to a general-purpose or special-purpose programmable computer, for configuring and operating the computer when the storage medium or device is read by the computer to perform the processing described herein. It is also contemplated that the system of the present invention may be implemented as a tangible, non-transitory medium configured with a computer program, wherein the medium so configured causes the computer to operate in a specific and predefined manner to perform one or more processing steps described herein.

Several embodiments of the present invention have been described. However, it should be understood that the foregoing description is intended to illustrate and not to limit the scope of the present invention, which is defined by the scope of the following claims. Therefore, other embodiments are also within the scope of the following claims. For example, various modifications may be made without departing from the scope of the present invention. Furthermore, some of the steps described above may be sequence-independent and, therefore, may be performed in an order different from that described.

Claims (21)

1. A method for translating software specifications, comprising: Receive the first software specification specified in the first programming language; Receive the second software specification specified in the second programming language; Receive a third software specification specified in a third programming language different from the first and second programming languages, the third software specification defining one or more data relationships between the first and second software specifications; The first software specification is represented using a fourth programming language that is different from the first, second, and third programming languages; The representation of the second software specification is formed using the fourth programming language; Analyze the third software specification to identify the one or more data relationships; and Forming a combined representation of the first software specification and the second software specification using the fourth programming language includes, based on the identified one or more data relationships, forming a connection between the representation of the first software specification in the fourth programming language and the representation of the second software specification in the fourth programming language. 2. The method according to claim 1, wherein the first programming language is a procedural programming language. 3. The method of claim 2, wherein the fourth programming language enables parallelism between different parts of the software specification. 4. The method of claim 3, wherein the fourth programming language enables multiple types of parallelism, including: The first type of parallelism enables multiple instances of a part of the software specification to operate on different portions of the input data stream; and The second type of parallelism enables different parts of the software specification to be executed simultaneously for different parts of the input data stream. 5. The method according to claim 2, wherein the second programming language is a procedural programming language. 6. The method according to claim 2, wherein the second programming language is the same as the first programming language. 7. The method according to claim 1, wherein the one or more data relationships between the first software specification and the second software specification include at least one data relationship, the at least one data relationship corresponding to the first software specification receiving data from the first dataset and the second software specification providing data to the first dataset. 8. The method according to claim 1, wherein the fourth programming language is a data flow graph-based programming language. 9. The method of claim 8, wherein the connection of the fourth programming language corresponds to a directed link representing a data stream. 10. The method of claim 1, wherein the first software specification is configured to interact with one or more datasets, each dataset having an associated dataset type from a plurality of dataset types of the first software specification, and the second software specification is configured to interact with one or more datasets, each dataset having an associated type from a plurality of dataset types of the second software specification, the method further comprising: Processing the first software specification, the processing includes: Identify one or more datasets of the first software specification, and determine the associated type of each of the identified one or more datasets in the first software specification; and Forming a representation of the first software specification using the fourth programming language includes: forming a specification for each of the one or more identified datasets using the fourth programming language, wherein the specification of the dataset in the fourth programming language has a type corresponding to the associated type of the dataset in the first programming language; The specification of the one or more datasets of the fourth programming language has at least one of the following: an input dataset type or an output dataset type; Processing the second software specification, the processing includes: Identify the one or more datasets of the second software specification, and determine the associated type of the dataset in the second software specification for each of the identified one or more datasets; and Forming a representation of the second software specification using the fourth programming language includes: forming a specification for each of the one or more identified datasets using the fourth programming language, wherein the specification of the dataset in the fourth programming language has a type corresponding to the associated type of the dataset in the first programming language; The specification of the one or more datasets of the fourth programming language enables input or output functionality. 11. The method of claim 10, wherein forming the combined representation comprises at least one of the following: One or more connections are formed to replace the connection between the specification of the input function enabled in one or more datasets of the second software specification of the fourth programming language and the representation of the second software specification of the fourth programming language with the connection between the first software specification of the fourth programming language and the representation of the second software specification of the fourth programming language; or One or more connections are formed to replace the connection between the specification of the one or more datasets that enable input functionality in the first software specification of the fourth programming language and the representation of the second software specification of the fourth programming language with the connection between the second software specification of the fourth programming language and the representation of the first software specification of the fourth programming language. 12. The method of claim 11, further comprising: The first software specification of the fourth programming language stores one or more datasets of the first software specification of the fourth programming language that enable output functionality, or The representation of the second software specification of the fourth programming language that enables output functionality stores one or more datasets of the second software specification of the fourth programming language. 13. The method of claim 10, wherein: The first software specification includes one or more data transformation operations, and analyzing the first software specification includes identifying at least some of the one or more data transformation operations and classifying the identified data transformation operations into corresponding data transformation types of the fourth programming language, and The representation of the first software specification forming the fourth programming language includes: forming a specification for each of the identified data transformation operations of the fourth programming language, wherein the specification of the data transformation operation of the fourth programming language enables a data transformation operation corresponding to the data transformation type of the identified data transformation operation of the first programming language. 14. The method of claim 10, wherein at least one of the specifications of the one or more datasets of the fourth programming language has a read-only random access dataset type. 15. The method of claim 10, wherein determining the associated type of the dataset in the first software specification includes analyzing parameters of the dataset definition and commands that access the dataset. 16. The method of claim 15, wherein the parameters include one or more of the following: file organization associated with the dataset, access mode associated with the dataset, mode for opening the dataset, and input/output operations. 17. The method according to claim 1, further comprising: The combined representation of the first software specification and the second software specification is stored in the storage medium. 18. The method of claim 1, wherein the first software specification defines one or more data processing operations that interact with one or more datasets, and the second software specification defines one or more data processing operations that interact with one or more datasets. 19. The method of claim 18, wherein the third software specification defines one or more data relationships between the one or more datasets of the first software specification and the one or more datasets of the second software specification. 20. Software stored in a non-transitory form on a computer-readable medium for translating software specifications, said software comprising instructions that cause a computing system to perform the following operations: Receive the first software specification specified in the first programming language; Receive the second software specification specified in the second programming language; Receive a third software specification specified in a third programming language different from the first and second programming languages, the third software specification defining one or more data relationships between the first and second software specifications; The first software specification is represented using a fourth programming language that is different from the first, second, and third programming languages; The representation of the second software specification is formed using the fourth programming language; Analyze the third software specification to identify the one or more data relationships; and Forming a combined representation of the first software specification and the second software specification using the fourth programming language includes, based on the identified one or more data relationships, forming a connection between the representation of the first software specification in the fourth programming language and the representation of the second software specification in the fourth programming language. 21. A computational system for translating software specifications, the computational system comprising: An input device or port is configured to receive software specifications, which include: The first software specification is defined using the first programming language; A second software specification defined using a second programming language; A third software specification defined in a third programming language different from the first and second programming languages, the third software specification defining one or more data relationships between the first and second software specifications; and At least one processor configured to process the received software specification, the processing including: The first software specification is represented using a fourth programming language that is different from the first, second, and third programming languages; The representation of the second software specification is formed using the fourth programming language; Analyze the third software specification to identify the one or more data relationships; and Forming a combined representation of the first software specification and the second software specification using the fourth programming language includes, based on the identified one or more data relationships, forming a connection between the representation of the first software specification in the fourth programming language and the representation of the second software specification in the fourth programming language.