[go: up one dir, main page]

TWI860073B - Data processing system - Google Patents

Data processing system Download PDF

Info

Publication number
TWI860073B
TWI860073B TW112133165A TW112133165A TWI860073B TW I860073 B TWI860073 B TW I860073B TW 112133165 A TW112133165 A TW 112133165A TW 112133165 A TW112133165 A TW 112133165A TW I860073 B TWI860073 B TW I860073B
Authority
TW
Taiwan
Prior art keywords
data
database
uploaded
type
processing system
Prior art date
Application number
TW112133165A
Other languages
Chinese (zh)
Other versions
TW202511936A (en
Inventor
陳維超
張明淇
吳卓叡
黃敬倫
Original Assignee
英業達股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 英業達股份有限公司 filed Critical 英業達股份有限公司
Priority to TW112133165A priority Critical patent/TWI860073B/en
Application granted granted Critical
Publication of TWI860073B publication Critical patent/TWI860073B/en
Publication of TW202511936A publication Critical patent/TW202511936A/en

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A data processing method is used for a data processing system processing an upload data. The data processing system comprises a relational database and a NoSQL database, and the upload data comprises a data content and a table name. The data processing method comprises determining a structure type of the upload data according to the data content; determining a first table of a first database according to the structure type, the data content and the table name of the upload data; and storing the upload data in the first table of the first database. The first database is the relational database or the NoSQL database.

Description

資料處理系統 Data processing system

本發明係指一種資料處理系統,尤指一種可根據資料內容自動化進行資料庫選擇、資料結構調整及資料移轉之資料處理系統。 The present invention refers to a data processing system, in particular, a data processing system that can automatically perform database selection, data structure adjustment and data transfer based on data content.

隨著資訊數位化以及網路的普及,各領域都有透過應用程式介面(Application Programming Interface,API)存取資料庫之需求。為了滿足資料應用的多元化,資料庫的種類也隨之蓬勃發展,如今主流的資料庫主要可分為關連性資料庫(Relational Database)以及非關聯性資料庫(NoSQL)兩種。關聯性資料庫如Oracle、MySQL、SQL Server等通常透過多個資料表來存放資料,資料表中的資料以及資料表之間具有預先定義且嚴謹的結構化關係。非關聯性資料庫如MangoDB、Redis等則不同於傳統的關連性資料庫,不須使用固定的資料表格式儲存資料,可隨意定義資料模型,具有更佳的彈性以及可延展性。透過API存取資料庫的技術使得使用者在面對多元的資料庫類型、多樣化的資料庫技術時,可以不用具備與資料庫相關的專業知識,只需根據API之定義即可進行資料庫的存取。 With the digitization of information and the popularization of the Internet, all fields have the need to access databases through application programming interfaces (APIs). In order to meet the diversification of data applications, the types of databases have also flourished. Today, mainstream databases can be mainly divided into two types: relational databases and non-relational databases (NoSQL). Relational databases such as Oracle, MySQL, and SQL Server usually store data through multiple tables. The data in the tables and the tables have pre-defined and strictly structured relationships. Non-relational databases such as MangoDB and Redis are different from traditional relational databases. They do not need to use a fixed table format to store data. Data models can be defined at will, and they have better flexibility and scalability. The technology of accessing databases through APIs allows users to access databases based on the definition of APIs without having to possess database-related expertise when faced with multiple database types and diverse database technologies.

然而,一般而言,使用者在上傳資料前,仍需先與資料庫技術人員進行需求確認,資料庫技術人員則根據使用者的實際需求選擇符合需求的資料 庫、規劃相應的資料關係架構以及建立相應的資料表,此外,還須提供用於存取資料之客製化API。當資料庫與資料表設定完成且儲存一定的資料後,若有後續的資料庫或資料表結構異動之需求發生時,仍須仰賴資料庫技術人員進行資料庫轉移以及資料結構的調整。在資訊量日益龐大的情況下,資料庫技術人員將面臨可觀的人力負擔以及資料庫異動成本。在使用關聯性資料庫的情況下,此問題尤甚嚴重。然而,因應需求變化而變動資料內容一直是本領域難以避免的問題,因此,亟需建立可自動判別之方法,可以根據使用者上傳之資料內容進行自動化的資料庫以及資料表調整,以節省人力、降低成本。 However, generally speaking, before uploading data, users still need to confirm their needs with database technicians. Database technicians will select a database that meets the user's actual needs, plan the corresponding data relational architecture, and establish the corresponding database table. In addition, they must provide a customized API for accessing data. After the database and database table are set up and a certain amount of data is stored, if there is a need to change the database or table structure in the future, database technicians are still required to transfer the database and adjust the data structure. With the increasing amount of information, database technicians will face considerable manpower burdens and database change costs. This problem is particularly serious when using a relational database. However, changing data content in response to changing needs has always been an unavoidable problem in this field. Therefore, it is urgent to establish an automatic identification method that can automatically adjust the database and table based on the data content uploaded by the user to save manpower and reduce costs.

因此,本發明之主要目的即在於提供一種可自動判別資料內容之資料處理系統,藉以改善習知技術之缺點。 Therefore, the main purpose of the present invention is to provide a data processing system that can automatically identify the content of data, so as to improve the shortcomings of the known technology.

本發明實施例揭露一種一種資料處理系統,用於處理一上傳資料,其中該上傳資料包含一資料內容及一資料表名稱。該資料處理系統包含一關聯性資料庫、一非關聯性資料庫、一資料處理單元以及一儲存單元。該資料處理單元耦接於該關聯性資料庫以及該非關聯性資料庫,用來執行一程式碼。該儲存單元耦接於該資料處理單元,用來儲存該程式碼,以指示該處理單元執行一資料處理方法。該資料處理方法包含有根據該資料內容,決定該上傳資料之一資料類型;根據該上傳資料之該資料類型、該資料內容及該資料表名稱,決定一第一資料庫之一第一資料表;以及將該上傳資料儲存於該第一資料庫之該第一資料表。其中,該第一資料庫為該關聯性資料庫或該非關聯性資料庫。 The present invention discloses a data processing system for processing an uploaded data, wherein the uploaded data includes a data content and a data table name. The data processing system includes a relational database, a non-relational database, a data processing unit and a storage unit. The data processing unit is coupled to the relational database and the non-relational database to execute a program code. The storage unit is coupled to the data processing unit to store the program code to instruct the processing unit to execute a data processing method. The data processing method includes determining a data type of the uploaded data according to the data content; determining a first data table of a first database according to the data type, the data content and the data table name of the uploaded data; and storing the uploaded data in the first data table of the first database. The first database is the relational database or the non-relational database.

100:API 100:API

110:使用者 110: User

120:資料庫 120: Database

130:資料 130: Data

20:資料處理系統 20:Data processing system

200:API 200:API

210:關聯式資料庫 210: Relational database

212:非關聯式資料庫 212: Non-relational database

220:資料處理單元 220: Data processing unit

230:儲存單元 230: Storage unit

232:程式碼 232:Program code

240:上傳資料 240: Upload data

30:資料處理處理流程 30: Data processing flow

300~308:步驟 300~308: Steps

40:資料類型判斷流程 40: Data type determination process

400~416:步驟 400~416: Steps

50、60:流程 50, 60: Process

500~510、600~618:步驟 500~510, 600~618: Steps

70:資料庫移轉流程 70: Database migration process

700~710:步驟 700~710: Steps

80:資料表移轉流程 80:Data table migration process

800~810:步驟 800~810: Steps

第1圖為使用者透過API存取資料庫之示意圖。 Figure 1 is a diagram showing a user accessing a database through an API.

第2圖為本發明實施例之一資料處理系統之示意圖。 Figure 2 is a schematic diagram of a data processing system according to one embodiment of the present invention.

第3圖為本發明實施例之一資料處理處理流程示意圖。 Figure 3 is a schematic diagram of the data processing flow of one embodiment of the present invention.

第4圖為本發明實施例之一資料類型判斷流程示意圖。 Figure 4 is a schematic diagram of the data type determination process of one embodiment of the present invention.

第5圖為本發明實施例之一流程示意圖。 Figure 5 is a schematic diagram of the process of one embodiment of the present invention.

第6圖為本發明實施例之一流程示意圖。 Figure 6 is a schematic diagram of the process of one embodiment of the present invention.

第7圖為本發明實施例之一資料庫移轉流程之示意圖。 Figure 7 is a schematic diagram of the database migration process of one embodiment of the present invention.

第8圖為本發明實施例之一資料表移轉流程之示意圖。 Figure 8 is a schematic diagram of the data table transfer process of one embodiment of the present invention.

在說明書及後續的申請專利範圍當中使用了某些詞彙來指稱特定的元件。所屬領域中具有通常知識者應可理解,硬體製造商可能會用不同的名詞來稱呼同一個元件。本說明書及後續的申請專利範圍並不以名稱的差異來做為區分元件的方式,而是以元件在功能上的差異來做為區分的準則。在通篇說明書及後續的申請專利範圍當中所提及的「包含」係為一開放式的用語,故應解釋成「包含但不限定於」。 Certain terms are used in this specification and subsequent patent applications to refer to specific components. A person with ordinary knowledge in the field should understand that hardware manufacturers may use different terms to refer to the same component. This specification and subsequent patent applications do not use differences in names as a way to distinguish components, but rather use differences in components' functions as the criteria for distinction. The term "including" mentioned throughout the specification and subsequent patent applications is an open term and should be interpreted as "including but not limited to".

請參考第1圖,第1圖為使用者透過API存取資料庫之示意圖。如第1圖所示,使用者110透過API 100送出存取資料庫120之請求並取得來自資料庫120的回應。在此過程中,使用者110僅需根據API 100之規範在請求中攜帶需要存放於資料庫120之資料130即可,無需知曉資料庫120的類型,亦無須具備與資料庫120相關的專業知識。然而,一般而言,API 100以及資料庫120需經資料庫技術人員根據使用者需求詳細規劃並實現為便於存取之型式,一旦系統上線使用後 便難於更改。資料130可以是結構化的表格,也可能是各種多媒體資料,更可能是具有任意結構的資料集合,且因應實際需求的改變,資料130的結構與內容更可能隨時需要更新。在此情況下,資料庫技術人員需要通盤考慮資料庫120已儲存的舊有資料結構以及新的資料130的需求,並據此更新資料庫120中的資料表結構、資料表關聯關係等以及API 100之內容,在特定需求下,還需考慮進行不同種類資料庫之間的資料移轉。此更新過程將耗費可觀的人力與物力。 Please refer to Figure 1, which is a schematic diagram of a user accessing a database through an API. As shown in Figure 1, a user 110 sends a request to access a database 120 through an API 100 and obtains a response from the database 120. In this process, the user 110 only needs to carry the data 130 that needs to be stored in the database 120 in the request according to the specification of the API 100, without knowing the type of the database 120 or having professional knowledge related to the database 120. However, in general, the API 100 and the database 120 need to be carefully planned by database technicians according to user needs and implemented in a form that is easy to access. Once the system is online, it is difficult to change. Data 130 may be a structured table, or may be various multimedia data, or may be a data set with any structure. In response to changes in actual needs, the structure and content of data 130 may need to be updated at any time. In this case, database technicians need to comprehensively consider the old data structure stored in database 120 and the needs of new data 130, and update the table structure, table associations, etc. in database 120 and the content of API 100 accordingly. Under specific needs, data migration between different types of databases must also be considered. This update process will consume considerable manpower and material resources.

針對上述問題,請參考第2圖,第2圖為本發明實施例之一資料處理系統20之示意圖。如第2圖所示,資料處理系統20包含一關聯式資料庫210、一非關聯式資料庫212、一資料處理單元220以及一儲存單元230。關聯式資料庫210可以是任意一種關聯式資料庫如Oracle、MySQL、SQL Server等,且不限於此。非關聯式資料庫212可以是任意一種非關聯式資料庫如MangoDB、Redis等,且不限於此。資料處理單元220可為通用處理器、微處理器、特定應用積體電路(Application Specific Integrated Circuit,ASIC)等或其組合。儲存單元230可為任一資料儲存裝置,用來儲存一程式碼232,並透過資料處理單元220讀取及執行程式碼232。舉例來說,儲存單元230可為唯讀式記憶體(ROM)、快閃記憶體(flash memory)、隨機存取記憶體(RAM)、硬碟及光學資料儲存裝置(optical data storage device)及非揮發性儲存單元等,但不限於此。 In response to the above problems, please refer to FIG. 2, which is a schematic diagram of a data processing system 20 of an embodiment of the present invention. As shown in FIG. 2, the data processing system 20 includes a relational database 210, a non-relational database 212, a data processing unit 220, and a storage unit 230. The relational database 210 can be any kind of relational database such as Oracle, MySQL, SQL Server, etc., but is not limited thereto. The non-relational database 212 can be any kind of non-relational database such as MangoDB, Redis, etc., but is not limited thereto. The data processing unit 220 can be a general-purpose processor, a microprocessor, an application-specific integrated circuit (ASIC), etc., or a combination thereof. The storage unit 230 can be any data storage device for storing a program code 232, and the data processing unit 220 reads and executes the program code 232. For example, the storage unit 230 can be a read-only memory (ROM), a flash memory, a random access memory (RAM), a hard disk, an optical data storage device, and a non-volatile storage unit, but is not limited thereto.

如第2圖所示,資料處理系統20提供一API 200供使用者110進行一上傳資料240之存取,上傳資料240至少須包含一資料表名稱以及其資料內容。資料處理系統20根據使用者110所上傳之上傳資料240執行一資料處理方法,據此判別適於存放上傳資料240的資料庫以及資料表等,達成自動化儲存資料之目的。如第3圖所示,本發明實施例之資料處理方法可歸納為一資料處理處理流程30並編 譯為程式碼232,其包含以下步驟: As shown in FIG. 2, the data processing system 20 provides an API 200 for the user 110 to access an uploaded data 240, and the uploaded data 240 must at least include a table name and its data content. The data processing system 20 executes a data processing method based on the uploaded data 240 uploaded by the user 110, and thereby determines the database and table suitable for storing the uploaded data 240, thereby achieving the purpose of automatically storing data. As shown in FIG. 3, the data processing method of the embodiment of the present invention can be summarized as a data processing process 30 and compiled into a program code 232, which includes the following steps:

步驟300:開始。 Step 300: Start.

步驟302:根據上傳資料240的資料內容,決定上傳資料240之一資料類型。 Step 302: Determine a data type of the uploaded data 240 according to the data content of the uploaded data 240.

步驟304:根據上傳資料240之資料類型、資料內容及資料表名稱,決定一第一資料庫之一第一資料表。 Step 304: Determine a first data table of a first database according to the data type, data content and data table name of the uploaded data 240.

步驟306:將上傳資料240的資料內容儲存於第一資料庫之第一資料表。 Step 306: Store the data content of the uploaded data 240 in the first data table of the first database.

步驟308:結束。 Step 308: End.

根據流程30,於步驟302中,資料處理系統20首先根據上傳資料240的資料內容決定其資料類型。接著,於步驟304中,資料處理系統20根據上傳資料240之資料表名稱搜尋關聯式資料庫210以及非關聯式資料庫212,以判斷是否已有相關資料儲存於資料處理系統20中,並據此決定用於最終儲存上傳資料240的第一資料庫以及第一資料表。最後,於步驟306中,資料處理系統20將上傳資料240的資料內容儲存於步驟304中決定的第一資料庫之第一資料表中。 According to process 30, in step 302, the data processing system 20 first determines the data type of the uploaded data 240 according to the data content. Then, in step 304, the data processing system 20 searches the relational database 210 and the non-relational database 212 according to the table name of the uploaded data 240 to determine whether relevant data has been stored in the data processing system 20, and accordingly determines the first database and the first table for finally storing the uploaded data 240. Finally, in step 306, the data processing system 20 stores the data content of the uploaded data 240 in the first table of the first database determined in step 304.

詳細來說,於步驟302中,資料處理系統20首先根據上傳資料240的資料內容的結構決定其資料類型。上傳資料240的資料類型可分為兩種:結構性資料類型以及非結構性資料類型。結構性資料類型的資料適於存放於關聯式資料庫210中,而非結構性資料類型的資料則適於存放於非關聯式資料庫212中,本發明實施例據此資訊進行進一步地判別以決定上傳資料240最終的存放方式。在一實施例中,資料處理系統20在接收上傳資料240的同時,可以接收由使用者110所指定的一資料庫指定參數。資料庫指定參數可以指定資料存放於關聯式資料 庫210或非關聯式資料庫212中。請參考第4圖,第4圖為於步驟302進行資料類型判斷之一資料類型判斷流程40,其包含以下步驟: Specifically, in step 302, the data processing system 20 first determines the data type of the uploaded data 240 according to the structure of the data content. The data type of the uploaded data 240 can be divided into two types: structured data type and unstructured data type. Data of structured data type is suitable for storage in the relational database 210, while data of unstructured data type is suitable for storage in the non-relational database 212. The embodiment of the present invention further determines based on this information to determine the final storage method of the uploaded data 240. In one embodiment, the data processing system 20 can receive a database specified parameter specified by the user 110 while receiving the uploaded data 240. The database-specified parameter can specify that the data is stored in a relational database 210 or a non-relational database 212. Please refer to FIG. 4, which is a data type determination process 40 for performing data type determination in step 302, which includes the following steps:

步驟400:開始。 Step 400: Start.

步驟402:判斷資料庫指定參數是否為真?若是,執行步驟406;若否,執行步驟404。 Step 402: Determine whether the database specified parameter is true? If so, execute step 406; if not, execute step 404.

步驟404:判斷上傳資料240的資料內容是否滿足一第一判斷條件?若是,執行步驟410;若否,執行步驟414。 Step 404: Determine whether the data content of the uploaded data 240 meets a first determination condition? If yes, execute step 410; if no, execute step 414.

步驟406:判斷資料庫指定參數指示使用關聯式資料庫或非關聯式資料庫?若使用關聯式資料庫,執行步驟408;若使用非關聯式資料庫,執行步驟414。 Step 406: Determine whether the database-specified parameter indicates the use of a relational database or a non-relational database? If a relational database is used, execute step 408; if a non-relational database is used, execute step 414.

步驟408:判斷上傳資料240的資料內容是否滿足一第一判斷條件?若是,執行步驟410;若否,執行步驟412。 Step 408: Determine whether the data content of the uploaded data 240 meets a first determination condition? If yes, execute step 410; if no, execute step 412.

步驟410:決定上傳資料240的資料類型為結構性資料類型。 Step 410: Determine that the data type of the uploaded data 240 is a structured data type.

步驟412:回傳「資料結構不符合」之錯誤訊息。 Step 412: Return an error message of "data structure does not match".

步驟414:決定上傳資料240的資料類型為非結構性資料類型。 Step 414: Determine that the data type of the uploaded data 240 is an unstructured data type.

步驟416:結束。 Step 416: End.

根據流程40,資料處理系統20可根據上傳資料240的資料內容以及資料庫指定參數決定上傳資料240儲存於資料處理系統20之資料結構。資料處理系統20首先判斷使用者110是否有設定資料庫指定參數(步驟402)。使用者110若無上傳資料庫指定參數,資料處理系統20根據上傳資料240的資料內容是否滿足一第一判斷條件,自動化決定上傳資料240的資料類型(步驟404)。當滿足第一判斷條件時,資料處理系統20決定上傳資料240的資料類型為結構性資料類型(步驟410);當不滿足第一判斷條件時,資料處理系統20決定上傳資料240的資料類 型為非結構性資料類型(步驟414)。另一方面,當使用者110有上傳資料庫指定參數(資料庫指定參數值為真),需根據資料庫指定參數以及上傳資料240的資料內容進一步判斷上傳資料240的資料類型(步驟406)。若資料庫指定參數指示使用關聯式資料庫時,需判斷上傳資料240的資料內容是否滿足第一判斷條件(步驟408)。當上傳資料240的資料內容滿足第一判斷條件時,資料處理系統20決定上傳資料240的資料類型為結構性資料類型(步驟410);當上傳資料240的資料內容不滿足第一判斷條件時,資料處理系統20需回傳一錯誤訊息給使用者110並提示其指定之資料庫類型與資料結構不符合(步驟412)。若資料庫指定參數指示使用非關聯式資料庫時,資料處理系統20可直接決定上傳資料240的資料類型為非結構性資料類型(步驟414)。 According to the process 40, the data processing system 20 can determine the data structure of the uploaded data 240 stored in the data processing system 20 according to the data content of the uploaded data 240 and the database specified parameters. The data processing system 20 first determines whether the user 110 has set the database specified parameters (step 402). If the user 110 has not uploaded the database specified parameters, the data processing system 20 automatically determines the data type of the uploaded data 240 according to whether the data content of the uploaded data 240 meets a first determination condition (step 404). When the first judgment condition is met, the data processing system 20 determines that the data type of the uploaded data 240 is a structured data type (step 410); when the first judgment condition is not met, the data processing system 20 determines that the data type of the uploaded data 240 is an unstructured data type (step 414). On the other hand, when the user 110 has a database-specified parameter for uploading (the database-specified parameter value is true), the data type of the uploaded data 240 needs to be further determined based on the database-specified parameter and the data content of the uploaded data 240 (step 406). If the database-specified parameter indicates that a relational database is used, it is necessary to determine whether the data content of the uploaded data 240 meets the first judgment condition (step 408). When the data content of the uploaded data 240 meets the first judgment condition, the data processing system 20 determines that the data type of the uploaded data 240 is a structured data type (step 410); when the data content of the uploaded data 240 does not meet the first judgment condition, the data processing system 20 needs to return an error message to the user 110 and prompt that the specified database type does not match the data structure (step 412). If the database specified parameter indicates the use of a non-relational database, the data processing system 20 can directly determine that the data type of the uploaded data 240 is a non-structured data type (step 414).

在流程40中,資料處理系統20係根據上傳資料240的資料內容是否滿足第一判斷條件,以判斷上傳資料240的資料格式與資料類型。在一實施例中,第一判斷條件可包括上傳資料240的資料內容結構,由包含欄位(column)名稱以及代表資料數值組的列(row)所組成,且其中每一欄位所對應資料的所有資料型態(data type)皆必須一致,以及每一欄位之資料型態皆為浮點數(包含整數)(Float)、日期時間(Datetime)以及字串(String)之一。當滿足以上條件時,資料處理系統20可判斷上傳資料240的資料內容為一結構性資料類型,否則為非結構性資料類型。相較於非結構性資料類型,結構性資料類型具有嚴格的格式限制,可存放於關聯式資料庫210或非關聯式資料庫212中。相對於結構性資料類型,非結構性資料類型具有較鬆散的結構,只能存放於非關聯式資料庫212中。當使用者110未設置資料庫指定參數時,資料處理系統20可直接根據第一判斷條件決定上傳資料240的資料類型。當使用者110有設置資料庫指定參數時,則必須同時考慮所指定資料庫類型以及上傳資料240的資料內容是否滿足第一判斷條 件。上傳資料240的資料內容若無法滿足第一判斷條件時,無法存放於資料結構較為嚴謹的關聯式資料庫210。因此,若使用者110於此條件下指定使用關聯式資料庫210時,資料處理系統20需回傳錯誤訊息並結束流程。 In the process 40, the data processing system 20 determines the data format and data type of the uploaded data 240 according to whether the data content of the uploaded data 240 satisfies the first judgment condition. In one embodiment, the first judgment condition may include the data content structure of the uploaded data 240, which is composed of a column name and a row representing a data value group, and all data types of the data corresponding to each column must be consistent, and the data type of each column is one of a floating point number (including an integer) (Float), a date and time (Datetime), and a string (String). When the above conditions are met, the data processing system 20 can determine that the data content of the uploaded data 240 is a structured data type, otherwise it is an unstructured data type. Compared with the unstructured data type, the structured data type has strict format restrictions and can be stored in the relational database 210 or the non-relational database 212. Compared with the structured data type, the non-structured data type has a looser structure and can only be stored in the non-relational database 212. When the user 110 does not set the database specified parameter, the data processing system 20 can directly determine the data type of the uploaded data 240 according to the first judgment condition. When the user 110 sets the database specification parameter, the specified database type and the data content of the uploaded data 240 must be considered at the same time to see if they meet the first judgment condition. If the data content of the uploaded data 240 cannot meet the first judgment condition, it cannot be stored in the relational database 210 with a more stringent data structure. Therefore, if the user 110 specifies to use the relational database 210 under this condition, the data processing system 20 needs to return an error message and terminate the process.

關於第一判斷條件中判別資料型態的方式,可採用現行軟體或套件,且不限於此。舉例來說,採用Python進行資料型態的判別時,可使用”dateutil.parser.parse”套件檢查日期時間類型,使用”float()”函式檢查浮點數類型,以及使用”isinstance(input,str)”函式檢查字串類型。採用R語言進行資料型態的判別時,可使用”as.Date”檢查日期時間類型,使用”float.TryParse(input,out valid_float)”檢查浮點數類型,以及使用”is.character()”檢查字串類型。本領域具通常知識者當可視實際需求使用合適之工具進行資料型態的判別,且不限於此。 Regarding the method of determining the data type in the first judgment condition, existing software or packages can be used, but are not limited to this. For example, when using Python to determine the data type, the "dateutil.parser.parse" package can be used to check the date and time type, the "float()" function can be used to check the floating point type, and the "isinstance(input,str)" function can be used to check the string type. When using R language to determine the data type, "as.Date" can be used to check the date and time type, "float.TryParse(input,out valid_float)" can be used to check the floating point type, and "is.character()" can be used to check the string type. People with ordinary knowledge in this field can use appropriate tools to determine the data type according to actual needs, but are not limited to this.

根據流程30,於步驟304中,資料處理系統20根據上傳資料240之資料類型、資料內容及資料表名稱,決定用於儲存上傳資料240的第一資料庫以及第一資料表。於此步驟中,資料處理系統20首先必須根據上傳資料240之資料表名稱搜尋關聯式資料庫210以及非關聯式資料庫212以判斷是否已有相關資料儲存於資料處理系統20中,並據此決定用於最終儲存上傳資料240的第一資料庫以及第一資料表。 According to process 30, in step 304, the data processing system 20 determines the first database and the first data table for storing the uploaded data 240 according to the data type, data content and data table name of the uploaded data 240. In this step, the data processing system 20 must first search the relational database 210 and the non-relational database 212 according to the data table name of the uploaded data 240 to determine whether there is related data stored in the data processing system 20, and accordingly determine the first database and the first data table for finally storing the uploaded data 240.

請參考第5圖,第5圖繪示當資料處理系統20未於關聯式資料庫210以及非關聯式資料庫212中搜尋到與上傳資料240之資料表名稱相符合的資料表時,根據上傳資料240之資料類型決定用於實際儲存上傳資料240的第一資料庫以及第一資料表之一流程50。在此情形下,資料處理系統20中未存放與上傳資料240 相關之資料,可直接根據步驟302中資料類型判斷流程40所決定的上傳資料240之資料類型自動化決定第一資料庫以及第一資料表。流程50包含以下步驟: Please refer to Figure 5, which shows a process 50 for determining a first database and a first data table for actually storing the uploaded data 240 according to the data type of the uploaded data 240 when the data processing system 20 fails to find a data table with a data table name matching the uploaded data 240 in the relational database 210 and the non-relational database 212. In this case, the data processing system 20 does not store data related to the uploaded data 240, and the first database and the first data table can be automatically determined directly according to the data type of the uploaded data 240 determined by the data type determination process 40 in step 302. Process 50 includes the following steps:

步驟500:開始。 Step 500: Start.

步驟502:判斷上傳資料240之資料類型。若為結構性資料類型,執行步驟504;若為非結構性資料類型,執行步驟506。 Step 502: Determine the data type of the uploaded data 240. If it is a structured data type, execute step 504; if it is an unstructured data type, execute step 506.

步驟504:決定第一資料庫為關聯式資料庫210。 Step 504: Determine that the first database is the relational database 210.

步驟506:決定第一資料庫為非關聯式資料庫212。 Step 506: Determine that the first database is a non-relational database 212.

步驟508:於第一資料庫建立具有上傳資料240之資料表名稱之第一資料表。 Step 508: Create a first data table with the data table name of the uploaded data 240 in the first database.

步驟510:結束。 Step 510: End.

根據流程50,資料處理系統20需根據步驟302中上傳資料240之資料類型決定第一資料庫(步驟502)。當上傳資料240之資料類型為結構性資料類型時,決定儲存上傳資料240的第一資料庫為關聯式資料庫210(步驟504)。當上傳資料240之資料類型為非結構性資料類型時,決定儲存上傳資料240的第一資料庫為非關聯式資料庫212(步驟506)。最後,資料處理系統20須於步驟504、506中決定的第一資料庫,根據上傳資料240之資料表名稱建立第一資料表,用以存放上傳資料240的資料內容。 According to the process 50, the data processing system 20 needs to determine the first database according to the data type of the uploaded data 240 in step 302 (step 502). When the data type of the uploaded data 240 is a structured data type, the first database storing the uploaded data 240 is determined to be the relational database 210 (step 504). When the data type of the uploaded data 240 is an unstructured data type, the first database storing the uploaded data 240 is determined to be the non-relational database 212 (step 506). Finally, the data processing system 20 must establish a first data table in the first database determined in steps 504 and 506 according to the data table name of the uploaded data 240 to store the data content of the uploaded data 240.

請參考第6圖,第6圖繪示當資料處理系統20於關聯式資料庫210以及非關聯式資料庫212中搜尋到與上傳資料240之資料表名稱相符合的資料表時,決定用於實際儲存上傳資料240的第一資料庫以及第一資料表之一流程60。假設處理系統20於一第二資料庫中找到與上傳資料240之資料表名稱相符合的一第二資料表。在此情形下,資料處理系統20需考慮第二資料庫、第二資料表以及上傳 資料240之相容性,決定第一資料庫以及第一資料表。流程60包含以下步驟: Please refer to Figure 6, which shows a process 60 for determining a first database and a first data table for actually storing the uploaded data 240 when the data processing system 20 searches for a data table that matches the data table name of the uploaded data 240 in the relational database 210 and the non-relational database 212. Assume that the processing system 20 finds a second data table that matches the data table name of the uploaded data 240 in a second database. In this case, the data processing system 20 needs to consider the compatibility of the second database, the second data table and the uploaded data 240 to determine the first database and the first data table. Process 60 includes the following steps:

步驟600:開始。 Step 600: Start.

步驟602:判斷上傳資料240之資料類型。若為結構性資料類型,執行步驟604;若為非結構性資料類型,執行步驟606。 Step 602: Determine the data type of the uploaded data 240. If it is a structured data type, execute step 604; if it is an unstructured data type, execute step 606.

步驟604:判斷第二資料庫為關聯式資料庫210或非關聯式資料庫212。若為關聯式資料庫210,執行步驟608:若為非關聯式資料庫212,則執行步驟610。 Step 604: Determine whether the second database is a relational database 210 or a non-relational database 212. If it is a relational database 210, execute step 608: If it is a non-relational database 212, execute step 610.

步驟606:判斷第二資料庫為關聯式資料庫210或非關聯式資料庫212。若為關聯式資料庫210,執行步驟612:若為非關聯式資料庫212,則執行步驟610。 Step 606: Determine whether the second database is a relational database 210 or a non-relational database 212. If it is a relational database 210, execute step 612: If it is a non-relational database 212, execute step 610.

步驟608:判斷上傳資料240的資料內容與第二資料表是否相容?若是,執行步驟616;若否,則執行步驟614。 Step 608: Determine whether the data content of the uploaded data 240 is compatible with the second data table? If yes, execute step 616; if not, execute step 614.

步驟610:決定第一資料庫為非關聯式資料庫212,第一資料表為第二資料表。 Step 610: Determine that the first database is a non-relational database 212 and the first data table is a second data table.

步驟612:決定第一資料庫為非關聯式資料庫212,進行一資料庫移轉流程。 Step 612: Determine that the first database is a non-relational database 212 and perform a database migration process.

步驟614:決定第一資料庫為關聯式資料庫210,進行一資料表移轉流程。 Step 614: Determine that the first database is the relational database 210 and perform a table migration process.

步驟616:決定第一資料庫為關聯式資料庫210,第一資料表為第二資料表。 Step 616: Determine that the first database is the relational database 210 and the first data table is the second data table.

步驟618:結束。 Step 618: End.

根據流程60,資料處理系統20需根據上傳資料240之資料類型以及第二資料庫的類型,判斷以及決定第一資料庫以及第一資料表(步驟602~606)。 當第二資料庫為非關聯式資料庫212時,無論上傳資料240之資料類型是否為結構性資料,上傳資料皆須存放於非關聯式資料庫212。在此情況下,資料處理系統20決定第一資料庫為非關聯式資料庫212,第一資料表為第二資料表(步驟610)。也就是說,上傳資料240可直接存放於現存的第二資料庫中的第二資料表中。 According to process 60, the data processing system 20 needs to judge and determine the first database and the first data table according to the data type of the uploaded data 240 and the type of the second database (steps 602-606). When the second database is a non-relational database 212, regardless of whether the data type of the uploaded data 240 is structured data, the uploaded data must be stored in the non-relational database 212. In this case, the data processing system 20 determines that the first database is a non-relational database 212 and the first data table is a second data table (step 610). In other words, the uploaded data 240 can be directly stored in the second data table in the existing second database.

更進一步地,當第二資料庫為關聯式資料庫210時,資料處理系統20需要根據上傳資料240的資料類型判斷上傳資料240是否適於與第二資料表的資料內容一同儲存於關聯式資料庫210中。舉例來說,當第二資料庫為關聯式資料庫210且上傳資料240的資料類型為非結構性資料類型時,關聯式資料庫210將無法儲存上傳資料240中的非結構性資料。據此,資料處理系統20需先將第二資料表的資料由關聯式資料庫210遷移至非關聯式資料庫212,再進行上傳資料240的資料儲存。因此,資料處理系統20決定第一資料庫為非關聯式資料庫212,並進行一資料庫移轉流程(步驟612)。 Furthermore, when the second database is a relational database 210, the data processing system 20 needs to determine whether the uploaded data 240 is suitable for being stored in the relational database 210 together with the data content of the second table according to the data type of the uploaded data 240. For example, when the second database is a relational database 210 and the data type of the uploaded data 240 is a non-structured data type, the relational database 210 will not be able to store the non-structured data in the uploaded data 240. Accordingly, the data processing system 20 needs to first migrate the data of the second table from the relational database 210 to the non-relational database 212, and then store the uploaded data 240. Therefore, the data processing system 20 determines that the first database is a non-relational database 212 and performs a database migration process (step 612).

另一方面,當第二資料庫為關聯式資料庫210且上傳資料240的資料類型為結構性資料類型時,上傳資料240將可以儲存於關聯式資料庫210中。由於關聯式資料庫對於資料的儲存方式有嚴格的資料庫綱要(schema)定義,因此,資料處理系統20需要先檢驗第二資料表以及上傳資料240的資料內容是否相容,不能直接於第二資料表儲存上傳資料240(步驟608)。若上傳資料240的資料內容的所有欄位及其資料型態皆與該第二資料表相容或相符合時,資料處理系統20可直接於第二資料表儲存上傳資料240的資料內容。因此,資料處理系統20決定第一資料庫為關聯式資料庫210,且第一資料表即為第二資料表(步驟616)。若相較於第二資料表,上傳資料240的資料內容除包含第二資料表的所有欄位外還有新增欄位時,上傳資料240無法直接儲存於第二資料表中。在此情況下,資料 處理系統20需先進行資料表欄位的擴充,再進行上傳資料240的資料儲存。因此,資料處理系統20決定第一資料庫為關聯式資料庫210,並進行一資料表移轉流程(步驟614)。 On the other hand, when the second database is a relational database 210 and the data type of the uploaded data 240 is a structured data type, the uploaded data 240 can be stored in the relational database 210. Since the relational database has a strict database schema definition for the storage method of data, the data processing system 20 needs to first check whether the data content of the second data table and the uploaded data 240 is compatible, and cannot directly store the uploaded data 240 in the second data table (step 608). If all fields of the data content of the uploaded data 240 and their data types are compatible or consistent with the second data table, the data processing system 20 can directly store the data content of the uploaded data 240 in the second data table. Therefore, the data processing system 20 determines that the first database is the relational database 210, and the first data table is the second data table (step 616). If the data content of the uploaded data 240 includes all the fields of the second data table and newly added fields compared to the second data table, the uploaded data 240 cannot be directly stored in the second data table. In this case, the data processing system 20 needs to expand the data table fields first, and then store the uploaded data 240. Therefore, the data processing system 20 determines that the first database is the relational database 210, and performs a data table migration process (step 614).

關於步驟612中,資料庫移轉流程請參考第7圖。第7圖為本發明實施例之一資料庫移轉流程70之示意圖,資料庫移轉流程70包含以下步驟: For the database migration process in step 612, please refer to Figure 7. Figure 7 is a schematic diagram of a database migration process 70 of an embodiment of the present invention. The database migration process 70 includes the following steps:

步驟700:開始。 Step 700: Start.

步驟702:匯出第二資料表之資料為一暫存資料。 Step 702: Export the data of the second data table as temporary data.

步驟704:刪除第二資料表。 Step 704: Delete the second data table.

步驟706:於非關聯性資料庫212建立具有上傳資料240之資料表名稱之第一資料表。 Step 706: Create a first table with the table name of the uploaded data 240 in the non-relational database 212.

步驟708:匯入暫存資料至第一資料表。 Step 708: Import temporary data into the first data table.

步驟710:結束。 Step 710: End.

根據資料庫移轉流程70,資料處理系統20將第二資料表的資料內容匯出並暫存為一暫存資料後(步驟702),即可刪除第二資料表(步驟704)。接著,資料處理系統20即可於非關聯性資料庫212建立具有上傳資料240之資料表名稱的第一資料表(步驟706)。最後,資料處理系統20再將暫存資料匯入新建的第一資料表中(步驟708)。據此,即可完成第二資料表的跨資料庫的資料移轉,並可於非關聯性資料庫212建立可與上傳資料240相容之第一資料表。 According to the database migration process 70, after the data processing system 20 exports the data content of the second data table and temporarily stores it as a temporary data (step 702), the second data table can be deleted (step 704). Then, the data processing system 20 can create a first data table with the data table name of the uploaded data 240 in the non-relational database 212 (step 706). Finally, the data processing system 20 imports the temporary data into the newly created first data table (step 708). Accordingly, the cross-database data migration of the second data table can be completed, and a first data table compatible with the uploaded data 240 can be created in the non-relational database 212.

在一實施例中,資料處理系統20在接收上傳資料240的同時,可以同時接收由使用者110所指定的一資料庫移轉參數。當資料庫移轉參數之值為真時,無論上傳資料240是否為非結構性資料類型,資料處理系統20皆可執行資料 庫移轉流程70以將儲存於關聯性資料庫210中的第二資料表資料轉移至非關聯性資料庫212中新建立之第一資料表。在使用者110未設定資料庫移轉參數的情形下,資料處理系統20可根據流程60進行判別以決定資料的處理方式。 In one embodiment, the data processing system 20 can simultaneously receive a database migration parameter specified by the user 110 while receiving the uploaded data 240. When the value of the database migration parameter is true, the data processing system 20 can execute the database migration process 70 to migrate the second table data stored in the relational database 210 to the newly created first table in the non-relational database 212 regardless of whether the uploaded data 240 is a non-structural data type. In the case where the user 110 does not set the database migration parameter, the data processing system 20 can make a judgment according to the process 60 to determine the data processing method.

關於步驟614中的資料表移轉流程請參考第8圖。第8圖為本發明實施例之一資料表移轉流程80之示意圖,資料表移轉流程80包含以下步驟: Please refer to Figure 8 for the data table transfer process in step 614. Figure 8 is a schematic diagram of a data table transfer process 80 of an embodiment of the present invention. The data table transfer process 80 includes the following steps:

步驟800:開始。 Step 800: Start.

步驟802:匯出第二資料表之資料為一暫存資料。 Step 802: Export the data of the second data table as temporary data.

步驟804:刪除第二資料表。 Step 804: Delete the second data table.

步驟806:根據上傳資料240之資料內容以及第二資料表之資料內容,於關聯性資料庫210建立具有上傳資料240之資料表名稱之第一資料表。 Step 806: Based on the data content of the uploaded data 240 and the data content of the second data table, a first data table with the data table name of the uploaded data 240 is created in the relational database 210.

步驟808:匯入暫存資料至第一資料表,其中新增欄位之資料設為一空值。 Step 808: Import the temporary data into the first data table, where the data of the newly added field is set to a null value.

步驟810:結束。 Step 810: End.

根據資料表移轉流程80,資料處理系統20將第二資料表的資料內容匯出並暫存為一暫存資料後(步驟802),即可刪除第二資料表(步驟804)。接著,資料處理系統20可根據上傳資料240以及暫存資料的資料結構,於關聯性資料庫210建立具有上傳資料240之資料表名稱的第一資料表(步驟806)。最後,資料處理系統20再將暫存資料匯入新建的第一資料表中(步驟808)。據此,即可完成第二資料表的資料移轉,並建立可與上傳資料240相容之第一資料表。 According to the data table migration process 80, after the data processing system 20 exports the data content of the second data table and temporarily stores it as a temporary data (step 802), the second data table can be deleted (step 804). Then, the data processing system 20 can create a first data table with the data table name of the uploaded data 240 in the associative database 210 according to the uploaded data 240 and the data structure of the temporary data (step 806). Finally, the data processing system 20 imports the temporary data into the newly created first data table (step 808). In this way, the data migration of the second data table can be completed, and a first data table compatible with the uploaded data 240 can be established.

據此,根據步驟304中的流程50、60、資料庫移轉流程70以及資料表移轉流程80,資料處理系統20可自動判別上傳資料240之資料類型並根據資料處 理系統20中的既存資料決定適合的資料庫以及資料表,減少因資料結構改變造成的人力、物力負擔。 Accordingly, according to the processes 50, 60, the database migration process 70 and the table migration process 80 in step 304, the data processing system 20 can automatically identify the data type of the uploaded data 240 and determine the appropriate database and table according to the existing data in the data processing system 20, thereby reducing the burden of manpower and material resources caused by the change of data structure.

最後,於步驟306中,資料處理系統20即可根據步驟304中決定的第一資料庫以及第一資料表,儲存上傳資料240。 Finally, in step 306, the data processing system 20 can store the uploaded data 240 according to the first database and the first data table determined in step 304.

綜上所述,本發明提供一種資料處理系統以及資料處理方法,使得使用者在上傳資料前無須與資料庫技術人員反覆確認需求,當上傳資料之有任何異動時,不須耗費龐大的人力成本與時間進行資料庫架構的調整、資料遷移以及API修正。因此,本發明可改善習知技術之缺點,並滿足於實際應用之需求。 In summary, the present invention provides a data processing system and a data processing method, so that users do not need to repeatedly confirm their needs with database technicians before uploading data. When there are any changes in the uploaded data, there is no need to spend huge manpower costs and time to adjust the database architecture, migrate data, and modify the API. Therefore, the present invention can improve the shortcomings of the known technology and meet the needs of practical applications.

以上所述僅為本發明之較佳實施例,凡依本發明申請專利範圍所做之均等變化與修飾,皆應屬本發明之涵蓋範圍。 The above is only the preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the patent application of the present invention shall fall within the scope of the present invention.

20:資料處理系統 20:Data processing system

200:API 200:API

210:關聯式資料庫 210: Relational database

212:非關聯式資料庫 212: Non-relational database

220:資料處理單元 220: Data processing unit

230:儲存單元 230: Storage unit

232:程式碼 232:Program code

240:上傳資料 240: Upload data

110:使用者 110: User

Claims (8)

一種資料處理系統,用於處理一上傳資料,其中該上傳資料包含一資料內容及一資料表名稱,該資料處理系統包含:一關聯性資料庫;一非關聯性資料庫;一資料處理單元,耦接於該關聯性資料庫以及該非關聯性資料庫,用來執行一程式碼;以及一儲存單元,耦接於該資料處理單元,用來儲存該程式碼,以指示該處理單元執行一資料處理方法,該資料處理方法包含有:根據該資料內容,決定該上傳資料之一資料類型;根據該上傳資料之該資料類型、該資料內容及該資料表名稱,決定一第一資料庫之一第一資料表;以及將該上傳資料儲存於該第一資料庫之該第一資料表;其中,該第一資料庫為該關聯性資料庫或該非關聯性資料庫;其中該資料類型分為一結構性資料類型以及一非結構性資料類型;其中根據該資料內容決定該上傳資料之該資料類型之步驟,包含當該上傳資料之該資料內容滿足一第一判斷條件時,決定該上傳資料之該資料類型為該結構性資料類型,否則決定該上傳資料之該資料類型為該非結構性資料類型,其中該第一判斷條件包含:該資料內容包含至少一個欄位;該至少一個欄位之每一欄位之對應資料的資料型態皆一致;以及該至少一個欄位之每一欄位之資料型態皆為浮點數、整數、日期時間以及字串之一。 A data processing system is provided for processing an uploaded data, wherein the uploaded data includes a data content and a data table name, and the data processing system includes: a relational database; a non-relational database; a data processing unit coupled to the relational database and the non-relational database, and used to execute a program code; and a storage unit coupled to the data processing unit, and used to store the program code to instruct the processing unit to execute a data processing method, wherein the data processing method includes: determining a data type of the uploaded data according to the data content; determining a first data table of a first database according to the data type, the data content and the data table name of the uploaded data; and storing the uploaded data in the first data table of the first database. wherein the first database is the associative database or the non-associative database; wherein the data type is divided into a structured data type and an unstructured data type; wherein the step of determining the data type of the uploaded data according to the data content comprises determining the data type of the uploaded data to be the structured data type when the data content of the uploaded data satisfies a first judgment condition. structured data type, otherwise the data type of the uploaded data is determined to be the unstructured data type, wherein the first judgment condition includes: the data content includes at least one field; the data type of the corresponding data of each field of the at least one field is consistent; and the data type of each field of the at least one field is one of a floating point number, an integer, a date and time, and a string. 如請求項1所述之資料處理系統,其中該資料處理系統另接收由一使用者指定之一資料庫指定參數,並根據該資料庫指定參數決定該資料類型,其中根據該資料內容決定該上傳資料之該資料類型之步驟另包括:響應於該資料庫指定參數指示使用該關聯式資料庫且該上傳資料之該資料內容滿足該第一判斷條件,決定該上傳資料之該資料類型為該結構性資料類型;響應於該資料庫指定參數指示使用該關聯式資料庫且該上傳資料之該資料類型不滿足該第一判斷條件,該資料處理系統回傳一錯誤訊息給該使用者;以及響應於該資料庫指定參數指示使用該非關聯式資料庫,決定該上傳資料之該資料類型為該非結構性資料類型。 The data processing system as described in claim 1, wherein the data processing system further receives a database-specified parameter specified by a user and determines the data type according to the database-specified parameter, wherein the step of determining the data type of the uploaded data according to the data content further includes: in response to the database-specified parameter indicating the use of the associative database and the data content of the uploaded data satisfying the first judgment condition, determining The data type of the uploaded data is determined to be the structured data type; in response to the database specified parameter indicating the use of the relational database and the data type of the uploaded data not satisfying the first judgment condition, the data processing system returns an error message to the user; and in response to the database specified parameter indicating the use of the non-relational database, the data type of the uploaded data is determined to be the unstructured data type. 如請求項1所述之資料處理系統,其中根據該上傳資料之該資料類型、該資料內容及該資料表名稱決定該第一資料庫之該第一資料表之步驟,包含:判斷該資料處理系統中是否儲存具有該資料表名稱之一第二資料表,其中該第二資料表儲存於該資料處理系統之一第二資料庫;響應於該第二資料表不存在,根據該上傳資料之該資料類型決定該第一資料庫,以及於該第一資料庫建立具有該資料表名稱之該第一資料表;以及響應於該第二資料表已存在,根據該上傳資料以及該第二資料表,決定該第一資料庫以及該第一資料表。 The data processing system as described in claim 1, wherein the step of determining the first data table of the first database according to the data type, the data content and the data table name of the uploaded data comprises: determining whether a second data table with the data table name is stored in the data processing system, wherein the second data table is stored in a second database of the data processing system; in response to the second data table not existing, determining the first database according to the data type of the uploaded data, and establishing the first data table with the data table name in the first database; and in response to the second data table already existing, determining the first database and the first data table according to the uploaded data and the second data table. 如請求項3所述之資料處理系統,其中響應於該第二資料表不存在,根據該上傳資料之該資料類型決定該第一資料庫之步驟,包含: 響應於該上傳資料之該資料類型為結構性資料類型,決定該第一資料庫為該關聯性資料庫;以及響應於該上傳資料之該資料類型為非結構性資料類型,決定該第一資料庫為該非關聯性資料庫。 The data processing system as described in claim 3, wherein in response to the second data table not existing, the step of determining the first database according to the data type of the uploaded data comprises: In response to the data type of the uploaded data being a structured data type, determining the first database to be the associative database; and in response to the data type of the uploaded data being a non-structured data type, determining the first database to be the non-associative database. 如請求項3所述之資料處理系統,其中響應於該第二資料表已存在,根據該上傳資料以及該第二資料表,決定該第一資料庫以及該第一資料表之步驟,包含:響應於該上傳資料之該資料類型為該結構性資料類型且該第二資料庫為該關聯性資料庫,根據該上傳資料之該資料內容以及該第二資料表決定該第一資料庫以及該第一資料表;響應於該上傳資料之該資料類型為該結構性資料類型且該第二資料庫為該非關聯性資料庫,決定該第一資料庫為該非關聯性資料庫,以及決定該第一資料表為該第二資料表;響應於該上傳資料之該資料類型為該非結構性資料類型且該第二資料庫為該關聯性資料庫,決定該第一資料庫為該非關聯性資料庫,以及進行一資料庫移轉;以及響應於該上傳資料之該資料類型為該非結構性資料類型且該第二資料庫為該非關聯性資料庫,決定該第一資料庫為該非關聯性資料庫,以及決定該第一資料表為該第二資料表。 The data processing system as described in claim 3, wherein in response to the second data table existing, the step of determining the first database and the first data table according to the uploaded data and the second data table comprises: in response to the data type of the uploaded data being the structural data type and the second database being the associative database, determining the first database and the first data table according to the data content of the uploaded data and the second data table; in response to the data type of the uploaded data being the structural data type and the second database being the non-associative database , determining the first database to be the non-relational database, and determining the first data table to be the second data table; in response to the data type of the uploaded data being the non-structured data type and the second database being the relational database, determining the first database to be the non-relational database, and performing a database migration; and in response to the data type of the uploaded data being the non-structured data type and the second database being the non-relational database, determining the first database to be the non-relational database, and determining the first data table to be the second data table. 如請求項5所述之資料處理系統,其中響應於該上傳資料之該資料類型為該結構性資料類型且該第二資料庫為該關聯性資料庫,根據該上傳資料之該資料內容以及該第二資料表決定該第一資料庫以及該第一資料表 之步驟,包含:響應於該上傳資料之該資料內容的所有欄位及其資料型態與該第二資料表符合時,決定該第一資料庫為該關聯性資料庫,以及決定該第一資料表為該第二資料表;以及響應於該第二資料表之所有欄位符合於該上傳資料之該資料內容的所有欄位之子集合以及該上傳資料之該資料內容具有新增欄位時,決定該第一資料庫為該關聯性資料庫,以及進行一資料表移轉。 The data processing system as described in claim 5, wherein the data type corresponding to the uploaded data is the structural data type and the second database is the associative database, and the step of determining the first database and the first data table according to the data content of the uploaded data and the second data table includes: all fields and their data types corresponding to the data content of the uploaded data and the second data table are determined. When the data tables match, the first database is determined to be the associative database, and the first data table is determined to be the second data table; and in response to all fields of the second data table matching a subset of all fields of the data content of the uploaded data and the data content of the uploaded data having a newly added field, the first database is determined to be the associative database, and a data table migration is performed. 如請求項6所述之資料處理系統,其中資料表轉移之步驟包含:匯出該第二資料表之資料為一暫存資料;刪除該第二資料表;根據該上傳資料之該資料內容以及該第二資料表之一資料內容,於該關聯性資料庫建立具有該資料表名稱之該第一資料表;以及匯入該暫存資料至該第一資料表,其中該新增欄位之資料設為一空值。 The data processing system as described in claim 6, wherein the step of transferring the data table includes: exporting the data of the second data table as temporary data; deleting the second data table; creating the first data table with the data table name in the relational database according to the data content of the uploaded data and a data content of the second data table; and importing the temporary data into the first data table, wherein the data of the newly added field is set to a null value. 如請求項5所述之資料處理系統,其中響應於該上傳資料之該資料類型為該非結構性資料類型且該第二資料庫為該關聯性資料庫,決定該第一資料庫為該資料處理系統之該非關聯性資料庫,以及進行該資料庫移轉之步驟,包含:匯出該第二資料表之資料為一暫存資料;刪除該第二資料表;於該非關聯性資料庫建立具有該資料表名稱之該第一資料表;以及匯入該暫存資料至該第一資料表。 The data processing system as described in claim 5, wherein in response to the data type of the uploaded data being the unstructured data type and the second database being the relational database, the first database is determined to be the non-relational database of the data processing system, and the steps of performing the database migration include: exporting the data of the second data table as temporary data; deleting the second data table; creating the first data table with the data table name in the non-relational database; and importing the temporary data into the first data table.
TW112133165A 2023-09-01 2023-09-01 Data processing system TWI860073B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW112133165A TWI860073B (en) 2023-09-01 2023-09-01 Data processing system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW112133165A TWI860073B (en) 2023-09-01 2023-09-01 Data processing system

Publications (2)

Publication Number Publication Date
TWI860073B true TWI860073B (en) 2024-10-21
TW202511936A TW202511936A (en) 2025-03-16

Family

ID=94084088

Family Applications (1)

Application Number Title Priority Date Filing Date
TW112133165A TWI860073B (en) 2023-09-01 2023-09-01 Data processing system

Country Status (1)

Country Link
TW (1) TWI860073B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120016901A1 (en) * 2010-05-18 2012-01-19 Google Inc. Data Storage and Processing Service
TWI475411B (en) * 2011-12-29 2015-03-01 Chunghwa Telecom Co Ltd Large data checking system and its method in cloud platform
CN106708993A (en) * 2016-12-16 2017-05-24 武汉中地数码科技有限公司 Spatial data storage processing middleware framework realization method based on big data technology
US20170228290A1 (en) * 2013-09-20 2017-08-10 Amazon Technologies, Inc. Backup of partitioned database tables
CN113934408A (en) * 2021-09-17 2022-01-14 华润置地控股有限公司 Data service platform and method for generating application program interface

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120016901A1 (en) * 2010-05-18 2012-01-19 Google Inc. Data Storage and Processing Service
TWI475411B (en) * 2011-12-29 2015-03-01 Chunghwa Telecom Co Ltd Large data checking system and its method in cloud platform
US20170228290A1 (en) * 2013-09-20 2017-08-10 Amazon Technologies, Inc. Backup of partitioned database tables
CN106708993A (en) * 2016-12-16 2017-05-24 武汉中地数码科技有限公司 Spatial data storage processing middleware framework realization method based on big data technology
CN113934408A (en) * 2021-09-17 2022-01-14 华润置地控股有限公司 Data service platform and method for generating application program interface

Also Published As

Publication number Publication date
TW202511936A (en) 2025-03-16

Similar Documents

Publication Publication Date Title
US6240422B1 (en) Object to relational database mapping infrastructure in a customer care and billing system
CN111712809A (en) Learn ETL Rules by Example
US9454572B2 (en) Out-of-place materialized view refresh
CN102054034A (en) Implementation method for business basic data persistence of enterprise information system
CN110928882A (en) Memory database indexing method and system based on improved red-black tree
CN114925142B (en) A multi-type database compatibility method, device, equipment and medium for an ORM framework
CN113761040A (en) Database and application program bidirectional mapping method, device, medium and program product
CN115794833A (en) Data processing method, server and computer storage medium
CN105447105A (en) NoSQL-based single field section index query method for distributed Internet of things data
TWI860073B (en) Data processing system
JP3777666B2 (en) Database processing method and system
CN114969054A (en) Method and device for version upgrade of a database
CN117194372B (en) Data migration methods, apparatus, electronic devices and storage media
CN115455023B (en) A method and device for processing view data table
US20070050420A1 (en) Method and apparatus for transferring data between databases
TWI879497B (en) Data processing system
US9361359B1 (en) Accessing schema-free databases
JP2024086652A (en) Optimizing text filtering queries on graph data
CN114138815B (en) A method, device and medium for implementing multi-database compatibility of an application program
JPH09305622A (en) Database management method and system having document retrieval function
US20250068614A1 (en) Data Processing System
WO2023130771A1 (en) Data management method and apparatus, and electronic device and storage medium
CN108897897B (en) Data processing method and device
CN115964385A (en) Data query method based on general data access framework
WO2019153547A1 (en) Database operation method, apparatus and device, and computer-readable storage medium