[go: up one dir, main page]

CN115134421B - Multi-source heterogeneous data cross-system collaborative management system and method - Google Patents

Multi-source heterogeneous data cross-system collaborative management system and method Download PDF

Info

Publication number
CN115134421B
CN115134421B CN202210507966.9A CN202210507966A CN115134421B CN 115134421 B CN115134421 B CN 115134421B CN 202210507966 A CN202210507966 A CN 202210507966A CN 115134421 B CN115134421 B CN 115134421B
Authority
CN
China
Prior art keywords
data
type
source
task
requirement
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210507966.9A
Other languages
Chinese (zh)
Other versions
CN115134421A (en
Inventor
赵薇薇
王艳
吕守业
赵美燕
王刚
陈雪华
冯鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Remote Sensing and Digital Earth of CAS
Original Assignee
Institute of Remote Sensing and Digital Earth of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Remote Sensing and Digital Earth of CAS filed Critical Institute of Remote Sensing and Digital Earth of CAS
Priority to CN202210507966.9A priority Critical patent/CN115134421B/en
Publication of CN115134421A publication Critical patent/CN115134421A/en
Application granted granted Critical
Publication of CN115134421B publication Critical patent/CN115134421B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Information Transfer Between Computers (AREA)

Abstract

本申请提出的多源异构数据跨系统协同管理系统、方法及存储介质中,该系统包括数据接入层、数据总线层、数据订阅层。数据订阅层,用于获取数据需求任务和数据需求任务的类型,并对获取的数据需求任务进行解析,根据数据请求的解析结果和数据需求任务的类型对数据总线层的数据进行订阅;数据总线层,用于将数据订阅层的解析结果、数据需求任务以及数据需求任务的类型,发送至数据接入层;数据接入层,用于根据数据总线层的解析结果,将数据需求任务以及数据需求任务的类型,发送至对应的数据源系统。其中,本申请有效地缩短了数据汇集及更新时间,实现了对多中心异构在线系统的多主体协调管理,从而解决了协同管理和集约服务效率低的问题。

In the multi-source heterogeneous data cross-system collaborative management system, method and storage medium proposed in this application, the system includes a data access layer, a data bus layer, and a data subscription layer. The data subscription layer is used to obtain data requirement tasks and types of data requirement tasks, parse the obtained data requirement tasks, and subscribe to data in the data bus layer based on the parsing results of the data request and the type of data requirement tasks; data bus The data access layer is used to send the parsing results, data requirement tasks and data requirement task types of the data subscription layer to the data access layer; the data access layer is used to send the data requirement tasks and data requirements based on the parsing results of the data bus layer. The type of required task is sent to the corresponding data source system. Among them, this application effectively shortens the data collection and update time, realizes multi-subject coordinated management of multi-center heterogeneous online systems, thereby solving the problem of low efficiency of collaborative management and intensive services.

Description

一种多源异构数据跨系统协同管理系统及方法A multi-source heterogeneous data cross-system collaborative management system and method

技术领域Technical field

本申请涉及计算机应用技术领域,尤其涉及一种多源异构数据跨系统协同管理系统、方法及存储介质。The present application relates to the field of computer application technology, and in particular to a multi-source heterogeneous data cross-system collaborative management system, method and storage medium.

背景技术Background technique

随着遥感和网络技术的快速发展,对卫星和航拍影像、地面跨时空视频、网络数据、电磁信息和地理信息等多源异构数据进行整合利用,实现天空地多源异构数据的跨时空、多尺度、多粒度汇聚,可以为公共安全事件的智能感知与理解提供可靠、可信、可拓展的数据支撑,对提高面向公共安全事件的大数据共享服务水平、扩展共享服务渠道、增强共享服务能力具有重要意义。With the rapid development of remote sensing and network technology, multi-source heterogeneous data such as satellite and aerial images, ground cross-time and space videos, network data, electromagnetic information and geographical information are integrated and utilized to realize cross-time and space multi-source heterogeneous data from the sky and the ground. , multi-scale, multi-granularity aggregation can provide reliable, credible, and scalable data support for intelligent perception and understanding of public security events, and can improve the level of big data sharing services for public security events, expand shared service channels, and enhance sharing. Service capabilities are of great significance.

但是,相关技术中,提供的信息交互方式单一,只是对业务系统内部的几类数据进行汇聚管理,无法通过一整套系统安全汇聚多在线系统的多源异构数据,实现对多中心异构在线系统的多主体协调管理,存在着协同管理和集约服务效率低的问题。However, related technologies provide a single information interaction method, which only aggregates and manages several types of data within the business system. It is impossible to safely aggregate multi-source heterogeneous data from multiple online systems through a complete set of systems to achieve multi-center heterogeneous online processing. The multi-agent coordinated management of the system has the problem of low efficiency in collaborative management and intensive services.

发明内容Contents of the invention

本申请提供一种多源异构数据跨系统协同管理系统及方法,以对多源异构数据跨安全等级的交互进行管理。This application provides a multi-source heterogeneous data cross-system collaborative management system and method to manage the interaction of multi-source heterogeneous data across security levels.

本申请第一方面实施例提出一种多源异构数据跨系统协同管理系统,包括:The first embodiment of the present application proposes a multi-source heterogeneous data cross-system collaborative management system, including:

数据订阅层,用于获取数据需求任务和所述数据需求任务的类型,并对获取的数据需求任务进行解析,根据数据请求的解析结果和所述数据需求任务的类型对数据总线层的数据进行订阅;The data subscription layer is used to obtain the data requirement task and the type of the data requirement task, parse the obtained data requirement task, and process the data in the data bus layer according to the parsing result of the data request and the type of the data requirement task. subscription;

数据总线层,用于将所述数据订阅层的解析结果、所述数据需求任务以及所述数据需求任务的类型,发送至数据接入层;The data bus layer is used to send the parsing results of the data subscription layer, the data requirement task, and the type of the data requirement task to the data access layer;

数据接入层,用于根据所述数据总线层的解析结果,将所述数据需求任务以及所述数据需求任务的类型,发送至对应的数据源系统。The data access layer is used to send the data requirement task and the type of the data requirement task to the corresponding data source system according to the analysis result of the data bus layer.

本申请第二方面实施例提出一种多源异构数据跨系统协同管理方法,包括:The second embodiment of the present application proposes a cross-system collaborative management method for multi-source heterogeneous data, including:

获取数据需求任务和所述数据需求任务的类型;Obtain the data requirement task and the type of the data requirement task;

对所述数据需求任务进行解析,基于解析结果和所述数据需求任务的类型,将数据需求发送至对应的数据源系统;Analyze the data requirement task, and send the data requirement to the corresponding data source system based on the analysis result and the type of the data requirement task;

接收所述数据源系统发送的所述数据需求任务对应的数据;Receive data corresponding to the data requirement task sent by the data source system;

基于接收数据的类型确定对应的传输方式并输出。The corresponding transmission method is determined based on the type of received data and output.

本申请第三方面实施例提出的非临时性计算机可读存储介质,其中,所述非临时性计算机可读存储介质存储有计算机程序;所述计算机程序被处理器执行时实现如上第二方面所示的方法。The non-transitory computer-readable storage medium proposed by the third aspect of the embodiment of the present application, wherein the non-transitory computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the above second aspect is implemented. method shown.

本申请的实施例提供的技术方案至少带来以下有益效果:The technical solutions provided by the embodiments of the present application at least bring the following beneficial effects:

本申请提出的多源异构数据跨系统协同管理系统、方法及存储介质中,该系统包括数据接入层、数据总线层、数据订阅层。数据订阅层,用于获取数据需求任务和数据需求任务的类型,并对获取的数据需求任务进行解析,根据数据请求的解析结果和数据需求任务的类型对数据总线层的数据进行订阅;数据总线层,用于将数据订阅层的解析结果、数据需求任务以及数据需求任务的类型,发送至数据接入层;数据接入层,用于根据数据总线层的解析结果,将数据需求任务以及数据需求任务的类型,发送至对应的数据源系统。其中,本申请提供了标准化、智能化的针对性多源异构数据调度和更新策略,有效地缩短了数据汇集及更新时间,实现了对多中心异构在线系统的多主体协调管理,满足了上层应用对多源异构数据的按需汇集需求,使有限的传输信道资源始终用来传输汇集用户最急需的数据产品,解决了协同管理和集约服务效率低的问题。In the multi-source heterogeneous data cross-system collaborative management system, method and storage medium proposed in this application, the system includes a data access layer, a data bus layer, and a data subscription layer. The data subscription layer is used to obtain data requirement tasks and types of data requirement tasks, parse the obtained data requirement tasks, and subscribe to data in the data bus layer based on the parsing results of the data request and the type of data requirement tasks; data bus The data access layer is used to send the parsing results, data requirement tasks and data requirement task types of the data subscription layer to the data access layer; the data access layer is used to send the data requirement tasks and data requirements based on the parsing results of the data bus layer. The type of required task is sent to the corresponding data source system. Among them, this application provides standardized and intelligent targeted multi-source heterogeneous data scheduling and update strategies, effectively shortening the data collection and update time, realizing multi-agent coordinated management of multi-center heterogeneous online systems, and meeting the needs of The upper-layer application's on-demand collection requirements for multi-source heterogeneous data enable limited transmission channel resources to always be used to transmit and collect the most urgently needed data products for users, solving the problem of low efficiency in collaborative management and intensive services.

本申请附加的方面和优点将在下面的描述中部分给出,部分将从下面的描述中变得明显,或通过本申请的实践了解到。Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

附图说明Description of the drawings

本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解,其中:The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

图1为根据本申请一个实施例提供的多源异构数据跨系统协同管理系统的结构示意图;Figure 1 is a schematic structural diagram of a multi-source heterogeneous data cross-system collaborative management system provided according to an embodiment of the present application;

图2为根据本申请一个实施例提供的数据服务总线中实体关系传输数据的流程示意图;Figure 2 is a schematic flowchart of data transmission between entities in a data service bus according to an embodiment of the present application;

图3为根据本申请一个实施例提供的多源异构数据跨系统协同管理方法的流程示意图。Figure 3 is a schematic flowchart of a multi-source heterogeneous data cross-system collaborative management method provided according to an embodiment of the present application.

具体实施方式Detailed ways

下面详细描述本申请的实施例,所述实施例的示例在附图中示出,其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的,旨在用于解释本申请,而不能理解为对本申请的限制。The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application, but should not be construed as limiting the present application.

下面参考附图描述本申请实施例的多源异构数据跨系统协同管理系统及方法。The multi-source heterogeneous data cross-system collaborative management system and method according to the embodiment of the present application will be described below with reference to the accompanying drawings.

实施例一Embodiment 1

图1为根据本申请一个实施例提供的一种多源异构数据跨系统协同管理系统的结构示意图,如图1所示,可以包括:Figure 1 is a schematic structural diagram of a multi-source heterogeneous data cross-system collaborative management system provided according to an embodiment of the present application. As shown in Figure 1, it may include:

数据订阅层,用于获取数据需求任务和数据需求任务的类型,并对获取的数据需求任务进行解析,根据数据请求的解析结果和数据需求任务的类型对数据总线层的数据进行订阅;The data subscription layer is used to obtain the data requirement task and the type of the data requirement task, parse the obtained data requirement task, and subscribe to the data in the data bus layer according to the parsing result of the data request and the type of the data requirement task;

数据总线层,用于将数据订阅层的解析结果、数据需求任务以及数据需求任务的类型,发送至数据接入层;The data bus layer is used to send the parsing results, data requirement tasks, and data requirement task types of the data subscription layer to the data access layer;

数据接入层,用于根据数据总线层的解析结果,将数据需求任务以及数据需求任务的类型,发送至对应的数据源系统;The data access layer is used to send data requirement tasks and types of data requirement tasks to the corresponding data source system based on the analysis results of the data bus layer;

数据接入层,还用于将不同数据源系统来源的多源异构数据进行安全接入;The data access layer is also used to securely access multi-source heterogeneous data from different data source systems;

数据总线层,还用于接收数据接入层汇集的多源异构数据,并根据多源异构数据的不同数据来源定义不同的数据域,根据不同数据域下的不同数据类型定义不同数据域下的不同主题,并将不同数据域中的数据进行发布;The data bus layer is also used to receive multi-source heterogeneous data collected by the data access layer, and to define different data domains based on different data sources of multi-source heterogeneous data, and to define different data domains based on different data types under different data domains. different topics and publish data in different data domains;

数据订阅层,还用于接收数据总线层中的发布的数据,并确定接收数据的类型,基于接收数据的类型确定对应的传输方式进行传输。The data subscription layer is also used to receive published data in the data bus layer, determine the type of received data, and determine the corresponding transmission method for transmission based on the type of received data.

其中,在本发明的实施例中,数据订阅层可以从数据管理平台获取数据需求任务和数据需求任务的类型。以及,数据管理平台可以对多源异构数据进行管理与分析。Among them, in the embodiment of the present invention, the data subscription layer can obtain the data requirement task and the type of the data requirement task from the data management platform. In addition, the data management platform can manage and analyze multi-source heterogeneous data.

以及,在本发明的实施例中,数据订阅层中获取的数据需求任务的类型包括常规服务模式需求、应急服务模式需求和专项服务模式需求。不同的类型需求中对数据调用及数据更新需求也有所不同。And, in the embodiment of the present invention, the types of data requirement tasks obtained in the data subscription layer include regular service mode requirements, emergency service mode requirements and special service mode requirements. Different types of requirements also have different requirements for data calling and data updating.

具体的,在本发明的实施例中,常规服务模式需求为在日常工作下或响应紧急程度一般的用户需求;应急服务模式需求为数据需要快速获取或需要在最短时间内调动所有可用和有效的资源的用户需求;专项服务模式需求为在以专项任务输入为核心的情况下或通过任务驱动持续对某一地区或某一时间进行跟踪的用户需求。Specifically, in the embodiment of the present invention, the regular service mode requirements are user needs in daily work or in response to a general degree of urgency; the emergency service mode requirements are that data needs to be obtained quickly or all available and effective data needs to be mobilized in the shortest time. User demand for resources; special service model demand is the user demand for continuous tracking of a certain area or a certain time with special task input as the core or through task driving.

以及,在本发明的实施例中,常规服务模式需求可以包括定期或者7*24小时实时数据更新需求;应急服务模式需求可以包括对具体资源类型、产品等级以及时间、区域等数据需求;专项服务模式需求可以包括任务明确的时间周期、频次、区域、影像数据载荷要求及视频、业务数据及其智能分析结果数据。And, in the embodiment of the present invention, the regular service mode requirements may include regular or 7*24 hours real-time data update requirements; the emergency service mode requirements may include data requirements for specific resource types, product levels, time, regions, etc.; special services Mode requirements can include clear task time periods, frequencies, regions, image data load requirements and video, business data and intelligent analysis result data.

进一步地,在本发明的实施例中,数据订阅层对获取的数据请求进行解析包括对数据需求任务进行解析,得出数据需求任务所涉及的数据源系统编号。Further, in the embodiment of the present invention, the data subscription layer parses the obtained data request, including parsing the data requirement task, and obtains the data source system number involved in the data requirement task.

以及,在本发明的实施例中,数据接入层可以根据数据订阅层解析的数据需求任务所涉及的数据源系统编号,将数据需求任务发送至对应的数据源系统。And, in the embodiment of the present invention, the data access layer can send the data requirement task to the corresponding data source system according to the data source system number involved in the data requirement task parsed by the data subscription layer.

需要说明的是,在本发明的实施例中,多源异构数据可以包括卫星遥感数据、航拍遥感数据、地面视频数据、网络信息数据、地理信息数据和业务信息数据。It should be noted that in embodiments of the present invention, multi-source heterogeneous data may include satellite remote sensing data, aerial remote sensing data, ground video data, network information data, geographical information data and business information data.

以及,在本发明的实施例中,多源异构数据中数据的不同时,对应数据的所处的系统也不相同。其中,在本发明的一个实施例中,表1为多源异构数据中数据类型与数据所处系统/网络的对应关系。Moreover, in the embodiment of the present invention, when the data in the multi-source heterogeneous data is different, the systems where the corresponding data are located are also different. Among them, in one embodiment of the present invention, Table 1 shows the corresponding relationship between data types in multi-source heterogeneous data and the system/network where the data is located.

表1Table 1

进一步地,在本发明的实施例中,数据源系统接收数据需求任务后,可以根据数据需求任务的类型,将数据需求任务对应的数据发送至多源异构数据跨系统协同管理系统。Further, in the embodiment of the present invention, after the data source system receives the data requirement task, it can send the data corresponding to the data requirement task to the multi-source heterogeneous data cross-system collaborative management system according to the type of the data requirement task.

具体地,在本发明的实施例中,当数据需求任务的类型为常规服务模式需求时,若数据类型为文件数据,则数据源系统定期主动推送相应地区编目信息,在数据接入层中的数据传输代理服务监控编目信息的变化并定期发送给多源异构数据跨系统协同管理系统;若数据类型为视频/业务数据,则数据源系统可以实时采集更新汇聚至多源异构数据跨系统协同管理系统;若数据类型为不同网络系统环境下的数据源数据,需要通过数据接入层中与数据总线层进行跨网数据传输和信息交互。Specifically, in the embodiment of the present invention, when the type of data requirement task is a regular service mode requirement, and if the data type is file data, the data source system actively pushes the corresponding regional cataloging information on a regular basis. In the data access layer The data transmission agent service monitors changes in cataloging information and regularly sends it to the multi-source heterogeneous data cross-system collaborative management system; if the data type is video/business data, the data source system can collect and update in real time and aggregate it into the multi-source heterogeneous data cross-system collaboration Management system; if the data type is data source data in different network system environments, cross-network data transmission and information interaction need to be carried out through the data access layer and the data bus layer.

以及,在本发明的实施例中,当数据需求任务的类型为应急服务模式需求时,若数据类型为航天遥感数据,则数据源系统接收到明确的实体数据需求加急任务后,一方面可通过系统配置,提高优先权限,另一方面完成与其他任务的冲突消解,及时调整卫星轨道或姿态安排拍摄任务,及时获取数据/信息服务,并简化相应处理流程。同时根据具体资源类型、产品等级以及时间、区域需求,由数据源系统后台配置实现自动分发服务发送至多源异构数据跨系统协同管理系统;若数据类型为无人机数据,则数据源系统在接收到应急任务后,提交空中飞行申请,完成飞行任务和数据的下传处理,经多源异构数据跨系统协同管理系统传输至数据管理平台;若数据类型为视频和业务数据,数据源系统在接收到紧急任务后,需要调动突发事件所在地范围内的所有视频采集设备,并根据需求调整视频采集的角度和焦距,全方位对事件进行监控,同时调动智能分析资源,根据事件性质,实时分析相关人、车、物,对事件进行追溯、分析和发展预测,并将分析结果传送至多源异构数据跨系统协同管理系统。And, in the embodiment of the present invention, when the type of data demand task is emergency service mode demand, if the data type is aerospace remote sensing data, after the data source system receives the clear entity data demand urgent task, on the one hand, it can Through system configuration, the priority authority is improved. On the other hand, conflicts with other tasks are resolved, satellite orbits or attitude are adjusted in time to arrange shooting tasks, data/information services are obtained in a timely manner, and corresponding processing procedures are simplified. At the same time, according to the specific resource type, product level, time, and regional requirements, the data source system background configuration realizes automatic distribution services and sends them to the multi-source heterogeneous data cross-system collaborative management system; if the data type is drone data, the data source system After receiving the emergency mission, submit an application for air flight, complete the download processing of the flight mission and data, and transmit it to the data management platform through the multi-source heterogeneous data cross-system collaborative management system; if the data type is video and business data, the data source system After receiving the emergency task, it is necessary to mobilize all video collection equipment within the location of the emergency, adjust the angle and focal length of the video collection according to needs, monitor the event in an all-round way, and mobilize intelligent analysis resources to conduct real-time monitoring according to the nature of the event. Analyze relevant people, vehicles, and objects, trace, analyze, and predict events, and transmit the analysis results to the multi-source heterogeneous data cross-system collaborative management system.

进一步地,在本发明的实施例中,当数据需求任务的类型为专项服务模式需求时,若数据类型为遥感数据,则数据源系统根据任务明确的时间周期、频次、区域及载荷要求获取相应数据的编目信息或者实体数据并推送至多源异构数据跨系统协同管理系统;若数据类型为视频和业务数据,则数据源系统获取特定时间段的视频、业务数据及其智能分析结果并推送至多源异构数据跨系统协同管理系统。Further, in the embodiment of the present invention, when the type of data requirement task is a special service mode requirement, and if the data type is remote sensing data, the data source system obtains the corresponding data according to the specific time period, frequency, area and load requirements of the task. The catalog information or entity data of the data is pushed to the multi-source heterogeneous data cross-system collaborative management system; if the data type is video and business data, the data source system obtains the video, business data and intelligent analysis results of a specific time period and pushes them to the multi-source heterogeneous data cross-system collaborative management system. Cross-system collaborative management system for source heterogeneous data.

以及,在本发明的实施例中,数据源系统发送至多源异构数据跨系统协同管理系统的数据源类别可以包括文件数据、流媒体数据和短报文数据。其中,在本发明的实施例中,文件数据包括编目信息数据和实体数据,两者编号一致。以及,实体数据容量大、传输时间长,中间可能发生网络中断或服务暂停等异常情况,在传输和交换服务设计上要支持断点续传等保护机制;流媒体数据可以包括视频、声音媒体数据,在传输过程中必须严格控制延迟、抖动和误码率关键性能;短报文数据可以包括目标报文、飞行报文及WiFi报文等,容量小、频次高,可以直接传输。And, in the embodiment of the present invention, the data source categories sent by the data source system to the multi-source heterogeneous data cross-system collaborative management system may include file data, streaming media data and short message data. Among them, in the embodiment of the present invention, the file data includes catalog information data and entity data, and the two numbers are consistent. In addition, the physical data capacity is large and the transmission time is long, and abnormal situations such as network interruption or service suspension may occur in the middle. The transmission and exchange service design must support protection mechanisms such as breakpoint resumption; streaming media data can include video and sound media data. , the key performance of delay, jitter and bit error rate must be strictly controlled during the transmission process; short message data can include target messages, flight messages and WiFi messages, etc., with small capacity and high frequency, and can be transmitted directly.

进一步地,在本发明的实施例中,数据接入层可以将不同系统来源的多源异构数据分别通过接入网关进行安全接入。其中接入网关可以包括外网汇聚单元、外网代理服务单元、隔离通信单元、内网汇聚单元、内网代理服务单元。Further, in the embodiment of the present invention, the data access layer can securely access multi-source heterogeneous data from different system sources through the access gateway. The access gateway may include an external network convergence unit, an external network proxy service unit, an isolation communication unit, an intranet convergence unit, and an intranet proxy service unit.

具体的,在本发明的实施例中,对于卫星遥感影像数据、无人机数据、地面监控视频数据通过外网汇聚单元将数据汇集到相应的汇集单元中来,然后再通过不同的网络传输,将数据发送到相应的外网代理服务单元,外网服务单元通过数据类型再反馈给不同等级的隔离通信单元进行差异化隔离处理,隔离单元处理完毕后将数据送往内网代理服务单元,完成从外网到内网的数据跨越,内网代理服务单元会将隔离过的安全数据发送至内网汇集单元。基于此,多源异构数据通过多个单元的协同处理,安全从外网送达至内网进行进一步数据处理。Specifically, in the embodiment of the present invention, for satellite remote sensing image data, drone data, and ground surveillance video data, the data is collected into the corresponding collection unit through the external network aggregation unit, and then transmitted through different networks. Send the data to the corresponding external network proxy service unit. The external network service unit then feeds back to the isolation communication units of different levels according to the data type for differentiated isolation processing. After the isolation unit completes the processing, it sends the data to the internal network proxy service unit. Complete For data crossing from the external network to the internal network, the internal network proxy service unit will send the isolated security data to the internal network aggregation unit. Based on this, multi-source heterogeneous data is safely delivered from the external network to the internal network for further data processing through the collaborative processing of multiple units.

以及,在本发明的实施例中,利用分布式数据服务总线完成数据总线层与数据订阅层之间的通信。其中,域代表逻辑上隔离的通信网络。具体的,在不同的域中,同一组主机上或相同主机组上运行的多个交互彼此隔离,不同域的数据写入者与数据读取者不会交换数据。And, in the embodiment of the present invention, the distributed data service bus is used to complete the communication between the data bus layer and the data subscription layer. Among them, domains represent logically isolated communication networks. Specifically, in different domains, multiple interactions running on the same host or the same host group are isolated from each other, and data writers and data readers in different domains will not exchange data.

具体的,在本发明的实施例中,数据总线层和数据订阅层是通过以数据为中心的发布/订阅进行通信,将数据从发布者传送到感兴趣的订阅者。以及,在通信之前需要建立实体来建立彼此间的发布/订阅通信。分布式数据服务总线中可以包含域、数据写入者、发布者、数据读取者、订阅者、主实体。Specifically, in the embodiment of the present invention, the data bus layer and the data subscription layer communicate through data-centered publish/subscribe to transmit data from the publisher to interested subscribers. Also, before communication, entities need to be established to establish publish/subscribe communication between each other. The distributed data service bus can contain domains, data writers, publishers, data readers, subscribers, and main entities.

本发明的实施例中,数据总线层可以作为发布者,数据订阅层可以作为订阅者。以及,在本发明的实施例中,数据总线层可以将不同来源的数据定义为不同的域,从而有效的对数据进行隔离。在一个数据域内,可以根据数据类型定义不同的主题,从而可以通过主题标识数据。以及,在本发明的实施例中,主题是一种特定的域段,主题有一个唯一的主题名称、数据类型和一套QOS策略。同一数据类型可以发布多个主题,每个主题只可以关联一种数据类型。In the embodiment of the present invention, the data bus layer can serve as a publisher, and the data subscription layer can serve as a subscriber. Moreover, in the embodiment of the present invention, the data bus layer can define data from different sources as different domains, thereby effectively isolating the data. Within a data domain, different topics can be defined based on the data type, so that the data can be identified by topic. And, in the embodiment of the present invention, the topic is a specific domain segment, and the topic has a unique topic name, data type and a set of QOS policies. The same data type can be published on multiple topics, and each topic can only be associated with one data type.

以及,在本发明实施例中,可以为每个大数据类型下的子类型的数据定义一个主题,并根据需要通信的信息定义主题的数据类型和QOS策略,为后续数据通信做好准备。Moreover, in the embodiment of the present invention, a topic can be defined for each subtype of data under the big data type, and the data type and QOS policy of the topic can be defined according to the information that needs to be communicated, so as to prepare for subsequent data communication.

进一步地,在本发明的实施例中,分布式数据服务总线由多个数据域组成,每个域包含一套发布者和订阅者,当一个确定的数据在一个域内发布时,将不会被任何其他域的订阅者接收。Further, in the embodiment of the present invention, the distributed data service bus is composed of multiple data domains. Each domain contains a set of publishers and subscribers. When a certain data is published in a domain, it will not be Subscribers to any other domain receive it.

图2为本发明实施例提供的一种数据服务总线中实体关系传输数据的流程示意图。FIG. 2 is a schematic flowchart of entity relationship data transmission in a data service bus provided by an embodiment of the present invention.

如图2所示,在本发明的实施例中,可以使用数据写入者发送数据,数据写入者与单独的主题相关联,一个特定的主题可以拥有多个对应的数据写入者。数据写入者负责为数据编列,并将其传送至发布者进行传输。As shown in Figure 2, in embodiments of the present invention, data writers can be used to send data. Data writers are associated with individual topics, and a specific topic can have multiple corresponding data writers. Data writers are responsible for organizing the data and delivering it to the publisher for transmission.

进一步地,在本发明的实施例中,发布者负责获取发布的数据,将其发布至域中所有的相关订阅者,发布者拥有和管理数据写入者。一个数据写入者仅可由单独的发布者拥有,一个发布者可以对应多个数据写入者,基于此,相同发布者可以为不同数据类型的多个不同主题发送数据。Further, in the embodiment of the present invention, the publisher is responsible for obtaining the published data and publishing it to all relevant subscribers in the domain. The publisher owns and manages the data writers. A data writer can only be owned by a single publisher, and a publisher can correspond to multiple data writers. Based on this, the same publisher can send data for multiple different topics of different data types.

进一步地,在本发明的实施例中,数据读取者访问数据服务总线上接收的数据,一个数据读取者关联一个单独主题,特定的主题可以对应超过一个数据读取者。Further, in the embodiment of the present invention, data readers access data received on the data service bus. One data reader is associated with a single topic, and a specific topic can correspond to more than one data reader.

进一步地,在本发明的实施例中,订阅者是负责发布数据实际接收的数据,订阅者拥有和管理数据读取者,数据读取者和订阅者间的关联是订阅,一个数据读取者只能有一个单独的订阅者拥有,一个订阅者可以对应多个数据读取者。基于此,相同订阅者可以为不同数据类型的很多不同主题接收数据。Further, in the embodiment of the present invention, the subscriber is responsible for publishing data and actually receiving the data. The subscriber owns and manages the data reader. The association between the data reader and the subscriber is a subscription. A data reader It can only be owned by a single subscriber, and one subscriber can correspond to multiple data readers. Based on this, the same subscriber can receive data for many different topics with different data types.

进一步地,在本发明的实施例中,多源异构数据在对应的数据域内进行发送和接收,发布者和订阅者分别用来管理单个的或多个的数据写入者和数据读取者,一个数据读取者和一个数据写入者必须以相同的主题相关联,基于此,数据写入者发布的数据才能被订阅的数据读取者接收。Further, in the embodiment of the present invention, multi-source heterogeneous data is sent and received in the corresponding data domain, and publishers and subscribers are used to manage single or multiple data writers and data readers respectively. , a data reader and a data writer must be associated with the same topic, based on this, the data published by the data writer can be received by the subscribed data reader.

在本发明的实施例中,与客户端/服务器架构相比,发布/订阅模式降低了在网络上发送数据的消耗,在低带宽下的偶然订阅请求替代大量的高带宽客户端请求。同时,对于定期数据交换来说,发布/订阅在反应时间和带宽上比客户端/服务器端的效率更高。In embodiments of the present invention, compared with client/server architecture, the publish/subscribe model reduces the consumption of sending data over the network, and occasional subscription requests at low bandwidth replace a large number of high-bandwidth client requests. At the same time, for periodic data exchange, publish/subscribe is more efficient than client/server in terms of response time and bandwidth.

以及,在本公开的实施例中,数据订阅层对应订阅者,数据订阅层中的订阅者接收来自发布者的数据,将其传送至与其连接的任何数据读取者,然后数据读取者获取来自订阅者的数据,将其为该主题反编列为适当的类型。And, in the embodiment of the present disclosure, the data subscription layer corresponds to the subscriber. The subscriber in the data subscription layer receives the data from the publisher and transmits it to any data reader connected to it, and then the data reader obtains Data from subscribers, decategorized into the appropriate type for the topic.

以及,在本发明的实施例中,消息订阅/发布接口可以仅需要订阅关注的数据,在有数据到达时会自动将数据发送给订阅方,数据的传输通过JSON格式封装,相比http接口可支持更高的并发,数据延迟也好于http接口。And, in the embodiment of the present invention, the message subscription/publishing interface only needs to subscribe to the data of interest. When data arrives, the data will be automatically sent to the subscriber. The transmission of the data is encapsulated in JSON format. Compared with the http interface, it can Supports higher concurrency and data latency is better than the http interface.

进一步地,在本发明的实施例中,数据订阅层中的数据类型可以包括文件数据、视频和业务数据、流媒体数据。Further, in the embodiment of the present invention, the data types in the data subscription layer may include file data, video and business data, and streaming media data.

以及,在本发明的实施例中,数据订阅层基于接收数据的类型确定对应的传输方式进行传输可以包括:And, in the embodiment of the present invention, the data subscription layer determines the corresponding transmission method based on the type of received data for transmission, which may include:

若接收数据的类型为文件数据,则根据文件数据的文件类型、文件大小、文件地址进行封装后进行传输;If the type of received data is file data, it will be encapsulated and transmitted according to the file type, file size, and file address of the file data;

若接收数据的类型为流媒体,则将流媒体数据实时进行传输;If the type of received data is streaming media, the streaming media data will be transmitted in real time;

若接收数据的类型为视频和业务数据,则周期性将视频和业务数据进行传输。If the type of received data is video and service data, the video and service data will be transmitted periodically.

本申请提出的多源异构数据跨系统协同管理系统中,该系统包括数据接入层、数据总线层、数据订阅层。数据订阅层,用于获取数据需求任务和数据需求任务的类型,并对获取的数据需求任务进行解析,根据数据请求的解析结果和数据需求任务的类型对数据总线层的数据进行订阅;数据总线层,用于将数据订阅层的解析结果、数据需求任务以及数据需求任务的类型,发送至数据接入层;数据接入层,用于根据数据总线层的解析结果,将数据需求任务以及数据需求任务的类型,发送至对应的数据源系统。其中,本申请提供了标准化、智能化的针对性多源异构数据调度和更新策略,有效地缩短了数据汇集及更新时间,实现了对多中心异构在线系统的多主体协调管理,满足了上层应用对多源异构数据的按需汇集需求,使有限的传输信道资源始终用来传输汇集用户最急需的数据产品,解决了协同管理和集约服务效率低的问题。In the multi-source heterogeneous data cross-system collaborative management system proposed in this application, the system includes a data access layer, a data bus layer, and a data subscription layer. The data subscription layer is used to obtain data requirement tasks and types of data requirement tasks, parse the obtained data requirement tasks, and subscribe to data in the data bus layer based on the parsing results of the data request and the type of data requirement tasks; data bus The data access layer is used to send the parsing results, data requirement tasks and data requirement task types of the data subscription layer to the data access layer; the data access layer is used to send the data requirement tasks and data requirements based on the parsing results of the data bus layer. The type of required task is sent to the corresponding data source system. Among them, this application provides standardized and intelligent targeted multi-source heterogeneous data scheduling and update strategies, effectively shortening the data collection and update time, realizing multi-agent coordinated management of multi-center heterogeneous online systems, and meeting the needs of The upper-layer application's on-demand collection requirements for multi-source heterogeneous data enable limited transmission channel resources to always be used to transmit and collect the most urgently needed data products for users, solving the problem of low efficiency in collaborative management and intensive services.

图3为根据本申请一个实施例提供的一种多源异构数据跨系统协同管理方法的流程示意图,如图3所示,可以包括:Figure 3 is a schematic flowchart of a multi-source heterogeneous data cross-system collaborative management method provided according to an embodiment of the present application. As shown in Figure 3, it may include:

步骤301、获取数据需求任务和数据需求任务的类型。Step 301: Obtain the data requirement task and the type of the data requirement task.

步骤302、对数据需求任务进行解析,基于解析结果和数据需求任务的类型,将数据需求发送至对应的数据源系统。Step 302: Analyze the data requirement task, and send the data requirement to the corresponding data source system based on the analysis result and the type of the data requirement task.

步骤303、接收数据源系统发送的数据需求任务对应的数据。Step 303: Receive data corresponding to the data requirement task sent by the data source system.

步骤304、基于接收数据的类型确定对应的传输方式并输出。Step 304: Determine the corresponding transmission method based on the type of received data and output it.

为了实现上述实施例,本公开还提出一种非临时性计算机可读存储介质。In order to implement the above embodiments, the present disclosure also proposes a non-transitory computer-readable storage medium.

本公开实施例提供的非临时性计算机可读存储介质,存储有计算机程序;计算机程序被处理器执行时,能够实现如图3所示的多源异构数据跨系统协同管理系统。The non-transitory computer-readable storage medium provided by the embodiment of the present disclosure stores a computer program; when the computer program is executed by the processor, the multi-source heterogeneous data cross-system collaborative management system as shown in Figure 3 can be implemented.

本申请提出的多源异构数据跨系统协同管理系统、方法及存储介质中,该系统包括数据接入层、数据总线层、数据订阅层。数据订阅层,用于获取数据需求任务和数据需求任务的类型,并对获取的数据需求任务进行解析,根据数据请求的解析结果和数据需求任务的类型对数据总线层的数据进行订阅;数据总线层,用于将数据订阅层的解析结果、数据需求任务以及数据需求任务的类型,发送至数据接入层;数据接入层,用于根据数据总线层的解析结果,将数据需求任务以及数据需求任务的类型,发送至对应的数据源系统。其中,本申请提供了标准化、智能化的针对性多源异构数据调度和更新策略,有效地缩短了数据汇集及更新时间,实现了对多中心异构在线系统的多主体协调管理,满足了上层应用对多源异构数据的按需汇集需求,使有限的传输信道资源始终用来传输汇集用户最急需的数据产品,解决了协同管理和集约服务效率低的问题。In the multi-source heterogeneous data cross-system collaborative management system, method and storage medium proposed in this application, the system includes a data access layer, a data bus layer, and a data subscription layer. The data subscription layer is used to obtain data requirement tasks and types of data requirement tasks, parse the obtained data requirement tasks, and subscribe to data in the data bus layer based on the parsing results of the data request and the type of data requirement tasks; data bus The data access layer is used to send the parsing results, data requirement tasks and data requirement task types of the data subscription layer to the data access layer; the data access layer is used to send the data requirement tasks and data requirements based on the parsing results of the data bus layer. The type of required task is sent to the corresponding data source system. Among them, this application provides standardized and intelligent targeted multi-source heterogeneous data scheduling and update strategies, effectively shortening the data collection and update time, realizing multi-agent coordinated management of multi-center heterogeneous online systems, and meeting the needs of The upper-layer application's on-demand collection requirements for multi-source heterogeneous data enable limited transmission channel resources to always be used to transmit and collect the most urgently needed data products for users, solving the problem of low efficiency in collaborative management and intensive services.

在本说明书的描述中,参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中,对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且,描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外,在不相互矛盾的情况下,本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。In the description of this specification, reference to the terms "one embodiment," "some embodiments," "an example," "specific examples," or "some examples" or the like means that specific features are described in connection with the embodiment or example. , structures, materials or features are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine different embodiments or examples and features of different embodiments or examples described in this specification unless they are inconsistent with each other.

流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为,表示包括一个或更多个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分,并且本申请的优选实施方式的范围包括另外的实现,其中可以不按所示出或讨论的顺序,包括根据所涉及的功能按基本同时的方式或按相反的顺序,来执行功能,这应被本申请的实施例所属技术领域的技术人员所理解。Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments, or portions of code that include one or more executable instructions for implementing customized logical functions or steps of the process. , and the scope of the preferred embodiments of the present application includes additional implementations in which functions may be performed out of the order shown or discussed, including in a substantially simultaneous manner or in the reverse order, depending on the functionality involved, which shall It should be understood by those skilled in the technical field to which the embodiments of this application belong.

尽管上面已经示出和描述了本申请的实施例,可以理解的是,上述实施例是示例性的,不能理解为对本申请的限制,本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。Although the embodiments of the present application have been shown and described above, it can be understood that the above-mentioned embodiments are illustrative and cannot be understood as limitations of the present application. Those of ordinary skill in the art can make modifications to the above-mentioned embodiments within the scope of the present application. The embodiments are subject to changes, modifications, substitutions and variations.

Claims (8)

1.一种多源异构数据跨系统协同管理系统,其特征在于,所述系统包括:1. A multi-source heterogeneous data cross-system collaborative management system, characterized in that the system includes: 数据订阅层,用于获取数据需求任务和所述数据需求任务的类型,并对获取的数据需求任务进行解析,根据数据请求的解析结果和所述数据需求任务的类型对数据总线层的数据进行订阅;The data subscription layer is used to obtain the data requirement task and the type of the data requirement task, parse the obtained data requirement task, and process the data in the data bus layer according to the parsing result of the data request and the type of the data requirement task. subscription; 数据总线层,用于将所述数据订阅层的解析结果、所述数据需求任务以及所述数据需求任务的类型,发送至数据接入层;The data bus layer is used to send the parsing results of the data subscription layer, the data requirement task, and the type of the data requirement task to the data access layer; 数据接入层,用于根据所述数据总线层的解析结果,将所述数据需求任务以及所述数据需求任务的类型,发送至对应的数据源系统;The data access layer is used to send the data requirement task and the type of the data requirement task to the corresponding data source system according to the analysis result of the data bus layer; 所述数据总线层,还用于接收所述数据接入层汇集的多源异构数据,并根据所述多源异构数据的不同数据来源定义不同的数据域,根据不同数据域下的不同数据类型定义所述不同数据域下的不同主题,并将所述不同数据域中的数据进行发布;The data bus layer is also used to receive multi-source heterogeneous data collected by the data access layer, and define different data domains according to different data sources of the multi-source heterogeneous data. The data type defines different topics under the different data domains, and publishes the data in the different data domains; 其中,所述数据源系统接收所述数据需求任务后,根据所述数据需求任务的类型,将所述数据需求任务对应的数据发送至所述多源异构数据跨系统协同管理系统;其中,所述多源异构数据包括卫星遥感数据、航拍遥感数据、地面视频数据、网络信息数据、地理信息数据和业务信息数据,数据订阅层中获取的数据需求任务的类型包括常规服务模式需求、应急服务模式需求和专项服务模式需求;Wherein, after receiving the data requirement task, the data source system sends the data corresponding to the data requirement task to the multi-source heterogeneous data cross-system collaborative management system according to the type of the data requirement task; wherein, The multi-source heterogeneous data includes satellite remote sensing data, aerial remote sensing data, ground video data, network information data, geographical information data and business information data. The types of data demand tasks obtained in the data subscription layer include regular service mode requirements, emergency Service model requirements and special service model requirements; 其中,所述数据源系统接收所述数据需求任务后,根据所述数据需求任务的类型,将所述数据需求任务对应的数据发送至所述多源异构数据跨系统协同管理系统,包括:Wherein, after receiving the data requirement task, the data source system sends the data corresponding to the data requirement task to the multi-source heterogeneous data cross-system collaborative management system according to the type of the data requirement task, including: 当数据需求任务的类型为常规服务模式需求时,若数据类型为文件数据,则数据源系统定期主动推送相应地区编目信息,在数据接入层中的数据传输代理服务监控编目信息的变化并定期发送给多源异构数据跨系统协同管理系统;若数据类型为视频/业务数据,则数据源系统实时采集更新汇聚至多源异构数据跨系统协同管理系统;若数据类型为不同网络系统环境下的数据源数据,则通过数据接入层中与数据总线层进行跨网数据传输和信息交互;When the type of data requirement task is a regular service mode requirement, and if the data type is file data, the data source system will actively push the corresponding regional catalog information regularly. The data transmission agent service in the data access layer monitors the changes in the catalog information and periodically Send to the multi-source heterogeneous data cross-system collaborative management system; if the data type is video/business data, the data source system collects and updates in real time and aggregates it to the multi-source heterogeneous data cross-system collaborative management system; if the data type is in different network system environments The data source data carries out cross-network data transmission and information interaction through the data access layer and the data bus layer; 当数据需求任务的类型为应急服务模式需求时,若数据类型为航天遥感数据,则数据源系统接收到明确的实体数据需求加急任务后,一方面通过系统配置,提高优先权限,另一方面完成与其他任务的冲突消解,及时调整卫星轨道或姿态安排拍摄任务,及时获取数据/信息服务,并简化相应处理流程,同时根据具体资源类型、产品等级以及时间、区域需求,由数据源系统后台配置实现自动分发服务发送至多源异构数据跨系统协同管理系统;若数据类型为无人机数据,则数据源系统在接收到应急任务后,提交空中飞行申请,完成飞行任务和数据的下传处理,经多源异构数据跨系统协同管理系统传输至数据管理平台;若数据类型为视频和业务数据,数据源系统在接收到紧急任务后,调动突发事件所在地范围内的视频采集设备,并根据需求调整视频采集的角度和焦距,全方位对事件进行监控,同时调动智能分析资源,根据事件性质,实时分析相关人、车、物,对事件进行追溯、分析和发展预测,并将分析结果传送至多源异构数据跨系统协同管理系统;When the type of data demand task is emergency service mode demand, and if the data type is aerospace remote sensing data, after the data source system receives a clear urgent task of entity data demand, on the one hand, it will increase the priority through system configuration, and on the other hand, Complete the resolution of conflicts with other tasks, timely adjust the satellite orbit or attitude to arrange shooting tasks, obtain data/information services in a timely manner, and simplify the corresponding processing procedures. At the same time, according to specific resource types, product levels, time and regional requirements, the data source system backend Configure the automatic distribution service to be sent to the multi-source heterogeneous data cross-system collaborative management system; if the data type is drone data, after receiving the emergency mission, the data source system submits an application for air flight and completes the flight mission and data download. Processed and transmitted to the data management platform through the multi-source heterogeneous data cross-system collaborative management system; if the data type is video and business data, after receiving the emergency task, the data source system mobilizes the video collection equipment within the location of the emergency. And adjust the angle and focal length of video collection according to needs, monitor the event in an all-round way, and mobilize intelligent analysis resources to analyze relevant people, vehicles, and objects in real time according to the nature of the event, trace, analyze, and predict the development of the event, and analyze the event The results are transmitted to the multi-source heterogeneous data cross-system collaborative management system; 当数据需求任务的类型为专项服务模式需求时,若数据类型为遥感数据,则数据源系统根据任务明确的时间周期、频次、区域及载荷要求获取相应数据的编目信息或者实体数据并推送至多源异构数据跨系统协同管理系统;若数据类型为视频和业务数据,则数据源系统获取特定时间段的视频、业务数据及其智能分析结果并推送至多源异构数据跨系统协同管理系统;When the type of data requirement task is a special service mode requirement, and if the data type is remote sensing data, the data source system will obtain the cataloging information or entity data of the corresponding data according to the task's clear time period, frequency, area and load requirements and push it to multiple sources. Heterogeneous data cross-system collaborative management system; if the data type is video and business data, the data source system obtains the video, business data and their intelligent analysis results in a specific time period and pushes them to the multi-source heterogeneous data cross-system collaborative management system; 所述数据订阅层,还用于对所述数据需求任务进行解析,得出所述数据需求任务所涉及的数据源系统编号;The data subscription layer is also used to analyze the data requirement task and obtain the data source system number involved in the data requirement task; 所述数据接入层,还用于根据数据订阅层解析的数据需求任务所涉及的数据源系统编号,将数据需求任务发送至对应的数据源系统。The data access layer is also used to send the data requirement task to the corresponding data source system according to the data source system number involved in the data requirement task parsed by the data subscription layer. 2.如权利要求1中所述的多源异构数据跨系统协同管理系统,其特征在于,2. The multi-source heterogeneous data cross-system collaborative management system as claimed in claim 1, characterized in that, 所述数据接入层,还用于将不同数据源系统来源的多源异构数据进行安全接入;The data access layer is also used to securely access multi-source heterogeneous data from different data source systems; 所述数据订阅层,还用于接收所述数据总线层中的发布的数据,并确定接收数据的类型,基于所述接收数据的类型确定对应的传输方式进行传输。The data subscription layer is also used to receive the published data in the data bus layer, determine the type of the received data, and determine the corresponding transmission method for transmission based on the type of the received data. 3.如权利要求2所述的多源异构数据跨系统协同管理系统,其特征在于,所述数据总线层为每个主题配置主题名称、数据类型和一套QOS策略。3. The multi-source heterogeneous data cross-system collaborative management system according to claim 2, wherein the data bus layer configures a topic name, data type and a set of QOS policies for each topic. 4.如权利要求2所述的多源异构数据跨系统协同管理系统,其特征在于,所述数据接入层,还用于将不同系统来源的多源异构数据分别通过接入网关进行安全接入。4. The multi-source heterogeneous data cross-system collaborative management system according to claim 2, characterized in that the data access layer is also used to process multi-source heterogeneous data from different system sources through access gateways. Secure access. 5.如权利要求2所述的多源异构数据跨系统协同管理系统,其特征在于,5. The multi-source heterogeneous data cross-system collaborative management system as claimed in claim 2, characterized in that, 所述数据订阅层中的数据类型包括:文件数据、视频和业务数据、流媒体数据。The data types in the data subscription layer include: file data, video and business data, and streaming media data. 6.如权利要求5所述的多源异构数据跨系统协同管理系统,其特征在于,所述数据订阅层基于所述接收数据的类型确定对应的传输方式进行传输,包括:6. The multi-source heterogeneous data cross-system collaborative management system according to claim 5, wherein the data subscription layer determines the corresponding transmission method for transmission based on the type of the received data, including: 若接收数据的类型为文件数据,则根据所述文件数据的文件类型、文件大小、文件地址进行封装后进行传输;If the type of received data is file data, it is encapsulated according to the file type, file size, and file address of the file data and then transmitted; 若接收数据的类型为流媒体,则将所述流媒体数据实时进行传输;If the type of received data is streaming media, the streaming media data is transmitted in real time; 若接收数据的类型为视频和业务数据,则周期性将所述视频和业务数据进行传输。If the type of received data is video and service data, the video and service data are periodically transmitted. 7.一种应用于如权利要求1-6任一所述的多源异构数据跨系统协同管理系统的方法,其特征在于,所述方法,包括:7. A method applied to the multi-source heterogeneous data cross-system collaborative management system according to any one of claims 1 to 6, characterized in that the method includes: 获取数据需求任务和所述数据需求任务的类型;Obtain the data requirement task and the type of the data requirement task; 对所述数据需求任务进行解析,基于解析结果和所述数据需求任务的类型,将数据需求发送至对应的数据源系统;Analyze the data requirement task, and send the data requirement to the corresponding data source system based on the analysis result and the type of the data requirement task; 接收所述数据源系统发送的所述数据需求任务对应的数据;Receive data corresponding to the data requirement task sent by the data source system; 基于接收数据的类型确定对应的传输方式并输出;Determine the corresponding transmission method based on the type of received data and output it; 所述方法,还包括:The method also includes: 接收所述数据接入层汇集的多源异构数据,并根据所述多源异构数据的不同数据来源定义不同的数据域,根据不同数据域下的不同数据类型定义所述不同数据域下的不同主题,并将所述不同数据域中的数据进行发布;Receive multi-source heterogeneous data collected by the data access layer, define different data domains according to different data sources of the multi-source heterogeneous data, and define different data domains according to different data types in the different data domains. different topics and publish the data in the different data domains; 其中,所述数据源系统接收所述数据需求任务后,根据所述数据需求任务的类型,将所述数据需求任务对应的数据发送至所述多源异构数据跨系统协同管理系统;Wherein, after receiving the data requirement task, the data source system sends the data corresponding to the data requirement task to the multi-source heterogeneous data cross-system collaborative management system according to the type of the data requirement task; 其中,所述多源异构数据包括卫星遥感数据、航拍遥感数据、地面视频数据、网络信息数据、地理信息数据和业务信息数据,数据订阅层中获取的数据需求任务的类型包括常规服务模式需求、应急服务模式需求和专项服务模式需求;Among them, the multi-source heterogeneous data includes satellite remote sensing data, aerial remote sensing data, ground video data, network information data, geographical information data and business information data. The types of data requirement tasks obtained in the data subscription layer include regular service mode requirements. , emergency service model needs and special service model needs; 其中,所述数据源系统接收所述数据需求任务后,根据所述数据需求任务的类型,将所述数据需求任务对应的数据发送至所述多源异构数据跨系统协同管理系统,包括:Wherein, after receiving the data requirement task, the data source system sends the data corresponding to the data requirement task to the multi-source heterogeneous data cross-system collaborative management system according to the type of the data requirement task, including: 当数据需求任务的类型为常规服务模式需求时,若数据类型为文件数据,则数据源系统定期主动推送相应地区编目信息,在数据接入层中的数据传输代理服务监控编目信息的变化并定期发送给多源异构数据跨系统协同管理系统;若数据类型为视频/业务数据,则数据源系统实时采集更新汇聚至多源异构数据跨系统协同管理系统;若数据类型为不同网络系统环境下的数据源数据,则通过数据接入层中与数据总线层进行跨网数据传输和信息交互;When the type of data requirement task is a regular service mode requirement, and if the data type is file data, the data source system will actively push the corresponding regional catalog information regularly. The data transmission agent service in the data access layer monitors the changes in the catalog information and periodically Send to the multi-source heterogeneous data cross-system collaborative management system; if the data type is video/business data, the data source system collects and updates in real time and aggregates it to the multi-source heterogeneous data cross-system collaborative management system; if the data type is in different network system environments The data source data carries out cross-network data transmission and information interaction through the data access layer and the data bus layer; 当数据需求任务的类型为应急服务模式需求时,若数据类型为航天遥感数据,则数据源系统接收到明确的实体数据需求加急任务后,一方面通过系统配置,提高优先权限,另一方面完成与其他任务的冲突消解,及时调整卫星轨道或姿态安排拍摄任务,及时获取数据/信息服务,并简化相应处理流程,同时根据具体资源类型、产品等级以及时间、区域需求,由数据源系统后台配置实现自动分发服务发送至多源异构数据跨系统协同管理系统;若数据类型为无人机数据,则数据源系统在接收到应急任务后,提交空中飞行申请,完成飞行任务和数据的下传处理,经多源异构数据跨系统协同管理系统传输至数据管理平台;若数据类型为视频和业务数据,数据源系统在接收到紧急任务后,调动突发事件所在地范围内的视频采集设备,并根据需求调整视频采集的角度和焦距,全方位对事件进行监控,同时调动智能分析资源,根据事件性质,实时分析相关人、车、物,对事件进行追溯、分析和发展预测,并将分析结果传送至多源异构数据跨系统协同管理系统;When the type of data demand task is emergency service mode demand, and if the data type is aerospace remote sensing data, after the data source system receives a clear urgent task of entity data demand, on the one hand, it will increase the priority through system configuration, and on the other hand, Complete the resolution of conflicts with other tasks, timely adjust the satellite orbit or attitude to arrange shooting tasks, obtain data/information services in a timely manner, and simplify the corresponding processing procedures. At the same time, according to specific resource types, product levels, time and regional requirements, the data source system backend Configure the automatic distribution service to be sent to the multi-source heterogeneous data cross-system collaborative management system; if the data type is drone data, after receiving the emergency mission, the data source system submits an application for air flight and completes the flight mission and data download. Processed and transmitted to the data management platform through the multi-source heterogeneous data cross-system collaborative management system; if the data type is video and business data, after receiving the emergency task, the data source system mobilizes the video collection equipment within the location of the emergency. And adjust the angle and focal length of video collection according to needs, monitor the event in an all-round way, and mobilize intelligent analysis resources to analyze relevant people, vehicles, and objects in real time according to the nature of the event, trace, analyze, and predict the development of the event, and analyze the event The results are sent to the multi-source heterogeneous data cross-system collaborative management system; 当数据需求任务的类型为专项服务模式需求时,若数据类型为遥感数据,则数据源系统根据任务明确的时间周期、频次、区域及载荷要求获取相应数据的编目信息或者实体数据并推送至多源异构数据跨系统协同管理系统;若数据类型为视频和业务数据,则数据源系统获取特定时间段的视频、业务数据及其智能分析结果并推送至多源异构数据跨系统协同管理系统;When the type of data requirement task is a special service mode requirement, and if the data type is remote sensing data, the data source system will obtain the cataloging information or entity data of the corresponding data according to the task's clear time period, frequency, area and load requirements and push it to multiple sources. Heterogeneous data cross-system collaborative management system; if the data type is video and business data, the data source system obtains the video, business data and their intelligent analysis results in a specific time period and pushes them to the multi-source heterogeneous data cross-system collaborative management system; 所述对所述数据需求任务进行解析,基于解析结果和所述数据需求任务的类型,将数据需求发送至对应的数据源系统,包括:The step of analyzing the data requirement task and sending the data requirement to the corresponding data source system based on the parsing result and the type of the data requirement task includes: 对所述数据需求任务进行解析,得出所述数据需求任务所涉及的数据源系统编号;Analyze the data requirement task and obtain the data source system number involved in the data requirement task; 根据解析的数据需求任务所涉及的数据源系统编号,将数据需求任务发送至对应的数据源系统。According to the data source system number involved in the parsed data requirement task, the data requirement task is sent to the corresponding data source system. 8.一种非临时性计算机可读存储介质,其上存储有计算机程序,其特征在于,所述计算机程序被处理器执行时实现如权利要求7中所述方法。8. A non-transitory computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, the method of claim 7 is implemented.
CN202210507966.9A 2022-05-10 2022-05-10 Multi-source heterogeneous data cross-system collaborative management system and method Active CN115134421B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210507966.9A CN115134421B (en) 2022-05-10 2022-05-10 Multi-source heterogeneous data cross-system collaborative management system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210507966.9A CN115134421B (en) 2022-05-10 2022-05-10 Multi-source heterogeneous data cross-system collaborative management system and method

Publications (2)

Publication Number Publication Date
CN115134421A CN115134421A (en) 2022-09-30
CN115134421B true CN115134421B (en) 2024-02-20

Family

ID=83376529

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210507966.9A Active CN115134421B (en) 2022-05-10 2022-05-10 Multi-source heterogeneous data cross-system collaborative management system and method

Country Status (1)

Country Link
CN (1) CN115134421B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115757343A (en) * 2022-11-21 2023-03-07 太极计算机股份有限公司 Data service acquisition method and device supporting multi-service collaboration
CN115904757A (en) * 2022-12-22 2023-04-04 广东南方新媒体科技有限公司 Heterogeneous data message delivery method, device, device and storage medium
CN118277460A (en) * 2022-12-29 2024-07-02 鼎捷软件股份有限公司 Data matching device and data matching method
CN116980449B (en) * 2023-09-14 2023-12-19 哈尔滨工程大学三亚南海创新发展基地 Collaborative management method, system, computer equipment and medium for data resource main body

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103944955A (en) * 2014-01-21 2014-07-23 中国科学院遥感与数字地球研究所 Handset space information service method based on wireless network
CN104639496A (en) * 2013-11-06 2015-05-20 北京航天长峰科技工业集团有限公司 Internet of things perception information acquisition method
CN106570081A (en) * 2016-10-18 2017-04-19 同济大学 Semantic net based large scale offline data analysis framework
CN107819824A (en) * 2017-10-09 2018-03-20 中国电子科技集团公司第二十八研究所 A kind of Urban Data opens and information service system and method for servicing
CN108737161A (en) * 2018-04-20 2018-11-02 中国人民解放军国防科技大学 Real-time data bus management method and system for intelligent agent cooperation system
CN109460952A (en) * 2018-10-25 2019-03-12 北京卫星信息工程研究所 Heterogeneous network converged communication device and communication means for Emergency Logistics
CN109992672A (en) * 2019-04-11 2019-07-09 华北科技学院 Knowledge Graph Construction Method Based on Disaster Scenario
CN110532493A (en) * 2019-08-29 2019-12-03 北京明略软件系统有限公司 Processing method and processing device, storage medium and the electronic device of data
WO2021222384A1 (en) * 2020-04-28 2021-11-04 Strong Force Intellectual Capital, Llc Digital twin systems and methods for transportation systems
CN114153920A (en) * 2021-11-19 2022-03-08 国家电投集团新疆能源化工有限责任公司 Big data edge platform and method
CN114398442A (en) * 2022-01-25 2022-04-26 中国电子科技集团公司第十研究所 Data-driven information processing system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11921672B2 (en) * 2017-07-31 2024-03-05 Splunk Inc. Query execution at a remote heterogeneous data store of a data fabric service
US20220027417A1 (en) * 2020-07-17 2022-01-27 Rapidsos, Inc. Modular application programming interface system

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104639496A (en) * 2013-11-06 2015-05-20 北京航天长峰科技工业集团有限公司 Internet of things perception information acquisition method
CN103944955A (en) * 2014-01-21 2014-07-23 中国科学院遥感与数字地球研究所 Handset space information service method based on wireless network
CN106570081A (en) * 2016-10-18 2017-04-19 同济大学 Semantic net based large scale offline data analysis framework
CN107819824A (en) * 2017-10-09 2018-03-20 中国电子科技集团公司第二十八研究所 A kind of Urban Data opens and information service system and method for servicing
CN108737161A (en) * 2018-04-20 2018-11-02 中国人民解放军国防科技大学 Real-time data bus management method and system for intelligent agent cooperation system
CN109460952A (en) * 2018-10-25 2019-03-12 北京卫星信息工程研究所 Heterogeneous network converged communication device and communication means for Emergency Logistics
CN109992672A (en) * 2019-04-11 2019-07-09 华北科技学院 Knowledge Graph Construction Method Based on Disaster Scenario
CN110532493A (en) * 2019-08-29 2019-12-03 北京明略软件系统有限公司 Processing method and processing device, storage medium and the electronic device of data
WO2021222384A1 (en) * 2020-04-28 2021-11-04 Strong Force Intellectual Capital, Llc Digital twin systems and methods for transportation systems
CN114153920A (en) * 2021-11-19 2022-03-08 国家电投集团新疆能源化工有限责任公司 Big data edge platform and method
CN114398442A (en) * 2022-01-25 2022-04-26 中国电子科技集团公司第十研究所 Data-driven information processing system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
esearch on multi-data sources data center architecture at UOIT;Xiaoli Meng;《IEEE Xplore》;全文 *
面向自然灾害应急任务的时空数据智能聚合方法;仇林遥;《CNKI中国知网》;正文第2.1-2.5节 *

Also Published As

Publication number Publication date
CN115134421A (en) 2022-09-30

Similar Documents

Publication Publication Date Title
CN115134421B (en) Multi-source heterogeneous data cross-system collaborative management system and method
US10367852B2 (en) Multiplexed demand signaled distributed messaging
AU2020325362B2 (en) Messaging platform for delivering real-time messages
KR102004160B1 (en) Apparatus and method for logical grouping method of iot connected client nodes using client identifier
US10484190B2 (en) Managing channels in an open data ecosystem
US8819080B2 (en) System and method for collection, retrieval, and distribution of data
TWI473455B (en) Airborne communication network
EP3861706B1 (en) Framework for dynamic brokerage and management of topics and data at the service layer
CN114531197B (en) On-orbit Distributed Information Resource Application Service System
CN101365119B (en) Video recording balance equalizing method used for network video monitoring system
US11070639B2 (en) Network infrastructure system and method for data processing and data sharing using the same
CN112118315A (en) Data processing system, method, apparatus, electronic device and storage medium
KR20140072044A (en) Distributing multi-source push notifications to multiple targets
US20160036876A1 (en) Bidirectional communication on live multimedia broadcasts
CN113364888B (en) Service scheduling method, system, electronic device and computer readable storage medium
US6442596B1 (en) Shared data views for large-scale multimode environments
KR102435830B1 (en) Method and architecture of Network Infrastructure for Optimal Application Service Processing and Data Sharing among Application domains
US20190297131A1 (en) System and Method for Querying and Updating a Live Video Stream Using a Structured Query Language
US20220269625A1 (en) Systems and methods for generic assurance framework
CN117272236B (en) Multi-source arrival time fusion method and system based on flight time sequence
Bano et al. A novel approach to distributed model aggregation using Apache Kafka
CN104572859B (en) A kind of distributed complex event handling system
CN115250279A (en) IoT Hub core processing module
CN116112560B (en) Data uplink and proxy method and system based on Reactive mechanism
CN118605361B (en) Data processing method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant