CN114520832B - Data stream processing method, device, equipment and computer storage medium - Google Patents
Data stream processing method, device, equipment and computer storage medium Download PDFInfo
- Publication number
- CN114520832B CN114520832B CN202011299080.7A CN202011299080A CN114520832B CN 114520832 B CN114520832 B CN 114520832B CN 202011299080 A CN202011299080 A CN 202011299080A CN 114520832 B CN114520832 B CN 114520832B
- Authority
- CN
- China
- Prior art keywords
- subscription
- data stream
- distributed
- data
- stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/542—Event management; Broadcasting; Multicasting; Notifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application provides a data stream processing method, a device, equipment and a computer storage medium, relates to the technical field of computers, and is used for improving the flexibility of a subscribing terminal for requesting a data stream through a data stream processing engine. The method comprises the following steps: the method comprises the steps that a data stream processing engine acquires a subscription broadcast stream of a data stream requested by a subscription end, and updates subscription configuration information based on the subscription broadcast stream, wherein the subscription broadcast stream carries subscription end indication information of the subscription end and data stream indication information of the data stream requested by the subscription end; and acquiring the data stream to be distributed, which is sent by the data generating end, determining each target subscribing end for requesting the data stream to be distributed based on the subscription configuration information, and sending the data stream to be distributed to each target subscribing end. According to the method, the subscription terminal can flexibly request the data stream provided by the data generating terminal only by recording and updating the subscription configuration information, so that the flexibility of the subscription terminal for requesting the data stream through the data stream processing engine is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data stream processing method, apparatus, device, and computer storage medium.
Background
In streaming data (streaming data) processing, a data stream provided by a data generating end is pushed to a subscribing end (i.e., an end requesting the data stream) typically through a streaming data processing framework provided by a data stream processing engine; however, the data stream provided by the data generating end at present is directly indicated to have the information of the subscribing end receiving the data stream, and then the data stream processing engine can only send the data stream to the subscribing end indicated in the data stream and can not send the data stream to other subscribing ends except the appointed subscribing end; therefore, if other subscribers want to request the data stream, the data generating end needs to be informed by the task, and then the data generating end regenerates the data stream, and the regenerated data stream is sent to the other subscribers by the data stream processing engine, so that the flexibility of delivering the data stream is poor, and the flexibility of requesting the data stream by the subscribers is low.
Disclosure of Invention
The embodiment of the application provides a data stream processing method, a device, equipment and a computer storage medium, which are used for improving the flexibility of a subscriber terminal for requesting a data stream through a data stream processing engine.
In a first aspect of the present application, a data stream processing method is provided, which is applied to a data stream processing engine, and includes:
acquiring a subscription broadcast stream of a data stream requested by a subscription end, and updating subscription configuration information based on the subscription broadcast stream, wherein the subscription broadcast stream carries subscription end indication information of the subscription end and data stream indication information of the data stream requested by the subscription end;
acquiring data streams to be distributed, which are sent by a data generation end, determining each target subscription end requesting the data streams to be distributed based on the subscription configuration information, and sending the data streams to be distributed to each target subscription end.
In a second aspect of the present application, there is provided a data stream processing apparatus comprising:
the broadcast stream processing unit is used for acquiring a subscription broadcast stream of a data stream requested by a subscription end, updating subscription configuration information based on the subscription broadcast stream, wherein the subscription broadcast stream carries subscription end indication information of the subscription end and data stream indication information of the data stream requested by the subscription end;
the data stream processing unit is used for acquiring the data stream to be distributed sent by the data generating end, determining each target subscribing end for requesting the data stream to be distributed based on the subscription configuration information, and sending the data stream to be distributed to each target subscribing end.
In a possible implementation manner, the data stream processing unit is specifically configured to:
if the to-be-distributed data stream carries the subscription end indication information, determining each target subscription end in the subscription ends indicated by the subscription end indication information based on the subscription configuration information.
In one possible implementation manner, the broadcast stream processing unit is further configured to obtain a subscribed broadcast stream from the first message middleware before obtaining a subscribed broadcast stream of the data stream requested by the subscriber end, and obtain a label of the data stream to be distributed to the obtained subscribed broadcast stream or one of the data streams to be distributed from the second message middleware to form a data stream to be processed; and
the broadcast stream processing unit is specifically configured to: acquiring the data stream to be processed, and separating the subscription broadcast stream from the data stream to be processed according to the tag;
the data stream processing unit is specifically configured to: and acquiring the data stream to be processed, and separating the data stream to be distributed from the data stream to be processed according to the tag.
In a third aspect the application provides a computer apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of the first aspect and any one of the possible implementations when executing the program.
In a fourth aspect of the application, a computer program product is provided, the computer program product comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from a computer-readable storage medium by a processor of a computer device, which executes the computer instructions, causing the computer device to perform the method provided in the various possible implementations of the first aspect described above.
In a fifth aspect of the application, there is provided a computer readable storage medium storing computer instructions that, when run on a computer, cause the computer to perform a method as described in any of the first aspect and any of the possible implementations.
Due to the adoption of the technical scheme, the embodiment of the application has at least the following technical effects:
according to the method and the device for processing the data streams, the data stream processing engine can dynamically acquire information of the data streams requested by the subscribing terminals based on the subscription broadcast streams, and then the data stream processing engine distributes the data streams to be distributed to each target subscribing terminal based on the recorded subscription configuration information.
In the second aspect, in the embodiment of the present application, subscription configuration information is updated in real time based on a subscription broadcast stream of a subscription end, so that accuracy of a determined target subscription end can be improved, and further accuracy of sending a data stream to be distributed is improved.
In the third aspect, in the embodiment of the application, the dynamic subscription of the data stream to be distributed is realized under the condition that no additional module component is added, the complexity of the system is reduced, and the efficiency of dynamically requesting the data stream by the subscription terminal through the data stream processing engine is improved.
Drawings
Fig. 1 is a schematic diagram of an application scenario provided in an embodiment of the present application;
fig. 2 is an exemplary diagram of another application scenario provided in an embodiment of the present application;
fig. 3 is a schematic diagram illustrating a procedure of a data stream processing method according to an embodiment of the present application;
fig. 4 is a schematic diagram of a process of sending a data stream to be distributed according to an embodiment of the present application;
fig. 5 is a schematic diagram of a process for determining a target subscription end according to an embodiment of the present application;
fig. 6 is a schematic process diagram of a data stream processing method according to an embodiment of the present application;
fig. 7 is a block diagram of a data stream processing apparatus according to an embodiment of the present application;
fig. 8 is a block diagram of a computer device according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions provided by the embodiments of the present application, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
In order to facilitate the technical solution of the present application to be better understood by those skilled in the art, the following description of technical terms related to the present application is provided.
1) Data stream processing engine
The data processing engine that processes streaming data (i.e., data streams) may include, but is not limited to, apache, flink, storm, samza, etc.; the Flink is a framework for stateful computing processing of unbounded and bounded data streams, and is a relatively efficient data stream processing engine that considers all data to exist in the form of streams.
2) Message queue
A Message Queue (MQ) is a container in which messages (data) are stored, i.e., a Queue in which data to be transmitted is stored; the message queue is an asynchronous communication mode between services, is an important component in a distributed system, can be used for publishing and subscribing messages, mainly solves the problems of application coupling, asynchronous message, flow peak clipping and the like, and realizes a high-performance, high-availability, scalable and final consistency architecture; common message queues are RocketMQ, rabbitMQ, kafka and the like; where Kafka is a distributed, multi-partitioned, multi-copy, multi-subscriber, zookeeper-based message queue (also known as a distributed log system).
3) Subscription terminal and data generation terminal
The subscription end in the embodiment of the application is equipment or service for requesting data flow (stream data), and the subscription end can be terminal equipment, a message queue and a service platform; the data generating end in the embodiment of the application is equipment or service for providing or generating data flow, and the data generating end can be but is not limited to a message queue, a service platform and the like; the service platform related to the embodiment of the application is also called as system service, and can include, but is not limited to, a game service platform, a video service platform, a live broadcast service platform, a content sharing service platform, a traffic service platform and the like.
4) Subscribing to broadcast streams
The subscription broadcast stream in the embodiment of the application is a broadcast message for requesting the data stream, the subscription broadcast stream can be triggered by the subscription terminal, the subscription broadcast stream can carry subscription terminal indication information for indicating the identity of the subscription terminal, and the subscription broadcast stream can also carry data stream indication information for indicating the data stream requested by the subscription terminal.
5) Data stream, data stream to be distributed, data stream to be processed
The data stream in the embodiment of the application is stream data provided by a data generating end; the data stream to be distributed is a data stream to be processed by the data stream processing engine, wherein the data generating end sends the data stream to the data stream processing engine; the data stream to be processed is formed by labeling one of the obtained subscription broadcast streams or the data streams to be distributed by the data stream processing engine.
6) Subscribing to configuration information
The subscription configuration information in the embodiment of the application is information for indicating the subscriber to request the data stream, and the subscription configuration information can include subscriber indication information of the subscriber and data stream indication information of the data stream requested by the subscriber.
7) Cloud service technology (Cloud technology)
The cloud service technology is a generic term of network technology, information technology, integration technology, management platform technology, application technology and the like based on cloud computing business model application, and can form a resource pool, and the cloud service technology is flexible and convenient as required. The cloud service technology is an important support; background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, and the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data need strong system rear shield support and can be realized only through a cloud service technology.
The following describes the design concept of the present application.
In streaming data (streaming data) processing, a data stream sent by a data generating end is pushed to a fixed subscribing end through a streaming data processing framework of a data stream processing engine; in the process of stream data processing, after a subscriber terminal subscribes to a data stream of a data generation terminal, a stream data processing framework indicates a fixed subscriber terminal subscribing to the data stream in a task; after the data generating end starts the task, determining a fixed subscription end for receiving the data stream based on the task, wherein the fixed subscription end is indicated in the data stream generated after the task is started, and the information of the fixed subscription end for receiving the data stream cannot be modified after the data generating end starts the processing task; and after receiving the data stream, the stream processing engine only supports the fixed subscription end indicated by the data stream based on the indication of the data stream, but cannot deliver the data stream to other subscription ends except the fixed subscription end, so that each subscription end cannot flexibly subscribe the data stream provided by the data generating end.
In view of this, the inventors devised a data stream processing method, apparatus, device, and computer storage medium; considering that a fixed subscription end for receiving the data stream is already designated in the data stream in the related technology, so that the data stream processing engine can only push the data stream to the fixed subscription end; meanwhile, in order to improve the accuracy of data stream distribution, the data stream processing engine updates subscription configuration information in real time based on the subscription broadcast stream of the subscriber end in the process of processing the data stream, and further distributes the received data stream to the subscriber end requesting the data stream based on the updated subscription configuration information.
In order to more clearly understand the design concept of the present application, the following description will be given by way of example to an application scenario of an embodiment of the present application.
Referring to fig. 1, an exemplary diagram of an application scenario for subscribing to a data stream is shown. The application scene comprises at least one data generating end 100, a data stream processing engine 200 and at least one subscribing end 300, wherein the data generating end 100 and the data stream processing engine 200 can communicate, and the data stream processing engine 200 and the subscribing end 300 can communicate; wherein:
The data generating end 100 may generate a data stream after starting a task and send the data stream to the data stream processing engine 200.
The data stream processing engine 200 (such as but not limited to including 200-1, 200-2, or 200-3 illustrated in the figure) may obtain a subscription broadcast stream of the data stream requested by the subscriber 300, and update the subscription configuration information based on the subscription broadcast stream; acquiring a data stream to be distributed, which is sent by the data generating end 100, and sending the data stream to be distributed to each target subscribing end requesting the data stream to be distributed based on the subscription configuration information; the above-described data stream to be distributed is a data stream generated by the data generating terminal 100.
The subscriber 300 may transmit a subscription broadcast stream requesting a data stream to the data stream processing engine 200 and receive a data stream to be distributed transmitted from the data stream processing engine 200.
As an embodiment, considering the network environment and the influence of the storage capacity and the processing capacity of the data stream processing engine 200, if the number of data streams and subscription broadcast streams reaching the data stream processing engine 200 in a short time is excessive, stacking or blocking of the data streams and subscription broadcast streams may occur, thereby having serious influence on the whole data stream processing system, so that the embodiment of the present application designs to use message middleware to cache the data streams and subscription broadcast streams to avoid adverse influence of a large amount of stream data on the data stream processing engine 200.
In particular, referring to fig. 2, another exemplary diagram of an application scenario for subscribing to a data stream is provided. The application scene comprises at least one data generating end 100, a data stream processing engine 200 and at least one subscribing end 300, wherein a first message middleware 210 is arranged between the data stream processing engine 200 and the subscribing end 300, and a second message middleware 220 is arranged between the data generating end 100 and the data stream processing engine 200; wherein:
the data generating terminal 100 (such as, but not limited to, the generating terminal 100-1, 100-2 or 100-3 illustrated in the figure) may generate a data stream after the task is started, and send the data stream to the second message middleware 220.
The subscriber end 300 (as may be, but not limited to including, 300-1 or 300-2 as illustrated in the figures) may send a subscription broadcast stream directly to the data stream processing engine 200; the subscriber 300 may further send the subscriber indication information and the data stream indication information to the service node 400, where the service node 400 processes the subscription request into a subscription broadcast stream, and sends the subscription broadcast stream to the data stream processing engine 200 through the first message middleware 210; the service node 400 may be a separate server, a server cluster, or the like.
The data stream processing engine 200 (such as but not limited to including 200-1, 200-2 or 200-3 shown in the figure) may directly receive the subscription broadcast stream sent by the subscription terminal 300 after triggering the processing of the data stream and the subscription broadcast stream, or may acquire the subscription broadcast stream from the first message middleware 210, and update the subscription configuration information based on the subscription broadcast stream; and obtaining the data stream to be distributed from the second message middleware 220, and sending the data stream to be distributed to the subscriber terminal 300 based on the subscription configuration information.
Further, the data stream processing engine 200 may acquire a plurality of subscription broadcast streams from the first message middleware 210 at a time, and when acquiring a plurality of data streams to be distributed from the second message middleware 220 at a time, store the acquired subscription broadcast streams and the acquired data streams to be distributed in a buffer space thereof; the data stream processing engine 200 can sequentially take out the subscription broadcast stream and the data stream to be distributed from the self cache space for processing; when the data in the own cache space is empty, the subscription broadcast stream is acquired again from the first message middleware 210, and the data stream to be distributed is acquired from the second message middleware 220.
Furthermore, when the data stream processing engine 200 sequentially takes out the subscription broadcast stream and the data stream to be distributed from its own buffer space for processing, one subscription broadcast stream or one data stream to be distributed may be taken out of the buffer space at a time for processing, or one subscription broadcast stream and one data stream to be distributed may be taken out of the buffer space at a time at the same time, and the subscription broadcast stream and the data stream to be distributed which are taken out at the same time form one data stream to be processed, so that the data stream processing engine 200 processes the data stream to be processed.
The data processing engine 200 in the embodiment of the present application may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, for example, may be a plurality of blockchain servers providing blockchain services in a blockchain technology, or may be a plurality of cloud servers providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms in a cloud service technology (for example, but not limited to, a server 200-1, a server 200-2 or a server 200-3 illustrated in the figure); the functions of the data processing engine 200 described above may be implemented by one or more cloud servers, by one or more clusters of cloud servers, etc.
The data generating end 100 in the embodiment of the present application may be, but not limited to, a message queue, a service platform, etc., and the subscribing end 300 may be, but not limited to, a terminal device, a message queue, a service platform, etc.
The message middleware (such as the first message middleware 210 or the second message middleware 220) in the embodiment of the present application may be, but not limited to, a message queue, an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a plurality of cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud storage, network services, cloud communication, middleware services, platforms, and the like in the cloud service technology; the functions of the message middleware can be realized by one or more cloud servers, one or more cloud server clusters, and the like.
The message queue and the service platform related in the embodiment of the application can be independent physical servers, can be a server cluster or a distributed system formed by a plurality of physical servers, and can also be a plurality of cloud servers for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligent platforms in the cloud service technology; the functions of the message queue or the service platform can be realized by one or more cloud servers, can also be realized by one or more cloud server clusters, and the like.
The terminal device involved in the embodiments of the present application may be, but is not limited to, a mobile terminal, a fixed terminal or a portable terminal, such as a mobile handset, a site, a unit, a device, a multimedia computer, a multimedia tablet, an internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a Personal Communication System (PCS) device, a personal navigation device, a Personal Digital Assistant (PDA), an audio/video player, a digital camera/video camera, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof.
Based on the application scenarios of fig. 1 and fig. 2, an exemplary description is given below of a data stream processing method according to an embodiment of the present application; referring to fig. 3, a data stream processing method applied to the data stream processing engine 200 is provided, which specifically includes the following steps:
step 301, acquiring a subscription broadcast stream of a data stream requested by a subscriber, and updating subscription configuration information based on the acquired subscription broadcast stream, wherein the subscription broadcast stream carries subscriber indication information of the subscriber and data stream indication information of the data stream requested by the subscriber.
Specifically, the obtained subscription broadcast stream is the subscription broadcast stream to be processed by the data stream processing engine 200; the data stream processing engine 200 may, but is not limited to, determine the subscription configuration information indicated by the subscription end indication information based on the subscription end indication information in the subscription broadcast stream, and further update the determined subscription configuration information based on the subscription broadcast stream total data stream indication information.
The subscription terminal indication information in the embodiment of the application is used for indicating the identity information of the subscription terminal, and the subscription terminal indication information can be, but is not limited to, identification information of the subscription terminal, wherein the identification information of the subscription terminal can be account information of the subscription terminal or an Internet Protocol (IP) address of the subscription terminal; when the subscription terminal is a service platform (for example, but not limited to, a website), the subscription terminal indication information may also be a website address or domain name of the website, etc., and those skilled in the art may set the subscription terminal indication information according to actual requirements.
The data flow indication information in the embodiment of the present application is used to indicate the data flow requested by the subscriber, and the data flow indication information may be, but is not limited to, a content theme of the data flow, a generating end identifier of a data generating end for generating the data flow, a data type of the data flow, a website or domain name of a service platform for providing the data flow, etc., and those skilled in the art may set the data flow indication information according to actual needs.
Step S302, obtaining the data stream to be distributed sent by the data generating end, determining each target subscribing end requesting the data stream to be distributed based on the subscription configuration information, and sending the data stream to be distributed to each target subscribing end.
Specifically, in the embodiment of the present application, the subscription end receiving the data stream to be distributed may not be specified, that is, the subscription end indication information of the subscription end does not need to be carried in the data stream to be distributed, and further in step S302, the data stream processing engine 200 may directly distribute the data stream to be distributed to each determined target subscription end.
As an embodiment, since the embodiment of the present application may include multiple subscription ends such as a terminal device, a message queue, or a service platform, etc., subscription types of data flows requested by different subscription ends may be different, for example, when the subscription ends are different message queues, data transmission protocols between different message queues and the data flow processing engine 200 are different, so in the embodiment of the present application, in order to improve a transmission success rate of a data flow to be distributed, the data flow to be distributed may also be sent to each target subscription end according to a data transmission protocol matched with a subscription type of each target subscription end based on a subscription type of the target subscription end, which will be described in detail later.
The following describes a procedure in which the data stream processing engine 200 acquires a subscription broadcast stream in step S301 and acquires a data stream to be distributed in step S302:
as an embodiment, the data stream processing engine 200 may determine the subscription broadcast stream sent by the subscriber 300 or the first message middleware 210 as the acquired subscription broadcast stream in step S301; the subscription broadcast stream may also be actively grabbed from the first message middleware 210, and the actively grabbed subscription broadcast stream is determined to be the acquired subscription broadcast stream; the subscription broadcast stream grabbed from the self cache space can also be determined to be the acquired subscription broadcast stream; for the application scenario illustrated in fig. 1, the subscription end 300 may directly send the subscription broadcast stream to the data stream processing engine 200, and further, the data stream processing engine 200 may determine the subscription broadcast stream sent by the subscription end as the acquired subscription broadcast stream; for the application scenario illustrated in fig. 2, the first message middleware 210 caches the subscription broadcast stream of one or more subscription ends, and the first message middleware 210 may actively send the subscription broadcast stream to the data stream processing engine 200, or the data stream processing engine 200 may actively capture the subscription broadcast stream from the first message middleware 210.
In step S302, the data stream processing engine 200 may determine the data stream sent by the data generating end 100 or the second message middleware 220 as an acquired data stream to be distributed; the data stream can be actively grabbed from the second message middleware 220, and the actively grabbed data stream is determined as the data stream to be distributed; the data stream grabbed from the self cache space can be determined as the data stream to be distributed; for the application scenario illustrated in fig. 1, the data generating end 100 may directly send a data stream to the data stream processing engine 200, and further, the data stream processing engine 200 may determine the data stream sent by the data generating end 100 as a data stream to be distributed; for the application scenario illustrated in fig. 2, the second message middleware 220 caches one or more data flows of the data generating end 100, and the second message middleware 220 may actively send the data flows to the data flow processing engine 200, or the data flow processing engine 200 may actively grab the data flows from the second message middleware 220 as the data flows to be distributed.
As an embodiment, in order to improve the processing efficiency of the data stream processing engine and improve the accuracy of sending the data stream to be distributed, before the data stream processing engine 200 acquires the subscription broadcast stream of the subscription end request data stream in step S301, the data stream processing engine 200 may also acquire the subscription broadcast stream and the data stream to be distributed at the same time, and tag one of the acquired subscription broadcast stream or the data stream to be distributed to form the data stream to be processed; when a subscription broadcast stream of a subscriber-side request data stream is acquired in step S301, acquiring the data stream to be processed, and separating the subscription broadcast stream from the data stream to be processed according to the tag; when the data stream to be distributed sent by the data generating end is obtained in step S302, the data stream to be distributed may be separated from the data stream to be processed according to the tag;
Wherein the data stream processing engine 200 may acquire the subscription broadcast stream from the first message middleware 210 and acquire the data stream to be distributed from the second message middleware 220 at the same time, or the data stream processing engine 200 may acquire the subscription broadcast stream or the data stream to be distributed from the buffer space at the same time; and the tag can be used as a connection character, and the subscription broadcast stream and the data stream to be distributed are directly spliced to obtain the data stream to be processed.
As an embodiment, in the embodiment of the present application, subscription configuration information of each subscription end is recorded in the data stream processing engine 200, that is, a subscription configuration information set is recorded in the data stream processing engine 200, where the subscription configuration information set includes subscription configuration information of each subscription end that has sent a subscription broadcast stream; where data stream processing engine 200 is a separate physical server, the subscription profile information set may be stored locally on the physical server; when the data stream processing engine 200 is a server cluster or a distributed system, a set of subscription configuration information may be recorded in the server cluster or the distributed system, each server in the server cluster or the distributed system may access the recorded set of subscription configuration information, and in step S301, subscription configuration information in the recorded set of subscription configuration information may be updated in real time according to the acquired subscription broadcast stream.
Wherein the data stream processing engine 200 may, but is not limited to, record the subscription configuration information of each subscriber in the subscription configuration information set in the form of key value pairs or fixed data types, and when the data stream processing engine 200 is a link, the link may record the subscription configuration information of each subscriber in the link state in the form of map < state, cnf >, where state includes the subscription indication information of the subscriber, and cnf may, but is not limited to, include the data stream indication information.
The following details of the process of sending the data stream to be distributed to each target subscription end in step S302 based on the subscription type of the target subscription end are described in the embodiments of the present application:
as an embodiment, the matching relationship between the subscription type and the data transmission protocol is not limited too much, and those skilled in the art may set the matching relationship between the subscription type and the data transmission protocol according to actual requirements, for example, in order to promote the message processed by the data stream, but not limited to preset matching relationship between the subscription type and the data transmission protocol, or set the matching relationship between the subscription type and the data transmission protocol based on the related information of the subscription end, etc.
As an embodiment, in order to quickly and accurately determine the subscription type of each target subscription end, the data stream processing engine 200 in the embodiment of the present application may further record subscription type information of each subscription end, where the subscription type information is used to indicate the subscription type of the subscription end; specifically, each subscription end may send respective subscription type information to the data stream processing engine 200 through a message other than the subscription broadcast stream, and each subscription end may also send respective subscription type information to the data stream processing engine 200 while carrying the subscription type information in the subscription broadcast stream; further, after receiving the subscription type information of each subscription end, the data stream processing engine 200 may record the subscription type information in the subscription configuration information, or record the subscription type information in information other than the subscription configuration information; if the data stream processing engine 200 is a link, the link may record subscription type information of the subscription end in cnf in the map < state, cnf >; the link may also record the subscription type information of the subscription end in the form of map < key, client >, where the key includes the subscription end indication information of the subscription end, and the client may but is not limited to include the subscription type information of the subscription end.
When the subscription type information of the subscription end can be carried in the subscription broadcast stream, after receiving the subscription broadcast stream, the data stream processing engine 200 can update the subscription type information of each subscription end based on the subscription type information of the subscription end carried in the subscription broadcast stream; when the data stream processing engine 200 sends the data stream to be distributed to each target subscription end, the subscription type of each target subscription end can be determined based on the recorded subscription type information; and respectively sending the data streams to be distributed to each target subscribing terminal according to a data transmission protocol matched with the subscription types of each target subscribing terminal.
For ease of understanding, the following describes an exemplary subscription type of the subscription end according to the present application:
when the subscription end is a different message queue, the subscription type can be determined according to the queue type of the message queue, if the subscription end comprises ActiveMQ, rabbitMQ, rocketMQ, kafka, ES, the subscription type can be an ActiveMQ type, a RabbitMQ type, a RocketMQ type, a Kafka type, an ES type and the like.
When the subscription terminal includes different service platforms, the subscription type may be, but is not limited to, service type based on the service platform or domain name determination of the corresponding website, etc.
The subscription terminal comprises different terminal devices, and the subscription type can be determined based on the type of the terminal device, the network where the terminal device is positioned, the position and the like; further, when the subscription end is a terminal device and the delivery of the subscription broadcast stream and the data stream to be distributed is performed between the terminal device and the data stream processing engine 200 through the first message middleware 210, the data stream processing engine 200 may, but is not limited to, send the data stream to be distributed to each target subscription end based on the following manner: determining message middleware corresponding to the subscription type of the target subscription end according to each target subscription end respectively; and respectively sending the data streams to be distributed to message middleware corresponding to the subscription types of the target subscription ends according to a data transmission protocol matched with the subscription types of the target subscription ends, so that the message middleware sends the data streams to be distributed to the target subscription ends corresponding to the subscription types.
Referring to fig. 4, for example, the determined target subscription end includes a subscription end 1, a subscription end 2, a subscription end 3, and a subscription end 4, where subscription types of the subscription end 1 to the subscription end 3 are Kafka type subscribers, and when subscription types of the subscription end 4 are dockmq, the data stream processing engine 200 determines that, when bypassing the data stream (i.e. sending the data stream to be distributed to the target subscription end), the data stream to be distributed bypassing the subscription end 1 to the subscription end 3 is sent to a Kafka type data stream channel, and sends the data to be distributed to the subscription end 1 to the subscription end 3 according to a Kafka data transmission protocol matched with the Kafka type; while the data stream processing engine 200 bypasses the data stream to be distributed of the subscriber terminal 4 to the data stream channel of the dockmq type and transmits the data to the subscriber terminal 4 according to the dockmq transport protocol matched with the dockmq type.
As an embodiment, there is a possibility that no subscription end currently requests the data stream to be distributed, in this case, in order to ensure that tasks related to the data stream can be performed normally, the data stream to be distributed may be directly discarded, or the data stream to be distributed may be stored for subsequent processing based on service requirements; specifically, if the target subscription end is not determined based on the subscription configuration information, discarding the data stream to be distributed.
As an embodiment, in some service scenarios, it may be specified that the data stream generated by the data generating end is desired to be consumed by the subscribing end (i.e., the data stream is requested by the subscribing end), where in this case, after determining that no subscribing end requests the data stream to be distributed, the data stream to be distributed may be cached to a storage duration, and further determining whether to distribute the data stream to be distributed to the corresponding subscribing end based on updating of subscription configuration information of each subscribing end in the storage duration;
specifically, referring to fig. 5, if the target subscription end is not determined based on the subscription configuration information, the data stream to be distributed is stored, and when the storage time length of the data stream to be distributed reaches the time length threshold value, whether the target subscription end exists is redetermined based on the latest subscription configuration information, if the target subscription end exists is redetermined, the stored data stream to be distributed is distributed to the redetermined target subscription end, otherwise, whether the redetermined times reach the set times, if the redetermined times do not reach the target subscription end, the stored data stream to be distributed is discarded, otherwise, the storage time length of the data stream to be distributed is redetected after the storage time length is cleared, and when the storage time length reaches the time length threshold value again, whether the target subscription end exists is redetermined based on the latest subscription configuration information, or not, and when the time difference between the storage time lengths of the time length and the latest redetermined time is up to the time length threshold value is redetermined, whether the target subscription end exists is redetermined based on the latest time length; the storage time length, the time length threshold value and the set times are not excessively limited, and can be set by a person skilled in the art according to actual requirements.
As an embodiment, in order to improve the flexibility of delivering a data stream by a data generating end, in the embodiment of the present application, the data generating end 100 may also carry subscription end indication information in a data stream to be distributed, so as to indicate to the data stream processing engine 200 that only a subscription end corresponding to the subscription end indication information carried in the data stream to be distributed has permission to acquire the data stream to be distributed; and when determining to request each target subscription end of the data stream to be distributed, if the data stream to be distributed carries subscription end indication information, the data stream processing engine 200 determines each target subscription end in the subscription ends indicated by the subscription end indication information carried in the data stream to be distributed based on the subscription configuration information.
Specifically, the subscription end for requesting the data stream to be distributed may be determined based on the latest subscription configuration information, and then, from the determined subscription ends, the subscription end indicated by the subscription end indication information carried in the data stream to be distributed is determined as the target subscription end; the subscription terminal indicated by the subscription terminal indication information carried in the data stream to be distributed can be determined first, the subscription configuration information of the determined subscription terminal is obtained from the latest subscription configuration information, and then the subscription terminal requesting the data stream to be distributed in the determined subscription terminal is determined to be the target subscription terminal based on the obtained subscription configuration information.
In order to facilitate understanding of the technical solution provided by the embodiments of the present application, a specific example of a data stream processing method is provided in the following.
In this example, a Flink is described as the data stream processing engine 200.
Referring to fig. 6, in a dynamic subscription process of a data stream based on a link, a first message middleware aggregates subscription configuration information of each subscription end into a subscription broadcast (broadcast) stream by means of a message queue and the like, and a second message middleware caches the data stream sent by each data generating end in the second middleware;
the method comprises the steps that a Flink engine acquires a subscription broadcast stream of a subscription end, acquires a data stream sent by a data generation end as a data stream to be distributed, further a task process (TaskManger) in the Flink executes a task through task resources (Taskslot), and connects the acquired subscription broadcast stream with the data stream to be distributed to obtain data to be processed, wherein the data stream to be processed is actually a new spliced data broadcast stream; the Flink engine can acquire the data stream to be processed, update and cache subscription configuration information map < state, cnf > according to subscription configuration information in the data stream to be processed, and update and cache client information map < key, client > corresponding to the subscription type; and finally, bypassing the data streams to be distributed requested by each target subscription end from the data streams to be processed according to the subscription types, and further sending the data streams to be distributed to each target subscription end subscribing the data streams to be distributed.
In the process of sending the data stream to be distributed, the Flink sends the data stream to be distributed to each target subscribing terminal according to a data transmission protocol matched with the subscribing type of each target subscribing terminal; specifically, the link may perform a copy bypass on the data stream to be distributed based on the total number of subscription types of the subscription end that is known in advance, to obtain copies of the data stream to be distributed with the total number as bypass data streams; and then determining each target subscription end based on the latest subscription configuration information map < state, cnf >, determining the subscription type (client information) of each target subscription end based on the latest map < key, client >, and then respectively sending each bypass data stream to each corresponding target subscription end according to the data transmission protocol matched with the subscription type of each target subscription end.
According to the embodiment of the application, only one piece of subscription configuration information of the subscription terminal is required to be recorded, and the recorded subscription configuration information is updated in real time according to the received subscription broadcast stream, so that the subscription terminal can flexibly request the data stream provided by the data generating terminal according to the recorded subscription configuration information, and the flexibility of the subscription terminal for requesting the data stream through the data stream processing engine is improved.
Referring to fig. 7, based on the same inventive concept, an embodiment of the present application provides a data stream processing apparatus 700, including:
a broadcast stream processing unit 701, configured to obtain a subscription broadcast stream of a data stream requested by a subscription end, and update subscription configuration information based on the subscription broadcast stream, where the subscription broadcast stream carries subscription end indication information of the subscription end and data stream indication information of the data stream requested by the subscription end;
the data stream processing unit 702 is configured to obtain a data stream to be distributed sent by a data generating end, determine each target subscribing end that requests the data stream to be distributed based on the subscription configuration information, and send the data stream to be distributed to each target subscribing end.
As an example, the apparatus of fig. 7 may be used to implement any of the data stream processing methods discussed above.
As an embodiment, the subscription broadcast stream further carries subscription type information of a subscription end, and the broadcast stream processing unit 701 is further configured to: updating the subscription type information of each subscription end based on the subscription type information of the subscription end carried in the subscription broadcast stream; and
the data stream processing unit 702 is specifically configured to:
Determining the subscription type of each target subscription terminal based on the recorded subscription type information;
and respectively sending the data streams to be distributed to each target subscribing terminal according to a data transmission protocol matched with the subscription types of each target subscribing terminal.
As an embodiment, the data stream processing unit 702 is further configured to:
if the target subscription end is not determined based on the subscription configuration information, discarding the data stream to be distributed; or storing the data stream to be distributed, and when the storage time length of the data stream to be distributed reaches the time length threshold value, re-determining whether a target subscription end exists or not based on the latest subscription configuration information, and discarding the stored data stream to be distributed if the target subscription end is not re-determined by the set times.
As an embodiment, the data stream processing unit 702 is specifically configured to:
if the data stream to be distributed carries the subscription terminal indication information, determining each target subscription terminal in the subscription terminals indicated by the subscription terminal indication information based on the subscription configuration information.
As an embodiment, before acquiring the subscription broadcast stream of the subscription end request data stream, the broadcast stream processing unit 701 is further configured to acquire the subscription broadcast stream from the first message middleware, and acquire the to-be-distributed data stream from the second message middleware, and form the to-be-processed data stream after labeling one of the acquired subscription broadcast stream or the to-be-distributed data stream; and
The broadcast stream processing unit 701 specifically is configured to: acquiring the data stream to be processed, and separating the subscription broadcast stream from the data stream to be processed according to the tag;
the data stream processing unit 702 is specifically configured to: and acquiring the data stream to be processed, and separating the data stream to be distributed from the data stream to be processed according to the tag.
As an example of a hardware entity, the data stream processing apparatus 700 is a computer device shown in fig. 8, where the computer device includes a processor 801, a storage medium 802, and at least one external communication interface 803; the processor 801, the storage medium 802, and the external communication interface 803 are all connected via a bus 804.
The storage medium 802 has stored therein a computer program;
the processor 801, when executing the computer program, implements one of the data stream processing methods discussed above.
One processor 801 is exemplified in fig. 8, but the number of processors 801 is not limited in practice.
Wherein the storage medium 802 may be a volatile memory (RAM) such as a random-access memory (RAM); the storage medium 802 may also be a non-volatile memory (non-volatile memory), such as a read-only memory, a flash memory (flash memory), a hard disk (HDD) or a Solid State Drive (SSD), or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. The storage medium 802 may be a combination of the storage media described above.
As an embodiment, the processor 801 in fig. 8 may be used to implement the functions of the broadcast stream processing unit 701 and the data stream processing unit 702 as in fig. 7.
As an example, each aspect of the video play control method provided by the embodiment of the present application may also be implemented in the form of a program product, which includes program code for causing a computer device to execute the steps in the data stream processing method according to the various exemplary embodiments of the present application described above in the present specification, when the program product is run on the computer device.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware associated with program instructions, where the foregoing program may be stored in a computer readable storage medium, and when executed, the program performs steps including the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk or an optical disk, or the like, which can store program codes.
Alternatively, the above-described integrated units of the application may be stored in a computer-readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solution of the embodiments of the present application may be embodied in essence or a part contributing to the prior art in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the above-mentioned methods of the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.
Based on the same technical idea, an embodiment of the present application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform a data stream processing method as previously discussed.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. A data stream processing method, applied to a data stream processing engine, comprising:
Acquiring a subscription broadcast stream of a data stream requested by a subscription end, and updating subscription configuration information based on the subscription broadcast stream, wherein the subscription broadcast stream carries subscription end indication information of the subscription end and data stream indication information of the data stream requested by the subscription end;
acquiring data streams to be distributed, which are sent by a data generation end, determining each target subscription end requesting the data streams to be distributed based on the subscription configuration information, and sending the data streams to be distributed to each target subscription end.
2. The method of claim 1, wherein the subscription broadcast stream further carries subscription type information of a subscription end, the method further comprising: updating the subscription type information of each subscription end based on the subscription type information of the subscription end carried in the subscription broadcast stream; and
the sending the data stream to be distributed to each target subscriber terminal includes:
determining the subscription type of each target subscription terminal based on the recorded subscription type information;
and respectively sending the data streams to be distributed to each target subscribing terminal according to a data transmission protocol matched with the subscription types of each target subscribing terminal.
3. The method according to claim 1 or 2, wherein if the target subscription end is not determined based on the subscription configuration information, the method further comprises:
Discarding the data stream to be distributed; or (b)
Storing the data stream to be distributed, and when the storage duration of the data stream to be distributed reaches a duration threshold value, re-determining whether a target subscription end exists or not based on the latest subscription configuration information, and discarding the stored data stream to be distributed if the target subscription end is not re-determined by the set times.
4. The method of claim 3, wherein determining each target subscriber requesting the data stream to be distributed based on the subscription configuration information, specifically comprises:
if the to-be-distributed data stream carries the subscription end indication information, determining each target subscription end in the subscription ends indicated by the subscription end indication information based on the subscription configuration information.
5. The method of claim 1 or 2, wherein prior to the acquiring the subscribed broadcast stream of the subscriber-side request data stream, the method further comprises:
acquiring a subscription broadcast stream from a first message middleware, and acquiring a to-be-distributed data stream from a second message middleware, wherein the acquired subscription broadcast stream or one label tag in the to-be-distributed data stream forms a to-be-processed data stream; and
The acquiring the subscription broadcast stream of the request data stream of the subscription terminal specifically comprises the following steps: acquiring the data stream to be processed, and separating the subscription broadcast stream from the data stream to be processed according to the tag;
the obtaining the data stream to be distributed sent by the data generating end specifically comprises the following steps: and acquiring the data stream to be processed, and separating the data stream to be distributed from the data stream to be processed according to the tag.
6. A data stream processing apparatus, comprising:
the broadcast stream processing unit is used for acquiring a subscription broadcast stream of a data stream requested by a subscription end, updating subscription configuration information based on the subscription broadcast stream, wherein the subscription broadcast stream carries subscription end indication information of the subscription end and data stream indication information of the data stream requested by the subscription end;
the data stream processing unit is used for acquiring the data stream to be distributed sent by the data generating end, determining each target subscribing end for requesting the data stream to be distributed based on the subscription configuration information, and sending the data stream to be distributed to each target subscribing end.
7. The apparatus of claim 6, wherein the subscription broadcast stream further carries subscription type information of a subscription end, and the broadcast stream processing unit is further configured to: updating the subscription type information of each subscription end based on the subscription type information of the subscription end carried in the subscription broadcast stream; and
The data stream processing unit is specifically configured to:
determining the subscription type of each target subscription terminal based on the recorded subscription type information;
and respectively sending the data streams to be distributed to each target subscribing terminal according to a data transmission protocol matched with the subscription types of each target subscribing terminal.
8. The apparatus of claim 6 or 7, wherein the data stream processing unit is further configured to:
if the target subscription end is not determined based on the subscription configuration information, discarding the data stream to be distributed; or storing the data stream to be distributed, and when the storage time length of the data stream to be distributed reaches a time length threshold value, re-determining whether a target subscription end exists or not based on the latest subscription configuration information, and discarding the stored data stream to be distributed if the target subscription end is not re-determined by the set times.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1-6 when the program is executed.
10. A computer readable storage medium storing computer instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011299080.7A CN114520832B (en) | 2020-11-19 | 2020-11-19 | Data stream processing method, device, equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011299080.7A CN114520832B (en) | 2020-11-19 | 2020-11-19 | Data stream processing method, device, equipment and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114520832A CN114520832A (en) | 2022-05-20 |
| CN114520832B true CN114520832B (en) | 2023-08-25 |
Family
ID=81594915
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011299080.7A Active CN114520832B (en) | 2020-11-19 | 2020-11-19 | Data stream processing method, device, equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114520832B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103516746A (en) * | 2012-06-26 | 2014-01-15 | 中兴通讯股份有限公司 | Method for pushing service change information, device and system thereof |
| CN104092767A (en) * | 2014-07-21 | 2014-10-08 | 北京邮电大学 | A publish/subscribe system with added message queue model and its working method |
| CN104133871A (en) * | 2014-07-21 | 2014-11-05 | 广州华多网络科技有限公司 | Method and device for processing subscription information |
| WO2016118876A1 (en) * | 2015-01-23 | 2016-07-28 | Ebay Inc. | Messaging and processing high volume data |
| CN106603394A (en) * | 2013-12-05 | 2017-04-26 | 华为技术有限公司 | Method and device for realizing subscription notification |
| CN106612301A (en) * | 2015-10-22 | 2017-05-03 | 招商局国际信息技术有限公司 | Updated data pushing method and updated data pushing device |
| CN108366096A (en) * | 2018-01-17 | 2018-08-03 | 深圳市金立通信设备有限公司 | A kind of information subscribing method, terminal and computer readable storage medium |
| CN109656725A (en) * | 2018-11-09 | 2019-04-19 | 北京字节跳动网络技术有限公司 | Message consumer's switching method, device, storage medium and electronic equipment |
| WO2019158049A1 (en) * | 2018-02-13 | 2019-08-22 | 中兴通讯股份有限公司 | Subscription information update method and apparatus |
| CN111163172A (en) * | 2019-12-31 | 2020-05-15 | 北京奇艺世纪科技有限公司 | Message processing system, method, electronic device and storage medium |
| US10833881B1 (en) * | 2017-11-06 | 2020-11-10 | Amazon Technologies, Inc. | Distributing publication messages to devices |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10491474B2 (en) * | 2017-02-17 | 2019-11-26 | Home Box Office, Inc. | Endpoint abstraction for service-to-service communication |
-
2020
- 2020-11-19 CN CN202011299080.7A patent/CN114520832B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103516746A (en) * | 2012-06-26 | 2014-01-15 | 中兴通讯股份有限公司 | Method for pushing service change information, device and system thereof |
| CN106603394A (en) * | 2013-12-05 | 2017-04-26 | 华为技术有限公司 | Method and device for realizing subscription notification |
| CN104092767A (en) * | 2014-07-21 | 2014-10-08 | 北京邮电大学 | A publish/subscribe system with added message queue model and its working method |
| CN104133871A (en) * | 2014-07-21 | 2014-11-05 | 广州华多网络科技有限公司 | Method and device for processing subscription information |
| WO2016118876A1 (en) * | 2015-01-23 | 2016-07-28 | Ebay Inc. | Messaging and processing high volume data |
| CN106612301A (en) * | 2015-10-22 | 2017-05-03 | 招商局国际信息技术有限公司 | Updated data pushing method and updated data pushing device |
| US10833881B1 (en) * | 2017-11-06 | 2020-11-10 | Amazon Technologies, Inc. | Distributing publication messages to devices |
| CN108366096A (en) * | 2018-01-17 | 2018-08-03 | 深圳市金立通信设备有限公司 | A kind of information subscribing method, terminal and computer readable storage medium |
| WO2019158049A1 (en) * | 2018-02-13 | 2019-08-22 | 中兴通讯股份有限公司 | Subscription information update method and apparatus |
| CN109656725A (en) * | 2018-11-09 | 2019-04-19 | 北京字节跳动网络技术有限公司 | Message consumer's switching method, device, storage medium and electronic equipment |
| CN111163172A (en) * | 2019-12-31 | 2020-05-15 | 北京奇艺世纪科技有限公司 | Message processing system, method, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114520832A (en) | 2022-05-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107566786B (en) | Method and device for acquiring monitoring video and terminal equipment | |
| US10637947B2 (en) | Scalable, real-time messaging system | |
| US20230164397A1 (en) | Distributed election-based end-to-end content distribution network system and distribution method | |
| CN108282529B (en) | System, method and device for publishing and subscribing data | |
| CN104581374B (en) | A kind of method, node and server for obtaining section file and generating sub- m3u8 files | |
| CN115794139B (en) | Mirror image data processing method, device, equipment and medium | |
| US8903972B2 (en) | Method and apparatus for sharing contents using information of group change in content oriented network environment | |
| CN103548315B (en) | Method and apparatus for high performance low latency real time notification delivery | |
| CN105657000A (en) | Message transmission method and device | |
| WO2017161757A1 (en) | Method and system for distributing streaming media file | |
| CN104243610A (en) | Distributed file transmission service method | |
| CN102984277B (en) | Prevent the system and method that malice connects | |
| WO2014101461A1 (en) | Picture download method and device | |
| US10268532B2 (en) | Application message processing system, method, and application device | |
| CN111835801B (en) | File downloading method, device, server, edge equipment, terminal and medium | |
| CN109347967B (en) | A method and device for acquiring audio and video data | |
| CN114520832B (en) | Data stream processing method, device, equipment and computer storage medium | |
| US11032583B2 (en) | Method and system for improving high availability for live content | |
| CN119211364A (en) | Source return processing method, device and computing equipment based on multi-process live broadcast architecture | |
| CN111090818B (en) | Resource management method, resource management system, server and computer storage medium | |
| CN115297095B (en) | Back source processing method, device, computing equipment and storage medium | |
| US20140149499A1 (en) | Remote request fulfillment and delivery | |
| CN112925946B (en) | A business data storage method, device and electronic equipment | |
| CN114900485B (en) | Method, electronic equipment and system for accessing network file storage | |
| US20080320141A1 (en) | Affiliate bandwidth management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |