US20210410005A1 - Real-time large volume data correlation - Google Patents
Real-time large volume data correlation Download PDFInfo
- Publication number
- US20210410005A1 US20210410005A1 US17/447,572 US202117447572A US2021410005A1 US 20210410005 A1 US20210410005 A1 US 20210410005A1 US 202117447572 A US202117447572 A US 202117447572A US 2021410005 A1 US2021410005 A1 US 2021410005A1
- Authority
- US
- United States
- Prior art keywords
- data
- data packet
- correlator
- data element
- packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
- H04L9/3242—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/02—Arrangements for optimising operational condition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/06—Optimizing the usage of the radio link, e.g. header compression, information sizing, discarding information
Definitions
- the present disclosure relates generally to systems and methods for correlating large volumes of data from a communication network.
- the system comprises a communication network and a computing system, the computing system comprising the data correlator system.
- the computing system comprises a data correlator and a first data feed queue that supplies a first data packet from the communication network to the data correlator.
- the first data packet comprises a key data element, such as a device network identifier, and a user plane data element, such as a portion of one of an image, a video frame, or text, or a control plane data element, such as a network performance measurement, a key performance indicator, or a location of a user device.
- the computing system also comprises a data mapper and a second data feed queue that supplies a second data packet to the data mapper.
- the second data packet can comprise the key data element and a user identifier data element, such as a telephone number, an international mobile equipment identity (IMEI) code, an international mobile subscriber identity (IMSI) code, a name, or an account number.
- the data mapper creates a hash value for the second data packet.
- the data correlator uses the hash value and the key data element to identify the second data packet and create a combined data packet by combining the first data packet and the second data packet.
- the data correlator can provide the combined data packet to an optimization engine to optimize the functioning of the communication network.
- the method uses a data correlator executing on a computing system and the method comprises receiving a first data packet at the data correlator from a communication network, wherein the first data packet comprises a key data element and a user plane data element or a control plane data element.
- a data mapper executing on the computing system receives a second data packet which can comprise the key data element and a user identifier data element.
- the data mapper can create a hash value for the second data packet.
- the data correlator uses the hash value and the key data element to identify the second data packet and to combine the first data packet and second data packet to create a combined data packet.
- the data correlator can provide the combined data packet to an optimization engine to optimized the functioning of the communication network.
- FIG. 1 is a block diagram illustrating an example communication network with data correlator systems in accordance with certain example embodiments.
- FIG. 2A illustrates an example data correlator system in accordance with certain example embodiments.
- FIG. 2B illustrates an alternate example of a data correlator system in accordance with certain example embodiments.
- FIGS. 3A, 3B, and 3C illustrate example hash tables used with a data correlator system in accordance with certain example embodiments.
- FIG. 4A illustrates an example method of a data correlator system in accordance with certain example embodiments.
- FIG. 4B illustrates an alternate example method of a data correlator system in accordance with certain example embodiments.
- FIG. 4C illustrates yet another alternative example method of a data correlator system in accordance with certain example embodiments.
- FIG. 5 is an example embodiment of a functional block diagram of a computing system for hosting a data correlator system in accordance with certain example embodiments.
- the present disclosure relates generally to systems and methods for correlating large volumes of data in connection with optimizing the performance of a communication network.
- a user device receiving user content such as video content
- the communication network may require certain information associated with the user device such as the device network identifier, the signal strength associated with the user device, the location of the user device, and the telephone number associated with the user device.
- the signal strength and the location of the user may be determined by the communication network.
- the telephone number associated with the user device may be retrieved from a separate database of user information.
- the data correlator system can use common information among the different pieces of information.
- data packets associated with each different piece of information include a key data element such as the device network identifier, an evolved Universal Terrestrial Radio Access Network (eUTRAN) device identifier, or a mobility management entity (MME) device identifier.
- eUTRAN evolved Universal Terrestrial Radio Access Network
- MME mobility management entity
- the foregoing examples of identifiers are generally referred to herein as device network identifiers for simplicity.
- the data correlator can use the key data element associated with the data packets of each different piece of information to correlate and combine the related data.
- the data correlator system can rely on data mappers that create a hash value associated with the different pieces of information. It is common for communication networks today to have millions of simultaneous users and each of those users can be receiving large volumes of data over the communication network. In order to quickly correlate millions of data packets being transmitted over the communication network and to minimize the data processing and memory requirements for such correlation, the data correlation system relies on the data mappers and hash values to operate quickly and efficiently.
- the foregoing example relating to handing off a user device from one cell to a neighboring cell of a communication network is only one example of the types of network optimization that can be performed.
- decisions relating to optimization of the communication network can be based on the type of user device or the signal strength of the user device. For instance, if a user is watching a video with the user device in a region of the communication network that has limited bandwidth, the network can suggest that the user switch to a lower resolution of the video in order to improve delivery of the video content to the user device.
- the data correlation techniques described herein can be used in other types of networks such as financial system networks.
- FIGS. 1-5 example embodiments in which the data correlator system can be implemented will be described. It should be understood that the example embodiments are intended to be non-limiting examples and that the data correlator system can be implemented in other systems. For example, in alternate embodiments, one of more of the components shown in the figures or described in the text may be eliminated, modified, or combined with another component.
- FIG. 1 illustrates an example communication network 100 for a data correlator system, in accordance with example embodiments of the present disclosure.
- the communication network 100 can be any of a variety of communication networks handling large volumes of data.
- the communication network 100 can be a cellular mobile radio network such as a Long Term Evolution (LTE) standard based cellular network, a 3G network, a 4G network, a 5G network, or another appropriate wireless network.
- LTE Long Term Evolution
- the communication network 100 may include an user device 102 that may be used by an end-user to communicate with the communication network using wireless services offered by the communication network 100 .
- the end-user may be an individual user, a group of users, an enterprise or organizational entity, etc.
- the user device 102 may be referred to as user equipment.
- the user device 102 may be referred to as a mobile station.
- the user device 102 may include all user equipment and software needed for communication with the communication network 100 . Examples of the user device 102 may include, but are not limited to, a smart phone, a cellular phone, a computer or laptop, a computing tablet, or any endpoint communication devices equipped with wireless capabilities.
- the user device 102 of the end-user may include a server, e.g., when the user is transmitting data for storage in a cloud server over the cellular network, or when the user is receiving streaming video content over the cellular network.
- the user device 102 may access the wireless services offered by the communication network 100 via the radio access network 104 .
- the user device 102 may wirelessly connect with the radio access network 104 over a wireless link 120 , and the radio access network 104 may couple the user device 102 to other parts of the communication network, such as the core network 108 .
- the radio access network of an LTE network may be an evolved Universal Terrestrial Radio Access Network (eUTRAN) that includes one or more evolved base stations.
- eUTRAN evolved Universal Terrestrial Radio Access Network
- the radio access network 104 may include base stations and/or base station controllers.
- the radio access network 104 may transmit all the communication from the endpoint device 102 to the core network 108 .
- the core network 108 of the communication network 100 may be responsible for the overall control of the actions requested by user device 102 that is wirelessly connected to the radio access network 104 .
- the core network 108 may be responsible for processing and routing the traffic (voice and/or data) from the user device 102 to other endpoint devices and/or external networks, such as Internet Protocol (IP) network 112 or public switched telephone network (PSTN) 110 .
- IP Internet Protocol
- PSTN public switched telephone network
- the core network 108 may also handle all the signaling in the control plane.
- the core network 108 of an LTE network may be an Evolved Packet Core (EPC) that provides various functions that support wireless services in the LTE environment, e.g., as specified by the 3GPP standards.
- the EPC network may include at least a serving gateway (S-GW), a packet data network gateway (P-GW), and/or a mobility management entity (MME).
- S-GW serving gateway
- P-GW packet data network gateway
- MME mobility management entity
- the MME is the control node that processes the signaling between the user device 102 and the core network 108 .
- the functions of the MME may be categorized as a bearer function, such as, establishment, maintenance, and release of the bearers; and/or connection management functions, such as, establishment of the connection and security between the EPC and the user device 102 .
- the S-GW may serve as a gateway that routes and forwards user data packets between the eUTRAN and the P-GW, while also acting as the mobility anchor for the user plane during handovers and as the anchor for mobility between LTE and other 3GPP technologies.
- the P-GW serves as a gateway that provides connectivity between the EPC and other external packet data networks.
- the P-GW may route data traffic to the IP network 112 and/or the PSTN 110 via an IMS network that includes a home subscriber server (HSS) database that is a central database that contains user-related and subscription-related information about all the network operator's subscribers (also referred to as users).
- HSS home subscriber server
- the foregoing network components are merely examples and other communication networks can have other components.
- data correlator systems 114 a and 114 b are illustrated in the example communication network 110 . Although only two data correlator systems are shown in the example of FIG. 1 , it should be understood that many data correlator systems can be distributed throughout the network to handle data correlation. Additionally, while the data correlator systems 114 a and 114 b are illustrated as in communication with the core network 108 , in other embodiments data correlator systems can communicate with other components of the communication network 100 .
- the data correlator system 114 can be implemented as a remote computing system or a collection of remote computing systems in a cloud computing environment.
- the example data correlator system 114 of FIG. 2A receives packets of data from the communication network 100 on data feed queues 202 and provides combined data packets in the form of correlated data via an output queue 219 to external clients 220 , such as an optimization engine.
- the data correlator system 114 can also receive data packets from other sources, such as a database of subscriber information, as illustrated by data feed queue 204 .
- the data feed queues 202 and 204 and output queue 219 can be implemented as any of a variety of network connections providing wired or wireless communication with the communication network 100 and other network components.
- the network connections of the data correlator system 114 can include one or more of a telecommunications network, a local area network (LAN), a wide area network (WAN), an intranet, an Internet, or any combination thereof. It will be appreciated that the network connections disclosed are examples and other means of establishing a communications link between the various network components can be used.
- the data correlator system 114 of FIG. 2A further comprises one or more data correlators 205 in communication with one or more data mappers 206 .
- the data correlator system 114 can also include one or more delayed data correlators 216 and one or more data writers 218 .
- the data correlator 205 receives sets of data packets in large volumes on the data feed queues 202 .
- each data feed queue can provide a different type of data.
- data set A can represent the user plane data and include data packets with user content data elements such as video or images the user is receiving at the user device 102 .
- Data set B can represent the control plane data which can include data packets with network performance measurements, key performance indicators, or location of a user device.
- Data set C can represent the data plane which can include data packets associated with the location of the user device 102 within the network 100 such as the signal strength of the user device 102 .
- Data set D represents a smaller volume of data than data sets A, B, and C.
- data set D can comprise user identifier data elements containing the device network identifier associated with the user's device along with one other data element, such as a telephone number, an international mobile equipment identity (IMEI) code, an international mobile subscriber identity (IMSI) code, a name, or an account number.
- IMEI international mobile equipment identity
- IMSI international mobile subscriber identity
- the device network identifier is typically included in the data packets in data sets A, B, and C.
- Including the device network identifier in the data packets of data set D as well as the data packets of data sets A, B, and C provides a key data element common among the data packets associated with a user device and allows the data correlator 205 to correlate data packets.
- the data mapper 206 receives data set D and creates a hash table enabling the data correlator 205 to more quickly identify and correlate data packets.
- the hash table will be explained further below in connection with FIGS. 3A, 3B, and 3C .
- the hash table can be stored in a key/value database 207 .
- the key/value database 207 can include one or more tangible computer-readable storage devices.
- the computer-readable storage devices can be stored on or coupled to the data correlator system 114 .
- the computer-readable storage devices can include a hard drive, an on-board flash memory and/or one or more removable memory cards or removable flash memory.
- the computer-readable storage devices and their associated computer-readable media can provide non-transitory, non-volatile storage of the hash tables as well as other data structures or computer-executable instructions.
- the data correlator can combine the data packets and provide them to external systems for use in optimizing the communication network 100 .
- an optional data writer 218 can write the combined data to files that can be used by a variety of external clients 220 that optimize the network.
- the data writer 218 and its associated files can be omitted and the data correlator 205 can provide correlated and combined data packets directly to an optimization engine.
- the combined data packets provide more complete information associated with the user device 102 thereby enabling the optimization engine to make optimization decisions.
- the large volumes of data packets received on the data feed queues 205 can be correlated and combined more quickly so that the optimization engine can provide real-time or near real-time optimizations to the communication network 100 .
- the term “real-time” refers to a user's perspective of the communication network and means that correlating data and making network optimization decisions can be completed within a few milliseconds to a few seconds, so that the network optimization appears to be happening in “real time” from the user's perspective.
- the combined data packets can also be stored for later analysis of historical network performance. Analyzing the historical network performance can be of value when enhancing or updating the communication network 100 , for example.
- the data correlator system 114 illustrated in FIG. 2A also includes an optional delayed data correlator 216 . While the data correlator system 114 is designed to expedite the correlation and combination of associated data packets, in some instances, associated data packets could arrive at the data correlator system 114 at different times, thereby, preventing the data correlator 205 from quickly correlating and combining the data packets. For example, a first data packet comprising a device network identifier and video content could arrive on data feed queue A several seconds after a second related data packet comprising the same device network identifier and a telephone number arrives on data feed queue D. Due to the difference in time, the data correlator 205 may be unable to correlate and combine the two data packets.
- the data mapper forwards the second data packet to the delayed data correlator 216 and, when it arrives, the data correlator 205 forwards the first data packet to the delayed data correlator 216 .
- the delayed data correlator works to correlate and combine related data packets that arrived at different times. Although the delayed combined data packet may be too late to assist with real-time optimization, it still can be stored for later analysis of historical network performance.
- FIG. 2B an alternate embodiment of a data correlator system 114 is illustrated. Similar to the data correlator 114 illustrated in FIG. 2A , the data correlator 114 illustrated in FIG. 2B is implemented in a computer system and comprises data queues 252 , 262 , and 272 which provide data packets to and receive data packets from data correlators 255 , 265 , and 275 .
- the data correlator system 114 of FIG. 2B also comprises data mappers 256 , 266 , and 276 associated with key/value databases 257 , 267 , and 277 , respectively.
- FIG. 2B operate in a similar manner to their counterpart components described in connection with FIG. 2A .
- the system of FIG. 2B implements a chain data correlation approach. That is, after data correlator 255 combines a first data packet from queue A and a second data packet from queue B, the combined data packet is sent to data correlator 265 where it is combined with a third related data packet from queue C. After data correlator 265 combines the third data packet with the first and second data packets, the combined data packet is sent to data correlator 275 where it is combined with a fourth related data packet from queue D. Therefore, the combined data packet provided to the output queue 279 is the combination of four related data packets and provides more complete information about the data transmitted than one of the combined data packets of FIG. 2A which only included two data packets.
- While components of the data correlator systems 114 illustrated in FIGS. 2A and 2B are implemented in software, such as the data correlator, the data mapper, the optional delayed data correlator, the optional data writer, and the external client, it will be appreciated that one or more acts or functions of the data correlator system 114 may be performed by hardware, software, or a combination thereof, and may be embodied in one or more computing systems. It should be understood that the data correlator, the data mapper, the optional delayed data correlator, the optional data writer, and the external client can include various routines, sub-routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types
- the hash table is stored in the key/value databases such as those illustrated in FIGS. 2A and 2B .
- One goal of the hash table 209 is to store data packets efficiently so that they do not require excessive memory and processing resources.
- the hash table 209 is empty and the first data packet received at the data mapper is stored in the fields at the top of the hash table.
- the key 305 represents a key data element in the received first data packet, such as a device identifier associated with the user device 102 .
- the value 307 represents another data element in the received first data packet, such as a telephone number, an international mobile equipment identity (IMEI) code, an international mobile subscriber identity (IMSI) code, a name, or an account number associated with the user device.
- the hash value 310 is a condensed version of the key 305 . In its condensed form, the hash value 310 provides a faster mechanism for the data correlator to correlate the received first data packet with a related data packet received on another data feed queue.
- the hash table 209 is now shown populated with data from multiple data packets that have been received by the data mapper.
- data from newer data packets arriving at the data mapper is inserted in fields below the older data so that older packet data remains at the top of the hash table 209 .
- FIG. 3B shows that because the hash value is a condensed version of the key, it is possible for different user devices to have the same hash value.
- key 305 and key 314 can represent device network identifiers for different user devices, but they have the same hash value because the hash value is a condensed version of the key.
- the data in the key and value fields is linked by link 312 .
- the data correlator receives the hash value 310 associated with two different user devices, the data correlator will examine the key in fields 305 and 314 in order to correctly identify the data packet.
- FIG. 3C illustrates when the hash table 209 becomes full with packet data and hash values due to the volume of data packets received at the data mapper.
- the hash table 209 is limited to a fixed size. Accordingly, when the hash table 209 becomes full, a key and value data associated with a newly received data packet will overwrite the oldest packet data at the top of the hash table 209 .
- FIGS. 4A, 4B, and 4C example flow charts are shown depicting methods for correlating data using the data correlator systems 114 of FIGS. 1-3C . It should be understood that in other example embodiments, the example methods of FIGS. 4A, 4B, and 4C can be modified to combine certain steps, add steps, or modify the order of certain steps.
- the data correlator receives on a data feed queue a first data packet comprising a key data element and at least one of a user plane data element or a control plane data element.
- the key data element can be a device network identifier associated with the user device 102 .
- the user plane data element can be, for example, a portion of a video frame being transmitted to the user device 102 .
- the control plane data element can be a network performance measurement, a key performance indicator, or a location of a user device.
- the data mapper of the data correlator system 114 receives a second data packet that is related to the first data packet in that the second data packet has the same key data element (e.g., device network identifier) as well as a user identifier data element comprising one of a telephone number, a name, or an account number.
- key data element e.g., device network identifier
- the data mapper creates a hash value for the second data packet and stores the hash value and the key and value of the second data packet in a hash table in a key/value database.
- the data correlator receives the hash value from the hash table and uses the hash value and the key data element to identify the second data packet. The data correlator can then combine the first data packet and the second data packet into a combined data packet.
- the data correlator provides the combined data packet to an optimization engine so that the optimization engine can use the data of the combined packets to make network optimization decisions and recommendations. Without the combination of the data from the two data packets, the optimization engine will be less able to make effective network optimization decisions and recommendations.
- the data correlation system 114 can comprise multiple data feed queues providing a variety of data packets to multiple data correlators.
- FIG. 4B illustrates an alternative example with multiple data correlators. The alternative example of FIG. 4B can build upon the example of FIG. 4A . In other words, the steps of the example method illustrated in 4 B can take place in parallel with the steps of the example method illustrated in FIG. 4A .
- a second data correlator receives a third data packet on a third data feed queue.
- the third data packet comprises the same key data element of the first and second data packets of FIG. 4A as well as one of the user plane data element or the control plane data element that was absent from the first data packet.
- the third data packet can comprise the key data element and the control plane data element. It should be understood that in alternate examples, the third data packet could comprise other information such as information from the data plane of the communication network.
- the second data correlator can use the same hash value referenced in FIG. 4A and the key data element to identify the third data packet and, in step 440 , combine it with the related second data packet having the same key data element.
- the second data correlator provides the combined second data packet (a combination of the second data packet and the third data packet) to the network optimization engine for use in optimizing the communications network 100 .
- FIG. 4C shows the chain data correlation approach illustrated in FIG. 2B .
- the method illustrated in FIG. 4C can begin after step 420 in FIG. 4A and before the combined data packet is forwarded to the optimization engine.
- a second data correlator receives from the first data correlator the combined data packet that was created in step 420 of FIG. 4A .
- a second data mapper receives a third data packet comprising the same key data element of the first and second data packets as well as one of the user plane data element or the control plane data element that was absent from the first data packet.
- the third data packet can comprise the key data element and the control plane data element. It should be understood that in alternate examples, the third data packet could comprise other information such as information from the data plane of the communication network.
- the second data mapper creates a second hash value for the third data packet.
- the second data correlator uses the second hash value and the key data element to identify the third data packet as related to the first and second data packets. The second data correlator combines the third data packet with the first and second data packets to create a second combined data packet.
- the second data correlator provides the second combined data packet to the optimization engine for use in optimizing the communication network.
- the second data correlator could provide the second combined data packet to yet another data correlator for combining additional data.
- the components of the example communication network 100 have been described with reference to the example systems and methods illustrated in FIGS. 1-4C .
- the example embodiments can include one or more computer programs that embody the functions described herein and illustrated in the appended flow charts.
- computer programs that embody the functions described herein and illustrated in the appended flow charts.
- FIGS. 1-4C The components of the example communication network 100 have been described with reference to the example systems and methods illustrated in FIGS. 1-4C .
- the example embodiments can include one or more computer programs that embody the functions described herein and illustrated in the appended flow charts.
- a skilled programmer would be able to write such computer programs to implement example embodiments based on the figures and associated description in the application text. Therefore, disclosure of a particular set of program code instructions is not considered necessary for an adequate understanding of how to make and use the example embodiment
- the example embodiments described herein can be used with computer hardware and software that perform the methods and processing functions described previously.
- the systems, methods, and procedures described herein can be embodied in a programmable computer, computer-executable software, or digital circuitry.
- the software can be stored on tangible computer-readable media.
- computer-readable media can include a floppy disk, RAM, ROM, hard disk, removable media, flash memory, memory stick, optical media, magneto-optical media, CD-ROM, etc.
- Digital circuitry can include integrated circuits, gate arrays, building block logic, field programmable gate arrays (FPGA), etc.
- FIG. 5 illustrates an example block diagram of a computing system 500 such as the computing system hosting the data correlator system 114 described previously.
- the computing device includes a hardware processor 504 , display/speaker/microphone 506 , wireless module 508 , memory 510 , and I/O 512 .
- elements of the computing device 500 are electrically and communicatively interconnected via a bus 514 .
- the processor 504 may comprise one or more specific or general purpose processors configured to execute instructions stored on the memory 510 that, when executed, control the processor 504 to execute various applications and perform various functions associated with the computing device 500 . Additionally or alternatively, the processor 504 may include a programmable gate array and operate, at least in part, based on firmware. As an example, the processor 504 may execute instructions stored on the memory 510 including instructions for an operating system of the computing device 500 and instructions for applications.
- the applications that may be executed by the computing device 500 include mobile applications and engines, data mapping and data correlation processing applications, network optimization applications, an e-mail application, a map viewer application, and a web-browser application, among others.
- the input commands control applications executing on the computing device 500 to perform various tasks or functions associated with the applications.
- the display/speaker/microphone 506 is configured to display applications executing on the processor 504 and provide visual and audible feedback to a user of the computing device 500 .
- the display/speaker/microphone 506 may comprise one or more display devices such as LCD, LED, OLED, and Electronic Ink displays, among others.
- the display/speaker/microphone 506 may also comprise one or more speakers and one or more microphones.
- the wireless communication module 508 is configured to provide wireless communication of data to and from the computing device 500 .
- the wireless communication module 508 may be configured for cellular communications using one or more of GSM, CDMA, TDMA, OFDM and other cellular communications protocols, wireless area network communications using one or more of the family of 802.11x protocols and other wireless area network communications protocols, and Bluetooth communications protocols.
- the memory 510 may comprise a Random Access Memory (RAM), Read Only Memory (ROM), or any other tangible storage memory configured to store software programs for execution by the processor 504 .
- the memory 510 may comprise one or more of dynamic, persistent, and semi-persistent solid state memories, magnetic memories, removable memories, or any other known memories suitable for the application of storing data and software programs for the computing device 500 .
- the I/O 512 includes inputs and outputs of the computing device 500 such as power connectors, data connectors, and other input and output devices.
- the I/O 512 may comprise, for example, wired data communication input and output interfaces, power charging interfaces, infra-red interfaces, light and proximity sensors, capacitive sensors, “soft” and “hard” buttons, switches, and other input/output interfaces of the computing device 500 .
- the bus 514 is configured to electrically and communicatively connect the processor 504 , the display/speaker/microphone 506 , the wireless module 508 , the memory 510 , and the I/O 512 for transfer of data and instructions between elements of the computing device 500 .
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Power Engineering (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
A system and method uses a data correlator system to associate large volumes of related data. The data correlator system receives data packets with user content from a mobile network. The data correlator system creates a hash value for data associated with the user content, such as user account information or the location of the user device. The data correlator system uses the hash value to combine the user content with the associated data. The data correlator system provides the combined data to an optimization engine for optimizing the performance of the mobile network.
Description
- This application is a continuation of U.S. patent application Ser. No. 16/552,729, filed Aug. 27, 2019, now U.S. Pat. No. 11,122,464, which is herein incorporated by reference in its entirety.
- The present disclosure relates generally to systems and methods for correlating large volumes of data from a communication network.
- As mobile devices such as cellular telephones, smart phones, and tablet computers continue to proliferate, the demands on communication networks continue to grow. For example, mobile network customers frequently access a variety of content, including text, images, and video using their mobile devices. This increased use of mobile devices has greatly increased the volume of data traffic handled by mobile networks. Techniques for optimizing the performance of the mobile network can assist the mobile network operator in handling increased volumes of data traffic. Optimizing the performance of the mobile network often requires quickly evaluating data associated with a particular content that a user is accessing over the mobile network. However, existing techniques for correlating data are not well-suited to handle large volumes of data. Additionally, existing techniques for correlating large volumes of data can be time consuming and can require substantial memory and processor resources. Accordingly, improved techniques for correlating large volumes of data associated with the data traffic on the mobile network can assist in improving the performance of the mobile network.
- One aspect of the present disclosure provides a system for correlating large volumes of data in connection with a communication network. The system comprises a communication network and a computing system, the computing system comprising the data correlator system. The computing system comprises a data correlator and a first data feed queue that supplies a first data packet from the communication network to the data correlator. The first data packet comprises a key data element, such as a device network identifier, and a user plane data element, such as a portion of one of an image, a video frame, or text, or a control plane data element, such as a network performance measurement, a key performance indicator, or a location of a user device. The computing system also comprises a data mapper and a second data feed queue that supplies a second data packet to the data mapper. The second data packet can comprise the key data element and a user identifier data element, such as a telephone number, an international mobile equipment identity (IMEI) code, an international mobile subscriber identity (IMSI) code, a name, or an account number. The data mapper creates a hash value for the second data packet. The data correlator uses the hash value and the key data element to identify the second data packet and create a combined data packet by combining the first data packet and the second data packet. The data correlator can provide the combined data packet to an optimization engine to optimize the functioning of the communication network.
- Another aspect of the present disclosure provides a method for correlating large volumes of data in connection with a communication network. The method uses a data correlator executing on a computing system and the method comprises receiving a first data packet at the data correlator from a communication network, wherein the first data packet comprises a key data element and a user plane data element or a control plane data element. A data mapper executing on the computing system receives a second data packet which can comprise the key data element and a user identifier data element. The data mapper can create a hash value for the second data packet. The data correlator uses the hash value and the key data element to identify the second data packet and to combine the first data packet and second data packet to create a combined data packet. The data correlator can provide the combined data packet to an optimization engine to optimized the functioning of the communication network.
- The foregoing aspects are example embodiments and are not meant to limit the scope of the present disclosure. These and other aspects will be described in connection with the following description and the accompanying drawings.
-
FIG. 1 is a block diagram illustrating an example communication network with data correlator systems in accordance with certain example embodiments. -
FIG. 2A illustrates an example data correlator system in accordance with certain example embodiments. -
FIG. 2B illustrates an alternate example of a data correlator system in accordance with certain example embodiments. -
FIGS. 3A, 3B, and 3C illustrate example hash tables used with a data correlator system in accordance with certain example embodiments. -
FIG. 4A illustrates an example method of a data correlator system in accordance with certain example embodiments. -
FIG. 4B illustrates an alternate example method of a data correlator system in accordance with certain example embodiments. -
FIG. 4C illustrates yet another alternative example method of a data correlator system in accordance with certain example embodiments. -
FIG. 5 is an example embodiment of a functional block diagram of a computing system for hosting a data correlator system in accordance with certain example embodiments. - The present disclosure relates generally to systems and methods for correlating large volumes of data in connection with optimizing the performance of a communication network. As one example, a user device receiving user content, such as video content, over the communication network may require transmission of a large number of data packets. In order to optimize the performance of the communication network, it may be desirable to hand off the user device from one cell to a neighboring cell within the communication network. In order to hand off the user device from one cell to another cell, the communication network may require certain information associated with the user device such as the device network identifier, the signal strength associated with the user device, the location of the user device, and the telephone number associated with the user device. These different pieces of information typically come from different sources. For example, the signal strength and the location of the user may be determined by the communication network. However, the telephone number associated with the user device may be retrieved from a separate database of user information.
- In order to correlate the different pieces of information for network optimization decisions, the data correlator system can use common information among the different pieces of information. For example, data packets associated with each different piece of information include a key data element such as the device network identifier, an evolved Universal Terrestrial Radio Access Network (eUTRAN) device identifier, or a mobility management entity (MME) device identifier. The foregoing examples of identifiers are generally referred to herein as device network identifiers for simplicity. The data correlator can use the key data element associated with the data packets of each different piece of information to correlate and combine the related data.
- As a further optimization, the data correlator system can rely on data mappers that create a hash value associated with the different pieces of information. It is common for communication networks today to have millions of simultaneous users and each of those users can be receiving large volumes of data over the communication network. In order to quickly correlate millions of data packets being transmitted over the communication network and to minimize the data processing and memory requirements for such correlation, the data correlation system relies on the data mappers and hash values to operate quickly and efficiently.
- The foregoing example relating to handing off a user device from one cell to a neighboring cell of a communication network is only one example of the types of network optimization that can be performed. In other examples, decisions relating to optimization of the communication network can be based on the type of user device or the signal strength of the user device. For instance, if a user is watching a video with the user device in a region of the communication network that has limited bandwidth, the network can suggest that the user switch to a lower resolution of the video in order to improve delivery of the video content to the user device. Moreover, the data correlation techniques described herein can be used in other types of networks such as financial system networks.
- Referring now to
FIGS. 1-5 , example embodiments in which the data correlator system can be implemented will be described. It should be understood that the example embodiments are intended to be non-limiting examples and that the data correlator system can be implemented in other systems. For example, in alternate embodiments, one of more of the components shown in the figures or described in the text may be eliminated, modified, or combined with another component. -
FIG. 1 illustrates anexample communication network 100 for a data correlator system, in accordance with example embodiments of the present disclosure. Thecommunication network 100 can be any of a variety of communication networks handling large volumes of data. For example, thecommunication network 100 can be a cellular mobile radio network such as a Long Term Evolution (LTE) standard based cellular network, a 3G network, a 4G network, a 5G network, or another appropriate wireless network. - The
communication network 100 may include an user device 102 that may be used by an end-user to communicate with the communication network using wireless services offered by thecommunication network 100. The end-user may be an individual user, a group of users, an enterprise or organizational entity, etc. As one example, in an LTE network, the user device 102 may be referred to as user equipment. However, in other networks, such as a 3G network, the user device 102 may be referred to as a mobile station. The user device 102 may include all user equipment and software needed for communication with thecommunication network 100. Examples of the user device 102 may include, but are not limited to, a smart phone, a cellular phone, a computer or laptop, a computing tablet, or any endpoint communication devices equipped with wireless capabilities. In some example embodiments, the user device 102 of the end-user may include a server, e.g., when the user is transmitting data for storage in a cloud server over the cellular network, or when the user is receiving streaming video content over the cellular network. - The user device 102 may access the wireless services offered by the
communication network 100 via theradio access network 104. In particular, the user device 102 may wirelessly connect with theradio access network 104 over awireless link 120, and theradio access network 104 may couple the user device 102 to other parts of the communication network, such as thecore network 108. As an example, the radio access network of an LTE network may be an evolved Universal Terrestrial Radio Access Network (eUTRAN) that includes one or more evolved base stations. However, in other example communication networks, such as 3G, 4G, or 5G networks, theradio access network 104 may include base stations and/or base station controllers. - The
radio access network 104 may transmit all the communication from the endpoint device 102 to thecore network 108. Thecore network 108 of thecommunication network 100 may be responsible for the overall control of the actions requested by user device 102 that is wirelessly connected to theradio access network 104. For example, thecore network 108 may be responsible for processing and routing the traffic (voice and/or data) from the user device 102 to other endpoint devices and/or external networks, such as Internet Protocol (IP)network 112 or public switched telephone network (PSTN) 110. Further, thecore network 108 may also handle all the signaling in the control plane. For example, thecore network 108 of an LTE network may be an Evolved Packet Core (EPC) that provides various functions that support wireless services in the LTE environment, e.g., as specified by the 3GPP standards. The EPC network may include at least a serving gateway (S-GW), a packet data network gateway (P-GW), and/or a mobility management entity (MME). - The MME is the control node that processes the signaling between the user device 102 and the
core network 108. In particular, the functions of the MME may be categorized as a bearer function, such as, establishment, maintenance, and release of the bearers; and/or connection management functions, such as, establishment of the connection and security between the EPC and the user device 102. The S-GW may serve as a gateway that routes and forwards user data packets between the eUTRAN and the P-GW, while also acting as the mobility anchor for the user plane during handovers and as the anchor for mobility between LTE and other 3GPP technologies. The P-GW serves as a gateway that provides connectivity between the EPC and other external packet data networks. The P-GW may route data traffic to theIP network 112 and/or thePSTN 110 via an IMS network that includes a home subscriber server (HSS) database that is a central database that contains user-related and subscription-related information about all the network operator's subscribers (also referred to as users). The foregoing network components are merely examples and other communication networks can have other components. - Also illustrated in the
example communication network 110 are data 114 a and 114 b (referred to generally as 114). Although only two data correlator systems are shown in the example ofcorrelator systems FIG. 1 , it should be understood that many data correlator systems can be distributed throughout the network to handle data correlation. Additionally, while the 114 a and 114 b are illustrated as in communication with thedata correlator systems core network 108, in other embodiments data correlator systems can communicate with other components of thecommunication network 100. - Referring now to
FIG. 2A , an exampledata correlator system 114 is illustrated. The data correlator system can be implemented as a remote computing system or a collection of remote computing systems in a cloud computing environment. The exampledata correlator system 114 ofFIG. 2A receives packets of data from thecommunication network 100 on data feed queues 202 and provides combined data packets in the form of correlated data via anoutput queue 219 toexternal clients 220, such as an optimization engine. Thedata correlator system 114 can also receive data packets from other sources, such as a database of subscriber information, as illustrated by data feedqueue 204. The data feedqueues 202 and 204 andoutput queue 219 can be implemented as any of a variety of network connections providing wired or wireless communication with thecommunication network 100 and other network components. In an example embodiment, the network connections of thedata correlator system 114 can include one or more of a telecommunications network, a local area network (LAN), a wide area network (WAN), an intranet, an Internet, or any combination thereof. It will be appreciated that the network connections disclosed are examples and other means of establishing a communications link between the various network components can be used. - The
data correlator system 114 ofFIG. 2A further comprises one ormore data correlators 205 in communication with one or more data mappers 206. Optionally, thedata correlator system 114 can also include one or more delayed data correlators 216 and one ormore data writers 218. The data correlator 205 receives sets of data packets in large volumes on the data feed queues 202. As one example, each data feed queue can provide a different type of data. For example, data set A can represent the user plane data and include data packets with user content data elements such as video or images the user is receiving at the user device 102. Data set B can represent the control plane data which can include data packets with network performance measurements, key performance indicators, or location of a user device. Data set C can represent the data plane which can include data packets associated with the location of the user device 102 within thenetwork 100 such as the signal strength of the user device 102. Data set D represents a smaller volume of data than data sets A, B, and C. For example, data set D can comprise user identifier data elements containing the device network identifier associated with the user's device along with one other data element, such as a telephone number, an international mobile equipment identity (IMEI) code, an international mobile subscriber identity (IMSI) code, a name, or an account number. Additionally, the device network identifier is typically included in the data packets in data sets A, B, and C. Including the device network identifier in the data packets of data set D as well as the data packets of data sets A, B, and C provides a key data element common among the data packets associated with a user device and allows the data correlator 205 to correlate data packets. - The data mapper 206 receives data set D and creates a hash table enabling the data correlator 205 to more quickly identify and correlate data packets. The hash table will be explained further below in connection with
FIGS. 3A, 3B, and 3C . The hash table can be stored in a key/value database 207. The key/value database 207 can include one or more tangible computer-readable storage devices. The computer-readable storage devices can be stored on or coupled to thedata correlator system 114. For example, the computer-readable storage devices can include a hard drive, an on-board flash memory and/or one or more removable memory cards or removable flash memory. The computer-readable storage devices and their associated computer-readable media can provide non-transitory, non-volatile storage of the hash tables as well as other data structures or computer-executable instructions. - Once data packets are correlated, the data correlator can combine the data packets and provide them to external systems for use in optimizing the
communication network 100. For example, anoptional data writer 218 can write the combined data to files that can be used by a variety ofexternal clients 220 that optimize the network. Alternatively, thedata writer 218 and its associated files can be omitted and the data correlator 205 can provide correlated and combined data packets directly to an optimization engine. The combined data packets provide more complete information associated with the user device 102 thereby enabling the optimization engine to make optimization decisions. - One benefit of using the techniques of the data correlator and data mapper of
FIG. 2A is that the large volumes of data packets received on the data feedqueues 205 can be correlated and combined more quickly so that the optimization engine can provide real-time or near real-time optimizations to thecommunication network 100. As used herein, the term “real-time” refers to a user's perspective of the communication network and means that correlating data and making network optimization decisions can be completed within a few milliseconds to a few seconds, so that the network optimization appears to be happening in “real time” from the user's perspective. Aside from real-time optimization of thecommunication network 100, the combined data packets can also be stored for later analysis of historical network performance. Analyzing the historical network performance can be of value when enhancing or updating thecommunication network 100, for example. - The
data correlator system 114 illustrated inFIG. 2A also includes an optional delayeddata correlator 216. While thedata correlator system 114 is designed to expedite the correlation and combination of associated data packets, in some instances, associated data packets could arrive at thedata correlator system 114 at different times, thereby, preventing the data correlator 205 from quickly correlating and combining the data packets. For example, a first data packet comprising a device network identifier and video content could arrive on data feed queue A several seconds after a second related data packet comprising the same device network identifier and a telephone number arrives on data feed queue D. Due to the difference in time, the data correlator 205 may be unable to correlate and combine the two data packets. In this situation, the data mapper forwards the second data packet to the delayeddata correlator 216 and, when it arrives, the data correlator 205 forwards the first data packet to the delayeddata correlator 216. The delayed data correlator works to correlate and combine related data packets that arrived at different times. Although the delayed combined data packet may be too late to assist with real-time optimization, it still can be stored for later analysis of historical network performance. - Referring now to
FIG. 2B , an alternate embodiment of adata correlator system 114 is illustrated. Similar to the data correlator 114 illustrated inFIG. 2A , the data correlator 114 illustrated inFIG. 2B is implemented in a computer system and comprises 252, 262, and 272 which provide data packets to and receive data packets fromdata queues 255, 265, and 275. Thedata correlators data correlator system 114 ofFIG. 2B also comprises 256, 266, and 276 associated with key/data mappers 257, 267, and 277, respectively. The foregoing components of thevalue databases data correlator system 114 ofFIG. 2B operate in a similar manner to their counterpart components described in connection withFIG. 2A . However, the system ofFIG. 2B implements a chain data correlation approach. That is, after data correlator 255 combines a first data packet from queue A and a second data packet from queue B, the combined data packet is sent todata correlator 265 where it is combined with a third related data packet from queue C. After data correlator 265 combines the third data packet with the first and second data packets, the combined data packet is sent todata correlator 275 where it is combined with a fourth related data packet from queue D. Therefore, the combined data packet provided to the output queue 279 is the combination of four related data packets and provides more complete information about the data transmitted than one of the combined data packets ofFIG. 2A which only included two data packets. - While components of the
data correlator systems 114 illustrated inFIGS. 2A and 2B are implemented in software, such as the data correlator, the data mapper, the optional delayed data correlator, the optional data writer, and the external client, it will be appreciated that one or more acts or functions of thedata correlator system 114 may be performed by hardware, software, or a combination thereof, and may be embodied in one or more computing systems. It should be understood that the data correlator, the data mapper, the optional delayed data correlator, the optional data writer, and the external client can include various routines, sub-routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types - Referring now to
FIGS. 3A, 3B, and 3C , an example of a hash table 209 is illustrated. The hash table is stored in the key/value databases such as those illustrated inFIGS. 2A and 2B . One goal of the hash table 209 is to store data packets efficiently so that they do not require excessive memory and processing resources. As illustrated inFIG. 3A , the hash table 209 is empty and the first data packet received at the data mapper is stored in the fields at the top of the hash table. The key 305 represents a key data element in the received first data packet, such as a device identifier associated with the user device 102. Thevalue 307 represents another data element in the received first data packet, such as a telephone number, an international mobile equipment identity (IMEI) code, an international mobile subscriber identity (IMSI) code, a name, or an account number associated with the user device. Thehash value 310 is a condensed version of the key 305. In its condensed form, thehash value 310 provides a faster mechanism for the data correlator to correlate the received first data packet with a related data packet received on another data feed queue. - Referring to
FIG. 3B , the hash table 209 is now shown populated with data from multiple data packets that have been received by the data mapper. As illustrated in the example ofFIG. 3B , data from newer data packets arriving at the data mapper is inserted in fields below the older data so that older packet data remains at the top of the hash table 209.FIG. 3B shows that because the hash value is a condensed version of the key, it is possible for different user devices to have the same hash value. In the example ofFIG. 3B , key 305 and key 314 can represent device network identifiers for different user devices, but they have the same hash value because the hash value is a condensed version of the key. When this occurs, the data in the key and value fields is linked bylink 312. When the data correlator receives thehash value 310 associated with two different user devices, the data correlator will examine the key in 305 and 314 in order to correctly identify the data packet.fields -
FIG. 3C illustrates when the hash table 209 becomes full with packet data and hash values due to the volume of data packets received at the data mapper. Given the goal of minimizing memory resources, the hash table 209 is limited to a fixed size. Accordingly, when the hash table 209 becomes full, a key and value data associated with a newly received data packet will overwrite the oldest packet data at the top of the hash table 209. - Referring now to
FIGS. 4A, 4B, and 4C , example flow charts are shown depicting methods for correlating data using thedata correlator systems 114 ofFIGS. 1-3C . It should be understood that in other example embodiments, the example methods ofFIGS. 4A, 4B, and 4C can be modified to combine certain steps, add steps, or modify the order of certain steps. Referring to step 405 ofFIG. 4A , the data correlator receives on a data feed queue a first data packet comprising a key data element and at least one of a user plane data element or a control plane data element. As mentioned previously, the key data element can be a device network identifier associated with the user device 102. The user plane data element can be, for example, a portion of a video frame being transmitted to the user device 102. The control plane data element can be a network performance measurement, a key performance indicator, or a location of a user device. Instep 410, the data mapper of thedata correlator system 114 receives a second data packet that is related to the first data packet in that the second data packet has the same key data element (e.g., device network identifier) as well as a user identifier data element comprising one of a telephone number, a name, or an account number. Instep 415, the data mapper creates a hash value for the second data packet and stores the hash value and the key and value of the second data packet in a hash table in a key/value database. Instep 420, the data correlator receives the hash value from the hash table and uses the hash value and the key data element to identify the second data packet. The data correlator can then combine the first data packet and the second data packet into a combined data packet. Instep 425, the data correlator provides the combined data packet to an optimization engine so that the optimization engine can use the data of the combined packets to make network optimization decisions and recommendations. Without the combination of the data from the two data packets, the optimization engine will be less able to make effective network optimization decisions and recommendations. - As illustrated in
FIG. 2A , thedata correlation system 114 can comprise multiple data feed queues providing a variety of data packets to multiple data correlators.FIG. 4B illustrates an alternative example with multiple data correlators. The alternative example ofFIG. 4B can build upon the example ofFIG. 4A . In other words, the steps of the example method illustrated in 4B can take place in parallel with the steps of the example method illustrated inFIG. 4A . Beginning withstep 430 ofFIG. 4B , a second data correlator receives a third data packet on a third data feed queue. The third data packet comprises the same key data element of the first and second data packets ofFIG. 4A as well as one of the user plane data element or the control plane data element that was absent from the first data packet. In other words, if the first data packet included the key data element and the user plane data element, the third data packet can comprise the key data element and the control plane data element. It should be understood that in alternate examples, the third data packet could comprise other information such as information from the data plane of the communication network. Instep 435, the second data correlator can use the same hash value referenced inFIG. 4A and the key data element to identify the third data packet and, instep 440, combine it with the related second data packet having the same key data element. Instep 445, the second data correlator provides the combined second data packet (a combination of the second data packet and the third data packet) to the network optimization engine for use in optimizing thecommunications network 100. - In contrast to the parallel method illustrated in
FIG. 4B ,FIG. 4C shows the chain data correlation approach illustrated inFIG. 2B . The method illustrated inFIG. 4C can begin afterstep 420 inFIG. 4A and before the combined data packet is forwarded to the optimization engine. Instep 470, a second data correlator receives from the first data correlator the combined data packet that was created instep 420 ofFIG. 4A . Instep 475, a second data mapper receives a third data packet comprising the same key data element of the first and second data packets as well as one of the user plane data element or the control plane data element that was absent from the first data packet. In other words, if the first data packet included the key data element and the user plane data element, the third data packet can comprise the key data element and the control plane data element. It should be understood that in alternate examples, the third data packet could comprise other information such as information from the data plane of the communication network. Instep 480, the second data mapper creates a second hash value for the third data packet. Instep 485, the second data correlator uses the second hash value and the key data element to identify the third data packet as related to the first and second data packets. The second data correlator combines the third data packet with the first and second data packets to create a second combined data packet. Lastly, instep 490, the second data correlator provides the second combined data packet to the optimization engine for use in optimizing the communication network. Alternatively, the second data correlator could provide the second combined data packet to yet another data correlator for combining additional data. - The components of the
example communication network 100 have been described with reference to the example systems and methods illustrated inFIGS. 1-4C . The example embodiments can include one or more computer programs that embody the functions described herein and illustrated in the appended flow charts. However, it should be apparent that there could be many different ways of implementing aspects of the example embodiments in computer programming, and these aspects should not be construed as limited to one set of computer instructions. Further, a skilled programmer would be able to write such computer programs to implement example embodiments based on the figures and associated description in the application text. Therefore, disclosure of a particular set of program code instructions is not considered necessary for an adequate understanding of how to make and use the example embodiments. Those skilled in the art will appreciate that one or more acts described may be performed by hardware, software, or a combination thereof, as may be embodied in one or more computing systems. Furthermore, those having ordinary skill in the art and the benefit of the present disclosure will appreciate that the foregoing payment network components can have any of several other suitable computer system configurations. - The example embodiments described herein can be used with computer hardware and software that perform the methods and processing functions described previously. The systems, methods, and procedures described herein can be embodied in a programmable computer, computer-executable software, or digital circuitry. The software can be stored on tangible computer-readable media. For example, computer-readable media can include a floppy disk, RAM, ROM, hard disk, removable media, flash memory, memory stick, optical media, magneto-optical media, CD-ROM, etc. Digital circuitry can include integrated circuits, gate arrays, building block logic, field programmable gate arrays (FPGA), etc.
-
FIG. 5 illustrates an example block diagram of acomputing system 500 such as the computing system hosting thedata correlator system 114 described previously. As illustrated inFIG. 5 , the computing device includes ahardware processor 504, display/speaker/microphone 506,wireless module 508,memory 510, and I/O 512. InFIG. 5 , elements of thecomputing device 500 are electrically and communicatively interconnected via abus 514. - The
processor 504 may comprise one or more specific or general purpose processors configured to execute instructions stored on thememory 510 that, when executed, control theprocessor 504 to execute various applications and perform various functions associated with thecomputing device 500. Additionally or alternatively, theprocessor 504 may include a programmable gate array and operate, at least in part, based on firmware. As an example, theprocessor 504 may execute instructions stored on thememory 510 including instructions for an operating system of thecomputing device 500 and instructions for applications. The applications that may be executed by thecomputing device 500 include mobile applications and engines, data mapping and data correlation processing applications, network optimization applications, an e-mail application, a map viewer application, and a web-browser application, among others. The input commands control applications executing on thecomputing device 500 to perform various tasks or functions associated with the applications. - The display/speaker/
microphone 506 is configured to display applications executing on theprocessor 504 and provide visual and audible feedback to a user of thecomputing device 500. To that end, the display/speaker/microphone 506 may comprise one or more display devices such as LCD, LED, OLED, and Electronic Ink displays, among others. The display/speaker/microphone 506 may also comprise one or more speakers and one or more microphones. - The
wireless communication module 508 is configured to provide wireless communication of data to and from thecomputing device 500. As a non-limiting group of examples, thewireless communication module 508 may be configured for cellular communications using one or more of GSM, CDMA, TDMA, OFDM and other cellular communications protocols, wireless area network communications using one or more of the family of 802.11x protocols and other wireless area network communications protocols, and Bluetooth communications protocols. - The
memory 510 may comprise a Random Access Memory (RAM), Read Only Memory (ROM), or any other tangible storage memory configured to store software programs for execution by theprocessor 504. As a non-limiting example group, thememory 510 may comprise one or more of dynamic, persistent, and semi-persistent solid state memories, magnetic memories, removable memories, or any other known memories suitable for the application of storing data and software programs for thecomputing device 500. - The I/
O 512 includes inputs and outputs of thecomputing device 500 such as power connectors, data connectors, and other input and output devices. The I/O 512 may comprise, for example, wired data communication input and output interfaces, power charging interfaces, infra-red interfaces, light and proximity sensors, capacitive sensors, “soft” and “hard” buttons, switches, and other input/output interfaces of thecomputing device 500. Thebus 514 is configured to electrically and communicatively connect theprocessor 504, the display/speaker/microphone 506, thewireless module 508, thememory 510, and the I/O 512 for transfer of data and instructions between elements of thecomputing device 500. - The example methods and acts described in the embodiments presented previously are illustrative, and, in alternative embodiments, certain acts can be performed in a different order, in parallel with one another, omitted entirely, and/or combined between different example embodiments, and/or certain additional acts can be performed, without departing from the scope and spirit of the invention. Accordingly, such alternative embodiments are included in the inventions described herein.
- Although specific embodiments have been described above in detail, the description is merely for purposes of illustration. It should be appreciated, therefore, that many aspects described above are not intended as required or essential elements unless explicitly stated otherwise. Modifications of, and equivalent acts corresponding to, the disclosed aspects of the example embodiments, in addition to those described above, can be made by a person of ordinary skill in the art, having the benefit of the present disclosure, without departing from the spirit and scope of the invention defined in the following claims, the scope of which is to be accorded the broadest interpretation so as to encompass such modifications and equivalent structures.
Claims (20)
1. A computing system comprising:
a data correlator;
a first data feed queue that supplies a first data packet from a network connection to the data correlator, wherein the first data packet comprises a key data element, wherein the first data packet further comprises a user plane data element or a control plane data element;
a data mapper; and
a second data feed queue that supplies a second data packet to the data mapper, wherein the second data packet comprises the key data element and a user identifier data element,
wherein the data mapper creates a hash value for the second data packet;
wherein the data correlator uses the hash value and the key data element to identify the second data packet and combines the first data packet and the second data packet to create a combined data packet comprising the user identifier data element and at least one of: the user plane data element or the control plane data element; and
wherein the data correlator provides the combined data packet to an external client.
2. The computing system of claim 1 , wherein the key data element comprises a device network identifier.
3. The computing system of claim 1 , wherein the user plane data element comprises a portion of one of: an image, a video frame, or text.
4. The computing system of claim 1 , wherein the user identifier data element comprises one of: a telephone number, an international mobile equipment identity code, an international mobile subscriber identity code, a name, or an account number.
5. The computing system of claim 1 , wherein the user identifier data element is retrieved from a user database.
6. The computing system of claim 1 , wherein the external client comprises an optimization engine, wherein the optimization engine uses the control plane data element and the key data element to identify a first cell location of a device and to transfer the device to a second cell location within a communication network.
7. The computing system of claim 1 , wherein the computing system further comprises:
a second data correlator; and
a third data feed queue that supplies a third data packet to the second data correlator, wherein the third data packet comprises the key data element and one of: the user plane data element or the control plane data element that is absent from the first data packet;
wherein the second data correlator uses the hash value and the key data element to identify the third data packet and to combine the third data packet and the second data packet to create a combined second data packet comprising the user identifier data element and the one of: the user plane data element or the control plane data element that is absent from the first data packet; and
wherein the second data correlator provides the combined second data packet to the external client.
8. The computing system of claim 1 , further comprising:
a second data correlator; and
a second data mapper;
wherein the second data correlator receives the combined data packet from the data correlator;
wherein the second data mapper receives a third data packet, wherein the third data packet comprises the key data element and one of: the user plane data element or the control plane data element that is absent from the first data packet;
wherein the second data mapper creates a second hash value for the third data packet;
wherein the second data correlator uses the second hash value and the key data element to identify the third data packet and to combine the combined data packet and the third data packet to create a second combined data packet comprising the user identifier data element, the user plane data element, and the control plane data element; and
wherein the second data correlator provides the second combined data packet to the external client.
9. The computing system of claim 8 , wherein the control plane data element comprises one of: a network performance measurement or location data.
10. The computing system of claim 1 , further comprising a delayed data correlator, wherein the delayed data correlator:
receives a third data packet from the data correlator when the data correlator is unable to combine the third data packet and a fourth data packet;
receives a second hash value from the data mapper;
uses the second hash value and the key data element to identify the third data packet and to combine the third data packet received from the data correlator and the fourth data packet received at the data mapper to create a delayed combined data packet comprising the user identifier data element and one of: the user plane data element or the control plane data element; and
provides the delayed combined data packet to the external client.
11. A method comprising:
receiving, from a network connection, a first data packet at a data correlator, wherein the first data packet comprises a key data element, wherein the first data packet further comprises a user plane data element or a control plane data element, and wherein the data correlator executes on at least one processor; and
receiving a second data packet at a data mapper, the data mapper executing on the at least one processor, wherein the second data packet comprises the key data element and a user identifier data element;
creating, by the data mapper, a hash value for the second data packet;
identifying, by the data correlator, the second data packet by using the hash value and the key data element;
combining, by the data correlator, the first data packet and the second data packet to create a combined data packet comprising the user identifier data element and the user plane data element or the control plane data element; and
providing the combined data packet to an external client.
12. The method of claim 11 , wherein the key data element comprises a device network identifier.
13. The method of claim 11 , wherein the user plane data element comprises a portion of one of: an image, a video frame, or text.
14. The method of claim 11 , wherein the user identifier data element comprises one of: a telephone number, an international mobile equipment identity code, an international mobile subscriber identity code, a name, or an account number.
15. The method of claim 11 , wherein the user identifier data element is retrieved from a user database.
16. The method of claim 11 , wherein the external client comprises an optimization engine, wherein the optimization engine uses the control plane data element and the key data element to identify a first cell location of a device and to transfer the device to a second cell location within a communication network.
17. The method of claim 11 , further comprising:
receiving a third data packet at a second data correlator, wherein the third data packet comprises the key data element and one of: the user plane data element or the control plane data element that is absent from the first data packet;
identifying, by the second data correlator, the third data packet by using the hash value and the key data element;
combining, by the second data correlator, the second data packet and the third data packet to create a combined second data packet comprising the user identifier data element and the one of: the user plane data element or the control plane data element that is absent from the first data packet; and
providing, by the second data correlator, the combined second data packet to the external client.
18. The method of claim 11 , further comprising:
receiving, at a second data correlator, the combined data packet from the data correlator;
receiving, at a second data mapper, a third data packet, wherein the third data packet comprises the key data element and one of: the user plane data element or the control plane data element that is absent from the first data packet;
creating, by the second data mapper, a second hash value for the third data packet;
identifying, by the second data correlator, the third data packet by using the second hash value and the key data element;
combining, by the second data correlator, the combined data packet and the third data packet to create a second combined data packet comprising the user identifier data element, the user plane data element, and the control plane data element; and
providing, by the second data correlator, the second combined data packet to the external client.
19. The method of claim 18 , wherein the control plane data element comprises one of:
a network performance measurement or location data.
20. A non-transitory computer-readable medium storing instructions which, when executed by a computing system including at least one processor, cause the computing system to perform operations, the operations comprising:
receiving, from a network connection, a first data packet, wherein the first data packet comprises a key data element, wherein the first data packet further comprises a user plane data element or a control plane data element; and
receiving a second data packet, wherein the second data packet comprises the key data element and a user identifier data element;
creating a hash value for the second data packet;
identifying the second data packet by using the hash value and the key data element;
combining the first data packet and the second data packet to create a combined data packet comprising the user identifier data element and the user plane data element or the control plane data element; and
providing the combined data packet to an external client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US17/447,572 US20210410005A1 (en) | 2019-08-27 | 2021-09-13 | Real-time large volume data correlation |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US16/552,729 US11122464B2 (en) | 2019-08-27 | 2019-08-27 | Real-time large volume data correlation |
| US17/447,572 US20210410005A1 (en) | 2019-08-27 | 2021-09-13 | Real-time large volume data correlation |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US16/552,729 Continuation US11122464B2 (en) | 2019-08-27 | 2019-08-27 | Real-time large volume data correlation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| US20210410005A1 true US20210410005A1 (en) | 2021-12-30 |
Family
ID=74682004
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US16/552,729 Active 2039-10-16 US11122464B2 (en) | 2019-08-27 | 2019-08-27 | Real-time large volume data correlation |
| US17/447,572 Abandoned US20210410005A1 (en) | 2019-08-27 | 2021-09-13 | Real-time large volume data correlation |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US16/552,729 Active 2039-10-16 US11122464B2 (en) | 2019-08-27 | 2019-08-27 | Real-time large volume data correlation |
Country Status (1)
| Country | Link |
|---|---|
| US (2) | US11122464B2 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US12004114B2 (en) | 2021-06-11 | 2024-06-04 | AT&T Intellectual Propety I, L.P. | Geolocation of wireless network users |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6356951B1 (en) * | 1999-03-01 | 2002-03-12 | Sun Microsystems, Inc. | System for parsing a packet for conformity with a predetermined protocol using mask and comparison values included in a parsing instruction |
| US6483804B1 (en) * | 1999-03-01 | 2002-11-19 | Sun Microsystems, Inc. | Method and apparatus for dynamic packet batching with a high performance network interface |
| US20100124239A1 (en) * | 2008-11-20 | 2010-05-20 | Silver Peak Systems, Inc. | Systems and methods for compressing packet data |
| US20100131607A1 (en) * | 2008-11-21 | 2010-05-27 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Correlating data indicating subjective user states associated with multiple users with data indicating objective occurrences |
| US20130227651A1 (en) * | 2012-02-28 | 2013-08-29 | Verizon Patent And Licensing Inc. | Method and system for multi-factor biometric authentication |
| US8762302B1 (en) * | 2013-02-22 | 2014-06-24 | Bottlenose, Inc. | System and method for revealing correlations between data streams |
| US20140297821A1 (en) * | 2013-03-27 | 2014-10-02 | Alcatel-Lucent Usa Inc. | System and method providing learning correlation of event data |
| US8874266B1 (en) * | 2012-01-19 | 2014-10-28 | Google Inc. | Enhancing sensor data by coordinating and/or correlating data attributes |
| US20150088841A1 (en) * | 2013-09-20 | 2015-03-26 | Oracle International Corporation | Techniques for correlating data in a repository system |
| US20160021070A1 (en) * | 2013-03-06 | 2016-01-21 | Amv Networks Gmbh | System and method for data-protection-compliant capture and forwarding of telemetry data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2510997B (en) * | 2012-05-21 | 2014-11-05 | Aceaxis Ltd | Detection of Intermodulation Products in a Wireless Network |
-
2019
- 2019-08-27 US US16/552,729 patent/US11122464B2/en active Active
-
2021
- 2021-09-13 US US17/447,572 patent/US20210410005A1/en not_active Abandoned
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6356951B1 (en) * | 1999-03-01 | 2002-03-12 | Sun Microsystems, Inc. | System for parsing a packet for conformity with a predetermined protocol using mask and comparison values included in a parsing instruction |
| US6483804B1 (en) * | 1999-03-01 | 2002-11-19 | Sun Microsystems, Inc. | Method and apparatus for dynamic packet batching with a high performance network interface |
| US20100124239A1 (en) * | 2008-11-20 | 2010-05-20 | Silver Peak Systems, Inc. | Systems and methods for compressing packet data |
| US20100131607A1 (en) * | 2008-11-21 | 2010-05-27 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Correlating data indicating subjective user states associated with multiple users with data indicating objective occurrences |
| US8874266B1 (en) * | 2012-01-19 | 2014-10-28 | Google Inc. | Enhancing sensor data by coordinating and/or correlating data attributes |
| US20130227651A1 (en) * | 2012-02-28 | 2013-08-29 | Verizon Patent And Licensing Inc. | Method and system for multi-factor biometric authentication |
| US8762302B1 (en) * | 2013-02-22 | 2014-06-24 | Bottlenose, Inc. | System and method for revealing correlations between data streams |
| US20160021070A1 (en) * | 2013-03-06 | 2016-01-21 | Amv Networks Gmbh | System and method for data-protection-compliant capture and forwarding of telemetry data |
| US20140297821A1 (en) * | 2013-03-27 | 2014-10-02 | Alcatel-Lucent Usa Inc. | System and method providing learning correlation of event data |
| US20150088841A1 (en) * | 2013-09-20 | 2015-03-26 | Oracle International Corporation | Techniques for correlating data in a repository system |
Also Published As
| Publication number | Publication date |
|---|---|
| US11122464B2 (en) | 2021-09-14 |
| US20210068006A1 (en) | 2021-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2654366B1 (en) | Session-aware GTPv2 load balancing | |
| US10686854B2 (en) | Streaming content using ad hoc networks of user devices | |
| US20130272136A1 (en) | Session-Aware GTPv1 Load Balancing | |
| US20200120551A1 (en) | Triggering selective fallback based on user subscription information | |
| CN105580444A (en) | Communication system, base station, communication method, and non-transitory computer readable medium storing program | |
| US10820231B2 (en) | Systems and methods for APN based CoS and QoS control for network services | |
| US10749965B2 (en) | Systems and methods for capturing and logging web application traffic | |
| CN109428866A (en) | Method, system and the computer readable storage medium of subscriber identity information are provided | |
| EP2763451B1 (en) | Monitoring 3g/4g handovers in telecommunications networks | |
| US10439935B2 (en) | Systems and methods for providing external services to core network traffic | |
| CN103476017B (en) | Method and device for association of LTE (Long Term Evolution) network user information | |
| US8270942B2 (en) | Method for the interception of GTP-C messages | |
| CN108605048A (en) | Selective calling with flight control connects system | |
| US20210410005A1 (en) | Real-time large volume data correlation | |
| CN100444690C (en) | Method for Realizing Update of Roaming Terminal Group Information in Trunking System | |
| CN104798397B (en) | Method and apparatus for hosting an application via an access node | |
| US20170005889A1 (en) | Self-localizing data distribution network | |
| US10149149B2 (en) | Method, system and device for accessing data storage in a telecommunications network | |
| US20240334318A1 (en) | System and method for steering network traffic for multiple core networks | |
| CN114362891B (en) | A communication method and device | |
| CN105493471A (en) | Transparent data transmission method and public service entity | |
| KR102273390B1 (en) | Method and Apparatus for Integrating Network Function | |
| US10003657B2 (en) | Data transmission processing method and apparatus | |
| CN109246589B (en) | Method and equipment for realizing positioning service of remote halo terminal | |
| EP3629618A1 (en) | High-volume low-impact quality of service management for internet of things |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
| STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE |