WO2010051020A1

WO2010051020A1 - System to create and maintain internet meta-maps

Info

Publication number: WO2010051020A1
Application number: PCT/US2009/005839
Authority: WO
Inventors: Jesse D. Hurley; Seth Voltz
Original assignee: Anamorphic Systems, Inc.
Priority date: 2008-10-29
Filing date: 2009-10-28
Publication date: 2010-05-06

Abstract

Various embodiments provide a system and method to generate a comprehensive meta-map of the Internet, enabling the creator of the meta-map to gather important data related to information routing capabilities at the network level. The embodiments providing the meta-map enable network operators and observers to perform advanced network management techniques through the accurate and timely presentation of highly granular network topologies. The meta-map documentation system records topological traversal, securely cataloging and profiling from backbone through access networks and into local area networks in homes and businesses.

Description

SYSTEM TO CREATE AND MAINTAIN INTERNET META-

MAPS

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority benefit of U.S. Patent Application

No. 12/260,285, entitled, "SYSTEM AND METHOD TO CREATE, POPULATE, AND MAINTAIN AN INTERNET META-MAP" filed October 29, 2008, which is hereby incorporated by reference in its entirety.

COPYRIGHT NOTICE

[0002] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever. The following notice applies to the software and data as described below and in the drawings that form a part of this document: Copyright 2007-2008, Jesse D. Hurley and Seth Voltz. AU Rights Reserved.

FIELD

[0003] The disclosed subject matter relates to the field of data processing devices, and more particularly to networked data processing devices, telecommunications, and Internetworking services.

BACKGROUND

[0004] Network mapping is a field of interest relevant to telecommunications and Internetworking services, hi traditional network topology mapping, a mapping service provides routing information about routes on known routers to queried end-points using ICMP (Internet Control Messaging Protocol) messaging frameworks. The tools traditionally used to accomplish this, ping and traceroute, are uni-directional in their service, resolving routers that do not restrict external ICMP reporting. Sophisticated network backbone maps have been generated using just traceroute servers and databases, e.g. Scriptroute and Rocketfuel, but these uni-directional systems do not extend to protected areas of access networks.

BRIEF DESCRIPTION OF THE DRAWINGS

[0005] FIG. 1 is a block diagram illustrating an example embodiment of a system for creating, populating and maintaining an Internet Meta Map in accordance with the described embodiments.

[0006] FIG. 2 is a block diagram illustrating an example embodiment of the creation process of the embodiment shown in FIG. 1.

[0007] FIG. 3 is a block diagram illustrating an example embodiment of the population process of the embodiment shown in FIG. 1. [0008] FIG. 4 is a block diagram illustrating an example embodiment of the maintenance process of the embodiment shown in FIG. 1.

[0009] FIG. 5 is a block diagram illustrating an example embodiment of the presentation process of the embodiment shown in FIG. 1.

[0010] FIG. 6 is a flow diagram illustrating the processing flow for a particular example embodiment.

[0011] FIG. 7 illustrates an example of a computing system on which processing for various embodiments can be implemented.

DESCRIPTION OF EXAMPLE EMBODIMENTS [0012] An apparatus and method for creating, populating, and maintaining an Internet Meta-Map is disclosed. In the following detailed description, reference is made to the accompanying drawings that form a part hereof, and in which are shown, by way of illustration, specific embodiments in which the disclosed subject matter can be practiced. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the disclosed subject matter. Overview

[0013] Internet Control Messaging Protocol (ICMP) is a network management service protocol that employs control codes managing router packet traversal of network architectures at the IP layer. Defined by the Internet Engineering Task Force (IETF) Request for Comment (RFC) 792, ICMP has remained a mainstay of Internet infrastructure.

[0014] User Datagram Protocol (UDP) is a packet protocol that allows nodes to send packets to other nodes on the network, supporting both broadcast and multicast on local networks. Defined by IETF RFC 768, the UDP protocol is a well documented mainstay of Internet infrastructure.

[0015] Transmission Control Protocol (TCP) is a packet protocol that like UDP, enables nodes to send packets to other nodes on the network, but additionally incorporates control and confirmation messaging routines enabling, among other things, error correction and timing of packet receipt. It is defined by IETF RFC 1122, and extends service on top of the IP layer in Internet network topologies.

[0016] Local Area Network(s) (LAN) are network topologies closest to the edge of machines (hosts) operating on the network architecture, frequently close in geography, subnet mask, and IP addressing range. Typically, in industry, the LAN is the protected side of a firewall, where the Internet facing side faces traffic restrictions to prevent unauthorized access to the network. [0017] Metropolitan Area Network(s) (MAN) are network topologies employed by large enterprises and by Internet Service Providers (ISP)'s to manage subnetworks of LANs that are connected to their network. Typically, MANs manage links and IP layer transmissions from the LANs to the Internet or to a Wide Area Network, (WAN).

[0018] Access Network topologies are components of ISP networks that provide data services to customers and Customer Premises Equipment (CPE) nodes using MANs. These access networks frequently split bandwidth across many CPE installations, diversifying cost and resources.

[0019] Backbone Network topologies are the primary carriers of data traffic for the Internet, with major Tier 1 ISPs multilaterally peering traffic originating across the backbone. This critical component to the Internet ensures that traffic can take multiple, diverse paths across the network, tying dispersed MANs and LANs to the infrastructure.

[0020] Peering is a practice employed by Tier 1 ISPs that passes traffic from other Tier 1 ISPs and their customers through their own networks at low or no cost. This community effort facilitates communication with nodes that are segregated to a single provider's network, connecting those nodes to every other node on the Internetwork. Without peering, Internet architectures and the services dependent upon them would not be technologically feasible. [0021] Ingress points are nodes on the Internetwork facing side of an ISP's infrastructure that provide traffic flows into the ISP's network, frequently using specialized gateway routers employing the BGP4 protocol. [0022] Egress points are nodes on the Internetwork facing side of an

ISP's infrastructure that provide traffic flows out of the ISP's network, which like ingress nodes employ BGP4 routing protocols. [0023] Border Gateway Protocol version 4 (BGP4) is a recent revision of the Border Gateway Protocol (BGP) that maintains backbone router communications, and directs traffic flows at ingress and egress nodes to ISP networks. It has been described as a path vector protocol because it seeks to optimize traffic on the Internetwork on the basis of shortest route and network QoS (Quality of Service) rulesets. It is codified by IETF RFC 4271.

[0024] Routing Information Protocol (RIP) is a service protocol employed by routers on LANs to facilitate contacting addressable hosts that the router controls. The RIP employs the Bellman-Ford algorithm to manage routes between hosts on the LAN. IPv4 standards are defined by IETF RFC 2453, and IPv6 standards are set by IETF RFC 2080.

[0025] Address Resolution Protocol (ARP) is a service protocol employed on routers which tabulates IP address mapping to Media Access Control (MAC) addresses of host Network Interface Cards (NIC)s. IPv4 employs ARP as specified in IETF RFC 826, which is being deprecated by IPv6 addressing with Neighbor Discovery Protocol (NDP) in IPv6 network architectures codified by IETF RFC 4861. Tools

[0026] A series of tools enable the creation of network maps, and are also instrumental as information gathering subcomponents used in the creation of the network topology layer of an Internet meta-map. [0027] Traceroute is a tool to determine network topologies by varying

"time-to-live" (TTL) attributes of User Datagram Protocol (UDP) packets directed to remote hosts on a network, wherein upon expiry of the TTL, the remote router reports a failure to the sender through Internet Control Message Protocol (ICMP) "time exceeded" or "destination unreachable" control packets. The traceroute tool has been updated to support IPv6 addressing, making it available for traversing networks wherein routers support the newer IPv6 framework. Modern traceroute utilities can additionally use TCP SYN packets in an attempt to get a return vector from the target host. Traceroutes are routinely blocked by BGP4 routers on the ingress and egress edge routers of many Internet Service Providers (ISP)s, rendering Traceroute by itself functionally useless for mapping Internet-connected metropolitan area networks (MAN)s, access networks, and Local Area Networks (LAN)s.

[0028] TCPTraceroute is a tool that functions similarly to traceroute in that it maps network topologies unidirectionally, but does so using TCP SYN requests. This TCP based implementation introduces an additional level of resilience and resolution to the directional mapping methodology employed. Like traceroute, this system can be blocked by network operators who are sensitive to network probing, making this tool subject to similar limitations as its UDP based traceroute sibling. [0029] HPing2 is a powerful packet forging tool enabling the construction of custom ICMP, TCP and UDP packets to test network architectures. It can enable MTU discovery, firewall port resolution, and network performance tests across diverse protocols. [0030] MTR is a combination tool that performs ping and traceroute processes simultaneously. It is written for linux operating systems, and can be useful for Maximum Transmission Unit (MTU) discovery. [0031] Otrace is a rapid network mapping tool for established connections. It enables "hop enumeration", across persisted TCP connections between nodes such as exists in HTTP or SMTP. It was developed from 2000-- 2006, and disclosed in a second version in January 2007. It was subsequently improved upon by another tool, iTrace by enhancing trace resolution quality in more restricted networks. [0032] Nessus is an automated software tool to identify, classify, and report on a list of known host system vulnerabilities. Part of a proprietary implementation for enterprise deployment, nessus can be used to rapidly identify security vulnerabilities across network topologies. Its database of security flaws can be updated with the Nessus Attack Scripting Language (NASL). [0033] Firewalk is a packet forging tool for firewall port discovery similar to HPing2, written by Michael D. Schiffman. The tool uses TTL variances to probe extents of controlled zones managed by firewalls. Like HPing2, it is unable to resolve firewall ports if ICMP responses are blocked by the infrastructure. [0034] PoF is a passive operating system characterization tool that uses system response signatures to classify the systems responding by OS type. It can also be used for verifying security configuration of network topologies to ensure that they are compliant with required policy. [0035] Ping is a tool to contact a remote host via an ICMP "echo request", requesting a lightweight ICMP "echo response". When the remote host returns the "echo response" ICMP packet, the routing round-trip time, packet loss and latency of delivery is recorded and reported to the requesting ping tool instance. Like traceroutes which employ ICMP packets for operation, pings are regularly blocked by ISPs on gateway routers facing access networks and MANs.

[0036] Nmap is a tool to determine host based services via automated discovery of service messaging, open network-facing ports, and common responses delivered against known templates for services and applications in Internet infrastructure, e.g. web servers or email servers. This powerful network service and host characterization tool can be quite dangerous if the information it collects is compromised and conveyed to malicious third parties. Typically, well-protected networks prevent and flag nmap scans, alerting network administrators of a potential intrusion attempt, making this tool functionally useless by itself for characterizing most Internet-facing networks. [0037] Dig is a tool to determine comprehensive information regarding a named domain's location on the Internet Protocol infrastructure. It reports information regarding "A records", "CNAME records", "LOC records", "MX records", "NS records", "PTR records", "SOA records", "SRV records", and "TXT records". The powerful tool is a utility for populating DNS caches on local client machines, making it possible to rapidly retrieve content on the Internet via a web browser or other domain name referencing tool. [0038] Whois is a tool that provides user name and domain name service identity resolution based upon registrar and system ID data sources. Reporting queries a whois server which reports either a comprehensive "authoritative" record, or refers to a secondary server which is listed as the primary registrar. [0039] Domain Name System - Service Discovery (DNS-SD) and Universal Plug 'n Play (UPnP) Browsers are tools that employ a UDP-based network protocol comprised of announce and browse messages using the DNS- SD and UPnP messaging frameworks. Compatible devices and software instances on the network report back to the requester their availability and their levels of compliance with DNS-SD and UPnP service frameworks. [0040] Rocketfuel is a tool developed by researchers working at the

University of Washington, presented at an Association of Computing Machinery (ACM) conference in 2002. It pro grammatically maps the topologies of backbone networks, using vantage point servers and looking glass servers, presenting a compelling representation of the core routers operating the Internetwork. Their work was later extended and superceded by their work in another tool, Scriptroute.

[0041] Scriptroute is a tool developed by researchers working at the

University of Washington, and now continued at the University of Maryland as part of a research group run by Neil Spring, funded by a grant from the Defense Advanced Research Projects Agency (DARPA). The tool was first publicly demonstrated at a Usenix conference in 2003, and described in a series of papers published by Spring's working group to the Usenix Symposium. Using the Scriptroute tool, the University of Washington team successfully deduced the network topologies of long obscured ISP backbone networks, later verified with the ISPs for accuracy. This demonstrated the first, competent, comprehensive, full-scale backbone map, albeit limited in its scan duration and durable accuracy. [0042] LanSurveyor is a tool sold commercially by SolarWinds that automatically maps local area networks (LANs), but is unusable in mapping backbone networks from external vantage points. The tool generates a network map of the local area controlled network, finding node and edge extents that sit behind controlled firewall routers. Integrated with Solarwinds' ipMonitor product, LanSurveyor purports to be a real-time map of host availability on managed LAN topologies.

[0043] Looking Glass Servers are nodes on ISP or hosting provider networks to provide traces to and from nodes for network health verification. These servers were used as "vantage points" in the Scriptroute and Rocketfuel projects, making automated discovery of the backbone possible. They can respond to BGP queries, as well as standard format ICMP packet requests. Typically they are placed on networks that are exposed so that they can provide valid network health statistics across the Internet. The technology has been well demonstrated, as continuous use of looking glass servers dates to the mid 1990s. [0044] Internet Mapping Project was a project set up at Carnegie Mellon University by Hal Burch and Bill Cheswick, formerly of Bell Laboratories in which they "brute force" tracerouted most of the backbone architecture present at the time. The demonstration proved that traceroutes can be used to comprehensively map the backbone network infrastructure, but the effort also strongly demonstrated the load and limitations of such a system. The project implementation employed UDP packets dispatched to targets with varying TTLs to probe networks using ICMP packet returns.

[0045] The Abilene Map is a project operated by the University of

Indiana to provide a network mapping tool for Internet2 traffic. It attempts to provide real-time data and visualization of the Internet2 backbone topology traffic flows, the first ever publicly available demonstration of such a capability for Internet2 backbone architecture. Backbone Mapping

[0046] Looking glass servers provided the first real resource to establish the health of a particular edge of the network architecture map by responding to routine ICMP probes and traceroute methods. These servers are now commonly used on the backbone of ISP and enterprise networks to assist in QoS and health monitoring efforts, as well as to feed data into dynamic ruleset validation for traffic shaping and management. These servers are regularly used by researchers in the backbone mapping field to get updated pictures of ISP topologies. The presence of these systems on the Internetwork has enabled important research into network neutrality issues, capacity planning for backbone architectures, and helped expose and classify the persistent threat of denial of service attacks. These systems represent a key component in the effort to form a defensive attack attribution architecture for characterizing network attacks. A good example of these backbone monitoring systems are the network attack monitoring services operated by CERT/CE and the SANS Institute.

[0047] Such looking glass dependent systems are limited in many ways, but primarily because information gathering processes for the backbone map were never automated. Research projects were funded by DARPA to rectify this, producing the Rocketfuel and Scriptroute tools. The reach of these tools and their ability to produce comprehensive backbone maps is extremely impressive, especially when contrasted to the brute force efforts of the Internet Mapping Project, but they remain limited in their ability to produce a real-time backbone map, and are completely incapable of mapping the access networks where the majority of traffic capacity limitations lie in the US Internetwork architecture. Further, like all ICMP based scans, such automated systems and tools are limited by route asymmetries and filtering.

[0048] The European Union has started a public project known as

MOMENT, which is an attempt to provide another method for backbone mapping to provide a unified framework for sharing information across national ISPs. The project currently remains in the proposal and design stage soliciting public comment. Local Area Network Mapping

[0049] LAN mapping is a procedure regularly used by network administrators to build, test and maintain LAN topologies. Tools such as LAN Surveyor and ethereal are instrumental in determining security vulnerabilities that exist so that these holes can be patched. Such tools are also very effective in troubleshooting activities where network communications are cut off or in which quality of service is poor, making the tool important in the proper maintenance of a LAN. However, with these tools, comprehensive management of these topologies becomes difficult for large installations, and is completely frustrated for Internet-dispersed topologies. Further, should a malicious outsider gain access to the LAN map, services which are not routinely secured because of an design choice to rely upon firewalls for security are subject to compromise, threatening the entire organization. Such information, therefore, must be safeguarded and access to sections of maps that characterize LANs restricted to authorized and authenticated users.

Supporting Information - Data Resolution & Reliability Quality [0050] The ideal extension of such mapping systems would be a comprehensive, automated, real-time map of the Internet from backbone through access networks, and into LANs. The current efforts, primarily focused on mapping the backbone OR the LAN, and never updated in real-time, are limited in scope and listed below:

• (backbone-to-LAN penetration, ICMP filtering, ingress-egress routing hierarchies, reverse-path trace) • Security Concerns (Firewall punching, map hijacking, DDoS, etc.)

• Private Networks (Corporate, Government, ISP Ops, Universities)

• Protocol Mapping (VoIP, HTTP, SSH, FTP, P2P)

• Historical Data (Time Interval, collection and collation of data)

• Quality of Service Information (Latency Deltas, Jitter Deltas, Jitter, Latency, Route permanence, DNS resolution lag times, DNS resolution cache times) Description of an Internet Meta Map

[0051] An Internet Meta Map is an information artifact that is the product of the various embodiments described herein, and can be dynamically updated by the system as conditions within the scope of the Meta Map's monitored networks change. The Internet Meta Map is a map of Internet topology of routers, hosts, devices, and users. The map can be updated real-time, on a recurring schedule, or based upon triggered events. The Meta Map can be linked to additional information such as user presence, geo-location data, network service availability on hosts, device capabilities, as well as traffic profiles of infrastructural elements in the backbone, MANs or access networks. The Meta Map can be extended to correlate new data with nodes and edges, e.g. connections among users, devices, locations, traffic patterns, and data access. As the Meta Map grows and builds a historiographic profile of activity within its scope, it can be further used to train intelligent agents to identify aberrations in patterns from baseline conditions established in the device and user set models.

The Internet Meta Map as a Framework (API extensibility)

[0052] The utility of the Meta Map is vastly enhanced by enabling credentialed access to its growing data set, and thus the incorporation of an API is a welcome addition. This will allow Meta Map data to be shared and analyzed by a diverse user base, each with a unique set of target objectives. This ability to repurpose the data of the Meta Map also enables access to the data set as a service, in itself providing opportunities for new innovations built on the Meta Map.

Privacy, Network Security & Auditing

[0053] The creation of the Internet Meta Map artifact by the described system introduces new questions and new responsibilities in information security, in areas including privacy, digital identity rights, responsible disclosure, and intellectual property management. The system does not enable "hacking", and in fact promotes the reverse by encouraging development of comprehensive knowledge bases of network patterns, thereby providing a powerful new baseline for active intrusion detection and network management. Issues at the national level

[0054] The Internet policy of the United States has largely supported the development of the backbone and access networks through the encouragement of commercial investment by ISPs. The extraordinary growth of the Internet as a staple of commerce has underscored the importance of the Internet infrastructure, evincing the need for a comprehensive Internet-wide monitoring and management utility. Recently, foreign governments and organizations hostile to the United States have used Internet based attacks to weaken or disrupt Internet access for critical government services, e.g. in Estonia and Georgia. These attacks were either directed via Distributed Denial of Service (DDoS) mechanisms, or by attack models identified by the Infosec Research Council report on network threats. Work at the national level to secure "cyberspace" has become a focus area for the Department of Homeland Security (DHS), codified in the "National Strategy to Secure Cyberspace." Highlighting the need for, and the present absence of, a comprehensive framework to monitor Internet service in real-time, the White House and the DHS have solicited private sector partnerships to perform the development of such systems. In a recent (July 2008) RFP put forth by DARPA, the Department of Defense (DoD) focused research agency, called for new tools to address emerging Internet threats, specifying the need for a real time system to monitor gateway links to other national networks on the Internet infrastructure.

[0055] The Internet Meta Map as an artifact provides critical-path information for developing a comprehensive framework for attack attribution in real time. The current state of the art in being limited to backbone networks cannot adequately provide provenance data for DDoS (Distributed Denial of Service) attack patterns, making modeling difficult and prevention impossible in real time without also potentially disrupting service for legitimate network users. As a system to develop the Internet Meta Map, the various embodiments address this critical need for secure, real-time network information collection, collation, and attack modeling.

[0056] The system in creating the Internet Meta Map enables the realtime modeling of traffic flows, establishing a baseline of legitimate activity against which rapidly deployable and controllable filters can be built to identify and block malicious traffic. Models that are responsive in real time can assist in identifying emerging threats, enabling network operators to mitigate damage to their own networks, and by their interconnected nature and peering practices, the Internet at large.

[0057] Once the Meta Map is populated, it will enable networks to become pro-active in network management rather than reactionary in response to network threats. A new system of dynamic filters, built on the Meta Map, would be capable of providing management and security support at gateway speeds (lOGbps or more), making the prospect of automated and intelligent network management a reality. The ultimate goal of such a system is to completely mitigate the spread of malicious attack patterns by integrating Meta Map based models with alerting, and dynamic immunity frameworks across the access networks, denying attackers the ability to infect zombies and marshal bots critical to mounting a DDoS attack.

[0058] With the population of a Meta Map, the information contained must be verifiable and the information provenance of the data within the system must be maintained. Future developments would protect the access integrity of the meta map, while performing "out-of-band" cross-referenced checking of map data source security.

Issues at the infrastructure level

[0059] The national infrastructure of Internet facilities includes backbone networks, MANs, access networks, and LANs. The interconnected routers of these systems use protocols that have long been clearly specified in IETF standards. The challenge to this infrastructure is that it is inflexible in its ability to cost effectively scale - linearly or exponentially - to increasing demand for bandwidth. US consumer and commercial demands for increasing bandwidth speeds have placed critical strain on the access networks, and are close to saturating the presently deployed backbone networks of the Tier-1 ISP's. The hardware is facing new stressors as demands for throughput increase by an order of magnitude every 5-7 years. The next growth cycle is known to be unstable, with the transition to video content distribution placing access networks close to critical failure points, particularly in the access networks operated by cable company MSO' s which provide greater than 60% of the broadband access offerings to the US market. Pirate distribution of content, facilitated by emerging P2P networks, further threatens network topologies not designed to handle the increased traffic. The problem underscores a critical need for a comprehensive information service that allows the ISPs to monitor overall network conditions in real time, while simultaneously providing tools for protection against DDoS attack, P2P induced access network failures, and traffic shaping activities. No Internet-wide framework for such a system at the infrastructural level presently exists. Efforts by organizations like the SANS Institute do provide backbone support, but due to technological limitations of their architecture SANS' systems cannot accurately monitor access networks and LANs.

Issues at the private network level [0060] Private networks are operated by enterprise corporations, universities and the like. These networks often employ sophisticated firewall and intrusion detection systems specifically designed to prevent malicious third parties from probing and attacking their internal networks. Because of this, extension of Meta Map topology analyses into these domains requires support from secured clients behind the firewall or behind the "Demilitarized Zone" (DMZ) of the private network. Inclusion in the Meta Map of these network topologies assists these networks by providing information metrics, helping network administrators to plan capacity upgrades and further strengthen their networks against intrusion by unauthorized outsiders while enabling access for properly credentialed users. Further, the integration of Meta Map data into these architectures will enable new services to be securely deployed that span LAN, MAN and WAN topologies, reducing the total management cost and security risk of geographically dispersed users in accessing the network.

Issues at the local area network level

[0061] The various embodiments described herein address a unique set of issues in LAN topologies. First, security of any network analysis tool is of paramount importance to LAN administrators, not only in its ability to expose latent vulnerabilities that would enable a malicious third party to compromise access controls, but also because it enables the same LAN administrators to make intelligent decisions about the security of their network in real time as user, client, and device behavior patterns change. Secured access to this information will provide real tools for the network administrator to mitigate "trusted insider" breaches of network security, as well as assist in identifying otherwise potentially legitimate user behaviors that inadvertently compromise network integrity, e.g. file sharing of corporate documents in working teams. The various embodiments, in operation at the LAN level, can provide valuable data about the hosts and devices acting within the LAN topology, providing the ability to address potential threats from infected machines or compromised systems in real time.

At the host/device level [0062] Integration of various embodiments with security software embedded in the host or device will enable the creation of new policy management systems for digital rights and content management, new auditing tools for tracing content and service usage, and will provide a framework for protecting privacy in a precise, and comprehensive manner. Limited device security and attachment of unprotected devices to enterprise networks not only exposes the network to potential compromise, but also may lead to infection from other devices that have been compromised on the network - a dangerous feedback loop. The various embodiments can provide information to devices about network condition, and future enhancements will make it possible to extend that real-time security protection to the host. Beyond security, the various embodiments can also enable the device to be notified of and consume network services, within a rapid framework, making file sharing, collaboration systems, and communication services easier to deploy and maintain.

At the user level

[0063] User awareness of network topology and context is a powerful ability that will enable that user to make accurate decisions about their privacy and the security of their communications. It will also allow that user to make use of new services that connect their devices, locations, and tools to a secure and personal, Internet-wide network. With a provenance verified and credentialed Meta Map, Internet wide security for personal communications is easier to preserve.

Example embodiments

[0064] The example embodiments described herein provide systems and methods for creating, populating and maintaining an Internet Meta Map by mapping routes and collating host data via methods arbitrated by a control manager that commands remote and local modules in a networked architecture that stores raw, parsed and analyzed data in connected storage systems. The methods control local and remote module activities in performing mapping functions, secure data transfer and module communications and control, enable route resolution, storing routing data, timing route information updating, providing event driven controls for mapping functions, parsing map data, and generating provenance metadata. Local modules extend the system by providing API access and support for marshaling of connected sets of control managers. [0065] A block diagram of a preferred embodiment is shown in FIG. 1, in which a control manager 120 which is comprised of several trusted modules (140, 150, 160, and 170) employing a push messaging based control system as is standard in the state of the art - incorporating by reference US utility patent application 2008/0010480 Al, invented by inventors common to the present patent application. The system additionally employs a subsystem and supporting methods in authentication, authorization, and access controls for identity based secure communications as is standard in the state of the art - incorporating by reference US utility patent application 2008/0031459 Al, invented by inventors common to the present patent application.

[0066] The system to create the Internet Meta Map is comprised of a remote module (105 through 109) connected across a network 130 to a trusted control manager 120 and local modules (110, 220) connected to the trusted control manager 120 that provide security services, communication services, data processing services, mapping services and other supporting services. The remote modules securely receive and process mapping commands, traceroute commands, authentication controls, authorization controls, access controls, and other commands from the control manager. The connection between the remote module and the trusted control manager is as described in the incorporated push communications system. Data collected by the remote modules is transferred to the control manager and then marshaled to the local modules for processing and/or storage. In mapping between remote modules, the control manager arbitrates communications and identifies targets for the remote clients to scan. [0067] The system provides methods for secure and verifiable population of the Internet Meta Map through remote module to control manager link route mapping, remote module to secondary remote module mapping secured and arbitrated by the control manager, remote module to vantage point mapping, secured local module to secured local module mapping, and secured local module to vantage point mapping. [0068] The system provides methods for timed and also event driven updating of the Internet Meta Map through the processes of event managers 140 and timer modules 170. The timed update to the Internet Meta Map enables the scalable processing of map data collection and analysis. Intervals to the timed mapping methods can be set dynamically by controls from the control manager 120 through the timer service. Triggered events in an "evented" model can be also used, and such events to activate mapping processes are provided by an event manager 140 embedded in the control manager 120. The event manager 140 also connects to the Mapper API 210 for external programmatic control of event triggering processes as permitted by authorized and authenticated methods. Alternative embodiments may include event managers and timers embedded in local modules that can coordinate communications and processes with the event manager and/or timer in the control manager and among other local modules. [0069] Data flows generally from the remote or local modules into the control manager, and from the control manager into the map parser 220 connected to transitive storage 230 systems where the data is temporarily stored pending analysis by the map analytic processor 240. Data from the map analytic processor once completed processing is stored in a structured storage system 250, e.g. memory or disk based databases. These structured data stores are then subsequently accessed by the map analytic processor 240 for secondary processing or correlation tasks, storing the new analytic data in the structured storage system 250. A presentation engine 260 that accesses the analytic data held within the structured storage system can expose the data for presentation to a credentialed user accessing the system via a remote module, the Mapping API 210 via credentialed services, or to a trusted reporting server that generates periodic summaries of map composition (not shown).

[0070] Mapping tools employed by remote and local modules in the preferred embodiment may include, but are not limited to well known traceroute routines, nmap utilities, pOf tools, ping tools, UPnP browsing and networking tools, DNS-SD network resolution and communication tools, and other host and network characterization tools that assist in providing data to the system via processes run on remote or local modules. Alternative embodiments may include improvements or enhancements in mapping processes by addition of tools not specified here and are automatically considered part of the system embodied herein.

Map Creation

[0071] A preferred embodiment for the collection and creation of map data is shown in FIG. 2. In step 250 the map system 200 sends a scan request message through the control manager 120 to a scanner local module 110 as in step 255 or a remote module (105, 109) as shown in steps 256 and 257. Upon receipt of the scan request message, the module (110, 105, or 109) performs the requested network scan operation with the specified scan type to the requested destination. In FIG. 2, this is shown where remote module 1 105 has been requested to scan to the scanner local module 110 as in step 260 and vice-versa in step 261. Upon the completion or an error in the scan, the modules (110, 105, or 109) report back their findings to the control manager 120 as in steps 265 and 266. The findings are then routed to the map parser 220 as in step 270. The parser performs a transformation of the data, which prepares the transformed data to be stored in the transitive data store 230 as is done in step 275. The scan data stays in the transitive store until the data is retrieved and processed in the population process described in relation to FIG. 3. Map Population

(0072] A preferred embodiment of the map population process is shown in FIG. 3. In step 300 the map analytic processor 240 requests new scans stored in the transitive data store 230 which are returned in step 305. The map analytic processor 240 converts the raw scan data into map objects (e.g., edges, nodes, and associated meta data) corresponding to network entities discovered by the network scan operation and by analyzing the relationships between the host representations in the scans with other data in the scan, and existing data in the map in step 310. As each scan is processed, the analytic processor 240 stores the data in the structured data store 250 as in step 315. The structured store reports back success or failure for the save in step 320 which may trigger a retry of step 315 or, upon success, the analytic processor can delete the processed raw scan data from the transitive store as in step 325.

Map Maintenance

[0073] In order to keep the map up to date the map analytic processor

260 must request new scans as described in FIG. 2. A preferred embodiment of the map maintenance process is described in FIG. 4. In step 350 the analytic processor queries the transitive data store 250 for map data that may be out of date or otherwise need refreshing. That data is returned in step 355 and analyzed in step 360 to create a set of scan directives determining source, destination and type of scan. Those directives are sent as push messages through the control manager 120 in step 365 to other modules which receive the messages and scan as described in FIG. 2.

Map Presentation

[0074] The map is maintained in its native format in the structured data store 250 which is not useful for humans to view. A preferred embodiment of the map presentation process, to make the data human-viewable, is shown in FIG. 5. In step 400, an external request for map presentation data is forwarded by the control manager 120 to the map presenter 260. The map presenter collects the relevant data to complete the request from the structured store 250 in step 405. The requested data is returned in step 410 and transformed from its native format into a presentation format (such as XML, a proprietary binary format or a format native to a third-party display engine like Mathematica) in step 415 before it is sent back to the requesting party via the control manager 120 as shown in step 420. [0075] Figure 6 illustrates a process flow diagram for an example embodiment. In the embodiment 610 shown, an apparatus and method for creating, populating, and maintaining an Internet Meta-Map includes: providing a control module (processing block 615); providing a plurality of remote modules in data communication with the control module via a network (processing block 620); using the control module to send a scan request to a requested one of the plurality of remote modules (processing block 625); using the requested one of the plurality of remote modules to perform a network scan operation and to gather scan data from the scan operation (processing block 630); and converting the scan data into map objects for display in a network meta-map (processing block 635).

[0076] Figure 7 shows a diagrammatic representation of a machine in the example form of a computer system 700 within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed. In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client machine in client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0077] The example computer system 700 includes a processor 702 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), a main memory 704 and a static memory 706, which communicate with each other via a bus 708. The computer system 700 may further include a video display unit 710 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system 700 also includes an input device 712 (e.g., a keyboard), a cursor control device 714 (e.g., a mouse), a disk drive unit 716, a signal generation device 718 (e.g., a speaker) and a network interface device 720. [0078] The disk drive unit 716 includes a machine-readable medium 722 on which is stored one or more sets of instructions (e.g., software 724) embodying any one or more of the methodologies or functions described herein. The instructions 724 may also reside, completely or at least partially, within the main memory 704, the static memory 706, and/or within the processor 702 during execution thereof by the computer system 700. The main memory 704 and the processor 702 also may constitute machine-readable media. The instructions 724 may further be transmitted or received over a network 726 via the network interface device 720.

[0079] Applications that may include the apparatus and systems of various embodiments broadly include a variety of electronic and computer systems. Some embodiments implement functions in two or more specific interconnected hardware modules or devices with related control and data signals communicated between and through the modules, or as portions of an application-specific integrated circuit. Thus, the example system is applicable to software, firmware, and hardware implementations, hi example embodiments, a computer system (e.g., a standalone, client or server computer system) configured by an application may constitute a "module" that is configured and operates to perform certain operations as described herein. In other embodiments, the "module" may be implemented mechanically or electronically. For example, a module may comprise dedicated circuitry or logic that is permanently configured (e.g., within a special-purpose processor) to perform certain operations. A module may also comprise programmable logic or circuitry (e.g., as encompassed within a general-purpose processor or other programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement a module mechanically, in the dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g. configured by software) may be driven by cost and time considerations. Accordingly, the term "module" should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in a certain manner and/or to perform certain operations described herein. While the machine-readable medium 722 is shown in an example embodiment to be a single medium, the term "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term "machine-readable medium" shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present description. The term "machine-readable medium" shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic media, and carrier wave signals. As noted, the software may be transmitted over a network using a transmission medium. The term "transmission medium" shall be taken to include any medium that is capable of storing, encoding or carrying instructions for transmission to and execution by the machine, and includes digital or analog communications signal or other intangible medium to facilitate transmission and communication of such software.

[0080] The illustrations of embodiments described herein are intended to provide a general understanding of the structure of various embodiments, and they are not intended to serve as a complete description of all the elements and features of apparatus and systems that might make use of the structures described herein. Many other embodiments will be apparent to those of ordinary skill in the art upon reviewing the above description. Other embodiments may be utilized and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. The figures provided herein are merely representational and may not be drawn to scale. Certain proportions thereof may be exaggerated, while others may be minimized. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. [0081] Although the present specification describes components and functions implemented in the embodiments with reference to particular standards and protocols, the disclosed subject matter may be not limited to such standards and protocols. Each of the standards for Internet and other packet-switched network transmission (e.g., transmission control protocol (TCP) / Internet Protocol (IP) (TCP/IP), User Datagram Protocol (UDP) / Internet Protocol (IP) (UDP/IP), Hypertext Markup Language (HTML), and Hypertext Transfer Protocol (HTTP) represent examples of the state of the art. Such standards are periodically superseded by faster or more efficient equivalents having essentially the same functions. Accordingly, replacement standards and protocols having the same functions are considered equivalents.

[0082] Thus, as described above, an apparatus and method for creating, populating, and maintaining an Internet Meta-Map is disclosed. Although the disclosed subject matter has been described with reference to several example embodiments, it may be understood that the words that have been used are words of description and illustration, rather than words of limitation. Changes may be made within the purview of the appended claims, as presently stated and as amended, without departing from the scope and spirit of the disclosed subject matter in all its aspects. Although the disclosed subject matter has been described with reference to particular means, materials, and embodiments, the disclosed subject matter is not intended to be limited to the particulars disclosed; rather, the subject matter extends to all functionally equivalent structures, methods, and uses such as are within the scope of the appended claims.

Claims

ClaimsWe claim:

1. A method comprising: providing a control module; providing a plurality of remote modules in data communication with the control module via a network; using the control module to send a scan request to a requested one of the plurality of remote modules; using the requested one of the plurality of remote modules to perform a network scan operation and to gather scan data from the scan operation; and converting the scan data into map objects for display in a network meta- map.

2. The method as claimed in Claim 1 wherein the network scan operation is one or more of the operations from the group: tracerouting, nmap utility operations, pOf tools, ping tools, Universal Plug 'n Play (UPnP) browsing and networking tools, and Domain Name System - Service Discovery (DNS-SD) network resolution and communication tools.

3. The method as claimed in Claim 1 wherein the scan request is sent using a push messaging based control system.

4. The method as claimed in Claim 1 wherein the control module is in secure data communications with the plurality of remote modules.

5. The method as claimed in Claim 1 wherein the control module identifies targets for the plurality of remote modules to scan.

6. The method as claimed in Claim 1 using the requested one of the plurality of remote modules to send the scan request to a secondary remote module, which can perform a secondary network scan operation and gather secondary scan data from the scan operation, the secondary scan data being sent back to the control module.

7. The method as claimed in Claim 1 using the requested one of the plurality of remote modules to send the scan request to a vantage point, which can perform a secondary network scan operation and gather secondary scan data from the scan operation, the secondary scan data being sent back to the control module.

8. The method as claimed in Claim 1 wherein at least one of the plurality of remote modules is a local module.

9. The method as claimed in Claim 1 including periodically updating the network scan data.

10. The method as claimed in Claim 1 wherein the map objects include edges, nodes, and meta data corresponding to network entities discovered by the network scan operation.

11. A system comprising: a control module; a plurality of remote modules in data communication with the control module via a network, wherein the control module being configured to send a scan request to a requested one of the plurality of remote modules, and wherein the requested one of the plurality of remote modules being configured to perform a network scan operation and to gather scan data from the scan operation; and a map analytic processor to convert the scan data into map objects for display in a network meta-map.

12. The system as claimed in Claim 11 wherein the network scan operation is one or more of the operations from the group: tracerouting, nmap utility operations, pOf tools, ping tools, Universal Plug 'n Play (UPnP) browsing and networking tools, and Domain Name System - Service Discovery (DNS-SD) network resolution and communication tools.

13. The system as claimed in Claim 11 wherein the scan request is sent using a push messaging based control system.

14. The system as claimed in Claim 1 1 wherein the control module is in secure data communications with the plurality of remote modules.

15. The system as claimed in Claim 11 wherein the control module identifies targets for the plurality of remote modules to scan.

16. The system as claimed in Claim 11 wherein the requested one of the plurality of remote modules being configured to send the scan request to a secondary remote module, which can perform a secondary network scan operation and gather secondary scan data from the scan operation, the secondary scan data being sent back to the control module.

17. The system as claimed in Claim 11 wherein the requested one of the plurality of remote modules being configured to send the scan request to a vantage point, which can perform a secondary network scan operation and gather secondary scan data from the scan operation, the secondary scan data being sent back to the control module.

18. The system as claimed in Claim 11 wherein at least one of the plurality of remote modules is a local module.

19. The system as claimed in Claim 11 wherein the control module being configured to periodically request updates to the network scan data.

20. The system as claimed in Claim 11 wherein the map objects include edges, nodes, and meta data corresponding to network entities discovered by the network scan operation.

21. An apparatus comprising: means for providing a control module; means for providing a plurality of remote modules in data communication with the control module via a network; means for using the control module to send a scan request to a requested one of the plurality of remote modules; means for using the requested one of the plurality of remote modules to perform a network scan operation and to gather scan data from the scan operation; and means for converting the scan data into map objects for display in a network meta-map.