HK1151368B

HK1151368B - Method for determining network proximity for global traffic load balancing using passive tcp performance instrumentation

Info

Publication number: HK1151368B
Application number: HK11105379.7A
Authority: HK
Inventors: 迈克尔‧克里斯蒂安; 大卫‧阿普盖尔; 杰彦斯‧威加亚拉戈哈凡
Original assignee: Jollify Management Limited
Priority date: 2007-12-21
Filing date: 2008-12-12
Publication date: 2014-07-04

Description

Method for determining network proximity for global traffic load balancing using passive TCP performance test equipment

Technical Field

The present invention relates to TCP connections, and more particularly to determining connection quality of a TCP connection.

Background

The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Accordingly, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.

The term "data center" is used herein to refer to a collaborative location (collaboration) of associated servers. Although the servers belonging to a particular data center are in the same building or complex, the data centers are typically located at geographically remote locations from each other. The geographical distance adds protection so that a sudden failure in one data center caused by a natural disaster does not cause a failure in the other data centers as well. For example, one data center may be located on the east coast of new york and another data center may be located on the west coast of san francisco.

Global load balancing or "GLB" is a mechanism for distributing client access to a particular service among multiple servers. For example, in the case where a particular service is provided by servers belonging to data centers in new york and san francisco, the GLB may assign client access such that the number of clients connected to the data center in new york is about the same as the number of clients connected to the data center in san francisco.

When used in the context of the internet, the GLB may use various active and passive monitoring techniques to generate a comprehensive mapping of the internet. Based on this mapping, the GLB makes traffic routing decisions to connect clients to the "closest" server. As used herein, "close" does not necessarily mean that the decision is based solely on geographic proximity. As used herein, a "close" server is a server that produces the fastest connection to a client. Thus, if a server is located 100 miles away, and it is slower for a client to reach that server than a server located 200 miles away because of severe congestion, the GLB will route the client to a "closer" server that is 200 miles away.

Many active and passive monitoring mechanisms establish a global mapping of internet proximity for the GLB. The protocols used by these mechanisms may include, but are not limited to: icmp (ping), BGP (border gateway protocol), and manual entry. The Internet Control Message Protocol (ICMP) is one of the core protocols of the internet. One important ICMP application is the ping tool. The ping tool sends and receives ICMP echo requests and response messages to determine whether a host is reachable (reachable) and to determine the length of time required for packets to travel to and from the host. The Border Gateway Protocol (BGP) is the core routing protocol of the internet. BGP works by maintaining an IP network table that specifies network reachability among autonomous systems. BGP makes routing decisions based on paths, network policies, and rule sets. Unfortunately, these mechanisms and protocols are unable to monitor the actual performance of a web connection employing the TCP protocol, and thus, are unable to achieve accurate routing determinations for TCP connections.

GLB systems may have difficulty maintaining a complete and accurate mapping of the internet due to dynamic changes in topology and connectivity. Inaccuracies in the mapping may result in erroneous routing decisions. Depending on what mapping protocol is employed, a significant amount of time may be required to correct these routing decisions.

Disclosure of Invention

According to a first aspect of the present invention, there is provided a method for measuring connection quality from a plurality of data centers to a client, comprising: receiving connection quality measurements for each end user client; wherein the connection quality measurements for the end-user client comprise, from each of the plurality of data centers, a measurement of connection quality between (a) the end-user client and (b) the data center; determining the end user clients and grouping the end user clients into net blocks based on the IP addresses of the end user clients; for each of the net-blocks, generating aggregated connection quality measurement data by aggregating connection quality measurements for those end-user clients belonging to that net-block; and outputting the aggregated connection quality measurement data to a plurality of load balancing servers for use by the load balancing servers in determining how to route messages between the end user client and the data center.

According to a second aspect of the present invention, there is provided a method for measuring connection quality from a plurality of data centers to a client, comprising: storing, from the server into a computer-readable non-transitory storage medium, connection quality measurements for each end-user client; wherein the connection quality measurements for the end-user client comprise, from each of the plurality of data centers, a measurement of connection quality between (a) the end-user client and (b) the data center; determining the end user clients and grouping the end user clients into net blocks based on the IP addresses of the end user clients; for each of the net blocks, generating aggregated connection quality measurement data by aggregating connection quality measurements of those end user clients belonging to that net block of a plurality of net blocks; wherein, the network block to which each end user client belongs is based on the IP address of the end user client; and outputting the aggregated connection quality measurement data to a plurality of load balancing servers for use by the plurality of load balancing servers in determining how to route messages between the end user client and the data center.

According to a third aspect of the present invention, there is provided a method for measuring connection quality from a plurality of data centers to a client, comprising: receiving requests from a plurality of end-user clients; storing the generated connection quality measurements for each end-user client based on the request; determining the end user clients and grouping the end user clients into net blocks based on the IP addresses of the end user clients; outputting aggregated connection quality measurement data to a plurality of load balancing servers for use by the plurality of load balancing servers in determining how to route messages between the end user client and the data center; wherein the aggregated connection quality measurement data is generated for each of the net blocks by aggregating connection quality measurements for those end user clients belonging to that net block; wherein the method is performed by one or more computing devices.

According to a fourth aspect of the present invention, there is provided a system for measuring connection quality from a plurality of data centers to a client, comprising: a first plurality of servers, each server of the first plurality of servers located in one of a plurality of data centers, wherein the first plurality of servers measure connection quality measurements for each end user client and generate connection quality measurements for the each end user client; wherein the connection quality measurements for the end-user client comprise, from each of the plurality of data centers, a measurement of connection quality between (a) the end-user client and (b) the data center; an aggregation server, wherein the aggregation server receives the connection quality measurements for each end-user client from the first plurality of servers; determining the end user clients based on the IP addresses of the end user clients and grouping the end user clients into net blocks; for each of the net-blocks, generating aggregated connection quality measurement data by aggregating connection quality measurements for those end-user clients belonging to that net-block; and a second plurality of servers, wherein the second plurality of servers receive the aggregated connection quality measurement data for use in determining how to route messages between the end-user client and the data center.

According to a fifth aspect of the present invention, there is provided an apparatus for measuring connection quality from a plurality of data centers to a client, comprising: means for receiving connection quality measurements for each end-user client; wherein the connection quality measurements for the end-user client comprise, from each of the plurality of data centers, a measurement of connection quality between (a) the end-user client and (b) the data center; means for determining and grouping the end user clients into net blocks based on their IP addresses; means for generating aggregated connection quality measurement data for each of the net blocks by aggregating connection quality measurements for those end user clients belonging to that net block; and means for outputting the aggregated connection quality measurement data to a plurality of load balancing servers for use by the load balancing servers in determining how to route messages between the end user client and the data center.

According to a sixth aspect of the present invention, there is provided an apparatus for measuring connection quality from a plurality of data centers to a client, comprising: means for storing connection quality measurements for each end-user client from the server into a computer-readable non-transitory storage medium; wherein the connection quality measurements for the end-user client comprise, from each of the plurality of data centers, a measurement of connection quality between (a) the end-user client and (b) the data center; means for determining and grouping the end user clients into net blocks based on their IP addresses; means for generating aggregated connection quality measurement data for each of the net blocks by aggregating connection quality measurements for those end user clients belonging to that net block of a plurality of net blocks; wherein, the network block to which each end user client belongs is based on the IP address of the end user client; and means for outputting the aggregated connection quality measurement data to a plurality of load balancing servers for use by the plurality of load balancing servers in determining how to route messages between the end user client and the data center.

According to a seventh aspect of the present invention, there is provided an apparatus for measuring connection quality from a plurality of data centers to a client, comprising: means for receiving requests from a plurality of end-user clients; means for storing the generated connection quality measurements for each end-user client based on the request; means for determining and grouping the end user clients into net blocks based on the IP addresses of the end user clients; means for outputting aggregated connection quality measurement data to a plurality of load balancing servers for use by the plurality of load balancing servers in determining how to route messages between the end user client and the data center; wherein the aggregated connection quality measurement data is generated for each of the net blocks by aggregating connection quality measurements for those end user clients belonging to that net block.

Drawings

The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:

FIG. 1 is a diagram showing clients connected to web beacons (beacons) located in different data centers, according to one embodiment of the invention;

fig. 2 is a diagram showing data aggregated from web beacons and processed data assigned to a GLB server according to an embodiment of the invention;

FIG. 3 is a diagram illustrating processed data grouped by a network block (netblock) according to an embodiment of the invention; and

FIG. 4 is a block diagram of a computer system upon which an embodiment of the invention may be implemented.

Detailed Description

Techniques for measuring connection quality of actual TCP connections and techniques for using measurement information in subsequent traffic routing decisions are described. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it is understood that the invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.

General overview

The transmission control protocol ("TCP") is an internet protocol that enables an application on a networked host to create a connection to another host. For example, a client requesting a web page may represent one host and a server providing web page content to the client may represent another host.

The TCP protocol has many properties related to the connection between hosts. TCP guarantees reliable and in-order delivery of data from a sender to a receiver. To achieve in-order delivery, TCP also provides for retransmission of lost packets and discarding of duplicate packets sent. TCP is also able to distinguish data of multiple connections of concurrent applications (e.g., web servers and email servers) running on the same host.

To initiate a TCP connection, the two hosts exchange initial sequence numbers. The initial sequence number identifies the order of bytes sent from each host so that the transferred data remains in order regardless of any breaks or out of order that may occur during the transfer. For each byte transmitted, the sequence number is incremented.

Each byte sent is assigned a sequence number by the sender and the receiver then sends an acknowledgement back to the sender to acknowledge the transmission. For example, if computer a (sender) sends 4 bytes with sequence number 50 (the four bytes in the packet have assigned sequence numbers 50, 51, 52, and 53), then computer B (receiver) sends an acknowledgement 54 back to computer a to indicate the next byte that computer B expects to receive. By sending an acknowledgement 54, computer B is informing that bytes 50, 51, 52 and 53 were received correctly. If it is not known why the last two bytes were corrupted, computer B sends an acknowledgement value 52 because bytes 50 and 51 were successfully received.

TCP is also capable of performing congestion control. For example, TCP may adjust the rate at which data enters the network to keep the data flow below a rate that would cause network congestion problems. The acknowledgement received or lack of acknowledgement of receipt for the transmitted data is measured by the transmitter to learn about the network conditions between the transmitter and the receiver. The TCP sender and receiver may then alter the rate of the data flow based on the network conditions encountered to ensure good traffic flow.

Unfortunately, actual TCP connection measurements to ensure accurate measurements of connection quality may not be performed by third parties or non-TCP protocols. For example, using a non-TCP protocol such as ICMP as a substitute for the actual TCP connection measurements may lead to erroneous results. One reason is that: ICMP traffic generally has a lower quality of service ("QoS") than TCP traffic. QoS is the priority given to traffic on the internet. QoS provides different priorities to different data flows by guaranteeing a certain level of performance for the data flows according to requests from applications. The QoS of ping traffic used by ICMP is given lower priority than TCP because ping is not guaranteed to always reach the ping's destination. Instead, TCP guarantees that all traffic will reach a particular destination. As a result, certain measurements may be affected by lower QoS, such as, but not limited to, larger packet drop rates and queuing times than normal. Thus, ICMP-based routing decisions for TCP connections will be erroneous.

It is difficult for third party services to measure TCP connection quality because TCP is a point-to-point protocol. In a point-to-point protocol, traffic is routed from one network point to another. For example, one network point may be a client requesting a web page, and another network point may be a server providing the web page to the client. A third party can only intercept TCP traffic of a point-to-point connection between two connection points if it is on the network and can actively view all packets in the connection between the two parties. Therefore, it is very difficult and impractical to make measurements of TCP connections by third party services.

Web beacon (beacon) and passive monitoring

In an embodiment, web beacons and passive server monitoring measure the connection quality of the actual TCP connection between the client and the server within the data center that provides the client content. web beacons are also known as pixel tags (pixel tags), transparent gif (clear gif), or zero-content images (zero-content images), and are transparent or invisible graphical images, typically no more than 1 x 1 pixels.

In one embodiment, the web beacon is generated by placing a small amount of code or beacon code into the generated web page. The client requests and is served a generated web page from a web server in the data center. When the generated web page is processed by the client, the beacon code causes the client browser to retrieve a web beacon or null content image from a beacon server located in each potential data center location. For example, if there are three different collaborative locations or data centers in the united states that can provide the web page, one on the west coast, another on the east coast, and yet another in the midwest, then the beacon code causes the client to request web beacons or zero-content images from beacon servers located at each collaborative location with TCP connections. These requests are performed in the background of the client so as not to interfere with the loading of the currently generated web page from the web server.

In an embodiment, a passive monitoring system in the kernel module of the beacon server retrieves statistics from the web beacon that measure and register TCP connections between the client and the data center. Network latency may be measured, for example, by timing the arrival and departure of packets during a TCP handshake sequence. As another example, the packet loss ratio (packet lossratio) over the lifetime of a TCP connection may be measured by counting the number of TCP retransmissions required for the connection. These two metrics, network latency and packet loss ratio, provide an excellent indication of the quality of the connection between a particular client and a particular data center. Any other measurements related to the TCP connection may also be measured and registered by the passive monitoring system.

Because the measurement of TCP connections from web beacons is based on actual web traffic from the data center to the client, accurate TCP connection quality measurements are ensured. These measurements can then be used to subsequently make intelligent global traffic routing decisions to connect the "closest" server in the data center to the client.

A diagram illustrating a web beacon and passive monitoring in accordance with an embodiment is shown in fig. 1. In fig. 1, client 100 may be routed to 3 different data centers. Data center 102 includes beacon server 108 and web server 110. Data center 104 includes beacon server 112 and web server 114. Data center 106 includes beacon server 116 and web server 118. The client 100 requests and receives a web page from the web server 110 (transmission 120). The web page provided by the web server 110 contains beacon code that instructs the client 100 to retrieve web beacons from the web beacon servers located in the various data centers. As a result, client 100 requests a web beacon from beacon server 108 in data center 102 via request 122, a web beacon from beacon server 112 in data center 104 via request 124, and a web beacon from beacon server 116 in data center 106 via request 126. To retrieve a web beacon, a TCP connection is established between the client 100 and each beacon server 108, 112, and 116. All of these requests for web beacons are made in the background of the client to not interfere with the page loading of the web server 110.

The passive monitoring system in each beacon server measures and registers statistics about the TCP connections established between the client 100 and each beacon server 108, 112 and 116 for retrieving web beacons. Measurements of the TCP connection are used to determine the quality between the client 100 and the data center corresponding to each beacon server. The registered and recorded measurement data may vary from implementation to implementation. In one embodiment, the measurement data includes, but is not limited to: round trip latency data and packet loss rate data. Measurements and statistics on each TCP connection for a web beacon are stored by each beacon server.

Measuring TCP connections by other methods

Various other methods may also be used to perform the measurement of the performance of the TCP connection. Beacons, or direct clients to connect to a server, are used to measure connection performance. A client may be directed to connect to multiple beacon servers. More than one beacon server is used to enable multiple endpoints of a connection to be measured. Further, the client may be directed to connect to a particular beacon server based on a random selection of beacon servers. The beacon server may be a dedicated server dedicated only to measuring connection data. The beacon server may also be a server that provides real traffic. Any type of TCP-based connection protocol may be used, including but not limited to: FTP (file transfer protocol), HTTP (hypertext transfer protocol), or IRC (internet online chat), and any application that has some control over the location to which the user is connected may be used.

In one embodiment, the application is placed on the computer of the client. When an application is started or run, the application attempts to connect to multiple beacon servers. In another embodiment, a DNS request is made from the parser for each occurrence of a beacon server, and a reply is sent for the client to connect to multiple beacon servers. The selection of the beacon server may vary from implementation to implementation. In one embodiment, the bootstrap client connects to a particular beacon server based on a random selection method. Even with random selection, the amount of data collected over time is sufficient to bring the random selection method close to the accuracy with which clients connect to the beacon server in each data center. Once connected to any of the beacon servers, the passive monitoring system in each beacon server measures and registers statistics about the established connection from the client to the beacon server.

Aggregating and processing data

In one embodiment, the database collects TCP measurements and associated client IP addresses from beacon servers located in each data center. The data may be requested by a database server or transmitted periodically by each web beacon server. The database may be located in a particular data center or remotely in a separate facility.

In one embodiment, the database server normalizes the data received from the beacon server into a set of discrete net blocks. A net block as used herein is a range of client IP addresses. For example, a netblock of "1.1.1.0" indicates all clients whose IP addresses start with "1.1.1. x", where "x" can be any value between 0 and 255. Clients in the same net block may use the same path to the data center. As a result, TCP connection measurements for clients in the same netblock may be aggregated to provide more complete information about the path from a particular client to a particular data center. To determine the best service destination, the connection performance of the paths from each data center to each possible net block is measured and maintained.

In an embodiment, the list of available servers or data centers is sorted by the quality of the connection to each particular net block (sort). In another embodiment, the database server normalizes the data based on TCP statistics received from the beacon server. In this case, the connections are ordered based on the quality of the connection with the associated IP address. In yet another embodiment, the database collects the measurement data and the corresponding IP address and does not perform any further processing on the data.

In one embodiment, measurement-based data is exported from a database to a distributed authority name server (authority name server) that acts as a global load balancer. The measurement-based data provides the global load balancer with information to make a correct decision about client routing. In an embodiment, measurement-based data may be exclusively used by the global load balancer to determine client routes. In another embodiment, the measurement-based data is additional data that the global load balancer considers for making routing decisions.

A diagram illustrating a central database collecting TCP statistics from a beacon server according to an embodiment is shown in fig. 2. In fig. 2, the beacon servers of the three data centers of fig. 1 (beacon server 108 of data center 102, beacon server 112 of data center 104, and beacon server 116 of data center 106) send data to central database 200. Central database 200 aggregates and classifies data based on various criteria. The processed data is then sent to GLB202, GLB 204, GLB 206 and GLB 208. In another embodiment, the processed data is transmitted from the central database 200 to other user devices (consumers) 210. Other user devices include applications other than web servers that may be hosted in a data center. For example, an internet company may have a standalone music or media application that allows clients to connect directly to the application. Because these applications are contained in the same data center as the beacon server, the TCP data accumulated by the central server can also be used to efficiently route traffic to these other applications.

Global load balancer utilizing measurement-based data

Examples of how the global load balancer uses measurement-based data are described later. Global load balancing is performed when a client requests access to a particular web page. A DNS lookup is performed to translate a web site URL (e.g., "www.sampledomain.com") entered by the client into an IP address (e.g., "1.2.3.4"). The lookup is directed to an authoritative domain name server (also a global load balancer). The global load balancer checks the request IP address. The request IP address is then compared with information for that particular netblock (or range of IP addresses) in the categorized measurement-based data. The global load balancer selects the first available web server on the list ordered by various TCP statistics (e.g., network latency and packet loss) and returns the IP address of that web server to the client. The client is thus routed to the web server with the best available TCP connectivity.

An illustration of processed data sorted by net blocks according to an embodiment of the invention is shown in fig. 3. The data has a "client" column 300, a "collaborative location a" column 302, a "collaborative location B" column 304, and a "collaborative location C" column 306. The "client" column 300 lists the IP addresses of the clients to which each collaborative location is connected. In one embodiment, clients are listed in a netblock. In line 308, the netblock "1.1.1.0/24" indicates that the subnet mask of the client at IP address "1.1.1. x" is 24, where x can be any number between 0-255. If the client wishes to connect to a collaborative location from this net-block, it takes 10ms to connect to collaborative location a, 50ms to connect to collaborative location B, and 80ms to connect to collaborative location C. Thus, the client from netblock "1.1.1.0" will be closest to the collaborative location a.

In line 310, the netblock "2.2.2.0/26" indicates that the subnet mask of the client at IP address "2.2.2. x" is 26, where x can be any number between 0-63. If the client wishes to connect to a collaborative location from this netblock, it takes 100ms to connect to collaborative location a, 40ms to connect to collaborative location B, and 5ms to connect to collaborative location C. Thus, the client from netblock "2.2.2.0" will be closest to the co-location C.

In line 312, the netblock "3.3.3.0/24" indicates that the subnet mask of the client at IP address "3.3.3. x" is 24, where x can be any number between 0-255. If the client wishes to connect to a collaborative location from this net-block, it takes 300ms to connect to collaborative location a, 1ms to connect to collaborative location B, and 500ms to connect to collaborative location C. Thus, the client from netblock "3.3.3.0" will be closest to the co-location B.

Closed circuit feedback

In one embodiment, a closed circuit feedback loop is generated that can dynamically respond to changes in internet topology and performance. As shown above, the global load balancer routes clients to particular web servers to fetch web content. The global load balancer determines routes based on TCP connection measurements by retrieving web beacons.

When the client retrieves the web page content from the server, the web page content contains beacon code that instructs the client to retrieve the web beacon from each data center again. By constantly getting more measurements and statistics from the web beacon TCP connection, the connection quality measurement system is able to quickly self-correct for situations that affect network availability. These situations include, but are not limited to: fiber breakage, equipment problems, server failover, or capacity problems.

Overview of hardware

FIG. 4 is a block diagram that illustrates a computer system 400 upon which an embodiment of the invention may be implemented. Computer system 400 includes a bus 402 or other communication mechanism for communicating information, and a processor 404 coupled with bus 402 for processing information. Computer system 400 also includes a main memory 406, such as a Random Access Memory (RAM) or other dynamic storage device, coupled to bus 402 for storing information and instructions to be executed by processor 404. Main memory 406 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 404. Computer system 400 also includes a Read Only Memory (ROM)408 or other static storage device coupled to bus 402 for storing static information and instructions for processor 404. A storage device 410, such as a magnetic disk or optical disk, is provided and coupled to bus 402 for storing information and instructions.

Computer system 400 may be coupled via bus 402 to a display 412, such as a Cathode Ray Tube (CRT), for displaying information to a computer user. An input device 414, including alphanumeric and other keys, is coupled to bus 402 for communicating information and command selections to processor 404. Another type of user input device is cursor control 416, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 404 and for controlling cursor movement on display 412. The input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allow the device to specify positions in a plane.

The invention is related to the use of computer system 400 for implementing the techniques described herein. According to one embodiment of the invention, those techniques are performed by computer system 400 in response to processor 404 executing one or more sequences of one or more instructions contained in main memory 406. Such instructions may be read into main memory 406 from another machine-readable medium, such as storage device 410. Execution of the sequences of instructions contained in main memory 406 causes processor 404 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.

The term "machine-readable medium" as used herein refers to any medium that participates in providing data that causes a machine to operation in a specific fashion. In an embodiment implemented using computer system 400, various machine-readable media are involved, for example, in providing instructions to processor 404 for execution. Such a medium may take many forms, including but not limited to storage media and transmission media. Storage media includes both non-volatile media and volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 410. Volatile media includes dynamic memory, such as main memory 406. Transmission media includes coaxial cables, copper wire or fiber optics, including the wires that comprise bus 402. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infrared data communications. All such media must be tangible to enable the instructions carried by the media to be deleted by a physical mechanism that reads the instructions into a machine.

Common forms of machine-readable media include, for example: a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM or any other optical medium, punch cards, paper tape, or any other physical medium with patterns of holes, a RAM, a ROM, an EPROM, a FLASH-EPROM, or any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.

Various forms of machine-readable media may be involved in carrying one or more sequences of one or more instructions to processor 404 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 400 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 402. Bus 402 carries the data to main memory 406, from which main memory 406 processor 404 retrieves and executes the instructions. The instructions received by main memory 406 may optionally be stored on storage device 410 either before or after execution by processor 404.

Computer system 400 also includes a communication interface 418 coupled to bus 402. Communication interface 418 provides a two-way data communication coupling to a network link 420, network link 420 being connected to a local network 422. For example, communication interface 418 may be an Integrated Services Digital Network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 418 may be a Local Area Network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 418 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

Network link 420 typically provides data communication through one or more networks to other data devices. For example, network link 420 provides a connection through local network 422 to a host computer 424 or to data equipment operated by an Internet Service Provider (ISP) 426. ISP 426 in turn provides data communication services through the world wide packet data communication network 428 now commonly referred to as the "Internet". Local network 422 and internet 428 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 420 and through communication interface 418, which carry the digital data to and from computer system 400, are exemplary forms of carrier waves transporting the information.

Computer system 400 can send messages and receive data, including program code, through the network(s), network link 420 and communication interface 418. In the Internet example, a server 430 might transmit a requested code for an application program through Internet 428, ISP 426, local network 422 and communication interface 418.

The received code may be executed by processor 404 as it is received, and/or stored in storage device 410, or other non-volatile storage for later execution. In this manner, computer system 400 may obtain application code in the form of a carrier wave.

In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Thus, the sole and exclusive indicator of what is the invention, and is intended by the applicants to be the invention, is the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Hence, no limitation, element, property, feature, advantage or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims

1. A method for measuring connection quality from a plurality of data centers to a client, comprising:

receiving connection quality measurements for each end user client;

wherein the connection quality measurements for the end-user client comprise, from each of the plurality of data centers, a measurement of connection quality between (a) the end-user client and (b) the data center;

determining the end user clients and grouping the end user clients into net blocks based on the IP addresses of the end user clients;

for each of the net-blocks, generating aggregated connection quality measurement data by aggregating connection quality measurements for those end-user clients belonging to that net-block; and

outputting the aggregated connection quality measurement data to a plurality of load balancing servers for use by the load balancing servers in determining how to route messages between the end user client and the data center.

2. The method of claim 1, wherein the measurement of connection quality is based on a TCP connection from the end-user client to a plurality of beacon servers, wherein each beacon server of the plurality of beacon servers corresponds to one of the plurality of data centers.

3. The method of claim 2, wherein the TCP connection is a result of providing web beacon code to each of the end-user clients.

4. The method of claim 1, wherein the aggregated connection quality measurement data is organized into a plurality of groups, wherein for each group, the IP address of the end-user client falls within the IP address range of the same netblock.

5. The method of claim 4, wherein the set of aggregated connection quality measurement data comprises data centers ordered by connection quality to an IP address range of the same netblock.

6. The method of claim 1, wherein the measurement of connection quality comprises network latency.

7. The method of claim 1, wherein the measure of connection quality comprises a packet loss ratio.

8. The method of claim 2, wherein the TCP connection originates from an end-user client that retrieves a null-content image corresponding to one of the plurality of data centers from each of the plurality of data centers.

9. A method for measuring connection quality from a plurality of data centers to a client, comprising:

storing, from the server into a computer-readable non-transitory storage medium, connection quality measurements for each end-user client;

for each of the net blocks, generating aggregated connection quality measurement data by aggregating connection quality measurements of those end user clients belonging to that net block of a plurality of net blocks;

wherein, the network block to which each end user client belongs is based on the IP address of the end user client; and

outputting the aggregated connection quality measurement data to a plurality of load balancing servers for use by the plurality of load balancing servers in determining how to route messages between the end user client and the data center.

10. The method of claim 9, wherein the measurement of connection quality is based on a TCP connection from the end-user client to a plurality of beacon servers, wherein each beacon server of the plurality of beacon servers corresponds to one of the plurality of data centers.

11. The method of claim 10, wherein the TCP connection is a result of providing web beacon code to each of the end-user clients.

12. The method of claim 9, wherein the aggregated connection quality measurement data is organized into a plurality of groups, wherein for each group, the IP address of the end-user client falls within the IP address range of the same netblock.

13. A method for measuring connection quality from a plurality of data centers to a client, comprising:

receiving requests from a plurality of end-user clients;

storing the generated connection quality measurements for each end-user client based on the request;

outputting aggregated connection quality measurement data to a plurality of load balancing servers for use by the plurality of load balancing servers in determining how to route messages between the end user client and the data center;

wherein the aggregated connection quality measurement data is generated for each of the net blocks by aggregating connection quality measurements for those end user clients belonging to that net block;

wherein the method is performed by one or more computing devices.

14. The method of claim 13, wherein the measurement of connection quality is based on a TCP connection from the end-user client to a plurality of beacon servers, wherein each beacon server of the plurality of beacon servers corresponds to one of the plurality of data centers.

15. The method of claim 14, wherein the TCP connection is a result of providing web beacon code to each of the plurality of end-user clients.

16. The method of claim 13, wherein the aggregated connection quality measurement data is organized into a plurality of groups, wherein for each group, the IP address of the end-user client falls within the IP address range of the same netblock.

17. An apparatus for measuring connection quality from a plurality of data centers to a client, comprising:

means for receiving connection quality measurements for each end-user client;

means for determining and grouping the end user clients into net blocks based on their IP addresses;

means for generating aggregated connection quality measurement data for each of the net blocks by aggregating connection quality measurements for those end user clients belonging to that net block; and

means for outputting the aggregated connection quality measurement data to a plurality of load balancing servers for use by the load balancing servers in determining how to route messages between the end user client and the data center.

18. The apparatus of claim 17, wherein the measurement of connection quality is based on a TCP connection from the end-user client to a plurality of beacon servers, wherein each beacon server of the plurality of beacon servers corresponds to one of the plurality of data centers.

19. The apparatus of claim 18, wherein the TCP connection is a result of providing web beacon code to each of the end-user clients.

20. The apparatus of claim 17, wherein the aggregated connection quality measurement data is organized into a plurality of groups, wherein for each group, the IP address of the end-user client falls within the IP address range of the same netblock.

21. The apparatus of claim 20, wherein the set of aggregated connection quality measurement data comprises data centers ordered by connection quality to an IP address range of the same netblock.

22. The apparatus of claim 17, wherein the measure of connection quality comprises network latency.

23. The apparatus of claim 17, wherein the measure of connection quality comprises a packet loss ratio.

24. The apparatus of claim 18, wherein the TCP connection originates from an end-user client that retrieves a null-content image corresponding to one of the plurality of data centers from each of the plurality of data centers.

25. An apparatus for measuring connection quality from a plurality of data centers to a client, comprising:

means for storing connection quality measurements for each end-user client from the server into a computer-readable non-transitory storage medium;

means for generating aggregated connection quality measurement data for each of the net blocks by aggregating connection quality measurements for those end user clients belonging to that net block of a plurality of net blocks;

means for outputting the aggregated connection quality measurement data to a plurality of load balancing servers for use by the plurality of load balancing servers in determining how to route messages between the end user client and the data center.

26. The apparatus of claim 25, wherein the measurement of connection quality is based on a TCP connection from the end-user client to a plurality of beacon servers, wherein each beacon server of the plurality of beacon servers corresponds to one of the plurality of data centers.

27. The apparatus of claim 26, wherein the TCP connection is a result of providing web beacon code to each of the end-user clients.

28. The apparatus of claim 25, wherein the aggregated connection quality measurement data is organized into a plurality of groups, wherein for each group, the IP address of the end-user client falls within the IP address range of the same netblock.

29. An apparatus for measuring connection quality from a plurality of data centers to a client, comprising:

means for receiving requests from a plurality of end-user clients;

means for storing the generated connection quality measurements for each end-user client based on the request;

means for determining and grouping the end user clients into net blocks based on the IP addresses of the end user clients;

means for outputting aggregated connection quality measurement data to a plurality of load balancing servers for use by the plurality of load balancing servers in determining how to route messages between the end user client and the data center;

wherein the aggregated connection quality measurement data is generated for each of the net blocks by aggregating connection quality measurements for those end user clients belonging to that net block.

30. The apparatus of claim 29, wherein the measurement of connection quality is based on a TCP connection from the end-user client to a plurality of beacon servers, wherein each beacon server of the plurality of beacon servers corresponds to one of the plurality of data centers.

31. The apparatus of claim 30, wherein the TCP connection is a result of providing web beacon code to each of the plurality of end-user clients.

32. The apparatus of claim 29, wherein the aggregated connection quality measurement data is organized into a plurality of groups, wherein for each group, the IP address of the end-user client falls within the IP address range of the same netblock.