ES2371404T3 - SYSTEM AND PROCEDURE TO ESTIMATE THE PREVALENCE OF DIGITAL CONTENT ON THE WORLD-WIDE-WEB. - Google Patents

Abstract

A procedure for estimating the prevalence of digital content in a network, the procedure comprising: receiving (710) an estimate of a number of times a web page has been accessed (110; 110 '; 110' '; 411-416; 421-424); repeatedly request (740) the website (110; 110 '; 110' '; 411-416; 421-424) and, in response, receive content files; determining (750) a number of times that a first content object (A1; A2; A3) is included in the content files; determining (770) a total number of times that the web page (110; 110 '; 110' '; 411-416; 421-424) has been requested; and estimating (773, 774) the number of times that the first content object (A1; A2; A3) has been presented to website visitors based on the number of times the first content object (A1; A2 ; A3) was included in the content files, the total number of times the website (110; 110 '; 110' '; 411-416; 421-424) was requested, and the estimate of the number of times it was You have accessed the website (110; 110 '; 110' '; 411-416; 421-424).

Description

System and procedure to estimate the prevalence of digital content on the World-Wide-Web

REFERENCE TO RELATED APPLICATIONS

The present application claims priority over, and incorporates by reference, the provisional invention patent application number 60 / 175,665, filed at the United States Patent and Trademark Office on January 12, 2000, and the provisional application for invention patent number 60 / 231,195, filed at the United States Patent and Trademark Office on September 7, 2000.

FIELD OF THE INVENTION

The present invention relates in general to a computer program system, method and product for tracking and measuring the digital content that is distributed over a computer network such as the Internet. More particularly, the present invention relates to a computer program system, method and product that collects data from online advertisements, analyzes the data, and uses the data to calculate measurements of the prevalence of those advertisements.

BACKGROUND OF THE INVENTION

The increase in popularity of the Internet and the World-Wide-Web ("Web") is due, in part, to the interactive technologies that a web page can use. These interactive technologies directly affect the Web as an advertising medium because the technologies introduced new advertising formats such as sponsorship of fixed icons, banners and rotating buttons, and interstitial ads (that is, online ads that interrupt the user's work and take over of a significant percentage of the screen display). Even when the creation of the ad is different, the effect on the viewer is similar to traditional advertising. For example, a banner ad or logo icon on a web page creates an impression of the product for the viewer that is equivalent to a traditional billboard advertisement that promotes a product by presenting the brand or slogan. Likewise, a sponsor logo on a web page creates a sponsor impression for the viewer that is equivalent to seeing a sponsor logo on the scoreboard in a college basketball game.

The rapid and unpredictable growth of the Internet over the past several years has created a high demand for quality statistics that quantify its magnitude and expansion rate. Several traditional measurement methodologies produce useful statistics about the Internet and its users, but the complexity of the Internet has left some of these methodologies unable to answer many important questions.

International application WO98 / 59309 discloses a cooperative electronic media measurement system that uses media handlers to obtain information about, or not to obtain information about, presented media objects, including identification tags, if any, for agent collection of gathering research data and subsequent dispatch to a centralized media research controller. The media research controller registers advertisements and other media for subsequent measurement and provides a unique identification tag that can be added to, or associated with, the existing media object. Media objects are presented to a panel member by means of a panel member computing device that can receive media objects through a network connection, from one or more local sources, or generate media objects in real time, or a combination thereof. One is assigned

or more investigative data collection agents to measure the exposure to, and interactions with, electronic media of each panel member. Cooperative media handlers automatically obtain information from, or if they do not obtain information about, media objects presented including identification tags, if any, and other information, for collection by research data collection agents. A research data collection agent will track a panel element, and collect such transmissions from cooperative media handlers, when a panel member is within the scope of the research data collection agent.

U.S. Pat. 5,995,943 discloses a system and procedure for aggregation and synthesis of information that provides aggregation and encapsulation of structured and unstructured information from disparate sources such as those available on a network such as the Internet. A compatible / addressable network interface device is operated by a user. The network interface device communicates with local data stores or data stores accessible by network through an addressing scheme such as the Uniform Resource Locator (URLs) addresses used by the Internet. The data that passes between the network interface device and the data stores is accessed and scanned and retrieved through an intermediary gateway system. Such aggregate information is then synthesized, adapted, customized and located to meet the requests for information resources specified by the user through the network interface device.

Online advertising is an area where traditional methodologies do not lend themselves well to measurement. Every day, thousands and thousands of electronic advertisements appear and then disappear from millions of web pages. The transitory nature of online advertising activity justifies a novel methodology to accurately measure advertising activity.

Existing advertising tracking and measurement systems automate the collection of web pages, but do not automate the collection of online advertisements. Since the content of an online ad changes or rotates over time, the precise reconstruction of the frequency of specific ads requires continuous sampling of relevant web pages in the correct proportions. In addition, due to the mere magnitude of the Web, sampling algorithms must be fine tuned to optimize resource allocation (i.e. network bandwidth, database storage, processor time, etc.) and simultaneously allow maximum Internet coverage. Existing advertising tracking and measurement systems do not meet these needs because they are not optimized for resource allocation and do not continually sample relevant web pages in the correct proportion.

In view of the deficiencies of the existing systems described above, there is a need for an advertising tracking and measurement system that uses resources more intelligently, is more compatible with the websites you visit, is scalable, and produces accurate measurements. The invention disclosed in this document addresses this need.

SUMMARY OF THE INVENTION

The present invention is a system, method, and software product for tracking and measuring the digital content that is distributed over a computer network such as the Internet as defined in the appended claims. The system collects data from online advertisements, analyzes the data, and uses the data to calculate measurements of the prevalence of those advertisements.

In the preferred embodiment, traffic data from a variety of complementary sources and methodologies feed the traffic analysis system, an intelligent agent (ie, software that interacts with, learns from, and adapts to an environment). The traffic analysis system processes the unprocessed traffic data by debugging and summarizing the traffic data before storing the processed data in a database. When the statistical summary system calculates the frequency of advertisements, impressions, and expenses, it is based on the processed data from the traffic analysis system.

The ad sampling system, also known as the "poller" or "cloud poller", uses a robust methodology that continually searches for the most significant and influential websites to be probed (that is, monitored). On the other hand, the selection and definition of the site made by the present invention dictates the web pages that comprise each web site to ensure that complete singular brand entities are reported as such. The ad sampling system uses intelligent agent technology to retrieve web pages at various frequencies to obtain a representative sample. This allows the "cloud poller" to assess exactly how often each ad appears in the traffic data. After the "cloud poller" searches for a web page, the ad sampling system extracts the ads from the web page. In the preferred embodiment, the ad extractor, also known as the "extractor", invokes an automatic ad detection process ("ADF"), a heuristic extraction process, to automatically extract all the ads from the web page.

After extracting the advertisements from the website, the ad sampling system invokes a classification engine to analyze the ad fragments. The classifier processes each fragment to determine a classification for the fragment and then stores the fragment and classification data in a database. The result of the analysis and processing performed by the ad sampling system is an abundant catalog of advertising activity that can be easily consulted by a customer.

The present invention uses a web client part and a user interface to access and update the data in the database. The web client part offers a client, or user, of the present invention a query interface to the database populated by traffic analysis, ad sampling, and statistical summary systems. The user interface is a graphical user interface that includes a separate component for system account management, site administration, taxonomy administration, advertising content classification, and collection of rate lists. The user interface allows an administrator and account operator to maintain and administer the present invention. The user interface also allows a media editor to review the data in the database to verify the accuracy and completeness of the vast amount of data collected by the present invention. This data integrity procedure routinely investigates unusual or peripheral data points to calibrate the system and adapt it to an ever-changing environment.

BRIEF DESCRIPTION OF THE DRAWINGS

The attached figures better illustrate the details of the present invention, both in terms of structure and operation. The same reference numbers and designations in these figures refer to equal elements.

Figure 1 is a network diagram representing the environment for an advertising prevalence system according to the present invention Figure 2 represents the network diagram of Figure 1, in greater detail, to show the relationships between the network environment and the elements that comprise the advertising prevalence system. Figure 3 represents the network diagram of Figure 2, in greater detail, to show the elements and Sub-elements that comprise the advertising prevalence system and the connections to the network environment. Figure 4A is an example website that illustrates the expected values used in the calculation of Advertising prevalence statistics. Figure 4B is an example website that illustrates the observed values used in the calculation of the Advertising prevalence statistics. Figure 4C is an example website that illustrates the weighted values used in the calculation of Advertising prevalence statistics. Figure 4D is an example website that illustrates an alternative procedure for calculating Advertising prevalence statistics. Figure 5 illustrates an example of a database structure that the prevalence system can use advertising Figure 6 is a functional block diagram of the advertising prevalence system showing the configuration of hardware and software components. Figure 7A is an organization chart of a process in the advertising prevalence system that measures the quality of the Online advertising and activity generated by an online ad. Figure 7B is an organization chart that describes, in greater detail, the process of sampling traffic data from of Figure 7A. Figure 7C is an organization chart that describes, in greater detail, the process of generating a survey map based on traffic data sampled from Figure 7A. Figure 7D is an organization chart that describes, in greater detail, the process of polling the Internet to gather data of samples from Figure 7A. Figure 7E is an organization chart that describes, in greater detail, the process of classifying advertising data to from Figure 7A. Figure 7F is an organization chart that describes, in greater detail, the process of calculating statistics advertising from Figure 7A.

DETAILED DESCRIPTION OF THE INVENTION

Figure 1 depicts the environment for the preferred embodiment of the present invention that includes the Internet 100, and a website 110, the traffic sampling system 120, the advertising prevalence system 130, and the client 140. The present invention uses Smart agent technology to gather data related to attributes, placement, and prevalence of online ads. This data provides a user with updated estimates of ad statistics and helps the user gain a competitive advantage.

As shown in Figure 1, the Internet 100 is a public communication network that allows the traffic sampling system 120 and the advertising prevalence system 130 to communicate with a client 140 and a website

110. Although the preferred embodiment uses Internet 100, the present invention contemplates the use of other public or private network architectures such as an intranet or extranet. An intranet is a private communication network that works similarly to the Internet 100. An organization, such as a corporation, creates an intranet to provide a secure means for members of the organization to access resources through the organization's network . An extranet is also a private communication network that works similar to Internet 100. Unlike an intranet, an extranet provides a secure means for the organization to authorize non-members of the organization to access certain resources through the Internet. Organization network. The present invention also contemplates the use of a network protocol such as Ethernet or Token Ring, as well as proprietary network protocols.

The traffic sampling system 120 is a program that monitors and records web activity on the Internet 100. The traffic sampling system 120 is an intermediate repository of traffic data between an Internet user (not shown) on the Internet 100 and a web server 112. Web server 112 shown in Figure 1 is a personal computer or conventional computer workstation that includes the appropriate operating system, hardware, communications protocol (for example, the Transmission Control Protocol / Internet Protocol), and web server software to host a collection of web pages. The Internet user (not shown) communicates with the web server 112 requesting a uniform resource locator ("URL") 114, 116, 118 associated with the website 110, typically using a web browser. Any program or device that can register a URL request made by a user (not shown) to a web server 112 can perform the functions that the present invention requires of the traffic sampling system 120. The traffic sampling system 120 adds then the traffic data for each website 110 for use by the advertising prevalence system 130.

The present invention can use any commercially available traffic sampling system that provides similar functionality to the Media Metrix audience measurement product. Other possible mechanisms to obtain a sample of traffic data include:

one.

 "Cache proxy sampling" gathers data such as user click sequence data, and web page requests from a global distributed hierarchy of cache proxy servers. This data passes through an intermediate mechanism that provides pre-search and cache services for web objects. As of May 1999, traffic statistics calculated by the present invention represent the synthesis of unprocessed data from nine first level caches and approximately 400 second level caches in the United States, as well as an additional 1100 worldwide .

2.

 “Client side panel collection” retrieves data from samples from each panelist through a client side mechanism and transfers the data to a collection repository. The client-side mechanism can monitor the browser's address bar, the user's browser, a client-side proxy, or TCP / IP battery hook functions.

3.

 A "transcoder" is a proxy that rewrites HTML, usually in order to add elements for ad revenue generation or page headers / footers. Free internet service providers (“ISPs”) typically use this technique.

Four.

 Any mechanism for distributing content that duplicates the website or site content in a way designed to reduce network congestion or improve the user experience.

5.

 Any content filtering mechanism that evaluates URL requests and takes actions to allow or deny such requests.

6.

 From server logs maintained by Internet service providers ("ISPs") or individual websites.

Figure 2 expands the detail of the advertising prevalence system 130 of Figure 1 to show the relationships between the network environment and the elements comprising the advertising prevalence system 130. The advertising prevalence system 130 includes a traffic analysis system 210, an ad sampling system 220, and a statistical summary system 230 that communicates data to the database 200 for storage. Account manager 260, operator 262, and media editor 264 can access database 200 through user interface 240 to perform administrative functions. The client 140 can access the database 200 through the web client part 250.

The traffic analysis system 210 receives unprocessed traffic data from the traffic sampling system 120. The traffic analysis system 210 purifies the unprocessed traffic data by removing the information from the traffic data that a user can identify particular on the Internet 100 and then stores the anonymous data in the database 200. The traffic analysis system 210 estimates the overall traffic to each significant website on the Internet 100. The present invention uses this data not only to calculate the number of advertising impressions given an estimate of the rotation rate on that page, but also in the polling map creation system 320. In one embodiment, the traffic analysis system 210 receives traffic data from a cache site in the Internet 100. The objective is to accurately measure the number of page views by individual users, and therefore the number of printing advertising ones.

The ad sampling system 220 uses the anonymous traffic data to determine which URLs to include in the sample retrieved from the web server 112. The ad sampling system 220 contacts the web server 112 through the Internet 100 to retrieve a URL 114, 116, 118 and extract the ads from it along with the accompanying features that describe the ads. The success rate for creative recovery is high. The analysis indicates that the present invention captures more than 95% of the creatives being served. The ad sampling system 220 stores these ad features in the database 200. The ad sampling system 220, for example, the Online Media Network Intelligent Agent Collection ("OMNIAC"), or the "cloud poller" ”, Repeatedly polls featured websites, extracts the ads from each web page returned by the probe, and classifies the ads on each web page by type, technology and advertiser.

The traffic analysis system 210 and the ad sampling system 220 also present the data retrieved from the Internet 100 to the statistical summary system 230 for periodic processing. The statistical summary system 230 calculates the frequency of advertisements, impressions, and expenses per site and per week.

The graphical user interface for the present invention includes the user interface 240 and the web client part 250. The account manager 260, the operator 262, and the media editor 264 access the user interface 240 to manage access by client part 140 web client part 250 (for example, account and password management), define polling sites and instructions, and manage advertising taxonomy, content classification, and compilation of tariff lists for the prevalence system advertising 130. The web client part 250 is the web browser interface that a client 140 uses to retrieve the measurement results of advertisements from the database 200 as generated by the traffic analysis system 210, the sampling system of ads 220, and the statistical summary system 230.

Figure 3 further expands the detail of the advertising prevalence system 130 to represent the logical components comprising the elements of the advertising prevalence system 130 shown in Figure 2. Figure 3 also represents the relationships between the network environment and those logical components .

The traffic analysis system 210 includes an anonymity system process 310 and traffic summary 312.

The anonymity system 310 purifies the data received from the traffic sampling system 120 by deleting the information that identifies a particular user on the Internet. The data is made anonymous by passing all user information (for example, the internet protocol number ("IP") of origin or cookies) through a cryptographically secure unidirectional hash function; This ensures greater privacy for web users without devaluing the resulting data. The anonymity system 310 presents the purified data to the traffic summary system 312 which in turn stores the URL count information added in the database 200.

The traffic summary process 312 receives the purified data from the anonymity system 310. The anonymous traffic data is summarized to produce traffic totals per week or month for individual URLs, domains and websites. The traffic summary process 312 scales the data by weighting factors to extrapolate the total global traffic from the sample.

The ad sampling system 220 of Figure 3 includes a survey map creation system 320, a web page retrieval system 322, a web browser emulation environment 324, an ad extractor 326, and a structural classifier 328

The survey map creation system 320 generates a survey map, that is, the URLs 114, 116, 118 that the ad sampling system 220 will visit. This survey map helps the ad sampling system 220 with the measurement of ad rotation on individual websites. The preferred embodiment of the present invention continuously searches for various web pages in the survey map. In an alternative embodiment, the present invention visits each URL of the survey map approximately every 6 minutes. Another embodiment may vary the search frequency considering several factors including the amount of traffic that the website visits as a whole and the individual web page in question, the number of ads historically viewed on the website, and the similarity of the rotation of Ads historically observed to other sampled pages.

The web page recovery system 322 uses the polling map generated by the polling map creation system 320 to determine which web pages to sample and the frequency of sampling. For each URL of the survey map generated by the survey map creation system 320, the web page retrieval system 322 searches for a web page, extracts each advertisement from the web page, and stores the attributes of the advertisement in the database base. 200 data. The data retrieved from each URL of the survey map is used to calculate the frequency with which each ad is displayed on a particular website.

For each web page, the 324 web browser emulation environment simulates the display of the web page in a browser. This simulation ensures that the present invention will detect not only static advertisements, but also dynamic advertisements generated by software programs written in a language such as JavaScript, Perl, Java, C, C ++, or HTML that may be embedded in a web page.

The ad extractor 326 extracts the online ads from the simulation result performed by the web browser emulation environment 324. Ad extractor 326 identifies characteristics of the advertising content (ie "fragments") extracted from the returned web pages for the 320 mapping map creation system that are of particular interest. Ads are the most interesting dynamic feature of extracting, however, an alternative embodiment of the present invention can use extraction technology to collect any type of digital content including promotions, surveys, and news articles. Ad extractor 326 can use various ad extraction procedures, including rule-based extraction, heuristic extraction, and comparison extraction.

Rule-based extraction relies on a media editor 264 to use user interface 240 to create rules. The user interface 240 stores the rules in the database 200 and the ad extractor 326 applies the rules to each web page that retrieves the web page recovery system 322. The effect of executing a rule is to identify and extract a fragment HTML of the web page (that is, the part of the page that contains the ad). Ad extractor 326 first converts the HTML representation of the searched web page into a perfectly formed XML representation. After this conversion, the rules are applied to the syntax analysis tree of the XML representation of the web page.

Heuristic extraction is based on the similarity of the ads at the HTML or XML source code level because the ads are typically inserted by an ad server when the web page is generated in response to the request of the web browser emulation environment 324 of View the web page. Heuristic extraction analyzes clues from the source code (for example, references to the names of known ad servers) and extracts the fragments surrounding those clues. The advantage of this procedure is that the extraction is automatic and the media editor does not have to create the rules.

Extraction by comparison repeatedly searches the same web page. This extraction procedure compares the different versions of the website to determine if the content varies from version to version. The portion of the website that varies with a certain degree of frequency is usually an advertisement and is extracted.

The structural classifier 328 parses each advertisement syntactically and stores the structural components in the database 200 and passes those components to the statistical summary system 230. Each ad fragment extracted by the ad extractor 326 is analyzed by the structural classifier 328. The process performed by structural classifier 328 includes the removal of duplicate fragments, the analysis of structural fragments, and the detection of duplicate advertisements.

Structural classifier 328 performs the removal of duplicate fragments compared to the current ad fragment with other fragments of the database 200. Two ad fragments are duplicated if the fragments are identical (for example, each fragment has exactly the same HTML content) . If structural classifier 328 determines that the current fragment is a duplicate of a fragment of the database, the ad sampling system 220 records another observation of the fragment and continues processing fragments.

The structural classifier 328 performs analysis of structural fragments on the XML representation of the web page by determining the "physical type" of the fragment (that is, the HTML source code used to build the ad). The physical types that the present invention recognizes include banner, form, individual link, and embedded content. Banner ad fragments include an individual HTML link that has one or two images attached and no FORM or IFRAME tag. Form ad fragments include an individual HTML form that does not have an IFRAME tag. Individual link ad fragments include a textual tag link, but not IMG, FORM or IFRAME tags. Ad fragments of embedded content refer to an external entity using an IFRAME tag. After performing this analysis, structural classifier 328 updates the ad fragment of the database. For a banner ad fragment, structural classifier 328 stores the link and image URLs in database 200. A form ad fragment requires the creation of a URL simulating a user presentation that puts each HTML control in its Default value. Structural classifier 328 stores this URL and the "form signature" (that is, a string that uniquely describes the content of all form controls) in database 200. For an individual text ad fragment, the classifier Structural 328 stores the URL for the link and all the text contained within the link in database 200. For embedded content ad fragments, structural classifier 328 stores the URL associated with the external reference in database 200. This URL is loaded by the system, and the document referred to is loaded. Once the loaded document has been structurally analyzed, the original fragment inherits any attribute that results from the analysis of the new fragment.

Structural classifier 328 detects duplicate advertisements on each ad fragment that has a known physical type because these fragments represent advertisements. Each unique advertisement has information, including what site definitions are associated with the fragment, stored in database 200. The determination of uniqueness of structural classifier 328 depends on different criteria for each type of fragment. The first stage for each type of definition is to resolve all the URLs associated with the registry. URLs that reference images are loaded, and duplicate images are noted. HTML link URLs, also known as "click URLs," are followed each time a new ad is created. The final destination for a click-through URL is noted, after following all HTTP redirects. This is also done for simulated link submission URLs associated with form definitions. Once all URLs have been resolved, structural classifier 328 determines if the ad is unique. Banner ad fragments are considered unique if they have the same number of images, if the images are identical, and if the destination URL is identical. Form ad fragments are considered unique if they have the same signature, and the same destination URL. Individual link ad fragments are considered unique if they have the same text content and the same destination URL.

The statistical summary system 230 calculates the ad statistics for each unique advertisement in the database 200. The present invention calculates, for each website, the advertising impressions (ie, the number of times a human being sees an advertisement ). The present invention calculates the advertising impressions, I, using the formula I = T x R, where T is the traffic going to the site, and R is the rotation of advertisements on that site. The present invention also calculates the expenses S, using the formula S = I x RC, where I are the advertising impressions for a website, and RC is the rate code for the website. Most advertising purchases are complicated agreements with discounts for purchase volume, so our numbers do not necessarily represent the actual cost of the total purchase.

The web client part 250 is a graphical user interface that provides a client 140 with a query interface to the database 200 populated by the traffic analysis system 210, the ad sampling system 220, and the summary system 230. The client 140 can use the web client part 250 to create, store, edit and download graphical and tabular reports for one or more industrial categories depending on the level of service that the client 140 selects.

The user interface 240 of Figure 3 includes a separate component for system account management 340, site administration 342, taxonomy administration 344, advertising content classification 346, and compilation of tariff lists 348.

The account manager 260 uses the account management module of the system 340 of the user interface 240 to simplify the administration of the web client part 250. The account manager 260 uses the account management module of the system 340 to create and delete user accounts, manage user account passwords, and check the overall health of the 250 client web part.

The operator 262 uses the site administration module 342 of the user interface 240 to simplify the administration of the site definitions. Internet Advertising Bureau analysts estimate that more than 90% of all web advertising dollars are spent on the first fifty websites. Site selection begins by choosing the first 100 ads considering data from Media Metrix, Nielsen / Net Ratings, and proxy traffic data from database 200. These lists are updated periodically to downgrade websites with low traffic levels and promote new sites with high levels of traffic. The present invention also includes websites that provide meaningful content in key industries. A site chosen for inclusion in the site definitions must have the structure of the site analyzed to remove sections that do not serve ads, come from foreign countries, or are part of a set of frameworks. Sites from a foreign country, such as yahoo.co.jp, sell advertising in the host country, and therefore are not applicable to the measurements calculated by the present invention. Websites that use a set of HTML frames are treated very carefully to only apply rotation frequencies to traffic from sections of the frameset that contain the ad. These combined exclusions are key to making accurate estimates of advertising impressions. The present invention also labels sections that cannot be measured directly, due to registration requirements (eg, mail pages). As websites change the frequency, this structural analysis is repeated periodically. Finally, the analysis phase will automatically mark the altered sites to allow even more timely updates.

Media editor 264 uses taxonomy administration modules 344, advertising content classification 346, and collection of tariff lists 348 from user interface 240. Taxonomy administration module 344 simplifies the creation and maintenance of attributes assigned to ads during content classification including industry, company and ad products. The taxonomy names each attribute and specifies its type, ancestry and membership segment. For example, a Honda company could be related to the Automotive industry and belong to the industrial segment Manufacturers of automobiles. The advertising content classification component 346 helps the media editor 264 to perform the content classification.

The structural classifier 328 performs the automated allocation of advertising products to determine which advertisement is being published. This process includes assigning "advertising products" (that is, attributes that describe each "thing" the ad is advertising) to each ad fragment. In another embodiment of the present invention, the ad sampling system 220 uses an expandable set of heuristics to assign advertising products to each advertisement. In the preferred embodiment, however, the only automatic procedure employed is the address classification. The address classification is based on the destination URL to assign a set of advertising products to an ad. A media editor 264 uses user interface 240 to maintain the set of classified addresses. For example, the first time a media editor observes an ad in which the click-through URL is www.honda.com, you can enter this URL as belonging to the advertiser “Honda Motors”. Any subsequent ad that includes the same click-through URL will also be recognized as a Honda ad. A classified address comprises a host, a URL route prefix, and a set of advertising products. The address classification assigns a classified address advertising product to an ad if the host in the destination URL matches the host of the classified address and the route prefix in the classified address matches the start of the route in the URL of the destination.

Structural classifier 328 performs the allocation and human verification of advertising products as a quality check of advertising product data. This phase is the one that requires more human work.

A media editor 264 uses a graphical user interface module in the user interface 240 to display each advertisement, verify automatic assignments of advertising products, and assign any other advertising product that seems appropriate after the inspection of the advertisement and the destination of the ad. The address classification database is also typically maintained at this time.

The media editor 264 uses the rate list collection module 348 to enter the contact information and rate list for a website identified by the traffic analysis system 210, as well as the designated advertisers. The introduction of the tariff list includes the applicable quarter (for example, Q4 2000), the dimensions of the ad in pixels, the structure of rates (for example, CPM, flat rate, or per click), the scale of costs for purchases of various quantities and duration. Media editor 264 also records the URL of the online media kit and if rates are published therein. Contact information for a website or advertiser includes the home page, name, telephone and fax numbers, email address, and physical address.

Figures 4A to 4C illustrate the preferred procedure for calculating advertising prevalence statistics. The calculation of advertising prevalence statistics is an iterative process that uses expected values deduced by traffic analysis system 210 and observed values deduced by advertising prevalence system 220 to calculate weighted values and advertising prevalence statistics. Each of Figures 4A to 4C represents a network on the Internet 100 that includes two websites served by the web server P 410 and the web server Q 420. Figure 4A illustrates example expected traffic values for the network. Figure 4B illustrates example observed traffic values for the network. Figure 4C illustrates sample weighted traffic values for the network.

The first stage in the process is to normalize the results from the traffic analysis system 210. The traffic analysis system 210 provides the traffic received by each web page in the traffic data sample. Figure 4A represents the sample traffic received on each web page 411-416, 421-424 on Internet 100 with the tag "Traffic =". The survey map generated by the survey map creation system 320 includes an entry for each web page 411-416, 421-424. The survey map also includes an “area” for each web page 411-416, 421-424 consumed on the survey map labeled “Area =”. Normalized results are calculated by dividing the area that a web page consumes on the survey map by the sum of the area for each web page in the traffic sample. In Figure 4A, the normalized value, or probability, for web page P1 411 is the area for web page P1 (i.e. 15) divided by the sum of the area for web page P1, P2, P3, P4, P5, P6, Q1, Q2, Q3 and Q4 (i.e. 120). The normalized value is, therefore, 0.125, or 12.5%. In addition to the normalized value, the system also determines the scale by dividing the traffic for a web page by the area for the web page. In Figure 4A, the scale for the P1 web page 411 is the traffic for the P1 web page (i.e. 150) divided by the area for the P1 web page (i.e. 15), therefore the scale for the P1 web page is 10. Table 1 summarizes the scale and probability values for the remaining web page in Figure 4A.

Table 1

Web page

Area Scale Probability

P1

fifteen 10 12.5%

P2

10 one 8.3%

P3

14 one 12%

P4

12 0.25 10%

P5

8 0.5 6.7%

P6

4 one 3.3%

Q1

30 0.5 25%

Q2

4 0.5 3.3%

Q3

fifteen 2 12.5%

Q4

8 0.5 6.7%

Figure 4B represents the example web page searches on each web page 411-416, 421-424 on the Internet 100 with the tag "Searches =". Figure 4B also represents the number of sample views of each ad on a web page 411-416, 421-424 with a tag such as "A1 Views =" to indicate the number of views of the A1 ad, "A2 Views = ”To indicate the number of views of ad A2, etc.

Figure 4C represents the weighted searches of sample web pages on each web page 411-416, 421-424 on Internet 100 with the tag "Searches =". Figure 4C also represents the number of sample views of each ad on a web page 411-416, 421-424 with a label such as "A1 Views =" to indicate the number of views of the A1 ad, "A2 Views = ”To indicate the number of views of ad A2, etc. The next stage in the calculation process is to calculate the scaled searches for each website 410, 420 by adding the product of the observed searches of Figure 4B and the scale of Figure 4A, for each website 411416, 421-424 in the website. Next, the calculation calculates the traffic for each website 410, 420 by adding the traffic of Figure 4A for each website 411-416, 421-424 on the website. The rate list, or CPM, is a value

assigned by media editor 264 for each website 410, 420. Table 2 summarizes the scaled searches, traffic, and CPM for Figures 4A to 4C.

Table 2

Site

Scale searches Traffic CPM

P

193.5 185 $ 35.00

Q

43 51 $ 50.00

The next thing in the calculation process is to calculate the Scale Observations for each ad on each website 410, 420 by adding the product of the ad views of Figure 4B and the scale of Figure 4A, for each website 411-416 , 421-424 on website 410, 420. The final stage in the calculation is to calculate advertising prevalence statistics (ie, frequency, impressions, and expenses) for each ad on each website 410, 420. The frequency is Calculated by dividing the observations at scale by the scaled searches for each advertisement on each website 410, 420. Impressions are calculated by multiplying the frequency by the traffic in Table 2 above for each advertisement on each website 410, 420. Expenses They are calculated by multiplying the impressions by the CPM of Table 2 above for each ad on website 410, 420. Table 3 summarizes the Scale Observations, Frequency, Impressions, and Expenses for the Q 420 website using the data of Figures 4A to 4C.

Table 3

Observations at scale

Frequency Impressions Expenses

A1

55.0 0.28 52.58 $ 1.84

A2

85.0 0.44 81.27 $ 2.84

A3

6.0 0.03 5.74 $ 0.20

A4

3.5 0.02 3.35 $ 0.12

TO 5

Table 4

Observations at scale

Frequency Impressions Expenses

A1

29.5 0.69 34.99 $ 1.75

A2

12.0 0.28 14.23 $ 0.71

A3

12.0 0.28 14.23 $ 0.71

A4

12.0 0.28 14.23 $ 0.71

TO 5

1.5 0.03 1.78 $ 0.09

Figure 4D illustrates an alternative embodiment for calculating advertising prevalence statistics. In this embodiment, the probe is tuned to optimize the accuracy of rotation measurement. Statistical estimates of accuracy in the field are difficult to make, due to the non-stationary nature of the advertising servers. When probed every 6 minutes, it has a resolution of 0.06% in rotation rate over a one-week measurement period.

Also in the alternative embodiment of Figure 4D, the probes are distributed between the sites to accurately measure the rotation of advertisements at each site. The number of poll URLs assigned to a site is determined from three variables. The first is a constant across all sites; a certain number of polling URLs are required to accurately measure rotation even at the smallest site. Half of the probes are assigned with this variable. The second variable, weighted to 40%, is the amount of traffic going to a site, since each polling URL represents a proportion of the total Internet traffic. The twenty largest sites receive more than 75% of these probes. Finally, the complexity of the site is taken into account, as measured by the total number of unique URLs found in our proxy traffic data, with the most complicated sites receiving extra polling URLs. This represents the remaining 10% of the probe distribution. Polling URLs can be chosen using a site shredder algorithm to break the site into zones (that is, sets of pages whose ad rotation characteristics are likely to be similar) for polling. Zone distribution is mathematically designed to maximize site coverage and, therefore, the accuracy of ad rotation. A single URL is chosen to represent the advertising rotation of each zone. This URL is chosen as the page with the heaviest traffic that contains ads in that area. The algorithm avoids pages of specific dates or pages that refer to a time-limited event such as the total lunar eclipse of August 1999.

The alternative embodiment of Figure 4D calculates ad impressions by combining rotation and traffic estimates for each 430 website. To do this, the system breaks down the site into its constituent roots using the site shredder algorithm. Ad rotation in each ad space is calculated and applied to estimate advertising impressions at its associated root. Root ad rotation without probes is estimated from a traffic-weighted average of the ad rotation of probes at a similar level.

For example, in Figure 4D, the sample site tree has five polling URLs 431-435, P1-5, placed in five main branches of a main page and 14 secondary branches. The number of each page is the sample traffic that goes to that page. The P1 probe on the homepage, “www.testsite.com”, measures the rotation, R, that must be applied to the traffic going to that main page, with traffic of 88 page views. Branch A has a single probe, P2, placed on the top level page of that branch with a poll URL “www.testsite.com/A/”. The rotation of this single poll URL is estimated as RA and is applied to traffic for that entire root, a total of 21 page views. Branch C has a probe, P3, on a secondary branch page with heavy traffic, with a poll URL “www.testsite.com/C/third.html”. The rotation, RC, of this page applies to all secondary branch pages in that root and also one level up in the tree, for a total of 25 page views. Branch E receives a large part of the traffic for the site, a total of 61 page views, and therefore two probes, P4 and P5, are assigned. These are on two secondary branch pages, "www.testsite.com/E/first.html" and "www.testsite.com/E/third.html." The traffic to each individual page is applied to the rotation of each one. For the remaining 18 page views in that branch (ten page views of two secondary pages and eight of the top level page of that branch) a weighted rotation is calculated, RE = ((13 x RE1) + (30 x RE3 )) / (13 + 30). Root rotation analysis results in advertising impressions for more than 96% of the site. Impressions for the two final branches, B and D, are calculated with an average rotation from adjacent branches, weighted by traffic,

RB = RD = ((21 x RA) + (25 x RC) + (61 x RE)) ÷ (21 + 25 + 61).

This analysis results in total impressions across the site for each unique ad. The final calculation made by the alternative embodiment of Figure 4D is the expenses, the product of the prints and the list of rates.

Figure 5 illustrates a database structure that the advertising prevalence system 130 can use to store information retrieved by the traffic sampling system 120 and the web page recovery system 322. The preferred embodiment segments the database 200 in partitions. Each partition can perform functions similar to a separate database such as database 200. In addition, a partitioned database simplifies the administration of partition data. Although the preferred embodiment uses database partitions, the present invention contemplates the consolidation of these partitions into a single database, as well as making each partition a separate database and distributing each database to a computer workstation. or separate general purpose server. The partitions for the database 200 of the present invention include sampling records 510, polling definitions 520, advertising support data 530, and advertising summary 540. The preferred embodiment of the present invention uses a database management system. relational, such as the Oracle8i product of Oracle Corporation, to create and manage the database and partitions. Although the preferred embodiment uses a relational database, the present invention contemplates the use of other database architectures as an object-oriented database management system.

The partition of sampling records 510 of database 200 comprises database tables that are logically segmented into areas of traffic data 512, advertisement view register 514, and advertisement structure 516.

The traffic data area 512 contains data processed by the traffic sampling system 120, the anonymity system 310, and the statistical summary system 230. The data stored in this scheme includes an "altered" URL, and the count of traffic that each URL receives per traffic source over a period of time. An "altered" URL is an ordinary URL with the protocol field removed and the order of the components with points in the inverted device name. For example, the present invention transforms an ordinary URL, such as http://www.somesite.com/food, into an altered URL by eliminating the protocol field (ie, "http: /") and reversing the order of the components with points in the team name (ie www.somesite.com). The altered URL resulting in this example is "com.somesite.www / food". The present invention uses this proprietary URL format to greatly improve the process of traffic data analysis. The traffic sampling system 120 populates the traffic data area 512 of the database 200. The polling map creation system 320 accesses the data from the traffic data area 512 to assist the page retrieval system web 322 and the statistical summary system 230 with the calculation of advertising printing and expense statistics.

The ad view registration area 514 records the time, URL and ad identifier for each ad found on the Internet 100. This area also records each time the system does not detect an ad on a web page that previously included the ad. In addition, the system records each time the system detects a potential ad, but does not recognize the ad during structural classification. The structural classifier 328 and the web page retrieval system 322 of the ad sampling system 222 populates the ad view registration area 514 of the database 200. The statistical summary system 230 accesses the data in the area of Ad view record 514 to determine how often each ad appears on each site.

Ad structure area 516 contains data that characterize each unique ad located by the system. This data includes the content of the ad, the type of ad (for example, image, HTML form, Flash, etc.), the destination URL linked to the ad, and various elements used during content classification and diagnosis, including where the ad was first seen, and what ad definition the ad originally produced. The structural classifier component 328 of the ad sampling system 220 populates the ad structure area 516 of the database 200. The user interface 240 accesses the data of the ad structure area 516 to present each advertisement to the publisher of 264 media during classification editing. The web client part 250 also accesses the data of the ad structure area 516 to present the announcements to the client 140.

The polling definition partition 520 of the database 200 comprises database tables that are logically segmented into site definition areas 522, polling map 524, and definition of ad extraction rules 526.

Site definition area 522 zones the part of Internet 100 that the system polls into zones. The definition of primary zone is a “site”, a coherent entity that the system has to analyze, sample and summarize. The system defines each site in terms of altered URL prefixes, both inclusive and exclusive. An "altered URL prefix" is an altered URL that represents the zone of all the altered URLs for which it is a prefix. An "inclusive altered URL prefix" specifies that a URL is part of some entity. An “exclusive altered URL prefix” specifies that a URL is not part of any entity, canceling the parts of the entity included by an inclusive prefix. As an illustration, what follows is the list of altered URLs that may result from the processing of a set of URLs in a traffic sample.

one.

 com.somesite /

2.

 com.somesite / foo

3.

 com.somesite / foo / bar

Four.

 com. somesite / foo / blah

5.

 com.someothersite /

If the site definition for “somesite” includes the inclusive URL prefix “com.somesite /” and the exclusive URL prefix “com.somesite / foo / bar”, the application of this site definition to the sample URLs listed above produces a system that includes URLs 1, 2 and 4. URL 3 is not part of the site definition due to the explicit exclusion of "com.somesite / foo / bar". URL 5 is not part of the site definition because it was never included in the inclusive URL prefix "com.somesite /". The user interface 240 populates the site definition area 522 of the database 200. The polling map creation system 320 accesses the data from the site definition area 522 to determine which URLs to poll. The statistical summary system 230 accesses the data of the site definition area 522 to determine the levels of traffic to the sites by adding the traffic to the URLs included in a site.

Poll Map Area 524 contains a weight for each URL of each site that the system is measuring. This weight determines the probability that the system chooses a URL for each probe. The system generates weights by executing complex iterative algorithms against traffic data and database polling records.

200. An analysis of traffic data can discern which URLs have been visited, how often users have visited those URLs. The result of the analysis ensures that the system samples ads of these URLs in similar proportions, given certain limitations such as a maximum number of probes to assign to any individual URL. The data from the partition of sampling records 510 of the database 200 is useful for determining which URLs need special treatment due to past behavior (for example, a URL is sampled less infrequently if the system has never detected an ad in the URL) The polling map creation system 320 populates the polling map areas 524 of the database 200. The polling map creation system 320 accesses the data from the polling map area 524 to assign the probes. The statistical summary system 230 accesses the data of the survey map area 524 to determine which URLs should have scaled their rotations to counteract the effect of the forced application of the probing map limitations.

The area of defining ad extraction rules 526 describes Extensible Markup Language ("XML") tags, which typically represent a standardized HTML document, indicating those parts of the content that the system considers to be ads. The system defines an extraction rule in terms of "XML structure" and "XML characteristics". The "XML structure" refers to the placement of various XML nodes in relation to other XML nodes. For example, an anchor node ("A") that contains an image node ("IMG") is probably an advertisement. After using this structural detection process to match the content of the advertisement, the system examines the characteristics of the content to determine if the content is an advertisement. To continue the previous example, if the image node contains a link characteristic ("href") that contains the sub-string "adserver", it is very likely an advertisement. The characteristics can be agreed based on a simple sub-string, as in the example, or a more complicated regular expression. Another form of extraction rule can point to a specific node in an XML structure using some form of XML path specification, such as an "Xpointer." The media editor 264 populates the extraction rules definition area 526 of the database 200. The ad extractor 526 of the ad sampling system 220 accesses the data of the ad extraction rules definition area 326 for Determine which parts of each page probed represent an ad.

The advertising support data partition 530 of the database 200 comprises database tables that are logically segmented in the area of advertising product taxonomy 532, advertising information 534, tariff list 536, and expanded advertising product information 538.

The taxonomy area of advertising products 532 contains a hierarchical taxonomy of advertising products, attributes that describe which advertisement is being advertised. This taxonomy includes industries, companies, products, websites, web sites, messages, etc. Each node in the hierarchy has a type that specifies what kind of entity it represents and a parent node. For example, the hierarchy can specify which products exist within companies, which in turn exist within industries. Media editor 264 populates the taxonomy area of advertising products 532 of database 200. User interface 240 accesses data from the taxonomy area of advertising products 532 to generate statistical data and record where companies tend to advertise , industries, etc. The web client part 250 also accesses the data of the taxonomy area of advertising products 532 to present this information to the client 140.

The advertising information area 534 contains the data that describes what each unique advertisement registered by the system announces. The tables of the advertising information area 534 associate the advertising products with the advertisements. For example, the system may associate a type of advertising product company with a specific ad to indicate that the ad is advertising the company. The system uses the following procedures to associate an advertising product with an advertisement:

one.

 A "direct classification" assigns an advertising product directly to the ad. For example, a media editor 264 creates a direct classification specifying that a particular advertisement advertises the advertising product "Honda."

2.

 An "address classification" assigns an advertising product to an address prefix that the system uses to match the ad destination. For example, a media editor 264 creates an address classification by specifying that the address "com.honda" indicates an advertisement for Honda. An ad that points to “com.honda.www / cars,” therefore, associates the ad with Honda.

3.

 An "ancestor classification" assigns an ancestor of the advertising product to an advertisement. For example, if a direct classification assigns Honda to an advertisement, the “automotive” industry advertising product is a predecessor of Honda. The ancestor classification uses this relationship to associate automotive to the advertisement.

The media editor 264 populates the advertising information area 534 of the database 200. The user interface 240 accesses the data of the advertising information area 534 to generate statistical data.

The tariff list area 536 contains data that describes the cost of advertisements on a website. These costs include monetary values for each specific form, size, or duration of execution that Internet 100 advertisers use to determine the cost of ad purchases. The system stores rate list data for each website that the system polls. The media editor 264 can the tariff list area 536 in the database 200. The user interface 240 accesses the data in the tariff list area 536 to generate statistical data.

The expanded advertising product information area 538 contains additional information about specific types of advertising products not immediately captured in the taxonomy hierarchy. Specifically, this includes additional information related to websites and companies, such as company contact information URLs, website and media kit. This information extends the utility of the system by providing additional information to the client 140 about the polled entities. For example, a client 140 may follow a hyperlink to the company's contact information directly from a system report. The media editor 264 populates the expanded advertising product information area 538 of the database 200. The web client part 250 accesses the data of the expanded advertising product information area 538 to provide added value information to a customer 140 .

The advertising summary partition 540 of the database 200 comprises database tables that are logically segmented into areas of advertising statistics 542, data integrity 544 and summary of advertising information 546.

Advertising statistics area 542 describes how often an ad appears on each website. The system calculates and stores the following statistics in this area.

one.

 The proportion of page views that present an ad relative to the total number of page views. The system determines this statistic by analyzing the polling records.

2.

 The number of impressions an ad received. The system determines this statistic by measuring traffic levels for the website using the site definition and traffic data, and multiplying that measurement by the proportion of page views calculated above.

3.

 The amount of expenses an ad received. The system determines this statistic by applying the rate list information to the number of impressions that the ad receives previously calculated.

The statistical summary system 230 populates the advertising statistics area 542 of the database 200. The web client part 250 accesses the data of the advertising statistics area 542 to report expenses, impressions and rotation of customer announcements. 140.

The data integrity area 544 contains in-depth information about atypical statistical values and other potential anomalies that result from analysis of trends and time segments. This automated monitoring and analysis ensures that the system will contain accurate analysis data. In addition, the system uses real-world advertising information, as input to the system, to verify the accuracy of the analysis data. The data integrity analysis system, performed by the statistical summary system 230, populates the data integrity area 544 of the database 200. The operator 262 accesses the data integrity area 544 to detect potential errors and monitor General system health.

The advertising information summary area 546 summarizes the advertising information in a format that is compact and easy to distribute. The system extracts data from this area from the advertising support data partition

530. Although the data is not as descriptive as the data in the 530 advertising support data partition, they provide the ability to quickly perform an accurate query. Advertising support data partition 530 associates each ad with a company, product, or industry. If the system associates multiple advertising products of the same type with an advertisement, a single advertising product is chosen to summarize those associates using an allocation priority system, as follows:

one.

 Advertising products associated with an ad using direct ranking receive the highest possible priority, "M".

2.

 Advertising products associated with an ad using address classification receive a priority equal to the length of the address prefix chain to which they are assigned, therefore, a long address prefix chain will receive a higher priority than a chain of address short address prefix.

3.

 Advertising products associated with an advertisement using ancestor classification receive priority from the assigned ancestor.

Four.

 The ad receives the highest priority advertising product in each type.

5.

 When two ancestors having the same type and priority are assigned to an advertisement, a conflict occurs and must be corrected by the media editor 264.

The statistical summary system 230 populates the advertising information summary area 546 of the database 200. The web client part 250 accesses the advertising information summary area 546 to generate reports for the client 140.

The following description deals with an embodiment of the database structure illustrated in Figure 5. This data model is encoded in an Oracle database. The structure of the table comprises three environments, the basic scheme, the analysis scheme, and the client part. The basic scheme describes the environment of the server part that allows the "cloud poller" to direct autonomous live processes that continuously search the entire Web by recording the advertising activity and the operators and media editors for the present invention to direct, monitor and increase the information provided by the "cloud poller". The analysis scheme is the environment of the server part that allows the ad sampling system, also known as OMNIAC, to apply rigorous data analysis procedures to the information gathered from the Web. The scheme of the client part helps a client of the present invention to access data, build database query strings, and generate reports.

Database objects that comprise the "basic scheme" are used more frequently by various components of the OMNIAC system. The code bases that are based on this scheme include the implementation of the server-side processes that take advertisements from the Web. In addition, the database schemas used by other components associated with OMNIAC are composed of some or all of the tables in the basic schema. The basic scheme is conceptually composed of four sub-schemes that include advertising, advertisements, polling and sites. The advertising sub-scheme contains information about "advertising products" entities along with which each advertisement is being advertised. The ad sub-scheme describes the ads that the system has located and analyzed. The polling sub-scheme defines “when,” “where,” and “how” for the polling process. The site sub-scheme describes websites, including structural site definitions and rate list information.

Of the four sub-schemes, Advertising serves the more general purpose and is, therefore, referred to more frequently. The primary table in this sub-scheme is ADVERTISABLE, which defines advertising products. Many of the conceptual entities in the OMNIAC universe are advertising products: industries, companies, products, services and websites are all defined here. The type field, which refers to the ADVERTISABLE_TYPE table, distinguishes between different types of advertising products, and the parent field organizes the records hierarchically, establishing such relationships as industry-contains-company and company-produce-product.

In addition to the inherent grouping involved by the parent-child relationship defined in ADVERTISABLE, ADVERTISABLE_GROUP_MEMBER is used to further group the advertising products. Examples of groups defined in this way include car classes, segments of the travel industry, and types of computer hardware.

Other tables of the advertising sub-scheme serve to define what is announced by each advertisement. ADVERTISES is used to associate advertising products directly with advertisements. LOCATION ADVERTISES, CLASSIFIED LOCATION and LOCATION_MATCHES also indirectly associate advertising products with ads through the ad's destination address.

The "ads" referred to above are references to records in AD, the primary table in the ad sub-scheme. The ad sub-scheme is used to define each ad in the OMNIAC universe. Each unique ad has one record in AD, along with one or more records in AD_DEFINITION. Ad definitions are unique pieces of XML that OMNIAC has retrieved from the Web. Ads are unique ads defined by sets of ad definitions determined to be equivalent during automated classification.

Other tables in Ads contain ads attributes, referred to by AD and AD_DEFINITION. AD_TECHNOLOGY describes known web technologies used to produce ads, while TEXT describes textual content for certain ads. FUZZY_WEB_LOCATION contains fuzzy addresses found in the ads. A fuzzy address is a URL that has to be processed by the system, such as an anchor or an image. Once OMNIAC has loaded a fuzzy address, a reference is made to MIME_CONTENT if the URL refers to an image, or DEST_WEB_LOCATION if the URL refers to another HTML page.

Moving on, the polling sub-scheme controls the behavior of the OMNIAC poll and the ad extraction components. The primary purpose of this scheme is to define sets of objectives. A set of objectives is a conceptual construction that orders OMNIAC to search a set of pages at certain intervals, extracting the ads using a set of rules called extraction rules. Each set of goals is defined by a row in TARGET_SET.

The frequencies, directions, and extraction rules that constitute each set of objectives are defined in STROBE, AD_WEB-LOCATION, and EXTRACTION_RULE, respectively. The many-to-many relationships between rows of these tables are defined in TS_RUNS_AT, TS_PROBES, and TS_APPLIES.

The fourth and final subscheme is Sites, which simply records information about websites. Each site or subsite defined in the hierarchy of advertising products has a corresponding record in SITE_INFO, along with a number of rows in SITE_DOMAIN and SITE_MONTHLY_DATA. SITE_DOMAIN describes the physical structure of a site in terms of inclusive and exclusive URL roots. SITE_MONTHLY_DATA records lists of ad rates, third-party traffic estimates, and cache statistics for each site on a monthly basis.

The analysis scheme is an extension to the basic scheme that includes several additional tables populated by the OMNIAC analysis module. The analysis module is the unit in charge of processing the information contained in the basic scheme, producing a balanced data set that accurately describes the advertising activity.

Like the basic scheme, the analysis scheme is composed of four conceptual sub-schemes composed of tables that implement the common functionality. These subschemes include ad breakdown, summary of ad views, space statistics, and site statistics. The advertising decomposition sub-scheme contains information about each ad in the system, including attributes and what the ad is advertising. The ad view summary sub-scheme summarizes the ad views, recording how many times each ad was viewed in each space over the course of a day. The space statistics sub-scheme describes the rotation of advertisements for each space during each period of time. The site statistics sub-scheme describes site information, including ad rotation for each period of time.

The primary table in the advertising decomposition subschema is AD_INFO, which contains denormalized records that describe ad attributes. AD_INFO records are taken as ID’s control input data in the AD table; There is one AD_INFO record for each AD record that has been fully classified and represents a valid ad. AD_INFO is populated by the advertising relations analysis module described in the basic schema tables ADVERTISES and LOCATION_ADVERTISES.

The AD_INFO fields that specify what is advertised by an ad are: CATEGORY (industry), ORGANIZATION (company), ORGANIZATION_GROUP (industrial segment), ORGANIZATION_OVERGROUP, COMMODITY (product / service), COMMODITY_GROUP (product / service segment), COMMODITY_OVERGROUP, and MESSAGE

AD_INFO also includes fields that describe various non-advertising attributes. FORMAT, which refers to AD_SLOT_TYPE.ID, specifies the form factor of an ad. TECHNOLOGY, which refers to AD_TECHNOLOGY2.ID, specifies the technology used to implement the ad. DEFINITION, IMAGE and DESTINATION specify the AD_DEFINITION, IMAGE and DEST_WEB_LOCATION records associated with the ad. These three fields reflect the fields in the AD table.

The advertising decomposition scheme contains a few tables in addition to AD_INFO. ADV_IMPLICATION is a cache of implications of advertising products derived from the hierarchy in ADVERTISABLE. This is used to accelerate the operation of the analysis module. AD_INFO_FLATTENED is a more easily consulted version of AD_INFO that contains ad / ad product pairs for each of the fields in AD_INFO that refer to ADVERTISABLE. Finally, AD_TECHNOLOGY2 describes ad technologies interpreted by the analysis module that are presentable to the user on the client side.

The ad view summary sub-scheme covers the only PLACEMENT_SUMMARY table. PLACEMENT_SUMMARY is taken as the control input of the day, the announcement and the space, and contains, in the CNT field, the number of times an ad was seen in a space on a particular day.

The analysis module populates PLACEMENT_SUMMARY by adding the accesses registered in the APD_n tables, of which there is one for each day, where n is the ID of the day in question. These tables are created and populated by the server side as an access flow to ads within the system.

The third sub-scheme in the Analysis scheme is Space Statistics. This sub-scheme describes the behavior of the ads in the context of the spaces as well as information about the spaces themselves. A space is an address on the Web where ads rotate, currently defined in terms of address ID (a reference to AD_WEB_LOCATION.ID) and extraction rule ID (a reference to EXTRACTION_RULE.ID).

The primary table in Space Statistics is SLOT_AD_VIEWS, which records the total views and relative frequency for each ad in each space during each time period. The primary key of this table is made up of the PERIOD_TYPE, PERIOD, LOCATION_ID, RULE_ID and AD_ID fields. There are two fields outside the primary key: CNT contains the total number of ad views, and FREQUENCY contains the relative frequency.

In this subscheme there is also SLOT_SUMMARY, which records general information of spaces outside the context of individual advertisements. Therefore, this table is taken as control input data from the PERIOD_TYPE, PERIOD, LOCATION_ID and RULE_ID fields. The CNT field records total ad views in the space; This field is divided into SLOT_AD_VIEWS.CNT to determine the relative frequency. Also in SLOT_SUMMARY is a SLOT_TYPE field that specifies the type of ad most frequently seen in the space, and SITE_ID, which specifies within which site the space resides.

The final table in the space statistics sub-scheme is SLOT_TYPE_COUNT. This table is used to determine what value to use in SLOT_SUMMARY.SLOT_TYPE. The number of times each ad format was viewed is recorded, and the type of space that most views receive is placed inside SLOT_SUMMARY.SLOT_TYPE.

Figure 6 is a functional block diagram of the advertising prevalence system 130. The memory 610 of the advertising prevalence system 130 stores the software components, in accordance with the present invention, which analyze the traffic data over the Internet 100, sample advertising data from that traffic data, and generate summary data that characterize the advertising data. The system bus 612 connects the memory 610 of the advertising prevalence system 130 to the network adapter of the transmission control protocol / internet protocol ("TCP / IP") 614, the database 200 and the central processor 616. The adapter Network TCP / IP 614 is the mechanism that facilitates the passage of network traffic between the advertising prevalence system 130 and the Internet 100. The central processor 616 executes the programmed instructions stored in memory 610.

Figure 6 shows the functional modules of the advertising prevalence system 130 arranged as an object model. The object model groups the object-oriented software programs into components that perform the fundamental functions and applications in the advertising prevalence system 130. Proper implementation of the components of the object-oriented software program of Figure 6 can use the specification Enterprise JavaBeans The book by Paul J. Perrone et al, entitled "Building Java Enterprise Systems with J2EE" (Sams Publishing, June 2000) provides a description of a Java business application developed using the Enterprise JavaBeans specification. Matthew Reynolds' book, entitled "Beginning E-Commerce" (Wrox Press Inc., 2000) provides a description of the use of an object model in the design of a web server for an e-commerce application.

The object model for the memory 610 of the advertising prevalence system 130 employs a three-layer architecture that includes the presentation layer 620, the partition of infrastructure objects 630, and the business logic layer 640. The object model in addition It divides the business logic layer 640 into two partitions, the partition of application service objects 650 and the partition of data objects 660.

The presentation layer 620 contains the programs that manage the interface between the advertising prevalence system 130 and the client 140, the account manager 260, the operator 262, and the media editor 264. In Figure 6, the presentation layer 620 includes the TCP / IP interface 622, the web client part 624, and the user interface 626. A suitable implementation of the presentation layer 620 can use Java servlets to interact with the client 140, the account manager 260, the operator 262 and the media editor 264 of the present invention through the hypertext transfer protocol ("HTTP"). Java servlets are executed within a request / response server that handles request messages from client 140, account manager 260, operator 262 and media editor 264 and returns response messages to client 140, the account manager 260, operator 262 and media editor 264. A Java servlet is a Java program that runs within a web server environment. A Java servlet takes a request as input, parses the data, performs logical operations, and issues a response back to client 140, account manager 260, operator 262 and media editor 264. The Java execution platform combines Java servlets to simultaneously serve many requests. A TCP / IP 622 interface using Java servlets functions as a web server that communicates with client 140, account manager 260, operator 262 and media editor 264 using the HTTP protocol. TCP / IP interface 622 accepts HTTP requests from client 140, account manager 260, operator 262 and media editor 264 and passes the information of the request to visit object 642 in business logic layer 640. The Visiting object 642 passes the results information returned from the business logic layer 640 to the TCP / IP interface 622. The TCP / IP interface 622 sends these results back to the client 140, the account manager 260, the operator 262 and media editor 264 in an HTTP response. The TCP / IP 622 interface exchanges data with the Internet 100 through the TCP / IP 614 network adapter.

The infrastructure object partition 630 contains the programs that perform administrative and system functions on behalf of the business logic layer 640. The infrastructure object partition 630 includes the operating system 636, and a software program component oriented to objects for the database management system interface (“DBMS”) 632, the administrator interface 634, and the Java 638 execution platform.

The business logic layer 640 contains the programs that perform the essentials of the present invention. The business logic layer 640 in Figure 6 includes multiple instances of the visit object 642. There is a separate instance of the visit object 642 for each client session initiated by the web client part 624 or the user interface 626 through the TCP / IP 622 interface. Each 642 visit object is a dynamic session pod that includes a persistent storage area from the beginning to the end of the client session, not only during a single interaction or procedure call. The persistent storage area contains information associated with URL 114, 116, 118 or client 140, account manager 260, operator 262 and media editor 264. In addition, the persistent storage area contains data exchanged between the system. of advertising prevalence 130 and the traffic sampling system 120 through the TCP / IP interface 622 as the sets of query results from a database query

200.

When the traffic sampling system 120 finishes collecting information about a URL 114, 116, 118, it sends a message to the TCP / IP interface 622 that invokes a procedure to create a visitor object 642 and stores the information about the connection in the state of the visit object 642. The visit object 642, in turn, invokes a procedure in the traffic analysis application 652 to process the information retrieved by the traffic sampling system 120. The application of traffic analysis Traffic 652 stores the processed data from the anonymity system 310 and the polling map creation system 320 in the state of the traffic analysis data 662 and the database 200. Figures 7A and 7B describe, in greater detail , the process followed by the traffic analysis application 652 for each URL 114, 116, 118 obtained from the traffic sampling system 120. Even though Figure 6 represents the central processor 616 contro When the traffic analysis application 652 is understood, it is to be understood that the function performed by the traffic analysis application 652 can be distributed to a separate system configured similarly to the advertising prevalence system 130.

After the traffic analysis application 652 processes a URL 114, 116, 118 identified by the traffic sampling system 120, the visit object 642 invokes a procedure in the advertisement sampling application 654 to retrieve the URL 114, 116, 118 of the website 110. The ad sampling application 654 processes the recovered web page by extracting the embedded ads and classifying those ads. The ad sampling application 654 stores the data retrieved by the web page recovery system 322 and processed by the web browser emulation environment 324, the ad extractor 326 and the structural classifier 328 in the state of the sampling data of ads 664 and database 200. Figures 7A, 7C, 7D and 7E describe, in greater detail, the process that follows the ad sampling application 654 for each URL 114, 116, 118 identified by the sampling system of traffic 120. Even though Figure 6 represents the central processor 616 controlling the ad sampling application 654, a person skilled in the art will realize that the processing performed by the ad sampling application 654 can be distributed to a separate system configured similarly to the advertising prevalence system 130.

After the traffic analysis application 652 and the ad sampling system 654 process the URL 114, 116, 118 identified by the traffic sampling system 120, the visitor object 642 invokes a procedure in the statistical summary application 656 to calculate summary statistics for the data. The statistical summary application 656 calculates advertising print statistics, expenses and valuation statistics for each ad embedded in URL 114, 116, 118. The statistical summary application 656 stores the statistical data in the state of the statistical summary data 666 and database 200. Figure 7F describes, in greater detail, the process followed by the statistical summary application 656 for each URL 114, 116, 118 identified by the traffic sampling system 120. Even though Figure 6 represents the central processor 616 controlling the statistical summary application 656, a person skilled in the art realizes that the function performed by the statistical summary application 656 can be distributed to a separate system configured similarly to the advertising prevalence system 130 .

Figure 7A is a flow chart of a process in the advertising prevalence system 130 that measures the value of online advertisements by tracking and comparing online advertising activity across all major industries, channels, advertising formats and types. Process 700 begins, in step 710, sampling traffic data from Internet 100. Figure 7B describes step 710 in greater detail. Step 720 uses the traffic data sampled from step 710 to perform site selection, and define and refine site definitions for advertising prevalence system 130. Step 730 uses the result of the site selection and definition process. to generate a survey map based on sampled traffic data. Figure 7C describes step 730 in more detail. Step 740 uses the survey map of step 730 to visit the Internet 100 to collect sample data from the survey sites identified in step 730. Figure 7D describes step 740 in more detail. For each URL retrieved in stage 740, stage 750 extracts the ads from the URL, stage 760 classifies each ad, and stage 770 calculates the statistics for each ad. Figures 7E and 7F describe, respectively, steps 760 and 770 in greater detail. Finally, process 700 performs data integrity checks in step 780 to verify data integrity and analysis results in the system.

Figure 7B is an organization chart describing, in greater detail, the traffic data sampling process of Figure 7A, step 710. The process 710 begins in step 711 gathering data from a web traffic monitor such as the traffic system. Traffic sampling 120. Process 710 empties the user information from the data retrieved by the web traffic monitor in step 712 to debug the data and ensure the anonymity of the sample. For each URL of the purified sample, step 713 measures the number of web page views observed in the traffic data. Step 714 completes process 710 statistically extrapolating the measured number of web page views in the sample to the entire Internet universe 100.

Figure 7C is a flow chart describing, in greater detail, the process of generating a survey map based on the traffic data sampled from Figure 7A, step 730. Process 730 begins in step 731 analyzing a subset of the Sampled traffic data that falls within the eligible site definitions. After the analysis of step 731, step 732 constructs an initial survey map based on the sample traffic data. Step 733 analyzes the historical results of ad measurements from database 200 for the URLs of the initial poll map. Step 734 uses these historical results as well as system parameters to optimize the sampling plan. Step 735 completes process 730 by monitoring the results of the samples and adjusting the system as necessary.

Figure 7D is a flow chart describing, in greater detail, the polling process of the Internet 100 for collecting sample data of Figure 7A, step 740. The process 740 begins in step 741, searching for a web page of the Internet. 100. The web page of step 741 is passed to a web browser emulation environment in step 742 to simulate the presentation of the web page in a browser. This simulation allows the advertising prevalence system 130 to detect the ads embedded in the website. These ads can be embedded in JavaScript code, Java applet or servlet code, or common gateway interface code such as Perl script. In addition, the simulation in step 742 allows the advertising prevalence system 130 to detect dynamic and interactive advertisements on the website. After the simulation in step 742, step 743 extracts the ad data from the web page and step 744 stores the ad data in the database 200. Step 745 determines whether process 740 has to search for another page. web to gather more sample data. In the preferred embodiment, process 740 continuously samples web pages of the Internet 100. A person skilled in the art realizes that the functionality carried out by step 745 can be associated with a planning system that will plan the sounding of the Internet 100 to collect sample advertising data.

Figure 7E is an organization chart that describes, in greater detail, the process of classifying the advertising data of Figure 7A, step 760. Process 760 begins the analysis of ad fragments in step 761 determining whether the fragment is a duplicate . When the advertising prevalence system 130 finds an advertisement fragment for the first time, step 762 analyzes the internal structure of the fragment. After step 762, or when step 761 determines that the ad fragment is a duplicate, step 763 retrieves the external content of the ad from the web page. Step 764 then compares the external content with the previously observed advertisements. Step 765 analyzes the result of the comparison of step 764 to determine if the ad is a duplicate. When the advertising prevalence system 130 finds an advertisement for the first time, step 766 begins processing the new advertisement by registering the structure of the new advertisement in database 200. Step 767 then performs the automated classification of advertisements and stores the types of classification in database 200. Step 768 completes the processing of a new advertisement by performing the human verification of the advertisement classifications. After step 768, or when step 765 determines that the ad is a duplicate, step 769 updates the record of ad views in the database 200 to indicate the observation of the ad.

Figure 7F is an organization chart that describes, in greater detail, the process of calculating advertising statistics of Figure 7A, step 770. Process 770 begins the calculation of advertising statistics in step 771 summarizing the results of the advertising measurements. In step 772, process 770 uses the survey map generated in step 730 to weigh the results of the advertising measurements. Ad frequency is calculated in step 773 for each web page request. Step 774 uses the sample traffic data from step 710 and the ad frequency from step 773 to calculate the advertising impressions for each ad. Step 775 completes process 770 by calculating ad expenses by combining the advertising impressions of step 774 and the rate list data entered by the media editor 264 with the rate list collection module 348 of the user interface 240 .

Although the embodiments disclosed in the present invention describe a fully functional system, it is to be understood that there are other embodiments that are equivalent to the embodiments disclosed herein. As many revisions and variations will occur to those who review the present application, the present invention is not limited to the exact construction and operation illustrated and described herein. Accordingly, it is intended that all modifications and suitable equivalents that may be resorted to fall within the scope of the claims.

Claims (18)

one.

 A procedure to estimate the prevalence of digital content in a network, the procedure comprising:

receive (710) an estimate of a number of times a web page has been accessed (110; 110 ’; 110”; 411416; 421-424); repeatedly request (740) the website (110; 110 ’; 110”; 411-416; 421-424) and, in response, receive content files; determining (750) a number of times that a first content object (A1; A2; A3) is included in the content files; determining (770) a total number of times that the website (110; 110 ’; 110”; 411-416; 421-424) has been requested; and estimating (773, 774) the number of times that the first content object (A1; A2; A3) has been presented to website visitors based on the number of times the first content object (A1; A2 ; A3) was included in the content files, the total number of times the web page (110; 110 '; 110 "; 411-416; 421424) was requested, and the estimate of the number of times that access to the website (110; 110 '; 110 "; 411416; 421-424).

2.

 A method according to claim 1, wherein at least a part of the estimate of the number of times the website has been accessed (110; 110 '; 110 "; 411-416; 421-424) is received from a proxy

3.

 A method according to claim 1, wherein the procedure is performed by an advertising prevalence system (130).

Four.

 A method according to claim 1, wherein at least a portion of the estimate of the number of times the website has been accessed (110; 110 '; 110 "; 411-416; 421-424) is received from at least a panelist computer.

5.

 A method according to claim 1, wherein the content object (A1; A2; A3) is an advertisement.

6.

 A method according to claim 1, wherein estimating the number of times the first content object has been presented to visitors comprises:

determining (773) a rotation rate for the content object by dividing the total number of times the web page was requested by the number of times the first content object was included in the content files; determining (774) the number of times that the first content object has been presented to visitors by multiplying the estimate of the number of times the website has been accessed by the rotation rate.

7.

 A system to estimate the prevalence of digital content in a network, the system comprising:

an estimation device (210) for determining an estimate of a number of times a web page has been accessed (110; 110 ’; 110”; 411-416; 421-424); a poller (220) to repeatedly request the web page (110; 110 ’; 110”; 411-416; 421-424) and, in response, receive content files; a statistical summary system (230) to determine a number of times that a first content object (A1; A2; A3) is included in the content files, determine a total number of times that the web page (110; 110 ' ; 110 ”; 411-416; 421-424) has been requested, and estimate the number of times the first content object (A1; A2; A3) has been submitted to visitors to the website based on the number of times that the first content object (A1; A2; A3) was included in the content files, the total number of times the web page (110; 110 '; 110 ”; 411-416; 421-424) was requested , and the estimate of the number of times the website has been accessed (110; 110 '; 110 "; 411-416; 421-424).

8.

 A system according to claim 7, wherein at least a portion of the estimate of the number of times the website has been accessed (110; 110 '; 110 "; 411-416; 421-424) is received from a proxy

9.

 A system according to claim 7, wherein the system is an advertising prevalence system (130).

10.

 A system according to claim 7, wherein at least a portion of the estimate of the number of times the website has been accessed (110; 110 '; 110 "; 411-416; 421-424) is received from at least a panelist computer.

eleven.

 A system according to claim 7, wherein the content object (A1; A2; A3) is an advertisement.

12.

 A system according to claim 7, wherein the statistical summary system estimates the number of times the first content object has been presented to visitors:

determining (773) a rotation rate for the content object by dividing the total number of times the web page was requested by the number of times the first content object was included with the content files; determining (774) the number of times that the first content object has been presented to visitors by multiplying the estimate of the number of times the website has been accessed by the rotation rate.

13.

 A medium readable by a machine that stores instructions that, when executed, cause a machine to at least:

receive (710) an estimate of a number of times a web page has been accessed (110; 110 ’; 110”; 411416; 421-424); repeatedly request (740) the web page and, in response, receive content files; determine (750) a number of times that a first content object (A1; A2; A3) is included in the content files; determine (770) a total number of times the web page (110; 110 ’; 110”; 411-416; 421-424) has been requested; and estimate (773, 774) the number of times that the first content object (A1; A2; A3) has been presented to website visitors based on the number of times the first content object (A1; A2 ; A3) was included in the content files, the total number of times the web page (110; 110 '; 110 ”; 411-416; 421424) was requested, and the estimate of the number of times that access to the website (110; 110 '; 110 "; 411416; 421-424).

14.

 A machine-readable medium according to claim 13, wherein at least a portion of the estimate of the number of times the website has been accessed (110; 110 '; 110 "; 411-416; 421-424) It is received from a proxy.

fifteen.

 A machine-readable medium according to claim 13, wherein the instructions stored in the machine-readable medium are executed by an advertising prevalence system (130).

16.

 A machine-readable medium according to claim 13, wherein at least a portion of the estimate of the number of times the website has been accessed (110; 110 '; 110 "; 411-416; 421-424) It is received from at least one panelist computer.

17.

 A machine-readable medium according to claim 13, wherein the content object (A1; A2; A3) is an advertisement.

18.

 A machine-readable medium according to claim 13, wherein the instructions stored in the machine-readable medium estimate the number of times the first content object has been presented to visitors:

determining (773) a rotation rate for the content object by dividing the total number of times the web page was requested by the number of times the first content object was included in the content files; determining (774) the number of times that the first content object has been presented to visitors by multiplying the estimate of the number of times the website has been accessed by the rotation rate.