HK1161001A - Simulcast resolution in content matching systems - Google Patents

Description

Simulcast parsing in a content matching system

Technical Field

The present invention relates to systems and methods for measuring the presence of an audience for a media presentation, and in particular, to systems and methods that use content matching techniques.

Background

Devices and methods for measuring the audience of a media presentation, such as a television program or radio program, are well known in the art. In order to rate the advertising time contained in a broadcast, knowledge of the size and composition of the audience of a television or radio broadcast associated with certain environments (e.g., a home) is of paramount importance to the broadcast industry.

The audience groups that provide coordination in a television audience survey are referred to as "panels," and each participating audience in a panel is referred to as a "panel member. The audience measurement devices cooperate with associated media presentation devices or display systems used by the panelists to view television broadcasts at their respective viewing locations. Such measuring devices have three main objectives, namely: a) determining content presented on their associated media devices; b) identifying a broadcast source (e.g., a television channel, or other audio or video broadcast stream) and a content distribution platform (e.g., analog terrestrial delivery, digital terrestrial delivery, analog satellite delivery, cable television, IPTV, etc.); and c) recording the presence of one or more panelists so that the impact of the content, broadcast source and platform can be accounted for to produce audience data.

Audience measurement systems typically include a set-top box connected to a media device (traditionally a television). To identify the content, broadcast source and platform of the viewed program, these measurement systems may use one or more of various available methods, such as tuner frequency measurement, or identification of embedded video or audio codes, service information, image feature identification, watermarking, and signatures, among others.

For signature recognition, many systems have been proposed which essentially comprise a measuring device that continuously takes a signature from the audio or video output of a television or display device (or both) and stores the signature along with an associated time stamp. The stored signatures generated by the measuring means are later transmitted via a modem or any other telecommunication means to a remote central base (or station) where they are processed to identify the entire content displayed on the monitored television or display device.

This functionality may be implemented through content recognition techniques, which include a set of techniques and methods that can identify an unknown segment of audio or video material from a plurality of reference segments generated from known broadcast sources. Those skilled in the art will recognize that there are methods and algorithms for content identification by generating and identifying signatures. The audio and/or video signals are converted into signatures that characterize the media content being analyzed. Next, a pattern correlation engine is used to identify the unknown content by scanning the signature of the unknown content against a number of previously generated reference signatures. The correlation values are then analyzed according to a suitable algorithm to determine what is being displayed in order to provide a broad media measurement and monitoring service, the most widely used service being "broadcast identification" (i.e., identifying the broadcast source being viewed on a television set; e.g., in the case of television audience measurement, the broadcast source is typically a television channel).

However, during certain periods of time, it may occur that two or more different broadcast sources include the same content. Such events are called simulcast transmissions, characterized by: the scan engine may find two or more reference signatures that match the signature of an unknown piece of content, creating an ambiguous situation that prevents the audience measurement system from unambiguously assigning the content to a broadcast source.

In the case of a content matching system in combination with a source detection measurement system such as that proposed by Wheeler et al (U.S. patent No.6,675,383), a method is implemented to address this problem. If the audience measurement devices are able to identify the broadcast platform associated with the source that provided the signal to the television or media playback device during the simulcast, the scanning process only takes into account reference matching signatures generated by broadcast sources transmitted on the identified platform. This may ultimately reduce or even eliminate the aforementioned ambiguity, except in the case where simulcast includes two or more broadcast sources transmitted on the same platform (e.g., two analog terrestrial channels or two digital satellite channels).

Another approach, for example, proposed by Neuhauser et al (international patent publication No.2004/062282), involves detecting ancillary codes or any other type of metadata present in the broadcast signal that ultimately can identify the content viewed by the panelist(s) and/or the source of the broadcast, where the audio data is identified based on both a signature characterizing the audio data and additional data obtained from the audio data (e.g., as source identification codes). However, those skilled in the art know that code detection is used as the primary audience measurement method if the code or metadata is present in the broadcast signal received by the panelist, and generating a signature is often the second option when no code or metadata stream or code or metadata stream is detected or is not present, as suggested by Thomas et al in U.S. patent No.5,481,294, which describes a home measurement device that records secondary codes or extracts program signatures from programs when no secondary code is found in the broadcast signal. As a general rule, content matching methods are used when the audience measurement system cannot rely on fully available codes or metadata.

A third approach, as suggested by Williams et al in U.S. patent No.5,945,988, may use known audience data from the monitored panelist(s) in order to enhance the identification of the audio samples. However, this solution can only provide the best guess based on historical data.

Finally, the case of simulcast transmission has been given by Lee et al and described in international patent publication No. 2005/006768. However, the solution described in this patent publication is directed to the following specific case: the signature is generated based on a Cyclic Redundancy Check (CRC) or other identification of the predetermined data packet portion of the digital broadcast signal. Thus, this approach does not solve the problem in the case of analog broadcast signals or in the case of audience measurement systems that generate signatures based on time-domain or frequency-domain characteristics of digital audio or video signals that may be transmitted over the air as sound or electromagnetic waves from a rendering device (e.g., a television) and received remotely.

Therefore, there is a need to address the problems faced by content matching techniques in simulcast situations where supplemental information provided from a platform detector is not available or sufficient to identify the broadcast source, or where broadcast source identification by ancillary codes or other metadata types is not feasible.

Disclosure of Invention

The invention is defined by the appended claims.

The audience measurement system generates signatures of unknown content being viewed by the panelists. The signatures of the unknown content are stored and transmitted to a central processing site where they are compared to reference signatures to identify the signatures.

The signature of the unknown content may be remotely acquired from sound or electromagnetic waves through the air from a media presentation device such as a television or radio. Alternatively, the signature may be obtained directly from the audio or video (or audio and video) component of the broadcast signal from the electrical output of the media presentation device.

The scanning engine looks for matches between the signature of the unknown content and the signatures of the known content and stores successive matches to establish a tracking segment, which is a string of matches indicating that the unknown content is perfectly consistent with one or more known content over a particular period of time.

In accordance with another aspect of the invention, when the signature of more than one known content matches the signature of the unknown content, the system associates the unknown content with the known content having the longest tracking segment.

Drawings

The invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

figure 1 is a schematic diagram of a typical content matching.

Figure 1a is a schematic view of the assembly of figure 1.

Figure 2 is a diagram of a series of viewing segments and their corresponding signatures detected by the measurement instrument, a stream of reference signatures to which the signatures are compared, the resulting tracking segments and broadcast source attributes.

Figure 3 shows a combination of simulcast transmissions with viewing periods of different lengths.

Figure 4 is a table with simulcast resolution factors for different viewing segment lengths.

Figure 4a is a table showing the results of the implementation of the method and system according to the invention.

Fig. 5 is a flow chart of the operations performed by a program for executing the longest segment rule.

Detailed Description

In a typical audience measurement system 1 (see fig. 1), a measurement device 2 (referred to as a "measuring instrument") is used to measure the viewing activity of one or more members of a household with respect to a predetermined media presentation device 3.

Fig. 1a shows the individual measuring device 2 in more detail. Each measuring device 2 (of the plurality of measuring devices in the measuring system 1) obtains data about a broadcast received from one of a plurality of broadcast sources 50 and used by a user on the media presentation apparatus 3. As shown in fig. 1a, the illustrated measuring device 2 comprises an audio transducer 2a that receives an audio signal (sound sent through the air from a media presentation apparatus 3). The audio signal is converted to an electrical signal from which a signature corresponding to the unknown content being viewed on the multimedia presentation device 3 is generated. Alternatively, the measuring device 2 may be directly connected to the video or audio electrical output of the media presentation apparatus 3.

Each measuring device 2 stores those generated signatures and sends them to the central processing site 4. May be sent over a telephone line, an internet connection (wireless or local area network), a cellular telephone network, or any other communication network that provides support for data transfer. The signature 5 of the unknown content of the measuring device and the reference signature 6 are then compared by a scanning engine 7 at the central processing site 4, the scanning engine 7 outputting a corresponding match 8.

The "viewing period" is defined as a period of time during which the multimedia presentation apparatus 3 is turned on and the panelist indicates his or her participation. An example of such a system is described in the applicant's co-pending international patent application publication No. 2008/072094, which is incorporated herein by reference. Fig. 2 shows a schematic view of the measured viewing period. In the example shown in FIG. 2, viewing period 10 is represented, with viewing period 10 beginning at time t1 and ending at time t 2. Each viewing session is in turn divided into "viewing segments" (21, 22, 23, 24, 25 in the figure), i.e. periods of time tuned to the same broadcast source at the multimedia presentation device 3. The minimum length of time considered by the audience measurement system 1 to be a viewing period is called the "persistence threshold," the value of which is defined during system setup. The value adopted by most countries is 15 seconds, which will be assumed in the following description.

In the case of a content matching system, each measuring device 2 generates a signature 30 of the content presented in the viewing clip, and then sends that signature 30 to the central processing site 4 for identification. The signature 30 of the viewing segment is compared to a "reference signature" 40 (the signatures of all possible broadcast sources 50 that the monitored media presentation device can view). Thus, the signed streams are stored in files in the system database for each broadcast source 50. The scanning engine 7 compares the signature 30 of the viewing clip with the reference signatures 40 of the respective broadcast sources 50 and outputs a corresponding match. In the case of the present invention, in this process, every match between the signature of the measuring instrument and the reference signature of any broadcast source is stored in memory. A series of consecutive matches between the signature of the measurement instrument and the signatures of the various broadcast sources is referred to as a "tracking segment" 60, and the "tracking segment" 60 is also stored and used by the system to identify the broadcast source of the content being viewed. In the example shown in FIG. 2, the matching engine outputs the tracking segment corresponding to broadcast source A during t1 and t2 (view segment 21), the tracking segment corresponding to broadcast source B between t3 and t4 (view segment 22), and the tracking segment corresponding to broadcast source C between t5 and t6 (view segment 23).

In the case of simulcasting, two or more tracking segments may be associated to a same viewing segment. In the example shown in fig. 2, during the viewing segment 24, the scan engine 7 outputs two tracking segments having different time lengths: a longer tracking segment 61 corresponding to broadcast source a and a shorter tracking segment 62 corresponding to broadcast source B. During t7 and t8, the two broadcast sources transmit different content, and during t8 and t9, simulcast occurs. The scanning engine 7 assigns the viewing segment to the broadcast source with the longest tracking segment because the tracking segment includes additional information that enables the system to identify the viewed broadcast source based on the matching signature of the non-simulcast state. In this case, information before the simulcast state is used to identify the broadcast source. Another example of multicasting in the case of viewing segment 25 is provided in which the scanner outputs two tracking segments, both beginning at the same time t 10: a shorter tracking segment 63 corresponding to broadcast source a and a longer tracking segment 64 corresponding to broadcast source B. In this example, the simulcast state corresponds to the period between t10 and t 11. As in the previous example, the scan engine 7 assigns the viewing segment to the broadcast source with the longest tracking segment (broadcast source B in this example) because this segment contains information about the viewing period in the non-simulcast period that follows between t11 and t 12.

In order to determine which of the plurality of tracking segments during the simulcast corresponds to the viewing segment, the method of the present invention makes the decision based on the "longest tracking segment rule" explained in the previous example. The method is further described below.

The broadcast environment is characterized by two different simulcast behaviors: continuous simulcast and decentralized simulcast. The continuous simulcast is characterized in that: two or more broadcast sources broadcast exactly the same content throughout the day. The scatter-and-play is characterized in that: two or more broadcast sources alternately broadcast the same content or different content for certain periods of time. For example, scatter-simulcasts occur when a local or regional broadcast source broadcasts the same content throughout the day (typically a national network) but introduces local content during advertising hours or other predetermined periods. The "longest segment rule" addresses the case of scatter-simulcasting, where the simulcast periods of two or more broadcast sources follow or precede periods where the content of each broadcast source is different. Other examples of scatter-simulcasts include situations where the same movie or portions of the same movie are available at more than one broadcast source at the same time. It is noted that time-shifted environments, like those provided by modern broadcast platforms, can produce a virtual simulcast state in which a user can select the same content that can be obtained at different times from different broadcast sources, or even from a library that provides content "on demand". In all of these cases, the task of the content matching engine (i.e., the need to identify the content consumed by the television being measured) has a greater likelihood of finding the same content broadcast from multiple sources at different times or patterns. The present invention addresses the problem of maximizing the likelihood of correctly identifying the correct broadcast source from which the content being used is derived. To this end, the matching engine is programmed to select the longest tracking segment whenever two or more tracking segment candidates for any unknown viewing segment are found.

An application example of the present invention is given below. Fig. 3 shows a typical case of joint delivery, in which two or more broadcast sources simulcast deliver a program for a predetermined length of time 70 (15 minutes in the present example), and an advertisement time 80 having another predetermined length of time (in the example shown in fig. 3, the advertisement time 80 is 4 minutes) belonging to these broadcast sources is inserted. A possibility is shown to describe the resolution of the simulcast situation given different viewing segment durations and assuming a "duration threshold" 90 of 15 seconds. In the first case, a duration 91 of one minute is assumed for the viewing segment. Under these conditions, a total of 63 viewing segments (one every 15 seconds) may overlap with a 15 minute simulcast period. Of the 63 segments, 57 segments will fall completely within the simulcast period, making it impossible for the matching engine to determine the identity of the viewed broadcast source. Accordingly, 6 of the 63 one-minute segments will include signatures for non-simulcast content, enabling the viewed broadcast source to be identified by the corresponding tracked segment.

The same analysis is repeated for viewing segments of two, three and four minutes in length (92, 93 and 94 in fig. 3, respectively), and is valid for viewing segments shorter than the simulcast period. The following formula gives the total number of segments that can contain a part of the simulcast (total simulcast segments):

wherein the content of the first and second substances,

TSS: total simulcast segments;

ST: simulcast time (seconds) (70 in fig. 3)

VSL: viewing segment length (seconds) (91, 92, 93, and 94 in fig. 3);

PT: duration threshold (seconds) (90 in fig. 3);

the following formula gives the number of segments (parsed simulcast segments) that include the non-simulcast portion and thus can be identified by the signature of the non-simulcast content:

wherein the content of the first and second substances,

SSS: parsed simulcast fragments

VSL: viewing segment length (seconds)

PT: continuous threshold (seconds)

Given the above formula, the Simulcast Resolution Factor (SRF) under the longest track fragment rule is:

fig. 4 shows a table with SRF values for viewing segment lengths (lengths of simulcast transmission periods) ranging from 1 minute to 15 minutes for the values in the first column. The second column shows the average simulcast resolution factor for a given range and duration of 15 seconds. The third column shows the percentage of the simulcast segment to the viewing segment length resolved by the longest track segment rule.

For example, FIG. 4a shows a table with results of implementing a system and method according to the invention in the television market. The first column includes four different viewing segment length ranges. The second column represents the share of those viewing segment length ranges in the total viewing time. The third column shows the average resolution factor corresponding to the range of the longest segment rule described with the present invention. The fourth column shows the percentage of the total simulcast viewing time that is parsed in each case. As can be seen from the example of fig. 4a, in television environments having a viewing mode similar to that provided in fig. 4, approximately 90.9% of the total simulcast viewing time in these environments can be correctly identified by applying the present invention.

Moreover, the longer the viewing segment, the higher the likelihood of identifying the correct source using the present invention, since the likelihood of encountering different portions at the head or tail of the segment becomes correspondingly higher. Since most media consumption of broadcast content tends to occur in segments of a few minutes in length, the present invention contributes to reducing the impact of simulcast on the accuracy of the output data. Other methods may be used to determine other segments (more likely short segments) that are not focused on a single broadcast source if other methods exist.

FIG. 5 illustrates a process 100 for implementing the longest track segment rule, according to an exemplary embodiment of the present invention, process 100. The process 100 is performed for a viewing segment of each sample for which a broadcast source needs to be identified. The process 100 is executed as a computer program, such as the scan engine 7, containing executable program instructions in a processor. The process 100 begins at block 101 in fig. 5, where at block 101 a signature of a viewing segment generated by a measuring device at home is loaded. At block 102, a file containing a reference signature of a broadcast source associated with the measured household is loaded. The total number of files to be loaded will be determined by the home location (signal availability, i.e. terrestrial, satellite and cable networks) and the receiving device (possibility to decode the aforementioned available signals). Once the measurement signature and the reference signature are loaded, the process begins the search process 103.

Assuming that the respective signatures of the respective viewing segments have both a timestamp and a number, in block 104 the program sets a counter to n-0 for each new viewing segment to be analyzed. In block 105, the program acquires the signature indicated by the counter until the end of the viewing segment is reached, as shown in block 111. At block 106, the process compares the previously loaded reference signatures for a certain broadcast source N, searching for a match between the signature N of the viewing segment and any reference signatures for broadcast source N. If the viewing segment has been previously identified as being viewed in real time (by a system process not included in the procedure described in figure 5), then the search is limited to a range given by the time of occurrence of the signature plus/minus (+/-) a predetermined time tolerance (e.g. 15 seconds) taking into account the timestamp of the signature n for the viewing segment, so that the reference signature is also synchronously timestamped. However, if the viewing segment has been previously identified as time-shifted viewing (by a system process not included in the procedure described in FIG. 5), then in block 102 the loaded file should include a reference signature for broadcast source N for a predetermined number of days (e.g., current day plus previous day, plus previous week, plus previous month, etc.).

If a match is found (block 107), the process proceeds to block 108, which checks whether the reference signature that matches the signature of the viewing segment is synchronized with the previous matching reference signature (i.e. the time interval between two signatures in the viewing segment is exactly the same as the time interval between two corresponding matching signatures in the reference signature file). If so (or always the case where the first signatures of the various viewing segments are synchronized), the process stores the matching signature, appends it to the previously saved signature, thus generating a tracking segment for broadcast source N (block 109). The process then increments the signature counter (block 110) and repeats the matching process for the next signature until the end of the viewing segment is reached (block 111). When the end of the viewing segment is reached, the process compares, in block 112, the time length value of the tracking segment N saved in block 110 (given by the difference between the timestamps of the last and first matching signatures) with a previously defined persistence threshold value. If the length of time is greater than or equal to the persistence threshold, at block 113, the process stores the tracking segments of broadcast source N for later comparison with tracking segments from other broadcast sources, increments the reference signature counter (block 114), checks for the presence of a new reference signature to be matched (block 115), loads the reference signature of the new broadcast source (block 102), and begins a new search to compare the signature of the viewing segment with the reference signature of broadcast source N +1 (block 103). At block 112, if the length of time of the tracking segment is less than the persistence threshold, the tracking segment is discarded and the process proceeds to block 114, which has been described.

When the end of the reference signature is reached at block 115, the process begins the comparison process for all of the tracking segments stored at block 113. At block 117, the process compares the time lengths of the stored tracking segments, searching for the tracking segment with the longest duration. At block 118, if the process does not find a longest tracking segment, the process cannot identify the viewing segment being analyzed and output a corresponding message (in the example of FIG. 5, "designated by an alternative method" at block 119, assuming another method is used in this case). If the longest tracking segment is found at block 118, the process identifies the broadcast source having the longest tracking segment at block 120. Next, the process assigns the viewing segment to the broadcast source identified in the previous step and outputs the result (in this example, at block 121, the longest tracking segment is "broadcast source _ N").

Once a viewing segment has been assigned to a broadcast source, the process restarts at block 100, a signature for a new viewing segment is loaded (block 101), and the steps of searching, matching and comparing are repeated until all viewing segments generated by the measuring device have been processed.

Without prejudice to the underlying principles of the invention, the details and the embodiments may vary, also significantly, with respect to what has been described and illustrated, purely by way of example, without departing from the scope of the invention as defined by the annexed claims.

It will be apparent to those skilled in the art that the present invention may find advantageous application in a variety of media formats including television and radio programs broadcast via a variety of communications means, such as a cable television network, satellite network, internet connection, and the like, in a variety of processes involving identification of broadcast sources.

Claims (18)

1. A method of identifying a broadcast source during a simulcast transmission, wherein the method comprises:

generating a signature from unknown broadcast content;

comparing the signature of the unknown broadcast content to signatures of known broadcast sources to find a matching signature;

combining the matching signatures of each known broadcast source into a corresponding tracking segment;

comparing the time lengths of the respective tracking segments;

identifying a longest tracking segment of the respective tracking segments;

and according to the longest tracking segment, assigning the corresponding segment of the unknown broadcast content to a known broadcast source.

2. The method of claim 1, wherein the unknown broadcast content corresponds to real-time viewing.

3. The method of claim 1, wherein the unknown broadcast content corresponds to time-shifted viewing.

4. The method of claim 1, wherein the unknown broadcast content is a television program.

5. The method of claim 1, wherein the unknown broadcast content is a radio broadcast program.

6. The method of claim 1, wherein the unknown broadcast content is conveyed by analog signals.

7. The method of claim 1, wherein the unknown broadcast content is conveyed by digital signals.

8. The method of claim 1, wherein the known broadcast source is a television channel.

9. The method of claim 1, wherein the known broadcast source is a radio station.

10. The method of claim 1, wherein the known broadcast source is transmitted by analog signals.

11. The method of claim 1, wherein the known broadcast source is transmitted by a digital signal.

12. The method of claim 1, wherein the signature is generated based on time domain characteristics of the unknown broadcast content.

13. The method of claim 1, wherein the signature is generated based on frequency domain characteristics of the unknown broadcast content.

14. The method of claim 1, wherein the signature is generated based on digital data stream characteristics of the unknown broadcast content.

15. A computer program executable on a processing device, the computer program comprising computer executable instructions for performing the steps of the method according to claim 1.

16. A recording medium containing the computer program according to claim 15.

17. An apparatus comprising a processor configured to perform the method of claim 1 or comprising the computer program of claim 15.

18. An audience measurement system, comprising:

a measurement device comprising an audio or video signature generator; and

a processor according to claim 17, in communication with an audience measurement instrument to identify a broadcast source from an audio or video signature generated by the audio or video generator during a simulcast transmission.