HK1186325A - Controlling a media program based on a media reaction - Google Patents

Description

Controlling media programs based on media responses

Technical Field

The invention relates to controlling media programs based on media responses.

Background

Currently, users enjoying media programs often desire to control those media programs during presentation of the programs, such as pausing, muting, or stopping presentation. For example, a user may wish to stop a television program when he or she gets up with a snack while watching the television program. To do this, a conventional controller, such as a remote control, may require the user to find the remote control, find a stop button on the remote control, press the stop button, and once he or she returns with the snack, find the remote control again, find a play button, and press the play button.

Alternatively, the user may be listening to a music program and a friend listening with the user may start talking to the user during the music program. In this case, the user may wish to pause, reduce the volume, or mute the music program until the discussion ends. Conventional approaches permit a user to pause, reduce volume, or mute a program through the user's intentional, active controls, such as through a volume dial on a stereo amplifier or through a remote control. However, controlling music programs by these methods can be slow, causing the user to miss what friends say, or otherwise not provide a good user experience.

Moreover, the user may have a very small son or daughter who unexpectedly steps into a room in which a song or movie is being played that the user does not want the son or daughter to hear or see. In this case, the user may attempt to stop the media program, but such an attempt at control may be undesirably slow.

These are just a few examples of ways in which conventional methods for controlling media programs may not provide a good user experience.

Disclosure of Invention

Techniques and apparatuses for controlling media programs based on media responses are described herein. In some embodiments, the techniques pause, mute, or stop a media program when a user leaves a room, when a user in the room is talking or otherwise not paying attention to the program, or when a child walks into the room.

This summary is provided to introduce simplified concepts for controlling media programs based on media responses, which are further described below in the detailed description. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

Drawings

Embodiments of techniques and apparatuses for controlling media programs based on media responses are described with reference to the following drawings. In the drawings, like reference numerals are used to refer to like features and components:

FIG. 1 illustrates an example environment in which techniques for controlling media programs based on media responses, as well as other techniques, may be implemented.

FIG. 2 is an illustration of an example computing device local to the viewer of FIG. 1.

FIG. 3 is an illustration of an example remote computing device that is remote to the viewer of FIG. 1.

FIG. 4 illustrates an example method for determining media reaction based on passive sensor data.

Fig. 5 shows a time-based plot of media responses, which are interest levels for one user and for 40 time periods during the presentation of a media program.

FIG. 6 illustrates an example method for constructing a reaction history.

FIG. 7 illustrates an example method for controlling a media program based on a media reaction.

Fig. 8 illustrates a scene of a media program and a media reaction of a person viewing the media program.

Fig. 9 illustrates an example method for controlling media programs based on a person's identity or demographics.

FIG. 10 illustrates an example method for controlling a media program based on media responses of a plurality of people.

FIG. 11 illustrates an example device in which techniques for controlling media programs based on media responses, as well as other techniques, may be implemented.

Detailed Description

Overview

Techniques and apparatuses for controlling media programs based on media responses are described herein. For example, consider the following: both Bob and Janet are watching television shows. Suppose Janet turns to Bob to talk. At this point, or in response to Bob turning to Janet to respond to her, the technique pauses the program. Neither person is looking for, finding, or selecting a remote control to pause the program. Also, assume that Bob or Bob and Janet turn the head back towards the television. At which point the technique recovers the program. Note again that neither person needs to use a remote control to restore the program.

The following is also considered: television shows are not suitable for children. When Bob and Janet do not want their children to see this program, they are watching it after the children have gone to sleep. However, assume that their six-year-old daughter, Abigail, gets up from bed and walks into the room where Bob and Janet are watching the television series. Rather than finding, and quickly pressing a stop button on the remote control, Abigail, upon walking into the room, the technique causes the screen to be blacked out and the audio to be muted. When Abigail leaves the room, the technique may wait a short period of time to rewind the television series to or before the time the series was blacked out and muted and resume the series.

These are merely two examples of how the described techniques and/or apparatus may control a media program based on media reactions, however many other examples are contemplated herein. Where the context permits, the techniques and/or apparatus are referred to herein separately or collectively as "techniques". Turning now to an example environment in which the techniques may be embodied, then to example methods that are capable of (but not required to) working with the techniques. Some of these various methods include methods for sensing and determining a reaction to media and constructing a reaction history for a user. Following these example methods, the present document turns to example methods for controlling media programs based on media reactions.

Example Environment

FIG. 1 is an illustration of an example environment 100 for receiving sensor data and determining media reactions based on the sensor data. These media responses may be used to control media programming, among other uses. The techniques may use these media responses, alone or in combination with other information, such as demographics, response history, and information about people and media programs or portions thereof.

The environment 100 includes a media presentation device 102, an audience sensing device 104, a status module 106, an interest module 108, an interface module 110, and a user interface 112.

The media presentation device 102 presents media programs to a viewer 114 having one or more users 116. Media programs may include, alone or in combination, television programs, movies, music videos, video clips, advertisements, blogs, web pages, electronic books, computer games, songs, song albums or programs, slideshows or other pictorial arrangements, tweets (tweets), or other audio and/or video media. The audience 114 may include one or more users 116 located in: the location allows for consumption of the media program presented by the media presentation device 102 and measurements made by the audience sensing devices 104, whether separately or within a group of viewers 114. Three users are shown in viewer 114: user 116-1, user 116-2, and user 116-3. Although only three users are shown, sensor data may be sensed and media responses determined in many locations and for tens, hundreds, thousands, or even millions of users.

The audience sensing devices 104 are capable of sensing the audience 114 and providing sensor data of the audience 114 to the status module 106 and/or the interest module 108 (the sensor data 118 is shown as being provided via an arrow). The sensed data may be sensed passively, actively, and/or in response to an explicit request.

Passively sensed data is passive by not requiring active participation by those users when measuring them. The actively sensed data includes data recorded by a user in the audience (such as a handwritten log) and data sensed from the user by a biometric sensor worn by the user in the audience. The sensor data sensed in response to an explicit request may be actively or passively sensed. One example is where the technique requests the user to perform a particular action to produce a particular result before or during control of the media program, such as raising a hand if the user wishes the technique to stop or pause the media program or mute the media program. In this case, the user is expressing a reaction to raise a hand, however this may be sensed passively by measurements that do not require the user to actively participate in the reaction. The technique senses this lifted hand in several ways as described below.

The sensor data may include data sensed using light emitted by the audience sensing device 104 or other signals transmitted, such as with an infrared sensor that bounces emitted infrared light off of a user or audience space (e.g., a sofa, a wall, etc.) and senses returned light. Examples of ways to measure sensor data of a user and to measure sensor data are provided in more detail below.

The audience sensing devices 104 may or may not process the sensor data prior to providing the sensor data to the status module 106 and/or the interest module 108. Thus, the sensor data may be or include raw or processed data such as: RGB (red, green, blue) frames; an infrared data frame; depth data; heart rate; a respiration rate; head orientation or movement of the user (e.g., three-dimensional coordinates x, y, z and three angular pitch (pitch), tilt (tilt), and yaw (yaw)); face (e.g., eyes, nose, and mouth) orientation, movement, or occlusion; orientation, movement, or occlusion of the skeleton; audio, which may include information indicating an orientation sufficient to determine from which user the audio originated or directly indicating which user or what was spoken (if someone is speaking); a thermal reading sufficient to determine or indicate the presence and location of one of the users 116; and distance from the audience sensing devices 104 or the media presentation devices 102. In some cases, the audience sensing devices 104 include infrared sensors (webcams, Kinect cameras), stereo or directional audio microphones, and thermal readouts (plus infrared sensors), although other sensing means may also or instead be used.

The status module 106 receives the sensor data and determines a status 120 (shown at arrow) of the user 116 in the audience 114 based on the sensor data. The states include: for example: sadness, talking, nausea, fear, smiling, frowning, calm, surprise, anger, laughing, screaming, clapping, shaking hands, cheering, looking away, leaning towards … …, sleeping, just arriving, or leaving, to name a few.

The talk state may be a general state indicating that the user is talking, however it may also include subcategories based on voice content, such as speaking about the media program (talk related) or speaking not related to the media program (talk unrelated). The status module 106 may determine which category of speech to speak via speech recognition.

Based on the sensor data, the status module 106 may also or instead determine the number of users, the identity and/or demographic data of the users (shown at 122), or the engagement during the presentation (shown at 124). The identity indicates a unique identity of one of the users 116 in the audience 114, such as SusanBrown. The demographic data categorizes one of the users 116, such as 5 feet 4 inches tall, children, and male or female. Engagement indicates whether the user may be focusing on the media program, such as based on the user's presence or head orientation. In some cases, engagement may be determined by the status module 106 with sensor data having a lower resolution or less processed than the sensor data used to determine the status. Even so, participation may still be useful in measuring audience, whether by itself or for determining the interests of the user using the interests module 108.

The interest module 108 determines a user's interest level 130 (shown at arrow) for a media program based on the sensor data 118 and/or the user's engagement or status (shown at arrow with engagement/status 126) and information about the media program (shown at arrow with media type 128). Interest module 108 may determine that, for example, a number of laugh states for a media program intended as a serious drama indicate a low interest level and, in turn, a number of laugh states for a media program intended as a comedy indicate a high interest level.

As shown in fig. 1, status module 106 and/or interest module 108 provides demographics/identities 122 and one or more of the following media responses: participation 124, status 120, or interest level 130, are all shown at the arrows in fig. 1. Based on one or more of these media responses, status module 106 and/or interest module 108 may also provide another type of media response, i.e., a total media response type to the media program, such as a rating (e.g., thumbing up or samsung). However, in some cases, the media reaction is instead received by the interface module 110 and the overall media reaction is determined.

The status module 106 and interest module 108 may be local to the viewer 114, and thus local to the media presentation device 102 and the audience sensing device 104, although this is not required. An example embodiment in which the status module 106 and the interest module 108 are local to the viewer 114 is shown in FIG. 2. However, in some cases, the status module 106 and/or the interest module 108 are remote to the viewer 114, which is shown in fig. 3.

The interface module 110 receives the media responses and demographic/identity information and determines or receives some indication as to which media program or which portion thereof the responses relate to. The interface module 110 may present the media reaction 132 to the media program or cause the media reaction 132 to the media program to be presented through the user interface 112, although this is not required.

The interface module 110 may be located locally with respect to the audience 114, such as in the case where a user is viewing his or her own media response or a family member's media response. However, in many cases, the interface module 110 receives the media reaction from a remote source.

Note that the sensor data 118 may include the context in which the user is reacting to the media or the current context of the user for whom a rating or recommendation of the media is requested. Thus, the audience sensing devices 104 may sense that a second person is in the room or otherwise physically near the first person, which may be the first person's context. The context may also be determined in other ways as described below in fig. 2.

Fig. 2 is an illustration of an example computing device 202 local to viewer 114. The computing device 202 includes, or has access to, the media presentation device 102, the audience sensing device 104, one or more processors 204, and a computer-readable storage medium ("CRM") 206.

The CRM206 includes an operating system 208, a status module 106, an interest module 108, media programs 210 (each of the media programs 210 may include or have associated program information 212 and portions 214), an interface module 110, a user interface 112, a history module 216, a reaction history 218, and a control module 220.

Each of the media programs 210 may have, include or be associated with program information 212 and portions 214. The program information 212 may indicate the title, episode, author or artist, genre, and other information of the program, including information about portions within each media program 210. Thus, the program information 212 may indicate that one of the media programs 210 is a music video, includes a harmony portion repeated 4 times, includes 4 verse (verse) portions, includes portions based on each visual presentation during the song, such as the artist singing, the vocal accompaniment singer dancing, the title of the music video, the artist, the year of manufacture, resolution and formatting data, and so forth.

Portions 214 of one of the media programs 210 make up the program, and each portion may have metadata or other information about each portion (e.g., an "R" rating for a particular portion but not other portions of the same movie). These portions may represent particular time ranges in the media program, such as 2 second, 5 second, or 15 second time periods. These portions may in turn represent cohesive portions in the media program, which may be based on content in the cohesive portions, such as a complete song played in a radio-like program, a ball or play in a sporting event, an action in a theater show, an advertisement in an advertisement block, or a scene in a movie, to name a few.

The history module 216 includes a reaction history 218 or has access to the reaction history 218. The history module 216 may construct and update a reaction history 218 based on ongoing reactions of the user (or others as noted below) to the media program. In some cases, the history module 216 determines the respective context of the user, however this may in turn be determined and received from other entities. Thus, in some cases, the history module 216 determines the time, the location, the weather of the location, and so forth during the user's reaction to the media program or request for a rating or recommendation of the media program. The history module 216 may determine ratings and/or recommendations for media based on the user's current context and the reaction history 218. As noted elsewhere herein, the reaction history 218 may be used with media reactions as a basis for controlling the presentation of media programs.

The control module 220 can control the presentation of the media program based on the media reaction. The control module 220 may implement control based on media responses that indicate as little information as "a person is present and has an identity or demographic," for example. Thus, the control module 220 may control the media program based on a media reaction indicating that the person has walked into the room and that the human child is. Alternatively, the media module 220 may obfuscate the scene of the movie showing a horse with broken legs that is playing based on the person's identity, which is indicated by the person's reaction history 218: the person may be sensitive to a scene showing an animal in distress.

As shown in fig. 2 and 3, the media program 210, the portion 214, or the control module 220 may be located locally or remotely to the computing device 202, and thus locally or remotely to one or more users having media reactions (e.g., the user 116-1 of the audience 114 of fig. 1).

Note that in this illustrated example, the entities including the media presentation device 102, the audience sensing device 104, the status module 106, the interest module 108, the interface module 110, the history module 216, and the control module 220 are included within a single computing device (such as a desktop computer with a display, a front-facing camera, a microphone, an audio output, and so forth). However, each of these entities may be separate from or integrated with each other in one or more computing devices or otherwise. As will be described in part below, the media presentation device 102 may be integrated with the audience sensing device 104 but separate from the status module 106, the interest module 108, the interface module 110, the history module 216, or the control module 220. Also, each of these modules may operate on separate devices or be combined in one device.

As shown in fig. 2, the computing devices 202 may each be one or a combination of various devices, here shown in six examples: a laptop computer 202-1, a tablet computer 202-2, a smart phone 202-3, a set-top box 202-4, a desktop computer 202-5, and a gaming system 202-6, although other computing devices and systems, such as televisions with computing capabilities, netbooks, and cellular telephones, may also be used. Note that three of these computing devices 202 include the media presentation device 102 and the audience sensing device 104 (laptop 202-1, tablet 202-2, smartphone 202-3). One device does not include but is in communication with the media presentation device 102 and the audience sensing device 104 (desktop computer 202-5). Two other devices do not include the media presentation device 102 and may or may not include the audience sensing device 104, such as in the case where the audience sensing device 104 is included within the media presentation device 120 (set top box 202-4 and gaming system 202-6).

Fig. 3 is an illustration of an example remote computing device 302 that is remote to viewer 114. Fig. 3 also illustrates a communication network 304 by which the remote computing device 302 communicates with the audience sensing devices 104 (not shown, but implemented within or in communication with the computing device 202), the interface module 110, the history module 216 (with or without the reaction history 218), or the control module 220, assuming these entities are in the computing device 202 as shown in fig. 2. The communication network 304 may be the internet, a local area network, a wide area network, a wireless network, a USB hub, a computer bus, another mobile communication network, or a combination of these.

The remote computing device 302 includes one or more processors 306 and a remote computer-readable storage medium ("remote CRM") 308. The remote CRM308 includes a status module 106, an interest module 108, media programs 210 (each of the media programs 210 may include or have associated program information 212 and/or portions 214), a history module 216, a reaction history 218, and a control module 220.

Note that in this illustrated example, the media presentation device 102 and the audience sensing device 104 are physically separate from the status module 106 and the interest module 108, with the first two operating locally to an audience viewing the media program and the second two operating remotely. Thus, sensor data is communicated from the audience sensing devices 104 to one or both of the status module 106 or interest module 108, which may be transmitted locally (FIG. 2) or remotely (FIG. 3). Moreover, after being determined by status module 106 and/or interest module 108, the respective media responses and other information may be communicated to the same or other computing devices 202 for receipt by interface module 110, history module 216, and/or control module 220. Thus, in some cases, a first one of the computing devices 202 may measure sensor data, transmit the sensor data to the remote device 302, and then the remote device 302 transmits the media response to another one of the computing devices 202, all over the network 304.

These and other capabilities, as well as the manner in which the entities of fig. 1-3 act and interact, are set forth in greater detail below. These entities may be further divided, combined, and the like. The environment 100 of fig. 1 and the detailed illustrations of fig. 2 and 3 show some of many possible environments in which the described techniques can be employed.

Example method

Determining media response based on passive sensor data

Fig. 4 depicts a method 400 of determining media response based on passive sensor data. These and other methods described herein are illustrated as sets of blocks that specify operations performed, but are not necessarily limited to the orders shown for performing the operations of the respective blocks. In portions of the following discussion, reference will be made to the environment 100 of fig. 1 and entities illustrated in detail in fig. 2-3, the reference being made thereto for exemplary purposes only. The techniques are not limited to being performed by one entity or multiple entities operating on one device.

Block 402 senses or receives sensor data of a viewer or user that is passively sensed during presentation of a media program to the viewer or user. This sensor data may include the context of the viewer or user, or the context may be received separately, although the context is not required.

For example, consider the following: the audience includes all three users 116 of FIG. 1: users 116-1, 116-2, and 116-3. Assume that the media presentation device 102 is an LCD display having speakers and through which the media programs are presented, and that the display is in communication with the set top box 202-4 of fig. 2. Here the audience sensing devices 104 are a Kinect capable of sensing sound and position, a front facing high resolution infrared sensor, a red-green-blue sensor, and two microphones integrated with the set top box 202-4 or the media presentation device 102. Assume also that the media program 210 being presented is an animated movie rated PG, named "superman family" (incorporated) and streamed from a remote source and through the set top box 202-4. The set top box 202-4 presents a family of superman with 6 ads, with the intervals occurring at the beginning of the movie-one, three in three ad blocks, two in one two ad block.

Receiving sensor data for all three users 116 in the audience 114; for this example, consider first user 116-1. It is assumed here that: during the superman family process, the audience sensing device 104 measures the following at various times for the user 116-1 and then provides the following at block 402:

at time 1, the head is oriented 3 degrees, no audio or low amplitude audio.

At time 2, the head is oriented 24 degrees with no audio.

At time 3, the skeleton moves (arm), high amplitude audio.

At time 4, the skeleton moves (arms and body), high amplitude audio.

Time 5, head movement, facial feature change (20%), medium amplitude audio.

At time 6, detailed face orientation data, no audio.

At time 7, the skeleton is oriented (missing), with no audio.

Time 8, face orientation, breathing rate.

Block 404 determines a status of the user during the media program based on the sensor data. In some cases, block 404 determines a probability of the state, or a plurality of probabilities of a plurality of states, respectively. For example, block 404 may determine that a state may be correct, but not completely deterministic (e.g., 40% of the chance users are laughing). Block 404 may also or instead determine, based on the sensor data, that multiple states are possible (such as sad or calm states) and the probability of each state (e.g., sad state 65%, calm state 35%).

Block 404 may also or instead determine demographics, identity, and/or engagement. Also, the method 400 may skip block 404 and proceed directly to block 406, as described later below.

In the example made, the status module 106 receives the sensor data listed above and determines the following corresponding status of the user 116-1:

time 1: and (4) looking at the direction.

Time 2: the eyes were removed.

Time 3: clapping the hands.

Time 4: and (4) cheering.

Time 5: and (5) laughing.

Time 6: smiling.

Time 7: and (4) leaving.

Time 8: the patient is asleep.

At time 1, the status module 106 determines that the status of the user 116-1 is looking at the media program based on sensor data indicating that the head of the user 116-1 is 3 degrees off looking directly at the LCD display and a rule indicating that the "looking at" status applies to less than 20 degrees off (as an example only). Similarly, at time 2, the status module 106 determines that the user 116-1 is looking away because the deviation is greater than 20 degrees.

At time 3, the status module 106 determines that the user 116-1 is clapping his hand based on sensor data indicating that the user 116-1 has skeletal movement of his arm and high amplitude audio. The status module 106 may distinguish claps from other statuses (such as cheering) based on the type of arm movement (not indicated above for simplicity). Similarly, at time 4, the status module 106 determines that the user 116-1 is cheering due to arm movement and high amplitude audio attributable to the user 116-1.

At time 5, the status module 106 determines that the user 116-1 is laughing based on sensor data indicating that the user 116-1 has head movement, 20% facial feature changes, and medium amplitude audio. Various sensor data may be used to distinguish between different states, such as screech based on audio being of medium amplitude rather than high amplitude, and facial feature changes, such as opening of the mouth and raising of both brows.

For time 6, the audience sensing devices 104 process the raw sensor data to provide processed sensor data, and in this case, perform facial recognition processing to provide detailed facial orientation data. In conjunction with the absence of audio, the status module 106 determines that the detailed face orientation data (here, raised mouth corners, amount of eye lid coverage) indicates that the user 116-1 is smiling.

At time 7, the status module 106 determines that the user 116-1 is away based on sensor data indicating that the user 116-1 has skeletal movement away from the audience sensing device 104. The sensor data may also indicate this directly, such as in the case where the audience sensing device 104 does not sense the presence of the user 116-1 (either by having no skeleton or head readings, or no longer receiving a thermal signature).

At time 8, status module 106 determines that user 116-1 is asleep based on sensor data indicating that the facial orientation of user 116-1 has not changed for a certain period of time (e.g., the user's eyes have not blinked), and a steady, slow breathing rate.

These eight sensor readings are a simplified example for purposes of explanation. The sensor data may include extensive data as noted elsewhere herein. Also, sensor data may be received that measures the audience once every fraction of a second, thereby providing detailed data for tens, hundreds, and thousands of time periods during the presentation of the media program, and from which status or other media responses may be determined.

Returning to method 400, block 404 may determine demographics, identity, and engagement in addition to the status of the user. The status module 106 may determine or receive sensor data, determine demographics and identities from the sensor data, or receive demographics or identities from the audience sensing devices 104. Continuing the ongoing example, the sensor data of the user 116-1 may indicate that the user 116-1 is John Brown, the user 116-2 is Lydia Brown, and the user 116-3 is Susan Brown. Alternatively, for example, the sensor data may indicate that the user 116-1 is 6 feet, 4 inches tall and male (based on skeletal orientation). The sensor data may be received with or include information indicative of portions of the sensor data that may be attributable to each of the users in the audience, respectively. However, in this current example, assume that the audience sensing device 104 provides three sets of sensor data, where each set of sensor data indicates the identity of the user along with the sensor data.

Still at block 404, the techniques may determine the participation of the audience or users in the audience. As noted, this determination may not be as fine as the determination of the user's status, but is still useful. Assume for the above example that sensor data is received for user 116-2 (Lydia Brown), and that this sensor data includes only head and skeleton orientation.

At time 1, the head is oriented 0 degrees and the skeleton is oriented with the upper torso in front of the lower torso.

At time 2, the head is oriented 2 degrees and the skeleton is oriented with the upper torso in front of the lower torso.

At time 3, the head is oriented 5 degrees and the skeleton is oriented with the upper torso approximately flush with the lower torso.

At time 4, the head is oriented 2 degrees and the skeleton is oriented up the torso behind the lower torso.

At time 5, the head was oriented 16 degrees and the skeleton was oriented with the upper torso behind the lower torso.

At time 6, the head was oriented 37 degrees and the skeleton was oriented with the upper torso behind the lower torso.

At time 7, the head is oriented 5 degrees and the skeleton is oriented with the upper torso in front of the lower torso.

At time 8, the head is oriented 1 degree and the skeleton is oriented with the upper torso in front of the lower torso.

The status module 106 receives this sensor data and determines the following corresponding participation of Lydia Brown:

time 1: the participation is high.

Time 2: the participation is high.

Time 3: participate in medium-high.

Time 4: participate in the middle.

Time 5: participate in medium-low.

Time 6: the participation is low.

Time 7: the participation is high.

Time 8: the participation is high.

At times 1, 2, 7, and 8, the status module 106 determines that Lydia is highly participating in the superman family at these times based on sensor data indicating that the head of the user 116-2 is 5 degrees or less off-center from direct view to the LCD display and the skeletal orientation of the upper torso before the lower torso (indicating that Lydia is leaning forward in the media presentation).

At time 3, the status module 106 determines that the participation level of Lydia has dropped because Lydia is no longer leaning forward. At time 4, the status module 106 determines that Lydia engagement has further decreased to moderate based on Lydia leaning backward, even though she is still looking almost directly at the superman family.

At times 5 and 6, the status module 106 determines that Lydia is less engaged, falls to medium-low, and then falls to low engagement based on Lydia still leaning backward and looking slightly away (16 degrees) and then looking significantly away (37 degrees), respectively. Note that at time 7, Lydia quickly returns to high participation, which may be of interest to the media creator because it indicates what is considered exciting or otherwise influential.

The method 400 may proceed directly from block 402 to block 406 or from block 404 to block 406 or block 408. If proceeding from block 404 to block 406, the technique determines the interest level based on the type of media being presented and the user's participation or status. If proceeding from block 402 to block 406, the technique determines the interest level based on the type of media being presented and the sensor data of the user without first or independently determining the user's engagement or status.

Continuing the above example for users 116-1 and 116-2, assume that block 406 receives the state determined by state module 106 for user 116-1 (John Brown) at block 404. Based on the state of John Brown and information about the media program, the interest module 108 determines the level of interest (overall or over time) for the superman family. It is assumed here that the superman family is an adventure program and a comedy program, and that parts of the movie are marked as having one of these media types. Although simplified, assume that times 1 and 2 are marked as comedy, times 3 and 4 are marked as adventure, times 5 and 6 are marked as comedy, and times 7 and 8 are marked as adventure. Revisiting the state determined by the state module 106, consider again the following:

time 1: and (4) looking at the direction.

Time 2: the eyes were removed.

Time 3: clapping the hands.

Time 4: and (4) cheering.

Time 5: and (5) laughing.

Time 6: smiling.

Time 7: and (4) leaving.

Time 8: the patient is asleep.

Based on these states, the state module 106 determines: for time 1, John Brown has a medium-low interest in the content of time 1-if this is an adventure or drama type, the status module 106 may determine that John Brown is instead highly interested. Here, however, since the content is comedy and thus is intended to evoke laughter or the like, the interest module 108 determines that John Brown has medium-low interest at time 1. Similarly, for time 2, interest module 108 determines that John Brown has low interest at time 2 because his status is not just laughing or smiling but looking away.

At times 3 and 4, the interest module 108 determines that John Brown has a high level of interest based on the type of adventure and the status of clapping and cheering at these times. At time 6, based on the comedy type and John Brown smile, he is determined to have moderate interest at this time.

At times 7 and 8, interest module 108 determines that John Brown has very low interest. The media type is here a hazard, however in this case the interest module 108 would determine that John Brown's level of interest is very low for most types of content.

It is readily seen that advertisers, media providers, constructors or augmentors of media, and media creators may benefit from knowing the user's level of interest. It is assumed here that the level of interest for the superman family is provided over time, as well as demographic information about John Brown. Using this information from multiple demographically similar users, the media creator may learn that adult males are interested in certain adventure content, but that most comedy parts are uninteresting, at least for this demographic group.

As a more detailed example, consider fig. 5, fig. 5 shows a time-based plot 500 having an interest level 502 of 40 time periods 504 over a portion of a media program. It is assumed here that the media program is a movie that includes other media programs, advertisements, at time slots 18 through 30. As shown, interest module 108 determines that the user started from a medium interest level and then bounced between medium and medium-high, and very high interest levels until time period 18. During the first advertisement covering time periods 18 through 22, interest module 108 determines that the user has a medium low interest level. However, for time periods 23 through 28, interest module 108 determines that the user has a very low level of interest (because, for example, he is looking away and talking or leaving the room). However, for the last advertisement covering time slots 28 through 32, interest module 108 determines that the user has an intermediate level of interest for time slots 29 through 32, the majority of the advertisement.

This may be valuable information-left for the first advertising user, left for the middle ad and the beginning user of the last ad, and returned (with moderate interest) for the majority of users of the last ad. This resolution and accuracy of interest is contrasted with certain conventional methods, which may not provide information about how many of the movie viewers actually viewed the advertisement, which advertisements were viewed, and in what amount of interest. If this example is a common trend for viewing the masses, the price of the advertisement in the middle of the block may be decreased, while other advertisement prices may be adjusted. Alternatively, advertisers and media providers may learn to play short blocks of advertisements, for example, with only two advertisements. Interest level 502 also provides valuable information about portions of the movie itself, such as by a very high interest level at time period 7 (e.g., a particularly influential scene of the movie) and a diminished interest at time periods 35-38.

Note that in some cases, the participation level, while useful, may be less useful or accurate compared to the status and interest level. For example, if the user's face is occluded (blocked) and thus not watching the media program, the status module 106 may determine that the user is not engaged only for the engagement level. If the user's face is occluded by the user's hand (skeletal orientation) and the audio indicates high volume audio, the status module 106 may determine that the user is screaming when determining the status. In combination with the content being terrorist or suspensory, screaming status indicates a very high level of interest. This is just one example of a situation where the level of interest may be significantly different from the level of engagement.

As indicated above, method 400 may proceed directly from block 402 to block 406. In this case, interest module 108, alone or in combination with status module 106, determines the interest level based on the type of media (including multiple media types for different portions of the media program) and the sensor data. As an example, for sensor data of John Brown at time 4, which data indicates skeletal movement (arm and body), and high amplitude audio, as well as comedy, sports, conflict-based talk show, adventure-based video game, twitter, or horror types, interest module 108 may determine that John Brown has a high level of interest at time 4. Conversely, for the same sensor data at time 4, interest module 108 may determine that John Brown has a low level of interest at time 4 for an episode, a comedy, or classical music. This may be performed based on the sensor data without first determining the engagement level or state, however the determination may also be performed.

Following block 404 or 406, block 408 provides demographics, identity, engagement, status, and/or interest level. The status module 106 or interest module 108 may provide this information to various entities, such as the interface module 110, the history module 216, the control module 220, and others.

Providing this information to the constructor of the highlight program may enable the constructor to construct the program with truly highlights, such as a well-scored joke in a favorite show or a superb sporting contest in a sporting event. Providing this information to the augmentor of the media program may enable the augmentor to add media reactions to the presentation of the media program, which may improve the user experience. The user enjoys comedies more when accompanied by a real laugh and at the correct moment of the comedy show, e.g., as compared to the laugh track.

Providing this information to advertisers after presenting an advertisement (determining the media reaction in the advertisement) may effectively enable advertisers to measure the value of the advertisements they have displayed during the media program. Providing this information to the media creator may effectively enable the media creator to evaluate the potential value of similar media programs or portions thereof. For example, prior to releasing the media program to the public, the media creator may determine the portion of the media program that is not reacting well and thereby modify the media program to improve it.

Providing this information to the rating entity may effectively enable the rating entity to automatically rate the media program for the user. Some other entity, such as control module 220, may use this information to control the presentation of media.

Providing media responses to the history module 216 may effectively enable the history module 216 to construct and update a response history 218. The history module 216 may construct the reaction history 218 based on one or more contexts in which each set of media reactions to the media program is received, or the media reactions may be accounted for in whole or in part in context. Thus, the context of the media response in which the user watched the television program after work on a wednesday night may be altered to reflect that the user may be tired from work.

As noted herein, the techniques may determine multiple states of a user during most media programs, even for 15 seconds of advertising or video clips. In this case, block 404 is repeated, such as over a one second period.

Moreover, the status module 106 may determine not only multiple statuses of the user over time, but also various different statuses at particular times. For example, a user may be both laughing and looking away, both of which are states that may be determined and provided or used to determine the user's level of interest.

Further, either or both of status module 106 and interest module 108 may determine engagement, status, and/or interest levels based on historical data as well as sensor data or media types. In one case, the user's historical sensor data is used to normalize the user's engagement, status, or interest level (e.g., dynamically for the current media reaction). For example, if Susan Brown is watching a media program and receives her sensor data, the techniques may normalize or otherwise learn how to best determine her engagement, status, and interest level based on her historical sensor data. If the historical sensor data of SusanBrown indicates that she is not a particularly expressive or talking user, the techniques may be adjusted for this history. Thus, a lower amplitude of audio may be sufficient to determine that Susan Brown laughed than the amplitude of audio used to determine that a typical user laughed.

In another case, the historical engagement, status, or interest level of the user for whom the sensor data was received is compared to the historical engagement, status, or interest level of others. Thus, based on data indicating that Lydia Brown exhibits high interest in almost every media program she watches as compared to the interest levels of others (either generally or for the same media program), a lower interest level for Lydia Brown may be determined. In any of these cases, the techniques learn over time and may thus normalize participation, status, and/or interest levels.

Method for constructing a reaction history

As noted above, the techniques may determine the user's participation, status, and/or interest level in various media programs. Moreover, these techniques may use passive or active sensor data to do so. Using these media responses, the techniques may construct a user's response history. This reaction history can be used in various ways as described elsewhere herein.

FIG. 6 depicts a method 600 for constructing a reaction history based on a user's reaction to a media program. Block 602 receives a user's reaction set sensed during presentation of a plurality of respective media programs and information about the respective media programs. An example reaction set for media programs is shown in fig. 5, showing a measure of interest level over time of presentation of the program to a user.

The information about the respective media programs may include, for example, the name of the media (e.g., office, 104 th collection) and its genre (e.g., song, television program, or advertisement), as well as other information described herein.

In addition to the media reactions and their corresponding media programs, block 602 may also receive a user's context during presentation of the media programs as described above.

Further, block 602 may receive media responses from other users, which are used to construct a response history. Thus, the history module 216 may determine other users having similar reactions to those of the user based on the user's media reactions (in part, or after constructing an initial or preliminary reaction history for the user). The history module 216 may determine other people who have reactions similar to the user's reaction and refine the user's reaction history using those other people's reactions to programs that the user has not seen or heard.

Block 604 constructs a reaction history for the user based on the user's reflection set and information about the corresponding media program. As noted, block 604 may also use the reaction history, context, etc. of others to construct the user's reaction history. In some embodiments described elsewhere herein, this reaction history may be used to control the media program.

Method for controlling media programs

Fig. 7 illustrates a method 700 for controlling a media program based on a media reaction. As noted, the techniques may control media programs without requiring active control by the user (e.g., a dial, keyboard, or remote control), although active control by the user is not necessarily precluded.

Block 702 receives a current media reaction to a media program currently being presented to an audience having one or more people, the media reaction determined based on sensor data passively sensed during presentation. These media responses may include one or more of the many media responses described herein, which may be determined as indicated above.

Block 704 determines that the person is not paying attention to the media program based on the current media reaction. Media responses that may indicate that a person is not paying attention include very low interest levels, low engagement, a status of away, an unrelated conversation status, and a shift from eye status, to name a few.

For example, assume that two people named Bob and Janet are watching a movie together in the room of their home. Suppose Janet goes to Bob and reviews the movie. It is assumed here that control module 220 receives the talk status from status module 106 of fig. 1-3. Assume that state module 106 indicates that the conversation is relevant and that an accompanying eye-loss state is also received based on an analysis of the first few words spoken by Janet. Here the control module 220 determines that the media reaction of Janet is not indicative of Janet being not of interest based on the conversation being relevant. The control module 220 may then wait to receive another media reaction at block 702.

Continuing the ongoing example, assume that Bob removes gaze from the movie and responds to Janet just after Janet turns to Bob and talks. In block 704, control module 220 receives Bob's gaze away status and conversation status soon after receiving Janet's related conversation and gaze away status. The control module 220 then determines that Bob is not paying attention to the movie, but instead, that he is paying attention to Janet. Control module 220 may determine that Bob is not paying attention based on his gaze-off status and his conversation status. Control module 220 may instead also consider the state of Janet or Bob's reaction history, such as a reaction history indicating, for example, that Bob rarely talks or looks away while watching a media program.

In response to determining that the person is not paying attention to the media program, block 706 controls presentation of the media program. This control may be performed in real time, quickly, and in various ways. The control module 220 may pause the presentation of the media program, mute or reduce the audio of the media program, or stop the presentation of the media program.

The control module 220 may also or instead record a mark in the media program at a time or location commensurate with the current media reaction. This flag may be used later when "rewinding" the media program, as noted below.

Continuing the ongoing example, the control module 220 pauses the movie. The control module 220 may wait for another media reaction, as indicated below. The control module 220 may also or instead explicitly request a media reaction in which the control module 220 stops controlling (e.g., pausing) the media program in response to the media reaction. It is assumed here that the control module 220 pauses the movie and presents a request stating "please wave your hand to continue the program" over the paused movie. Bob or Janet may wave their hands to continue the movie, however here we assume that they continue their conversation while the movie remains paused.

Block 708 receives a second media response for the person. The media reaction being received may be the same or a similar media reaction, and the method 700 continues to control the presentation of the media program in response to the media reaction. Thus, method 700 may repeat blocks 702, 704, and 706. Control module 220 continues to pause the movie while, for example, Bob and Janet continue talking and remove the eye from the movie.

At some point, however, assume that Bob looks back to the display presenting the paused movie. In this case, the control module 220 receives the on state rather than receiving the more off-gaze state.

Block 710 determines that the person is paying attention or is ready to pay attention to the media program based on the second media reaction. Block 710 determines that the media reactions upon which a person is paying attention or is ready to pay attention may be different, including based on the reaction history of the person. The media reactions that may indicate that the user is focusing on include a medium or higher level of interest, a medium or higher engagement, a look-at state, a tilt-to- … … state, and a just-reached state, to name a few.

Continuing the ongoing example, at block 710, the control module 220 receives Bob's look-at status. The control module 220 determines that Bob is now paying attention because he is looking at the paused movie.

In response to determining that the user is or is about to focus on the media program, block 712 ceases control and/or resumes the media program. As noted above, controlling the presentation of the media program may include pausing, muting, or stopping the media program, among other things. Thus, the control module 220 may stop pausing the media program, stop muting the audio of the media program, or resume the media program.

However, in some cases, the control module 220 rewinds the media program for a particular amount of time (such as 2 seconds), or presents the media program at the beginning of the cohesive portion during which the control of the media program occurred.

In the above example of Bob and Janet, it is assumed that control module 220 does not pause the movie during the period during which Bob is not in the particular scene of interest. The control module 220 may rewind the media program back to the beginning of the same scene. This may depend on the length of distraction of the viewer. At more than one instant of distraction, control module 220 may rewind and begin playing at the beginning of a scene. Thus, assume that Bob and Janet talk for 5 minutes. In this case, the control module 220 may rewind to the beginning of the scene. If Bob and Janet converse for only 4 seconds, control module 220 may instead simply stop pausing the movie or rewinding for only a few seconds.

As noted above, the control module 220 may record a mark at a time or location in the media program. This flag may assist the control module 220, such as where the control module 220 does not pause the media program, but instead mutes or dims the volume down but allows the media program to continue to be presented.

As a different example, assume that two people are listening to an album with thirty songs. In response to the two people starting to talk, control module 220 may turn the volume down, but not turn the volume off or stop the song. The control module 220 may mark this location and if the lack of attention lasts for several minutes or more, rewind to the beginning of the song to replay the song at normal volume when the conversation is stopped.

The control module 220 may analyze metadata associated with the location or time to determine the cohesive portion in which the control is implemented. Thus, the control module 220 may determine that the current portion of the media program is part of a scene in the media program having a beginning of a particular time based on the metadata of the movie at the marked location in the media program. The control module 220 may then resume presentation of the media program at the particular moment in time to replay the beginning of the scene.

By way of illustration, consider fig. 8, which shows a media program 800, scenes 802 for the first five minutes of a television series named guilt and penalty (here, scenes 802-1, 802-2, 802-3, and 802-4), and participation 804 of a person watching the television series. Assume that during scenario 802-3, control module 220 receives a low participation media reaction. Based on this low engagement, control module 220 decreases the volume and marks a point at which the volume is decreased (shown at marker 806). The control module 220 determines that this reduced volume control, shown at reference 806, is within the scene 802-3. In response to determining to play the scene again, the control module 220 presents the series at the beginning of the scene 802-3 (shown as scene start 808).

The method 700 may operate alone or in combination with other methods described herein, such as the methods 400, 600, 900, and/or 1000. This description continues with other methods that also describe techniques for controlling media programs.

Fig. 9 illustrates a method 900 for controlling media programs based on the identity or demographics of a person. Method 900 may operate in conjunction with method 700 or similarly to method 700, such as by recording a mark or determining a cohesive portion of the media program where the presentation of the media program is to continue after it is no longer desired to control the presentation, and so forth.

Block 902 receives an identity or demographic of a person, which is determined based on sensor data passively sensed during a current presentation of a media program. As noted herein, the identity or demographic may be determined from the media response, however in some cases it may also be determined based on sensor data without also having to determine the person's media response.

Block 904 determines to control the current presentation of the media program based on the identity or demographics and information about the media program. This information may indicate, for example, that the media program is of a particular type, such as a horror or suspicion program, or that the program is rated as unsuitable for children, and so forth.

Block 906 controls the current presentation of the media program. This control over the current presentation may include those noted for method 700, however block 906 may alter the presentation in additional ways not set forth above.

For example, assume that a person's identity or demographics indicates that the person is a minority group (minors) and that the information indicates that the media program or a currently presented portion of the media program is not suitable for presentation to the minority group. In this case, the control module 220 may pause, stop, or mute the media program as described above, however, the control module 220 may instead alter the presentation by blacking its screen or significantly reducing the resolution (e.g., pixelation of the video) and reducing the volume of the video portion of the presentation. For the movie example above, assume that Bob and Janet's 6-year-old daughter unexpectedly walked into the room while the movie was playing. In response, the control module 220 may partially blur or reduce the resolution of the movie while also reducing the volume, but may forego stopping or pausing the program.

Also, if an identity is received, the control module 220 may determine a reaction history associated with the person. In this case, control module 220 may determine whether to control the current presentation at block 904 and, if control is determined, how to control the presentation. For example, assume that the information about the portion of the media program indicates that boldness is to be presented for the media program and that the reaction history indicates that the person in the viewer has a history of dislikes or being sensitive to boldness. In such an example, the control module 220 may decrease the volume of the presentation during the coarse.

Consider again Bob and Janet from the movie example above. It is assumed here that the control module 220 receives or determines the identity of Janet and the associated reaction history. Based on this history of response, control module 220 determines that Janet is very sensitive to or would be offended by an animal shown to be in distress. For a large portion of the media program, the control module 220 may determine that no control is needed. Assume, however, that for a particular upcoming scene, control module 220 determines that a dog is hit by a car in this upcoming scene based on the metadata of the media program associated with the portion of the movie (e.g., information 212 about portions 214 of all media programs 210 of fig. 2). The control module 220 may blur the video presentation, reduce the resolution of the video presentation, or otherwise alter the video presentation. Note that in doing so, the techniques may not only permit people to better enjoy media programs by enabling people to forgo actively controlling programs in many cases, but may also enable people to enjoy programs that they may not have seen at all.

Thus, suppose a child's parent has decided that a popular comedy show will be good for the child to watch, except for boldness, because the comedy would otherwise have many compensatory qualities. It may be almost impossible for a child to enjoy the comedy without hearing the boldness-an alternative is for the parent to deliberately hold the mute button on the remote control (which may still not mute all boldness) or view the comedy without any audio (which may make viewing the comedy meaningless). However, the techniques enable a child to view the comedy without the parent actively controlling the program.

After the controlled portion of the media program is no longer presented (such as by being fully presented or skipped), the control module 220 stops controlling the media program. Thus, the control module 220 may stop muting the gross cut after the gross cut ends or is skipped, or stop blurring the scene showing the animal in distress after the scene ends.

Fig. 10 illustrates a method 1000 for controlling a media program based on media responses of a plurality of people. Method 1000 may operate in conjunction with or similarly to methods 700 and/or 900, however this is not required.

Block 1002 receives a first media reaction to a media program currently being presented to an audience having one or more people, the first media reaction determined based on sensor data that is passively sensed during a current presentation and is a first person of the one or more people.

Block 1004 determines that the first person is not paying attention to the media program based on the first media reaction. The control module 220 may do so in one or more of the various ways described above. Some media responses do not indicate that a person is not paying attention, while some media responses that do indicate that a person is not paying attention may be ignored in some cases. For example, the state of departure may indicate that control of the media program is warranted, although this is not always the case. If an advertisement has just started, the away state may not indicate that stopping the current presentation of the media program (and thus the advertisement) is warranted. Similarly, a low interest level during an advertisement may not indicate that the control module 220 should stop presentation of the media program.

In response to determining that the first person is not paying attention to the media program, block 1006 stops or alters the current presentation of the media program, such as by stopping, pausing or muting the current presentation.

Block 1008 receives a second media reaction of the first person. The control module 220 may determine that this media reaction is from the same person as the first media reaction, although the received media reaction may also be tagged or otherwise include an indicator that associates the media reaction with one of the viewers, whether or not the identities of those persons are known.

Block 1010 determines that the first person is paying attention or will pay attention to the media program based on the second media reaction.

Block 1012 presents a second presentation of the media program at or before the point at which the current presentation stopped or changed. As described above, presenting a media program at or before the point at which the current presentation stopped or changed may be performed based on determining the cohesive portion (such as a scene or song) at which to begin the presentation. Presenting the media program may be performed automatically and without user interaction, although this is not required.

Block 1014 receives or determines that a second person of the one or more persons in the audience is of an approximate age. The approximate age may be based on an identity or demographic determined for the second person, which may be received or determined. If determined, the approximate age may be based on the second person's media reaction, which may be as simple as receiving a "new presence status" as indicated above.

Block 1016 determines to stop or alter the second presentation of the media program based on the information about the media program and the second person is approximately aged.

Block 1018 stops or alters the second presentation of the media program. The control module 220 may resume the media program by re-presenting the media program in response to determining that the second person is no longer present (such as by receiving a departure status from the status module 106 of fig. 1-3) in the various manners set forth above.

These are just a few of the many ways in which the techniques may enable people to better enjoy or control media programs.

The foregoing discussion describes methods related to controlling media programs based on media responses, as well as other methods and techniques. Aspects of these methods may be implemented in hardware (e.g., fixed logic circuitry), firmware, software, manual processing, or any combination thereof. A software implementation represents program code that performs specified tasks when executed by a computer processor. The example methods may be described in the general context of computer-executable instructions, which may include software, applications, routines, programs, objects, components, data structures, procedures, modules, functions, and the like. The program code can be stored in one or more computer-readable memory devices, both local and/or remote to a computer processor. The method may also be implemented by multiple computing devices in a distributed computing mode. Furthermore, the features described herein are platform-independent and may be implemented on a variety of computing platforms having a variety of processors.

These techniques may be embodied at one or more of the entities shown in fig. 1-3 and 11 (device 1100 is described below), which may be further divided, combined, etc. Accordingly, these figures illustrate some of the many possible systems or devices capable of employing the described techniques. The entities in these figures generally represent software, firmware, hardware, entire devices or networks, or a combination thereof. For example, in the case of a software implementation, the entities (e.g., status module 106, interest module 108, interface module 110, history module 216, and control module 220) represent program code that performs specified tasks when executed on a processor (e.g., processors 204 and/or 306). The program code can be stored in one or more computer-readable memory devices, such as CRM206 and/or remote CRM308 or computer-readable storage media 1114 in fig. 11.

Example apparatus

Fig. 11 illustrates various components of an example device 1100 that can be implemented as any type of client, server, and/or computing device described with reference to previous fig. 1-10 to implement techniques for controlling media programs based on media reactions. In various embodiments, device 1100 can be implemented as one or a combination of wired and/or wireless devices, such as any form of a television mobile computing device (e.g., a television set-top box, a Digital Video Recorder (DVR), etc.), a consumer device, a computer device, a server device, a portable computer device, a user device, a communication device, a video processing and/or rendering device, an appliance device, a gaming device, an electronic device, a system on a chip (SoC), and/or another type of device or portion thereof. Device 1100 can also be associated with a user (e.g., a person) and/or an entity that operates the device such that a device describes logical devices that include users, software, firmware, and/or a combination of devices.

Device 1100 includes communication devices 1102 that enable wired and/or wireless communication of device data 1104 (e.g., received data, data that is being received, data scheduled for broadcast, data packets of the data, etc.). The device data 1104 or other device content can include configuration settings of the device, media content (e.g., media programs 210) stored on the device, and/or information associated with a user of the device. Media content stored on device 1100 can include any type of audio, video, and/or image data. Device 1100 includes one or more data inputs 1106 via which any type of data, media content, and/or inputs can be received, such as human utterances, user-selectable inputs, messages, music, television media content, media responses, recorded video content, and any other type of audio, video, and/or image data received from any content and/or data source.

Device 1100 also includes communication interfaces 1108, which can be implemented as any one or more of a serial and/or parallel interface, a wireless interface, any type of network interface, a modem, and as any other type of communication interface. Communication interfaces 1108 provide a connection and/or communication links between device 1100 and a communication network by which other electronic, computing, and communication devices communicate data with device 1100.

Device 1100 includes one or more processors 1110 (e.g., any of microprocessors, controllers, and the like) which process various computer-executable instructions to control the operation of device 1100 and to implement techniques for controlling media programs and other methods described herein based on media responses. Additionally or alternatively, device 1100 can be implemented with any one or combination of hardware, firmware, or fixed logic circuitry that is implemented in connection with processing and control circuits which are generally identified at 1112. Although not shown, device 1100 can include a system bus or data transfer system that couples the various components within the device. A system bus can include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or local bus that utilizes any of a variety of bus architectures.

Device 1100 also includes computer-readable storage media 1114, such as one or more memory devices, examples of which include Random Access Memory (RAM), non-volatile memory (e.g., any one or more of a read-only memory (ROM), flash memory, EPROM, EEPROM, etc.), and a disk storage device, that enables persistent and/or non-transitory data storage (i.e., as opposed to mere signal transmission). A disk storage device may be implemented as any type of magnetic or optical storage device, such as a hard disk drive, a recordable and/or rewriteable Compact Disc (CD), any type of a Digital Versatile Disc (DVD), and the like. Device 1100 may also include a mass storage device 1116.

Computer-readable storage media 1114 provides data storage mechanisms to store the device data 1104, as well as various device applications 1118 and any other types of information and/or data related to operational aspects of device 1100. For example, an operating system 1120 can be maintained as a computer application with the computer-readable storage media 1114 and executed on processors 1110. The device applications 1118 may include a device manager, such as any form of a control application, software application, signal processing and control module, code that is native to a particular device, a hardware abstraction layer for a particular device, and so forth.

The device applications 1118 also include any system components, engines, or modules to implement techniques for controlling media programs based on media reactions. In this example, the device applications 1118 may include the status module 106, the interest module 108, the interface module 110, the history module 216, and/or the control module 220.

Final phrase

Although embodiments of techniques and apparatus for controlling media programs based on media responses have been described in language specific to structural features and/or methods, it is to be understood that the subject of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations for controlling media programs based on media responses.

Claims (10)

1. A computer-implemented method, comprising:

receiving a current media reaction to a media program currently being presented to an audience having one or more people, the media reaction determined based on sensor data that is passively sensed during the presentation and is of at least one of the one or more people [702 ];

determining that the person is not paying attention to the media program [704] based on the current media reaction; and

in response to determining that the person is not paying attention to the media program, controlling the presentation of the media program [706 ].

2. The computer-implemented method of claim 1, wherein controlling the presentation of the media program comprises pausing the media program or muting audio of the media program.

3. The computer-implemented method of claim 1, further comprising, after controlling the presentation of the media program:

receiving a second media response of the person;

determining that the person is paying attention or is ready to pay attention to the media program based on the second media reaction; and

in response to determining that the person is paying attention or is ready to pay attention to the media program, ceasing to control the presentation of the media program.

4. The computer-implemented method of claim 3, wherein controlling the presentation of the media program stops or pauses the presentation of the media program, and stopping controlling the presentation resumes the presentation of the media program at a beginning of a cohesive portion of the media program during which the stopping or pausing occurred.

5. A computer-implemented method, comprising:

receiving an identity or demographic of a person, the identity or demographic determined based on sensor data passively sensed during a current presentation of a media program [902 ];

determining to control the current presentation of the media program based on the identity or the demographic and information about the media program [904 ]; and

controlling the current presentation of the media program [906 ].

6. The computer-implemented method of claim 5, wherein controlling the current presentation stops or changes the current presentation.

7. The computer-implemented method of claim 5, wherein the identity or the demographic of the person indicates that the person is a minority group and the information indicates that the media program or the currently presented portion of the media program is not suitable for presentation to a minority group.

8. The computer-implemented method of claim 5, wherein the identity of the person or the demographic is the identity of the person, the method further comprising determining a user history associated with the identity, and wherein determining to control the current presentation of the media program is based on the user history.

9. The computer-implemented method of claim 8, wherein determining to control the current presentation determines to alter the current presentation to blur or reduce a visual portion of the presentation in resolution, but not stop the current presentation.

10. The computer-implemented method of claim 5, wherein controlling the current presentation controls a portion of the media program and the method further comprises ceasing to control the media program when presentation of the portion of the media program is complete.