CN1695111A - Declaratve markup for scoring multiple time-based assets and events within a scene composition system - Google Patents

Abstract

A system and method for declarative markup that allows temporal manipulation of media assets. The media assets can be audio media, video media, animations, audio-visual media, images or events. Using the, present invention a media sequence can be formed by playing more than one medium in series, in parallel or in any other temporal combination wherein a medium is cued to another medium. A media sequence created using the present invention can become part of a new media sequence, and the rate of playing the media sequence can be controlled (26E) by fields associated with the new media sequence. Also, using present invention, a media sequence can be cued to start playing at a fixed time before the end of a first media sequence, and in this instance the length of the first media sequence can be varied while still maintaining the fixed time from the end of the first media sequence.

Description

Descriptive tagging of multiple time-based assets and events for scoring within scene compositing systems

technical field

The present invention relates generally to modeling languages for 3D graphics, and more particularly to temporal manipulation of media assets.

Background technique

Conventional modeling languages for real-time 3D scene rendering have traditionally focused on scene structure, geometry, external features and (to some extent) on animation and interactivity. The aforementioned aspects of concentration are influenced by the following two factors. First, 3D computer graphics applications have been adapted for user-driven experiences so that they are easily structured around rendered responses to events. Second, most of these applications take a "reproduce it as fast as possible" approach to scene updates, paying less attention to timebase fidelity. Conventional modeling languages cannot provide the accuracy of the temporal relationship between two media assets. For example, if a video asset and an audio asset are to start at the same time, this can be achieved by specifying a start time for each asset independently of the other assets. This allows for slightly different start times. It is desirable to have the same field control the start time of each asset, thereby achieving accurate synchronization of the assets. Media assets include audio media, video media, animation, audiovisual media, images, or events.

Due to the incorporation of full motion video and high-fidelity audio into a hybrid rendering scenario, it is desirable to deliver a high-quality television-like viewing experience while supporting viewer interactivity. The hope is to provide a passive viewing experience, one that is more TV-like than web-like.

In declarative markup languages, the semantics required to achieve a desired output are implicit in the description of that output. It is not necessary to provide a separate process (ie, write a script) to obtain the desired output. An example of a declarative language is Hypertext Markup Language (HTML).

Various methods for scoring animation and playback have been previously developed in other computer-based media, including Macromedia Director and the W3C's Synchronized Multimedia Integration Language (SMIL). However, these existing scoring systems do not allow for the illustrative synthesis of real-time scenes, where independent scores are dynamically synthesized and decomposed hierarchically, employing a spatial scene graph-like fashion to construct time. For example, construct time blocks that are close to each other, or construct time blocks that are parallel (synchronous) to each other. Conventional scoring systems do not allow for declarative evaluation of fractional rates and directions of change, and they do not allow for declarative implementation of modulo calculation strategies in terms of standardized "piece complete" output, which lends itself to rapid compilation and reuse of properties animation.

Contents of the invention

A system and method for declarative tagging that allows temporal manipulation of media assets is presented. The media assets may be audio media, video media, animations, audiovisual media, images or events. Using the present invention, a sequence of media can be formed by playing more than one media in sequence, in parallel, or any other time combination, where media is inserted into another media. The media sequence created by using the present invention can become a part of a new media sequence, and the rate of playing the media sequence can be controlled through fields associated with the new media sequence. Furthermore, using the present invention, media sequences can be inserted to start playing at a fixed time before the end of the first media sequence, and in this case, the length of the first media sequence can be changed while still maintaining a distance from said first media sequence A fixed time to end.

Description of drawings

Figure 1A shows the basic architecture of Blendo.

FIG. 1B is a flow diagram illustrating the flow of content through the Blendo engine.

Figure 2A shows the temporal relationship between media sequences in a score.

Figure 2B illustrates synchronization of the media sequence of Figure 2 requiring preloading.

Figure 3 shows the temporal relationship between the various constituent media presented interactively.

Detailed ways

Blendo is an exemplary embodiment of the present invention that allows temporal manipulation of media assets, including controlling animation and visual imaging, and inserting audio media, video media, animation, and event data into the media asset being played. Figure 1A shows the basic Blendo architecture. A comprehensive description of Blendo can be found in Appendix A. At the core of the Blendo architecture is a core runtime (Core Runtime) module 10 (hereinafter referred to as the core), which provides various application program interface (hereinafter referred to as API) elements and object models to a set of objects existing in the system 11. During normal operation, the file is parsed by the analyzer 14 into a raw scene graph 16 and passed to the core 10, where its objects are given instances and a runtime scene graph is built. The objects may be built-in objects 18, author-defined objects 20, native objects 24, and the like. The object obtains platform services 32 using a set of managers 26 available. These platform services 32 include event handling, asset loading, media playback, and more. The object uses the rendering layer 28 to compose an intermediate or final image for display. The page integration component 30 is used to connect Blendo to the external environment, such as HTML or XML pages.

Blendo contains system objects according to this group manager 26 . Each manager 26 provides a set of APIs to control certain aspects of the system 11 . Event manager 26D provides access to input system events or environmental events initiated by user input. The load manager 26C facilitates the loading of Blendo files and native node implementations. Media manager 26E provides the ability to load, control and play audio, image and video media assets. The rendering manager 26G allows creation and management of objects for rendering scenes. The scene manager 26A controls the scene graph. Surface Manager 26F allows for the creation and management of surfaces upon which scene elements and other assets can be composited. Thread manager 26B gives authors the ability to create and control threads and communicate between them.

Figure 1B illustrates a conceptual description of content flow through the Blendo engine in a flowchart. In block 50, rendering begins from a source comprising a file or stream 34 (FIG. 1A) of content brought into analyzer 14 (FIG. 1A). The source may be in a text format like native VRML, a native binary format, an XML-based format, and the like. Regardless of the format of the source, in block 55 the source is converted to the original scene graph 16 (FIG. 1A). The raw scene graph 16 may represent nodes, fields and other objects in the content, as well as field initial values. A description of object prototypes, external prototype indexes, and routing declarations may also be included in the stream 34 .

The top level of the original scene graph 16 consists of nodes, top level fields and functions, prototypes and routes contained in the file. In addition to traditional components, Blendo also allows other fields and functions to be included at the top level. Use these to provide an interface to the external environment, such as HTML pages. They also provide an object interface when streams 34 are used as the content of external prototypes.

Each primitive node includes a list of fields initialized within its environment. Each primitive field entry includes the field's name, type (if given), and data value. Each data value includes numbers, strings, primitive nodes, and/or primitive fields, which can represent a field value of an unambiguous type.

In block 60, the prototype is extracted from the top level of the original scene graph 16 (FIG. 1A) and used to populate a database of object prototypes accessible by the scene.

The raw scene graph 16 is then sent through the build traversal. During this traversal, each object is built using the database of object prototypes (block 65).

In block 70 , the route is established in flow 34 . Then in block 75, each field in the scene is initialized. This is done by sending initial events to non-default fields of the object. Since the scene graph structure is implemented by using node fields, block 75 also constructs the scene hierarchy. Events are fired using in-order traversal. The first node encountered counts the fields in that node. If the field is a node, then that node is traversed first.

As a result, nodes on a particular branch of the tree are initialized. Then, send the event to the node field with the initial value of the node field.

After the fields of a given node have been initialized, the author is allowed to add initialization logic (block 80) to the prototype object to ensure that the node is fully initialized when called. The above blocks generate the root scene. In block 85, the scene is delivered to the scene manager 26A (FIG. 1A) created for the scene. In block 90, the scene manager 26A is used to render and perform property processing either implicitly or under author control.

Scenes rendered by the scene manager 26A may be constructed using objects from the Blendo object hierarchy. Appendix B shows the object hierarchy and provides a detailed description of the objects in Blendo. Objects can derive some of their functionality from their parent objects, and subsequently extend or modify their functionality. At the base of the hierarchy is the Object. The two main classes of objects derived from said object are nodes and fields. where the node contains a render method that is called as part of the render traversal. The data attributes of a node are called fields. Between the Blendo object hierarchy are object classes called Timing Objects, described in detail below. The following code sections are for exemplary purposes. It should be noted that the line numbers in each code section only represent the line numbers of that particular code section, not the line numbers in the original source code.

timing object

Timing objects include TimeBase nodes. It is included as a field of the Timing node and provides the media with a common set of timing semantics. Through node instances, the TimeBase node can be used for many related media nodes in order to ensure time synchronization. Blendo also provides a set of nodes including Scores, which are used to sort media events. The Score is a timing node and its timing is derived from TimeBase. The Score includes a list of Cue nodes that emit events at specified times. Various timing objects including Score are described below.

TimedNodes

The following code part illustrates the TimeNode node. Functions in the nodes are described hereafter.

1) TimedNode: ChildNode

2) field TimeBaseNode timeBase NULL

3) function Time getDuration()

4) function void updateStartTime(Time now, Time mediaTime, Float rate)

5) function void updateStopTime(Time now, Time mediaTime, Float rate)

6) function void updateMediaTime(Time now, Time mediaTime, Float rate)

}

This object is the parent node of all nodes controlled by TimeBaseNode.

In line 2 of the code section, the TimeBase field contains the controlling TimeBaseNode which makes the following appropriate function calls when the time base is started, stopped or advanced.

In line 3, the getDuration function returns the duration of the TimedNode. If not available, then return value -1. This function will normally be overridden by the exported object.

Line 4 lists the updateStartTime function. When called, the function begins advancing its event or the media it controls, with a start offset specified by the mediaTime variable. The updateStartTime function will generally be overridden by the exported object.

Line 5 lists the updateStopTime function, which, when called, stops advancing its events or the media it controls. This function will normally be overridden by the exported object.

In line 6, the updateMediaTime function is called whenever mediaTime is updated by the TimeBaseNode. Use the updateMediaTime function by exported objects to further control their media or send additional events.

IntervalSensor

The following code section illustrates the IntervalSensor node. Fields in the nodes are described hereafter.

1) IntervalSensor: TimedNode

2) field Time cycle Interval 1

3) field Float fraction 0

4) field Float time 0

}

As time elapses, the IntervalSensor node generates events.

IntervalSensor nodes can be used for many purposes including but not limited to:

Drive continuous simulation and animation;

Controlling periodic activity (e.g., once every minute);

• Start a single occurring event such as an alarm bell.

When the updateStartTime() function of the IntervalSensor node is called, the IntervalSensor node sends the initial segment and time events. The node also sends a fragment and time event each time updateMediaTime() is called. Finally, when the updateStopTime() function is called, the final segment and time events are sent.

In line 2 of the code section, the cycleInterval field is set by the author to determine the length of time in seconds that progresses the segment from 0 to 1 . This value is returned when the getDuration() function is called.

Line 3 lists the Fragment fields that generate an event whenever the TimeBaseNode runs using the following equation (1), which is:

fraction=max(min(mediaTime/cycleInterval, 1), 0) Equation (1)

Line 4 lists the Time field, which generates an event whenever the TimeBaseNode runs. The value of the time field is the current wall clock time.

Score

The following code section illustrates the Score node. Fields in the nodes are described hereafter.

1) Score: TimedNode

2) field MF CueNode cue[]

}

This object calls each entry in the insert field for each updateStartTime(), updateMediaTime(), updateStopTime() call received. The call for each inserted entry returns the current accumulated relative time. This value is passed to subsequent inserted entries to allow calculation of relative offsets between inserted entries.

In line 2 of the code section, the insert field holds a list of CueNode entries to be invoked with the mediaTime passed.

TimeBaseNode

The following code part illustrates the TimeBaseNode node. Fields and functions in the nodes are described hereafter.

1) TimeBaseNode: Node

2) rield Time mediaTime 0

3) function void evaluate(Time time)

4) function void addClient(TimedNode node)

5) function void removeClient(TimedNode node)

6) function Int32 getNumClients()

7) function TimedNode getClient(Int32 index)

}

The object is the parent node of all nodes that generate media Time.

Line 2 of the code section lists the mediaTime field, which generates an event whenever mediaTime advances. Typically the MediaTime field is controlled by the exported object.

Line 3 lists the evaluation function that is called by the Scene Manager when time advances if the TimeBaseNode is registered to be interested in receiving time events.

Line 4 lists the addClient function that is called by each TimedNode when the TimeBaseNode is set in its timeBase field. Each client in the list is called when mediaTime starts, advances or stops. If the passed node is already a client, this function does nothing.

Line 5 lists the removeClient function that is called by each TimedNode when the TimeBaseNode is no longer set in its timeBase field. If the transferred node is not in the client list, then this function does nothing.

Line 6 lists the getNumClients function, which returns the number of clients currently in the client list.

Line 7 lists the getClient function, which returns the client at the passed index. If the index is out of bounds, a NULL value is returned.

TimeBase

The following code section illustrates the TimeBase node. Fields in the nodes are described hereafter.

           
1) TimeBase: TimeBaseNode

2) field Bool loop false

3) field Time startTime 0

4) field Time playTime 0

5) field Time stopTime 0

6) field Time mediaStartTime 0

7) field Time mediaStopTime 0

8) field Float rate 1

9) field Time duration 0

10) field Bool enabled true

11) field Bool isActive false

}

        

This object controls the advancement of mediaTime. TimeBase can start, stop and resume the value and make mediaTime loop continuously. TimeBase allows mediaTime to play within a subset of its range.

In line 2 of the code section, the loop field controls whether mediaTime repeats its advancement when mediaTime reaches the end of its run.

In line 3, the startTime field controls when the mediaTime starts advancing. The TimeBase starts running when the startTime in units of wall-clock time is reached. True as long as stopTime is less than startTime. When this happens, if the rate is greater than or equal to 0, set mediaTime to the value of mediaStartTime. If mediaStartTime is out of range (see description for valid ranges of mediaStartTime), then mediaTime is set to 0. If the rate is less than 0, set mediaTime to mediaStopTime. If mediaStopTime is out of range, set mediaTime to the duration. TimeBase continues to run until stopTime is reached, or mediaStopTime (mediaStartTime, if rate is less than 0) is reached. If the startTime event is received while the TimeBase is running, it is ignored.

In lines 4 and 5, the playTime field works the same as startTime, except mediaTime is not reset when activated. The playTime field allows mediaTime to continue advancing after the TimeBase was stopped with stopTime. If playTime and startTime have the same value, then startTime dominates. If a playTime event is received while the TimeBase is running, the event is ignored. The stopTime field controls when the TimeBase stops.

In line 6, the mediaStartTime field sets the start of the mediaTime subrange of the media duration to run. The range of mediaStartTime is from zero to the end of the duration (0..duration). If the value of the mediaStartTime field is out of range, then 0 is substituted in its place.

In line 7, the mediaStopTime field sets mediaTime to run to the end of the subrange of the media duration. The range of mediaStopTime is from zero to the end of the duration (0..duration). If the value of mediaStopTime is out of range, a duration is substituted in its place.

In line 8, the rate field allows mediaTime to run at a rate other than one-half a second of wall-clock time. The rate provided in the rate field is used as the instantaneous rate. When the call evaluates, the time elapsed since the last call is multiplied by the rate, and the result is added to the current mediaTime.

In line 9, the duration field generates an event when the durations of all clients of this TimeBase determine their duration. The duration field has the same value as the client with the longest duration.

In line 10, the enable field allows TimeBase operation. When the enable field becomes false, isActive becomes false if it was true, and mediaTime stops advancing. When false, startTime and playTime are ignored. When the enable field becomes true, startTime and playTime are evaluated to determine if the TimeBase should start running. If yes, then perform the behavior as described in startTime or playTime.

Line 11 lists the isActive field, which generates a true event when TimeBase becomes active, and a false event when timeBase becomes invalid.

CueNode

The following code snippet illustrates the CueNode node. Fields in the nodes are described hereafter.

           
1) CueNode: Node

2) field Float offset-1

3) field Float delay 0

4) field Bool enabled true

5) field Int32 direction 0

6) function void updateStartTime(Time now, TimemediaTime, Float rate)

7) function void updateStopTime(Time now, TimemediaTime, Float rate)

8) function Time evaluate(Time accumulated, Time now, TimemediaTime,
Float rate)

9) function Time getAccumulatedTime(Time accumulated)

10) function void fire(Time now, TimemediaTime)

}

        

This object is the parent object of all objects in Score's insertion list.

In line 2 of the code section, the offset field establishes a relative offset of 0 from the beginning of the sequence. For example, a value of 5 will fire a CueNode when the input mediaTime reaches a value of 5.

In line 3, the delay field establishes the relative delay before the CueNode fires. If offset is a value other than -1 (default), the delay is measured against the offset. Otherwise, the delay is measured against the end of the previous CueNode or against 0 if this is the first CueNode. For example, if the offset has a value of 5 and the delay has a value of 2, then the node will fire when mediaTime reaches 7. If the offset has a value of -1 and the delay has a value of 2, then this node will fire 2 seconds after the previous CueNode ended.

In line 4, if the enable field is false, disable the CueNode. The CueNodes behave as if the offset and delay were at their default values, and do not fire events. If it is true, then the CueNode is functioning normally.

In line 5, the direction field controls how this node fires relative to the mediaTime's travel direction. If this field is 0, then the node fires when the node's offset and/or delay is reached, regardless of whether mediaTime is increasing (rate greater than zero) or decreasing (rate less than zero). If the direction field is less than zero, then the node will only fire if the mediaTime is lowered, the offset and/or the delay to reach said node. If the direction field is greater than zero, the node fires only when the mediaTime is incremented, and the offset and/or delay to the node is reached.

Line 6 lists the updateStartTime function that is called when the parent Score receives an updateStartTime() function call. Call each CueNode in turn.

Line 7 lists the updateStopTime function that is called when the parent Score receives an updateStopTime() function call. Call each CueNode in turn.

Line 8 lists the evaluation function that is called when the parent Score receives an updateMediaTime() function call. Each CueNode is called in turn and must return its cumulative time. For example, if the offset is 5 and the delay is 2, then the CueNode will always return a value of 7. If the offset is -1 and the delay is 2, then the CueNode will always return the accumulated time of the input plus 2. This is the default behavior. Certain CueNodes (such as IntervalCues) have well-defined durations and firing times.

In line 9, the getAccumulatedTime function returns the accumulated time using the same calculation as in the evaluate() function.

Line 10 lists the fire function that is called according to the default evaluate() function when the CueNode reaches its fire time. The activation function is intended to be overridden by a specific derived object in order to perform the appropriate action.

MediaCue

The following code section illustrates the MediaCue node. Fields in the nodes are described hereafter.

1) MediaCue: CueNode TimeBaseNode{

2) field Time mediaStartTime 0

3) field Time mediaStopTime 0

4) field Time duration 0

5) field Bool is Active false

}

When this CueNode is active, the object controls advancing mediaTime. MediaCue allows mediaTime to play within a subset of its range. The MediaCue is valid from the time determined by the Offset and/or Delay fields and for the length of time determined by mediaStopTime minus mediaStartTime. The value MediaCue returns from getAccumulatedTime() is the value calculated by adding the default function to mediaStopTime and subtracting mediaStartTime. When active this node generates a mediaTime, which is calculated by subtracting the fire time from which the mediaStartTime from the input mediaTime has been added. So the MediaCue advances the mediaTime at the same rate as the input mediaTime.

In line 2 of the code section, the mediaStartTime field sets the start of the mediaTime subrange for which the media duration will run. The range of mediaStartTime is from zero to the end of the duration (0..duration). If the value of the mediaStartTime field is out of range, then 0 is used in its place.

In line 3, the mediaStopTime field sets mediaTime to run to the end of the subrange of media durations. The range of mediaStopTime is from zero to the end of the duration (0..duration). If the value of the mediaStopTime field is out of range, a duration is used in its place instead.

In line 4, the duration field generates an event when the durations of all clients of this TimeBaseNode determine their duration. The duration field has the same value as the client with the longest duration.

Line 5 lists the isActive field, which generates a true event when the node becomes active, and a false event when the node becomes inactive.

IntervalCue

The following code section illustrates the IntervalCue node. Fields in the nodes are described hereafter.

1) IntervalCue: CueNode

2) field Float period 1

3) field Bool rampUp true

4) field Float fraction 0

5) field Bool isActive false

}

As time progresses, this object sends fragment events from 0 to 1 (or from 1 to 0 if rampUp is false).

Line 2 of the code snippet lists the period field, which determines the time in which the snippet is ramped.

In line 3, if the rampUp field is true (default), then the segment ramps up from 0 to 1 for the duration of the IntervalCue. If false, then the slice goes down from 1 to 0. If mediaTime runs backwards (when the rate is less than zero), then the clip goes down from 1 to 0 when the rampUp field is true, and goes up from 0 to 1 when the rampUp field is false .

In line 4, the fragment field sends an event to evaluate() with each call while the node is valid. If mediaTime advances, the fragment starts outputting when the node fires, and stops outputting when the node reaches its firetime plus the period. Describe the fragment value as:

fraction=(mediaTime-firing time)*period Equation (2)

Line 5 lists the node isActive field, which sends a true event when the node becomes active, and a false event when the node becomes inactive. If mediaTime advances, the node becomes active when mediaTime becomes greater than or equal to firetime. When the mediaTime becomes greater than or equal to the firing time plus the period, the node becomes invalid. If mediaTime goes backwards, the node becomes active when mediaTime becomes less than or equal to the fire time plus the period, and becomes inactive when mediaTime becomes less than or equal to the fire time. The firing of these events is affected by the direction field.

FieldCue

The following code section illustrates the FieldCue node. Fields in the nodes are described hereafter.

1) FieldCue: CueNode

2) field Field cueValue NULL

3) field Field cueOut NULL

}

When FieldCue fires, this object sends cueValue as an event to cueOut. FieldCue allows setting and emitting any field type. A cueOut value can be routed to any type of field. If the current type of cueValue is inconsistent with the destination field, undefined results may occur.

In line 2 of the code section, the cueValue field is the edited value that is emitted when the node fires.

Line 3 lists the cueOut field that sends an event with a value of cueValue when this node fires.

TimeCue

The following code section exemplifies the TimeCue node. Fields in the nodes are described hereafter.

1) TimeCue: CueNode

2) field Time cueTime 0

}

When TimeCue fires, this object sends the current wall-clock time as an event to cueTime.

Line 2 of the code section lists the cueTime field that sends an event with the current wall clock time when this node fires.

The scoring structure within the context of real-time scene composition enables authors to descriptively describe the timing control for most rendering and playback techniques, including: image flipbooks and image synthesis animations (e.g., animated GIFs); video and Audio clips and streams; geometric animation clips and streams, such as chain transformations, geometry, and structural coordinates; animation of rendering parameters, such as brightness, blur, and transparency; modulation of parameters for behavioral, simulation, or regeneration systems; and dynamic control assets Loading, event routing and logic functions. For example, the following example emits a string to preload an image asset, then performs an animation using that image, and then runs a movie. It is also possible to run the string in the example below in reverse (ie first the movie plays backwards, then the animation plays in reverse and the image disappears).

           
1) Score {

2) timeBase DEF TB TimeBase {}

3) cue [

4) Fieldcue {

5) cueValue String " "

6) cueOut TO ISURF.URL

7) direction-1

  8)        }

9) FieldCue{

10) cueValue String "imagel.png"

11) cueOut TO ISURF.url

12) direction-10
        <!-- SIPO <DP n="14"> -->
        <dp n="d14"/>
13) }

14) IntervalCue{

15) delay 0.5

16) period 2.5 #2.5 second animation

17) fraction TO PI. fraction

18) }

19) DEF MC MediaCue {

20) offset 2

  twenty one)        }

22) Fieldcue {

23) cueValue String " "

24) cueOut TO ISURF.URL

25) direction 1

26) delay -0.5

27) }

28) Fieldcue {

29) cueValue String "imagel.png"

30) cueOut TO ISURF.URL

31) direction-1

32) delay -0.5

33) }

34) ]

35) }

36) #Slide out image

37) DEF T Transform {

38) children Shape {

39) appearance Appearance {

40) texture Texture {

41) surface DEF ISURF ImageSurface { }

42) }

43) }

44) geometry IndexedFaceSet {...}

45) }

46) }

47) DEF PI Position Interpolator {

48) key...

49) keyValue...

50) value TO T. translation

51) }

52) #Movie

53) Shape {

54) appearance Appearance{

55) texture Texture{

56) surface MovieSurface {

57) url "myMovie.mpg"
        <!-- SIPO <DP n="15"> -->
        <dp n="d15"/>
58) timeBase USE MC

59) }

60) }

61) }

62) geometry IndexedFaceSet {...}

63) }

        

All Cue nodes in the Score fire relative to the media time of the TimeBase, providing a common index and thereby creating an accurate relationship between the timing of the various media assets. In the code snippet above, once the TimeBase starts, the FieldCue (line 9) fires because the FieldCue has default offset and delay fields whereby the image appears. Lines 35-45 of the code section load the image on the surface (200, Figure 2A). The IntervalCue (line 13) then starts 0.5 seconds later and runs for the next 2.5 seconds, incrementing its segment output from 0 to 1. Firing of the IntervalCue starts animation of the image (202, Figure 2A). Lines 46-50 control the animation. The MediaCue (line 18) starts 2 seconds after the TimeBase starts, or when the IntervalCue is 1.5 seconds into its animation, thereby starting the movie. Lines 51-62 load the first frame of the movie on the surface (204, Figure 2A). When the string is played backwards, first the movie is played in reverse. The image then appears 0.5 seconds later and the animation starts 0.5 seconds after the image appears. When the animation stops and 0.5 seconds after the image disappears, play the animation in reverse for 2.5 seconds. This example shows that Cues can be offset from each other or from a TimeBase, and that a subsequent Cue can start before the previous one ends.

The MediaCue provides synchronization tools for authors. MediaCue is a form of Cue that works just like TimeBase. In fact, MediaCue can be used wherever TimeBase can be used, as shown in the above example. But since the MediaCue is embedded in the time sequence of events, there is enough information in the implementation to request a preloaded asset. Figure 2B illustrates synchronization of the media sequence of Figure 2A requiring preloading. For example, in the above example, if the implementation knows that the movie took 0.5 seconds to preload and play immediately, then after the TimeBase starts and after waiting (block 210) 1.5 seconds, in block 215, send a "ready "Signal. Once the ready signal is received, the movie is preloaded in block 220 . This tends to give it the requested 0.5 seconds to preload. In block 225, a request to start is received, and once the request to start is received, block 230 starts the movie immediately.

It is possible to create presentations with complex timing thanks to the combination of TimeBase and the media sequencing capabilities allowed in Blendo. Figure 3 shows the time relationship of various components presented by Blendo. When the viewer selects a news presentation (360), the viewer sees a screen where he can select a story (362). When the user selects story S3 from a selection of five stories S1, S2, S3, S4 and S5, a welcome screen with the announcer is displayed (364). On the welcome screen, the viewer can choose to switch to another story (374), thereby aborting story S3. After the welcome statement, the screen changes to the story position (366) and the selected story is played (368). Here, the viewer can turn to the next story, previous story, replay the current story or choose to play an extended version of story S3 (370), or jump to eg another story S5 (372). After playing the selected story, the user can make the next selection.

It should be understood that the present invention is not dependent on Blendo, and that it may be part of a Blendo-independent embodiment. It should also be understood that the present invention is equally applicable to 2D scene rendering and 3D scene rendering.

While particular embodiments of the invention have been described, it will be obvious to those skilled in the art that changes and modifications may be made in its broader aspects without departing from the invention in its broader aspects, and therefore, the appended claims All such changes and modifications which fall within the true spirit and scope of the invention are intended to be embraced within its scope.

Claims (32)

1. a computer-readable medium comprises computer instruction, is used for:

Come illustrative ground to synthesize the first time-based media sequence according to first media sequence, second media sequence; Wherein said second media sequence is followed described first media sequence; And

Begin second media sequence in variable time with respect to described first media sequence.

2. computer-readable medium as claimed in claim 1 also comprises the computer instruction that is used for illustrative ground control animation.

3. computer-readable medium as claimed in claim 1 also comprises the computer instruction that is used for the synthetic described first time-based media sequence, wherein can be integrated into first to incident based in the time series in real time.

4. computer system as claimed in claim 1 also comprises:

Be used to receive the computer instruction of first rate value, the speed of the described first time-based media sequence of described first rate value controls playing,

Wherein the variation according to the first rate value changes the speed of playing the described first time-based media sequence.

5. computer system as claimed in claim 1 also comprises:

The computer instruction that is used for the synthetic second time-based media sequence; And

Be used to receive the computer instruction of second rate value, the speed of the described first time-based media sequence of the described second rate value controls playing; Wherein

The described first time-based media sequence is the part of the described second time-based media sequence, and changes the speed of playing the described first time-based media sequence according to the variation of described second rate value.

6. computer-readable medium as claimed in claim 1 also comprises:

Be used to receive the computer instruction of third speed value, the speed of described second media sequence of described third speed value controls playing;

Wherein the variation according to the third speed value changes the speed of playing described second media sequence.

7. computer-readable medium as claimed in claim 1 is wherein play described first media sequence on the subclass of duration of first media sequence.

8. computer-readable medium as claimed in claim 1 wherein postpones the broadcast of second media sequence according to the end of playing first media sequence.

9. computer-readable medium as claimed in claim 1 wherein can be play described time-based sequence forward or backwards.

10. computer-readable medium as claimed in claim 1 also comprises:

Be used for illustrative ground and receive instruction so that the time before the end of described second media sequence begins the computer instruction of the 3rd media sequence.

11. computer-readable medium as claimed in claim 10, wherein the length of second media sequence is not by user's appointment.

12. computer-readable medium as claimed in claim 1 also comprises:

Be used for from jump to the computer instruction of the second place on the described first time-based media sequence in the primary importance on the first time-based media sequence.

13. computer-readable medium as claimed in claim 12, wherein said primary importance on described first media sequence and the described second place on described second media sequence.

14. computer-readable medium as claimed in claim 1, wherein said computer instruction are used to begin to load described second media sequence.

15. computer-readable medium as claimed in claim 1, wherein said computer instruction are used to begin to play described second media sequence.

16. one kind is used for the method that illustrative ground synthesizes the first time-based sequence, described method comprises:

Come illustrative ground to synthesize the first time-based sequence according to first media sequence and second media sequence; Wherein

Second media sequence illustrative ground is inserted described first media sequence.

17. method as claimed in claim 16 is wherein selected described first media sequence and described second media sequence from the group of being made of video sequence, tonic train, animation or audiovisual sequence.

18. method as claimed in claim 16, wherein described first media sequence of prestrain and described second media sequence are so that can begin immediately in the medium of each prestrain.

19. method as claimed in claim 16 is wherein being inserted second media sequence with respect to the variable time of described first media sequence.

20. an illustrative ground synthesizes the method for the first time-based media sequence, described method comprises:

Play first media sequence and second media sequence so that form the described first time-based media sequence;

Play described the 3rd medium with described first concurrently based on time series.

21. method as claimed in claim 20 also comprises:

Illustrative ground control visual picture, described image are the parts of the media sequence selected from the group of being made up of first media sequence, second media sequence and the 3rd media sequence.

22. method as claimed in claim 21 also comprises:

Provide the first rate value, the speed of the described first time-based media sequence of described first rate value controls playing; And

Variation according to the first rate value changes the speed of playing the described first time-based media sequence.

23. method as claimed in claim 21 also comprises:

Provide second rate value that is associated with the second time-based media sequence, the speed of the described first time-based media sequence of the described second rate value controls playing; And

Variation according to second rate value changes the speed of playing the described first time-based media sequence; Wherein

The described first time-based media sequence is the part of the described second time-based media sequence.

24. method as claimed in claim 21 also comprises:

Provide the third speed value, the speed of described second media sequence of described third speed value controls playing; And

Variation according to the third speed value changes the speed of playing described second media sequence.

25. method as claimed in claim 21 also comprises:

In the subclass of duration of first media sequence, play described first media sequence.

26. method as claimed in claim 21 also comprises:

Postpone to play second media sequence according to the end of playing first media sequence.

27. method as claimed in claim 21 also comprises:

The reverse-play described first time-based media sequence.

28. method as claimed in claim 21 also comprises:

Fourth media sequence is provided; And

Before finishing, described the 3rd media sequence loads described fourth media sequence; Wherein

Any length for described the 3rd media sequence loaded described fourth media sequence before described the 3rd media sequence finishes.

29. computer system as claimed in claim 21 also comprises:

From jump to the second place on the described first time-based media sequence in the primary importance on the first time-based media sequence.

30. method as claimed in claim 29, wherein said primary importance on described first media sequence and the described second place on described second media sequence.

31. the method that synthetic medium present, described method comprises:

First media sequence and second media sequence are provided; And

How the explanation that provides at least one to be used to define timing relationship, described explanation makes in variable time that to play first media sequence relevant with in second media sequence each if having defined.

32. method as claimed in claim 31 is wherein selected described first media sequence and described second media sequence from the group of being made of video sequence, tonic train, animation or audiovisual sequence.

Images