[go: up one dir, main page]

WO2000043999A2 - Method and apparatus for synchronizing playback of multiple media types over networks having different transmission characteristics - Google Patents

Method and apparatus for synchronizing playback of multiple media types over networks having different transmission characteristics Download PDF

Info

Publication number
WO2000043999A2
WO2000043999A2 PCT/US2000/001232 US0001232W WO0043999A2 WO 2000043999 A2 WO2000043999 A2 WO 2000043999A2 US 0001232 W US0001232 W US 0001232W WO 0043999 A2 WO0043999 A2 WO 0043999A2
Authority
WO
WIPO (PCT)
Prior art keywords
data
media
primary
primary media
secondary media
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2000/001232
Other languages
French (fr)
Other versions
WO2000043999A3 (en
Inventor
Rolf Henri Toft
Masahiro Nakano
Robert Bradley Rosin
Toshiya Fujii
Kent Lawrence Headley
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Electronics Inc
Original Assignee
Sony Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Electronics Inc filed Critical Sony Electronics Inc
Priority to AU24158/00A priority Critical patent/AU2415800A/en
Publication of WO2000043999A2 publication Critical patent/WO2000043999A2/en
Publication of WO2000043999A3 publication Critical patent/WO2000043999A3/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10—Digital recording or reproducing
    • G11B20/10527—Audio or video recording; Data buffering arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4782—Web browsing, e.g. WebTV
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/858—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/858—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
    • H04N21/8586—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot by using a URL
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/16—Analogue secrecy systems; Analogue subscription systems
    • H04N7/162—Authorising the user terminal, e.g. by paying; Registering the use of a subscription channel, e.g. billing
    • H04N7/163—Authorising the user terminal, e.g. by paying; Registering the use of a subscription channel, e.g. billing by receiver means only
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B2220/00—Record carriers by type
    • G11B2220/20—Disc-shaped record carriers
    • G11B2220/25—Disc-shaped record carriers characterised in that the disc is based on a specific recording technology
    • G11B2220/2537—Optical discs
    • G11B2220/2545—CDs

Definitions

  • This invention generally relates to consumer device control systems and more particularly, to a method and apparatus for synchronizing playback of multiple media types over networks with different transmission characteristics.
  • the audio-video systems include audio devices that can playback sounds from audio media such as compact disks (CD), minidisks, tape cassettes, and records.
  • the video devices in these systems can display images on a display screen such as a television set and generate accompanying voice, music, or sound.
  • Video information can be stored on video media such as video cassettes, digitally encoded laser disks, and digital video disks (DVD).
  • the audio and video devices are generally connected to the audio-video system using a local bus or other direct connection. These devices transmit audio and connection is reserved for use by the directly attached audio and video devices. As a result, there is a relatively short time period between requesting playback of media and the time playback of the media begins. Further, the delivery rate of the audio and video information is predictable because the direct connection delivers information to the audio-video system at essentially a constant transmission rate. For example, a CD player can deliver audio information over a direct connection to an audio-video system at a constant transmission rate because bandwidth on the direct connection is not shared with other devices. By connecting an audio-video system to a network such as the Internet, many more audio and video titles are available for playback.
  • the Internet provides an everchanging variety of media and is not limited to the media stored on traditional devices such as CDS, videos, laser disks, and minidisks.
  • media delivered over a network like the Internet may include variable delays and transmit data at variable data rates. Variations in delivery may occur because the audio or video information being transmitted has different compression/decompression methods that take different times periods to process or because the information is delivered by computer systems transmitting information over networks with different bandwidths. As an additional factor, data delivery rates may fluctuate on public networks such as the Internet as thousands of users sporadically access the Internet and consume the available bandwidth.
  • a method for synchronizing the playback of data on a primary media with data on a secondary media is provided.
  • the data on the primary media is identified using a unique identifier.
  • the unique identifier is generated based upon information associated with data on the primary media such as the table of contents.
  • This unique identifier is used to locate data on the secondary media stored remotely over a network.
  • data on the secondary media could be a hypertext link associated with a particular track on a compact disk (CD).
  • Data on the secondary media matching the unique identifier is downloaded over the network and cached in a local storage area. Timing information is then used to synchronize playback of data on the primary data on media with the cached data from the secondary media.
  • Another aspect of the present invention provides a method of using the timing information to synchronize playback of data from the primary media with data from the secondary media.
  • data on the primary media is identified using a unique identifier.
  • Data from the secondary media cached on a local storage device is located using the unique identifier.
  • Timing information associated with the data from secondary media is used to set triggers for launching data from the secondary media.
  • the time period for launching data from the secondary media is based upon an elapsed time period data on the primary media has played. Playing data from the primary media over a time period triggers interrupts, launches data on the secondary media, and synchronizes playback of data from the secondary media with the data on primary media.
  • FIG. 1 is a block diagram of an audio-video system consistent with the present invention including data from primary media and secondary media;
  • FIG. 2 is a block diagram of a set-top device shown in FIG. 1 consistent with the present invention
  • FIG. 3 is a flowchart consistent with the present invention for playing data on secondary media downloaded from a network with data on primary media stored on a local audio- video device;
  • FIG. 4 is a flowchart for synchronizing the playback of data on a secondary media and data on a primary media consistent with the present invention
  • FIG. 5 is a block diagram displaying one technique for organizing synchronization information associated with data on the primary media and secondary media consistent with one implementation of the present invention.
  • FIG. 1 is a block diagram of an audio-video system 100 operatively coupled to a network capable of combining various media into a multimedia presentation.
  • Audiovideo system 100 includes audio-video devices 102, a display device 104, a set-top device 106, an input device 107 for communicating with set-top device 106, and a network 108 accessible to set-top device 106.
  • a server 110 is operatively coupled to network 108 and a remote database with secondary media 1 12.
  • a local database stored on primary media 1 14 is operatively coupled to audio-video devices 102.
  • Network 108 can include the Internet and computer resources associated with the Internet.
  • Audio-video devices 102 include devices capable of playing different audio and video media.
  • these devices may include a compact disk player, a video tape player, a DVD player, a mini disk player, a television receiver, an audio receiver, and an audio playback device such as a stereo receiver/amplifier and audio tape player.
  • These devices can be connected together using a bus 103 such as IEEE- 1394 ("Firewire") or Universal Serial Bus (USB).
  • Bus 103 transmits data stored on a media quickly between set- top device 106 and audio-video devices 102. Because each device connected to bus 103 can be uniquely addressed, multiple identical devices can be configured into the same system. For example, DVD players could be separately addressed on bus 103 as DVD-1 and DVD-2.
  • Video images are displayed on display device 104 when a portion of data on the media includes visual display information.
  • a television tuner or video playback device in audio-video devices 102 generates images displayed on display device 104.
  • the television tuner or video playback device may also generate audio information played back through an audio device such as a stereo receiver/amplifier and audio tape player.
  • Display device 104 can be any device used to display images such as a computer screen, a television, or a video-monitor device.
  • Set-top device, or box, (STB) 106 is a computer-based device used to control audio- video devices 102 and display device 104.
  • set-top device 106 has access to server 110 and secondary media 112 through network 108.
  • set-top device 106 coordinates playback of data on primary media 114 and secondary media 112.
  • Primary media 114 is typically a storage medium such as a CD-ROM associated with an audio-video device while secondary media 112 is a storage medium such as a hard-disk associated with server 100.
  • a communication protocol enables STB 106 to communicate with audio-video devices 102 and display device 104.
  • STB 106 sends commands to audio-video devices 102 to play the audio media, video media, and other information associated with primary media 114 and secondary media 112. These commands can be in the format of the control protocol called Control A-l or S-Link developed by the Sony Corporation based in Tokyo, Japan.
  • Control A-l or S-Link developed by the Sony Corporation based in Tokyo, Japan.
  • other control protocols such as Home Audio Video Interoperability Architecture (HAVi) can be used along with other control protocols that are compatible with audio-video devices 102 and display device 104.
  • HAVi was developed by a consortium including many consumer electronics companies including Sony, Hitachi, Matsushita, Phillips, Toshiba. Sharp, and Grundig.
  • HAVi Home/Nideo Interoperability
  • Input device 107 can communicate with STB 106 through a wireless communication mechanism using infrared signals or radio frequency signals. This allows a user to operate STB 106 and accompanying audio-video devices 102 without cumbersome wires.
  • Input device 107 can be a keyboard, a keyboard and a mouse, a hand-held remote control device, or any other input device useful in controlling STB 106.
  • server 110 facilitates access to secondary media 112.
  • Secondary media 112 includes data or information related to data on primary media 114.
  • data on secondary media 112 may include hypertext links to locations on the Internet, static images, or video clips that compliment and enhance the playback of data on primary media 114.
  • Server 110 transmits this data from secondary media 112 through network 108 to STB 106.
  • STB 106 stores data from the secondary media internally in a local database on a storage device such as a hard disk or other cache device. STB 106 then coordinates the playback of data on primary media 114 and data from secondary media 112.
  • Coordination is necessary because network 108 transmits data stored on secondary media 112 at a slower transmission rate to STB 106 than data transmitted between audio-video-devices 102, primary media 114, and STB 106.
  • a local storage area such as cache
  • implementations of the present invention provide a technique for synchronizing the playback of data from secondary media 1 12 transmitted over network 108 and data from primary media 114 stored on audio-video devices 102.
  • Data transmitted over a slower transmission link, such as network 108 is cached in a local database and played back at a predetermined time corresponding to the playback of data transmitted over a faster transmission link, such as bus 103, thus synchronizing the playback of data on secondary media 112 and primary media 114. Additional information on synchronizing data provided over transmission links with different transmission speeds and operating characteristics is discussed in further detail later herein.
  • FIG. 2 is a block diagram of STB 106 designed in accordance with the present invention.
  • STB 106 includes a processor 202, an input-output interface 204, a network interconnection 206, a wireless interface 207, a memory 208, an audio-video interface 210, a secondary storage 212, and an internal bus 214 facilitating communication between each of the aforementioned elements.
  • Processor 202 can be a general purpose processor such as an Intel Pentium processor or an application specific integrated circuit (ASIC) designed to execute one or more applications.
  • processor 202 can be an ASIC designed specifically to run web browser software compatible with the World Wide Web (WWW).
  • Input-output interface 204 can be connected to a number of different peripheral devices such as an external storage device or a printer.
  • Network interconnect 206 may be a modem, cable modem, or other device to provide an interface between a network such as the Internet and STB 106.
  • Wireless interface 207 facilitates receiving infra-red (IR) or radio frequency (RF) signals from input devices not physically attached to STB- 106.
  • IR infra-red
  • RF radio frequency
  • a keyboard device and mouse can communicate with set-top box 106 through a wireless infrared interface without the use of cables or other physical attachments.
  • Memory 208 includes a user interface 218. a media download module 220. a media cache module 222, a media synchronization module 224, synchronization data 226, a local database application 228. a remote database application 230, a development kit 232, and a run time environment 234.
  • User interface 218 is a graphical user interface supporting hypertext links and suitable for displaying and controlling the playback of data from multiple media types in accordance with the present invention.
  • user interface 218 is a web browser capable of displaying and controlling static images, video, audio, and providing hypertext links to other locations on the World Wide Web (WWW).
  • WWW World Wide Web
  • Media download module 220 uses a communication protocol such as TCP/IP to download data from secondary media 112 over network 108 to STB 106.
  • Server 110 uses a unique identifier associated with data on primary media 114 to search secondary media 112 for data corresponding to primary media 114.
  • the data associated with secondary media 112 may include hypertext links, static images, video clips, and audio clips suitable for playback with different types of data stored on primary media 114.
  • Data on primary media 114 and secondary media 112 is organized using database storage and retrieval techniques well known in the art. For example, this data could be stored on EXCEL databases.
  • Data on secondary media 112 matching the unique identifier is downloaded into local database 216 associated with STB 106.
  • Local database 216 acts as a temporary cache location to store data from secondary media 112 before playback on audio-video devices 102.
  • each entry of data from secondary media 112 stored in local database 216 is also indexed according to the same unique identifier.
  • Media cache module 222 matches entries in local database 216 to the data on primary media 114 being played back over audio-video devices 102.
  • Media cache module 222 searches data from secondary media 112 cached in local database 216 for a unique identifier associated with data on primary media 114 being played on an audio or video device. Matching entries of data from secondary media 112 in local database 216 are played back with data on primary media 114 consistent with the present invention.
  • Media synchronization module 224 sends commands to different audio and video devices to play data on primary media 114 and data on secondary media 112 at predetermined times.
  • Timing information indicates when the data on secondary media 112 should be played. For example, timing information can indicate that data from secondary media 112 should be played after data on primary media 1 14 has been played for a predetermined number of seconds. This timing information can be stored in local database 216 along with data from secondary media 112 or may be stored separately in a file containing synchronization information.
  • Media download module 220, media cache module 222, and media synchronization module 224 can be implemented in C, JavaScript, or an object-oriented programming language such as the Java programming language.
  • JavaScript or an object-oriented programming language
  • standard versions of the Java programming language and the JavaScript scripting language do not generally allow applications to access storage areas associated with a computer system. Consequently, modules 220, 222, and 224 require extensions to JavaScript or the Java Programming language to access secondary storage 212, memory 208, and other storage areas where data from secondary media 112, primary media 114, and related information may be stored.
  • Synchronization data 226 provides timing information for playing back data from primary media 114 and secondary media 112.
  • the timing information is generally downloaded over network 108 along with data from secondary media 112. Playback of data from secondary media 112 depends on the elapsed time primary media 114 has played.
  • timing information can indicate that a secondary media such as a hypertext link should be displayed on display device 104 after a selected primary media such as a music on a CD-ROM has been playing for 20 seconds.
  • synchronization data 226 can be stored in a separate file or separate record in a database.
  • Local database application 228 and remote database application 230 are software components used to search and retrieve data stored on local database 216 and a remote database storing data from secondary media 112.
  • Remote database application 230 provides database operations necessary for media download module 220 to access and retrieve data from secondary media 112.
  • Local database application 228 provides similar database functionality enabling media cache module 222 to retrieve downloaded data from secondary media 112 stored on local database 216.
  • Development kit 232 includes software libraries and application programming interfaces (APIs) used by media download module 220, media cache module 222. and media synchronization module 224 to access static images, audio, and video media information stored on local databases, remote databases, and other storage areas associated with set-top box 106.
  • development kit 232 provides access to HAVi or Sony's Control A-l or S-Link protocol to send commands to audio-video devices 102.
  • These software libraries and APIs in development kit 232 may be compatible with a wide range of programming languages including C, JavaScript, the Java programming language, and other object- oriented programming languages.
  • User interface 218 may include extensions to hypertext markup language (HTML) that access development kit 232 and process static images, audio, and video media.
  • HTML hypertext markup language
  • Run-time environment 234 facilitates execution on processor 202 of the above mentioned software subsystems located in memory 208.
  • Different run-time environments 228 compatible with implementations of the present invention include real-time operating systems (RTOS) for real-time processing, UNIX-based operating systems, or Microsoft Windows.
  • RTOS real-time operating systems
  • UNIX-based operating systems UNIX-based operating systems
  • Microsoft Windows Microsoft Windows
  • Audio-video interface 210 provides a connection to audio-video devices 102 and display device 104.
  • STB 106 transmits commands through audio-video interface 210 to display device 104 or the appropriate device in audio-video devices 102 in accordance with the present invention. These commands control operation of each device in audio-video devices 102.
  • audio-visual interface 210 can transmit commands requesting a device to send descriptive information on the media stored in the device or data stored on the media. For example, the descriptive information could include the title of a video or audio track.
  • These commands can also include commands that cause the audio and video devices to play audio and video media in a desired sequence.
  • Secondary storage 212 preferably comprises a hard disk drive or other mass storage device formatted to store information such as local database 216.
  • local database 216 can be used as a cache for storing data from secondary media 112 such as hypertext links, video clips, audio or video information as well as synchronization or timing information.
  • FIG. 3 is a flow chart of the method steps for synchronizing the playback of data on primary media 112 and secondary media 114 in accordance with the present invention.
  • set-top box 106 sends commands to audio-video devices 102 to determine which devices are available and determine the configuration of audio-video system 100 (stage 302).
  • Set-top box device 106 identifies each audio-video device according to a unique address assigned by the manufacturer thereof. Because each audio or video device has a unique address, STB 106 can distinguish between multiple audio-video devices that play the same media format. For example, STB will identify two CD players as CD-I and CD-2.
  • STB 106 uses the unique address associated with each audio-video device to interrogate audio-video devices 102 and identify the primary media associated with each audio-video device (stage 304).
  • Primary media 114 associated with these devices includes CDS, video cassettes, laser disks, DVDs, and minidisks.
  • the commands used to perform this interrogation are based on HAVi or the Control A-l or S-Link protocol developed by the Sony Corporation.
  • STB 106 extracts table of contents information from the data associated with media on each audio-video device.
  • the table of contents can be used as a "fingerprint" to uniquely identify the data because the table of contents for data on different audio or video media is generally unique.
  • One technique for identifying audio CDS is contained in a specification known in the industry as the "Red Book”. Additional techniques for identifying data stored on CDS are also described in U.S. Patent, "Compact Disc Changer Utilizing Disc Database", Serial No. 08/507.544, by Carl J. Yankowski, assigned to the assignee of the present invention, filed July 26, 1995 which is incorporated by reference in its entirety. Those skilled in the art will appreciate that the techniques for identifying data on CDS can also be used to identify data stored on video cassettes and other storage mediums such as DVD and minidisks.
  • STB 106 uses table of contents data extracted from primary media 1 14 to generate a unique identifier (stage 306). Thereafter, the unique identifier identifies data on primary media 114 and is used to cross reference data stored on secondary media 112 available over network 108.
  • STB 106 loads media download module 220 into memory 208 to obtain data from secondary media 112.
  • Data on secondary media 112 is indexed according to the unique identifier associated with primary media 114.
  • Media download module 220 uses the unique identifier to locate data on secondary media 112 associated with a remote database over a network such as the Internet (stage 308).
  • a database developer indexes information such as hypertext links from secondary media 112 by one or more unique identifiers corresponding to data on primary media 114.
  • media download module 220 uses the unique identifier for the data stored on the CD to locate the corresponding data such as a hypertext link or a static image stored on secondary media 112.
  • media download module 220 downloads the data over network 108 and into a cache or database such as local database 216 (stage 310).
  • the database or cache used to hold data from secondary media 112 can transmit information at a transmission rate equal to or greater than the transmission rate associated with transmitting data from the primary media.
  • set-top box can synchronize playback of data from primary media 114 with data on secondary media 112 (stage 312). For example, data from secondary media such as hypertext links previously downloaded from the Internet can be displayed on display device 104 while data stored on a storage medium such as a CD is being played on a CD player.
  • FIG. 4 is a flowchart diagram illustrating the stages associated with the process of synchronizing the playback of data on primary media 114 and secondary media 112 in accordance with the present invention. These additional stages are associated with stage 312 in FIG. 3.
  • STB 106 loads media cache module 222 in memory 208 for processing by processor 202.
  • Media cache module 222 identifies data on primary media 114 selected for playback, such as audio media stored on a CD, using the unique identifier discussed above (stage 402). Using the unique identifier, media cache module 222 locates cached data from secondary media 112 corresponding to the unique identifier stored in local database 216 (stage 404).
  • cached data from secondary media 112 may include a series of hypertext links related to audio tracks on a CD being played on a CD player.
  • data from secondary media 112 such as a hypertext link is cached before playback of data from primary media 114 to avoid the delay introduced downloading data over a network such as the Internet.
  • processor 202 executes media synchronization module 224 to coordinate the playback.
  • Media synchronization module 224 sets triggers to play data on the secondary media 112 based upon the elapsed time period data on primary media 114 has played (stage 406).
  • synchronization data 226 holds information used for timing the playback of the different types of media. This synchronization information is loaded into media synchronization module 224 before playing data from primary media 1 14. Triggers are set in media synchronization module 224 to launch data from secondaiy media 112 such as hypertext links at predetermined time intervals.
  • Synchronization begins when data stored on primary media 114 such as a CD begins playing (stage 408). After the data plays for an elapsed time period, media synchronization module 224 triggers interrupts causing cached data from secondary media 112, such as a hypertext link, to be launched and processed by the appropriate audio-visual devices (stage 410). Media synchronization module 224 then locates subsequent data in cache based on the media unique identifier (stage 412). At the appropriate time period, synchronization module 224 launches the next data from secondary media 1 12 for playback and the process repeats until all data from primary media 114 and corresponding data from secondary media 112 has been played (stage 414).
  • FIG. 5 illustrates an exemplary organization of synchronization data 226 used for two tracks on a CD, data from primary media 114, and the hypertext links, data from secondary media 112, corresponding to these tracks.
  • synchronization data 226 includes a synchronization header 500, a track identifier 501, a first link 502, a second link 504, a third link 506, and a fourth link 508.
  • Synchronization header 500 includes information to describe the synchronization file including the unique identifier ("file name”) to associate the hypertext links with the tracks on a CD.
  • Track identifier 501 indicates which track of the CD should be played while the hypertext links are displayed on a display device for selection. Those skilled in the art will understand that track identifier 501 includes sufficient information to identify and load a CD in a CD player for use with a command language such as HAVi or the Control A-l or S-Link developed by the Sony Corporation.
  • Each link subsequent to track identifier 501 includes a time stamp and data specifying when to launch the link.
  • first link 502 has a time stamp of zero indicating that the corresponding link is available and can be displayed immediately when the CD begins playing the first track (i.e. the CD has played zero seconds).
  • the " ⁇ type>" entry in first link 502 can indicate the type of information including a universal record locator (URL) type pointing to any valid URL type or a JavaScript type pointing to a separate file containing JavaScript commands.
  • the ten (10) second time stamp on second link 504 indicates that the hypertext link is displayed after the CD has played ten (10) seconds of the first track.
  • the forty (40) second time stamp associated with the third link 506 and the one-hundred sixty (160) second time stamp associated with the fourth link 508 indicate that the third and fourth links have been downloaded and available for display after the CD has played forty (40) and one-hundred sixty (160) seconds, respectively.
  • Synchronization data 226 also includes information for synchronizing the second track of the CD with several hypertext links.
  • the synchronization information for the second track includes a track identifier for the second track 510, a first link 512 for the second track, a second link 514 for the second track, and a third link 516 for the second track.
  • Synchronization data 226 for the second track on the CD is organized in a similar manner to the corresponding information for the first track on the CD discussed above.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A method for synchronizing the playback of data on a primary media (114) with data on a secondary media (112) stored remotely over a network (108) is provided. Generally, the network (108) can not deliver data on the secondary media (112) as quickly to an audio or video device as data on the primary media (114) stored in the audio or video device (102) is delivered. This makes it difficult to synchronize playback of data on the primary and secondary media. To overcome this problem, implementations of the present invention download data on the secondary media (112) in advance and coordinates the playback of the data. Initially, data on the primary media (114) is identified using a unique identifier. The unique identifier is used to locate data on the secondary media (112) stored remotely over a network (108). Data on the secondary media (112) matching the unique identifier is downloaded over the networek (108) and stored in a storage area local to the audio or video device (102) such as a cache. Timing information is then used to synchronize playback of data on the primary media (114) with data on the secondary media (112) stored in cache.

Description

Title of the Invention
METHOD AND APPARATUS FOR SYNCHRONIZING PLAYBACK OF
MULTIPLE MEDIA TYPES OVER NETWORKS HAVING DIFFERENT
TRANSMISSION CHARACTERISTICS
Background of the Invention Field of the Invention
This invention generally relates to consumer device control systems and more particularly, to a method and apparatus for synchronizing playback of multiple media types over networks with different transmission characteristics. Description of the Related Art
Conventional audio-video systems are designed to play audio and video information from a variety of different media. The audio-video systems include audio devices that can playback sounds from audio media such as compact disks (CD), minidisks, tape cassettes, and records. The video devices in these systems can display images on a display screen such as a television set and generate accompanying voice, music, or sound. Video information can be stored on video media such as video cassettes, digitally encoded laser disks, and digital video disks (DVD).
The audio and video devices are generally connected to the audio-video system using a local bus or other direct connection. These devices transmit audio and connection is reserved for use by the directly attached audio and video devices. As a result, there is a relatively short time period between requesting playback of media and the time playback of the media begins. Further, the delivery rate of the audio and video information is predictable because the direct connection delivers information to the audio-video system at essentially a constant transmission rate. For example, a CD player can deliver audio information over a direct connection to an audio-video system at a constant transmission rate because bandwidth on the direct connection is not shared with other devices. By connecting an audio-video system to a network such as the Internet, many more audio and video titles are available for playback. The Internet provides an everchanging variety of media and is not limited to the media stored on traditional devices such as CDS, videos, laser disks, and minidisks. Unfortunately, media delivered over a network like the Internet may include variable delays and transmit data at variable data rates. Variations in delivery may occur because the audio or video information being transmitted has different compression/decompression methods that take different times periods to process or because the information is delivered by computer systems transmitting information over networks with different bandwidths. As an additional factor, data delivery rates may fluctuate on public networks such as the Internet as thousands of users sporadically access the Internet and consume the available bandwidth.
For at least these reasons, it is difficult to combine media stored on a network such as the Internet with information stored on a local device because the transmission characteristics of a network are not as predictable as a local direct connection. For example, the difference in transmission speed makes it difficult to play back a CD from a local CD player while simultaneously downloading and displaying images from the Internet in realtime. Even though audio information on the CD is ready for playback immediately, images cannot be downloaded from a network as quickly. Further, sudden variations in data traffic on a network such as the Internet may cause the transmission rate for downloading information to change unpredictably. This also makes it difficult to synchronize playback of media on a local device, such as a CD player, with media being downloaded over a network.
Accordingly, there is a need to coordinate the delivery and playback of data stored on different media types delivered to an audio or video device when the data is being transmitted with different transmission characteristics to an audio- video system.
Summary of The Invention
In accordance with the present invention, as embodied and broadly described herein, a method for synchronizing the playback of data on a primary media with data on a secondary media is provided. Initially, the data on the primary media is identified using a unique identifier. The unique identifier is generated based upon information associated with data on the primary media such as the table of contents. This unique identifier is used to locate data on the secondary media stored remotely over a network. For example, data on the secondary media could be a hypertext link associated with a particular track on a compact disk (CD). Data on the secondary media matching the unique identifier is downloaded over the network and cached in a local storage area. Timing information is then used to synchronize playback of data on the primary data on media with the cached data from the secondary media.
Another aspect of the present invention provides a method of using the timing information to synchronize playback of data from the primary media with data from the secondary media. Initially, data on the primary media is identified using a unique identifier. Data from the secondary media cached on a local storage device is located using the unique identifier. Timing information associated with the data from secondary media is used to set triggers for launching data from the secondary media. The time period for launching data from the secondary media is based upon an elapsed time period data on the primary media has played. Playing data from the primary media over a time period triggers interrupts, launches data on the secondary media, and synchronizes playback of data from the secondary media with the data on primary media.
Brief Description of The Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate systems and methods consistent with the invention and, together with the description, serve to explain the advantages, and principles of the invention.
In the drawings:
FIG. 1 is a block diagram of an audio-video system consistent with the present invention including data from primary media and secondary media;
FIG. 2 is a block diagram of a set-top device shown in FIG. 1 consistent with the present invention;
FIG. 3 is a flowchart consistent with the present invention for playing data on secondary media downloaded from a network with data on primary media stored on a local audio- video device;
FIG. 4 is a flowchart for synchronizing the playback of data on a secondary media and data on a primary media consistent with the present invention; and FIG. 5 is a block diagram displaying one technique for organizing synchronization information associated with data on the primary media and secondary media consistent with one implementation of the present invention.
Detailed Description Audio Visual System And Network
FIG. 1 is a block diagram of an audio-video system 100 operatively coupled to a network capable of combining various media into a multimedia presentation. Audiovideo system 100 includes audio-video devices 102, a display device 104, a set-top device 106, an input device 107 for communicating with set-top device 106, and a network 108 accessible to set-top device 106. A server 110 is operatively coupled to network 108 and a remote database with secondary media 1 12. A local database stored on primary media 1 14 is operatively coupled to audio-video devices 102. Network 108 can include the Internet and computer resources associated with the Internet.
Audio-video devices 102 include devices capable of playing different audio and video media. For example, these devices may include a compact disk player, a video tape player, a DVD player, a mini disk player, a television receiver, an audio receiver, and an audio playback device such as a stereo receiver/amplifier and audio tape player. These devices can be connected together using a bus 103 such as IEEE- 1394 ("Firewire") or Universal Serial Bus (USB). Bus 103 transmits data stored on a media quickly between set- top device 106 and audio-video devices 102. Because each device connected to bus 103 can be uniquely addressed, multiple identical devices can be configured into the same system. For example, DVD players could be separately addressed on bus 103 as DVD-1 and DVD-2.
Video images are displayed on display device 104 when a portion of data on the media includes visual display information. For example, a television tuner or video playback device in audio-video devices 102 generates images displayed on display device 104. The television tuner or video playback device may also generate audio information played back through an audio device such as a stereo receiver/amplifier and audio tape player. Display device 104 can be any device used to display images such as a computer screen, a television, or a video-monitor device.
Set-top device, or box, (STB) 106 is a computer-based device used to control audio- video devices 102 and display device 104. In addition, set-top device 106 has access to server 110 and secondary media 112 through network 108. Consistent with the present invention, set-top device 106 coordinates playback of data on primary media 114 and secondary media 112. Primary media 114 is typically a storage medium such as a CD-ROM associated with an audio-video device while secondary media 112 is a storage medium such as a hard-disk associated with server 100.
A communication protocol enables STB 106 to communicate with audio-video devices 102 and display device 104. STB 106 sends commands to audio-video devices 102 to play the audio media, video media, and other information associated with primary media 114 and secondary media 112. These commands can be in the format of the control protocol called Control A-l or S-Link developed by the Sony Corporation based in Tokyo, Japan. Alternatively, other control protocols such as Home Audio Video Interoperability Architecture (HAVi) can be used along with other control protocols that are compatible with audio-video devices 102 and display device 104. HAVi was developed by a consortium including many consumer electronics companies including Sony, Hitachi, Matsushita, Phillips, Toshiba. Sharp, and Grundig. The document describing HAVi, "The HAVi Architecture: Specification of the Home/Nideo Interoperability (HAVi) Architecture", Version 0.8, authored by Sony. Hitachi, Matsushita. Phillips, Toshiba, Sharp, Grundig is incorporated by reference herein.
Input device 107 can communicate with STB 106 through a wireless communication mechanism using infrared signals or radio frequency signals. This allows a user to operate STB 106 and accompanying audio-video devices 102 without cumbersome wires. Input device 107 can be a keyboard, a keyboard and a mouse, a hand-held remote control device, or any other input device useful in controlling STB 106.
In FIG. 1, server 110 facilitates access to secondary media 112. Secondary media 112 includes data or information related to data on primary media 114. For example, data on secondary media 112 may include hypertext links to locations on the Internet, static images, or video clips that compliment and enhance the playback of data on primary media 114. Server 110 transmits this data from secondary media 112 through network 108 to STB 106. STB 106 stores data from the secondary media internally in a local database on a storage device such as a hard disk or other cache device. STB 106 then coordinates the playback of data on primary media 114 and data from secondary media 112. Coordination is necessary because network 108 transmits data stored on secondary media 112 at a slower transmission rate to STB 106 than data transmitted between audio-video-devices 102, primary media 114, and STB 106. Using a local storage area such as cache, implementations of the present invention provide a technique for synchronizing the playback of data from secondary media 1 12 transmitted over network 108 and data from primary media 114 stored on audio-video devices 102. Data transmitted over a slower transmission link, such as network 108, is cached in a local database and played back at a predetermined time corresponding to the playback of data transmitted over a faster transmission link, such as bus 103, thus synchronizing the playback of data on secondary media 112 and primary media 114. Additional information on synchronizing data provided over transmission links with different transmission speeds and operating characteristics is discussed in further detail later herein.
FIG. 2 is a block diagram of STB 106 designed in accordance with the present invention. STB 106 includes a processor 202, an input-output interface 204, a network interconnection 206, a wireless interface 207, a memory 208, an audio-video interface 210, a secondary storage 212, and an internal bus 214 facilitating communication between each of the aforementioned elements. Processor 202 can be a general purpose processor such as an Intel Pentium processor or an application specific integrated circuit (ASIC) designed to execute one or more applications. For example, processor 202 can be an ASIC designed specifically to run web browser software compatible with the World Wide Web (WWW). Input-output interface 204 can be connected to a number of different peripheral devices such as an external storage device or a printer. Network interconnect 206 may be a modem, cable modem, or other device to provide an interface between a network such as the Internet and STB 106. Wireless interface 207 facilitates receiving infra-red (IR) or radio frequency (RF) signals from input devices not physically attached to STB- 106. For example, a keyboard device and mouse can communicate with set-top box 106 through a wireless infrared interface without the use of cables or other physical attachments.
Memory 208 includes a user interface 218. a media download module 220. a media cache module 222, a media synchronization module 224, synchronization data 226, a local database application 228. a remote database application 230, a development kit 232, and a run time environment 234. User interface 218 is a graphical user interface supporting hypertext links and suitable for displaying and controlling the playback of data from multiple media types in accordance with the present invention. Preferably, user interface 218 is a web browser capable of displaying and controlling static images, video, audio, and providing hypertext links to other locations on the World Wide Web (WWW).
Media download module 220 uses a communication protocol such as TCP/IP to download data from secondary media 112 over network 108 to STB 106. Server 110 uses a unique identifier associated with data on primary media 114 to search secondary media 112 for data corresponding to primary media 114. The data associated with secondary media 112 may include hypertext links, static images, video clips, and audio clips suitable for playback with different types of data stored on primary media 114. Data on primary media 114 and secondary media 112 is organized using database storage and retrieval techniques well known in the art. For example, this data could be stored on EXCEL databases. Data on secondary media 112 matching the unique identifier is downloaded into local database 216 associated with STB 106. Local database 216 acts as a temporary cache location to store data from secondary media 112 before playback on audio-video devices 102. Preferably, each entry of data from secondary media 112 stored in local database 216 is also indexed according to the same unique identifier.
Media cache module 222 matches entries in local database 216 to the data on primary media 114 being played back over audio-video devices 102. Media cache module 222 searches data from secondary media 112 cached in local database 216 for a unique identifier associated with data on primary media 114 being played on an audio or video device. Matching entries of data from secondary media 112 in local database 216 are played back with data on primary media 114 consistent with the present invention.
Media synchronization module 224 sends commands to different audio and video devices to play data on primary media 114 and data on secondary media 112 at predetermined times. Timing information indicates when the data on secondary media 112 should be played. For example, timing information can indicate that data from secondary media 112 should be played after data on primary media 1 14 has been played for a predetermined number of seconds. This timing information can be stored in local database 216 along with data from secondary media 112 or may be stored separately in a file containing synchronization information.
Media download module 220, media cache module 222, and media synchronization module 224 can be implemented in C, JavaScript, or an object-oriented programming language such as the Java programming language. For security reasons, standard versions of the Java programming language and the JavaScript scripting language do not generally allow applications to access storage areas associated with a computer system. Consequently, modules 220, 222, and 224 require extensions to JavaScript or the Java Programming language to access secondary storage 212, memory 208, and other storage areas where data from secondary media 112, primary media 114, and related information may be stored.
Synchronization data 226 provides timing information for playing back data from primary media 114 and secondary media 112. The timing information is generally downloaded over network 108 along with data from secondary media 112. Playback of data from secondary media 112 depends on the elapsed time primary media 114 has played. For example, timing information can indicate that a secondary media such as a hypertext link should be displayed on display device 104 after a selected primary media such as a music on a CD-ROM has been playing for 20 seconds. As previously discussed, synchronization data 226 can be stored in a separate file or separate record in a database.
Local database application 228 and remote database application 230 are software components used to search and retrieve data stored on local database 216 and a remote database storing data from secondary media 112. Remote database application 230 provides database operations necessary for media download module 220 to access and retrieve data from secondary media 112. Local database application 228 provides similar database functionality enabling media cache module 222 to retrieve downloaded data from secondary media 112 stored on local database 216.
Development kit 232 includes software libraries and application programming interfaces (APIs) used by media download module 220, media cache module 222. and media synchronization module 224 to access static images, audio, and video media information stored on local databases, remote databases, and other storage areas associated with set-top box 106. Preferably, development kit 232 provides access to HAVi or Sony's Control A-l or S-Link protocol to send commands to audio-video devices 102. These software libraries and APIs in development kit 232 may be compatible with a wide range of programming languages including C, JavaScript, the Java programming language, and other object- oriented programming languages. User interface 218 may include extensions to hypertext markup language (HTML) that access development kit 232 and process static images, audio, and video media.
Run-time environment 234 facilitates execution on processor 202 of the above mentioned software subsystems located in memory 208. Different run-time environments 228 compatible with implementations of the present invention include real-time operating systems (RTOS) for real-time processing, UNIX-based operating systems, or Microsoft Windows.
Audio-video interface 210 provides a connection to audio-video devices 102 and display device 104. STB 106 transmits commands through audio-video interface 210 to display device 104 or the appropriate device in audio-video devices 102 in accordance with the present invention. These commands control operation of each device in audio-video devices 102. For example, audio-visual interface 210 can transmit commands requesting a device to send descriptive information on the media stored in the device or data stored on the media. For example, the descriptive information could include the title of a video or audio track. These commands can also include commands that cause the audio and video devices to play audio and video media in a desired sequence.
Secondary storage 212 preferably comprises a hard disk drive or other mass storage device formatted to store information such as local database 216. As previously discussed, local database 216 can be used as a cache for storing data from secondary media 112 such as hypertext links, video clips, audio or video information as well as synchronization or timing information. Media Synchronization
FIG. 3 is a flow chart of the method steps for synchronizing the playback of data on primary media 112 and secondary media 114 in accordance with the present invention. Initially, set-top box 106 sends commands to audio-video devices 102 to determine which devices are available and determine the configuration of audio-video system 100 (stage 302). Set-top box device 106 identifies each audio-video device according to a unique address assigned by the manufacturer thereof. Because each audio or video device has a unique address, STB 106 can distinguish between multiple audio-video devices that play the same media format. For example, STB will identify two CD players as CD-I and CD-2.
STB 106 uses the unique address associated with each audio-video device to interrogate audio-video devices 102 and identify the primary media associated with each audio-video device (stage 304). Primary media 114 associated with these devices includes CDS, video cassettes, laser disks, DVDs, and minidisks. Preferably, the commands used to perform this interrogation are based on HAVi or the Control A-l or S-Link protocol developed by the Sony Corporation.
STB 106 extracts table of contents information from the data associated with media on each audio-video device. The table of contents can be used as a "fingerprint" to uniquely identify the data because the table of contents for data on different audio or video media is generally unique. One technique for identifying audio CDS is contained in a specification known in the industry as the "Red Book". Additional techniques for identifying data stored on CDS are also described in U.S. Patent, "Compact Disc Changer Utilizing Disc Database", Serial No. 08/507.544, by Carl J. Yankowski, assigned to the assignee of the present invention, filed July 26, 1995 which is incorporated by reference in its entirety. Those skilled in the art will appreciate that the techniques for identifying data on CDS can also be used to identify data stored on video cassettes and other storage mediums such as DVD and minidisks.
Accordingly, STB 106 uses table of contents data extracted from primary media 1 14 to generate a unique identifier (stage 306). Thereafter, the unique identifier identifies data on primary media 114 and is used to cross reference data stored on secondary media 112 available over network 108.
STB 106 loads media download module 220 into memory 208 to obtain data from secondary media 112. Data on secondary media 112 is indexed according to the unique identifier associated with primary media 114. Media download module 220 uses the unique identifier to locate data on secondary media 112 associated with a remote database over a network such as the Internet (stage 308). For example, a database developer indexes information such as hypertext links from secondary media 112 by one or more unique identifiers corresponding to data on primary media 114. When a user loads a CD in a CD player, media download module 220 uses the unique identifier for the data stored on the CD to locate the corresponding data such as a hypertext link or a static image stored on secondary media 112.
Once related data on secondary media is located, media download module 220 downloads the data over network 108 and into a cache or database such as local database 216 (stage 310). Preferably, the database or cache used to hold data from secondary media 112 can transmit information at a transmission rate equal to or greater than the transmission rate associated with transmitting data from the primary media. Because data on primary media 114 and cached data from secondary media 112 can be transmitted at essentially the same rate, set-top box can synchronize playback of data from primary media 114 with data on secondary media 112 (stage 312). For example, data from secondary media such as hypertext links previously downloaded from the Internet can be displayed on display device 104 while data stored on a storage medium such as a CD is being played on a CD player.
FIG. 4 is a flowchart diagram illustrating the stages associated with the process of synchronizing the playback of data on primary media 114 and secondary media 112 in accordance with the present invention. These additional stages are associated with stage 312 in FIG. 3. Initially, STB 106 loads media cache module 222 in memory 208 for processing by processor 202. Media cache module 222 identifies data on primary media 114 selected for playback, such as audio media stored on a CD, using the unique identifier discussed above (stage 402). Using the unique identifier, media cache module 222 locates cached data from secondary media 112 corresponding to the unique identifier stored in local database 216 (stage 404). For example, cached data from secondary media 112 may include a series of hypertext links related to audio tracks on a CD being played on a CD player. As previously discussed, data from secondary media 112 such as a hypertext link is cached before playback of data from primary media 114 to avoid the delay introduced downloading data over a network such as the Internet.
Once data stored on primary media 114 and corresponding data stored on secondary media 112 is identified, processor 202 executes media synchronization module 224 to coordinate the playback. Media synchronization module 224 sets triggers to play data on the secondary media 112 based upon the elapsed time period data on primary media 114 has played (stage 406). As previously discussed, synchronization data 226 holds information used for timing the playback of the different types of media. This synchronization information is loaded into media synchronization module 224 before playing data from primary media 1 14. Triggers are set in media synchronization module 224 to launch data from secondaiy media 112 such as hypertext links at predetermined time intervals. Those skilled in the art will understand how traditional interrupt mechanisms capable of setting, triggering, and processing interrupts can be used to implement this aspect of the present invention. Synchronization begins when data stored on primary media 114 such as a CD begins playing (stage 408). After the data plays for an elapsed time period, media synchronization module 224 triggers interrupts causing cached data from secondary media 112, such as a hypertext link, to be launched and processed by the appropriate audio-visual devices (stage 410). Media synchronization module 224 then locates subsequent data in cache based on the media unique identifier (stage 412). At the appropriate time period, synchronization module 224 launches the next data from secondary media 1 12 for playback and the process repeats until all data from primary media 114 and corresponding data from secondary media 112 has been played (stage 414).
FIG. 5 illustrates an exemplary organization of synchronization data 226 used for two tracks on a CD, data from primary media 114, and the hypertext links, data from secondary media 112, corresponding to these tracks. For the first track of the CD, synchronization data 226 includes a synchronization header 500, a track identifier 501, a first link 502, a second link 504, a third link 506, and a fourth link 508. Synchronization header 500 includes information to describe the synchronization file including the unique identifier ("file name") to associate the hypertext links with the tracks on a CD. In this example, the filename is "850f970b.SCN" wherein the prefix "850f970b" corresponds to the primary data and the suffix ".SCN" indicates that this file is a synchronization file. Track identifier 501 indicates which track of the CD should be played while the hypertext links are displayed on a display device for selection. Those skilled in the art will understand that track identifier 501 includes sufficient information to identify and load a CD in a CD player for use with a command language such as HAVi or the Control A-l or S-Link developed by the Sony Corporation.
Each link subsequent to track identifier 501 includes a time stamp and data specifying when to launch the link. For example, first link 502 has a time stamp of zero indicating that the corresponding link is available and can be displayed immediately when the CD begins playing the first track (i.e. the CD has played zero seconds). The "<type>" entry in first link 502 can indicate the type of information including a universal record locator (URL) type pointing to any valid URL type or a JavaScript type pointing to a separate file containing JavaScript commands. The ten (10) second time stamp on second link 504 indicates that the hypertext link is displayed after the CD has played ten (10) seconds of the first track. Similarly, the forty (40) second time stamp associated with the third link 506 and the one-hundred sixty (160) second time stamp associated with the fourth link 508 indicate that the third and fourth links have been downloaded and available for display after the CD has played forty (40) and one-hundred sixty (160) seconds, respectively.
Synchronization data 226 also includes information for synchronizing the second track of the CD with several hypertext links. The synchronization information for the second track includes a track identifier for the second track 510, a first link 512 for the second track, a second link 514 for the second track, and a third link 516 for the second track. Synchronization data 226 for the second track on the CD is organized in a similar manner to the corresponding information for the first track on the CD discussed above.
While specific embodiments have been described herein for purposes of illustration, various modifications may be made without departing from the spirit and scope of the invention. For example, the figures and examples discussed above utilize a primary and a secondary media however implementations of the present invention could be used to synchronize more than two types of media. Implementations of the present invention can also use command languages other than HAVi or the Control A-l or S-Link protocol developed by the Sony Corporation. Further, although aspects of the present invention are described as being stored in memory and other storage mediums, one skilled in the art will appreciate that these aspects of the present invention can also be stored on or read from other types of computer-readable media, such as secondary storage devices, like hard disks, floppy disks, or CD-ROM; a carrier wave from the Internet, or other forms of RAM. Accordingly, the invention is not limited to the above described embodiments, but instead is defined by the appencted claims in light of their full scope of equivalents.

Claims

Claims
1. A method, performed on a processor, for synchronizing the playback of data on a primary media with data on a secondary media, comprising: identifying data on the primary media; locating data on the secondary media available remotely over a network where data on the secondary media corresponds to the identified data on the primary media; storing data from the secondary media available over the network into a storage area associated with the processor; and synchronizing the playback of data from the primary media with data from the secondary media stored in the storage area.
2. The method of claim 1 wherein the identifying stage further includes generating a unique identifier associated with data on the primary media.
3. The method of claim 2 wherein the locating stage further includes using the unique identifier to identify data on the secondary media.
4. The method of claim 1 wherein the storing stage further includes selecting a storage area capable of delivering data from the secondary media at a transmission rate at least equal to the transmission rate associated with data on the primary media.
5. The method of claim 1 wherein the storing stage further includes downloading data from the secondary media over a network.
6. The method of claim 1 wherein downloading data on the secondary media over a network occurs at a slower transmission rate than the transmission rate associated with delivering data the primary media.
7. The method in claim 1 wherein the synchronizing stage further comprises: identifying data on the primary media; locating data on the secondary media stored in the storage area and corresponding to the identified data on primary media; setting a trigger to launch data on the secondary media based upon an elapsed time period data on the primary media has played; playing data on the primary media over a time period: and triggering the trigger at the expiration of the elapsed time period to synchronize playback of data on the secondary media with data on the primary media.
8. The method in claim 7 wherein the identifying stage further includes generating a unique identifier associated with data on the primary media.
9. The method of claim 8 wherein locating data on the secondary media stored in the storage area includes using the unique identifier to identify data on the secondary media.
10. A playback method, performed on a audio- video system, for synchronizing the playback of data on a primary media with data on a secondary media, comprising the stages of identifying data on the primary media with a unique identifier; locating data on the secondary media stored remotely over a network where data on the secondary media is associated with the unique identifier; downloading data on the secondary media located on the network; storing data on the secondary media from the network into a storage area; and synchronizing the playback of data on the primary media with the stored data.
11. The playback method in claim 10 wherein the synchronizing stage further comprises the stages of: identifying data on the primary media according to a unique identifier; locating the stored data corresponding to the unique identifier; setting a trigger to playback the stored data based upon an elapsed time period data on the primary media has played; playing data on the primary media over a time period; and triggering the trigger at he elapsed time period to synchronize playback of the stored data with data on the primary media.
12. An apparatus for synchronizing the playback of data from at least two media located on different locations of a data processing system, comprising: data on a primary media (1 14) identified by an identifier and associated with the data processing system; data on a secondary media (112) available over a network (108) remote to the primary media (114) that corresponds to data on the primary media (114); a storage area associated with the data processing system capable of storing data on the secondary media; and a synchronization module (106) capable of synchronizing the playback of data on the primary media (114) with data on the secondary media (112) stored in the storage area.
13. The apparatus of claim 12 wherein the identifier is uniquely associated with data on the primary media.
14. The apparatus of claim 12 wherein data on the secondary media (112) corresponds to data on the primary media (114) according to the identifier.
15. The apparatus of claim 12 wherein the storing area is capable of delivering data from the secondary media (112) to the synchronization module (106) at a transmission rate at least equal to the corresponding transmission rate associated with data on the primary media (114).
16. The apparatus of claim 12 wherein data on the storing area is capable of receiving data on the secondary media (112) downloaded over the network (108).
17. The apparatus of claim 12 wherein the network (108) downloads data on the secondary media (112) at a slower transmission rate than the corresponding transmission rate associated with providing data on the primary media (114).
18. The apparatus of claim 12 wherein the synchronization module (106) further comprises: identifier module configured to identify data on the primary media; a locator module configured to locate data on the secondary media stored in the storage area that corresponds to the identified data; a trigger setting module configured.to set a trigger to launch data on the secondary media (112) based upon an elapsed time period the data on the primary media (114) has played; a playback module that plays data on the primary media (114) over a time period; and a trigger module configured to launch a trigger at the expiration of the elapsed time period and synchronize playback of data from the secondary media (112) with data on the primary media (114).
19. The apparatus in claim 18 wherein the identifier module generates a unique identifier associated with data on the primary media (114).
20. The apparatus of claim 18 wherein data from the secondary media (112) stored in the storage area is located using the unique identifier associated with data on the primary media (114).
21. A computer-readable medium containing instructions executable on a processor that synchronizes the playback of data on a primary media with data on a secondary media, the instructions capable of: identifying data on the primary media; locating data on the secondary media available remotely over a network where data on the secondary media corresponds to the identified data on the primary media; storing data on the secondary media available over the network into a storage area associated with the processor; and synchronizing playback of data on the primary media with data on the secondary media stored in the storage area.
22. The computer-readable medium of claim 21 further including instructions capable of generating a unique identifier associated with data on the primary media.
23. The computer-readable medium of claim 21 further including instructions to locate data on the secondary media according to the unique identifier.
24. The computer-readable medium of claim 21 further including instructions capable of facilitating the delivery of data from the secondary media at a transmission rate at least equal the transmission rate associated with data on the primary media.
25. The computer readable medium of claim 21 further including instructions capable of facilitating the download of data on the secondary media over a network.
26. The computer-readable medium of claim 21 further including instructions capable of facilitating the download of data on the secondary media over a network at a slower transmission rate than the transmission rate associated with delivering data on the primary media.
27. The computer-readable medium of claim 21 further including instructions to synchronize playback of data on the primary media and data on the secondary media, the instructions capable of: identifying data on the primary media; locating data on the secondary media stored in the storage area and corresponding to the identified data on the primary media: setting a trigger to launch data from the secondary media based upon an elapsed time period data on the primary media has played; playing data on the primary media over a time period; and triggering the trigger at the expiration of the elapsed time period to synchronize playback of data on the secondary media with data on the primary media.
28. An apparatus for synchronizing the playback of data on a primary media with data on a secondary media on a data processing system, comprising: means for identifying data on the primary media; means for locating data on the secondary media available remotely over a network where data on the secondary media corresponds to the identified data on the primary media; means for storing data on the secondary media available over the network into a storal area associated with the processor; and means for synchronizing the playback of data on the primary media with data on the secondary media stored in the storage area.
PCT/US2000/001232 1999-01-22 2000-01-20 Method and apparatus for synchronizing playback of multiple media types over networks having different transmission characteristics Ceased WO2000043999A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU24158/00A AU2415800A (en) 1999-01-22 2000-01-20 Method and apparatus for synchronizing playback of multiple media types over networks having different transmission characteristics

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US23531399A 1999-01-22 1999-01-22
US09/235,313 1999-01-22

Publications (2)

Publication Number Publication Date
WO2000043999A2 true WO2000043999A2 (en) 2000-07-27
WO2000043999A3 WO2000043999A3 (en) 2000-11-30

Family

ID=22884980

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2000/001232 Ceased WO2000043999A2 (en) 1999-01-22 2000-01-20 Method and apparatus for synchronizing playback of multiple media types over networks having different transmission characteristics

Country Status (2)

Country Link
AU (1) AU2415800A (en)
WO (1) WO2000043999A2 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002017618A3 (en) * 2000-08-23 2002-05-16 Imagicast Inc Distributed publishing network
WO2005106881A1 (en) * 2004-04-29 2005-11-10 Kamchung Choi Multimedia system capable of positioning to play multimedia and playing multimedia synchronously
US7100191B1 (en) 1999-08-23 2006-08-29 Xperex Corporation Distributed publishing network
US7254825B1 (en) 1999-08-23 2007-08-07 Xperex Corporation Method and device with DVD and HTML control environments
EP1437739A3 (en) * 2003-01-10 2008-08-13 Kabushiki Kaisha Toshiba Information playback apparatus and information playback method
EP1804244A3 (en) * 2005-12-28 2009-04-22 Kabushiki Kaisha Toshiba Reproduction apparatus and content reproduction method
GB2471871A (en) * 2009-07-15 2011-01-19 Sony Comp Entertainment Europe Apparatus and method for a virtual dance floor
US8010690B2 (en) * 2008-06-26 2011-08-30 Microsoft Corporation Arrangement for connectivity within an advanced multimedia content framework
US9609179B2 (en) 2010-09-22 2017-03-28 Thomson Licensing Methods for processing multimedia flows and corresponding devices
CN106649297A (en) * 2015-07-21 2017-05-10 腾讯科技（深圳）有限公司 Method, browser and server for playing webpage audio file

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2140850C (en) * 1994-02-24 1999-09-21 Howard Paul Katseff Networked system for display of multimedia presentations
KR0138845B1 (en) * 1994-12-13 1998-06-01 양승택 Synchronization controller and control method for multimedia object in mheg engine
US5929849A (en) * 1996-05-02 1999-07-27 Phoenix Technologies, Ltd. Integration of dynamic universal resource locators with television presentations
US5987525A (en) * 1997-04-15 1999-11-16 Cddb, Inc. Network delivery of interactive entertainment synchronized to playback of audio recordings

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7100191B1 (en) 1999-08-23 2006-08-29 Xperex Corporation Distributed publishing network
US7254825B1 (en) 1999-08-23 2007-08-07 Xperex Corporation Method and device with DVD and HTML control environments
WO2002017618A3 (en) * 2000-08-23 2002-05-16 Imagicast Inc Distributed publishing network
EP1437739A3 (en) * 2003-01-10 2008-08-13 Kabushiki Kaisha Toshiba Information playback apparatus and information playback method
WO2005106881A1 (en) * 2004-04-29 2005-11-10 Kamchung Choi Multimedia system capable of positioning to play multimedia and playing multimedia synchronously
EP1804244A3 (en) * 2005-12-28 2009-04-22 Kabushiki Kaisha Toshiba Reproduction apparatus and content reproduction method
US8010690B2 (en) * 2008-06-26 2011-08-30 Microsoft Corporation Arrangement for connectivity within an advanced multimedia content framework
GB2471871A (en) * 2009-07-15 2011-01-19 Sony Comp Entertainment Europe Apparatus and method for a virtual dance floor
GB2471871B (en) * 2009-07-15 2011-12-14 Sony Comp Entertainment Europe Apparatus and method for a virtual dance floor
US9609179B2 (en) 2010-09-22 2017-03-28 Thomson Licensing Methods for processing multimedia flows and corresponding devices
CN106649297A (en) * 2015-07-21 2017-05-10 腾讯科技（深圳）有限公司 Method, browser and server for playing webpage audio file

Also Published As

Publication number Publication date
AU2415800A (en) 2000-08-07
WO2000043999A3 (en) 2000-11-30

Similar Documents

Publication Publication Date Title
US20020194260A1 (en) Method and apparatus for creating multimedia playlists for audio-visual systems
US8381109B2 (en) Method and apparatus for managing bookmark information for content stored in a networked media server
US5990884A (en) Control of multimedia information with interface specification stored on multimedia component
RU2225029C2 (en) Entertainment system data control
US20070198654A1 (en) Network Server
TWI297993B (en) Method for processing connection request of a disk player
JPH1196098A5 (en)
WO2000043999A2 (en) Method and apparatus for synchronizing playback of multiple media types over networks having different transmission characteristics
JP3702525B2 (en) Interactive video recording / playback system
US20030055914A1 (en) Communication apparatus and method, recording medium, program, and content providing apparatus
KR100509162B1 (en) System and method for sharing CODEC in peer-to-peer network
HK1013601A (en) System and method controlling multimedia information components

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
AK Designated states

Kind code of ref document: A3

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A3

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase