TW201739264A - Method and system for automatically embedding interactive elements into multimedia content - Google Patents

Abstract

The disclosure is related to a method and a system for automatically embedding interactive elements into multimedia content. In the method, a computer-implemented system first receives a multimedia content. The features in the content can be obtained by resolving the multimedia. The feature information including video features and audio features and the corresponding interactive events are stored in a memory in form of a comparison table. The comparison table is stored for comparing with the features resolved in the content. After the comparison, one or more time codes in conformity with the conditions can be obtained. One or more interactive elements corresponding to the time codes are embedded to the multimedia content. Every interactive event correlated to the interactive element is such as opening a webpage, executing software, displaying message or initiating an interactive screen. A new multimedia content is therefore created.

Description

Method and system for embedding interactive components in automatic multimedia content

The present invention is a method and system for embedding interactive components to multimedia content, and more particularly to a method and system for automatically embedding interactive components based on multimedia content and content features.

In the traditional process of watching TV or audio-visual content, the content provided by the TV station or the audio-visual content provider is always received in one direction. When the viewer sees the interested piece, for example, wants to know the audio-visual content, an attraction, a character, and a clothing. The details of the items need to be spent on finding and there is no effective way to get relevant information.

In the prior art, if the video content provider wants the viewer to obtain the specific information, the video content is manually viewed, and the specific information is manually inserted in a location where additional information needs to be generated, such as a certain time segment, by software means. When the audio and video content is played later, the player will detect this specific information while playing the audio and video content, and broadcast the related information.

The invention discloses a method and a system for embedding an interactive component to a multimedia content, which can automatically arrange a time code embedded with a specific triggering information by detecting a feature of the multimedia content, corresponding to a time period, and then output one or more embedded components. Trigger The new multimedia content of the newsletter can be used to broadcast the related content according to the triggering information when the multimedia content is played later.

According to one embodiment, in the process of the method for embedding an interactive component in an automatic multimedia content, a multimedia content is first received, and the multimedia content is parsed to obtain a video and audio feature, and then the audio and video features and the trigger feature information are compared. One or more time codes of the multimedia content that match the trigger feature information. The trigger feature information is a video feature that records a movie frame for comparing the audio and video features, and an audio feature. Each time code corresponds to an embedded interactive component, and sets an interaction event associated with the interactive component, for example, in the terminal device. Opening a web page, executing a software, displaying a message, or initiating an interactive picture, after completing the multimedia content of the embedded interactive component, outputting a new multimedia content including the interactive component and the associated interactive event.

In an embodiment, the new multimedia content is output to an audio-visual content service platform that provides a multimedia content service, and provides that each terminal device can access the multimedia content including the interactive component and the associated interactive event, when playing on the terminal device. In the case of multimedia content, the playback software detects the trigger feature, activates the interactive component in the time code that matches the set trigger feature, and executes the corresponding interactive event.

In an embodiment of the system for embedding interactive components in the automatic multimedia content, the system can be implemented by a computer system, wherein the software and/or the hardware implemented functional unit includes an analysis unit that parses the multimedia content to obtain the audio and video features therein; For comparing units of audio and video features and triggering feature information, one or more time codes of the multimedia content that meet the triggering feature information may be obtained; an editing unit in which each of the matching triggering feature information is obtained The time code corresponds to embedding an interactive component and setting an interaction event associated with the interactive component; and further comprising a video forming unit, forming a new multimedia content including the at least one interactive component and the associated interactive event.

The system can further include a memory unit in which the video features and/or audio features of the audiovisual features and corresponding interactive events are recorded.

In order to further understand the techniques, methods, and effects of the present invention for achieving the intended purpose, refer to the following detailed description, drawings, and The objects, features, and characteristics of the invention are to be understood as a part of the invention.

11‧‧‧Video Library

12‧‧‧Video and audio content service platform

10‧‧‧Automatic audio and video content embedded interactive component system

101,102,103‧‧‧ Terminal devices

201‧‧‧ analytical unit

202‧‧‧ comparison unit

203‧‧‧editing unit

204‧‧‧Video-forming unit

205‧‧‧Database

30‧‧‧Database

60‧‧‧ display screen

601‧‧‧ Jump out of the screen

602‧‧‧Mobile device screen

Step S301~S317‧‧‧ Flow of embedded interactive components in automatic multimedia content

Step S401~S413‧‧‧ Flow of embedded interactive components in automatic multimedia content

Step S501~S507‧‧‧ Flow of embedded interactive components in automatic multimedia content

1 is a schematic diagram of a system for embedding interactive components of an automatic multimedia content according to the present invention; FIG. 2 is a diagram showing an embodiment of a system function block for embedding interactive components of an automatic multimedia content according to the present invention; and FIG. 3 is a diagram showing a method for embedding interactive components of an automatic multimedia content according to the present invention; FIG. 4 is a second flowchart of an embodiment of a method for embedding an interactive component in an automatic multimedia content according to the present invention; FIG. 5 is a third flowchart of a method for embedding an interactive component in an automatic multimedia content according to the present invention; 6 shows a system embodiment diagram of the embedded interactive component of the automatic multimedia content of the present invention.

In order to automatically process audio and video content and embed specific events, the present invention provides a method and system for embedding interactive components in automatic audio and video content. The system is preferably implemented by a computer system through which multimedia content input to the system is input. Interactive components that can trigger specific events can be automatically embedded as needed, such as a marker that conforms to segments of embedded interactive components such as specific images (video), images, audio, time points, etc., such that when playing this multimedia content, The time code embedded with the interactive component will trigger the associated one or more interactive events. In the embodiment, the interactive event, such as displaying a QR code, can be linked to a web page (URL). Interactive events can also display information about an object (product), merchandising information, advertisements, etc., or broadcast another video or audio, or wireless signal. Assume When the interaction event is determined, the trigger mode may be set, that is, the mode for setting the event, including directly displaying on the display device, or playing the audio signal through the peripheral device of the display device, or through an external device. Perform interactive events.

1 shows a system diagram of an embedded interactive component of an automatic multimedia content of the present invention.

In the system diagram, in which the interactive content component system 10 is embedded in the automatic audio and video content, the system can input the multimedia content by the film library 11, and after processing by the system, the time code of the embedded interactive component is automatically compared, and can be in the multimedia content. The interactive component and the interactive event to be triggered are executed, and finally a new multimedia content is formed, which can be output to the audio-visual content service platform 12 as a digital content platform for providing access to the terminal device 101, 102, 103, so that the user can broadcast the content. In the case of new multimedia content, the broadcast program can perform corresponding interactive events when detecting interactive components, such as opening a web page on a terminal device, executing software, displaying a message, or initiating an interactive screen.

2 is a diagram showing an embodiment of a system function block of an embedded interactive component of an automatic multimedia content of the present invention.

In this example, the system is a system implemented by a computer system, wherein the function block is a functional unit that performs certain functions, and is a functional unit implemented by software and/or hardware, and an automatic interactive component system is embedded in the audio and video content. The video library 11 is connected to the video library 11 at one end, and the multimedia content input therein can be not only the video library 11, but also the audio and video content received by the instant streaming, and the audio and video content input from a specific computer device. The system 10 is connected to the audio-visual content service platform 12, for example, to transmit new multimedia content formed through the network to the audio-visual content service platform 12, to provide viewing by each terminal user, and to obtain additional information from the generated interactive event when the content is played. .

According to an embodiment, the function performed by the automatic audio-visual content embedded interactive component system 10 can be implemented by a software of a computer device or with a software, and includes a parsing unit 201 for parsing one of the multimedia contents received by the system 10 to The audio and video features are derived. When parsing, for example, parsing the multimedia content to get the information of each frame, the package Including picture information, audio information, you can get the characteristics of the image or audio.

The system may be provided with a database 205, which is a storage medium for recording system information, and may also be a memory unit in which the trigger feature information for comparing the audio and video features obtained by parsing the multimedia content is recorded, and is also a video feature and/or audio. The feature, and each trigger feature information is compared with an associated interaction event, and the trigger feature information and the interaction event may be represented by a comparison table and stored in the database 205 (or memory unit).

Then, the system compares the obtained audio and video features and various trigger feature information set by the system with a matching unit 202 to obtain one or more time codes of the multimedia content that meet the trigger feature information. The trigger feature information is a screen (video), an audio signal, or a specific time point to trigger a specific interactive event according to requirements. Then, the system is provided with an editing unit 203, and the above-mentioned information corresponding to the triggering feature can correspond to an interactive event, which can be preset by the system, and can be recorded in a comparison table in the foregoing manner, and the time interval corresponding to the triggering feature information is set. For a time code, the editing unit 203 can correspondingly embed an interactive component and set an interaction event associated with the interactive component. The interactive component is a tag, which can be a signal message, a picture, a piece of sound, a piece of text, a time stamp, or a digital watermark.

For example, the trigger feature information is a certain attraction image. When the frame image is automatically parsed out from the multimedia content, and the frame-by-frame or video segment is compared, the image segment matching the feature of the attraction image can be obtained. Corresponding to the time code of a certain time period, at this time, the embedded interactive component can be edited, and when the interactive component is detected, an attraction information, a travel suggestion, a related advertisement content, and the like are generated; For example, if the trigger feature information is set as a song segment, when the feature of the video segment is automatically parsed in the multimedia content, after the audio comparison, the video segment corresponding to the song segment is found, and the time code corresponding to the segment can also be obtained. At this time, the interactive components are embedded, and information, purchase information, associated advertisements, etc., which can generate related music content, are set. In addition, the trigger feature information can be set as a time point, that is, when the user views the multimedia content for a time, through the embedded interactive element. Initiate an interactive event, such as jumping out of an advertising message.

After completing the above interactive function and setting interaction event according to the trigger feature information, the video forming unit 204 rewrites or encodes the multimedia content to form a new multimedia content including at least one interactive component and the associated interactive event.

In the method flow of the embedded interactive component in the automatic multimedia content, as shown in the embodiment shown in FIG. 3, the foregoing system may input the audio and video content in the movie library or a specific source, as in step S301; and then analyze the audio and video content, as in step S303, including The video is parsed to obtain an image signal of each frame or video clip, or an audio signal or frequency in the audio; at this time, the audio and video characteristics of each video or audio can be obtained, as in step S305.

The audio and video features will be frame-by-frame or the trigger feature set by each video segment comparison system, such as the above-mentioned preset setting of the screen (video) to trigger a specific interactive event, audio signal, or a specific time point according to requirements, such as steps S307; and determining whether the trigger feature is met, as in step S309. When the comparison triggers the specificity, one of the embodiments is as described in the process, and the specific trigger feature information to be triggered is set in the comparison database 30, and after the comparison is matched, a mark is recorded, and the time code at this time can be recorded. Record it. The comparison of the interactive events to be triggered corresponding to the triggering features may be recorded in the database 30 or the specific memory device in a comparison table form.

The comparison step can be performed in various ways, such as instant parsing, comparison and editing when playing multimedia content, and each time a matching video or audio is obtained, or may include time information, and immediately associated with a specific interactive event. The interactive component; in another way, after the system completely parses the entire multimedia content, the time code of the matched video segment is recorded, and when the multimedia content is completely parsed and the video clip matching the trigger feature is obtained, the time is again according to each time. The code embeds the interactive component in the multimedia content.

In the comparison step of step S309, if there is no matching video segment in the process of instant parsing and comparison, the steps S303 and the like are repeated; until there is a video segment that matches the trigger feature information (Yes), the record is consistent. a certain time period (time code), as in step S311, the system automatically embeds the interactive component in this paragraph; At this time, the associated interaction event is set according to the matching trigger feature information, that is, the interaction event associated with setting the interaction component in one embodiment is to open a webpage, execute a software, display a message, or start a terminal device. Interactive screens (eg, voting, questionnaires, audience feedback), etc., as in step S313. In step S313, when the associated interaction event is set, the database 30 can be accessed again to set an interaction event corresponding to the triggering feature that is met at this time.

The step continues to determine whether the action of the embedded interactive component of the entire multimedia content is completed, such as step S315; if not, the step S303 and the like are continuously repeated; if it is completed, it indicates that the new multimedia content can be output, as in step S317.

In the flowchart of the method embodiment shown in FIG. 4, this example lists the final formed multimedia content to be output to the video service platform.

In this example, it is indicated that the audio and video content can be input from a specific video source (step S401), and the video content is automatically parsed by software or algorithm (step S403) to obtain each frame feature in the audio and video content, or each piece of audio. Feature (step S405), where the system can perform a specific video processing technology to obtain image features of each frame or image segment, or obtain each audio feature, refer to the prior knowledge of the related art, and not This statement.

The system continues to compare the trigger feature information that has been previously set according to the specific requirements with the parsed audio and video features. In step S407, a paragraph matching the feature is obtained, and one or more time codes of the multimedia content that meet the trigger feature information are obtained. The specific requirements described therein may set in advance the audio and video features in which the interactive components need to be embedded according to the attributes of the multimedia content. For example, if the multimedia content involves eco-teaching, the triggering features (video features, audio features) of the embedded interactive components may be used for images (plants, animals) of various organisms, or sounds generated by specific creatures (different types). Birds, animals, howling, etc.). When the comparison is made to obtain the matching feature information, that is, the interactive component of the trigger event embedded in the paragraph (covering a certain segment) (step S409), another embodiment may further include setting a trigger mode when setting the interactive component, and finally Forming new audio and video content (step S411), and outputting to the audio and video service platform (Step S413).

The setting of the trigger mode includes a manner in which a specific interaction event can be triggered by the system. When the interactive event associated with the interactive component is to open a webpage, execute a software, display a message, or initiate an interactive screen to perform some interaction with the viewer, the system can It is determined that the related information is displayed at a certain position, size, special effect indicating the message, volume of the played audio, and the like on the display of the terminal device, and is not limited to these methods. Even in a multi-screen environment it is possible to decide whether it can be synchronized on a particular mobile device.

When the user terminal plays the video and audio content in a playing software on the specific audio and video service platform by the terminal device, the playing process continuously obtains the video and audio information by the playing software, and compares whether there is an interactive component, and if there is an interactive component, an interactive event is triggered. A flow chart of a method for embedding an interactive component of an automatic multimedia content of the present invention, such as shown in FIG.

In this embodiment, the terminal device plays the video content (step S501), and continuously detects the interactive component. When the embedded interactive component is detected according to the audio-visual feature (step S503), the interactive event is triggered (step S505). At this time, the interactive event can be performed according to a specific trigger mode, including directly displaying the message on the terminal device display (step S507).

6 is a diagram showing an embodiment of a system for embedding interactive components of an automatic multimedia content of the present invention.

In this example, a display screen 60 of a terminal device is displayed. When viewing a multimedia screen in which an interactive component has been embedded, the player continuously detects the interactive component. When an interactive component is detected, the trigger mode is determined according to the setting. The interactive event is displayed or played out. In this example, the screen 601 can be displayed to display related information, such as displaying specific information, pictures, videos, advertisement contents, etc. Another possible trigger mode allows the user to trigger display in another mobile device screen 602. This interactive event.

Other interactive events include providing a link to a web page, such as a QR Code, which allows viewers to scan the QR code with a mobile device to direct access to specific URLs, as well as to pop out advertisements, product information, and other associated audio and video content. system End-to-end or multimedia content providers can also interact with the audience through this interactive event, such as making specific topics for voting, feedback, and so on.

In summary, the present invention provides a related method and system for automatically embedding interactive components according to preset trigger feature information, so that the system provider can automatically embed specific information in the multimedia content according to the preset requirements. It replaces the traditional one-way viewing method and the manpower and time spent manually setting up simple information, and the present invention proposes a more diverse interactive mode, in which a specific event can be triggered automatically according to the video clips that meet the requirements. Interactive components allow viewers to easily access additional information about the audio and video content, especially for interaction and advertising.

The preferred embodiments of the present invention are not intended to limit the scope of the present invention, and equivalent structural changes in the description and the drawings of the present invention are included in the scope of the present invention. Bright.

11‧‧‧Video Library

10‧‧‧Automatic audio and video content embedded interactive component system

201‧‧‧ analytical unit

202‧‧‧ comparison unit

203‧‧‧editing unit

204‧‧‧Video-forming unit

205‧‧‧Database

12‧‧‧Video and audio content service platform

Claims (10)

A method for embedding an interactive component in an automatic multimedia content, comprising: receiving a multimedia content; parsing the multimedia content to obtain a video and audio feature; comparing the audio and video features with the trigger feature information, and obtaining the multimedia that meets the trigger feature information One or more time codes of the content; each time code corresponds to embedding an interactive component and setting an interaction event associated with the interactive component; and outputting a new multimedia content including the at least one interactive component and the associated interaction event. A method of embedding an interactive element in an automatic multimedia content as claimed in claim 1, wherein the audiovisual feature in the multimedia content comprises a video feature of a movie frame in the multimedia content. A method of embedding an interactive element in an automatic multimedia content as claimed in claim 1, wherein the audiovisual feature in the multimedia content comprises an audio feature in the multimedia content. The method for embedding an interactive component in the automatic multimedia content according to claim 1, wherein the interactive event associated with the interactive component is set to open a webpage, execute a software, display a message, or initiate an interactive screen. The method for embedding an interactive component in an automatic multimedia content according to claim 4, wherein setting the interactive component further comprises setting a trigger mode. The method for embedding an interactive component in an automatic multimedia content according to any one of claims 1 to 5, wherein the triggering feature information of the audiovisual feature is a video feature and/or an audio feature, and an interaction event corresponding to each trigger feature information. Stored in a comparison table. A method of embedding an interactive component in an automatic multimedia content according to any one of claims 1 to 5, wherein the new multimedia content is output to a video content distribution platform that provides a multimedia content service. A system for embedding interactive components in an automatic multimedia content is a system implemented by a computer system. The functional unit implemented by a software and/or hardware in the system comprises: a parsing unit that parses a multimedia received by the system. Content, to obtain a video and audio feature; a comparison unit for comparing each of the audio and video features and the trigger feature information to obtain one or more time codes of the multimedia content that meet the trigger feature information; an editing unit Corresponding to each time code corresponding to the trigger feature information, an interactive component is embedded, and an interaction event associated with the interaction component is set; and a video and audio forming unit is formed to form at least one interactive component and the associated interactive event. New multimedia content. A system for embedding an interactive component in an automatic multimedia content according to claim 8 further comprising a memory unit, wherein a comparison table is stored, the comparison table recording video features and/or audio features of the audiovisual feature, and The corresponding interactive event. The system for embedding an interactive component in an automatic multimedia content according to claim 8 or 9, wherein setting an interactive event associated with the interactive component is to open a webpage, execute a software, display a message, or initiate a session on a terminal device. Interactive screen.