JP4177820B2 - Data input device and data input system - Google Patents

Description

  The present invention relates to a data input device and a data input system for inputting video attribute information (metadata), and more particularly, to a data input device and a data input system for inputting information on events occurring in a video.

  In recent years, many contents (particularly video contents) have been distributed due to the development of information infrastructure such as multi-channel broadcasting by digital satellite broadcasting. On the other hand, with the improvement of computer processing capacity and the spread of broadband networks such as ADSL and FTTH, it is common to record and view videos on a storage device on a personal computer or portable terminal.

  For home video, instead of the conventional tape media, the advent of a device with a built-in large-capacity hard disk allows multiple contents to be stored together and interactive viewing and editing utilizing random access. It has become.

  In order to efficiently handle a large amount of content, information such as an index is essential. Attribute information added to content such as an index is collectively referred to as metadata. In recent years, standardization has been performed as represented by MPEG-7 so that metadata can be interoperated in various environments and applications, and it becomes possible to edit and search content using metadata as a clue. ing.

A summarization technique for collecting and reproducing scenes of high interest according to user preferences has also been proposed. In these summarization techniques, events in a video are described as attribute information, and are automatically summarized according to matching with user preferences. For example, the importance for each event is obtained, and a summary video is created from the section of the corresponding event (see, for example, Patent Document 1).
Japanese Patent No. 3176893

  There are two methods for automatically inputting event information, such as a feature amount of content, and a method for manually inputting event information. Since automatic generation is limited in terms of quality, it is necessary to rely on manual input to provide high-quality services. However, manual input of event information is expensive and time consuming. For this reason, even if the video can be obtained by TV recording or the like, a service using metadata cannot be provided immediately.

  In view of the above-described problems, an object of the present invention is to provide a data input device and a data input system that smoothly provide metadata to a user.

  According to the data input device of the present invention, the time input means for inputting the time of the event occurring in the video as a temporary event occurrence time, the associating means for associating the event occurrence time with the video, and the video features A pre-stored reference feature amount indicating a feature of the target image including an event cause corresponding to an event result included in the image associated with the provisional event occurrence time; A first storage unit; a search unit that searches for a video whose difference between the reference feature value and the video feature value is equal to or less than a threshold value as the target video; and a time of an event associated with the target video as a true event occurrence time It is characterized by comprising second storage means for storing as:

According to the data input system of the present invention, in the data input system in which the time input terminal and the event content input terminal are connected via a network, the time input terminal temporarily calculates the time of an event occurring in the video. A time input means for inputting the event occurrence time, and a transmission means for transmitting the temporary event occurrence time to the event input terminal,
The event content input terminal includes a receiving means for receiving the temporary event occurrence time from the time input terminal, an associating means for associating the event occurrence time with a video, and a calculating means for calculating a video feature amount indicating a video feature. A first storage unit that stores in advance a reference feature amount indicating a feature of a target video including an event cause corresponding to an event result included in the video associated with the provisional event occurrence time; and the reference feature Search means for searching for a video whose difference between the amount and the video feature value is equal to or less than a threshold value as the target video is provided.

The data input system of the present invention is a data input system in which a time input terminal and an event content input terminal are connected via a network.
The time input terminal includes a time input means for inputting the time of an event occurring in the video as a temporary event occurrence time, an associating means for associating the event occurrence time with the video, and a video feature amount indicating a video feature And a first storage means for preliminarily storing a reference feature amount indicating a feature of a target video including an event cause corresponding to an event result included in a video associated with the provisional event occurrence time. Search means for searching for a video whose difference between the reference feature value and the video feature value is equal to or less than a threshold value as the target video; and a time of an event associated with the target video as a true event occurrence time. 2 storing means, and transmitting means for transmitting the true event occurrence time to the event input terminal,
The event content input terminal includes a receiving means for receiving the true event occurrence time from the time input terminal, a content input means for inputting event contents of the event, the true event occurrence time, and the event contents. And a related means for associating with each other.

Furthermore, the data input system of the present invention is a data input system in which a time input terminal and an event content input terminal are connected via a network.
The time input terminal includes provisional time input means for inputting the time of an event occurring in the video as a provisional event occurrence time, and transmission means for transmitting the provisional event occurrence time to the event input terminal. ,
The event content input terminal includes a receiving means for receiving the temporary event occurrence time from the time input terminal, an association means for associating the event occurrence time with the video, and a video associated with the temporary event occurrence time. True time input means for inputting the time of the event associated with the target video including the event cause corresponding to the included event result as the true event occurrence time, and second storage means for storing the true event occurrence time And content input means for inputting event contents of the event, and related means for associating the true event occurrence time with the event contents.

  According to the data input device and data input system of the present invention, metadata can be provided smoothly.

Hereinafter, a data input device and a data input system according to embodiments of the present invention will be described in detail with reference to the drawings.
As shown in FIG. 1A, the data input device of this embodiment includes a processor 101, a memory 102, a display unit 103, a video input unit 104, a video storage unit 105, an operation unit 106, and an event information storage unit 109. . The operation unit 106 includes a time input unit 107 and an event content input unit 108.

  The processor 101 performs overall control of the data input device according to the present embodiment, including activation and execution of software. The processor 101 searches for the event video that causes the phenomenon shown in the video from the video feature amount in the video at the time input by the time input unit 107. The processor 101 searches for an event video based on the similarity between the reference feature value and the video feature value. The video feature amount is, for example, at least one of video color information, video shape information, motion information indicating the degree of motion of an object in the video, and sound information accompanying the video. The processor 101 corrects the tentative event occurrence time input by the time input unit 107 with the time of the searched video as the true event occurrence time. Details of the control performed by the processor 101 will be described later with reference to FIGS. 2, 6, 11, 22, 24, 25, 27, 29, and 33.

  The memory 102 stores temporary data necessary for executing the software. The memory 102 stores a reference feature value for finding a video feature value of a certain video. The reference feature amount is, for example, at least one of color information, shape information, motion information, and sound information.

  The display unit 103 is for displaying information such as images, graphics, and characters to a data input person who is a user of the data input device of the present embodiment. The display unit 103 includes, for example, a control unit (not shown) for controlling the monitor and information to be displayed. The display unit 103 displays an image as shown in 110, for example. Reference numeral 110 denotes an example in which the display unit 103 displays a baseball broadcast video.

  The video input unit 104 inputs a video to be displayed on the display unit 103 from the video storage unit 105. This video is in an electronic file format, for example, and the video input unit 104 inputs this electronic file and acquires the video.

  The video storage unit 105 stores video in a predetermined format. The video storage unit 105 is composed of, for example, a hard disk. The video storage unit 105 stores video so that an arbitrary position of the video can be randomly accessed from the outside.

  The operation unit 106 is a part that is operated by a data input person, and is, for example, a device such as a keyboard, a mouse, or a remote controller.

  The time input unit 107 is for the data input person to register the time when the event occurred in the video. The data input person uses the time input unit 107 to input (register) the time in association with the video. The time input unit 107 operates when a data input person presses a button as indicated by 111 in FIG.

  The event content input unit 108 is for a data input person to input event content corresponding to a video. The event content is information indicating the content of the event (event) indicated by the video. For example, when the video is a baseball broadcast, there are hits, hits in the field, etc. as shown at 112 in FIG. 1B. Note that the method for inputting the contents of the event is not limited to selection by a button. Any method may be used as long as it is a method of inputting contents such as text input or selection from a list. Further, for the sake of simplification, in 112, only “play” is input for one event, but a plurality of items such as “team name + player name + play” may be input.

The event information storage unit 109 is a data storage unit that stores input event information. Here, the event information is information in which a true event occurrence time and event contents are set as one set. The true event occurrence time is the time when the corresponding event has occurred. As a term of this pair, there is a temporary event occurrence time. The provisional event occurrence time is different from the time when the event corresponding to the data input person occurs, but is relatively close to the time when the event occurred.
This temporary event occurrence time may be stored as event information including only time information in the event information storage unit 109 or may be stored in the memory 102.
As described above, the data input using the data input device of the present embodiment is event information generated in the video.

  The basic configuration shown in FIG. 1A is a configuration in which the display unit 103, the time input unit 107, and the event content input unit 108 exist in the same screen as 110, 111, and 112, respectively. Individual functions may exist on different devices. In FIG. 1B, each unit is configured by software, but may be configured by hardware such as a switch that can be actually pressed instead of a button on the screen.

Next, an example of the operation of the data input device according to the present embodiment will be described with reference to FIG. FIG. 2 is an example of a baseball game relay video for explaining the event.
In the example of FIG. 2, the pitcher pitches at the time of the video frame 201, and the batter hits the hit at the time of the video frame 202. In this example, for the sake of simplification, only the hit result is described as event information. Of course, each pitch may be an event. That is, in this example, an event has occurred in the video frame 202.

  The data input person views the video frame 202 displayed as 110 on the display unit 103, operates the time input unit 107 at timing 203, and the processor 101 registers a temporary event occurrence time. The temporary event occurrence time to be registered may be the moment when the time input unit 107 is operated, or may use a time that goes back a certain time in consideration of the time from the event occurrence to the operation. Further, this time may be changed according to the event contents described later. For example, the home run may be set to go back in the past for a longer time than the hit (single shot).

  Next, the event content input unit 108 is operated in accordance with the content of the event, and an appropriate item is selected and input. The registration of the event content takes a certain amount of time to select an item, and therefore is delayed from the time registration timing 203.

  In this example, for example, the time of the video frame 202 that is the scene of the hit result is registered as the time of the “hit” event. However, there are cases in which it is desired to view from the video frame 201, which is a pitching scene, which is a semantic delimiter, in such a case that a scene is searched for and viewed using event information. In this case, the processor 101 refers to the reference feature amount stored in the memory 102 and searches for a video frame having a video feature amount that is close to this reference feature amount to some extent. When searching for this video frame, the processor 101 compares the video feature quantity of each video frame with the reference feature quantity in the past from the timing 203. Then, when the difference between the feature amounts of the video feature amount and the reference feature amount is smaller than a certain threshold value, the time corresponding to the video frame having the video feature amount is assumed to be a true event occurrence time. This operation will be described later with reference to FIG.

Next, a description example of event information will be described with reference to FIG.
The event information includes true event occurrence time information 301 and event content information 302. Since the items of the event content information 302 change depending on the type of video and the purpose of use, these are merely examples. Of course, even in the case of baseball, contents having items other than those shown in FIG. 3 may be described. Further, if necessary, not only the result of the batting but also events related to the progress of the game such as throwing event information and changes and player changes may be described.

Next, a first modification of the data input device according to this embodiment will be described with reference to FIG.
A display unit 401 is a part of the display unit 103 and displays an image. The display unit 401 is, for example, a television receiver that displays a program being broadcast. The display unit 402 displays the operation units of the time input unit 107 and the event content input unit 108, and the data input person inputs data via the input devices 403 and 404 included in the operation unit 106. Others may be the same as in FIG. 1A, but the video input unit 104 and the video storage unit 105 may not be provided when processing using video is not required.

Next, a second modification of the data input device according to this embodiment will be described with reference to FIG.
A display unit 501 is a part of the display unit 103 and displays an image. The input device 503 is an input device having the functions of the time input unit 107 and the event content input unit 108. The input device 503 has, for example, a time registration button corresponding to the time input unit 107 on the remote controller, and the data input person operates these buttons while looking at the display unit 501. The control terminal 502 is a terminal that controls event input processing and stores event information, and takes the form of a computer, a hard disk recorder, a set-top box, or the like. If the event content information is unnecessary, the event content input unit 108 may not be provided. That is, event information including only time information may be stored in the event information storage unit 109 with only the display unit 501 for displaying video and the time input unit 107.

Next, an operation of the processor 101 searching for a video frame having a desired video feature amount will be described with reference to FIG. FIG. 6 is a processing flow for detecting a video frame having a desired feature amount using the time input by the time input unit 107 as a base point.
The processor 101 acquires the video feature quantity of the video frame to be detected from the memory 102 as a reference feature quantity (step S601). As the video feature amount, for example, color information such as a color histogram or distribution of a desired video frame, shape information such as an edge of the video frame, motion information such as a motion vector, or the like is used. The acquisition of the video feature amount may be performed by a data input person inputting the value of the video feature amount itself, or by inputting a video frame and extracting the feature amount from the video frame.

  The processor 101 sets the initial value of the video frame to be compared with the video feature amount acquired in step S601 as the time input by the time input unit 107 (step S602). That is, the first video frame to be compared with the video feature amount acquired in step S601 is the video frame at the time input by the time input unit 107.

  The processor 101 extracts the video feature quantity of the video frame to be compared with the video feature quantity acquired in step S601, compares the video feature quantity, and calculates the difference between the video feature quantities (step S603). If the difference is less than or equal to the threshold value, the process is terminated assuming that the compared video frame is a video frame to be detected. If the difference is not less than or equal to the threshold value, the process proceeds to step S604, and the video frame to be compared is shifted to the previous n (n = 1, 2,.

  If there is no video frame whose difference is equal to or less than the threshold, the value of n in step S604 may be reduced and the process may be continued from step S102 again. Alternatively, a process may be performed in which the video frame having the smallest difference is regarded as a video frame to be detected.

Next, an interface screen displayed on the display unit 103 for displaying a list of event information will be described with reference to FIG.
Each event display includes a thumbnail image 701, time information 702, and event content information 703. Symbol 704 indicates whether the event has been input only for time information or event content information has been input. The event information 705 indicates that the data input person is currently selected to input event content information.
In this way, event information 705 including only time information, and event information including time information 702 and event content information 703 are input separately, so that time information of event candidates is provisionally registered and suitable as an event. Only when the event content information is input, the main registration can be performed. Therefore, real-time input is possible even when the occurrence of events frequently occurs and the content input is not in time.

  In this example, the symbol 704 is used to distinguish event information including only time information and event information for which event content information has been input. However, the display method is not limited to this as long as both can be distinguished. As another example, for example, a frame, a background color, a character color, an icon, and the like may be changed.

Next, an interface screen different from FIG. 7 displayed on the display unit 103 for displaying a list of event information will be described with reference to FIG. FIG. 8 is another interface screen for distinguishing and displaying event information including only time information and event information including event content information.
The window 801 displays a list of event information that includes only time information and does not include event content information. The window 802 displays a list of event information including event content information. Each event information is composed of thumbnails, time information and the like as in FIG. The event information 803 is selected to input event content information. When the event content information is input, the event information 803 is deleted from the window 801 and displayed in the window 802.

Next, a case where a new function is added to the basic configuration described with reference to FIG. 1B will be described with reference to FIG. FIG. 9 shows screens 901 and 905 displayed on the display unit 103.
Buttons for performing functions added to the basic configuration shown in FIG. 1B are a rewind unit 902, an event deletion unit 903, and a return unit 904. Reference numeral 901 denotes a button in which these buttons are added to 110 and 111. A reference numeral 905 is obtained by adding these buttons to 112.
The rewinding unit 902 is for selecting an event in which time information is registered and playing a video from the corresponding time. The data input person can check the video at the time of the event occurrence in order to input the event content information.

  The event deletion unit 903 is for deleting an event for which time information is registered. First, a data input person temporarily registers time information of event candidates. When the data input person determines that the event is not appropriate, the data input person can delete the event, and when the data input person determines that the data input is not appropriate, the data input person can input the event content information and perform the main registration. Of course, the event deletion unit 903 may be used for deleting the registered event.

  The return unit 904 is for returning to the video frame immediately before the event input after performing the rewind for the event input. Details of the operation of the return unit 904 will be described later with reference to FIG.

  In FIG. 9, for the sake of convenience, the configuration added to the configuration in FIG. 1B is used, but individual windows and menus may be arranged differently from those in FIG. Moreover, the structure which does not have either part may be sufficient.

Next, event deletion will be described with reference to FIG. 10 using a baseball video as an example. The event information of the batting result is input as in FIG. 2, but the time information is the pitching time.
Both the video frame 1001 and the video frame 1002 are pitching videos and are event candidates. Therefore, the data input person inputs time at timings 1004 and 1006, and temporarily registers event information. However, the data input person regards the video frame 1001 as unnecessary because no hit suitable for an event has occurred, and deletes the event at timing 1005. On the other hand, the data input person inputs a content and registers the video frame 1002 at timing 1007 because a hit (hit) suitable for an event has occurred in the video frame 1003 thereafter.

Next, the operation of the return unit 904 will be described with reference to FIG.
The processor 101 waits for a rewind instruction from the data input person using the rewind unit 902 (step S1101), and proceeds to the next step when the rewind instruction is issued. The memory 102 stores the position of the video frame immediately before rewinding (step S1102). Next, the processor 101 waits for the input of the event information to end, and proceeds to the next step when it ends (step S1103). The processor 101 moves the playback video frame to the video frame stored in step S202, and resumes playback (step S1104).

(network)
Hereinafter, a data input system according to the present embodiment when inputting time information and event content information via a network will be described with reference to the drawings.
A case where the time input terminal 1201, the content input terminal 1202, and the event information database 1203 are connected via a network will be described with reference to FIG. The data input system includes a time input terminal 1201, a content input terminal 1202, and an event information database 1203.
The time input terminal 1201 is a terminal for inputting time information of an event occurring in the video. The time input terminal 1201 has the functions 110 and 111 in FIG.

  The content input terminal 1202 is a terminal for inputting event content information. The content input terminal 1202 has the functions 110 and 112 in FIG.

  The event information database 1203 is a database that stores input event information. The event information storage unit 109 shown in FIG.

Next, the operation of the data input system shown in FIG. 12 will be described with reference to FIG.
First, the time input terminal 1201 inputs event time information for a new event (step S1301). The time information input by the time input terminal 1201 is transmitted to the content input terminal 1202 via the network (step S1302).
The content input terminal 1202 receives time information from the time input terminal 1201 (step S1303). The event information displayed on the content input terminal 1202 is updated (step S1304). For example, the event information is displayed as shown in FIG. Further, the content input terminal 1202 has a function of correcting the time information, and when the time information needs to be corrected, the time received in step S1305 is corrected. The content input terminal 1202 inputs event content information (step S1306). Finally, the content input terminal 1202 registers event time information and event content information as event information in the event information database 1203 via the network.

Next, the operation of the data input system shown in FIG. 12 will be described with reference to FIG. Steps similar to those in FIG. 13 are denoted by the same reference numerals and description thereof is omitted.
The time input terminal 1201 registers the time information input in step S1301 in the event information database 1203 via the network (step S1401).
The content input terminal 1202 reads the database from the event information database 1203 via the network (step S1402). If a new event is registered in the database, the process proceeds to step S1304, and if not registered, the database is periodically read (step S1403).

(Network: Multiple terminals)
Each terminal installed on the network is not necessarily one, and there may be a plurality of content input terminals (1503, 1504,...) As shown in FIG. By adopting such a configuration, it is possible to input data by sharing among a plurality of data input persons.

Next, the operation of the data input system shown in FIG. 15 will be described with reference to FIG. FIG. 16 is a processing flow for distributing processing for inputting event content information.
First, the time input terminal 1502 transmits the input time information to each content input terminal (step S1601). Each content input terminal 1503, 1504 receives time information from the time input terminal 1502, and updates the display of event information (for example, the list display of FIG. 7 or FIG. 8) (step S1602). When displaying the event information, it may be displayed that another content input terminal is editing the event information. That is, it may indicate that the event information cannot be edited (edit lock) because another content input terminal is editing. In order to distinguish whether the editing is locked, the frame, background color, character color, icon, etc. are changed according to whether they are locked.

  The content input terminal 1503 selects one of the event information that can be edited from these event information, and starts editing (or input) (step S1603). The content input terminal 1503 transmits to the other content input terminal an instruction to lock the event information whose editing has been started so that the other input terminal cannot edit (step S1604). Upon receiving the lock instruction from the content input terminal 1503, the content input terminal 1504 locks the event information that the content input terminal 1503 has started editing so that it cannot be edited (step S1605). The content input terminal 1503 performs content input (step S1606), and when the content input is completed, transmits an editing lock release instruction to another content input terminal (step S1607). The content input terminal 1504 receives the edit lock release instruction, releases the edit lock, and updates the display of the event information (step S1608).

  In the example shown in FIG. 16, the content input at the content input terminal with the earliest input (editing) start timing is stored, but the present invention is not limited to this. For example, a content input terminal that can be edited or input may be determined based on the input end timing and the timing that the input is undertaken. Further, event information stored in the event information database 1203 may be determined by majority vote, first-come-first-served basis, or the like of the same input contents.

  Next, a processing flow different from FIG. 16 in which processing for inputting event content information is distributed will be described with reference to FIG. In this example, the time input terminal selects a content input terminal in charge of event content input. For example, the time input terminal selects a content input terminal that is not currently being edited. In addition, the time input terminal may select the content input terminals so that the number of events input by each content input terminal in the past and the time required for input by each content input terminal are equal. The time input terminal may monitor the load of each content input terminal and select the terminal with the smallest load.

  The time input terminal 1502 selects a content input terminal (step S1701). In the example of FIG. 17, the time input terminal 1502 selects the content input terminal 1503. The time input terminal 1502 notifies the content input terminal selected in step S1501 that it is in charge of input next (step S1702). The content input terminal 1503 displays that the terminal notified to the time input terminal 1502 is the content input terminal 1503 (step S1703). The time input terminal 1502 transmits the input time information to the input terminal (content input terminal 1503 in the example of FIG. 17) while permitting editing of the input time information (step S1704).

  The content input terminal 1503 receives the time information and updates the display (step S1705). The time input terminal 1502 transmits time information in a state where editing is locked to another content input terminal that is not in charge of input (the content input terminal 1504 in the example of FIG. 17) (step S1706). The content input terminal 1504 receives the time information and updates the display (step S1707). The content input terminal 1503 in charge of the input inputs the event content (step S1708), and when the content input is completed, information indicating that the input is complete is transmitted to the time input terminal 1502 (step S1709). Upon receiving the content input completion notification, the time input terminal 1502 transmits edit lock release instruction information to the content input terminal 1504 which is a non-input terminal (step S1710). The content input terminal 1504 receives the release instruction, releases the editing lock, and updates the display (step S1711).

  Note that steps S1702 and S1703 for giving a notice may be omitted. Further, the time information may be transmitted only to the terminal through which the time input terminal 1502 inputs contents, and in this case, steps S1706, S1707, S1709, S1710, and S1711 can be omitted. In this case, the content input terminal 1504 not in charge of content input periodically reads the database, and updates the display by notifying that another database has updated the database. Alternatively, it may be set so that only event information in charge of content input is displayed and no event information in charge is displayed. In this case, it is not necessary to update the display.

Next, a processing flow different from that in FIGS. 16 and 17 in which processing for inputting event content information is distributed will be described with reference to FIG.
The time input terminal 1502 registers the time information in the event information database 1203 (step S1801). The content input terminal 1503 searches the event information database 1203 for editable event information (step S1802). Here, “editable” means that editing is not locked by another input terminal. The content input terminal 1503 reads the event information found by searching in step S1802 from the event information database 1203, and edits and locks it so that other input terminals cannot edit it (step S1803). The content input terminal 1503 inputs event content information (step S1804). The content input terminal 1503 registers the event content information in the event information database 1203 and releases the edit lock.

As described above, the time input terminal inputs a temporary event occurrence time, and the event content input terminal obtains the true event occurrence time corresponding to the event based on the video feature amount, but is not limited to this configuration.
For example, not only the provisional event occurrence time is input at the time input terminal, but the true event occurrence time corresponding to the event may be obtained based on the video feature amount. In this case, the event content input terminal simply inputs the event content. As another example, at the content input terminal, the user may search for the true event time based on the video and input the event content of this event.

(Automatic detection)
The time input means described so far has been manually input by a data input person in accordance with the occurrence of an event while observing the video displayed on the display unit 103. Here, a method of automatically detecting the occurrence of an event and acquiring the time will be described with reference to FIG.
In a video, particularly a sports video, information indicating the score progress is often superimposed. As shown in the upper diagram of FIG. 19, for example, in a baseball video, the time, front and back, scores, ball count, and output status are displayed as 1901. In addition, at the time of scoring, as shown in the lower diagram of FIG. These are updated or newly displayed when the situation changes, that is, when an event occurs. Therefore, in the data input device of the present embodiment, the occurrence of an event is automatically detected by detecting the change in these images. The video is changed by setting a region in advance and comparing the feature values.

Next, a method for designating an arbitrary area of the screen in order to detect an event from information displayed on the screen will be described with reference to FIG. FIG. 20 shows an example of an interface for designating an arbitrary area of the screen in order to input such event detection information.
As shown in the upper diagram of FIG. 20, the data input person drags the pointer 2001 to set, for example, the detection area 2002. In addition, as shown in the lower diagram of FIG. 20, the data input person may determine the range of the detection area in advance, and the data input person may set the detection area by designating one point of the area with the pointer 2003. .
Alternatively, the user may specify information such as the detection area and coordinates in an electronic file, for example, and read the file into the memory 102 or the like to detect the event.

Next, an example of the data structure of event detection information will be described with reference to FIG. This event detection information is stored in the memory 102, for example.
The data of one detection area includes coordinate information 2101 (for example, upper left and lower right coordinates), feature amount information 2102, and corresponding event information 2103. The data structure is such that a plurality of these data can exist for each detection area. As the feature amount information, the feature amount of the immediately preceding video frame or the feature amount given in advance is used. In the case of the feature amount of the immediately preceding video frame, it is possible to detect a change in ball count and output status as in 1901 in FIG. When the feature amount is given in advance, a change in score such as 1902 can be detected.

Next, a flow of processing for automatically detecting an event will be described with reference to FIG.
The video input unit 104 reads one video frame (step S2201). The processor 101 reads information for one area from the event detection information stored in the memory 102 (step S2202). The processor 101 gives an instruction to compare the feature quantity of the area designated by the event detection information in the video frame with the feature quantity of the event detection area (step S2203). If the processor 101 detects that there is a change compared to the feature amount of the previous video frame, the event is less than the threshold value. An instruction for notifying the corresponding event is issued (step S2204). These steps S2202, S2203, and S2204 are performed for all event detection information (step S2205), and upon completion, the process proceeds to step S2206. These processes are repeated until all video frames are completed (step S2206).

Next, an example of an interface for displaying a detected event will be described with reference to FIGS. 23 (A) and 23 (B).
In the examples of FIGS. 23A and 23B, detected events that are input with event content information and those that are not input with event content information are displayed separately. For example, FIG. 23A shows an interface in which the horizontal axis 2301 is a time axis and the detected event position is marked. An event for which no event content information has been input is displayed with the mark changed as 2302.

  FIG. 23B shows an interface 2303 that displays a list of thumbnail images of detected events. An event for which no event content information has been input is displayed by changing the frame, background color, character color, and the like as 2304. In this way, the event content information that has not been input and the event that has already been input are displayed separately to eliminate oversight.

  Even when an event is automatically detected, the event time may be manually input. It is possible to confirm whether or not the detected event is input by performing event detection after manually inputting the event first.

Next, a processing flow for checking whether or not event information is input to an automatically detected event will be described with reference to FIG.
In FIG. 24, it can be used for checking both time information and event content information. That is, when checking the time information, it is used to check whether the event content information of the detected event is input in the check of the presence of the event information manually input previously and the check of the event content information.

  The processor 101 reads one event information automatically detected (step S2401). The processor 101 checks whether or not the automatically detected event information is stored in the event information storage unit 109 (step S2402). As described above, one or both of the time information check and the event content information check is checked. If no event information is input, the process advances to step S2403 to notify a non-input event (step S2403), and the display is updated. If event information has been input, the process proceeds to step S2404, and the processes from step S2401 to step S2403 are repeated until all detected events are checked.

Next, a processing flow for detecting a video frame having a desired feature amount based on the time of the detected event will be described with reference to FIG. FIG. 25 is similar to FIG. 6, but FIG. 6 shows a case where a video frame having a desired feature amount is detected based on the time of the input event. Steps similar to those in FIG. 6 are denoted by the same reference numerals and description thereof is omitted.
For example, the video frame of the pitch immediately before is detected from the time when the score is displayed as a base point. The processor 101 sets the initial value of the video frame to be compared with the video feature amount acquired in step S601 as the video frame at the time obtained by the event detection (step S2501). The processor 101 extracts the video feature quantity of the video frame obtained in step S2501 to be compared with the video feature quantity acquired in step S601, compares the video feature quantity, and calculates the difference between the video feature quantities (step S2502). If the difference is less than or equal to the threshold value, the process is terminated assuming that the compared video frame is a video frame to be detected. If the difference is not less than or equal to the threshold value, the process proceeds to step S604, the video frame to be compared is shifted n video frames before, and the processing step S2502 for performing comparison again is repeated.

(Special playback)
Next, a case where a new function is added to the basic configuration described with reference to FIG. 1B will be described with reference to FIG. FIG. 26 shows a screen displayed on the display unit 103. Buttons for performing functions added to the basic configuration shown in FIG. 1B are a special playback unit 2601 and a return unit 2602.
The special playback unit 2601 is a part that searches for an event occurrence location using special playback (trick play) such as video rewind or fast forward. In order to perform special reproduction, the video storage unit 105 that stores video is a hard disk or the like.

  The return unit 2602 returns to the video frame immediately before special playback (trick play) after the event input is completed. FIG. 27 shows a processing flow of the return unit 2602.

  The processor 101 waits for an instruction for special reproduction from the data input person (step S2701), and proceeds to the next step when an instruction for special reproduction is given. The memory 102 stores the video frame immediately before special reproduction is performed (step S2702). The processor 101 waits for the input of event information to end, and when it ends, proceeds to the next step (step S1103). The processor 101 moves the playback video frame to the video frame stored in step S2701, and resumes playback (step S2703).

  Note that special playback may use an external controller or the like without depending on the special playback unit 2601 and the return unit 2602. Further, instead of having the independent return unit 2602, it may be returned that the event input is completed when the event information is registered by the content registration unit 2603.

(1 button input)
Next, a case where a new function is added to the basic configuration described with reference to FIGS. 1A and 1B will be described with reference to FIG.
In FIG. 1A, the time input unit 107 and the event content input unit 108 are independent. However, in the configuration shown in FIG. 28, the time input unit 107 is not provided, but only the event content input unit 108 is provided. The time when the unit 108 is pressed is defined as time information. In addition, registration is not performed after selecting contents as shown in FIG. 1B, but registration is performed immediately upon selection.

Next, the processing flow in the case shown in FIG. 28 will be described with reference to FIG.
The processor 101 waits for the button 2802 connected to the event content input unit 108 to be pressed (step S2901). When the button 2802 is pressed, the processor 101 acquires the time of the video frame at the current position (step S2902). The processor 101 registers the acquired time information and the selected event content information (step S2903).

  For example, when the button 2802 corresponding to “hit” is pressed, event information having the time of pressing immediately as time information and “hit” as event content information is registered in the event information storage unit 109. The button portion 2801 can register not only a single item such as “hit” but also a plurality of items (for example, “XX player, hit”) at a time.

(2-screen display)
Next, an example in which the display unit 103 has two screens will be described with reference to FIGS. FIG. 30 shows an example of an interface of the data input device according to this embodiment.
The example shown in FIG. 30 has an event input screen 3001 and an event discovery screen 3002. The event input screen 3001 is a screen for inputting event time information. The event discovery screen 3002 displays an image of a future time from the event input screen. That is, the relationship between the video time on the event input screen 3001 and the video time on the event discovery screen 3002 is as shown in FIG. 31, for example.

  For example, the event discovery screen 3002 displays the latest video broadcast in real time, and the event input screen 3001 displays a video delayed by 5 seconds. The event discovery screen 3002 may display the latest video broadcast in real time, and the event input screen 3001 may display a video during special playback such as rewinding for event input.

  When an event occurs while viewing the event discovery screen 3002, the time at which the event occurred is input by observing the event input screen 3001. The timing of event input is measured using the interface shown in FIG. 30, and the button 2801 shown in FIG. 28 is pressed to register the time at which the event occurred and the contents of the event with one button.

  Next, a display method different from FIG. 30 will be described with reference to FIG. FIG. 32 shows another interface example of the data input device according to the present embodiment. The event input screen 3201 and the event discovery screen 3202 are switched and displayed. This is the same except that one screen is displayed at a time, which is different from FIG.

(After player information etc.)
Next, a process flow for inputting player information after inputting event information will be described with reference to FIG.
When performing sport video event input, the player name is not always known at the time of event input. There are many cases where players are introduced during the game. FIG. 33 is a processing flow for post-inputting player information in such a case.

  Player identification information that is information such as a player name is stored in the memory 102 in advance. The data input person registers the event information of the player whose player name is unknown at the time of the event using the player identification information (step S3301). When the data input person knows the player name, the player identification information stored previously is designated and player information such as the player name is input (step S3302). The processor 101 searches the event information for which the corresponding player identification information is input from the already registered event information, and updates the content information included in the event information (S3303). Next, when an event related to the player occurs, the data input person registers the event information using the input player information (S3304). In this way, the data input operation can be started without investigating player information before the game starts.

  According to the embodiment described above, event information input work necessary for providing metadata is broadcast by linking event occurrence time information and event content information based on the video feature amount included in the video. Can be input in real time or at a speed in accordance with real time. This makes it possible to provide metadata smoothly, and services and applications that use metadata can be used.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

(A) is a block diagram of the data input device concerning embodiment of this invention, (B) is the operation button which the display part of (A) displays. The figure for demonstrating the relationship between generation | occurrence | production of an event, and time. The figure which shows the example of description of event information. The figure which shows the 1st modification of the data input device concerning this embodiment. The figure which shows the 2nd modification of the data input device concerning this embodiment. The flowchart of the process for detecting the video frame which has a desired feature-value. The figure which shows an example of the interface displayed on the display part which displays a list of event information. The figure which shows an example of the interface different from FIG. 7 displayed on the display part 103 which displays event information as a list. The figure which shows the button displayed when a new function is added to the basic composition of FIG. The figure which shows the case where an event is deleted. 10 is a flowchart showing the operation of the return unit of FIG. The figure which shows the data input system with which a time input terminal, a content input terminal, and an event information database are connected via a network. The flowchart which shows an example of the operation | movement of FIG. The flowchart which shows another example of the operation | movement of FIG. The figure which shows the case where there are two or more content input terminals in FIG. The flowchart which shows an example of the operation | movement of FIG. The flowchart which shows another example of the operation | movement of FIG. The flowchart which shows another example of the operation | movement of FIG. The figure for demonstrating the automatic detection of an event. The figure for demonstrating the method of designating the area | region on a screen for event automatic detection. The figure which shows the data structure of the event detection information for event automatic detection. The flowchart which shows the operation | movement of event automatic detection. The figure which shows an example of the interface which displays the event detected automatically. The flowchart for checking whether event information is input into the automatically detected event. The flowchart of the process for detecting the video frame which has a desired feature-value on the basis of the time of the event detected automatically. The figure which shows the button displayed when a new function is added to the basic composition of FIG. The flowchart for returning to the video frame immediately before performing special reproduction (trick play). The figure which shows the button displayed when a new function is added to the basic composition of FIG. 29 is a flowchart for registering event information in relation to FIG. 28. The figure which shows that the display part of FIG. 1 (A) displays 2 screens. The figure which shows the position of each time of 2 screens of FIG. The figure which shows the case where 2 screens are switched and displayed. The flowchart for inputting player information after inputting event information.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Processor, 102 ... Memory, 103, 401, 402, 501 ... Display unit, 104 ... Video input unit, 105 ... Video storage unit, 106 ... Operation unit, 107 ... Time input unit, 108 ... Event content input unit, 109 ... event information storage unit, 201, 202, 1002, 1003 ... video frame, 203 ... time registration timing, 301 ... event occurrence time information, 302, 703 ... event content information, 403, 503 ... input device, 502 ... control terminal, 701 ... Thumbnail image, 702 ... Time information, 704 ... Symbol, 705, 803 ... Event information, 801, 802 ... Window, 901, 905 ... Screen, 902 ... Rewind part, 903 ... Event deletion part, 904 ... Return part, 1001 ... Video frame, 1201, 1502 ... Time input terminal, 1202,1503 1504 ... Content input terminal, 1203 ... Event information database, 2001, 2003 ... Pointer, 2002 ... Detection area, 2101 ... Coordinate information, 2102 ... Feature information, 2103 ... Corresponding event information, 2303 ... Interface, 2601 ... Special playback section, 2602... Restoration unit, 2603... Content registration unit, 2801, 2802... Button part, 3001, 3201 ... Event input screen, 3002, 3202.

Claims (18)

Time input means for inputting the time of an event occurring in the video as a temporary event occurrence time;
Means for associating the event occurrence time with the video;
A calculation means for calculating a video feature amount indicating a video feature;
First storage means for preliminarily storing a reference feature amount indicating a feature of a target video including an event cause corresponding to an event result included in a video associated with the provisional event occurrence time;
Search means for searching for a video whose difference between the reference feature and the video feature is equal to or less than a threshold as the target video;
A data input device comprising: a second storage means for storing the event time associated with the target video as a true event occurrence time. Content input means for inputting the event content of the event;
The data input device according to claim 1, further comprising association means for associating the true event occurrence time with the event content.   The searching means is based on at least one of video color information, video shape information, sound information accompanying the video, and motion information indicating the degree of motion of an object in the video as the video feature quantity and the reference feature quantity. 3. The data input device according to claim 1, wherein a target video is searched.   The search means searches for the target video in a range between the temporary event occurrence time and a temporary event occurrence time input before the temporary event occurrence time. The data input device according to any one of claims 1 to 3. A designation means for designating an area in which a change in the video feature amount is known in advance according to the occurrence of an event in the screen displaying the video;
5. The data input device according to claim 1, further comprising an acquisition unit configured to acquire a tentative event occurrence time based on a change in a video feature amount in the area. First reproducing means for reproducing the first video;
Second playback means for playing back a second video corresponding to a time before the first video;
6. The display apparatus according to claim 1, further comprising display means for displaying the first video and the second video simultaneously or by switching between the first video and the second video. The data input device according to item. A display means for displaying a list of events by distinguishing an event in which the event content is input and an event in which the event content is not input among a plurality of events in which the event occurrence time is stored;
A deletion unit for selecting an event from events for which no event content has been input, and deleting the event from the second storage unit;
A third playback means for selecting an event from events for which no event content has been input and playing back video from an event occurrence time corresponding to the event;
Third storage means for storing the position of the video frame that was displayed immediately before starting the reproduction;
The data input device according to any one of claims 2 to 6, further comprising first return means for returning to the video frame at the position. Change means for changing the position of the video frame to input the event content;
Position storage means for storing the position of the video frame immediately before the change when changing the position of the video frame;
The data input according to any one of claims 2 to 7, further comprising a second return means for returning to the position of the video frame immediately before the change when the input of the event contents is completed. apparatus. At least one assigned button corresponding to the content of the event content to be entered,
The data according to any one of claims 2 to 8, further comprising time determination means for determining the selected time as a true event occurrence time when the button is selected. Input device. Storage means for storing in advance identification information related to the event;
10. The apparatus according to claim 2, further comprising a capturing unit that captures, as event content, information included in the identification information corresponding to the event when the event content is input. 11. The data input device described. In a data input system in which a time input terminal and an event content input terminal are connected via a network,
The time input terminal is
Time input means for inputting the time of an event occurring in the video as a temporary event occurrence time;
Comprising transmission means for transmitting the temporary event occurrence time to the event input terminal;
The event content input terminal is:
Receiving means for receiving the temporary event occurrence time from the time input terminal;
Means for associating the event occurrence time with the video;
A calculation means for calculating a video feature amount indicating a video feature;
First storage means for preliminarily storing a reference feature amount indicating a feature of a target video including an event cause corresponding to an event result included in a video associated with the provisional event occurrence time;
A data input system comprising: search means for searching for a video whose difference between the reference feature value and the video feature value is a threshold value or less as the target video.   The time input terminal further comprises selection means for selecting a terminal that transmits the temporary event occurrence time based on the load of each event content input terminal when there are a plurality of event content input terminals. The data input system according to claim 11.   When there are a plurality of event content input terminals, the time input terminal selects a terminal that transmits the temporary event occurrence time, and indicates that the selection has been made prior to transmitting the temporary event occurrence time. The data input system according to claim 11 or 12, wherein the data input system transmits the selected data to a selected terminal.   12. The data input system according to claim 11, wherein, when there are a plurality of event content input terminals, the transmission means transmits the temporary event occurrence time to each event content input terminal. Each event content input terminal
An editing means for editing the content about the event,
Of the plurality of event content input terminals, when editing the event content corresponding to the temporary event occurrence time transmitted by the transmitting means first, edit the event content on another event content input terminal 15. The data input system according to claim 14, further comprising an edit lock means for locking the edit so as not to be performed. Each event content input terminal
An editing means for editing the content about the event,
Among the plurality of event content input terminals, the event content corresponding to the tentative event occurrence time transmitted by the transmission means, the timing to start editing, the timing to end editing, the timing to announce when editing, and the edited content are 15. The apparatus according to claim 14, further comprising editing lock means for locking editing so that other event content input terminals cannot edit the event content based on any one of the majority of similar things. The data input system described in 1. In a data input system in which a time input terminal and an event content input terminal are connected via a network,
The time input terminal is
Time input means for inputting the time of an event occurring in the video as a temporary event occurrence time;
Means for associating the event occurrence time with the video;
A calculation means for calculating a video feature amount indicating a video feature;
First storage means for preliminarily storing a reference feature amount indicating a feature of a target video including an event cause corresponding to an event result included in a video associated with the provisional event occurrence time;
Search means for searching for a video whose difference between the reference feature and the video feature is equal to or less than a threshold as the target video;
Second storage means for storing a time of an event associated with the target video as a true event occurrence time;
Comprising transmission means for transmitting the true event occurrence time to the event input terminal;
The event content input terminal is:
Receiving means for receiving the true event occurrence time from the time input terminal;
Content input means for inputting the event content of the event;
A data input system comprising a related means for associating the true event occurrence time with the event content. In a data input system in which a time input terminal and an event content input terminal are connected via a network,
The time input terminal is
Provisional time input means for inputting the time of an event occurring in the video as a provisional event occurrence time;
Comprising transmission means for transmitting the temporary event occurrence time to the event input terminal;
The event content input terminal is:
Receiving means for receiving the temporary event occurrence time from the time input terminal;
Means for associating the event occurrence time with the video;
A true time input means for inputting, as a true event occurrence time, an event time associated with a target image including an event cause corresponding to an event result included in an image associated with the provisional event occurrence time;
Second storage means for storing the true event occurrence time;
Content input means for inputting the event content of the event;
A data input system comprising a related means for associating the true event occurrence time with the event content.

Images