JP2010021638A - Device and method for adding tag information, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate tag information for searching the desired image frame of a user automatically and to add the tag information for a plurality of image frames. <P>SOLUTION: Based on the current position and map data detected by a GPS section 14, a moving picture processing device 16 extracts a geographical name near the current position as a perimeter keyword and recognizes the voice by means of a voice recognition unit 15, adds the perimeter keyword as the tag information to an image frame at a time point when the current position is detected if the voice is not recognized, and adds a noun extracted from the recognized voice as the tag information to an image frame at a time point when the voice is detected if the voice is recognized. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

The present invention relates to a tag information adding device, a tag information adding method, and a computer program for adding tag information for searching a user's desired image frame to a plurality of image frames.
The present invention particularly relates to tag information for searching for a desired image frame of a user for a plurality of image frames constituting a moving image file shot by a video camera or a still image file shot by a still camera or the like. The present invention relates to a tag information adding device, a tag information adding method, and a computer program.

As a conventional technique, Patent Document 1 below describes a method in which an operator performs an input operation for adding a tag to a frame in order to highlight a desired frame of a moving image being reproduced. Further, as another conventional technique, Patent Document 2 below describes a method in which an operator performs an input operation for adding a key image to a desired partial area in an image frame.
Japanese Patent Laying-Open No. 2005-181599 (FIG. 13) Japanese Patent Laying-Open No. 2007-19768 (FIG. 10)

  However, the above-described conventional technique has a problem that it takes an enormous amount of time for the input operation work because the operator needs to perform an input operation for adding a tag.

  In view of the above problems of the prior art, the present invention automatically generates tag information for searching for a desired image frame for a plurality of image frames constituting a moving image file or a still image file. It is an object to provide a tag information adding device, a tag information adding method, and a computer program that can be added.

In order to achieve the above object, the present invention automatically generates and adds tag information for searching for a user's desired image frame to a plurality of image frames captured and generated by the imaging means. A tag information adding device,
A current position detecting means for detecting a current position of the imaging means during imaging by the imaging means;
Based on the current position detected by the current position detection means, using map data having a geographical name, a peripheral keyword extraction means for extracting a geographical name near the current position as a peripheral keyword;
Voice recognition means for recognizing a user's voice during imaging by the imaging means;
First tag information addition for adding the peripheral keyword extracted by the peripheral keyword extraction unit as tag information to the image frame at the time when the current position is detected when no voice is recognized by the voice recognition unit Means,
Second tag information adding means for extracting a noun from the recognized voice and adding it as tag information to the image frame at the time when the voice is recognized when the voice is recognized by the voice recognition means; ,
It is characterized by having.

Further, the second tag information adding means compares the extracted noun with the peripheral keyword extracted by the peripheral keyword extracting means, and matches the noun to the image frame at the time when the voice is recognized. It is added as tag information.
Further, the first and second tag information adding means respectively have the same tag information for a predetermined number of image frames from the image frame when the current position is detected and the image frame when the sound is recognized. Is added.
Further, the plurality of image frames are moving image data shot by a video camera used as the imaging means.
Further, the plurality of image frames are still image files taken by a still camera used as the imaging unit or a camera built in a mobile phone.

In order to achieve the above object, the present invention automatically generates and adds tag information for searching for a user's desired image frame to a plurality of image frames captured and generated by the imaging means. Tag information adding method,
A current position detecting step for detecting a current position of the imaging means during imaging by the imaging means;
Based on the current position detected by the current position detection means and map data, a peripheral keyword extraction step for extracting a geographical name near the current position as a peripheral keyword;
A voice recognition step of recognizing a user's voice during imaging by the imaging means;
First tag information addition for adding the peripheral keyword extracted by the peripheral keyword extraction unit as tag information to the image frame at the time when the current position is detected when no voice is recognized by the voice recognition unit Steps,
A second tag information adding step for extracting a noun from the recognized voice and adding it as tag information to the image frame at the time when the voice is recognized when the voice is recognized by the voice recognition means; ,
It is characterized by having.

In order to achieve the above object, in the present invention, a computer automatically generates tag information for searching for a desired image frame of a user for a plurality of image frames captured and generated by an imaging means. A computer program for adding,
A current position detecting step for detecting a current position of the imaging means during imaging by the imaging means;
Based on the current position detected by the current position detection means and map data, a peripheral keyword extraction step for extracting a geographical name near the current position as a peripheral keyword;
A voice recognition step of recognizing a user's voice during imaging by the imaging means;
First tag information addition for adding the peripheral keyword extracted by the peripheral keyword extraction unit as tag information to the image frame at the time when the current position is detected when no voice is recognized by the voice recognition unit Steps,
A second tag information adding step for extracting a noun from the recognized voice and adding it as tag information to the image frame at the time when the voice is recognized when the voice is recognized by the voice recognition means; ,
It is characterized by having.

  With this configuration, tag information for searching for a user's desired image frame can be automatically generated and added to a plurality of image frames constituting a moving image file or a still image file.

  According to the present invention, tag information for searching for a desired image frame of a user is automatically added to a plurality of image frames constituting a moving image file or a still image file generated by being imaged by an imaging means. Can be generated and added.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a tag information adding apparatus according to the present invention, and FIG. 2 is a flowchart for explaining the operation of the moving picture processing apparatus of FIG.

  In FIG. 1, an outside camera 11 that is an image capturing unit captures a surrounding moving image of a vehicle (not shown) such as a front, and an in-vehicle camera 12 that is another image capturing unit captures an in-vehicle moving image such as a driver inside the vehicle. Are arranged as follows. The in-vehicle microphone 13 records the voice of the driver in the vehicle, and the GPS (Global Positioning System) unit 14 acquires the current position (latitude, longitude) of the vehicle. The voice recognition device 15 recognizes the voice recorded by the in-vehicle microphone 13. The moving image processing device 16 is configured to obtain audio data obtained by the in-vehicle microphone 13 and the voice recognition device 15 for an arbitrary frame among a plurality of frames constituting each moving image captured by the in-vehicle camera 11 and the in-vehicle camera 12; A search tag or the like is automatically generated and added based on the current position acquired by the GPS unit 14 and recorded in the moving image recording device 17 as tagged moving image data. Here, the moving image processing device 16 stores map data in advance, and acquires a geographical name (name of place, name of mountain, river, bridge, building, etc.) in the vicinity of the current position as a peripheral key code from the map data. be able to. The number of peripheral key codes to be acquired is preferably a plurality, for example, about 20.

  The operation of the moving image processing device 16 will be described with reference to FIG. Here, the structure of the moving image information of the moving image data includes a moving image data name (for example, a number indicating the shooting order), a keyword serving as a tag for searching for a user's desired image frame, and a start frame to which the same tag is assigned. Number, end frame number, position data (latitude, longitude) and the like are included. Here, the number of frames to which the same tag is assigned is uniquely 100 frames (= 1 block) so that it can be easily seen at the time of search display.

  First, it is checked whether there is moving image data from the outside camera 11 and the inside camera 12 (step S1). If the moving image data exists in step S1, the process proceeds to step S2, and the moving image data name is described in the moving image information. Next, it is checked whether or not position data from the GPS unit 14 exists (step S3). If there is no position data, the process proceeds to step S11, and the moving image information is recorded in the moving image recording device 17. If position data exists in step S3, the process proceeds to step S4, and the position data is described in the moving image information. Next, one or more geographical names near the current position are extracted as peripheral keywords based on the position data and map data (step S5).

  Next, it is checked whether or not there is voice data from the voice recognition device 15 (step S6). If there is no audio data, the process proceeds to step S12, where the moving image frame number at the time when the position data is acquired is set as the start frame number, and the frame after 100 frames is set as the end frame number, and one or more extracted from the position data in step S5 The peripheral keyword is described in the tag of the moving image information, and then the process proceeds to step S11 to record the moving image information in the moving image recording device 17.

  If there is speech data in step S6, the process proceeds to step S7, where a noun is extracted from the speech data, and then the extracted noun is matched with one or more peripheral keywords extracted from the location data in step S5 ( Step S8). If not matched, one or more peripheral keywords extracted from the position data branching to step S9 are described in the video information tag. On the other hand, if matched, the process proceeds to step S10 and the matched noun is converted to the video information. Describe in the tag. In steps S9 and S10, the moving image frame number at the time when the audio data is acquired is set as the starting frame number, and the frame after 100 frames is set as the ending frame number. Then, the process proceeds to step S11 and moving image information is recorded in the moving image recording device 17.

  FIG. 3 shows an example of moving image information given to moving image data shot in the vicinity of Mt. Fuji. In FIG. 3, latitude data (latitude) and longitude data (Longitude) and a keyword = Gotemba are assigned to one block from the start frame number = 1 to the end frame number = 100 of the video data of Mt. Fuji. An example is shown in which latitude data (latitude) and longitude data (Longitude) and keyword = Mt. Fuji are assigned to one block from frame number = 90 to end frame number = 189.

  FIG. 4 shows an example of moving image information given to moving image data of Mt. Fuji for which no audio data exists. In FIG. 4, latitude and longitude data and peripheral keywords extracted from position data in step S5 in FIG. 3 = Gotemba, Mt. Fuji, and Ashigara are assigned to one block from start frame number = 1 to end frame number = 100. An example is shown.

  FIG. 5 shows an example of the moving image information given to the moving image data of Mt. Fuji in which the audio data “Today's Mt. Fuji, beautiful” exists from the middle of the moving image data (frame number = 90). In FIG. 5, latitude and longitude data and peripheral keywords extracted from the position data in step S5 = Gotemba are assigned to one block from start frame number = 1 to end frame number = 100, and start frame number = An example in which latitude and longitude data and the noun “Mt. Fuji” extracted by speech recognition are assigned to one block from 90 to end frame number = 189.

<Application example of the present invention>
FIG. 6 shows a system to which the tag information adding apparatus 10 according to the present invention is applied. In the system shown in FIG. 6, the tag information adding device 10 according to the present invention is mounted on a vehicle V and used as a moving image database (DB) 10a, and constitutes a surrounding moving image and an in-vehicle moving image shot while the vehicle V is traveling. A search tag is automatically generated and added to any frame of a plurality of frames that are recorded while the vehicle V is traveling based on the current position and recorded as tagged moving image data. To do. The moving image data recorded in the tag information adding device 10 (moving image DB 10a) is uploaded to the server 1 and downloaded from the server 1 to the mobile phone 2 of the user Y, the PC (personal computer) 3, and the in-vehicle information terminal 4.

  The imaging means of the present invention is not limited to a video camera, but can be applied to a normal portable video camera, a digital still camera, and a camera with a built-in mobile phone. Also, the tag addition target video is not limited to a moving image, and can be applied to an image file composed of a plurality of still images.

  The present invention automatically generates tag information for searching for a user's desired image frame for a plurality of image frames constituting a moving image file or a still image file generated by being picked up by an image pickup means. And can be used for a video camera, a still camera, a mobile phone, and the like.

It is a block diagram which shows one Embodiment of the tag information addition apparatus which concerns on this invention. 3 is a flowchart for explaining the operation of the moving image processing apparatus of FIG. 1. It is explanatory drawing which shows moving image information. It is explanatory drawing which shows the moving image information when there is no audio | voice data. It is explanatory drawing which shows the moving image information when there exists audio | voice data. It is explanatory drawing which shows the system to which the tag information addition apparatus which concerns on this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Out-of-vehicle camera 12 In-vehicle camera 13 In-vehicle microphone 14 GPS part 15 Voice recognition device 16 Movie processing device 17 Movie recording device

Claims (7)

A tag information adding device that automatically generates and adds tag information for searching for a desired image frame of a user to a plurality of image frames that are captured and generated by an imaging unit,
A current position detecting means for detecting a current position of the imaging means during imaging by the imaging means;
Based on the current position detected by the current position detection means, using map data having a geographical name, a peripheral keyword extraction means for extracting a geographical name near the current position as a peripheral keyword;
Voice recognition means for recognizing a user's voice during imaging by the imaging means;
First tag information addition for adding the peripheral keyword extracted by the peripheral keyword extraction unit as tag information to the image frame at the time when the current position is detected when no voice is recognized by the voice recognition unit Means,
Second tag information adding means for extracting a noun from the recognized voice and adding it as tag information to the image frame at the time when the voice is recognized when the voice is recognized by the voice recognition means; ,
Provided tag information adding device.   The second tag information adding means compares the extracted noun with the peripheral keyword extracted by the peripheral keyword extracting means, and matches the noun with respect to the image frame at the time when the voice is recognized. The tag information adding device according to claim 1, wherein the tag information adding device is added.   The first and second tag information adding means respectively add the same tag information to a predetermined number of image frames from the image frame at the time when the current position is detected and the image frame at the time when the sound is recognized. The tag information adding device according to claim 1, wherein the tag information adding device is a tag information adding device.   The tag information adding apparatus according to any one of claims 1 to 3, wherein the plurality of image frames are moving image data photographed by a video camera used as the imaging unit.   The plurality of image frames are still image files photographed by a still camera used as the imaging unit or a camera built in a mobile phone. 4. Tag information adding device. A tag information adding method for automatically generating and adding tag information for searching for a user's desired image frame to a plurality of image frames captured and generated by an imaging means,
A current position detecting step for detecting a current position of the imaging means during imaging by the imaging means;
Based on the current position detected by the current position detection means and map data, a peripheral keyword extraction step for extracting a geographical name near the current position as a peripheral keyword;
A voice recognition step of recognizing a user's voice during imaging by the imaging means;
First tag information addition for adding the peripheral keyword extracted by the peripheral keyword extraction unit as tag information to the image frame at the time when the current position is detected when no voice is recognized by the voice recognition unit Steps,
A second tag information adding step for extracting a noun from the recognized voice and adding it as tag information to the image frame at the time when the voice is recognized when the voice is recognized by the voice recognition means; ,
Provided tag information addition method. A computer program for automatically generating and adding tag information for searching for a user's desired image frame to a plurality of image frames captured and generated by an imaging means,
A current position detecting step for detecting a current position of the imaging means during imaging by the imaging means;
Based on the current position detected by the current position detection means and map data, a peripheral keyword extraction step for extracting a geographical name near the current position as a peripheral keyword;
A voice recognition step of recognizing a user's voice during imaging by the imaging means;
First tag information addition for adding the peripheral keyword extracted by the peripheral keyword extraction unit as tag information to the image frame at the time when the current position is detected when no voice is recognized by the voice recognition unit Steps,
A second tag information adding step for extracting a noun from the recognized voice and adding it as tag information to the image frame at the time when the voice is recognized when the voice is recognized by the voice recognition means; ,
Computer program provided.

Images