JP2019197210A - Speech recognition error correction support device and its program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice recognition error correction support device capable of promptly correcting a voice recognition error while suppressing a waiting time at a plurality of correction terminals. SOLUTION: A voice recognition error correction support device 1 acquires a correction content for a recognition result via a voice recognition means 11 for generating a voice recognition result and time information for each word, and a plurality of correction terminals 2. A plurality of editing means 15 for correcting an error by editing, and the editing means 15 displays the editing screen control means 151 for displaying the editing screen including the recognition result, and the time of the word in the text specified on the editing screen. Based on the information, the recognition result is updated with the postscript priority by using the sound reproduction unit 153 that outputs a corresponding sound and the correction content corrected on the editing screen, and the other updating unit is notified of the update notification. Error correction means 152 for reflecting the information. [Selection diagram] Figure 2

Description

  The present invention relates to a speech recognition error correction support device that supports error correction in speech recognition and a program thereof.

  When using the sound of audio material (including video / audio material) recorded in a program interview, etc. as a character, the transcription work of the audio is indispensable. Usually, this work is performed by the operator listening to the sound of the recorded material and inputting characters using a keyboard or the like of the terminal. At this time, the worker frequently repeats playback and stop of the sound or listens to the sound at the same location many times. It is said that it takes about six times as much work time.

Disclosed is a technology that supports real-time voice transcription work by recognizing live streaming content in real time, and allowing multiple unspecified users to input correct character strings for errors in the speech recognition results. (See Patent Document 1).
In addition, as a technology to support the transcription work, speech recognition processing is performed for each sentence (cell) in which the input speech is divided into arbitrary units, and the worker recognizes the recognition result after the speech recognition processing and responds to this. A technique for correcting an error in speech recognition processing by comparing with speech to be performed is disclosed (see Patent Document 2). In this technology, voice is reproduced in units of cells in the speech recognition process, the operator corrects the recognition result in units of cells, and the cells are corrected by a general text editor operation. Further, in this technique, after the cell is corrected, the worker reproduces the sound from the head of the cell and confirms whether or not the recognition result is correctly corrected.

JP 2013-29684 A Japanese Patent Laying-Open No. 2015-184564

In the technique disclosed in Patent Document 1, since the transcription target is live streaming content, it is not possible to check the content contents retroactively. Therefore, this technique has a problem that errors remain when there are many speech recognition errors, or when the utterance content is unclear and it is necessary to listen again and again.
Since the technique disclosed in Patent Document 2 performs sound reproduction and correction of recognition results in cell units, even if there are few correction points, it is necessary to check whether the correction points and the correction results match. In addition, it is necessary to reproduce audio from the beginning of the cell. For this reason, this technique has a problem that a waiting time occurs until the sound of the correction target portion in the middle of the cell is reproduced.

  Therefore, the present invention enables speech reproduction in units of words, and even if the speech recognition has been completed or the content is being speech-recognized, the speech recognition error can be quickly performed by suppressing waiting time at a plurality of correction terminals. It is an object of the present invention to provide a speech recognition error correction support device and a program thereof that can correct the error.

  In order to solve the above problems, a speech recognition error correction support device according to the present invention inputs a material content, and corrects a speech recognition error for a speech included in the material content via a plurality of correction terminals. A correction support device comprising a voice recognition means, a storage means, and a plurality of editing means, wherein the editing means comprises an editing screen control means, a voice output means, and an error correction means. .

In such a configuration, the speech recognition error correction support device recognizes speech by the speech recognition means, generates a recognition result that is text data, and generates time information for each word constituting the recognition result. This time information is a relative elapsed time from the top of the material content. Thereby, the voice recognition means associates the relative elapsed time from the top of the material content with each recognized word.
Further, the speech recognition error correction support device stores the recognition result and the time information in the storage unit together with the material content.
Then, the speech recognition error correction support device acquires the correction content for the recognition result via the correction terminal by the editing means corresponding to the plurality of correction terminals, and corrects the speech recognition error.

Here, the editing means displays the editing screen including the recognition result of the material content to be corrected on the correction terminal by the editing screen control means.
The editing means outputs the corresponding voice to the correction terminal based on the time information of the word in the text designated on the editing screen. As a result, the operator can check the speech-recognized text and speech for each word.
Then, the speech recognition error correction support device updates the recognition result stored in the storage means with the postscript priority (overwrite) using the correction content corrected on the editing screen by the error correction means, and the recognition result is An update notification to the effect that the postscript has been written is sent to the other editing means to reflect the correction content corresponding to the correction terminal. As a result, the operator can recognize the corrected portion corrected by another correction terminal.

Note that the voice recognition error correction support device may include a voice recognition unit outside.
Further, the speech recognition error correction support apparatus can be operated by a speech recognition error correction support program for causing a computer to function as each of the above-described means.

The present invention has the following excellent effects.
According to the present invention, the recognition result obtained by voice recognition of the material content can be displayed on the editing screen of the correction terminal, and the recognition result can be grasped by reproducing the voice at an arbitrary word position. Can share corrections and correct recognition errors.
Further, according to the present invention, by performing correction by a plurality of correction terminals with priority on the postscript, the waiting time of the correction terminals can be reduced without performing exclusive control between the correction terminals.

It is a block block diagram which shows the structure of the speech recognition error correction system which concerns on embodiment of this invention. It is a block block diagram which shows the structure of the speech recognition error correction assistance apparatus which concerns on embodiment of this invention. It is explanatory drawing for demonstrating the memory content which a raw material information storage means memorize | stores. It is explanatory drawing for demonstrating the content of voice recognition information. It is explanatory drawing for demonstrating the content of segment information. It is a figure which shows the example of the screen which selects material content, Comprising: (a) is an example of a selection screen, (b) is an example of a list box, (c) is an example of a selection screen when a current time setting button is pressed. . It is explanatory drawing for demonstrating the start time and end time at the time of selecting material content. It is a screen block diagram which shows an example of the item list screen which shows the list of the item which divided | segmented the voice recognition result of material content. It is a screen block diagram which shows the example which expanded the voice recognition result to the edit area | region on the item list screen. It is explanatory drawing for demonstrating the edit screen in a different correction terminal, and demonstrating the state which is correcting the same object. It is explanatory drawing for demonstrating an example of the edit operation | work in an edit area. It is explanatory drawing for demonstrating an example of the correction result which a correction result output means produces | generates. It is a flowchart which shows the segment information production | generation operation | movement which produces | generates the speech recognition result of the speech recognition error correction assistance apparatus which concerns on embodiment of this invention per segment. It is a flowchart which shows the segment information presentation operation | movement which shows the speech recognition result of the speech recognition error correction assistance apparatus which concerns on embodiment of this invention to a correction terminal per segment. It is a flowchart which shows the segment correction operation | movement which corrects a recognition result, performing voice reproduction of the speech recognition error correction assistance apparatus which concerns on embodiment of this invention. It is a block block diagram which shows the structure of the speech recognition error correction assistance apparatus which concerns on embodiment of the modification of this invention. It is a screen block diagram which shows the example which displayed time information on the edit area | region on the item list screen. It is explanatory drawing for demonstrating the difference in the display time by the presence or absence of a time code. It is a screen block diagram which shows an example of an item list screen, Comprising: (a) is an example without a time code, (b) shows an example when there is a time code. It is a screen block diagram which shows an example of the item list screen which displayed the context menu. It is a screen block diagram which shows the example which encodes and displays a selection character string with a two-dimensional code, Comprising: (a) is a screen which displayed the two-dimensional code, (b) is a figure which shows the content of the encoded character string It is.

Embodiments of the present invention will be described below with reference to the drawings.
[Outline of speech recognition error correction system]
First, an outline of a speech recognition error correction system S including a speech recognition error correction support device 1 according to an embodiment of the present invention will be described with reference to FIG.

The speech recognition error correction system S corrects a speech recognition error in material content including at least speech. In the present embodiment, the material content is content composed of video and audio, for example, broadcasting material.
The speech recognition error correction system S includes a speech recognition error correction support device 1 and a plurality of correction terminals 2 (2,..., 2).

The speech recognition error correction support apparatus 1 inputs material content and corrects speech recognition errors for speech included in the material content via a plurality of correction terminals 2.
This speech recognition error correction support device 1 sequentially recognizes material content, associates recognition results with speech for each word, and can perform speech playback from an arbitrary word position designated by the operator on the correction terminal 2. And In addition, the voice recognition error correction support device 1 sequentially reflects the corrections made by the plurality of correction terminals 2, and if there are different corrections at the plurality of correction terminals 2, the subsequent correction is made effective without performing exclusive control. .

The correction terminal 2 corrects an error included in a recognition result which is text data recognized by the voice recognition error correction support apparatus 1. The correction terminal 2 includes an input device such as a keyboard, a display device that displays an editing screen, a speaker that displays sound, and the like. The display device may include a touch panel.
The speech recognition error correction support apparatus 1 and the correction terminal 2 are connected via the Internet, an intranet, or the like.
Thus, the speech recognition error correction support apparatus 1 corrects speech recognition errors by the plurality of correction terminals 2 without performing exclusive control on the same material content.

As a result, the speech recognition error correction system S can confirm the speech and the recognition result in units of words even for the material content during speech recognition, and does not perform exclusive control. The terminal 2 can quickly correct a speech recognition error.
Hereinafter, the configuration and operation of the speech recognition error correction support apparatus 1 will be described.

[Configuration of voice recognition error correction support device]
First, the configuration of the speech recognition error correction support device 1 will be described with reference to FIG.
As shown in FIG. 2, the speech recognition error correction support apparatus 1 includes a material content input unit 10, a speech recognition unit 11, a recognition result division unit 12, an item information extraction unit 13, a material information storage unit 14, Editing means 15 (15,..., 15) and information deletion means 16 are provided.

The material content input means 10 inputs material content.
The material content input unit 10 may input material content from an external storage medium, for example, or input via a communication line.
The material content input unit 10 outputs the audio of the input material content to the voice recognition unit 11. The material content input means 10 writes and stores the input material content (video / audio) in the material information storage means 14 for use in correction work in the editing means 15 described later.

The voice recognition unit 11 recognizes the voice of the material content input by the material content input unit 10 and generates a recognition result as text data and time information for each word constituting the recognition result.
The voice recognition unit 11 performs voice recognition using a language model, an acoustic model, and a pronunciation dictionary (not shown), and generates a recognized word and time information indicating an elapsed time from the head of the voice of the word. The voice recognition unit 11 sequentially notifies the recognition result dividing unit 12 of the generated recognition result words and time information, and writes and stores them in the material information storage unit 14.
Note that the speech recognition method in the speech recognition means 11 may be, for example, a method of recognizing a word string from speech disclosed in JP 2010-175765 A and outputting the result.

The recognition result dividing unit 12 divides the recognition result (word string) recognized by the voice recognition unit 11 according to a predetermined reference. Hereinafter, each block (word string) of the divided recognition results generated by the recognition result dividing unit 12 is referred to as a segment.
As a division criterion used by the recognition result dividing unit 12, an arbitrary criterion can be determined in advance.
For example, a silent section of speech can be used as a division criterion. In this case, the recognition result dividing unit 12 uses the time information of the speech recognition result acquired from the speech recognition unit 11, and determines that the interval between words is longer than a predetermined time as a silent section, Split before and after the section.

  Further, for example, a cut point of the video can be used as a reference for division. In this case, the recognition result dividing unit 12 detects, from the video stored in the material information storage unit 14, a frame whose image feature of an adjacent frame is significantly different from a predetermined reference as a cut point, and the time of the cut point Divide the recognition result before and after.

  Further, for example, meta information previously added to the material content may be used as a division criterion. The meta information includes GPS (Global Positioning System) position information (geotag) and the like. In this case, the recognition result dividing unit 12 divides the recognition result at a point in time where the location where the material content is shot or collected differs according to the position information.

  The recognition result dividing unit 12 outputs the segment obtained by dividing the speech recognition result to the item information extracting unit 13. The recognition result dividing unit 12 writes and stores the word and its time information in the material information storage unit 14 for each segment.

The item information extraction unit 13 extracts a feature word included in the segment as an item for each segment divided by the recognition result division unit 12.
This characteristic word is a characteristic word included in the segment. For example, the item information extraction unit 13 extracts a word characterizing a segment by a TF-IDF method (TF: Term Frequency, word appearance frequency, IDF: Inverse Document Frequency, inverse document frequency). The TF-IDF is a kind of weight related to a word in a document (in this embodiment, a segment), and is mainly used in fields such as information retrieval and sentence summarization.
Specifically, the item information extraction unit 13 calculates the appearance frequency tf (w, s) of the word w in the segment s by the following equation (1).

In this equation (1), n _{w, s} represents the number of appearances of a word w in the segment s, and _Σtεs n _{t, s} represents the sum of the number of occurrences of all words in the segment s.
Further, the item information extraction unit 13 calculates the inverse document frequency idf (w) of a certain word w by the following equation (2).

In this equation (2), N represents the total number of segments in the material content, and df (w) represents the number of material content segments in which a certain word w appears (total number of segments [total number of documents]).
Then, for each word in the segment, the item information extraction means 13 is the word having the largest product of the tf value of the expression (1) and the idf value of the expression (2), as shown in the following expression (3): Alternatively, a word larger than a predetermined reference value is set as a feature word of the segment.

The item information extraction unit 13 writes and stores the extracted item in the material information storage unit 14 in association with the segment.
In addition, the item information extraction means 13 is good also as extracting a noun and a proper noun as a feature word by carrying out the morphological analysis of a segment, without using TF-IDF method.

  In addition, when the material content includes video, the item information extraction unit 13 may extract a thumbnail image from the video in the time section corresponding to the segment, in addition to the feature word. For example, the item information extraction unit 13 extracts the first frame of the video in the time interval corresponding to the segment as a thumbnail image. The item information extraction unit 13 writes and stores the extracted thumbnail image in the material information storage unit 14 in association with the segment.

  The material information storage unit (storage unit) 14 stores material content that is a target for correcting an error in speech recognition and various types of information obtained by dividing the material content into segments. This material information storage means 14 can be constituted by a general storage medium such as a hard disk or a semiconductor memory.

Here, the material information stored in the material information storage unit 14 will be specifically described with reference to FIGS.
As shown in FIG. 3, the material information storage unit 14 stores material content (video / audio) Con that is a target for correcting a voice recognition error. The material content (video / audio) Con is stored by the material content input means 10.

  Further, as shown in FIG. 3, the material information storage unit 14 stores the voice recognition information Rec and the segment information Seg in association with each material content Con. In addition, identification information (ID etc.) and time information of the material content Con are added (not shown) to the material content Con. The time information is a predetermined reference time, and is, for example, a time (input start time (year, month, date, etc.)) input to the speech recognition error correction support device 1 or the like.

The speech recognition information Rec is the word and time information recognized and associated with the speech recognition means 11.
For example, as shown in FIG. 4, the voice recognition information Rec associates time information t with a correction flag f for each word w in association with identification information (material content identification information) for identifying material content. Here, the correction flag f is information indicating whether or not correction has been made by the correction terminal 2 (for example, “1” with correction, “0” without correction).

The segment information Seg is information indicating the content obtained by dividing the speech recognition information Rec into segments by the recognition result dividing unit 12.
For example, as shown in FIG. 5, the segment information Seg is associated with the material content identification information, for each identification information (number etc.) for identifying each segment, a start time ts, an end time te, and an item k. Is associated with the thumbnail image g.
Here, the start time ts and the end time te indicate the time interval of the segment. The item k is a feature word extracted by the item information extraction unit 13. The thumbnail image g is a frame image extracted from the material content video corresponding to the time information at the head of the segment by the item information extraction unit 13.

In the example of FIG. 3, for the material content Con3, all the speech recognition is not completed and the segment information Seg is not determined.
Here, the material content Con, the voice recognition information Rec, and the segment information Seg are stored in the same storage unit, but may be stored in separate storage units.
Returning to FIG. 2, the description of the configuration of the speech recognition error correction support apparatus 1 will be continued.

The editing means 15 is for the operator to correct the speech recognition result stored in the material information storage means 14 by using the correction terminal 2 connected to the outside. There are a plurality of editing means 15 for each of the plurality of correction terminals 2. Note that the editing unit 15 may operate as one process for the correction terminal 2 and the process may be duplicated each time the correction terminal 2 is connected.
As shown in FIG. 2, the editing unit 15 includes a correction target selection unit 150, an editing screen control unit 151, an error correction unit 152, a video / audio reproduction unit 153, and a correction result output unit 154.

The correction target selection means 150 selects the material content that is the target of correcting the voice recognition error. The correction target selection unit 150 displays a screen for selecting the material content to be corrected on the correction terminal 2 and accepts a selection from the correction terminal 2.
Here, an example of a screen for selecting material content will be described with reference to FIG. For example, the correction target selection unit 150 displays a selection screen 30 as shown in FIG.

  The selection screen 30 is a screen for specifying material content to be corrected by specifying a time section. The selection screen 30 includes a time setting area 301, a current time setting button 302, and a correction start button 303.

  The time setting area 301 is an area for setting a start time and an end time. For example, the time setting area 301 is a list box for setting the date, hour, and minute. For example, as shown in FIG. 6B, the time setting area 301 displays a candidate list with a scroll bar 301b when the button 301a is pressed. The same applies to hours and minutes. The material content to be corrected is specified by the start time and the end time.

The current time setting button 302 is a setting button for setting the material content currently being input (or later) from the current time to be corrected.
For example, as shown in FIG. 6C, when the current time setting button 302 is pressed, the current time is set as the start time, and the end time is not specified. , And later) are subject to correction.
The correction start button 303 is a button for starting correction of the material content to be corrected after setting the start time and the end time.

Here, with reference to FIG. 7, the time interval of the material content specified by the start time and the end time selected by the correction target selection means 150 will be described.
FIG. 7 shows the time from today before n days (a predetermined number of days) to today, in time series.

  The time interval T1 is an example when a certain time two days before today is a start time and a certain time one day before today is an end time. The time interval T1 is set to a time earlier than the set time by a predetermined time bt (for example, several tens of seconds to several minutes). As a result, the operator of the correction terminal 2 can check the voice before the point of actual correction.

  The time interval T2 is an example when the start time and the end time are set on the oldest day of the material content stored in the material information storage unit 14. In the time section T2, when there is no material content at a time earlier than the set time by a predetermined time bt, the disclosure time is set as the time corresponding to the oldest material content.

The time interval T3 is an example when a certain time of today is set as the start time and the end time is not set. Also for the time section T3, the start time is a time earlier than the set time by a predetermined time bt.
Returning to FIG. 2, the description of the configuration of the speech recognition error correction support apparatus 1 will be continued.
The correction target selection unit 150 outputs identification information such as the file name of the set material content to the editing screen control unit 151.

  For each segment, the edit screen control unit 151 displays an item and a selection button for designating whether or not to display a word string included in the segment, and whether to display the word string of the segment by pressing the selection button. Whether or not.

Here, referring to FIG. 8 and FIG. 9 (refer to FIG. 2 as appropriate), an example of the screen displayed by the editing screen control means 151 will be described together with its control contents.
As shown in FIG. 8, the edit screen control means 151 displays on the screen of the correction terminal 2 displayed by the edit screen control means 151.

  The item list screen 31 includes a selection button 311, an item display field 312, a thumbnail image display area 313, a time table display field 314, a scroll bar display field 315, and a correction result acquisition button 316.

The selection button 311 is a button for selecting whether to display a word string for each segment.
The item display field 312 is an area for displaying items extracted in the segment. The editing screen control unit 151 reads the item corresponding to the segment (item k in FIG. 5) from the material information storage unit 14 and displays it in the item display field 312. If the segment information is not being confirmed while currently being input, such as live material, the editing screen control unit 151 sets the item display field 312 to be blank.

  The thumbnail image display area 313 is an area for displaying thumbnail images extracted in the segment. The edit screen control unit 151 reads out a thumbnail image (thumbnail image g in FIG. 5) corresponding to the segment from the material information storage unit 14 and displays it in the thumbnail image display area 313. If the segment information corresponding to the material content is not confirmed, the editing screen control unit 151 displays the first frame of the corresponding material content as a thumbnail image.

The time table display field 314 is a field for displaying a time table indicating the segment position on the time axis of the material content. The editing screen control unit 151 generates and displays a time table with reference to the segment time information (start time ts and end time te in FIG. 5) in the material information storage unit 14. If the segment information corresponding to the material content is not confirmed, the editing screen control unit 151 refers to the time information t (see FIG. 4) of the first word and the last word of the corresponding material content.
The scroll bar display column 315 is a column for displaying a scroll bar indicating which segment is being displayed when the item list does not fit on the screen. The edit screen control means 151 updates the item list on the screen by moving the scroll bar up and down.

The correction result acquisition button 316 is a button for requesting a correction result for the material content to be corrected selected by the correction target selection unit 150. When the correction result acquisition button 316 is pressed, the editing screen control unit 151 outputs the identification information (material content identification information) of the material content to be corrected selected by the correction target selection unit 150 to the correction result output unit 154. .
Thus, by displaying the item list screen 31, the operator can check the items, and easily select a segment for which the speech recognition result is to be confirmed as compared to displaying the speech recognition result at a time. can do.

  In this item list screen 31, when the operator clicks the mouse of the correction terminal 2 or presses the selection button (“open” in FIG. 8) 311 by touching the touch panel, the editing screen control means 151 displays the item. On the list screen 31, an editing area 317 (see FIG. 9) for correcting the word string of the segment is displayed.

FIG. 9 is a screen example showing an item list screen 31 </ b> B displaying the editing area 317.
In the item list screen 31B, a moving image display area 313B and an edit area 317 are displayed in the selected segment with respect to the item list screen 31 described in FIG.

The moving image display area 313B is an area for reproducing material content corresponding to a segment. The edit screen control unit 151 refers to the time information (start time ts and end time te in FIG. 5) of the segment in the material information storage unit 14 at the timing when the segment is selected, and the video section of the corresponding material content Is displayed in the moving image display area 313B. If the segment information corresponding to the material content is not fixed, the editing screen control unit 151 refers to the time information t (see FIG. 4) of the first word of the corresponding material content, and the corresponding material content. The first frame is displayed in the moving image display area 313B.
When the image area of the moving image display area 313B is clicked with a mouse or the like or the reproduction start button st is pressed, the editing screen control means 151 instructs the video / audio reproduction means 153 to reproduce the material content.

  The editing area 317 is an area for displaying a word string corresponding to a segment and performing editing. The editing screen control unit 151 expands a word string corresponding to the segment stored in the material information storage unit 14 in the editing area 317.

Note that the edit screen control unit 151 sequentially adds a character string to be displayed each time a word of the speech recognition information Rec (see FIG. 3) is added when the edit area 317 is displayed. Accordingly, the editing screen control unit 151 expands the editing area 317. Thereby, the voice recognition error correction support apparatus 1 can correct errors in the voice recognition of the material content in real time.
The editing of the editing area 317 is performed by an error correction unit 152 described later. When an arbitrary word is clicked with the mouse or the like in the editing area 317, the editing screen control unit 151 instructs the video / audio reproduction unit 153 to reproduce the material content corresponding to the time of the clicked word.
Also, the edit screen control unit 151 depresses the selection button (“close” in FIG. 9) 311 to hide the edit area 317 and the moving image display area 313B as the thumbnail image display area 313, and the list of items in FIG. The display is returned to the screen 31.
Returning to FIG. 2, the description of the configuration of the speech recognition error correction support apparatus 1 will be continued.

The error correction unit 152 corrects an error in the word string of the segment in the editing area 317 (FIG. 9) by the editing operation of the operator of the correction terminal 2. The error correcting means 152 functions as a general text editor (screen editor) in an editing operation for correcting a word string.
Further, the error correction unit 152 notifies the editing unit 15 (an editing unit activated in a separate process) corresponding to the other correction terminal 2 of the correction content, so that the editing area 317 of the other correction terminal 2 is notified. To reflect. On the other hand, when the correction content is notified from the other editing means 15, the error correction means 152 reflects the correction content in the editing area 317 of its correction terminal 2.

When the correction is made, the error correction unit 152 overwrites the word of the voice recognition information Rec (FIG. 4) and corrects it, and sets the correction flag f. This error correction unit 152 does not perform complicated exclusive control by updating by overwriting (prior writing priority) even when the error correction unit 152 of the other editing unit 15 corrects the same correction target. To do.
Then, the error correction means 152 notifies the other editing means 15 that the correction has been made, so that the corresponding error correction means 152 refers to the voice recognition information Rec (FIG. 4), and the correction contents. Reflect. Or the error correction means 152 is good also as notifying the word before a change and the word after a change.
At this time, the error correction unit 152 displays the corrected word with a different color attribute (for example, red) from the uncorrected word.
Thereby, the error correction means 152 can make an operator visually recognize the content corrected by the other correction terminal 2.

For example, as shown in FIG. 10, it is assumed that the same correction target is corrected by different correction terminals 2 ₁ and 2 ₂ in the editing area 317.
In this case, as shown in FIG. 10 (a), the editing terminals 2 ₁ operator may modify the "Kyo" to "day" in the edit area 317. Further, as shown in FIG. 10 (b), the editing terminals 2 ₂ operator may modify the "Hata" to "skin" in the edit area 317.
Then, the respective error correction means 152 corresponding to the correction terminals 2 ₁ and 2 ₂ notify other correction contents to reflect the other correction contents.
As a result, it is possible to quickly correct an error in speech recognition with a plurality of correction terminals 2.

In addition, the error correcting unit 152 has a function of reproducing sound when correcting a word string.
Specifically, the error correction unit 152 reproduces sound from the selected word by clicking the mouse or touching the touch panel in the editing area 317 (FIG. 9). In addition, when an arbitrary position is selected again during the audio reproduction, the error correction unit 152 stops the audio reproduction.

FIG. 11 is an explanatory diagram for explaining an example of editing work in the editing area.
For example, in the editing area 317 of FIG. 11, when “March” is selected by mouse click or the like, the error correction unit 152 displays the time information of the words included in the segment of the material information storage unit 14 (time information of FIG. 4). Referring to t), the video / audio reproduction means 153 is instructed to reproduce audio from the position of the corresponding material segment. At this time, a video corresponding to the audio playback time may be played back in the moving image display area 313B in conjunction with the audio.

Then, as shown in FIG. 11, the error correction unit 152 links each word corresponding to the reproduced voice in conjunction with the reproduction of the voice so as to clearly indicate the reproduction position of the voice and the presence / absence of the reproduction in the segment. Change the display attribute of the display part. For example, the error correction means 152 displays the word corresponding to the voice in black and white inversion or in a predetermined color.
Returning to FIG. 2, the description of the configuration of the speech recognition error correction support apparatus 1 will be continued.

  The video / audio reproduction means 153 outputs the video and audio of the material content to the correction terminal 2. The video / audio reproduction means 153 reads the video / audio from the material information storage means 14 based on the position designated by the editing screen control means 151 or the error correction means 152 and outputs it to the correction terminal 2.

The correction result output means 154 outputs the voice recognition result (transcription result) corrected by the error correction means 152 to the outside.
The correction result output unit 154 reads the corrected speech recognition result for the material content to be corrected output from the editing screen control unit 151 from the material information storage unit 14 and outputs it to the corresponding correction terminal 2.

  For example, the correction result output unit 154 generates text data including a title T, time information / item K, and a correction result W, as shown in FIG. There are a plurality of pieces of time information / items K and correction results W corresponding to the number of segments.

In the title T, for example, a correction date is described.
In the time information / item K, for example, a segment start time ts and an item k (see FIG. 5) associated with the segment are described. If the segment information is not finalized for the material content currently being input, the correction result output means 154 describes the time information t (see FIG. 4) of the first word in the time information, and sets the item as blank. To do.

  In the correction result W, for example, a corrected word w string (see FIG. 4) corresponding to the segment is described. If the segment information is not finalized in the currently input material content, the correction result output means 154 displays the word w string up to the present time of the speech recognition information corresponding to the currently input material content (see FIG. 4). Is described.

  The correction result output means 154 may output the corrected speech recognition result as text data to the correction terminal 2, or a word for which the correction flag f (see FIG. 4) is set in the speech recognition information. The data may be output as data in a predetermined format to which color information is added, for example, PDF (Portable Document Format).

  The information deleting unit 16 periodically deletes information stored in the material information storage unit 14. For example, the information deletion means 16 deletes information (material content Con, voice recognition information Rec, segment information Seg (see FIG. 3)) that has been stored in advance and has exceeded the number of days in advance.

By configuring the speech recognition error correction support device 1 as described above, the speech recognition error correction support device 1 enables the correction performed later by the plurality of correction terminals 2 even if it is the same correction target. By doing so, the operator's correction can be quickly reflected without performing exclusive control. In addition, since the voice recognition error correction support device 1 can notify the other correction terminal 2 of the correction made by a certain correction terminal 2, it can prevent the same correction from being made by a plurality of correction terminals 2. As a whole, the correction time can be shortened, and the voice recognition error can be corrected in real time for the input material content.
In addition, the speech recognition error correction support device 1 can correct a speech recognition error while confirming a recognition result word and a speech that is a source of the recognition result by a simple text editing operation.
Note that the speech recognition error correction support apparatus 1 can be operated by a speech recognition error correction support program for causing a computer to function as each of the above-described means.

[Operation of voice recognition error correction support device]
Next, the operation of the speech recognition error correction support apparatus 1 according to the embodiment of the present invention will be described with reference to FIGS. Here, as the operation of the speech recognition error correction support apparatus 1, the segment information generation operation for generating the recognition result by the speech recognition for the material content in the segment unit and the recognition result in the segment unit are presented to the correction terminal 2. The segment information presenting operation and the segment correcting operation for correcting the recognition result while performing audio reproduction will be described.

(Segment information generation operation)
First, the segment information generation operation of the speech recognition error correction support apparatus 1 will be described with reference to FIG. 13 (refer to FIG. 2 as appropriate).

In step S1, the material content input means 10 inputs material content for voice recognition. At this time, the material content input unit 10 writes and stores the input material content in the material information storage unit 14.
In step S2, the speech recognition means 11 recognizes the speech of the material content input in step S1, associates the recognition result as text data with the time information for each word constituting the recognition result, and divides the recognition result. The information is notified to the means 12 and written and stored in the material information storage means 14.

  In step S3, the recognition result dividing unit 12 divides the recognition result recognized in step S2 into segments according to predetermined criteria, for example, video cut points, audio silence intervals, and the like.

In step S4, the item information extraction unit 13 extracts a feature word included in the segment as an item for each segment divided in step S3, and extracts a thumbnail image from the video corresponding to the segment. At this time, the item information extraction unit 13 writes and stores the extracted items and thumbnail images in the material information storage unit 14 in association with the segments.
Here, when the input of the material content is not completed (No in step S5), the speech recognition error correction support apparatus 1 returns to step S1 and continues the input of the material content.
On the other hand, when the input of the material content is completed (Yes in step S5), the speech recognition error correction support apparatus 1 ends the operation.
Through the above operation, the speech recognition error correction support apparatus 1 associates material content, speech recognition information, and segment information with the material information storage unit 14 as shown in FIG.

(Segment information presentation operation)
Next, the segment information presentation operation of the speech recognition error correction support apparatus 1 will be described with reference to FIG. 14 (refer to FIG. 2 as appropriate). The segment information presentation operation is an operation of the editing unit 15 after the connection with the correction terminal 2 is started.

In step S <b> 10, the correction target selection unit 150 displays on the correction terminal 2 a selection screen 30 (see FIG. 6) for selecting material content to be corrected for speech recognition errors stored in the material information storage unit 14.
In step S11, the correction target selection unit 150 sets the start time and end time of the material content to be corrected on the screen, and waits until the correction start button is pressed (No in step S11). Then, when the correction start button is pressed (Yes in step S11), the correction target selection unit 150 transfers control to the edit screen control unit 151 that performs the control after step S12.

  In step S12, the edit screen control unit 151 designates whether to display an item and a word string included in the segment for each segment based on various types of information stored in the material information storage unit 14. An item list screen 31 (see FIG. 8) including a selection button is displayed on the correction terminal 2.

In step S13, the edit screen control unit 151 waits until the selection button (open) is pressed on the item list screen (No in step S13).
On the other hand, when the selection button (open) is pressed (Yes in step S13), in step S14, the edit screen control means 151 displays an edit area 317 corresponding to the selected segment as shown in FIG. Then, the word string which is the recognition result corresponding to the segment stored in the material information storage unit 14 is developed in the editing area 317.

After this operation, the editing means 15 shifts to a state in which the operator can correct the editing result on the screen of the correction terminal 2. When the editing area 317 is displayed by pressing the selection button (open), the editing screen control unit 151 can hide the editing area 317 by pressing the selection button (close) at an arbitrary timing. The non-display operation is not shown. The description of the material content playback operation in the moving image display area 313B of the item list screen 31B (see FIG. 9) is also omitted here.
With the above operation, the speech recognition error correction support apparatus 1 can correct a speech recognition error for each segment of material content.

(Segment correction operation)
Next, the segment correction operation of the speech recognition error correction support apparatus 1 will be described with reference to FIG. 15 (refer to FIG. 2 as appropriate). Since the segment correction operation is an arbitrary procedure performed by the operator, here, an example of an operation performed in combination with audio reproduction and correction operation will be described.

  In step S20, the error correction unit 152 selects a word or a word string for which the voice in the editing area 317 (FIG. 9) is to be reproduced by clicking the operator's mouse or touching the touch panel. At this time, the error correction unit 152 refers to the time information of the segment in the material information storage unit 14 via the video / audio reproduction unit 153 and outputs the audio of the time corresponding to the word or the word string to the correction terminal 2. To do. As a result, the sound is reproduced by the correction terminal 2, and the operator can confirm the sound by comparing the sound and the recognized word string.

  In step S <b> 21, the error correction unit 152 accepts designation of the position of the correction part by clicking the operator's mouse or touching the touch panel. At this time, the error correction means 152 stops the output to the correction terminal 2 if the sound is being reproduced. Thereby, the reproduction of the sound is stopped.

In step S22, the error correction unit 152 displays a cursor at a specified position in the editing area, and corrects the recognition error by an operator's editing operation such as character deletion or character insertion. Here, the error correction unit 152 updates the word in the material information storage unit 14 with the correction result.
In step S <b> 23, the error correction unit 152 notifies the other editing unit 15 of the correction content, thereby reflecting the content displayed on the corresponding correction terminal 2 by the error correction unit 152 of the other editing unit 15.

  In step S <b> 24, the error correction unit 152 accepts designation of the position of the correction by clicking the operator's mouse or touching the touch panel. At this time, the error correction unit 152 refers to the time information of the segment in the material information storage unit 14 via the video / audio reproduction unit 153 and outputs the audio of the time corresponding to the word or the word string to the correction terminal 2. To do. As a result, the sound is reproduced on the correction terminal 2, and the operator can confirm whether or not the correction result is correct.

Although illustration is omitted, if the corrected part has not been corrected correctly by the operator's confirmation in step S24, the process returns to step S22 and the operation is repeated.
With the above operation, the speech recognition error correction support apparatus 1 can quickly correct a speech recognition error with the plurality of correction terminals 2.

Although the embodiment of the present invention has been described above, the present invention is not limited to this embodiment.
Here, the material content has been described as including video and audio, but it may be audio-only material content.
In that case, the item information extraction means 13 may extract only the items and not the thumbnail images. Further, the video / audio reproduction unit 153 may be an audio output unit that outputs only audio.

In addition, here, the correction target of the voice recognition error is the material content that has already been voice-recognized and the material content that is currently being voice-recognized. For example, when only the material content that is currently undergoing speech recognition is the target of speech recognition error correction, the speech recognition error correction support device 1 omits the recognition result division unit 12, the item information extraction unit 13, and the correction target selection unit 150. It may be configured simply.
Here, the correction target selection unit 150 selects the material content to be corrected on the basis of the date and time when the material content is stored. However, the selection criterion is not limited to the date and time. For example, when the material content is associated with a genre of an electronic program guide such as news or sports as a broadcast material, the genre may be specified and the material content may be selected. Alternatively, when event information when the material is recorded is added to the material content, the event may be specified and the material content may be selected.

Further, the voice recognition error correction support device 1 may include voice recognition means 11 outside.
For example, it is good also as a structure of the speech recognition error correction assistance apparatus 1B shown in FIG. The speech recognition error correction support device 1B includes the speech recognition unit 11 of the speech recognition error correction support device 1 (FIG. 2) as a speech recognition device. In this case, the recognition result input means 19 which is an input interface stores the word and time information in association with each other in the material information storage means 14 and notifies the recognition result division means 12 of them.
Note that the speech recognition error correction support apparatus 1B can also be operated by a speech recognition error correction support program for causing a computer to function as each of the above-described means.

Further, the edit screen control means 151 may display time information in association with each item displayed on the item list screens 31 and 31B (see FIGS. 8 and 9).
For example, as shown in FIG. 17, the edit screen control means 151 displays time information 318 and 318B for each item. Thereby, it is possible to easily access the item of the time to be referred to from among a plurality of displayed items.
At this time, the edit screen control means 151 displays the start time ts (see FIG. 5), which is the time information of the first word of the material content, as time information 318 for the item whose selection button is “open”. . Further, the edit screen control unit 151 sets the time corresponding to the playback time of the audio output by the video / audio playback unit 153 (sound output unit) from the start time of the material content for the item whose selection button is “close”. The time information 318B is variably displayed.

Here, the time input to the speech recognition error correction support apparatus 1 is used as the time information. However, when displaying the time information as shown in FIG. 17, the time code added to the material content is used. Also good.
In this case, the material content input unit 10 sets “00:00:00” as the start time ts (see FIG. 5) of the first item in the material information storage unit 14. When the time code is added as tag information to the material content, for example, in the case of material content in a moving image format such as XDCAM (registered trademark), the material content input means 10 uses the material information for each material content. Store in the storage means 14.

  Then, as shown in FIG. 18, the edit screen control unit 151 adds the time code time for the material content to which the time code is added, based on the time information to which the time code is not added. In FIG. 18, only “hour: minute: second” of the time code is displayed.

For example, as shown in FIG. 19A, the edit screen control unit 151 displays the start start time 314a of the material content in the time table display field 314 for the material content to which no time code is added. The time information 318 is displayed in the item.
Further, the edit screen control means 151 sets the start time of the material content and the time of the time code in the time table display field 314 for the material content to which the time code is added, as shown in FIG. The added time is displayed as the head start time 314a, and the time information 318 of each item is also displayed by adding the time of the time code to the time information of each item.
This makes it possible to clarify the correspondence between the utterance content and the time code.

Further, the error correction unit 152 may select an editing operation from a context menu.
For example, as shown in FIG. 20, the error correction unit 152 displays a context menu 319 by a right click key of the mouse or the like.
In the context menu 319, a function (copy, etc.) and a shortcut key (Ctrl + C, etc.) are displayed together. When the function is selected by left-clicking the mouse, the error correction means 152 performs the corresponding editing operation. Execute. Alternatively, when the shortcut key corresponding to the function is pressed, the correction unit 152 executes the corresponding editing operation.
As a result, the operator can learn the operation method and the operation procedure can be minimized.

Here, the correction result output means 154 outputs the corrected speech recognition result to the correction terminal 2.
However, the correction result output means 154 may output an arbitrary character string in a form that can be displayed on another device at the time of correction.
For example, the editing screen control unit 151 selects a character string to be output with a two-dimensional code (for example, QR code [registered trademark]) with the mouse while editing the editing area 317 (see FIG. 9), and outputs a correction result. The means 154 is instructed to output a character string with a two-dimensional code.

  In this case, the edit screen control means 151 selects a character string and selects “QR display of selected character string” from the context menu 319 shown in FIG. 20 or presses a shortcut key (here, Ctrl + Q). As a result, the selected character string is output to the correction result output means 154. At this time, the editing screen control unit 151 outputs time information corresponding to the first word of the selected character string to the correction result output unit 154 together with the selected character string.

Then, as shown in FIG. 21A, the correction result output unit 154 generates a QR code having the time information and the selected character string as text, and displays the QR code as a modal window 320, for example. Note that the edit screen control unit 151 displays the modal window 320 until an arbitrary place in the modal window 320 is clicked with the mouse, and hides the modal window 320 after the click.
Thereby, a device (smart phone, tablet, etc.) that can read the QR code can display the selected character string by reading and decoding the QR code by the QR reading function.
For example, as shown in FIG. 21B, the selected character string and time information corresponding to the first word (here, “Tohoku”) are displayed on the screen of the smartphone.
Thereby, the correction result output means 154 can transmit the voice recognition result (transcription result) to a device not connected to the network.

1, 1B Voice recognition error correction support device 10 Material content input means 11 Voice recognition means 12 Recognition result division means 13 Item information extraction means 14 Material information storage means (storage means)
DESCRIPTION OF SYMBOLS 15 Editing means 150 Editing object selection means 151 Editing screen control means 152 Error correction means 153 Video / audio reproduction means (audio output means)
154 Correction result output means 19 Recognition result input means

Claims (9)

A speech recognition error correction support device that inputs material content and corrects an error in speech recognition for speech included in the material content via a plurality of correction terminals,
Voice recognition means for recognizing the voice and generating a recognition result that is text data and time information for each word constituting the recognition result;
Storage means for associating time information for each word included in the recognition result for each of the recognition results, and storing together with the material content and the time information of the material content;
A plurality of editing means for acquiring correction contents for the recognition result via the plurality of correction terminals and correcting an error in the speech recognition; and
The editing means includes
An editing screen control means for displaying an editing screen including the recognition result of the material content to be corrected on the correction terminal;
A voice output means for outputting a corresponding voice to the correction terminal based on the time information of the word in the text designated on the editing screen;
The recognition result stored in the storage unit is updated with priority on the postscript using the correction content corrected on the editing screen, and an update notification to the effect that the recognition result has been postwritten is sent to other editing units. And error correction means for reflecting the correction content corresponding to the correction terminal;
A speech recognition error correction support apparatus comprising: A speech recognition error correction support device that inputs material content and corrects an error in speech recognition for speech included in the material content via a plurality of correction terminals,
Storage means for storing a recognition result, which is text data recognized by the external voice recognition means, and time information for each word constituting the recognition result;
A plurality of editing means for acquiring correction contents for the recognition result via the plurality of correction terminals and correcting an error in the speech recognition; and
The editing means includes
An editing screen control means for displaying an editing screen including the recognition result of the material content to be corrected on the correction terminal;
A voice output means for outputting a corresponding voice to the correction terminal based on the time information of the word in the text designated on the editing screen;
Update the recognition result stored in the storage unit using the correction content corrected on the editing screen, and notify the other editing unit of an update notification indicating that the recognition result has been postwritten. Error correction means for reflecting the correction content corresponding to the correction terminal;
A speech recognition error correction support apparatus comprising:   The speech recognition error correction support apparatus according to claim 1 or 2, wherein the error correction means displays the correction content with a color attribute different from that of the text that has not been corrected. The editing means further includes a correction target selection means for displaying a correction target selection screen for selecting one or more material contents to be corrected on the correction terminal,
The said edit screen control means displays the edit screen containing the said recognition result of the 1 or more correction object selected by the said correction object selection means on the said correction terminal, The Claim 1 to Claim 3 characterized by the above-mentioned. The speech recognition error correction support device according to any one of the above.   The editing means further includes a correction result output means for outputting the correction result of the correction target selected by the correction target selection means to the correction terminal in response to an instruction to acquire the correction result from the correction terminal. The speech recognition error correction support device according to claim 4.   6. The correction result output means generates a QR code (registered trademark) having the character string as text from the character string selected on the editing screen, and displays the QR code on the correction terminal. The speech recognition error correction support device according to claim 1.   The edit screen control means displays time information of the first word of the recognition result on the correction terminal in association with the recognition result of the material content to be corrected. The speech recognition error correction support device according to any one of the above.   8. The voice recognition error correction support according to claim 7, wherein the editing screen control unit updates time information displayed on the correction terminal in accordance with a playback time of the voice output by the voice output unit. apparatus.   A speech recognition error correction support program for causing a computer to function as the speech recognition error correction support device according to any one of claims 1 to 8.

Images