JP2000047683A - Segmentation aids and media - Google Patents

Abstract

(57) [Problem] A technology relating to speech synthesis requires high-precision speech segmentation as preprocessing, but it is extremely difficult to automatically and perfectly perform continuous speech segmentation of spontaneous speech. At the same time, there is no necessity for immediate processing and complete automation, so human work is involved. An object of the present invention is to construct an interface that enables a person without a high level of specialized knowledge to perform high-precision segmentation of speech and the like. SOLUTION: An automatic segmentation unit 12 for calculating segmentation candidates is provided, and the candidates are displayed on a screen so that an operator can select or correct them, or perform a segmentation operation while checking by listening or reading. It comprises a correction unit 13 under GUI control. With the above configuration, it is possible to easily achieve high-precision segmentation even for an operator who does not have a high level of specialized knowledge.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal segmentation method and, more particularly, to a segmentation assisting device for an operator who performs a voice segmentation operation to perform a phoneme segmentation required when synthesizing artificial speech.

[0002]

2. Description of the Related Art In a technique for generating synthesized speech and a technique relating to automatic speech recognition, performing segmentation and labeling on speech is extremely important as preprocessing of each technique. There have been many reports on techniques related to automatic segmentation. For example, the publications of IEICE Transactions D-II Vol. 73 No. 1 pages 1 to 9 and Japanese Patent Publication No. 6-337692 and Japanese Patent Publication No. 7-13587. Here, a detailed description will be given by taking, as an example, the one published on the first to ninth pages of the IEICE Transactions DII Vol.73 No.1. In this paper, we report an example of constructing an expert system that performs phoneme segmentation in continuous speech using techniques and knowledge used by experts. In the proposed system, phonemes in input speech are detected according to the knowledge and strategy of a ruled expert, and their boundaries are determined. It performs hypothesis inference using certainty factors to express ambiguous knowledge that depends on the phonological environment, that is, the type of the preceding and succeeding phonological units and typical phonological transformations, and extracts acoustic features based on the hypothesis of the phonological environment .
FIG. 15 shows a strategy for determining phonemes used by the system. The input speech is subjected to spectrum analysis, and certainty, which is a certain phoneme, is calculated based on the characteristics of the power spectrum. This is the detection of phonemic candidates. Further, a phonological environment is hypothesized for each hypothesized phonological candidate. Based on the hypothesis of the phonological environment, the certainty of the phonological boundary is obtained. The certainty of the phoneme candidate hypothesis detected in this way and the certainty of the phoneme boundary are integrated to determine the certainty of the phoneme boundary. When a plurality of phoneme boundaries are candidates, the phoneme boundary with the highest certainty is determined. As a result of the voice segmentation performed in this manner, 90% or more phoneme boundaries, which are only consonants, were detected with an accuracy within 10 ms of the boundary shift. However, the addition error ranged from 30% to 40%. Here, the addition error means an error in which a phoneme label that does not actually exist is added. For example, if the label “kaNri” is correct, “kaeNr”
If the label "i" is given, "e" is an erroneously added one, so that an addition error occurs. The additional error is an error that cannot be expressed by a deviation of the position from the correct answer, and thus is separately evaluated.

[0003]

With the prior art, it is extremely difficult to automatically and perfectly perform the segmentation and labeling of a continuous speech of a natural utterance. In particular, when generating synthetic speech, humans hear the synthesized speech using the segmentation results, so high segmentation accuracy is required, but real-time processing like an automatic speech recognition device requires It is only necessary to accumulate the result of segmentation in advance as a database without being required. As described above, there is no necessity of complete automation, and when high-precision segmentation is required, human work can be interposed. In such a situation, a segmentation operation is conventionally performed based on a speech waveform or a power spectrum, but such a highly accurate segmentation operation requires a high degree of expertise.

[0004] The present invention takes into account the problem that such a conventional high-precision segmentation operation requires a high degree of specialized knowledge, and enables a person who does not have high-level expertise to perform high-precision voice segmentation. It is an object to provide a fragmentation assisting device.

[0005]

In order to solve the above-mentioned problems, a first aspect of the present invention (corresponding to claim 1) is a signal input means for inputting an audio signal or an image signal, and the signal input means comprises: Automatic segmentation is performed on the input audio signal or image signal, and automatic segmentation means for calculating segment boundary candidates, and the segment boundary candidates calculated by the automatic segmentation means are displayed on a screen. And a correction means for performing segmentation by selecting or correcting the candidate while confirming the candidate for the segment boundary by GUI control.

A second invention (corresponding to claim 2) is:
A text which is provided separately from the signal input means and is used when performing automatic segmentation by the automatic segmentation means and / or used when selecting or correcting the segment boundary candidate by GUI control by the correction means. The segmentation assisting device according to the first aspect, further comprising a text input unit for inputting.

A third aspect of the present invention (corresponding to claim 3) is:
Phoneme symbol notation conversion means for converting the text input by the text input means into phoneme symbol notation, and using the phoneme symbol notation in the automatic correction means instead of using the text, and / or Alternatively, the segmentation assisting device according to the second invention is used in the correcting means.

A fourth invention (corresponding to claim 4) is:
The automatic segmentation means according to any one of the first to third inventions, wherein the phoneme boundary candidate is calculated by an audio signal processing using a zero crossing number, a pitch, a power, an audio formant, a cepstrum or a phase of the audio signal. It is a segmentation assistance device as described. .

A fifth invention (corresponding to claim 5) provides:
The segmentation assistance according to any of the second or third invention, wherein the automatic segmentation means calculates phoneme boundary candidates by syllable or phoneme matching using the text or phoneme symbol notation. Device.

A sixth aspect of the present invention (corresponding to claim 6) is:
The second invention or the third invention, wherein the automatic segmentation means calculates a phoneme boundary candidate by the acoustic signal processing, and calculates a phoneme boundary candidate by syllable or phoneme matching. It is a segmentation assistance device as described.

A seventh aspect of the present invention (corresponding to claim 7) is:
The automatic segmentation means converts the number of phoneme boundary candidates given by the sound signal processing using the number of zero crosses, pitch, power, sound formant, cepstrum or phase of the sound signal, and converts the text input by the text input means. And comparing the number of phoneme boundary candidates obtained from the obtained phoneme symbol notation with the number of phoneme boundary candidates given by the acoustic signal processing based on a predetermined criterion. It is a segmentation assistance device according to the invention.

An eighth aspect of the present invention (corresponding to claim 8) is:
The method according to claim 7, wherein the automatic segmentation means reduces the number of the phoneme boundary candidates given by the acoustic signal processing using the phoneme boundary candidates calculated by the syllable or phoneme matching. It is a segmentation aid.

According to a ninth aspect of the present invention (corresponding to claim 9),
An input means for inputting an audio signal or an image signal and a segmentation candidate segmented based on some reference thereto, and a worker selecting, moving, deleting or adding the segmentation candidate through a GUI to perform a segmentation correction operation. Correction means for performing the segmentation.

The tenth invention (corresponding to claim 10)
The correction means selects any of the phoneme boundary candidates,
The segmentation assisting device according to any one of the first to third inventions or the ninth invention, wherein a segmentation operation is performed by moving, deleting, or adding.

The eleventh invention (corresponding to claim 11)
The segmentation assisting device according to any one of the first to third inventions or the ninth invention, wherein the correction means corrects a speech segment between any of the phoneme boundary candidates by listening. is there.

The twelfth invention (corresponding to claim 12)
Wherein the correcting means performs speech synthesis based on the segmentation result selected by the operator via a GUI and performs another correction by listening.
A segmentation assisting device according to any one of the above inventions or the ninth invention.

The thirteenth invention (corresponding to claim 13)
Is a medium storing a program for realizing all or a part of the function of each means in any one of the first to twelfth inventions.

[0018]

Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1) In FIG. 1, reference numeral 11 denotes a signal input unit for inputting an audio sound signal, 12 an automatic segmentation unit for automatically performing phoneme segmentation and calculating segment boundary candidates, and 13 for GUI control. A correction unit 14 that corrects the segment boundary candidates calculated by the automatic segmentation unit 12 is a segmentation result output unit that outputs a phoneme segmentation result corrected by GUI control.

Here, FIG. 10 will be described. 102 is a display area for displaying the progress of the segmentation work and the segmentation result superimposed on the waveform of the audio sound signal, 101 is a Y-axis scale that determines the display magnification of the Y-axis of the display area, and 105 is a segmentation candidate selected with a mouse , A work area for correction, movement, and deletion, 103 is an X-axis scale that determines the display magnification in the X-axis (time axis) direction of the work area 105, and 104 is a Y-axis scale that determines the Y-axis display magnification. . As will be described later, reference numeral 106 denotes a delimiter for displaying the input phoneme symbol notation. In FIG. 10, "Ka-soruid" is used as the delimiter 106.
o-, Ka-do ga me N "(cursor movement, card screen) and phoneme symbol notation are displayed. Reference numeral 110 denotes an initialization button for initializing a segmentation operation, and 111 denotes a recognition result clear button for erasing a result obtained by automatically performing phoneme segmentation in the automatic segmentation unit 12 or a result obtained by correcting a segmentation candidate in the work area 105. , 112 are label candidate confirmation buttons for confirming and outputting the result of correcting the phoneme segmentation performed by the automatic segmentation unit 12, and 107 is a playback button for playing back the input audio signal. As will be described later, reference numeral 108 denotes a confirmation playback button for actually synthesizing speech using the result of the phoneme segmentation corrected in the work area 105 and confirming by listening. Reference numeral 109 denotes a recognition execution button for automatically performing phoneme segmentation to calculate a segment candidate. Reference numerals 113 to 116 label a segmentation candidate. 117 denotes a candidate correction for moving a segment boundary candidate along the X axis (time axis). Button, 118
Is a candidate delete button for deleting a segment boundary candidate, 11
Reference numeral 9 denotes a candidate selection button for selecting a candidate which is actually a segment boundary from the segment boundary candidates. As described later, reference numeral 120 denotes a candidate confirmation button for confirming by listening by actually reproducing between segment boundary candidates.
Reference numeral 121 denotes a candidate addition button for adding a new segment boundary candidate.

Next, the operation of the embodiment will be described. In FIG. 1, an automatic segmentation unit 12 automatically performs a phoneme segmentation on a voice acoustic signal input by a signal input unit 11.
The correction unit 13 based on GUI control displays the result of the automatic segmentation on the screen as shown in FIG. 10, for example. The operator can correct the segmentation result by operating a mouse or the like. The segmentation result corrected by the operator is sent to the segmentation result output unit 14 and output.

Further, the above-described operation will be described with reference to FIG. 10 and FIG. After the initialization button 110 is clicked on with a mouse to delete the previous phoneme segmentation work, a file name is specified, and a voice acoustic signal is read. This corresponds to reading a voice acoustic signal in the signal input unit 11. Next, the recognition execution button 109 is clicked on with a mouse to perform automatic segmentation and calculate a segmentation boundary candidate. This corresponds to automatically performing phoneme segmentation in the automatic segmentation unit 12. In FIG. 10, work area 1
Many vertical lines are displayed in the display area 05 and the display area 101 so as to be superimposed on the audio sound signal. These are the segment boundary candidates calculated by the automatic segmentation unit 12. Next, by clicking the candidate correction button 117 with a mouse or pressing the function key “F5” of the keyboard, the screen enters a correction mode, and the work area 10
The segment boundary candidates displayed in 5 can be selected with the mouse. The selected segment boundary candidates are
By moving the mouse, it becomes possible to move along the X axis. In FIG. 10, the candidate correction button has been selected. Furthermore, when the candidate deletion button 118 is clicked with the mouse or the function key “F6” of the keyboard is pressed, the screen enters a deletion mode, and the segment boundary candidate displayed in the work area 105 is selected with the mouse. Can be deleted.
Further, by clicking the candidate selection button 119 with a mouse or pressing the function key F7 of the keyboard, the screen enters a selection mode, and a segment boundary candidate displayed in the work area 105 can be selected with a mouse. The selected segment boundary candidate becomes a segment boundary and is displayed in the work area 105 and the display area 102 with different display colors. Further, the candidate confirmation button 120 is clicked with a mouse or the function key “F
Pressing "8" causes the screen to enter confirmation mode and
The sound between the selected boundary candidates among the segment boundary candidates displayed in 5 is reproduced and can be confirmed by listening. Further, when the candidate addition button 121 is clicked with the mouse or the function key “F9” of the keyboard is pressed, the screen enters an addition mode. Can be added. As described above, the segment boundary candidates are corrected using the functions 117 to 121. This corresponds to correcting the segment boundary candidate in the correction unit 13 based on the GUI control.

As described above, according to the present embodiment, the phoneme boundary candidates calculated by the automatic segmentation unit 12 are presented on the screen, and the correction unit 13 by GUI control is displayed.
By performing the selection and correction according to the above, it is possible to perform high-precision voice segmentation even without a skilled operator.

(Embodiment 2) This embodiment is obtained by adding a text input unit to the first embodiment.

In FIG. 2, 11 is a signal input unit, 21 is a text input unit, 12 is an automatic segmentation unit, 1
Reference numeral 3 denotes a correction unit based on GUI control, and reference numeral 14 denotes a segmentation result output unit. In the present embodiment, text corresponding to the voice input to signal input unit 11 is input to text input unit 21. Automatic segmentation unit 12
The phoneme segmentation can be performed using not only the sound and audio information obtained from the signal input unit 11 but also the text information obtained from the text input unit 21. Similarly, the correction unit 13 based on the GUI control includes a text input unit 21
The segment boundary candidates can be corrected with reference to the text input from the.

(Embodiment 3) This embodiment is obtained by adding a phoneme symbol notation conversion unit 31 to the second embodiment.
In FIG. 3, 11 is a signal input unit, 21 is a text input unit, 12 is an automatic segmentation unit, 31 is a phoneme symbol notation conversion unit that automatically converts input text into phoneme symbol notation, and 13 is correction by GUI control. Reference numeral 14 denotes a segmentation result output unit. The phoneme symbol notation conversion unit 31 converts the text input to the text input unit 21 into phoneme symbol notation. The phoneme symbol notation conversion unit 31 converts a text into a phoneme symbol notation using a dictionary in which the word and the grammar of the phoneme symbol are paired. If, for example, “cursor movement, card screen” is input to the text input unit 21, this is automatically changed to “Ka” as shown by the delimiter 106 in FIG.
ー so ruido, Ka-do
ga me N ".

As described above, according to the present embodiment, it is possible to input text that is not a phoneme notation, such as a sentence containing kana and kanji, into the text input unit 21.

(Embodiment 4) In this embodiment, an example will be described in which the automatic segmentation in the automatic segmentation unit 12 of the above-described second or third embodiment is performed by acoustic signal processing. The automatic segmentation unit 12 in FIG. 3 extracts acoustic parameters that often change characteristically at phoneme boundaries. The extracted parameters are, for example, the number of zero crosses, the pitch, the power, the audio formant, the cepstrum, etc. of the audio signal. A temporal change rate is calculated for each of the extracted parameters. The time or sample point at which these rates of change are maximal is calculated. These times and sample points are output from the automatic segmentation unit 14 as phoneme boundary candidates.

The parameters extracted in the audio signal processing of the present invention are not limited to the number of zero crossings, power, voice formant or cepstrum in the above-described embodiment, but are characteristically changed at phoneme boundaries, such as phases. You only have to.

Further, the parameters to be extracted in the audio signal processing of the present invention are not limited to those for which the maximum point of the rate of change is calculated in the above-described embodiment, but are simply phoneme values such as using the values of the audio parameters themselves. What is necessary is just a calculation method that can detect the boundary.

(Embodiment 5) In this embodiment, an example will be described in which the automatic segmentation in the automatic segmentation unit of the above-described second or third embodiment is performed by syllable or phoneme matching. The automatic segmentation unit 12 in FIG. 3 performs syllable or phoneme matching. Matching is performed on training data for speech recognition prepared in advance and speech data input by the signal input unit 11 using a hidden Markov model (HMM). Then, the automatic segmentation unit 12 outputs information on the sample point or time of the audio sound signal to which each syllable or phoneme is assigned as a candidate for a phoneme boundary.

Note that the syllable or phoneme matching of the present invention is not limited to the hidden Markov model in the above-described embodiment, and is even a matching method capable of performing time expansion and contraction (not depending on speech time), such as DP matching. do it.

(Embodiment 6) Hereinafter, a sixth embodiment of the present invention will be described with reference to the drawings.

In FIG. 4, 11 is a signal input unit, 21 is a text input unit, 31 is a phoneme symbol notation conversion unit, 41 is an audio signal processing unit, 42 is a syllable or phoneme matching unit,
Reference numeral 3 denotes a correction unit based on GUI control, and reference numeral 12 denotes a segmentation result output unit.

In this embodiment, the automatic segmentation section 14 of the second embodiment comprises an acoustic signal processing section 41 and a syllable or phoneme matching section 42.
As described above, the segmentation is performed by the automatic segmentation unit 12 using a plurality of different methods, so that it is possible to prevent missing of the phoneme boundary candidates.

(Embodiment 7) Hereinafter, a seventh embodiment of the present invention will be described with reference to the drawings.

In FIG. 5, 11 is a signal input unit, 21 is a text input unit, 41 is an audio signal processing unit, 31 is a phoneme symbol notation conversion unit, 51 is a candidate reduction unit, 42 is a syllable or phoneme matching unit, and 13 is Correction unit by GUI control, 14
Is a segmentation result output unit.

In the present embodiment, the operability can be improved by introducing the candidate reduction unit 51 to narrow down the number of phoneme boundary candidates handled by the operator in advance on the GUI. The specific operation of the candidate reducing unit 51 will be described with reference to FIG. 9 as an example of a method of reducing candidates centering on the boundary between a voiced part and an unvoiced part. It is assumed that the text and the audio sound signal of “joy” are input to the text input unit 21 and the signal input unit 11, respectively, and the output of the phoneme symbol notation conversion unit 31 is “KaNki”. Each of the phoneme boundary candidates output from the acoustic signal processing unit 41 corresponds to one of (1) to (6) in FIG. Assuming that there are four places where the maximum point of the change in the number of zero crossings is present in the output of the acoustic signal processing unit 41, the four places are uniquely identified in (1), (2), (4), and (5) of FIG. Is done. Furthermore, when the sample point giving the boundary of (5) is determined in this way, since there is only a voiced sound i thereafter, candidates other than the candidate giving the end of the voiced part can be deleted. When the sample point that gives the boundary between (2) and (4) is determined, it is sufficient to leave a candidate that gives the boundary between a and N. In this case, the candidate that gives this boundary cannot be identified robustly. , Conversely, reduce the number of candidates by deleting candidates that give very little this boundary. Also, as in this example, the number of relatively robust candidates and the number of corresponding phoneme symbols do not always match.
If they do not match, syllable or phoneme matching unit 4
A similar candidate reduction is performed by assigning a candidate close to the result of the rough segmentation using the result of the second segmentation.

The correction unit 13 based on the GUI control receives the output of the phoneme boundary candidates received from the candidate reduction unit 51, the syllable or phoneme matching unit 42, and the phoneme symbol notation conversion unit 31, and displays them on the screen. FIG. 11 shows a display example.

In FIG. 11, the work area 105 and the display area 102 have the waveform of the audio sound signal and the candidate
Many short vertical lines that are phoneme boundary candidates received from
A long vertical line that is a phoneme boundary candidate received from the syllable or phoneme matching unit 42 and a result of labeling of the syllable or phoneme matching are displayed as phoneme symbol notation. The operator may correct these phoneme boundary candidates to determine an actual phoneme boundary.

As described above, in the present embodiment, the operator's operation can be made simpler by automatically narrowing down the number of phoneme boundary candidates displayed on the GUI screen in advance.

Embodiment 8 Hereinafter, an eighth embodiment of the present invention will be described with reference to the drawings.

In FIG. 6, 13 is a correction unit by GUI control, 60 is an input unit, 61 is a display unit, 62 is a storage unit, 6
3 is an instruction detecting unit, 64 is an instruction interpreting unit, and 14 is a segmentation result output unit. The input unit 60 receives a speech signal, a result of phoneme segmentation performed using some means, a phoneme symbol, and the like. The storage unit 62 stores the data input to the input unit 61. Display section 61
Displays the data stored in the storage unit 62 on the screen of the GUI. An operator who performs voice segmentation selects a candidate that is considered to represent a phoneme boundary using a mouse on the GUI. The instruction detecting unit 63 detects the mouse operation of the worker. The instruction interpreting unit 64 interprets the selected candidate as a phoneme boundary, and stores it in the storage unit 62 as the determined segmentation position. The display unit 61 redisplays the data on the screen based on the newly stored data.

FIG. 12 shows an example of the result of voice segmentation performed by an operator according to this procedure. In FIG. 12, long vertical lines displayed in the work area 105 and the display area 102 are the results of phoneme segmentation, and are phoneme symbol notations in which alphabets are labeled. The voice segmentation result finally determined by the operator in this way is sent to the segmentation result output unit 14 and output.

As shown in FIG. 14, the processing as described in the seventh embodiment can be put together in place of the input unit 60. That is, in the seventh embodiment, the speech unit and the phoneme segmentation result or the phoneme symbol performed by using some means are input by the input unit 60. Instead, the correction by the GUI is performed by the processing as shown in FIG. A voice signal, a phoneme segmentation result, or a phoneme symbol can also be input to the unit 13.

In the present embodiment, the instruction detecting unit 63
Detects mouse movements, but is not limited to mouse movements. Other operations such as touching the touch panel may be detected, or the operator's instructions may be understood by recognizing the operator's voice.

(Embodiment 9) This embodiment is characterized in that, in addition to the function of the embodiment 8, the operator can perform the segmentation work while confirming by listening to the sound.

FIG. 7 shows the modification unit 13 by GUI control according to the seventh embodiment in detail.
Is a storage unit, 63 is an instruction detecting unit, 64 is an instruction interpreting unit,
Reference numeral 71 denotes an audio cutout unit, and reference numeral 72 denotes an audio reproduction unit.

An example of the operation according to the configuration shown in FIG. 7 will be described in detail below.

The operator can use, for example, G as shown in FIG.
By selecting the candidate confirmation button 120 on the UI screen using a mouse, it is possible to listen to the voice segment sandwiched between the phoneme boundary candidates and the voice segment sandwiched between the selected phoneme boundaries. At this time, the instruction interpreting unit 74 interprets a voice section to be reproduced. The voice cutout unit 75 cuts out a voice section to be reproduced from the voice acoustic signal called from the storage unit 72. This voice section is a voice playback unit 7
At 6 the audio signal is reproduced.

As described above, in the present embodiment, it is possible for the operator to check whether the correct segmentation has been performed by listening to the reproduced sound.

(Embodiment 10) In this embodiment, in addition to the function of the embodiment 8, by applying the segmentation result by the operator to speech synthesis, the segmentation can be performed with the accuracy required for speech synthesis. Can be confirmed by hearing.

FIG. 8 shows the details of the correction unit 13 based on the GUI control according to the seventh embodiment.
Is a storage unit, 63 is an instruction detecting unit, 64 is an instruction interpreting unit,
81 is a voice synthesizing unit, and 82 is a voice reproducing unit.

An example of the operation according to the configuration shown in FIG. 8 will be described in detail below.

For example, the operator can use G as shown in FIG.
By selecting the confirmation playback button 108 on the UI screen with the mouse, the user can listen to the synthesized speech using the information of the pre-selected phoneme boundaries, and the accuracy of the phoneme boundaries can be reduced to a practical level in advance. Can be checked whether it is enough. At this time, the operator interprets the instruction 6
Reference numeral 4 indicates that the information of the segmentation position in the storage unit 62 is sent to the voice synthesizing unit 81, and the synthesized voice synthesized by the voice synthesizing unit 81 is reproduced from the voice reproducing unit 82.

Note that by adding the voice cutout unit 71 described in the ninth embodiment to the correction unit 13 based on the GUI control shown in the present embodiment, the functions described in the ninth embodiment and the functions described in the present embodiment are added. It can have the functions described in. FIG. 13 shows the configuration of the correction unit 13 based on the GUI control at this time. FIG. 13 illustrates the modification unit 13 by GUI control according to the seventh embodiment in detail, where 61 is a display unit,
62 is a storage unit, 63 is an instruction detection unit, 64 is an instruction interpretation unit, 71 is a voice cutout unit, 81 is a voice synthesis unit, 131
Is an audio reproducing unit. With this configuration, it is possible to confirm by listening between the segmentation candidates, and by listening to the actually synthesized speech, it is possible to confirm whether or not the accuracy of the phoneme boundary is sufficient at a practical level. Another correction can be made based on the result.

The signal input unit is an example of the signal input unit of the present invention, the automatic segmentation unit is an example of the automatic segmentation unit of the present invention, and the correction unit by GUI control is an example of the correction unit of the present invention. The text input unit is an example of the text input unit of the present invention, and the phoneme symbol notation conversion unit is an example of the phoneme symbol notation conversion unit of the present invention.

Further, the signal input means of the present invention is not limited to the one for inputting the audio and sound signals in the above-mentioned embodiment, but can also input an image signal. When the signal input means inputs an image signal, the automatic segmentation means can perform segmentation on the image. That is, processing such as separation of the background of the image from the object can be performed. The correction means can correct the segmentation result for the image by GUI control.

Further, in the present embodiment, the language type has been described using Japanese as an example. However, the language type is not limited to Japanese, but may be English, German, French, Korean, Chinese or the like. It only needs to be a language type that can be targeted.

Further, the present invention is also a medium characterized by storing a program for realizing all or a part of each means for realizing the function.

[0061]

As is apparent from the above description, it is possible to provide a segmentation assisting device that can perform a high-accuracy segmentation even by a person without advanced technical knowledge.

[Brief description of the drawings]

FIG. 1 is a block diagram of a first embodiment of a segmentation assisting device according to the present invention.

FIG. 2 is a block diagram of a segmentation assisting device according to a second embodiment of the present invention.

FIG. 3 is a block diagram of a segmentation assisting device according to third, fourth, and fifth embodiments of the present invention.

FIG. 4 is a block diagram of a segmentation assisting device according to a sixth embodiment of the present invention.

FIG. 5 is a block diagram of a segmentation assisting device according to a seventh embodiment of the present invention.

FIG. 6 is a block diagram of a segmentation assisting device according to an eighth embodiment of the present invention.

FIG. 7 is a block diagram of a ninth embodiment of a segmentation assisting device according to the present invention.

FIG. 8 shows a tenth embodiment of the segmentation aid according to the present invention.
FIG. 2 is a block diagram according to the embodiment.

FIG. 9 shows locations of phoneme boundaries in a seventh embodiment of the segmentation assisting apparatus according to the present invention.

FIG. 10 shows a first example of the segmentation assisting device according to the present invention.
7 is an example of a GUI screen according to the embodiment.

FIG. 11 shows a seventh embodiment of the segmentation assistance device according to the present invention.
7 is an example of a GUI screen according to the embodiment.

FIG. 12 shows an eighth embodiment of the segmentation assisting apparatus according to the present invention.
7 is an example of a GUI screen according to the embodiment.

FIG. 13 shows a first example of the segmentation assisting device according to the present invention.
0 is a block diagram according to the embodiment.

FIG. 14 shows an eighth embodiment of the segmentation assisting apparatus according to the present invention.
15 is a replacement example of the input unit in the embodiment.

FIG. 15 is a diagram showing a conventional strategy for determining phonemes of an expert system for performing phoneme segmentation in continuous speech.

[Explanation of symbols]

 Reference Signs List 11 signal input unit 12 automatic segmentation unit 13 correction unit by GUI control 14 segmentation result output unit 21 text input unit 31 phoneme symbol notation conversion unit 41 acoustic signal processing unit 42 syllable or phoneme matching unit 51 candidate reduction unit 61 display unit 62 storage unit 63 Instruction detecting unit 64 Instruction interpreting unit 71 Voice cutout unit 72 Voice reproducing unit 81 Voice synthesizing unit 82 Voice reproducing unit

Claims (13)

[Claims] 1. A signal input unit for inputting an audio signal or an image signal, and a segmentation is automatically performed on the audio signal or the image signal input by the signal input unit to calculate a segment boundary candidate. Automatic segmentation means; and displaying the segment boundary candidates calculated by the automatic segmentation means on a screen, selecting and correcting the candidates while confirming the segment boundary candidates by GUI control, and performing segmentation by selecting or correcting the candidates. And a means for assisting segmentation. 2. The method according to claim 1, wherein the automatic segmentation means is provided separately from the signal input means, and is used when performing automatic segmentation by the automatic segmentation means.
2. The segmentation assisting device according to claim 1, further comprising text input means for inputting text used when selecting or correcting the segment boundary candidate by UI control. 3. A phoneme symbol notation conversion means for converting the text inputted by the text input means into a phoneme symbol notation, wherein the phoneme symbol notation is replaced by the automatic correction means instead of using the text. 3. The method according to claim 2, wherein the correction means is used.
A segmentation aid as described. 4. The method according to claim 1, wherein the automatic segmentation means calculates phoneme boundary candidates by acoustic signal processing using the number of zero crossings, pitch, power, audio formant, cepstrum or phase of the audio signal.
The segmentation assisting device according to any one of claims 1 to 3. 5. The method according to claim 2, wherein the automatic segmentation means calculates phoneme boundary candidates by syllable or phoneme matching using the text or phoneme symbol notation. Segmentation aids. 6. The method according to claim 2, wherein the automatic segmentation means calculates a phoneme boundary candidate by the acoustic signal processing, and calculates a phoneme boundary candidate by syllable or phoneme matching. A segmentation aid according to any of the preceding claims. 7. The automatic segmentation means inputs the number of phoneme boundary candidates given by the audio signal processing using the number of zero crosses, pitch, power, audio formant, cepstrum or phase of the audio signal, and the text input means. Comparing the converted text with the number of phoneme boundary candidates obtained from the converted phoneme symbol notation, and reducing the number of the phoneme boundary candidates given by the acoustic signal processing based on a predetermined criterion. The segmentation assisting device according to claim 6, wherein 8. The automatic segmentation means for reducing the number of phoneme boundary candidates given by the acoustic signal processing using the phoneme boundary candidates calculated by the syllable or phoneme matching. 8. The segmentation assistance device according to 7. 9. An input means for inputting an audio signal or an image signal and a segmentation candidate segmented based on the audio signal or the image signal, and an operator selecting, moving, deleting or adding the segmentation candidate via a GUI. And a correcting means for performing a segmentation correcting operation by the segmentation assisting device. 10. The segmentation work according to claim 1, wherein said correcting means performs a segmentation operation by selecting, moving, deleting or adding any of said phoneme boundary candidates.
10. The segmentation aid according to any one of 3 or 9. 11. The segmentation assisting device according to claim 1, wherein the correction unit corrects a speech segment between any of the phoneme boundary candidates by listening. 12. The method according to claim 1, wherein the correction unit performs voice synthesis based on a segmentation result selected by a worker via a GUI and performs another correction by listening. A segmentation aid according to any of the preceding claims. 13. A medium storing a program for realizing all or a part of the function of each of the means according to claim 1. Description: