JP2014062970A - Voice synthesis, device, and program - Google Patents

Abstract

Kind Code: A1 In a speech synthesis process for generating synthesized speech of an utterance style with a large change in prosody and voice quality including colloquial speech, the sound quality (feeling of real voice) of the characteristic part (spoken expression part) of spoken speech Make it high.
A (quasi) real voice selection unit (103) selects a voice or a voice based on a real voice as a characteristic part of a colloquial speech based on designation by a (quasi) real voice ID (114), and a voice quality information extraction unit (105) Based on the result of analyzing the prosody and voice quality of the voice, the voice quality-considered waveform synthesis unit 110 adjusts the prosody of the other synthesized voice parts, and performs a synthesized voice waveform generation process considering the analyzed voice quality. As a result, it is possible to generate a synthesized speech in which the prosody and voice quality are not greatly shifted between the real voice portion and the synthesized speech portion.
[Selection] Figure 1A

Description

  The present invention relates to a speech synthesizer, and relates to a speech synthesizer technique for generating speech style synthesized speech.

  There is a text-to-speech synthesis technology that converts text into speech and reads it out, and a text-to-speech synthesis system that uses it. Applications of such technologies and systems include, for example, guide voice for car navigation, email reading and voice interaction interfaces on mobile phones and smartphones, screen readers for visually impaired people, and reading functions for electronic books. .

  The importance of speech synthesis technology has increased in recent years. In addition to the widespread use of car navigation, mobile phones and smartphones, the user interface using voice has been used more than ever due to the universal design orientation for disabled and elderly people. Furthermore, in recent years, the spread of electronic book terminals has started, and the need for speech synthesis technology essential for speech reading has expanded.

  In recent needs for speech synthesis technology, there are an increasing number of examples used in speech dialogue processing such as car navigation and mobile phones. In addition, in such applications, there is an increasing need to synthesize spoken / spoken speech so that users can naturally advance conversations. Even conventional speech synthesis techniques exist that can generate colloquial speech by devising the prosody (inflection, rhythm, strength, etc.) of the synthesized speech (see Patent Document 1).

JP 2011-28131 A JP 2008-107454 A JP 2009-020264 A

  However, in the actual speech of colloquial tone, the prosody and voice quality of the characteristic expression unique to the colloquial tone (for example, “Kana” at the end of the sentence) is large. Reproduction of actual colloquial speech has not been fully realized in terms of voice quality because of emphasis on reproduction.

  An object of the present invention is to solve the above-mentioned problems and to generate a characteristic part of spoken tone speech (spoken language) in speech synthesis processing for generating synthesized speech of an utterance style with a large change in prosody and voice quality, including spoken tone speech. The object is to provide a speech synthesis method, device, and program capable of improving the sound quality (feeling of real voice) of (expression part).

  Furthermore, an object of the present invention is to provide a speech synthesis method, apparatus, and program that does not cause a great difference in prosody and voice quality between a real voice part and a synthesized voice part when a real voice is inserted into the spoken phrase part. Is to provide.

  In order to achieve the above object, in the present invention, a speech synthesis method in a processing unit that outputs synthesized speech, the processing unit is a voice data corresponding to a characteristic part of an utterance style from input text, or Select prosody information and voice quality information from the selected (quasi) real voice data, select quasi real voice data (hereinafter referred to as (quasi) real voice data) according to the real voice in terms of prosody and voice quality, and extract the prosodic information The synthesized speech waveform data corresponding to the input text is generated in consideration of the extracted voice quality information by adjusting the prosody information of the synthesized speech corresponding to the input text based on the Provided is a speech synthesis method for joining (quasi) real voice data at connection end points.

  In order to achieve the above object, according to the present invention, the speech synthesizer includes a processing unit and a storage unit, and the processing unit corresponds to a characteristic part of the utterance style from the input text (semi-standard). ) Select real voice data, extract prosodic information and voice quality information from the selected (quasi) real voice data, and adjust and extract prosody information of synthesized speech corresponding to the input text based on the extracted prosodic information The synthesized speech waveform data corresponding to the input text is generated in consideration of the voice quality information, and the generated synthesized speech waveform data and the selected (quasi) real voice data are joined and output at the connection end points. Providing equipment.

  Furthermore, in order to achieve the above object, according to the present invention, there is provided a speech synthesis program executed by a processing unit of a speech synthesizer. Select (quasi) real voice data, extract prosodic information and voice quality information from the selected (quasi) real voice data, and adjust the prosodic information of the synthesized speech corresponding to the input text based on the extracted prosodic information In consideration of the extracted voice quality information, synthesized speech waveform data corresponding to the input text is generated, and the synthesized speech waveform data generated and the selected (quasi) real voice data are joined at the connection end point. Provide a program.

  According to the present invention, in speech synthesis processing for generating synthesized speech of an utterance style with a large change in prosody and voice quality including colloquial speech, a voice or a real voice is included in a portion characteristic of the utterance style (a spoken expression portion). The overall sound quality (feeling of real voice) can be improved by using sound data of conforming quality. Furthermore, the difference in prosody and voice quality between the (quasi) real voice and other synthesized voice parts can be reduced as much as possible, and the uncomfortable feeling of the synthesized voice felt in the prior art can be reduced.

It is a figure explaining an example of the whole function structure of the speech synthesizer of Example 1. FIG. FIG. 2 is a diagram illustrating an example of a hardware configuration of a speech synthesizer according to the first embodiment. It is a figure which shows the example of data stored in the (quasi) real voice audio | voice storage part based on Example 1. FIG. It is a figure which shows the example of output data of the input analysis part based on Example 1. FIG. It is a figure which shows the example of output data of the prosody generation part based on Example 1. FIG. It is a figure which shows the example of data stored in the (quasi) real voice audio | voice storage part based on Example 1. FIG. It is a figure which shows the output data example of the prosody information extraction part based on Example 1. FIG. It is a figure which shows the whole process flowchart of the prosody generation part based on Example 1. FIG. It is a figure which shows the flowchart of the prosody information generation process of the prosody generation part based on Example 1. FIG. 6 is a schematic diagram of prosody adjustment processing in the prosody generation unit according to Embodiment 1. FIG. It is a figure which shows the example of output data of the prosody generation part based on Example 1. FIG. It is a figure which shows the process flowchart of the voice quality extraction part based on Example 1. FIG. It is a figure which shows the example of data stored in the (quasi) real voice audio | voice storage part based on Example 1. FIG. It is a figure which shows the process flowchart of the voice quality consideration waveform synthetic | combination part based on Example 1. FIG. It is a figure explaining the whole function structure of the speech synthesizer based on Example 2. FIG. It is a figure which shows the example of data of the reading-out scenario storage part based on Example 2. FIG. It is a figure which shows the example of data of the whole area voice quality information storage part based on Example 2. FIG. It is a figure which shows the process flowchart of the voice quality consideration waveform synthetic | combination part based on Example 2. FIG. It is a figure explaining the whole function structure of the speech synthesizer based on Example 3. FIG. It is a figure which shows an example of a structure of the content data based on Example 3. FIG. It is a figure which shows an example of a structure of the content data based on Example 3. FIG. It is a figure which shows an example of a structure of the whole area voice quality information based on Example 1. FIG.

  Hereinafter, various embodiments will be sequentially described with reference to the drawings. In the present specification, “real voice” refers to a real voice in a normal sense, and “quasi-real voice” refers to a synthetic real voice similar to a real voice in terms of prosody and voice quality, “(quasi) real voice”. The term “means” means both a real voice in a normal sense and a synthetic voice that conforms to the real voice in terms of prosody and voice quality. For example, “real voice data” means real voice data in a normal sense, and “quasi-real voice data” means synthetic real voice data that conforms to real voice in terms of prosody and voice quality. “Real voice data” means both real voice data and semi-voice data.

  The first embodiment relates to a speech synthesizer and system. With the configuration of the speech synthesizer and the system of the present embodiment, a basic function for converting the text read out by the user into synthesized speech is realized. In the following, each unit of the internal processing of the speech synthesizer and the system is referred to by the expression “part”. However, the form of a system or program realized as a software program without being realized by a device as hardware. It is possible to realize the function.

  FIG. 1A is a diagram for explaining the overall functions of the speech synthesis apparatus and system of this embodiment. In the speech synthesizer of the present embodiment, when a (quasi) real voice ID 114 is given as an input in addition to the input text (Kanji sentence) 112 to be read out, the speech synthesis process is performed and the synthesized speech 113 is obtained. Is output. In order to realize this speech synthesis processing, a normal speech synthesizer such as an input analysis unit 106, prosody generation unit 107, voice quality-considered waveform synthesis unit 110, connection synthesis unit 111, prosody model 108, and segment database (DB) 109 is used. A processing unit and a function processing unit are provided. In addition to this, the (quasi) real voice voice storage unit 101, the quasi real voice voice generation unit 102 that generates a synthetic real voice according to the real voice in terms of prosody and voice quality, characteristic of the configuration of the present embodiment, (Quasi) real voice selection unit 103, prosody information extraction unit 104, voice quality information extraction unit 105, and voice quality-considered waveform synthesis unit 110 is specially selected based on the voice quality information extracted by voice quality information extraction unit 105. It is a functional processing unit that performs a simple waveform synthesis process. These processing units and function processing units in the apparatus of FIG. 1A and the system can be realized, for example, by executing a program in the processing unit described below.

  In the following, processing performed by each function processing unit constituting this embodiment will be described first. Prior to that, FIG. 1B shows a specific example of the hardware configuration of the speech synthesizer and system that realizes this overall functional configuration. Show. Note that this hardware configuration can be used not only as the first embodiment but also as a hardware configuration of a speech synthesizer and system of other embodiments described later.

  In the configuration of FIG. 1B, reference numeral 120 denotes a normal computer such as a personal computer, and a central processing unit (CPU) 121 as a processing unit and a memory (MM) as a storage unit, which are interconnected by an internal bus 128. ) 122, a network interface 123 connected to the network 124, and an input / output unit (I / O) 126. In addition, a hard disk drive (HDD) 125 serving as a storage unit and a speaker 127 connected to the input / output unit 126 are provided outside thereof. The HDD 125 as an external storage unit stores various types of function programs for speech synthesis that are loaded into the MM 122 during operation and executed by the CPU 121 as a processing unit, and various databases such as a segment database (DB). The The speaker 127 connected to the input / output unit 126 outputs synthesized speech that is a result of speech synthesis by the speech synthesizer and system.

  Returning to the functional configuration diagram of the first embodiment shown in FIG. 1A, the input text 112, which is a kanji sentence, is a text to be read out such as “Thank you for your consideration” or “Is the weather today sunny?”. In the above example, the number of sentences is one, but it is possible to input a long text obtained by combining a plurality of sentences. In FIG. 1A, a kanji sentence is used. Of course, a foreign language such as English or Chinese may be used. In that case, internal speech synthesis processing must use program data corresponding to the foreign language.

  The (quasi) real voice ID 114 is information that specifies which (quasi) real voice is used for which character string portion in the input text 112. FIG. 2 shows an example of the database 200 in the (semi) voice voice storage unit 101 storing the character string corresponding to the ID and the corresponding (semi) voice voice. Here, the (quasi) real voice means a normal real voice or a synthetic real voice similar to the real voice in terms of prosody and voice quality as described above. The user of the speech synthesizer inputs information “2”, for example, as the (quasi) real voice ID 114 together with the input text 112 “Is the weather today sunny?”. The user refers to the (quasi) real voice database 200 shown in FIG. 2 to determine an ID that designates a (quasi) real voice to be used as a part of the input text. For example, for a colloquial expression such as “Kana”, there can be “Kana” uttered with various prosody (inflections). The user of the speech synthesizer needs to select a (quasi) real voice having the desired prosody and specify “1” or “2” as the corresponding (quasi) real voice ID.

  The input analysis unit 106 receives the input text 112 and the (quasi) real voice ID 114 and performs an analysis process thereof. The input text 112 is subjected to language analysis processing for determining the pronunciation and accent of words included in the text, and the analysis result is passed to the prosody generation unit 107.

  On the other hand, the (semi) voice voice ID 114 is passed to the (semi) voice voice selection unit 103 as it is. In another embodiment to be described later, a case where the (quasi) real voice ID 114 is not designated by a simple ID or the case where the (quasi) real voice ID itself is not designated will be described. In this case, the input analysis unit 106 passes information necessary for selecting the (quasi) real voice from the analysis result of the input text 112 to the (semi) real voice selection unit 103.

  The language analysis processing performed by the input analysis unit 106 is information output from the input analysis unit 106 that has already been performed by various speech synthesizers, for example, information 300 shown in FIG. Of course, the information 300 in FIG. 3 is the minimum necessary, and there may be a case where various information is added and output.

  The prosody generation unit 107 determines prosodic information that the synthesized speech should have using the input analysis result (FIG. 3). The determined prosodic information includes, for example, the length (ms), height (Hz), and strength (dB) of each phoneme constituting the input text. The prosody generation unit 107 uses the prosody model 108 to determine the prosody information.

  FIG. 4 shows an example of prosody information output from the prosody generation unit 107 as information 400. However, the prosody generation unit 107 of the present embodiment adjusts the prosody information 400 of the generated input text based on the prosody information of the (quasi) real voice output from the prosody information extraction unit 104. This process will be described later.

  The voice quality-considered waveform synthesis unit 110 generates synthesized speech data based on the prosody information 400 of FIG. 4 output from the prosody generation unit 107. As the basic processing performed here, the waveform synthesis processing performed in the conventional speech synthesis processing can be used. In the waveform synthesizing process, synthesized speech is usually generated with reference to speech data called the segment DB 109. The technique used here is, for example, a large number of speech waveform data (segments) in units of phonemes and syllables, stored in the unit DB, and an optimal sequence of segments for constructing an input text from them. A technique of selecting (arrangement) and then performing waveform connection processing or waveform superimposition processing to generate synthesized speech can be used. An algorithm such as a superposition process using crossfading can be used for the waveform connection process, and an algorithm such as a time domain pitch synchronization overlapping process (TD-PSOLA) can be used for the waveform superimposition process. In addition to this technique, the speech analysis parameters are stored in the DB in units of phonemes and syllables, and the speech analysis parameters corresponding to the phonemes constituting the input text are extracted and passed through the inverse filtering process. A parameter synthesis technique for converting into a speech waveform can also be used. In recent years, there has been a method of modeling speech analysis parameters using a probabilistic model called HMM (Hidden Markov Model) to realize smoother parameter changes.

  The waveform synthesizing process described above describes the basic process of the waveform synthesizing process performed by the voice quality considering waveform synthesizing unit 110. The voice quality-considered waveform synthesizer 110 of this embodiment uses the voice quality information of the (quasi) real voice output from the voice quality information extractor 105 and uses the segment selection process or the voice analysis parameter adjustment process described above. To implement. As a result, it is possible to generate a synthesized voice having a voice quality close to that of a (semi) voice voice. This process will be described later.

  The connection synthesizer 111 performs a process of combining the synthesized speech output from the waveform synthesizer 110 and the (semi) voiced voice data output from the (semi) voiced voice selection unit 103. In response to the input text 112 “Is it fine today” and (quasi) real voice ID 114 “2”, the synthesized speech corresponding to “I am fine today” is output from the voice quality-considered waveform synthesizing unit 110 and at the same time ( The quasi-speech voice selection unit 103 outputs quasi-speech voice data corresponding to the ID “2”. In the connection synthesis unit 111, the portion corresponding to “Kana” among the synthesized speech of “Today is sunny” output from the voice quality-considered waveform synthesis unit 110 is output from the (quasi) real voice selection unit 103. The process of replacing with (quasi) real voice data of “Kana” is implemented. For example, a cross-fade overlay process can be used for this replacement process. In addition, the connection processing of the real voice and the synthesized voice disclosed in Patent Document 2 and Patent Document 3 by the present inventors can be used for the replacement processing. As a result, the synthesized speech 113 with higher sound quality is output by using the designated (quasi) real voice.

  The flow of the main speech synthesis process for converting the input text 112 into the synthesized speech 113 has been described above. Hereinafter, a function processing unit characteristic of the speech synthesizer according to the present embodiment will be described.

  The (semi) voice voice storage unit 101 stores (semi) voice voice waveform data corresponding to various expressions (Japanese character strings). For example, a list 500 shown in FIG. 5 is an example of data stored in the (quasi) real voice storage unit 101. In this example, the corresponding (quasi) real voice segment data is stored for the expression used as the end of a spoken tone (spoken language) speech. Of course, it is not necessary to limit the expression to the end of a sentence, and (quasi) real voice data can be stored in association with an expression that requires the quality of real voice. The voice waveform stored here stores the voice length (ms) and the pitch (Hz) at both end points as additional information to be used in the connection synthesis unit 111 later. .

  The quasi-real voice generation unit 102 is a processing unit for generating quasi-real voice that is synthesized voice data according to the quality of the real voice. The purpose of this processing unit is to increase the types of real voices that can be used other than those stored in the (quasi) real voice storage unit 101. However, if it is determined that the required sound quality can be ensured only by the (semi) voice voice data stored in the (semi) voice voice storage unit 101, the quasi-speech voice generation unit 102 is not necessary. Therefore, processing in the semi-real voice generation unit 102 will be described in the second embodiment. In the description of the present embodiment, the description will be continued on the assumption that the quasi-real voice generation unit 102 does not operate. However, as a modification of the present embodiment, (quasi-) real voice data stored in the (quasi) real voice storage unit 101 is used. If it is determined that the necessary sound quality cannot be secured, the quasi-speech voice generation unit 102 can generate the required quasi-speech voice, that is, a synthetic real voice that is similar to the real voice in terms of prosody and voice quality. .

  Subsequently, in the (semi) voice voice selection unit 103, based on the (semi) voice voice ID 114 passed through the input analysis unit, the corresponding (semi) speech voice data is converted to the (semi) speech voice storage unit 101, Alternatively, it is obtained from the quasi-real voice generation unit 102. In the above example, when “2” is designated as the (quasi) real voice ID 114, it corresponds to ID = 2 from the data stored in the (quasi) real voice storage unit 101 shown in FIG. ) Real voice data and its additional information are selected. The data selected here is passed to the prosodic information extraction unit 104, the voice quality information extraction unit 105, and the connection synthesis unit 111.

  The prosody information extraction unit 104 performs prosody information extraction processing on the real voice data output from the (quasi) real voice selection unit 103. Prosodic information is information such as the length (ms), height (Hz), and strength (dB) of each phoneme constituting speech data, and uses various speech signal processing algorithms that have been widely used in the past. Can be analyzed. For example, a pitch extraction algorithm based on autocorrelation function peak detection, an LPC residual waveform autocorrelation function peak detection algorithm, or the like can be used for sound pitch (Hz) analysis. There are also power analysis algorithms such as the mean square value for sound intensity.

  However, it is not easy to determine a phoneme sequence that constitutes speech and to analyze its time position. By using speech recognition algorithms disclosed in various documents, it is possible to convert (transcribe) real voice data into a phoneme string and determine the time position of each phoneme. However, the current speech recognition algorithm does not have very high recognition accuracy of transcription. Therefore, it is possible to take a technique of converting the character string representation transferred together with the (quasi) real voice data from the (quasi) real voice selection unit 103 into a phoneme string and determining only the time position by the speech recognition algorithm. Even with the current speech recognition algorithm, as long as the phoneme string is determined, the time position of each phoneme can be determined with high accuracy to some extent. By using such various speech signal processing algorithms, the prosody information extraction unit 104 generates prosody information as shown in FIG. 6 for real voice data. Of course, this prosodic information is extracted (and further manually corrected) from the speech data list 500 stored in the (semi) voice voice storage unit 101, and the (semi) speech voice together with the speech data. A configuration for storing in the storage unit 101 is also possible.

  The prosody information 600 shown in FIG. 6 output from the prosody information extraction unit 104 is transferred to the prosody generation unit 107. The prosody generation unit 107 generates the prosody information 400 of the synthesized speech based on the input text analysis result 300 passed from the input analysis unit 106. In the speech synthesizer of the present embodiment, this prosody information extraction unit The prosody information 400 of synthesized speech is adjusted using the prosody information 600 of the real voice data passed from 104. This adjustment process analyzes how the prosody (intonation, rhythm, etc.) when the input text 112 is directly speech-synthesized is combined with the specified (quasi) real voice ID 114, and is generated first. An adjustment result obtained by changing the prosodic information is output as final prosodic information 400. As a result, the prosodic information is adjusted so as to be smoothly connected to the selected real voice data. Details of the prosody information adjustment processing in the prosody generation unit 107 will be described later.

  The voice quality information extraction unit 105 performs voice quality information extraction processing on the real voice data output from the (quasi) real voice selection unit 103. The voice quality information is information indicating the voice color of the voice data, and corresponds to, for example, a cepstrum coefficient, a frequency analysis coefficient by FFT, or the like. Various configurations can be considered for the configuration of the voice quality information, and some embodiments will be described in detail in another example described later. The voice quality information extracted by the voice quality information extraction unit 105 includes at least short-time voice quality information (for both ends) indicating the voice quality near both ends of the real voice data, and the entire area indicating the voice quality for the entire real voice data. There are two types of voice quality information. As the short time voice quality information of the former, short time voice data (referred to as a frame) having a certain length (for example, 20 ms) is cut out from both ends of the real voice data, and the above voice quality analysis process is performed on the voice. Can be applied. On the other hand, there are various conceivable methods for configuring the latter global voice quality information. For example, the simplest method is to perform the above voice quality analysis process on the entire real voice data. In other embodiments described later, other configurations will be described. In this description, the voice quality information extraction unit 105 outputs the short-time frame cepstrum coefficient vectors at both ends of the real voice data as short-time voice quality information, and the cepstrum coefficient vector for the entire real voice data as global voice quality information. to continue.

  The voice quality-considered waveform synthesis unit 110 receives voice quality information (short time and whole area) output from the voice quality information extraction unit 105 and uses it to perform waveform synthesis processing of synthesized speech. As already described above, in this waveform synthesis process, the segment data corresponding to each phoneme (syllable) constituting the input text is extracted from the segment DB 109, and the closeness of the sound between the segment data called connection cost is shown. Search for a unit sequence (optimal unit sequence) whose value is minimum in the entire unit sequence. In calculating the connection cost, by considering the voice quality information (short time and whole area), it is possible to generate a synthesized voice that smoothly connects the designated (quasi-) real voice data and the voice quality. Details of the processing here will be described later.

  The outline of each functional process of the speech synthesizer according to this embodiment shown in FIG. 1A has been described above. As a result of this processing, it is possible to generate a synthesized voice obtained by voice synthesis of the designated input text 112 by replacing and connecting with the voice voice data corresponding to the designated (quasi) real voice ID. By adjusting the voice based on the voice quality and prosody of the real voice data part, it is possible to make a synthesized voice in which the voice quality and prosody are smoothly connected to the real voice data. As a specific example, when the speech of ID = 2 in FIG. 5 is designated as the input text “Is it sunny today” and the real voice data corresponding to “Kana”, the output synthesized speech is “ The (semi) voice data of Fig. 5 is used for the "Nana" part, and the synthesized voice corresponding to "Today is fine" in the previous part is the voice quality and prosody of the real voice data of "Kana" at the end. It will be adjusted to fit. As a result, the synthesized voice of “Kana is fine today” with the prosody and voice quality matched with the real voice “Kana” at the end is output.

Next, the characteristic part of the speech synthesizer of the present embodiment will be described in detail including the processing flow.
An example of the detailed processing flow of the prosody generation unit 107 will be described using the flowcharts shown in FIGS.

  When the prosodic generation process starts, first, the language analysis result 300 output from the input analysis unit 106 is received, and the prosodic information (length, height) of each phoneme (syllable) constituting the input text based on the language analysis result. , Strength). This prosodic information generation processing 703 can directly use prosodic information generation processing implemented in various speech synthesis programs currently used. For example, the process shown in the flowchart of FIG. 8 is performed.

  In FIG. 8, when the language analysis result 300 is input, the pronunciation of each word is broken down into phoneme strings (or syllable strings) (step 802). As a result, the phoneme symbol sequence shown in the “phoneme” column of FIG. 4 is converted.

  Subsequently, an environment information vector is generated for each phoneme (step 803). The environment information vector is obtained by extracting various language environment information possessed by phonemes at respective positions in a vector format based on the information 300 analyzed by language analysis. The language environment information included in the vector includes, for example, whether the phoneme is a consonant / vowel (0 or 1), the number of syllables (number of mora) of a word (accent phrase) including the phoneme, the phoneme The part of speech of the word containing the word, whether the phoneme immediately before the phoneme is a consonant / vowel (0 or 1), whether the phoneme immediately after the phoneme is a consonant / vowel (0 or 1) Whether a word (accent phrase) boundary exists / does not exist immediately before the phoneme (0 or 1), or does a word (accent phrase) boundary exist / does not exist immediately after the phoneme (0 or 1) There can be.

  Subsequently, referring to the generated environment information vector, a continuation length determination process for each phoneme constituting the input text is performed (step 804). There are various examples of this determination algorithm, for example, a method using a decision tree, a method of predicting with a regression line by quantification type I or multiple regression analysis. When the continuation length determination method using a decision tree is used, decision tree data that has been machine-learned using a large amount of speech data in advance is referred to. In the decision tree data, starting from the root node (starting point) of the tree, questions set to each node of the tree (whether the Nth element of the environment information vector is larger / smaller than the value k, etc.) Branching is performed using the extracted environment information vector, and the continuation length value (ms) set for the finally reached leaf node is output as the continuation length of the phoneme having the environment information vector. Similarly, in the case of prediction based on the quantification type I or the multiple regression line, the continuation length of the phoneme is determined by calculating the inner product between the environment information vector of each phoneme and a certain weight coefficient vector. The continuation length value (ms) of the entire input text is determined by executing the continuation length determination process for each phoneme for all phonemes constituting the input text.

  Subsequently, a process for determining the pitch (Hz) of each phoneme is performed (step 805). Similarly, the pitch (Hz) of each phoneme is determined from the extracted environment information vector by using a method such as a decision tree or multiple regression prediction. Finally, in step 806, the strength (dB) for each phoneme is determined. The same method as described above can be used for the processing here.

  In the above description, the continuation length determination process (step 804) is performed, and then the pitch determination process (step 805) and the strength determination process (step 806) are performed. Depending on the type of information included, an embodiment in which the duration is determined after the pitch is determined first, or an embodiment in which the duration and the pitch / strength are determined at the same time are also conceivable. This order is not important for the implementation of this embodiment.

  The prosodic information generation process (step 703) in the flowchart of FIG. 7 can be realized by the process flow of FIG. As a result, the prosodic information sequence for each phoneme as shown in FIG. 4 is output from the prosodic information generation process (step 703).

  Following the prosody information generation process (step 703), the prosody information 600 of the (quasi) real voice data extracted by the prosody information extraction unit 104 is received, and based on the information, the prosody information generation process in step 703 is performed. Adjustment processing of the generated prosodic information 400 is performed. This adjustment process is a process for adjusting the prosodic information 400 of the generated input text so as to match the prosody (overall speech speed, pitch, strength, etc.) of the (quasi) real voice data. is there.

  In the flowchart shown in FIG. 7, first, a continuation length adjustment process (step 705) is executed. Various adjustment methods are possible, but the simplest is that the average duration of the whole sentence (or breath phrase phrase unit etc.) is the average duration of the prosodic information 600 of (quasi) real voice data shown in FIG. There can be a method of increasing or decreasing the duration value of each phoneme so as to be close to the length. The increase / decrease here can be performed by performing the increase / decrease process only on the vowel phoneme duration based on the knowledge that the difference in speech speed greatly affects the vowel phoneme in Japanese speech. Of course, a method of quantifying the influence of the change in speech speed on the consonant phonemes and the influence on the vowel phonemes, and performing the increase / decrease process by switching each of them is also conceivable. Furthermore, the (semi) voiced voice data corresponding to the corresponding section of the sentence speech from which the (semi) voiced voice data was cut out (for example, the (semi) voiced voice data corresponding to “Kanaa” was originally “soon to be finished”). If it has been cut out, the average duration of the portion immediately before “Kana” (the portion corresponding to “Now”) is stored as additional information in the (quasi) real voice storage unit 101 shown in FIG. In addition, there may be a method of performing adjustment processing using the value.

  Subsequently, an overall pitch adjustment process (step 706) is executed. Similarly, in this pitch (pitch) adjustment processing, adjustment processing is performed so that the pitch value of the entire sentence (or breath phrase phrase unit, etc.) is smoothly connected to the pitch of the start point of the (quasi) real voice data. Is called. Although various embodiments can be considered for this adjustment process, it is necessary to make an adjustment such that the pitches coincide at the phoneme locations corresponding to the end points connected to the (quasi) real voice data (FIG. 9a). To match this, as shown in FIG. 9b, the pitch curve for the entire input text generated in step 703 is translated up and down so that the pitches match at the phoneme location at the connection end point. It ’s fine. When the pitch of the head phoneme or the like becomes lower than a certain value due to this parallel movement, the pitch curve is further lowered by rotating the pitch curve with the connection end point as the rotation center point as shown in FIG. 9c. Not too high / not too high. Furthermore, more appropriately, as shown in FIG. 9d, the pitch change can be adjusted by fixing the heights of the start point position and the connection end point position of the pitch curve and expanding / reducing the vertical width of the curve. . The enlargement / reduction ratio in this case is determined using an appropriate prediction method (regression prediction, prediction using a decision tree, etc.) based on the pitch change width of the (quasi) real voice data portion. Further, as described above, a method for predicting based on the value of the pitch change width of the corresponding section of the original sentence voice from which the real voice data is cut out is also conceivable.

  Finally, the prosodic information 1000 shown in FIG. 10 is output as a result of adjusting the continuation length and pitch (and not shown in the flowchart of FIG. 7 but the intensity of the sound) as described above.

  By executing the processing of the flowchart shown in FIG. 7 as described above, the prosody generation unit 107 generates the prosody information of the synthesized speech that smoothly connects the prosody with the (semi) real voice data of the specified (semi) real voice ID. Can be generated.

  As a characteristic part of the speech synthesizer of the present embodiment, a description will be given next with reference to FIG.

  The voice quality information extraction processing according to the flowchart shown in FIG. 11 is an embodiment different from the processing content of the voice quality information extraction unit already described above. In this voice quality information extraction process, in addition to calculating short-time voice quality information at the start and end of (quasi) real voice data output from the (quasi) real voice selection unit 103, Calculates voice quality information for all voice data. In this description, a cepstrum coefficient vector (for example, 16 dimensions) is used as a specific algorithm for calculating voice quality information. Furthermore, as a method of configuring the global voice quality information, a plurality of time positions that are characteristic of the voice color (dominant to the overall voice quality impression) in the (quasi) real voice data are set in advance, and each time position is set. Voice quality information (cepstrum coefficient vector) as a result of weighted addition of short-time voice quality information at all time positions by multiplying the short-time voice quality information (cepstrum coefficient vector) with the specified weighting coefficient as global voice quality information Output.

  Hereinafter, the processing of the voice quality information extraction unit 105 will be described with reference to the flowchart of FIG.

  First, when the process starts, in step 1102, (quasi) real voice data output from the (quasi) real voice selection unit 103 is received. In step 1103, short-time voice quality information is calculated for the (quasi) real voice data. The short-time voice quality information is calculated at two points of the start position and the end position of the (quasi) real voice data. Specifically, a short-time audio frame (for example, 20 ms) is cut out at the start position and the end position (actually, the silent portion is skipped and the sound power exceeds a certain threshold), and the audio frame is extracted. Perform cepstrum analysis. As a result, two pieces of voice quality information (cepstrum coefficient vectors) corresponding to the start and end points are output as short time voice quality information.

  Subsequently, a calculation process step for global voice quality information is performed.

  First, the global voice quality information vector V output in step 1106 is initialized, and in step 1107, the number of voice quality feature points for the (quasi) real voice data is acquired. Here, the voice quality feature point represents a time position where the voice quality (timbre) is characteristic in the (quasi) real voice data. For each (semi) voice voice data stored in the (semi) voice voice storage unit 101, the number of feature points and the time and weight of each feature point position are added as shown in the information 1200 of FIG. And store it. In step 1107, the related information of the corresponding feature points is transferred from the (quasi) real voice selection unit 103 together with the (quasi) real voice data, and the number of feature points is acquired therefrom.

  Similarly to step 1107, step 1108 also acquires feature point related information (time Ti and weight Wi (1 ≦ i ≦ N) of each feature point) that has already been delivered.

  The subsequent steps 1110 to 1113 are repeatedly executed while increasing the variable i from 1 to N. In step 1110, a short time voice frame (for example, 20 ms) is cut out from the (quasi) real voice data at the i-th feature point time. In step 1111, the short-time voice quality information (cepstrum coefficient vector) is calculated, and the value is multiplied by the weight Wi for this feature point and added to the output variable V.

  By repeating these steps, it is possible to calculate global voice quality information that is the result of weighted addition of short-time voice quality information at all feature points specified in the (quasi) real voice data.

  Finally, in step 1104, the calculated short-term voice quality information of the start and end points and global voice quality information (each 16-dimensional cepstrum coefficient vector) are output from the voice quality extraction unit 105.

  In addition, as shown in FIG. 21, another configuration of the calculation method of global voice quality information of the present embodiment can be adopted. In the example of FIG. 21, (quasi) real voice data, as shown in the figure, there is no division (= division 1), two divisions, three divisions,. While running, short-time voice quality information is calculated in each divided section. In the division number i, i pieces of short time voice quality information are output, and representative voice quality information representing the whole is determined from these. As a representative point calculation method, a method using an average value which is a statistical method, a method using a median (median value), a method using a central cluster (centroid) by a clustering method, and the like can be considered. In this way, when k representative voice quality information is calculated, the representative voice quality information of each division is collected and output as k-dimensional global voice quality information which is a global voice quality information vector.

Next, processing of the voice quality-considered waveform synthesis unit 110 will be described as a characteristic part of the speech synthesis apparatus according to the present embodiment with reference to the flowchart of FIG.
This flowchart is obtained by adding some processes characteristic to the present embodiment to the basic configuration of the unit selection type speech synthesis (Unit Selection Speech Synthesis) technique. First, the basic configuration of the unit selection type speech synthesis technology will be described with reference to FIG. 13, and then additional steps characteristic to the speech synthesis apparatus of this embodiment will be described.

  The flowchart on the left side of FIG. 13 shows the flow of the waveform synthesis process using the segment selection method. First, in step 1302, all the segment candidates are listed for each phoneme (syllable or special unit) constituting the reading sentence. The prosody information 1000 as shown in FIG. 10 is passed from the prosody generation unit 107 to the waveform synthesis unit 110. For each phoneme such as “KY” and “O” specified in the phoneme field of the prosodic information, a plurality of segment candidates exist in the segment DB 109. In step 1302, a segment candidate set is searched and enumerated for each segment unit constituting the reading sentence, such as a segment candidate set for “KY” and a segment candidate set for “O”. Subsequently, in step 1303, cost values called target costs are calculated for all of the listed segment candidates.

  The details of the cost value calculation process are described in the central flowchart of FIG. The target cost is calculated based on how far the length / height / strength of each segment candidate is different from the length / height / strength of each phoneme specified in the prosodic information 1000. To calculate the cost value. The calculation formula here may have various embodiments, but since it does not greatly affect the essence of the apparatus of the present embodiment, details are omitted here. Thus, after calculating the cost values for length, height, and strength between the values set in the prosodic information and the segment values, the cost calculation processing for the whole voice quality is performed as a characteristic step in this embodiment. 1315 is performed. Here, the cost value is calculated between the global voice quality information of the (quasi) real voice data output from the voice quality extraction unit 105 and the global voice quality information of the segment to be calculated. The global voice quality information of the (quasi) real voice data is the data of the embodiment as described above. On the other hand, several embodiments can be considered for the calculation method of global voice quality information possessed by the segment. Most simply, the segment data is regarded as one (quasi) real voice data, and the above-described method is used as the global voice quality information extraction method in the voice quality information extraction unit 105, so that the global voice quality for the segment data is obtained. Information can be acquired. Then, distance calculation (such as Euclidean distance) can be performed between two pieces of global voice quality information (for example, a 16th-order LPC cepstrum coefficient vector).

  Further, it is possible to realize a method in which the consonant phoneme segment and the vowel phoneme segment in the segment data are distinguished, and the above-described global voice quality information extraction method is applied only to the vowel phoneme segment. This is because the vowel phoneme section has more information than the consonant phoneme section and has information for determining the voice quality of the speaker and the voice section of the voice section. In general, it is possible to adopt a method in which a weight value is set in accordance with the type of phoneme constituting the segment data, and the contribution to the entire voice quality information of the entire segment data is set.

  In addition, as another method of calculating the whole area voice quality information for the segment data, in the original recorded voice from which the segment data is cut out, for a certain length of voice segment (for example, accent phrase) where the segment data is located Thus, a method of extracting the entire voice quality information already explained by the voice quality information extracting unit 105 and storing the entire voice quality information in the segment DB 109 in association with the segment data is also conceivable. In this case, when searching for and enumerating the segment candidates in step 1302, it is necessary to take out the entire voice quality information associated with the segments.

  Further, when the global voice quality information corresponding to FIG. 21 is adopted, the global voice quality information for the segment data calculated here must also be configured accordingly. There is also a method of simply calculating k-dimensional global voice quality information by the above-described k division for each piece of data and calculating a distance between it and (quasi) real voice data. However, in the case of segment data, the voice length is often shorter than the voice length of (quasi) real voice data, and if the same division number k is simply used, the analysis time length of each division becomes too short. The problem arises. For this, after calculating the whole voice quality information of k division for the original recorded voice from which the segment data is cut out, the divided voice length is equal to or less than the length of the segment data. At that time, it is also possible to adopt a method of limiting the division range to only the segment data. In other words, if the division unit is shorter than the segment data length, the short-term voice quality information is calculated within the segment data. This is a method of calculating voice quality information for speech.

  Further, in the method described here, the difference in length between the (quasi) real voice data and the original recorded voice from which the segment data is cut out is ignored, so the length of the division unit does not correspond even in the same k division. The problem also arises. For this, it is possible to adopt a method in which the time length of the division unit is reduced to, for example, 1000 milliseconds, 500 milliseconds, and 250 milliseconds instead of increasing the number of divisions. As a result, the problem that the analysis time length in each division unit is different can be solved.

  Returning to the description of the target cost calculation step 1303, as described above, the cost value for the length, height, and strength calculated between the segment and the prosodic information (target), and the step 1315, the present implementation. The cost value for the global voice quality information calculated uniquely for the example is finally weighted and the target cost for the segment is determined (step 1316). By repeating the above processing for all segment candidates (step 1317), the target cost calculation processing (step 1303) is performed.

  Subsequently, a connection cost calculation process (step 1304) is performed. Details of this step are shown as a flowchart on the right side of FIG. In this flowchart, there is a part changed from the basic configuration in the unit selection type speech synthesis by the implementation of this embodiment.

  In the calculation of the connection cost in the flowchart on the right side of FIG. 13, first, one adjacent segment candidate x and y is extracted from the read-out sentence (step 1322). That is, it is a set of segments such that a synthesized speech is generated by combining a segment y immediately after the segment x. Since a plurality of segment candidates are searched and enumerated for each segment unit (for example, phoneme) constituting the reading sentence, the number of combinations is enormous. Various methods for reducing the number of combinations have been disclosed and implemented, but the description is omitted because they are not related to the gist of the present embodiment.

  In step 1323, short time voice quality information at the end position is extracted from the left segment x. The method of constructing short-time voice quality information has already been described. In this connection cost calculation process, since it is necessary to calculate the distance between the short-time voice quality information extracted for the (quasi) real voice data described above and the short-time voice quality information of the segment extracted here, These two types of short-time voice quality information are desirably information analyzed by the same method.

  Next, the process is divided depending on whether or not the segment y is a phoneme (unit unit) corresponding to the head part of the (quasi) real voice data. First, if the determination result is NO, that is, if it is not a junction point with (quasi) real voice data, it is the same as the normal voice synthesis process, and in step 1325, the short-time voice quality at the head position of the segment y The information is calculated, and the distance between the short-time voice quality information of the above-mentioned calculated segments x and y is calculated in step 1326.

  On the other hand, when the position of the segment y corresponds to the head part of the (quasi) real voice data, the short time voice quality information at the end of the segment x and the (quasi) real voice calculated by the voice quality extraction unit 105. It is necessary to calculate the distance with the short-time voice quality information at the start point of the data. In step 1327, the short-time voice quality information of the (quasi) real voice data is extracted, and in step 1328, the distance to the short-time voice quality information of the segment x is calculated.

  Finally, in step 1329, a certain connection cost value is output by multiplying the calculated distance between the short-time voice quality information by a certain weight value. This weight value is not constant, and it may be possible to change the weight value depending on whether it is calculated on the left or right route in the connection cost calculation flowchart, and the weight is changed depending on the type of phoneme of the element x or y. There may also be embodiments.

  The above processing is carried out for all unit combinations. Thereby, the connection cost between all the combinations of the segment is set (step 1304).

  Next, a search process for the segment series that provides the minimum cost is performed (step 1305). A plurality of segment candidates are listed for each phoneme (syllable) constituting the reading sentence, and this is a process for determining one of them. In the determination process, the combination of the segment having the smallest cost sum is determined by adding the target cost of the segment and the connection cost between the segments corresponding to the left and right phonemes (syllables). This search process can be realized using a Viterbi algorithm or the like widely used in speech synthesis technology.

  Finally, in step 1306, final synthesized speech generation processing is performed using the determined combination of segments. Here, it is realizable by using the audio | voice signal processing method called a waveform superimposition process or a waveform connection process for every element | segment or between elements. Although there are various embodiments in these methods, the description is omitted because it is not related to the gist of the present embodiment.

  Through the above processing, in the speech synthesis method and apparatus according to the present embodiment, in speech synthesis processing for generating synthesized speech of an utterance style with a large change in prosody and voice quality including colloquial speech, a part characteristic to the utterance style By using the voice data of the quality according to the real voice or the real voice for the (spoken expression part), the overall sound quality (feeling of real voice) can be improved. Furthermore, the difference in the prosody and voice quality between the (quasi) real voice and other synthesized voice parts can be reduced as much as possible, and the uncomfortable feeling of the synthesized voice felt in the prior art can be reduced.

  In this embodiment, an embodiment of a system that synthesizes a plurality of sentences continuously, such as a dialogue system, using the basic configuration of the speech synthesizer described in the first embodiment will be described.

  In this embodiment, a plurality of sentences are synthesized continuously or alternately with a user response based on a scenario or the like, or a mechanism for determining the next reading sentence of the system according to a user response. Realize a system that

  FIG. 14 is a diagram for explaining the entire speech synthesis system of the present embodiment. In this configuration, in addition to the basic configuration of the first embodiment (FIGS. 1A and 1B), a reading scenario storage unit 1401, a reading sentence generation unit 1402, and a global voice quality information storage unit 1403 are provided.

Hereinafter, the flow of processing will be described focusing on the newly added function processing unit in FIG.
In the present embodiment, the input to the speech synthesis system (text and (quasi) real voice ID) is not a single input each time, but is sequentially synthesized from within a scenario composed of a plurality of sentences. The sentence to be decided is decided.

  The reading scenario storage unit 1401 stores such scenario data (data specifying the flow of reading sentences). This reading scenario storage unit 1401 can be formed in a part of the storage unit. There are various ways to compose the scenario data, but the simplest way is to store a set of aloud text and the corresponding (quasi) real voice ID in the form of a list entered in order along the flow of the scenario. It is a technique. In this case, the reading scenario storage unit 1401 stores scenario data 1500 as shown in FIG.

  The read-out sentence generation unit 1402 performs processing of taking out (quasi) real voice IDs corresponding to the read-out sentences in order from the read-out scenario storage unit 1401 and inputting them to the speech synthesizer of this embodiment. Needless to say, this read-out sentence generation unit 1402 can be realized, for example, by executing a program in a CPU that is a processing unit.

  If scenario data 1500 as shown in FIG. 15 is stored, a sentence to be synthesized next and a (quasi) real voice ID may be extracted and input at the timing when the immediately preceding speech synthesis process is completed. . Of course, scenario data can be saved in a more complex format. For example, it is conceivable to use scenario data based on standardized standards such as VoiceXML and SSML (Speed Synthesis Markup Language). In this case, since only the text of the read-out sentence can be specified in these standard data, another correspondence table (FIG. 15 is also an example thereof) is used to determine the (quasi) real voice ID corresponding to the sentence. become. In the preferred embodiment of this embodiment, the input text is analyzed, and (quasi) real voice data is determined using the correspondence table of FIG.

  The global voice quality information storage unit 1403 added by the system of this embodiment adds and stores the global voice quality information extracted by the voice quality information extraction unit 105 each time. The configuration method of the global voice quality information stored here is not particularly limited. In this storage unit 1403, it is necessary to store at least several times of global voice quality information synthesized immediately before. For example, FIG. 16 shows an example 1600 of information stored in the global voice quality information storage unit 1403. In this information 1600, the entire voice quality information relating to the read-out sentence up to the past three times is stored with the current read-out sentence as history 0. In addition, a read-out sentence and a corresponding (quasi) real voice ID are also stored. The information that needs to be stored here also depends on the processing method used for the entire voice quality information of past read-out sentences in the voice quality-considered waveform synthesis processing, as with the length of the previous history. .

  In the voice quality considering waveform synthesis unit 110, in addition to the voice quality information regarding the current reading sentence output from the voice quality information extraction unit 105, the whole voice quality information for the past reading sentence stored in the whole voice quality information storage unit 1403 is also referred to. Then, a synthesized speech for the current reading sentence is generated. This will be described with reference to FIG.

  FIG. 17 is an example of a processing flowchart of the voice quality considering waveform synthesis unit 110 in the present embodiment.

  The processing flowchart of FIG. 17 differs from the processing flowchart of the voice quality-considered waveform synthesis unit 110 of the first embodiment shown in FIG. In FIG. 17, the cost calculation process for the global voice quality information is realized as several more detailed processing steps.

  In step 1701, global voice quality information (hereinafter referred to as C1) for the sentence synthesized immediately before is obtained from the information stored in the global voice quality information storage unit 1403. In step 1702, the current voice quality information output from the voice quality extraction unit 105 is acquired. The whole voice quality information (hereinafter referred to as C0) for the read-out sentence ((quasi) real voice data) is acquired.

  In step 1703, position information P indicating where the segment for which the target cost is currently calculated is located in the reading sentence is calculated. This position P is used as information for evaluating to what position of the current sentence the global voice quality information of the (quasi) real voice data in the immediately preceding reading sentence affects. For the calculation method of the position P, for example, a value indicating the position in the phoneme sequence constituting the reading sentence can be used, and the position is located at the second from the head of the reading sentence. It is also possible to use a value indicating such from the prosodic information 1000. In addition, when a breath pose is included between the beginning and the position occupied by this segment in the read-out sentence, an implementation form in which a large value is added to the position P is also conceivable. When a breath pose appears, it is assumed that there will be a significant change (or reset) with respect to the voice quality, so that the above phenomenon can be taken into account by making such a non-linear change in the value of the position P as well. Become. In another embodiment, as the position P, a breathing phrase including the segment, or a value indicating what number the accent phrase is from the head can be used. It is conceivable that the influence of the global voice quality information changes step by step for each accent phrase or phrase that is a semantically united speech unit.

  In step 1704, based on the position information P, weight values for the whole area voice quality information C1 and C0 are calculated. These weight values are information for determining whether the influence of the (quasi) real voice data of the previous sentence or the (quasi) real voice data of the current sentence is strong. Calculated based on the value. Here, various methods for calculating the weight value can be considered, but the simplest calculation method is W1 = (1-P / Pmax), W0 = (1-W1). Here, Pmax is the position information P calculated for the segment corresponding to the last phoneme (syllable) constituting the reading sentence. In this calculation formula, the influence of the global voice quality information of the (quasi) real voice data of the immediately preceding sentence becomes larger as the segment located at the head of the reading sentence, and its influence decreases proportionally as the position P increases. That is, there is an effect that the influence of the entire voice quality information of the (quasi) real voice data of the current sentence becomes large. This formula is an example and is the simplest case. In a more practical embodiment, there may be a method of referring to a table in which the weight values W1 and W0 are set in advance for each value of the position P (or for each value width). As described above, it is difficult to think that the overall voice quality of the (quasi) real voice data of the immediately preceding sentence and the influence of this sentence change linearly. In order to simulate actual speech, it is desirable to set a weight table adjusted in advance.

  In Step 1705, the global voice quality information on the (quasi) real voice data side used in the distance calculation of the global voice quality information for this segment is corrected. Although the global voice quality information of the current sentence is C0, as described above, the influence of the global voice quality information of the immediately preceding sentence may be necessary depending on the position in the text to be read. Therefore, the global voice quality information used in the distance calculation is calculated by the equation C = W0 · C0 + W1 · C1.

  Finally, in step 1706, a distance is calculated between the calculated global voice quality information C and the global voice quality information of this segment (the configuration method will be described in the first embodiment), and this is calculated as a cost value for the global voice quality. And

  Through the above flow, it is possible to correct the voice quality of the synthesized speech for the current sentence by referring to the entire voice quality information of the immediately preceding sentence and the previous sentence stored in the global voice quality storage unit 1403 It becomes.

  According to the second embodiment, in the case where a plurality of sentences are continuously synthesized, the head of the synthesized speech of this time is matched with the overall voice quality of the (quasi) real voice data of the sentence synthesized immediately before. You can adjust the voice quality of the part. As a result, it is possible to generate synthesized speech in which the voice quality is smoothly connected.

  In the present embodiment, the configuration of the second embodiment is applied to describe an embodiment of a system in which read-out content is distributed from a server to a terminal such as a personal computer or an electronic book terminal, and the read-out content is read out by a speech synthesis system. To do.

  In this embodiment, for example, the whole data in which a (quasi) real voice data group is added to a text to be read out in an electronic book or the like is distributed. The speech synthesis process according to the present embodiment is performed while using. In addition to electronic books, when generating dialogue data (scenarios) on the server side in a voice dialogue interface on a smartphone, etc., the server side will be able to display the entire dialogue scenario or a part of the scenario generated according to the dialogue flow from the server side. It can also be used as a mechanism for distributing to the side.

  FIG. 18 is a diagram for explaining the entire speech synthesis system of the present embodiment. The server 1800 has a normal computer configuration, a storage unit 1801 in its storage unit, and a content transmission unit 1802 can be configured by a CPU that is a processing unit or a network interface.

  On the other hand, the terminal-side computer 120 includes a content receiving unit 1803, a content storage unit 1804, a content selection unit 1806, and an input generation unit 1805 in addition to the configuration of the second embodiment (FIG. 14). In addition to the content storage unit 1804 being a storage unit, these are function processing units that can be realized by the network interface described above and the program execution of the CPU that is the processing unit. On the other hand, the real voice voice storage unit, the quasi-voice voice generation unit, and the (semi) real voice selection unit are not included, and the input generation unit 1805 performs a part of these processes. In addition, the above-described device on the terminal side is connected to the content transmission unit 1802 and the content storage unit 1801 on the server side via the network 124.

In the following description, only the processing unit on the terminal side will be described.
The content receiving unit 1803 receives content data from the server via the network and stores it in the content storage unit 1804. The content data assumed here includes, in electronic books and voice conversation scenario data, in addition to the reading text, a (quasi) real voice data group, and a list indicating the correspondence between the read text and the (quasi) real voice data It is data. FIG. 19 shows content data 1900 as an example. (A) of FIG. 19 is a list of the read-out sentence itself, and corresponds to the text itself of the book content in the electronic book data. FIG. 19C is a list of (quasi) real voice data. As shown here, (quasi) real voice data is embedded in the content itself and distributed. FIG. 19B shows data indicating the correspondence between the read-out sentence and the (quasi) real voice data to be used there.

  FIG. 20 shows an example of content data 2000 in still another embodiment. In the content data 2000 of this form, a (quasi) real voice ID is designated with a special expression (* N *) in a reading sentence. In this way, by storing necessary data together in one content data, it is possible to change the way the synthesized speech is spoken according to the content.

  The content selection unit 1806 is a mechanism for selecting data that the user wants to use from among a plurality of content data 2000 stored in the content storage unit 1804. When implemented as an electronic book terminal, this is a mechanism (such as a touch panel) for selecting electronic book content that the user wants to read (want to hear) from among electronic books stored in the terminal. Further, in the case of interactive content used in the voice interactive interface, the content selection process may be a mechanism that is automatically determined by the voice interactive system instead of the user's own selection work.

  The content data thus selected is transferred to the input generation unit 1805. The processing of the input generation unit 1806 is a combination of the reading sentence generation unit 1402, the input analysis unit 106, and the (semi) real voice selection unit 103 in FIG. 14 of the second embodiment. When the content data as shown in FIG. 19 is used, the input generation unit 1806 takes out the reading sentences sequentially from the reading sentence list in the contents data shown in FIG. 19A, and corresponds to the sentence ID. The (quasi) real voice data (and accompanying information) is determined based on the correspondence table in FIG. 19 (b) and the voice data list in FIG. 19 (c). The voice data (accompanying information) is transferred to the prosody generation unit, prosody information extraction unit, voice quality information extraction unit, and connection synthesis unit.

  By using such a configuration, it becomes possible to read out the content with higher quality synthesized speech using (quasi) real voice data set for each content.

  While various embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the above-described embodiments have been described in detail for better understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  Further, each of the above-described configurations, functions, processing units, etc. has been described mainly with reference to an example of creating a program to be executed by a processing unit for realizing part or all of them. Needless to say, it may be realized by hardware, for example, by designing with an integrated circuit.

101 (quasi) real voice storage unit 102 quasi real voice generation unit 103 (quasi) real voice selection unit 104 prosodic information extraction unit 105 voice quality information extraction unit 106 input analysis unit 107 prosody generation unit 108 prosodic model 109 segment database (DB)
110 Voice quality-considered waveform synthesis unit 111 Connection synthesis unit 112 Input text
113 synthetic voice 114 (semi) voice voice ID
120 Computer 121 Central Processing Unit (CPU)
122 memory (MM)
123 interface (I / F)
124 network 125 hard disk controller (HDD)
126 Input / output unit (I / O)
127 Speaker 128 Bus 200 (Quasi) Real voice database 901, 903 Curve 902 Connection end point 1401 Reading scenario storage unit 1402 Reading sentence generation unit 1403 Global voice quality information storage unit 1800 Server 1801 Content storage unit 1802 Content transmission unit 1803 Content reception unit 1804 Content Storage unit 1805 Input generation unit 1806 Content selection unit

Claims (15)

A speech synthesis method in a processing unit for outputting synthesized speech,
The processor is
Select real voice data corresponding to the characteristic part of the utterance style from the input text, or quasi-real voice data (hereinafter referred to as (quasi) real voice data) according to prosody and voice quality,
Extract prosodic information and voice quality information from the selected (quasi) real voice data,
Based on the extracted prosodic information, adjust the prosody information of the synthesized speech corresponding to the input text, and considering the extracted voice quality information, generate synthesized speech waveform data corresponding to the input text,
A speech synthesis method comprising joining the generated synthesized speech waveform data and the selected (quasi) real voice data at a connection end point. The speech synthesis method according to claim 1,
The processor is
A speech synthesizing method for determining the (quasi) real voice data to be selected by analyzing the input text in order to select the (quasi) real voice data. The speech synthesis method according to claim 1,
The processor is
In order to extract the voice quality information, for the (quasi) real voice data, short time voice quality information corresponding to a short interval at both ends thereof, and global voice quality information corresponding to the whole (quasi) real voice data A speech synthesis method characterized by extracting. The speech synthesis method according to claim 3,
The processor is
A speech synthesis method using frequency domain coefficient information obtained by a frequency analysis method for the entire (quasi) real voice data as the global voice quality information. The speech synthesis method according to claim 3,
The processor is
Using the time and weight information of voice quality feature points previously added to the (quasi) real voice data as the global voice quality information, information obtained by weighting and adding short-time voice quality information at each time position with a specified weight A speech synthesis method characterized by being used. The speech synthesis method according to claim 3,
The processor is
As the global voice quality information, global voice quality information or short-time voice quality information at a specific position is extracted in k speech sections obtained by dividing the (quasi) real voice data into k parts, and from the extracted k voice quality information. A speech synthesis method using N pieces of representative voice quality information obtained by changing the process of determining representative voice quality information from k = 1 to N. The speech synthesis method according to claim 3,
The processor is
When the synthesized voice waveform data is generated in consideration of the extracted voice quality information in consideration of the extracted global voice quality information, the waveform synthesis processing is also performed in consideration of the last several global voice quality information that has been saved. A speech synthesis method characterized by: The speech synthesis method according to claim 2,
The processor is
In order to extract the voice quality information, the global voice quality information corresponding to the entire (quasi) real voice data is extracted, and the extracted global voice quality information is stored,
A speech synthesis method, wherein, when generating the synthesized speech waveform data in consideration of the extracted voice quality information, a waveform synthesis process is performed in consideration of the last several pieces of global voice quality information stored. A speech synthesizer,
A processing unit and a storage unit;
The processor is
Select (semi) voice data from the input text corresponding to the characteristic part of the utterance style,
Extract prosodic information and voice quality information from the selected (quasi) real voice data,
Based on the extracted prosodic information, adjust the prosody information of the synthesized speech corresponding to the input text, and considering the extracted voice quality information, generate synthesized speech waveform data corresponding to the input text,
A speech synthesizer characterized in that the generated synthesized speech waveform data and the selected (quasi) real voice data are joined at a connection end point and output. The speech synthesizer according to claim 9,
The processor is
In order to select the (quasi) real voice data, the speech synthesizer determines the (quasi) real voice data to be selected by analyzing the input text. The speech synthesizer according to claim 9,
The processor is
In order to extract the voice quality information, for the (quasi) real voice data, short time voice quality information corresponding to a short interval at both ends thereof, and global voice quality information corresponding to the whole (quasi) real voice data A speech synthesizer characterized by extracting. The speech synthesizer according to claim 11,
The processor is
Storing the extracted global voice quality information in the storage unit;
When generating the synthesized speech waveform data in consideration of the extracted voice quality information, the waveform synthesis processing is performed in consideration of the most recent whole area voice quality information stored in the storage unit. A speech synthesizer. The speech synthesizer according to claim 10,
The processor is
In order to extract the voice quality information, global voice quality information corresponding to the entire (quasi) real voice data is extracted,
Storing the extracted global voice quality information in the storage unit;
A speech synthesizer characterized in that, when generating the synthesized speech waveform data in consideration of the extracted voice quality information, waveform synthesis processing is performed in consideration of the last several pieces of global voice quality information stored. A speech synthesis program executed by the processing unit of the speech synthesizer,
The processing unit is
Select (semi) voice data from the input text corresponding to the characteristic part of the utterance style,
Extract prosodic information and voice quality information from the selected (quasi) real voice data,
Based on the extracted prosodic information, adjust the prosody information of the synthesized speech corresponding to the input text, and considering the extracted voice quality information, generate synthesized speech waveform data corresponding to the input text,
A speech synthesis program that operates to join the generated synthesized speech waveform data and the selected (quasi) real speech data at a connection end point. The speech synthesis program according to claim 14,
The processing unit is
A speech synthesis program which operates to determine the (quasi) real voice data to be selected by analyzing the input text in order to select the (quasi) real voice data.

Images