JP2003208188A - Japanese text-to-speech synthesis method - Google Patents

Abstract

(57) [Summary] An object of the present invention is to provide a new Japanese text synthesis method using not only phoneme units but also diphone units having a boundary at the center of a phoneme. In addition, the present invention provides a method of synthesizing Japanese text that can synthesize a more natural sound and more effectively use a corpus than a conventional method using only phoneme units. The purpose is to provide. SOLUTION: In the Japanese text speech synthesis method,
In the waveform connection in a chain of vowels and vowels, a speech synthesis unit is selected in consideration of both the connection at the boundary and the connection at the center of the vowel.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a text-to-speech synthesis method capable of reading Japanese text information in synthetic speech.

[0002]

2. Description of the Related Art [1] Description of conventional Japanese text-to-speech synthesizer

FIG. 1 shows a schematic configuration of a conventional Japanese text-to-speech synthesizer.

The input Japanese text is subjected to morphological analysis and dependency analysis in the language processing unit 1, and phoneme symbols,
Converted to accent marks, etc.

The prosody pattern generation unit 2 uses the phoneme symbol, the accent symbol string, and the part-of-speech information of the input text obtained from the morpheme analysis result to determine the phoneme duration (voice length), the fundamental frequency (voice pitch F). ₀ ), the power of the vowel center (loudness of voice) and the like are estimated.

The phoneme unit selector 3 is closest to the estimated phoneme duration, fundamental frequency, vowel center power, etc.
In addition, the combination of the synthesis units that minimizes the distortion when the synthesis units (phoneme pieces) stored in the waveform dictionary 5 are connected is selected by using the dynamic programming method.

In the phoneme waveform generator 4, a voice is generated by connecting the phonemes while converting the pitch according to the selected combination of the phonemes.

By the way, in the unit selection performed by the phoneme unit selection unit 3, it is important to use a scale (cost) that matches the perceptual characteristics (see Reference 1).

Reference 1: E. Klabbers and R. Veldhuis,
"Reducing audible spectral discontinuities", IEEE
Trans. Speech and Audio Processing, vol. 9, no.
1, pp. 39-51, 2001.

Therefore, it is necessary to perform the mapping from the observable feature quantity (physical quantity, language information) to the psychological quantity by the cost function. FIG. 2 shows a conceptual diagram of the cost function.

Since the cost function needs to reflect the auditory impression, it should be estimated based on perceptual experiments.
When the space of the explanatory variables is relatively simple like the fundamental frequency F ₀ and the phoneme duration Duration, it is possible to estimate the mapping from the physical quantity to the psychological quantity by the perceptual experiment. However, in a complex space where higher-dimensional feature quantities are used as explanatory variables, estimation by perceptual experiments is difficult to realize. Also, many studies have been conducted on physical quantities that directly reflect psychological quantities as direct expressions of psychological quantities, but physical quantities with sufficient accuracy have not yet been found (see References 2 and 3).

Reference 2: Y. Stylianou and AK Syrdal,
"Perceptual and objective detection of discontinu
ities in concatenative speech synthesis ", Proc. IC
ASSP, pp. 837-840, Salt Lake City, USA, May. 200
1. Reference 3: M. Tsuzaki, "Feature extraction by auditor
y modeling for unitselection in concatenative spee
ch synthesis ", Proc. EUROSPEECH, pp. 2223-2226, Aa
lborg, Denmark, Sep. 2001.

As for the mapping from linguistic information to psychological information, it is comparatively easy to estimate it by a perception experiment. By using this mapping, it may be possible to capture perceptual features that cannot be sufficiently expressed by physical quantities. However, since only categorical features can be expressed only by linguistic information, and samples within the same category cannot be compared, it is necessary to use them together with physical quantities that can express the features of each sample.

In this example, the physical quantity and the result of the perceptual experiment are directly used.
Use the cost function Wc using the psychological amount
It The cost function Wc has five sub-cost functions shown in Table 1.
Composed of numbers. C_proAnd C_FoIs based on sound source information
Sub-cost function _typ, C_envAnd C
_specIs a sub-cost function based on vocal tract information.

[0015]

[Table 1]

That is, as shown in FIG. 3, u _i-1 ,
Let u _i and u _{i + 1 be} phonemes of unit candidates, and t _i−1 , t _i ,
Assuming that t _{i + 1} is the actual environment (target) to be used,
The sub-cost function for u _i is C _pro (u _i ,
t _i ), C _typ (u _i , t _i ), C _{en v} (u _i ,
u _i-1 ), C _spec (u _i , u _i-1 ) and C _Fo (u _i ,
u _i-1 ).

In FIG. 3, u _i-1 and u _i are not continuous in the corpus, and mean one unit candidate for the i-1 and i-th targets t _i-1 and t _i . .

C _pro (u _i , t _i ) and C
_typ (u _i , t _i ) represents the distortion between the unit candidate (u _i ) extracted for the i-th phoneme and the environment (target t _i ) that is actually used. Also, C _env (u
_i , u _i-1 ), C _spec (u _i , u _i-1 ) and C _Fo (u
_i , u _i−1 ) is the i-th unit candidate (u _i ) and i−1
It represents the distortion that occurs when the th unit candidate (u _i-1 ) is connected.

When the preceding unit candidate is u _i-1 , the cost function WC (u _i , t _i ) of the unit candidate u _i for the phoneme target t _i to be synthesized is expressed by the following equation (1).

[0020]

[Equation 1]

Where C _pro , C _F0 , C _env , C _spec ,
C _typ is a sub-cost function, which will be described in detail below.
Each sub-cost is normalized to have a substantially equal dynamic range.

W _pro , w _F0 , w _env , w _spec , and w _typ represent weights for the respective sub-costs, and their sum is 1 as shown in the following equation (2). In this example, equal weights (0.2) are used for all sub-costs.

[0023]

[Equation 2]

The phoneme unit selection unit 3 selects a combination of unit candidates having the minimum total cost TC expressed by the following equation (3) by using the dynamic programming method, targeting each phoneme in the input sentence. To do.

[0025]

[Equation 3]

Here, N represents the number of target phonemes in the sentence. WC (u _i, t _i ) indicates the cost of the unit candidate u _i for the target t _i to be combined.

Each sub-cost function will be described below.

(1) Subcost function for prosody: C
_{The pro} sub-cost function C _pro (u _i, t _i ) indicates deterioration of naturalness caused by a difference in prosody (F ₀ locus, phoneme duration) between the unit candidate u _i and the target t _i .
It is expressed by the following equation (4).

[0029]

[Equation 4]

Where D_F0(u_i,t_i,m) is a unit candidate
u_iAnd target t_iIn the m-th division section in
KE logF₀Shows the difference in the average value of. Also, M is
The number of phoneme divisions is shown. Also, D_d(u_i,t_i) Is simple
Candidate u_iAnd target t_iDifference in phoneme duration in
Is shown. Also, P is D_F0(u_i,t_i,m) and D _d
(u_i,t_i) And are non-linear functions with variables.

In this example, it is assumed that the prosody transformation is performed at the time of synthesis, and the function P is determined from the result of the perceptual experiment on the deterioration of naturalness caused by the prosody transformation. When prosody transformation is not performed, it is necessary to perform a perceptual experiment on naturalness deterioration caused by using a prosody different from the target, and determine the function P from the result.

(2) Sub-cost function for F ₀ discontinuity: C _F0 sub-cost function CF ₀ (u _i, u _i-1 ) depends on the discontinuity of F _{0 at} the connection boundary between u _i-1 and u _i. It shows the deterioration of naturalness that occurs and is expressed by the following equation (5).

[0033]

[Equation 5]

Here, D _F0 (u _i, u _i-1 ) is u _i-1 and u
_The distance based on the difference of LogF _{0 at} the connection boundary with _i is shown. u _i-1 and u _i are continuous in the corpus,
This sub-cost is 0 between unit candidates in which no connection occurs.

(3) Sub-cost function relating to phoneme environment substitution: C _{env The} sub-cost function C _env (u _i, u _i-1 ) represents the deterioration of naturalness caused by the substitution of the phoneme environment between the unit candidate and the target. , Determined from the results of perception experiments (Reference 4
reference). This sub-cost function C _env (u _i, u _i-1 ) is expressed by the following equation (6).

Reference 4: Tsune Kawai, Minoru Tsuzaki, Takeshi Masuda, Hidenori Iwasawa, "Perceptual Evaluation of Natural Deterioration by Substituting Phoneme Environments when Connecting Waveform Elements", IEEJ, SP2001-22, pp. 51- 57,
May, 2001.

[0037]

[Equation 6]

Where S_s(u_i,E_s(u_i-1), t_i)
Indicates the sub-cost function generated by the substitution of the subsequent environment,
S_p(u_i,E_P(u_i), t_i-1) Is due to the substitution of the preceding environment
The resulting sub-cost function is shown. E_s(u_i-1) Is the corpus
The following phonemes in _P (u_i) Indicates the preceding phoneme
is doing.

For example, "a" of "ae" (u _i-1 = / a
/, E _s (u _i-1 ) ＝ / e /), and “o” (u
_{When i} = / o / and E _p (u _i ) = / N /) are connected and "ao" (t _i-1 = / a /, t _i = / o /) is combined, The function C _env (u _i, u _i-1 ) is expressed by the following equation (7).
It is represented by.

[0040]

[Equation 7]

Even if the environments match, S _s and S _p do not always become 0. In that case, the cost value is (u _i,
u _i-1 ) represents the difficulty of connection due to ambiguity in labeling, etc. Note that u _i−1 and u _i are continuous in the corpus, and the sub cost is 0 between unit candidates in which no connection occurs.

(4) Sub-cost function relating to spectral discontinuity: C _{spec The} sub-cost function C _spec (u _i, u _i-1 ) is a natural result of spectral discontinuity at the connection boundary between u _i-1 and u _i. It represents the deterioration of the sex and is expressed by the following equation (8).

[0043]

[Equation 8]

Where h (f) is a triangular window of length ω.
is doing. MCD (u_i, U_i-1, F) are unit candidates u
_i-1The f-th frame from the end and the unit candidate u_i
From the beginning of the frame to the f-th frame
Shows tram distortion. u _i-1And u_iAnd in the corpus
Between the unit candidates that are connected in
, The sub-cost function becomes 0.

(5) Subcost function related to phoneme suitability
Number: C_typ Sub-cost function C_typ(u_i,t_i) Indicates the phoneme suitability
And unit candidate u _iAnd target t_iBetween
Shows the natural degradation caused by the average spectral difference
is doing. This sub-cost function C_typ(u_i,t_i) Is next
It is expressed by equation (9).

[0046]

[Equation 9]

Here, Cen represents a logarithmic spectrum centroid. Further, MCD shows mel cepstrum distortion between the centroids Cen (u _i) and the centroid Cen target t _i (t _i) of the unit candidate u _i. For the calculation of the centroid Cen (t _i ) of the target t _i, the mel cepstrum output by the speech synthesis method by HMM (see Reference 5) is used.

Reference 5: Takakatsu Yoshimura, Keiichi Tokuda, Takashi Mashiko,
Takao Kobayashi, Tadashi Kitamura, "Simultaneous Modeling of Spectrum, Pitch, and Duration in HMM-based Speech Synthesis", Theory of Communication
(D-II), vol. J83-D-II, no. 11, pp. 2099-2107, 200
0.

[2] Description of various conventional unit selection methods

In recent years, text-to-speech synthesis (TTS: Text-to-Sp
A corpus-based device is mainly used as an eech) device, and it is becoming possible to obtain high-quality synthesized speech by connecting speech unit waveforms to synthesize speech. However, its quality is not good enough, and there are many points left to be improved even if it is limited to the composition of reading-aloud sentences. The present inventors aim to construct a higher quality Japanese text-to-speech synthesizer for synthesizing read-aloud sentences.

In Japanese, if vowel devoicing is excluded, CV
Since a syllable is composed of (C: consonant, V: vowel) and V, the Japanese text-to-speech synthesizer can efficiently construct a speech corpus by considering CV as a synthesis unit. . Further, since the transition from C to V is considered to be important for the perception of phonology, the CV unit is suitable as the synthesis unit in the Japanese text-to-speech synthesizer.

However, when synthesizing speech using the CV unit, the connection from V to V often causes discontinuity. This is because the transition from V to V is a section in which the formants make a smooth transition, and if discontinuity of the formants occurs due to the connection, the naturalness is greatly deteriorated. Therefore, realization of smooth connection between V and V is an important issue.

Longer units have been proposed in order to avoid connecting V-V composite units (see documents 6, 7 and 8).

Reference 6: H. Kawai, N. Higuchi, T. Shimi
zu and S. Yamamoto, "Developmentof a text-to-speec
h system for Japanese based on waveform splicing ",
Proc. ICASSP, pp. 569-572, Adelaide, Australia, A
pr. 1994. Reference 7: S. Takano, K. Tanaka, H. Mizuno, M. Abe an
d S. Nakajima, "A Japanese TTS system based on mul
tiform units and a speech modification algorithm w
ith harmonics reconstruction ", IEEE Trans. Speech
and Audio Processing, vol. 9, no. 1, pp. 3-10, 200
1. Reference 8: N. Iwahashi, N. Kaiki and Y. Sagisaka, "Sp.
eech segment selection for concatenative synthesis
based on spectral distortion minimization ", IEICE
Trans. Fundamentals, vol. E76-A, no. 11, pp. 1942
-1948, 1993.

As shown in Document 6, the CV unit proposed by Kawai et al. Regards a vowel chain following a CV having a high frequency of appearance as one unit, and is an extension of the CV unit.

As another unit, there is a variable length unit proposed by Iwahashi et al. As shown in Reference 8. In this method, a cost function for a phoneme sequence to be synthesized is determined, and an optimum unit is selected from the speech corpus according to a criterion that minimizes the sum. By performing a search by dynamic programming based on phoneme units, it is possible to select units consisting of phoneme chains of various lengths (see Reference 9).

Reference 9: A Black and N. Campbell, "Opti
mising selection of units from speech databese for
concatenative synthesis ", Proc. EUROSPEECH, pp. 5
81-584, Madrid, Spain, Sept. 1995.

Therefore, if there is a vowel chain to be synthesized in the corpus and it is selected as a unit, connection of the synthesis unit between V and V can be avoided. However, since there are an unlimited number of vowel chain types in Japanese, it is impossible to construct a corpus containing all vowel chains.

Also, prosodic coverage (see reference 10)
Considering all the above, the amount of corpus becomes even larger. Therefore, it can be said that there is an unavoidable problem of connection of composition units between V and V.

Reference 10: Tsune Kawai, Nobuo Higuchi, Seiichi Yamamoto,
"Creation of Waveform Fragment Dataset for Speech Synthesis Considering Fundamental Frequency and Phoneme Connection Time", IEICE (D-II), vol.
J82-D-II, no. 8, pp. 1229-1238, 1999.

In the vowel chain, the formant transition near the vowel center is more stable than the formant transition at the vowel boundary. Therefore, when the connection is made at the vowel center, the discontinuity can be reduced in many cases as compared with the case where the connection is made at the vowel boundary. This has been clarified in a preliminary experiment conducted by the present inventors. The VCV unit shown in Reference 11 is a synthetic unit devised based on this idea.

Reference 11: Yamato Sato, "Speech synthesis method using PAECOR-VCV concatenation", IEICE (D), vol. J61-D, no. 11,
pp. 858-865, 1978.

However, there is a possibility that there is a synthesis unit that allows smooth connection even at the vowel boundary, so it is not always better to connect only at the vowel center. Therefore, in the vowel chain, both the unit for connecting at the vowel center and the unit for connecting at the vowel boundary should be considered.

[0064]

The present invention has been made based on the above consideration, and a new Japanese text synthesis using not only a phoneme unit but also a diphone unit with the center of the phoneme as a boundary. The purpose is to provide a method.

Further, according to the present invention, more natural speech can be synthesized and the corpus can be used more effectively as compared with the conventional method using only the phoneme unit. It is an object to provide a text synthesizing method.

[0066]

According to a first aspect of the present invention, in a Japanese text-to-speech synthesis method, in the waveform connection in the chain of vowels and vowels, the connection at their boundaries and their vowel centers are used. It is characterized in that the voice synthesis unit is selected in consideration of both the connection and the.

According to the invention described in claim 2, in the invention described in claim 1, in the waveform connection in the chain of the vowel and the semivowel, the connection at the boundary between them and the connection at the center of the vowel before the semivowel. It is characterized in that the voice synthesis unit is selected in consideration of both of the above.

According to a third aspect of the invention, in the invention according to the second aspect, in the waveform connection in the chain of vowels and nasal sounds, the connection at the boundary between them and the connection at the center of the vowel before the nasal sound. It is characterized in that the voice synthesis unit is selected in consideration of both of the above.

[0069]

BEST MODE FOR CARRYING OUT THE INVENTION Referring to FIGS.
An embodiment of the present invention will be described. First, after considering the connection in the vowel center, the unit selection method using the phoneme unit and the diphone unit in the Japanese text-to-speech method, which is a feature of the present invention, will be described.

[1] Consideration on connection at vowel center

When the waveform connection is made at the vowel boundary in the vowel chain, a larger discontinuity of formant transition appears as compared with the case where the waveform connection is made at the vowel center in the vowel chain. This is because the waveform connection at the vowel center is more stable in spectrum than the waveform connection at the vowel boundary, and statistically the static feature has a small variance and a small change. To be

From this, it is expected that in the vowel chain, better connection can be achieved by performing the waveform connection at the vowel center than at the vowel boundary.
This conjecture is verified by conducting experiments using the mel-cepstral distortion as an objective evaluation scale.

[1-1] Experimental method

The mel-cepstral distortion around the connection boundary when the waveform connection is made at the vowel boundary and when the waveform connection is made at the vowel center will be examined.

Each connection method in the vowel chain is shown in FIG. FIG. 4A shows a waveform connection method at a vowel boundary.
(B) shows the waveform connection method at the vowel center.

FIG. 4 shows a case where the vowel V1 and the vowel V2 are connected. In FIG. 4, V ^* indicates all vowels. V1 _fh indicates the first half of the vowel V1, and V1 _lh indicates the second half of the vowel V1. Figure 4 (a)
Then, V1 of Unit1 and V2 of Unit2 are connected. In FIG. 4B, V1 _{fh of} Unit1 and
It is connected to (V1 _lh + V2) of Unit2.

ATR labeled by hand in the Japanese speech corpus uttered by one Japanese male speaker
450 sentences (about 30 minutes) in the phoneme balance sentence are used. Connections are made at the phoneme boundaries and the phoneme centers in all vowel chains existing in the corpus, and the distribution of the weighted sum of mel-cepstrum distortions (represented by the above expression (8), referred to as connection distortions) in each case is obtained. In the above equation (8), the number of frames ω for calculating the mel cepstrum is 4 and the frame shift is 5 ms. The mel cepstrum distortion in each frame is calculated using the following equation (10).

[0078]

[Equation 10]

Here, mc _i ^(p) represents the p-dimensional mel-cepstral coefficient obtained from the F ₀ adaptive smoothing spectrum (STRAIGHT spectrum) (see Reference 12) in the frame with the following vowel. Similarly, mc _i-1 ^(p) represents the mel cepstrum coefficient in the preceding vowel. The sampling frequency is 16 kHz.

Reference 12: H. Kawahara, I. Masuda-Kats
use and A.de Cheveign ₀ , "Restructuring speech repr
esentations using a pitch-adaptive time-frequency
smoothing and an instantaneous-frequency-based F ₀
extraction: possible roleof a repetitive structure
in sounds ", Speech Communication, vol. 27, no.3-
4, pp. 187-207, 1999.

In the following description, sound repellency / N /
Will also be regarded as a vowel.

[1-2] When vowels having different phoneme environments are included

The connection in the vowel chain when the phoneme environments include vowels different from each other will be examined. As the preceding vowel,
All vowels in the succeeding environment are used in the corpus (including the case where V ^* ≠ V2 in FIG. 4).

The frequency distribution of connection distortion is shown in FIG. It can be seen from FIG. 5 that the connection distortion at the vowel center (Vowelceter; solid line) can reduce the connection distortion in many cases compared to the connection at the vowel boundary (Vowel boundary; broken line). In addition, Av. Represents the average value and Sd. Indicates the standard deviation.

In selecting a unit, it is important to find a unit that reduces not only the discontinuity of the spectrum but also the sum of all sub-costs. Therefore, if the distribution is closer to the smaller connection distortion, it means that the number of unit candidates that can possibly reduce the cost is increased. From this, it can be seen that when the phoneme environment is different, the connection with the vowel center can make the connection with a smaller spectrum discontinuity, and more good unit candidates exist. .

[1-3] When vowels having the same phoneme environment are used

Next, the case where vowels having the same phoneme environment are used will be examined. This corresponds to the case of V ^* = V2 in FIG. The number of unit candidates in the corpus in this case is about 1/3 as compared with the case of the previous experiment (including the case of V ^* ≠ V2).

FIG. 6 shows the frequency distribution of connection distortion when vowels having the same phoneme environment are used. As can be seen from FIG. 6, the connection distortion frequency distribution is almost the same for both the connection at the vowel center (Vowel ceter; solid line) and the connection at the vowel boundary (Vowel boundary; broken line).

Therefore, when the phoneme environments are the same, there is no great difference between the connection at the vowel center and the connection at the vowel boundary.

Therefore, in each unit candidate, the connection at the vowel boundary and the connection at the center of the vowel are compared, and it is considered to use the connection that can further reduce the connection distortion. The frequency distribution of connection distortion at this time, that is, the frequency distribution of connection distortion when one of the connection at the vowel boundary and the connection at the center of the vowel has the smaller connection distortion is shown in FIG.
Is indicated by a chain line (Vowel ceter & Vowel boundary). From this frequency distribution, it can be seen that in this case, the connection distortion can be made smaller than in the case where only one of the connection at the vowel boundary and the connection at the vowel center is used.

This is because the number of unit candidates is increased by considering both the connection at the vowel boundary and the connection at the vowel center, and it is possible to use the unit candidate which can make the connection with smaller spectral discontinuity. It means that

From the above, it can be said that in the vowel chain, a better unit selection can be performed by using both the connection at the vowel center and the connection at the vowel boundary.

[2] Description of Japanese text-to-speech method according to the present invention

In the Japanese text-to-speech synthesis method according to the present invention, in the vowel chain (VV chain), connection at the vowel center (unit selection using diphone units),
It is characterized in that suitable unit selection is performed in combination with connection at the vowel boundary (conventional unit selection using phoneme units). It should be noted that in the other chains, the conventional unit selection using the phoneme unit is used.

Further, in the present embodiment, the connection by the diphone unit is considered not only for the V-V chain, but also for the V-semi-vowel chain and the V-nasal sound chain. Regarding the V-semi-vowel chain and the V-nasal sound chain, the connection at the center of the vowel before the half-vowel and the nasal sound is taken into consideration when the connection by the diphone unit is considered. Also, in order to prevent an increase in the number of connection points, a unit consisting of half vowels is not used.
However, by considering silence as one phoneme, it is an exception when the preceding or following half vowel is silent.

[2-1] Unit selection method using phoneme unit and diphone unit

The sub-cost function used in the unit selection using the voice unit has already been described in the section of the description of the prior art, so the description thereof will be omitted here.

The sub-cost function used in unit selection using the diphone unit will be described below.

With regard to vowel unit candidates for which connection in the vowel center is considered, unit selection is performed by dividing into first and second half phonemes in order to consider diphone units. Here, a semiphoneme is assumed to have half the duration of the original phoneme. The calculation of vowel cost considering the diphone unit is based on the phoneme unit, and a combination of the first half phoneme unit candidate and the second half semiphoneme unit candidate is regarded as one phoneme.

The first half-phoneme unit candidate is u _i ^f (before dividing into half-phonemes, u _1i ), and the second half-phoneme unit candidate is u _i.
Each sub-cost for a target t _i (t _i ^{f for} the first half and t _i ^l for the second half) is calculated as follows, where ^l (before splitting into _semiphonemes is u _2i ).

(1) Subcost function C _{pro for} prosody
The (u _i , t _i ) sub-cost function C _pro (u _i , t _i ) is expressed by the following equation (11),
As shown in (12), the calculation is performed for each semiphoneme, and the weighting is performed according to the connection time dur of each semiphoneme.

[0102]

[Equation 11]

[0103]

[Equation 12]

Here, C _pro (u _i ^f , t _i ^f ), C
_The number of phoneme divisions in _pro (u _i ^l , t _i ^l ) is M / 2.

(2) Sub cost function C _F0 (u _i , u _i-1 ) for F ₀ discontinuity The sub cost function C _F0 (u _i , u _i-1 ) is a phoneme as shown by the following equation (13). It is calculated as the sum of the subcosts at the boundaries (u _i-1 and u _i ^f ) and the phoneme centers (u _i ^f and u _i ^l ).

[0106]

[Equation 13]

(3) Sub-cost function C _env (u _i , u _i-1 ) related to phoneme environment substitution The sub _- cost function C _env (u _i , u _i-1 ) is expressed by the following equation (14).
Indicated by. The preceding phoneme environment and the following phoneme environment of the semiphoneme are equal to the phoneme environment for the phoneme before being divided into the semiphonemes. However, the cost functions S _s ^d and S _p ^d in the phoneme center are different from those between phonemes.

[0108]

[Equation 14]

(4) Sub-cost function C _spec (u _i , u _i-1 ) relating to spectral discontinuity The sub _- cost function C _spec (u _i , u _i-1 ) is expressed by the following equation (15).
It is calculated as the sum of the subcosts at the phoneme boundaries (u _i-1 and u _i ^f ) and the phoneme centers (u _i ^f and u _i ^l ), as shown in.

[0110]

[Equation 15]

(5) Sub-cost function C _typ (u _i , t _i ) relating to phoneme suitability The sub-cost function C _typ (u _i , t _i ) is calculated for each semiphoneme as shown by the following equation (16). Is performed, and the weighting is performed according to the connection time dur of each semiphoneme, thereby calculating.

[0112]

[Equation 16]

Here, ω _f and ω _l are equal to the above equation (12).

Target t _i considering diphone units
FIG. 7 shows targets and unit candidates used for the calculation of each sub-cost in the cost calculation of the unit candidates u _i ^f and u _i ^l for. Here, u _i-1 , u _i ^f , and u _i ^l are not continuous in the corpus, and the i−1 th,
It means one unit candidate for the i-th first half target and the i-th second half target.

When the diphone unit is used (u
_i-1, U_i ^fAre consecutive in the corpus), C
_env(U_i ^f, U_i-1), C_spec(U_i ^f, U_i-1),
C_F0(U _i ^f, U_i-1) Becomes 0. Also, the phoneme unit is
When used (u_i ^f, U_i ^lAre continuously in the corpus
C)_env(U_i ^l, U_i ^f), C
_spec(U _i ^l, U_i ^f), C_F0(U_i ^l, U_i ^f) Is 0
Becomes

In consideration of the above, the combination of unit candidates that minimizes the total cost TC expressed by the above equation (3) is selected by using the dynamic programming method. For vowels that consider diphone units, the number of candidates in the first half is F
Assuming that the number of candidates in the latter half is L, F × L paths are calculated, and as a result, L paths remain.

FIG. 8 shows an example of unit selection using a phoneme unit and a diphone unit.

In this example, the input sentence is easy (/ ts [C] u [V] i [V] y [C] a
[V] S [C] /). In addition, y is a half vowel.

In FIG. 8, "<V ^* ]" (V ^* represents all vowels) indicates the latter half of the vowel, and "[V ^* >"
Indicates the first half of the vowel.

In this example, / ts-u /, / ui / and / iy /
In each chain, unit selection is performed considering not only phoneme units but also diphone units. Other chains / ya /
In / as /, unit selection is performed only by phoneme unit. As a result of unit selection, a variable length unit is selected that is allowed to be connected not only at the phoneme boundary but also at the vowel center.

[3] Evaluation experiment

In order to evaluate the performance of the Japanese text-to-speech synthesis method (hereinafter referred to as the proposed method) according to the above-described embodiment, a comparison with the conventional method based on the phoneme unit is performed.

[3-1] Subjective evaluation experiment

[3-1-1] Experimental conditions

The same speech corpus as used in the experimental method of [1-1] above (in the Japanese speech corpus uttered by one Japanese male speaker, of the manually labeled ATR phoneme balance sentence) The experiment was performed using 450 sentences (about 30 minutes). Not only labeling but F
_{0 is} also manually corrected.

A pair comparison experiment was conducted using synthetic speech of 10 Japanese sentences. These sentences are not included in the corpus used for unit selection. Using the proposed method and the conventional method, we select units and synthesize speech. In order to evaluate only the performance of the unit selection method, unit selection was performed using the natural prosody information extracted from the original speech and the mel cepstrum sequence.

In speech synthesis, a prosody (F ₀
The locus, phoneme duration, and power) were controlled. The subjects are 10 Japanese adult men and women. In each trial, synthetic speech by the proposed method and synthetic speech by the conventional method were presented as a pair in a random order, and the subject was asked to select a synthetic speech that felt more unnatural. At that time, it is allowed to listen to the synthesized voice not only once but also as many times as necessary.

[3-1-2] Experimental Results

The synthesized 10 sentences are composed of 366 phonemes, and in the proposed method, 145 phoneme boundary connections (VC connection: 125, VV connection: 6, V-semi-vowel connection: 3, V).
-Nasal connection: 11) and 25 vowel-centered connections are made. In the conventional method, 163 phoneme boundary connections (VC connection:
124, V-V connection: 16, V-semi-vowel connection: 3, V-
Nasal connection: 20).

The results of the paired comparison experiment are shown in FIG. In FIG. 9, the ratio of the naturally-perceived one of the synthesized speech by the proposed method and the synthesized speech by the conventional method is shown as a Preference score. It is clear from FIG. 9 that the proposed method can synthesize speech more naturally than the conventional method.

[3-2] Objective evaluation experiment

[3-2-1] Experimental conditions

Since the proposed method can increase the number of unit candidates by considering the diphone unit as well, it is expected that the corpus can be used more effectively. Therefore, we calculated the corpus reduction rate of the proposed method over the conventional method. The corpus reduction rate is expressed as a value obtained by dividing the difference between the corpus sizes of the conventional method and the proposed method (corpus reduction amount) when the average cost values are equal by the corpus size of the conventional method at that time. The corpus used for unit selection is about 8 hours (about 10,000 sentences)
2 ^{-n / 2} (n = 0,1,2, ..., 9)
It has been reduced to. 53 sentences not included in this voice corpus are used as the evaluation sentences.

[3-2-2] Experimental results

FIG. 10 shows the relationship between the corpus size and the corpus reduction rate. It can be seen from FIG. 10 that the proposed method can reduce the corpus as compared with the conventional method, and it can be said that the corpus can be used more effectively. It can be seen that the result becomes even larger as the corpus becomes larger.

[0136]

According to the present invention, it is possible to obtain a new Japanese text synthesizing method using not only a phoneme unit but also a diphone unit having the center of a phoneme as a boundary.

Further, according to the present invention, compared to the conventional method using only the phoneme unit, more natural speech can be synthesized and the corpus can be used more effectively. .

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a conventional Japanese text-to-speech synthesizer.

FIG. 2 is a schematic diagram showing the concept of a cost function.

[3] In the cost calculation unit candidate u _i for phoneme targets t _i, is a schematic diagram showing a target and a unit candidate to be used in the calculation of each sub-costs.

FIG. 4 is a schematic diagram showing a waveform connection method at a vowel boundary and a waveform connection method at a vowel center.

FIG. 5 is a graph showing a frequency distribution of connection distortion when vowels having different phoneme environments are included.

FIG. 6 is a graph showing a frequency distribution of connection distortion when vowels having the same phoneme environment are used.

FIG. 7 is a schematic diagram showing targets and unit candidates used for calculation of each sub-cost in cost calculation of unit candidates u _i ^f and u _i ^l for a target t _i considering a diphone unit.

FIG. 8 is a schematic diagram showing an example of unit selection using a phoneme unit and a diphone unit.

FIG. 9 is a graph showing the results of a subjective evaluation experiment.

FIG. 10 is a graph showing the results of an objective evaluation experiment.

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Minoru Tsuzaki             2-2 Kodai, Seika-cho, Soraku-gun, Kyoto             International Telecommunications Basic Technology Research Institute Co., Ltd. F-term (reference) 5D045 AB02 AB30

Claims (3)

[Claims] 1. A Japanese text-to-speech method,
In the waveform connection in the chain of vowels and vowels, the speech synthesis unit is selected in consideration of both the connection at the boundary and the connection at the vowel center. Text-to-speech method. 2. In a waveform connection in a chain of vowels and semivowels, a speech synthesis unit is selected in consideration of both the connection at the boundary between them and the connection at the center of the vowel before the half vowel. The Japanese text-to-speech synthesis method according to claim 1, wherein 3. In a waveform connection in a chain of vowels and nasal sounds, a voice synthesis unit is selected in consideration of both the connection at the boundary between them and the connection at the center of the vowel before the nasal sound. The Japanese text-to-speech synthesis method according to claim 2, wherein