TWI858505B - Voice data generation method and system and computer program product - Google Patents

Abstract

A method for generating voice data implemented by a voice data generating system comprises: (A) calculating the number of characters corresponding to a sentence text portion according to a sentence text portion contained in a text data; (B) determining a sentence end pause duration corresponding to the sentence text portion according to at least the number of characters; and (C) generating voice data corresponding to the text data and used for being output in the form of sound according to at least the sentence text portion and the sentence end pause duration, wherein the voice data comprises a sentence voice portion for indicating the sentence text portion by voice, and a sentence end pause portion following the sentence voice portion, not indicating any sentence text portion and lasting for a time length that matches the sentence end pause duration.

Description

Voice data generation method and system and computer program product

The present invention relates to a data generation method, in particular to a voice data generation method suitable for being applied to computer voice output. The present invention also relates to a voice data generation system suitable for being applied to computer voice output, and a computer program product.

Converting text into computer speech is a common function in existing technologies, but making computer speech sound more natural is always a very challenging goal. Therefore, how to make computer speech closer to the way real people speak has become the topic to be explored in this case.

In order to make computer speech closer to the way real people speak, one of the purposes of the present invention is to provide a method for generating speech data.

The voice data generation method of the present invention is implemented by a voice data generation system on a text data; the voice data generation method comprises: (A) calculating a number of characters corresponding to a sentence text part according to a sentence text part contained in the text data; (B) determining a sentence end pause duration corresponding to the sentence text part at least according to the number of characters; (C) generating a voice data corresponding to the text data and used for being output in the form of sound according to at least the sentence text part and the sentence end pause duration, wherein the voice data comprises a sentence voice part used for indicating the sentence text part by voice, and a sentence end pause part which follows the sentence voice part, does not indicate any sentence text part and lasts for a time length that matches the sentence end pause duration.

In some embodiments of the speech data generation method of the present invention, the speech data generation system stores a speech speed parameter, and the speech speed parameter is related to the speech speed when the speech data is output in the form of sound. In step (B), the speech data generation system calculates an expected time corresponding to the text part of the sentence based on the number of characters and the speech speed parameter, and then determines the pause length at the end of the sentence based on at least the expected time.

In some embodiments of the voice data generation method of the present invention, the voice data generation system further stores a preset available time parameter. In step (B), when the sentence text portion is the first sentence text portion in a paragraph of the text data, the voice data generation system determines the pause duration at the end of the sentence based on a remaining time value, wherein the remaining time value is the difference between the preset available time parameter and the expected consuming time.

In some embodiments of the voice data generation method of the present invention, the voice data generation system further stores a first pause duration parameter and a second pause duration parameter greater than the first pause duration parameter. In step (B), when the sentence text portion is the first sentence text portion in the paragraph of the text data, the voice data generation system determines the pause duration at the end of the sentence in the following manner: if the remaining time value is greater than or equal to a threshold value, the pause duration at the end of the sentence is set to the value of the first pause duration parameter; if the remaining time value is less than the threshold value, the pause duration at the end of the sentence is set to the value of the second pause duration parameter.

In some embodiments of the voice data generation method of the present invention, the voice data generation system further stores a preset available time parameter. In step (B), when the sentence text portion is the Nth sentence text portion in a paragraph of the text data (N is an integer greater than 1), the voice data generation system determines the end-of-sentence pause duration by determining the end-of-sentence pause duration based on a remaining time value, wherein the remaining time value is related to the preset available time parameter and is also related to another remaining time value and another end-of-sentence pause duration corresponding to the (N-1)th sentence text portion of the paragraph of the sentence text portion.

In some implementations of the voice data generating method of the present invention, the voice data generating system further stores a minimum pause duration parameter, a first pause duration parameter greater than the shortest pause duration parameter, and a second pause duration parameter greater than the first pause duration parameter. In step (B), when the sentence text portion is the Nth sentence text portion in the paragraph of the text data, the voice data generation system determines the pause duration at the end of the sentence in the following manner: if the remaining time value is greater than or equal to a first threshold value which is a positive number, the pause duration at the end of the sentence is set to the value of the shortest pause duration parameter; if the remaining time value is less than the first threshold value and greater than or equal to a second threshold value which is a negative number, the pause duration at the end of the sentence is set to the value of the first pause duration parameter; if the remaining time value is less than the second threshold value, the pause duration at the end of the sentence is set to the value of the second pause duration parameter.

In some implementations of the voice data generation method of the present invention, in step (B), when the remaining time value is less than a negative threshold value, the voice data generation system also splits the text portion of the sentence into multiple sequential sentence segments, sets the pause duration at the end of the sentence to the value of a pause duration parameter, and determines the pause duration in a sentence. In step (C), when the remaining time value is less than the threshold value, the speech voice portion of the sentence includes a plurality of speech segments that are sequential and are respectively used to indicate the sentence segments by voice, and M mid-sentence pause portions that do not indicate any sentence segment and whose duration matches the duration of the mid-sentence pause, wherein M is an integer greater than or equal to 1, and the M mid-sentence pause portions are respectively subsequent to the first M speech segments in the speech segments.

In some embodiments of the speech data generation method of the present invention, the speech data generation method further includes a step before step (A): (D) defining the sentence text portion from the text data based on one or more specific symbols contained in the text data.

Another object of the present invention is to provide a voice data generation system that helps make computer voice closer to the way real people speak.

The speech data generation system of the present invention comprises a storage unit and a processing unit electrically connected to the storage unit, and the processing unit is used to: calculate the number of characters corresponding to a sentence text part according to a sentence text part contained in a text data; determine a sentence end pause length corresponding to the sentence text part at least according to the number of characters; generate a speech data corresponding to the text data and used to be output in the form of sound according to at least the sentence text part and the sentence end pause length, wherein the speech data comprises a sentence speech part used to indicate the sentence text part by voice, and a sentence end pause part that follows the sentence speech part, does not indicate any sentence text part and lasts for a time length that matches the sentence end pause length.

In some embodiments of the speech data generation system of the present invention, the storage unit stores a speech speed parameter, and the speech speed parameter is related to the speech speed when the speech data is output in the form of sound. The processing unit is used to calculate an expected time corresponding to the text part of the sentence according to the number of characters and the speech speed parameter, and then determine the pause length at the end of the sentence at least according to the expected time.

In some embodiments of the speech data generation system of the present invention, the storage unit further stores a preset available time parameter. When the sentence text portion is the first sentence text portion in a paragraph of the text data, the processing unit determines the pause duration at the end of the sentence by a remaining time value, wherein the remaining time value is the difference between the preset available time parameter and the estimated consuming time.

In some embodiments of the speech data generation system of the present invention, the storage unit further stores a first pause duration parameter and a second pause duration parameter greater than the first pause duration parameter. When the sentence text portion is the first sentence text portion in the paragraph of the text data, the processing unit determines the pause duration at the end of the sentence in a manner including: if the remaining time value is greater than or equal to a threshold value, setting the pause duration at the end of the sentence to the value of the first pause duration parameter; if the remaining time value is less than the threshold value, setting the pause duration at the end of the sentence to the value of the second pause duration parameter.

In some implementations of the speech data generation system of the present invention, the storage unit further stores a preset available time parameter. When the sentence text portion is the Nth sentence text portion in a paragraph of the text data (N is an integer greater than 1), the processing unit determines the end-of-sentence pause duration by determining the end-of-sentence pause duration based on a remaining time value, wherein the remaining time value is related to the preset available time parameter and is also related to another remaining time value and another end-of-sentence pause duration corresponding to the (N-1)th sentence text portion of the paragraph of the sentence text portion.

In some implementations of the voice data generating system of the present invention, the storage unit further stores a shortest pause duration parameter, a first pause duration parameter greater than the shortest pause duration parameter, and a second pause duration parameter greater than the first pause duration parameter. In the case where the sentence text portion is the Nth sentence text portion in the paragraph of the text data (N is an integer greater than 1), the processing unit determines the pause duration at the end of the sentence in the following manner: if the remaining time value is greater than or equal to a first threshold value which is a positive number, the pause duration at the end of the sentence is set to the value of the shortest pause duration parameter; if the remaining time value is less than the first threshold value and greater than or equal to a second threshold value which is a negative number, the pause duration at the end of the sentence is set to the value of the first pause duration parameter; if the remaining time value is less than the second threshold value, the pause duration at the end of the sentence is set to the value of the second pause duration parameter.

In some implementations of the speech data generation system of the present invention, the processing unit is also used to split the text portion of the sentence into multiple sequential sentence segments when the remaining time value is less than a negative threshold value, set the pause duration at the end of the sentence to the value of a pause duration parameter, and determine the pause duration in a sentence. When the remaining time value is less than the threshold value, the speech voice portion of the sentence includes a plurality of speech segments that are sequential and are respectively used to indicate the sentence segments by voice, and M mid-sentence pause portions that do not indicate any sentence segment and whose duration matches the duration of the mid-sentence pause, wherein M is an integer greater than or equal to 1, and the M mid-sentence pause portions are respectively subsequent to the first M speech segments in the speech segments.

In some implementations of the speech data generation system of the present invention, the processing unit is further used to define the sentence text part from the text data according to one or more specific symbols contained in the text data before calculating the number of characters.

Another object of the present invention is to provide a computer program product that helps make computer voice closer to the way real people speak.

The computer program product of the present invention includes an application program, wherein the application program is suitable for being loaded and run by an electronic device, so that the electronic device implements the voice data generation method described in any of the above-mentioned implementation forms for a text data.

The effect of the present invention is that the voice data generation system can determine the length of the sentence-end pause corresponding to the text part of the sentence at least according to the number of characters in the text part of the sentence of the text data, thereby setting the duration of the sentence-end pause part following the voice part of the sentence in the voice data. In this way, the voice data generated by the voice data generation system can be based on the length of the sentence, thereby simulating the pauses of different lengths caused by breathing when a real person speaks. Therefore, the voice data generation system helps to make computer voice closer to the way a real person speaks.

Before the invention is described in detail, it should be noted that, if not specifically defined, the "electrical connection" described in this patent specification generally refers to "wired electrical connection" achieved by connecting multiple electronic devices/devices/components to each other through conductive materials, and "radio connection" for unidirectional/bidirectional wireless signal transmission through wireless communication technology. In addition, the "electrical connection" described in this patent specification also generally refers to "direct electrical connection" formed by direct connection between multiple electronic devices/devices/components, and "indirect electrical connection" formed by indirect connection between multiple electronic devices/devices/components through other electronic devices/devices/components.

Referring to FIG. 1 , an embodiment of the voice data generating system 1 of the present invention is suitable for being electrically connected to a plurality of user terminals 5 (only one of which is shown in FIG. 1 ) via a network. Each user terminal 5 is a mobile phone, tablet computer, laptop computer or desktop computer that can be operated by a user, and for ease of description, the operation of the present embodiment is described below using only the user terminal 5 shown in FIG. 1 as an example.

In this embodiment, the voice data generation system 1 is implemented as a server device, and the voice data generation system 1 includes a processing unit 11 and a storage unit 12 electrically connected to the processing unit 11. The processing unit 11 is a central processor with data calculation and processing functions, and the processing unit 11 is suitable for being electrically connected to the user end 5 through a network for communication. The storage unit 12 is a data storage device (such as a hard disk) for storing digital data.

However, in another embodiment, the voice data generation system 1 can also be implemented as a plurality of servo devices electrically connected to each other, the processing unit 11 is implemented as a combination of a plurality of central processors respectively possessed by the servo devices, and the storage unit 12 is implemented as a combination of a plurality of storage devices respectively possessed by the servo devices. In addition, in another embodiment, the voice data generation system 1 is an electronic device that can be operated by a user, and can be implemented as a mobile phone, a tablet computer, a laptop computer, or a desktop computer. Therefore, it should be understood that the actual implementation of the voice data generation system 1 in terms of hardware is not limited to this embodiment.

In this embodiment, the storage unit 12 stores a speech rate parameter P1, a preset available time parameter P2, and a plurality of pause time parameters P3, wherein the "time" mentioned in this patent specification represents "time length". The speech rate parameter P1, the preset available time parameter P2, and the pause time parameters P3 are used by the processing unit 11 to generate voice data that can be played out as computer voice (for example, played out by the user terminal 5), and the relevant details of the voice data will be described later.

The speech rate parameter P1 is used to set the text playback speed of the voice data itself generated by the processing unit 11. In the present embodiment, the unit represented by the speech rate parameter P1 is "seconds/character". In other words, the speech rate parameter P1 represents the time required for a single character to be played, but is not limited to this. For example, if the value of the speech rate parameter P1 is "0.2", it means that when the voice data is played as computer voice, each character in it takes an average of 0.2 seconds to be played. Therefore, assuming that an example of a sentence text part in the voice data is composed of ten characters (for example, "Today's weather is sunny and cloudy"), this sentence text part will take 2 seconds to be played as computer voice.

The default available time parameter P2 is used to indicate the upper limit of the ideal time length for which multiple characters are played continuously during the playback of the voice data. More specifically, the default available time parameter P2 can be understood as a parameter used to simulate the lung capacity of a real person. For example, if the default available time parameter P2 is set to 3.5 seconds, the speaking style presented by the voice data generated according to the default available time parameter P2 is to simulate a real person who can speak easily for 3.5 seconds in one breath, but will gradually feel tired after more than 3.5 seconds.

In this embodiment, the number of the pause duration parameters P3 is three, and the three pause duration parameters P3 are respectively a shortest pause duration parameter P30, a first pause duration parameter P31 greater than the shortest pause duration parameter P30, and a second pause duration parameter P32 greater than the first pause duration parameter P31. Specifically, the shortest pause time parameter P30 is used to simulate the length of time a real person pauses slightly between sentences during speaking, the first pause time parameter P31 is used to simulate the length of time a real person pauses due to taking a small breath during speaking, and the second pause time parameter P32 is used to simulate the length of time a real person pauses due to taking a large breath during speaking. Exemplarily, the shortest pause time parameter P30 is, for example, 0.2 seconds, the first pause time parameter P31 is, for example, 0.4 seconds, and the second pause time parameter P32 is, for example, 0.8 seconds, but it is not limited thereto.

It is to be noted that the values of the speech rate parameter P1, the default available time parameter P2 and the pause time parameters P3 can be freely adjusted according to the different speaking styles to be presented through the voice data. For example, if the speaking style of an adult male is to be simulated, the default available time parameter P2 can be set to 3.5 seconds, and if the speaking style of a child is to be simulated, the default available time parameter P2 can be set to 1.9 seconds, but the present invention is not limited thereto. Depending on the desired speaking style, the speech rate parameter P1, the default available time parameter P2 and the pause time parameters P3 can all be freely set to various values greater than 0. Therefore, the actual values of the speech rate parameter P1, the default available time parameter P2 and the pause time parameters P3 are not limited to the above examples.

Referring to FIG. 1 and FIG. 2 (composed of FIG. 2A and FIG. 2B ), the following is an exemplary description of how the voice data generation system 1 of the present embodiment implements a voice data generation method for a text data. The text data may be received by the processing unit 11 from the user end 5 (i.e., transmitted by the user end 5 to the processing unit 11), or the text data may be pre-stored in the storage unit 12 and read by the processing unit 11. Since the way in which the processing unit 11 obtains the text data does not affect the implementation of the voice data generation method, the present embodiment does not limit the source of the text data.

For the sake of ease of description and understanding, let's assume that the content of the text data is: "When we were students, our teachers would carefully plan activities for us. Halloween is one of the most anticipated holidays of the year. Parents would rack their brains to coordinate with school activities and dress up their children. Sometimes they would ask for what they want to dress up as, but more often they would be reluctant and dress up to cater to their parents' tastes. Even so, it is very happy to celebrate with classmates at school and go to the streets to ask for candy together!" It should be noted that the text data shown above has only a single paragraph, but the voice data generation method can also be applied to text data with multiple paragraphs.

First, in step S1, the processing unit 11 determines a plurality of threshold values and a plurality of supplementary time values according to the value of the preset available time parameter P2 (eg, "3.5").

In this embodiment, the threshold values include, for example, a first threshold value that is a positive number, a second threshold value that is a negative number, and a third threshold value that is less than the second threshold value and is also a negative number. On the other hand, the supplementary time values include, for example, a first supplementary time value corresponding to the first pause duration parameter P31, and a second supplementary time value corresponding to the second pause duration parameter P32.

Specifically, in the present embodiment, the first threshold value is 0.5 times (e.g., "1.75") the value of the default available time parameter P2 (e.g., "3.5"), the second threshold value is -0.5 times (e.g., "-1.75") the value of the default available time parameter P2, and the third threshold value is -1 times (e.g., "-3.5") the value of the default available time parameter P2. On the other hand, in the present embodiment, the first supplementary time value is 0.5 times (e.g., "1.75") the value of the default available time parameter P2, and the second supplementary time value matches the value of the default available time parameter P2 (e.g., "3.5"). It should be noted that the threshold values and the supplementary time values are calculated by the processing unit 11 according to the value of the preset available time parameter P2 with a preset multiplier. However, the multiplier relationship between each of them and the preset available time parameter P2 can be freely set and adjusted. Therefore, the actual states of the threshold values and the supplementary time values are certainly not limited to the examples mentioned above.

After the processing unit 11 determines the threshold values and the replenishment time values, the process proceeds to step S2.

In step S2, the processing unit 11 defines a plurality of sequential sentence text parts from the text data according to one or more specific symbols contained in the text data. More specifically, each sentence text part is composed of a plurality of characters that can be played by computer voice in this embodiment, and the specific symbols are a plurality of predefined specific punctuation marks, including but not limited to commas, periods, hyphens, space marks, semicolons, question marks, exclamation marks, quotation marks, colons, etc., which are used to indicate pauses.

Taking the text data shown above as an example, the first sentence text part defined by the processing unit 11 would be "When we were students, our teachers would carefully plan activities for us", the second sentence text part would be "Halloween is one of the most anticipated holidays of the year", the third sentence text part would be "Mom and Dad also racked their brains to cooperate with school activities", and so on.

After the processing unit 11 defines the sentence text parts contained in the text data, the process proceeds to step S3.

In step S3, for the first sentence text part (hereinafter referred to as "the first sentence text part"), the processing unit 11 calculates the number of characters corresponding to the first sentence text part according to all the characters in the first sentence text part. And, for the convenience of description, the number of characters corresponding to the first sentence text part is taken as a first character number. Taking the text data shown above as an example, the first sentence text part is "When we were students, our teachers would carefully plan activities for us", so the first character number would be "18", that is, the first sentence text part is composed of 18 characters in total.

After the processing unit 11 calculates the first character quantity, the process proceeds to step S4.

In step S4, for the first sentence text portion, the processing unit 11 calculates an expected time corresponding to the first sentence text portion according to the first character number and the speech rate parameter P1. And, for the convenience of description, the expected time corresponding to the first sentence text portion is referred to as a first expected time. More specifically, the first expected time represents the length of time required for the first sentence text portion to be played as computer voice. Moreover, in the present embodiment, the processing unit 11 multiplies the first character number (for example, "18") and the speech rate parameter P1 (for example, "0.2") to calculate the first expected time (for example, "3.6"), but is not limited thereto.

After the processing unit 11 calculates the first estimated time duration, the process proceeds to step S5.

In step S5, for the first sentence text portion, the processing unit 11 calculates a remaining time value corresponding to the first sentence text portion according to the first expected consuming time and the preset available time parameter P2. And, for the convenience of description, the remaining time value corresponding to the first sentence text portion is referred to as a first remaining time value. More specifically, in this embodiment, the processing unit 11 calculates the first remaining time value (e.g., "-0.1") by subtracting the first expected consuming time (e.g., "3.6" cited in step S4) from the value of the preset available time parameter P2 (e.g., "3.5"); in other words, the first remaining time value is the difference between the preset available time parameter P2 and the first expected consuming time.

In this embodiment, the first remaining time value can be understood as the lung capacity of a real person after speaking the first sentence text part in one breath at the speech speed represented by the speech speed parameter P1, but it is not limited to this. In this embodiment, if the first remaining time value is greater than or equal to 0, it can be understood as a real person can easily speak the first sentence text part in one breath. On the contrary, if the first remaining time value is less than 0 (i.e., a negative number), it can be understood as a real person will feel tired due to insufficient lung capacity after speaking the first sentence text part in one breath. Moreover, when the remaining time value is less than 0, the size of its absolute value can be understood as the degree of effort, that is, the larger the absolute value, the higher the degree of effort.

After the processing unit 11 calculates the first remaining time value, the process proceeds to step S6.

In step S6, the processing unit 11 determines a sentence end pause duration corresponding to the first sentence text portion according to the first remaining time value, the second threshold value, and the third threshold value. And, for ease of description, the sentence end pause duration corresponding to the first sentence text portion is referred to as a first sentence end pause duration.

The first sentence end pause duration indicates the length of time that the computer voice pauses after playing the first sentence text portion and before continuing to play the next sentence text portion. Moreover, in this embodiment, the processing unit 11 determines the first sentence end pause duration by comparing the first remaining time value with the second threshold value (e.g., "-1.75" as exemplified in step S1) and the third threshold value (e.g., "-3.5" as exemplified in step S1), and determining the first sentence end pause duration based on the comparison result, but the present invention is not limited thereto.

Specifically, if the processing unit 11 determines that the first remaining time value is greater than or equal to the second threshold value, the processing unit 11 sets the first end-of-sentence pause duration to the value of the first pause duration parameter P31 (e.g., "0.4"). On the other hand, if the processing unit 11 determines that the first remaining time value is less than the second threshold value and greater than or equal to the third threshold value, the processing unit 11 sets the first end-of-sentence pause duration to the value of the second pause duration parameter P32 (e.g., "0.8"). On the other hand, if the processing unit 11 determines that the first remaining time value is less than the third threshold value, the processing unit 11 not only sets the pause duration at the end of the first sentence to the value of the second pause duration parameter P32, but also splits the first sentence text portion into a plurality of sentence segments that are sequential and can together constitute the first sentence text portion, and further determines a mid-sentence pause duration corresponding to the first sentence text portion. More specifically, for the first sentence text portion, the processing unit 11 will split the first sentence text portion into the sentence segments and determine the pause duration in the sentence only when the first remaining time value is less than the third threshold value. Moreover, in this embodiment, the processing unit 11 sets the pause duration in the sentence to the value of the first pause duration parameter P31 (e.g., "0.4"), but is not limited to this.

After the processing unit 11 determines the pause duration at the end of the first sentence, the process proceeds to step S7.

In step S7, for the second sentence text part (hereinafter referred to as "the second sentence text part"), the processing unit 11 calculates the number of characters corresponding to the second sentence text part according to all the characters in the second sentence text part. And, for the convenience of description, the number of characters corresponding to the second sentence text part is taken as a second character number. Taking the text data shown above as an example, the second sentence text part is "Halloween is one of the most anticipated festivals in a year", therefore, the second character number will be "16", that is, the second sentence text part is composed of 16 characters in total.

After the processing unit 11 calculates the second character quantity, the process proceeds to step S8.

In step S8, for the second sentence text portion, the processing unit 11 calculates an expected time corresponding to the second sentence text portion according to the second character number and the speech rate parameter P1. And, for the convenience of description, the expected time corresponding to the second sentence text portion is referred to as a second expected time. Similar to the first expected time, the second expected time represents the length of time required for the second sentence text portion to be played as computer voice. Moreover, in the present embodiment, the processing unit 11 multiplies the second character number (for example, "16") and the speech rate parameter P1 (for example, "0.2") to calculate the second expected time (for example, "3.2"), but is not limited thereto.

After the processing unit 11 calculates the second estimated time duration, the process proceeds to step S9.

In step S9, for the second sentence text part, the processing unit 11 calculates a remaining time value corresponding to the second sentence text part according to the second estimated consuming time, the remaining time value corresponding to the previous sentence text part (i.e., the first sentence text part) of the second sentence text part (i.e., the first remaining time value), and one of the supplementary time values related to the end-of-sentence pause duration corresponding to the previous sentence text part (i.e., the first end-of-sentence pause duration). And, for the convenience of description, the remaining time value corresponding to the second sentence text part is taken as a second remaining time value. Similar to the first remaining time value, the second remaining time value can be understood as the lung capacity of a real person after simulating the speaking speed of the second sentence in one breath at the speaking speed parameter P1.

In this embodiment, the processing unit 11 calculates the second remaining time value by adding the first remaining time value (e.g., "-0.1" as exemplified in step S5) to the supplementary time value related to the pause duration at the end of the first sentence (e.g., the first supplementary time value, whose value is, for example, "1.75" as exemplified in step S1), and then subtracting the second expected duration (e.g., "3.2" as exemplified above), thereby calculating the second remaining time value (e.g., "-1.55"). Furthermore, if the pause duration at the end of the sentence corresponding to the previous text portion is set to the value of the first pause duration parameter P31 (for example, "0.4" as cited in step S6), then when the processing unit 11 calculates the second remaining time value, it is the remaining time value corresponding to the previous text portion plus the first supplementary time value corresponding to the first pause duration parameter P31 (for example, "1.75" as cited in step S1), and then subtracts the second expected time. If the pause duration at the end of the sentence corresponding to the previous text portion is set to the value of the second pause duration parameter P32 (for example, "0.8" as cited in step S6), then when the processing unit 11 calculates the second remaining time value, it adds the remaining time value corresponding to the previous text portion to the second supplementary time value corresponding to the second pause duration parameter P32 (for example, "3.5" as cited in step S1), and then subtracts the second expected time.

More specifically, which supplementary time value the processing unit 11 uses to calculate the second remaining time value depends on whether the previous end-of-sentence pause duration is set to the value of the first pause duration parameter P31 or the value of the second pause duration parameter P32. Each supplementary time value can be understood as simulating the lung capacity restored by a real person through breathing after speaking a sentence. Therefore, if the pause duration at the end of the previous sentence is set to a relatively small value of the first pause duration parameter P31, it can be understood as simulating a real person using only a relatively short time to breathe. In this case, the processing unit 11 will use the relatively small first supplement time value to calculate the second remaining time value, thereby simulating that a real person can only recover a relatively small amount of lung capacity before speaking the next sentence. On the contrary, if the pause duration at the end of the previous sentence is set to a relatively large value of the second pause duration parameter P32, it can be understood as simulating a real person taking a relatively long time to breathe. In this case, the processing unit 11 will use the relatively large second supplement time value to calculate the second remaining time value, thereby simulating a real person to recover relatively more lung capacity before speaking the next sentence.

After the processing unit 11 calculates the second remaining time value, the process proceeds to step S10.

In step S10, the processing unit 11 determines a sentence end pause duration corresponding to the second sentence text portion according to the second remaining time value, the first threshold value, the second threshold value, and the third threshold value. And, for ease of description, the sentence end pause duration corresponding to the second sentence text portion is referred to as a second sentence end pause duration.

The second end-of-sentence pause duration indicates the length of time that the computer voice pauses after playing the second sentence text portion and before continuing to play the next sentence text portion. Moreover, in this embodiment, the processing unit 11 determines the second end-of-sentence pause duration by comparing the second remaining time value with the first threshold value (e.g., "1.75" as exemplified in step S1), the second threshold value (e.g., "-1.75" as exemplified in step S1), and the third threshold value (e.g., "-3.5" as exemplified in step S1), and determining the second end-of-sentence pause duration based on the comparison result, but the present invention is not limited thereto.

More specifically, on the one hand, if the processing unit 11 determines that the second remaining time value is greater than or equal to the first threshold value (for example, greater than or equal to 1.75), the processing unit 11 sets the second end-of-sentence pause duration to the value of the shortest pause duration parameter P30 (for example, "0.2").

On the other hand, if the processing unit 11 determines that the second remaining time value is less than the first threshold value and greater than or equal to the second threshold value (for example, less than 1.75 and greater than or equal to -1.75), the processing unit 11 sets the second end-of-sentence pause duration to the value of the first pause duration parameter P31 (for example, "0.4").

On the other hand, if the processing unit 11 determines that the second remaining time value is less than the second threshold value and greater than or equal to the third threshold value (for example, less than -1.75 and greater than or equal to -3.5), the processing unit 11 sets the second end-of-sentence pause duration to the value of the second pause duration parameter P32 (for example, "0.8").

On the other hand, if the processing unit 11 determines that the second remaining time value is less than the third threshold value (for example, less than -3.5), the processing unit 11 not only sets the pause duration at the end of the second sentence to the value of the second pause duration parameter P32, but also splits the second sentence text portion into a plurality of sentence segments that are sequential and can together constitute the second sentence text portion, and further determines another mid-sentence pause duration corresponding to the second sentence text portion. Moreover, for the second sentence text portion, the processing unit 11 will split the second sentence text portion into the sentence segments and determine the pause duration in the sentence only when the second remaining time value is less than the third threshold value. Moreover, in this embodiment, the processing unit 11 sets the pause duration in the sentence to the value of the first pause duration parameter P31 (for example, "0.4"), but is not limited to this.

After the processing unit 11 determines the pause duration at the end of the second sentence, the process proceeds to step S11.

In step S11, for each of the third sentence text part to the second to last sentence text part of the text data, the processing unit 11 calculates the number of characters, the estimated time consumption, the remaining time value, and the pause time at the end of the sentence corresponding to each sentence text part. The calculation method of the processing unit 11 for each number of characters, each estimated time consumption, each remaining time value, and each pause time at the end of the sentence in this step is substantially the same as the calculation method described in step S7, step S8, step S9, and step S10, and therefore will not be repeated here one by one.

After the processing unit 11 calculates the sentence end pause duration corresponding to each of the third sentence text portion to the second to last sentence text portion, the process proceeds to step S12.

In step S12, for the last sentence text portion of the text data (in this embodiment, it is regarded as an eleventh sentence text portion, that is, "also very happy" in the text data shown above), the processing unit 11 calculates a remaining time value corresponding to the eleventh sentence text portion (in this embodiment, it is regarded as an eleventh remaining time value). The processing unit 11 calculates the eleventh remaining time value in the same manner as the second remaining time value calculated in steps S7 to S9, so it will not be repeated here.

After the processing unit 11 calculates the eleventh remaining time value corresponding to the eleventh sentence text portion, the process proceeds to step S13.

In step S13, the processing unit 11 determines a sentence-end pause duration corresponding to the eleventh sentence text portion (in this embodiment, it is taken as an eleventh sentence-end pause duration). In addition, the processing unit 11 compares the eleventh remaining time value with the third threshold value (e.g., "-3.5" as exemplified in step S1), and determines whether to split the eleventh sentence text portion according to the comparison result.

In this embodiment, the processing unit 11 directly sets the sentence-end pause duration corresponding to the last sentence text portion to the value of the second pause duration parameter P32 (e.g., "0.8"). In other words, in this embodiment, the value of the eleventh sentence-end pause duration does not involve the number of characters in the eleventh sentence text portion and the eleventh remaining time value. However, in other embodiments, the processing unit 11 may determine the sentence-end pause duration corresponding to the last sentence text portion in a manner that is substantially the same as that for determining the second sentence-end pause duration, and is not limited to this embodiment.

In addition, if the processing unit 11 determines that the eleventh remaining time value is less than the third threshold value (for example, less than -3.5), the processing unit 11 splits the eleventh sentence text part into a plurality of sentence segments that are sequential and can together constitute the eleventh sentence text part, and determines a sentence pause length corresponding to the eleventh sentence text part. Moreover, in the present embodiment, the processing unit 11, for example, directly uses the value of the first pause length parameter P31 (for example, "0.4") as the sentence pause length corresponding to the eleventh sentence text part, but is not limited to this. On the contrary, if the processing unit 11 determines that the eleventh remaining time value is not less than the third threshold value, the processing unit 11 will not split the eleventh sentence text portion, and will not set the mid-sentence pause length for the eleventh sentence text portion.

After the processing unit 11 sets the pause time at the end of the sentence corresponding to the text portion of the eleventh sentence, the process proceeds to step S14.

In step S14, the processing unit 11 generates a voice data corresponding to the text data and used to be played in the form of computer voice according to all the sentence text parts and the sentence end pause durations corresponding to the sentence text parts. The voice data includes a plurality of sentence voice parts corresponding to the sentence text parts, and a plurality of sentence end pause parts corresponding to the sentence voice parts. In addition, the sentence end pause parts correspond to the sentence text parts of the text data in addition to the sentence voice parts.

In more detail, each sentence voice part is used to present the sentence text part corresponding to the sentence voice part itself in the process of playing the voice data. On the other hand, for each sentence end pause part, the sentence end pause part is continuous with the sentence voice part corresponding to itself, and is a section of silence or a part with breathing effect in the voice data. In other words, the sentence end pause part will not indicate any of the sentence text parts in computer voice. Furthermore, the duration of the pause at the end of the sentence is consistent with the duration of the pause at the end of the sentence corresponding to the text portion of the sentence to which it corresponds. For example, if the duration of the pause at the end of the sentence corresponding to the text portion of the first sentence (i.e., the duration of the pause at the end of the first sentence) is 0.4 seconds, then the duration of the pause at the end of the sentence corresponding to the text portion of the first sentence in the voice data will be 0.4 seconds.

Furthermore, assuming that the processing unit 11 splits one or more of the sentence text parts into multiple sentence segments, and sets corresponding mid-sentence pause durations for the sentence text parts, then for each sentence text part that is split into multiple sentence segments, the sentence voice part corresponding to the sentence text part in the voice data will include multiple sequential voice segments and M mid-sentence pause parts, and M is an integer greater than or equal to 1. More specifically, the voice segments are the sentence segments that respectively correspond to the sentence text part, and are used to indicate the character contents of the sentence segments respectively by computer voice. On the other hand, the number of the M mid-sentence pauses is the number of the speech segments minus one, and the M mid-sentence pauses are respectively connected to the first M speech segments in the speech segments. Further, for each mid-sentence pause, the mid-sentence pause is a section of silence or a section with a breathing effect in the speech portion of the sentence. In other words, the mid-sentence pause will not indicate any sentence segment of the speech portion of the sentence in computer voice. Moreover, the duration of the mid-sentence pause is consistent with the duration of the mid-sentence pause corresponding to the text portion of the sentence. For example, assuming that the processing unit 11 splits the first sentence text portion in the text data into two sentence segments, and sets the pause duration corresponding to the first sentence text portion to 0.4 seconds, then in the voice data, the sentence voice portion corresponding to the first sentence text portion will include two corresponding voice segments, and a single pause portion between the two voice segments with a duration of 0.4 seconds. It should be noted that splitting a single sentence text portion into multiple sentence segments can be achieved using existing technologies, so the details are not described in detail here.

Continuing with the text data shown above, two exemplary forms of the voice data corresponding to the text data are shown below. Among them, each sentence voice part of the voice data is represented by adding curly brackets "{}" to the corresponding sentence text part, and each end-of-sentence pause part is represented by a half-width underscore symbol "_". In addition, for the sentence voice part including a voice segment and a pause part in the sentence, each voice segment is represented by adding a plurality of characters corresponding to it to a middle bracket "[]", and each pause part in the sentence is represented by a half-width pound sign "#".

First, assume that the processing unit 11 does not split any of the text parts of the text data into multiple voice segments. In this case, the first exemplary state of the voice data is: "{When we were students, teachers would carefully plan activities for us}_{Halloween is one of the most anticipated holidays of the year}_{Moms and dads also racked their brains to cooperate with school activities}_{dress up their children}_{sometimes they would suggest what they want to dress up as}_{but more often they would be reluctant}_{dress up to their parents to suit their tastes}_{even so}_{celebrate with classmates at school}_{go to the streets to ask for candy together}_{and are very happy}_".

Next, it is assumed that the processing unit 11 splits the first, second, third, fifth, sixth, seventh, and ninth sentence text parts in the text data and sets the pause duration in the sentence. In this case, the second exemplary state of the voice data is: "{[When we were students, teachers would]#[carefully plan activities for us]}_{[Halloween is]#[one of the most anticipated holidays of the year]}_{[Moms and dads]#[also rack their brains to coordinate school activities]}_{dress up their children}_{[Sometimes]#[they come up with what they want to dress up as]}_{[but more often]#[they are reluctant]}_{[cater to their parents' tastes]#[dress up what they want]}_{even so}_{[when we go to school]#[celebrate with classmates]}_{go to the streets to ask for candy together}_{it's still very happy}_".

After the processing unit 11 generates the voice data corresponding to the text data, the process proceeds to step S15.

In step S15, the processing unit 11 transmits the voice data to the user terminal 5, so that the user terminal 5 plays the voice data in the form of computer voice for the user to listen to. It should be noted that in the embodiment where the voice data generating system 1 is implemented as an electronic device, the voice data can also be played by the voice data generating system 1 itself.

Continuing the first exemplary aspect of the voice data described above, based on the voice parts of the sentences and the pause parts at the end of the sentences contained in the voice data, assuming that the pause duration at the end of the first sentence is set to 0.4 seconds and the pause duration at the end of the second sentence is set to 0.8 seconds, when the voice data is played by the user terminal 5, the user terminal 5 plays the video "Teachers will carefully plan activities for us during our student years". After the first speech part, there will be a pause of 0.4 seconds (i.e., the pause at the end of the first sentence is played), and then the second speech part of "Halloween is one of the most anticipated festivals of the year" will be played. Moreover, after the user terminal 5 finishes playing the second speech part, there will be a pause of 0.8 seconds (i.e., the pause at the end of the second sentence is played), and then the next speech part will be played, and so on. In this way, the voice data can simulate the situation of breathing pauses of different lengths when a real person speaks according to the length of the sentence spoken.

The above is an example of how the voice data generating system 1 of this embodiment implements the voice data generating method.

It should be noted that in other embodiments, for each sentence text part, the processing unit 11 may also directly multiply the first character number by a preset weight value after calculating the character number corresponding to the sentence text part to determine the sentence end pause length corresponding to the sentence text part. Alternatively, the processing unit 11 may also directly multiply the expected time consumption corresponding to the sentence text part by a preset weight value after calculating the expected time consumption corresponding to the sentence text part. Therefore, the way in which the processing unit 11 determines the sentence end pause length corresponding to each sentence text part is not limited to this embodiment.

It should be understood that steps S1 to S15 of the present embodiment and the flowchart of FIG2 are only used to illustrate one of the practicable ways of the voice data generating method of the present invention. It should be understood that even if steps S1 to S15 are merged, split or adjusted in sequence, if the process after merging, splitting or adjusting in sequence achieves substantially the same effect as the present embodiment in substantially the same way, it still belongs to the practicable aspects of the voice data generating method of the present invention. Therefore, steps S1 to S15 of the present embodiment and the flowchart of FIG2 are not used to limit the practicable scope of the present invention.

The present invention also provides an embodiment of a computer program product. The computer program product can be stored in a computer-readable recording medium (such as a hard disk, a flash drive, and a memory card, etc.), and includes an application program. The application program includes the speech rate parameter P1, the preset available time parameter P2, and the pause time parameters P3 shown in FIG1, and can be loaded and run by an electronic device (such as a mobile phone, a tablet computer, a laptop computer, and a desktop computer, etc.). Moreover, when the application program is loaded and run by the electronic device, the application program can enable the electronic device to be used as a voice data generation system provided by the present invention (such as the voice data generation system 1 shown in FIG1), and implement the voice data generation method provided by the present invention on a text data.

In summary, by implementing the voice data generation method on the text data, the voice data generation system 1 can determine the length of the sentence-end pause corresponding to the text portion of the sentence at least according to the number of characters in each sentence text portion of the text data, thereby setting the duration of the sentence-end pause portion following the voice portion of each sentence in the voice data. In this way, the voice data generated by the voice data generation system 1 can be based on the length of the sentence, thereby simulating pauses of different lengths caused by breathing when a real person speaks. Therefore, the voice data generation system 1 helps to make computer voice closer to the way a real person speaks, and can indeed achieve the purpose of the present invention.

However, the above is only an example of the implementation of the present invention, and it should not be used to limit the scope of the implementation of the present invention. All simple equivalent changes and modifications made according to the scope of the patent application of the present invention and the content of the patent specification are still within the scope of the patent of the present invention.

1··········· Voice data generation system 11········· Processing unit 12········· Storage unit P1········· Speech rate parameter P2········· Default available time parameter P3········· Pause time parameter P30········ Shortest pause time parameter P31········ First pause time parameter P32········ Second pause time parameter 5············ User end S1~S15·········· Steps

Other features and functions of the present invention will be clearly presented in the implementation method with reference to the drawings, wherein: FIG. 1 is a block diagram, exemplarily showing an implementation of the voice data generation system of the present invention, and a user terminal suitable for use with the implementation; and FIG. 2 (composed of FIG. 2A and FIG. 2B) is a flow chart, which is used to exemplarily illustrate how the implementation implements a voice data generation method for a text data.

S1~S15·········· Steps

Claims (8)

A method for generating speech data is implemented by a speech data generating system on a text data, wherein the speech data generating system stores a speech speed parameter and a preset available time parameter, and the speech speed parameter is related to the speech speed when the data is output in the form of sound; the method for generating speech data comprises: (A) calculating the number of characters corresponding to a sentence text portion according to a sentence text portion contained in the text data; (B) calculating an expected time corresponding to the sentence text portion according to the number of characters and the speech speed parameter, and then determining a sentence end pause length corresponding to the sentence text portion at least according to the expected time, wherein the sentence text portion is a paragraph of the text data. In the case of the first sentence text part in the text, the voice data generation system determines the sentence end pause duration in a manner that the sentence end pause duration is determined according to a remaining time value, and the remaining time value is the difference between the preset available time parameter and the expected time duration; and (C) generating a voice data corresponding to the text data and used to be output in the form of sound based on at least the sentence text part and the sentence end pause duration, wherein the voice data includes a sentence voice part for indicating the sentence text part by voice, and a sentence end pause part that follows the sentence voice part, does not indicate any sentence text part, and lasts for a time length that matches the sentence end pause duration. The voice data generation method as described in claim 1, wherein: the voice data generation system also stores a first pause duration parameter and a second pause duration parameter greater than the first pause duration parameter; In step (B), when the sentence text portion is the first sentence text portion in the paragraph of the text data, the voice data generation system determines the pause duration at the end of the sentence in the following manner: if the remaining time value is greater than or equal to a threshold value, the pause duration at the end of the sentence is set to the value of the first pause duration parameter; and if the remaining time value is less than the threshold value, the pause duration at the end of the sentence is set to the value of the second pause duration parameter. A method for generating speech data is implemented by a speech data generating system on a text data. The speech data generating system stores a speech speed parameter and a preset available time parameter, and the speech speed parameter is related to the speech speed when the data is output in the form of sound. The method for generating speech data comprises: (A) calculating the number of characters corresponding to a sentence text part according to a sentence text part included in the text data; (B) calculating an expected time corresponding to the sentence text part according to the number of characters and the speech speed parameter, and determining a sentence end pause time corresponding to the sentence text part at least according to the expected time, wherein, when the sentence text part is the Nth sentence text part in a paragraph of the text data (N is an integer greater than 1), The voice data generation system determines the end-of-sentence pause duration in a manner that determines the end-of-sentence pause duration based on a remaining time value, and the remaining time value is related to the preset available time parameter, and is also related to another remaining time value and another end-of-sentence pause duration corresponding to the (N-1)th sentence text portion of the paragraph of the sentence text portion; and (C) generating a voice data corresponding to the text data and used to be output in the form of sound based on at least the sentence text portion and the end-of-sentence pause duration, wherein the voice data includes a sentence voice portion for indicating the sentence text portion by voice, and a sentence end-of-sentence pause portion that follows the sentence voice portion, does not indicate any sentence text portion, and lasts for a time length that matches the end-of-sentence pause duration. The method for generating speech data as described in claim 3, wherein: the speech data generating system further stores a minimum pause duration parameter, a first pause duration parameter greater than the minimum pause duration parameter, and a second pause duration parameter greater than the first pause duration parameter; in step (B), when the sentence text portion is the Nth sentence text portion in the paragraph of the text data, the speech data generating system determines the sentence end pause duration The method includes: if the remaining time value is greater than or equal to a first threshold value which is a positive number, the pause duration at the end of the sentence is set to the value of the shortest pause duration parameter; if the remaining time value is less than the first threshold value and greater than or equal to a second threshold value which is a negative number, the pause duration at the end of the sentence is set to the value of the first pause duration parameter; and if the remaining time value is less than the second threshold value, the pause duration at the end of the sentence is set to the value of the second pause duration parameter. The method for generating speech data as claimed in any one of claims 1 and 3, wherein: in step (B), when the remaining time value is less than a negative threshold value, the speech data generating system further splits the text portion of the sentence into a plurality of sequential sentence segments, sets the pause duration at the end of the sentence to the value of a pause duration parameter, and determines the pause duration in the sentence; and in step (C), when the remaining time value is less than a negative threshold value, the speech data generating system further splits the text portion of the sentence into a plurality of sequential sentence segments, sets the pause duration at the end of the sentence to the value of a pause duration parameter, and determines the pause duration in the sentence; When the time value is less than the threshold value, the speech part of the sentence includes a plurality of speech segments that are sequential and are used to indicate the sentence segments by voice, and M mid-sentence pause portions that do not indicate any sentence segment and whose duration is consistent with the duration of the mid-sentence pause, where M is an integer greater than or equal to 1, and the M mid-sentence pause portions are respectively connected to the first M speech segments in the speech segments. The speech data generation method as described in any one of claim items 1 and 3 further includes a step before step (A): (D) defining the text portion of the sentence from the text data according to one or more specific symbols contained in the text data. A speech data generation system comprises: a storage unit storing a speech speed parameter and a preset available time parameter, wherein the speech speed parameter is related to the speech speed when the data is output in the form of sound; and a processing unit electrically connected to the storage unit and configured to implement the speech data generation method as described in any one of claims 1 to 6 for a text data. A computer program product includes an application program, wherein the application program includes a speech rate parameter and a preset available time parameter, and the speech rate parameter is related to the speech speed when outputting data in the form of sound, and the application program is suitable for being loaded and run by an electronic device, so that the electronic device implements the speech data generation method described in any one of claim items 1 to 6 for a text data.

Images