JP7659721B2 - Audio reproduction method, apparatus, electronic device and storage medium - Google Patents

Description

This application relates to the technical field of display control, and more specifically to audio playback methods, devices, electronic devices, and storage media.

With the rapid development of audio playback technology, there are more and more types of audio players. Audio synchronization technology is an important basis for audio playback. In order to ensure synchronous audio playback, the audio server needs to calibrate the reference time of the system connected to the audio player before sending audio data.

However, after receiving the audio data, the audio player needs to perform back-end data processing such as decoding the audio data, so it needs to reserve decoding time for processing the audio data before playback.

It is difficult to standardize and identify the back-end decoding processing time of different audio data. Moreover, some audio data has a huge data volume, which requires a long decoding processing time. The decoding processing rate places high requirements on the product hardware. After the hardware performance reaches a bottleneck, the desired audio playback time cannot be accurately synchronized with the reference time of the terminal where the audio player is located, which makes it easy for audio playback to cause phenomena such as audio-image asynchronization, dropped frames, and playback delays.

Embodiments of the present invention provide an audio playback method, device, electronic device, and storage medium that can increase the reservation time for back-end decoding processing and achieve synchronization of audio playback.

An audio reproducing method provided by an embodiment of the present invention includes:
decoding an audio data set to be played on a terminal device, the audio data set including at least one frame of audio data;
recording a reference time for starting decoding and an actual decoding process time for each frame of the audio data of the at least one frame of audio data;
obtaining a reference offset time and a desired playback time corresponding to the audio data;
adjusting the decode start reference time for each frame of the audio data of the at least one frame based on the reference offset time to obtain an adjusted decode start time;
determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time;
updating the reference playing time based on the time change to obtain an adjusted reference playing time ;
When the adjusted reference playback time reaches the desired playback time, the audio data is played back according to the terminal device.

Accordingly, an audio playback device provided by an embodiment of the present invention comprises:
a decoding module for decoding an audio data set to be played on a terminal device, said audio data set including at least one frame of audio data;
a recording module for recording a reference time for starting a decoding process and an actual decoding process time of each frame of the audio data of the at least one frame of the audio data;
an acquisition module for acquiring a reference offset time and a desired playback time corresponding to the audio data;
an adjustment module for adjusting the decoding start reference time for each frame of the audio data of the at least one frame based on the reference offset time to obtain a decoding start adjustment time;
a determining module for determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time;
a timing module for updating the reference play time based on the time change to obtain an adjusted reference play time ;
and a playback module for playing the audio data according to a terminal device when the adjusted reference playback time reaches a desired playback time.

Optionally, in some embodiments of the present application, the audio data includes current audio data, and the acquisition module:
a sampling unit for sampling at least one frame of historical audio data, said historical audio data being of the same data type as said current audio data;
a first acquiring unit for acquiring a historical reference playback time and a desired playback time after decoding processing for each frame of the historical audio data of the at least one frame of the historical audio data;
a first determination unit for respectively determining an offset time of each frame of the historical audio data of the at least one frame according to the historical reference playback time and the desired playback time;
and a second identifying unit for identifying a reference offset time of the current audio data based on at least one of the offset times.

Optionally, in some embodiments of the present application, the second identification unit is
a first specifying subunit for specifying an average value of at least one of the offset times to obtain an average offset time;
and a second identifying subunit for identifying a reference offset time of the current audio data based on the average offset time.

Optionally, in some embodiments of the present application, the second identification unit is
determining an average value of at least one of the offset times to obtain an average offset time;
taking an integer number of the average offset times to obtain an average integer offset time corresponding to the historical audio data;
The average integer offset time is determined to be a reference offset time of the current audio data, and is further used.

Optionally, in some embodiments of the present application, the second identification unit is
taking an integer number of the at least one offset time to obtain at least one integer offset time;
determining an average integer offset time of each frame of audio data of the at least one frame of audio data based on the at least one integer offset time;
The integer average offset time is further used to identify the integer average offset time as a reference offset time of the current audio data.

Optionally, in some embodiments of the present application, the sampling unit comprises:
obtaining historical audio data based on a predetermined frequency, the historical audio data including at least one frame of historical audio data;
Sampling the at least one frame of historical audio data is further utilized.

Optionally, in some embodiments of the present application, the reference offset time includes an actual offset time, and the adjustment module:
a second acquiring unit for acquiring a reservation processing time of the audio data;
a third determination unit for determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time;
and a third obtaining unit for adjusting the decoding start reference time based on the actual offset time to obtain a decoding start adjustment time.

Optionally, in some embodiments of the present application, the second acquisition unit is
The third specifying subunit specifies a reservation processing time for the audio data based on the reference time for starting decoding of the audio data and the desired playback time.

Optionally, in some embodiments of the present application, the second acquisition unit is
obtaining a reservation processing time for the audio data;
determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time;
determining an error offset time of the audio data based on the reference offset time and the actual offset time;
The decoding start reference time is adjusted based on the reference offset time and the error offset time to obtain a decoding start adjustment time.

Optionally, in some embodiments of the present application, the recording module comprises:
performing timing feedback based on a crystal oscillation to obtain a decoding start time and a decoding end time of the audio data;
The method further includes determining an actual processing time for decoding the audio data based on the decoding start time and the decoding end time.

Optionally, in some embodiments of the present application, the specific module may include:
obtaining a decoding start time and a decoding end time of a decoding process for the audio data;
determining a base reference playback time of the audio data based on the decoding start adjustment time and the decoding start time;
and determining a reference playback time of the audio data based on the base reference playback time and the decoding end time.

Optionally, in some embodiments of the present application, the acquisition module is
It is used to directly extract the desired playback time of the audio data from the audio data.

Optionally, in some embodiments of the present application, the acquisition module is
identifying an audio data type of the audio data;
Obtaining a mapping relationship set, the mapping relationship set including a mapping relationship between a predetermined audio data type and a predetermined reference offset time;
Obtaining a reference offset time corresponding to the audio data based on the mapping relationship set and the audio data type is further used.

In the embodiment of the present application, the reference playback time of the audio data is shortened, so that the reference playback time changes over time, and the audio data is played back after it reaches the desired playback time after the change; that is, after the reference playback time of the audio data meets the desired playback time, the audio data can be played back synchronously. Of these, the reference playback time is shortened by shortening the reference decoding start time, but the actual decoding processing time of the audio data is not affected or limited. Therefore, shortening the reference playback time corresponding to the audio data provides sufficient decoding time for the decoding processing of the audio data, and avoids phenomena such as audio-video asynchrony, dropped frames, or playback delays in the playback of the audio data due to the decoding processing time being too long.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific embodiments of the present application will be described in detail in conjunction with the accompanying drawings to make the technical solutions and advantages of the present invention clearer.
FIG. 2 is a scene schematic diagram of the audio playback method provided by the present invention; FIG. 2 is a flow diagram of an audio playback method provided by the present invention; FIG. 2 is another flow diagram of the audio playback method provided by the present invention; FIG. 2 is another flow diagram of the audio playback method provided by the present invention; 1 is a schematic diagram of an audio playback device provided by the present invention; 1 is a schematic diagram of an electronic device provided by the present invention;

The realization of the object, functional features and advantages of the present application will be further explained with reference to the embodiments and the accompanying drawings.
It should be understood that the specific examples described herein are merely for the purpose of interpreting the present application, and are not intended to limit the present application.

The present application provides an audio playback method, device, electronic device, and storage medium. Specifically, the audio playback method provided in the embodiments of the present invention can be executed by an electronic device including a terminal device or a server. The terminal device may be a television, a mobile phone, a notebook, a desktop, or a tablet. The server may be a standalone physical server, a server cluster or a distributed system consisting of multiple physical servers, and may be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), big data, and artificial intelligence platforms, and may be directly or indirectly connected via wired or wireless communications.

Specifically, the embodiment of the present application will be described taking as an example a case where a terminal device and a server jointly execute the audio playback method. Data transmission between the terminal device and the server can be performed in the form of a wired network, a wireless network, or a broadcast. In addition to the terminal device and the server, other devices that auxiliary execute the audio playback method can also be added, and the type of other devices is not limited here. Among them, the specific procedure for the terminal device and the server to jointly execute the audio playback method is as follows:

The terminal device receives an audio data set (audio data packets) to be played from the server, and then the processor in the terminal device decodes the received audio data set, and records the reference decode start time of each frame of audio data and the actual decode processing time of each frame of audio data. The terminal device also obtains a reference offset time and a desired playback time of each frame of audio data from the audio data, and then adjusts the reference decode start time for each frame of audio data based on the reference offset time to obtain an adjusted decode start time. Then, based on the adjusted decode start time and the actual decode processing time, the terminal device determines the reference playback time for the end of the decode processing of the audio data. Finally, if the reference playback time changes over time and the reference playback time is equal to the desired playback time, the terminal device plays the audio data. Here, if the terminal device is a television, the audio data can be played directly without using an additional audio player.

Here, the reference time of the audio data (e.g., the reference time to start decoding) is synchronized with the reference clock of the terminal device. The reference clock is adjusted and synchronized to the correct time by the Network Time Protocol (NTP). The source of the correct time is Universal Time Coordinated (UTC). For example, when the reference clock of the terminal device indicates Beijing time, if the start time of the decoding of the audio data by the terminal device is 6 o'clock Beijing time, the reference time to start decoding is also 6 o'clock.

The reference playback time is the time that can be played back after the audio data is decoded, and in theory, this reference playback time is also synchronized with Beijing time. That is, the Beijing time after the audio data is decoded is the reference playback time of the audio data. However, the timing of the reference clock is obtained by updating after data transmission, and since data transmission consumes time, the timing of the reference clock has a time error. Therefore, in the embodiment of the present application, the reference playback time is determined based on the reference time to start decoding and the actual decoding processing time of the decoding. Among them, the actual decoding processing time is timed by feedback of the crystal oscillation, so the time recording is very accurate, so that the actual reference playback time after the audio data decoding processing can be obtained, and at this time, the time stamp of the obtained reference playback time is not synchronized with the time stamp of the reference clock, but the increase of the reference playback time increases with time (Beijing time). For example, if the reference playback time after the audio data decoding processing is exactly 8 o'clock, the Beijing time will be 8:01, that is, the reference playback time is not the same as the time stamp of the Beijing time.

Here, if the reference playback time of the audio data is equal to the desired playback time, the audio data can be played back; however, decoding the audio data takes time, and the reference playback time after the audio data is decoded often exceeds the desired playback time corresponding to the audio data. Therefore, the reference playback time after the audio data is decoded is adjusted back so as to be synchronized with the desired playback time, or the reference playback time increases over time and is then synchronized with the desired playback time before the audio data is played back.

Here, the Network Time Protocol (NTP) is used for computer time synchronization, allowing computers to synchronize with servers or clock sources (e.g., quartz clocks, GPS, etc.) and providing highly accurate time correction (less than 1 millisecond difference from the standard on LANs, tens of milliseconds on WANs). Here, NTP may obtain UTC from acquisition time sources such as atomic clocks, observatories, satellites, or the Internet.

Here, the desired playback time is the desired playback time of the audio data, for example, a presentation time label (PTS, Presentation Time Stamp). When the desired playback time of the audio data is synchronized with the reference playback time of the terminal, the audio data is played back. In other words, when the desired playback time of the audio data is synchronized with international standard time and the desired playback time of the audio data lags behind the reference playback time of the terminal, the audio data cannot be played back accurately, for example, phenomena such as audio-video asynchronization, dropped frames, and playback delays occur.

In the embodiment of the present application, the reference playback time of the audio data is shortened, so that the reference playback time changes over time, and the audio data is played back after it reaches the desired playback time after the change; that is, after the reference playback time of the audio data meets the desired playback time, the audio data can be played back synchronously. Of these, the reference playback time is shortened by shortening the reference decoding start time, but the actual decoding processing time of the audio data is not affected or limited. Therefore, shortening the reference playback time corresponding to the audio data provides sufficient decoding time for the decoding processing of the audio data, and avoids phenomena such as audio-video asynchrony, dropped frames, or playback delays in the playback of the audio data due to the decoding processing time being too long.

Each of the embodiments will be described in detail below. Note that the order of description of the embodiments below does not limit the priority of the embodiments.
2 is a flow diagram of the audio playback method provided by the present invention. The specific flow of the audio playback method is as follows.

Step 101: Decode an audio data set to be played on a terminal device, the audio data set including at least one frame of audio data.

The audio data received and played by the terminal device is often encoded and compressed data. For example, in digital television, the transmitted packet TLV is an ISDB S3 standard packet, and the TLV packet corresponding to a 4K Super Hi-Vision program is generally encoded and compressed to improve transmission efficiency since the data volume of the program content is generally very large. The audio data received by the terminal device is compressed data that is decoded, and the terminal device can decode the audio data before playing it back.

Step 102: Record a reference time for starting decoding and an actual decoding process time for each frame of the audio data of the at least one frame of audio data.

Here, the decode start reference time is synchronized with a reference clock of the terminal device, that is, when the decoding process of the audio data is started, the time of the reference clock is the decode start reference time of the audio data.
Here, the actual audio data decoding processing time is the actual time required for the audio data decoding processing.

Here, the actual decoding process time of the audio data can be accurately measured by a hardware clock device. That is, optionally, in some embodiments of the present application, "recording the actual decoding process time for the decoding process of the audio data of each frame of the at least one frame of audio data" can be:
performing timing feedback based on a crystal oscillation to obtain a decoding start time and a decoding end time of the audio data;
The method further includes determining an actual processing time for decoding the audio data based on the decoding start time and the decoding end time.

For example, time can be measured by feedback from a quartz crystal. For example, an oscillator is used, and when the terminal device is turned on, time is measured from 0:00:00. The time measurement process is timing feedback using 27M quartz crystal oscillation, among which measuring time using an oscillator is relatively accurate.

Here, the time is measured when the decoding of the audio data starts, and the decoding start time is obtained. When the decoding of the audio data ends, the decoding end time of the audio data is obtained. The actual decoding processing time for the decoding of the audio data can be obtained by the difference between the decoding end time and the decoding start time.

Step 103: Obtain the reference offset time and desired playback time corresponding to the audio data.

Here, if the reference playback time of the audio data is longer than the desired playback time, the audio data cannot be played back accurately, resulting in phenomena such as audio-video asynchronization, dropped frames, and playback delays, so it is necessary to adjust the reference playback time of the audio data. That is, the adjustment of audio data playback is made by referring to the data of the reference playback time, without adjusting the data corresponding to the reference clock, i.e., the display time of the terminal is not adjusted.

Here, the reference offset time may be determined by the difference between the reference playback time and the desired playback time of the audio data, and the reference playback time is backed up to the difference so that the adjusted reference playback time is equal to or less than the desired playback time.

Here, since the decoding time of the audio data is at the millisecond level, the adjustment of the reference time is also a difference of milliseconds that the human body cannot easily perceive. For example, if the original reference playback time of the audio data is 7:500 ms, that is, the time displayed on the reference clock of the terminal device is exactly 7:00, and the reference offset time is 200 ms, the reference playback time after adjustment is 7:300 ms. At this time, the time displayed on the reference clock is exactly 7:00, and the millisecond-level adjustment of 200 ms is difficult for the user to perceive, but since the time of 200 ms cannot be ignored for audio decoding, if the original reference playback time exceeds the desired playback time, the audio data will not be accurately played. Therefore, the reference offset time is adjusted by that amount to adjust the reference playback time, making it possible to play the audio data, providing sufficient time for decoding the audio data, and reducing the performance requirements of the hardware product, making it difficult for the user to perceive the difference in time.

In some embodiments of the present application, the reference offset time of the current audio data may be determined based on the offset status of the past history data. That is, optionally, in some embodiments, "obtaining a reference offset time corresponding to the audio data" specifically means:
sampling at least one frame of historical audio data, said historical audio data being of the same data type as said current audio data;
obtaining a historical reference playback time and a historical desired playback time after decoding processing for each frame of the historical audio data of the at least one frame of the historical audio data;
determining an offset time of each frame of the at least one frame of the historical audio data based on the historical reference playback time and the historical desired playback time;
determining a reference offset time of the current audio data based on at least one of the offset times.

Here, the reference offset time to which the reference playback time of the current audio data should be adjusted is estimated based on obtaining the offset time of historical audio data of the same data type, and since the historical audio data with the same data type as the current audio data is sampled, obtaining the reference offset time is accurate.

In order to ensure the accuracy of the reference offset time, the sampled historical audio data may be exchanged according to a certain frequency, i.e., in some embodiments, "sampling at least one frame of historical audio data" specifically means:
obtaining historical audio data based on a predetermined frequency, the historical audio data including at least one frame of historical audio data;
and sampling the at least one frame of historical audio data.

Here, the different or new historical audio data has a certain timeliness and accuracy as a reference for obtaining the reference offset time.

Here, determining a reference offset time of the current audio data by averaging offset times corresponding to the historical audio data; that is, optionally, in some embodiments, "determining a reference offset time of the current audio data based on at least one of the offset times" specifically means:
determining an average value of at least one of the offset times to obtain an average offset time;
determining a reference offset time of the current audio data based on the average offset time.

Here, the average value indicates the overall offset of the historical audio data, which is the same data type as the current audio data, and this overall offset can reflect the adjustment range of the reference playback time of the current audio data, so taking the average offset time of the historical audio data as the reference offset time of the current audio data has a certain degree of accuracy and reference value.

Here, since the decoding time is at the millisecond level, in order to improve calculation efficiency, a time of 1 millisecond or less can be ignored, and therefore an offset value at an integer level can be adjusted for the offset time. That is, optionally, in some embodiments, "determining a reference offset time of the current audio data based on at least one of the offset times" specifically means:
determining an average value of at least one of the offset times to obtain an average offset time;
taking an integer number of the average offset times to obtain an average integer offset time corresponding to the historical audio data;
determining the average integer offset time as a reference offset time of the current audio data.

Here, the method of obtaining an integer may be a method of converting a decimal to an integer, for example, adjusting the value after the decimal point to 1, and making the reference playback time after adjustment shorter than the desired playback time, allowing the audio data to be played back smoothly.

Here, you can intuitively know the adjustment interval for the reference playback time of the audio data from the integer reference offset time.

Here, since the decoding time is at the millisecond level, in order to improve the calculation efficiency, the time of 1 millisecond or less can be ignored, i.e., only the integer part of each offset time can be taken into consideration for selecting the reference offset time, and the integer of each offset time is taken first, and the average value of these offset times is calculated. That is, optionally, in some embodiments, "determining the reference offset time of the current audio data based on at least one of the offset times" specifically means:
taking an integer number of the at least one offset time to obtain at least one integer offset time;
determining an average integer offset time of each frame of audio data of the at least one frame of audio data based on the at least one integer offset time;
determining the integer average offset time as a reference offset time of the current audio data.

Here, to ensure that the adjusted reference playback time is equal to or less than the desired playback time, each offset time may be converted from a decimal to an integer. For example, the decimal part of each offset data may be adjusted to 1, i.e., the integer part increases by 1 ms.

Here, the reference offset time indicates the basic offset data or overall offset of historical audio data that has the same data type as the current audio data, so that the reference playback time can generally be adjusted to be less than or equal to the desired playback time by adjusting the reference offset time relative to the reference playback time.

Here, the reference offset time is obtained based on obtaining an average value, and the average value of each audio data offset time represents the offset of the entire audio data set, so the reference offset time obtained by the above method can satisfy the adjustment needs of the reference playback time to a certain extent.

However, for audio data requiring higher accuracy, in addition to the reference offset time, an error offset time that still exists can be identified based on the adjustment of the reference offset time of the audio data based on the actual offset time corresponding to the audio data, and the reference playback time of the audio data can be adjusted simultaneously based on the reference offset time and the error offset time. That is, optionally, in some embodiments, "adjusting the decoding start reference time based on the reference offset time to obtain the decoding start adjustment time" can be
obtaining a reservation processing time for the audio data;
determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time;
determining an error offset time of the audio data based on the reference offset time and the actual offset time;
adjusting the decoding start reference time based on the reference offset time and the error offset time to obtain a decoding start adjusted time.

Here, since the reference offset time is calculated from the average offset time of the historical audio data, there will be an error even if the reference playback time is adjusted using the reference offset time. Therefore, the error offset time is determined using the actual offset time and the reference offset time. After the reference playback time is adjusted based on the reference offset time, further adjustment is made using the error offset time. This improves the accuracy of the adjustment of the reference playback time of the audio data, and ensures that the adjusted reference playback time will be less than or equal to the desired playback time.

For example, if the reference offset time is an integer, precision will be lost, so to meet the precision requirement, the actual audio data corresponding to the audio data of each frame is identified, the error offset time is identified based on the actual audio data and the reference offset time, the reference playback time is adjusted based on the reference offset time, and then further adjusted based on the error offset time.

Here, for audio data requiring higher accuracy, the offset time is adjusted based on the actual offset time after the audio data is decoded, and the reference offset time further includes the actual offset time. That is, optionally, in some embodiments, "adjusting the decoding start reference time based on the reference offset time to obtain the decoding start adjustment time" can be
obtaining a reservation processing time for the audio data;
determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time;
adjusting the decoding start reference time based on the actual offset time to obtain a decoding start adjusted time.

Here, after the audio data is decoded and before the audio data is output, the reference time for starting decoding of the audio data is adjusted based on the actual offset time of the audio data.

Here, the reserved processing time is a processing time reserved for decoding audio data, and the reserved processing time may be acquired according to the reference decoding start time and the desired playback time. That is, optionally, in some embodiments of the present application, "acquiring the reserved processing time of audio data" may be
The method includes specifying a reserved processing time for the audio data based on a reference time for starting decoding of the audio data and the desired playback time.

Here, the scheduled processing time for audio data can be obtained based on the difference between the desired playback time and the reference decoding start time, and the actual offset time for audio data decoding can be obtained based on a comparison between the scheduled processing time and the actual decoding processing time.

Here, the desired playback time of the audio data can be directly extracted from the data information of the audio data. That is, optionally, in some embodiments, "obtaining the desired playback time corresponding to the audio data" specifically means:
Extracting a desired playback duration of the audio data directly from the audio data.

Here, the reference offset time of the audio data may be determined based on the type of the audio data. That is, selectively, in some embodiments, "obtaining a reference offset time corresponding to the audio data" specifically means:
identifying an audio data type of the audio data;
Obtaining a mapping relationship set, the mapping relationship set including a mapping relationship between a predetermined audio data type and a predetermined reference offset time;
and obtaining a reference offset time corresponding to the audio data based on the mapping relationship set and the audio data type.

Here, the mapping relationship set may be determined based on a conventional relationship between historical audio data type and offset time. For example, a reference offset time corresponding to the data type of audio data is jointly determined based on statistics of historical audio data type and offset time.

Step 104: For each frame of audio data of the at least one frame of audio data, adjust the decode start reference time based on the reference offset time to obtain an adjusted decode start time.

Here, the reference time to start decoding before the audio data decoding process and the reference playback time after the decoding process are on the same timeline. In other words, a change in the reference time to start decoding results in a change in the reference playback time, so adjusting the reference time to start decoding when the decoding process started is equivalent to adjusting the reference playback time after the decoding process.

Here, reference is made to Fig. 3. Fig. 3 is a flow diagram of the reference playback time adjustment in the embodiment of the present invention, in which an adjustment can be made to the decode start reference time at the start of audio data processing, for example, after the decode start reference time is adjusted backward, a change adjustment is also made to the final reference playback time after audio data processing. That is, this corresponds to an update adjustment of the final reference playback time after the adjustment of the decode start reference time, and is specifically as follows.
Step 111: Obtain an audio data set, which is an audio data packet (TLV stream).
Step 112: When starting to decode audio data in the audio data packet, the real time of the reference clock of the terminal device at the start of decoding the audio data (i.e., the reference time to start decoding the audio data) and the actual decoding processing time are recorded.
Step 113: Obtain a desired playback time corresponding to the audio data.
Step 114: By adjusting the decoding start reference time based on the reference offset time and the error offset time, the sum of the adjusted decoding start adjustment time and the decoding actual processing time (i.e., the reference playback time) is equal to or shorter than the desired playback time corresponding to the audio data.
Step 115: The reference playback time after adjustment (i.e., after the audio data is decoded) is updated with the continuing timing, the reference playback time is changed to the desired playback time corresponding to the audio data, and the audio data is played back at this time.

Here, by adjusting the reference decode start time when decoding of the audio data starts, the hardware device that plays the audio data can obtain a reference playback time that is less than or equal to the desired playback time. In other words, after adjusting the reference decode start time, the reference playback time obtained by the hardware device after decoding is the adjusted reference playback time.

Step 105: Identify a reference playback time of the audio data based on the decode start adjustment time and the decode actual processing time, and the reference playback time is less than or equal to the desired playback time.

Here, the sum of the reference time to start decoding the audio data and the actual decoding processing time is the reference time after the audio data is decoded, and since it refers to the offset time of the historical audio data, by adjusting the reference time to start the current audio data, the reference playback time after the audio data is decoded becomes equal to or shorter than the desired playback time of the audio data.

Here, the reference playback time of the audio data is determined based on the time information before the audio data is decoded and the time information after the audio data is decoded. That is, optionally, in some embodiments, "determining the reference playback time of the audio data based on the decode start adjustment time and the decode actual processing time" specifically means:
obtaining a decoding start time and a decoding end time of a decoding process for the audio data;
determining a base reference playback time of the audio data based on the decoding start adjustment time and the decoding start time;
determining a reference playback time of the audio data based on the base reference playback time and the decoding end time.

Here, the base reference playback time of the audio data can be obtained from the difference between the decode start adjustment time and the decode start time, and the reference playback time of the audio data can be obtained from the sum of the base reference playback time and the decode end time. In other words, the difference between the reference playback time and the decode start adjustment time is equal to the difference between the decode end time and the decode start time.

Here, by obtaining the reference playback time using the base reference playback time, the adjustment of the reference offset time can be adjusted at the start of decoding the audio data. In other words, the reference time for starting decoding the audio data is adjusted.

Here, by adjusting the base reference playback time before the start of decoding the audio data, the adjusted decode start adjustment time can be synchronized with the final reference playback time.

Step 106: Update the reference playing time according to time change to obtain an adjusted reference playing time .

Here, the reference playback time is the reference playback time of the audio data, and when the reference playback time reaches the desired playback time, the audio data can be played back.

Here, the reference offset time of the current audio data can be inferred from the offset time of the historical audio data, so by adjusting it back to the reference playback time using the reference offset time, the reference playback time is adjusted so that it is not synchronized with the time of the reference clock and is below the desired playback time.

Step 107: When the adjusted reference playback time reaches the desired playback time, the audio data is played according to the terminal device.

Here, as the reference playback time continues to be clocked, the audio data can be played back after the timestamp corresponding to the reference playback time becomes the same as the timestamp of the desired playback time. Since the decoding of audio data is at the millisecond level, the time difference between the adjusted reference playback time and the desired playback time should also be at the millisecond level.

Here, since the difference in the timestamps corresponding to the desired playback times of each audio data in the audio data set is relatively constant, the difference in the reference playback times corresponding to each audio data after adjustment is also relatively constant, making continuous playback of audio possible.

Now, refer to FIG. 4. In the embodiment of the present application, the specific process of adjusting the reference playback time of the audio data according to the reference offset time is as follows:

Step 121: Obtain an audio data set, i.e., an audio data packet (TLV stream), obtain a timestamp of the reference time for starting decoding of the audio data from the audio data set, verify the timestamp of the reference time for starting decoding, and determine the validity of the timestamp data. If it is valid, proceed to step 122; if it is invalid, continue to step 121.

Step 122: Obtain a timestamp of the desired playback time of each frame of audio data from the audio data set, verify the timestamp of the desired playback time, and determine the validity of the timestamp data. If it is valid, proceed to step 123; if it is invalid, continue to step 122.

Step 123: Record the decoding start time of the audio data decoding, and verify the time stamp of the decoding start time to determine the validity of the time stamp data. If it is valid, proceed to step 124; if it is invalid, continue to step 123.
Step 124: A base reference playback time is obtained.
Step 125: Adjust the base reference playback time based on the reference offset time (effectively adjusting the decoding start reference time).
Step 126: The actual offset time is obtained.
Step 127: Based on the reference offset time and the actual offset time, determine the error offset time of the reference playback time, adjust the reference offset time, and then make further adjustments to the base reference playback time based on the error offset time (effectively making adjustments to the decoding start reference time), obtain the base reference playback adjustment time, and verify the validity of the base reference playback adjustment time. If it is invalid, proceed to step 127; if it is valid, proceed to step 128.
Step 128: Update the base reference playback adjustment time in the hardware device of audio playback, and determine the adjusted reference playback time according to the decoding actual processing time.

Here, in the adjustment process, the reference decode start time is first adjusted based on the reference offset time, and then the adjusted error offset time is determined based on the actual decode processing time. Then, by continuing the adjustment based on the error offset time, the accuracy of the adjustment of the reference decode start time, i.e., the accuracy of the final reference playback time of the audio data, is improved.

In the embodiment of the present application, the reference playback time of the audio data is shortened, so that the reference playback time changes over time, and the audio data is played back after it reaches the desired playback time after the change; that is, after the reference playback time of the audio data meets the desired playback time, the audio data can be played back synchronously. Of these, the reference playback time is shortened by shortening the reference decoding start time, but the actual decoding processing time of the audio data is not affected or limited. Therefore, shortening the reference playback time corresponding to the audio data provides sufficient decoding time for the decoding processing of the audio data, and avoids phenomena such as audio-video asynchrony, dropped frames, or playback delays in the playback of the audio data due to the decoding processing time being too long.

To better implement the audio playback method described in the present disclosure, the present disclosure also provides an audio playback device according to the audio playback method. Here, the meanings of the nouns are the same as those in the above-mentioned audio playback method, and specific implementation details may refer to the description in the embodiment of the method.

Please refer to Fig. 5. Fig. 5 is a schematic block diagram of an audio playback device provided in the present application, which includes a decoding module 201, a recording module 202, an acquisition module 203, an adjustment module 204, a determination module 205, a timing module 206, and a playback module 207. Specifically, the following is included:
The decoding module 201 is adapted to decode an audio data set to be played on a terminal device, said audio data set comprising at least one frame of audio data.
Here, the audio data received and played by the terminal device is often coded and compressed data. For example, in digital television, the transmitted packet TLV is a packet of ISDB S3 standard, and the TLV packet corresponding to a 4K super high vision program is generally very large in data amount of the program content, so in order to improve the transmission efficiency, the audio data is generally coded and compressed, and the audio data received by the terminal device is compressed data to be decoded, and the terminal device can play the audio data after decoding the audio data.
The recording module 202 records a reference time for starting decoding and an actual decoding process time for each frame of the audio data of the at least one frame of audio data.
Here, the decode start reference time is synchronized with a reference clock of the terminal device, that is, when the decoding process of the audio data is started, the time of the reference clock is the decode start reference time of the audio data.
Here, the actual audio data decoding processing time is the actual time required for the audio data decoding processing.

Optionally, in some embodiments of the present application, the recording module 202 may further include:
performing timing feedback based on a crystal oscillation to obtain a decoding start time and a decoding end time of the audio data;
and determining an actual processing time for decoding the audio data based on the decoding start time and the decoding end time.

Here, the reference time is updated by receiving the time from the reference clock, but since the process of transmitting and updating data consumes time, it is inaccurate to obtain the actual decoded time of the audio data based on the reference time. Therefore, in this application, the timing of the actual decoded time of the audio data is realized based on a quartz crystal oscillation timing device, and it is more accurate to use a timing process using quartz crystal oscillation feedback.

The acquisition module 203 acquires the reference offset time and desired playback time corresponding to the audio data.

Here, if the reference playback time of the audio data is longer than the desired playback time, the audio data cannot be played back accurately, resulting in phenomena such as audio-video asynchronization, dropped frames, and playback delays, so it is necessary to adjust the reference playback time of the audio data. That is, the adjustment of audio data playback is made by referring to the data of the reference playback time, without adjusting the data corresponding to the reference clock, i.e., the display time of the terminal is not adjusted.

Here, the reference offset time may be determined by the difference between the reference playback time and the desired playback time of the audio data, and the reference playback time is backed up to the difference so that the adjusted reference playback time is equal to or less than the desired playback time.

Here, since the decoding time of the audio data is at the millisecond level, the adjustment of the reference time is also a difference of milliseconds that the human body cannot easily perceive. For example, if the original reference playback time of the audio data is 7:500 ms, that is, the time displayed on the reference clock of the terminal device is exactly 7:00, and the reference offset time is 200 ms, the reference playback time after adjustment is 7:300 ms. At this time, the time displayed on the reference clock is exactly 7:00, and the millisecond-level adjustment of 200 ms is difficult for the user to perceive, but since the time of 200 ms cannot be ignored for audio decoding, if the original reference playback time exceeds the desired playback time, the audio data will not be accurately played. Therefore, the reference offset time is adjusted by that amount to adjust the reference playback time, making it possible to play the audio data, providing sufficient time for decoding the audio data, and reducing the performance requirements of the hardware product, making it difficult for the user to perceive the difference in time.

Optionally, in some embodiments of the present application, the audio data includes current audio data, and the acquisition module 203:
a sampling unit for sampling at least one frame of historical audio data, said historical audio data being of the same data type as said current audio data;
a first acquiring unit for acquiring a historical reference playback time and a desired playback time after decoding processing for each frame of the historical audio data of the at least one frame of the historical audio data;
a first determination unit for respectively determining an offset time of each frame of the historical audio data of the at least one frame according to the historical reference playback time and the desired playback time;
and a second identifying unit for identifying a reference offset time of the current audio data based on at least one of the offset times.

Here, the reference offset time to which the reference playback time of the current audio data should be adjusted is estimated based on obtaining the offset time of historical audio data of the same data type, and since the historical audio data with the same data type as the current audio data is sampled, obtaining the reference offset time is accurate.

Optionally, in some embodiments of the present application, the second identification unit is
a first specifying subunit for specifying an average value of at least one of the offset times to obtain an average offset time;
and a second identifying subunit for identifying a reference offset time of the current audio data based on the average offset time.

Here, the average value indicates the overall offset of the historical audio data, which is the same data type as the current audio data, and this overall offset can reflect the adjustment range of the reference playback time of the current audio data, so taking the average offset time of the historical audio data as the reference offset time of the current audio data has a certain degree of accuracy and reference value.

Optionally, in some embodiments of the present application, the second identification unit is
determining an average value of at least one of the offset times to obtain an average offset time;
taking an integer number of the average offset times to obtain an average integer offset time corresponding to the historical audio data;
The average integer offset time is determined to be a reference offset time of the current audio data, and is further used.

Optionally, in some embodiments of the present application, the second identification unit is
taking an integer number of the at least one offset time to obtain at least one integer offset time;
determining an average integer offset time of each frame of audio data of the at least one frame of audio data based on the at least one integer offset time;
The integer average offset time is further used to identify the integer average offset time as a reference offset time of the current audio data.

Optionally, in some embodiments of the present application, the sampling unit comprises:
obtaining historical audio data based on a predetermined frequency, the historical audio data including at least one frame of historical audio data;
Sampling the at least one frame of historical audio data is further utilized.

Optionally, in some embodiments of the present application, the acquisition module 203 may further include:
It is used to directly extract the desired playback time of the audio data from the audio data.

Optionally, in some embodiments of the present application, the acquisition module 203 may further include:
identifying an audio data type of the audio data;
Obtaining a mapping relationship set, the mapping relationship set including a mapping relationship between a predetermined audio data type and a predetermined reference offset time;
Obtaining a reference offset time corresponding to the audio data based on the mapping relationship set and the audio data type is further used.

The adjustment module 204 adjusts the decode start reference time for each frame of audio data of the at least one frame of audio data based on the reference offset time to obtain an adjusted decode start time.

Here, the adjustment of the reference decoding start time may be based on a reference offset time, or may be based on an actual offset time, or the error offset time may be adjusted based on the adjustment of the reference offset time, thereby ensuring the accuracy of the adjustment of the reference playback time.

Optionally, in some embodiments of the present application, the reference offset time includes an actual offset time, and the adjustment module 204:
a second acquiring unit for acquiring a reservation processing time of the audio data;
a third determination unit for determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time;
and a third obtaining unit for adjusting the decoding start reference time based on the actual offset time to obtain a decoding start adjustment time.

Optionally, in some embodiments of the present application, the second acquisition unit is
The third specifying subunit specifies a reservation processing time for the audio data based on the reference time for starting decoding of the audio data and the desired playback time.

Optionally, in some embodiments of the present application, the second acquisition unit is
obtaining a reservation processing time for the audio data;
determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time;
determining an error offset time of the audio data based on the reference offset time and the actual offset time;
The decoding start reference time is adjusted based on the reference offset time and the error offset time to obtain a decoding start adjustment time.

The determination module 205 determines the reference playback time of the audio data based on the decode start adjustment time and the actual decode processing time.

Optionally, in some embodiments of the present application, the identification module 205 may include:
obtaining a decoding start time and a decoding end time of a decoding process for the audio data;
determining a base reference playback time of the audio data based on the decoding start adjustment time and the decoding start time;
and determining a reference playback time of the audio data based on the base reference playback time and the decoding end time.

The timing module 206 updates the reference play time according to the time change to obtain an adjusted reference play time .

When the adjusted reference playback time reaches the desired playback time, the playback module 207 plays the audio data according to the terminal device.

In this embodiment, first, the decoding module 201 performs a decoding process on the audio data acquired by the terminal device, and at the same time, the recording module 202 records the decoding start reference playback time before the audio data decoding process and the decoding actual processing time of the decoding process; then, the acquisition module 203 extracts the desired playback time of each frame audio data and the reference offset time corresponding to each frame audio data from the audio data; then, the adjustment module 204 adjusts the decoding start reference playback time (adjusts it to be smaller) based on the reference offset time, so that the decoding start adjustment time is not synchronized with the reference clock time; and, based on the decoding start adjustment time and the decoding actual processing time, the identification module 204 identifies the reference playback time after the audio data decoding process (i.e., the reference playback time is determined based on the decoding start adjustment time and the decoding actual processing time, and is not synchronized with the reference clock time); at the same time, the timing module 206 retimes the adjusted reference playback time, and when the current time corresponding to the retimed reference playback time is equal to the desired playback time of the audio data, the playback module 207 plays the corresponding audio data. In the embodiment of the present application, the reference playback time of the audio data is reduced, so that the reference playback time is changed over time, and the audio data is played back after the reference playback time reaches the desired playback time after the change; that is, after the reference playback time of the audio data meets the desired playback time, the audio data can be played back synchronously. Among these, the reference playback time is reduced by reducing the reference decoding start time, but the actual decoding processing time of the audio data is not affected or limited. Therefore, reducing the reference playback time corresponding to the audio data provides sufficient decoding time for the decoding processing of the audio data, and prevents phenomena such as audio-video asynchronous, dropped frames, or playback delays from occurring in the playback of the audio data due to the decoding processing time being too long.

The present application also provides an electronic device, and as shown in FIG. 6, a structural schematic diagram of the electronic device according to the present application is provided, which is specifically described as follows:
The electronic device may include components such as a processor 401 of one or more processing cores, a memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that the electronic device structure shown in FIG. 6 does not constitute a limitation of the electronic device, which may include more or fewer components than shown, combine some components, or include a different arrangement of components, including:
The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect each part of the entire electronic device, execute or run software programs and/or modules stored in the memory 402, and call up data stored in the memory 402 to execute various functions of the electronic device and process data, thereby monitoring the entire electronic device. Optionally, the processor 401 may include one or more processing cores. Preferably, the processor 401 may integrate an application processor that mainly processes an operating system, a user interface, and application programs, and a modulation/demodulation processor that mainly processes wireless communication. Note that the above-mentioned modulation/demodulation processor does not have to be integrated into the processor 401.
The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and decoding processes by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area that can store an operating system, an application program required for at least one function (e.g., a sound playback function, an image playback function, etc.), and a storage data area. The storage data area may store data created according to the use of the electronic device, etc. The memory 402 may also include a high-speed random access memory, and may further include at least one non-volatile memory such as a magnetic disk storage device, a flash memory device, or other volatile solid-state storage device. Correspondingly, the memory 402 may further include a memory controller that provides access to the memory 402 by the processor 401.
The electronic device includes a power source 403 that provides power to the various components, and preferably the power source 403 is logically connected to the processor 401 via a power management system, thereby enabling functions such as charging, discharging, and managing power consumption via the power management system. The power source 403 may include any of the following components: one or more DC or AC power sources, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, etc.

The electronic device may also include an input unit 404 that is used to receive input numeric or textual information and generate keyboard, mouse, joystick, optical or trackball signal inputs associated with user settings and function control.

Although not shown, the electronic device may also include a display unit and the like, which will not be described here. Specifically, in this embodiment, the processor 401 of the electronic device loads executable files corresponding to the processes of one or more application programs into memory 402 in accordance with the following instructions, and the processor 401 executes the application programs stored in memory 402, thereby achieving various functions as follows:

Decode an audio data set to be played on a terminal device, the audio data set including at least one frame of audio data; record a reference decode start time and an actual decode processing time of a decoding process for each frame of audio data of the at least one frame of audio data; obtain a reference offset time and a desired playback time corresponding to the audio data; adjust the reference decode start time for each frame of audio data of the at least one frame of audio data based on the reference offset time to obtain an adjusted decode start time; determine a reference playback time of the audio data based on the adjusted decode start time and the actual decode processing time, the reference playback time being less than or equal to the desired playback time; update the reference playback time based on a time change to obtain an adjusted reference playback time ; and play back the audio data according to the terminal device when the adjusted reference playback time reaches the desired playback time.

For specific implementations of the above operations, please refer to the above embodiments, and duplicated descriptions will not be given here.
By reducing the reference playback time of the audio data, the reference playback time is changed with time, and after the change, the audio data is played back when the desired playback time is reached, i.e., after the reference playback time of the audio data meets the desired playback time, the audio data can be played back synchronously. Among these, the reference playback time is reduced by reducing the reference decoding start time, but the actual decoding processing time of the audio data is not affected or limited. Therefore, reducing the reference playback time corresponding to the audio data provides a sufficient decoding time for the decoding processing of the audio data, and prevents phenomena such as audio-picture asynchronous, dropped frames, or playback delays from occurring in the playback of the audio data due to the decoding processing time being too long.

Those skilled in the art will appreciate that all or part of the steps in the various methods of the above-described embodiments may be performed by instructions stored in a computer-readable storage medium and loaded and executed by a processor, or by instructions that control associated hardware.

The present application therefore provides a storage medium having stored thereon a number of instructions that can be loaded by a processor to perform steps in any of the audio playback methods described herein. For example, the instructions can perform steps such as:

Decode an audio data set to be played on a terminal device, the audio data set including at least one frame of audio data; record a reference decode start time and an actual decode processing time of a decoding process for each frame of audio data of the at least one frame of audio data; obtain a reference offset time and a desired playback time corresponding to the audio data; adjust the reference decode start time for each frame of audio data of the at least one frame of audio data based on the reference offset time to obtain an adjusted decode start time; determine a reference playback time of the audio data based on the adjusted decode start time and the actual decode processing time, the reference playback time being less than or equal to the desired playback time; update the reference playback time based on a time change to obtain an adjusted reference playback time ; and play back the audio data according to the terminal device when the adjusted reference playback time reaches the desired playback time.
For specific implementations of the above operations, please refer to the above embodiments, and duplicated descriptions will not be given here.

Here, the storage medium may include a read only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, or the like.

The instructions stored on the storage medium can perform steps in any of the audio playback methods provided in this specification, and can therefore achieve beneficial effects that can be achieved by any of the audio playback methods provided in this specification, which will not be described in detail herein with reference to the aforementioned embodiments.
The audio playback method, device, electronic device, and storage medium provided in this specification have been described in detail above, and the specification has used specific examples to explain the principles and embodiments of the present invention. However, the description of the above embodiments is only used to help understand the method of the present invention and its core concept. At the same time, for those skilled in the art, there are changes in the specific embodiments and application scope based on the concept of the present invention, so the contents of this specification should not be understood as limitations of the present invention.
The above is only a preferred embodiment of the present application, which does not limit the patent scope of the present application. Any equivalent structural or flow conversion made by utilizing the contents of the description and accompanying drawings of the present application, or operation directly or indirectly in other related technical fields, is also within the patent protection scope of the present application.

Claims (20)

1. A method for audio reproduction, comprising:
decoding an audio data set to be played on a terminal device, the audio data set including at least one frame of audio data;
recording a reference time for starting decoding and an actual decoding process time for each frame of the audio data of the at least one frame of audio data;
obtaining a reference offset time and a desired playback time corresponding to the audio data;
adjusting the decode start reference time for each frame of the audio data of the at least one frame based on the reference offset time to obtain an adjusted decode start time;
determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time, and determining whether the reference playback time is equal to or shorter than the desired playback time;
updating the reference play time based on a time change to obtain an adjusted reference play time ;
When the adjusted reference playback time reaches a desired playback time, the audio data is played back according to the terminal device.
13. An audio playback method comprising: The audio data includes current audio data, and obtaining a reference offset time corresponding to the audio data includes:
sampling at least one frame of historical audio data, said historical audio data being of the same data type as said current audio data;
obtaining a historical reference playback time and a historical desired playback time after decoding processing for each frame of the historical audio data of the at least one frame of the historical audio data;
determining an offset time of each frame of the at least one frame of the historical audio data based on the historical reference playback time and the historical desired playback time;
determining a reference offset time of the current audio data based on at least one of the offset times;
2. The audio reproducing method according to claim 1. determining a reference offset time of the current audio data based on at least one of the offset times,
determining an average value of at least one of the offset times to obtain an average offset time;
determining a reference offset time of the current audio data based on the average offset time.
3. The audio reproducing method according to claim 2. determining a reference offset time of the current audio data based on at least one of the offset times,
determining an average value of at least one of the offset times to obtain an average offset time;
taking an integer number of the average offset times to obtain an average integer offset time corresponding to the historical audio data;
determining the average integer offset time to a reference offset time of the current audio data;
3. The audio reproducing method according to claim 2. determining a reference offset time of the current audio data based on at least one of the offset times,
taking an integer number of at least one of the offset times to obtain at least one integer offset time;
determining an average integer offset time of each frame of audio data of the at least one frame of audio data based on at least one of the integer offset times;
determining the integer average offset time as a reference offset time of the current audio data;
3. The audio reproducing method according to claim 2. Sampling at least one frame of historical audio data
obtaining historical audio data based on a predetermined frequency, the historical audio data being historical audio data of at least one frame;
and sampling the at least one frame of historical audio data.
3. The audio reproducing method according to claim 2. The reference offset time includes an actual offset time, and the decoding start reference time is adjusted based on the reference offset time to obtain a decoding start adjustment time.
obtaining a reservation processing time for the audio data;
determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time;
adjusting the decoding start reference time based on the actual offset time to obtain a decoding start adjustment time.
2. The audio reproducing method according to claim 1. The acquiring of the reservation processing time of the audio data includes:
specifying a reservation processing time for the audio data based on a reference time for starting decoding of the audio data and the desired playback time;
8. The audio reproducing method according to claim 7. Adjusting the decoding start reference time based on the reference offset time to obtain a decoding start adjustment time,
obtaining a reservation processing time for the audio data;
determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time;
determining an error offset time of the audio data based on the reference offset time and the actual offset time;
adjusting the decoding start reference time based on the reference offset time and the error offset time to obtain a decoding start adjustment time.
2. The audio reproducing method according to claim 1. Recording an actual decoding time of the decoding process for each frame of the audio data of the at least one frame of the audio data
performing timing feedback based on a crystal oscillation to obtain a decoding start time and a decoding end time of the audio data;
determining an actual processing time for decoding the audio data based on the decoding start time and the decoding end time;
2. The audio reproducing method according to claim 1. determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time,
obtaining a decoding start time and a decoding end time of a decoding process for the audio data;
determining a base reference playback time of the audio data based on the decoding start adjustment time and the decoding start time;
determining a reference playback time of the audio data based on the base reference playback time and the decoding end time,
2. The audio reproducing method according to claim 1. Obtaining a desired playback time corresponding to the audio data includes:
extracting a desired playback duration of said audio data directly from said audio data;
2. The audio reproducing method according to claim 1. Obtaining a reference offset time corresponding to the audio data includes:
identifying an audio data type of the audio data;
Obtaining a mapping relationship set, the mapping relationship set including a mapping relationship between a predetermined audio data type and a predetermined reference offset time;
and obtaining a reference offset time corresponding to the audio data based on the mapping relationship set and the audio data type.
2. The audio reproducing method according to claim 1. 1. An audio playback device, comprising:
a decoding module for decoding an audio data set to be played on a terminal device, said audio data set including at least one frame of audio data;
a recording module for recording a reference time for starting a decoding process and an actual decoding process time of each frame of the audio data of the at least one frame of the audio data;
an acquisition module for acquiring a reference offset time and a desired playback time corresponding to the audio data;
an adjustment module for adjusting the decoding start reference time for each frame of the audio data of the at least one frame based on the reference offset time to obtain a decoding start adjustment time;
a determining module for determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time;
a timing module for updating the reference play time according to a time change to obtain an adjusted reference play time ;
a playback module for playing the audio data according to a terminal device when the adjusted reference playback time reaches a desired playback time;
1. An audio playback device comprising: The audio data includes current audio data, and the acquisition module is further configured to:
a sampling unit for sampling at least one frame of historical audio data, said historical audio data being of the same data type as said current audio data;
a first acquiring unit for acquiring a historical reference playback time and a historical desired playback time after decoding processing for each frame of the historical audio data of the at least one frame ;
a first determination unit for respectively determining an offset time of each frame of the historical audio data of the at least one frame according to the historical reference playback time and the desired playback time;
a second determining unit for determining a reference offset time of the current audio data based on at least one of the offset times;
15. An audio reproducing device according to claim 14. The second identification unit is
a first specifying subunit for specifying an average value of at least one of the offset times to obtain an average offset time;
a second specifying subunit for specifying a reference offset time of the current audio data based on the average offset time;
16. An audio reproducing device according to claim 15. the reference offset time includes an actual offset time, and the adjustment module
a second acquiring unit for acquiring a reservation processing time of the audio data;
a third determination unit for determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time;
a third obtaining unit for adjusting the decoding start reference time according to the actual offset time to obtain a decoding start adjustment time;
15. An audio reproducing device according to claim 14. The second acquisition unit is
a third specifying subunit for specifying a reservation processing time of the audio data based on a reference time for starting decoding of the audio data and the desired playback time;
20. An audio reproducing device according to claim 17. An electronic device including a memory, a processor, and a computer program stored on the memory and executable by the processor, the program, when executed by the processor,
decoding an audio data set to be played on a terminal device, the audio data set including at least one frame of audio data;
recording a reference time for starting decoding and an actual decoding process time for each frame of the audio data of the at least one frame of audio data;
obtaining a reference offset time and a desired playback time corresponding to the audio data;
adjusting the decode start reference time for each frame of the audio data of the at least one frame based on the reference offset time to obtain an adjusted decode start time;
determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time;
updating the reference playing time based on the time change to obtain an adjusted reference playing time ;
When the adjusted reference playback time reaches a desired playback time, the audio data is played back according to the terminal device.
1. An electronic device comprising: A computer-readable storage medium having a computer program stored thereon, the computer program being capable of, when executed by a processor,
decoding an audio data set to be played on a terminal device, the audio data set including at least one frame of audio data;
recording a reference time for starting decoding and an actual decoding process time for each frame of the audio data of the at least one frame of audio data;
obtaining a reference offset time and a desired playback time corresponding to the audio data;
adjusting the decode start reference time for each frame of the audio data of the at least one frame based on the reference offset time to obtain an adjusted decode start time;
determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time;
updating the reference play time based on a time change to obtain an adjusted reference play time ;
When the adjusted reference playback time reaches a desired playback time, the audio data is played back according to the terminal device.
A computer-readable storage medium comprising:

Images