WO2017186015A1 - Method and device for dubbing audio-visual digital media - Google Patents

Abstract

Disclosed are a method and device for dubbing audio-visual digital media, for use in resolving the problem in the prior art of lowered user experience due to low convenience in dubbing audio-visual digital media. The method comprises: introducing an audio into an editing interface; generating, according to the introduced audio, an audio view control used for indicating the audio, and displaying the audio view control in the editing interface, the position of the audio view control reflecting a matching relationship between the audio and the audio-visual digital media; and when a position movement instruction for the audio view control is received, moving the audio view control according to the position movement instruction, so as to adjust the matching relationship.

Description

Method and device for dubbing audiovisual digital media Technical field

The present application relates to the field of multimedia technologies, and in particular, to a method and apparatus for dubbing audiovisual digital media.

Background technique

With the rapid development of information technology and multimedia technology, the information presentation on the network is also increasingly rich. Compared with the previous text-based or picture-based, relatively static information presentation methods, many users prefer audio-visual digital media. Relatively dynamic information presents information. Among them, the audiovisual digital media may include video, animation, and the like.

In the prior art, the content of the audiovisual digital media can be enriched by adding dubbing to the audiovisual digital media. However, if the user is not satisfied with the existing dubbing effect, for example, the video and audio are not well matched, etc., the dubbing in the audiovisual digital media cannot be adjusted.

It can be seen that in the prior art, the convenience of dubbing the audiovisual digital media is poor, and the user experience is reduced.

Summary of the invention

The embodiment of the present invention provides a method and a device for dubbing audiovisual digital media, which are used to solve the problem of poor convenience in audio recording of audiovisual digital media in the prior art, and reduce user experience.

A method for dubbing audiovisual digital media provided by an embodiment of the present application includes:

Import an audio in the editing interface;

Generating an audio view control for representing the audio according to the imported audio, and displaying the audio view control in the editing interface, the position of the audio view control reflecting the audio and the audiovisual number Media cooperation relationship;

When the position move instruction for the audio view control is received, the audio view control is moved to adjust the mating relationship according to the position move instruction.

An apparatus for dubbing an audiovisual digital medium provided by an embodiment of the present application includes:

An audio import module for importing an audio in the editing interface;

An audio view control module, configured to generate an audio view control for representing the audio according to the imported audio, and display the audio view control in the editing interface, where a position of the audio view control reflects The relationship between the audio and the audiovisual digital media;

And an audio adjustment module, configured to: when receiving a position move instruction for the audio view control, move the audio view control to adjust the mating relationship according to the position move instruction.

The embodiment of the present application can visualize the audio used for dubbing the audiovisual digital media into an audio view control by using at least one of the foregoing technical solutions, and the user can adjust the audiovisual digital media by moving the audio view control by, for example, dragging. The cooperation relationship with the audio can improve the convenience of dubbing the audiovisual digital media and improve the user experience, and therefore, the problems in the prior art can be partially or completely solved.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the present application, and are intended to be a part of this application. In the drawing:

1 is a schematic flowchart of a method for dubbing audiovisual digital media according to an embodiment of the present application;

FIG. 2 is a schematic diagram of an editing interface in an actual application scenario according to an embodiment of the present disclosure;

3 is a partial schematic diagram of an editing interface according to an embodiment of the present application;

4 is a partial schematic diagram of another editing interface provided by an embodiment of the present application;

FIG. 5 is a partial schematic diagram of still another editing interface according to an embodiment of the present application;

FIG. 6 is a partial schematic diagram of still another editing interface according to an embodiment of the present application;

FIG. 7 is a schematic diagram of adjusting a coordination relationship between audiovisual digital media and audio according to an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of another coordination relationship between audiovisual digital media and audio according to an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of another coordination relationship between audiovisual digital media and audio according to an embodiment of the present disclosure;

FIG. 10 is a schematic diagram of adjusting a matching relationship between audiovisual digital media and audio according to an embodiment of the present disclosure;

FIG. 11 is a schematic structural diagram of an apparatus for dubbing audiovisual digital media according to an embodiment of the present application.

detailed description

The technical solutions of the present application will be clearly and completely described in the following with reference to the specific embodiments of the present application and the corresponding drawings. It is apparent that the described embodiments are only a part of the embodiments of the present application, and not all of them. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

The scheme of the present application will be described below.

FIG. 1 is a schematic flowchart diagram of a method for dubbing audiovisual digital media according to an embodiment of the present disclosure, where the audiovisual digital media may be video or animation.

In the embodiment of the present application, the audiovisual digital media itself may not include audio, and the audiovisual digital media may be dubbed by the method; the audiovisual digital media itself may also contain audio, and the audiovisualization may be performed by the method. Digital media adds new audio, or if the audio contained in the audiovisual digital media itself can be separated, the audio contained in the audiovisual digital media itself can also be adjusted by this method, etc., and the enumerated operations can belong to The audio-visual digital media is dubbed as described in this application.

The execution subject of the flow in FIG. 1 may be an audio and video related device. The device may specifically be: a personal computer, a large and medium-sized computer, a computer cluster, a mobile phone, a tablet computer, a smart watch, a car mobile station, and the like. The enumerated devices are merely examples of the execution subject and do not constitute a limitation on the execution subject.

The process in Figure 1 can include the following steps:

S101: Import an audio in the editing interface.

In the embodiment of the present application, the editing interface may be an operation interface used when dubbing audiovisual digital media.

In addition to the imported audio, there are audiovisual digital media to be dubbed in the editing interface.

For example, you can import audio-visual digital media into the editing interface and import audio. Alternatively, you can import audio into the editing interface and then import the audio-visual digital media. Alternatively, you can use the audio-visual digital media through predetermined operations. The other interface switches to the editing interface. After switching, the audiovisual digital media is already available in the editing interface, and then the audio is imported in the editing interface.

In the embodiment of the present application, the audio may be audio recorded by a user, or may be a soundtrack downloaded from the Internet or audio selected from an audio library, etc., and the source and specific content of the audio in the present application. Not limited.

The one audio may refer to a piece of audio that is continuous in time. When it is necessary to configure multi-segment audio for audiovisual digital media, the steps in FIG. 1 can be performed for each piece of audio. For convenience of description, the following mainly describes the scheme of the present application by taking an audio of the audiovisual digital medium as an example.

S102: Generate an audio view control for representing the audio according to the imported audio, and display the audio view control in the editing interface, where the position of the audio view control reflects the audio and the audiovisual The relationship between digital media.

In the embodiment of the present application, the imported audio can be converted into a Visualized audio view control. The present application does not limit the display shape of the audio view control. The audio view control can generally be displayed as a rectangle. Of course, it can also be displayed as a graphic other than a rectangle depending on the specific business situation.

In the embodiment of the present application, an audio view control for representing the audio may be generated according to data of the imported audio; or an audio format conversion may be performed on the imported audio, and the representation data may be generated according to the converted data. Audio view control for audio.

For example, the format of the imported audio can be converted to a Pulse Code Modulation (PCM) format at a certain sampling rate, and the value of each sample point of the converted audio can be read and normalized, and the value can be converted. The amplitude of each sample point for the view (ie, waveform).

In the embodiment of the present application, the cooperation relationship between the audio and the audiovisual digital media can indicate how the audio and the audiovisual digital media cooperate with each other in the playing time during synchronization, that is, the time axis and audiovisual of the audio. How does the timeline of digital media work together? Specifically, in synchronous playback, is the audio-visual digital media first broadcasted, or is the audio first broadcasted, or the audio-visual digital and audio simultaneously broadcast; when not simultaneously broadcast, how long does the first broadcast party play? After that, the other party starts broadcasting again; when the playback ends after the broadcast; and so on. It should be noted that the synchronous play means that the audio and the audio-visual digital media are played together according to a certain cooperation relationship, and they can be broadcasted at the same time or at different times.

In the embodiment of the present application, the position of the audio view control can be used to intuitively reflect the cooperation relationship between the audio and the audiovisual digital media. Specifically, a play time progress bar such as an audiovisual digital media or an audiovisual digital media view control for representing the audiovisual digital media may also be displayed in the editing interface. When the position of the audio view control is different, the audio view control may be Aligning to different portions of the play time progress bar, or aligning to different portions of the audiovisual digital media view control, the alignment relationship that can be determined by the position of the audio view control can be directly mapped to the mating relationship Or indirect mapping, the location of the audio view control The cooperation relationship can be reflected.

S103: When receiving a position move instruction for the audio view control, move the audio view control to adjust the mating relationship according to the position move instruction.

In the embodiment of the present application, the position movement instruction may be sent by the user by dragging the operation of the audio view control, wherein, for the touch screen device, the user may directly move the audio view control by means of a finger drag, for the non-touch screen Device, the user can move the audio view control by clicking and dragging with the mouse.

The location move command can also be sent by the user by clicking on a predetermined control. For example, suppose that there are two predetermined controls, a "left shift" button control and a "right shift" button control, the user can send a left shift position movement instruction for the audio view control by clicking the "left shift" button control to make the audio The view control is accordingly shifted left by a preset unit distance; similarly, the user can move the audio view control to the right by a predetermined unit distance by clicking the "Shift Right" button control.

In the embodiment of the present application, when the user drags, when the audio view control is moved, the audio view control can be completely moved according to the user's drag track. However, in practical applications, it is also possible to restrict the movable range of the audio view control in advance. In this case, the audio view control may not be moved completely according to the user's drag track, for example, when it is movable. When the range is only one audio track, when the user's drag track deviates from the audio track, the audio view control can be moved on the audio track only according to the component direction of the user's drag track on the audio track.

Further, since the audio view control is visible, it is convenient for the user to drag the audio view control relatively freely according to his own idea, and when the position of the audio view control changes, the cooperation relationship changes accordingly, thereby realizing The adjustment of the relationship between audio and audiovisual digital media.

Through the above method, the audio used for dubbing the audiovisual digital media can be visualized as an audio view control, and the user can move the audio view control by means such as dragging to adjust the cooperation relationship between the audiovisual digital media and the audio. Therefore, the convenience in dubbing audio-visual digital media can be improved, and the user experience can be improved, and therefore, problems in the prior art can be partially or completely solved.

Based on the above method, the embodiments of the present application further provide some specific implementation manners of the foregoing methods, and an extended solution, which will be described below.

In the embodiment of the present application, for the step S101, before generating the audio view control for indicating the audio, performing: receiving the audiovisual digital media, turning on a control instruction of the audio editing mode; according to the control instruction Opening an audio editing mode for the audiovisual digital media and displaying the editing interface. Taking a touch screen device as an example, the user can send the control of turning on the audio editing mode by performing corresponding hardware operations on the touch screen (for example, physical buttons on the edge of the touch screen, etc.) or software (for example, an application installed in the touch device). instruction.

It should be noted that, if the audio editing mode is not enabled, after the audio is imported, the imported audio can be synchronously played from the current playback position of the audiovisual digital media, or from the beginning of the audiovisual digital media. Play synchronously in a matching manner.

In the embodiment of the present application, in order to facilitate user operations, the editing interface may include:

An audio track on which the audio view control is located.

The audio track can be used to limit the movable range of the audio view control. In this case, for the step S103, the audio view control is moved to adjust the cooperation relationship according to the position movement instruction, which may specifically include: The position movement instruction moves the audio view control on the audio track to adjust the mating relationship.

The audio track itself may have a time axis, which may be visualized into the audio track itself, in which case the length of the audio track may characterize the length of time of the time axis, The following examples are all based on this case.

As mentioned above, the mating relationship can indicate how the time axis of the audio and the time axis of the audiovisual digital medium match when the audio and audiovisual digital media are played simultaneously. For the audio view control used to represent the audio, the audio view control also has the timeline of the audio accordingly. When the audio view control is in an audio track, the timeline of the audio is: in the timeline of the audio track, the audio view control falls on the portion of the audio track. This section mainly explains the relationship between the audio view control and the time axis of the audio, and the relationship between the audio timeline and the audio track.

Further, the time axis of the audiovisual digital media can be aligned or fully aligned with the portion of the audio track; alternatively, the time axis of the audiovisual digital media can be the audio track itself. The embodiment of the present application is mainly based on the former case. When the audio view control moves on the audio track, when the position changes, the time axis of the audio is aligned with the time axis of the audiovisual digital media. The ground changes, so that the interaction between the audio and the audiovisual digital media can be adjusted by moving the audio view control, and during the adjustment process, the user can visually see the time axis of the audio, which is convenient for the user to operate. This section mainly explains the relationship between the timeline of the audiovisual digital media and the audio track, and combines the previous paragraph to explain why the audio and audiovisual digital media can be adjusted by moving the audio view control on the audio track. The relationship of cooperation.

The concept of "alignment" is mentioned above, and for ease of understanding, "alignment" is explained here. In the embodiment of the present application, “alignment” may be specifically divided into alignment in terms of time and alignment in length.

For time alignment, for example, "the time axis of the audiovisual digital media can be aligned or fully aligned with the audio track" as described above is a temporal alignment. Suppose that some two are aligned in terms of time, the specific meaning is: the duration of the two (that is, the length of the time axis) is equal, and the start time of the time axis of the two is the same Knot The beam moment is also the same.

For alignment in terms of length. Assume that some two are aligned in terms of length, the specific meaning is: the length of the two (the length here is the distance unit, not the time unit, for example, the length of the two is 20 The centimeters, etc. are equal, and the starting positions of the two are vertically aligned, and the ending positions are also vertically aligned.

In the embodiment of the present application, in addition to the audio track, the editing interface may further include:

An audiovisual digital media playing interface for playing the content of the audiovisual digital media;

The audiovisual digital media playing time progress bar, wherein the audiovisual digital media playing time progress bar may further have a corresponding playing indicator for indicating the playing time progress.

The audio-visual digital media playback time progress bar can be regarded as the visualization of the time axis of the audio-visual digital media. By viewing the time progress bar of the digital media playback, the user can intuitively see the time axis of the audio-visual digital media, combining Based on the audio track and the time axis of the audio, the user can intuitively and accurately adjust the coordination relationship between the audio and the audiovisual digital media.

The above includes the contents included in the editing interface and a brief introduction. Of course, the content included in the editing interface may not be limited to the enumerated content, and in the actual application, the editing interface may not completely include the enumerated content, for example, may not include the audiovisual digital media playing interface, and the like. The embodiment of the present application does not limit the layout and style of the content in the editing interface, and the matching relationship between the audio and the audiovisual digital media can be adjusted through the editing interface.

For ease of understanding, the embodiment of the present application provides a schematic diagram of an editing interface in an actual application scenario, as an example of the editing interface described in this application, as shown in FIG. 2 .

In FIG. 2, the editing interface includes a window 10, a window 20, an audio view control 21, an audio track 22, an audiovisual digital media playback time progress bar 23 (hereinafter simply referred to as a time progress bar 23), and a play indicator 24. The window 10 is an area for playing audio-visual digital media, and the window 20 is an operation area for adjusting audio, audio-visual digital media playback, and matching relationship. The frequency view control 21, the audio track 22, and the time progress bar 23 are all disposed in the window 20.

Then, based on the method in FIG. 1, after the audio editing mode is started, the audio track 22 can be displayed in the editing interface. At this time, the audio imported in step S101 can be loaded, the audio data can be read, and the visualized audio view control 21 can be generated according to the audio. And displayed on the audio track 22.

The time progress bar 23 is a reflection of the progress of the audiovisual digital media playback, and can also be used to control the progress of the audiovisual digital media playback. The time progress bar 23 is further provided with a play indicator 24, and the specific shape of the play indicator 24 can also select other graphics depending on the specific business situation. The initial position of the play indicator 24 can be selected according to the specific business situation. The starting point or center point of the strip 23 or other location. In practical applications, the progress controller can be separated on the basis of the time progress bar 23. In this case, the time progress bar 23 can only reflect the progress of the audiovisual digital media playback, and is not responsible for controlling the audiovisual digital media playback. Progress, the progress controller is responsible for controlling the progress of audiovisual digital media playback.

In FIG. 2, the time progress bar 23 and the play indicator 24 are disposed in the window 20. In addition to this arrangement, the time progress bar 23 and the play indicator 24 may be disposed in the window 10, and the like. The time progress bar 23 and the play indicator 24 may be resident on the screen (always displayed in the editing interface), or may be in a hidden state at first, and may be displayed in the editing interface after triggering an event in a certain operation process, and may be Certain conditions (for example, after a certain period of time) are hidden again.

It should be noted that the editing interface in FIG. 2 is only an example, and the layout and style of the content can be changed. For example, the size and orientation of the window 10 and the window 20 can be adjusted according to actual business conditions, and the like.

In the embodiment of the present application, when the audio and the audiovisual digital media are synchronously played, all of the audio may be played, or only a part of the audio may be played. In the latter case, it is equivalent to using only a part of the audio for audiovisualization. Digital media dubbing can reduce the amount of work required for users to edit audio. In order to achieve the functions corresponding to these two situations, it can be divided on the audio track. The area, as well as the portion for the audio view control to fall on different areas, correspondingly play the audio portion corresponding to each part in different ways.

Specifically, the audio track may include an effective loading area and an adjustment area, the effective loading area is a continuous area, and the adjustment area is disposed at both ends of the effective loading area, in this case, for the step S103. The moving the audio view control to adjust the mate relationship according to the position movement instruction may include: moving the audio view control according to the position movement instruction, where the audio view control falls within the valid loading Portions of the area will cooperate with the audiovisual digital media playback, and portions of the audio view control that fall within the adjustment area will not play.

In practical applications, in order to make the interface beautiful, when the audio track is displayed, the effective loading area and the adjustment area may not be displayed, only the effective loading area is displayed, and the adjustment area is hidden.

In the embodiment of the present application, according to the foregoing description, the audio view control can be moved on the audio track, that is, the length of the movable range of the audio view control is the length of the audio track. In order to facilitate the intuitive determination and adjustment of the relationship between the audio and the audiovisual digital media, in the editing interface, the length of the audiovisual digital media playback time progress bar, the audio track, and the audio view control may be mutually adapted. The lengths indicated by the lengths of the persons can also be adapted to each other. Although the lengths and durations belong to different dimensions, the numerical values can be correspondingly proportional to each other.

In the embodiment of the present application, the ratio of the unit length of the audio track to the length of the unit length of the audiovisual digital media playing time progress bar and the length of the audiovisual digital media playing time progress bar may be used as a predetermined reference standard. Determining a range of values for the length and/or length of the above three suitable for the actual application scenario.

x为所述音频的时长与所述视听化数字媒体的时长的比值，所述音频轨道满足以下条件：Specifically, when the ratio of the unit length of the audio track to the length indicated by the unit length of the audio-visual digital media playing time progress bar is y, the length of the valid loading area and the audiovisual number The ratio of the length of the media play time progress bar

x is the ratio of the duration of the audio to the duration of the audiovisual digital medium, the audio track satisfying the following conditions:

Where b is the length of the play time progress bar, d is the length of the effective loading area, d ₁ is the length of the adjustment area set at one end of the effective loading area, and d ₂ is set to the effective The length of the adjustment area at the other end of the loading area.

For ease of understanding, the two types of typical values of y (y = y = 1, y is a positive number not equal to 1), and the editing interface in Figure 2 are taken as an example, combined with Figures 3 to 6, for audio. The above conditions for the track to be satisfied are explained.

The first category assumes y=1.

For an audio track, the length of its effective loading area is the same as the length of the audiovisual digital media playing time progress bar. 3 is a partial schematic diagram of an editing interface according to an embodiment of the present application, specifically, part of the editing interface in FIG. 2 in the case where the length of the effective loading area is the same as the length of the viewing period of the audiovisual digital media playing time. schematic diagram.

In FIG. 3, c is the length of the audio view control 21, b is the length of the time progress bar 23, d is the length of the effective loading area M of the audio track 22, and d ₁ is the adjustment area N ₁ of one end of the audio track 22. The length, d ₂ is the length of the adjustment area N ₂ at the other end of the audio track 22, and the reference numerals in Fig. 3 are also used in the subsequent figures of Fig. 3, and will not be described later.

这个条件即是为了使有效加载区域M与时间进度条23在时长方面两端对齐，从而可以达到这样的技术效果：若音频的时长不大于视听化数字媒体的时长，可以通过将音频视图控件21在有效加载区域M内移动，使音频可以对齐于视听化数字媒体的、与音频时长相等的任意连续部分。In FIG. 3, d=b, a is the length of the audio track 22, a=d+d ₁ +d ₂ =b+d ₁ +d ₂ , that is, the length of the audio track 22 is: the effective loading area M The length (equal to the length of the time progress bar 23) and the sum of the lengths of the adjustment areas N ₁ , N ₂ at both ends. At this time, the effective loading area M and the time progress bar 23 are not only end-aligned in terms of duration, but also end-aligned in terms of length. abovementioned

This condition is to make the effective loading area M and the time progress bar 23 both end aligned in terms of duration, so that the technical effect can be achieved: if the duration of the audio is not greater than the duration of the audiovisual digital medium, the audio view control 21 can be Moving within the effective loading area M allows the audio to be aligned to any contiguous portion of the audiovisual digital medium that is equal in duration to the audio.

其中，

The latter two conditions are also explained. Intuitively, the range of values of d ₁ and d ₂ can be made as large as possible to provide a wider adjustment range for the user, but in FIG. 3, when d ₁ and / or d _{2 is} greater than the length c of the audio view control 21 itself, it is easy to appear that the audio view control 21 is completely out of the effective loading area M, which is disadvantageous to the user operation, therefore, the condition can be set.

among them,

In order to facilitate the understanding of the value of c, some specific values can be substituted in FIG. 3, as shown in FIG. 4 and FIG. 5.

由于

则在生成音频视图控件21时，可以将音频视图控件21的长度设置为

也即，音频视图控件21的长度为时间进度条23的长度的

FIG. 4 is a partial schematic diagram of another editing interface according to an embodiment of the present application, specifically, the duration of the audiovisual digital media is 6 minutes, and the duration of the audio is 1 minute. The same portions in FIG. 4 and subsequent drawings as those in the foregoing drawings will not be described again, and for FIG. 4,

due to

When the audio view control 21 is generated, the length of the audio view control 21 can be set to

That is, the length of the audio view control 21 is the length of the time progress bar 23.

由于

则在生成音频视图控件21时，可以将音频视图控件21的长度设置为

也即，音频视图控件21的长度为时间进度条24的长度的

FIG. 5 is a partial schematic diagram of still another editing interface according to an embodiment of the present application, where the duration of the audiovisual digital media is 10 minutes, and the duration of the audio is 5 minutes. Then for Figure 5,

due to

When the audio view control 21 is generated, the length of the audio view control 21 can be set to

That is, the length of the audio view control 21 is the length of the time progress bar 24.

In the second category, assume that y is a positive number that is not equal to one.

此时，有效加载区域M与时间进度条23在时长方面仍是两端对齐的，但是在长度方面不是两端对齐的。In this case, the length of the effective loading area M is different from the length of the time progress bar 23, specifically,

At this time, the effective loading area M and the time progress bar 23 are still aligned in terms of duration, but are not aligned in terms of length.

视听化数字媒体的时长为6分钟、音频的时长为1分钟为例的。FIG. 6 is a partial schematic diagram of still another editing interface according to an embodiment of the present disclosure, specifically

The case where the audiovisual digital medium has a duration of 6 minutes and the duration of the audio is 1 minute is taken as an example.

由于

则在生成音频视图控件21时，可以将音频视图控件21的长度设置为

也即，音频视图控件21的长度为时间进度条24的长度的

In Figure 6,

due to

When the audio view control 21 is generated, the length of the audio view control 21 can be set to

That is, the length of the audio view control 21 is the length of the time progress bar 24.

It should be noted that, in practical applications, y ≤ 1 is generally caused, because in the case of y ≤ 1, in the case of y ≤ 1, it is equivalent to amplifying the audio track 22, which is advantageous for the user for the audio track 22 The audio view control 21 on the screen performs a finer drag operation, thereby improving the fineness of the adjustment of the coordination relationship between the audio and the audiovisual digital media.

Further, the range of values of the length of the audio track can be estimated accordingly. Specifically, for the step S102, generating an audio view control for indicating the audio, specifically, the method may include: An audio view control that represents the audio and that satisfies the following conditions is generated:

Where a is the length of the audio track, that is, the length of the effective loading area and the length of the adjustment area; b is the length of the play time progress bar, and c is the length of the audio view control.

In the embodiment of the present application, the value of y may be preset or the default is y=1, and then the audio view control is generated and displayed by performing step S102. Further, after the audio view control is generated and displayed, the user can also be allowed to change the value of y by issuing an instruction (for example, by clicking a preset touch button or other means), thereby lengthening the audio view control and the audio track. Make an adaptive adjustment.

The length of the audio view control, the audio track, the audiovisual digital media play time progress bar, etc. are analyzed and described above. The following is an analysis of the playback of audio and audiovisual digital media.

In the embodiment of the present application, in order to improve the accuracy when the user adjusts the cooperation relationship between the audio and the audiovisual digital media, in step S103, when the audio view control is moved according to the position movement instruction, the real-time operation may also be performed: An image in the audiovisual digital media corresponding to the location of the audio view control on the audio track is displayed. Specifically, the image can be displayed by using an audiovisual digital media playing interface in the editing interface. Through the real-time displayed image, the user can conveniently align the audio with the image in the audiovisual digital media, thereby improving the accuracy of the user adjusting the cooperation relationship.

In the embodiment of the present application, synchronous playback of audio and audiovisual digital media has been mentioned many times in the foregoing, and synchronous playback may be performed automatically or based on an operation of manual playback by a user. Simultaneous broadcast The playback can be started after the audio view control is generated and displayed; in the moving process of step S103, the synchronous playback can be paused; after the moving process of step S103 is finished, the synchronized playback can be started again.

Specifically, when the audiovisual digital media play time progress bar includes a play indicator for indicating the play time progress, for step S102, after the audio view control is displayed in the edit interface, Starting at a position of the play indicator, synchronously playing the audiovisual digital medium and the audio according to the cooperation relationship; when the moving process starts, suspending the synchronous play; when the moving process At the end, the audiovisual digital media and the audio are played synchronously according to the adjusted cooperation relationship.

Further, when the moving process ends, the synchronized playing of the audiovisual digital media and the audio according to the adjusted cooperation relationship may specifically include:

When the moving process ends, the play indicator is adjusted to a time on the play time progress bar corresponding to the position where the audio view control is located, and according to the adjusted cooperation relationship, after the adjustment The position of the play indicator begins, and the audiovisual digital media and the audio are played synchronously (for this case, the play indicator moves correspondingly with the movement of the audio view control); or,

When the moving process ends, according to the adjusted cooperation relationship, continuing to play the audiovisual digital media and the audio from the position of the play indicator (for this case, playing) The indicator does not move with the movement of the audio view control).

For ease of understanding, the following two types of typical values of y are used to illustrate the playback of audio and audiovisual digital media.

The first category assumes y=1.

Suppose the user sends a position move instruction for the audio view control by dragging the audio view control with the finger, then the audiovisual number is displayed when the user's finger presses the audio view control The media and audio can be paused. At this time, the user can drag the audio view control on the audio track. When the user's finger leaves the audio view control, the audiovisual digital media and audio can be played from the position of the play indicator.

为例的。Referring to FIG. 7, FIG. 7 is a schematic diagram of adjusting the coordination relationship between audiovisual digital media and audio according to an embodiment of the present application. Specifically, the duration of the audiovisual digital media shown in FIG. 5 is 10 minutes, and the duration of the audio is 5 minutes,

For example.

In FIG. 7, before the user drags, the position of the audio view control 21 is indicated by a broken line, at the position A, the play indicator 24 is at the position C; after the user drags, the position of the audio view control 21 is indicated by a solid line, at the position B. The position of the play indicator 24 is at position C. At this time, the position of the play indicator 24 is still at the position before the user drags the audio view control 21, so when the user's finger leaves the audio view control 21, the audiovisual digital media starts playing from the position of the play indicator 24, and plays. During the playback time of the indicator 24 moving from the position C to the position D, only the corresponding portion of the audiovisual digital medium is played without playing the audio (because the start time of the audio has not yet arrived); when the play indicator 24 is moved to the position At time D, the corresponding portion of the audiovisual digital media and audio is simultaneously played; when the play indicator 24 is moved to the position E, the audio is played and the corresponding portion of the audiovisual digital media is played until the audiovisual digital media is played. It can be seen that during the course of the segment, the play indicator 24 does not move with the movement of the audio view control 21.

With continued reference to FIG. 7, a description will be given of a case where the play indicator 24 moves correspondingly with the movement of the audio view control 21. In this case, before the user drags, the position of the audio view control 21 is indicated by a broken line, at the position A, the play indicator 24 is at the position C; after the user drags, the position of the audio view control 21 is indicated by a solid line, Position B, the play indicator 24 moves to position D accordingly. Therefore, when the user's finger leaves the audio view control 21, the audiovisual digital media starts playing from the position D of the play indicator 24, and also starts playing audio; when the play indicator 24 moves to the position E, the audio is played and continues. Play audiovisual digital media Corresponding part until the audiovisual digital media is played.

In the embodiment of the present application, as already mentioned, the audio track may include an effective loading area and an adjustment area. In this case, the audio view control may partially fall into the adjustment area, and the following is an example for synchronous playback of the audiovisual digital media and audio in this case.

Referring to FIG. 8 , FIG. 8 is a schematic diagram of adjusting another relationship between audiovisual digital media and audio according to an embodiment of the present disclosure. Specifically, the duration of the audiovisual digital media shown in FIG. 5 is 10 minutes, and the duration of the audio. For 5 minutes, For example.

In FIG. 8, before the user drags, the position of the audio view control 21 is indicated by a broken line, which is at position A; after the user drags, the position of the audio view control 21 is indicated by a solid line, at position B. The position of the play indicator 24 is not limited herein, and the position before the drag can be maintained, or can be moved along with the audio view control 21. After dragging, the Z portion of the audio view control 21 falls in the adjustment area N ₂ of the audio track 22, and when the play indicator 24 is moved to correspond to the position B where the audio view control 21 is located, the audiovisual number is simultaneously played. The corresponding part of the media and audio, the audio is played synchronously with the audiovisual digital media until the audiovisual digital media is played, and the audio portion corresponding to the Z portion of the audio view control 21 is not played.

为例的。Similarly, referring to FIG. 9 , FIG. 9 is a schematic diagram of adjusting the matching relationship between the audiovisual digital media and the audio provided by the embodiment of the present application, specifically, the duration of the audiovisual digital media shown in FIG. 5 is 10 minutes. The duration of the audio is 5 minutes.

For example.

In FIG. 9, in the view control audio adjustment area Y N ₁ falls portion 21 audio tracks, therefore, the corresponding portion of the audio portion Y may not play.

In the embodiment of the present application, in order to facilitate the user to identify the portion of the audio view control that falls within the adjustment area, the portion may be displayed differently from the portion of the audio view control that falls within the effective loading area. For example, the portion falling on the adjustment area and the portion falling on the effective loading area may be displayed with different transparency or different brightness or different colors, or may be displayed separately, or The adjustment area for the audio track is labeled with a label, and so on.

In the second category, assume that y is a positive number that is not equal to one.

The way of playing is similar to the assumption that y=1, except that the audiovisual digital media playing time progress bar is not aligned with the effective loading area in terms of length.

For example, when y<1, the length of the valid loading area is greater than the length of the time progress bar. In this case, when the audio view control is moved on the audio track, when the audio view control enters the adjustment area, in order to facilitate the user Operation, the audio track can be moved correspondingly to meet the current moving direction of the audio view control, so that during the process of moving the audio view control, the audio view control can be in the middle of the editing interface relatively close to the middle, and try not to be edited. The edge position in the interface.

In the above embodiments, the case where the duration of the audio is not greater than the duration of the audio-visual digital medium is taken as an example, and the solution of the present application is described. In practical applications, the duration of the audio is greater than the duration of the audio-visual digital medium. In the case of this application, the solution of the present application is equally applicable.

Referring to FIG. 10, FIG. 10 is a schematic diagram of adjusting the matching relationship between the audiovisual digital media and the audio provided by the present application, specifically, the duration of the audio is greater than the duration of the audiovisual digital media, y=1, d ₁ =d ₂ ≤ c is an example.

In FIG. 10, before the user drags, the position of the audio view control 21 is indicated by a dotted line. After the user drags, the position of the audio view control 21 is indicated by a solid line. It should be noted that, actually, the audio view after dragging The control 21 is still located on the audio track 22, but for ease of viewing, the dragged audio view control 21 is not shown on the audio track 22, but is moved vertically downward from the audio track 22, indicating that it is before the drag. Below the audio view control 21. The audio portion of the portion of the audio view control 21 that falls within the adjustment areas N ₁ , N ₂ may not be played during the simultaneous playback of the audio and the audiovisual digital media, both before and after the drag.

The above is a method for dubbing audiovisual digital media provided by the embodiment of the present application. Based on the same idea, the embodiment of the present application further provides a corresponding device for dubbing audiovisual digital media, as shown in FIG. 11 .

FIG. 11 is a schematic structural diagram of an apparatus for dubbing an audiovisual digital medium according to an embodiment of the present disclosure, where the apparatus includes:

The audio importing module 1101 is configured to import an audio in the editing interface;

The audio view control module 1102 is configured to generate an audio view control for representing the audio according to the imported audio, and display the audio view control in the editing interface, where the position of the audio view control reflects Coordinating relationship between the audio and the audiovisual digital media;

The audio adjustment module 1103 is configured to: when receiving a position move instruction for the audio view control, move the audio view control to adjust the mating relationship according to the position move instruction.

Optionally, the device further includes:

The editing mode opening module 1104 is configured to: before the audio view control module 1102 generates an audio view control for indicating the audio, receive the audiovisual digital media, and start a control instruction of the audio editing mode; according to the control instruction, The audiovisual digital medium turns on an audio editing mode and displays the editing interface.

Optionally, the editing interface further includes:

An audio track, the audio view control being located on the audio track, the audio track including an effective loading area and an adjustment area;

An audiovisual digital media playing interface for playing the content of the audiovisual digital media;

Audiovisual digital media playback time progress bar;

The audio adjustment module 1103 is specifically configured to: move the audio view control on the audio track according to the position movement instruction to adjust the mating relationship.

Optionally, the effective loading area is a continuous area, and the adjustment area is disposed in the The audio adjustment module 1103 is specifically configured to: move the audio view control according to the position movement instruction, where a portion of the audio view control falling in the effective loading area will cooperate with the audiovisual number Media playback, the portion of the audio view control that falls within the adjustment area will not play.

x为所述音频的时长与所述视听化数字媒体的时长的比值，所述音频轨道满足以下条件：Optionally, when the ratio of the unit length of the audio track to the duration represented by the unit length of the audiovisual digital media playing time progress bar is y, the length of the valid loading area and the audiovisualization The ratio of the length of the digital media playback time progress bar

x is the ratio of the duration of the audio to the duration of the audiovisual digital medium, the audio track satisfying the following conditions:

Where b is the length of the play time progress bar, d is the length of the effective loading area, d ₁ is the length of the adjustment area set at one end of the effective loading area, and d ₂ is set to the effective The length of the adjustment area at the other end of the loading area.

Optionally, the audio view control module 1102 is specifically configured to: generate an audio view control for representing the audio and satisfying the following conditions:

Where a is the length of the audio track, that is, the length of the effective loading area and the length of the adjustment area; b is the length of the play time progress bar, and c is the length of the audio view control.

Optionally, the play time progress bar includes a play indicator for indicating the play time progress;

The device also includes:

a playing module 1105, after the audio view control module displays the audio view control in the editing interface, starting from a position of the play indicator, according to the cooperation relationship, the audiovisual number And playing the audio synchronously; and when the moving process starts, suspending the synchronous play; when the moving process ends, according to the adjusted cooperation relationship, the audiovisual digital media and the The audio is played synchronously.

Optionally, the playing module 1105 is specifically configured to: when the moving process ends, adjust the play indicator to a time on the play time progress bar corresponding to a location where the audio view control is located, and according to And the adjusted matching relationship starts from the position of the adjusted play indicator to synchronously play the audiovisual digital medium and the audio.

Optionally, the playing module 1105 is further configured to: when the audio adjustment control moves the audio view control according to the position moving instruction, display the position of the audio view control on the audio track in real time. Corresponding images in the audiovisual digital media.

Optionally, the location movement instruction is sent by a user dragging the audio view control;

The audio adjustment module 1103 is specifically configured to: the audio view control is dragged on the audio track according to the drag operation corresponding to the position movement instruction.

Optionally, the audiovisual digital media is video and/or animation.

The device in Figure 11 can be specifically located on an audiovisual related device.

The device provided by the present application is in one-to-one correspondence with the method provided by the present application. Therefore, the device also has similar beneficial technical effects as the method, since the beneficial technical effects of the method have been described in detail above. Therefore, the beneficial technical effects of the device will not be described again here.

Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include non-persistent memory, random access memory (RAM), and/or non-volatile memory in a computer readable medium, such as read only memory (ROM) or flash memory. Memory is an example of a computer readable medium.

Computer readable media includes both permanent and non-persistent, removable and non-removable media. Information storage can be implemented by any method or technology. The information can be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory. (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disk read only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, Magnetic tape cartridges, magnetic tape storage or other magnetic storage devices or any other non-transportable media can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary storage of computer readable media, such as modulated data signals and carrier waves.

It is also to be understood that the terms "comprises" or "comprising" or "comprising" or any other variations are intended to encompass a non-exclusive inclusion, such that a process, method, article, Other elements not explicitly listed, or elements that are inherent to such a process, method, commodity, or equipment. An element defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device including the element.

The above description is only an embodiment of the present application and is not intended to limit the application. Various changes and modifications can be made to the present application by those skilled in the art. Any modifications, equivalents, improvements, etc. made within the spirit and scope of the present application are intended to be included within the scope of the appended claims.

Claims (22)

A method for dubbing audiovisual digital media, characterized by comprising: Import an audio in the editing interface; Generating an audio view control for representing the audio according to the imported audio, and displaying the audio view control in the editing interface, the position of the audio view control reflecting the audio and the audiovisual number Media cooperation relationship; When the position move instruction for the audio view control is received, the audio view control is moved to adjust the mating relationship according to the position move instruction. The method of claim 1 wherein before the audio view control for representing the audio is generated, the method further comprises: Receiving the audiovisual digital medium, and turning on a control instruction of the audio editing mode; And according to the control instruction, turning on an audio editing mode for the audiovisual digital medium, and displaying the editing interface. The method of claim 1, wherein the editing interface further comprises: An audio track, the audio view control being located on the audio track, the audio track including an effective loading area and an adjustment area; An audiovisual digital media playing interface for playing the content of the audiovisual digital media; Audiovisual digital media playback time progress bar; And the moving the view control to adjust the mate relationship according to the position movement instruction, specifically: The audio view control is moved on the audio track to adjust the mating relationship according to the position movement instruction. The method according to claim 3, wherein the effective loading area is a continuous area, and the adjustment area is disposed at both ends of the effective loading area, and the audio is moved according to the position moving instruction. The view control adjusts the cooperation relationship, and specifically includes: Moving the audio view control according to the position movement instruction, a portion of the audio view control falling in the effective loading area will be played in conjunction with the audiovisual digital media, and the audio view control falls on a part of the adjustment area Will not play. The method according to claim 4, wherein said effective ratio when said unit length of said audio track and said duration of said unit length of said audiovisual digital media playing time progress bar are y The ratio of the length of the loading area to the length of the audiovisual digital media playing time progress bar x is the ratio of the duration of the audio to the duration of the audiovisual digital medium, the audio track satisfying the following conditions:

Where b is the length of the play time progress bar, d is the length of the effective loading area, d ₁ is the length of the adjustment area set at one end of the effective loading area, and d ₂ is set to the effective The length of the adjustment area at the other end of the loading area. The method of claim 5, wherein generating an audio view control for representing the audio comprises: An audio view control that represents the audio and that satisfies the following conditions is generated:

Where a is the length of the audio track, that is, the length of the effective loading area and the length of the adjustment area; b is the length of the play time progress bar, and c is the length of the audio view control. The method of claim 6 wherein said play time progress bar includes a play indicator for indicating said play time progress; After the audio view control is displayed in the editing interface, the method further includes: Starting from a position of the play indicator, synchronously playing the audiovisual digital medium and the audio according to the cooperation relationship; Suspending the synchronized play when the moving process starts; When the moving process ends, the audiovisual digital media and the audio are synchronously played according to the adjusted cooperation relationship. The method of claim 7, wherein when the moving process ends, the audio-visual digital media and the audio are played synchronously according to the adjusted cooperation relationship, which specifically includes: When the moving process ends, the play indicator is adjusted to a time on the play time progress bar corresponding to the position where the audio view control is located, and according to the adjusted cooperation relationship, after the adjustment The location of the play indicator begins, and the audiovisual digital media and the audio are played synchronously. The method according to any one of claims 1 to 8, wherein when the audio view control is moved according to the position movement instruction, the method further comprises: An image in the audiovisual digital media corresponding to the location of the audio view control on the audio track is displayed in real time. The method according to any one of claims 1 to 8, wherein the position movement instruction is sent by a user dragging the audio view control; And moving the audio view control according to the location movement instruction, specifically: And the audio view control is dragged on the audio track according to the drag operation corresponding to the position movement instruction. The method according to any one of claims 1 to 8, wherein said audiovisual number Word media is video and/or animation. An apparatus for dubbing audiovisual digital media, comprising: An audio import module for importing an audio in the editing interface; An audio view control module, configured to generate an audio view control for representing the audio according to the imported audio, and display the audio view control in the editing interface, where a position of the audio view control reflects The relationship between the audio and the audiovisual digital media; And an audio adjustment module, configured to: when receiving a position move instruction for the audio view control, move the audio view control to adjust the mating relationship according to the position move instruction. The device of claim 12, wherein the device further comprises: An editing mode opening module, configured to: before the audio view control module generates an audio view control for representing the audio, receive the audiovisual digital media, and start a control instruction of an audio editing mode; according to the control instruction, The audiovisual digital medium turns on an audio editing mode and displays the editing interface. The device of claim 12, wherein the editing interface further comprises: An audio track, the audio view control being located on the audio track, the audio track including an effective loading area and an adjustment area; An audiovisual digital media playing interface for playing the content of the audiovisual digital media; Audiovisual digital media playback time progress bar; The audio adjustment module is specifically configured to: move the audio view control on the audio track according to the position movement instruction to adjust the cooperation relationship. The apparatus according to claim 14, wherein the effective loading area is a continuous area, and the adjustment area is disposed at two ends of the effective loading area, and the audio adjustment module is specifically configured to: The position movement instruction moves the audio view control, and the portion of the audio view control that falls within the valid loading area will cooperate with the audiovisual digital medium During playback, the portion of the audio view control that falls within the adjustment area will not play. The apparatus according to claim 15, wherein said effective ratio is y when a ratio of a unit length of said audio track to a length indicated by said unit length of said audiovisual digital media playing time progress bar is y The ratio of the length of the loading area to the length of the audiovisual digital media playing time progress bar

x is the ratio of the duration of the audio to the duration of the audiovisual digital medium, the audio track satisfying the following conditions:

Where b is the length of the play time progress bar, d is the length of the effective loading area, d ₁ is the length of the adjustment area set at one end of the effective loading area, and d ₂ is set to the effective The length of the adjustment area at the other end of the loading area. The device according to claim 16, wherein the audio view control module is specifically configured to: generate an audio view control for representing the audio and satisfying the following conditions:

Where a is the length of the audio track, that is, the length of the effective loading area and the length of the adjustment area; b is the length of the play time progress bar, and c is the length of the audio view control. The apparatus according to claim 17, wherein said play time progress bar includes a play indicator for indicating said play time progress; The device also includes: a playing module, configured to: after the audio view control module displays the audio view control in the editing interface, start from the position of the play indicator, according to the cooperation relationship, the audiovisual digital media Synchronizing play with the audio; suspending the synchronous play when the moving process starts; and when the moving process ends, the audiovisual digital media and the audio according to the adjusted mating relationship Play synchronously. The device according to claim 18, wherein the playing module is specifically configured to: when the moving process ends, adjust the play indicator to the play time progress bar and the audio view control At the time corresponding to the position, and according to the adjusted cooperation relationship, the audiovisual digital media and the audio are synchronously played from the position of the adjusted play indicator. The device according to any one of claims 12 to 19, wherein the playing module is further configured to display in real time when the audio adjustment module moves the audio view control according to the position moving instruction. An image in the audiovisual digital media corresponding to the location of the audio view control on the audio track. The device according to any one of claims 12 to 19, wherein the position movement instruction is sent by a user dragging the audio view control; The audio adjustment module is specifically configured to: the audio view control is dragged on the audio track according to the drag operation corresponding to the position movement instruction. Apparatus according to any one of claims 12 to 19, wherein the audiovisual digital medium is video and/or animation.

Images