WO2023174066A1 - Video generation method and apparatus, electronic device, and storage medium - Google Patents

Abstract

Disclosed in embodiments of the present disclosure are a video generation method and apparatus, an electronic device, and a storage medium. The method comprises: determining content features of target pictures; determining target music on the basis of the content features, the rhythm of the target music matching the content features; synthesizing the target music and the target pictures to obtain a target video, the switching rhythm of image materials in the target video matching the rhythm of the target music.

Description

Video generation method, device, electronic device and storage medium

Cross-references to related applications

This application is based on the application with Chinese application number 202210270283.6 and the filing date is March 18, 2022, and claims its priority. The disclosure content of the Chinese application is hereby incorporated into this application as a whole.

Technical field

The present disclosure relates to the field of information technology, and in particular, to a video generation method, device, electronic device and storage medium.

Background technique

With the rapid development of network technology and mobile terminal technology, more and more applications have emerged. For example, applications related to picture editing, through which users can edit pictures, including generating videos related to pictures.

Currently, when editing a picture to generate a video related to the picture, the user needs to manually select different music to control the playback rhythm of the video or picture.

Contents of the invention

Embodiments of the present disclosure provide a video generation method, which includes:

Determine the content characteristics of the target image;

Determine target music based on the content characteristics, the rhythm of the target music matches the content characteristics;

The target music and the target picture are synthesized to obtain a target video, and the switching rhythm of the image material in the target video matches the rhythm of the target music.

An embodiment of the present disclosure also provides a video generation device, which includes:

The first determination module is used to determine the content characteristics of the target image;

a second determination module, configured to determine target music based on the content characteristics, and the rhythm of the target music matches the content characteristics;

A processing module configured to synthesize the target music and the target picture to obtain a target video, where the switching rhythm of the image material in the target video matches the rhythm of the target music.

An embodiment of the present disclosure also provides an electronic device, where the electronic device includes:

one or more processors;

A storage device for storing one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the video generation method as described above.

Embodiments of the present disclosure also provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the video generation method as described above is implemented.

An embodiment of the present disclosure also provides a computer program, including: instructions that, when executed by a processor, cause the processor to perform the video generation method as described above.

Description of the drawings

The above and other features, advantages, and aspects of various embodiments of the present disclosure will become more apparent with reference to the following detailed description taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It is to be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.

Figure 1 is a flow chart of a video generation method in an embodiment of the present disclosure;

Figure 2 is a schematic diagram of a video track and an audio track in an embodiment of the present disclosure;

Figure 3 is a schematic diagram of a video track in an embodiment of the present disclosure;

Figure 4 is a schematic diagram of an audio track in an embodiment of the present disclosure;

Figure 5 is a schematic diagram of a video track in an embodiment of the present disclosure;

Figure 6 is a schematic diagram of an audio track and a video track in an embodiment of the present disclosure;

Figure 7 is a flow chart of a video generation method in an embodiment of the present disclosure;

Figure 8 is a schematic structural diagram of a video generation device in an embodiment of the present disclosure;

FIG. 9 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure.

Detailed ways

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the disclosure are shown in the drawings, it should be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, which rather are provided for A more thorough and complete understanding of this disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of the present disclosure.

It should be understood that the various steps described in the method implementations of the present disclosure can be performed in different orders. and/or parallel execution. Furthermore, method embodiments may include additional steps and/or omit performance of illustrated steps. The scope of the present disclosure is not limited in this regard.

As used herein, the term "include" and its variations are open-ended, ie, "including but not limited to." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; and the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below.

It should be noted that concepts such as “first” and “second” mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units. Or interdependence.

It should be noted that the modifications of "one" and "plurality" mentioned in this disclosure are illustrative and not restrictive. Those skilled in the art will understand that unless the context clearly indicates otherwise, it should be understood as "one or Multiple”.

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are for illustrative purposes only and are not used to limit the scope of these messages or information.

After conducting analysis, the inventor found that the related technology had problems such as cumbersome user operations and low intelligence.

Figure 1 is a flow chart of a video generation method in an embodiment of the present disclosure. The method can be executed by a video generation device. The device can be implemented in software and/or hardware. The device can be configured in electronic equipment, such as Terminals specifically include but are not limited to smartphones, PDAs, tablets, wearable devices with displays, desktops, laptops, all-in-one computers, smart home devices, etc.

As shown in Figure 1, the method specifically includes the following steps.

Step 110: Determine the content characteristics of the target image.

The target picture can be an image material captured by the user in real time through the camera, an image material imported locally from the terminal, or an image material downloaded from the Internet.

The content characteristics of the target picture may include the main color of the target picture, the object category in the target picture (such as people or scenery), etc., and the actions of the object (such as running, jumping, sleeping, opening mouth, etc.).

Optionally, determining the content characteristics of the target image includes:

Determine the picture characteristics of the target picture through the picture feature determination algorithm; determine the emotional characteristics of the target picture based on the picture characteristics of the target picture; determine the content characteristics of the target picture based on the emotional characteristics of the target picture.

The picture features include at least one of the color of the target picture, the object category in the target picture (such as people or scenery), and the actions of the objects in the target picture (such as running, jumping, sleeping, opening mouth, etc.). Optional, can The color recognition algorithm is used to identify the color of the target picture; the target picture is processed through a neural network model based on machine learning, and the object category and object action in the target picture are identified.

The color of the target image specifically refers to the main color of the target image, such as red, yellow, blue, etc. Specifically, the main color of the target image can be determined by counting the number of pixels of each color. For example, the target image includes two colors: red and blue. The number of red pixels is 10,000 and the number of blue pixels is 20,000. Then it can be determined that the main color of the target image is blue, which is the picture feature. is blue.

Further, determining the emotional characteristics of the target picture based on the picture characteristics of the target picture includes:

Based on the mapping relationship between the picture features and the emotional features, the emotional features that have a mapping relationship with the picture features of the target picture are determined. The mapping relationship may be set before determining the picture characteristics of the target picture, for example, manually or automatically by a device or algorithm.

For example, there is a mapping relationship between the picture characteristics of "red" and the emotional characteristics of "strong and bold"; there is a mapping relationship between the picture characteristics of "yellow" and the emotional characteristics of "bright and brisk"; there is a mapping relationship between the picture characteristics of "blue" and the emotional characteristics of "gentle and graceful". There is a mapping relationship between the emotional characteristics; there is a mapping relationship between the "green" picture characteristics and the "calm and indifferent" emotional characteristics; there is a mapping relationship between the "gray" picture characteristics and the "dark and low" emotional characteristics. After determining the picture characteristics of the target picture, the emotional characteristics that have a mapping relationship with the picture characteristics of the target picture can be determined based on the above preset mapping relationship.

Or, for example, there is a mapping relationship between the picture feature that the object's action in the target picture is "jumping" and the emotional feature "bright and brisk"; there is a mapping relationship between the picture feature that the object's action in the target picture is "sleeping" and the emotional feature "gentle and graceful" Mapping relationships, etc.

In some embodiments, when the number of target pictures is one, determining the content characteristics of the target picture according to the emotional characteristics of the target picture includes:

The emotional characteristics of the target picture are determined as the content characteristics of the target picture.

When the number of target pictures is two, determining the content characteristics of the target picture based on the emotional characteristics of the target picture includes:

If the emotional characteristics of the two target pictures are the same, the emotional characteristics of the target picture are determined as the content characteristics of the target picture;

If the emotional characteristics of the two target pictures are not the same, the emotional characteristics of any target picture are determined as the content characteristics of the target picture.

When the number of target pictures is at least three, the target picture is determined according to the emotional characteristics of the target picture. Content characteristics of the film include:

If the emotional characteristics of each target picture are different from each other, the emotional characteristics of any target picture are determined as the content characteristics of the target picture. For example, the number of target pictures is three, which are respectively recorded as the first target picture, the second target picture and the third target picture. Assuming that the emotional characteristics of the first target picture are "strong and bold", the emotional characteristics of the second target picture are "bright and brisk", and the emotional characteristics of the third target picture are "gentle and graceful", then "strong and bold" and "bright" can be Either one of "bright" and "gentle and graceful" is determined as the content feature of the two target pictures, that is, the content feature of the target picture can be "strong and bold", "bright and light" or "gentle and graceful".

When the number of target pictures is at least three, determining the content characteristics of the target picture according to the emotional characteristics of the target picture includes:

If there are situations where the emotional features of different target pictures are the same, the emotional feature that appears most frequently will be determined as the content feature of the target picture. For example, the number of target pictures is three, which are respectively recorded as the first target picture, the second target picture and the third target picture. Assume that the emotional characteristics of the first target picture are "strong and bold", the emotional characteristics of the second target picture are "strong and bold", and the emotional characteristics of the third target picture are "gentle and graceful", then the most frequent "strong and bold" will be Determine the content characteristics of the target image. For another example, the number of target pictures is 6. Assume that the emotional features of 4 of the target pictures are all "gentle and graceful", then it can be determined that the most frequent emotional feature is "gentle and graceful", that is, "gentle and graceful" is determined as this Content characteristics of three target images.

In summary, when there are multiple target pictures, each target picture corresponds to an emotional feature, and the content features of the target picture are determined based on the emotional features of the target picture, including: selecting the emotional feature with the largest number of corresponding target pictures as the content feature. For example, the number of target pictures is 3, which are marked as the first target picture, the second target picture and the third target picture respectively. Among them, the emotional feature corresponding to the first target picture is A, and the emotional feature corresponding to the second target picture is B. , the emotional feature corresponding to the third target picture is A, that is, the number of target pictures corresponding to emotional feature A is 2, and the number of target pictures corresponding to emotional feature B is 1, so emotional feature A is determined as the content feature.

Step 120: Determine target music based on the content characteristics, and the rhythm of the target music matches the content characteristics.

Optionally, the rhythm of the target music matches the content characteristics, so as to control the carousel of pictures and the switching rhythm of the target pictures through the rhythm of the target music. For example, when the content feature is "strong and bold", the target music can be passionate and fast-paced music; when the content feature is "dark and low", the target music can be music with slow-paced changes.

Optional, for example, the target picture includes people, and further includes smiling faces, then the target music can be compared Cheerful music; for example, if the target picture includes a couple, the target music can be music about sweet emotions, etc.

Optionally, a mapping relationship between content features and target music can be preset. After obtaining the content features of the target picture, the corresponding target music can be determined by searching for the mapping relationship.

Step 130: Synthesize the target music and the target picture to obtain a target video, and the switching rhythm of the image material in the target video matches the rhythm of the target music.

The image material in the target video can be a target picture, or it can also be a special effect, such as a fast-paced flash cut effect, a slow-paced blur to clear transition effect, etc.

Exemplarily, synthesizing the target music and the target picture to obtain a target video includes: determining the position of each target picture in the target video based on at least one of the playback duration of the target music, the number of target pictures, or the rhythm of the target music. Switching time; generate target video based on the switching time.

Optionally, determining the switching moment of each target picture in the target video based on at least one of the playback duration of the target music, the number of the target pictures, or the rhythm of the target music includes: The playback duration of the target music is evenly divided according to the number of target pictures; each split point is determined as the switching moment of the target picture in the target video, so that the playback duration of each target picture in the target video is the same.

In other words, each target picture is switched and displayed at a constant speed according to the playing time of the target music. For example, the playing time of the target music is 3 seconds, and the number of target pictures is three, which are recorded as target picture A, target picture B and target picture C respectively. The target music and the target picture are synthesized to obtain the target video. The target video is a 3s video including target picture A, target picture B and target picture C. For example, refer to the schematic diagram of a video track and an audio track as shown in Figure 2. Play the target picture A in the 0-1s of the target video, that is, the video clip corresponding to the target picture A in the 0-1s of the target video. When the target video is played to the 1s, switch the video picture from the target picture A to the target picture B. . Play the target picture B in the 1-2s of the target video, and the 1-2s of the target video correspond to the video clip of the target picture B. When the target video is played to the 2s, the video picture is switched from the target picture B to the target picture C. Play the target picture C in the 2-3s of the target video, and the 2-3s of the target video correspond to the video clip of the target picture C. Each target picture plays for the same duration in the target video, that is, each target picture is played at a constant speed. For example, the target music is relatively soft slow-tempo music.

It should be noted that the order in which target picture A, target picture B and target picture C are played in the target video can be pre-specified by the user, can be determined randomly, or can be determined dynamically based on some strategies, such as according to the shooting order of each target picture. The time determines the playback order.

In some embodiments, the switching moment of each target picture in the target video is determined based on at least one of the playback duration of the target music, the number of the target pictures, or the rhythm of the target music, include:

According to the target duration of each target picture played in the target video, the switching moment of each target picture in the target video is determined so that the playing duration of each target picture in the target video is the target duration. The target duration can be preset. That is, based on the target duration of each target picture played in the target video, the switching moment of each target picture in the target video is determined so that the playing duration of each target picture in the target video is the target duration. For example, pre-specify that the playback time of each target picture in the target video is 1s. Assume that there are 5 target pictures in total, namely target picture A, target picture B, target picture C, target picture D and target picture E. Example As shown in Figure 3, the target picture A is played in the 0-1s of the target video. When the target video is played to the 1s, the video picture is switched from the target picture A to the target picture B; in the 1-2s of the target video Play the target picture B. When the target video is played to the 2s, the video screen will be switched from the target picture B to the target picture C. Play the target picture C at the 2-3s of the target video. When the target video is played to the 3s, the video screen will be switched to the target picture C. The video picture is switched from the target picture C to the target picture D; the target picture D is played in the 3-4s of the target video. When the target video is played to the 4s, the video picture is switched from the target picture D to the target picture E; in the target video The first 4-5s of playback target picture E. That is, the total duration of the target video is 5s. If the total duration of the target music is greater than 5s, it will automatically end at the 5th second. If the total duration of the target music is less than 5s, assuming it is 3s, it will automatically repeat from the beginning at the 3rd second. .

In some embodiments, the switching moment of each target picture in the target video is determined based on at least one of the playback duration of the target music, the number of the target pictures, or the rhythm of the target music, Including: determining the accent rhythm point of the target music according to the rhythm of the target music; determining the accent rhythm point as the switching moment of the target picture in the target video. That is, the switching moment of each target picture in the target video is determined according to the stuck moment of the target music, so that the switching moment of the target picture in the target video is the stuck moment.

For example, the target music includes five accent rhythm points, that is, stuck moments, which are 1.5s, 2.3s, 3.0s, 3.9s and 4.4s respectively. Figure 4 shows a schematic diagram of an audio track. When applying the target music to generate a target video based on the target picture (assuming the number of target pictures is 5), play the target picture A in the 0-1.5s time period of the target video, and change the video picture at the 1.5s time period. Switch from target picture A to target picture B; play target picture B in the time period from 1.5s to 2.3s, and switch the video picture from target picture B to target picture C at 2.3s; in 2.3s- Play the target picture C in the 3.0s time period, and switch the video picture from the target picture C to the target picture D at the 3.0s time period; play the target picture D in the 3.0s-3.9s time period. At the 3.9s moment, the video screen is switched from the target picture D to the target picture E; the target picture E is played during the time period from the 3.9s to the 4.4s. When the 4.4s is reached, the target video playback ends. Figure 5 shows a schematic diagram of a video track. The target video starts playing from 0s, and the track information on the corresponding timeline is read at the same time, that is Pictures + music, and the start and end times of playback are based on the main video track. Therefore, the video ends when the 4.4s is played, and the overall presentation effect is that the playback is stuck and the rhythm changes briskly. The target music may be faster tempo music.

Furthermore, when the video rhythm is controlled by playing at a constant speed, the target music and the target picture are synthesized to obtain the target video, including: assuming that the number of target pictures is 5, the target music is a 12s long slow-tempo music, and the length of each target picture is specified. If the playback duration is 2s, a new audio track will be generated inside the video player for management, consistent with the starting time of the video, that is, starting from 0s, playing to the 10th second, and the target music will also play to the 10th second. The overall effect is a uniform slow-tempo playback. Correspondingly, reference can be made to the schematic diagram of an audio track and a video track as shown in Figure 6.

Optionally, synthesizing the target music and the target picture to obtain the target video also includes: determining the switching moment of the special effects in the target video based on the rhythm of the target music; and based on the switching moment of the target picture in the target video and the special effects. The target video is generated at the switching moment. The special effects include, for example, fast-paced flash-cut special effects, slow-paced blur-to-clear transition special effects, etc.

The video generation method provided by the embodiment of the present disclosure realizes personalized control of the playback rhythm of the target picture in the target video according to the content characteristics of the target picture, so that the presentation effect of the target video is better and the user experience is improved. For example, if the content feature of the target image is bright and cheerful, a video with a strong rhythm will eventually be generated; if the content feature of the target image is dark and gloomy, a video with a slow rhythm will be generated. As a result, the playback effect of the target video is associated with the content characteristics of the target picture, and the generation of personalized target videos is achieved. Moreover, it realizes the automation of video generation, simplifies the operation of users to generate videos based on pictures, and improves the efficiency and user experience of video generation.

On the basis of the above embodiments, with reference to the flow diagram of a video generation method shown in Figure 7, specifically, the content characteristics are determined based on the target picture, and then the target music is determined based on the content characteristics, and the rhythm change of the video is controlled based on the target music. Obtain the target video including the target picture.

Figure 8 is a schematic structural diagram of a video generation device in an embodiment of the present disclosure. The video generation device specifically includes: a first determination module 810, a second determination module 820 and a processing module 830.

The first determination module 810 is used to determine the content characteristics of the target picture; the second determination module 820 is used to determine the target music based on the content characteristics, and the rhythm of the target music matches the content characteristics; the processing module 830 is used to The target music and the target picture are synthesized to obtain a target video, and the switching rhythm of the image material in the target video matches the rhythm of the target music.

Optionally, the first determination module 810 includes: a first determination unit, configured to determine the picture characteristics of the target picture through a picture characteristics determination algorithm; and a second determination unit, configured to determine based on the picture characteristics of the target picture. Determine the emotional characteristics of the target picture; a third determination unit is used to determine the content characteristics of the target picture according to the emotional characteristics of the target picture.

Optionally, the picture characteristics include at least one of the color of the target picture, the category of the object in the target picture, or the action of the object in the target picture.

Optionally, the second determination unit is specifically configured to: based on the mapping relationship between the picture features and the emotional features, determine the emotional features that have a mapping relationship with the picture features of the target picture.

Optionally, when there are multiple target pictures, each target picture corresponds to an emotional feature, and the third determination unit is specifically configured to: select the emotional feature with the largest number of corresponding target pictures as the content feature. .

Optionally, the processing module 830 includes: a determining unit configured to determine, based on at least one of the playing duration of the target music, the number of the target pictures, or the rhythm of the target music, that each of the target pictures is in the target A switching moment in the video; a generating unit configured to generate the target video according to the switching moment.

Optionally, the determining unit is specifically configured to: evenly divide the playback duration of the target music according to the number of the target pictures; determine each dividing point as the number of the target pictures in the target video. The switching time is such that the playback duration of each target picture in the target video is the same.

Optionally, the determination unit is specifically configured to: determine the switching moment of each target picture in the target video according to the target duration of playback of each target picture in the target video, so that each The playback duration of the target picture in the target video is the target duration.

Optionally, the determining unit is specifically configured to: determine the accent rhythm point of the target music according to the rhythm of the target music; determine the accent rhythm point as the switching moment of the target picture in the target video. .

Optionally, the determining unit is further configured to determine the switching moment of the special effects in the target video based on the rhythm of the target music; and the generating unit is configured to: switch the target picture in the target video based on the rhythm of the target music. The target video is generated at the time and the switching time of the special effects.

The video generation device provided by the embodiments of the present disclosure can execute the steps in the method provided by the method embodiments of the present disclosure. The execution steps and beneficial effects will not be described again here.

FIG. 9 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure. Referring specifically to FIG. 9 below, a schematic structural diagram of an electronic device 500 suitable for implementing an embodiment of the present disclosure is shown. The electronic device 500 in the embodiment of the present disclosure may include, but is not limited to, mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMP (portable multimedia players), vehicle-mounted terminals ( Such as car navigation terminals), wearable electronic devices, etc., as well as mobile terminals such as digital TVs, desktop computers, Fixed terminals for smart home devices and more. The electronic device shown in FIG. 9 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 9, the electronic device 500 may include a processing device (eg, central processing unit, graphics processor, etc.) 501, which may be loaded into a random access device according to a program stored in a read-only memory (ROM) 502 or from a storage device 508. The program in the memory (RAM) 503 performs various appropriate actions and processes to implement the video generation method according to the embodiments of the present disclosure. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, ROM 502 and RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

Generally, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 507 such as a computer; a storage device 508 including a magnetic tape, a hard disk, etc.; and a communication device 509. Communication device 509 may allow electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although FIG. 9 illustrates electronic device 500 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, the computer program including program code for executing the method shown in the flowchart, thereby achieving the above The video generation method described. In such embodiments, the computer program may be downloaded and installed from the network via communication device 509, or from storage device 508, or from ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmed read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. In this disclosure, computer readable A signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device . Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and server can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can communicate with digital data in any form or medium. Communications (e.g., communications network) interconnections. Examples of communications networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or developed in the future network of.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.

The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: determines the content characteristics of the target picture; determines the target music based on the content characteristics, The rhythm of the target music matches the content characteristics; the target music and the target picture are synthesized to obtain a target video, and the switching rhythm of the image material in the target video matches the rhythm of the target music.

Optionally, when one or more of the above programs are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages—such as "C" or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In situations involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through Internet connection).

The flowcharts and block diagrams in the figures illustrate systems, methods, and computer programs in accordance with various embodiments of the present disclosure. Possible architecture, functionality, and operations of a program product. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.

The units involved in the embodiments of the present disclosure can be implemented in software or hardware. Among them, the name of a unit does not constitute a limitation on the unit itself under certain circumstances.

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

According to one or more embodiments of the present disclosure, the present disclosure provides a video generation method, including: determining content features of a target picture; determining target music based on the content features, and the rhythm of the target music is consistent with the content features Match; synthesize the target music and the target picture to obtain a target video, and the switching rhythm of the image material in the target video matches the rhythm of the target music.

According to one or more embodiments of the present disclosure, in the video generation method provided by the present disclosure, optionally, determining the content characteristics of the target picture includes: determining the picture characteristics of the target picture through a picture characteristic determination algorithm. ; Determine the emotional characteristics of the target picture based on the picture characteristics of the target picture; Determine the content characteristics of the target picture based on the emotional characteristics of the target picture.

According to one or more embodiments of the present disclosure, in the video generation method provided by the present disclosure, optionally, the picture characteristics include the color of the target picture, the object category in the target picture, or the action of the object in the target picture. At least one.

According to one or more embodiments of the present disclosure, in the video generation method provided by the present disclosure, optionally, determining the emotional characteristics of the target picture based on the picture characteristics of the target picture includes: based on the picture characteristics and The mapping relationship between emotional features determines the emotional features that have a mapping relationship with the picture features of the target picture.

According to one or more embodiments of the present disclosure, in the video generation method provided by the present disclosure, optionally, when the number of target pictures is multiple, each target picture corresponds to an emotional feature, and the Determining the content features of the target picture according to the emotional features of the target picture includes: selecting the emotional feature with the largest number of corresponding target pictures as the content feature.

According to one or more embodiments of the present disclosure, in the video generation method provided by the present disclosure, optionally, the synthesis of the target music and the target picture to obtain a target video includes: At least one of the playback duration, the number of the target pictures, or the rhythm of the target music determines the switching moment of each of the target pictures in the target video; the target video is generated according to the switching moment.

According to one or more embodiments of the present disclosure, in the video generation method provided by the present disclosure, optionally, based on the playback duration of the target music, the number of the target pictures, or the rhythm of the target music At least one of determining the switching moment of each target picture in the target video includes: evenly dividing the playback duration of the target music according to the number of the target pictures; determining each dividing point as the The switching time of the target picture in the target video is such that the playback duration of each target picture in the target video is the same.

According to one or more embodiments of the present disclosure, in the video generation method provided by the present disclosure, optionally, based on the playback duration of the target music, the number of the target pictures, or the rhythm of the target music At least one of determining the switching moment of each of the target pictures in the target video includes: determining the switching moment of each of the target pictures in the target video according to the target duration of playing of each of the target pictures in the target video. The switching time is such that the playback duration of each target picture in the target video is the target duration.

According to one or more embodiments of the present disclosure, in the video generation method provided by the present disclosure, optionally, based on the playback duration of the target music, the number of the target pictures, or the rhythm of the target music At least one of determining the switching moment of each target picture in the target video includes: determining the accent rhythm point of the target music according to the rhythm of the target music; determining the accent rhythm point as the target picture The switching moment in the target video.

According to one or more embodiments of the present disclosure, in the video generation method provided by the present disclosure, optionally, the synthesis of the target music and the target picture to obtain the target video includes: based on the rhythm of the target music Determine the switching moment of the special effects in the target video; generate the target video according to the switching moment of the target picture in the target video and the switching moment of the special effects.

According to one or more embodiments of the present disclosure, the present disclosure provides a video generation device, including: a first determination module for determining content characteristics of a target picture; a second determination module for determining based on the content characteristics Target music, the rhythm of the target music matches the content characteristics; a processing module, used to synthesize the target music and the target picture to obtain a target video, the switching rhythm of the image material in the target video matches the Match the tempo of the target music.

According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

one or more processors;

Memory, used to store one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement any of the video generation methods provided by this disclosure.

According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the video generation as described in any one provided by the present disclosure is implemented. method.

According to one or more embodiments of the present disclosure, the present disclosure provides a computer program including: instructions that, when executed by a processor, cause the processor to perform the video generation method as described above.

The above description is only a description of the preferred embodiments of the present disclosure and the technical principles applied. Those skilled in the art should understand that the disclosure scope involved in the present disclosure is not limited to technical solutions composed of specific combinations of the above technical features, but should also cover solutions composed of the above technical features or without departing from the above disclosed concept. Other technical solutions formed by any combination of equivalent features. For example, a technical solution is formed by replacing the above features with technical features with similar functions disclosed in this disclosure (but not limited to).

Furthermore, although operations are depicted in a specific order, this should not be understood as requiring that these operations be performed in the specific order shown or performed in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims (15)

A video generation method including: Determine the content characteristics of the target image; Determine target music based on the content characteristics, the tempo of the target music matches the content characteristics; and The target music and the target picture are synthesized to obtain a target video, and the switching rhythm of the image material in the target video matches the rhythm of the target music. The video generation method according to claim 1, wherein determining the content characteristics of the target picture includes: Determine the picture characteristics of the target picture through a picture characteristics determination algorithm; Determine the emotional characteristics of the target picture based on the picture characteristics of the target picture; and The content characteristics of the target picture are determined according to the emotional characteristics of the target picture. The video generation method according to claim 2, wherein the picture characteristics include at least one of a color of the target picture, an object category in the target picture, or an action of the object in the target picture. The video generation method according to claim 2 or 3, wherein the determining the emotional characteristics of the target picture based on the picture characteristics of the target picture includes: Based on the mapping relationship between the picture features and the emotional features, the emotional features that have a mapping relationship with the picture features of the target picture are determined. The video generation method according to any one of claims 2 to 4, wherein the number of the target pictures is multiple, each target picture corresponds to an emotional feature, and the emotional feature according to the target picture Determine the content characteristics of the target image, including: The emotional feature with the largest number of corresponding target images is selected as the content feature. The video generation method according to any one of claims 1 to 5, wherein said synthesizing the target music and the target picture to obtain the target video includes: Determine the switching moment of each of the target pictures in the target video based on at least one of the playback duration of the target music, the number of the target pictures, or the rhythm of the target music; and The target video is generated according to the switching moment. The video generation method according to claim 6, wherein each of the target pictures is determined based on at least one of a playing duration of the target music, a number of the target pictures, or a rhythm of the target music. The switching moments in the target video include: Evenly divide the playback duration of the target music according to the number of the target pictures; Each segmentation point is determined as the switching moment of the target picture in the target video, so that the playback duration of each target picture in the target video is the same. The video generation method according to claim 6 or 7, wherein the determination of each of the target pictures is based on at least one of the playback duration of the target music, the number of the target pictures, or the rhythm of the target music. The switching moments in the target video include: According to the target playing time of each target picture in the target video, the switching moment of each target picture in the target video is determined, so that the playing time of each target picture in the target video is Target duration. The video generation method according to any one of claims 6 to 8, wherein the determination of each location is based on at least one of a playback duration of the target music, a number of the target pictures, or a rhythm of the target music. The switching moment of the target picture in the target video includes: Determine the accent rhythm point of the target music according to the rhythm of the target music; and The accent rhythm point is determined as the switching moment of the target picture in the target video. The video generation method according to any one of claims 6 to 9, wherein said synthesizing the target music and the target picture to obtain the target video includes: Determine the switching moment of the special effects in the target video based on the rhythm of the target music; and The target video is generated according to the switching moment of the target picture in the target video and the switching moment of the special effect. The video generation method according to any one of claims 1 to 10, wherein the image material includes at least one of a target picture or a special effect. A video generation device including: The first determination module is used to determine the content characteristics of the target image; a second determination module, configured to determine target music based on the content characteristics, the rhythm of the target music matching the content characteristics; and A processing module configured to synthesize the target music and the target picture to obtain a target video, where the switching rhythm of the image material in the target video matches the rhythm of the target music. An electronic device including: memory; and A processor coupled to the memory, the processor configured to perform the method of any one of claims 1-11 based on instructions stored in the memory. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the video generation method according to any one of claims 1-11 is implemented. A computer program consisting of: Instructions that, when executed by a processor, cause the processor to perform the video generation method according to any one of claims 1-11.