WO2019023953A1 - Video editing method and video editing system based on intelligent terminal - Google Patents

Abstract

A video editing method based on a smart terminal and a video editing system are provided. The video editing method comprises the steps of: acquiring a video file to be edited and storing in the smart terminal; acquiring a portrait image of a person having a portrait element, and extracting a character portrait feature of the character portrait element; acquiring a video segment of the character to be clipped that matches the character portrait feature; and remaining video of the video clip or the video to be clipped except the video clip The segments are stitched together. After adopting the above technical solution, the smart picture can be realized according to the user's demand combined with the picture of the person imported by the user, and a complete video with or without a certain character or multiple characters can be formed, and an interactive interface is provided, and the extracted video segment is again Screening to improve accuracy.

Description

Video editing method and video editing system based on intelligent terminal Technical field

The present invention relates to the field of smart devices, and in particular, to a video editing method and a video editing system based on a smart terminal.

Background technique

With the diversified development of video programs, TV programs, movies and other video programs have become an indispensable part of people's lives. People have certain preferences for plots and characters in video programs, and often want to see an actor but People don't want to watch the whole drama or the whole movie, people generally take the means of fast-forward, but this method is not only a waste of time, but also easy to miss the story. In the prior art, there is an algorithm for extracting video segments of a person of interest, but the accuracy is not high.

Therefore, the image recognition algorithm distinguishes the back of the target person from the front picture, separately compares and provides a method of video editing in combination with the user's needs, and is applicable not only to the video program but also to the life of the user. Video, you can edit a video containing a specific character, or you can edit a video that contains multiple characters, or a video that does not contain a certain character, and provide a video containing the back of the target person, saving time and accuracy. And by filtering the extracted content by interacting with the user, again improving the accuracy.

Summary of the invention

In order to overcome the above technical deficiencies, an object of the present invention is to provide a video editing method and a video editing system based on a smart terminal, which can perform a dedicated video clip with or without a certain character or a plurality of characters according to the needs of the user, and Fast and convenient, high precision and time saving.

The invention discloses a video editing method based on a smart terminal, comprising the following steps:

Obtaining a video file to be edited and storing in the smart terminal;

Obtaining a portrait image of a person having a portrait element and extracting a portrait feature of the portrait element;

Obtaining, in the video to be edited, a video segment of a character that matches the character portrait feature;

Splicing the video clip or the remaining video clips other than the video clip in the video to be clipped.

Preferably, the step of acquiring a person portrait picture having a portrait element and extracting the person portrait feature of the person portrait element comprises:

Obtaining a portrait picture of a person having a portrait element and storing it in the smart terminal;

Identifying a portrait element in the portrait image of the person;

The body contour feature of the character portrait element is extracted as a first body contour feature, and the facial portrait feature of the person portrait element is extracted as a first facial portrait feature.

Preferably, the step of acquiring the video clip of the character to be clipped that matches the character portrait feature comprises:

Splitting the video to be clipped, obtaining each frame of the picture, and extracting the to-be-obtained portrait elements in the frame of each frame, including the character back element and the character face element;

Extracting a body contour feature of the character back element as a second body shape feature, and extracting a face portrait feature of the character face element as a second face portrait feature;

Comparing the second body contour feature with the first body contour feature, and obtaining a picture corresponding to the second body contour feature when the similarity is greater than or equal to the first similarity threshold is a character back image;

Comparing the second facial portrait feature with the first facial portrait feature, and acquiring a picture corresponding to the second facial portrait feature when the similarity is greater than or equal to the second similarity threshold is a front view of the character;

Cutting the character back picture from the character front picture from the to-be-edited video to form a video segment.

Preferably, the video clip or the remaining video clips other than the video clip in the video to be clipped are spelled The steps to follow include:

Separating the audio information and the video information in the remaining video segments except the video segment in the video clip or the video to be clipped to form an audio portion and a video portion;

Separating the audio portion and the video portion separately to form a complete audio portion and a complete video portion;

Synchronizing the complete audio portion with the full video portion.

Preferably, the step of acquiring a video segment of the character in the video to be clipped that matches the character portrait feature is splicing the remaining video segments other than the video segment in the video segment or the video to be clipped Between the steps, the video editing method further includes:

The video clip is pushed to the user and filtered by the user to remove irrelevant video clips.

The invention also discloses a video editing system based on a smart terminal, comprising:

a video acquisition module, which acquires a video file to be edited and stores the video file in the smart terminal;

a portrait feature extraction module, acquiring a portrait image of a person having a portrait element, and extracting a portrait feature of the portrait element;

a video segment acquisition module, configured to connect to the video acquisition module and the portrait feature extraction module, and acquire a video segment of the to-be-edited video that includes a character that matches the character portrait feature;

The video splicing module is connected to the video segment obtaining module to splicing the video clip or the remaining video segments except the video segment in the video to be clipped.

Preferably, the portrait feature extraction module comprises:

a picture obtaining unit, which acquires a portrait picture of a person having a portrait element and stores it in the smart terminal;

a portrait element identification unit connected to the picture acquisition unit to identify a person portrait element in the person portrait picture;

a portrait feature extraction unit, coupled to the portrait element recognition unit, extracting a body profile feature of the person portrait element as a first body profile feature, and extracting a face portrait feature of the person portrait element as a first face portrait feature .

Preferably, the video segment obtaining module includes:

An element extracting unit splits the video to be clipped, obtains a frame of each frame, and extracts a pair of portrait elements in the frame of each frame, including a character back element and a character face element;

a feature extraction unit, connected to the element extraction unit, extracting a body contour feature of the character back image element as a second body shape contour feature, and extracting a face portrait feature of the character face element as a second face portrait feature;

a back image acquisition unit, connected to the feature extraction unit, and comparing the second body contour feature with the first body contour feature, and acquiring the second when the similarity is greater than or equal to the first similarity threshold The picture corresponding to the figure outline feature is a character back picture;

a front view acquiring unit, connected to the feature extracting unit, comparing the second facial portrait feature with the first facial portrait feature, and acquiring the second facial when the similarity is greater than or equal to a second similarity threshold The picture corresponding to the portrait feature is the front view of the character;

The cutting unit is connected to the back view picture acquiring unit and the front picture acquiring unit, and cuts the character back picture from the character front picture from the to-be-edited video to form a video segment.

Preferably, the video splicing module comprises:

a separating unit, separating audio information and video information in the video segment to form an audio portion and a video portion;

a splicing unit, connected to the separating unit, splicing the audio portion and the video portion separately to form a complete audio portion and a complete video portion;

a synchronization unit coupled to the tiling unit to synchronize the complete audio portion with the complete video portion.

Preferably, between the video segment acquisition module and the video splicing module, the video editing system further includes:

a video clip screening module that pushes the video clip to a user, and the user filters the video to eliminate irrelevant video Fragment.

After adopting the above technical solution, compared with the prior art, the following beneficial effects are obtained:

1. Meet the user's need to acquire a dedicated video with or without a character or multiple characters;

2. The back of the target person can be identified to obtain a video clip containing or not including a character or a plurality of characters;

3. Fast and convenient, high accuracy;

DRAWINGS

1 is a flow chart showing a video editing method in accordance with a preferred embodiment of the present invention;

2 is a flow chart showing a method for extracting a portrait feature of a video editing method in accordance with a preferred embodiment of the present invention;

3 is a flow chart showing a method for acquiring a video clip by a video editing method in accordance with a preferred embodiment of the present invention;

4 is a schematic flow chart of a method for splicing a video clip or a video clip other than a video clip in a video clip or a video to be clipped according to a preferred embodiment of the present invention;

FIG. 5 is a schematic flow chart of a video editing method according to another preferred embodiment of the present invention; FIG.

Figure 6 is a block diagram showing the structure of a video editing system in accordance with a preferred embodiment of the present invention.

FIG. 7 is a schematic structural diagram of a portrait feature extraction module of a video editing system in accordance with a preferred embodiment of the present invention.

FIG. 8 is a schematic structural diagram of a video clip acquiring module of a video editing system in accordance with a preferred embodiment of the present invention.

FIG. 9 is a schematic structural diagram of a video splicing module of a video editing system in accordance with a preferred embodiment of the present invention.

FIG. 10 is a schematic structural diagram of a system of a video editing system in accordance with another preferred embodiment of the present invention.

Reference mark:

100-video editing system; 11-video acquisition module; 12-video segment acquisition module; 13-portrait feature extraction module; 14-video splicing module; 15-video segment screening module.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. The following description refers to the same or similar elements in the different figures unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with aspects of the present disclosure as detailed in the appended claims.

The terms used in the present disclosure are for the purpose of describing particular embodiments only, and are not intended to limit the disclosure. The singular forms "a", "the" and "the" It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

In the description of the present invention, the specific meaning of the above terms may be understood by one of ordinary skill in the art, unless otherwise specified and defined.

In the following description, the use of suffixes such as "module", "component" or "unit" for indicating an element is merely an explanation for facilitating the present invention, and does not have a specific meaning per se. Therefore, "module" and "component" can be used in combination.

The mobile terminal can be implemented in various forms. For example, the terminal described in the present invention may include a mobile terminal such as a mobile phone, a smart phone, a notebook computer, a PDA (Personal Digital Assistant), a PAD (Tablet), a PMP (Portable Multimedia Player), a navigation device, and the like, and such as Fixed terminal for digital TV, desktop computer, etc.

1 is a schematic flowchart of a video clip method based on a smart terminal according to a preferred embodiment of the present invention. In this embodiment, the video editing method specifically includes the following steps:

S100: Acquire a video file to be edited and store it in the smart terminal.

In order to implement the video clip, the video file to be edited must first be obtained. The method for obtaining the video file to be edited includes importing the video in the smart terminal, and importing the video from outside the smart terminal and storing it in the smart terminal. The video file to be edited imported here must contain the target person that the user wants to edit. If the user imports the video error, that is, does not include the target person who wants to edit, there will be no result when the subsequent image recognition acquires the video clip, and Remind users that they have not obtained the relevant video. Please check if the clip video file or the target portrait image is imported incorrectly.

S200: Acquire a portrait image of a person with a portrait element, and extract a portrait feature of the portrait element

After obtaining the video to be edited, in order to realize the video clip centered on the target person, it is necessary to acquire a portrait image having a portrait element and extract the portrait feature of the portrait element in the image. The method of obtaining a portrait picture includes not only importing the picture in the smart terminal, but also importing the picture from outside the smart terminal and storing it in the smart terminal. The portrait image imported here must be closely integrated with the user's needs. If the user needs a frontal portrait video clip of the target person, the user-provided portrait image must contain the facial element of the target person, if the user needs a video containing the target person's back. Fragment, the user-provided portrait image must contain the back of the target character. At the same time, when the number of target characters is 1, the user-provided portrait image must also be a picture containing the target person alone; when the number of target characters is greater than one The user needs to provide the corresponding portrait image of the target person, and no other person except the target person may appear in all the pictures, but may be a portrait picture of the person who simultaneously includes only a plurality of target characters. In order to provide the accuracy of the results, it is recommended that the user intercepts the image satisfying the above requirements from the video to be clipped as a portrait image, and the comparison result is more accurate.

S300: Obtain a video segment of the video to be edited that includes a character that matches the character portrait feature

After obtaining the video file to be edited and the portrait image and extracting the portrait feature, the characters of the picture in the video need to be compared according to the extracted portrait feature to obtain a video segment containing the character corresponding to the extracted portrait feature. In this process, it is necessary to establish a strategy according to the needs of the user, including the distinction between the frontal image of the target person and the back image of the target person, the distinction between the number of target characters, and the distinction between the target person and other characters. First, if the video clip that the user needs is a screen containing only the front portrait of the target person, only the front portrait of the target person needs to be obtained. If the video clip that the user needs is only the image of the target person's back, only the target person needs to be acquired. The picture of the back, if both are needed, there should be a logical OR relationship between the picture containing the frontal portrait of the target person and the picture containing the back of the target person. Secondly, when the number of target characters of the user exceeds 1, the relationship between the characters needs to be considered. When the user needs the screen of the simultaneous appearance of each target character, the relationship between the portrait features of each target character should be logical and when the user needs When any of the target characters appears, the relationship between the portrait features of each target person should be logical or. In addition, the logical relationship between the target characters can be determined by the user's needs, such as two of them. Logic and relationship, the other is a logical OR relationship with the two. Take TV dramas as an example. If the user needs all the highlights of a supporting actor and the heroine, then the two characters should appear at the same time. The relationship between the two should be logical and the only picture of both characters appears. Both the back and the front. Finally, regarding the distinction between the target person and other characters, if the user needs to include only the picture of the target person, the number of characters extracted in the picture should be consistent with the number of target characters in the picture.

The process of obtaining a video clip of a character to be clipped that matches the character of the target person is as follows: the clipped video is framed, and each frame is acquired, through image transform technology, image enhancement technology, image recognition technology, and image segmentation technology. Extracting the portrait elements in each frame, and extracting the portrait features in the portrait elements by sampling, and comparing them with the portrait elements extracted from the target figures, if the two match, the picture That is, the picture containing the target person, the connected picture forms a video segment.

In the above TV drama example, the portrait feature is divided into a facial portrait feature and a body contour feature. If the facial portrait feature of the supporting role is A1, the body contour feature is A2, and the facial character feature of the heroine is B1, the body contour feature For B2, the number of portrait characters in the picture is N, then only the supporting actor plays against the heroine, including the front. Also includes the back, and the logical relationship of the features that should not be included in the picture of others should be (A1 or A2) and (B1 or B2) and (N=2).

S400: splicing the video clip or the remaining video clips except the video clip in the video to be clipped

After obtaining the video clip of the character to be clipped that matches the character of the target person, the clip needs to be stitched, and the stitching should be in a certain order, either in chronological order or according to the characters in the screen. More or less in the order, in which the change of characters from small to many is the change from only the target person to the other person, from the opposite to the other, it can also be small according to the proportion of the target person in the video picture. In the order of large or small to large, the latter two sequences should be supplemented by chronological order. For example, the order of the target person's proportion of the video screen is taken as an example. The ratio of the character to the video screen can be calculated by dividing the area of the portrait element in the video screen by the area of the video screen, and calculating the ratio. It should be done after identifying each frame of the picture. When the proportion of the character of a certain picture is relatively high, if the picture is in a selected video segment, the video segment is regarded as one body, and the time ratio is spliced regardless of the proportion of the other pictures in the video segment. Prevent the splicing of the screen from being discontinuous due to the splicing of the ratio. It is equivalent to comparing the maximum ratio of the pictures in the video clip, and splicing in the order of the largest ratio. At the same time, when the ratio is the same, it is also spliced in chronological order.

In addition to splicing the acquired video clips, when the user needs to include a video of a certain character or a plurality of characters, it is necessary to splicing the remaining video clips other than the obtained video clips in the video to be clipped, and removing the acquired video clips. Video clip.

With the above configuration, the user can quickly and accurately obtain the exclusive video with or without a certain character or a plurality of characters according to the user's needs, and can accurately identify the video segment containing the back of the target person.

Referring to FIG. 2, in a preferred embodiment, the step of acquiring a portrait image of a person with a portrait element and extracting the portrait feature of the portrait element includes:

S201: Acquire a portrait picture of a person having a portrait element and store it in the smart terminal.

After obtaining the video to be edited, in order to realize the video clip centered on the target person, it is necessary to obtain a portrait image having the portrait element, and the method of obtaining the portrait image includes both importing the image in the smart terminal and importing from outside the smart terminal. The picture is stored in the smart terminal.

S202: Identify a portrait element in the portrait image of the person

After obtaining a portrait image with a portrait element, since there may be a background with disturbing factors in the image, it is necessary to extract the portrait element in the image. Here, first, through image transformation, such as Fourier transform, Wo The Ershi-Adama transform and the discrete Kafner-Levy transform transform the image from the time domain to the frequency domain, and then enhance the high-frequency abrupt components in the frequency domain image by image enhancement technology to enhance the edge of the image and enhance the edge of the image. After that, image recognition technology is needed to extract the portrait elements in the picture by extracting features, establishing indexes, and query steps, and extracting the portrait elements by image segmentation technology. The extraction feature operation here is based on the external corresponding portrait database, and the recognition model of different portrait elements is established by sampling the portrait elements in the portrait database to distinguish different portrait elements, such as a large number of facial portraits in the database. The recognition model for creating a facial portrait is sampled, and the model is used for recognition in the process of recognizing a portrait. When a portion consistent with the model is recognized in the image, the portion is considered to be a facial portrait element.

S203: extracting a body contour feature of the character portrait element as a first body contour feature, and extracting a facial portrait feature of the character portrait element as a first facial portrait feature

After extracting the portrait element, the portrait feature needs to be extracted. Here, the portrait image should be distinguished. When the portrait element in the portrait image is the back of the character, the outline feature of the character's back should be extracted, including the body contour. The proportion of each part and other features form a first figure outline feature; when the portrait element in the portrait picture is a frontal portrait of the person, the facial portrait features of the frontal portrait of the person should be extracted, including facial skin color, facial features, size, position distance Relationships and features with recognizable features on the face, such as black squats in the corners of the mouth, can also be uniformly sampled on the face and the size of the face portrait in the picture.

Referring to FIG. 3, in a preferred embodiment, acquiring the video to be edited includes matching the character portrait feature. The steps of the video clip of the character may specifically include:

S301: split the video to be clipped, obtain a frame of each frame, and extract a to-be-obtained portrait element in each frame, including a character back element and a character face element.

In order to identify the portrait of a person in a clip video, it is necessary to first frame the video to form a frame of one frame and one frame, and extract the portrait element in each frame, first through image transformation, such as Fourier transform. The Walsh-Hadamard transform and the discrete Kafner-Levy transform transform the image from the time domain to the frequency domain, and then enhance the high-frequency abrupt component in the frequency domain image by image enhancement technology to enhance the image edge and image edge. After being enhanced, it is necessary to identify the portrait elements in the image by extracting features, indexing and query steps through image recognition technology, and finally extracting the portrait elements by image segmentation technology, and the extracted portrait elements include the characters of the characters and the facial elements of the characters. . The extraction feature operation here is based on the external corresponding portrait database, and the recognition model of different portrait elements is established by sampling the portrait elements in the portrait database to distinguish different portrait elements, such as a large number of facial portraits in the database. The recognition model for creating a facial portrait is sampled, and the model is used for recognition in the process of recognizing a portrait. When a portion consistent with the model is recognized in the image, the portion is considered to be a facial portrait element.

S302: extracting a body contour feature of the character back element as a second body shape feature, and extracting a face portrait feature of the character face element as a second face portrait feature

After extracting the back elements of the character in the picture to be edited, it is necessary to sample the back elements of the character, extract the contours of the body contour and the proportion of each part to form the second body contour feature. The extraction method here should be The method of extracting the first figure outline feature of the character is consistent; after extracting the face part of the character in the picture of the video to be edited, the face part of the character is sampled, and the facial skin color, the facial features shape, the position distance relationship, and the face are extracted. A feature having a recognizable feature, such as a facial portrait feature of a black scorpion of the corner of the mouth, forms a second facial portrait feature, and the extraction method herein should be consistent with the method of extracting the first facial portrait feature of the character. There is a possibility that the target person may wear sunglasses, so it is necessary to delete the sunglasses portion when extracting the facial portrait feature, and only the remaining facial portrait portion is considered. The step of deleting the sunglasses portion includes forming features such as contour lines and colors by sampling a large number of sunglasses database. The constructed sunglasses model is indexed by the sunglasses model indexing. When the part matching the sunglasses model is inquired, the part is considered to be the sunglasses part, and the part is deleted.

S303: Align the second body contour feature with the first body contour feature, and obtain a picture corresponding to the second body contour feature when the similarity is greater than or equal to the first similarity threshold.

After obtaining the second body contour feature and the first body contour feature, the first body contour feature is indexed according to the first body contour feature, and the second body contour feature is scaled to the same size as the first body contour feature, and according to the index pair The scaled second body contour feature is sampled and searched, such as whether the body contour line is consistent, whether the proportion of each part of the body is consistent, etc., and the first similarity threshold is 90%, and the coincidence degree is greater than or equal to the first similarity. The threshold value is considered to be greater than or equal to the first similarity threshold value, and the person corresponding to the second figure-shaped contour feature and the first figure-shaped contour feature is considered to be The same person obtains a picture corresponding to the contour feature of the second figure, and is a picture of the character's back. Here, the standard of the first similarity threshold can be adjusted up and down to meet a certain recognition accuracy. Since the back recognition is difficult and easy to identify errors, setting a higher similarity threshold is beneficial to improve the accuracy. To prevent the missing picture when the similarity threshold is too high, the similarity can be set between 85% and 90%. This screen pops up, and the user selects whether to obtain the screen, thereby reducing the omission of the screen.

S304: Align the second facial portrait feature with the first facial portrait feature, and obtain a screen corresponding to the second facial portrait feature when the similarity is greater than or equal to the second similarity threshold.

After acquiring the second facial portrait feature and the first facial portrait feature, indexing is performed according to the first facial portrait feature, and the second facial portrait feature is scaled to the same size as the first facial portrait feature, and is scaled according to the index pair The second facial portrait feature is compared, such as whether the facial skin color is the same, whether the facial shape shape, the position distance relationship are consistent, and the facial recognition feature, such as whether the black corner of the mouth is the same, etc., and the second similarity threshold is taken. 85%, when the degree of coincidence is greater than or equal to the second similarity threshold, the second facial portrait feature is considered to be the first facial portrait The similarity of the feature is greater than or equal to the second similarity threshold, and the person corresponding to the second facial portrait feature is considered to be the same person as the first facial portrait feature, and the image corresponding to the second facial portrait feature is obtained as the front of the character. Picture. Here, the standard of the second similarity threshold can be adjusted up and down to meet a certain recognition accuracy. Since the facial portrait features are more and the alignment is more accurate, the facial portrait feature comparison threshold, that is, the second similarity threshold, is slightly lower than the body contour feature comparison threshold, that is, the first similarity threshold. Similarly, for the comparison of facial portrait features, setting a higher similarity threshold is beneficial to improve the accuracy. In order to prevent missing images when the similarity threshold is too high, the screen can be popped up when the similarity is between 80% and 85%. The user selects whether to obtain the picture, thereby reducing the omission of the picture.

S305: Cut the character back picture from the character to be cut from the to-be-edited video to form a video segment.

After obtaining the back image of the character and the front picture of the character, whether the picture of the adjacent frame is also acquired is obtained. When the picture of the adjacent frame is acquired, the continuous frame picture is regarded as an integral cut from the video to be clipped to form a video segment. .

Referring to FIG. 4, in a preferred embodiment, the step of splicing the video clips or the remaining video clips other than the video clips in the video to be clipped includes:

S401: Separating the audio information and the video information in the remaining video segments except the video segment in the video clip or the video to be clipped to form an audio portion and a video portion.

After obtaining the remaining video segments of the video clip or the video to be clip except the video clip, the audio information in each clip needs to be separated from the video information, where the video information does not include audio information, and the audio information needs to be recorded. The positional relationship of the video information, and the audio information and the video information are extracted to form an audio part and a video part.

S402: splicing the audio portion and the video portion separately to form a complete audio portion and a complete video portion

After acquiring the audio portion and the video portion of each segment, the audio portion and the video portion unit need to be sequentially stitched together to form a complete audio portion composed entirely of the audio portion and a complete video portion composed entirely of the video portion.

S403: Synchronize the complete audio portion with the complete video portion

After obtaining the complete audio portion and the complete video portion, the complete audio portion is synchronized with the complete video portion according to the recorded positional relationship between the audio information and the video information to form a final complete video.

5 is a schematic flowchart of a video clip method based on a smart terminal according to another preferred embodiment of the present invention. In this embodiment, the step of acquiring a video segment of the character corresponding to the character portrait feature in the video to be edited and performing the remaining video segment except the video segment in the video segment or the video to be clipped Between the stitching steps, the video editing method further includes the following steps:

S500: Push the video clip to a user, and the user performs screening to remove irrelevant video clips.

After obtaining the video clip, in order to further improve the accuracy, the user can perform screening by pushing the obtained video clip to the user, and can perform the delete operation to eliminate the discriminating unrelated video clip.

Referring to FIG. 6, a smart terminal-based video editing system 100 in accordance with a preferred embodiment of the present invention specifically includes the following components:

In order to implement the video clip, the video capture module 11 must first obtain the video file to be edited, and the method for obtaining the video file to be edited includes not only importing the video in the smart terminal, but also importing the video from outside the smart terminal and storing it in the smart terminal. Inside. The video file to be edited imported here must contain the target person that the user wants to edit. If the user imports the video error, that is, does not include the target person who wants to edit, there will be no result when the subsequent image recognition acquires the video clip, and Remind users that they have not obtained the relevant video. Please check if the clip video file or the target portrait image is imported incorrectly.

The portrait feature extraction module 13 is configured to acquire a character portrait image having a portrait element and extract a character portrait of the portrait element in the image after the video to be clipped is acquired. Sign. The method of obtaining a portrait picture includes not only importing the picture in the smart terminal, but also importing the picture from outside the smart terminal and storing it in the smart terminal. The portrait image imported here must be closely integrated with the user's needs. If the user needs a frontal portrait video clip of the target person, the user-provided portrait image must contain the facial element of the target person, if the user needs a video containing the target person's back. Fragment, the user-provided portrait image must contain the back of the target character. At the same time, when the number of target characters is 1, the user-provided portrait image must also be a picture containing the target person alone; when the number of target characters is greater than one The user needs to provide the corresponding portrait image of the target person, and no other person except the target person may appear in all the pictures, but may be a portrait picture of the person who simultaneously includes only a plurality of target characters. In order to provide the accuracy of the results, it is recommended that the user intercepts the image satisfying the above requirements from the video to be clipped as a portrait image, and the comparison result is more accurate.

The video segment obtaining module 12 is connected to the video acquiring module 11 and the portrait feature extracting module 13 to obtain a to-be-edited video file and a portrait image and extract the character portrait feature, and then needs to be based on the extracted character portrait feature in the video. The characters of the screen are compared to obtain a video clip containing the character corresponding to the extracted portrait feature. In this process, it is necessary to establish a strategy according to the needs of the user, including the distinction between the frontal image of the target person and the back image of the target person, the distinction between the number of target characters, and the distinction between the target person and other characters. First, if the video clip that the user needs is a screen containing only the front portrait of the target person, only the front portrait of the target person needs to be obtained. If the video clip that the user needs is only the image of the target person's back, only the target person needs to be acquired. The picture of the back, if both are needed, there should be a logical OR relationship between the picture containing the frontal portrait of the target person and the picture containing the back of the target person. Secondly, when the number of target characters of the user exceeds 1, the relationship between the characters needs to be considered. When the user needs the screen of the simultaneous appearance of each target character, the relationship between the portrait features of each target character should be logical and when the user needs When any of the target characters appears, the relationship between the portrait features of each target person should be logical or. In addition, the logical relationship between the target characters can be determined by the user's needs, such as two of them. Logic and relationship, the other is a logical OR relationship with the two. Take TV dramas as an example. If the user needs all the highlights of a supporting actor and the heroine, then the two characters should appear at the same time. The relationship between the two should be logical and the only picture of both characters appears. Both the back and the front. Finally, regarding the distinction between the target person and other characters, if the user needs to include only the picture of the target person, the number of characters extracted in the picture should be consistent with the number of target characters in the picture.

The process of obtaining a video clip of a character to be clipped that matches the character of the target person is as follows: the clipped video is framed, and each frame is acquired, through image transform technology, image enhancement technology, image recognition technology, and image segmentation technology. Extracting the portrait elements in each frame, and extracting the portrait features in the portrait elements by sampling, and comparing them with the portrait elements extracted from the target figures, if the two match, the picture That is, the picture containing the target person, the connected picture forms a video segment.

In the above TV drama example, the portrait feature is divided into a facial portrait feature and a body contour feature. If the facial portrait feature of the supporting role is A1, the body contour feature is A2, and the facial character feature of the heroine is B1, the body contour feature For B2, the number of portrait characters in the picture is N, then only the supporting actor plays against the heroine, including both the front and the back, and the logical relationship of the features that should not be included in the picture of other people should be (A1 or A2) And(B1 or B2)and(N=2).

The video splicing module 14 is connected to the video segment obtaining module 12, and after acquiring the video segment of the character to be clipped that matches the character of the target person, the spliced segment needs to be spliced, and the splicing should be in a certain order. It can be in chronological order or in the order of the characters in the picture from less to more or from more to less. The change of characters from small to many is the change from only the target person to other people, from more to less. Conversely, the order of the target characters in the video screen may be in the order of small to large or large to small, and the latter two sequences shall be supplemented by chronological order. For example, the order of the target person's proportion of the video screen is taken as an example. The ratio of the character to the video screen can be calculated by dividing the area of the portrait element in the video screen by the area of the video screen, and calculating the ratio. It should be done after identifying each frame of the picture. When the proportion of the character of a certain picture is high, if the picture is in a selected video segment, the video segment is regarded as one body, regardless of the proportion of other pictures in the video segment. Low, all spliced in chronological order, preventing the splicing of the picture by the ratio splicing to cause the picture to be broken. It is equivalent to comparing the maximum ratio of the pictures in the video clip, and splicing in the order of the largest ratio. At the same time, when the ratio is the same, it is also spliced in chronological order.

In addition to splicing the acquired video clips, when the user needs to include a video of a certain character or a plurality of characters, it is necessary to splicing the remaining video clips other than the obtained video clips in the video to be clipped, and removing the acquired video clips. Video clip.

Referring to FIG. 7, in a preferred embodiment, the portrait feature extraction module 13 specifically includes:

After the image acquisition unit obtains the video to be edited, in order to realize the video clip centered on the target person, it is necessary to obtain the portrait image of the person having the portrait element, and the manner of obtaining the portrait image includes both the image imported into the smart terminal and the The smart terminal externally imports the picture and stores it in the smart terminal.

The portrait element identification unit is connected to the picture acquisition unit, and after acquiring the person portrait picture having the person portrait element, since there may be a background with interference factors in the picture, the person portrait element in the picture needs to be extracted, here Firstly, the image is transformed from the time domain to the frequency domain by image transform, such as Fourier transform, Walsh-Hadamard transform and discrete Kafner-Levy transform, and then the image in the frequency domain image is high. The frequency mutation component is strengthened, and the edge of the image is strengthened. After the image edge is strengthened, the image recognition technology is used to extract the character portrait element in the image by extracting the feature, establishing the index build and the query step, and extracting the portrait element by the image segmentation technology. The extraction feature operation here is based on the external corresponding portrait database, and the recognition model of different portrait elements is established by sampling the portrait elements in the portrait database to distinguish different portrait elements, such as a large number of facial portraits in the database. The recognition model for creating a facial portrait is sampled, and the model is used for recognition in the process of recognizing a portrait. When a portion consistent with the model is recognized in the image, the portion is considered to be a facial portrait element.

a portrait feature extraction unit is connected to the portrait element recognition unit, and after extracting the portrait element, the portrait feature needs to be extracted, where the portrait image should be distinguished, and when the portrait element in the portrait image is the back of the character, The silhouette of the figure of the character should be extracted, including the contour of the body, the proportion of each part, etc., to form the first figure outline feature; when the portrait element in the portrait picture is the front portrait of the character, the front portrait of the character should be extracted. The facial portrait features, including the facial skin color, the size of the facial features, the positional distance relationship, and the features of the facial recognition, such as the black scorpion of the corner of the mouth, can also be uniformly sampled on the face and record the size of the face portrait in the picture.

Referring to FIG. 8, in a preferred embodiment, the video segment obtaining module 12 specifically includes:

The element extraction unit, in order to recognize the portrait of the person in the clip video, needs to first frame the video to form a frame of one frame and one frame, and extract the portrait element in each frame, first through image transformation, such as Fourier transform, Walsh-Hadamard transform and discrete Kafner-Levy transform transform the image from time domain to frequency domain, and then enhance the high frequency mutation component in the frequency domain image by image enhancement technology to enhance the image. Edge, after the image edge is strengthened, it is necessary to identify the portrait element in the image by extracting features, building an index build and query step through image recognition technology, and finally extracting the portrait element by image segmentation technology, and the extracted character portrait element includes the character back view. Elements and character facial elements. The extraction feature operation here is based on the external corresponding portrait database, and the recognition model of different portrait elements is established by sampling the portrait elements in the portrait database to distinguish different portrait elements, such as a large number of facial portraits in the database. The recognition model for creating a facial portrait is sampled, and the model is used for recognition in the process of recognizing a portrait. When a portion consistent with the model is recognized in the image, the portion is considered to be a facial portrait element.

The feature extraction unit is connected to the element extraction unit, and after extracting the back view element of the character in the picture of the video to be edited, the back view element of the character is sampled, and the outline of the body, the proportion of each part, and the like are extracted to form a second shape. The outline contour feature, the extraction method here should be consistent with the method of extracting the first figure outline feature of the character; after extracting the face element of the character in the picture of the video to be edited, the face element of the character needs to be sampled, and the extraction includes Facial skin color, facial shape shape, position distance relationship, and facial recognition features, such as the corner of the mouth The facial portrait features of features such as black scorpion form a second facial portrait feature, and the extraction method here should be consistent with the method of extracting the first facial portrait feature of the character. There is a possibility that the target person may wear sunglasses, so it is necessary to delete the sunglasses portion when extracting the facial portrait feature, and only the remaining facial portrait portion is considered. The step of deleting the sunglasses portion includes forming features such as contour lines and colors by sampling a large number of sunglasses database. The constructed sunglasses model is indexed by the sunglasses model indexing. When the part matching the sunglasses model is inquired, the part is considered to be the sunglasses part, and the part is deleted.

a back image acquisition unit is connected to the feature extraction unit, and after acquiring the second body contour feature and the first body contour feature, the index is established according to the first body contour feature, and the second body contour feature is scaled to The shape of the contour is the same size, and the scaled second contour contour feature is sampled according to the index, such as whether the body contour line is consistent, whether the proportion of each part of the body is consistent, etc., taking the first similarity threshold 90%, when the degree of coincidence is greater than or equal to the first similarity threshold, the similarity between the second body contour feature and the first body contour feature is considered to be greater than or equal to the first similarity threshold, and the second body contour feature is considered to correspond to The character corresponding to the first figure contour feature is the same person, and the picture corresponding to the second body shape contour feature is obtained, which is a character back picture. Here, the standard of the first similarity threshold can be adjusted up and down to meet a certain recognition accuracy. Since the back recognition is difficult and easy to identify errors, setting a higher similarity threshold is beneficial to improve the accuracy. To prevent the missing picture when the similarity threshold is too high, the similarity can be set between 85% and 90%. This screen pops up, and the user selects whether to obtain the screen, thereby reducing the omission of the screen.

a front picture acquiring unit, connected to the feature extracting unit, after acquiring the second facial portrait feature and the first facial portrait feature, indexing according to the first facial portrait feature, and scaling the second facial portrait feature to the first face The portrait features the same size, and the scaled second facial portrait features are compared according to the index, such as whether the facial skin color is the same, the facial features, the positional distance relationship, and the facial recognition feature, such as the black corner of the mouth. Whether the 相似 is the same or the like, the second similarity threshold is 85%, and when the degree of coincidence is greater than or equal to the second similarity threshold, the similarity between the second facial portrait feature and the first facial portrait feature is considered to be greater than or equal to the second similarity. The degree threshold is that the person corresponding to the second facial portrait feature and the person corresponding to the first facial portrait feature are the same person, and the screen corresponding to the second facial portrait feature is acquired as the front view of the character. Here, the standard of the second similarity threshold can be adjusted up and down to meet a certain recognition accuracy. Since the facial portrait features are more and the alignment is more accurate, the facial portrait feature comparison threshold, that is, the second similarity threshold, is slightly lower than the body contour feature comparison threshold, that is, the first similarity threshold. Similarly, for the comparison of facial portrait features, setting a higher similarity threshold is beneficial to improve the accuracy. In order to prevent missing images when the similarity threshold is too high, the screen can be popped up when the similarity is between 80% and 85%. The user selects whether to obtain the picture, thereby reducing the omission of the picture.

The cutting unit is connected to the back image acquiring unit and the front screen acquiring unit, and after acquiring the back image of the character and the front view of the character, whether the image of the adjacent frame is also acquired, when the image of the adjacent frame is acquired The continuous frame picture is regarded as an integral cut from the video to be clipped to form a video segment.

Referring to FIG. 9, in a preferred embodiment, the video splicing module 14 specifically includes:

The separating unit, after acquiring the video clip or the remaining video clips except the video clip in the video to be clipped, separates the audio information in each segment from the video information, where the video information does not include the audio information, and needs to be recorded. The positional relationship between the audio information and the video information, and the audio information and the video information are extracted to form an audio part and a video part.

After the splicing unit obtains the audio portion and the video portion of each segment, the audio portion and the video portion unit need to be spliced in order to form a complete audio portion composed entirely of the audio portion and a complete video portion composed entirely of the video portion.

The synchronization unit, after acquiring the complete audio part and the complete video part, synchronizes the complete audio part with the complete video part according to the recorded position relationship of the audio information and the video information to form a final complete video.

Referring to FIG. 10, in accordance with a smart terminal-based video editing system 100 in accordance with another preferred embodiment of the present invention, between the video segment acquisition module 12 and the video splicing module 14, the video editing system 100 further includes the following components:

The video segment screening module 15 obtains the video segment, and further improves the accuracy. By pushing the obtained video segment to the user, the user performs screening, and the deletion operation may be performed to remove the unrelated video segment that identifies the error.

Other embodiments of the present disclosure will be apparent to those skilled in the <RTIgt; The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which are in accordance with the general principles of the present disclosure and include common general knowledge or conventional technical means in the art that are not disclosed in the present disclosure. . The specification and examples are to be regarded as illustrative only,

It should be noted that the embodiments of the present invention are preferred embodiments, and are not intended to limit the scope of the present invention. Any one skilled in the art may use the above-disclosed technical contents to change or modify the equivalent embodiments. Any modification or equivalent changes and modifications of the above embodiments in accordance with the technical spirit of the present invention are still within the scope of the technical solutions of the present invention.

Claims (10)

A video editing method based on a smart terminal, comprising the following steps: Obtaining a video file to be edited and storing in the smart terminal; Obtaining a portrait image of a person having a portrait element and extracting a portrait feature of the portrait element; Obtaining, in the video to be edited, a video segment of a character that matches the character portrait feature; Splicing the video clip or the remaining video clips other than the video clip in the video to be clipped. A video editing method according to claim 1, wherein The steps of obtaining a portrait image of a person portrait element and extracting the portrait feature of the portrait element include: Obtaining a portrait picture of a person having a portrait element and storing it in the smart terminal; Identifying a portrait element in the portrait image of the person; The body contour feature of the character portrait element is extracted as a first body contour feature, and the facial portrait feature of the person portrait element is extracted as a first facial portrait feature. A video editing method according to claim 2, wherein Obtaining, in the video to be edited, a video segment of the character that matches the character portrait feature includes: Splitting the video to be clipped, obtaining each frame of the picture, and extracting the to-be-obtained portrait elements in the frame of each frame, including the character back element and the character face element; Extracting a body contour feature of the character back element as a second body shape feature, and extracting a face portrait feature of the character face element as a second face portrait feature; Comparing the second body contour feature with the first body contour feature, and obtaining a picture corresponding to the second body contour feature when the similarity is greater than or equal to the first similarity threshold is a character back image; Comparing the second facial portrait feature with the first facial portrait feature, and acquiring a picture corresponding to the second facial portrait feature when the similarity is greater than or equal to the second similarity threshold is a front view of the character; Cutting the character back picture from the character front picture from the to-be-edited video to form a video segment. A video editing method according to claim 1, wherein The step of splicing the video clip or the remaining video clips other than the video clip in the video to be clipped includes: Separating the audio information and the video information in the remaining video segments except the video segment in the video clip or the video to be clipped to form an audio portion and a video portion; Separating the audio portion and the video portion separately to form a complete audio portion and a complete video portion; Synchronizing the complete audio portion with the full video portion. A video editing method according to any one of claims 1 to 4, wherein the step of acquiring a video clip of a character corresponding to the person portrait feature in the video to be clipped and the step of placing the video clip or Between the steps of splicing the remaining video segments except the video segment in the clip video, the video editing method further includes: The video clip is pushed to the user and filtered by the user to remove irrelevant video clips. A video editing system based on a smart terminal, comprising: a video acquisition module, which acquires a video file to be edited and stores the video file in the smart terminal; a portrait feature extraction module, acquiring a portrait image of a person having a portrait element, and extracting a portrait feature of the portrait element; a video segment acquisition module, configured to connect to the video acquisition module and the portrait feature extraction module, and acquire a video segment of the to-be-edited video that includes a character that matches the character portrait feature; The video splicing module is connected to the video segment obtaining module to splicing the video clip or the remaining video segments except the video segment in the video to be clipped. The video editing system of claim 6 wherein: The portrait feature extraction module includes: a picture obtaining unit, which acquires a portrait picture of a person having a portrait element and stores it in the smart terminal; a portrait element identification unit connected to the picture acquisition unit to identify a person portrait element in the person portrait picture; a portrait feature extraction unit, coupled to the portrait element recognition unit, extracting a body profile feature of the person portrait element as a first body profile feature, and extracting a face portrait feature of the person portrait element as a first face portrait feature . The video editing system of claim 7 wherein: The video segment obtaining module includes: An element extracting unit splits the video to be clipped, obtains a frame of each frame, and extracts a pair of portrait elements in the frame of each frame, including a character back element and a character face element; a feature extraction unit, connected to the element extraction unit, extracting a body contour feature of the character back image element as a second body shape contour feature, and extracting a face portrait feature of the character face element as a second face portrait feature; a back image acquisition unit, connected to the feature extraction unit, and comparing the second body contour feature with the first body contour feature, and acquiring the second when the similarity is greater than or equal to the first similarity threshold The picture corresponding to the figure outline feature is a character back picture; a front view acquiring unit, connected to the feature extracting unit, comparing the second facial portrait feature with the first facial portrait feature, and acquiring the second facial when the similarity is greater than or equal to a second similarity threshold The picture corresponding to the portrait feature is the front view of the character; The cutting unit is connected to the back view picture acquiring unit and the front picture acquiring unit, and cuts the character back picture from the character front picture from the to-be-edited video to form a video segment. The video editing system of claim 6 wherein: The video splicing module includes: a separating unit, separating audio information and video information in the video segment to form an audio portion and a video portion; a splicing unit, connected to the separating unit, splicing the audio portion and the video portion separately to form a complete audio portion and a complete video portion; a synchronization unit coupled to the tiling unit to synchronize the complete audio portion with the complete video portion. The video editing system of any of claims 6-9, wherein between the video segment acquisition module and the video splicing module, the video editing system further comprises: The video clip screening module pushes the video clip to the user, and the user filters the unrelated video clips.

Images