KR102780326B1 - Method for Generating Sports Highlight Videos - Google Patents

Abstract

A method for generating a sports highlight video according to the present invention comprises: (a) a step in which a video preprocessing unit receives an original sports video and a commentary text of the original sports video for generating a highlight video; (b) a step in which the video preprocessing unit divides and sets the entire length of the original sports video into a plurality of unit chunks according to an arbitrarily set criterion; (c) a step in which the video preprocessing unit derives volume information of commentary voices and semantic information of commentary texts included in the plurality of unit chunks; (d) a step in which the video generation unit extracts a point of occurrence of a game event based on the volume information and semantic information of commentary voices included in the plurality of unit chunks; (e) a step in which the video generation unit determines the length of a main event video by setting a start point and an end point before and after the point of occurrence of the game event as a midpoint; and (f) a step in which the video generation unit generates a sports highlight video including the main event video.

Description

{Method for Generating Sports Highlight Videos}

The present invention relates to a method for generating a sports highlight video, and more particularly, to a method for automatically and accurately generating a sports highlight video without human intervention.

As people's interest in sports has increased recently, the demand for sports highlight videos that allow for quick and easy viewing of important moments in the game is also increasing. In addition, this is important not only for the convenience of fans watching the game, but also for various related parties such as sports analysts, coaches, and players.

In the past, it was common to record the entire game and manually edit the highlights. However, this method took a lot of time and effort from the editor, and there was a high possibility of missing important moments.

Additionally, since subjective judgment can be involved in the manual editing process, there was also a problem of discrepancies between the highlights that viewers wanted and the highlights that were actually provided.

In addition, when the editor's subjectivity is involved in the manual editing process, there are cases where highlights are produced that are favorable to a specific team or player, which can lead to controversy over fairness.

And the quality of sports highlight videos can vary greatly depending on the editor's fatigue and experience, which can lead to inconsistencies in highlight videos.

Therefore, a method to solve these problems is required.

Korean Patent Publication No. 10-2018-0118936

The present invention is an invention devised to solve the problems of the above-described prior art, and has the purpose of providing a method for automatically generating consistent and accurate sports highlight videos based on original sports videos and commentary text.

The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the description below.

In order to achieve the above object, the method for generating a sports highlight video of the present invention comprises: (a) a step in which a video preprocessing unit receives an original sports video for generating a highlight video and a commentary text of the original sports video; (b) a step in which the video preprocessing unit divides and sets the entire length of the original sports video into a plurality of unit chunks according to an arbitrarily set criterion; (c) a step in which the video preprocessing unit derives volume information of commentary voices and semantic information of commentary texts included in the plurality of unit chunks; (d) a step in which the video generation unit extracts a point of occurrence of a game event based on the volume information and semantic information of commentary voices included in the plurality of unit chunks; (e) a step in which the video generation unit determines the length of a main event video by setting a start point and an end point before and after the point of occurrence of the game event as a midpoint; and (f) a step in which the video generation unit generates a sports highlight video including the main event video.

At this time, the step (b) may include a step (b-1) in which the video preprocessing unit sets a temporary chunk from a point where any utterance of the commentary voice included in the original sports video starts to a point where the utterance ends, a step (b-2) in which the video preprocessing unit analyzes whether there is a point where the voice of the commentary voice included in the temporary chunk set in the step (b-1) changes, a step (b-3) in which the video preprocessing unit divides the temporary chunk into a plurality of unit chunks based on the point where the voice of the commentary voice included in the temporary chunk changes if it is determined by the step (b-2) that there is a point where the voice of the commentary voice included in the temporary chunk changes, and a step (b-4) in which the video preprocessing unit sets each blank portion between the temporary chunks set in the step (b-1) as a unit chunk.

And the step (c) above may include a step (c-1) in which the image preprocessing unit sets the volume level for the commentary voice included in each unit chunk, and a step (c-2) in which the image preprocessing unit interprets the meaning of the commentary text included in each unit chunk.

In addition, the step (d) may include a step (d-1) in which the video generation unit extracts a unit chunk including a commentary text indicating an event related to the corresponding sport, a step (d-2) in which the video generation unit determines whether the volume of the commentary voice included in the unit chunk extracted in the step (d-1) is equal to or higher than a preset reference value, and a step (d-3) in which the video generation unit determines a unit chunk having a volume equal to or higher than the preset reference value in the step (d-2) as the time of occurrence of the game event.

In addition, the step (e) may include a step (e-1) in which the video generation unit sets a first preceding point in time and a first succeeding point in time having a preset time interval based on the unit chunk set as the point of occurrence of the match event, a step (e-2) in which the video generation unit determines whether a commentary voice is generated at the first preceding point in time and the first succeeding point in time, a step (e-3a) in which the video generation unit includes a portion from the first preceding point in time to the starting point of the unit chunk set as the point of occurrence of the match event in the main event video if the video generation unit determines in step (e-2) that a commentary voice is not generated at the first succeeding point in time, and a step (e-4a) in which the video generation unit includes a portion from the ending point of the unit chunk set as the point of occurrence of the match event to the first succeeding point in the main event video if the video generation unit determines in step (e-2) that a commentary voice is not generated at the first succeeding point in time.

And the step (e) may further include a step (e-3b) of setting the first preceding time point to an earlier time point according to a preset criterion when the image generation unit determines in the step (e-2) that the narration voice is generated at the first preceding time point, and a step (e-3c) of re-performing the steps (e-2) and (e-3b) n times until the step (e-3a) is reached.

In addition, the step (e) may further include a step (e-4b) of setting the first subsequent time point to a later time point according to a preset criterion when the video generation unit determines in the step (e-2) that the narration voice is generated at the first subsequent time point, and a step (e-4c) of re-performing the steps (e-2) and (e-4b) n times until the step (e-4a) is reached.

Meanwhile, between the above steps (e) and (f), an (add) step may be further performed in which the video generation unit determines the length of at least one of a pre-event video played before the main event video and a post-event video played after the main event video.

At this time, the (add) step may include an (add-1) step in which the video generation unit sets a second preceding time point and a second succeeding time point having a preset time interval based on the main event video, an (add-2) step in which the video generation unit determines whether a commentary voice is generated at the second preceding time point and the second succeeding time point, a (add-3a) step in which the video generation unit determines that a commentary voice is not generated at the second preceding time point in the (add-2) step, determines a region from the second preceding time point to the starting point of the main event video as a pre-event video, and a (add-4a) step in which the video generation unit determines that a commentary voice is not generated at the second succeeding time point in the (add-2) step, determines a region from the ending point of the main event video to the second succeeding time point as a post-event video.

And the (add) step may further include an (add-3b) step of setting the second preceding time point to an earlier time point according to a preset standard when the image generation unit determines in the (add-2) step that the narration voice is generated at the second preceding time point, and an (add-3c) step of re-performing the (add-2) step and the (add-3b) step n times until the (add-3a) step is reached.

In addition, the (add) step may further include an (add-4b) step of setting the second subsequent time point to a later time point according to a preset standard when the image generation unit determines in the (add-2) step that the narration voice is generated at the second subsequent time point, and an (add-4c) step of re-performing the (add-2) step and the (add-4b) step n times until the (add-4a) step is reached.

Additionally, the step (f) above can generate the sports highlight video including the pre-event video and the post-event video along with the main event video.

In addition, the present invention may further include a step (ex1) in which the memory unit stores a plurality of data generated in steps (a) to (f) in a database.

And after the above step (f), a step (ex2) may be further performed in which the data verification unit verifies whether the sports highlight video matches the point where the game event of the corresponding sport occurred.

The sports highlight video generation method of the present invention for solving the above-mentioned problem has the advantage of being able to quickly and automatically extract important game events based on original sports videos and commentary text.

This can minimize the intervention of subjective judgment that may occur during the manual editing process, provide more objective and consistent highlight footage, and effectively highlight important moments that viewers may miss by accurately detecting the point where game events occur.

In addition, since the present invention has a method of judging the importance of a game event by utilizing the volume information and semantic information of the commentary voice, it can identify important moments more accurately than other methods and has the advantage of being universally applicable to various sports.

In addition, the present invention has the advantage of being able to flexibly set the length and components of a sports highlight video, thereby satisfying various needs of viewers and providing a more personalized sports highlight video.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

FIG. 1 is a diagram conceptually illustrating the structure of a sports highlight video generation device for performing a sports highlight video generation method according to one embodiment of the present invention.
FIG. 2 is a diagram showing the entire process of a method for generating a sports highlight video according to one embodiment of the present invention.
FIG. 3 is a drawing showing a detailed process of step (b) in a method for generating a sports highlight video according to one embodiment of the present invention.
FIG. 4 is a drawing showing a detailed process of step (c) in a method for generating a sports highlight video according to one embodiment of the present invention.
FIG. 5 is a drawing showing a detailed process of step (d) in a method for generating a sports highlight video according to one embodiment of the present invention.
FIG. 6 is a drawing showing a detailed process of step (e) in a method for generating a sports highlight video according to one embodiment of the present invention.
FIG. 7 is a drawing schematically showing the structure of a sports highlight video in a method for generating a sports highlight video according to one embodiment of the present invention.
FIG. 8 is a drawing showing a detailed process of the (add) step in a method for generating a sports highlight video according to one embodiment of the present invention.
FIG. 9 is a drawing showing a detailed process of step (ex2) in a method for generating a sports highlight video according to one embodiment of the present invention.

In this specification, when a component (or region, layer, portion, etc.) is referred to as being “on,” “connected to,” or “coupled to” another component, it means that it can be directly disposed/connected/coupled to the other component, or that a third component may be disposed between them.

Identical drawing symbols refer to identical components. Also, in the drawings, the thicknesses, proportions, and dimensions of the components are exaggerated for the purpose of effectively explaining the technical contents.

“And/or” includes any combination of one or more of the associated constructs that can be defined.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The singular expression includes the plural expression unless the context clearly indicates otherwise.

Additionally, terms such as "below," "lower," "above," and "upper," are used to describe the relationships between components depicted in the drawings. These terms are relative concepts and are described based on the directions indicated in the drawings.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. In addition, terms that are defined in commonly used dictionaries, such as terms, should be interpreted as having a meaning consistent with the meaning in the context of the relevant art, and are explicitly defined herein, unless interpreted in an idealized or overly formal sense.

It should be understood that the terms "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

Additionally, when it is stated in this specification that a first component operates or is executed on a second component, it should be understood that the first component operates or is executed in an environment in which the second component operates or is executed, or that the first component operates or is executed through direct or indirect interaction with the second component.

When a component, device, or system is referred to as including a component consisting of a program or software, even if there is no explicit mention, it should be understood that the component, device, or system includes hardware (e.g., memory, CPU, etc.) or other programs or software (e.g., an operating system or drivers necessary to operate hardware) necessary for the program or software to execute or operate.

Additionally, unless otherwise specifically stated, a component may be implemented in software, hardware, or both software and hardware.

Also, the terminology used herein is for the purpose of describing embodiments only and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. The words 'comprises' and/or 'comprising' as used in the specification do not exclude the presence or addition of one or more other components mentioned.

In addition, terms such as 'part', 'device', etc. in this specification may be intended to refer to a functional and structural combination of hardware and software driven by or for driving the hardware. For example, the hardware herein may be a data processing device including a CPU or other processor. In addition, software driven by the hardware may refer to a running process, an object, an executable, a thread of execution, a program, etc.

In addition, it can be easily inferred by an average expert in the technical field of the present invention that the above terms may mean a logical unit of a given code and hardware resources for executing the given code, and do not necessarily mean physically connected code or a type of hardware.

Hereinafter, the specific technical contents to be implemented in the present invention will be described in detail with reference to the attached drawings.

The method for generating a sports highlight video according to the present invention can be driven by processors of various servers and terminals, and can be output through a video output device such as a display module to provide visible information through a visualized graphical user interface.

In particular, the method for generating a sports highlight video according to the present invention can be programmed and installed in a server, terminal, etc. using a removable disk or a communication network, and the method for generating a sports highlight video can enable the server, terminal, etc. to be operated by various functional means.

That is, in the present invention, information processing by software is specifically realized through hardware.

Hereinafter, a method for generating a sports highlight video according to one embodiment of the present invention will be described with reference to the attached drawings.

FIG. 1 is a diagram conceptually illustrating the structure of a sports highlight video generation device for performing a sports highlight video generation method according to one embodiment of the present invention.

As illustrated in FIG. 1, a sports highlight video generation device according to one embodiment of the present invention may include an image preprocessing unit (10), an image generation unit (20), a memory unit (30), and a data verification unit (40).

The image preprocessing unit (10) is a component that preprocesses the original sports video so that a sports highlight video can be generated, and the image generation unit (20) generates a sports highlight video based on the original sports video preprocessed by the image preprocessing unit (10).

The memory unit (30) is provided to store all data generated in the process of generating a sports highlight video according to the present invention, and may include a database for storing data for this purpose.

The data verification unit (40) is a component that verifies whether the sports highlight video generated by the video preprocessing unit (10) and the video generation unit (20) matches the point where the game event of the corresponding sport occurred.

Each of the components illustrated in FIG. 1 may be functionally and logically separated, and it will be readily apparent to an average expert in the technical field of the present invention that this does not necessarily mean that each component is separated into a separate physical device or written in separate code.

FIG. 2 is a diagram showing the entire process of a method for generating a sports highlight video according to one embodiment of the present invention.

As illustrated in FIG. 2, a method for generating a sports highlight video according to one embodiment of the present invention includes steps (a) to (f), and may further include steps (add), (ex1), and (ex2).

Step (a) is a step in which the video preprocessing unit (10) receives an original sports video and annotation text of the original sports video to generate a highlight video.

At this time, the original sports video is a video recording of the entire game, and the type of sport shown in the original sports video is not limited to a specific sport and can of course be applied to any sport.

Additionally, original sports footage like this includes commentary voices uttered by the sport's commentators.

Commentary text is the commentary voice spoken by the commentator during the game converted into text. It can be automatically generated, for example, through speech recognition technology (Speech to Text, STT), but it can also be created manually.

Such original sports footage and commentary text are used as the basis for creating sports highlight videos, and through each step below, important moments are identified and used to create sports highlight videos based on these.

Step (b) is a step in which the video preprocessing unit (10) divides the entire length of the original sports video into multiple unit chunks according to arbitrarily set criteria.

The criteria for dividing a unit chunk can be set by, for example, a preset or by considering the narrator's speech pattern, etc. For example, a video can be divided into chunks at regular time intervals or chunks can be set based on the narrator's speech unit.

Additionally, each unit chunk can be analyzed and processed independently, allowing an image divided into multiple unit chunks to be efficiently processed using parallel processing techniques.

Step (c) is a step in which the image preprocessing unit (10) derives volume information of the commentary voice and semantic information of the commentary text included in multiple unit chunks.

The volume information of the commentary voice indicates the volume of the commentator's voice in each unit chunk, which is an important indicator for identifying the point where an event in the corresponding sport occurs.

In addition, the semantic information of the commentary text provides important information for understanding specific events or important moments by analyzing the commentator's speech content. Such semantic information can be extracted using natural language processing technology and can include information such as keywords and sentence structures of the commentary text.

Step (d) is a step in which the video generation unit (20) extracts the point of occurrence of a game event based on the volume information and semantic information of the commentary voice included in multiple unit chunks.

As described above, if the volume of the commentary voice is high or the commentary text contains content related to a specific event, the unit chunk may be judged as the point where a match event occurred.

For example, a loud part of a commentator's voice is likely to be an important moment in the game, so this process can identify the point where important events occur based on this.

In addition, the location of a game event can be determined based on keywords related to a specific event, or by comprehensively considering keywords and volume information.

At this time, keywords can be set arbitrarily by the administrator (e.g. goals, assists, hits, home runs, specific player names, etc.), or sometimes the administrator can set new keywords through manual search.

There are various ways to identify the point where a game event occurs, and more detailed information about the process will be described later.

Step (e) is a step in which the video generation unit (20) determines the length of the main event video by setting the start and end points before and after the point where the game event occurs as the midpoint.

The main event video is the most essential part of a sports highlight video, and the start and end points of the main event video can be set flexibly to sufficiently convey the context of the game. This will be described later.

Step (f) is a process in which the video generation unit (20) generates a sports highlight video including a main event video.

Through this process, an objective and consistent highlight video is produced, eliminating the subjective judgment of manual editing.

Below, each of the steps described above will be explained in more detail.

FIG. 3 is a drawing showing a detailed process of step (b) in a method for generating a sports highlight video according to one embodiment of the present invention.

As illustrated in FIG. 3, step (b) may include steps (b-1) to (b-4).

Step (b-1) is a step in which the video preprocessing unit (10) sets the point from which any speech among the commentary voices included in the original sports video starts to the point where the speech ends as a temporary chunk.

The reason for doing this is to prevent the narration voice from being cut off and split in the middle. This is because if one utterance is split in the middle, it is difficult to accurately grasp the intention, overall atmosphere, and volume of the narration voice.

Step (b-2) is a step in which the image preprocessing unit (10) analyzes whether there is a point where the voice of the narration voice included in the temporary chunk set in step (b-1) changes.

This is because, when there are multiple narrators, other narrators may speak consecutively without a gap after a specific narrator speaks, so the purpose is to divide and separate that part.

For example, this course may utilize voice frequency analysis techniques, voice activity detection (VAD) techniques, and pitch analysis techniques.

Step (b-3) is a step in which, if the image preprocessing unit (10) determines that there is a point where the voice of the narration voice included in the temporary chunk changes in step (b-2), the temporary chunk is divided into multiple unit chunks based on that point.

And step (b-4) is a step in which the image preprocessing unit (10) sets each blank portion existing between each temporary chunk set in step (b-1) as a unit chunk.

According to the above process, the entire original sports video can be segmented into multiple unit chunks separated by the commentator's utterance and multiple unit chunks located between the unit chunks containing the commentator's utterance.

FIG. 4 is a drawing showing a detailed process of step (c) in a method for generating a sports highlight video according to one embodiment of the present invention.

As illustrated in FIG. 4, step (c) may include steps (c-1) and (c-2).

As described above, step (c) is a step in which the image preprocessing unit (10) derives volume information of the narration voice included in multiple unit chunks and semantic information of the narration text. First, in step (c-1), the image preprocessing unit (10) sets the volume level for the narration voice included in each unit chunk. At this time, the volume level can be set in various ways.

For example, the image preprocessing unit (10) can set the average value of the loudness within each unit chunk. This is because the narrator's narration voice can vary widely even within one unit chunk.

Next, step (c-2) is a step in which the image preprocessing unit (10) interprets the meaning of the explanatory text included in each unit chunk.

This process is designed to provide important information for understanding specific events or important moments by analyzing the content of the commentator's speech, and as mentioned above, semantic information can be interpreted using natural language processing technology.

FIG. 5 is a drawing showing a detailed process of step (d) in a method for generating a sports highlight video according to one embodiment of the present invention.

As illustrated in FIG. 5, step (d) may include steps (d-1) to (d-3).

Step (d-1) is a step in which the video generation unit (20) extracts a unit chunk containing a commentary text meaning an event related to the sport.

Since the semantic information for each unit chunk was analyzed in the above-described step (c-2), in this process, the unit chunk that includes the explanatory text meaning the event related to the corresponding sport is extracted from each unit chunk.

For example, if the sport is soccer, unit chunks containing meanings such as 'goal', 'score', and 'go in' can be extracted, and if the sport is baseball, unit chunks containing meanings such as 'home run', 'hit', and 'strikeout' can be extracted. Of course, this can be preset in various ways depending on the sport.

Step (d-2) is a step in which the image generation unit (20) determines whether the volume of the commentary voice included in the unit chunk extracted in step (d-1) is equal to or higher than a preset reference value.

In this process, the unit chunks selected in step (d-1) are judged to determine whether the volume of the commentary voice included in the unit chunks is above the standard value.

At this time, the reference value for the volume level can be set in various ways.

For example, since the normal volume level tends to appear most frequently during a sports game, the reference value can be set to the volume level that appears most frequently when synthesizing each unit chunk.

Alternatively, the reference value may be an average loudness derived by compiling the loudness levels of all unit chunks set in step (c-1).

Step (d-3) is the step in which the video generation unit (20) determines a unit chunk having a volume level higher than the standard value set in step (d-2) as the time of occurrence of a game event.

In other words, this process determines that the part where the commentator's volume is higher than the reference value is an important moment in the game, and considers it as the point at which a game event occurs. This is because the commentator's volume tends to increase when a game event occurs.

FIG. 6 is a drawing showing a detailed process of step (e) in a method for generating a sports highlight video according to one embodiment of the present invention.

As illustrated in FIG. 6, step (e) may include steps (e-1) to (e-4), and steps (e-3) and (e-4) may also be divided into detailed steps.

Step (e) is the process of determining the main event video, which is the most essential part of a sports highlight video, and sets the start and end points of the main event video so that it can sufficiently convey the context of the game.

Step (e-1) is a step in which the video generation unit (20) sets a first preceding point in time and a first succeeding point in time with a preset time interval based on a unit chunk set as a game event occurrence point.

This is to ensure that the scene that led to the match event and the result of the match event are included in the sports highlight video based on the moment the match event occurred.

Step (e-2) is a step in which the image generation unit (20) determines whether the commentary voice is generated at the first preceding time point and the first succeeding time point set in step (e-1).

This process is intended to prevent commentary voices from being cut off in sports highlight videos and to process the videos smoothly. And depending on the result of this judgment, the process can proceed to steps (e-3a) to (e-3c) or steps (e-4a) to (e-4c).

Step (e-3a) is a step in which, if the video generation unit (20) determines that no commentary voice is generated at the first preceding time point in step (e-2), it includes the starting point of the unit chunk set as the point of occurrence of the game event from the first preceding time point in the main event video.

That is, if the commentary voice does not occur at the first preceding point, that point is set as the starting point of the main event video.

Step (e-3b) is a step in which, if the video generation unit (20) determines that the commentary voice is generated at the first preceding point in time in step (e-2), the first preceding point in time is set to an earlier point in time according to the preset criteria.

That is, if the commentary voice is generated at the first preceding point in time, the first preceding point in time can be reset to an earlier time than the present. In this case, when the first preceding point in time is reset, the time interval can be set to be shorter than the time interval of the first preceding point in time that was set immediately before.

Step (e-3c) is a step in which steps (e-2) and (e-3b) are re-performed n times until step (e-3a) is reached if step (e-3b) has been performed.

As a result, we can ultimately set the starting point of the main event video to the point where the commentary voice does not occur.

And step (e-4a) is a step for including the main event video from the end point of the unit chunk set as the game event occurrence point to the first subsequent time point when the video generation unit (20) determines that the commentary voice does not occur at the first subsequent time point in step (e-2).

That is, if the commentary voice does not occur at the first trailing point, that point is set as the end point of the main event video.

Step (e-4b) is a step in which, if the video generation unit (20) determines that the commentary voice is generated at the first subsequent time point in step (e-2), the first subsequent time point is set to a later time point according to the preset criteria.

That is, if the commentary voice is generated at the first lag time point, the first lag time point can be re-set to a time later than the present. In this case, when re-setting the first lag time point, it can be set to have a shorter time interval than the time interval of the first lag time point that was set immediately before.

Step (e-4c) is a step in which steps (e-2) and (e-4b) are re-performed n times until step (e-4a) is reached. This allows the optimal follow-up point to be set.

As a result, we can ultimately set the point in time when the commentary voice stops being heard as the end point of the main event video.

However, in steps (e-3a) to (e-3c) or steps (e-4a) to (e-4c), there may be cases where voice is continuously generated in all parts selected as the first preceding time point even if steps (e-3c) and (e-4c) are repeated multiple times.

In such a case, n is preset to a preset value, and when the number of repetitions reaches the n value, the first preceding point in time with the lowest volume of the commentary voice among all the first preceding points in time selected until the n value is reached can be determined as the start point or end point of the main event video.

Through the above step (e), the main event video of the sports highlight video can be determined. However, in the present embodiment, the sports highlight video may further include a pre-event video and a post-event video in addition to the main event video.

FIG. 8 is a drawing schematically showing the structure of a sports highlight video in a method for generating a sports highlight video according to one embodiment of the present invention.

As illustrated in Figure 8, the pre-event video is placed in front so that it is played before the main event video, so that the process prior to the occurrence of the main event video can be included in the sports highlight video to make it easier to identify the cause of the occurrence of the main event video.

In addition, the post-event video is placed in front so that it is played after the main event video, so that the situations after the main event video occurs can be included in the sports highlight video to make it easier to understand the atmosphere after the main event video occurs.

And for this purpose, between the step (e) described above and the step (f) described below, an (add) step may be further performed in which the video generation unit (20) determines the length of at least one of the pre-event video played before the main event video and the post-event video played after the main event video.

FIG. 8 is a drawing showing a detailed process of the (add) step in a method for generating a sports highlight video according to one embodiment of the present invention.

As illustrated in FIG. 8, the (add) step may include steps (add-1) to (add-4), and the (add-3) step and the (add-4) step may also be divided into detailed steps.

The (add-1) step is a step in which the video generation unit (20) sets a second preceding time point and a second succeeding time point with a preset time interval based on the main event video.

The (add-2) step is a step in which the image generation unit (20) determines whether the commentary voice is generated at the second preceding time point and the second succeeding time point set in the (add-1) step.

This process is also intended to prevent commentary voices from being cut off in sports highlight videos, like the aforementioned step (e-2), and to process the videos smoothly. And depending on the judgment result, the process can proceed to the following steps (add-3a) to (add-3c) or (add-4a) to (add-4c).

The (add-3a) step is a step in which, if the video generation unit (20) determines that no commentary voice is generated at the second preceding time point in the (add-2) step, the time from the second preceding time point to the start point of the main event video is determined to be the pre-event video.

That is, if the commentary voice does not occur at the second preceding point, that point is set as the starting point of the pre-event video.

The (add-3b) step is a step in which, if the video generation unit (20) determines that the commentary voice is generated at the second preceding time point in the (add-2) step, the second preceding time point is set to an earlier time point according to the preset criteria.

In other words, if the commentary voice is generated at the second preceding point in time, the second preceding point in time can be reset to an earlier time than the present. In this case, when the second preceding point in time is reset, the time interval can be set to be shorter than the time interval of the first second preceding point in time that was set immediately before.

The (add-3c) step is a step in which, if the (add-3b) step has been performed, the (add-2) and (add-3b) steps are re-performed n times until the (add-3a) step is reached.

As a result, we can ultimately set the starting point of the pre-event video to the point where no commentary voice occurs.

And the (add-4a) step is a step in which, if the video generation unit (20) determines that the commentary voice does not occur at the second subsequent time point in the (add-2) step, it determines the time from the end point of the main event video to the second subsequent time point as the post-event video.

That is, if the commentary voice does not occur at the second follow-up point, that point is set as the end point of the post-event video.

The (add-4b) step is a step in which, if the video generation unit (20) determines that the commentary voice is generated at the second subsequent time point in the (add-2) step, the second subsequent time point is set to a later time point according to the preset criteria.

In other words, if the commentary voice is generated at the second lag time, the second lag time can be set again later than the present. In this case, when the second lag time is set again, it can be set to have a shorter time interval than the time interval of the first second lag time set immediately before.

Step (add-4c) is a step that re-performs steps (add-2) and (add-4b) n times until step (add-4a) is reached.

As a result, the point in time when the commentary voice stops being generated can be set as the end point of the post-event video.

As described above, the total length of the sports highlight video can be determined according to the process of step (e) and step (add).

Next, step (f) is a step in which the video generation unit (20) generates a sports highlight video including a main event video.

As described above, this process creates a sports highlight video, including pre-event video and post-event video along with the main event video.

Meanwhile, the present invention may further include step (ex2) performed after step (ex1) and step (f).

Step (ex1) is a process in which the memory unit (30) stores multiple data generated in steps (a) to (f) in a database, enabling it to retain and manage all data that may be generated in all processes of the present invention.

And step (ex2) is the process in which the data verification unit (40) verifies whether the sports highlight video matches the point where the game event of the corresponding sport occurred.

This process can be performed to verify and confirm that the sports highlight video generated by all of the processes described above is actually the point where the event occurred in the game.

FIG. 9 is a drawing showing a detailed process of step (ex2) in a method for generating a sports highlight video according to one embodiment of the present invention.

As illustrated in Fig. 9, step (ex2) may include the following detailed processes.

Step (ex2-1) is a process in which the data verification unit (40) detects a point in time corresponding to a sport highlight video from the original sports video.

In this process, the generated sport highlight video is compared with the original sports video to determine at what point in time the event occurred.

(ex2-2) Step is a process in which the data verification unit (40) detects a scoreboard that appears within the field of view at a time point before and after the time point corresponding to the sports highlight video to be verified among the original sports video. This process can be performed through various video interpretation techniques.

At this time, the scoreboard may include not only the on-site scoreboard shown within the video field of view, but also any object that can display the score of the current game, such as an edited scoreboard overlaid on the video screen.

Step (ex2-3) is a process in which the data verification unit (40) determines the change in the score that appears on the scoreboard detected at points before and after the point corresponding to the sports highlight video.

In this process, when judging changes in scores, a method that only considers information on the scoreboard may be applied, but the method for judging changes in scores is not limited to this, and a method for comprehensively judging by considering volume information of the commentary voice, tone, and semantic information of the commentary text may also be applied.

Step (ex2-4) is a process in which the data verification unit (40) determines the corresponding sports highlight video as a normal video if a change in the score is detected as a result of the judgment in step (ex2-3).

And step (ex2-5) is a process in which the data verification unit (40) determines the corresponding sports highlight video as an abnormal video if no change in the score is detected as a result of the judgment in step (ex2-3).

For example, if the sport is soccer, a sports highlight video may be created showing the point in time when a goal was scored, but there may be cases where a decision is corrected after that point due to an offside or foul.

Therefore, in step (ex2-5), if there is no change in the score, the data verification unit (40) determines that a judgment correction has been made for the event in question, and thus determines that the sports highlight video in question is an abnormal video and not an actual event.

However, sports highlight videos judged as abnormal videos in this way are not completely discarded, but may be treated as temporarily stored videos and may be marked as having lower importance than normal videos and stored.

And later, the temporary archive footage can be used alongside regular footage in content that requires a larger number of sports highlight footage.

As described above, preferred embodiments according to the present invention have been examined, and it is obvious to those skilled in the art that the present invention can be embodied in other specific forms in addition to the embodiments described above without departing from the spirit or scope thereof. Therefore, the above-described embodiments should be considered as illustrative rather than restrictive, and accordingly, the present invention is not limited to the above description, but may be modified within the scope of the appended claims and their equivalents.

10: Image preprocessing section
20: Video generation section
30: Memory section
40: Data Verification Department

Claims (14)

(a) step in which a video preprocessing unit receives original sports video and commentary text of the original sports video for generating a highlight video;
(b) step in which the above video preprocessing unit divides the entire length of the original sports video into a plurality of unit chunks according to arbitrarily set criteria;
(c) step in which the above image preprocessing unit derives volume information of the commentary voice and semantic information of the commentary text included in the plurality of unit chunks;
(d) step in which the video generation unit extracts the occurrence point of a game event based on the volume information and semantic information of the commentary voice included in the plurality of unit chunks;
(e) step of determining the length of the main event video by setting the start point and end point before and after the point where the game event occurs as the midpoint by the above video generation unit; and
(f) step of generating a sports highlight video including the main event video by the above video generation unit;
Including,
Step (e) above,
(e-1) step in which the above video generation unit sets a first preceding point in time and a first succeeding point in time having a preset time interval based on the unit chunk set as the point of occurrence of the game event;
(e-2) step in which the above video generation unit determines whether an explanation voice is generated at the first preceding time point and the first succeeding time point;
If the video generation unit determines that the commentary voice does not occur at the first preceding time point in the step (e-2), the step (e-3a) of including the main event video from the first preceding time point to the starting point of the unit chunk set as the game event occurrence point; and
If the above video generation unit determines that the commentary voice does not occur at the first subsequent time point in the above step (e-2), the step (e-4a) of including the main event video from the end point of the unit chunk set as the game event occurrence point to the first subsequent time point;
As including,
In order to include in the main event video the cause of occurrence of the game event that appears from the first preceding point in time to the starting point of the unit chunk set as the game event occurrence point, and the result of occurrence of the game event that appears from the ending point of the unit chunk set as the game event occurrence point to the first succeeding point in time, together with the unit chunk set as the game event occurrence point.
How to create a sports highlight video. In the first paragraph,
Step (b) above,
(b-1) step in which the above video preprocessing unit sets a temporary chunk from the point where any utterance among the commentary voices included in the original sports video starts to the point where the utterance ends;
Step (b-2) in which the above video preprocessing unit analyzes whether there is a point where the voice of the commentary voice included in the temporary chunk set in step (b-1) changes;
If the above video preprocessing unit determines that there is a point where the voice of the narration voice included in the temporary chunk changes through the above step (b-2), the step (b-3) of dividing the temporary chunk into multiple unit chunks based on the point; and
Step (b-4) in which the image preprocessing unit sets each blank portion existing between each temporary chunk set in step (b-1) as a unit chunk;
Including,
How to create a sports highlight video. In the first paragraph,
Step (c) above,
(c-1) step in which the above image preprocessing unit sets the volume level for the commentary voice included in each unit chunk; and
(c-2) step in which the above image preprocessing unit interprets the meaning of the explanatory text included in each unit chunk;
Including,
How to create a sports highlight video. In the third paragraph,
Step (d) above,
Step (d-1) in which the above video generation unit extracts a unit chunk containing a commentary text meaning an event related to the corresponding sport;
(d-2) step in which the above video generation unit determines whether the volume of the commentary voice included in the unit chunk extracted in (d-1) is greater than a preset reference value; and
Step (d-3) in which the video generation unit determines a unit chunk having a volume level higher than the preset reference value in step (d-2) as the point in time when the game event occurs;
Including,
How to create a sports highlight video. delete In the first paragraph,
Step (e) above,
If the above video generation unit determines that the commentary voice is generated at the first preceding time point in the above step (e-2), the step (e-3b) sets the first preceding time point to an earlier time point according to the preset criteria; and
Step (e-3c) of re-performing steps (e-2) and (e-3b) n times until step (e-3a) is reached;
Including more,
How to create a sports highlight video. In the first paragraph,
Step (e) above,
If the above video generation unit determines that the commentary voice is generated at the first subsequent time point in the above step (e-2), the step (e-4b) sets the first subsequent time point to a later time point according to the preset criteria; and
Step (e-4c) of re-performing steps (e-2) and (e-4b) n times until step (e-4a) is reached;
Including more,
How to create a sports highlight video. In the first paragraph,
Between steps (e) and (f) above,
The above video generation unit further performs an (add) step of determining the length of at least one of a pre-event video played before the main event video and a post-event video played after the main event video.
How to create a sports highlight video. In Article 8,
The above (add) step is,
(add-1) step in which the above video generation unit sets a second preceding time point and a second succeeding time point having a preset time interval based on the above main event video;
(add-2) step in which the above video generation unit determines whether an explanation voice is generated at the second preceding time point and the second succeeding time point;
If the above video generation unit determines that the commentary voice does not occur at the second preceding time point in the above (add-2) step, the (add-3a) step of determining the time from the second preceding time point to the starting point of the main event video as the pre-event video; and
If the above video generation unit determines that the commentary voice does not occur at the second subsequent time point in the above (add-2) step, the (add-4a) step determines the time from the end point of the main event video to the second subsequent time point as the post-event video;
Including,
How to create a sports highlight video. In Article 9,
The above (add) step is,
If the above video generation unit determines that the commentary voice is generated at the second preceding time point in the above (add-2) step, the (add-3b) step sets the second preceding time point to an earlier time point according to the preset criteria; and
Step (add-3c) of re-performing steps (add-2) and (add-3b) n times until step (add-3a) is reached;
Including more,
How to create a sports highlight video. In Article 9,
The above (add) step is,
If the above video generation unit determines that the commentary voice is generated at the second subsequent time point in the above (add-2) step, the (add-4b) step sets the second subsequent time point to a later time point according to the preset criteria; and
Step (add-4c) of re-performing steps (add-2) and (add-4b) n times until step (add-4a) is reached;
Including more,
How to create a sports highlight video. In Article 8,
The above step (f) is,
Generating said sports highlight video, including said pre-event video and said post-event video along with said main event video;
How to create a sports highlight video. In the first paragraph,
The memory unit further includes a step (ex1) of storing a plurality of data generated in steps (a) to (f) in a database.
How to create a sports highlight video. In the first paragraph,
After step (f) above,
(ex2) The data verification department further performs a step of verifying whether the sports highlight video matches the point where the game event of the sport occurred.
How to create a sports highlight video.