JP7751921B1 - Animation Creation Device - Google Patents

Abstract

An animation creation device is provided that creates animations that can be used to guide players.
[Solution] An animation creation device acquires video data generated by filming a game. The animation creation device includes a teacher data storage unit that stores teacher data. The animation creation device detects the positions of multiple players and the ball in the video data. The animation creation device assigns identification information to each of the multiple players. The animation creation device creates images representing the following (a) to (c), and creates an animation using these images: (a) the positions of the multiple players and the ball; (b) the identification information assigned to the multiple players; and (c) the positions and identification information of the multiple players included in the teacher data.
[Selected figure] Figure 8

Description

This disclosure relates to an animation creation device.

Patent Document 1 describes an event log generation device. The event log generation device detects events by analyzing video images generated by filming a soccer match. Events include kicking the ball, tackling another player, etc.

JP 2010-41725 A

Video data generated by filming a sport such as soccer can be used to provide guidance to players. It is particularly desirable to provide guidance to help players move to the correct position depending on the situation of the game. However, such guidance has been difficult because the video data does not show the player's correct position.

In one aspect of the present disclosure, it is preferable to provide an animation creation device that creates animations that can be used to guide players.

One aspect of the present disclosure is an animation creation device. The animation creation device has the following configuration: A video data acquisition unit configured to acquire video data generated by filming a game played on a field by multiple players using a ball.

A training data storage unit that stores multiple pieces of training data for each position of the ball on the field, associating the position of the ball on the field with the positions of the multiple players on the field and the identification information assigned to each of the multiple players.

A position detection unit configured to detect the positions of the players and the ball in each of a plurality of frames included in the video data. A position conversion unit configured to convert the positions of the players and the ball in each of the plurality of frames into positions of the players and the ball on the field.

A first identification information assignment unit configured to assign the identification information to each of the multiple players in a specific frame that is one of the frames included in the video data. A tracking unit configured to associate the same player detected in each of the multiple frames by tracking.

A second identification information assignment unit configured to assign the identification information assigned to the player in the specific frame by the first identification information assignment unit to players in frames other than the specific frame who are associated with the player in the specific frame through the tracking.

An image creation unit configured to create images representing the following (a) to (c) for each of the plurality of frames. An animation creation unit configured to create an animation using the plurality of images created by the image creation unit.

(a) the positions of the players and the ball on the field;
(b) the identification information assigned to the plurality of players;
(c) In the training data, the positions of the multiple players associated with the position of the ball in (a), and the identification information assigned to each of the multiple players in the training data.

One aspect of the present disclosure is an animation creation device that can create animations that can be used to guide players.

FIG. 1 is a plan view showing a half court. FIG. 1 is a block diagram showing a configuration of an animation creation device. FIG. 2 is a block diagram showing the functional configuration of a calculation unit. FIG. 2 is an explanatory diagram showing the contents of teacher data. 10 is a flowchart showing a process executed by a calculation unit. 6A and 6B are explanatory diagrams illustrating a process for detecting multiple players and a process for detecting a ball in a frame, respectively. 7A and 7B are diagrams illustrating a process for locating the four corners of a half court in a frame, and a process for mapping the four corners in a top-down view of the half court. FIG. 2 is an explanatory diagram illustrating an image created by an image creating unit.

Exemplary embodiments of the present disclosure will now be described with reference to the drawings.
First Embodiment
1. Configuration of animation creation device 1 The configuration of the animation creation device 1 will be described with reference to Figures 1 to 4. The animation creation device 1 is a device that creates animation from video data generated by filming a soccer match or practice. Soccer corresponds to a sport played on a field by multiple players using a ball.

The video data was generated by capturing footage of the half court 101 shown in Figure 1, which is part of the field where soccer is played. The camera that generated the video data was located outside the half court 101, at a certain height.

As shown in FIG. 2, the animation creation device 1 includes a calculation unit 3 and a teacher data storage unit 5. The calculation unit 3 is connected to both the cloud 7 and the teacher data storage unit 5. The calculation unit 3 includes a microcomputer having a CPU 11 and semiconductor memory such as RAM or ROM (hereinafter referred to as memory 13).

Each function of the calculation unit 3 is realized by the CPU 11 executing a program stored on a non-transient physical recording medium. In this example, the memory 13 corresponds to the non-transient physical recording medium storing the program. Furthermore, by executing this program, a method corresponding to the program is performed. The calculation unit 3 may include one microcomputer or multiple microcomputers.

As shown in FIG. 3, the calculation unit 3 functionally comprises a video data acquisition unit 21, a frame extraction unit 23, a position detection unit 25, a position conversion unit 27, a first identification information assignment unit 29, a tracking unit 31, a second identification information assignment unit 33, an image creation unit 35, and an animation creation unit 37.

As shown in FIG. 2, the teacher data storage unit 5 stores multiple pieces of teacher data 41. As shown in FIG. 4, the teacher data 41 is data that associates the position of the ball 109 on the half court 101, the positions of multiple players 107 on the half court 101, and identification information assigned to each of the multiple players 107. In this embodiment, there are eight players 107. The teacher data storage unit 5 stores multiple pieces of teacher data 41 that differ for each position of the ball 109.

For example, the teacher data 41 at the top in Figure 4 is the teacher data 41 when the ball 109 is positioned at "6-I." Also, the teacher data 41 second from the top in Figure 4 is the teacher data 41 when the ball 109 is positioned at "6-H."

In the teacher data 41 at the top of Figure 4, the player 107 at position "3-C" is associated with the identification information "1." In addition, in that teacher data 41, the player 107 at position "7-D" is associated with the identification information "2." For example, the teacher data 41 can be created by someone with extensive knowledge of soccer.

The position of the ball 109 and the position of the player 107 on the half court 101 are represented by positions in the X and Y directions shown in Figure 1. In the X direction, the half court 101 is divided into rows 1 to 15. In the Y direction, the half court 101 is divided into columns A to I. The half court 101 is divided into 135 grids 102 by the rows in the X direction and the columns in the Y direction. A player's position on the half court 101 is represented by which grid 102 they are in. For example, a position represented by "3-C" is a position in the grid 102 whose row in the X direction is "3" and whose column in the Y direction is "C," as shown in Figure 1.

One identification information is assigned to each of the multiple players 107. The identification information is different for each player 107. The identification information assigned to any one player 107 does not change over time. In this embodiment, the identification information corresponds to the player's 107 position in the game (e.g., right-most forward, central midfielder, left-most defender, etc.). The multiple players 107 assigned identification numbers 1 to 8 are players on the defensive team guarding the goal 103 in the half court 101.

2. Processing Executed by the Animation Creation Device 1 The processing executed by the animation creation device 1 will be described with reference to FIGS.
In step 1, the video data acquisition unit 21 acquires video data that has been uploaded to the cloud 7. The video data has 24 frames 105 per second. Note that the user can save the video data in the cloud 7 in advance.

In step 2, the frame extraction unit 23 extracts a portion of the frames 105 from all of the frames 105 in the video data. Specifically, all of the frames 105 in the video data are divided into groups of three consecutive frames 105. One frame 105 is then extracted from each group, and the other two frames 105 are deleted. As a result, the extracted video data has eight frames 105 per second.

In step 3, the position detection unit 25 detects the positions of the multiple players 107 and the ball 109 in each of the multiple frames 105 included in the video data extracted in step 2.

For example, as shown in FIG. 6A, the position detection unit 25 recognizes multiple players 107 in each frame 105 and detects the positions of the multiple players 107 in the frame 105. The position in the frame 105 is a position represented by coordinates in the vertical direction x and horizontal direction y of the frame 105, as shown in FIG. 6A.

The position detection unit 25 can distinguish between the players 107 and the players 108 of the offensive team who are attacking towards the goal 103 in the following manner. The position detection unit 25 recognizes the colors of all detected players and divides the players into two groups based on their colors. Of the two groups, the player in the group with the largest number of players in the half court 101 is designated as player 107.

Since the colors of the uniforms, bibs, etc. of the two teams are usually different, the players can be divided into two groups based on color. One of the two groups is the offensive team that attacks toward goal 103, and the other is the defensive team that protects goal 103. Also, since the number of players on the defensive team in half court 101 is usually greater than the number of players on the offensive team, the players in the larger group are likely to be players from the defensive team (i.e., players 107).

In addition, as shown in Figure 6B, the position detection unit 25 recognizes the ball 109 in each frame 105 and detects the position of the ball 109 in the frame 105.

In step 4, the positions of the multiple players 107 and the ball 109 are transformed in each of the multiple frames 105. Transforming the positions means converting the positions in the frames 105 to positions in the half court 101.

This transformation can be performed, for example, as follows. As shown in Figure 7A, the positions of the four corners 111a to 111d of the half court 101 are identified in the frame 105. Next, as shown in Figure 7B, the positions of the corners 111a to 111d are mapped in a drawing of the half court 101 as seen from directly above. Next, affine transformation and perspective transformation matrices are calculated. These matrices can be used to transform positions in the frame 105 into positions on the half court 101.

In step 5, the first identification information assignment unit 29 acquires the first frame 105 (hereinafter referred to as the specific frame 105A) from the video data. The specific frame 105A is the frame 105 at the start of the competition or the start of the second half of the competition. The first identification information assignment unit 29 assigns identification information to each of the multiple players 107 detected in the specific frame 105A.

It is highly likely that at the start of the competition or the start of the second half of the competition, each of the multiple players 107 will be in a position corresponding to their identification information. For example, at the start of the competition or the start of the second half of the competition, a forward player 107 will be more likely to be in a more forward position (i.e., a position corresponding to the forward's identification information) than other players 107. Therefore, it is highly likely that the correct identification information can be assigned to each of the multiple players 107 detected in the specific frame 105A, depending on the position of the player 107 in the specific frame 105A.

In step 6, the tracking unit 31 tracks the player 107 in multiple frames 105, thereby associating the same player 107 detected in each of the multiple frames 105.

Tracking can be performed using known methods. For example, if a player 107 detected in a certain frame 105 and a player 107 detected in the next frame 105 are close in position, movement direction, etc., the tracking unit 31 determines that there is a high probability that the players 107 are the same, and associates them.

In step 7, the second identification information assignment unit 33 assigns identification information to players 107 detected in frames 105 other than the specific frame 105A. Specifically, the following processing is performed.

For example, suppose that in step 5, identification information a is assigned to one player 107a detected in a specific frame 105A. In step 6, the second identification information assignment unit 33 assigns the same identification information a to players 107 detected in each frame 105 other than the specific frame 105A who are associated with player 107a. This process is performed for all players 107 and for all frames 105. By performing step 7, a single identification information is assigned to the same player 107 across all of the multiple frames 105.

In step 8, the image creation unit 35 creates an image 113 shown in FIG. 8 for each of the multiple frames 105. The number of images 113 created is equal to the number of frames 105 extracted in step 2. One image 113 is created from any one frame 105. Any one image 113 is created using the information obtained for the corresponding frame 105 in the processing of steps 3 to 7. The image 113 is a drawing that represents the half court 101 as seen from directly above.

Image 113 displays as many reality marks 115 as there are multiple players 107. Each reality mark 115 corresponds to one player 107. The reality mark 115 represents the real-world position of the corresponding player 107 and the identification information of that player 107. The reality mark 115 is displayed at the position of the corresponding player 107 in the half court 101. The position of the reality mark 115 is the position of the player 107 calculated in the processing of step 4. The numerical value included in the reality mark 115 is the identification information assigned to the corresponding player 107 in step 5 or 7.

The real marks 115 represent the positions of the multiple players 107 on the half court 101 and the identification information assigned to the multiple players 107. The image 113 displays as many teacher marks 117 as there are multiple players 107. Each teacher mark 117 corresponds to one player 107. The numerical value included in the teacher mark 117 is the identification information assigned to the corresponding player 107.

The image creation unit 35 displays the teacher mark 117 on the image 113 in the following manner. First, the image creation unit 35 reads the teacher data 41 corresponding to the position of the ball 109 obtained by conversion in step 4 from the teacher data storage unit 5. For example, if the position of the ball 109 is "6-I," the image creation unit 35 reads the teacher data 41 located at the top in Figure 4.

Next, the image creation unit 35 displays teacher marks 117 in the image 113 at the positions of each of the multiple players 107 included in the read teacher data 41. For example, if the teacher data 41 at the top of Figure 4 is read, teacher marks 117 are displayed at the positions "3-C," "7-D," "12-C," "4-E," "8-E," "13-E," "6-H," and "12-C."

In addition, each teacher mark 117 displays identification information associated with the position of that teacher mark 117 (i.e., the position of the corresponding player 107) in the read teacher data 41. For example, when the topmost teacher data 41 in Figure 4 is read, the teacher mark 117 at position "3-C" displays "1," which is the identification information associated with the position of that teacher mark 117 (i.e., the position of the corresponding player 107) in the read teacher data 41.

The teacher marks 117 represent the positions of multiple players 107 associated with the position of the ball 109 obtained by conversion in step 4 in the teacher data 41, and the identification information assigned to each of the multiple players 107 in the teacher data 41. One ball mark 119 is displayed in the image 113. The ball mark 119 is displayed at the position of the ball 109 obtained by conversion in step 4.

In step 9, the animation creation unit 37 creates an animation using the multiple images 113 created in step 8. When the animation is displayed, the multiple images 113 are displayed one by one in sequence. When the animation is displayed, the real mark 115, teacher mark 117, and ball mark 119 appear to move continuously on the half court 101 over time.

3. Effects of the Animation Creation Device 1 (1A) The animation created by the animation creation device 1 is composed of images 113 representing the following (a) to (c):

(a) The positions of multiple players 107 and the ball 109 in the half court 101 (i.e., the position of the real mark 115 and the position of the ball mark 119).
(b) Identification information assigned to multiple players 107 (i.e., identification information attached to reality marks 115).

(c) In the teacher data 41, the positions of the multiple players 107 associated with the position of the ball 109 in (a) above, and the identification information assigned to each of the multiple players 107 in the teacher data 41 (i.e., the position of the teacher mark 117 and the identification information assigned thereto).

The animation can therefore simultaneously display the actual position of each player 107 (i.e., the position of the real mark 115) and the correct position of each player 107 (i.e., the position of the teacher mark 117). For example, a user of the animation creation device 1 can use this animation to instruct each player 107 so that they move to the correct position.

Since each reality mark 115 and teacher mark 117 is assigned identification information, viewers of the animation can use the identification information as a clue to easily recognize which reality mark 115 and which teacher mark 117 correspond (i.e., whether they represent the same player 107).

(1B) The sport is, for example, soccer. In this case, animation can be used to provide instruction to soccer player 107.
(1C) Image 113 is an image representing a half court 101 of a soccer field. Furthermore, multiple players 107 are players on the defensive team who are guarding a goal 103 on the half court 101. In this case, animation can be used to instruct the players 107 on the defensive team.

(1D) In the processing from step 3 onwards, a portion of frames 105 extracted from all of the frames 105 contained in the video data is used. This reduces the processing load on the calculation unit 3 when performing the processing from step 3 onwards.

<Other Embodiments>
Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be implemented in various modified forms.

(1) In the above embodiment, the sport is soccer, but the present invention is not limited to this. For example, the sport may be futsal, rugby, American football, or the like.
(2) The image 113 may represent the entire field of play. The players 107 may be players on an attacking team attacking the goal 103. The players 107 may also represent all players on the team.

(3) The appearance of the reality mark 115 and the teacher mark 117 is not particularly limited, and their shape, size, color, pattern, etc. can be set as appropriate. For example, the reality mark 115 or the teacher mark 117 may be a grid 102 with a different color or pattern, etc., from the other grids 102.

(4) The image 113 may further include other elements. For example, it may further include an indication that associates the reality mark 115 and the teacher mark 117, which have the same identification information. An example of an association indication is a line connecting the reality mark 115 and the teacher mark 117.

The display may further include a display showing the distance between the reality mark 115 and the teacher mark 117, which have the same identification information. Examples of the display showing the distance include a display of a numerical value of the distance.
(5) The animation creation device 1 may further include other functions. For example, a video may be created by combining multiple frames 105 in which markers are attached to multiple players 107, as shown in FIG. 3A. In the video, a marker may also be displayed on the ball 109, as shown in FIG. 6B. The video may be in MP4 format, for example.

(6) The specific frame 105A may be any frame 105 that shows the relationship between the player's 107 position and identification information, and does not have to be the frame 105 at the start of the competition or the start of the second half of the competition.

(7) The calculation unit 3 and the method described in the present disclosure may be realized by a special-purpose computer provided by configuring a processor and memory programmed to execute one or more functions embodied in a computer program. Alternatively, the calculation unit 3 and the method described in the present disclosure may be realized by a special-purpose computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the calculation unit 3 and the method described in the present disclosure may be realized by one or more special-purpose computers configured by combining a processor and memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored in a computer-readable non-transitory tangible recording medium as instructions to be executed by a computer. The method for realizing the functions of each unit included in the calculation unit 3 does not necessarily need to include software, and all of the functions may be realized using one or more pieces of hardware.

(8) The function of one component in each of the above embodiments may be shared among multiple components, or the functions of multiple components may be performed by one component. Furthermore, part of the configuration of each of the above embodiments may be omitted. Furthermore, at least part of the configuration of each of the above embodiments may be added to, replaced with, or otherwise applied to the configuration of another of the above embodiments.

(9) In addition to the animation creation device 1 described above, the present disclosure can also be realized in various forms, such as a system including the animation creation device 1 as a component, a program for causing a computer to function as the calculation unit 3, a non-transient tangible recording medium such as a semiconductor memory on which this program is recorded, a method for creating animation, a method for processing frames 105, etc.

[Technical idea disclosed in this specification]
[Item 1]
a video data acquisition unit configured to acquire video data generated by filming a game played on a field by a plurality of players using a ball;
a teacher data storage unit that stores, for each position of the ball on the field, a plurality of pieces of teacher data that associate the position of the ball on the field, the positions of the plurality of players on the field, and identification information assigned to each of the plurality of players;
a position detection unit configured to detect positions of the players and the ball in each of a plurality of frames included in the video data;
a position conversion unit configured to convert, in each of the plurality of frames, positions of the plurality of players and the ball in the frame into positions of the plurality of players and the ball on the field;
a first identification information assignment unit configured to assign the identification information to each of the plurality of players in a specific frame that is any one of frames included in the video data;
a tracking unit configured to perform tracking to associate the same player detected in each of the plurality of frames;
a second identification information assignment unit configured to assign the identification information assigned to the player of the specific frame by the first identification information assignment unit to players in frames other than the specific frame who are associated with the player of the specific frame through the tracking;
an image creation unit configured to create, for each of the plurality of frames, an image representing:
an animation creation unit configured to create an animation using the plurality of images created by the image creation unit;
An animation creation device comprising:
(a) the positions of the players and the ball on the field;
(b) the identification information assigned to the plurality of players;
(c) In the training data, the positions of the multiple players associated with the position of the ball in (a), and the identification information assigned to each of the multiple players in the training data.
[Item 2]
Item 1 is an animation creation device according to the present invention,
The sport is soccer,
Animation creation device.
[Item 3]
Item 2: The animation creation device according to item 2,
the image is an image representing a half court,
the plurality of players are players of a team defending a goal in the half court;
Animation creation device.
[Item 4]
The animation creation device according to any one of items 1 to 3,
The specific frame is a frame at the start of the competition or a frame at the start of the second half of the competition.
Animation creation device.
[Item 5]
The animation creation device according to any one of items 1 to 4,
the plurality of frames are a portion of frames extracted from all frames included in the video data;
Animation creation device.

1...Animation creation device, 3...Calculation unit, 5...Teacher data storage unit, 7...Cloud, 11...CPU, 13...Memory, 21...Video data acquisition unit, 23...Frame extraction unit, 25...Position detection unit, 27...Position conversion unit, 29...First identification information assignment unit, 31...Tracking unit, 33...Second identification information assignment unit, 35...Image creation unit, 37...Animation creation unit, 41...Teacher data, 101...Half court, 102...Grid, 103...Goal, 105...Frame, 107, 108...Player, 109...Ball, 111a-111d...Corner, 113...Image, 115...Real mark, 117...Teacher mark, 119...Ball mark

Claims (5)

a video data acquisition unit configured to acquire video data generated by filming a game played on a field by a plurality of players using a ball;
a teacher data storage unit that stores, for each position of the ball on the field, a plurality of pieces of teacher data that associate the position of the ball on the field, the positions of the plurality of players on the field, and identification information assigned to each of the plurality of players;
a position detection unit configured to detect positions of the players and the ball in each of a plurality of frames included in the video data;
a position conversion unit configured to convert, in each of the plurality of frames, positions of the plurality of players and the ball in the frame into positions of the plurality of players and the ball on the field;
a first identification information assignment unit configured to assign the identification information to each of the plurality of players in a specific frame that is any one of frames included in the video data;
a tracking unit configured to perform tracking to associate the same player detected in each of the plurality of frames;
a second identification information assignment unit configured to assign the identification information assigned to the player of the specific frame by the first identification information assignment unit to players in frames other than the specific frame who are associated with the player of the specific frame through the tracking;
an image creation unit configured to create, for each of the plurality of frames, an image representing:
an animation creation unit configured to create an animation using the plurality of images created by the image creation unit;
An animation creation device comprising:
(a) the positions of the players and the ball on the field;
(b) the identification information assigned to the plurality of players;
(c) In the training data, the positions of the multiple players associated with the position of the ball in (a), and the identification information assigned to each of the multiple players in the training data. 2. The animation creation device according to claim 1,
The sport is soccer,
Animation creation device. 3. The animation creation device according to claim 2,
the image is an image representing a half court,
the plurality of players are players of a team defending a goal in the half court;
Animation creation device. 3. The animation creation device according to claim 1,
The specific frame is a frame at the start of the competition or a frame at the start of the second half of the competition.
Animation creation device. 3. The animation creation device according to claim 1,
the plurality of frames are a portion of frames extracted from all frames included in the video data;
Animation creation device.