JP2025044237A - system - Google Patents

Abstract

To provide a system.SOLUTION: A system includes means for receiving video data and audio data of a match, means for analyzing the received video data; means for analyzing the received audio data, means for extracting scoring situations and foul situations from results of analysis of the video data and the audio data, means for generating a running score based on the extracted scoring situations and foul situations, means for updating and transmitting the generated running score in real time, means for displaying the transmitted running score, and means for recognizing a user's emotions and adjusting a display of the running score based on the user's emotions.SELECTED DRAWING: Figure 1

Description

The technology disclosed herein relates to a system.

Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description of the chatbot character and an associated instruction sentence, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

JP 2022-180282 A

When humans provide commentary for live broadcasts of basketball games, it is difficult to cover youth and mini-basketball games across the country. It is also difficult to automatically generate running scores that are created by hand for each official and team. Furthermore, it is difficult to create a database of teams and players across the country to visualize Japan's registered basketball players and use it to develop them.

We provide a means to provide commentary for live broadcasts of basketball games using AI. We provide a means to distribute commentary for youth and mini-basketball games across the country with live broadcasts by AI. We provide a means to automatically generate running scores using AI that are currently created by humans for each official and team. We provide a means to create a database of teams and players across the country, and provide a means to use the information in the database to visualize and develop registered basketball players in Japan.

1 is a conceptual diagram showing an example of a configuration of a data processing system according to a first embodiment. 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. FIG. 11 is a conceptual diagram showing an example of a configuration of a data processing system according to a second embodiment. FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. FIG. 13 is a conceptual diagram showing an example of a configuration of a data processing system according to a third embodiment. FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. 1 shows an emotion map onto which multiple emotions are mapped. 1 shows an emotion map onto which multiple emotions are mapped. FIG. 11 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. FIG. 11 is a sequence diagram showing a process flow of the data processing system in application example 1 of embodiment 1. FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second form example. FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third embodiment of the third aspect. FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 3 of embodiment 3. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system according to the first embodiment of the first form example when an emotion engine is combined. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in application example 1 of embodiment 1 when combined with an emotion engine. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment of the second form example when an emotion engine is combined. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in application example 2 of embodiment 2 when combined with an emotion engine. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system according to the third embodiment of the third form example when an emotion engine is combined. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in application example 3 of embodiment 3 when combined with an emotion engine.

Below, an example of an embodiment of a system related to the technology disclosed herein is described with reference to the attached drawings.

First, let us explain the terminology used in the following explanation.

In the following embodiments, the coded processor (hereinafter simply referred to as the "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. The processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

In the following embodiments, a signed random access memory (RAM) is a memory in which information is temporarily stored and is used as a working memory by the processor.

In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

In the following embodiments, a communication I/F (Interface) with a code is an interface including a communication processor and an antenna, etc. The communication I/F controls communication between multiple computers. Examples of communication standards applied to the communication I/F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

In the following embodiments, "A and/or B" is synonymous with "at least one of A and B." In other words, "A and/or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same concept as "A and/or B" is also applied when three or more things are expressed by connecting them with "and/or."

[First embodiment]

Figure 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with an indicator (e.g., a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and/or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

As shown in FIG. 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. Data generation model 58 and emotion identification model 59 are used by the identification processing unit 290.

In the smart device 14, the reception output process is performed by the processor 46. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the specific processing performed by the specific processing unit 290 of the data processing device 12.

"Example 1"

Example 1 of the present invention is a system in which AI provides commentary for a live broadcast of a basketball game. In this system, AI analyzes the video and audio data of the game to understand the situation of the game and the movements of the players. Then, based on the results of this analysis, a commentary of the game is automatically generated and distributed along with the live broadcast.

"Example 2"

Example 2 of the present invention is a system that distributes commentary of youth and mini-basketball games from around the country, with live coverage by AI. This system collects video and audio data from youth and mini-basketball games held around the country, which are analyzed by AI in the same way as example 1 to generate commentary. Then, by distributing the commentary together with the game footage, games from around the country are made available to a wide audience.

"Example 3"

Example 3 of the present invention is a system that uses AI to automatically generate running scores that are created manually by officials or for each team. In this system, AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game, and automatically generates a running score. This makes it possible to grasp the progress of the game in real time.

"Example 4"

Example 4 of the present invention is a system that creates a database of teams and players from across the country, and uses the information in the database to visualize and develop registered basketball players in Japan. This system collects information such as game results and stats for teams and players from across the country, and creates a database. The database is then used to provide feedback for analyzing players' performance and developing them.

The process flow for each example is explained below.

"Example 1"

Step 1: AI collects video and audio data from basketball games.

Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

Step 3: Based on the analysis results, the AI automatically generates commentary for the match.

Step 4: Distribute the generated commentary along with the live broadcast.

"Example 2"

Step 1: Collect video and audio data from youth and mini-basketball matches held across the country.

Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

Step 3: Based on the analysis results, the AI automatically generates commentary for the match.

Step 4: Distribute the generated commentary along with the game footage.

"Example 3"

Step 1: AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game.

Step 2: Based on the analysis results, the AI automatically generates a running score.

"Example 4"

Step 1: Collect information such as match results and stats for teams and players across the country.

Step 2: Create a database of the collected information.

Step 3: Use the databased information to analyze player performance and provide feedback for development.

(Example 1)

Next, a first embodiment of the first embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

Traditional live broadcasts of basketball games depend on the quality and quantity of commentators, making it difficult to accurately convey the detailed situation of the game and the movements of the players in real time. In addition, commentary on youth and mini-basketball games across the country is required by many people, which is costly and labor intensive. Furthermore, running scores created by officials and people for each team are done manually, so they take time and can lack accuracy. To solve these issues, an automated system that utilizes AI is needed.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for collecting video data and audio data of the match in real time, means for analyzing the collected video data and audio data, means for generating commentary on the match based on the analysis results, and means for distributing the generated commentary together with a live broadcast. This makes it possible to accurately convey the detailed situation of the match and the movements of the players in real time. In addition, commentary on youth and mini-basketball matches from all over the country can be distributed with live broadcasts by AI, reducing costs and labor. Furthermore, by automatically generating running scores by AI instead of manually created by officials and for each team, it is possible to reduce time and improve accuracy.

"Game video data" refers to digital data that visually records the progress of a basketball game.

"Audio data" refers to data that is a digital recording of the sounds generated during a basketball game.

"Means of collecting data in real time" refers to technology and devices that allow for the acquisition of video and audio data as the match progresses.

"Means of analysis" refers to technology and software that processes collected video and audio data to understand the situation of the game and the movements of the players.

"Means for generating commentary" refers to technology or software that automatically creates text that explains the situation of the game and the movements of players based on the analysis results.

"Means of distribution along with live broadcast" refers to the technology and devices that deliver the generated commentary to viewers in real time along with the video and audio of the match.

"Youth and mini-basketball game commentary" is commentary on basketball games aimed at young people and elementary school students.

"Running score" is data that records the scores and player performances that are updated in real time during the game.

"AI automatic generation means" refers to technology and software that uses artificial intelligence to automate manual tasks and generate data.

"Means of database creation" refers to technology and software for organizing collected information and storing it in a format that is easy to search and use.

"Visualization" refers to techniques and methods for visually displaying data and making it easier to understand.

"Means to be used in development" refers to technologies and methods that use data to support the training and development of athletes.

This invention is a system that uses AI to automatically generate commentary on a basketball game and deliver it in real time during live broadcasts. A specific embodiment of this system is described below.

Server operation

1. Data collection

The server collects video and audio data of the match in real time. Specifically, it uses a camera and microphone to capture video and audio of the match. Video data is collected at a resolution of 1080p, and audio data is collected at a sampling rate of 44.1kHz.

2. Data analysis

The server analyzes the collected video and audio data. This analysis uses image recognition software (e.g. OpenCV) and voice recognition software (e.g. Google (registered trademark) Speech-to-Text API). This allows the server to understand the situation of the game and the movements of the players. For example, it detects the action of Player A taking a shot and converts the live audio of that moment into text.

3. Description generation

The server generates commentary on the game based on the analysis results. A generative AI model (e.g. GPT-4 (registered trademark)) is used to generate this commentary. The generative AI model receives information about the game situation and player movements as input, and generates commentary in natural language. For example, it inputs information such as "Player A made a successful three-point shot" and generates commentary such as "Player A made a brilliant three-point shot."

4. Distribution

The server distributes the generated commentary along with the live broadcast. A streaming server (e.g. Wowza Streaming Engine) is used for distribution. This allows users to watch the game footage and commentary in real time over the Internet.

Device operation

1. Data reception

The device receives real-time game footage and commentary from the server using an internet connection and streaming software (e.g. VLC media player).

2. Display

The device displays the received video and commentary to the user. For display purposes, hardware such as a display and speakers are used. For example, if the user is watching on a smartphone, the video will be displayed on the smartphone screen and the commentary audio will be played through the built-in speaker.

User actions

1. Viewing

The user watches the game footage and commentary through the terminal. The user can get detailed commentary on the progress of the game and the movements of the players in real time. For example, if the user is watching on a television in the living room, the game footage will be displayed on the television screen and commentary will be played from the television speakers.

Specific examples

Example 1

Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

Player Action: Home team player A makes a three-point shot.

Example prompts for generative AI models

Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

Player Action: Home team player A makes a three-point shot.

Example of generated description

"With five minutes left in the third quarter, the home team is tied at 75 points and the away team at 70 points. Home team player A makes a great three-point shot, widening the gap even further."

In this way, users can watch detailed commentary on the progress of the match and the players' movements in real time.

The flow of the identification process in Example 1 is explained using Figure 11.

Step 1:

The server collects video and audio data from the match in real time.

Specific tasks include using cameras and microphones to capture video and audio of the game.

The input is video and audio from the camera and microphone, and the output is video and audio data in digital format.

Video data is collected at 1080p resolution, and audio data is collected at a sampling rate of 44.1kHz.

Step 2:

The server analyzes the collected video and audio data.

Specific operations include using image recognition software (e.g. OpenCV) to detect player movements from video data, and using voice recognition software (e.g. Google Speech-to-Text API) to convert the audio data into text to provide commentary.

The input is the video and audio data collected in step 1, and the output is the analyzed player movement information and the text commentary.

For example, it detects when Player A is about to take a shot and converts the audio commentary at that moment into text.

Step 3:

The server generates commentary for the match based on the analysis results.

Specifically, the prompt text is input into a generative AI model (e.g. GPT-4) to generate an explanatory text.

The input is the player movement information obtained in step 2 and the text commentary, and the output is the generated commentary.

For example, input information such as "Player A made a successful three-point shot" and generate an explanatory text such as "Player A made a brilliant three-point shot."

Step 4:

The server distributes the generated commentary along with the live broadcast.

Specifically, the video and commentary audio are streamed in real time using a streaming server (e.g. Wowza Streaming Engine).

The input is the explanatory text generated in step 3 and the video data collected in step 1, and the output is the video and explanatory audio delivered in real time.

Step 5:

The device receives game footage and commentary streamed from the server in real time.

Specific actions include using an internet connection and streaming software (e.g. VLC media player).

The input is streaming data from the server, and the output is the video displayed on the terminal and the commentary audio played.

Step 6:

The device displays the received video and commentary to the user.

Specific operations include showing images on the display and playing explanatory audio from the speaker.

The input is the video and commentary audio received in step 5, and the output is a live broadcast of the game provided to the user in both visual and audio.

Step 7:

Users can watch game footage and commentary through their devices.

Specific actions include watching the game footage on the device display and listening to commentary coming from the speaker.

The input is the video displayed in step 6 and the commentary audio played, and the output is the user's viewing experience.

For example, if a user is watching on a television in their living room, the game footage will be displayed on the television screen and commentary will be played from the television speakers.

(Application example 1)

Next, application example 1 of embodiment example 1 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

Existing basketball game commentary systems do not adequately provide real-time commentary, post-game highlight generation, or detailed analysis of players' performance. In addition, there is a problem that it is difficult to provide appropriate commentary immediately when a user asks about a specific player or play.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 1 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games nationwide with AI commentary, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for analyzing game video data and generating commentary in real time, a means for automatically extracting important scenes after the game ends and generating a highlight video, a means for analyzing player movements and performance and providing detailed statistical information, and a means for AI to provide commentary when a user asks a question about a specific player or play. This makes it possible to provide commentary in real time, generate post-game highlights, analyze players' performance in detail, and provide immediate commentary in response to user questions.

"A means for AI to provide commentary for live broadcasts of basketball games" refers to technology in which AI analyzes video and audio data from basketball games, understands the situation of the game and the movements of the players, and automatically generates commentary.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that uses AI to provide commentary on youth and mini-basketball games from around the country and broadcasts the games in real time with that commentary.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses AI to automatically generate scores that are manually recorded by officials and teams during a match.

"Means for analyzing game video data and generating commentary in real time" refers to technology that analyzes game video data in real time and instantly generates commentary based on the analysis results.

"Means for automatically extracting important scenes after the end of a match and generating a highlight video" refers to a technology that analyzes video data of a match after the end of the match, automatically extracts important scenes, and generates a highlight video.

"Means of analyzing player movements and performance and providing detailed statistical information" refers to technology that analyzes player movements and performance and provides detailed statistical information based on the results.

"A means for an AI to provide commentary when a user asks a question about a specific player or play" refers to technology that enables an AI to provide an appropriate commentary in response to a question when a user asks about a specific player or play.

To implement this invention, the following system configuration and processing steps are required.

System configuration

Hardware

Server: A high-performance computer for running AI models and analyzing data.

Smartphone: The device on which users watch and interact with the game commentary.

Camera: A device used to capture footage of the match.

Software

OpenCV: A library used to read and analyze video data.

TENSORFLOW (registered trademark): A framework used to run AI models.

Transformers: A library that generates explanations using the GPT-2 model.

Processing flow

1. Loading video data

The server uses OpenCV to read the game video data obtained from the camera frame by frame.

2. Frame analysis

The server analyzes the loaded frames to detect player movements and important actions. This analysis uses a deep learning model using TensorFlow.

3. Generate explanations

The server generates an explanatory text based on the analysis results. Specifically, it runs the GPT-2 model using the Transformers library and generates an explanatory text using the prompt text as input.

4. Delivering commentary

The generated commentary is delivered to smartphones in real time, allowing users to watch and listen to commentary on the match via their smartphones.

5. Highlight video generation

After the match ends, the server automatically extracts important scenes and generates a highlight video. This highlight video is also distributed to smartphones.

6. Player performance analysis

The server performs detailed analysis of the players' movements and performance and generates statistical information that users can view on their smartphones.

7. User interaction

When a user asks a question about a particular player or play, the server generates an appropriate commentary for that question and delivers it to the smartphone.

Specific examples

Scoring scene in the game: Detect the moment when Player A makes a three-point shot and generate a commentary such as "Player A scores a three-pointer."

Defensive highlights: Detects a scene where Player B successfully blocks a shot and generates a commentary such as "Player B makes a great block."

Example of a prompt

"Player A scores a three-pointer. Generate a commentary for this action."

"Player B makes a great block. Generate a commentary for this action."

In this way, the server can analyze the game video data in real time and automatically provide commentary to the user. This allows for real-time commentary, post-game highlights, detailed analysis of player performance, and instant commentary in response to user questions.

The flow of the specific processing in application example 1 is explained using Figure 12.

Step 1:

The server uses OpenCV to read the game video data captured by the camera frame by frame. The input is the video data from the camera, and the output is the individual frame images. Specifically, the server splits the video data into a series of still images and stores each frame in memory.

Step 2:

The server analyzes the loaded frames to detect player movements and important actions. A deep learning model using TensorFlow is used for this analysis. The input is the frame image obtained in step 1, and the output is information about the player movements and actions. Specifically, the server detects the positions and movements of players within the frames and identifies important actions such as points and blocks.

Step 3:

The server generates a commentary based on the analysis results. Specifically, it runs the GPT-2 model using the Transformers library and generates a commentary using the prompt as input. The input is the action information and prompt obtained in step 2, and the output is the generated commentary. The server generates a prompt corresponding to the player's action and inputs it into the GPT-2 model to obtain the commentary.

Step 4:

The server delivers the generated commentary to the smartphone in real time. The input is the commentary generated in step 3, and the output is the commentary displayed on the smartphone. Specifically, the server transmits the commentary to the smartphone via the network so that the user can view it.

Step 5:

After the match ends, the server automatically extracts important scenes and generates a highlight video. The input is the video data of the entire match, and the output is the highlight video. Specifically, the server selects highlight scenes based on important actions detected during the match and links them together to create a highlight video.

Step 6:

The server performs detailed analysis of players' movements and performance and generates statistical information. The input is frame images from the game and information about players' actions, and the output is detailed statistical information. Specifically, the server compiles data such as points, rebounds, and assists for each player and compiles it as statistical information.

Step 7:

When a user asks a question about a specific player or play, the server generates an appropriate commentary for that question and delivers it to the smartphone. The input is the user's question and prompt text, and the output is the generated commentary text. Specifically, the server analyzes the user's question, generates a corresponding prompt text, inputs it into the GPT-2 model, obtains the commentary text, and sends it to the smartphone.

(Example 2)

Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

Traditionally, commentary on basketball games required a specialized commentator, and there was the problem of difficulty in securing commentators for youth and mini-basketball games held all over the country. It was also difficult to convey the progress of the game and the movements of the players to viewers in real time, limiting the viewing experience. Furthermore, the collection and analysis of game data was done manually, which was inefficient and sometimes lacked consistency and accuracy.

The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

In this invention, the server includes means for collecting video and audio data of matches held throughout the country, means for storing the collected data in cloud storage, means for transmitting the stored data to a generative AI model and generating commentary for the match, means for synchronizing the generated commentary with the match video, means for streaming the match video with commentary, means for providing an interface for users to view, means for receiving the match video with commentary streamed from the server, and means for displaying the received video to the user. This enables live broadcasts with high-quality commentary by AI even for youth and mini-basketball matches held throughout the country, improving the viewing experience. In addition, efficient and accurate data management can be achieved because the collection and analysis of match data is automated.

"Means for collecting video and audio data from matches held around the country" refers to devices and systems that use cameras and microphones installed at match venues around the country to capture video and audio data from matches in real time and transmit it to a server.

"Means for storing collected data in cloud storage" refers to devices or systems that convert collected video and audio data into an appropriate format and store the data using cloud services.

"Means for sending stored data to a generative AI model and generating commentary on a match" refers to a device or system that retrieves data stored in cloud storage, sends prompt text to the generative AI model to analyze the data, and generates commentary on a match.

"Means for synchronizing the generated commentary with the game video" refers to a device or system that inserts the generated commentary text at a specific timestamp in the game video, allowing the video and commentary to be played in a synchronized state.

"Means for streaming game footage with commentary" refers to a device or system that encodes synchronized video data in real time and transmits it to a streaming platform for distribution to viewers.

"Means for providing an interface for users to view" refers to a device or system for displaying a viewing interface to users through a device such as a smartphone, tablet, or PC.

"Means for receiving game footage with commentary streamed from a server" refers to a device or system for receiving game footage with commentary streamed from a server in real time via an Internet connection.

"Means for displaying received video to a user" refers to a device or system for decoding received video data and displaying it on the user's device screen.

This invention is a system that broadcasts youth and mini-basketball matches held around the country with commentary by AI. A specific embodiment of this system is described below.

Server role

The server collects video and audio data from matches held around the country. Cameras and microphones installed at match venues around the country capture video and audio from matches in real time and send this data to the server. The collected data is stored in cloud storage such as Amazon S3 and Google Cloud Storage.

The server then sends the stored data to a generative AI model. Examples of generative AI models that can be used include OpenAI's (registered trademark) GPT-4 and Google's BERT. The server retrieves the data from cloud storage and sends a prompt to the generative AI model. The prompt includes instructions to analyze the progress of the game, the movements of the players, the score, etc.

The generative AI model analyzes the data based on the prompt text and generates commentary in natural language. The generated commentary is sent back to the server in text format. The server then synchronizes the generated commentary with the game footage. This synchronization process is performed using video editing software such as FFmpeg. The server inserts the commentary text at specific timestamps in the footage, allowing the footage and commentary to be played in sync.

Finally, the server streams the game footage with commentary to a streaming platform such as YouTube Live or Twitch. The server encodes the synchronized video data in real time and sends it to the streaming platform.

Role of the device

The terminal provides the user with an interface for viewing. Devices such as smartphones, tablets, and PCs are used here. The terminal displays the viewing interface to the user through a dedicated application or web browser. The interface includes a list of matches and a search function.

The device receives game footage with commentary streamed from the server. An Internet connection is required for this reception. The device receives video data from the streaming platform in real time and buffers it. The device decodes the received video data and displays it on the user's device screen.

User Roles

Users access the system using a terminal. To do so, they use a dedicated application or a web browser. Users launch a dedicated application on their smartphone or PC, or access the system's URL using a web browser.

The user selects the match they wish to watch. For this selection, a list of matches and a search function are provided. The user scrolls through the list of matches on the interface and clicks on the match they wish to watch. They can also use the search function to find a specific match.

The user watches the commentary-enhanced video of the selected match. While watching, the user can play, pause, rewind, and perform other operations. The user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed.

Specific examples

For example, if a user wants to watch a national youth basketball tournament game on their smartphone, they would follow these steps:

1. The user launches the dedicated app on their smartphone.

2. Use the in-app search function and type in "National Youth Basketball Tournament."

3. Select the match you want to watch from the search results.

4. The app streams game footage with commentary from the server.

5. Users can watch the game on their smartphone screen and enjoy the commentary.

An example of a prompt sentence to be input to the generative AI model is, "Analyze the game video and audio data of the National Youth Basketball Tournament and generate commentary including the progress of the game, player movements, score status, etc." Using this prompt sentence, the AI analysis system generates commentary of the game.

The flow of the identification process in Example 2 is explained using Figure 13.

Step 1:

The server collects video and audio data from matches held around the country. As input, it receives real-time data from cameras and microphones installed at match venues around the country. Specifically, it transmits the video and audio captured by these devices to the server via the internet. As output, it obtains the collected video and audio data.

Step 2:

The server stores the collected data in cloud storage. As input, it receives the video and audio data collected in step 1. Specifically, it converts the data into an appropriate format and uploads it to a cloud service such as Amazon S3 or Google Cloud Storage. As output, it obtains the data stored in cloud storage.

Step 3:

The server sends the saved data to the generative AI model to generate commentary on the match. As input, it receives video and audio data obtained from cloud storage. Specifically, it sends a prompt to the generative AI model to analyze the data. The prompt includes instructions to analyze the progress of the match, player movements, the score, etc. The output is the commentary text returned from the generative AI model.

Step 4:

The server synchronizes the generated commentary with the game footage. As input, it receives the commentary text obtained in step 3 and the video data obtained from the cloud storage. Specifically, it uses video editing software such as FFmpeg to insert the commentary text at a specific timestamp in the video. As output, it obtains the game footage synchronized with the commentary.

Step 5:

The server streams the game footage with commentary. As input, it receives the game footage synchronized with the commentary obtained in step 4. Specifically, it encodes the video data in real time and transmits it to a streaming platform such as YouTube Live or Twitch. As output, it obtains the streaming video that is distributed to the viewers.

Step 6:

The terminal provides an interface for users to watch. It receives user operations as input. Its specific operation is to display the viewing interface through a dedicated application or web browser. The interface includes a list of matches and a search function. As output, it provides information that allows the user to select the match they wish to watch.

Step 7:

The terminal receives game footage with commentary that is streamed from the server. As input, it receives streaming data from the server. Specifically, it receives video data in real time via an Internet connection and buffers it. As output, it obtains the received video data.

Step 8:

The terminal displays the received video to the user. As input, it receives the video data received in step 7. Specifically, it decodes the video data and displays it on the user's device screen. As output, it obtains the video that the user watches.

Step 9:

The user accesses the system using a terminal. A dedicated application or a web browser is used as input. Specifically, the user launches a dedicated application on a smartphone or PC, or accesses the system's URL in a web browser. Access to the system is obtained as output.

Step 10:

The user selects the match they want to watch. As input, they use the match list and search functionality on the viewing interface. They scroll through the match list and click on the match they want to watch. They can also use the search functionality to search for a specific match. As output, they get the selected match.

Step 11:

The user watches the commentary-enhanced video of the selected match. As input, the match selected in step 10 is received. Specifically, the user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed. As output, the video of the match being watched is obtained.

(Application example 2)

Next, application example 2 of embodiment example 2 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

In conventional basketball game commentary systems, live broadcasts and commentary of games were dependent on human labor, making it difficult to provide youth and mini-basketball games held all over the country to a wide audience. In addition, there was a lack of game highlights and replay functions, and post-game statistical data and analysis reports, making it difficult to increase viewer satisfaction. Furthermore, there was no function for users to follow their favorite teams or players, making it difficult to meet individual needs.

The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for AI to provide commentary on the live broadcast of a basketball game, a means for distributing commentary of youth and mini-basketball games nationwide with AI live broadcast, a means for AI to automatically generate running scores created by officials and people for each team, a means for collecting video and audio data of the game and analyzing it with AI to generate commentary, a means for distributing the game in real time through an application installed on a smartphone and providing commentary by AI, a means for providing highlights and replay functions of the game, a means for providing a function that allows users to follow their favorite teams and players, and a means for providing statistical data and analysis reports after the game. This makes it possible to widely provide youth and mini-basketball games held all over the country to viewers and increase the satisfaction of the viewers.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to generate commentary in real time during a basketball game and provide it to viewers.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to technology that broadcasts youth and mini-basketball games held around the country in real time with commentary by artificial intelligence.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Method of collecting video and audio data from the match and analyzing it with AI to generate commentary" refers to technology that collects video and audio data from the match and uses artificial intelligence to analyze it and generate commentary.

"Means of broadcasting matches in real time and providing commentary by AI through an application installed on a smartphone" refers to a technology that uses a smartphone application to broadcast matches in real time and provide commentary by artificial intelligence.

"Means for providing match highlight and replay features" refers to technology that provides the functionality to compile important scenes and plays during a match as highlights or play them back as replays.

"Means for providing a function that enables users to follow their favorite teams and players" refers to technology that provides a function that enables users to select and follow their favorite teams and players and receive information about them on a priority basis.

"Means for providing post-match statistical data and analytical reports" refers to technology that generates match statistical data and analytical reports after the match ends and provides them to users.

The following system configuration is described as a form for implementing this invention.

System configuration

The system includes a server, a smartphone, and an AI model. The server collects video and audio data from the match, analyzes it using the AI model, and generates commentary. The smartphone provides an application that enables users to watch the match in real time and receive commentary by the AI.

Hardware and software used

Server: A server with high-performance data processing capabilities (e.g. AWS (registered trademark) EC2 instance)

Smartphone: iOS or ANDROID (registered trademark) device

AI model: OpenAI's GPT-4

Data processing library: OpenCV (for image processing), Python (programming language)

Data processing and data calculations

1. Collecting video and audio data from the match:

The server collects video and audio data in real time from youth and mini-basketball matches held across the country. This is done using input devices such as cameras and microphones.

2. AI-based analysis and explanation generation:

The server uses OpenCV to analyze the collected video and audio data and extracts important scenes and plays from the match. Based on the extracted information, it uses OpenAI's GPT-4 to generate commentary. Specifically, it uses the following prompt sentence:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

3. Real-time broadcasting and commentary:

The server delivers the game footage along with the generated commentary to smartphones in real time. An application installed on the smartphone allows users to receive the AI commentary while watching the game.

4. Providing highlight and replay features:

The server compiles highlights of key scenes during the match and provides a replay function, allowing users to rewatch key moments of the match.

5. User following feature:

The smartphone application provides users with the ability to follow their favorite teams and players. This allows users to receive information about the teams and players they follow preferentially.

6. Providing post-match statistics and analysis reports:

After the match ends, the server generates statistical data and analysis reports and provides them to users via a smartphone application, allowing users to obtain detailed analysis information about the match.

Specific examples

For example, when a user is watching a youth basketball game through a smartphone application, the server delivers real-time game footage and AI commentary. The moment Player A scores a goal during the game, the server analyzes the scene and generates the following prompt:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

Based on this prompt, the AI model will generate commentary and provide it to the user. Users can enjoy the AI commentary in real time while watching the match. In addition, after the match, users can obtain detailed information about the match through statistical data and analysis reports.

The flow of the specific processing in application example 2 is explained using Figure 14.

Step 1:

The server collects video and audio data from youth and mini-basketball matches held around the country. As input, it receives real-time data from input devices such as cameras and microphones, and as output, it stores this data in storage. In concrete terms, the server receives video and audio data sent from each match venue, converts it into an appropriate format, and stores it.

Step 2:

The server analyzes the collected video and audio data. It uses the video and audio data saved in step 1 as input, and extracts information about important scenes and plays in the match as output. Specifically, the server uses OpenCV to analyze the video data frame by frame and identify the movements of players and the position of the ball. It also extracts events such as goals and fouls from the audio data.

Step 3:

The server generates a prompt sentence based on the analysis results. As input, it uses information about important scenes and plays in the match extracted in step 2, and as output, it generates a prompt sentence to pass to the AI model. Specifically, the server converts information such as player movements and scoring status into text format and generates a prompt sentence like the following:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

Step 4:

The server sends the generated prompt text to the AI model and generates an explanation. It uses the prompt text generated in step 3 as input and receives the explanation text generated from the AI model as output. In concrete terms, the server calls OpenAI's GPT-4 API, sends the prompt text, and generates an explanation.

Step 5:

The server delivers the generated commentary and game footage to smartphones in real time. It uses the commentary text generated in step 4 and the video data collected in step 1 as input, and generates data to be delivered to the user's smartphone as output. In concrete terms, the server overlays the commentary text on the video data and transmits it to the smartphone in streaming format.

Step 6:

Users watch the game through a smartphone application and receive commentary by AI. As input, it uses video and commentary data distributed from the server, and as output, it displays the game video and commentary to the user. In concrete terms, the smartphone application decodes the received data and displays it on the screen.

Step 7:

The server summarizes important scenes during the match as highlights and provides a replay function. It uses the information of important scenes extracted in step 2 as input and generates highlight videos as output. In concrete terms, the server edits the important scenes and saves them in a format that users can replay.

Step 8:

Users follow their favorite teams and players through a smartphone application. The application uses the user's selection information as input, and prioritizes displaying information about the followed teams and players as output. In concrete terms, the application records the user's selection, and prioritizes retrieving and displaying related information.

Step 9:

After the match ends, the server generates statistical data and analysis reports and provides them to the user. It uses data collected during the match as input and generates statistical data and analysis reports as output. In concrete terms, the server aggregates the match data, creates a report in a visually easy-to-understand format, and provides it to the user through a smartphone application.

(Example 3)

Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

In traditional basketball games, in order to grasp the score and foul situation in real time, it was necessary to manually create a running score, which was time-consuming and labor-intensive. In addition, there was a lack of a system to effectively collect and analyze data on teams and players across the country and use it for player development. This meant that there was insufficient visualization of player performance and feedback for development.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes a means for receiving video data and audio data of a match, a means for analyzing the received video data, a means for analyzing the received audio data, a means for extracting the score situation and the foul situation from the analysis result, a means for generating a running score based on the extracted information, a means for updating the generated running score in real time and transmitting it to the terminal, a means for displaying the running score transmitted to the terminal, a means for collecting information such as the match results and stats of teams and players nationwide, a means for database-izing the collected information, a means for analyzing the databased information and evaluating the performance of players, a means for generating feedback for player development based on the analysis result, a means for transmitting the generated feedback to the terminal, and a means for displaying the feedback transmitted to the terminal. This makes it possible to grasp the progress of the match in real time, and effectively visualize the performance of players and provide feedback for development.

"Game video data" refers to video files captured by a camera showing a basketball game.

"Audio data" refers to audio files recorded by a microphone during a basketball game.

"Means of receiving" refers to the function that allows the server to obtain data from outside.

"Means of analysis" refers to the function for processing received data and extracting necessary information.

"Scoring status" is information that shows how many points each team has scored during a match.

"Foul situation" is information that shows which players committed what type of fouls during the game.

The "running score" is a record of points and fouls that is updated in real time as the game progresses.

"Means of updating in real time" refers to a function that instantly reflects data as the match progresses.

"Terminal" refers to the device that a user uses to view information.

"Means of display" refers to the functionality for visually presenting information on a terminal.

"Match results" refers to information showing the final score and outcome after the match has ended.

"Stats" are statistical data that quantify the performance of players and teams.

"Means of collection" refers to the function for collecting the necessary data.

"Means of database creation" refers to the function of organizing and storing collected data.

"Means of analysis" refers to the function of processing databased information and deriving meaningful results.

"Feedback" refers to advice or evaluation provided based on the analysis results.

"Visualization" refers to displaying data visually to make it easier to understand.

"Development" refers to training and guidance to improve players' abilities.

This invention is a system for grasping the score and foul situation in a basketball game in real time and evaluating and developing the performance of players. A specific embodiment of this system is described below.

System configuration

Hardware

Server: A computer with high-performance computing power that receives, analyzes, generates, and transmits data.

Device: The device on which the user checks information, including smartphones, tablets, and computers.

Camera: A device used to capture video data of a match.

Microphone: A device used to capture audio data from a match.

Software

Video analysis software: Analyzes video data using libraries such as OpenCV.

Speech analysis software: Use libraries such as LibROSA to analyze speech data.

Database management system: Collected data is stored in a database using MySQL (registered trademark) or similar.

Data analysis tools: Analyze data using tools such as Pandas and NumPy.

Generative AI models: Use models such as GPT-4 to generate running scores and feedback.

System operation

Data reception and analysis

The server receives video and audio data of the game in real time from the cameras and microphones. The received video data is analyzed using video analysis software to detect goals and fouls. The audio data is analyzed using audio analysis software to analyze the commentary and crowd reactions.

Information extraction and running score generation

The server extracts the scoring and foul situations from the analysis results. Based on the extracted information, a generative AI model is used to generate a running score. The generated running score is updated in real time and sent to the terminal.

Display data

The device displays the received running score, allowing the user to follow the progress of the match in real time.

Data collection and database creation

The server collects information such as match results and stats for teams and players across the country. The collected information is stored in a database using a database management system.

Data analysis and feedback generation

The server analyzes the databased information and evaluates the player's performance. Based on the analysis results, a generative AI model is used to generate feedback for the player's development. The generated feedback is sent to the terminal, which displays it. This allows the user to check the analysis results of the player's performance and advice for development.

Specific examples

When a user is watching a basketball game, the server analyzes the video and audio of the game and automatically extracts information about scores and fouls.

The server generates information in real time, such as "Team A has scored" or "Team B has committed a foul," and sends it to the terminal.

The device displays this information, allowing users to check the progress of the match in real time.

Example of a prompt

"Analyze video and audio data from basketball games, extract the scoring and foul situations, and generate a running score."

"Analyze the game results and stats of basketball teams and players nationwide, evaluate the performance of players, and generate feedback for training." The flow of the specific process in Example 3 is explained with reference to FIG. 15.

Step 1:

The server receives video and audio data from the match.

Input: Real-time data from camera and microphone

Specific operation: Receives video and audio transmitted from the camera and microphone via the network.

Output: Received video and audio data

Step 2:

The server analyzes the received video data.

Input: Received video data

Specific operation: Use video analysis software (e.g. OpenCV) to detect scoring and foul scenes from video data.

Output: Data with timestamps for goals and fouls

Step 3:

The server analyzes the received audio data.

Input: Received audio data

Specific actions: Use audio analysis software (e.g. LibROSA) to analyze commentary and audience reactions from audio data.

Output: Timestamped data of commentary and audience reactions

Step 4:

The server extracts the scoring and foul situations from the analysis results.

Input: Timestamped data on goals and fouls, as well as timestamped data on commentary and crowd reactions

Specific operation: Integrate the results of video and audio analysis to extract the occurrence of goals and fouls.

Output: Scoring and foul situation data

Step 5:

The server generates a running score based on the extracted information.

Input: Scoring and foul situation data

Specific behavior: Uses a generative AI model (e.g. GPT-4) to generate points and foul information in text format.

Output: Text data of running score

Step 6:

The server updates the generated running score in real time and sends it to the device.

Input: Text data of running score

Specific operation: Sends the running score to the device using WebSocket or HTTP protocol.

Output: Running score sent to the device

Step 7:

The device will display the received running score.

Input: Running score sent to the device

Specific operation: Visually display the running score on the device display.

Output: A running score that the user can visually check

Step 8:

The server collects information such as match results and stats for teams and players across the country.

Input: Data from each team's official website and sports data services

Specific actions: Collect match results and stats using web scraping and APIs.

Output: Collected match results and stats data

Step 9:

The server stores the collected information in a database.

Input: Collected match results and stats data

Specific action: Store data in a table using a database management system (e.g. MySQL).

Output: Match results and stats data stored in a database

Step 10:

The server analyzes the databased information and evaluates the players' performance.

Input: Match results and stats data stored in the database

Specific operations: Use data analysis tools (e.g. Pandas, NumPy) to calculate players' shooting success rates, defensive strength, etc.

Output: Player performance evaluation data

Step 11:

The server generates feedback for player development based on the analysis results.

Input: Player performance evaluation data

Specific actions: Use a generative AI model (e.g. GPT-4) to generate textual feedback for player development.

Output: Text data of feedback for training

Step 12:

The server sends the generated feedback to the device.

Input: Text data for feedback for training

Specific behavior: Sends feedback to the device using WebSocket or HTTP protocol.

Output: Feedback sent to the device

Step 13:

The device will display the feedback it receives.

Input: Feedback sent to device

Specific action: Visually display feedback on the device display.

Output: Visual feedback for the user

(Application example 3)

Next, application example 3 of embodiment example 3 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

Traditional live broadcasts and commentary of basketball games rely on human commentators, which can make it difficult to provide accurate information in real time. Commentary on youth and mini-basketball games across the country also relies on human input, making it difficult to provide information efficiently. Furthermore, when monitoring production lines in factories, detecting abnormalities and improving production efficiency are issues. To solve these issues, an automated system that utilizes AI is needed.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 3 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games from around the country with AI live coverage, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for analyzing camera footage and audio data within the factory to detect production status and abnormal occurrences in real time and provide scores, and a means for storing past production data in a database and providing feedback for efficiency. This allows for accurate understanding of the progress of the game and the status of the production line in real time, making it possible to provide efficient information and improve production efficiency.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to automatically provide commentary on the progress of a basketball game and plays in real time.

"A means of broadcasting commentary on youth and mini-basketball games nationwide with live coverage by AI" refers to a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games nationwide and broadcasts that commentary in real time.

"A means of using AI to automatically generate running scores that are created manually by officials and for each team" refers to technology in which artificial intelligence automatically analyzes points, fouls, and other scores as the game progresses, and generates scores in real time.

"A means of analyzing camera footage and audio data within a factory to detect and score production status and abnormalities in real time" refers to a technology in which artificial intelligence analyzes video and audio data obtained from cameras and microphones installed within the factory, detects the status and abnormalities of the production line in real time, and displays the results as a score.

"Means of compiling past production data into a database and providing feedback for improving efficiency" refers to a technology that stores past production data collected within a factory in a database, analyzes the data, and provides feedback for improving production efficiency.

The following system configuration is described as a form for implementing this invention.

System configuration

Hardware

Camera: Cameras installed in the factory capture video data.

Microphones: Microphones installed in the factory capture audio data.

Server: A server for analyzing and storing data.

Device: The device (PC, smartphone, etc.) used by the administrator to check the results.

Software

OpenCV: A library for acquiring and preprocessing camera images.

TensorFlow/Keras: Libraries for loading AI models and making predictions.

SQLite: A database management system for storing production data.

Data processing and data calculations

The server acquires camera footage and audio data from within the factory in real time and performs preprocessing using OpenCV. The preprocessed data is input into an AI model using TensorFlow/Keras to detect production status and abnormalities. The detection results are displayed as a score and stored in a SQLite database.

Specific examples

For example, if an abnormality occurs in a factory, it will be detected from camera footage and notified to managers in real time. It can also analyze past production data and provide feedback to improve efficiency.

Example of a prompt

An example of a prompt to input to the generative AI model is as follows:

"Please create an AI model that analyzes camera footage in the factory and detects abnormalities on the production line in real time. Please also add a function to notify the administrator if an abnormality is detected. Also, please create a system that stores past production data in a database and provides feedback to improve efficiency."

In this way, the server can monitor the status of the production line in real time and detect any anomalies. It can also provide feedback for efficiency improvements based on past data.

The flow of the specific processing in application example 3 is explained using Figure 16.

Step 1:

The server captures video and audio data in real time from cameras and microphones installed in the factory. The input is the raw data from the cameras and microphones, and the output is the captured video and audio data. In concrete terms, the server connects to the cameras and microphones and starts the data stream.

Step 2:

The server preprocesses the acquired video data using OpenCV. The input is the video data acquired in step 1, and the output is the preprocessed video data. Specifically, the server resizes the video data to 224x224 pixels and normalizes it.

Step 3:

The server inputs the preprocessed video data into the AI model using TensorFlow/Keras to make predictions. The input is the video data preprocessed in step 2, and the output is the predicted results of production status and abnormality occurrence. In concrete terms, the server loads the AI model, inputs the preprocessed data into the model, and makes predictions.

Step 4:

The server displays the prediction result as a score and saves it in a SQLite database. The input is the prediction result obtained in step 3, and the output is the displayed score and the data saved in the database. In concrete terms, the server converts the prediction result into a score, displays it on the screen, and inserts it into the database.

Step 5:

The terminal receives the score and anomaly detection notifications sent from the server and displays them to the administrator. The input is the score and notifications sent from the server, and the output is the display information that the administrator can check. In concrete terms, the terminal receives data from the server and displays it on the user interface.

Step 6:

The server analyzes past production data and generates feedback for efficiency improvements. The input is past production data stored in a database, and the output is feedback information for efficiency improvements. In concrete terms, the server retrieves past data from the database and applies an analysis algorithm to generate feedback.

Step 7:

The terminal receives feedback information sent from the server and displays it to the administrator. The input is the feedback information sent from the server, and the output is the display information that the administrator can check. In concrete terms, the terminal receives feedback information from the server and displays it on the user interface.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

"Example 1"

In one embodiment of the present invention, a system in which AI provides commentary on a live broadcast of a basketball game is combined with an emotion engine that recognizes the user's emotions. Specifically, the emotion engine recognizes the emotions (happiness, sadness, surprise, etc.) that the user expresses while watching the game, and adjusts the content and tone of the commentary according to those emotions. For example, if the user expresses joy at how the game is progressing, the commentary will also share that joy. This allows the user to hear commentary that reflects their own emotions, further improving their satisfaction with watching the game.

"Example 2"

The emotion engine is also incorporated into a system that broadcasts commentary on youth and mini-basketball games from around the country, complete with live coverage by AI. Specifically, the emotion engine recognizes the emotions of users watching youth and mini-basketball games, and adjusts the content and tone of the commentary according to those emotions. For example, if a user is impressed by their child's performance, the commentary will be one that shares that emotion. This allows users to hear commentary that reflects their own emotions, further increasing their satisfaction with watching the games.

"Example 3"

Furthermore, emotion engines are also being incorporated into systems that use AI to automatically generate running scores that are manually created by officials and for each team. Specifically, the emotion engine recognizes the user's emotions regarding the score situation of the match, and adjusts the way the score is displayed according to those emotions. For example, if the user is happy about a score, the score display will also be displayed in a way that emphasizes that joy. This allows users to see a score display that reflects their own emotions, further increasing their satisfaction when watching the match.

The process flow for each example is explained below.

"Example 1"

Step 1: The AI begins commentating on a live broadcast of a basketball game.

Step 2: The emotion engine recognizes the user's emotions in real time.

Step 3: The AI adjusts the content and tone of the commentary based on the emotion recognized by the emotion engine.

"Example 2"

Step 1: AI will begin providing live commentary for youth and mini-basketball matches across the country.

Step 2: The emotion engine recognizes the emotions of users watching the game in real time.

Step 3: The AI adjusts the content and tone of the commentary based on the emotion recognized by the emotion engine.

"Example 3"

Step 1: AI automatically generates a running score based on the game's scoring situation.

Step 2: The emotion engine recognizes the user's emotions in real time regarding the score of the game.

Step 3: The AI adjusts the way the score is displayed depending on the emotion recognized by the emotion engine.

(Example 1)

Next, a first embodiment of the first embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

In conventional basketball game commentary systems, it was difficult to analyze the game situation and player movements in real time and provide commentary that corresponded to the user's emotions. In addition, there was a lack of means to efficiently distribute commentary of youth and mini-basketball games across the country, or to automatically generate running scores that are created manually by officials and for each team. Furthermore, there was insufficient use of emotion recognition technology to improve the user's viewing experience.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for providing commentary on live coverage of basketball games using artificial intelligence, means for delivering commentary on youth and mini-basketball games across the country with live coverage by artificial intelligence, means for automatically generating running scores using artificial intelligence that are created manually for each official and team, means for analyzing game video data, means for analyzing game audio data, means for receiving and analyzing user emotion data, means for automatically generating commentary based on the analysis results and adjusting the content and tone of the commentary according to the user's emotions, and means for delivering the adjusted commentary together with the live coverage. This makes it possible to analyze the situation of the game and the movements of the players in real time and provide commentary according to the user's emotions.

"Means for artificial intelligence to provide commentary for live broadcasts of basketball games" refers to technology that analyzes video and audio data of basketball games, and automatically generates commentary based on the results of that analysis using artificial intelligence, which is then distributed along with the live broadcast.

"A means of broadcasting commentary on youth and mini-basketball games nationwide with live coverage by artificial intelligence" refers to a technology in which artificial intelligence automatically generates commentary for youth and mini-basketball games nationwide and broadcasts that commentary together with live coverage.

"Method of automatically generating running scores using artificial intelligence that are created manually for each official or team" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Means for analyzing game video data" refers to technology that analyzes game video data frame by frame to identify the positions and movements of players, the position of the ball, etc.

"Means for analyzing audio data from a match" refers to technology that analyzes audio data from a match and identifies the content of the commentary and the reactions of the audience.

"Means for receiving and analyzing user emotion data" refers to technology that captures the user's facial expressions and voice, and analyzes that data to identify the user's emotions.

"Means for automatically generating commentary based on analysis results and adjusting the content and tone of the commentary according to the user's emotions" refers to a technology that automatically generates commentary based on the analysis results of video and audio data, and further adjusts the content and tone of the commentary taking into account the user's emotional data.

"Means for delivering adjusted commentary together with live broadcast" refers to technology that delivers adjusted commentary in real time to a user's device together with the live broadcast.

Form for implementing the invention

This invention is a system in which artificial intelligence provides commentary for live coverage of basketball games, analyzing video and audio data from the game and providing commentary that reflects the user's emotions. A specific embodiment of this system is shown below.

Hardware and software used

Hardware: High-performance servers, user devices (PCs, smartphones, tablets, etc.)

Software: Video analysis software (e.g. OpenCV), audio analysis software (e.g. Google Cloud Speech-to-Text API), emotion engines (e.g. Microsoft® Azure® Emotion API), generative AI models (e.g. GPT-4)

System configuration

1. The server receives the game video and audio data in real time. It uses a high-speed network connection to receive the streaming data sent from the game venue.

2. The server uses video analysis software to analyze the received video data frame by frame. Specifically, computer vision techniques are used to identify the positions of players, their movements, and the position of the ball. For example, OpenCV is used.

3. The server uses voice analysis software to analyze the received voice data. Specifically, it uses voice recognition technology to identify the content of the commentary and the reactions of the audience. For example, it uses the Google Cloud Speech-to-Text API.

4. The server uses a generative AI model to automatically generate commentary on the match based on the results of video and audio analysis. Specifically, it uses a natural language generation model such as GPT-4 to generate sentences that explain the situation of the match and the movements of the players.

5. The device captures the user's facial expressions and voice in real time and transmits them as emotion data to the server. Specifically, the device uses a camera and microphone.

6. The server uses an emotion engine to analyze the received emotion data. Specifically, it uses emotion recognition technology to identify emotions such as happiness, sadness, surprise, etc. from the user's facial expressions and voice. For example, it uses the Microsoft Azure Emotion API.

7. The server adjusts the content and tone of the generated commentary according to the user's emotions analyzed by the server. Specifically, if the user is happy, the tone of the commentary is brightened and more positive expressions are included.

8. The server delivers the adjusted commentary along with the live broadcast to the user's device. Specifically, it delivers in real time using streaming technology. For example, WebRTC is used.

Specific examples

For example, if a user expresses joy while watching a game, the emotion engine will recognize that joy. Based on that information, the server will lighten the tone of the commentary generated by the generative AI model and adjust it to share the joy. This allows users to hear commentary that reflects their own emotions, improving their satisfaction watching the game.

Example of a prompt

"Design an AI system to provide commentary for live broadcasts of basketball games. It must analyze video and audio data from the game, understand the game situation and the movements of the players, and incorporate an emotion engine that adjusts the content and tone of the commentary according to the user's emotions."

The flow of the identification process in Example 1 is explained using Figure 17.

Step 1:

The server receives the video and audio data of the match in real time. The input is streaming data sent from the match venue, and the output is the video and audio data stored in the server. Specifically, the server receives the data using a high-speed network connection and stores it in an appropriate format.

Step 2:

The server uses video analysis software to analyze the received video data frame by frame. The input is the saved video data, and the output is the analysis results, such as the positions and movements of the players and the position of the ball. In concrete terms, it uses OpenCV to analyze each frame and identify the positions of the players and the ball.

Step 3:

The server uses voice analysis software to analyze the received voice data. The input is the stored voice data, and the output is the analysis results in the form of text that contains the commentary and the reactions of the audience. Specifically, the Google Cloud Speech-to-Text API is used to convert the voice data into text and extract key information.

Step 4:

The server uses a generative AI model to automatically generate commentary on the match based on the results of video and audio analysis. The input is the results of video and audio analysis, and the output is the generated commentary. Specifically, it uses a natural language generation model such as GPT-4 to generate sentences that explain the situation of the match and the movements of the players.

Step 5:

The device captures the user's facial expressions and voice in real time and sends them to the server as emotion data. The input is the user's facial expressions and voice, and the output is the emotion data sent to the server. In concrete terms, the device uses a camera and microphone to capture the user's facial expressions and voice, and sends them to the server in real time.

Step 6:

The server uses an emotion engine to analyze the received emotion data. The input is the emotion data sent, and the output is the analysis results that identify the user's emotion. Specifically, it uses the Microsoft Azure Emotion API to identify emotions such as joy, sadness, and surprise from the user's facial expressions and voice.

Step 7:

The content and tone of the generated commentary is adjusted according to the user's emotions analyzed by the server. The input is the result of emotion analysis and the generated commentary, and the output is the adjusted commentary. Specifically, if the user shows joy, the tone of the commentary is brightened and more positive expressions are used.

Step 8:

The server delivers the adjusted commentary to the user's device along with the live broadcast. The input is the adjusted commentary and the live broadcast data, and the output is the live broadcast with real-time commentary delivered to the user's device. In concrete terms, it uses WebRTC to deliver the adjusted commentary in real time.

(Application example 1)

Next, application example 1 of embodiment example 1 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

Conventional basketball game commentary systems had the problem that the commentary was uniform and could not flexibly respond to the emotions of the viewers. In addition, improving the quality of the live broadcast and commentary of the game required a large number of people, which was costly and laborious. Furthermore, there was a lack of efficient means to provide commentary on youth and mini-basketball games across the country.

The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games nationwide with AI live coverage, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions, a means for generating commentary using a generation AI model, and a means for generating commentary by inputting a prompt sentence. This enables flexible commentary according to the viewer's emotions, improving the quality of live coverage and commentary of games. In addition, commentary on youth and mini-basketball games nationwide can be efficiently performed.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to a means that uses artificial intelligence technology to analyze video and audio data of basketball games, understand the situation of the game and the movements of the players, and automatically generate commentary.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a means of broadcasting live coverage including commentary on youth and mini-basketball games from around the country with AI providing commentary.

"Means of using AI to automatically generate running scores that are created manually by officials and teams" refers to a means of using AI to automatically generate scores that are manually created by officials and teams during a match.

"Means for adjusting the content and tone of commentary using an emotion engine that recognizes the user's emotions" refers to means that use emotion recognition technology to recognize the user's emotions (happiness, sadness, surprise, etc.) and adjust the content and tone of the commentary in accordance with those emotions.

"Means for generating commentary using a generative AI model" refers to means for generating commentary based on the situation of a match using a generative AI model (such as GPT-3 (registered trademark)).

"Means for inputting a prompt sentence and generating an explanation" refers to means for inputting a specific prompt sentence into a generative AI model and generating an explanation based on that prompt.

A system for implementing this invention includes a means for AI to provide commentary for live coverage of basketball games, a means for distributing commentary for youth and mini-basketball games across the country with AI commentary, a means for AI to automatically generate running scores that are created manually for each official or team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions, a means for generating commentary using a generative AI model, and a means for generating commentary by inputting a prompt text.

Hardware and software used

Hardware: Smartphone camera and microphone

Software: OpenCV, Keras, Transformers

Data processing and data calculations

Server

The server receives and analyzes video and audio data from the basketball game in real time. It uses OpenCV to acquire the video data and Keras to run an emotion recognition model. The emotion recognition model analyzes the user's facial expressions and recognizes emotions such as happiness, sadness, and surprise.

Terminal

The device generates commentary using a generative AI model (e.g., GPT-3). A prompt sentence is input into the generative AI model, which generates commentary based on the prompt. Examples of prompt sentences include "The user is happy. Generate a joyful commentary for the basketball game." and "The user is sad. Generate a comforting commentary for the basketball game."

User

Users watch the game through their smartphone's camera and microphone, and their emotions are recognized. The emotion engine adjusts the content and tone of the commentary based on the user's emotions. For example, if the user expresses joy at a highlight scene in the game, the commentary might say something like, "Great play! We've been waiting for this moment!" Conversely, if the user expresses sadness at a comeback scene in the game, the commentary might say something like, "It's a disappointing result, but there's still a chance. Let's look forward to the next play."

In this way, flexible commentary based on the viewer's emotions becomes possible, improving the quality of live game coverage and commentary. It also enables efficient commentary of youth and mini-basketball games across the country.

The flow of the specific processing in application example 1 is explained using Figure 18.

Step 1:

The server receives video and audio data from the smartphone's camera and microphone in real time. The input is the video and audio data from the camera and microphone, and the output is the data ready for analysis.

Step 2:

The server processes the video data using OpenCV to detect the user's face. The input is the video data received in step 1, and the output is the detected face area. Specifically, the server converts the video data to grayscale and applies a face detection algorithm.

Step 3:

The server runs an emotion recognition model using Keras to recognize the user's emotions. The input is the face region detected in step 2, and the output is the recognized emotion (happiness, sadness, surprise, etc.). Specifically, the server resizes the face region to an appropriate size and inputs it into the emotion recognition model.

Step 4:

The server generates a prompt sentence to be input to the generative AI model based on the recognized emotion. The input is the emotion recognized in step 3, and the output is the generated prompt sentence. Specifically, it selects a template sentence according to the emotion and generates the prompt sentence.

Step 5:

The server generates an explanation using a generative AI model (e.g., GPT-3). The input is the prompt text generated in step 4, and the output is the generated explanation text. In concrete terms, the prompt text is input into the generative AI model, and an explanation text is generated.

Step 6:

The server delivers the generated explanatory text to the user in real time. The input is the explanatory text generated in step 5, and the output is the explanatory audio delivered to the user's smartphone. In concrete terms, the explanatory text is input into a voice synthesis engine, and audio data is generated and delivered.

Step 7:

Users watch the game and listen to the commentary through their smartphones. The input is the commentary audio delivered from the server, and the output is an improvement to the user's viewing experience. In concrete terms, the commentary audio is played through the smartphone's speakers.

(Example 2)

Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

Traditional basketball game commentary relied on the commentator's subjective judgment, making it difficult to provide commentary that reflected the emotions of viewers. In addition, there was a lack of systems to provide youth and mini-basketball games from across the country to a wide audience, making it difficult to grasp the progress of the game and the performance of the players in real time. Furthermore, manually creating a running score for the game was time-consuming and labor-intensive, so efficient management was required.

The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

In this invention, the server includes means for providing commentary on live coverage of basketball games using artificial intelligence, means for distributing commentary on youth and mini-basketball games from around the country with live coverage by artificial intelligence, means for automatically generating running scores using artificial intelligence that are created manually for each official and team, means for collecting game video and audio data, means for analyzing the collected data and generating commentary, means for recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions, and means for distributing the generated commentary together with the game video. This makes it possible to provide commentary that corresponds to the viewer's emotions and improve satisfaction with viewing the game.

"Means for artificial intelligence to provide commentary for live broadcasts of basketball games" refers to technology in which artificial intelligence analyzes the progress of a basketball game and the performance of players, and generates commentary in real time.

"A means of broadcasting commentary on youth and mini-basketball games across the country with live coverage by artificial intelligence" refers to a technology in which artificial intelligence analyzes youth and mini-basketball games held across the country, generates commentary, and broadcasts it in real time.

"Means for automatically generating running scores using artificial intelligence that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Means for collecting game video and audio data" refers to technology that uses cameras and microphones installed at the game venue to collect game video and audio data in real time.

"Means of analyzing collected data and generating commentary" refers to technology in which artificial intelligence analyzes collected video and audio data and generates commentary based on the progress of the match and the performance of the players.

"Means of recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to technology that analyzes the viewer's facial expressions and tone of voice, and adjusts the content and tone of the commentary according to those emotions.

"Means for distributing generated commentary together with game footage" refers to technology for distributing commentary generated by artificial intelligence together with game footage via a streaming platform.

This invention is a system in which artificial intelligence (AI) broadcasts live basketball games, and also distributes live commentary by AI for youth and mini-basketball games across the country. It also includes a function that allows the AI to automatically generate running scores that are currently created manually by officials and for each team.

The server collects video and audio data from the match in real time using cameras and microphones installed at the match venue. Specifically, it uses hardware such as Sony's 4K cameras and Shure's high-sensitivity microphones. The collected data is stored in a database on the server.

The server then inputs the stored video and audio data into an AI model. This AI model uses a generative AI model such as OpenAI's GPT-4 or Google's BERT. The AI model analyzes the input data and generates commentary based on the progress of the match and the performance of the players. For example, it analyzes the moment when Player A makes a shot and generates commentary such as, "Player A made a great shot!".

In addition, the device will recognize the emotions of the user watching the game. Specifically, it will use the device's camera and microphone to capture the user's facial expressions and tone of voice. For example, Apple's Face ID technology or Amazon's Alexa will be used for this purpose. The device will then send the captured data to a server.

The server receives the user's emotion data sent from the terminal and analyzes it using an emotion engine. The emotion engine uses, for example, Affectiva or IBM Watson (registered trademark) emotion analysis APIs. If the user is emotional, the server adjusts the tone of the commentary to be more emotional and generates a commentary such as, "Player A's shot was really amazing! The crowd is really excited!".

Finally, the server distributes the generated commentary along with the game footage. Streaming platforms such as YouTube and Twitch are used for distribution. Users can watch the game through these platforms and enjoy the commentary according to their emotions.

As a concrete example, let's say a youth basketball game is taking place. The server collects video and audio data in real time from Sony's 4K cameras and Shure's high-sensitivity microphones installed at the game venue. The collected data is then input into OpenAI's GPT-4, which analyzes the moment Player A makes a shot. The AI model generates a commentary such as, "Player A made a great shot!".

Meanwhile, the device uses Apple's Face ID technology to capture the facial expressions of the user watching the game and recognizes that they are emotionally moved. The server then uses Affectiva's emotion analysis API to analyze the user's emotions and adjust the tone of the commentary to be more emotional. It generates a commentary such as, "Player A's shot was really amazing! The crowd is really excited!" and distributes it via YouTube.

Examples of prompts might include:

"Analyze video and audio data from youth basketball games to generate commentary on game progress and player performance. Also recognize viewer emotions and adjust the content and tone of the commentary accordingly."

In this way, the server can provide users with commentary that reflects their emotions, improving their satisfaction with watching the game.

The flow of the identification process in Example 2 is explained using Figure 19.

Step 1:

The server uses cameras and microphones installed at the venue to collect video and audio data of the match in real time. Specifically, it uses Sony 4K cameras and Shure high-sensitivity microphones. The input is the video and audio data of the match, and the output is the collected video and audio data. This data is stored in a database on the server.

Step 2:

The server inputs the stored video and audio data into an AI model. This AI model uses, for example, OpenAI's GPT-4. The input is the collected video and audio data, and the output is analyzed data on the progress of the match and the performance of the players. The AI model analyzes the movements of the players and the progress of the match from the video data, and analyzes the commentary and audience reactions from the audio data.

Step 3:

The server generates commentary on the match based on the data analyzed by the AI model. The input is the analyzed data on the progress of the match and the performance of the players, and the output is the generated commentary. For example, the server analyzes the moment when Player A makes a shot and generates a commentary such as, "Player A made a great shot!".

Step 4:

The device recognizes the emotions of users watching a game. Specifically, it uses the device's camera and microphone to capture the user's facial expressions and tone of voice. The input is data about the user's facial expressions and tone of voice, and the output is the recognized user's emotion data. This uses Apple's Face ID technology and Amazon's Alexa.

Step 5:

The server receives the user's emotion data sent from the terminal and analyzes it using an emotion engine. The input is the user's emotion data, and the output is the analyzed emotion information. The emotion engine uses, for example, Affectiva's emotion analysis API. If the user is emotional, the server adjusts the tone of the commentary to be emotional.

Step 6:

The server distributes the generated commentary together with the game footage. The input is the generated commentary and the game footage, and the output is the streaming data to be distributed. Streaming platforms such as YouTube and Twitch are used for distribution. Users can watch the game through these platforms and enjoy commentary according to their emotions.

(Application example 2)

Next, application example 2 of embodiment example 2 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

Conventional live basketball game broadcasting systems had the problem that commentary was uniform and did not allow for flexible responses to viewer emotions. In addition, there was a lack of game highlights, replay functions, and chat functions between viewers, so there was a need to improve the viewing experience. Furthermore, there were limited ways to widely provide viewers with information about youth and mini-basketball games across the country, resulting in information disparities between regions.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 2 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary of youth and mini-basketball games from around the country with AI commentary, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for recognizing the emotions of viewers and adjusting the content and tone of the commentary according to those emotions, a means for providing highlights and replay functions for games, and a means for providing a chat function between viewers. This allows for flexible commentary according to the emotions of viewers, improving the viewing experience and making it possible to widely provide youth and mini-basketball games from around the country to viewers.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to generate commentary in real time for basketball games and provide it to viewers.

"A means of broadcasting commentary on youth and mini-basketball games across the country with live coverage by AI" refers to technology that broadcasts youth and mini-basketball games held across the country in real time with commentary by artificial intelligence.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually recorded by officials and teams during a match.

"Means of recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to technology that recognizes emotions from the viewer's facial expressions, voice, etc., and changes the content and tone of the commentary according to those emotions.

"Means for providing match highlight and replay features" refers to technology that provides a highlight feature that extracts and plays important scenes from a match, and a replay feature that replays specific scenes.

"Means for providing a chat function between viewers" refers to technology that provides a chat function that enables users watching a match to exchange messages in real time.

The system for implementing this invention includes the following main components: a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games across the country with AI commentary, a means for AI to automatically generate running scores that are created manually for each official and team, a means for recognizing the emotions of viewers and adjusting the content and tone of the commentary according to those emotions, a means for providing game highlights and replay functions, and a means for providing a chat function between viewers.

Hardware and software used

Hardware: Smartphone (camera, microphone, display)

Software: OpenCV (image processing library), EmotionRecognizer (emotion recognition model), AICommentary (AI commentary generation model), LiveStreaming (live streaming library)

Data processing and data calculations

1. Frame acquisition: The server acquires frames from the live streaming, which allows video data of the match to be collected in real time.

2. Emotion Recognition: The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer. This obtains the viewer's emotion data.

3. Commentary generation: The server generates commentary using AICommentary based on the recognized emotions. This allows commentary to be generated in real time according to the viewer's emotions.

4. Add commentary: The server adds the generated commentary to the live stream, allowing viewers to see emotional commentary in real time.

5. Highlights and Replays: The server automatically extracts important scenes from the match and provides highlights and replays, allowing viewers to rewatch key moments of the match.

6. Chat function: The server provides a chat function that allows viewers to exchange messages with each other in real time. This promotes communication between viewers.

Specific examples

For example, during a youth basketball game, if the viewer is impressed, the server might generate a commentary like, "Great play! I'll never forget this moment!" Or, if the viewer is excited, the server might generate a commentary like, "You won't want to miss this! Watch the next play!"

Example of a prompt

"Create an AI model that recognizes the emotions of your viewers and adjusts the content and tone of your commentary accordingly. For example, if your viewer is emotional, make sure your commentary reflects that emotion."

The flow of the specific processing in application example 2 is explained using Figure 20.

Step 1:

The server obtains frames from live streaming. The input is real-time game video data, and the output is individual frame images. Specifically, the server receives the video data sent from the camera and splits it into frames using OpenCV.

Step 2:

The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer. The input is the frame image obtained in step 1, and the output is the viewer's emotion data. Specifically, the server detects the face in the frame and inputs it into the emotion recognition model to classify the emotion.

Step 3:

The server generates a commentary using AICommentary based on the recognized emotion. The input is the emotion data obtained in step 2, and the output is commentary text corresponding to the emotion. Specifically, the server inputs the emotion data as a prompt into the AI commentary generation model to generate an appropriate commentary.

Step 4:

The server adds the generated commentary to the live stream. The input is the commentary text generated in step 3, and the output is the live video with the commentary added. Specifically, the server synthesizes the commentary text into speech and overlays it on the video.

Step 5:

The server automatically extracts important scenes from a match and provides highlight and replay functions. The input is real-time match video data, and the output is highlight and replay footage. Specifically, the server detects events in the match, extracts important scenes, and saves them.

Step 6:

The server provides a chat function that allows viewers to exchange messages with each other in real time. The input is message data from the viewer, and the output is the message to be sent to other viewers. Specifically, the server receives messages and distributes them to other viewers in the chat room.

(Example 3)

Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

In conventional basketball games, creating a running score and understanding the progress of the game were done manually, making it difficult to update in real time. In addition, there was little progress in creating a database of teams and players nationwide, and feedback for analyzing player performance and training was not provided sufficiently. Furthermore, the score display did not reflect the user's feelings about the score situation in the game, resulting in low satisfaction when watching the game.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for receiving video data and audio data of the match, means for analyzing the received video data, means for analyzing the received audio data, means for extracting the score situation and foul situation from the analysis results, means for generating a running score based on the extracted information, means for updating and transmitting the generated running score in real time, means for displaying the transmitted running score, and means for recognizing the user's emotions and adjusting the score display. This makes it possible to grasp the progress of the match in real time and provide a score display that reflects the user's emotions.

"Game video data" refers to video data that records the progress of a basketball game.

"Audio data" refers to audio data that records commentary and commentary during a match, audience reactions, etc.

"Means of receiving" refers to the function for acquiring video and audio data of the match from an external source.

"Means of analysis" refers to the function of processing received video and audio data and extracting specific information.

"Scoring status" is information that shows how many points each team scored during the game.

"Foul situation" is information that shows which players committed what type of fouls during the game.

The "running score" is a scoreboard that shows the score and foul situation in real time as the game progresses.

"Means for updating and transmitting in real time" refers to a function for instantly updating the generated running score and transmitting it to the user's device.

"Display means" is a function for visually presenting the submitted running score to the user.

"Means of recognizing user emotions" refers to a function for analyzing and understanding the user's emotions regarding the score of a match, etc.

"Means for adjusting score display" is a function for changing the way the scoreboard is displayed depending on the recognized user's emotions.

"Game results and stats information for teams and players nationwide" refers to game results for basketball teams and players nationwide and statistical data for individual players.

"Means of saving to a database" refers to the function for organizing collected match results and stats information and storing it in a database.

"Means of analyzing performance" refers to a function for evaluating a player's performance and abilities based on the information stored in the database.

"Means of generating feedback" refers to the function that creates reports based on the analysis results to suggest player development and areas for improvement.

"Means for sending and displaying" refers to the function for sending the generated feedback to the user's device and presenting it visually.

"Visualization" refers to displaying data visually to make it easier to understand.

"Means for utilizing in development" refers to methods for supporting the training and development of players based on data.

This invention is a system that automatically generates running scores for basketball games and analyzes player performance. A specific embodiment of this system is described below.

System configuration

Hardware and software

1. Server

Hardware: Server with high-performance processor and large memory capacity

Software: Video analysis software (e.g. OpenCV), audio analysis software (e.g. Google Cloud Speech-to-Text), database management system (e.g. MySQL), sentiment analysis engine (e.g. IBM Watson Tone Analyzer)

2. Terminal

Hardware: PCs, tablets, smartphones, etc.

Software: Web browser, dedicated application

Program processing

Receiving and analyzing match data

The server receives video and audio data of the match. The video data is obtained from cameras and streaming services, and the audio data includes commentary and narration. The received video data is analyzed using OpenCV to detect scoring and foul scenes. The audio data is converted to text using Google Cloud Speech-to-Text, and information about scores and fouls is extracted from the commentary.

Extraction of scoring and foul situations

The server combines the results of video and audio analysis to extract the score and foul situations. This makes it possible to grasp the progress of the game in real time.

Generate and display running scores

The server generates a running score based on the extracted information. The generated running score is updated in real time and sent to the terminal using WebSocket or HTTP. The terminal displays the received running score to the user, allowing the user to check the progress of the match in real time.

Recognizing user emotions and adjusting score display

The server uses IBM Watson Tone Analyzer to recognize the user's emotions regarding the score of the game. Depending on the recognized emotion, the way the score is displayed is adjusted. For example, if the user is happy about a score, the score display is emphasized with bright colors or animations. The adjusted score display is sent to the terminal and displayed to the user.

Database of teams and players nationwide and performance analysis

The server collects match results and stats for teams and players across the country, and stores them in a MySQL database. Based on the stored information, the performance of players is analyzed using Python's Pandas library. Based on the analysis results, feedback for player development is generated and sent to the terminal. The terminal displays the received feedback to the user.

Specific examples

When the video data of the match is sent to the server, the server uses OpenCV to detect scoring scenes and uses Google Cloud Speech-to-Text to extract foul information from the commentary audio. For example, the prompt text can be "Analyze the scoring scenes and foul information of this match to generate a running score."

The server collects the results of matches from around the country and stores them in a MySQL database. It uses Python's Pandas library to analyze players' performance and generate feedback. For example, enter the following prompt statement: "Collect the results of matches from around the country, analyze players' performance, and generate feedback."

The server recognizes the user's joy at the scoring scene using IBM Watson Tone Analyzer, and adjusts the score display to emphasize the joy. For example, the prompt text is input as "Analyze the user's emotions at the scoring scene and adjust the score display." The flow of the specific process in Example 3 is explained with reference to FIG. 21.

Step 1:

Receive match data

The server receives video and audio data of the match. As input, it obtains video and audio data provided in real time from cameras and streaming services. As output, it stores the received video and audio data in its internal memory. Specifically, the server obtains the data through a network connection and stores it in a specified format.

Step 2:

Video data analysis

The server analyzes the received video data. As input, it uses the video data saved in step 1. As output, it obtains the results of detecting scoring and foul scenes. In concrete terms, the server uses OpenCV to analyze the video data frame by frame and executes an algorithm to detect specific events (scoring and fouls).

Step 3:

Analysis of audio data

The server analyzes the received audio data. As input, it uses the audio data stored in step 1. As output, it obtains text information extracted from the audio data. Specifically, the server converts the audio data into text using Google Cloud Speech-to-Text and extracts information about scores and fouls from the commentary.

Step 4:

Extraction of scoring and foul situations

The server integrates the video analysis results and audio analysis results to extract the scoring and foul situations. As input, it uses the outputs of steps 2 and 3. As output, it obtains data including the scoring and foul situations. In concrete terms, the server integrates the video analysis results and audio analysis results in a chronological order to identify the timing of scoring and fouls.

Step 5:

Generate running scores

The server generates a running score based on the extracted information. As input, it uses the score and foul status obtained in step 4. As output, it obtains data including the running score. Specifically, the server organizes the score and foul information in chronological order and compiles it in a scoreboard format.

Step 6:

Update and send running scores

The server updates the generated running score in real time and sends it to the terminal. As input, it uses the running score generated in step 5. As output, it sends the updated running score to the terminal. Specifically, the server sends data using WebSocket or HTTP.

Step 7:

Display running score

The terminal displays the received running score to the user. As input, it uses the running score sent in step 6. As output, it obtains a scoreboard that is visually presented to the user. In concrete terms, the terminal displays the scoreboard using a web browser or a dedicated application.

Step 8:

Recognizing user emotions

The server recognizes the user's emotions regarding the score of the game. As input, it uses the user's comments and reaction data. As output, it obtains the recognized emotion data. Specifically, the server analyzes the user's emotions using IBM Watson Tone Analyzer.

Step 9:

Score display adjustments

The server adjusts the way the score is displayed depending on the recognized emotion. As input, it uses the emotion data obtained in step 8 and the running score generated in step 5. As output, it obtains the adjusted score display data. As a specific operation, the server changes the color and animation of the score display.

Step 10:

Send adjusted score display

The server sends the adjusted score display to the terminal. As input, it uses the score display data adjusted in step 9. As output, it sends the adjusted score display to the terminal. Specifically, the server sends the data using WebSocket or HTTP.

Step 11:

Displaying the adjusted score

The terminal displays the adjusted score display to the user. As input, it uses the adjusted score display data sent in step 10. As output, it obtains the adjusted scoreboard that is visually presented to the user. In specific operation, the terminal displays the adjusted scoreboard using a web browser or a dedicated application.

Step 12:

Database of teams and players nationwide

The server collects match results and stats information for teams and players across the country and stores it in a database. As input, it uses data obtained from each team's official website and sports data services. As output, it obtains match results and stats information stored in the database. Specifically, the server organizes the data and stores it in a MySQL database.

Step 13:

Performance analysis

The server analyzes the players' performance based on the information stored in the database. As input, it uses the database information stored in step 12. As output, it obtains the analysis results that evaluate the players' performance and abilities. Specifically, the server processes the data using Python's Pandas library and evaluates the players' performance.

Step 14:

Generating feedback

The server generates feedback for player development based on the analysis results. As input, it uses the analysis results obtained in step 13. As output, it obtains a feedback report that points out the player's strengths and weaknesses and suggests areas for improvement. In concrete terms, the server creates the report and provides it to the player and coach.

Step 15:

Submit and view feedback

The server sends the generated feedback to the terminal, which displays it to the user. As input, it uses the feedback report generated in step 14. As output, it obtains a feedback report that is visually presented to the user. In concrete terms, the server sends the data via email or a dedicated application, and the terminal displays it.

(Application example 3)

Next, application example 3 of embodiment example 3 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

Traditionally, live broadcasts and commentary of basketball games relied on human commentators, making it difficult to grasp the score and foul situation in real time. In addition, analysis of player performance and feedback on development were also done manually, which was inefficient. Furthermore, it was not possible to display scores that reflected the user's emotions, making it difficult to improve the viewing experience.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 3 is realized by the following means.

In this invention, the server includes: a means for AI to provide commentary on live coverage of basketball games; a means for distributing commentary on youth and mini-basketball games nationwide with AI live coverage; a means for AI to automatically generate running scores that are created manually for each official and team; a means for analyzing game video and audio data in real time and automatically generating the score situation and foul situation; a means for recognizing the user's emotions and displaying the score according to the emotions; and a means for providing feedback for player performance analysis and development using a database of teams and players nationwide. This makes it possible to grasp the progress of the game in real time and display scores that reflect the user's emotions, improving the efficiency of player performance analysis and development.

"A means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to provide real-time commentary on the progress and play of a basketball game.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games from around the country, and broadcasts live coverage including that commentary.

"A means of using AI to automatically generate running scores that are created manually by officials and for each team" refers to technology that uses artificial intelligence to automatically generate scores such as points and fouls as the game progresses.

"Means for analyzing game video and audio data in real time and automatically generating scoring and foul status" refers to technology that analyzes game video and audio data in real time and automatically generates scoring and foul status.

"Means for recognizing the user's emotions and displaying a score according to those emotions" refers to technology that recognizes emotions from the user's facial expressions and voice, and adjusts the score display according to those emotions.

"Means of utilizing a database of teams and players nationwide to analyze player performance and provide feedback for development" refers to technology that utilizes a database of basketball teams and players nationwide to analyze player performance and provide feedback for development.

The system for implementing this invention uses the following hardware and software.

Hardware and software used

Hardware: Smartphone, head-mounted display

Software: TensorFlow (AI model), OpenCV (video analysis), NLTK (natural language processing), Emotion API (emotion recognition)

System configuration

The server includes the following means:

1. How AI can provide commentary for live broadcasts of basketball games:

The server receives video and audio data from the match in real time, and uses TensorFlow to analyze the progress of the match and the content of the play, generating commentary.

2. How to provide live commentary of youth and mini-basketball games nationwide with AI:

The server receives footage of youth and mini-basketball matches from across the country, and similarly uses AI to generate commentary, which it distributes in real time.

3. A way to use AI to automatically generate running scores that are currently created by officials and each team:

The server analyzes the video and audio data of the match and automatically generates the score and foul situations.

4. A method for analyzing game video and audio data in real time and automatically generating scoring and foul situations:

The server uses OpenCV to split the video data into frames and extract important scenes. The audio data is converted to text using NLTK and important keywords are extracted.

5. How to recognize user emotions and display a score according to those emotions:

The server uses the Emotion API to recognize emotions from the user's facial expressions and voice, and adjusts the score display accordingly.

6. Using a national team and player database to analyze player performance and provide feedback for development:

The server accesses a database of teams and players from across the country, displays analysis of player performance, and provides feedback for the next match.

Specific examples

When a user using a smartphone during a match sees a goal being scored, the server uses AI to analyze the scene and display the score in real time. If the user shows an expression of joy, the server will recognize it using an emotion engine and make the score display more colorful. After the match ends, the server also refers to a nationwide database to display an analysis of the players' performance and provide feedback for the next match.

Example of a prompt

"Please analyze the video and audio data of the game in real time, and use an AI model that automatically generates the score and foul situation to display a running score. Also, please recognize the user's emotions and display the score according to those emotions. Furthermore, please use a database of teams and players from all over the country to provide feedback for analyzing player performance and development."

The flow of the specific processing in application example 3 is explained using Figure 22.

Step 1:

Acquisition of game video and audio data

The server receives video and audio data of the game in real time from a smartphone or head-mounted display. The input is the video and audio data of the game, and the output is preprocessed data for analysis. In concrete terms, the server receives the data stream, splits the video data into frames, and converts the audio data into text.

Step 2:

Data preprocessing

The server uses OpenCV to split the video data into frames and extract important scenes. It also uses NLTK to convert the audio data into text and extracts important keywords (e.g., goals, fouls). The input is the preprocessed video and audio data, and the output is important scenes and keywords for analysis. Specifically, the server analyzes video frames and identifies scoring and foul scenes.

Step 3:

Analysis using AI models

The server uses TensorFlow to analyze the scoring and foul situations from video and audio data. The input is important scenes and keywords, and the output is data on the scoring and foul situations. In concrete terms, the server runs an AI model and automatically determines the scores and fouls for each scene.

Step 4:

Automatic generation of running scores

The server automatically generates a running score based on the analysis results. The input is data on the scoring and foul situations, and the output is a real-time running score. Specifically, the server tallys up the score and foul information and updates the scoreboard.

Step 5:

Emotion recognition

The server uses the Emotion API to recognize emotions from the user's facial expressions and voice. The input is the user's facial expression data and voice data, and the output is the recognized emotion data. In concrete terms, the server obtains user data from the camera and microphone and analyzes emotions.

Step 6:

Score display based on emotions

The server adjusts the score display according to the recognized emotion. The input is emotion data and the running score, and the output is a score display according to the emotion. In concrete terms, the server changes the design and effects of the score display based on the user's emotion.

Step 7:

Utilizing the database

The server refers to a database of teams and players from around the country, displays the results of player performance analysis, and provides feedback for the next match. The input is the match results and player data, and the output is performance analysis results and feedback. In concrete terms, the server retrieves relevant information from the database and displays the analysis results.

The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input for the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (registered trademark) (Internet search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (registered trademark) (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart device 14.

[Second embodiment]

Figure 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

As shown in FIG. 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical able-bodied person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is performed in a secure state.

Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. Data generation model 58 and emotion identification model 59 are used by the identification processing unit 290.

In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the specific processing performed by the specific processing unit 290 of the data processing device 12.

"Example 1"

Example 1 of the present invention is a system in which AI provides commentary for a live broadcast of a basketball game. In this system, AI analyzes the video and audio data of the game to understand the situation of the game and the movements of the players. Then, based on the results of this analysis, a commentary of the game is automatically generated and distributed along with the live broadcast.

"Example 2"

Example 2 of the present invention is a system that distributes commentary of youth and mini-basketball games from around the country, with live coverage by AI. This system collects video and audio data from youth and mini-basketball games held around the country, which are analyzed by AI in the same way as example 1 to generate commentary. Then, by distributing the commentary together with the game footage, games from around the country are made available to a wide audience.

"Example 3"

Example 3 of the present invention is a system that uses AI to automatically generate running scores that are created manually by officials or for each team. In this system, AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game, and automatically generates a running score. This makes it possible to grasp the progress of the game in real time.

"Example 4"

Example 4 of the present invention is a system that creates a database of teams and players from across the country, and uses the information in the database to visualize and develop registered basketball players in Japan. This system collects information such as game results and stats for teams and players from across the country, and creates a database. The database is then used to provide feedback for analyzing players' performance and developing them.

The process flow for each example is explained below.

"Example 1"

Step 1: AI collects video and audio data from basketball games.

Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

Step 3: Based on the analysis results, the AI automatically generates commentary for the match.

Step 4: Distribute the generated commentary along with the live broadcast.

"Example 2"

Step 1: Collect video and audio data from youth and mini-basketball matches held across the country.

Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

Step 3: Based on the analysis results, the AI automatically generates commentary for the match.

Step 4: Distribute the generated commentary along with the game footage.

"Example 3"

Step 1: AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game.

Step 2: Based on the analysis results, the AI automatically generates a running score.

"Example 4"

Step 1: Collect information such as match results and stats for teams and players across the country.

Step 2: Create a database of the collected information.

Step 3: Use the databased information to analyze player performance and provide feedback for development.

(Example 1)

Next, a first embodiment of the first embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

Traditional live broadcasts of basketball games depend on the quality and quantity of commentators, making it difficult to accurately convey the detailed situation of the game and the movements of the players in real time. In addition, commentary on youth and mini-basketball games across the country is required by many people, which is costly and labor intensive. Furthermore, running scores created by officials and people for each team are done manually, so they take time and can lack accuracy. To solve these issues, an automated system that utilizes AI is needed.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for collecting video data and audio data of the match in real time, means for analyzing the collected video data and audio data, means for generating commentary on the match based on the analysis results, and means for distributing the generated commentary together with a live broadcast. This makes it possible to accurately convey the detailed situation of the match and the movements of the players in real time. In addition, commentary on youth and mini-basketball matches from all over the country can be distributed with live broadcasts by AI, reducing costs and labor. Furthermore, by automatically generating running scores by AI instead of manually created by officials and for each team, it is possible to reduce time and improve accuracy.

"Game video data" refers to digital data that visually records the progress of a basketball game.

"Audio data" refers to data that is a digital recording of the sounds generated during a basketball game.

"Means of collecting data in real time" refers to technology and devices that allow for the acquisition of video and audio data as the match progresses.

"Means of analysis" refers to technology and software that processes collected video and audio data to understand the situation of the game and the movements of the players.

"Means for generating commentary" refers to technology or software that automatically creates text that explains the situation of the game and the movements of players based on the analysis results.

"Means of distribution along with live broadcast" refers to the technology and devices that deliver the generated commentary to viewers in real time along with the video and audio of the match.

"Youth and mini-basketball game commentary" is commentary on basketball games aimed at young people and elementary school students.

"Running score" is data that records the scores and player performances that are updated in real time during the game.

"AI automatic generation means" refers to technology and software that uses artificial intelligence to automate manual tasks and generate data.

"Means of database creation" refers to technology and software for organizing collected information and storing it in a format that is easy to search and use.

"Visualization" refers to techniques and methods for visually displaying data and making it easier to understand.

"Means to be used in development" refers to technologies and methods that use data to support the training and development of athletes.

This invention is a system that uses AI to automatically generate commentary on a basketball game and deliver it in real time during live broadcasts. A specific embodiment of this system is described below.

Server operation

1. Data collection

The server collects video and audio data of the match in real time. Specifically, it uses a camera and microphone to capture video and audio of the match. Video data is collected at a resolution of 1080p, and audio data is collected at a sampling rate of 44.1kHz.

2. Data analysis

The server analyzes the collected video and audio data. This analysis uses image recognition software (e.g. OpenCV) and voice recognition software (e.g. Google Speech-to-Text API). This allows the system to understand the situation of the game and the movements of the players. For example, it detects the action of Player A taking a shot and converts the live audio of that moment into text.

3. Description generation

The server generates commentary on the game based on the analysis results. A generative AI model (e.g. GPT-4) is used to generate this commentary. The generative AI model receives information about the game situation and player movements as input, and generates commentary in natural language. For example, it inputs information such as "Player A made a successful three-point shot" and generates commentary such as "Player A made a brilliant three-point shot."

4. Distribution

The server distributes the generated commentary along with the live broadcast. A streaming server (e.g. Wowza Streaming Engine) is used for distribution. This allows users to watch the game footage and commentary in real time over the Internet.

Device operation

1. Data reception

The device receives real-time game footage and commentary from the server using an internet connection and streaming software (e.g. VLC media player).

2. Display

The device displays the received video and commentary to the user. For display purposes, hardware such as a display and speakers are used. For example, if the user is watching on a smartphone, the video will be displayed on the smartphone screen and the commentary audio will be played through the built-in speaker.

User actions

1. Viewing

The user watches the game footage and commentary through the terminal. The user can get detailed commentary on the progress of the game and the movements of the players in real time. For example, if the user is watching on a television in the living room, the game footage will be displayed on the television screen and commentary will be played from the television speakers.

Specific examples

Example 1

Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

Player Action: Home team player A makes a three-point shot.

Example prompts for generative AI models

Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

Player Action: Home team player A makes a three-point shot.

Example of generated description

"With five minutes left in the third quarter, the home team is tied at 75 points and the away team at 70 points. Home team player A makes a great three-point shot, widening the gap even further."

In this way, users can watch detailed commentary on the progress of the match and the players' movements in real time.

The flow of the identification process in Example 1 is explained using Figure 11.

Step 1:

The server collects video and audio data from the match in real time.

Specific tasks include using cameras and microphones to capture video and audio of the game.

The input is video and audio from the camera and microphone, and the output is video and audio data in digital format.

Video data is collected at 1080p resolution, and audio data is collected at a sampling rate of 44.1kHz.

Step 2:

The server analyzes the collected video and audio data.

Specific operations include using image recognition software (e.g. OpenCV) to detect player movements from video data, and using voice recognition software (e.g. Google Speech-to-Text API) to convert the audio data into text to provide commentary.

The input is the video and audio data collected in step 1, and the output is the analyzed player movement information and the text commentary.

For example, it detects when Player A is about to take a shot and converts the audio commentary at that moment into text.

Step 3:

The server generates commentary for the match based on the analysis results.

Specifically, the prompt text is input into a generative AI model (e.g. GPT-4) to generate an explanatory text.

The input is the player movement information obtained in step 2 and the text commentary, and the output is the generated commentary.

For example, input information such as "Player A made a successful three-point shot" and generate an explanatory text such as "Player A made a brilliant three-point shot."

Step 4:

The server distributes the generated commentary along with the live broadcast.

Specifically, the video and commentary audio are streamed in real time using a streaming server (e.g. Wowza Streaming Engine).

The input is the explanatory text generated in step 3 and the video data collected in step 1, and the output is the video and explanatory audio delivered in real time.

Step 5:

The device receives game footage and commentary streamed from the server in real time.

Specific actions include using an internet connection and streaming software (e.g. VLC media player).

The input is streaming data from the server, and the output is the video displayed on the terminal and the commentary audio played.

Step 6:

The device displays the received video and commentary to the user.

Specific operations include showing images on the display and playing explanatory audio from the speaker.

The input is the video and commentary audio received in step 5, and the output is a live broadcast of the game provided to the user in both visual and audio.

Step 7:

Users can watch game footage and commentary through their devices.

Specific actions include watching the game footage on the device display and listening to commentary coming from the speaker.

The input is the video displayed in step 6 and the commentary audio played, and the output is the user's viewing experience.

For example, if a user is watching on a television in their living room, the game footage will be displayed on the television screen and commentary will be played from the television speakers.

(Application example 1)

Next, application example 1 of embodiment example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Existing basketball game commentary systems do not adequately provide real-time commentary, post-game highlight generation, or detailed analysis of players' performance. In addition, there is a problem that it is difficult to provide appropriate commentary immediately when a user asks about a specific player or play.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 1 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games nationwide with AI commentary, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for analyzing game video data and generating commentary in real time, a means for automatically extracting important scenes after the game ends and generating a highlight video, a means for analyzing player movements and performance and providing detailed statistical information, and a means for AI to provide commentary when a user asks a question about a specific player or play. This makes it possible to provide commentary in real time, generate post-game highlights, analyze players' performance in detail, and provide immediate commentary in response to user questions.

"A means for AI to provide commentary for live broadcasts of basketball games" refers to technology in which AI analyzes video and audio data from basketball games, understands the situation of the game and the movements of the players, and automatically generates commentary.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that uses AI to provide commentary on youth and mini-basketball games from around the country and broadcasts the games in real time with that commentary.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses AI to automatically generate scores that are manually recorded by officials and teams during a match.

"Means for analyzing game video data and generating commentary in real time" refers to technology that analyzes game video data in real time and instantly generates commentary based on the analysis results.

"Means for automatically extracting important scenes after the end of a match and generating a highlight video" refers to a technology that analyzes video data of a match after the end of the match, automatically extracts important scenes, and generates a highlight video.

"Means of analyzing player movements and performance and providing detailed statistical information" refers to technology that analyzes player movements and performance and provides detailed statistical information based on the results.

"A means for an AI to provide commentary when a user asks a question about a specific player or play" refers to technology that enables an AI to provide an appropriate commentary in response to a question when a user asks about a specific player or play.

To implement this invention, the following system configuration and processing steps are required.

System configuration

Hardware

Server: A high-performance computer for running AI models and analyzing data.

Smartphone: The device on which users watch and interact with the game commentary.

Camera: A device used to capture footage of the match.

Software

OpenCV: A library used to read and analyze video data.

TensorFlow: A framework used to run AI models.

Transformers: A library that generates explanations using the GPT-2 model.

Processing flow

1. Loading video data

The server uses OpenCV to read the game video data obtained from the camera frame by frame.

2. Frame analysis

The server analyzes the loaded frames to detect player movements and important actions. This analysis uses a deep learning model using TensorFlow.

3. Generate explanations

The server generates an explanatory text based on the analysis results. Specifically, it runs the GPT-2 model using the Transformers library and generates an explanatory text using the prompt text as input.

4. Delivering commentary

The generated commentary is delivered to smartphones in real time, allowing users to watch and listen to commentary on the match via their smartphones.

5. Highlight video generation

After the match ends, the server automatically extracts important scenes and generates a highlight video. This highlight video is also distributed to smartphones.

6. Player performance analysis

The server performs detailed analysis of the players' movements and performance and generates statistical information that users can view on their smartphones.

7. User interaction

When a user asks a question about a particular player or play, the server generates an appropriate commentary for that question and delivers it to the smartphone.

Specific examples

Scoring scene in the game: Detect the moment when Player A makes a three-point shot and generate a commentary such as "Player A scores a three-pointer."

Defensive highlights: Detects a scene where Player B successfully blocks a shot and generates a commentary such as "Player B makes a great block."

Example of a prompt

"Player A scores a three-pointer. Generate a commentary for this action."

"Player B makes a great block. Generate a commentary for this action."

In this way, the server can analyze the game video data in real time and automatically provide commentary to the user. This allows for real-time commentary, post-game highlights, detailed analysis of player performance, and instant commentary in response to user questions.

The flow of the specific processing in application example 1 is explained using Figure 12.

Step 1:

The server uses OpenCV to read the game video data captured by the camera frame by frame. The input is the video data from the camera, and the output is the individual frame images. Specifically, the server splits the video data into a series of still images and stores each frame in memory.

Step 2:

The server analyzes the loaded frames to detect player movements and important actions. A deep learning model using TensorFlow is used for this analysis. The input is the frame image obtained in step 1, and the output is information about the player movements and actions. Specifically, the server detects the positions and movements of players within the frames and identifies important actions such as points and blocks.

Step 3:

The server generates a commentary based on the analysis results. Specifically, it runs the GPT-2 model using the Transformers library and generates a commentary using the prompt as input. The input is the action information and prompt obtained in step 2, and the output is the generated commentary. The server generates a prompt corresponding to the player's action and inputs it into the GPT-2 model to obtain the commentary.

Step 4:

The server delivers the generated commentary to the smartphone in real time. The input is the commentary generated in step 3, and the output is the commentary displayed on the smartphone. Specifically, the server transmits the commentary to the smartphone via the network so that the user can view it.

Step 5:

After the match ends, the server automatically extracts important scenes and generates a highlight video. The input is the video data of the entire match, and the output is the highlight video. Specifically, the server selects highlight scenes based on important actions detected during the match and links them together to create a highlight video.

Step 6:

The server performs detailed analysis of players' movements and performance and generates statistical information. The input is frame images from the game and information about players' actions, and the output is detailed statistical information. Specifically, the server compiles data such as points, rebounds, and assists for each player and compiles it as statistical information.

Step 7:

When a user asks a question about a specific player or play, the server generates an appropriate commentary for that question and delivers it to the smartphone. The input is the user's question and prompt text, and the output is the generated commentary text. Specifically, the server analyzes the user's question, generates a corresponding prompt text, inputs it into the GPT-2 model, obtains the commentary text, and sends it to the smartphone.

(Example 2)

Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

Traditionally, commentary on basketball games required a specialized commentator, and there was the problem of difficulty in securing commentators for youth and mini-basketball games held all over the country. It was also difficult to convey the progress of the game and the movements of the players to viewers in real time, limiting the viewing experience. Furthermore, the collection and analysis of game data was done manually, which was inefficient and sometimes lacked consistency and accuracy.

The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

In this invention, the server includes means for collecting video and audio data of matches held throughout the country, means for storing the collected data in cloud storage, means for transmitting the stored data to a generative AI model and generating commentary for the match, means for synchronizing the generated commentary with the match video, means for streaming the match video with commentary, means for providing an interface for users to view, means for receiving the match video with commentary streamed from the server, and means for displaying the received video to the user. This enables live broadcasts with high-quality commentary by AI even for youth and mini-basketball matches held throughout the country, improving the viewing experience. In addition, efficient and accurate data management can be achieved because the collection and analysis of match data is automated.

"Means for collecting video and audio data from matches held around the country" refers to devices and systems that use cameras and microphones installed at match venues around the country to capture video and audio data from matches in real time and transmit it to a server.

"Means for storing collected data in cloud storage" refers to devices or systems that convert collected video and audio data into an appropriate format and store the data using cloud services.

"Means for sending stored data to a generative AI model and generating commentary on a match" refers to a device or system that retrieves data stored in cloud storage, sends prompt text to the generative AI model to analyze the data, and generates commentary on a match.

"Means for synchronizing the generated commentary with the game video" refers to a device or system that inserts the generated commentary text at a specific timestamp in the game video, allowing the video and commentary to be played in a synchronized state.

"Means for streaming game footage with commentary" refers to a device or system that encodes synchronized video data in real time and transmits it to a streaming platform for distribution to viewers.

"Means for providing an interface for users to view" refers to a device or system for displaying a viewing interface to users through a device such as a smartphone, tablet, or PC.

"Means for receiving game footage with commentary streamed from a server" refers to a device or system for receiving game footage with commentary streamed from a server in real time via an Internet connection.

"Means for displaying received video to a user" refers to a device or system for decoding received video data and displaying it on the user's device screen.

This invention is a system that broadcasts youth and mini-basketball matches held around the country with commentary by AI. A specific embodiment of this system is described below.

Server role

The server collects video and audio data from matches held around the country. Cameras and microphones installed at match venues around the country capture video and audio from matches in real time and send this data to the server. The collected data is stored in cloud storage such as Amazon S3 and Google Cloud Storage.

The server then sends the stored data to a generative AI model. Examples of generative AI models that can be used include OpenAI's GPT-4 and Google's BERT. The server retrieves the data from cloud storage and sends a prompt to the generative AI model. The prompt includes instructions for analyzing the progress of the game, the movements of players, the score, etc.

The generative AI model analyzes the data based on the prompt text and generates commentary in natural language. The generated commentary is sent back to the server in text format. The server then synchronizes the generated commentary with the game footage. This synchronization process is performed using video editing software such as FFmpeg. The server inserts the commentary text at specific timestamps in the footage, allowing the footage and commentary to be played in sync.

Finally, the server streams the match footage with commentary to a streaming platform such as YouTube Live or Twitch. The server encodes the synchronized video data in real time and sends it to the streaming platform.

Role of the device

The terminal provides the user with an interface for viewing. Devices such as smartphones, tablets, and PCs are used here. The terminal displays the viewing interface to the user through a dedicated application or web browser. The interface includes a list of matches and a search function.

The device receives game footage with commentary streamed from the server. An Internet connection is required for this reception. The device receives video data from the streaming platform in real time and buffers it. The device decodes the received video data and displays it on the user's device screen.

User Roles

Users access the system using a terminal. To do so, they use a dedicated application or a web browser. Users launch a dedicated application on their smartphone or PC, or access the system's URL using a web browser.

The user selects the match they wish to watch. For this selection, a list of matches and a search function are provided. The user scrolls through the list of matches on the interface and clicks on the match they wish to watch. They can also use the search function to find a specific match.

The user watches the commentary-enhanced video of the selected match. While watching, the user can play, pause, rewind, and perform other operations. The user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed.

Specific examples

For example, if a user wants to watch a national youth basketball tournament game on their smartphone, they would follow these steps:

1. The user launches the dedicated app on their smartphone.

2. Use the in-app search function and type in "National Youth Basketball Tournament."

3. Select the match you want to watch from the search results.

4. The app streams game footage with commentary from the server.

5. Users can watch the game on their smartphone screen and enjoy the commentary.

An example of a prompt sentence to be input to the generative AI model is, "Analyze the game video and audio data of the National Youth Basketball Tournament and generate commentary including the progress of the game, player movements, score status, etc." Using this prompt sentence, the AI analysis system generates commentary of the game.

The flow of the identification process in Example 2 is explained using Figure 13.

Step 1:

The server collects video and audio data from matches held around the country. As input, it receives real-time data from cameras and microphones installed at match venues around the country. Specifically, it transmits the video and audio captured by these devices to the server via the internet. As output, it obtains the collected video and audio data.

Step 2:

The server stores the collected data in cloud storage. As input, it receives the video and audio data collected in step 1. Specifically, it converts the data into an appropriate format and uploads it to a cloud service such as Amazon S3 or Google Cloud Storage. As output, it obtains the data stored in cloud storage.

Step 3:

The server sends the saved data to the generative AI model to generate commentary on the match. As input, it receives video and audio data obtained from cloud storage. Specifically, it sends a prompt to the generative AI model to analyze the data. The prompt includes instructions to analyze the progress of the match, player movements, the score, etc. The output is the commentary text returned from the generative AI model.

Step 4:

The server synchronizes the generated commentary with the game footage. As input, it receives the commentary text obtained in step 3 and the video data obtained from the cloud storage. Specifically, it uses video editing software such as FFmpeg to insert the commentary text at a specific timestamp in the video. As output, it obtains the game footage synchronized with the commentary.

Step 5:

The server streams the game footage with commentary. As input, it receives the game footage synchronized with the commentary obtained in step 4. Specifically, it encodes the video data in real time and transmits it to a streaming platform such as YouTube Live or Twitch. As output, it obtains the streaming video that is distributed to the viewers.

Step 6:

The terminal provides an interface for users to watch. It receives user operations as input. Its specific operation is to display the viewing interface through a dedicated application or web browser. The interface includes a list of matches and a search function. As output, it provides information that allows the user to select the match they wish to watch.

Step 7:

The terminal receives game footage with commentary that is streamed from the server. As input, it receives streaming data from the server. Specifically, it receives video data in real time via an Internet connection and buffers it. As output, it obtains the received video data.

Step 8:

The terminal displays the received video to the user. As input, it receives the video data received in step 7. Specifically, it decodes the video data and displays it on the user's device screen. As output, it obtains the video that the user watches.

Step 9:

The user accesses the system using a terminal. A dedicated application or a web browser is used as input. Specifically, the user launches a dedicated application on a smartphone or PC, or accesses the system's URL in a web browser. Access to the system is obtained as output.

Step 10:

The user selects the match they want to watch. As input, they use the match list and search functionality on the viewing interface. They scroll through the match list and click on the match they want to watch. They can also use the search functionality to search for a specific match. As output, they get the selected match.

Step 11:

The user watches the commentary-enhanced video of the selected match. As input, the match selected in step 10 is received. Specifically, the user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed. As output, the video of the match being watched is obtained.

(Application example 2)

Next, application example 2 of embodiment example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In conventional basketball game commentary systems, live broadcasts and commentary of games were dependent on human labor, making it difficult to provide youth and mini-basketball games held all over the country to a wide audience. In addition, there was a lack of game highlights and replay functions, and post-game statistical data and analysis reports, making it difficult to increase viewer satisfaction. Furthermore, there was no function for users to follow their favorite teams or players, making it difficult to meet individual needs.

The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for AI to provide commentary on the live broadcast of a basketball game, a means for distributing commentary of youth and mini-basketball games nationwide with AI live broadcast, a means for AI to automatically generate running scores created by officials and people for each team, a means for collecting video and audio data of the game and analyzing it with AI to generate commentary, a means for distributing the game in real time through an application installed on a smartphone and providing commentary by AI, a means for providing highlights and replay functions of the game, a means for providing a function that allows users to follow their favorite teams and players, and a means for providing statistical data and analysis reports after the game. This makes it possible to widely provide youth and mini-basketball games held all over the country to viewers and increase the satisfaction of the viewers.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to generate commentary in real time during a basketball game and provide it to viewers.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to technology that broadcasts youth and mini-basketball games held around the country in real time with commentary by artificial intelligence.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Method of collecting video and audio data from the match and analyzing it with AI to generate commentary" refers to technology that collects video and audio data from the match and uses artificial intelligence to analyze it and generate commentary.

"Means of broadcasting matches in real time and providing commentary by AI through an application installed on a smartphone" refers to a technology that uses a smartphone application to broadcast matches in real time and provide commentary by artificial intelligence.

"Means for providing match highlight and replay features" refers to technology that provides the functionality to compile important scenes and plays during a match as highlights or play them back as replays.

"Means for providing a function that enables users to follow their favorite teams and players" refers to technology that provides a function that enables users to select and follow their favorite teams and players and receive information about them on a priority basis.

"Means for providing post-match statistical data and analytical reports" refers to technology that generates match statistical data and analytical reports after the match ends and provides them to users.

The following system configuration is described as a form for implementing this invention.

System configuration

The system includes a server, a smartphone, and an AI model. The server collects video and audio data from the match, analyzes it using the AI model, and generates commentary. The smartphone provides an application that enables users to watch the match in real time and receive commentary by the AI.

Hardware and software used

Server: A server with high-performance data processing capabilities (e.g. AWS EC2 instance)

Smartphone: iOS or Android device

AI model: OpenAI's GPT-4

Data processing library: OpenCV (for image processing), Python (programming language)

Data processing and data calculations

1. Collecting video and audio data from the match:

The server collects video and audio data in real time from youth and mini-basketball matches held across the country. This is done using input devices such as cameras and microphones.

2. AI-based analysis and explanation generation:

The server uses OpenCV to analyze the collected video and audio data and extracts important scenes and plays from the match. Based on the extracted information, it uses OpenAI's GPT-4 to generate commentary. Specifically, it uses the following prompt sentence:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

3. Real-time broadcasting and commentary:

The server delivers the game footage along with the generated commentary to smartphones in real time. An application installed on the smartphone allows users to receive the AI commentary while watching the game.

4. Providing highlight and replay features:

The server compiles highlights of key scenes during the match and provides a replay function, allowing users to rewatch key moments of the match.

5. User following feature:

The smartphone application provides users with the ability to follow their favorite teams and players. This allows users to receive information about the teams and players they follow preferentially.

6. Providing post-match statistics and analysis reports:

After the match ends, the server generates statistical data and analysis reports and provides them to users via a smartphone application, allowing users to obtain detailed analysis information about the match.

Specific examples

For example, when a user is watching a youth basketball game through a smartphone application, the server delivers real-time game footage and AI commentary. The moment Player A scores a goal during the game, the server analyzes the scene and generates the following prompt:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

Based on this prompt, the AI model will generate commentary and provide it to the user. Users can enjoy the AI commentary in real time while watching the match. In addition, after the match, users can obtain detailed information about the match through statistical data and analysis reports.

The flow of the specific processing in application example 2 is explained using Figure 14.

Step 1:

The server collects video and audio data from youth and mini-basketball matches held around the country. As input, it receives real-time data from input devices such as cameras and microphones, and as output, it stores this data in storage. In concrete terms, the server receives video and audio data sent from each match venue, converts it into an appropriate format, and stores it.

Step 2:

The server analyzes the collected video and audio data. It uses the video and audio data saved in step 1 as input, and extracts information about important scenes and plays in the match as output. Specifically, the server uses OpenCV to analyze the video data frame by frame and identify the movements of players and the position of the ball. It also extracts events such as goals and fouls from the audio data.

Step 3:

The server generates a prompt sentence based on the analysis results. As input, it uses information about important scenes and plays in the match extracted in step 2, and as output, it generates a prompt sentence to pass to the AI model. Specifically, the server converts information such as player movements and scoring status into text format and generates a prompt sentence like the following:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

Step 4:

The server sends the generated prompt text to the AI model and generates an explanation. It uses the prompt text generated in step 3 as input and receives the explanation text generated from the AI model as output. In concrete terms, the server calls OpenAI's GPT-4 API, sends the prompt text, and generates an explanation.

Step 5:

The server delivers the generated commentary and game footage to smartphones in real time. It uses the commentary text generated in step 4 and the video data collected in step 1 as input, and generates data to be delivered to the user's smartphone as output. In concrete terms, the server overlays the commentary text on the video data and transmits it to the smartphone in streaming format.

Step 6:

Users watch the game through a smartphone application and receive commentary by AI. As input, it uses video and commentary data distributed from the server, and as output, it displays the game video and commentary to the user. In concrete terms, the smartphone application decodes the received data and displays it on the screen.

Step 7:

The server summarizes important scenes during the match as highlights and provides a replay function. It uses the information of important scenes extracted in step 2 as input and generates highlight videos as output. In concrete terms, the server edits the important scenes and saves them in a format that users can replay.

Step 8:

Users follow their favorite teams and players through a smartphone application. The application uses the user's selection information as input, and prioritizes displaying information about the followed teams and players as output. In concrete terms, the application records the user's selection, and prioritizes retrieving and displaying related information.

Step 9:

After the match ends, the server generates statistical data and analysis reports and provides them to the user. It uses data collected during the match as input and generates statistical data and analysis reports as output. In concrete terms, the server aggregates the match data, creates a report in a visually easy-to-understand format, and provides it to the user through a smartphone application.

(Example 3)

Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

In traditional basketball games, in order to grasp the score and foul situation in real time, it was necessary to manually create a running score, which was time-consuming and labor-intensive. In addition, there was a lack of a system to effectively collect and analyze data on teams and players across the country and use it for player development. This meant that there was insufficient visualization of player performance and feedback for development.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes a means for receiving video data and audio data of a match, a means for analyzing the received video data, a means for analyzing the received audio data, a means for extracting the score situation and the foul situation from the analysis result, a means for generating a running score based on the extracted information, a means for updating the generated running score in real time and transmitting it to the terminal, a means for displaying the running score transmitted to the terminal, a means for collecting information such as the match results and stats of teams and players nationwide, a means for database-izing the collected information, a means for analyzing the databased information and evaluating the performance of players, a means for generating feedback for player development based on the analysis result, a means for transmitting the generated feedback to the terminal, and a means for displaying the feedback transmitted to the terminal. This makes it possible to grasp the progress of the match in real time, and effectively visualize the performance of players and provide feedback for development.

"Game video data" refers to video files captured by a camera showing a basketball game.

"Audio data" refers to audio files recorded by a microphone during a basketball game.

"Means of receiving" refers to the function that allows the server to obtain data from outside.

"Means of analysis" refers to the function for processing received data and extracting necessary information.

"Scoring status" is information that shows how many points each team has scored during a match.

"Foul situation" is information that shows which players committed what type of fouls during the game.

The "running score" is a record of points and fouls that is updated in real time as the game progresses.

"Means of updating in real time" refers to a function that instantly reflects data as the match progresses.

"Terminal" refers to the device that a user uses to view information.

"Means of display" refers to the functionality for visually presenting information on a terminal.

"Match results" refers to information showing the final score and outcome after the match has ended.

"Stats" are statistical data that quantify the performance of players and teams.

"Means of collection" refers to the function for collecting the necessary data.

"Means of database creation" refers to the function of organizing and storing collected data.

"Means of analysis" refers to the function of processing databased information and deriving meaningful results.

"Feedback" refers to advice or evaluation provided based on the analysis results.

"Visualization" refers to displaying data visually to make it easier to understand.

"Development" refers to training and guidance to improve players' abilities.

This invention is a system for grasping the score and foul situation in a basketball game in real time and evaluating and developing the performance of players. A specific embodiment of this system is described below.

System configuration

Hardware

Server: A computer with high-performance computing power that receives, analyzes, generates, and transmits data.

Device: The device on which the user checks information, including smartphones, tablets, and computers.

Camera: A device used to capture video data of a match.

Microphone: A device used to capture audio data from a match.

Software

Video analysis software: Analyzes video data using libraries such as OpenCV.

Speech analysis software: Use libraries such as LibROSA to analyze speech data.

Database management system: Collected data is stored in a database using a system such as MySQL.

Data analysis tools: Analyze data using tools such as Pandas and NumPy.

Generative AI models: Use models such as GPT-4 to generate running scores and feedback.

System operation

Data reception and analysis

The server receives video and audio data of the game in real time from the cameras and microphones. The received video data is analyzed using video analysis software to detect goals and fouls. The audio data is analyzed using audio analysis software to analyze the commentary and crowd reactions.

Information extraction and running score generation

The server extracts the scoring and foul situations from the analysis results. Based on the extracted information, a generative AI model is used to generate a running score. The generated running score is updated in real time and sent to the terminal.

Display data

The device displays the received running score, allowing the user to follow the progress of the match in real time.

Data collection and database creation

The server collects information such as match results and stats for teams and players across the country. The collected information is stored in a database using a database management system.

Data analysis and feedback generation

The server analyzes the databased information and evaluates the player's performance. Based on the analysis results, a generative AI model is used to generate feedback for the player's development. The generated feedback is sent to the terminal, which displays it. This allows the user to check the analysis results of the player's performance and advice for development.

Specific examples

When a user is watching a basketball game, the server analyzes the video and audio of the game and automatically extracts information about scores and fouls.

The server generates information in real time, such as "Team A has scored" or "Team B has committed a foul," and sends it to the terminal.

The device displays this information, allowing users to check the progress of the match in real time.

Example of a prompt

"Analyze video and audio data from basketball games, extract the scoring and foul situations, and generate a running score."

"Analyze the game results and stats of basketball teams and players nationwide, evaluate the performance of players, and generate feedback for training." The flow of the specific process in Example 3 is explained with reference to FIG. 15.

Step 1:

The server receives video and audio data from the match.

Input: Real-time data from camera and microphone

Specific operation: Receives video and audio transmitted from the camera and microphone via the network.

Output: Received video and audio data

Step 2:

The server analyzes the received video data.

Input: Received video data

Specific operation: Use video analysis software (e.g. OpenCV) to detect scoring and foul scenes from video data.

Output: Data with timestamps for goals and fouls

Step 3:

The server analyzes the received audio data.

Input: Received audio data

Specific actions: Use audio analysis software (e.g. LibROSA) to analyze commentary and audience reactions from audio data.

Output: Timestamped data of commentary and audience reactions

Step 4:

The server extracts the scoring and foul situations from the analysis results.

Input: Timestamped data on goals and fouls, as well as timestamped data on commentary and crowd reactions

Specific operation: Integrate the results of video and audio analysis to extract the occurrence of goals and fouls.

Output: Scoring and foul situation data

Step 5:

The server generates a running score based on the extracted information.

Input: Scoring and foul situation data

Specific behavior: Uses a generative AI model (e.g. GPT-4) to generate points and foul information in text format.

Output: Text data of running score

Step 6:

The server updates the generated running score in real time and sends it to the device.

Input: Text data of running score

Specific operation: Sends the running score to the device using WebSocket or HTTP protocol.

Output: Running score sent to the device

Step 7:

The device will display the received running score.

Input: Running score sent to the device

Specific operation: Visually display the running score on the device display.

Output: A running score that the user can visually check

Step 8:

The server collects information such as match results and stats for teams and players across the country.

Input: Data from each team's official website and sports data services

Specific actions: Collect match results and stats using web scraping and APIs.

Output: Collected match results and stats data

Step 9:

The server stores the collected information in a database.

Input: Collected match results and stats data

Specific action: Store data in a table using a database management system (e.g. MySQL).

Output: Match results and stats data stored in a database

Step 10:

The server analyzes the databased information and evaluates the players' performance.

Input: Match results and stats data stored in the database

Specific operations: Use data analysis tools (e.g. Pandas, NumPy) to calculate players' shooting success rates, defensive strength, etc.

Output: Player performance evaluation data

Step 11:

The server generates feedback for player development based on the analysis results.

Input: Player performance evaluation data

Specific actions: Use a generative AI model (e.g. GPT-4) to generate textual feedback for player development.

Output: Text data of feedback for training

Step 12:

The server sends the generated feedback to the device.

Input: Text data for feedback for training

Specific behavior: Sends feedback to the device using WebSocket or HTTP protocol.

Output: Feedback sent to the device

Step 13:

The device will display the feedback it receives.

Input: Feedback sent to device

Specific action: Visually display feedback on the device display.

Output: Visual feedback for the user

(Application example 3)

Next, application example 3 of embodiment example 3 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Traditional live broadcasts and commentary of basketball games rely on human commentators, which can make it difficult to provide accurate information in real time. Commentary on youth and mini-basketball games across the country also relies on human input, making it difficult to provide information efficiently. Furthermore, when monitoring production lines in factories, detecting abnormalities and improving production efficiency are issues. To solve these issues, an automated system that utilizes AI is needed.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 3 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games from around the country with AI live coverage, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for analyzing camera footage and audio data within the factory to detect production status and abnormal occurrences in real time and provide scores, and a means for storing past production data in a database and providing feedback for efficiency. This allows for accurate understanding of the progress of the game and the status of the production line in real time, making it possible to provide efficient information and improve production efficiency.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to automatically provide commentary on the progress of a basketball game and plays in real time.

"A means of broadcasting commentary on youth and mini-basketball games nationwide with live coverage by AI" refers to a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games nationwide and broadcasts that commentary in real time.

"A means of using AI to automatically generate running scores that are created manually by officials and for each team" refers to technology in which artificial intelligence automatically analyzes points, fouls, and other scores as the game progresses, and generates scores in real time.

"A means of analyzing camera footage and audio data within a factory to detect and score production status and abnormalities in real time" refers to a technology in which artificial intelligence analyzes video and audio data obtained from cameras and microphones installed within the factory, detects the status and abnormalities of the production line in real time, and displays the results as a score.

"Means of compiling past production data into a database and providing feedback for improving efficiency" refers to a technology that stores past production data collected within a factory in a database, analyzes the data, and provides feedback for improving production efficiency.

The following system configuration is described as a form for implementing this invention.

System configuration

Hardware

Camera: Cameras installed in the factory capture video data.

Microphones: Microphones installed in the factory capture audio data.

Server: A server for analyzing and storing data.

Device: The device (PC, smartphone, etc.) used by the administrator to check the results.

Software

OpenCV: A library for acquiring and preprocessing camera images.

TensorFlow/Keras: Libraries for loading AI models and making predictions.

SQLite: A database management system for storing production data.

Data processing and data calculations

The server acquires camera footage and audio data from within the factory in real time and performs preprocessing using OpenCV. The preprocessed data is input into an AI model using TensorFlow/Keras to detect production status and abnormalities. The detection results are displayed as a score and stored in a SQLite database.

Specific examples

For example, if an abnormality occurs in a factory, it will be detected from camera footage and notified to managers in real time. It can also analyze past production data and provide feedback to improve efficiency.

Example of a prompt

An example of a prompt to input to the generative AI model is as follows:

"Please create an AI model that analyzes camera footage in the factory and detects abnormalities on the production line in real time. Please also add a function to notify the administrator if an abnormality is detected. Also, please create a system that stores past production data in a database and provides feedback to improve efficiency."

In this way, the server can monitor the status of the production line in real time and detect any anomalies. It can also provide feedback for efficiency improvements based on past data.

The flow of the specific processing in application example 3 is explained using Figure 16.

Step 1:

The server captures video and audio data in real time from cameras and microphones installed in the factory. The input is the raw data from the cameras and microphones, and the output is the captured video and audio data. In concrete terms, the server connects to the cameras and microphones and starts the data stream.

Step 2:

The server preprocesses the acquired video data using OpenCV. The input is the video data acquired in step 1, and the output is the preprocessed video data. Specifically, the server resizes the video data to 224x224 pixels and normalizes it.

Step 3:

The server inputs the preprocessed video data into the AI model using TensorFlow/Keras to make predictions. The input is the video data preprocessed in step 2, and the output is the predicted results of production status and abnormality occurrence. In concrete terms, the server loads the AI model, inputs the preprocessed data into the model, and makes predictions.

Step 4:

The server displays the prediction result as a score and saves it in a SQLite database. The input is the prediction result obtained in step 3, and the output is the displayed score and the data saved in the database. In concrete terms, the server converts the prediction result into a score, displays it on the screen, and inserts it into the database.

Step 5:

The terminal receives the score and anomaly detection notifications sent from the server and displays them to the administrator. The input is the score and notifications sent from the server, and the output is the display information that the administrator can check. In concrete terms, the terminal receives data from the server and displays it on the user interface.

Step 6:

The server analyzes past production data and generates feedback for efficiency improvements. The input is past production data stored in a database, and the output is feedback information for efficiency improvements. In concrete terms, the server retrieves past data from the database and applies an analysis algorithm to generate feedback.

Step 7:

The terminal receives feedback information sent from the server and displays it to the administrator. The input is the feedback information sent from the server, and the output is the display information that the administrator can check. In concrete terms, the terminal receives feedback information from the server and displays it on the user interface.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

"Example 1"

In one embodiment of the present invention, a system in which AI provides commentary on a live broadcast of a basketball game is combined with an emotion engine that recognizes the user's emotions. Specifically, the emotion engine recognizes the emotions (happiness, sadness, surprise, etc.) that the user expresses while watching the game, and adjusts the content and tone of the commentary according to those emotions. For example, if the user expresses joy at how the game is progressing, the commentary will also share that joy. This allows the user to hear commentary that reflects their own emotions, further improving their satisfaction with watching the game.

"Example 2"

The emotion engine is also incorporated into a system that broadcasts commentary on youth and mini-basketball games from around the country, complete with live coverage by AI. Specifically, the emotion engine recognizes the emotions of users watching youth and mini-basketball games, and adjusts the content and tone of the commentary according to those emotions. For example, if a user is impressed by their child's performance, the commentary will be one that shares that emotion. This allows users to hear commentary that reflects their own emotions, further increasing their satisfaction with watching the games.

"Example 3"

Furthermore, emotion engines are also being incorporated into systems that use AI to automatically generate running scores that are manually created by officials and for each team. Specifically, the emotion engine recognizes the user's emotions regarding the score situation of the match, and adjusts the way the score is displayed according to those emotions. For example, if the user is happy about a score, the score display will also be displayed in a way that emphasizes that joy. This allows users to see a score display that reflects their own emotions, further increasing their satisfaction when watching the match.

The process flow for each example is explained below.

"Example 1"

Step 1: The AI begins commentating on a live broadcast of a basketball game.

Step 2: The emotion engine recognizes the user's emotions in real time.

Step 3: The AI adjusts the content and tone of the commentary based on the emotion recognized by the emotion engine.

"Example 2"

Step 1: AI will begin providing live commentary for youth and mini-basketball matches across the country.

Step 2: The emotion engine recognizes the emotions of users watching the game in real time.

Step 3: The AI adjusts the content and tone of the commentary based on the emotion recognized by the emotion engine.

"Example 3"

Step 1: AI automatically generates a running score based on the game's scoring situation.

Step 2: The emotion engine recognizes the user's emotions in real time regarding the score of the game.

Step 3: The AI adjusts the way the score is displayed depending on the emotion recognized by the emotion engine.

(Example 1)

Next, a first embodiment of the first embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

In conventional basketball game commentary systems, it was difficult to analyze the game situation and player movements in real time and provide commentary that corresponded to the user's emotions. In addition, there was a lack of means to efficiently distribute commentary of youth and mini-basketball games across the country, or to automatically generate running scores that are created manually by officials and for each team. Furthermore, there was insufficient use of emotion recognition technology to improve the user's viewing experience.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for providing commentary on live coverage of basketball games using artificial intelligence, means for delivering commentary on youth and mini-basketball games across the country with live coverage by artificial intelligence, means for automatically generating running scores using artificial intelligence that are created manually for each official and team, means for analyzing game video data, means for analyzing game audio data, means for receiving and analyzing user emotion data, means for automatically generating commentary based on the analysis results and adjusting the content and tone of the commentary according to the user's emotions, and means for delivering the adjusted commentary together with the live coverage. This makes it possible to analyze the situation of the game and the movements of the players in real time and provide commentary according to the user's emotions.

"Means for artificial intelligence to provide commentary for live broadcasts of basketball games" refers to technology that analyzes video and audio data of basketball games, and automatically generates commentary based on the results of that analysis using artificial intelligence, which is then distributed along with the live broadcast.

"A means of broadcasting commentary on youth and mini-basketball games nationwide with live coverage by artificial intelligence" refers to a technology in which artificial intelligence automatically generates commentary for youth and mini-basketball games nationwide and broadcasts that commentary together with live coverage.

"Means for automatically generating running scores using artificial intelligence that are created manually for each official or team" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Means for analyzing game video data" refers to technology that analyzes game video data frame by frame to identify the positions and movements of players, the position of the ball, etc.

"Means for analyzing audio data from a match" refers to technology that analyzes audio data from a match and identifies the content of the commentary and the reactions of the audience.

"Means for receiving and analyzing user emotion data" refers to technology that captures the user's facial expressions and voice, and analyzes that data to identify the user's emotions.

"Means for automatically generating commentary based on analysis results and adjusting the content and tone of the commentary according to the user's emotions" refers to a technology that automatically generates commentary based on the analysis results of video and audio data, and further adjusts the content and tone of the commentary taking into account the user's emotional data.

"Means for delivering adjusted commentary together with live broadcast" refers to technology that delivers adjusted commentary in real time to a user's device together with the live broadcast.

Form for implementing the invention

This invention is a system in which artificial intelligence provides commentary for live coverage of basketball games, analyzing video and audio data from the game and providing commentary that reflects the user's emotions. A specific embodiment of this system is shown below.

Hardware and software used

Hardware: High-performance servers, user devices (PCs, smartphones, tablets, etc.)

Software: Video analysis software (e.g. OpenCV), audio analysis software (e.g. Google Cloud Speech-to-Text API), emotion engines (e.g. Microsoft Azure Emotion API), generative AI models (e.g. GPT-4)

System configuration

1. The server receives the game video and audio data in real time. It uses a high-speed network connection to receive the streaming data sent from the game venue.

2. The server uses video analysis software to analyze the received video data frame by frame. Specifically, computer vision techniques are used to identify the positions of players, their movements, and the position of the ball. For example, OpenCV is used.

3. The server uses voice analysis software to analyze the received voice data. Specifically, it uses voice recognition technology to identify the content of the commentary and the reactions of the audience. For example, it uses the Google Cloud Speech-to-Text API.

4. The server uses a generative AI model to automatically generate commentary on the match based on the results of video and audio analysis. Specifically, it uses a natural language generation model such as GPT-4 to generate sentences that explain the situation of the match and the movements of the players.

5. The device captures the user's facial expressions and voice in real time and transmits them as emotion data to the server. Specifically, the device uses a camera and microphone.

6. The server uses an emotion engine to analyze the received emotion data. Specifically, it uses emotion recognition technology to identify emotions such as happiness, sadness, surprise, etc. from the user's facial expressions and voice. For example, it uses the Microsoft Azure Emotion API.

7. The server adjusts the content and tone of the generated commentary according to the user's emotions analyzed by the server. Specifically, if the user is happy, the tone of the commentary is brightened and more positive expressions are included.

8. The server delivers the adjusted commentary along with the live broadcast to the user's device. Specifically, it delivers in real time using streaming technology. For example, WebRTC is used.

Specific examples

For example, if a user expresses joy while watching a game, the emotion engine will recognize that joy. Based on that information, the server will lighten the tone of the commentary generated by the generative AI model and adjust it to share the joy. This allows users to hear commentary that reflects their own emotions, improving their satisfaction watching the game.

Example of a prompt

"Design an AI system to provide commentary for live broadcasts of basketball games. It must analyze video and audio data from the game, understand the game situation and the movements of the players, and incorporate an emotion engine that adjusts the content and tone of the commentary according to the user's emotions."

The flow of the identification process in Example 1 is explained using Figure 17.

Step 1:

The server receives the video and audio data of the match in real time. The input is streaming data sent from the match venue, and the output is the video and audio data stored in the server. Specifically, the server receives the data using a high-speed network connection and stores it in an appropriate format.

Step 2:

The server uses video analysis software to analyze the received video data frame by frame. The input is the saved video data, and the output is the analysis results, such as the positions and movements of the players and the position of the ball. In concrete terms, it uses OpenCV to analyze each frame and identify the positions of the players and the ball.

Step 3:

The server uses voice analysis software to analyze the received voice data. The input is the stored voice data, and the output is the analysis results in the form of text that contains the commentary and the reactions of the audience. Specifically, the Google Cloud Speech-to-Text API is used to convert the voice data into text and extract key information.

Step 4:

The server uses a generative AI model to automatically generate commentary on the match based on the results of video and audio analysis. The input is the results of video and audio analysis, and the output is the generated commentary. Specifically, it uses a natural language generation model such as GPT-4 to generate sentences that explain the situation of the match and the movements of the players.

Step 5:

The device captures the user's facial expressions and voice in real time and sends them to the server as emotion data. The input is the user's facial expressions and voice, and the output is the emotion data sent to the server. In concrete terms, the device uses a camera and microphone to capture the user's facial expressions and voice, and sends them to the server in real time.

Step 6:

The server uses an emotion engine to analyze the received emotion data. The input is the emotion data sent, and the output is the analysis results that identify the user's emotion. Specifically, it uses the Microsoft Azure Emotion API to identify emotions such as joy, sadness, and surprise from the user's facial expressions and voice.

Step 7:

The content and tone of the generated commentary is adjusted according to the user's emotions analyzed by the server. The input is the result of emotion analysis and the generated commentary, and the output is the adjusted commentary. Specifically, if the user shows joy, the tone of the commentary is brightened and more positive expressions are used.

Step 8:

The server delivers the adjusted commentary to the user's device along with the live broadcast. The input is the adjusted commentary and the live broadcast data, and the output is the live broadcast with real-time commentary delivered to the user's device. In concrete terms, it uses WebRTC to deliver the adjusted commentary in real time.

(Application example 1)

Next, application example 1 of embodiment example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional basketball game commentary systems had the problem that the commentary was uniform and could not flexibly respond to the emotions of the viewers. In addition, improving the quality of the live broadcast and commentary of the game required a large number of people, which was costly and laborious. Furthermore, there was a lack of efficient means to provide commentary on youth and mini-basketball games across the country.

The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games nationwide with AI live coverage, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions, a means for generating commentary using a generation AI model, and a means for generating commentary by inputting a prompt sentence. This enables flexible commentary according to the viewer's emotions, improving the quality of live coverage and commentary of games. In addition, commentary on youth and mini-basketball games nationwide can be efficiently performed.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to a means that uses artificial intelligence technology to analyze video and audio data of basketball games, understand the situation of the game and the movements of the players, and automatically generate commentary.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a means of broadcasting live coverage including commentary on youth and mini-basketball games from around the country with AI providing commentary.

"Means of using AI to automatically generate running scores that are created manually by officials and teams" refers to a means of using AI to automatically generate scores that are manually created by officials and teams during a match.

"Means for adjusting the content and tone of commentary using an emotion engine that recognizes the user's emotions" refers to means that use emotion recognition technology to recognize the user's emotions (happiness, sadness, surprise, etc.) and adjust the content and tone of the commentary in accordance with those emotions.

"Means for generating commentary using a generative AI model" refers to means for generating commentary based on the situation of a match using a generative AI model (e.g., GPT-3).

"Means for inputting a prompt sentence and generating an explanation" refers to means for inputting a specific prompt sentence into a generative AI model and generating an explanation based on that prompt.

A system for implementing this invention includes a means for AI to provide commentary for live coverage of basketball games, a means for distributing commentary for youth and mini-basketball games across the country with AI commentary, a means for AI to automatically generate running scores that are created manually for each official or team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions, a means for generating commentary using a generative AI model, and a means for generating commentary by inputting a prompt text.

Hardware and software used

Hardware: Smartphone camera and microphone

Software: OpenCV, Keras, Transformers

Data processing and data calculations

Server

The server receives and analyzes video and audio data from the basketball game in real time. It uses OpenCV to acquire the video data and Keras to run an emotion recognition model. The emotion recognition model analyzes the user's facial expressions and recognizes emotions such as happiness, sadness, and surprise.

Terminal

The device generates commentary using a generative AI model (e.g., GPT-3). A prompt sentence is input into the generative AI model, which generates commentary based on the prompt. Examples of prompt sentences include "The user is happy. Generate a joyful commentary for the basketball game." and "The user is sad. Generate a comforting commentary for the basketball game."

User

Users watch the game through their smartphone's camera and microphone, and their emotions are recognized. The emotion engine adjusts the content and tone of the commentary based on the user's emotions. For example, if the user expresses joy at a highlight scene in the game, the commentary might say something like, "Great play! We've been waiting for this moment!" Conversely, if the user expresses sadness at a comeback scene in the game, the commentary might say something like, "It's a disappointing result, but there's still a chance. Let's look forward to the next play."

In this way, flexible commentary based on the viewer's emotions becomes possible, improving the quality of live game coverage and commentary. It also enables efficient commentary of youth and mini-basketball games across the country.

The flow of the specific processing in application example 1 is explained using Figure 18.

Step 1:

The server receives video and audio data from the smartphone's camera and microphone in real time. The input is the video and audio data from the camera and microphone, and the output is the data ready for analysis.

Step 2:

The server processes the video data using OpenCV to detect the user's face. The input is the video data received in step 1, and the output is the detected face area. Specifically, the server converts the video data to grayscale and applies a face detection algorithm.

Step 3:

The server runs an emotion recognition model using Keras to recognize the user's emotions. The input is the face region detected in step 2, and the output is the recognized emotion (happiness, sadness, surprise, etc.). Specifically, the server resizes the face region to an appropriate size and inputs it into the emotion recognition model.

Step 4:

The server generates a prompt sentence to be input to the generative AI model based on the recognized emotion. The input is the emotion recognized in step 3, and the output is the generated prompt sentence. Specifically, it selects a template sentence according to the emotion and generates the prompt sentence.

Step 5:

The server generates an explanation using a generative AI model (e.g., GPT-3). The input is the prompt text generated in step 4, and the output is the generated explanation text. In concrete terms, the prompt text is input into the generative AI model, and an explanation text is generated.

Step 6:

The server delivers the generated explanatory text to the user in real time. The input is the explanatory text generated in step 5, and the output is the explanatory audio delivered to the user's smartphone. In concrete terms, the explanatory text is input into a voice synthesis engine, and audio data is generated and delivered.

Step 7:

Users watch the game and listen to the commentary through their smartphones. The input is the commentary audio delivered from the server, and the output is an improvement to the user's viewing experience. In concrete terms, the commentary audio is played through the smartphone's speakers.

(Example 2)

Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

Traditional basketball game commentary relied on the commentator's subjective judgment, making it difficult to provide commentary that reflected the emotions of viewers. In addition, there was a lack of systems to provide youth and mini-basketball games from across the country to a wide audience, making it difficult to grasp the progress of the game and the performance of the players in real time. Furthermore, manually creating a running score for the game was time-consuming and labor-intensive, so efficient management was required.

The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

In this invention, the server includes means for providing commentary on live coverage of basketball games using artificial intelligence, means for distributing commentary on youth and mini-basketball games from around the country with live coverage by artificial intelligence, means for automatically generating running scores using artificial intelligence that are created manually for each official and team, means for collecting game video and audio data, means for analyzing the collected data and generating commentary, means for recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions, and means for distributing the generated commentary together with the game video. This makes it possible to provide commentary that corresponds to the viewer's emotions and improve satisfaction with viewing the game.

"Means for artificial intelligence to provide commentary for live broadcasts of basketball games" refers to technology in which artificial intelligence analyzes the progress of a basketball game and the performance of players, and generates commentary in real time.

"A means of broadcasting commentary on youth and mini-basketball games across the country with live coverage by artificial intelligence" refers to a technology in which artificial intelligence analyzes youth and mini-basketball games held across the country, generates commentary, and broadcasts it in real time.

"Means for automatically generating running scores using artificial intelligence that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Means for collecting game video and audio data" refers to technology that uses cameras and microphones installed at the game venue to collect game video and audio data in real time.

"Means of analyzing collected data and generating commentary" refers to technology in which artificial intelligence analyzes collected video and audio data and generates commentary based on the progress of the match and the performance of the players.

"Means of recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to technology that analyzes the viewer's facial expressions and tone of voice, and adjusts the content and tone of the commentary according to those emotions.

"Means for distributing generated commentary together with game footage" refers to technology for distributing commentary generated by artificial intelligence together with game footage via a streaming platform.

This invention is a system in which artificial intelligence (AI) broadcasts live basketball games, and also distributes live commentary by AI for youth and mini-basketball games across the country. It also includes a function that allows the AI to automatically generate running scores that are currently created manually by officials and for each team.

The server collects video and audio data from the match in real time using cameras and microphones installed at the match venue. Specifically, it uses hardware such as Sony's 4K cameras and Shure's high-sensitivity microphones. The collected data is stored in a database on the server.

The server then inputs the stored video and audio data into an AI model. This AI model uses a generative AI model such as OpenAI's GPT-4 or Google's BERT. The AI model analyzes the input data and generates commentary based on the progress of the match and the performance of the players. For example, it analyzes the moment when Player A makes a shot and generates commentary such as, "Player A made a great shot!".

In addition, the device will recognize the emotions of the user watching the game. Specifically, it will use the device's camera and microphone to capture the user's facial expressions and tone of voice. For example, Apple's Face ID technology or Amazon's Alexa will be used for this purpose. The device will then send the captured data to a server.

The server receives the user's emotion data sent from the terminal and analyzes it using an emotion engine. The emotion engine uses, for example, Affectiva or IBM Watson's emotion analysis API. If the user is emotional, the server adjusts the tone of the commentary to be more emotional and generates a commentary such as, "Player A's shot was really amazing! The crowd is really excited!".

Finally, the server distributes the generated commentary along with the game footage. Streaming platforms such as YouTube and Twitch are used for distribution. Users can watch the game through these platforms and enjoy the commentary according to their emotions.

As a concrete example, let's say a youth basketball game is taking place. The server collects video and audio data in real time from Sony's 4K cameras and Shure's high-sensitivity microphones installed at the game venue. The collected data is then input into OpenAI's GPT-4, which analyzes the moment Player A makes a shot. The AI model generates a commentary such as, "Player A made a great shot!".

Meanwhile, the device uses Apple's Face ID technology to capture the facial expressions of the user watching the game and recognizes that they are emotionally moved. The server then uses Affectiva's emotion analysis API to analyze the user's emotions and adjust the tone of the commentary to be more emotional. It generates a commentary such as, "Player A's shot was really amazing! The crowd is really excited!" and distributes it via YouTube.

Examples of prompts might include:

"Analyze video and audio data from youth basketball games to generate commentary on game progress and player performance. Also recognize viewer emotions and adjust the content and tone of the commentary accordingly."

In this way, the server can provide users with commentary that reflects their emotions, improving their satisfaction with watching the game.

The flow of the identification process in Example 2 is explained using Figure 19.

Step 1:

The server uses cameras and microphones installed at the venue to collect video and audio data of the match in real time. Specifically, it uses Sony 4K cameras and Shure high-sensitivity microphones. The input is the video and audio data of the match, and the output is the collected video and audio data. This data is stored in a database on the server.

Step 2:

The server inputs the stored video and audio data into an AI model. This AI model uses, for example, OpenAI's GPT-4. The input is the collected video and audio data, and the output is analyzed data on the progress of the match and the performance of the players. The AI model analyzes the movements of the players and the progress of the match from the video data, and analyzes the commentary and audience reactions from the audio data.

Step 3:

The server generates commentary on the match based on the data analyzed by the AI model. The input is the analyzed data on the progress of the match and the performance of the players, and the output is the generated commentary. For example, the server analyzes the moment when Player A makes a shot and generates a commentary such as, "Player A made a great shot!".

Step 4:

The device recognizes the emotions of users watching a game. Specifically, it uses the device's camera and microphone to capture the user's facial expressions and tone of voice. The input is data about the user's facial expressions and tone of voice, and the output is the recognized user's emotion data. This uses Apple's Face ID technology and Amazon's Alexa.

Step 5:

The server receives the user's emotion data sent from the terminal and analyzes it using an emotion engine. The input is the user's emotion data, and the output is the analyzed emotion information. The emotion engine uses, for example, Affectiva's emotion analysis API. If the user is emotional, the server adjusts the tone of the commentary to be emotional.

Step 6:

The server distributes the generated commentary together with the game footage. The input is the generated commentary and the game footage, and the output is the streaming data to be distributed. Streaming platforms such as YouTube and Twitch are used for distribution. Users can watch the game through these platforms and enjoy commentary according to their emotions.

(Application example 2)

Next, application example 2 of embodiment example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional live basketball game broadcasting systems had the problem that commentary was uniform and did not allow for flexible responses to viewer emotions. In addition, there was a lack of game highlights, replay functions, and chat functions between viewers, so there was a need to improve the viewing experience. Furthermore, there were limited ways to widely provide viewers with information about youth and mini-basketball games across the country, resulting in information disparities between regions.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 2 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary of youth and mini-basketball games from around the country with AI commentary, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for recognizing the emotions of viewers and adjusting the content and tone of the commentary according to those emotions, a means for providing highlights and replay functions for games, and a means for providing a chat function between viewers. This allows for flexible commentary according to the emotions of viewers, improving the viewing experience and making it possible to widely provide youth and mini-basketball games from around the country to viewers.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to generate commentary in real time for basketball games and provide it to viewers.

"A means of broadcasting commentary on youth and mini-basketball games across the country with live coverage by AI" refers to technology that broadcasts youth and mini-basketball games held across the country in real time with commentary by artificial intelligence.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually recorded by officials and teams during a match.

"Means of recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to technology that recognizes emotions from the viewer's facial expressions, voice, etc., and changes the content and tone of the commentary according to those emotions.

"Means for providing match highlight and replay features" refers to technology that provides a highlight feature that extracts and plays important scenes from a match, and a replay feature that replays specific scenes.

"Means for providing a chat function between viewers" refers to technology that provides a chat function that enables users watching a match to exchange messages in real time.

The system for implementing this invention includes the following main components: a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games across the country with AI commentary, a means for AI to automatically generate running scores that are created manually for each official and team, a means for recognizing the emotions of viewers and adjusting the content and tone of the commentary according to those emotions, a means for providing game highlights and replay functions, and a means for providing a chat function between viewers.

Hardware and software used

Hardware: Smartphone (camera, microphone, display)

Software: OpenCV (image processing library), EmotionRecognizer (emotion recognition model), AICommentary (AI commentary generation model), LiveStreaming (live streaming library)

Data processing and data calculations

1. Frame acquisition: The server acquires frames from the live streaming, which allows video data of the match to be collected in real time.

2. Emotion Recognition: The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer. This obtains the viewer's emotion data.

3. Commentary generation: The server generates commentary using AICommentary based on the recognized emotions. This allows commentary to be generated in real time according to the viewer's emotions.

4. Add commentary: The server adds the generated commentary to the live stream, allowing viewers to see emotional commentary in real time.

5. Highlights and Replays: The server automatically extracts important scenes from the match and provides highlights and replays, allowing viewers to rewatch key moments of the match.

6. Chat function: The server provides a chat function that allows viewers to exchange messages with each other in real time. This promotes communication between viewers.

Specific examples

For example, during a youth basketball game, if the viewer is impressed, the server might generate a commentary like, "Great play! I'll never forget this moment!" Or, if the viewer is excited, the server might generate a commentary like, "You won't want to miss this! Watch the next play!"

Example of a prompt

"Create an AI model that recognizes the emotions of your viewers and adjusts the content and tone of your commentary accordingly. For example, if your viewer is emotional, make sure your commentary reflects that emotion."

The flow of the specific processing in application example 2 is explained using Figure 20.

Step 1:

The server obtains frames from live streaming. The input is real-time game video data, and the output is individual frame images. Specifically, the server receives the video data sent from the camera and splits it into frames using OpenCV.

Step 2:

The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer. The input is the frame image obtained in step 1, and the output is the viewer's emotion data. Specifically, the server detects the face in the frame and inputs it into the emotion recognition model to classify the emotion.

Step 3:

The server generates a commentary using AICommentary based on the recognized emotion. The input is the emotion data obtained in step 2, and the output is commentary text corresponding to the emotion. Specifically, the server inputs the emotion data as a prompt into the AI commentary generation model to generate an appropriate commentary.

Step 4:

The server adds the generated commentary to the live stream. The input is the commentary text generated in step 3, and the output is the live video with the commentary added. Specifically, the server synthesizes the commentary text into speech and overlays it on the video.

Step 5:

The server automatically extracts important scenes from the match and provides highlight and replay functions. The input is real-time match video data, and the output is highlight and replay footage. Specifically, the server detects events in the match, extracts important scenes, and saves them.

Step 6:

The server provides a chat function that allows viewers to exchange messages with each other in real time. The input is message data from the viewer, and the output is the message to be sent to other viewers. Specifically, the server receives messages and distributes them to other viewers in the chat room.

(Example 3)

Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

In conventional basketball games, creating a running score and understanding the progress of the game were done manually, making it difficult to update in real time. In addition, there was little progress in creating a database of teams and players nationwide, and feedback for analyzing player performance and training was not provided sufficiently. Furthermore, the score display did not reflect the user's feelings about the score situation in the game, resulting in low satisfaction when watching the game.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for receiving video data and audio data of the match, means for analyzing the received video data, means for analyzing the received audio data, means for extracting the score situation and foul situation from the analysis results, means for generating a running score based on the extracted information, means for updating and transmitting the generated running score in real time, means for displaying the transmitted running score, and means for recognizing the user's emotions and adjusting the score display. This makes it possible to grasp the progress of the match in real time and provide a score display that reflects the user's emotions.

"Game video data" refers to video data that records the progress of a basketball game.

"Audio data" refers to audio data that records commentary and commentary during a match, audience reactions, etc.

"Means of receiving" refers to the function for acquiring video and audio data of the match from an external source.

"Means of analysis" refers to the function of processing received video and audio data and extracting specific information.

"Scoring status" is information that shows how many points each team scored during the game.

"Foul situation" is information that shows which players committed what type of fouls during the game.

The "running score" is a scoreboard that shows the score and foul situation in real time as the game progresses.

"Means for updating and transmitting in real time" refers to a function for instantly updating the generated running score and transmitting it to the user's device.

"Display means" is a function for visually presenting the submitted running score to the user.

"Means of recognizing user emotions" refers to a function for analyzing and understanding the user's emotions regarding the score of a match, etc.

"Means for adjusting score display" is a function for changing the way the scoreboard is displayed depending on the recognized user's emotions.

"Game results and stats information for teams and players nationwide" refers to game results for basketball teams and players nationwide and statistical data for individual players.

"Means of saving to a database" refers to the function for organizing collected match results and stats information and storing it in a database.

"Means of analyzing performance" refers to a function for evaluating a player's performance and abilities based on the information stored in the database.

"Means of generating feedback" refers to the function that creates reports based on the analysis results to suggest player development and areas for improvement.

"Means for sending and displaying" refers to the function for sending the generated feedback to the user's device and presenting it visually.

"Visualization" refers to displaying data visually to make it easier to understand.

"Means for utilizing in development" refers to methods for supporting the training and development of players based on data.

This invention is a system that automatically generates running scores for basketball games and analyzes player performance. A specific embodiment of this system is described below.

System configuration

Hardware and software

1. Server

Hardware: Server with high-performance processor and large memory capacity

Software: Video analysis software (e.g. OpenCV), audio analysis software (e.g. Google Cloud Speech-to-Text), database management system (e.g. MySQL), sentiment analysis engine (e.g. IBM Watson Tone Analyzer)

2. Terminal

Hardware: PCs, tablets, smartphones, etc.

Software: Web browser, dedicated application

Program processing

Receiving and analyzing match data

The server receives video and audio data of the match. The video data is obtained from cameras and streaming services, and the audio data includes commentary and narration. The received video data is analyzed using OpenCV to detect scoring and foul scenes. The audio data is converted to text using Google Cloud Speech-to-Text, and information about scores and fouls is extracted from the commentary.

Extraction of scoring and foul situations

The server combines the results of video and audio analysis to extract the score and foul situations. This makes it possible to grasp the progress of the game in real time.

Generate and display running scores

The server generates a running score based on the extracted information. The generated running score is updated in real time and sent to the terminal using WebSocket or HTTP. The terminal displays the received running score to the user, allowing the user to check the progress of the match in real time.

Recognizing user emotions and adjusting score display

The server uses IBM Watson Tone Analyzer to recognize the user's emotions regarding the score of the game. Depending on the recognized emotion, the way the score is displayed is adjusted. For example, if the user is happy about a score, the score display is emphasized with bright colors or animations. The adjusted score display is sent to the terminal and displayed to the user.

Database of teams and players nationwide and performance analysis

The server collects match results and stats for teams and players across the country, and stores them in a MySQL database. Based on the stored information, the performance of players is analyzed using Python's Pandas library. Based on the analysis results, feedback for player development is generated and sent to the terminal. The terminal displays the received feedback to the user.

Specific examples

When the video data of the match is sent to the server, the server uses OpenCV to detect scoring scenes and uses Google Cloud Speech-to-Text to extract foul information from the commentary audio. For example, the prompt text can be "Analyze the scoring scenes and foul information of this match to generate a running score."

The server collects the results of matches from around the country and stores them in a MySQL database. It uses Python's Pandas library to analyze players' performance and generate feedback. For example, enter the following prompt statement: "Collect the results of matches from around the country, analyze players' performance, and generate feedback."

The server recognizes the user's joy at the scoring scene using IBM Watson Tone Analyzer, and adjusts the score display to emphasize the joy. For example, the prompt text is input as "Analyze the user's emotions at the scoring scene and adjust the score display." The flow of the specific process in Example 3 is explained with reference to FIG. 21.

Step 1:

Receive match data

The server receives video and audio data of the match. As input, it obtains video and audio data provided in real time from cameras and streaming services. As output, it stores the received video and audio data in its internal memory. Specifically, the server obtains the data through a network connection and stores it in a specified format.

Step 2:

Video data analysis

The server analyzes the received video data. As input, it uses the video data saved in step 1. As output, it obtains the results of detecting scoring and foul scenes. In concrete terms, the server uses OpenCV to analyze the video data frame by frame and executes an algorithm to detect specific events (scoring and fouls).

Step 3:

Analysis of audio data

The server analyzes the received audio data. As input, it uses the audio data stored in step 1. As output, it obtains text information extracted from the audio data. Specifically, the server converts the audio data into text using Google Cloud Speech-to-Text and extracts information about scores and fouls from the commentary.

Step 4:

Extraction of scoring and foul situations

The server integrates the video analysis results and audio analysis results to extract the scoring and foul situations. As input, it uses the outputs of steps 2 and 3. As output, it obtains data including the scoring and foul situations. In concrete terms, the server integrates the video analysis results and audio analysis results in a chronological order to identify the timing of scoring and fouls.

Step 5:

Generate running scores

The server generates a running score based on the extracted information. As input, it uses the score and foul status obtained in step 4. As output, it obtains data including the running score. Specifically, the server organizes the score and foul information in chronological order and compiles it in a scoreboard format.

Step 6:

Update and send running scores

The server updates the generated running score in real time and sends it to the terminal. As input, it uses the running score generated in step 5. As output, it sends the updated running score to the terminal. Specifically, the server sends data using WebSocket or HTTP.

Step 7:

Display running score

The terminal displays the received running score to the user. As input, it uses the running score sent in step 6. As output, it obtains a scoreboard that is visually presented to the user. In concrete terms, the terminal displays the scoreboard using a web browser or a dedicated application.

Step 8:

Recognizing user emotions

The server recognizes the user's emotions regarding the score of the game. As input, it uses the user's comments and reaction data. As output, it obtains the recognized emotion data. Specifically, the server analyzes the user's emotions using IBM Watson Tone Analyzer.

Step 9:

Score display adjustments

The server adjusts the way the score is displayed depending on the recognized emotion. As input, it uses the emotion data obtained in step 8 and the running score generated in step 5. As output, it obtains the adjusted score display data. As a specific operation, the server changes the color and animation of the score display.

Step 10:

Send adjusted score display

The server sends the adjusted score display to the terminal. As input, it uses the score display data adjusted in step 9. As output, it sends the adjusted score display to the terminal. Specifically, the server sends the data using WebSocket or HTTP.

Step 11:

Displaying the adjusted score

The terminal displays the adjusted score display to the user. As input, it uses the adjusted score display data sent in step 10. As output, it obtains the adjusted scoreboard that is visually presented to the user. In specific operation, the terminal displays the adjusted scoreboard using a web browser or a dedicated application.

Step 12:

Database of teams and players nationwide

The server collects match results and stats information for teams and players across the country and stores it in a database. As input, it uses data obtained from each team's official website and sports data services. As output, it obtains match results and stats information stored in the database. Specifically, the server organizes the data and stores it in a MySQL database.

Step 13:

Performance analysis

The server analyzes the players' performance based on the information stored in the database. As input, it uses the database information stored in step 12. As output, it obtains the analysis results that evaluate the players' performance and abilities. Specifically, the server processes the data using Python's Pandas library and evaluates the players' performance.

Step 14:

Generating feedback

The server generates feedback for player development based on the analysis results. As input, it uses the analysis results obtained in step 13. As output, it obtains a feedback report that points out the player's strengths and weaknesses and suggests areas for improvement. In concrete terms, the server creates the report and provides it to the player and coach.

Step 15:

Submit and view feedback

The server sends the generated feedback to the terminal, which displays it to the user. As input, it uses the feedback report generated in step 14. As output, it obtains a feedback report that is visually presented to the user. In concrete terms, the server sends the data via email or a dedicated application, and the terminal displays it.

(Application example 3)

Next, application example 3 of embodiment example 3 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Traditionally, live broadcasts and commentary of basketball games relied on human commentators, making it difficult to grasp the score and foul situation in real time. In addition, analysis of player performance and feedback on development were also done manually, which was inefficient. Furthermore, it was not possible to display scores that reflected the user's emotions, making it difficult to improve the viewing experience.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 3 is realized by the following means.

In this invention, the server includes: a means for AI to provide commentary on live coverage of basketball games; a means for distributing commentary on youth and mini-basketball games nationwide with AI live coverage; a means for AI to automatically generate running scores that are created manually for each official and team; a means for analyzing game video and audio data in real time and automatically generating the score situation and foul situation; a means for recognizing the user's emotions and displaying the score according to the emotions; and a means for providing feedback for player performance analysis and development using a database of teams and players nationwide. This makes it possible to grasp the progress of the game in real time and display scores that reflect the user's emotions, improving the efficiency of player performance analysis and development.

"A means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to provide real-time commentary on the progress and play of a basketball game.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games from around the country, and broadcasts live coverage including that commentary.

"A means of using AI to automatically generate running scores that are created manually by officials and for each team" refers to technology that uses artificial intelligence to automatically generate scores such as points and fouls as the game progresses.

"Means for analyzing game video and audio data in real time and automatically generating scoring and foul status" refers to technology that analyzes game video and audio data in real time and automatically generates scoring and foul status.

"Means for recognizing the user's emotions and displaying a score according to those emotions" refers to technology that recognizes emotions from the user's facial expressions and voice, and adjusts the score display according to those emotions.

"Means of utilizing a database of teams and players nationwide to analyze player performance and provide feedback for development" refers to technology that utilizes a database of basketball teams and players nationwide to analyze player performance and provide feedback for development.

The system for implementing this invention uses the following hardware and software.

Hardware and software used

Hardware: Smartphone, head-mounted display

Software: TensorFlow (AI model), OpenCV (video analysis), NLTK (natural language processing), Emotion API (emotion recognition)

System configuration

The server includes the following means:

1. How AI can provide commentary for live broadcasts of basketball games:

The server receives video and audio data from the match in real time, and uses TensorFlow to analyze the progress of the match and the content of the play, generating commentary.

2. How to broadcast live commentary of youth and mini-basketball games nationwide with AI:

The server receives footage of youth and mini-basketball matches from across the country, and similarly uses AI to generate commentary, which it distributes in real time.

3. A way to use AI to automatically generate running scores that are currently created manually by officials and teams:

The server analyzes the video and audio data of the match and automatically generates the score and foul situations.

4. A method for analyzing game video and audio data in real time and automatically generating scoring and foul situations:

The server uses OpenCV to split the video data into frames and extract important scenes. It uses NLTK to convert the audio data into text and extract important keywords.

5. How to recognize user emotions and display a score according to those emotions:

The server uses the Emotion API to recognize emotions from the user's facial expressions and voice, and adjusts the score display accordingly.

6. Using a national team and player database to analyze player performance and provide feedback for development:

The server accesses a database of teams and players from across the country, displays analysis of player performance, and provides feedback for the next match.

Specific examples

When a user using a smartphone during a match sees a goal being scored, the server uses AI to analyze the scene and display the score in real time. If the user shows an expression of joy, the server will recognize it using an emotion engine and make the score display more colorful. After the match ends, the server also refers to a nationwide database to display an analysis of the players' performance and provide feedback for the next match.

Example of a prompt

"Please analyze the video and audio data of the game in real time, and use an AI model that automatically generates the score and foul situation to display a running score. Also, please recognize the user's emotions and display the score according to those emotions. Furthermore, please use a database of teams and players from all over the country to provide feedback for analyzing player performance and development."

The flow of the specific processing in application example 3 is explained using Figure 22.

Step 1:

Acquisition of game video and audio data

The server receives video and audio data of the game in real time from a smartphone or head-mounted display. The input is the video and audio data of the game, and the output is preprocessed data for analysis. In concrete terms, the server receives the data stream, splits the video data into frames, and converts the audio data into text.

Step 2:

Data preprocessing

The server uses OpenCV to split the video data into frames and extract important scenes. It also uses NLTK to convert the audio data into text and extracts important keywords (e.g., goals, fouls). The input is the preprocessed video and audio data, and the output is important scenes and keywords for analysis. Specifically, the server analyzes video frames and identifies scoring and foul scenes.

Step 3:

Analysis using AI models

The server uses TensorFlow to analyze the scoring and foul situations from video and audio data. The input is important scenes and keywords, and the output is data on the scoring and foul situations. In concrete terms, the server runs an AI model and automatically determines the scores and fouls for each scene.

Step 4:

Automatic generation of running scores

The server automatically generates a running score based on the analysis results. The input is data on the scoring and foul situations, and the output is a real-time running score. Specifically, the server tallys up the score and foul information and updates the scoreboard.

Step 5:

Emotion recognition

The server uses the Emotion API to recognize emotions from the user's facial expressions and voice. The input is the user's facial expression data and voice data, and the output is the recognized emotion data. In concrete terms, the server obtains user data from the camera and microphone and analyzes emotions.

Step 6:

Score display based on emotions

The server adjusts the score display according to the recognized emotion. The input is emotion data and the running score, and the output is a score display according to the emotion. In concrete terms, the server changes the design and effects of the score display based on the user's emotion.

Step 7:

Utilizing the database

The server refers to a database of teams and players from around the country, displays the results of player performance analysis, and provides feedback for the next match. The input is the match results and player data, and the output is performance analysis results and feedback. In concrete terms, the server retrieves relevant information from the database and displays the analysis results.

The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input for the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart glasses 214.

[Third embodiment]

Figure 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

As shown in FIG. 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical able-bodied person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is performed in a secure state.

Figure 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Figure 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. Data generation model 58 and emotion identification model 59 are used by the identification processing unit 290.

In the headset terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the specific processing performed by the specific processing unit 290 of the data processing device 12.

"Example 1"

Example 1 of the present invention is a system in which AI provides commentary for a live broadcast of a basketball game. In this system, AI analyzes the video and audio data of the game to understand the situation of the game and the movements of the players. Then, based on the results of this analysis, a commentary of the game is automatically generated and distributed along with the live broadcast.

"Example 2"

Example 2 of the present invention is a system that distributes commentary of youth and mini-basketball games from around the country, with live coverage by AI. This system collects video and audio data from youth and mini-basketball games held around the country, which are analyzed by AI in the same way as example 1 to generate commentary. Then, by distributing the commentary together with the game footage, games from around the country are made available to a wide audience.

"Example 3"

Example 3 of the present invention is a system that uses AI to automatically generate running scores that are created manually by officials or for each team. In this system, AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game, and automatically generates a running score. This makes it possible to grasp the progress of the game in real time.

"Example 4"

Example 4 of the present invention is a system that creates a database of teams and players from across the country, and uses the information in the database to visualize and develop registered basketball players in Japan. This system collects information such as game results and stats for teams and players from across the country, and creates a database. The database is then used to provide feedback for analyzing players' performance and developing them.

The process flow for each example is explained below.

"Example 1"

Step 1: AI collects video and audio data from basketball games.

Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

Step 3: Based on the analysis results, the AI automatically generates commentary for the match.

Step 4: Distribute the generated commentary along with the live broadcast.

"Example 2"

Step 1: Collect video and audio data from youth and mini-basketball matches held across the country.

Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

Step 3: Based on the analysis results, the AI automatically generates commentary for the match.

Step 4: Distribute the generated commentary along with the game footage.

"Example 3"

Step 1: AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game.

Step 2: Based on the analysis results, the AI automatically generates a running score.

"Example 4"

Step 1: Collect information such as match results and stats for teams and players across the country.

Step 2: Create a database of the collected information.

Step 3: Use the databased information to analyze player performance and provide feedback for development.

(Example 1)

Next, a first embodiment of the first embodiment will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

Traditional live broadcasts of basketball games depend on the quality and quantity of commentators, making it difficult to accurately convey the detailed situation of the game and the movements of the players in real time. In addition, commentary on youth and mini-basketball games across the country is required by many people, which is costly and labor intensive. Furthermore, running scores created by officials and people for each team are done manually, so they take time and can lack accuracy. To solve these issues, an automated system that utilizes AI is needed.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for collecting video data and audio data of the match in real time, means for analyzing the collected video data and audio data, means for generating commentary on the match based on the analysis results, and means for distributing the generated commentary together with a live broadcast. This makes it possible to accurately convey the detailed situation of the match and the movements of the players in real time. In addition, commentary on youth and mini-basketball matches from all over the country can be distributed with live broadcasts by AI, reducing costs and labor. Furthermore, by automatically generating running scores by AI instead of manually created by officials and for each team, it is possible to reduce time and improve accuracy.

"Game video data" refers to digital data that visually records the progress of a basketball game.

"Audio data" refers to data that is a digital recording of the sounds generated during a basketball game.

"Means of collecting data in real time" refers to technology and devices that allow for the acquisition of video and audio data as the match progresses.

"Means of analysis" refers to technology and software that processes collected video and audio data to understand the situation of the game and the movements of the players.

"Means for generating commentary" refers to technology or software that automatically creates text that explains the situation of the game and the movements of players based on the analysis results.

"Means of distribution along with live broadcast" refers to the technology and devices that deliver the generated commentary to viewers in real time along with the video and audio of the match.

"Youth and mini-basketball game commentary" is commentary on basketball games aimed at young people and elementary school students.

"Running score" is data that records the scores and player performances that are updated in real time during the game.

"AI automatic generation means" refers to technology and software that uses artificial intelligence to automate manual tasks and generate data.

"Means of database creation" refers to technology and software for organizing collected information and storing it in a format that is easy to search and use.

"Visualization" refers to techniques and methods for visually displaying data and making it easier to understand.

"Means to be used in development" refers to technologies and methods that use data to support the training and development of athletes.

This invention is a system that uses AI to automatically generate commentary on a basketball game and deliver it in real time during live broadcasts. A specific embodiment of this system is described below.

Server operation

1. Data collection

The server collects video and audio data of the match in real time. Specifically, it uses a camera and microphone to capture video and audio of the match. Video data is collected at a resolution of 1080p, and audio data is collected at a sampling rate of 44.1kHz.

2. Data analysis

The server analyzes the collected video and audio data. This analysis uses image recognition software (e.g. OpenCV) and voice recognition software (e.g. Google Speech-to-Text API). This allows the server to understand the situation of the game and the movements of the players. For example, it detects the action of Player A taking a shot and converts the live audio of that moment into text.

3. Description generation

The server generates commentary on the game based on the analysis results. A generative AI model (e.g. GPT-4) is used to generate this commentary. The generative AI model receives information about the game situation and player movements as input, and generates commentary in natural language. For example, it inputs information such as "Player A made a successful three-point shot" and generates commentary such as "Player A made a brilliant three-point shot."

4. Distribution

The server distributes the generated commentary along with the live broadcast. A streaming server (e.g. Wowza Streaming Engine) is used for distribution. This allows users to watch the game footage and commentary in real time over the Internet.

Device operation

1. Data reception

The device receives real-time game footage and commentary from the server using an internet connection and streaming software (e.g. VLC media player).

2. Display

The device displays the received video and commentary to the user. For display purposes, hardware such as a display and speakers are used. For example, if the user is watching on a smartphone, the video will be displayed on the smartphone screen and the commentary audio will be played through the built-in speaker.

User actions

1. Viewing

The user watches the game footage and commentary through the terminal. The user can get detailed commentary on the progress of the game and the movements of the players in real time. For example, if the user is watching on a television in the living room, the game footage will be displayed on the television screen and commentary will be played from the television speakers.

Specific examples

Example 1

Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

Player Action: Home team player A makes a three-point shot.

Example prompts for generative AI models

Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

Player Action: Home team player A makes a three-point shot.

Example of generated description

"With five minutes left in the third quarter, the home team is tied at 75 points and the away team at 70 points. Home team player A makes a great three-point shot, widening the gap even further."

In this way, users can watch detailed commentary on the progress of the match and the players' movements in real time.

The flow of the identification process in Example 1 is explained using Figure 11.

Step 1:

The server collects video and audio data from the match in real time.

Specific tasks include using cameras and microphones to capture video and audio of the game.

The input is video and audio from the camera and microphone, and the output is video and audio data in digital format.

Video data is collected at 1080p resolution, and audio data is collected at a sampling rate of 44.1kHz.

Step 2:

The server analyzes the collected video and audio data.

Specific operations include using image recognition software (e.g. OpenCV) to detect player movements from video data, and using voice recognition software (e.g. Google Speech-to-Text API) to convert the audio data into text to provide commentary.

The input is the video and audio data collected in step 1, and the output is the analyzed player movement information and the text commentary.

For example, it detects when Player A is about to take a shot and converts the audio commentary at that moment into text.

Step 3:

The server generates commentary for the match based on the analysis results.

Specifically, the prompt text is input into a generative AI model (e.g. GPT-4) to generate an explanatory text.

The input is the player movement information obtained in step 2 and the text commentary, and the output is the generated commentary.

For example, input information such as "Player A made a successful three-point shot" and generate an explanatory text such as "Player A made a brilliant three-point shot."

Step 4:

The server distributes the generated commentary along with the live broadcast.

Specifically, the video and commentary audio are streamed in real time using a streaming server (e.g. Wowza Streaming Engine).

The input is the explanatory text generated in step 3 and the video data collected in step 1, and the output is the video and explanatory audio delivered in real time.

Step 5:

The device receives game footage and commentary streamed from the server in real time.

Specific actions include using an internet connection and streaming software (e.g. VLC media player).

The input is streaming data from the server, and the output is the video displayed on the terminal and the commentary audio played.

Step 6:

The device displays the received video and commentary to the user.

Specific operations include showing images on the display and playing explanatory audio from the speaker.

The input is the video and commentary audio received in step 5, and the output is a live broadcast of the game provided to the user in both visual and audio.

Step 7:

Users can watch game footage and commentary through their devices.

Specific actions include watching the game footage on the device display and listening to commentary coming from the speaker.

The input is the video displayed in step 6 and the commentary audio played, and the output is the user's viewing experience.

For example, if a user is watching on a television in their living room, the game footage will be displayed on the television screen and commentary will be played from the television speakers.

(Application example 1)

Next, application example 1 of embodiment example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

Existing basketball game commentary systems do not adequately provide real-time commentary, post-game highlight generation, or detailed analysis of players' performance. In addition, there is a problem that it is difficult to provide appropriate commentary immediately when a user asks about a specific player or play.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 1 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games nationwide with AI commentary, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for analyzing game video data and generating commentary in real time, a means for automatically extracting important scenes after the game ends and generating a highlight video, a means for analyzing player movements and performance and providing detailed statistical information, and a means for AI to provide commentary when a user asks a question about a specific player or play. This makes it possible to provide commentary in real time, generate post-game highlights, analyze players' performance in detail, and provide immediate commentary in response to user questions.

"A means for AI to provide commentary for live broadcasts of basketball games" refers to technology in which AI analyzes video and audio data from basketball games, understands the situation of the game and the movements of the players, and automatically generates commentary.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that uses AI to provide commentary on youth and mini-basketball games from around the country and broadcasts the games in real time with that commentary.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses AI to automatically generate scores that are manually recorded by officials and teams during a match.

"Means for analyzing game video data and generating commentary in real time" refers to technology that analyzes game video data in real time and instantly generates commentary based on the analysis results.

"Means for automatically extracting important scenes after the end of a match and generating a highlight video" refers to a technology that analyzes video data of a match after the end of the match, automatically extracts important scenes, and generates a highlight video.

"Means of analyzing player movements and performance and providing detailed statistical information" refers to technology that analyzes player movements and performance and provides detailed statistical information based on the results.

"A means for an AI to provide commentary when a user asks a question about a specific player or play" refers to technology that enables an AI to provide an appropriate commentary in response to a question when a user asks about a specific player or play.

To implement this invention, the following system configuration and processing steps are required.

System configuration

Hardware

Server: A high-performance computer for running AI models and analyzing data.

Smartphone: The device on which users watch and interact with the game commentary.

Camera: A device used to capture footage of the match.

Software

OpenCV: A library used to read and analyze video data.

TensorFlow: A framework used to run AI models.

Transformers: A library that generates explanations using the GPT-2 model.

Processing flow

1. Loading video data

The server uses OpenCV to read the game video data obtained from the camera frame by frame.

2. Frame analysis

The server analyzes the loaded frames to detect player movements and important actions. This analysis uses a deep learning model using TensorFlow.

3. Generate explanations

The server generates an explanatory text based on the analysis results. Specifically, it runs the GPT-2 model using the Transformers library and generates an explanatory text using the prompt text as input.

4. Delivering commentary

The generated commentary is delivered to smartphones in real time, allowing users to watch and listen to commentary on the match via their smartphones.

5. Highlight video generation

After the match ends, the server automatically extracts important scenes and generates a highlight video. This highlight video is also distributed to smartphones.

6. Player performance analysis

The server performs detailed analysis of the players' movements and performance and generates statistical information that users can view on their smartphones.

7. User interaction

When a user asks a question about a particular player or play, the server generates an appropriate commentary for that question and delivers it to the smartphone.

Specific examples

Scoring scene in the game: Detect the moment when Player A makes a three-point shot and generate a commentary such as "Player A scores a three-pointer."

Defensive highlights: Detects a scene where Player B successfully blocks a shot and generates a commentary such as "Player B makes a great block."

Example of a prompt

"Player A scores a three-pointer. Generate a commentary for this action."

"Player B makes a great block. Generate a commentary for this action."

In this way, the server can analyze the game video data in real time and automatically provide commentary to the user. This allows for real-time commentary, post-game highlights, detailed analysis of player performance, and instant commentary in response to user questions.

The flow of the specific processing in application example 1 is explained using Figure 12.

Step 1:

The server uses OpenCV to read the game video data captured by the camera frame by frame. The input is the video data from the camera, and the output is the individual frame images. Specifically, the server splits the video data into a series of still images and stores each frame in memory.

Step 2:

The server analyzes the loaded frames to detect player movements and important actions. A deep learning model using TensorFlow is used for this analysis. The input is the frame image obtained in step 1, and the output is information about the player movements and actions. Specifically, the server detects the positions and movements of players within the frames and identifies important actions such as points and blocks.

Step 3:

The server generates a commentary based on the analysis results. Specifically, it runs the GPT-2 model using the Transformers library and generates a commentary using the prompt as input. The input is the action information and prompt obtained in step 2, and the output is the generated commentary. The server generates a prompt corresponding to the player's action and inputs it into the GPT-2 model to obtain the commentary.

Step 4:

The server delivers the generated commentary to the smartphone in real time. The input is the commentary generated in step 3, and the output is the commentary displayed on the smartphone. Specifically, the server transmits the commentary to the smartphone via the network so that the user can view it.

Step 5:

After the match ends, the server automatically extracts important scenes and generates a highlight video. The input is the video data of the entire match, and the output is the highlight video. Specifically, the server selects highlight scenes based on important actions detected during the match and links them together to create a highlight video.

Step 6:

The server performs detailed analysis of players' movements and performance and generates statistical information. The input is frame images from the game and information about players' actions, and the output is detailed statistical information. Specifically, the server compiles data such as points, rebounds, and assists for each player and compiles it as statistical information.

Step 7:

When a user asks a question about a specific player or play, the server generates an appropriate commentary for that question and delivers it to the smartphone. The input is the user's question and prompt text, and the output is the generated commentary text. Specifically, the server analyzes the user's question, generates a corresponding prompt text, inputs it into the GPT-2 model, obtains the commentary text, and sends it to the smartphone.

(Example 2)

Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

Traditionally, commentary on basketball games required a specialized commentator, and there was the problem of difficulty in securing commentators for youth and mini-basketball games held all over the country. It was also difficult to convey the progress of the game and the movements of the players to viewers in real time, limiting the viewing experience. Furthermore, the collection and analysis of game data was done manually, which was inefficient and sometimes lacked consistency and accuracy.

The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

In this invention, the server includes means for collecting video and audio data of matches held throughout the country, means for storing the collected data in cloud storage, means for transmitting the stored data to a generative AI model and generating commentary for the match, means for synchronizing the generated commentary with the match video, means for streaming the match video with commentary, means for providing an interface for users to view, means for receiving the match video with commentary streamed from the server, and means for displaying the received video to the user. This enables live broadcasts with high-quality commentary by AI even for youth and mini-basketball matches held throughout the country, improving the viewing experience. In addition, efficient and accurate data management can be achieved because the collection and analysis of match data is automated.

"Means for collecting video and audio data from matches held around the country" refers to devices and systems that use cameras and microphones installed at match venues around the country to capture video and audio data from matches in real time and transmit it to a server.

"Means for storing collected data in cloud storage" refers to devices or systems that convert collected video and audio data into an appropriate format and store the data using cloud services.

"Means for sending stored data to a generative AI model and generating commentary on a match" refers to a device or system that retrieves data stored in cloud storage, sends prompt text to the generative AI model to analyze the data, and generates commentary on a match.

"Means for synchronizing the generated commentary with the game video" refers to a device or system that inserts the generated commentary text at a specific timestamp in the game video, allowing the video and commentary to be played in a synchronized state.

"Means for streaming game footage with commentary" refers to a device or system that encodes synchronized video data in real time and transmits it to a streaming platform for distribution to viewers.

"Means for providing an interface for users to view" refers to a device or system for displaying a viewing interface to users through a device such as a smartphone, tablet, or PC.

"Means for receiving game footage with commentary streamed from a server" refers to a device or system for receiving game footage with commentary streamed from a server in real time via an Internet connection.

"Means for displaying received video to a user" refers to a device or system for decoding received video data and displaying it on the user's device screen.

This invention is a system that broadcasts youth and mini-basketball matches held around the country with commentary by AI. A specific embodiment of this system is described below.

Server role

The server collects video and audio data from matches held around the country. Cameras and microphones installed at match venues around the country capture video and audio from matches in real time and send this data to the server. The collected data is stored in cloud storage such as Amazon S3 and Google Cloud Storage.

The server then sends the stored data to a generative AI model. Examples of generative AI models that can be used include OpenAI's GPT-4 and Google's BERT. The server retrieves the data from cloud storage and sends a prompt to the generative AI model. The prompt includes instructions for analyzing the progress of the game, the movements of players, the score, etc.

The generative AI model analyzes the data based on the prompt text and generates commentary in natural language. The generated commentary is sent back to the server in text format. The server then synchronizes the generated commentary with the game footage. This synchronization process is performed using video editing software such as FFmpeg. The server inserts the commentary text at specific timestamps in the footage, allowing the footage and commentary to be played in sync.

Finally, the server streams the match footage with commentary to a streaming platform such as YouTube Live or Twitch. The server encodes the synchronized video data in real time and sends it to the streaming platform.

Role of the device

The terminal provides the user with an interface for viewing. Devices such as smartphones, tablets, and PCs are used here. The terminal displays the viewing interface to the user through a dedicated application or web browser. The interface includes a list of matches and a search function.

The device receives game footage with commentary streamed from the server. An Internet connection is required for this reception. The device receives video data from the streaming platform in real time and buffers it. The device decodes the received video data and displays it on the user's device screen.

User Roles

Users access the system using a terminal. To do so, they use a dedicated application or a web browser. Users launch a dedicated application on their smartphone or PC, or access the system's URL using a web browser.

The user selects the match they wish to watch. For this selection, a list of matches and a search function are provided. The user scrolls through the list of matches on the interface and clicks on the match they wish to watch. They can also use the search function to find a specific match.

The user watches the commentary-enhanced video of the selected match. While watching, the user can play, pause, rewind, and perform other operations. The user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed.

Specific examples

For example, if a user wants to watch a national youth basketball tournament game on their smartphone, they would follow these steps:

1. The user launches the dedicated app on their smartphone.

2. Use the in-app search function and type in "National Youth Basketball Tournament."

3. Select the match you want to watch from the search results.

4. The app streams game footage with commentary from the server.

5. Users can watch the game on their smartphone screen and enjoy the commentary.

An example of a prompt sentence to be input to the generative AI model is, "Analyze the game video and audio data of the National Youth Basketball Tournament and generate commentary including the progress of the game, player movements, score status, etc." Using this prompt sentence, the AI analysis system generates commentary of the game.

The flow of the identification process in Example 2 is explained using Figure 13.

Step 1:

The server collects video and audio data from matches held around the country. As input, it receives real-time data from cameras and microphones installed at match venues around the country. Specifically, it transmits the video and audio captured by these devices to the server via the internet. As output, it obtains the collected video and audio data.

Step 2:

The server stores the collected data in cloud storage. As input, it receives the video and audio data collected in step 1. Specifically, it converts the data into an appropriate format and uploads it to a cloud service such as Amazon S3 or Google Cloud Storage. As output, it obtains the data stored in cloud storage.

Step 3:

The server sends the saved data to the generative AI model to generate commentary on the match. As input, it receives video and audio data obtained from cloud storage. Specifically, it sends a prompt to the generative AI model to analyze the data. The prompt includes instructions to analyze the progress of the match, player movements, the score, etc. The output is the commentary text returned from the generative AI model.

Step 4:

The server synchronizes the generated commentary with the game footage. As input, it receives the commentary text obtained in step 3 and the video data obtained from the cloud storage. Specifically, it uses video editing software such as FFmpeg to insert the commentary text at a specific timestamp in the video. As output, it obtains the game footage synchronized with the commentary.

Step 5:

The server streams the game footage with commentary. As input, it receives the game footage synchronized with the commentary obtained in step 4. Specifically, it encodes the video data in real time and transmits it to a streaming platform such as YouTube Live or Twitch. As output, it obtains the streaming video that is distributed to the viewers.

Step 6:

The terminal provides an interface for users to watch. It receives user operations as input. Its specific operation is to display the viewing interface through a dedicated application or web browser. The interface includes a list of matches and a search function. As output, it provides information that allows the user to select the match they wish to watch.

Step 7:

The terminal receives game footage with commentary that is streamed from the server. As input, it receives streaming data from the server. Specifically, it receives video data in real time via an Internet connection and buffers it. As output, it obtains the received video data.

Step 8:

The terminal displays the received video to the user. As input, it receives the video data received in step 7. Specifically, it decodes the video data and displays it on the user's device screen. As output, it obtains the video that the user watches.

Step 9:

The user accesses the system using a terminal. A dedicated application or a web browser is used as input. Specifically, the user launches a dedicated application on a smartphone or PC, or accesses the system's URL in a web browser. Access to the system is obtained as output.

Step 10:

The user selects the match they want to watch. As input, they use the match list and search functionality on the viewing interface. They scroll through the match list and click on the match they want to watch. They can also use the search functionality to search for a specific match. As output, they get the selected match.

Step 11:

The user watches the commentary-enhanced video of the selected match. As input, the match selected in step 10 is received. Specifically, the user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed. As output, the video of the match being watched is obtained.

(Application example 2)

Next, application example 2 of embodiment example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

In conventional basketball game commentary systems, live broadcasts and commentary of games were dependent on human labor, making it difficult to provide youth and mini-basketball games held all over the country to a wide audience. In addition, there was a lack of game highlights and replay functions, and post-game statistical data and analysis reports, making it difficult to increase viewer satisfaction. Furthermore, there was no function for users to follow their favorite teams or players, making it difficult to meet individual needs.

The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for AI to provide commentary on the live broadcast of a basketball game, a means for distributing commentary of youth and mini-basketball games nationwide with AI live broadcast, a means for AI to automatically generate running scores created by officials and people for each team, a means for collecting video and audio data of the game and analyzing it with AI to generate commentary, a means for distributing the game in real time through an application installed on a smartphone and providing commentary by AI, a means for providing highlights and replay functions of the game, a means for providing a function that allows users to follow their favorite teams and players, and a means for providing statistical data and analysis reports after the game. This makes it possible to widely provide youth and mini-basketball games held all over the country to viewers and increase the satisfaction of the viewers.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to generate commentary in real time during a basketball game and provide it to viewers.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to technology that broadcasts youth and mini-basketball games held around the country in real time with commentary by artificial intelligence.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Method of collecting video and audio data from the match and analyzing it with AI to generate commentary" refers to technology that collects video and audio data from the match and uses artificial intelligence to analyze it and generate commentary.

"Means of broadcasting matches in real time and providing commentary by AI through an application installed on a smartphone" refers to a technology that uses a smartphone application to broadcast matches in real time and provide commentary by artificial intelligence.

"Means for providing match highlight and replay features" refers to technology that provides the functionality to compile important scenes and plays during a match as highlights or play them back as replays.

"Means for providing a function that enables users to follow their favorite teams and players" refers to technology that provides a function that enables users to select and follow their favorite teams and players and receive information about them on a priority basis.

"Means for providing post-match statistical data and analytical reports" refers to technology that generates match statistical data and analytical reports after the match ends and provides them to users.

The following system configuration is described as a form for implementing this invention.

System configuration

The system includes a server, a smartphone, and an AI model. The server collects video and audio data from the match, analyzes it using the AI model, and generates commentary. The smartphone provides an application that enables users to watch the match in real time and receive commentary by the AI.

Hardware and software used

Server: A server with high-performance data processing capabilities (e.g. AWS EC2 instance)

Smartphone: iOS or Android device

AI model: OpenAI's GPT-4

Data processing library: OpenCV (for image processing), Python (programming language)

Data processing and data calculations

1. Collecting video and audio data from the match:

The server collects video and audio data in real time from youth and mini-basketball matches held across the country. This is done using input devices such as cameras and microphones.

2. AI-based analysis and explanation generation:

The server uses OpenCV to analyze the collected video and audio data and extracts important scenes and plays from the match. Based on the extracted information, it uses OpenAI's GPT-4 to generate commentary. Specifically, it uses the following prompt sentence:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

3. Real-time broadcasting and commentary:

The server delivers the game footage along with the generated commentary to smartphones in real time. An application installed on the smartphone allows users to receive the AI commentary while watching the game.

4. Providing highlight and replay features:

The server compiles highlights of key scenes during the match and provides a replay function, allowing users to rewatch key moments of the match.

5. User following feature:

The smartphone application provides users with the ability to follow their favorite teams and players. This allows users to receive information about the teams and players they follow preferentially.

6. Providing post-match statistics and analysis reports:

After the match ends, the server generates statistical data and analysis reports and provides them to users via a smartphone application, allowing users to obtain detailed analysis information about the match.

Specific examples

For example, when a user is watching a youth basketball game through a smartphone application, the server delivers real-time game footage and AI commentary. The moment Player A scores a goal during the game, the server analyzes the scene and generates the following prompt:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

Based on this prompt, the AI model will generate commentary and provide it to the user. Users can enjoy the AI commentary in real time while watching the match. In addition, after the match, users can obtain detailed information about the match through statistical data and analysis reports.

The flow of the specific processing in application example 2 is explained using Figure 14.

Step 1:

The server collects video and audio data from youth and mini-basketball matches held around the country. As input, it receives real-time data from input devices such as cameras and microphones, and as output, it stores this data in storage. In concrete terms, the server receives video and audio data sent from each match venue, converts it into an appropriate format, and stores it.

Step 2:

The server analyzes the collected video and audio data. It uses the video and audio data saved in step 1 as input, and extracts information about important scenes and plays in the match as output. Specifically, the server uses OpenCV to analyze the video data frame by frame and identify the movements of players and the position of the ball. It also extracts events such as goals and fouls from the audio data.

Step 3:

The server generates a prompt sentence based on the analysis results. As input, it uses information about important scenes and plays in the match extracted in step 2, and as output, it generates a prompt sentence to pass to the AI model. Specifically, the server converts information such as player movements and scoring status into text format and generates a prompt sentence like the following:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

Step 4:

The server sends the generated prompt text to the AI model and generates an explanation. It uses the prompt text generated in step 3 as input and receives the explanation text generated from the AI model as output. In concrete terms, the server calls OpenAI's GPT-4 API, sends the prompt text, and generates an explanation.

Step 5:

The server delivers the generated commentary and game footage to smartphones in real time. It uses the commentary text generated in step 4 and the video data collected in step 1 as input, and generates data to be delivered to the user's smartphone as output. In concrete terms, the server overlays the commentary text on the video data and transmits it to the smartphone in streaming format.

Step 6:

Users watch the game through a smartphone application and receive commentary by AI. As input, it uses video and commentary data distributed from the server, and as output, it displays the game video and commentary to the user. In concrete terms, the smartphone application decodes the received data and displays it on the screen.

Step 7:

The server summarizes important scenes during the match as highlights and provides a replay function. It uses the information of important scenes extracted in step 2 as input and generates highlight videos as output. In concrete terms, the server edits the important scenes and saves them in a format that users can replay.

Step 8:

Users follow their favorite teams and players through a smartphone application. The application uses the user's selection information as input, and prioritizes displaying information about the followed teams and players as output. In concrete terms, the application records the user's selection, and prioritizes retrieving and displaying related information.

Step 9:

After the match ends, the server generates statistical data and analysis reports and provides them to the user. It uses data collected during the match as input and generates statistical data and analysis reports as output. In concrete terms, the server aggregates the match data, creates a report in a visually easy-to-understand format, and provides it to the user through a smartphone application.

(Example 3)

Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

In traditional basketball games, in order to grasp the score and foul situation in real time, it was necessary to manually create a running score, which was time-consuming and labor-intensive. In addition, there was a lack of a system to effectively collect and analyze data on teams and players across the country and use it for player development. This meant that there was insufficient visualization of player performance and feedback for development.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes a means for receiving video data and audio data of a match, a means for analyzing the received video data, a means for analyzing the received audio data, a means for extracting the score situation and the foul situation from the analysis result, a means for generating a running score based on the extracted information, a means for updating the generated running score in real time and transmitting it to the terminal, a means for displaying the running score transmitted to the terminal, a means for collecting information such as the match results and stats of teams and players nationwide, a means for database-izing the collected information, a means for analyzing the databased information and evaluating the performance of players, a means for generating feedback for player development based on the analysis result, a means for transmitting the generated feedback to the terminal, and a means for displaying the feedback transmitted to the terminal. This makes it possible to grasp the progress of the match in real time, and effectively visualize the performance of players and provide feedback for development.

"Game video data" refers to video files captured by a camera showing a basketball game.

"Audio data" refers to audio files recorded by a microphone during a basketball game.

"Means of receiving" refers to the function that allows the server to obtain data from outside.

"Means of analysis" refers to the function for processing received data and extracting necessary information.

"Scoring status" is information that shows how many points each team has scored during a match.

"Foul situation" is information that shows which players committed what type of fouls during the game.

The "running score" is a record of points and fouls that is updated in real time as the game progresses.

"Means of updating in real time" refers to a function that instantly reflects data as the match progresses.

"Terminal" refers to the device that a user uses to view information.

"Means of display" refers to the functionality for visually presenting information on a terminal.

"Match results" refers to information showing the final score and outcome after the match has ended.

"Stats" are statistical data that quantify the performance of players and teams.

"Means of collection" refers to the function for collecting the necessary data.

"Means of database creation" refers to the function of organizing and storing collected data.

"Means of analysis" refers to the function of processing databased information and deriving meaningful results.

"Feedback" refers to advice or evaluation provided based on the analysis results.

"Visualization" refers to displaying data visually to make it easier to understand.

"Development" refers to training and guidance to improve players' abilities.

This invention is a system for grasping the score and foul situation in a basketball game in real time and evaluating and developing the performance of players. A specific embodiment of this system is described below.

System configuration

Hardware

Server: A computer with high-performance computing power that receives, analyzes, generates, and transmits data.

Device: The device on which the user checks information, including smartphones, tablets, and computers.

Camera: A device used to capture video data of a match.

Microphone: A device used to capture audio data from a match.

Software

Video analysis software: Analyzes video data using libraries such as OpenCV.

Speech analysis software: Use libraries such as LibROSA to analyze speech data.

Database management system: Collected data is stored in a database using a system such as MySQL.

Data analysis tools: Analyze data using tools such as Pandas and NumPy.

Generative AI models: Use models such as GPT-4 to generate running scores and feedback.

System operation

Data reception and analysis

The server receives video and audio data of the game in real time from the cameras and microphones. The received video data is analyzed using video analysis software to detect goals and fouls. The audio data is analyzed using audio analysis software to analyze the commentary and crowd reactions.

Information extraction and running score generation

The server extracts the scoring and foul situations from the analysis results. Based on the extracted information, a generative AI model is used to generate a running score. The generated running score is updated in real time and sent to the terminal.

Display data

The device displays the received running score, allowing the user to follow the progress of the match in real time.

Data collection and database creation

The server collects information such as match results and stats for teams and players across the country. The collected information is stored in a database using a database management system.

Data analysis and feedback generation

The server analyzes the databased information and evaluates the player's performance. Based on the analysis results, a generative AI model is used to generate feedback for the player's development. The generated feedback is sent to the terminal, which displays it. This allows the user to check the analysis results of the player's performance and advice for development.

Specific examples

When a user is watching a basketball game, the server analyzes the video and audio of the game and automatically extracts information about scores and fouls.

The server generates information in real time, such as "Team A has scored" or "Team B has committed a foul," and sends it to the terminal.

The device displays this information, allowing users to check the progress of the match in real time.

Example of a prompt

"Analyze video and audio data from basketball games, extract the scoring and foul situations, and generate a running score."

"Analyze the game results and stats of basketball teams and players nationwide, evaluate the performance of players, and generate feedback for training." The flow of the specific process in Example 3 is explained with reference to FIG. 15.

Step 1:

The server receives video and audio data from the match.

Input: Real-time data from camera and microphone

Specific operation: Receives video and audio transmitted from the camera and microphone via the network.

Output: Received video and audio data

Step 2:

The server analyzes the received video data.

Input: Received video data

Specific operation: Use video analysis software (e.g. OpenCV) to detect scoring and foul scenes from video data.

Output: Data with timestamps for goals and fouls

Step 3:

The server analyzes the received audio data.

Input: Received audio data

Specific actions: Use audio analysis software (e.g. LibROSA) to analyze commentary and audience reactions from audio data.

Output: Timestamped data of commentary and audience reactions

Step 4:

The server extracts the scoring and foul situations from the analysis results.

Input: Timestamped data on goals and fouls, as well as timestamped data on commentary and crowd reactions

Specific operation: Integrate the results of video and audio analysis to extract the occurrence of goals and fouls.

Output: Scoring and foul situation data

Step 5:

The server generates a running score based on the extracted information.

Input: Scoring and foul situation data

Specific behavior: Uses a generative AI model (e.g. GPT-4) to generate points and foul information in text format.

Output: Text data of running score

Step 6:

The server updates the generated running score in real time and sends it to the device.

Input: Text data of running score

Specific operation: Sends the running score to the device using WebSocket or HTTP protocol.

Output: Running score sent to the device

Step 7:

The device will display the received running score.

Input: Running score sent to the device

Specific operation: Visually display the running score on the device display.

Output: A running score that the user can visually check

Step 8:

The server collects information such as match results and stats for teams and players across the country.

Input: Data from each team's official website and sports data services

Specific actions: Collect match results and stats using web scraping and APIs.

Output: Collected match results and stats data

Step 9:

The server stores the collected information in a database.

Input: Collected match results and stats data

Specific action: Store data in a table using a database management system (e.g. MySQL).

Output: Match results and stats data stored in a database

Step 10:

The server analyzes the databased information and evaluates the players' performance.

Input: Match results and stats data stored in the database

Specific operations: Use data analysis tools (e.g. Pandas, NumPy) to calculate players' shooting success rates, defensive strength, etc.

Output: Player performance evaluation data

Step 11:

The server generates feedback for player development based on the analysis results.

Input: Player performance evaluation data

Specific actions: Use a generative AI model (e.g. GPT-4) to generate textual feedback for player development.

Output: Text data of feedback for training

Step 12:

The server sends the generated feedback to the device.

Input: Text data for feedback for training

Specific behavior: Sends feedback to the device using WebSocket or HTTP protocol.

Output: Feedback sent to the device

Step 13:

The device will display the feedback it receives.

Input: Feedback sent to device

Specific action: Visually display feedback on the device display.

Output: Visual feedback for the user

(Application example 3)

Next, application example 3 of embodiment example 3 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

Traditional live broadcasts and commentary of basketball games rely on human commentators, which can make it difficult to provide accurate information in real time. Commentary on youth and mini-basketball games across the country also relies on human input, making it difficult to provide information efficiently. Furthermore, when monitoring production lines in factories, detecting abnormalities and improving production efficiency are issues. To solve these issues, an automated system that utilizes AI is needed.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 3 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games from around the country with AI live coverage, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for analyzing camera footage and audio data within the factory to detect production status and abnormal occurrences in real time and provide scores, and a means for storing past production data in a database and providing feedback for efficiency. This allows for accurate understanding of the progress of the game and the status of the production line in real time, making it possible to provide efficient information and improve production efficiency.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to automatically provide commentary on the progress of a basketball game and plays in real time.

"A means of broadcasting commentary on youth and mini-basketball games nationwide with live coverage by AI" refers to a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games nationwide and broadcasts that commentary in real time.

"A means of using AI to automatically generate running scores that are created manually by officials and for each team" refers to technology in which artificial intelligence automatically analyzes points, fouls, and other scores as the game progresses, and generates scores in real time.

"A means of analyzing camera footage and audio data within a factory to detect and score production status and abnormalities in real time" refers to a technology in which artificial intelligence analyzes video and audio data obtained from cameras and microphones installed within the factory, detects the status and abnormalities of the production line in real time, and displays the results as a score.

"Means of compiling past production data into a database and providing feedback for improving efficiency" refers to a technology that stores past production data collected within a factory in a database, analyzes the data, and provides feedback for improving production efficiency.

The following system configuration is described as a form for implementing this invention.

System configuration

Hardware

Camera: Cameras installed in the factory capture video data.

Microphones: Microphones installed in the factory capture audio data.

Server: A server for analyzing and storing data.

Device: The device (PC, smartphone, etc.) used by the administrator to check the results.

Software

OpenCV: A library for acquiring and preprocessing camera images.

TensorFlow/Keras: Libraries for loading AI models and making predictions.

SQLite: A database management system for storing production data.

Data processing and data calculations

The server acquires camera footage and audio data from within the factory in real time and performs preprocessing using OpenCV. The preprocessed data is input into an AI model using TensorFlow/Keras to detect production status and abnormalities. The detection results are displayed as a score and stored in a SQLite database.

Specific examples

For example, if an abnormality occurs in a factory, it will be detected from camera footage and notified to managers in real time. It can also analyze past production data and provide feedback to improve efficiency.

Example of a prompt

An example of a prompt to input to the generative AI model is as follows:

"Please create an AI model that analyzes camera footage in the factory and detects abnormalities on the production line in real time. Please also add a function to notify the administrator if an abnormality is detected. Also, please create a system that stores past production data in a database and provides feedback to improve efficiency."

In this way, the server can monitor the status of the production line in real time and detect any anomalies. It can also provide feedback for efficiency improvements based on past data.

The flow of the specific processing in application example 3 is explained using Figure 16.

Step 1:

The server captures video and audio data in real time from cameras and microphones installed in the factory. The input is the raw data from the cameras and microphones, and the output is the captured video and audio data. In concrete terms, the server connects to the cameras and microphones and starts the data stream.

Step 2:

The server preprocesses the acquired video data using OpenCV. The input is the video data acquired in step 1, and the output is the preprocessed video data. Specifically, the server resizes the video data to 224x224 pixels and normalizes it.

Step 3:

The server inputs the preprocessed video data into the AI model using TensorFlow/Keras to make predictions. The input is the video data preprocessed in step 2, and the output is the predicted results of production status and abnormality occurrence. In concrete terms, the server loads the AI model, inputs the preprocessed data into the model, and makes predictions.

Step 4:

The server displays the prediction result as a score and saves it in a SQLite database. The input is the prediction result obtained in step 3, and the output is the displayed score and the data saved in the database. In concrete terms, the server converts the prediction result into a score, displays it on the screen, and inserts it into the database.

Step 5:

The terminal receives the score and anomaly detection notifications sent from the server and displays them to the administrator. The input is the score and notifications sent from the server, and the output is the display information that the administrator can check. In concrete terms, the terminal receives data from the server and displays it on the user interface.

Step 6:

The server analyzes past production data and generates feedback for efficiency improvements. The input is past production data stored in a database, and the output is feedback information for efficiency improvements. In concrete terms, the server retrieves past data from the database and applies an analysis algorithm to generate feedback.

Step 7:

The terminal receives feedback information sent from the server and displays it to the administrator. The input is the feedback information sent from the server, and the output is the display information that the administrator can check. In concrete terms, the terminal receives feedback information from the server and displays it on the user interface.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

"Example 1"

In one embodiment of the present invention, a system in which AI provides commentary on a live broadcast of a basketball game is combined with an emotion engine that recognizes the user's emotions. Specifically, the emotion engine recognizes the emotions (happiness, sadness, surprise, etc.) that the user expresses while watching the game, and adjusts the content and tone of the commentary according to those emotions. For example, if the user expresses joy at how the game is progressing, the commentary will also share that joy. This allows the user to hear commentary that reflects their own emotions, further improving their satisfaction with watching the game.

"Example 2"

The emotion engine is also incorporated into a system that broadcasts commentary on youth and mini-basketball games from around the country, complete with live coverage by AI. Specifically, the emotion engine recognizes the emotions of users watching youth and mini-basketball games, and adjusts the content and tone of the commentary according to those emotions. For example, if a user is impressed by their child's performance, the commentary will be one that shares that emotion. This allows users to hear commentary that reflects their own emotions, further increasing their satisfaction with watching the games.

"Example 3"

Furthermore, emotion engines are also being incorporated into systems that use AI to automatically generate running scores that are manually created by officials and for each team. Specifically, the emotion engine recognizes the user's emotions regarding the score situation of the match, and adjusts the way the score is displayed according to those emotions. For example, if the user is happy about a score, the score display will also be displayed in a way that emphasizes that joy. This allows users to see a score display that reflects their own emotions, further increasing their satisfaction when watching the match.

The process flow for each example is explained below.

"Example 1"

Step 1: The AI begins commentating on a live broadcast of a basketball game.

Step 2: The emotion engine recognizes the user's emotions in real time.

Step 3: The AI adjusts the content and tone of the commentary based on the emotion recognized by the emotion engine.

"Example 2"

Step 1: AI will begin providing live commentary for youth and mini-basketball matches across the country.

Step 2: The emotion engine recognizes the emotions of users watching the game in real time.

Step 3: The AI adjusts the content and tone of the commentary based on the emotion recognized by the emotion engine.

"Example 3"

Step 1: AI automatically generates a running score based on the game's scoring situation.

Step 2: The emotion engine recognizes the user's emotions in real time regarding the score of the game.

Step 3: The AI adjusts the way the score is displayed depending on the emotion recognized by the emotion engine.

(Example 1)

Next, a first embodiment of the first embodiment will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

In conventional basketball game commentary systems, it was difficult to analyze the game situation and player movements in real time and provide commentary that corresponded to the user's emotions. In addition, there was a lack of means to efficiently deliver commentary on youth and mini-basketball games nationwide, or to automatically generate running scores that are created manually by officials and for each team. Furthermore, there was insufficient use of emotion recognition technology to improve the user's viewing experience.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for providing commentary on live coverage of basketball games using artificial intelligence, means for delivering commentary on youth and mini-basketball games across the country with live coverage by artificial intelligence, means for automatically generating running scores using artificial intelligence that are created manually for each official and team, means for analyzing game video data, means for analyzing game audio data, means for receiving and analyzing user emotion data, means for automatically generating commentary based on the analysis results and adjusting the content and tone of the commentary according to the user's emotions, and means for delivering the adjusted commentary together with the live coverage. This makes it possible to analyze the situation of the game and the movements of the players in real time and provide commentary according to the user's emotions.

"Means for artificial intelligence to provide commentary for live broadcasts of basketball games" refers to technology that analyzes video and audio data of basketball games, and automatically generates commentary based on the results of that analysis using artificial intelligence, which is then distributed along with the live broadcast.

"A means of delivering commentary on youth and mini-basketball games nationwide with live coverage by artificial intelligence" refers to a technology in which artificial intelligence automatically generates commentary for youth and mini-basketball games nationwide and delivers that commentary along with live coverage.

"Method of automatically generating running scores using artificial intelligence that are created manually for each official or team" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Means for analyzing game video data" refers to technology that analyzes game video data frame by frame to identify the positions and movements of players, the position of the ball, etc.

"Means for analyzing audio data from a match" refers to technology that analyzes audio data from a match and identifies the content of the commentary and the reactions of the audience.

"Means for receiving and analyzing user emotion data" refers to technology that captures the user's facial expressions and voice, and analyzes that data to identify the user's emotions.

"Means for automatically generating commentary based on analysis results and adjusting the content and tone of the commentary according to the user's emotions" refers to a technology that automatically generates commentary based on the analysis results of video and audio data, and further adjusts the content and tone of the commentary taking into account the user's emotional data.

"Means for delivering adjusted commentary together with live broadcast" refers to technology that delivers adjusted commentary in real time to a user's device together with the live broadcast.

Form for implementing the invention

This invention is a system in which artificial intelligence provides commentary for live coverage of basketball games, analyzing video and audio data from the game and providing commentary that reflects the user's emotions. A specific embodiment of this system is shown below.

Hardware and software used

Hardware: High-performance servers, user devices (PCs, smartphones, tablets, etc.)

Software: Video analysis software (e.g. OpenCV), audio analysis software (e.g. Google Cloud Speech-to-Text API), emotion engines (e.g. Microsoft Azure Emotion API), generative AI models (e.g. GPT-4)

System configuration

1. The server receives the game video and audio data in real time. It uses a high-speed network connection to receive the streaming data sent from the game venue.

2. The server uses video analysis software to analyze the received video data frame by frame. Specifically, computer vision techniques are used to identify the positions of players, their movements, and the position of the ball. For example, OpenCV is used.

3. The server uses voice analysis software to analyze the received voice data. Specifically, it uses voice recognition technology to identify the content of the commentary and the reactions of the audience. For example, it uses the Google Cloud Speech-to-Text API.

4. The server uses a generative AI model to automatically generate commentary on the match based on the results of video and audio analysis. Specifically, it uses a natural language generation model such as GPT-4 to generate sentences that explain the situation of the match and the movements of the players.

5. The device captures the user's facial expressions and voice in real time and transmits them as emotion data to the server. Specifically, the device uses a camera and microphone.

6. The server uses an emotion engine to analyze the received emotion data. Specifically, it uses emotion recognition technology to identify emotions such as happiness, sadness, surprise, etc. from the user's facial expressions and voice. For example, it uses the Microsoft Azure Emotion API.

7. The server adjusts the content and tone of the generated commentary according to the user's emotions analyzed by the server. Specifically, if the user is happy, the tone of the commentary is brightened and more positive expressions are included.

8. The server delivers the adjusted commentary along with the live broadcast to the user's device. Specifically, it delivers in real time using streaming technology. For example, WebRTC is used.

Specific examples

For example, if a user expresses joy while watching a game, the emotion engine will recognize that joy. Based on that information, the server will lighten the tone of the commentary generated by the generative AI model and adjust it to share the joy. This allows users to hear commentary that reflects their own emotions, improving their satisfaction watching the game.

Example of a prompt

"Design an AI system to provide commentary for live broadcasts of basketball games. It must analyze video and audio data from the game, understand the game situation and the movements of the players, and incorporate an emotion engine that adjusts the content and tone of the commentary according to the user's emotions."

The flow of the identification process in Example 1 is explained using Figure 17.

Step 1:

The server receives the video and audio data of the match in real time. The input is streaming data sent from the match venue, and the output is the video and audio data stored in the server. Specifically, the server receives the data using a high-speed network connection and stores it in an appropriate format.

Step 2:

The server uses video analysis software to analyze the received video data frame by frame. The input is the saved video data, and the output is the analysis results, such as the positions and movements of the players and the position of the ball. In concrete terms, it uses OpenCV to analyze each frame and identify the positions of the players and the ball.

Step 3:

The server uses voice analysis software to analyze the received voice data. The input is the stored voice data, and the output is the analysis results in the form of text that contains the commentary and the reactions of the audience. Specifically, the Google Cloud Speech-to-Text API is used to convert the voice data into text and extract key information.

Step 4:

The server uses a generative AI model to automatically generate commentary on the match based on the results of video and audio analysis. The input is the results of video and audio analysis, and the output is the generated commentary. Specifically, it uses a natural language generation model such as GPT-4 to generate sentences that explain the situation of the match and the movements of the players.

Step 5:

The device captures the user's facial expressions and voice in real time and sends them to the server as emotion data. The input is the user's facial expressions and voice, and the output is the emotion data sent to the server. In concrete terms, the device uses a camera and microphone to capture the user's facial expressions and voice, and sends them to the server in real time.

Step 6:

The server uses an emotion engine to analyze the received emotion data. The input is the emotion data sent, and the output is the analysis results that identify the user's emotion. Specifically, it uses the Microsoft Azure Emotion API to identify emotions such as joy, sadness, and surprise from the user's facial expressions and voice.

Step 7:

The content and tone of the generated commentary is adjusted according to the user's emotions analyzed by the server. The input is the result of emotion analysis and the generated commentary, and the output is the adjusted commentary. Specifically, if the user shows joy, the tone of the commentary is brightened and more positive expressions are used.

Step 8:

The server delivers the adjusted commentary to the user's device along with the live broadcast. The input is the adjusted commentary and the live broadcast data, and the output is the live broadcast with real-time commentary delivered to the user's device. In concrete terms, it uses WebRTC to deliver the adjusted commentary in real time.

(Application example 1)

Next, application example 1 of embodiment example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

Conventional basketball game commentary systems had the problem that the commentary was uniform and could not flexibly respond to the emotions of the viewers. In addition, improving the quality of the live broadcast and commentary of the game required a large number of people, which was costly and laborious. Furthermore, there was a lack of efficient means to provide commentary on youth and mini-basketball games across the country.

The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games nationwide with AI live coverage, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions, a means for generating commentary using a generation AI model, and a means for generating commentary by inputting a prompt sentence. This enables flexible commentary according to the viewer's emotions, improving the quality of live coverage and commentary of games. In addition, commentary on youth and mini-basketball games nationwide can be efficiently performed.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to a means that uses artificial intelligence technology to analyze video and audio data of basketball games, understand the situation of the game and the movements of the players, and automatically generate commentary.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a means of broadcasting live coverage including commentary on youth and mini-basketball games from around the country with AI providing commentary.

"Means of using AI to automatically generate running scores that are created manually by officials and teams" refers to a means of using AI to automatically generate scores that are manually created by officials and teams during a match.

"Means for adjusting the content and tone of commentary using an emotion engine that recognizes the user's emotions" refers to means that use emotion recognition technology to recognize the user's emotions (happiness, sadness, surprise, etc.) and adjust the content and tone of the commentary in accordance with those emotions.

"Means for generating commentary using a generative AI model" refers to means for generating commentary based on the situation of a match using a generative AI model (e.g., GPT-3).

"Means for inputting a prompt sentence and generating an explanation" refers to means for inputting a specific prompt sentence into a generative AI model and generating an explanation based on that prompt.

A system for implementing this invention includes a means for AI to provide commentary for live coverage of basketball games, a means for distributing commentary for youth and mini-basketball games across the country with AI commentary, a means for AI to automatically generate running scores that are created manually for each official or team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions, a means for generating commentary using a generative AI model, and a means for generating commentary by inputting a prompt text.

Hardware and software used

Hardware: Smartphone camera and microphone

Software: OpenCV, Keras, Transformers

Data processing and data calculations

Server

The server receives and analyzes video and audio data from the basketball game in real time. It uses OpenCV to acquire the video data and Keras to run an emotion recognition model. The emotion recognition model analyzes the user's facial expressions and recognizes emotions such as happiness, sadness, and surprise.

Terminal

The device generates commentary using a generative AI model (e.g., GPT-3). A prompt sentence is input into the generative AI model, which generates commentary based on the prompt. Examples of prompt sentences include "The user is happy. Generate a joyful commentary for the basketball game." and "The user is sad. Generate a comforting commentary for the basketball game."

User

Users watch the game through their smartphone's camera and microphone, and their emotions are recognized. The emotion engine adjusts the content and tone of the commentary based on the user's emotions. For example, if the user expresses joy at a highlight scene in the game, the commentary might say something like, "Great play! We've been waiting for this moment!" Conversely, if the user expresses sadness at a comeback scene in the game, the commentary might say something like, "It's a disappointing result, but there's still a chance. Let's look forward to the next play."

In this way, flexible commentary based on the viewer's emotions becomes possible, improving the quality of live game coverage and commentary. It also enables efficient commentary of youth and mini-basketball games across the country.

The flow of the specific processing in application example 1 is explained using Figure 18.

Step 1:

The server receives video and audio data from the smartphone's camera and microphone in real time. The input is the video and audio data from the camera and microphone, and the output is the data ready for analysis.

Step 2:

The server processes the video data using OpenCV to detect the user's face. The input is the video data received in step 1, and the output is the detected face area. Specifically, the server converts the video data to grayscale and applies a face detection algorithm.

Step 3:

The server runs an emotion recognition model using Keras to recognize the user's emotions. The input is the face region detected in step 2, and the output is the recognized emotion (happiness, sadness, surprise, etc.). Specifically, the server resizes the face region to an appropriate size and inputs it into the emotion recognition model.

Step 4:

The server generates a prompt sentence to be input to the generative AI model based on the recognized emotion. The input is the emotion recognized in step 3, and the output is the generated prompt sentence. Specifically, it selects a template sentence according to the emotion and generates the prompt sentence.

Step 5:

The server generates an explanation using a generative AI model (e.g., GPT-3). The input is the prompt text generated in step 4, and the output is the generated explanation text. In concrete terms, the prompt text is input into the generative AI model, and an explanation text is generated.

Step 6:

The server delivers the generated explanatory text to the user in real time. The input is the explanatory text generated in step 5, and the output is the explanatory audio delivered to the user's smartphone. In concrete terms, the explanatory text is input into a voice synthesis engine, and audio data is generated and delivered.

Step 7:

Users watch the game and listen to the commentary through their smartphones. The input is the commentary audio delivered from the server, and the output is an improvement to the user's viewing experience. In concrete terms, the commentary audio is played through the smartphone's speakers.

(Example 2)

Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

Traditional basketball game commentary relied on the commentator's subjective judgment, making it difficult to provide commentary that reflected the emotions of viewers. In addition, there was a lack of systems to provide youth and mini-basketball games from across the country to a wide audience, making it difficult to grasp the progress of the game and the performance of the players in real time. Furthermore, manually creating a running score for the game was time-consuming and labor-intensive, so efficient management was required.

The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

In this invention, the server includes means for providing commentary on live coverage of basketball games using artificial intelligence, means for distributing commentary on youth and mini-basketball games from around the country with live coverage by artificial intelligence, means for automatically generating running scores using artificial intelligence that are created manually for each official and team, means for collecting game video and audio data, means for analyzing the collected data and generating commentary, means for recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions, and means for distributing the generated commentary together with the game video. This makes it possible to provide commentary that corresponds to the viewer's emotions and improve satisfaction with viewing the game.

"Means for artificial intelligence to provide commentary for live broadcasts of basketball games" refers to technology in which artificial intelligence analyzes the progress of a basketball game and the performance of players, and generates commentary in real time.

"A means of broadcasting commentary on youth and mini-basketball games across the country with live coverage by artificial intelligence" refers to a technology in which artificial intelligence analyzes youth and mini-basketball games held across the country, generates commentary, and broadcasts it in real time.

"Means for automatically generating running scores using artificial intelligence that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Means for collecting game video and audio data" refers to technology that uses cameras and microphones installed at the game venue to collect game video and audio data in real time.

"Means of analyzing collected data and generating commentary" refers to technology in which artificial intelligence analyzes collected video and audio data and generates commentary based on the progress of the match and the performance of the players.

"Means of recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to technology that analyzes the viewer's facial expressions and tone of voice, and adjusts the content and tone of the commentary according to those emotions.

"Means for distributing generated commentary together with game footage" refers to technology for distributing commentary generated by artificial intelligence together with game footage via a streaming platform.

This invention is a system in which artificial intelligence (AI) broadcasts live basketball games, and also distributes live commentary by AI for youth and mini-basketball games across the country. It also includes a function that allows the AI to automatically generate running scores that are currently created manually by officials and for each team.

The server collects video and audio data from the match in real time using cameras and microphones installed at the match venue. Specifically, it uses hardware such as Sony's 4K cameras and Shure's high-sensitivity microphones. The collected data is stored in a database on the server.

The server then inputs the stored video and audio data into an AI model. This AI model uses a generative AI model such as OpenAI's GPT-4 or Google's BERT. The AI model analyzes the input data and generates commentary based on the progress of the match and the performance of the players. For example, it analyzes the moment when Player A makes a shot and generates commentary such as, "Player A made a great shot!".

In addition, the device will recognize the emotions of the user watching the game. Specifically, it will use the device's camera and microphone to capture the user's facial expressions and tone of voice. For example, Apple's Face ID technology or Amazon's Alexa will be used for this purpose. The device will then send the captured data to a server.

The server receives the user's emotion data sent from the terminal and analyzes it using an emotion engine. The emotion engine uses, for example, Affectiva or IBM Watson's emotion analysis API. If the user is emotional, the server adjusts the tone of the commentary to be more emotional and generates a commentary such as, "Player A's shot was really amazing! The crowd is really excited!".

Finally, the server distributes the generated commentary along with the game footage. Streaming platforms such as YouTube and Twitch are used for distribution. Users can watch the game through these platforms and enjoy the commentary according to their emotions.

As a concrete example, let's say a youth basketball game is taking place. The server collects video and audio data in real time from Sony's 4K cameras and Shure's high-sensitivity microphones installed at the game venue. The collected data is then input into OpenAI's GPT-4, which analyzes the moment Player A makes a shot. The AI model generates a commentary such as, "Player A made a great shot!".

Meanwhile, the device uses Apple's Face ID technology to capture the facial expressions of the user watching the game and recognizes that they are emotionally moved. The server then uses Affectiva's emotion analysis API to analyze the user's emotions and adjust the tone of the commentary to be more emotional. It generates a commentary such as, "Player A's shot was really amazing! The crowd is really excited!" and distributes it via YouTube.

Examples of prompts might include:

"Analyze video and audio data from youth basketball games to generate commentary on game progress and player performance. Also recognize viewer emotions and adjust the content and tone of the commentary accordingly."

In this way, the server can provide users with commentary that reflects their emotions, improving their satisfaction with watching the game.

The flow of the identification process in Example 2 is explained using Figure 19.

Step 1:

The server uses cameras and microphones installed at the venue to collect video and audio data of the match in real time. Specifically, it uses Sony 4K cameras and Shure high-sensitivity microphones. The input is the video and audio data of the match, and the output is the collected video and audio data. This data is stored in a database on the server.

Step 2:

The server inputs the stored video and audio data into an AI model. This AI model uses, for example, OpenAI's GPT-4. The input is the collected video and audio data, and the output is analyzed data on the progress of the match and the performance of the players. The AI model analyzes the movements of the players and the progress of the match from the video data, and analyzes the commentary and audience reactions from the audio data.

Step 3:

The server generates commentary on the match based on the data analyzed by the AI model. The input is the analyzed data on the progress of the match and the performance of the players, and the output is the generated commentary. For example, the server analyzes the moment when Player A makes a shot and generates a commentary such as, "Player A made a great shot!".

Step 4:

The device recognizes the emotions of users watching a game. Specifically, it uses the device's camera and microphone to capture the user's facial expressions and tone of voice. The input is data about the user's facial expressions and tone of voice, and the output is the recognized user's emotion data. This uses Apple's Face ID technology and Amazon's Alexa.

Step 5:

The server receives the user's emotion data sent from the terminal and analyzes it using an emotion engine. The input is the user's emotion data, and the output is the analyzed emotion information. The emotion engine uses, for example, Affectiva's emotion analysis API. If the user is emotional, the server adjusts the tone of the commentary to be emotional.

Step 6:

The server distributes the generated commentary together with the game footage. The input is the generated commentary and the game footage, and the output is the streaming data to be distributed. Streaming platforms such as YouTube and Twitch are used for distribution. Users can watch the game through these platforms and enjoy commentary according to their emotions.

(Application example 2)

Next, application example 2 of embodiment example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

Conventional live basketball game broadcasting systems had the problem that commentary was uniform and did not allow for flexible responses to viewer emotions. In addition, there was a lack of game highlights, replay functions, and chat functions between viewers, so there was a need to improve the viewing experience. Furthermore, there were limited ways to widely provide viewers with information about youth and mini-basketball games across the country, resulting in information disparities between regions.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 2 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary of youth and mini-basketball games from around the country with AI commentary, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for recognizing the emotions of viewers and adjusting the content and tone of the commentary according to those emotions, a means for providing highlights and replay functions for games, and a means for providing a chat function between viewers. This allows for flexible commentary according to the emotions of viewers, improving the viewing experience and making it possible to widely provide youth and mini-basketball games from around the country to viewers.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to generate commentary in real time for basketball games and provide it to viewers.

"A means of broadcasting commentary on youth and mini-basketball games across the country with live coverage by AI" refers to technology that broadcasts youth and mini-basketball games held across the country in real time with commentary by artificial intelligence.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually recorded by officials and teams during a match.

"Means of recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to technology that recognizes emotions from the viewer's facial expressions, voice, etc., and changes the content and tone of the commentary according to those emotions.

"Means for providing match highlight and replay features" refers to technology that provides a highlight feature that extracts and plays important scenes from a match, and a replay feature that replays specific scenes.

"Means for providing a chat function between viewers" refers to technology that provides a chat function that allows users watching a match to exchange messages in real time.

The system for implementing this invention includes the following main components: a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games across the country with AI commentary, a means for AI to automatically generate running scores that are created manually for each official and team, a means for recognizing the emotions of viewers and adjusting the content and tone of the commentary according to those emotions, a means for providing game highlights and replay functions, and a means for providing a chat function between viewers.

Hardware and software used

Hardware: Smartphone (camera, microphone, display)

Software: OpenCV (image processing library), EmotionRecognizer (emotion recognition model), AICommentary (AI commentary generation model), LiveStreaming (live streaming library)

Data processing and data calculations

1. Frame acquisition: The server acquires frames from the live streaming, which allows video data of the match to be collected in real time.

2. Emotion Recognition: The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer. This obtains the viewer's emotion data.

3. Commentary generation: The server generates commentary using AICommentary based on the recognized emotions. This allows commentary to be generated in real time according to the viewer's emotions.

4. Add commentary: The server adds the generated commentary to the live stream, allowing viewers to see emotional commentary in real time.

5. Highlights and Replays: The server automatically extracts important scenes from the match and provides highlights and replays, allowing viewers to rewatch key moments of the match.

6. Chat function: The server provides a chat function that allows viewers to exchange messages with each other in real time. This promotes communication between viewers.

Specific examples

For example, during a youth basketball game, if the viewer is impressed, the server might generate a commentary like, "Great play! I'll never forget this moment!" Or, if the viewer is excited, the server might generate a commentary like, "You won't want to miss this! Watch the next play!"

Example of a prompt

"Create an AI model that recognizes the emotions of your viewers and adjusts the content and tone of your commentary accordingly. For example, if your viewer is emotional, make sure your commentary reflects that emotion."

The flow of the specific processing in application example 2 is explained using Figure 20.

Step 1:

The server obtains frames from live streaming. The input is real-time game video data, and the output is individual frame images. Specifically, the server receives the video data sent from the camera and splits it into frames using OpenCV.

Step 2:

The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer. The input is the frame image obtained in step 1, and the output is the viewer's emotion data. Specifically, the server detects the face in the frame and inputs it into the emotion recognition model to classify the emotion.

Step 3:

The server generates a commentary using AICommentary based on the recognized emotion. The input is the emotion data obtained in step 2, and the output is commentary text corresponding to the emotion. Specifically, the server inputs the emotion data as a prompt into the AI commentary generation model to generate an appropriate commentary.

Step 4:

The server adds the generated commentary to the live stream. The input is the commentary text generated in step 3, and the output is the live video with the commentary added. Specifically, the server synthesizes the commentary text into speech and overlays it on the video.

Step 5:

The server automatically extracts important scenes from a match and provides highlight and replay functions. The input is real-time match video data, and the output is highlight and replay footage. Specifically, the server detects events in the match, extracts important scenes, and saves them.

Step 6:

The server provides a chat function that allows viewers to exchange messages with each other in real time. The input is message data from the viewer, and the output is the message to be sent to other viewers. Specifically, the server receives messages and distributes them to other viewers in the chat room.

(Example 3)

Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

In conventional basketball games, creating a running score and understanding the progress of the game were done manually, making it difficult to update in real time. In addition, there was little progress in creating a database of teams and players nationwide, and feedback for analyzing player performance and training was not provided sufficiently. Furthermore, the score display did not reflect the user's feelings about the score situation in the game, resulting in low satisfaction when watching the game.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for receiving video data and audio data of the match, means for analyzing the received video data, means for analyzing the received audio data, means for extracting the score situation and foul situation from the analysis results, means for generating a running score based on the extracted information, means for updating and transmitting the generated running score in real time, means for displaying the transmitted running score, and means for recognizing the user's emotions and adjusting the score display. This makes it possible to grasp the progress of the match in real time and provide a score display that reflects the user's emotions.

"Game video data" refers to video data that records the progress of a basketball game.

"Audio data" refers to audio data that records commentary and commentary during a match, audience reactions, etc.

"Means of receiving" refers to the function for acquiring video and audio data of the match from an external source.

"Means of analysis" refers to the function of processing received video and audio data and extracting specific information.

"Scoring status" is information that shows how many points each team scored during the game.

"Foul situation" is information that shows which players committed what type of fouls during the game.

The "running score" is a scoreboard that shows the score and foul situation in real time as the game progresses.

"Means for updating and transmitting in real time" refers to a function for instantly updating the generated running score and transmitting it to the user's device.

"Display means" is a function for visually presenting the submitted running score to the user.

"Means of recognizing user emotions" refers to a function for analyzing and understanding the user's emotions regarding the score of a match, etc.

"Means for adjusting score display" is a function for changing the way the scoreboard is displayed depending on the recognized user's emotions.

"Game results and stats information for teams and players nationwide" refers to game results for basketball teams and players nationwide and statistical data for individual players.

"Means of saving to a database" refers to the function for organizing collected match results and stats information and storing it in a database.

"Means of analyzing performance" refers to a function for evaluating a player's performance and abilities based on the information stored in the database.

"Means of generating feedback" refers to the function that creates reports based on the analysis results to suggest player development and areas for improvement.

"Means for sending and displaying" refers to the function for sending the generated feedback to the user's device and presenting it visually.

"Visualization" refers to displaying data visually to make it easier to understand.

"Means for utilizing in development" refers to methods for supporting the training and development of players based on data.

This invention is a system that automatically generates running scores for basketball games and analyzes player performance. A specific embodiment of this system is described below.

System configuration

Hardware and software

1. Server

Hardware: Server with high-performance processor and large memory capacity

Software: Video analysis software (e.g. OpenCV), audio analysis software (e.g. Google Cloud Speech-to-Text), database management system (e.g. MySQL), sentiment analysis engine (e.g. IBM Watson Tone Analyzer)

2. Terminal

Hardware: PCs, tablets, smartphones, etc.

Software: Web browser, dedicated application

Program processing

Receiving and analyzing match data

The server receives video and audio data of the match. The video data is obtained from cameras and streaming services, and the audio data includes commentary and narration. The received video data is analyzed using OpenCV to detect scoring and foul scenes. The audio data is converted to text using Google Cloud Speech-to-Text, and information about scores and fouls is extracted from the commentary.

Extraction of scoring and foul situations

The server combines the results of video and audio analysis to extract the score and foul situations. This makes it possible to grasp the progress of the game in real time.

Generate and display running scores

The server generates a running score based on the extracted information. The generated running score is updated in real time and sent to the terminal using WebSocket or HTTP. The terminal displays the received running score to the user, allowing the user to check the progress of the match in real time.

Recognizing user emotions and adjusting score display

The server uses IBM Watson Tone Analyzer to recognize the user's emotions regarding the score of the game. Depending on the recognized emotion, the way the score is displayed is adjusted. For example, if the user is happy about a score, the score display is emphasized with bright colors or animations. The adjusted score display is sent to the terminal and displayed to the user.

Database of teams and players nationwide and performance analysis

The server collects match results and stats for teams and players across the country, and stores them in a MySQL database. Based on the stored information, the performance of players is analyzed using Python's Pandas library. Based on the analysis results, feedback for player development is generated and sent to the terminal. The terminal displays the received feedback to the user.

Specific examples

When the video data of the match is sent to the server, the server uses OpenCV to detect scoring scenes and uses Google Cloud Speech-to-Text to extract foul information from the commentary audio. For example, the prompt text can be "Analyze the scoring scenes and foul information of this match to generate a running score."

The server collects the results of matches from around the country and stores them in a MySQL database. It uses Python's Pandas library to analyze players' performance and generate feedback. For example, enter the following prompt statement: "Collect the results of matches from around the country, analyze players' performance, and generate feedback."

The server recognizes the user's joy at the scoring scene using IBM Watson Tone Analyzer, and adjusts the score display to emphasize the joy. For example, the prompt text is input as "Analyze the user's emotions at the scoring scene and adjust the score display." The flow of the specific process in Example 3 is explained with reference to FIG. 21.

Step 1:

Receive match data

The server receives video and audio data of the match. As input, it obtains video and audio data provided in real time from cameras and streaming services. As output, it stores the received video and audio data in its internal memory. Specifically, the server obtains the data through a network connection and stores it in a specified format.

Step 2:

Video data analysis

The server analyzes the received video data. As input, it uses the video data saved in step 1. As output, it obtains the results of detecting scoring and foul scenes. In concrete terms, the server uses OpenCV to analyze the video data frame by frame and executes an algorithm to detect specific events (scoring and fouls).

Step 3:

Analysis of audio data

The server analyzes the received audio data. As input, it uses the audio data stored in step 1. As output, it obtains text information extracted from the audio data. Specifically, the server converts the audio data into text using Google Cloud Speech-to-Text and extracts information about scores and fouls from the commentary.

Step 4:

Extraction of scoring and foul situations

The server integrates the video analysis results and audio analysis results to extract the scoring and foul situations. As input, it uses the outputs of steps 2 and 3. As output, it obtains data including the scoring and foul situations. In concrete terms, the server integrates the video analysis results and audio analysis results in a chronological order to identify the timing of scoring and fouls.

Step 5:

Generate running scores

The server generates a running score based on the extracted information. As input, it uses the score and foul status obtained in step 4. As output, it obtains data including the running score. Specifically, the server organizes the score and foul information in chronological order and compiles it in a scoreboard format.

Step 6:

Update and send running scores

The server updates the generated running score in real time and sends it to the terminal. As input, it uses the running score generated in step 5. As output, it sends the updated running score to the terminal. Specifically, the server sends data using WebSocket or HTTP.

Step 7:

Display running score

The terminal displays the received running score to the user. As input, it uses the running score sent in step 6. As output, it obtains a scoreboard that is visually presented to the user. In concrete terms, the terminal displays the scoreboard using a web browser or a dedicated application.

Step 8:

Recognizing user emotions

The server recognizes the user's emotions regarding the score of the game. As input, it uses the user's comments and reaction data. As output, it obtains the recognized emotion data. Specifically, the server analyzes the user's emotions using IBM Watson Tone Analyzer.

Step 9:

Score display adjustments

The server adjusts the way the score is displayed depending on the recognized emotion. As input, it uses the emotion data obtained in step 8 and the running score generated in step 5. As output, it obtains the adjusted score display data. As a specific operation, the server changes the color and animation of the score display.

Step 10:

Send adjusted score display

The server sends the adjusted score display to the terminal. As input, it uses the score display data adjusted in step 9. As output, it sends the adjusted score display to the terminal. Specifically, the server sends the data using WebSocket or HTTP.

Step 11:

Displaying the adjusted score

The terminal displays the adjusted score display to the user. As input, it uses the adjusted score display data sent in step 10. As output, it obtains the adjusted scoreboard that is visually presented to the user. In specific operation, the terminal displays the adjusted scoreboard using a web browser or a dedicated application.

Step 12:

Database of teams and players nationwide

The server collects match results and stats information for teams and players across the country and stores it in a database. As input, it uses data obtained from each team's official website and sports data services. As output, it obtains match results and stats information stored in the database. Specifically, the server organizes the data and stores it in a MySQL database.

Step 13:

Performance analysis

The server analyzes the players' performance based on the information stored in the database. As input, it uses the database information stored in step 12. As output, it obtains the analysis results that evaluate the players' performance and abilities. Specifically, the server processes the data using Python's Pandas library and evaluates the players' performance.

Step 14:

Generating feedback

The server generates feedback for player development based on the analysis results. As input, it uses the analysis results obtained in step 13. As output, it obtains a feedback report that points out the player's strengths and weaknesses and suggests areas for improvement. In concrete terms, the server creates the report and provides it to the player and coach.

Step 15:

Submit and view feedback

The server sends the generated feedback to the terminal, which displays it to the user. As input, it uses the feedback report generated in step 14. As output, it obtains a feedback report that is visually presented to the user. In concrete terms, the server sends the data via email or a dedicated application, and the terminal displays it.

(Application example 3)

Next, application example 3 of embodiment example 3 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

Traditionally, live broadcasts and commentary of basketball games relied on human commentators, making it difficult to grasp the score and foul situation in real time. In addition, analysis of player performance and feedback on development were also done manually, which was inefficient. Furthermore, it was not possible to display scores that reflected the user's emotions, making it difficult to improve the viewing experience.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 3 is realized by the following means.

In this invention, the server includes: a means for AI to provide commentary on live coverage of basketball games; a means for distributing commentary on youth and mini-basketball games nationwide with AI live coverage; a means for AI to automatically generate running scores that are created manually for each official and team; a means for analyzing game video and audio data in real time and automatically generating the score situation and foul situation; a means for recognizing the user's emotions and displaying the score according to the emotions; and a means for providing feedback for player performance analysis and development using a database of teams and players nationwide. This makes it possible to grasp the progress of the game in real time and display scores that reflect the user's emotions, improving the efficiency of player performance analysis and development.

"A means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to provide real-time commentary on the progress and play of a basketball game.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games from around the country, and broadcasts live coverage including that commentary.

"A means of using AI to automatically generate running scores that are created manually by officials and for each team" refers to technology that uses artificial intelligence to automatically generate scores such as points and fouls as the game progresses.

"Means for analyzing game video and audio data in real time and automatically generating scoring and foul status" refers to technology that analyzes game video and audio data in real time and automatically generates scoring and foul status.

"Means for recognizing the user's emotions and displaying a score according to those emotions" refers to technology that recognizes emotions from the user's facial expressions and voice, and adjusts the score display according to those emotions.

"Means of utilizing a database of teams and players nationwide to analyze player performance and provide feedback for development" refers to technology that utilizes a database of basketball teams and players nationwide to analyze player performance and provide feedback for development.

The system for implementing this invention uses the following hardware and software.

Hardware and software used

Hardware: Smartphone, head-mounted display

Software: TensorFlow (AI model), OpenCV (video analysis), NLTK (natural language processing), Emotion API (emotion recognition)

System configuration

The server includes the following means:

1. How AI can provide commentary for live broadcasts of basketball games:

The server receives video and audio data from the match in real time, and uses TensorFlow to analyze the progress of the match and the content of the play, generating commentary.

2. How to broadcast live commentary of youth and mini-basketball games nationwide with AI:

The server receives footage of youth and mini-basketball matches from across the country, and similarly uses AI to generate commentary, which it distributes in real time.

3. A way to use AI to automatically generate running scores that are currently created manually by officials and teams:

The server analyzes the video and audio data of the match and automatically generates the score and foul situations.

4. A method for analyzing game video and audio data in real time and automatically generating scoring and foul situations:

The server uses OpenCV to split the video data into frames and extract important scenes. It uses NLTK to convert the audio data into text and extract important keywords.

5. How to recognize user emotions and display a score according to those emotions:

The server uses the Emotion API to recognize emotions from the user's facial expressions and voice, and adjusts the score display accordingly.

6. Using a national team and player database to analyze player performance and provide feedback for development:

The server accesses a database of teams and players from across the country, displays analysis of player performance, and provides feedback for the next match.

Specific examples

When a user using a smartphone during a match sees a goal being scored, the server uses AI to analyze the scene and display the score in real time. If the user shows an expression of joy, the server will recognize it using an emotion engine and make the score display more colorful. After the match ends, the server also refers to a nationwide database to display an analysis of the players' performance and provide feedback for the next match.

Example of a prompt

"Please analyze the video and audio data of the game in real time, and use an AI model that automatically generates the score and foul situation to display a running score. Also, please recognize the user's emotions and display the score according to those emotions. Furthermore, please use a database of teams and players from all over the country to provide feedback for analyzing player performance and development."

The flow of the specific processing in application example 3 is explained using Figure 22.

Step 1:

Acquisition of game video and audio data

The server receives video and audio data of the game in real time from a smartphone or head-mounted display. The input is the video and audio data of the game, and the output is preprocessed data for analysis. In concrete terms, the server receives the data stream, splits the video data into frames, and converts the audio data into text.

Step 2:

Data preprocessing

The server uses OpenCV to split the video data into frames and extract important scenes. It also uses NLTK to convert the audio data into text and extracts important keywords (e.g., goals, fouls). The input is the preprocessed video and audio data, and the output is important scenes and keywords for analysis. Specifically, the server analyzes video frames and identifies scoring and foul scenes.

Step 3:

Analysis using AI models

The server uses TensorFlow to analyze the scoring and foul situations from video and audio data. The input is important scenes and keywords, and the output is data on the scoring and foul situations. In concrete terms, the server runs an AI model and automatically determines the scores and fouls for each scene.

Step 4:

Automatic generation of running scores

The server automatically generates a running score based on the analysis results. The input is data on the scoring and foul situations, and the output is a real-time running score. Specifically, the server tallys up the score and foul information and updates the scoreboard.

Step 5:

Emotion recognition

The server uses the Emotion API to recognize emotions from the user's facial expressions and voice. The input is the user's facial expression data and voice data, and the output is the recognized emotion data. In concrete terms, the server obtains user data from the camera and microphone and analyzes emotions.

Step 6:

Score display based on emotions

The server adjusts the score display according to the recognized emotion. The input is emotion data and the running score, and the output is a score display according to the emotion. In concrete terms, the server changes the design and effects of the score display based on the user's emotion.

Step 7:

Utilizing the database

The server refers to a database of teams and players from around the country, displays the results of player performance analysis, and provides feedback for the next match. The input is the match results and player data, and the output is performance analysis results and feedback. In concrete terms, the server retrieves relevant information from the database and displays the analysis results.

The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input for the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the headset-type terminal 314.

[Fourth embodiment]

Figure 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

As shown in FIG. 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical able-bodied person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is performed in a secure state.

The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and legs. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414.

As shown in FIG. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. Data generation model 58 and emotion identification model 59 are used by the identification processing unit 290.

In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the specific processing performed by the specific processing unit 290 of the data processing device 12.

"Example 1"

Example 1 of the present invention is a system in which AI provides commentary for a live broadcast of a basketball game. In this system, AI analyzes the video and audio data of the game to understand the situation of the game and the movements of the players. Then, based on the results of this analysis, a commentary of the game is automatically generated and distributed along with the live broadcast.

"Example 2"

Example 2 of the present invention is a system that distributes commentary of youth and mini-basketball games from around the country, with live coverage by AI. This system collects video and audio data from youth and mini-basketball games held around the country, which are analyzed by AI in the same way as example 1 to generate commentary. Then, by distributing the commentary together with the game footage, games from around the country are made available to a wide audience.

"Example 3"

Example 3 of the present invention is a system that uses AI to automatically generate running scores that are created manually by officials or for each team. In this system, AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game, and automatically generates a running score. This makes it possible to grasp the progress of the game in real time.

"Example 4"

Example 4 of the present invention is a system that creates a database of teams and players from across the country, and uses the information in the database to visualize and develop registered basketball players in Japan. This system collects information such as game results and stats for teams and players from across the country, and creates a database. The database is then used to provide feedback for analyzing players' performance and developing them.

The process flow for each example is explained below.

"Example 1"

Step 1: AI collects video and audio data from basketball games.

Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

Step 3: Based on the analysis results, the AI automatically generates commentary for the match.

Step 4: Distribute the generated commentary along with the live broadcast.

"Example 2"

Step 1: Collect video and audio data from youth and mini-basketball matches held across the country.

Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

Step 3: Based on the analysis results, the AI automatically generates commentary for the match.

Step 4: Distribute the generated commentary along with the game footage.

"Example 3"

Step 1: AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game.

Step 2: Based on the analysis results, the AI automatically generates a running score.

"Example 4"

Step 1: Collect information such as match results and stats for teams and players across the country.

Step 2: Create a database of the collected information.

Step 3: Use the databased information to analyze player performance and provide feedback for development.

(Example 1)

Next, a first embodiment of the first embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

Traditional live broadcasts of basketball games depend on the quality and quantity of commentators, making it difficult to accurately convey the detailed situation of the game and the movements of the players in real time. In addition, commentary on youth and mini-basketball games across the country is required by many people, which is costly and labor intensive. Furthermore, running scores created by officials and people for each team are done manually, so they take time and can lack accuracy. To solve these issues, an automated system that utilizes AI is needed.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for collecting video data and audio data of the match in real time, means for analyzing the collected video data and audio data, means for generating commentary on the match based on the analysis results, and means for distributing the generated commentary together with a live broadcast. This makes it possible to accurately convey the detailed situation of the match and the movements of the players in real time. In addition, commentary on youth and mini-basketball matches from all over the country can be distributed with live broadcasts by AI, reducing costs and labor. Furthermore, by automatically generating running scores by AI instead of manually created by officials and for each team, it is possible to reduce time and improve accuracy.

"Game video data" refers to digital data that visually records the progress of a basketball game.

"Audio data" refers to data that is a digital recording of the sounds generated during a basketball game.

"Means of collecting data in real time" refers to technology and devices that allow for the acquisition of video and audio data as the match progresses.

"Means of analysis" refers to technology and software that processes collected video and audio data to understand the situation of the game and the movements of the players.

"Means for generating commentary" refers to technology or software that automatically creates text that explains the situation of the game and the movements of players based on the analysis results.

"Means of distribution along with live broadcast" refers to the technology and devices that deliver the generated commentary to viewers in real time along with the video and audio of the match.

"Youth and mini-basketball game commentary" is commentary on basketball games aimed at young people and elementary school students.

"Running score" is data that records the scores and player performances that are updated in real time during the game.

"AI automatic generation means" refers to technology and software that uses artificial intelligence to automate manual tasks and generate data.

"Means of database creation" refers to technology and software for organizing collected information and storing it in a format that is easy to search and use.

"Visualization" refers to techniques and methods for visually displaying data and making it easier to understand.

"Means to be used in development" refers to technologies and methods that use data to support the training and development of athletes.

This invention is a system that uses AI to automatically generate commentary on a basketball game and deliver it in real time during live broadcasts. A specific embodiment of this system is described below.

Server operation

1. Data collection

The server collects video and audio data of the match in real time. Specifically, it uses a camera and microphone to capture video and audio of the match. Video data is collected at a resolution of 1080p, and audio data is collected at a sampling rate of 44.1kHz.

2. Data analysis

The server analyzes the collected video and audio data. This analysis uses image recognition software (e.g. OpenCV) and voice recognition software (e.g. Google Speech-to-Text API). This allows the server to understand the situation of the game and the movements of the players. For example, it detects the action of Player A taking a shot and converts the live audio of that moment into text.

3. Description generation

The server generates commentary on the game based on the analysis results. A generative AI model (e.g. GPT-4) is used to generate this commentary. The generative AI model receives information about the game situation and player movements as input, and generates commentary in natural language. For example, it inputs information such as "Player A made a successful three-point shot" and generates commentary such as "Player A made a brilliant three-point shot."

4. Distribution

The server distributes the generated commentary along with the live broadcast. A streaming server (e.g. Wowza Streaming Engine) is used for distribution. This allows users to watch the game footage and commentary in real time over the Internet.

Device operation

1. Data reception

The device receives real-time game footage and commentary from the server using an internet connection and streaming software (e.g. VLC media player).

2. Display

The device displays the received video and commentary to the user. For display purposes, hardware such as a display and speakers are used. For example, if the user is watching on a smartphone, the video will be displayed on the smartphone screen and the commentary audio will be played through the built-in speaker.

User actions

1. Viewing

The user watches the game footage and commentary through the terminal. The user can get detailed commentary on the progress of the game and the movements of the players in real time. For example, if the user is watching on a television in the living room, the game footage will be displayed on the television screen and commentary will be played from the television speakers.

Specific examples

Example 1

Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

Player Action: Home team player A makes a three-point shot.

Example prompts for generative AI models

Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

Player Action: Home team player A makes a three-point shot.

Example of generated description

"With five minutes left in the third quarter, the home team is tied at 75 points and the away team at 70 points. Home team player A makes a great three-point shot, widening the gap even further."

In this way, users can watch detailed commentary on the progress of the match and the movements of the players in real time.

The flow of the identification process in Example 1 is explained using Figure 11.

Step 1:

The server collects video and audio data from the match in real time.

Specific tasks include using cameras and microphones to capture video and audio of the game.

The input is video and audio from the camera and microphone, and the output is video and audio data in digital format.

Video data is collected at 1080p resolution, and audio data is collected at a sampling rate of 44.1kHz.

Step 2:

The server analyzes the collected video and audio data.

Specific operations include using image recognition software (e.g. OpenCV) to detect player movements from video data, and using voice recognition software (e.g. Google Speech-to-Text API) to convert the audio data into text to provide commentary.

The input is the video and audio data collected in step 1, and the output is the analyzed player movement information and the text commentary.

For example, it detects when Player A is about to take a shot and converts the audio commentary at that moment into text.

Step 3:

The server generates a commentary of the match based on the analysis results.

Specifically, the prompt text is input into a generative AI model (e.g. GPT-4) to generate an explanatory text.

The input is the player movement information obtained in step 2 and the text commentary, and the output is the generated commentary.

For example, input information such as "Player A made a successful three-point shot" and generate an explanatory text such as "Player A made a brilliant three-point shot."

Step 4:

The server distributes the generated commentary along with the live broadcast.

Specifically, the video and commentary audio are streamed in real time using a streaming server (e.g. Wowza Streaming Engine).

The input is the explanatory text generated in step 3 and the video data collected in step 1, and the output is the video and explanatory audio delivered in real time.

Step 5:

The device receives game footage and commentary streamed from the server in real time.

Specific actions include using an internet connection and streaming software (e.g. VLC media player).

The input is streaming data from the server, and the output is the video displayed on the terminal and the commentary audio played.

Step 6:

The device displays the received video and commentary to the user.

Specific operations include showing images on the display and playing explanatory audio from the speaker.

The input is the video and commentary audio received in step 5, and the output is a live broadcast of the game provided to the user in both visual and audio.

Step 7:

Users can watch game footage and commentary through their devices.

Specific actions include watching the game footage on the device display and listening to commentary coming from the speaker.

The input is the video displayed in step 6 and the commentary audio played, and the output is the user's viewing experience.

For example, if a user is watching on a television in their living room, the game footage will be displayed on the television screen and commentary will be played from the television speakers.

(Application example 1)

Next, application example 1 of embodiment example 1 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

Existing basketball game commentary systems do not adequately provide real-time commentary, post-game highlight generation, or detailed analysis of players' performance. In addition, there is a problem that it is difficult to provide appropriate commentary immediately when a user asks about a specific player or play.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 1 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games nationwide with AI commentary, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for analyzing game video data and generating commentary in real time, a means for automatically extracting important scenes after the game ends and generating a highlight video, a means for analyzing player movements and performance and providing detailed statistical information, and a means for AI to provide commentary when a user asks a question about a specific player or play. This makes it possible to provide commentary in real time, generate post-game highlights, analyze players' performance in detail, and provide immediate commentary in response to user questions.

"A means for AI to provide commentary for live broadcasts of basketball games" refers to technology in which AI analyzes video and audio data from basketball games, understands the situation of the game and the movements of the players, and automatically generates commentary.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that uses AI to provide commentary on youth and mini-basketball games from around the country and broadcasts the games in real time with that commentary.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses AI to automatically generate scores that are manually recorded by officials and teams during a match.

"Means for analyzing game video data and generating commentary in real time" refers to technology that analyzes game video data in real time and instantly generates commentary based on the analysis results.

"Means for automatically extracting important scenes after the end of a match and generating a highlight video" refers to a technology that analyzes video data of a match after the end of the match, automatically extracts important scenes, and generates a highlight video.

"Means of analyzing player movements and performance and providing detailed statistical information" refers to technology that analyzes player movements and performance and provides detailed statistical information based on the results.

"A means for an AI to provide commentary when a user asks a question about a specific player or play" refers to technology that enables an AI to provide an appropriate commentary in response to a question when a user asks about a specific player or play.

To implement this invention, the following system configuration and processing steps are required.

System configuration

Hardware

Server: A high-performance computer for running AI models and analyzing data.

Smartphone: The device on which users watch and interact with the game commentary.

Camera: A device used to capture footage of the match.

Software

OpenCV: A library used to read and analyze video data.

TensorFlow: A framework used to run AI models.

Transformers: A library that generates explanations using the GPT-2 model.

Processing flow

1. Loading video data

The server uses OpenCV to read the game video data obtained from the camera frame by frame.

2. Frame analysis

The server analyzes the loaded frames to detect player movements and important actions. This analysis uses a deep learning model using TensorFlow.

3. Generate explanations

The server generates an explanatory text based on the analysis results. Specifically, it runs the GPT-2 model using the Transformers library and generates an explanatory text using the prompt text as input.

4. Delivering commentary

The generated commentary is delivered to smartphones in real time, allowing users to watch and listen to commentary on the match via their smartphones.

5. Highlight video generation

After the match ends, the server automatically extracts important scenes and generates a highlight video. This highlight video is also distributed to smartphones.

6. Player performance analysis

The server performs detailed analysis of the players' movements and performance and generates statistical information that users can view on their smartphones.

7. User interaction

When a user asks a question about a particular player or play, the server generates an appropriate commentary for that question and delivers it to the smartphone.

Specific examples

Scoring scene in the game: Detect the moment when Player A makes a three-point shot and generate an explanation such as "Player A scores a three-pointer."

Defensive highlights: Detects a scene where Player B successfully blocks a shot and generates a commentary such as "Player B makes a great block."

Example of a prompt

"Player A scores a three-pointer. Generate a commentary for this action."

"Player B makes a great block. Generate a commentary for this action."

In this way, the server can analyze the game video data in real time and automatically provide commentary to the user. This allows for real-time commentary, post-game highlights, detailed analysis of player performance, and instant commentary in response to user questions.

The flow of the specific processing in application example 1 is explained using Figure 12.

Step 1:

The server uses OpenCV to read the game video data captured by the camera frame by frame. The input is the video data from the camera, and the output is the individual frame images. Specifically, the server splits the video data into a series of still images and stores each frame in memory.

Step 2:

The server analyzes the loaded frames to detect player movements and important actions. A deep learning model using TensorFlow is used for this analysis. The input is the frame image obtained in step 1, and the output is information about the player movements and actions. Specifically, the server detects the positions and movements of players within the frames and identifies important actions such as points and blocks.

Step 3:

The server generates a commentary based on the analysis results. Specifically, it runs the GPT-2 model using the Transformers library and generates a commentary using the prompt as input. The input is the action information and prompt obtained in step 2, and the output is the generated commentary. The server generates a prompt corresponding to the player's action and inputs it into the GPT-2 model to obtain the commentary.

Step 4:

The server delivers the generated commentary to the smartphone in real time. The input is the commentary generated in step 3, and the output is the commentary displayed on the smartphone. Specifically, the server transmits the commentary to the smartphone via the network so that the user can view it.

Step 5:

After the match ends, the server automatically extracts important scenes and generates a highlight video. The input is the video data of the entire match, and the output is the highlight video. Specifically, the server selects highlight scenes based on important actions detected during the match and links them together to create a highlight video.

Step 6:

The server performs detailed analysis of players' movements and performance and generates statistical information. The input is frame images from the game and information about players' actions, and the output is detailed statistical information. Specifically, the server compiles data such as points, rebounds, and assists for each player and compiles it as statistical information.

Step 7:

When a user asks a question about a specific player or play, the server generates an appropriate commentary for that question and delivers it to the smartphone. The input is the user's question and prompt text, and the output is the generated commentary text. Specifically, the server analyzes the user's question, generates a corresponding prompt text, inputs it into the GPT-2 model, obtains the commentary text, and sends it to the smartphone.

(Example 2)

Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

Traditionally, commentary on basketball games required a specialized commentator, and there was the problem of difficulty in securing commentators for youth and mini-basketball games held all over the country. It was also difficult to convey the progress of the game and the movements of the players to viewers in real time, limiting the viewing experience. Furthermore, the collection and analysis of game data was done manually, which was inefficient and sometimes lacked consistency and accuracy.

The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

In this invention, the server includes means for collecting video and audio data of matches held throughout the country, means for storing the collected data in cloud storage, means for transmitting the stored data to a generative AI model and generating commentary for the match, means for synchronizing the generated commentary with the match video, means for streaming the match video with commentary, means for providing an interface for users to view, means for receiving the match video with commentary streamed from the server, and means for displaying the received video to the user. This enables live broadcasts with high-quality commentary by AI even for youth and mini-basketball matches held throughout the country, improving the viewing experience. In addition, efficient and accurate data management can be achieved because the collection and analysis of match data is automated.

"Means for collecting video and audio data from matches held around the country" refers to devices and systems that use cameras and microphones installed at match venues around the country to capture video and audio data from matches in real time and transmit it to a server.

"Means for storing collected data in cloud storage" refers to devices or systems that convert collected video and audio data into an appropriate format and store the data using cloud services.

"Means for sending stored data to a generative AI model and generating commentary on a match" refers to a device or system that retrieves data stored in cloud storage, sends prompt text to the generative AI model to analyze the data, and generates commentary on a match.

"Means for synchronizing the generated commentary with the game video" refers to a device or system that inserts the generated commentary text at a specific timestamp in the game video, allowing the video and commentary to be played in a synchronized state.

"Means for streaming game footage with commentary" refers to a device or system that encodes synchronized video data in real time and transmits it to a streaming platform for distribution to viewers.

"Means for providing an interface for users to view" refers to a device or system for displaying a viewing interface to users through a device such as a smartphone, tablet, or PC.

"Means for receiving game footage with commentary streamed from a server" refers to a device or system for receiving game footage with commentary streamed from a server in real time via an Internet connection.

"Means for displaying received video to a user" refers to a device or system for decoding received video data and displaying it on the user's device screen.

This invention is a system that broadcasts youth and mini-basketball matches held around the country with commentary by AI. A specific embodiment of this system is described below.

Server role

The server collects video and audio data from matches held around the country. Cameras and microphones installed at match venues around the country capture video and audio from matches in real time and send this data to the server. The collected data is stored in cloud storage such as Amazon S3 and Google Cloud Storage.

The server then sends the stored data to a generative AI model. Examples of generative AI models that can be used include OpenAI's GPT-4 and Google's BERT. The server retrieves the data from cloud storage and sends a prompt to the generative AI model. The prompt includes instructions for analyzing the progress of the game, the movements of players, the score, etc.

The generative AI model analyzes the data based on the prompt text and generates commentary in natural language. The generated commentary is sent back to the server in text format. The server then synchronizes the generated commentary with the game footage. This synchronization process is performed using video editing software such as FFmpeg. The server inserts the commentary text at specific timestamps in the footage, allowing the footage and commentary to be played in sync.

Finally, the server streams the match footage with commentary to a streaming platform such as YouTube Live or Twitch. The server encodes the synchronized video data in real time and sends it to the streaming platform.

Role of the device

The terminal provides the user with an interface for viewing. Devices such as smartphones, tablets, and PCs are used here. The terminal displays the viewing interface to the user through a dedicated application or web browser. The interface includes a list of matches and a search function.

The device receives game footage with commentary streamed from the server. An Internet connection is required for this reception. The device receives video data from the streaming platform in real time and buffers it. The device decodes the received video data and displays it on the user's device screen.

User Roles

Users access the system using a terminal. To do so, they use a dedicated application or a web browser. Users launch a dedicated application on their smartphone or PC, or access the system's URL using a web browser.

The user selects the match they wish to watch. For this selection, a list of matches and a search function are provided. The user scrolls through the list of matches on the interface and clicks on the match they wish to watch. They can also use the search function to find a specific match.

The user watches the commentary-enhanced video of the selected match. While watching, the user can play, pause, rewind, and perform other operations. The user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed.

Specific examples

For example, if a user wants to watch a national youth basketball tournament game on their smartphone, they would follow these steps:

1. The user launches the dedicated app on their smartphone.

2. Use the in-app search function and type in "National Youth Basketball Tournament."

3. Select the match you want to watch from the search results.

4. The app streams game footage with commentary from the server.

5. Users can watch the game on their smartphone screen and enjoy the commentary.

An example of a prompt sentence to be input to the generative AI model is, "Analyze the game video and audio data of the National Youth Basketball Tournament and generate commentary including the progress of the game, player movements, score status, etc." Using this prompt sentence, the AI analysis system generates commentary of the game.

The flow of the identification process in Example 2 is explained using Figure 13.

Step 1:

The server collects video and audio data from matches held around the country. As input, it receives real-time data from cameras and microphones installed at match venues around the country. Specifically, it transmits the video and audio captured by these devices to the server via the internet. As output, it obtains the collected video and audio data.

Step 2:

The server stores the collected data in cloud storage. As input, it receives the video and audio data collected in step 1. Specifically, it converts the data into an appropriate format and uploads it to a cloud service such as Amazon S3 or Google Cloud Storage. As output, it obtains the data stored in cloud storage.

Step 3:

The server sends the saved data to the generative AI model to generate commentary on the match. As input, it receives video and audio data obtained from cloud storage. Specifically, it sends a prompt to the generative AI model to analyze the data. The prompt includes instructions to analyze the progress of the match, player movements, the score, etc. The output is the commentary text returned from the generative AI model.

Step 4:

The server synchronizes the generated commentary with the game footage. As input, it receives the commentary text obtained in step 3 and the video data obtained from the cloud storage. Specifically, it uses video editing software such as FFmpeg to insert the commentary text at a specific timestamp in the video. As output, it obtains the game footage synchronized with the commentary.

Step 5:

The server streams the game footage with commentary. As input, it receives the game footage synchronized with the commentary obtained in step 4. Specifically, it encodes the video data in real time and transmits it to a streaming platform such as YouTube Live or Twitch. As output, it obtains the streaming video that is distributed to the viewers.

Step 6:

The terminal provides an interface for users to watch. It receives user operations as input. Its specific operation is to display the viewing interface through a dedicated application or web browser. The interface includes a list of matches and a search function. As output, it provides information that allows the user to select the match they wish to watch.

Step 7:

The terminal receives game footage with commentary that is streamed from the server. As input, it receives streaming data from the server. Specifically, it receives video data in real time via an Internet connection and buffers it. As output, it obtains the received video data.

Step 8:

The terminal displays the received video to the user. As input, it receives the video data received in step 7. Specifically, it decodes the video data and displays it on the user's device screen. As output, it obtains the video that the user watches.

Step 9:

The user accesses the system using a terminal. A dedicated application or a web browser is used as input. Specifically, the user launches a dedicated application on a smartphone or PC, or accesses the system's URL in a web browser. Access to the system is obtained as output.

Step 10:

The user selects the match they want to watch. As input, they use the match list and search functionality on the viewing interface. They scroll through the match list and click on the match they want to watch. They can also use the search functionality to search for a specific match. As output, they get the selected match.

Step 11:

The user watches the commentary-enhanced video of the selected match. As input, the match selected in step 10 is received. Specifically, the user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed. As output, the video of the match being watched is obtained.

(Application example 2)

Next, application example 2 of embodiment example 2 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

In conventional basketball game commentary systems, live broadcasts and commentary of games were dependent on human labor, making it difficult to provide youth and mini-basketball games held all over the country to a wide audience. In addition, there was a lack of game highlights and replay functions, and post-game statistical data and analysis reports, making it difficult to increase viewer satisfaction. Furthermore, there was no function for users to follow their favorite teams or players, making it difficult to meet individual needs.

The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for AI to provide commentary on the live broadcast of a basketball game, a means for distributing commentary of youth and mini-basketball games nationwide with AI live broadcast, a means for AI to automatically generate running scores created by officials and people for each team, a means for collecting video and audio data of the game and analyzing it with AI to generate commentary, a means for distributing the game in real time through an application installed on a smartphone and providing commentary by AI, a means for providing highlights and replay functions of the game, a means for providing a function that allows users to follow their favorite teams and players, and a means for providing statistical data and analysis reports after the game. This makes it possible to widely provide youth and mini-basketball games held all over the country to viewers and increase the satisfaction of the viewers.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to generate commentary in real time during a basketball game and provide it to viewers.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to technology that broadcasts youth and mini-basketball games held around the country in real time with commentary by artificial intelligence.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Method of collecting video and audio data from the match and analyzing it with AI to generate commentary" refers to technology that collects video and audio data from the match and uses artificial intelligence to analyze it and generate commentary.

"Means of broadcasting matches in real time and providing commentary by AI through an application installed on a smartphone" refers to a technology that uses a smartphone application to broadcast matches in real time and provide commentary by artificial intelligence.

"Means for providing match highlight and replay features" refers to technology that provides the functionality to compile important scenes and plays during a match as highlights or play them back as replays.

"Means for providing a function that enables users to follow their favorite teams and players" refers to technology that provides a function that enables users to select and follow their favorite teams and players and receive information about them on a priority basis.

"Means for providing post-match statistical data and analytical reports" refers to technology that generates match statistical data and analytical reports after the match ends and provides them to users.

The following system configuration is described as a form for implementing this invention.

System configuration

The system includes a server, a smartphone, and an AI model. The server collects video and audio data from the match, analyzes it using the AI model, and generates commentary. The smartphone provides an application that enables users to watch the match in real time and receive commentary by the AI.

Hardware and software used

Server: A server with high-performance data processing capabilities (e.g. AWS EC2 instance)

Smartphone: iOS or Android device

AI model: OpenAI's GPT-4

Data processing library: OpenCV (for image processing), Python (programming language)

Data processing and data calculations

1. Collecting video and audio data from the match:

The server collects video and audio data in real time from youth and mini-basketball matches held across the country. This is done using input devices such as cameras and microphones.

2. AI-based analysis and explanation generation:

The server uses OpenCV to analyze the collected video and audio data and extracts important scenes and plays from the match. Based on the extracted information, it uses OpenAI's GPT-4 to generate commentary. Specifically, it uses the following prompt sentence:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

3. Real-time broadcasting and commentary:

The server delivers the game footage along with the generated commentary to smartphones in real time. An application installed on the smartphone allows users to receive the AI commentary while watching the game.

4. Providing highlight and replay features:

The server compiles highlights of key scenes during the match and provides a replay function, allowing users to rewatch key moments of the match.

5. User following feature:

The smartphone application provides users with the ability to follow their favorite teams and players. This allows users to receive information about the teams and players they follow preferentially.

6. Providing post-match statistics and analysis reports:

After the match ends, the server generates statistical data and analysis reports and provides them to users via a smartphone application, allowing users to obtain detailed analysis information about the match.

Specific examples

For example, when a user is watching a youth basketball game through a smartphone application, the server delivers real-time game footage and AI commentary. The moment Player A scores a goal during the game, the server analyzes the scene and generates the following prompt:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

Based on this prompt, the AI model will generate commentary and provide it to the user. Users can enjoy the AI commentary in real time while watching the match. In addition, after the match, users can obtain detailed information about the match through statistical data and analysis reports.

The flow of the specific processing in application example 2 is explained using Figure 14.

Step 1:

The server collects video and audio data from youth and mini-basketball matches held around the country. As input, it receives real-time data from input devices such as cameras and microphones, and as output, it stores this data in storage. In concrete terms, the server receives video and audio data sent from each match venue, converts it into an appropriate format, and stores it.

Step 2:

The server analyzes the collected video and audio data. It uses the video and audio data saved in step 1 as input, and extracts information about important scenes and plays in the match as output. Specifically, the server uses OpenCV to analyze the video data frame by frame and identify the movements of players and the position of the ball. It also extracts events such as goals and fouls from the audio data.

Step 3:

The server generates a prompt sentence based on the analysis results. As input, it uses information about important scenes and plays in the match extracted in step 2, and as output, it generates a prompt sentence to pass to the AI model. Specifically, the server converts information such as player movements and scoring status into text format and generates a prompt sentence like the following:

Example prompt:

Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A has scored a goal.

Step 4:

The server sends the generated prompt text to the AI model and generates an explanation. It uses the prompt text generated in step 3 as input and receives the explanation text generated from the AI model as output. In concrete terms, the server calls OpenAI's GPT-4 API, sends the prompt text, and generates an explanation.

Step 5:

The server delivers the generated commentary and game footage to smartphones in real time. It uses the commentary text generated in step 4 and the video data collected in step 1 as input, and generates data to be delivered to the user's smartphone as output. In concrete terms, the server overlays the commentary text on the video data and transmits it to the smartphone in streaming format.

Step 6:

Users watch the game through a smartphone application and receive commentary by AI. As input, it uses video and commentary data distributed from the server, and as output, it displays the game video and commentary to the user. In concrete terms, the smartphone application decodes the received data and displays it on the screen.

Step 7:

The server summarizes important scenes during the match as highlights and provides a replay function. It uses the information of important scenes extracted in step 2 as input and generates highlight videos as output. In concrete terms, the server edits the important scenes and saves them in a format that users can replay.

Step 8:

Users follow their favorite teams and players through a smartphone application. The application uses the user's selection information as input, and prioritizes displaying information about the followed teams and players as output. In concrete terms, the application records the user's selection, and prioritizes retrieving and displaying related information.

Step 9:

After the match ends, the server generates statistical data and analysis reports and provides them to the user. It uses data collected during the match as input and generates statistical data and analysis reports as output. In concrete terms, the server aggregates the match data, creates a report in a visually easy-to-understand format, and provides it to the user through a smartphone application.

(Example 3)

Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

In traditional basketball games, in order to grasp the score and foul situation in real time, it was necessary to manually create a running score, which was time-consuming and labor-intensive. In addition, there was a lack of a system to effectively collect and analyze data on teams and players across the country and use it for player development. This meant that there was insufficient visualization of player performance and feedback for development.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes a means for receiving video data and audio data of a match, a means for analyzing the received video data, a means for analyzing the received audio data, a means for extracting the score situation and the foul situation from the analysis result, a means for generating a running score based on the extracted information, a means for updating the generated running score in real time and transmitting it to the terminal, a means for displaying the running score transmitted to the terminal, a means for collecting information such as the match results and stats of teams and players nationwide, a means for database-izing the collected information, a means for analyzing the databased information and evaluating the performance of players, a means for generating feedback for player development based on the analysis result, a means for transmitting the generated feedback to the terminal, and a means for displaying the feedback transmitted to the terminal. This makes it possible to grasp the progress of the match in real time, and effectively visualize the performance of players and provide feedback for development.

"Game video data" refers to video files captured by a camera showing a basketball game.

"Audio data" refers to audio files recorded by a microphone during a basketball game.

"Means of receiving" refers to the function that allows the server to obtain data from outside.

"Means of analysis" refers to the function for processing received data and extracting necessary information.

"Scoring status" is information that shows how many points each team has scored during a match.

"Foul situation" is information that shows which players committed what type of fouls during the game.

The "running score" is a record of points and fouls that is updated in real time as the game progresses.

"Means of updating in real time" refers to a function that instantly reflects data as the match progresses.

"Terminal" refers to the device that a user uses to view information.

"Means of display" refers to the functionality for visually presenting information on a terminal.

"Match results" refers to information showing the final score and outcome after the match has ended.

"Stats" are statistical data that quantify the performance of a player or team.

"Means of collection" refers to the function for collecting the necessary data.

"Means of database creation" refers to the function of organizing and storing collected data.

"Means of analysis" refers to the function of processing databased information and deriving meaningful results.

"Feedback" refers to advice or evaluation provided based on the analysis results.

"Visualization" refers to displaying data visually to make it easier to understand.

"Development" refers to training and guidance to improve players' abilities.

This invention is a system for grasping the score and foul situation in a basketball game in real time and evaluating and developing the performance of players. A specific embodiment of this system is described below.

System configuration

Hardware

Server: A computer with high-performance computing power that receives, analyzes, generates, and transmits data.

Device: The device on which the user checks information, including smartphones, tablets, and computers.

Camera: A device used to capture video data of a match.

Microphone: A device used to capture audio data from a match.

Software

Video analysis software: Analyzes video data using libraries such as OpenCV.

Speech analysis software: Use libraries such as LibROSA to analyze speech data.

Database management system: Collected data is stored in a database using a system such as MySQL.

Data analysis tools: Analyze data using tools such as Pandas and NumPy.

Generative AI models: Use models such as GPT-4 to generate running scores and feedback.

System operation

Data reception and analysis

The server receives video and audio data of the game in real time from the cameras and microphones. The received video data is analyzed using video analysis software to detect goals and fouls. The audio data is analyzed using audio analysis software to analyze the commentary and crowd reactions.

Information extraction and running score generation

The server extracts the scoring and foul situations from the analysis results. Based on the extracted information, a generative AI model is used to generate a running score. The generated running score is updated in real time and sent to the terminal.

Display data

The device displays the received running score, allowing the user to follow the progress of the match in real time.

Data collection and database creation

The server collects information such as match results and stats for teams and players across the country. The collected information is stored in a database using a database management system.

Data analysis and feedback generation

The server analyzes the databased information and evaluates the player's performance. Based on the analysis results, a generative AI model is used to generate feedback for the player's development. The generated feedback is sent to the terminal, which displays it. This allows the user to check the analysis results of the player's performance and advice for development.

Specific examples

When a user is watching a basketball game, the server analyzes the video and audio of the game and automatically extracts information about scores and fouls.

The server generates information in real time, such as "Team A has scored" or "Team B has committed a foul," and sends it to the terminal.

The device displays this information, allowing users to check the progress of the match in real time.

Example of a prompt

"Analyze video and audio data from basketball games, extract the scoring and foul situations, and generate a running score."

"Analyze the game results and stats of basketball teams and players nationwide, evaluate the performance of players, and generate feedback for training." The flow of the specific process in Example 3 is explained with reference to FIG. 15.

Step 1:

The server receives video and audio data from the match.

Input: Real-time data from camera and microphone

Specific operation: Receives video and audio transmitted from the camera and microphone via the network.

Output: Received video and audio data

Step 2:

The server analyzes the received video data.

Input: Received video data

Specific operation: Use video analysis software (e.g. OpenCV) to detect scoring and foul scenes from video data.

Output: Data with timestamps for goals and fouls

Step 3:

The server analyzes the received audio data.

Input: Received audio data

Specific actions: Use audio analysis software (e.g. LibROSA) to analyze commentary and audience reactions from audio data.

Output: Timestamped data of commentary and audience reactions

Step 4:

The server extracts the scoring and foul situations from the analysis results.

Input: Timestamped data on goals and fouls, as well as timestamped data on commentary and crowd reactions

Specific operation: Integrate the results of video and audio analysis to extract the occurrence of goals and fouls.

Output: Scoring and foul situation data

Step 5:

The server generates a running score based on the extracted information.

Input: Scoring and foul situation data

Specific behavior: Uses a generative AI model (e.g. GPT-4) to generate points and foul information in text format.

Output: Text data of running score

Step 6:

The server updates the generated running score in real time and sends it to the device.

Input: Text data of running score

Specific operation: Sends the running score to the device using WebSocket or HTTP protocol.

Output: Running score sent to the device

Step 7:

The device will display the received running score.

Input: Running score sent to the device

Specific operation: Visually display the running score on the device display.

Output: A running score that the user can visually check

Step 8:

The server collects information such as match results and stats for teams and players across the country.

Input: Data from each team's official website and sports data services

Specific actions: Collect match results and stats using web scraping and APIs.

Output: Collected match results and stats data

Step 9:

The server stores the collected information in a database.

Input: Collected match results and stats data

Specific action: Store data in a table using a database management system (e.g. MySQL).

Output: Match results and stats data stored in a database

Step 10:

The server analyzes the databased information and evaluates the players' performance.

Input: Match results and stats data stored in the database

Specific operations: Use data analysis tools (e.g. Pandas, NumPy) to calculate players' shooting success rates, defensive strength, etc.

Output: Player performance evaluation data

Step 11:

The server generates feedback for player development based on the analysis results.

Input: Player performance evaluation data

Specific actions: Use a generative AI model (e.g. GPT-4) to generate textual feedback for player development.

Output: Text data of feedback for training

Step 12:

The server sends the generated feedback to the device.

Input: Text data for feedback for training

Specific behavior: Sends feedback to the device using WebSocket or HTTP protocol.

Output: Feedback sent to the device

Step 13:

The device will display the feedback it receives.

Input: Feedback sent to device

Specific action: Visually display feedback on the device display.

Output: Visual feedback for the user

(Application example 3)

Next, application example 3 of embodiment example 3 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

Traditional live broadcasts and commentary of basketball games rely on human commentators, which can make it difficult to provide accurate information in real time. Commentary on youth and mini-basketball games across the country also relies on human input, making it difficult to provide information efficiently. Furthermore, when monitoring production lines in factories, detecting abnormalities and improving production efficiency are issues. To solve these issues, an automated system that utilizes AI is needed.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 3 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games from around the country with AI live coverage, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for analyzing camera footage and audio data within the factory to detect production status and abnormal occurrences in real time and provide scores, and a means for storing past production data in a database and providing feedback for efficiency. This allows for accurate understanding of the progress of the game and the status of the production line in real time, making it possible to provide efficient information and improve production efficiency.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to automatically provide commentary on the progress of a basketball game and plays in real time.

"A means of broadcasting commentary on youth and mini-basketball games nationwide with live coverage by AI" refers to a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games nationwide and broadcasts that commentary in real time.

"A means of using AI to automatically generate running scores that are created manually by officials and for each team" refers to technology in which artificial intelligence automatically analyzes points, fouls, and other scores as the game progresses, and generates scores in real time.

"A means of analyzing camera footage and audio data within a factory to detect and score production status and abnormalities in real time" refers to a technology in which artificial intelligence analyzes video and audio data obtained from cameras and microphones installed within the factory, detects the status and abnormalities of the production line in real time, and displays the results as a score.

"Means of compiling past production data into a database and providing feedback for improving efficiency" refers to a technology that stores past production data collected within a factory in a database, analyzes the data, and provides feedback for improving production efficiency.

The following system configuration is described as a form for implementing this invention.

System configuration

Hardware

Camera: Cameras installed in the factory capture video data.

Microphones: Microphones installed in the factory capture audio data.

Server: A server for analyzing and storing data.

Device: The device (PC, smartphone, etc.) used by the administrator to check the results.

Software

OpenCV: A library for acquiring and preprocessing camera images.

TensorFlow/Keras: Libraries for loading AI models and making predictions.

SQLite: A database management system for storing production data.

Data processing and data calculations

The server acquires camera footage and audio data from within the factory in real time and performs preprocessing using OpenCV. The preprocessed data is input into an AI model using TensorFlow/Keras to detect production status and abnormalities. The detection results are displayed as a score and stored in a SQLite database.

Specific examples

For example, if an abnormality occurs in a factory, it will be detected from camera footage and notified to managers in real time. It can also analyze past production data and provide feedback to improve efficiency.

Example of a prompt

An example of a prompt to input to the generative AI model is as follows:

"Please create an AI model that analyzes camera footage in the factory and detects abnormalities on the production line in real time. Please also add a function to notify the administrator if an abnormality is detected. Also, please create a system that stores past production data in a database and provides feedback to improve efficiency."

In this way, the server can monitor the status of the production line in real time and detect any anomalies. It can also provide feedback for efficiency improvements based on past data.

The flow of the specific processing in application example 3 is explained using Figure 16.

Step 1:

The server captures video and audio data in real time from cameras and microphones installed in the factory. The input is the raw data from the cameras and microphones, and the output is the captured video and audio data. In concrete terms, the server connects to the cameras and microphones and starts the data stream.

Step 2:

The server preprocesses the acquired video data using OpenCV. The input is the video data acquired in step 1, and the output is the preprocessed video data. Specifically, the server resizes the video data to 224x224 pixels and normalizes it.

Step 3:

The server inputs the preprocessed video data into the AI model using TensorFlow/Keras to make predictions. The input is the video data preprocessed in step 2, and the output is the predicted results of production status and abnormality occurrence. In concrete terms, the server loads the AI model, inputs the preprocessed data into the model, and makes predictions.

Step 4:

The server displays the prediction result as a score and saves it in a SQLite database. The input is the prediction result obtained in step 3, and the output is the displayed score and the data saved in the database. In concrete terms, the server converts the prediction result into a score, displays it on the screen, and inserts it into the database.

Step 5:

The terminal receives the score and anomaly detection notifications sent from the server and displays them to the administrator. The input is the score and notifications sent from the server, and the output is the display information that the administrator can check. In concrete terms, the terminal receives data from the server and displays it on the user interface.

Step 6:

The server analyzes past production data and generates feedback for efficiency improvements. The input is past production data stored in a database, and the output is feedback information for efficiency improvements. In concrete terms, the server retrieves past data from the database and applies an analysis algorithm to generate feedback.

Step 7:

The terminal receives feedback information sent from the server and displays it to the administrator. The input is the feedback information sent from the server, and the output is the display information that the administrator can check. In concrete terms, the terminal receives feedback information from the server and displays it on the user interface.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

"Example 1"

In one embodiment of the present invention, a system in which AI provides commentary on a live broadcast of a basketball game is combined with an emotion engine that recognizes the user's emotions. Specifically, the emotion engine recognizes the emotions (happiness, sadness, surprise, etc.) that the user expresses while watching the game, and adjusts the content and tone of the commentary according to those emotions. For example, if the user expresses joy at how the game is progressing, the commentary will also share that joy. This allows the user to hear commentary that reflects their own emotions, further improving their satisfaction with watching the game.

"Example 2"

The emotion engine is also incorporated into a system that broadcasts commentary on youth and mini-basketball games from around the country, complete with live coverage by AI. Specifically, the emotion engine recognizes the emotions of users watching youth and mini-basketball games, and adjusts the content and tone of the commentary according to those emotions. For example, if a user is impressed by their child's performance, the commentary will be one that shares that emotion. This allows users to hear commentary that reflects their own emotions, further increasing their satisfaction with watching the games.

"Example 3"

Furthermore, emotion engines are also being incorporated into systems that use AI to automatically generate running scores that are manually created by officials and for each team. Specifically, the emotion engine recognizes the user's emotions regarding the score situation of the match, and adjusts the way the score is displayed according to those emotions. For example, if the user is happy about a score, the score display will also be displayed in a way that emphasizes that joy. This allows users to see a score display that reflects their own emotions, further increasing their satisfaction when watching the match.

The process flow for each example is explained below.

"Example 1"

Step 1: The AI begins commentating on a live broadcast of a basketball game.

Step 2: The emotion engine recognizes the user's emotions in real time.

Step 3: The AI adjusts the content and tone of the commentary based on the emotion recognized by the emotion engine.

"Example 2"

Step 1: AI will begin providing live commentary for youth and mini-basketball matches across the country.

Step 2: The emotion engine recognizes the emotions of users watching the game in real time.

Step 3: The AI adjusts the content and tone of the commentary based on the emotion recognized by the emotion engine.

"Example 3"

Step 1: AI automatically generates a running score based on the game's scoring situation.

Step 2: The emotion engine recognizes the user's emotions in real time regarding the score of the game.

Step 3: The AI adjusts the way the score is displayed depending on the emotion recognized by the emotion engine.

(Example 1)

Next, a first embodiment of the first embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

In conventional basketball game commentary systems, it was difficult to analyze the game situation and player movements in real time and provide commentary that corresponded to the user's emotions. In addition, there was a lack of means to efficiently distribute commentary of youth and mini-basketball games across the country, or to automatically generate running scores that are created manually by officials and for each team. Furthermore, there was insufficient use of emotion recognition technology to improve the user's viewing experience.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for providing commentary on live coverage of basketball games using artificial intelligence, means for delivering commentary on youth and mini-basketball games across the country with live coverage by artificial intelligence, means for automatically generating running scores using artificial intelligence that are created manually for each official and team, means for analyzing game video data, means for analyzing game audio data, means for receiving and analyzing user emotion data, means for automatically generating commentary based on the analysis results and adjusting the content and tone of the commentary according to the user's emotions, and means for delivering the adjusted commentary together with the live coverage. This makes it possible to analyze the situation of the game and the movements of the players in real time and provide commentary according to the user's emotions.

"Means for artificial intelligence to provide commentary for live broadcasts of basketball games" refers to technology that analyzes video and audio data of basketball games, and automatically generates commentary based on the results of that analysis using artificial intelligence, which is then distributed along with the live broadcast.

"A means of delivering commentary on youth and mini-basketball games nationwide with live coverage by artificial intelligence" refers to a technology in which artificial intelligence automatically generates commentary for youth and mini-basketball games nationwide and delivers that commentary along with live coverage.

"Method of automatically generating running scores using artificial intelligence that are created manually for each official or team" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Means for analyzing game video data" refers to technology that analyzes game video data frame by frame to identify the positions and movements of players, the position of the ball, etc.

"Means for analyzing audio data from a match" refers to technology that analyzes audio data from a match and identifies the content of the commentary and the reactions of the audience.

"Means for receiving and analyzing user emotion data" refers to technology that captures the user's facial expressions and voice, and analyzes that data to identify the user's emotions.

"Means for automatically generating commentary based on analysis results and adjusting the content and tone of the commentary according to the user's emotions" refers to a technology that automatically generates commentary based on the analysis results of video and audio data, and further adjusts the content and tone of the commentary taking into account the user's emotional data.

"Means for delivering adjusted commentary together with live broadcast" refers to technology that delivers adjusted commentary in real time to a user's device together with the live broadcast.

Form for implementing the invention

This invention is a system in which artificial intelligence provides commentary for live coverage of basketball games, analyzing video and audio data from the game and providing commentary that reflects the user's emotions. A specific embodiment of this system is shown below.

Hardware and software used

Hardware: High-performance servers, user devices (PCs, smartphones, tablets, etc.)

Software: Video analysis software (e.g. OpenCV), audio analysis software (e.g. Google Cloud Speech-to-Text API), emotion engines (e.g. Microsoft Azure Emotion API), generative AI models (e.g. GPT-4)

System configuration

1. The server receives the game video and audio data in real time. It uses a high-speed network connection to receive the streaming data sent from the game venue.

2. The server uses video analysis software to analyze the received video data frame by frame. Specifically, computer vision techniques are used to identify the positions of players, their movements, and the position of the ball. For example, OpenCV is used.

3. The server uses voice analysis software to analyze the received voice data. Specifically, it uses voice recognition technology to identify the content of the commentary and the reactions of the audience. For example, it uses the Google Cloud Speech-to-Text API.

4. The server uses a generative AI model to automatically generate commentary on the match based on the results of video and audio analysis. Specifically, it uses a natural language generation model such as GPT-4 to generate sentences that explain the situation of the match and the movements of the players.

5. The device captures the user's facial expressions and voice in real time and transmits them as emotion data to the server. Specifically, the device uses a camera and microphone.

6. The server uses an emotion engine to analyze the received emotion data. Specifically, it uses emotion recognition technology to identify emotions such as happiness, sadness, surprise, etc. from the user's facial expressions and voice. For example, it uses the Microsoft Azure Emotion API.

7. The server adjusts the content and tone of the generated commentary according to the user's emotions analyzed by the server. Specifically, if the user is happy, the tone of the commentary is brightened and more positive expressions are included.

8. The server delivers the adjusted commentary along with the live broadcast to the user's device. Specifically, it delivers in real time using streaming technology. For example, WebRTC is used.

Specific examples

For example, if a user expresses joy while watching a game, the emotion engine will recognize that joy. Based on that information, the server will lighten the tone of the commentary generated by the generative AI model and adjust it to share the joy. This allows users to hear commentary that reflects their own emotions, improving their satisfaction watching the game.

Example of a prompt

"Design an AI system to provide commentary for live broadcasts of basketball games. It must analyze video and audio data from the game, understand the game situation and the movements of the players, and incorporate an emotion engine that adjusts the content and tone of the commentary according to the user's emotions."

The flow of the identification process in Example 1 is explained using Figure 17.

Step 1:

The server receives the video and audio data of the match in real time. The input is streaming data sent from the match venue, and the output is the video and audio data stored in the server. Specifically, the server receives the data using a high-speed network connection and stores it in an appropriate format.

Step 2:

The server uses video analysis software to analyze the received video data frame by frame. The input is the saved video data, and the output is the analysis results, such as the positions and movements of the players and the position of the ball. In concrete terms, it uses OpenCV to analyze each frame and identify the positions of the players and the ball.

Step 3:

The server uses voice analysis software to analyze the received voice data. The input is the stored voice data, and the output is the analysis results in the form of text that contains the commentary and the reactions of the audience. Specifically, the Google Cloud Speech-to-Text API is used to convert the voice data into text and extract key information.

Step 4:

The server uses a generative AI model to automatically generate commentary on the match based on the results of video and audio analysis. The input is the results of video and audio analysis, and the output is the generated commentary. Specifically, it uses a natural language generation model such as GPT-4 to generate sentences that explain the situation of the match and the movements of the players.

Step 5:

The device captures the user's facial expressions and voice in real time and sends them to the server as emotion data. The input is the user's facial expressions and voice, and the output is the emotion data sent to the server. In concrete terms, the device uses a camera and microphone to capture the user's facial expressions and voice, and sends them to the server in real time.

Step 6:

The server uses an emotion engine to analyze the received emotion data. The input is the emotion data sent, and the output is the analysis results that identify the user's emotion. Specifically, it uses the Microsoft Azure Emotion API to identify emotions such as joy, sadness, and surprise from the user's facial expressions and voice.

Step 7:

The content and tone of the generated commentary is adjusted according to the user's emotions analyzed by the server. The input is the result of emotion analysis and the generated commentary, and the output is the adjusted commentary. Specifically, if the user shows joy, the tone of the commentary is brightened and more positive expressions are used.

Step 8:

The server delivers the adjusted commentary to the user's device along with the live broadcast. The input is the adjusted commentary and the live broadcast data, and the output is the live broadcast with real-time commentary delivered to the user's device. In concrete terms, it uses WebRTC to deliver the adjusted commentary in real time.

(Application example 1)

Next, application example 1 of embodiment example 1 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

Conventional basketball game commentary systems had the problem that the commentary was uniform and could not flexibly respond to the emotions of the viewers. In addition, improving the quality of the live broadcast and commentary of the game required a large number of people, which was costly and laborious. Furthermore, there was a lack of efficient means to provide commentary on youth and mini-basketball games across the country.

The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games nationwide with AI live coverage, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions, a means for generating commentary using a generation AI model, and a means for generating commentary by inputting a prompt sentence. This enables flexible commentary according to the viewer's emotions, improving the quality of live coverage and commentary of games. In addition, commentary on youth and mini-basketball games nationwide can be efficiently performed.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to a means that uses artificial intelligence technology to analyze video and audio data of basketball games, understand the situation of the game and the movements of the players, and automatically generate commentary.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a means of broadcasting live coverage including commentary on youth and mini-basketball games from around the country with AI providing commentary.

"Means of using AI to automatically generate running scores that are created manually by officials and teams" refers to a means of using AI to automatically generate scores that are manually created by officials and teams during a match.

"Means for adjusting the content and tone of commentary using an emotion engine that recognizes the user's emotions" refers to means that use emotion recognition technology to recognize the user's emotions (happiness, sadness, surprise, etc.) and adjust the content and tone of the commentary in accordance with those emotions.

"Means for generating commentary using a generative AI model" refers to means for generating commentary based on the situation of a match using a generative AI model (e.g., GPT-3).

"Means for inputting a prompt sentence and generating an explanation" refers to means for inputting a specific prompt sentence into a generative AI model and generating an explanation based on that prompt.

A system for implementing this invention includes a means for AI to provide commentary for live coverage of basketball games, a means for distributing commentary for youth and mini-basketball games across the country with AI commentary, a means for AI to automatically generate running scores that are created manually for each official or team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions, a means for generating commentary using a generative AI model, and a means for generating commentary by inputting a prompt text.

Hardware and software used

Hardware: Smartphone camera and microphone

Software: OpenCV, Keras, Transformers

Data processing and data calculations

Server

The server receives and analyzes video and audio data from the basketball game in real time. It uses OpenCV to acquire the video data and Keras to run an emotion recognition model. The emotion recognition model analyzes the user's facial expressions and recognizes emotions such as happiness, sadness, and surprise.

Terminal

The device generates commentary using a generative AI model (e.g., GPT-3). A prompt sentence is input into the generative AI model, which generates commentary based on the prompt. Examples of prompt sentences include "The user is happy. Generate a joyful commentary for the basketball game." and "The user is sad. Generate a comforting commentary for the basketball game."

User

Users watch the game through their smartphone's camera and microphone, and their emotions are recognized. The emotion engine adjusts the content and tone of the commentary based on the user's emotions. For example, if the user expresses joy at a highlight scene in the game, the commentary might say something like, "Great play! We've been waiting for this moment!" Conversely, if the user expresses sadness at a comeback scene in the game, the commentary might say something like, "It's a disappointing result, but there's still a chance. Let's look forward to the next play."

In this way, flexible commentary based on the viewer's emotions becomes possible, improving the quality of live game coverage and commentary. It also enables efficient commentary of youth and mini-basketball games across the country.

The flow of the specific processing in application example 1 is explained using Figure 18.

Step 1:

The server receives video and audio data from the smartphone's camera and microphone in real time. The input is the video and audio data from the camera and microphone, and the output is the data ready for analysis.

Step 2:

The server processes the video data using OpenCV to detect the user's face. The input is the video data received in step 1, and the output is the detected face area. Specifically, the server converts the video data to grayscale and applies a face detection algorithm.

Step 3:

The server runs an emotion recognition model using Keras to recognize the user's emotions. The input is the face region detected in step 2, and the output is the recognized emotion (happiness, sadness, surprise, etc.). Specifically, the server resizes the face region to an appropriate size and inputs it into the emotion recognition model.

Step 4:

The server generates a prompt sentence to be input to the generative AI model based on the recognized emotion. The input is the emotion recognized in step 3, and the output is the generated prompt sentence. Specifically, it selects a template sentence according to the emotion and generates the prompt sentence.

Step 5:

The server generates an explanation using a generative AI model (e.g., GPT-3). The input is the prompt text generated in step 4, and the output is the generated explanation text. In concrete terms, the prompt text is input into the generative AI model, and an explanation text is generated.

Step 6:

The server delivers the generated explanatory text to the user in real time. The input is the explanatory text generated in step 5, and the output is the explanatory audio delivered to the user's smartphone. In concrete terms, the explanatory text is input into a voice synthesis engine, and audio data is generated and delivered.

Step 7:

Users watch the game and listen to the commentary through their smartphones. The input is the commentary audio delivered from the server, and the output is an improvement to the user's viewing experience. In concrete terms, the commentary audio is played through the smartphone's speakers.

(Example 2)

Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

Traditional basketball game commentary relied on the commentator's subjective judgment, making it difficult to provide commentary that reflected the emotions of viewers. In addition, there was a lack of systems to provide youth and mini-basketball games from across the country to a wide audience, making it difficult to grasp the progress of the game and the performance of the players in real time. Furthermore, manually creating a running score for the game was time-consuming and labor-intensive, so efficient management was required.

The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

In this invention, the server includes means for providing commentary on live coverage of basketball games using artificial intelligence, means for distributing commentary on youth and mini-basketball games from around the country with live coverage by artificial intelligence, means for automatically generating running scores using artificial intelligence that are created manually for each official and team, means for collecting game video and audio data, means for analyzing the collected data and generating commentary, means for recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions, and means for distributing the generated commentary together with the game video. This makes it possible to provide commentary that corresponds to the viewer's emotions and improve satisfaction with viewing the game.

"Means for artificial intelligence to provide commentary for live broadcasts of basketball games" refers to technology in which artificial intelligence analyzes the progress of a basketball game and the performance of players, and generates commentary in real time.

"A means of broadcasting commentary on youth and mini-basketball games across the country with live coverage by artificial intelligence" refers to a technology in which artificial intelligence analyzes youth and mini-basketball games held across the country, generates commentary, and broadcasts it in real time.

"Means for automatically generating running scores using artificial intelligence that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

"Means for collecting game video and audio data" refers to technology that uses cameras and microphones installed at the game venue to collect game video and audio data in real time.

"Means of analyzing collected data and generating commentary" refers to technology in which artificial intelligence analyzes collected video and audio data and generates commentary based on the progress of the match and the performance of the players.

"Means of recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to technology that analyzes the viewer's facial expressions and tone of voice, and adjusts the content and tone of the commentary according to those emotions.

"Means for distributing generated commentary together with game footage" refers to technology for distributing commentary generated by artificial intelligence together with game footage via a streaming platform.

This invention is a system in which artificial intelligence (AI) broadcasts live basketball games, and also distributes live commentary by AI for youth and mini-basketball games across the country. It also includes a function that allows the AI to automatically generate running scores that are currently created manually by officials and for each team.

The server collects video and audio data from the match in real time using cameras and microphones installed at the match venue. Specifically, it uses hardware such as Sony's 4K cameras and Shure's high-sensitivity microphones. The collected data is stored in a database on the server.

The server then inputs the stored video and audio data into an AI model. This AI model uses a generative AI model such as OpenAI's GPT-4 or Google's BERT. The AI model analyzes the input data and generates commentary based on the progress of the match and the performance of the players. For example, it analyzes the moment when Player A makes a shot and generates commentary such as, "Player A made a great shot!".

In addition, the device will recognize the emotions of the user watching the game. Specifically, it will use the device's camera and microphone to capture the user's facial expressions and tone of voice. For example, Apple's Face ID technology or Amazon's Alexa will be used for this purpose. The device will then send the captured data to a server.

The server receives the user's emotion data sent from the terminal and analyzes it using an emotion engine. The emotion engine uses, for example, Affectiva or IBM Watson's emotion analysis API. If the user is emotional, the server adjusts the tone of the commentary to be more emotional and generates a commentary such as, "Player A's shot was really amazing! The crowd is really excited!".

Finally, the server distributes the generated commentary along with the game footage. Streaming platforms such as YouTube and Twitch are used for distribution. Users can watch the game through these platforms and enjoy the commentary according to their emotions.

As a concrete example, let's say a youth basketball game is taking place. The server collects video and audio data in real time from Sony's 4K cameras and Shure's high-sensitivity microphones installed at the game venue. The collected data is then input into OpenAI's GPT-4, which analyzes the moment Player A makes a shot. The AI model generates a commentary such as, "Player A made a great shot!".

Meanwhile, the device uses Apple's Face ID technology to capture the facial expressions of the user watching the game and recognizes that they are emotionally moved. The server then uses Affectiva's emotion analysis API to analyze the user's emotions and adjust the tone of the commentary to be more emotional. It generates a commentary such as, "Player A's shot was really amazing! The crowd is really excited!" and distributes it via YouTube.

Examples of prompts might include:

"Analyze video and audio data from youth basketball games to generate commentary on game progress and player performance. Also recognize viewer emotions and adjust the content and tone of the commentary accordingly."

In this way, the server can provide users with commentary that reflects their emotions, improving their satisfaction with watching the game.

The flow of the identification process in Example 2 is explained using Figure 19.

Step 1:

The server uses cameras and microphones installed at the venue to collect video and audio data of the match in real time. Specifically, it uses Sony 4K cameras and Shure high-sensitivity microphones. The input is the video and audio data of the match, and the output is the collected video and audio data. This data is stored in a database on the server.

Step 2:

The server inputs the stored video and audio data into an AI model. This AI model uses, for example, OpenAI's GPT-4. The input is the collected video and audio data, and the output is analyzed data on the progress of the match and the performance of the players. The AI model analyzes the movements of the players and the progress of the match from the video data, and analyzes the commentary and audience reactions from the audio data.

Step 3:

The server generates commentary on the match based on the data analyzed by the AI model. The input is the analyzed data on the progress of the match and the performance of the players, and the output is the generated commentary. For example, the server analyzes the moment when Player A makes a shot and generates a commentary such as, "Player A made a great shot!".

Step 4:

The device recognizes the emotions of users watching a game. Specifically, it uses the device's camera and microphone to capture the user's facial expressions and tone of voice. The input is data about the user's facial expressions and tone of voice, and the output is the recognized user's emotion data. This uses Apple's Face ID technology and Amazon's Alexa.

Step 5:

The server receives the user's emotion data sent from the terminal and analyzes it using an emotion engine. The input is the user's emotion data, and the output is the analyzed emotion information. The emotion engine uses, for example, Affectiva's emotion analysis API. If the user is emotional, the server adjusts the tone of the commentary to be emotional.

Step 6:

The server distributes the generated commentary together with the game footage. The input is the generated commentary and the game footage, and the output is the streaming data to be distributed. Streaming platforms such as YouTube and Twitch are used for distribution. Users can watch the game through these platforms and enjoy commentary according to their emotions.

(Application example 2)

Next, application example 2 of embodiment example 2 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

Conventional live basketball game broadcasting systems had the problem that commentary was uniform and did not allow for flexible responses to viewer emotions. In addition, there was a lack of game highlights, replay functions, and chat functions between viewers, so there was a need to improve the viewing experience. Furthermore, there were limited ways to widely provide viewers with information about youth and mini-basketball games across the country, resulting in information disparities between regions.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 2 is realized by the following means.

In this invention, the server includes a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary of youth and mini-basketball games from around the country with AI commentary, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for recognizing the emotions of viewers and adjusting the content and tone of the commentary according to those emotions, a means for providing highlights and replay functions for games, and a means for providing a chat function between viewers. This allows for flexible commentary according to the emotions of viewers, improving the viewing experience and making it possible to widely provide youth and mini-basketball games from around the country to viewers.

"Means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to generate commentary in real time for basketball games and provide it to viewers.

"A means of broadcasting commentary on youth and mini-basketball games across the country with live coverage by AI" refers to technology that broadcasts youth and mini-basketball games held across the country in real time with commentary by artificial intelligence.

"Method of using AI to automatically generate running scores that are created manually by officials and teams" refers to technology that uses artificial intelligence to automatically generate scores that are manually recorded by officials and teams during a match.

"Means of recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to technology that recognizes emotions from the viewer's facial expressions, voice, etc., and changes the content and tone of the commentary according to those emotions.

"Means for providing match highlight and replay features" refers to technology that provides a highlight feature that extracts and plays important scenes from a match, and a replay feature that replays specific scenes.

"Means for providing a chat function between viewers" refers to technology that provides a chat function that allows users watching a match to exchange messages in real time.

The system for implementing this invention includes the following main components: a means for AI to provide commentary on live coverage of basketball games, a means for distributing commentary on youth and mini-basketball games across the country with AI commentary, a means for AI to automatically generate running scores that are created manually for each official and team, a means for recognizing the emotions of viewers and adjusting the content and tone of the commentary according to those emotions, a means for providing game highlights and replay functions, and a means for providing a chat function between viewers.

Hardware and software used

Hardware: Smartphone (camera, microphone, display)

Software: OpenCV (image processing library), EmotionRecognizer (emotion recognition model), AICommentary (AI commentary generation model), LiveStreaming (live streaming library)

Data processing and data calculations

1. Frame acquisition: The server acquires frames from the live streaming, which allows video data of the match to be collected in real time.

2. Emotion Recognition: The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer. This obtains the viewer's emotion data.

3. Commentary generation: The server generates commentary using AICommentary based on the recognized emotions. This allows commentary to be generated in real time according to the viewer's emotions.

4. Add commentary: The server adds the generated commentary to the live stream, allowing viewers to see emotional commentary in real time.

5. Highlights and Replays: The server automatically extracts important scenes from the match and provides highlights and replays, allowing viewers to rewatch key moments of the match.

6. Chat function: The server provides a chat function that allows viewers to exchange messages with each other in real time. This promotes communication between viewers.

Specific examples

For example, during a youth basketball game, if the viewer is impressed, the server might generate a commentary like, "Great play! I'll never forget this moment!" Or, if the viewer is excited, the server might generate a commentary like, "You won't want to miss this! Watch the next play!"

Example of a prompt

"Create an AI model that recognizes the emotions of your viewers and adjusts the content and tone of your commentary accordingly. For example, if your viewer is emotional, make sure your commentary reflects that emotion."

The flow of the specific processing in application example 2 is explained using Figure 20.

Step 1:

The server obtains frames from live streaming. The input is real-time game video data, and the output is individual frame images. Specifically, the server receives the video data sent from the camera and splits it into frames using OpenCV.

Step 2:

The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer. The input is the frame image obtained in step 1, and the output is the viewer's emotion data. Specifically, the server detects the face in the frame and inputs it into the emotion recognition model to classify the emotion.

Step 3:

The server generates a commentary using AICommentary based on the recognized emotion. The input is the emotion data obtained in step 2, and the output is commentary text corresponding to the emotion. Specifically, the server inputs the emotion data as a prompt into the AI commentary generation model to generate an appropriate commentary.

Step 4:

The server adds the generated commentary to the live stream. The input is the commentary text generated in step 3, and the output is the live video with the commentary added. Specifically, the server synthesizes the commentary text into speech and overlays it on the video.

Step 5:

The server automatically extracts important scenes from a match and provides highlight and replay functions. The input is real-time match video data, and the output is highlight and replay footage. Specifically, the server detects events in the match, extracts important scenes, and saves them.

Step 6:

The server provides a chat function that allows viewers to exchange messages with each other in real time. The input is message data from the viewer, and the output is the message to be sent to other viewers. Specifically, the server receives messages and distributes them to other viewers in the chat room.

(Example 3)

Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

In conventional basketball games, creating a running score and understanding the progress of the game were done manually, making it difficult to update in real time. In addition, there was little progress in creating a database of teams and players nationwide, and feedback for analyzing player performance and training was not provided sufficiently. Furthermore, the score display did not reflect the user's feelings about the score situation in the game, resulting in low satisfaction when watching the game.

The specific processing by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for receiving video data and audio data of the match, means for analyzing the received video data, means for analyzing the received audio data, means for extracting the score situation and foul situation from the analysis results, means for generating a running score based on the extracted information, means for updating and transmitting the generated running score in real time, means for displaying the transmitted running score, and means for recognizing the user's emotions and adjusting the score display. This makes it possible to grasp the progress of the match in real time and provide a score display that reflects the user's emotions.

"Game video data" refers to video data that records the progress of a basketball game.

"Audio data" refers to audio data that records commentary and commentary during a match, audience reactions, etc.

"Means of receiving" refers to the function for acquiring video and audio data of the match from an external source.

"Means of analysis" refers to the function of processing received video and audio data and extracting specific information.

"Scoring status" is information that shows how many points each team scored during the game.

"Foul situation" is information that shows which players committed what type of fouls during the game.

The "running score" is a scoreboard that shows the score and foul situation in real time as the game progresses.

"Means for updating and transmitting in real time" refers to a function for instantly updating the generated running score and transmitting it to the user's device.

"Display means" is a function for visually presenting the submitted running score to the user.

"Means of recognizing user emotions" refers to a function for analyzing and understanding the user's emotions regarding the score of a match, etc.

"Means for adjusting score display" is a function for changing the way the scoreboard is displayed depending on the recognized user's emotions.

"Game results and stats information for teams and players nationwide" refers to game results for basketball teams and players nationwide and statistical data for individual players.

"Means of saving to a database" refers to the function for organizing collected match results and stats information and storing it in a database.

"Means of analyzing performance" refers to a function for evaluating a player's performance and abilities based on the information stored in the database.

"Means of generating feedback" refers to the function that creates reports based on the analysis results to suggest player development and areas for improvement.

"Means for sending and displaying" refers to the function for sending the generated feedback to the user's device and presenting it visually.

"Visualization" refers to displaying data visually to make it easier to understand.

"Means for utilizing in development" refers to methods for supporting the training and development of players based on data.

This invention is a system that automatically generates running scores for basketball games and analyzes player performance. A specific embodiment of this system is described below.

System configuration

Hardware and software

1. Server

Hardware: Server with high-performance processor and large memory capacity

Software: Video analysis software (e.g. OpenCV), audio analysis software (e.g. Google Cloud Speech-to-Text), database management system (e.g. MySQL), sentiment analysis engine (e.g. IBM Watson Tone Analyzer)

2. Terminal

Hardware: PCs, tablets, smartphones, etc.

Software: Web browser, dedicated application

Program processing

Receiving and analyzing match data

The server receives video and audio data of the match. The video data is obtained from cameras and streaming services, and the audio data includes commentary and narration. The received video data is analyzed using OpenCV to detect scoring and foul scenes. The audio data is converted to text using Google Cloud Speech-to-Text, and information about scores and fouls is extracted from the commentary.

Extraction of scoring and foul situations

The server combines the results of video and audio analysis to extract the score and foul situations. This makes it possible to grasp the progress of the game in real time.

Generate and display running scores

The server generates a running score based on the extracted information. The generated running score is updated in real time and sent to the terminal using WebSocket or HTTP. The terminal displays the received running score to the user, allowing the user to check the progress of the match in real time.

Recognizing user emotions and adjusting score display

The server uses IBM Watson Tone Analyzer to recognize the user's emotions regarding the score of the game. Depending on the recognized emotion, the way the score is displayed is adjusted. For example, if the user is happy about a score, the score display is emphasized with bright colors or animations. The adjusted score display is sent to the terminal and displayed to the user.

Database of teams and players nationwide and performance analysis

The server collects match results and stats for teams and players across the country, and stores them in a MySQL database. Based on the stored information, the performance of players is analyzed using Python's Pandas library. Based on the analysis results, feedback for player development is generated and sent to the terminal. The terminal displays the received feedback to the user.

Specific examples

When the video data of the match is sent to the server, the server uses OpenCV to detect scoring scenes and uses Google Cloud Speech-to-Text to extract foul information from the commentary audio. For example, the prompt text can be "Analyze the scoring scenes and foul information of this match to generate a running score."

The server collects the results of matches from around the country and stores them in a MySQL database. It uses Python's Pandas library to analyze players' performance and generate feedback. For example, enter the following prompt statement: "Collect the results of matches from around the country, analyze players' performance, and generate feedback."

The server recognizes the user's joy at the scoring scene using IBM Watson Tone Analyzer, and adjusts the score display to emphasize the joy. For example, the prompt text is input as "Analyze the user's emotions at the scoring scene and adjust the score display." The flow of the specific process in Example 3 is explained with reference to FIG. 21.

Step 1:

Receive match data

The server receives video and audio data of the match. As input, it obtains video and audio data provided in real time from cameras and streaming services. As output, it stores the received video and audio data in its internal memory. Specifically, the server obtains the data through a network connection and stores it in a specified format.

Step 2:

Video data analysis

The server analyzes the received video data. As input, it uses the video data saved in step 1. As output, it obtains the results of detecting scoring and foul scenes. In concrete terms, the server uses OpenCV to analyze the video data frame by frame and executes an algorithm to detect specific events (scoring and fouls).

Step 3:

Analysis of audio data

The server analyzes the received audio data. As input, it uses the audio data stored in step 1. As output, it obtains text information extracted from the audio data. Specifically, the server converts the audio data into text using Google Cloud Speech-to-Text and extracts information about scores and fouls from the commentary.

Step 4:

Extraction of scoring and foul situations

The server integrates the video analysis results and audio analysis results to extract the scoring and foul situations. As input, it uses the outputs of steps 2 and 3. As output, it obtains data including the scoring and foul situations. In concrete terms, the server integrates the video analysis results and audio analysis results in a chronological order to identify the timing of scoring and fouls.

Step 5:

Generate running scores

The server generates a running score based on the extracted information. As input, it uses the score and foul status obtained in step 4. As output, it obtains data including the running score. Specifically, the server organizes the score and foul information in chronological order and compiles it in a scoreboard format.

Step 6:

Update and send running scores

The server updates the generated running score in real time and sends it to the terminal. As input, it uses the running score generated in step 5. As output, it sends the updated running score to the terminal. Specifically, the server sends data using WebSocket or HTTP.

Step 7:

Display running score

The terminal displays the received running score to the user. As input, it uses the running score sent in step 6. As output, it obtains a scoreboard that is visually presented to the user. In concrete terms, the terminal displays the scoreboard using a web browser or a dedicated application.

Step 8:

Recognizing user emotions

The server recognizes the user's emotions regarding the score of the game. As input, it uses the user's comments and reaction data. As output, it obtains the recognized emotion data. Specifically, the server analyzes the user's emotions using IBM Watson Tone Analyzer.

Step 9:

Score display adjustments

The server adjusts the way the score is displayed depending on the recognized emotion. As input, it uses the emotion data obtained in step 8 and the running score generated in step 5. As output, it obtains the adjusted score display data. As a specific operation, the server changes the color and animation of the score display.

Step 10:

Send adjusted score display

The server sends the adjusted score display to the terminal. As input, it uses the score display data adjusted in step 9. As output, it sends the adjusted score display to the terminal. Specifically, the server sends the data using WebSocket or HTTP.

Step 11:

Displaying the adjusted score

The terminal displays the adjusted score display to the user. As input, it uses the adjusted score display data sent in step 10. As output, it obtains the adjusted scoreboard that is visually presented to the user. In specific operation, the terminal displays the adjusted scoreboard using a web browser or a dedicated application.

Step 12:

Database of teams and players nationwide

The server collects match results and stats information for teams and players across the country and stores it in a database. As input, it uses data obtained from each team's official website and sports data services. As output, it obtains match results and stats information stored in the database. Specifically, the server organizes the data and stores it in a MySQL database.

Step 13:

Performance analysis

The server analyzes the players' performance based on the information stored in the database. As input, it uses the database information stored in step 12. As output, it obtains the analysis results that evaluate the players' performance and abilities. Specifically, the server processes the data using Python's Pandas library and evaluates the players' performance.

Step 14:

Generating feedback

The server generates feedback for player development based on the analysis results. As input, it uses the analysis results obtained in step 13. As output, it obtains a feedback report that points out the player's strengths and weaknesses and suggests areas for improvement. In concrete terms, the server creates the report and provides it to the player and coach.

Step 15:

Submit and view feedback

The server sends the generated feedback to the terminal, which displays it to the user. As input, it uses the feedback report generated in step 14. As output, it obtains a feedback report that is visually presented to the user. In concrete terms, the server sends the data via email or a dedicated application, and the terminal displays it.

(Application example 3)

Next, application example 3 of embodiment example 3 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

Traditionally, live broadcasts and commentary of basketball games relied on human commentators, making it difficult to grasp the score and foul situation in real time. In addition, analysis of player performance and feedback on development were also done manually, which was inefficient. Furthermore, it was not possible to display scores that reflected the user's emotions, making it difficult to improve the viewing experience.

The specific processing by the specific processing unit 290 of the data processing device 12 in application example 3 is realized by the following means.

In this invention, the server includes: a means for AI to provide commentary on live coverage of basketball games; a means for distributing commentary on youth and mini-basketball games nationwide with AI live coverage; a means for AI to automatically generate running scores that are created manually for each official and team; a means for analyzing game video and audio data in real time and automatically generating the score situation and foul situation; a means for recognizing the user's emotions and displaying the score according to the emotions; and a means for providing feedback for player performance analysis and development using a database of teams and players nationwide. This makes it possible to grasp the progress of the game in real time and display scores that reflect the user's emotions, improving the efficiency of player performance analysis and development.

"A means for AI to provide commentary for live broadcasts of basketball games" refers to technology that uses artificial intelligence to provide real-time commentary on the progress and play of a basketball game.

"A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games from around the country, and broadcasts live coverage including that commentary.

"A means of using AI to automatically generate running scores that are created manually by officials and for each team" refers to technology that uses artificial intelligence to automatically generate scores such as points and fouls as the game progresses.

"Means for analyzing game video and audio data in real time and automatically generating scoring and foul status" refers to technology that analyzes game video and audio data in real time and automatically generates scoring and foul status.

"Means for recognizing the user's emotions and displaying a score according to those emotions" refers to technology that recognizes emotions from the user's facial expressions and voice, and adjusts the score display according to those emotions.

"Means of utilizing a database of teams and players nationwide to analyze player performance and provide feedback for development" refers to technology that utilizes a database of basketball teams and players nationwide to analyze player performance and provide feedback for development.

The system for implementing this invention uses the following hardware and software.

Hardware and software used

Hardware: Smartphone, head-mounted display

Software: TensorFlow (AI model), OpenCV (video analysis), NLTK (natural language processing), Emotion API (emotion recognition)

System configuration

The server includes the following means:

1. How AI can provide commentary for live broadcasts of basketball games:

The server receives video and audio data from the match in real time, and uses TensorFlow to analyze the progress of the match and the content of the play, generating commentary.

2. How to broadcast live commentary of youth and mini-basketball games nationwide with AI:

The server receives footage of youth and mini-basketball matches from across the country, and similarly uses AI to generate commentary, which it distributes in real time.

3. A way to use AI to automatically generate running scores that are currently created manually by officials and teams:

The server analyzes the video and audio data of the match and automatically generates the score and foul situations.

4. A method for analyzing game video and audio data in real time and automatically generating scoring and foul situations:

The server uses OpenCV to split the video data into frames and extract important scenes. The audio data is converted to text using NLTK and important keywords are extracted.

5. How to recognize user emotions and display a score according to those emotions:

The server uses the Emotion API to recognize emotions from the user's facial expressions and voice, and adjusts the score display accordingly.

6. Using a national team and player database to analyze player performance and provide feedback for development:

The server accesses a database of teams and players from across the country, displays analysis of player performance, and provides feedback for the next match.

Specific examples

When a user using a smartphone during a match sees a goal being scored, the server uses AI to analyze the scene and display the score in real time. If the user shows an expression of joy, the server will recognize it using an emotion engine and make the score display more colorful. After the match ends, the server also refers to a nationwide database to display an analysis of the players' performance and provide feedback for the next match.

Example of a prompt

"Please analyze the video and audio data of the game in real time, and use an AI model that automatically generates the score and foul situation to display a running score. Also, please recognize the user's emotions and display the score according to those emotions. Furthermore, please use a database of teams and players from all over the country to provide feedback for analyzing player performance and development."

The flow of the specific processing in application example 3 is explained using Figure 22.

Step 1:

Acquisition of game video and audio data

The server receives video and audio data of the game in real time from a smartphone or head-mounted display. The input is the video and audio data of the game, and the output is preprocessed data for analysis. In concrete terms, the server receives the data stream, splits the video data into frames, and converts the audio data into text.

Step 2:

Data preprocessing

The server uses OpenCV to split the video data into frames and extract important scenes. It also uses NLTK to convert the audio data into text and extracts important keywords (e.g., goals, fouls). The input is the preprocessed video and audio data, and the output is important scenes and keywords for analysis. Specifically, the server analyzes video frames and identifies scoring and foul scenes.

Step 3:

Analysis using AI models

The server uses TensorFlow to analyze the scoring and foul situations from video and audio data. The input is important scenes and keywords, and the output is data on the scoring and foul situations. In concrete terms, the server runs an AI model and automatically determines the scores and fouls for each scene.

Step 4:

Automatic generation of running scores

The server automatically generates a running score based on the analysis results. The input is data on the scoring and foul situations, and the output is a real-time running score. Specifically, the server tallys up the score and foul information and updates the scoreboard.

Step 5:

Emotion recognition

The server uses the Emotion API to recognize emotions from the user's facial expressions and voice. The input is the user's facial expression data and voice data, and the output is the recognized emotion data. In concrete terms, the server obtains user data from the camera and microphone and analyzes emotions.

Step 6:

Score display based on emotions

The server adjusts the score display according to the recognized emotion. The input is emotion data and the running score, and the output is a score display according to the emotion. In concrete terms, the server changes the design and effects of the score display based on the user's emotion.

Step 7:

Utilizing the database

The server refers to a database of teams and players from around the country, displays the results of player performance analysis, and provides feedback for the next match. The input is the match results and player data, and the output is performance analysis results and feedback. In concrete terms, the server retrieves relevant information from the database and displays the analysis results.

The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating the user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the robot 414.

The emotion identification model 59, which serves as an emotion engine, may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. Emotions that represent states and actions arising from a state of mind are arranged on the outer sides of the concentric circles. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

These emotions are distributed in the three o'clock direction of emotion map 400, and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness takes precedence over internal sensations, resulting in a sense of calm.

The inside of emotion map 400 represents what is going on inside the mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible (more expressed in behavior) the emotions become.

Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and remaining battery power, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https://ci.nii.ac.jp/naid/500000375379). The left half of the emotion map is lined with emotions that belong to an area called "reaction" where sensation is dominant. The right half of the emotion map is lined with emotions that belong to an area called "situation" where situation recognition is dominant.

The emotion map defines two emotions that encourage learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

The emotion identification model 59 inputs user input to a pre-trained neural network, obtains emotion values indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple learning data that are combinations of user input and emotion values indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Figure 10. Figure 10 shows an example in which multiple emotions, "peace of mind," "calm," and "reassuring," have similar emotion values.

In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

In the above embodiment, an example has been described in which the specific processing program 56 is stored in the storage 32, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

The specific processing program 56 may also be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed in the computer 22 in response to a request from the data processing device 12.

It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

The various processors listed below can be used as hardware resources for executing specific processes. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource for executing specific processes by executing software, i.e., a program. Examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically to execute specific processes. All of these processors have built-in or connected memory, and all of these processors execute specific processes by using the memory.

The hardware resource that executes the specific process may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). The hardware resource that executes the specific process may also be a single processor.

As an example of a configuration using a single processor, first, there is a configuration in which one processor is configured by combining one or more CPUs with software, and this processor functions as a hardware resource that executes a specific process. Secondly, there is a configuration in which a processor is used that realizes the functions of the entire system, including multiple hardware resources that execute a specific process, on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the various processors mentioned above as hardware resources.

More specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processing described above is merely an example. It goes without saying that unnecessary steps can be deleted, new steps can be added, and the processing order can be changed without departing from the spirit of the invention.

The above description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above explanation of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. Also, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

All publications, patent applications, and technical standards described in this specification are incorporated by reference into this specification to the same extent as if each individual publication, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

The following is further disclosed regarding the above embodiment.

(Claim 1)

A system that includes a means for AI to provide commentary for live broadcasts of basketball games, a means for AI to provide commentary for youth and mini-basketball games across the country with live broadcasts, and a means for AI to automatically generate running scores that are currently created manually by officials and for each team.

(Claim 2)

The system according to claim 1, further comprising a means for creating a database of teams and players nationwide.

(Claim 3)

The system according to claim 2 further includes a means for utilizing the databased information to visualize and develop registered basketball players in Japan.

(Claim 4)

The system of claim 1, further comprising an emotion engine that recognizes the user's emotions.

(Claim 5)

The system of claim 2 further includes an emotion engine that recognizes the user's emotions.

(Claim 6)

The system of claim 3 further includes an emotion engine that recognizes the user's emotions.

"Example 1"
(Claim 1)
A means for collecting video and audio data of the match in real time;
A means for analyzing the collected video and audio data;
A means for generating a match commentary based on the analysis results;
A means for distributing the generated commentary together with a live broadcast;
A way for AI to provide commentary on live broadcasts of basketball games,
A method to broadcast live commentary of youth and mini-basketball games nationwide with AI,
A system that includes a means of using AI to automatically generate running scores that are currently created manually by officials and each team.

(Claim 2)
2. The system according to claim 1, further comprising means for creating a database of teams and players nationwide.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the database information for visualizing and developing registered basketball players in Japan.

"Application example 1"
(Claim 1)
A way for AI to provide commentary on live broadcasts of basketball games,
A method to broadcast live commentary of youth and mini-basketball games nationwide with AI,
A means to automatically generate running scores using AI, which are currently created by officials and each team manually,
A means to analyze game video data and generate commentary in real time;
A means to automatically extract important scenes after the game ends and generate highlight videos,
A means to analyze player movements and performance and provide detailed statistics,
Users can ask questions about specific players or plays and the AI will provide commentary.
A system including:

(Claim 2)
2. The system according to claim 1, further comprising means for creating a database of teams and players nationwide.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the database information for visualizing and developing registered basketball players in Japan.

"Example 2"
(Claim 1)
A means to collect video and audio data from matches held across the country,
A means for storing collected data in cloud storage;
A means for transmitting the stored data to a generative AI model to generate match commentary; and
a means for synchronizing the generated commentary with game footage; and
A means to stream game footage with commentary;
means for providing an interface for a user to view;
A means for receiving game video with commentary streamed from a server;
The system includes a means for displaying the received video to a user.

(Claim 2)
2. The system according to claim 1, further comprising means for creating a database of teams and players nationwide.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the databased information for visualizing and developing registered players.

"Application Example 2"
(Claim 1)
A way for AI to provide commentary on live broadcasts of basketball games,
A method to broadcast live commentary of youth and mini-basketball games nationwide with AI,
A means to automatically generate running scores using AI, which are currently created by officials and each team manually,
A means to collect video and audio data from the match and have AI analyze it to generate commentary,
A means to broadcast matches in real time and provide AI commentary through an application installed on smartphones,
A means to provide match highlights and replay functionality;
A means to provide users with the ability to follow their favorite teams and players;
The system includes a means for providing post-match statistical and analytical reports.

(Claim 2)
2. The system according to claim 1, further comprising means for creating a database of teams and players nationwide.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the database information for visualizing and developing registered basketball players in Japan.

"Example 3"
(Claim 1)
A means for receiving video data and audio data of a match;
A means for analyzing the received video data;
means for analyzing the received voice data;
A means for extracting the scoring situation and foul situation from the analysis results,
A means for generating a running score based on the extracted information;
A means for updating the generated running score in real time and transmitting it to the terminal;
A means for displaying the running score transmitted to the terminal;
A system including:

(Claim 2)
A means to collect information such as game results and stats of teams and players nationwide,
A means for compiling the collected information into a database;
A means of analyzing the databased information and evaluating the performance of athletes;
A means for generating feedback for player development based on the analysis results;
means for transmitting the generated feedback to the terminal;
means for displaying the feedback sent to the terminal;
The system of claim 1 , comprising:

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the databased information for visualizing and developing registered basketball players in Japan.

"Application example 3"
(Claim 1)
A way for AI to provide commentary on live broadcasts of basketball games,
A method to broadcast live commentary of youth and mini-basketball games nationwide with AI,
A means to automatically generate running scores using AI, which are currently created by officials and each team manually,
A method to analyze camera footage and audio data in the factory to detect and score production status and abnormalities in real time,
A system that includes a means of compiling past production data into a database and providing feedback to improve efficiency.

(Claim 2)
2. The system according to claim 1, further comprising means for creating a database of teams and players nationwide.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the database information for visualizing and developing registered basketball players in Japan.

"Example 1 of combining emotion engines"
(Claim 1)
A way for artificial intelligence to provide commentary for live broadcasts of basketball games,
A method to broadcast live commentary of youth and mini-basketball games nationwide using artificial intelligence,
A method to automatically generate running scores using artificial intelligence, which are currently created by officials and teams.
A means of analyzing game video data;
A means for analyzing audio data of the match;
A means for receiving and analyzing emotion data of a user;
A means for automatically generating commentary based on the analysis results and adjusting the content and tone of the commentary according to the user's emotions;
The system includes a means for delivering the adjusted commentary along with the play-by-play coverage.

(Claim 2)
2. The system according to claim 1, further comprising means for creating a database of teams and players nationwide.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the databased information for visualizing and developing registered basketball players.

"Application example 1 when combining emotion engines"
(Claim 1)
A way for AI to provide commentary on live broadcasts of basketball games,
A method to broadcast live commentary of youth and mini-basketball games nationwide with AI,
A means to automatically generate running scores using AI, which are currently created by officials and each team manually,
A means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions;
A means for generating commentary using a generative AI model;
A means for inputting a prompt sentence and generating a commentary;
A system including:

(Claim 2)
2. The system according to claim 1, further comprising means for creating a database of teams and players nationwide.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the database information for visualizing and developing registered basketball players in Japan.

"Example 2 when combining emotion engines"
(Claim 1)
A way for artificial intelligence to provide commentary for live broadcasts of basketball games,
A method to broadcast live commentary of youth and mini-basketball games nationwide using artificial intelligence,
A method to automatically generate running scores using artificial intelligence, which are currently created by officials and teams.
A means for collecting game video and audio data;
A means for analyzing the collected data and generating commentary;
A means of recognizing audience emotions and adjusting the content and tone of commentary accordingly;
The system includes a means for distributing the generated commentary together with the game footage.

(Claim 2)
2. The system according to claim 1, further comprising means for creating a database of teams and players nationwide.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the databased information for visualizing and developing registered basketball players in the country.

"Application example 2 when combining emotion engines"
(Claim 1)
A way for AI to provide commentary on live broadcasts of basketball games,
A method to broadcast live commentary of youth and mini-basketball games nationwide with AI,
A means to automatically generate running scores using AI, which are currently created by officials and each team manually,
A means of recognizing audience emotions and adjusting the content and tone of commentary accordingly;
A means to provide match highlights and replay functionality;
A means for providing a chat function between viewers;
A system including:

(Claim 2)
2. The system according to claim 1, further comprising means for creating a database of teams and players nationwide.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the database information for visualizing and developing registered basketball players in Japan.

"Example 3 when combining emotion engines"
(Claim 1)
A means for receiving video data and audio data of a match;
A means for analyzing the received video data;
means for analyzing the received voice data;
A means for extracting a score situation and a foul situation from the analysis result;
A means for generating a running score based on the extracted information;
A means for updating and transmitting the generated running score in real time;
A means for displaying the submitted running score;
The system includes a means for recognizing a user's emotions and adjusting the score display.

(Claim 2)
A means to collect game results and stats information for teams and players nationwide,
A means for storing the collected information in a database; and
A means of analyzing the performance of players based on the information stored in the database;
A means for generating feedback for player development based on the analysis results;
The system of claim 1 further comprising means for transmitting and displaying the generated feedback.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the databased information for visualizing and developing players.

"Application example 3 when combining emotion engines"
(Claim 1)
A way for AI to provide commentary on live broadcasts of basketball games,
A method to broadcast live commentary of youth and mini-basketball games nationwide with AI,
A means to automatically generate running scores using AI, which are currently created by officials and each team manually,
A method for analyzing video and audio data of a match in real time and automatically generating the score and foul situation.
A means for recognizing a user's emotion and displaying a score according to the emotion;
A means to utilize a database of teams and players across the country to provide feedback for player performance analysis and development;
A system including:

(Claim 2)
2. The system according to claim 1, further comprising means for creating a database of teams and players nationwide.

(Claim 3)
2. The system according to claim 1, further comprising a means for utilizing the database information for visualizing and developing registered basketball players in Japan.

10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset type terminal 414 Robot

Claims (3)

A means for receiving video data and audio data of a match;
means for analyzing the received video data;
means for analyzing the received voice data;
means for extracting a score situation and a foul situation from an analysis result of the video data and the audio data;
a means for generating a running score based on the extracted score situation and foul situation;
A means for updating and transmitting the generated running score in real time;
A means for displaying the transmitted running score;
a means for recognizing a user's emotion and adjusting the display of the running score based on the user's emotion;
A system including: A means for collecting game results and stats information for teams and players nationwide;
A means for storing the collected match results and stats information in a database;
A means for analyzing player performance based on the information stored in said database;
A means for generating feedback for player development based on the analysis results;
a means for transmitting and displaying the generated feedback;
The system of claim 1 further comprising: The system according to claim 1 further includes a means for utilizing the information stored in the database to visualize and develop players.

Images