KR102460595B1 - Method and apparatus for providing real-time chat service in game broadcasting - Google Patents

Abstract

An embodiment of the present invention is a method of providing a chat service in real time in a game broadcast conducted by a streamer, comprising the steps of: obtaining a game image in the game broadcast; extracting information about a game state from the game image; Determining a first situation regarding the current game state based on information about the game state, determining a second situation regarding the streamer based on audio information in the game broadcast, the first situation and the first situation 2 Based on the situation, determining an output phrase from among a plurality of candidate phrases stored in the DB and outputting the output phrase to a chat window in the game broadcast.

Description

Method and apparatus for providing real-time chat service in game broadcasting

The present invention relates to a method and apparatus for providing a real-time chat service in game broadcasting, and more particularly, to determine a game situation based on a game broadcasting image and a streamer's voice, etc. , relates to a method and apparatus for providing a chat service in which a phrase suitable for the determined situation is automatically entered into a chat window.

Recently, an Internet broadcasting service that provides video content to other anonymous users through the Internet in real time has been attracting attention. These Internet broadcasts provide content on a variety of topics as much as people's various interests, and in particular, game broadcasts that provide real-time game play images are gaining popularity.

Game broadcasting refers to broadcasting in which a video streamer plays an online game and transmits the playing video as a real-time broadcast.

Due to the great popularity of game broadcasting, research and discussion on more effective game broadcasting are continuing, such as recently developed systems for smoothly providing game broadcasting or systems for assisting in sharing opinions with viewers.

Korean Patent Publication No. 10-2020-0127658 (2020.11.11.) Korean Patent Publication No. 10-2020-0048772 (2020.05.08.)

An object of the present invention to solve the above problems is to provide a method for providing a real-time chat service in game broadcasting.

Another object of the present invention to solve the above problems is to provide an apparatus for providing a real-time chat service in game broadcasting.

A method of providing a chat service in real time in a game broadcast conducted by a streamer performed by a server according to an embodiment of the present invention for achieving the above object includes: acquiring a game image in the game broadcast; extracting state information, determining a first situation regarding the current game state based on the information on the game state, determining a second situation regarding the streamer based on audio information in the game broadcast and determining an output phrase from among a plurality of candidate phrases stored in a DB based on the first situation and the second situation, and outputting the output phrase to a chat window in the game broadcast. .

Here, the information about the game state may include a health bar area extracted from the game image, a system message area, and location information between characters, and the system message area may include a system window area and a system notification area.

Here, the step of determining the second situation regarding the streamer based on the audio information in the game broadcast is improved audio information in which noise is partially removed from the audio information by using the audio information and a pre-stored clip for removing noise. deriving, determining an audio reference intensity on a frequency corresponding to the streamer based on the improved initial audio information from the start time of the game broadcasting to a predetermined time among the improved audio information, and the improved audio information The method may include determining the state of the streamer as an excited state when an intensity on a frequency greater than or equal to the audio reference intensity is detected in the improved measurement target audio information after the predetermined time.

Here, deriving a chat area included in another game video in another game broadcast before the same game as the game broadcast, the number of phrases generated during a preset chat window size in the chat area in the other game video is equal to or greater than a first threshold Deriving phrases within a first time interval corresponding to the chatting window, determining whether the number of words most shared among the phrases within the first time interval is equal to or greater than a second threshold, and the number of words is the second The method may further include including, in the plurality of candidate phrases stored in the DB, phrases in which a keyword corresponding to the word is shared among the phrases within the first time interval, if the threshold is higher.

Here, among the phrases in which the keyword is shared, a phrase within a second time interval of a preset size from the first phrase is determined as a phrase suitable for the context of the first phrase, and a phrase outside the second time interval is determined as the first phrase It may further include the step of determining that the context is out of context for the phrase.

Here, determining whether a preset positive word or negative word other than the keyword in the second phrase is included among the phrases in which the keyword is shared, and when the affirmative word is included in the second phrase, the phrases in which the keyword is shared The method may further include determining positive phrases, and when the negative word is included in the second phrase, determining phrases in which the keyword is shared as negative phrases.

Here, calculating a generality index of each of the phrases in which the keyword is shared based on the plurality of candidate phrases stored in the DB, and a phrase in which the generality index exceeds a third threshold among the phrases in which the keyword is shared The method may further include excluding from the plurality of candidate phrases.

Here, the method further comprising the step of calculating a response rate corresponding to the influence of the output phrase based on the input phrases for a preset time from the time when the output phrase is output, wherein the response rate is a first method regarding whether a word matches; It may be calculated using at least one of the second method related to the similarity and the third method related to the question and answer.

According to the present invention, it is possible to automatically generate a dialogue that can elicit a response from a large number of viewers in live video streaming, thereby providing the same effect as input by a real person.

According to the present invention, it is possible to select and provide a phrase optimized for the current state by considering the location information between the health bar area, the system message area and the character in the broadcast, and also the voice of the streamer.

According to the present invention, it is possible to select the most effective phrase from among a plurality of candidate phrases in various situations through various response rate calculation methods.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to aid understanding of the present invention, provide various embodiments and, together with the detailed description, explain technical features of the various embodiments.
1 is a block diagram of terminals performing a method of providing a real-time chat service in a game broadcast according to an embodiment of the present invention.
2 is a block diagram of a chat service providing server according to an embodiment of the present invention.
3 is a flowchart of a method of providing a real-time chatting service in a game broadcast according to an embodiment of the present invention.
4 is a diagram for explaining a method of extracting information about a game state according to an embodiment of the present invention.
5 is a diagram for explaining a method of deriving audio information according to an embodiment of the present invention.
6 is a diagram for explaining a method of calculating a response rate corresponding to an influence on an output phrase according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that all modifications, equivalents and substitutes included in the spirit and scope of the present invention are included. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, A, B, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term "and/or" includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In describing the present invention, in order to facilitate the overall understanding, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

1 is a block diagram of terminals performing a method of providing a real-time chat service in a game broadcast according to an embodiment of the present invention.

Referring to FIG. 1 , a method for providing a real-time chat service according to an embodiment of the present invention includes a chat service providing server (hereinafter, server 100), a streamer terminal 200, a broadcast server 300, and a viewer terminal 400. ) can be performed through transmission and reception of information between

For example, the streamer terminal 200 may access the broadcast server 300 to perform broadcasting, and the viewer terminal 400 may access the broadcast server 300 to view the broadcast of the streamer terminal 200 . can Also, the chat service providing server 100 according to an embodiment may access the broadcast server 300 similarly to the viewer terminal 400 and participate in the broadcast of the streamer terminal 200 . That is, the chat service providing server 100 may connect to the broadcast server 300 to output a phrase by participating in the chatting in the broadcast of the streamer terminal 200 .

For example, at least a portion of the streamer terminal 200 and the viewer terminal 400 may be referred to as a user terminal, and the user terminal may represent a communicable and computingable electronic device, for example, a communicable desktop. Desktop computer, laptop computer, notebook, smart phone, tablet PC, mobile phone, smart watch, smart glasses glass), e-book reader, PMP (portable multimedia player), portable game console, navigation device, digital camera, DMB (digital multimedia broadcasting) player, digital audio recorder, digital voice It may be a digital audio player, a digital video recorder, a digital video player, a personal digital assistant (PDA), or the like.

For example, in the case of the streamer terminal 200, it may be limited to a device equipped with a photographing device capable of photographing an image for broadcasting, but may be linked with a photographing device through a network to obtain an image for broadcasting A device may also be included, but is not limited thereto.

2 is a block diagram of a chat service providing server according to an embodiment of the present invention.

Referring to FIG. 2 , the chat service providing server 100 according to an embodiment of the present invention may be referred to as a chatting service providing apparatus 100 . Here, the chat service providing apparatus 100 may include at least one processor 110 and a memory 120 , and may include a transceiver 130 , an input interface device 140 , an output interface device 150 , and a storage device. At least one of (160) may be further included. In addition, each component included in the chat service providing apparatus 100 may be connected by a bus 170 to communicate with each other.

For example, the at least one processor 110 may be a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. can mean Each of the memory 120 and the storage device 160 may be configured as at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory 120 may be one of a read only memory (ROM) and a random access memory (RAM), and the storage device 160 is a flash-memory. , a hard disk drive (HDD), a solid state drive (SSD), or various memory cards (eg, micro SD card).

For example, the memory 120 or the storage device 160 may store at least one instruction that can be executed by the at least one processor 110 . For example, the at least one command may include a command for obtaining a game image in the game broadcast, a command for extracting information about a game state from the game image, and a first information about a current game state based on the information about the game state. A command for determining a situation, a command for determining a second situation regarding the streamer based on audio information in the game broadcast, and outputting a plurality of candidate phrases stored in the DB based on the first situation and the second situation It may include a command for determining a phrase and a command for outputting the output phrase to a chat window in the game broadcast.

For example, the information about the game state may include a health bar area extracted from the game image, a system message area, and location information between characters, and the system message area may include a system window area and a system notification area. have.

For example, the command for determining the second situation regarding the streamer based on the audio information in the game broadcast is an improved improved method in which noise is partially removed from the audio information using the audio information and a pre-stored clip for removing noise. A command for deriving audio information, a command for determining an audio reference intensity on a frequency corresponding to the streamer based on improved initial audio information from the start of the game broadcast to a predetermined time among the improved audio information, and the improvement The improved measurement target audio information after the predetermined time among the audio information may include a command for determining the state of the streamer as an excited state when an intensity on a frequency greater than or equal to the audio reference intensity is detected.

For example, the at least one command is a command for deriving a chat area included in another game video in another game broadcast before another game broadcast of the same game as the game broadcast, and occurs during a chat window size preset in the chat area in the other game video A command for deriving phrases within a first time interval corresponding to the chat window in which the number of phrases is greater than or equal to a first threshold, a command for determining whether the number of words most shared among phrases within the first time interval is equal to or greater than a second threshold, and When the number of words is equal to or greater than the second threshold, a command for including, among the phrases within the first time period, a keyword corresponding to the word is shared, is included in the plurality of candidate phrases stored in the DB. can

For example, the at least one command determines a phrase within a second time interval of a preset size from a first phrase among phrases in which the keyword is shared as a phrase matching the context for the first phrase, and the second time The method may further include instructions for determining the out-of-interval phrase as a phrase out of context with respect to the first phrase.

For example, the at least one command includes a command for determining whether a preset positive word or negative word excluding the keyword in the second phrase is included among the phrases in which the keyword is shared, and when the affirmative word is included in the second phrase , may further include a command for determining phrases in which the keyword is shared as positive phrases, and determining phrases in which the keyword is shared as negative phrases when the negative word is included in the second phrase.

For example, the at least one command includes a command for calculating a generality index of each of the phrases in which the keyword is shared based on the plurality of candidate phrases stored in the DB and the generality index among the phrases in which the keyword is shared. The method may further include a command to exclude a phrase exceeding a third threshold from the plurality of candidate phrases.

For example, the at least one command further includes a command for calculating a response rate corresponding to the influence of the output phrase based on the inputted phrases for a preset time from the time when the output phrase is output, and the response rate is a word The calculation may be performed using at least one of a first method relating to whether or not matching, a second method relating to similarity, and a third method relating to question and answer.

3 is a flowchart of a method of providing a real-time chatting service in a game broadcast according to an embodiment of the present invention.

Referring to FIG. 3 , in step S310, an embodiment may acquire a game image in the game broadcast. That is, according to an embodiment, a game image broadcast by a streamer may be obtained.

In step S320, an embodiment may extract information about the game state from the game image. In more detail, it will be described in conjunction with FIG. 4 .

4 is a diagram for explaining a method of extracting information about a game state according to an embodiment of the present invention.

Referring to FIG. 4 , according to an embodiment, the health bar area 410 , the system message areas 421 , 422 , and 423 , and the location information 431 and 432 between the characters can be extracted from the game image, and the system message area may include a system window area 421 and a system notification area 422 .

For example, the position information 431 and 432 between the characters should track the movements of the characters. In this case, the mini health bar located at the top of the character, which maintains a constant appearance regardless of the movement of the character, can be determined as the tracking target. have. Here, the mini health bar may be specified for each character in the image through a template matching method, and according to an embodiment, information of the characters in the image may be obtained from an area corresponding to the mini health bar. Here, the information of the characters may include peer identification information, level information, nickname information, physical strength information, and magical power information regarding whether the character is an ally or an enemy of the streamer. For example, the stamina information and the horse power information may be acquired as information about the remaining stamina ratio and the remaining horse power ratio. The template matching method may indicate a method of extracting a portion matching a preset template image from within an image, and the method is obvious to those skilled in the art, and a detailed operation method will be omitted.

For example, an embodiment may also extract the chat window area 441 and the streamer area 442 from the game image. Also, for example, in one embodiment, the status window area 450 may be further extracted to determine whether the image is a game image or a non-game image. Here, the status window area 450 may be information that is always displayed on the screen, and through this, it is possible to accurately determine whether a game is being played or not in any situation, and a phrase suitable for whether a game is being played or not can be output. For example, the status window area 450 may include a numerical value, and the numerical value may have a characteristic that changes depending on the situation. You can accurately determine whether you are in the game or not in response to a change in a numerical value within.

According to an embodiment, the characters in the health bar area 410 , the system window area 421 , the system notification area 422 , and the chat area 441 may be acquired through an OCR (Optical Character Reader) technique, and the acquired characters Based on these, information about the game state can be determined. Here, the OCR technique is obvious to those skilled in the art, and a detailed description thereof will be omitted.

For example, the characters in the system window area 421 may be used to obtain information about the game situation, and the characters in the system notification area 422 may be used to obtain an event occurring in the game. In particular, event information determined based on text in the system notification area 422 may be used to obtain information about a game situation that is not visible in the game image. Characters in the chatting area 441 may be used to identify a context for obtaining an output phrase, which will be described later. Characters in the health bar area 410 may be used to obtain information on a character played by a streamer. For example, the character information may include the character's attack power, vitality, stat, gold, and skill cooldown time, and the character information is a preset image corresponding to each information. After the location of each piece of information is specified in the health bar area 410 based on , it may be obtained based on characters obtained from the specified location.

For example, the system image notification area 423 may provide information about an event generated through an image rather than text. For example, the system image notification area 423 may include a plurality of images, and each of the plurality of images may represent an image related to a specific character. In addition, the plurality of images may include images having different sizes, and may indicate event information generated between characters displayed in the plurality of images based on a color and a relative positional relationship between the plurality of images. That is, according to an embodiment, information on the character involved in the occurrence of an event may be obtained based on a plurality of images in the system image notification area 423 and participate in the occurrence of an event through the size and relative positional relationship between the plurality of images. Event information between one character can be acquired. That is, according to an embodiment, information about which character killed which character and which character helped kill, etc. may be acquired through the size and relative positional relationship between the plurality of images.

According to an embodiment, death information of the character of the streamer may be acquired based on the color change of the image. That is, according to an embodiment, when the color of the image disappears, death information indicating that the character of the streamer has died may be acquired. For example, in one embodiment, when only S (saturation) representing saturation among HSV (hue saturation value) values of an image is decreased, it may be determined that the character of the streamer is dead.

All of the obtained information may be included in the information about the game state, and a first situation regarding the current game state may be derived based on the obtained information.

As described above, for deriving a specific area from a game image or detecting an object corresponding to a specific area, an embodiment may use various pre-trained models, and the learned model may be pre-trained with various algorithms through a learning dataset. . The training data set may include a training image and information indicating a learning area in the training image. For example, various neural networks or algorithms such as convolutional neural networks (CNN), artificial neural networks (ANN), recurrent neural networks (RNN), etc. may be used for learning, but are limited thereto It is not, and a conventional learning neural network or algorithm may be used in various ways.

Referring back to FIG. 3 , in step S330, an embodiment may determine a first situation regarding the current game state based on the information on the game state.

In step S340, an embodiment may determine the second situation regarding the streamer based on the audio information in the game broadcast. In more detail, it will be described in conjunction with FIG. 5 .

5 is a diagram for explaining a method of deriving audio information according to an embodiment of the present invention.

Referring to FIG. 5 , according to an embodiment, improved audio information in which noise is partially removed from the audio information may be derived by using the audio information and a pre-stored clip for removing noise. The pre-stored clip for noise control is for noise removal at a certain level, not for perfect noise control, and one for each game type may be prepared, but clips suitable for various situations may be prepared for high efficiency.

Or, for example, in one embodiment, unnecessary noise may be collected to remove unnecessary noise according to broadcasting characteristics, such as a keyboard sound, a mouse sound, or a background music sound of a game, and based on the collected unnecessary noise, a noise mask ( noise mask), and noise can be reduced using the generated noise mask.

In addition, according to an embodiment, the streamer's usual level of audio may be first collected to determine the streamer's excited state, and when audio of a level out of this is detected, the streamer's state may be determined as an excited state. To this end, an embodiment may determine an audio reference intensity on a frequency corresponding to the streamer based on improved initial audio information from the start of the game broadcast to a predetermined time among the improved audio information, and When an intensity on a frequency equal to or greater than the audio reference intensity is detected in the improved measurement target audio information after the predetermined time among audio information, the streamer may be determined to be an excited state.

Referring back to FIG. 3 , in step S350, an embodiment may determine an output phrase among a plurality of candidate phrases stored in the DB based on the first situation and the second situation.

For example, in one embodiment, a keyword may be derived based on at least one of the first situation and the second situation, and an output phrase may be determined from among a plurality of candidate phrases stored in the DB based on the keyword. . Here, in one embodiment, phrases may be derived based on a chat area included in another game video in another game broadcast before the same game as the game broadcast, and these may be stored in the DB as candidate phrases, and the DB stored in this way can be used to determine the output text.

For example, in one embodiment, the number of phrases generated during a preset chat window size (eg, 10 seconds) in the chat area in the other game video is a phrase within a first time interval corresponding to a chat window equal to or greater than a first threshold value can be derived, and it can be determined whether the number of words most shared among the phrases in the first time interval is equal to or greater than a second threshold, and when the number of words is equal to or greater than the second threshold, the first time interval Phrases in which a keyword corresponding to the word is shared among the phrases may be included in the plurality of candidate phrases stored in the DB. That is, in one embodiment, when collecting chatting/phrase from other game broadcasts, it is possible to construct a DB with more effective phrases by extracting a section in which the viewer's reaction intensively occurs.

Also, according to an embodiment, a phrase that fits the existing context/context and includes a keyword may be determined as an output phrase. For example, one embodiment determines a phrase within a second time interval of a preset size from the first phrase among the phrases in which the keyword is shared as a phrase matching the context for the first phrase, and the second time interval The method may further include determining the out of context phrase as a phrase out of context with respect to the first phrase. In other words, with respect to a specific phrase, a phrase located within the same time frame as the specific phrase is determined as a phrase matching the context of the specific phrase, and a phrase located within a time frame that is not the same as the specific phrase with respect to the specific phrase is selected as the context of the specific phrase. It can be determined as a phrase that does not fit into the context, and a BERT (Bidirectional Encoder Representations from Transformers) model can be trained using a specific phrase, a phrase that fits the context, and a phrase that does not fit the context, and the current chat area using the trained BERT model. From the phrases in the context, it is possible to determine a phrase including a keyword as an output phrase.

In addition, an embodiment may determine whether a preset affirmative word or negative word other than the keyword in the second phrase is included among the phrases in which the keyword is shared, and when the affirmative word is included in the second phrase, the keyword may determine shared phrases as positive phrases, and when the negative word is included in the second phrase, phrases shared with the keyword may be determined as negative phrases. In one embodiment, by labeling positive or negative phrases with respect to the phrases as above, when the phrases in the current chatting area are in a positive situation, it is possible to determine the output phrase among the positive phrases, and in a negative situation, output the negative phrases You can decide the text. In other words, according to an embodiment, whether a specific phrase is a positive phrase or a negative phrase can be labeled, and the above-described BERT-based emotion type classification model can be learned using data labeled for positive/negative. In one embodiment, based on the learned BERT-based emotion type classification model, it may be determined whether the phrase in the current chatting area is a positive phrase or a negative phrase, and it may be determined whether the current situation is positive or negative through the determined positive/negative phrase Accordingly, it is possible to determine an appropriate output phrase in consideration of positive/negative among the candidate segments.

Here, calculating a generality index of each of the phrases in which the keyword is shared based on the plurality of candidate phrases stored in the DB, and a phrase in which the generality index exceeds a third threshold among the phrases in which the keyword is shared The method may further include excluding from the plurality of candidate phrases. Here, the generality index may be expressed as perplexity, and when the generality index of a specific phrase is statistically below the third threshold, the specific phrase may be determined as a general phrase, and when the generality index of the specific phrase exceeds the third threshold, the specific phrase A phrase can be judged as an unusual, unusual phrase. Accordingly, an embodiment may exclude a phrase having a high generality index from candidate phrases so as to provide a most general answer.

In one embodiment, when setting keywords, keywords related to characters that do not appear in the game may be excluded, and in order to prevent determination of duplicate output phrases after outputting the output phrases in the game broadcast, the output phrases in the DB may be temporarily excluded.

6 is a diagram for explaining a method of calculating a response rate corresponding to an influence on an output phrase according to an embodiment of the present invention.

Referring to FIG. 6 , an embodiment may calculate a response rate for the output phrase in order to evaluate the influence on the output phrase. In other words, an embodiment may calculate a response rate corresponding to the influence of the output phrase based on the input phrases for a preset time from the time the output phrase is output. Here, for calculating the response rate, at least one of a first method regarding word matching, a second method regarding similarity, and a third method regarding question and answer may be used.

For example, in the first method among the three methods, the number of phrases in which the output phrase and at least one word overlap among the phrases input for a first time after the output phrase in the chat area including the output phrase, and the second It may be a method of calculating a response rate corresponding to the output phrase based on a ratio between the number of phrases for one hour. In addition, in the second method among the three methods, the number of phrases having a similarity with the output phrase among the phrases input for a second time after the output phrase in the chat area including the output phrase is equal to or greater than a preset value and during the second time It may be a method of calculating a response rate corresponding to the output phrase based on a ratio between the number of phrases. In addition, in the third method among the three methods, the last syllable of the last word among the phrases input for a third time after the output phrase in the chat area including the output phrase is one of the preset ending syllables, and the output phrase and It may be a method of calculating a response rate corresponding to the output phrase based on a ratio between the number of phrases in which at least one word overlaps and the number of phrases during the third time period. Here, the output phrase may indicate a phrase in which the end of the phrase is a question mark.

Referring back to FIG. 3 , in step S360, an embodiment may output the output phrase to a chat window in the game broadcast. That is, in one embodiment, the determined output phrase may be output to the chatting window area 441 together with the viewer.

The operation according to the embodiment of the present invention can be implemented as a computer-readable program or code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored. In addition, the computer-readable recording medium may be distributed in a network-connected computer system to store and execute computer-readable programs or codes in a distributed manner.

In addition, the computer-readable recording medium may include a hardware device specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. The program instructions may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although some aspects of the invention have been described in the context of an apparatus, it may also represent a description according to a corresponding method, wherein a block or apparatus corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method may also represent a corresponding block or item or a corresponding device feature. Some or all of the method steps may be performed by (or using) a hardware device such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

In embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In embodiments, the field programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by some hardware device.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it can be done.

Claims (8)

As a method of providing a chat service in real time in a game broadcast conducted by a streamer performed by a server,
acquiring a game image in the game broadcast;
extracting information about a game state from the game image;
determining a first situation regarding the current game state based on the information about the game state;
determining a second situation regarding the streamer based on audio information in the game broadcast;
determining an output phrase from among a plurality of candidate phrases stored in a DB based on the first situation and the second situation; and
Comprising the step of outputting the output phrase to a chat window in the game broadcast,
Further comprising the step of calculating a response rate corresponding to the influence of the output phrase based on the input phrases for a preset time from the time the output phrase is output,
The response rate is calculated using a first method for word matching, a second method for similarity, and a third method for question and answer,
The first method includes the number of phrases in which the output phrase and at least one word from among the phrases input for a first time after the output phrase in the chat area including the output phrase, and the number of phrases for the first time represents a method of calculating the response rate corresponding to the output phrase based on the ratio between the numbers,
In the second method, the number of phrases having a similarity with the output phrase equal to or greater than a preset value among phrases input for a second time after the output phrase in the chat area including the output phrase and the number of phrases for the second time represents a method of calculating a response rate corresponding to the output phrase based on the ratio between
In the third method, in the chat area including the output phrase, the last syllable of the last word among the phrases input for a third time after the output phrase is one of the preset ending syllables, and the output phrase and at least one word represents a method of calculating a response rate corresponding to the output phrase based on a ratio between the number of overlapping phrases and the number of phrases for the third time,
When the third method is used, the output phrase indicates a phrase where the end of the phrase is a question mark,
How to provide live chat service. The method according to claim 1,
Information about the game state,
It contains the location information between the health bar area, the system message area and the character extracted from the game image,
The system message area includes a system window area and a system notification area,
How to provide live chat service. The method according to claim 1,
The step of determining the second situation regarding the streamer based on the audio information in the game broadcast,
deriving improved audio information in which noise is partially removed from the audio information by using the audio information and a pre-stored noise removal clip;
determining an audio reference intensity on a frequency corresponding to the streamer based on the improved initial audio information from the start of the game broadcast to a predetermined time among the improved audio information; and
Comprising the step of determining the state of the streamer as an excited state when the improved measurement target audio information after a predetermined time among the improved audio information detects an intensity on a frequency equal to or greater than the audio reference intensity,
How to provide live chat service. The method according to claim 1,
deriving a chat area included in another game image in another game broadcasting before the same game as the game broadcasting;
deriving phrases within a first time period corresponding to the chat window in which the number of phrases generated during the chat window size preset in the chat area in the other game image is equal to or greater than a first threshold;
determining whether the number of words most shared among the phrases within the first time interval is equal to or greater than a second threshold; and
When the number of words is equal to or greater than the second threshold, including phrases in which a keyword corresponding to the word is shared among the phrases within the first time interval in the plurality of candidate phrases stored in the DB ,
How to provide live chat service. 5. The method according to claim 4,
Among the phrases in which the keyword is shared, a phrase within a second time interval of a preset size from the first phrase is determined as a phrase appropriate to the context for the first phrase, and a phrase outside the second time interval is determined for the first phrase. further comprising determining which phrase is out of context;
How to provide live chat service. 5. The method according to claim 4,
determining whether a preset positive word or negative word other than the keyword is included in a second phrase among the phrases in which the keyword is shared; and
When the affirmative word is included in the second phrase, phrases in which the keyword is shared are determined as positive phrases, and when the negative word is included in the second phrase, phrases in which the keyword is shared are determined as negative phrases further comprising the step of
How to provide live chat service. 5. The method according to claim 4,
calculating a generality index of each of the phrases in which the keyword is shared based on the plurality of candidate phrases stored in the DB; and
The method further comprising the step of excluding, from the plurality of candidate phrases, a phrase in which the generality index exceeds a third threshold among phrases in which the keyword is shared.
How to provide live chat service. delete

Images