KR102114457B1 - Method and apparatus for processing real-time character streaming contents - Google Patents

Abstract

Disclosed are a method and a device for providing real-time character streaming content. According to an embodiment, a method for processing real-time character streaming content comprises the steps of: generating a content providing channel and a content viewing channel for providing real-time character streaming content; recognizing connections through the content providing channel from a content providing terminal of a content provider; receiving a first character selection input for selecting a first character from the content providing terminal; recognizing connections through the content viewing channel from content viewing terminals of content viewers; receiving a recorded video stream of the content provider from the content providing terminal; generating a first character video stream in which a subject of the recorded video stream is replaced with the first character based on expression, motion, and sound information of the subject of the recorded video stream received from the content provider; and transmitting the generated first character video stream to the content viewing terminals.

Description

Method and device for processing real-time character streaming content {METHOD AND APPARATUS FOR PROCESSING REAL-TIME CHARACTER STREAMING CONTENTS}

The embodiments below relate to a technology for providing a real-time character streaming content service.

The character is a product that symbolizes a specific person or is given a personality and image by using animals or plants as a motif, and is utilized to help individuals and companies approach consumers more intimately. In recent years, with the development of communication technology, the way of consuming characters is expanding from traditional television media to various video platforms such as YouTube ^TM using a mobile platform. With the spread of 5G (5 generation) communication technology, virtual reality (VR) or augmented reality (AR) based content that can maximize the sense of presence and experience is also increasing.

As the size of data that can be exchanged in real time increases, technologies that provide video content in a streaming manner are being realized in real life, and services that communicate with videos beyond text and images are becoming common. With the development of communication technology and the expansion of video content services, the character content is usable, experience, by consuming characters through two-way interaction between characters and consumers, rather than one-way character consumption methods from existing character producers to consumers. Research on technology to increase convenience is required.

KR 1020060092625 KR 101987377B1 KR100881371B1 KR101111172B1

The embodiments are intended to provide viewers with real-time character streaming content.

The embodiments are intended to provide differentiated content for each viewer by generating video streaming content in which the subject of the captured video stream is replaced with a selected character.

The embodiments are intended to provide a real-time experience for a character desired by a viewer by generating a different character video stream according to the selected character type.

The embodiments generate streaming content in real time based on the viewer's action on the selected character, so that the viewer can be provided with an experience to interact with the character.

The embodiments generate streaming content in real time based on the number of viewers interacting with the selected character, thereby enabling active interaction between the viewer and the character.

The embodiments classify a group of content viewers for each selected character, so that the group of viewers and the character of the content provider can interact.

According to an embodiment, a method for processing real-time character streaming content includes generating a content providing channel and a content viewing channel for providing real-time character streaming content; Recognizing connections through the content providing channel from a content providing terminal of a content provider; Receiving a first character selection input for selecting a first character from the content providing terminal; Recognizing connections through the content viewing channel from content viewing terminals of content viewers; Receiving a video stream of the content provider from the content providing terminal; And generating a first character video stream in which the subject of the photographed video stream is replaced with the first character based on the expression, motion, and sound information of the subject of the photographed video stream received from the content provider. And transmitting the generated first character video stream to the content viewing terminals.

According to an embodiment, the method for processing real-time character streaming content, the step of receiving the first character selection input includes receiving a priority selection input for a plurality of characters, and generating the first character image stream The step of changing the first character to a character corresponding to the next priority selection input of the priority selection input when recognizing a preset first operation of the subject of the captured video stream; Removing the first character when recognizing a preset second operation of the subject of the captured video stream; Alternatively, when recognizing a preset third operation of the subject of the captured image stream, the method may include one or more of reinserting the removed first character.

According to an embodiment, receiving a second character selection input from a first content viewing terminal among the content viewing terminals; Generating a second character video stream in which the subject of the photographed video stream is replaced with the second character based on the expression, motion, and sound information of the subject of the photographed video stream; Transmitting the second character video stream to the first content viewing terminal; Receiving a third character selection input from a second content viewing terminal among the content viewing terminals; Generating a third character video stream in which the subject of the photographed video stream is replaced with the third character based on the expression, motion, and sound information of the subject of the photographed video stream; And transmitting the third character video stream to the second content viewing terminal, and facial expressions of each of the second character and the third character of the generated second character video stream and the third character video stream. , The motion and sound information may be different from the facial expression, motion, and sound information of the first character of the generated first character video stream.

According to an embodiment, generating the first character video stream may include: recognizing a preset operation of a subject of the captured video stream; Receiving a viewing video stream from the content viewing terminals; Recognizing at least one of a response operation, a non-response operation, and a non-response from the received viewing video stream, wherein the response operation is an operation performed in response to the preset operation; And generating at least one of an image stream in which the first character performs a first operation, an image stream in which a second operation is performed, and an image stream in which a third operation is performed, and the generated first character image. The step of transmitting the streams to the content viewing terminals may include one or more of the generated video streams in which the first character performs a first operation, a video stream in a second operation, and a video stream in a third operation. And transmitting the viewing video stream in which the response action, the non-response action, and the non-response are recognized, to each of the content viewing terminals, and wherein the video stream performing the first action comprises: When a response operation is recognized, the first character of the first character image stream is an image stream performing the first operation, and the second operation is performed. The video stream to be performed is an image stream in which the first character of the first character video stream performs a second operation when an operation other than the response is recognized, and the video stream performing the third operation is the When no response is recognized, the first character of the first character image stream may be an image stream performing a third operation.

According to one embodiment, the step of opening content chat rooms in each of the content viewing terminals; The method further includes receiving a chat input from the content viewing terminals and displaying it through the content chat room, and generating the first character video stream comprises: recognizing a preset operation of a subject of the captured video stream. ; Adding a video effect in which the first character performs a first operation based on a first keyword included in the chat input-the first keyword is a keyword corresponding to a preset operation of a subject of the captured video stream Im-; And adding a video effect in which the first character performs a second operation based on the second keyword included in the chat input.-The second keyword does not correspond to a preset operation of the subject of the captured video stream. This keyword can contain-;

According to an embodiment, generating the first character video stream may include: recognizing a preset operation of a subject of the captured video stream; Receiving a viewing video stream from the content viewing terminals; Recognizing a response operation from the received viewing video stream, wherein the response operation is an operation performed in response to the preset operation; And a video effect in which the first character performs the first operation when the ratio of the number of the content viewing terminals recognized by the response operation and the total number of content viewing terminals accessed through the content viewing channel is equal to or greater than a predetermined ratio. It may include the step of adding.

According to one embodiment, the step of opening content chat rooms in each of the content viewing terminals; The method further includes receiving a chat input from the content viewing terminals and displaying it through the content chat room, and generating the first character video stream comprises viewing the content inputting a first keyword through the chat input. When the ratio of the number of terminals and the number of total content watching terminals connected through the content viewing channel is equal to or greater than a predetermined ratio, the method may include adding a video effect in which the first character performs a first operation.

According to an embodiment, further comprising collecting each access time of the content viewing terminals connected through the content viewing channel, and generating the first character video stream comprises: each of the content viewing terminals The method may include adding a video effect in which the first character performs the first operation based on the access time of.

According to an embodiment, the step of generating the first character video stream may include summing costs equally paid by the first content viewing terminals; And adding a video effect in which the first character performs a first operation based on the summation result.

According to an embodiment, the content viewers include a first content viewer group and a second content viewer group, and receiving a second character selection input through content viewing terminals of the first content viewer group; And receiving a third character selection input through the content viewing terminals of the second content viewer group, and generating the first character video stream comprises the second character and the third character. It may include the step of generating a character video stream.

According to an embodiment, generating a first character image stream including the second character and the third character may include: recognizing a preset operation of a subject of the captured image stream; Receiving a viewing video stream from viewing terminals of each of the first content viewer group and the second content viewer group; Recognizing a first response operation from the received viewing video stream of the first group of content viewers, wherein the first response operation is an operation performed in response to the preset operation; Recognizing a second response operation from the received viewing video stream of the second group of content viewers, wherein the second response operation is a response operation that does not correspond to the preset operation; When the ratio of the total number of content viewing terminals of the first content viewer group and the total number of content viewing terminals of the first content viewer group performing the first response operation is greater than or equal to a first predetermined rate, the second character is Adding a video effect to perform the first operation; And when the ratio of the total number of content viewing terminals of the second content viewer group and the total number of content viewing terminals of the second content viewer group performing the second response operation is greater than or equal to a second predetermined rate, the third character. It may include adding a video effect to perform the first operation.

According to an embodiment, the step of establishing first content chat rooms in each of the content viewing terminal of the first content viewer group; Establishing second content chat rooms in each of the content viewing terminals of the second content viewer group; Receiving a first chat input from the content viewing terminal of the first content viewer group and displaying it through the first content chat room; And receiving a second chat input from the content viewing terminal of the second content viewer group and displaying it through the second content chat room, the first character video stream including the second character and the third character. In the generating, when the ratio of the total number of the content viewing terminals of the first content viewer group and the number of the content viewing terminals input the first keyword included in the first chat input is greater than or equal to the first predetermined rate , Adding a video effect in which the second character performs a first operation; And when the ratio of the total number of content viewing terminals and the total number of content viewing terminals of the second content viewer group inputting the second keyword included in the second chat input is greater than or equal to a second predetermined rate, the third character. It may include adding a video effect to perform the first operation.

According to an embodiment, the step of receiving the first character selection input may include receiving an input for selecting the first character from a user-created character, a licensed character, and a default character. step; And it may include the step of paying the cost to the character creator corresponding to the first character.

The apparatus according to an embodiment may be controlled by a computer program stored in a medium in order to execute the method of any one of the above-described methods in combination with hardware.

A real-time character streaming content processing apparatus according to an embodiment generates a content providing channel and a content viewing channel for providing real-time character streaming content; Recognize connections through the content providing channel from the content providing terminal of the content provider; Receiving a first character selection input for selecting a first character from the content providing terminal; Recognize connections through the content viewing channel from content viewing terminals of content viewers; Receiving a video stream of the content provider from the content providing terminal; The first character video stream is generated by replacing the subject of the photographed video stream with the first character based on the expression, motion, and sound information of the subject of the photographed video stream received from the content provider, and the generated first And a processor that transmits a character video stream to the content viewing terminals.

Embodiments may provide real-time character streaming content to viewers.

Embodiments may provide differentiated content for each viewer by generating video streaming content in which the subject of the captured video stream is replaced with a selected character.

Embodiments may provide a real-time experience for a character desired by the viewer by generating a different character video stream according to the type of the selected character.

Embodiments may generate streaming content in real time based on the viewer's action on the selected character, thereby providing the viewer with a sense of experience to interact with the character.

The embodiments may generate streaming content in real time based on the number of viewers interacting with the selected character, leading to active interaction between the viewer and the character.

Embodiments classify a group of content viewers for each selected character, so that the viewers' groups and the content provider's character can interact.

1 is a view for explaining a system for providing real-time character streaming content according to an embodiment.
2 is a flowchart illustrating a method for processing real-time character streaming content according to an embodiment.
3 to 8 are diagrams for describing a real-time character streaming content processing method according to an embodiment.
9 is an exemplary view of a configuration of an apparatus according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, various changes may be made to the embodiments, and the scope of the patent application right is not limited or limited by these embodiments. It should be understood that all modifications, equivalents, or substitutes for the embodiments are included in the scope of rights.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosure form, and the scope of the present specification includes modifications, equivalents, or substitutes included in the technical spirit.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from other components. For example, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

When an element is said to be "connected" to another element, it should be understood that other elements may be present, either directly connected to or connected to the other element.

The terms used in the examples are used for illustrative purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, the terms "include" or "have" are intended to indicate the presence of features, numbers, steps, actions, components, parts or combinations thereof described herein, one or more other features. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the embodiment belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined in the present application. Does not.

In addition, in the description with reference to the accompanying drawings, the same reference numerals are assigned to the same components regardless of reference numerals, and redundant descriptions thereof will be omitted. In describing the embodiments, when it is determined that detailed descriptions of related known technologies may unnecessarily obscure the subject matter of the embodiments, detailed descriptions thereof will be omitted.

Embodiments may be implemented in various types of products, such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent cars, kiosks, and wearable devices.

1 is a view for explaining a system for providing real-time character streaming content according to an embodiment.

Referring to FIG. 1, a system for providing real-time character streaming content is a system for providing video content using characters in real time, including a server 101, a network 102, a content provider group 103, and a content viewer group 104 ). The content provider group 103 is composed of subjects for providing real-time character streaming content to the content viewing terminals 151, 152, and 153 of the content viewers 141, 142, and 143. The real-time character streaming content may be content generated by replacing each subject of the captured video streams generated by the content provider 111 with a predetermined character. The real-time character streaming content may be content including a video stream in which the facial expression, motion, and sound information of the character is generated based on the subject's facial expression, motion, and sound information. For example, content providers can create video content by taking a picture of themselves with their respective smartphones, and the captured videos are replaced with character image video instead of the content provider's image video and provided in streaming form to viewers Video content. The content provider group 103 is a subject providing content, and may include, for example, a terminal transmitting a captured image for generating content and a subject that is a target of the captured image. The content viewer group 104 is a group associated with a subject who views content, and may include, for example, terminals that play or display real-time character streaming content and subjects who view or communicate content through the terminals. Here, for convenience, the content providing subject in the content provider group 103 is shown as a single one, but is not limited thereto, and may include a plurality of subjects.

The subject of the system according to an embodiment may be an electronic device including a communication function. For example, the electronic device includes a smart phone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop personal computer (PC), and a laptop. Laptop personal computer (PC), netbook computer, personal digital assistant (PDA), portable multimedia player (PMP), MP3 player, mobile medical device, camera, or wearable device (e.g. It may include at least one of a head-mounted-device (HMD) such as electronic glasses, an electronic garment, an electronic bracelet, an electronic necklace, an electronic appcessory, an electronic tattoo, a smart car, or a smart watch. Can be.

According to one embodiment, the electronic device may be a smart home appliance with a communication function. For smart home appliances, for example, electronic devices are televisions, digital video disk (DVD) players, audio, refrigerators, air conditioners, cleaners, ovens, microwaves, washing machines, air cleaners, set-top boxes, TVs. It may include at least one of a box (eg, Samsung HomeSync ^TM , Apple TV ^TM , or Google TV ^TM ), game consoles, electronic dictionaries, electronic keys, camcorders, or electronic picture frames.

According to an embodiment, the electronic device includes various medical devices (eg, magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), computed tomography (CT), photographing device, ultrasound device, etc.), navigation device, GPS receiver ( Global positioning system receiver (EDR), event data recorder (EDR), flight data recorder (FDR), automotive infotainment devices, marine electronic equipment (e.g. marine navigation devices and gyro compasses, etc.), avionics, security It may include at least one of a device, a head unit for a vehicle, an industrial or household robot, an automatic teller's machine (ATM) of a financial institution, or a point of sales (POS) of a store.

According to one embodiment, the electronic device is a furniture or part of a building / structure including a communication function, an electronic board, an electronic signature receiving device, a projector, or various measurement devices. It may include at least one of a device (for example, water, electricity, gas, or radio wave measurement devices). The electronic device according to an embodiment may be a combination of one or more of the various devices described above. Also, the electronic device according to an embodiment may be a flexible device. Also, it is apparent to those skilled in the art that the electronic device according to an embodiment is not limited to the above-described devices. The term user used in various embodiments may refer to a person using an electronic device or a device using an electronic device (eg, an artificial intelligence electronic device).

An electronic device according to an embodiment includes a processor, a memory, a user interface, and a communication interface, and may be connected to another electronic device through the network 102. The communication interface may transmit and receive data to and from other electronic devices within a predetermined distance through a wired, wireless network, or wired serial communication. The network 102 enables wired and wireless communication between various entities in a real-time character streaming content providing system according to an embodiment. The subjects in the system can communicate with each other through the network 102, and the network 102 can use standard communication technologies and / or protocols. At this time, the network 102 is the Internet (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), WCDMA (Wideband Code Division Multiple Access) , WiBro (Wireless Broadband), 3G (3rd Generation), LTE (Long-Term Evolution), WiMAX (Worldwide Interoperability for Microwave Acces) and 5G (5th generation). Anyone with ordinary knowledge in the field of telecommunications technology knows that it can be any other kind of network.

The network 102 according to an embodiment adopts a 5G standard, so that subjects in the system can broadcast a high-definition image using various equipment such as a 360-degree wearable camera and a drone, and the 5G communication standard has a large capacity through an ultra-high-speed large-capacity network. It can support data transmission and high resolution video service. The characteristics of 5G are summarized into ultra-high-speed (enhanced mobile broadband), ultra-reliable and low latency communication, and massive machine type communication. 5G can transmit more data faster by using a high-frequency band with a large amount of data transmission, and can experience 10 times faster sensation speed compared to 4G (LTE). Subjects in the system can transmit and receive large-scale content such as ultra-high-definition video or VR / AR quickly, implement real-time services through instant response and response, and greatly increase the number of terminals and sensors that can be connected to the Internet to create a large-scale IoT environment. Can be implemented.

According to one embodiment, the content provider group 103 includes a content provider 111. The content provider 111 may be an object of shooting, for example, a creator who wants to communicate with viewers using an image of a character, a character creator for promoting a character, or a content creator using a character for excellence have. The content provider 111 photographs an image through the content photographing device 131 and connects to the server 101 through the content providing terminal 121 to provide its own photographed image as streaming to viewers. Alternatively, the content provider 111 may take an image of itself using only the content providing terminal 121 regardless of time and place, and provide the captured images to viewers through the network 102.

According to one embodiment, content viewer group 104 includes content viewers 141, 142 and 143. The content viewers 141, 142, and 143 select the video stream transmitted from the content providing terminal 121, and the subject video stream generated by replacing the subject of the captured video stream with a character is the content viewing terminals 151, 152, and 153). For example, the content viewers 141, 142, and 143 may be fans of a character who wants to watch the video stream generated by the character of his or her favorite character instead of the subject. Content viewers 141, 142, and 143 connect to the server 101 through the content viewing terminals 151, 152, and 153, and view the video stream generated by the content provider 111 by the content viewing terminals 151 , 152 and 153). For example, the content viewers 141, 142, and 143 are video streams replaced by their favorite characters among the video streams respectively corresponding to a plurality of content providers (not shown) including the content provider 111. You can watch by selecting.

According to an embodiment, the content provider 111 may provide a captured video stream for providing video content in which the facial expression and motion of a character streamed according to the keyword, facial expression, and motion of the subject of the captured video stream are changed. Content viewers 141, 142, and 143 can experience characters that change in real time according to the capabilities of the subject of the captured video stream. The system for providing video content according to the present disclosure can generate a video stream in which the expression, motion, and sound of the subject of the photographed video stream provided by the content provider are replaced with the facial expression, motion, and sound of the character. The subject of the replaced video stream can be a character.

According to an embodiment, the content provider 111 wears hardware equipment (hereinafter referred to as 'motion capture equipment') including a processor, a memory, and a communication unit that performs a motion capture function to a part of the body to stream a character image Can shoot. A real-time video shooting application is stored in the memory of the motion capture device, and the application can be executed by a processor, and the executed real-time video shooting application can generate a character video stream using face rigging technology. The real-time video shooting application can acquire image data using the image sensor included in the motion capture device, and generate a character video stream based on the acquired image data. It may include an in-app module that enables. The generated character video stream may transmit the generated character video stream to the content viewing terminals 151, 152, and 153 of the content viewers 141, 142, and 143 whose communication channel is established by the communication unit.

According to an embodiment, the content provider 111 may provide a motion capture device equipped with a real-time video recording application in a space physically separated from each other, such as his room, his vehicle, and a studio specialized in video production (for example, a part of his body). For example, it can be worn on the head) to provide a character video stream to content viewers 41, 142, and 143. As described above, the content provider 111 can easily generate a character video stream regardless of time and stop by using the motion capture equipment, and can interact with the content viewers 141, 142, and 143 in real time. In addition, a motion capture device equipped with a real-time video shooting application can increase the accessibility of content providers to character content production.

According to an embodiment, a character may be inserted, changed, removed, or reinserted according to a specific operation or keyword of the subject of the captured video stream. For example, a character image may be inserted into the image of the content provider by the O display hand gesture of the content provider 111, and the subject of the streaming content may be replaced with the character, and the content provider 111 may clap once. The character may be changed, the character may be removed by the X display hand gesture of the content provider 111, and the removed character may be reinserted by the applause twice clapping action of the content provider 111.

According to an embodiment, the content viewers 141, 142, and 143 may select a preferred character according to the character selection mode, and the characters selected by each content viewer may watch video content that is the subject of the video stream. That is, for the same video content shot by the content provider 111, the video content streamed to the viewing terminals 151, 152, and 153 of the content viewers 141, 142, and 143 are different from each other according to the type of the selected character can do. As such, even if the expression, motion, and sound of the subject of the recorded video stream are the same, different contents may be generated according to the characters selected by the content viewers 141, 142, and 143, thereby providing an individual experience that suits each character preference of the viewers. Can be.

According to one embodiment, the content viewers 141, 142, and 143 are not just watching the character video content generated based on the captured image generated by the content provider 111, but are being streamed, and the character being streamed. You can interact with and create new content. For example, the content viewers 141, 142, and 143 select and watch the video of the content provider 111 that streams the video related to their favorite character, and previews the video in response to a predetermined operation of the character being watched. It is possible to perform a predetermined response operation. The content processing apparatus recognizes a predetermined response operation by the image sensor or motion capture equipment of the viewing terminals 151, 152 and 153, and each content of the predetermined response operation performed by the content viewers 141, 142 and 143 Depending on whether or not viewers interact with each other, character streaming content can be generated by generating different video streams. As such, the embodiments provide content that can interact with the character's facial expressions, movements, and sounds to provide differentiated content for each content viewer (141, 142, and 143), and provide an experience of individually communicating with the content provider. Can be.

According to one embodiment, a group 104 of content viewers 141, 142, and 143 may interact with the character being streamed to generate new content. For example, the group of content viewers 104 may be a group of viewers who have selected the same character, and may perform a predetermined response action in response to a predetermined action of the character being watched. At this time, when the ratio of the number of viewers who have performed a response action corresponding to a predetermined action of the character and the total number of content viewers belonging to the group 104 of content viewers is greater than or equal to a predetermined ratio (for example, a majority) In response to the operation, new character content can be generated. As such, embodiments may induce more than a majority of viewers to participate in order to consume new content about the character.

According to an embodiment, the content viewers 141, 142, and 143 may select a character from among predetermined characters capable of interacting with a character selected by the content provider 111 by a character selection mode. Content viewers 141, 142, and 143 may be grouped based on a selected character, and each grouped content viewer may perform a response action in response to a predetermined action of the character. When the ratio of the number of viewers performing the response operation among the total number of grouped content viewers is equal to or greater than a predetermined ratio (eg, a majority or more), new character streaming content corresponding to the response operation may be generated. As such, embodiments may group viewers into selected characters and induce more than a majority of the grouped viewers to consume new content related to the character.

According to an embodiment, a program for providing real-time character streaming content may be installed in the content providing terminal 121 and the content viewing terminals 151, 152 and 153. Depending on the installed program, real-time character streaming content service transmission and reception of information, processing of commands, and recording of information may be performed. The real-time character streaming content processing device is a device that performs operations for providing real-time character streaming content, and includes at least one of the server 101, the content providing terminal 121, and the content viewing terminals 151, 152, and 153, or It can be implemented in combination. The operation of the server 101 may include the operation of the motion capture device described above. The operation performed by the real-time character streaming content processing device may be the operation of the server 101, but as described above, at least one of the server 101, the content providing terminal 121, and the content viewing terminals 151, 152, and 153. It may be a linked operation by one. Hereinafter, an embodiment in which real-time character streaming content is provided will be described with reference to FIGS. 2 to 15.

2 is a flowchart illustrating a method for processing real-time character streaming content according to an embodiment.

According to an embodiment, the real-time character streaming content processing apparatus may generate a content providing channel and a content viewing channel for providing real-time character streaming content (201). A real-time character streaming content processing device (hereinafter referred to as a content processing device) is a device that provides a character video stream in real time, and may be implemented by, for example, a software module, a hardware module, or a combination thereof. The character streaming content may be a video stream generated by replacing the subject of the captured video stream with a character. As described above, the content processing apparatus may be implemented as a server that can be accessed through a network, but may also be implemented as a combination of a terminal and a server capable of communicating with a server. The content processing apparatus may execute instructions executed by programs installed on the server and the terminal.

According to an embodiment, the content providing channel is a channel for content providers to access the server through the content providing terminals, and the captured video stream of the content provider may be transmitted from the content providing terminal through the content providing channel. The content viewing channel is a channel for content viewers to access the server through the content viewing terminals, and the captured video stream of the content provider may be transmitted to the content viewing terminal through the content viewing channel. The content processing device may create a content providing channel and a content viewing channel through a network, and maintain a session for access of terminals. The content providing terminal and the content viewing terminal can transmit and receive information through the generated content providing channel and the content viewing channel.

According to an embodiment, the content processing apparatus may recognize connections through content providing channels from content providing terminals of content providers (202). The content processing device may maintain a channel for establishing a session, and the channel may correspond to a specific character. The content processing apparatus may recognize whether a content providing terminal of a content provider classified as a specific character accesses through a corresponding channel, and identify the recognized content providing terminal.

According to an embodiment, the content processing apparatus may receive a selection input for one of characters provided by predetermined character producers through a content providing terminal from a content provider (203). After recognizing the connection through the content providing channel from the content providing terminal, the content provider provides the character producers with a content processing device or a program for providing real-time character streaming content installed in the content providing terminal (hereinafter referred to as 'real-time character streaming program'). It may be asked to select or select a priority for one of the characters produced by. In short, this may be referred to as a character selection mode, and a content provider may select a character according to the character selection mode or select a priority for the character, and the content processing device may receive a selection input for the character.

According to an embodiment, the content processing apparatus may recognize connections through content viewing channels from content viewing terminals of content viewers (204). The content processing device may maintain a channel for establishing a session, and the channel may correspond to a character selected by content providers. The content processing apparatus may recognize whether the viewing terminals of content viewers classified into groups corresponding to the characters selected by the content providers access through the channel corresponding to the character, and identify the recognized content viewing terminals. The content providing channel and the content viewing channel may be matched with each other, and the content processing device may connect the content providing terminal and the content viewing terminals through the matched content providing channel and the content viewing channel.

According to an embodiment, the content processing apparatus may receive a captured video stream of the content provider from the content providing terminal (205). The content provider may take an image using an image sensor, the content providing terminal transmits the video stream shot in real time through the content providing channel, and the content processing apparatus may receive the captured video streams transmitted from the content provider.

According to an embodiment, the content processing apparatus may transmit the generated video stream to the content viewing terminal by replacing the subject of the captured video stream received from the content provider with a character selected by the content provider (206). The content viewing terminal may receive and play a video stream for a character in real time through a channel matched with a content provider. For example, the content processing apparatus replaces the image of the subject A selected by the content provider with the subject of the video stream (e.g., the content provider if the content provider photographed himself), so that the character A becomes the subject of the video stream. A character image stream may be generated, and the facial expression, operation, and sound information of the character may be generated based on the subject's facial expression, motion, and sound information. Content viewers can watch the video stream where the character A is not the actual shooting subject but the shooting subject. The generation of the character image stream may be performed by a character image stream generation program utilizing face rigging technology, and the character image stream generation program may be stored in the memory of the content processing device or stored in an external server of the content processing device to process the content. A character image stream may be generated by being communicatively coupled with a communication unit (not shown) of the device.

On the other hand, character producers provide characters to real-time character streaming programs, and provide characters created directly by real users, licensed characters, and default characters. can do. Content providers may be charged to pay the character creator who produced the character according to the character selection. For example, in the case of using the licensed character, if the content providers do not have a legitimate right to use the selected character, the usage fee for the character can be paid to the producer, and when using the basically provided character, Content providers can use the character for content creation for free. On the other hand, content providers can directly create new characters that match the subject's image and use them for content creation.

3 is a view for explaining a method of processing real-time character streaming content according to an embodiment.

According to an embodiment, the content processing apparatus may generate content providing channels and content viewing channels in order to provide a real-time character streaming content service. The content providing channels and the content viewing channels can be connected through the network 307. The content providing channels and the content viewing channels may be matched to each other and connected.

According to an embodiment, the content processing apparatus may recognize connections through the first content providing channel from the first content providing terminal of the first content provider 301.

According to an embodiment, the content processing apparatus may recognize connections through the first content viewing channel from the first content viewing terminals of the first content viewers 304. The first content viewers 304 may be grouped and classified as subjects who watch character streaming content. The content processing device may provide the character video stream generated based on the captured video streams of the first content provider 301 to the first content viewing terminals of the first content viewers 304 through the first content viewing channel. have. For example, the first content provider 301 may be a subject for promoting the character A, the first content viewers 304 may be a group of fans who prefer the character A, and the content processing device may include the first content. The provision channel and the first content viewing channel may be connected to enable interaction between the content provider and the viewer.

According to an embodiment, the content processing apparatus may provide a real-time character video stream through content viewing channels by connecting content providing channels and content viewing channels to each other. The content processing apparatus may connect the second content providing terminal of the second content provider 302 and the second content viewing terminals of the second content viewers 305 to each other through the second content providing channel and the second content viewing channel. The content processing apparatus may connect the third content providing terminal of the third content provider 303 and the third content viewing terminals of the third content viewers 306 to each other through the third content providing channel and the third content viewing channel.

For example, the second content provider 302 and the third content provider 303 are the subjects that provide character-related streaming content by selecting characters B and C, respectively, and the second content viewers 305 and the third content Viewers 306 may be a group of fans who like character B and character C, respectively. The content processing device may provide a content platform for grouping content providers and content viewers and matching each other to meet differentiated needs, and to participate in generating and viewing various types of content.

4 is a view for explaining a method of processing real-time character streaming content according to an embodiment.

According to one embodiment, the content processing device receives respective captured image streams through the network 403 from the content providing terminals 421 and 422 of the content providers 401 and 402 who want to generate character streaming content. Can be. The content processing apparatus may recognize a connection through the network 403 of the first content viewer group 404 to the second content viewer group 405. Content viewer groups may be classified by content providers and connected through a content providing channel and a content viewing channel. For example, the first content viewer group 404 may correspond to the content provider 401 and the second content viewer group 405 may correspond to the content provider 402.

According to an embodiment, the content processing apparatus may recognize a connection through a content providing channel by the first content provider 401 and receive an image stream photographed from the first content provider 401. The content processing device may input a first selection input for a first character or a priority for a character through the user interface 432 of the first content providing terminal 421 of the first content provider 401. Can receive. The first character selection mode may be a mode in which an image in which a subject of an image stream provided by content providers is replaced with a first character is displayed.

According to one embodiment, the content processing apparatus is the first for the first character among the character selection modes from the first content viewing terminals of the first content viewers 411 to 413 belonging to the first content viewer group 404. A selection input or a default character selection input can be received. The first character selection mode may be a mode in which an image in which a subject of an image stream provided by content providers is replaced with a first character is displayed. The default character selection mode is a selection mode in which a video stream of a first character selected by a content provider can be provided when no character is selected in the character selection mode.

According to an embodiment, the first content viewer group 404 including the first content viewers 411 to 413 may be a group classified by the first selection input for the first character selection mode. For example, the first content viewer 411 may input a first character selection mode in which an image is displayed around a puppy character through the user interface 435 of the first content viewing terminal 423. The first content viewers 411 to 413 may be users who have selected the first character selection mode.

According to an embodiment, the content processing apparatus may identify the first captured video stream of the content provider 401 and generate a first character video stream in response to the first selection input for the first character selection mode. Specifically, the content processing apparatus may generate a first character image stream by replacing the subject of the first captured image stream with the first character based on the identified first captured image stream. For example, the subject of the video stream captured by the content provider A 401 is A, but in response to the first selection input (ie, the puppy character) for the first character selection mode of the content provider A 401, the content The subject of the video stream can be converted from A to a puppy character by a processing device. Here, the first character video stream is a video stream in which facial expression, motion, and sound information of a subject photographed in the first captured video stream is converted into facial expression, motion, and sound information of the first character and played back, as described above. The rigging technique may be used, and the subject of shooting (progress) of the first character image stream may be the first character (eg, a puppy).

According to one embodiment, the content processing apparatus converts the video stream of the content provider into a first character video stream to the first content viewing terminals of the first content viewers 411 to 413 in the first content viewer group 404 Can transmit. The first content viewing terminals may reproduce and display the first character video stream. For example, the first content viewing terminal 423 of the first content viewer 411 may display the first character video stream 436 corresponding to the first character selection mode through the user interface 435.

According to an embodiment, the content processing apparatus may receive a second selection input for the second character selection mode among the character selection modes from the second content viewing terminal 422 of the second content provider 402. The second character selection mode may be a mode in which an image in which the subject of the video stream photographed by the second content provider 402 is replaced with the second character is displayed.

According to an embodiment, the content processing apparatus may select a second character selection mode among the character selection modes from the second content viewing terminals of the second content viewers 414 to 417 belonging to the second content viewer group 405. A second selection input may be received. The second character selection mode may be a mode in which an image is displayed as a second character as a photographing subject.

According to an embodiment, the second content viewer group 405 including the second content viewers 414 to 417 may be a group classified by a second selection input for the second character selection mode. For example, the second content viewer 417 may input a second character selection mode in which an image is displayed around a baby character through the user interface 437 of the second content viewing terminal 424. The second content viewers 414 to 417 may be users who have selected the second character selection mode.

According to an embodiment, the content processing apparatus may identify a second captured video stream of the content provider 402 corresponding to the second selection input among the content providers 401 and 402. The content processing device may generate a second character video stream that converts the content provider B, which is a photographing subject of the identified second photographed video stream, into a second character (eg, a baby character).

According to an embodiment, the content providing terminals 421 and 422 may encode the captured video streams and transmit them to the content processing device. The content processing device may decode the captured image streams to generate a character image. The content processing apparatus may convert a photographing subject of the decoded photographed image streams to a specific character to generate a character-oriented image stream. The content processing apparatus may encode the character-centric video stream and transmit the encoded character-centric video stream to the content viewing terminal. The content viewing terminal may decode, play, and display the received character-centric video stream.

According to an embodiment, the content provider 401 may be a shooting subject of the first captured video stream, and one of characters being inserted, changed, removed, and reinserted according to the first operation of the content provider 401 Can be executed. For example, a puppy character according to the first selection input of the content provider A 401 may be inserted in place of the image of the content provider A by the O display hand gesture of the content provider A 401 as a shooting subject. In addition, the puppy character displayed on the screen of the viewing terminal 423 of the content viewer 411 may be changed to a cat character according to the character priority selection input by the clapping action of the content provider A 401, The puppy character may be removed by the X display hand gesture of the content provider A 401, and the puppy character removed by the clapping twice action of the content provider A 401 may be reinserted. Also, the content provider 402 may be a subject of the second captured video stream, and one of characters inserted, changed, removed, and reinserted according to the second operation of the content provider 402 may be executed. . As such, the content provider may provide viewers with an experience to experience various characters through predetermined actions without performing character selection input each time according to a specific character selection or priority of character selection input.

On the other hand, according to another embodiment, the content provider 401 may be a subject of the first captured video stream, and a character may be inserted, changed, or removed as a predetermined keyword of the content provider 401 is entered into the chat room. And re-insertion may be performed. For example, as the content provider A (401), which is a shooting subject, inputs the keyword 'character insertion', a puppy character according to the first selection input of the content provider A (401) is inserted in place of the image of the content provider A. Can be. In addition, as the content provider A 401 inputs the keyword 'character change', the puppy character displayed on the screen of the viewing terminal 423 of the content viewer 411 is changed to a cat character according to the character priority selection input. The puppy character may be removed as the content provider A 401 enters the keyword 'remove character', and the puppy character removed as the content provider A 401 enters the keyword 'reinsert character' Can be reinserted.

5 is a view for explaining a method of processing real-time character streaming content according to an embodiment.

According to an embodiment, the content processing apparatus may provide a character streaming content customized for the viewer according to the character selection input of the content viewer. The content processing apparatus may receive an additional character selection input of the content viewer, and may change the character displayed on the screen of the viewing terminal according to the additional character selection input of the content viewer.

According to an embodiment, the content provider 501 may be a subject that provides the first character streaming content to the content viewer group 504 according to the first character selection input. The content viewer group 504 is a group that selects the first character and can watch the first character streaming content of the content provider 501. In addition, the content viewers 511 to 514 of the content viewer group 504 can additionally select the second character from the character selection input mode from the user interface of the content viewing terminal to select the second character. When the content processing apparatus receives the additional second character selection input from the content viewers 511 to 514, the content processing device may transmit the second character streaming content to the viewing terminal of the content viewer who has selected the second character.

For example, the puppy character video stream of the content provider 501 is the video of the viewing terminals 522 to 523 of the content viewer group 504 through the matched content providing channel and the content viewing channel as the puppy character is selected. It can be played back as streams 531 to 532. The content viewer 514 of the content viewer group 504 can select the baby character (ie, the second character) through the additional second character selection mode from the user interface of the content viewing terminal 523. The content processing apparatus changes the subject of the video stream photographed by the content provider 501 from the puppy character to the baby character based on the selection input to the baby character of the content viewing terminal 523 to view the terminal of the content viewer 514 (523).

According to an embodiment, the content processing apparatus may differently change the expression, motion, and sound of the image stream subject photographed by the content provider 501 according to the selected character type. For example, when the content viewer 514 selects the baby character as the second character according to the additional character selection mode, the video stream displayed on the viewing terminal 523 may be baby character streaming content. Meanwhile, the content viewer 512 without additional character selection can still watch the first selected puppy character streaming content. As an embodiment, the video stream generated according to the specific operation of the content provider 501 is transmitted to the content processing device, and the content processing device may generate the video stream generated by different actions for each character. For example, the wink action video stream of the content provider 501 may be generated as a puppy wink action video stream in the viewing terminal 522 of the content viewer 512, but the viewing terminal 523 of the content viewer 514 ) May be generated as a stream of motion images of a baby character. As such, even if the expression, motion, and sound of the video stream subject photographed by the content provider are the same, different content may be generated according to the character selected by the content viewer, thereby providing an individual sense of experience suitable for each character preference of the viewer.

6 is a view for explaining a method of processing real-time character streaming content according to an embodiment.

According to an embodiment, the content processing apparatus may dynamically change the character video stream based on the interaction between the content viewer and the character to generate the video stream and transmit it to the content viewers. The content processing apparatus may generate an image stream for providing to the viewer based on various variables such as the expression, voice and motion of the content viewer, feedback and statistics of the content viewer, the situation, the content providing environment, and the time interval.

According to an embodiment, the content processing apparatus may recognize a predetermined operation performed by a content provider or a predetermined keyword input into a chat room of the content providing terminal, and provide a video stream in which the character performs a predetermined operation.

According to one embodiment, content viewers may perform a predetermined response action to the viewing terminal or the motion capture device in response to a predetermined action of the character being watched. Character content content displayed on the viewing terminal may vary for each viewer according to each content of a predetermined response action performed by the content viewers.

For example, the content provider 601 may perform the high-five operation 621 while photographing the person's appearance with the content providing terminal 622. The content processing apparatus may transmit a video stream in which the high five action 621 of the content provider 601 is replaced by the high five action 621 of the character to the content viewer group 603. The content viewers 611 to 613 belonging to the content viewer group 603 may perform predetermined actions 631 and 641 corresponding to the high-five action 621 of the character toward the content viewing terminals 632 and 642. . Specifically, the content viewers 611 to 613 can capture the facial expressions and motions of the person displayed through the image sensors of the content viewing terminals 632 and 642 (the viewing terminal of the content viewer 612 is not shown), and the content processing device May recognize the facial expressions and actions of the content viewers 611 to 613 based on the video stream captured by the content viewing terminals 632 and 642. For example, the content viewer 611 may perform a high five operation 631 toward the content viewing terminal 632, and the content viewer 613 may raise an thumb toward the content viewing terminal 642 ( 641). The content processing apparatus receives video streams including the actions of the content viewers 611 and 613 from the content viewing terminals 632 and 642, and the characters are different for each response action of the content viewer 611 and the content viewer 613 An image stream performing an operation may be generated.

In one embodiment, the generated character video stream may be determined in advance according to an operation performed by a content provider and a content viewer, and facial expression, voice and motion, and character property changes to induce reward for active participation of content viewers And the like. When a response action that does not correspond to a predetermined response action of the content viewer is recognized, a default character image stream may be generated. The default character content is an image stream that can be generated when a content viewer performs facial expressions, voices, and gestures that are not matched with facial expressions, voices, and gestures performed by a content provider, and induces viewers to interact with characters It may include facial expressions, voice and gestures.

For example, the content processing device is a puppy to the viewing terminal 632 of the content viewer 611 that has performed the high-five operation 631, which is a predetermined response operation, to the high-five operation 621 of the content provider 601. The video stream 633 of a character's delighted expression can be transmitted. On the other hand, a puppy character grieves at the viewing terminal 642 of the content viewer 613 who performs an operation 641 for raising a thumb rather than a predetermined response operation for the high five operation 621 of the content provider 601. The video stream 643 of the facial expression can be transmitted.

As another embodiment, the content processing apparatus may generate a character video stream based on keywords included in a chat room of a content viewer's viewing terminal. The content processing apparatus may open content chat rooms in each of the content viewing terminals of the content viewers, receive chat input from the content viewing terminals, and the chat inputs may be displayed on the content viewing terminal. The content processing device may recognize a preset operation of the subject of the shooting stream provided by the content provider 601, and the content viewers 611 to 613 input keywords corresponding to the preset operation into the content chat room of the content viewing terminal can do. When the first keyword is recognized among the chat inputs, the content processing device may generate a character video stream by adding a video effect in which the character performs the first operation. In addition, when the second keyword is recognized among the chat inputs, the content processing device may generate a character video stream by adding a video effect in which the character performs a second operation. Here, the first keyword may be a keyword corresponding to a preset operation of the subject of the captured video stream, and the second keyword may be a keyword that does not correspond to a preset operation of the subject of the stream.

For example, the content processing device may recognize the high five operation 621 of the content provider 601. In response to this, the content viewer 611 may input the keyword 'high five' into the chat room of the content viewing terminal 632, and the content processing device may input the keyword 'high five' corresponding to the high five operation 621. Upon reception, a video stream to which a puppy character is pleased can be added. In addition, the content viewer 613 may input the keyword 'merong' into the chat room of the content viewing terminal 642, and the content processing device receives the keyword 'merong' that does not correspond to the high five operation 621. , You can create a video stream that adds a video effect that a dog character grieves.

As such, content viewers can not only watch the character video content provided by the content provider in a uniform and unidirectional manner, but also create a new video stream as they interact with characters being streamed, actions, and chat keywords, etc. It can provide differentiated content for each and provide an experience of individually communicating with content providers.

7 is a view for explaining a real-time character streaming content processing method according to an embodiment.

According to an embodiment, the content processing apparatus includes facial expressions, voices and gestures, content feedback and statistics of content viewers, content viewers interacting with characters over a predetermined percentage of the total number of content viewers belonging to the group of content viewers, It is possible to create new character content by recognizing various variables such as the content providing environment and time intervals. According to an embodiment, the grouped content viewers may perform a predetermined response action to the viewing terminal or the motion capture device in response to a predetermined action of the character being watched, and the content processing device may perform a content response action. When the ratio of the total number of content viewers in the content viewer group to the number of viewers is greater than or equal to a predetermined rate, a character video stream corresponding to the response operation may be generated and transmitted to the viewer terminal.

For example, the content provider 701 may perform a high-five operation 721 on the content providing terminal 722, and the content processing apparatus may image the high-five operation 721 received from the content provider 701. The stream can be replaced with a high-five action of the character and transmitted to the content viewing terminals 741 to 745. The content viewers 711 to 714 of the content viewer group 703 may perform the high five operations 731 to 734 corresponding to the high five operation 721 of the content provider to the viewing terminals 741 to 734, The content processing apparatus may receive the viewing video stream of the high five operations 731 to 734 of the content viewers 711 to 714. When the ratio of the number of content viewing terminals and the total number of content viewing terminals of the content viewer group in which the high-five operations 731 to 734 are recognized is higher than a predetermined ratio (for example, a majority or more), the content processing apparatus responds to the high five A character image stream to which an image effect according to an operation is added may be generated.

In one embodiment, the generated character content may be predetermined according to an operation performed by the content provider and the content viewer group, and facial expressions, voice and motion, and character attributes to induce compensation for active participation of the content viewer group And the like. Video streams that include facial expressions, voice and gestures, and character attribute changes to induce rewards for active participation of a group of content viewers include items that viewers can purchase at cost, character upgrades, and changes in character abilities. It can contain. Meanwhile, if an operation other than a predetermined operation of the content viewer group is recognized, default character content may be generated. The default character content is a video stream that can be generated when a content viewer group performs facial expressions, voices, gestures, and keywords that are not matched with facial expressions, voice, and gestures performed by a content provider, and interacts with characters to viewers It may include facial expressions, voice and motion to induce.

For example, the content processing apparatus may include the number of content viewers 711 to 714 and the content viewers performing the high five actions 731 to 734, which are predetermined response actions to the high five action 721 of the content provider 701. When the ratio of the number of content viewers in the group is equal to or greater than a predetermined ratio (for example, a majority or more), a video stream 752 to which a video effect changed by a character wearing a character is added may be generated. In addition, when the number of content viewers in which a predetermined response action is recognized for the high-five action 721 of the content provider 701 does not satisfy the above-described ratio, an image effect of a character's sad expression is added. The video stream can be transmitted. In addition, the ratio of the number of content viewers and the number of content viewing terminals in the content viewer group that the response action that does not correspond to the predetermined response action to the high-five action 721 of the content provider 701 is greater than or equal to a predetermined rate In this case, a video stream to which a video effect of a character's sad expression is added may be transmitted.

As another embodiment, the content processing apparatus recognizes a keyword input into a chat room of the content viewing terminal, and based on the ratio of the number of content viewing terminals and the total number of content viewing terminals that input the keyword is greater than or equal to a predetermined rate A character video stream can be generated. The content processing device may open content chat rooms in each of the content viewing terminals of the content viewers, receive chat input from the content viewing terminals, and display the content chat rooms. When the ratio of the number of content viewing terminals inputting the first keyword through the content chat room of the content viewing terminal and the total number of content viewing terminals accessed through the content viewing channel is greater than or equal to a predetermined ratio, the first character performs the first operation. A video stream can be generated by adding a video effect to be performed.

For example, the content processing device may recognize the high five operation 721 of the content provider 701. Correspondingly, the content viewers 711 to 714 can input the keyword 'high five' into the chat room of the content viewing terminals 741 to 744, and the content processing device corresponds to the 'high' operation corresponding to the high five operation 721. As the ratio of the number of content viewing terminals 741 to 744 input the keyword 'five' and the number of total content viewing terminals connected through the content viewing channel is greater than or equal to a predetermined ratio (for example, a majority or more), a puppy character It is possible to create a video stream with video effects that are pleasing.

As another embodiment, the content processing apparatus may generate a character video stream according to the connection time of each of the content viewing terminals. The content processing apparatus may collect each access time of content viewing terminals connected through the content viewing channel. The content processing apparatus may generate a character video stream by adding a video effect in which the character performs a specific operation based on each access time of the content viewing terminals.

For example, the content processing apparatus collects the access time of each of the content viewing terminals 741 to 744 and the facial expression of the character rejoicing on the basis that each access time is greater than or equal to a predetermined time (eg, 20 minutes or more). A character video stream to which a video effect of is added may be generated. As such, embodiments may induce more than a majority of viewers to participate, so as to provide new content about the character.

As another embodiment, the content processing apparatus may generate a character video stream by summing the costs equally paid by the content viewing terminals. The content processing device may sum the information of costs that are evenly paid from the content viewing terminals. According to the summed cost information, the content processing device may generate a character video stream to which a video effect in which a character performs a specific operation is added. Rather than simply a charging method paid by a specific consumer or viewer, the content processing device may provide a sense of experience in which content viewers who prefer characters participate in generating additional image effects of the characters by paying equally.

8 is a view for explaining a method of processing real-time character streaming content according to an embodiment.

According to an embodiment, the content processing apparatus may group content viewers according to a character selection input, and generate a new character video stream by interaction of characters of a content provider character with groups of content viewers. Specifically, the content viewers can select a character through the character selection mode from the content viewing terminal, and the content processing device can receive the character selection input of the content viewers. The content processing apparatus groups content viewers among viewers who select the same character according to the character selection mode of the content viewers, thereby generating an image stream capable of interacting with the selected character.

For example, the content provider 811 may select a puppy character in the character selection mode 812, and the content processing device generates a puppy character image stream based on the captured image stream of the content provider 811, and content viewers It can be transmitted to the viewing terminal. Content viewers watching the video stream of the content provider 811 can select a character according to the character selection modes 841 and 851, and the content processing device selects a cat character according to each character selection input. As a group 803, content viewers who have selected a ghost character may be classified into a content viewer group 804. The puppy character selected by the content provider 811, the cat character selected by the content viewer group 803, and the ghost character selected by the content viewer group 804 can be simultaneously expressed on the content providing terminal and the content viewing terminals, and each character They can interact with the selected subject's predefined expressions, actions, sounds, and keywords.

According to an embodiment, grouped content viewers may perform a response operation corresponding to a predetermined operation of a character of a content provider. The content processing device generates a new video stream for the character selected by the content viewer group when the ratio of the number of content viewers performing the response operation to the total number of viewers belonging to the content viewer group is equal to or greater than a predetermined ratio (for example, a majority). can do.

For example, the content provider 811 may perform the high-five operation 813 on the content providing terminal 814, and the content processing apparatus may image the high-five operation 813 received from the content provider 811. The stream can be replaced with the high-five action of the puppy character and transmitted to the content viewing terminals 822b and 833b. The content viewers 821 to 824 among the content viewer group 803 may perform high-five operations 821a to 824a on the viewing terminal 822b. The content processing apparatus recognizes that the ratio of the number of the content viewers recognized by the high-five operations 821a to 824a and the total number of content viewers in the content viewer group is equal to or greater than a predetermined ratio (for example, a majority), and the high-five response Depending on the operation, a video stream in which a puppy character and a cat character interact can be generated.

In addition, in response to the high-five operation 813 of the content provider 811, the content viewers 831 to 834 of the content viewer group 804, the high-five operation of the content provider 811 to the viewing terminal 833b ( It is possible to perform the operation 831a to 834a of raising the thumb that does not correspond to 813). The content processing device recognizes that the ratio of the number of content viewers and the total number of content viewers in the content viewer group in which the gestures for raising the thumbs 831a to 834a are recognized is greater than or equal to a predetermined ratio (for example, a majority) You can create a video stream in which the character and the ghost character interact. The generated puppy character, cat character, and ghost character interact with the video streams can be displayed in the same video stream in the content viewing terminals 822b and 833b, and the content processing device can display various videos according to the combination of response actions of the content viewers. You can create a stream.

In one embodiment, the generated character content may be predetermined according to an operation performed by the content provider and the content viewer group, and facial expressions, voice and motion, and character attributes to induce compensation for active participation of the content viewer group And the like. Video streams that include facial expressions, voice and gestures, and character attribute changes to induce rewards for active participation of a group of content viewers include items that viewers can purchase at cost, character upgrades, and changes in character abilities. It can contain. Meanwhile, if an operation other than a predetermined operation of the content viewer group is recognized, default character content may be generated. The default character content is a video stream that can be generated when a content viewer group performs facial expressions, voices, gestures, and keywords that are not matched with facial expressions, voice, and gestures performed by a content provider, and interacts with characters to viewers It may include facial expressions, voice and motion to induce.

For example, in response to the high-five operation of the content provider 811, the content processing apparatus performs a high-five operation in which a majority of the content viewers in the content viewer group 803 corresponds to the high-five operation of the content provider 811. When performed, an image stream in which a high-five action is performed by a dog character and a cat character may be generated. In addition, in response to a high five operation of the content provider 811, when a majority of the content viewers in the content viewer group 804 perform an operation of raising a thumb that does not correspond to the high five operation of the content provider 811 , It is possible to generate a video stream that performs a grieving action between a dog character and a ghost character. That is, the content processing apparatus may generate various video streams according to a combination of response actions of content viewers. As such, embodiments may group viewers by selected characters so as to provide new content about the character, and induce more than a majority of the grouped viewers to participate.

In another embodiment, the content processing apparatus may group content viewers according to the character selection input and generate a character video stream based on keyword input of the content viewers. The content processing apparatus may include a first content viewer group and a second content viewer group according to the character selection input, and a character video stream including a second character and a third character corresponding to the character selected by each content viewer group Can generate The content processing apparatus may open the first and second content chat rooms in each of the content viewing terminals of the first and second content viewer groups, and receive first and second chat inputs input to the first and second content chat rooms Thus, it can be displayed in chat rooms. When the ratio of the number of content viewing terminals input the first keyword through the first chat input and the total number of content viewing terminals in the first content viewer group is equal to or greater than a predetermined ratio, the content processing device selects the first content viewer group A video stream may be generated by adding a video effect in which the second character performs a specific operation. In addition, the content processing apparatus, if the ratio of the number of content viewing terminals input the second keyword through the second chat input and the total number of content viewing terminals in the second content viewer group is more than a predetermined ratio, the second content viewer group A video stream may be generated by adding a video effect in which the selected third character performs a specific operation.

For example, in response to the high five action 813 of the content provider 811, more than half of the content viewers in the content viewer group 803 correspond to the high five action of the content provider 811 in the chat room of the content viewing terminal. When the keyword 'high five' is input, the content processing device may generate a video stream in which the dog character and the cat character perform a high five operation. In addition, in response to the high-five operation of the content provider 811, more than half of the content viewer group 804 inputs the keyword 'Meron' that does not correspond to the high-five operation of the content provider 811 in the chat room of the content viewing terminal. In this case, the content processing device may generate a video stream that performs an operation of grieving between the puppy character and the ghost character. As such, embodiments may group viewers by selected characters so as to provide new content about the character, and induce participation of more than half of the grouped viewers.

9 is an exemplary view of a configuration of an apparatus according to an embodiment.

The device 901 according to an embodiment includes a processor 911 and a memory 912. The device 901 according to an embodiment may be the above-described server or terminal. The processor 911 may include at least one device described above with reference to FIGS. 1 to 8, or may perform at least one method described above with reference to FIGS. 1 to 8. The memory 912 may store information related to the above-described method or a program implemented with the above-described method. The memory 9123 may be a volatile memory or a nonvolatile memory.

The processor 911 may execute a program and control the device 901. The code of the program executed by the processor 911 may be stored in the memory 912. The device 901 is connected to an external device (for example, a personal computer or a network) through an input / output device (not shown), and can exchange data.

The embodiments described above may be implemented by hardware components, software components, and / or combinations of hardware components and software components. For example, the devices, methods, and components described in the embodiments include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors (micro signal processors), microcomputers, and field programmable gates (FPGAs). It can be implemented using one or more general purpose computers or special purpose computers, such as arrays, programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specifically configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and / or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodied in the transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

As described above, although the embodiments have been described with limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and / or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if substituted or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (15)

A real-time character streaming content processing method performed by a real-time character streaming content processing device,
Generating a content providing channel and a content viewing channel for providing real-time character streaming content;
Recognizing connections through the content providing channel from a content providing terminal of a content provider;
Receiving a first character selection input for selecting a first character from the content providing terminal;
Recognizing connections through the content viewing channel from content viewing terminals of content viewers;
Receiving a video stream of the content provider from the content providing terminal;
Generating a first character video stream in which the subject of the photographed video stream is replaced with the first character based on the expression, motion, and sound information of the subject of the photographed video stream received from the content provider; And
Transmitting the generated first character video stream to the content viewing terminals
Including,
Generating the first character video stream,
Recognizing a preset operation of a subject of the captured video stream;
Receiving a viewing video stream from the content viewing terminals;
Recognizing a response operation from the received viewing video stream, wherein the response operation is an operation performed in response to the preset operation; And
When the ratio of the number of the content viewing terminals recognized by the response operation and the total number of content viewing terminals accessed through the content viewing channel is equal to or greater than a predetermined ratio, the first character performs a video effect of performing the first operation. Steps to add
Containing,
Real-time character streaming content processing method.
According to claim 1,
The step of receiving the first character selection input,
And receiving a priority selection input for a plurality of characters,
Generating the first character video stream,
Changing the first character to a character corresponding to the next priority selection input of the priority selection input when recognizing a preset first operation of the subject of the captured video stream;
Removing the first character when recognizing a preset second operation of the subject of the captured video stream; or
Re-inserting the removed first character when recognizing a preset third motion of the subject of the captured video stream
Containing one or more of the
Real-time character streaming content processing method.
According to claim 1,
Receiving a second character selection input from a first content viewing terminal among the content viewing terminals;
Generating a second character video stream in which the subject of the photographed video stream is replaced with the second character based on the expression, motion, and sound information of the subject of the photographed video stream;
Transmitting the second character video stream to the first content viewing terminal;
Receiving a third character selection input from a second content viewing terminal among the content viewing terminals;
Generating a third character video stream in which the subject of the photographed video stream is replaced with the third character based on the expression, motion, and sound information of the subject of the photographed video stream; And
Transmitting the third character video stream to the second content viewing terminal;
Further comprising,
Expressions, motions, and sound information of each of the second character and the third character of the generated second character image stream and the third character image stream include facial expressions of the first character of the generated first character image stream, Different from the motion and sound information,
Real-time character streaming content processing method.
According to claim 1,
Generating the first character video stream,
Recognizing a preset operation of a subject of the captured video stream;
Receiving a viewing video stream from the content viewing terminals;
Recognizing at least one of a response operation, a non-response operation, and a non-response from the received viewing video stream, wherein the response operation is an operation performed in response to the preset operation; And
Generating at least one of a video stream in which the first character performs a second operation, a video stream in a third operation, and a video stream in a fourth operation;
Including,
Transmitting the generated first character video stream to the content viewing terminals,
Each of the generated one or more first character video streams performing the second operation, the video stream performing the third operation, and each of the video streams performing the fourth operation are the response operation and the operation other than the response operation And transmitting each of the content viewing terminals that have transmitted the viewing video stream in which the non-response was recognized.
Including,
The video stream performing the second operation,
When the response operation is recognized, the first character of the first character image stream is an image stream performing a second operation,
The video stream performing the third operation,
When an operation other than the response operation is recognized, the first character of the first character image stream is an image stream that performs a third operation,
The video stream performing the fourth operation,
When the non-response is recognized, the first character of the first character video stream is a video stream performing a fourth operation,
Real-time character streaming content processing method.
According to claim 1,
Establishing content chat rooms in each of the content viewing terminals;
Receiving a chat input from the content viewing terminals and displaying it through the content chat room
Further comprising,
Generating the first character video stream,
Recognizing a preset operation of a subject of the captured video stream;
Adding a video effect in which the first character performs a second operation based on the first keyword included in the chat input-the first keyword is a keyword corresponding to a preset operation of a subject of the captured video stream Im-; And
Adding a video effect in which the first character performs a third operation based on the second keyword included in the chat input-the second keyword does not correspond to a preset operation of the subject of the captured video stream Keywords-;
Containing,
Real-time character streaming content processing method.
delete According to claim 1,
Establishing content chat rooms in each of the content viewing terminals;
Receiving a chat input from the content viewing terminals and displaying it through the content chat room
Further comprising,
Generating the first character video stream,
When the ratio of the number of the content viewing terminals inputting the first keyword through the chat input and the total number of content viewing terminals accessed through the content viewing channel is equal to or greater than a predetermined ratio, the first character performs the second operation. Steps to add video effects to perform
Containing,
Real-time character streaming content processing method.
According to claim 1,
Collecting each access time of the content viewing terminals connected through the content viewing channel
Further comprising,
Generating the first character video stream,
Adding a video effect in which the first character performs a second operation based on each access time of the content viewing terminals
Containing,
Real-time character streaming content processing method.
According to claim 1,
Generating the first character video stream,
Summing the costs equally paid by the content viewing terminals; And
Adding a video effect in which the first character performs a second operation based on the summed costs.
Containing,
Real-time character streaming content processing method.
According to claim 1,
The content viewers include a first content viewer group and a second content viewer group,
Receiving a second character selection input through content viewing terminals of the first content viewer group;
Receiving a third character selection input through content viewing terminals of the second content viewer group;
Further comprising,
Generating the first character video stream,
And generating a first character video stream including the second character and the third character.
Real-time character streaming content processing method.
The method of claim 10,
Generating a first character video stream including the second character and the third character,
Recognizing a preset operation of a subject of the captured video stream;
Receiving a viewing video stream from viewing terminals of each of the first content viewer group and the second content viewer group;
Recognizing a first response operation from the received viewing video stream of the first group of content viewers, wherein the first response operation is an operation performed in response to the preset operation;
Recognizing a second response operation from the received video stream of the second group of content viewers, wherein the second response operation is a response operation that does not correspond to the preset operation;
When the ratio of the total number of content viewing terminals of the first content viewer group and the total number of content viewing terminals of the first content viewer group in which the first response action is recognized is greater than or equal to a first predetermined rate, the second character Adding a video effect to perform the first operation; And
If the ratio of the total number of content viewing terminals of the second content viewer group and the total number of content viewing terminals of the second content viewer group in which the second response operation is recognized is greater than or equal to a second predetermined rate, the third character Adding a video effect to perform the first operation
Containing,
Real-time character streaming content processing method.
The method of claim 10,
Establishing first content chat rooms in each of the content viewing terminals of the first content viewer group;
Establishing second content chat rooms in each of the content viewing terminals of the second content viewer group;
Receiving a first chat input from the content viewing terminal of the first content viewer group and displaying it through the first content chat room; And
Receiving a second chat input from the content viewing terminal of the second content viewer group and displaying it through the second content chat room
Further comprising,
Generating a first character video stream including the second character and the third character,
When the ratio of the total number of the content viewing terminals and the content viewing terminals of the first content viewer group that input the first keyword included in the first chat input is greater than or equal to the first predetermined rate, the second character Adding a video effect to perform the first operation; And
If the ratio of the total number of content viewing terminals and the total number of content viewing terminals of the second content viewer group inputting the second keyword included in the second chat input is greater than or equal to a second predetermined rate, the third character Adding a video effect to perform the first operation
Containing,
Real-time character streaming content processing method.
According to claim 1,
The step of receiving the first character selection input,
Receiving an input for selecting the first character from a user-made character, a licensed character, and a default character; And
Paying a cost to a character creator corresponding to the first character
Containing,
Real-time character streaming content processing method.
A computer program stored in a medium for executing the method of any one of claims 1 to 5, 7 to 13 in combination with hardware.
Create a content providing channel and a content viewing channel for providing real-time character streaming content;
Recognize connections through the content providing channel from the content providing terminal of the content provider;
Receiving a first character selection input for selecting a first character from the content providing terminal;
Recognize connections through the content viewing channel from content viewing terminals of content viewers;
Receiving a video stream of the content provider from the content providing terminal;
Generating a first character video stream in which the subject of the captured video stream is replaced with the first character based on the expression, motion, and sound information of the subject of the captured video stream received from the content provider; And
Transmitting the generated first character video stream to the content viewing terminals;
Generating the first character video stream,
Recognize a preset operation of a subject of the captured video stream;
Receiving a viewing video stream from the content viewing terminals;
Recognize a response operation from the received viewing video stream-the response operation is an operation performed in response to the preset operation-; And
When the ratio of the number of the content viewing terminals recognized by the response operation and the total number of content viewing terminals accessed through the content viewing channel is equal to or greater than a predetermined ratio, the first character performs a video effect of performing the first operation. To add
A real-time character streaming content processing device comprising a processor.

Images