JP2019091450A - Method and system for providing real-time feedback information related to content of user input - Google Patents

Abstract

A method and system for providing real-time feedback information associated with user-input content. SOLUTION: Content input from a user terminal is sensed, content input is analyzed based on the result of machine learning based on artificial intelligence, and feedback associated with the content input is analyzed based on the analysis result. A method of generating and providing information to a user terminal is provided. [Selection diagram]

Description

  The present embodiment relates to a method and system for providing feedback information associated with content, and more particularly, to sensing content input from a user terminal, and inputting feedback information associated with the sensed content. Methods and systems provided in real time by

  A user who has viewed or experienced the content or service concerned with a movie or other content or service may evaluate the content or service he / she used by a comment (or short sentence) and a score. Such evaluation by the user also serves as a clue when deciding whether the other user uses the corresponding content or service by disclosing them to the other user.

  In order for the user to input comments and scores for such content or service, it was first required that the user input comments from his / her user terminal and additional scores. That is, in order to evaluate the content or service, the user is required to additionally input a score that the user wants to give to the content or service, in addition to inputting specific evaluation content as text. The

  In order to input a score for a movie, two user inputs, short text input and score input, have been required since a star rating must be added after the comment is first input.

  Patent Document 1 (Publication Date December 27, 2016) relates to a keyword-based review service system, and the user is a web / application service, a game, a book, a movie, music, a person, lessons, a food, a gourmet store, a hotel, a pension Based on the experience of cars, home appliances, etc., create a review for a specific keyword on the web or app, thereby enabling to provide clues in information acquisition and product selection of other users. Describes the review service system.

  The above-mentioned information is only for the purpose of understanding, and may include contents that do not form a part of the prior art, or that the prior art does not include matters that can be presented to ordinary technicians. is there.

Korean Registered Patent No. 10-1690055

  In one embodiment, content input from a user terminal is analyzed in real time based on results of machine learning performed using data stored in a database, and feedback associated with the input content according to the analysis result. Provided are methods and systems for providing information to a user terminal in real time.

  One embodiment receives the text input from the user terminal in a predetermined unit (for example, a letter unit), analyzes it in real time based on the result of machine learning, and changes feedback information according to the result of the real time analysis The present invention provides a method and system for providing

  In one embodiment, based on a result of machine learning of a comment and a score from a user on a predetermined content / service stored in a database, a score associated with a comment on the predetermined content / service input from the user terminal is automatically made. Provided is a method and system for calculating and providing.

  In one aspect, sensing an input of first content from a user terminal, based on a result of machine learning performed using the second content stored in the database and feedback information associated with the second content. Analyzing the first content in real time, generating feedback information associated with the first content based on a result of the analysis, and providing the generated feedback information to the user terminal An information provision method is provided.

  The second content and the feedback information associated with the second content may be a comment from a user for a predetermined content or a predetermined service and a score associated with the comment, which are stored in advance in the database.

  The first content and feedback information associated with the first content may be a comment from the user of the user terminal for the predetermined content or a predetermined service, and a score associated with the comment, respectively.

  The first content may include text input from the user terminal.

  The analyzing may include receiving the text input from the user terminal in a predetermined unit, and analyzing the text received in the predetermined unit in real time based on the result of the machine learning. Good.

  The generated feedback information may change according to the result of the real time analysis.

  The analyzing in real time may include determining whether the text includes input restriction information or how much the input restriction information is included.

  The generated feedback information may include information indicating whether the input restriction information is included or how much the input restriction information is included.

  The information indicating whether the input restriction information is included or how much the input restriction information is included is information indicating that registration or upload of the text is restricted, and the text includes the input restriction information, or It may include at least one of visually varying indicators depending on how much the input restriction information is included.

  The analyzing in real time may comprise determining the character of the text.

  The generated feedback information may include an indicator that changes visually according to the determined nature of the text.

  The analyzing in real time may include determining a category of the text.

  The generated feedback information may include information indicating the determined category of the text.

  The analyzing in real time may include determining a group of users who are predicted to be interested in the text.

  The generated feedback information may include information indicating the determined group of users.

  The feedback information may be a score displayed in a predetermined form, a visual change on the first content to be input, a visual change on a background of the first content to be input, according to a result of the analysis. A visually changing slide bar, a visually changing instrument needle, a user interface associated with the first visually changing content, a visually changing emoticon, and a visually changing icon And at least one of

  The first content may be a message for transmitting from the user terminal to another user terminal.

  The feedback information includes information enabling the user of the other user terminal that has received the message to recognize the nature or at least a part of the content of the message without browsing the entire content of the message. Good.

  The machine learning may be performed using data further including noise, or performed using data from which a part of the second content and feedback information associated with the second content is deleted. It may be done.

  The machine learning may include, for each of the comments, preprocessing including separating each comment by Unicode parse and indexing each separated element of each comment.

  The machine learning is a drop chunk that executes learning by deleting data of the chunk and processing it as if there is no data for a chunk consisting of at least one item included in a sequence constituting the comment The method may include using at least one of a (Drop Chunk) normalization algorithm and a Permute Chunk normalization algorithm that randomly arranges items in the chunk.

  The method for providing feedback information may further include receiving correction information from the user terminal with respect to feedback information provided to the user terminal.

  The feedback information reflecting the correction information and the first content input from the user terminal may be used in performing the machine learning.

  In another aspect, a content input sensing unit that senses an input of first content from a user terminal, a second content stored in a database, and a machine learning performed using feedback information associated with the second content. A content analysis unit analyzing the first content in real time based on a result, a feedback information generation unit generating feedback information associated with the first content based on the analysis result, and the generated feedback information There is provided a feedback information providing system including a feedback information providing unit provided to a user terminal.

  The second content and the feedback information associated with the second content may be a comment from a user for a predetermined content or a predetermined service and a score associated with the comment, which are stored in advance in the database.

  The first content and feedback information associated with the first content may be a comment from the user of the user terminal for the predetermined content or a predetermined service, and a score associated with the comment, respectively.

  The first content may include text input from the user terminal.

  The content analysis unit may receive the text input from the user terminal in a predetermined unit, and analyze the text received in the predetermined unit in real time based on the result of the machine learning.

  The feedback information generation unit may generate feedback information that changes according to the result of the real-time analysis.

  By automatically calculating and providing a score associated with a comment for a predetermined content / service input from the user terminal, the user does not have to add a score after the comment is input, thereby, User input can be reduced in rating for content / service.

  By automatically changing and providing feedback information associated with the content in accordance with the content of the content input from the user terminal, it is possible to enhance the user's interest in the input and production of the content.

  Input by which the registration of the corresponding content by the user or uploading is restricted or the registration or uploading is restricted by the feedback information provided in real time to the user terminal according to the content included in the content input from the user terminal The user can be notified in real time that the restriction information is included.

FIG. 7 is a diagram illustrating a method of providing feedback information associated with first content according to first content input from a user terminal, according to an embodiment of the present invention. It is a figure showing structure of a feedback information providing system and a user terminal in one embodiment. 5 is a flowchart illustrating a method of providing feedback information associated with first content according to first content input from a user terminal, according to an embodiment; 5 is a flowchart illustrating a method of analyzing first content including text input from a user terminal and generating feedback information associated with the first content, according to an embodiment; FIG. 7 is a diagram illustrating an example of analyzing first content including text input from a user terminal, generating feedback information associated with the first content, and providing the feedback information to the user terminal. FIG. 7 is a diagram illustrating an example of analyzing first content including text input from a user terminal, generating feedback information associated with the first content, and providing the feedback information to the user terminal. FIG. 7 is a diagram illustrating an example of analyzing first content including text input from a user terminal, generating feedback information associated with the first content, and providing the feedback information to the user terminal. FIG. 7 is a diagram illustrating an example of analyzing first content including text input from a user terminal, generating feedback information associated with the first content, and providing the feedback information to the user terminal. FIG. 7 is a diagram illustrating an example in which a message transmitted from a user terminal to another user terminal includes feedback information according to an embodiment. FIG. 7 is a diagram illustrating a data set used in machine learning for analyzing first content input from a user terminal and a preprocessing process according to an embodiment. FIG. 10 is a diagram illustrating a methodology used in machine learning to analyze first content input from a user terminal, according to an embodiment. It is the figure which showed the performance evaluation of the machine learning by the applied methodology.

  Hereinafter, embodiments will be described in detail with reference to the attached drawings. The same reference numerals as shown in the figures indicate the same elements.

  FIG. 1 is a diagram illustrating a method of providing feedback information associated with a first content according to the first content input from a user terminal in an embodiment.

  In FIG. 1, the feedback information providing system 100 detects the input of the first content by the user terminal 110 with respect to the first content input by the user terminal 110, analyzes the first content input in real time, and A method for providing feedback information associated with content to the user terminal 110 will be described.

  The user terminal 110 may be an electronic device for inputting the first content and registering the first content in the feedback information providing system 100 or another content registration server (not shown).

  The first content may include text as content (user-input content) input by the user of the user terminal 110. For example, the first content may be for evaluating a specific content or service, such as a movie, and may include a comment for evaluating the specific content or service.

  The feedback information providing system 100 may be an apparatus for sensing an input of the first content by the user terminal 110 and analyzing the input first content in real time to generate feedback information associated with the first content. The feedback information providing system 100 includes first content based on a second content stored in the content and feedback information database 120 (hereinafter referred to as a database) and a result of machine learning based on feedback information associated with the second content. May be generated to generate feedback information associated with the first content. The feedback information providing system 100 may generate feedback information that changes according to real-time analysis by analyzing the first content input from the user terminal 110 in real time. The feedback information providing system 100 may provide the generated feedback information to the user terminal 110. The feedback information associated with the first content may include, for example, a score for evaluating a particular content or service.

  The database 120 may store the content previously input from the user and the feedback information associated therewith as the second content and the feedback information associated with the second content, respectively. For example, database 120 may store a user's comment on a particular content or service and a score associated with that comment.

  The embodiments will be described in more detail with reference to the examples shown in the figures. First, it is assumed that the user inputs a comment such as "somewhat interesting ..." for a specific movie while using the user terminal 110. The feedback information providing system 100 may sense the input of the comment by the user terminal 110 and may analyze the input comment in real time. The feedback information providing system 100 is a user terminal based on a result of machine learning performed using a user's comment for a specific movie (or the entire movie category) previously stored in the database 120 and a score associated with the corresponding comment. The comments entered by 110 may be analyzed. The feedback information providing system 100 may generate the score 130 as feedback information corresponding to the comment according to the result of analyzing the comment, and may transmit the generated score 130 to the user terminal 110. The user terminal 110 may output the communicated scores 130, for example, in the form of a star rating shown in the figure.

  Therefore, the user does not need to separately input the grade 130 associated with the comment input from the user terminal 110, and is automatically calculated and provided to the user terminal 110. In addition, since the feedback information providing system 100 analyzes the input comment in real time to generate the score 130, the score 130 changes in response to the comment input from the user terminal 110. In other words, the score 130 is adaptively changed and calculated according to the content of the input comment and is output to the user terminal 110. For example, the relationship between the comment included in the existing data stored in the database 120 and the score related thereto is machine-learned based on deep learning, and the result of the machine learning is used to newly generate the user terminal 110. When the comment is input, the comment associated with the comment is calculated and provided to the user terminal in real time as each character of the comment is input.

  More detailed configurations of the feedback information providing system 100 and the user terminal 110 and more specific examples of the first content and the feedback information will be described in more detail with reference to FIGS.

  FIG. 2 is a diagram showing the structure of a feedback information providing system and a user terminal in an embodiment.

  The more detailed configurations of the feedback information providing system 100 and the user terminal 110 described above will be described with reference to FIG.

  The feedback information providing system 100 may be a server or other computer device that analyzes the first content input by the user terminal 110 and generates and provides feedback information associated with the first content. For example, the feedback information providing system 100 may be a server that manages registration of the first content input from the user terminal 110 and associates and stores the first content and feedback information. However, registration and management of the first content may be performed by another external server that is not the feedback information providing system 100.

  The feedback information providing system 100 may include a communication unit 210 and a control unit 220.

  The communication unit 210 may be a device for the feedback information providing system 100 to communicate with another server or the user terminal 110. In other words, the communication unit 210 transmits / receives data and / or information to another server or user terminal 110, hardware such as a network interface card, a network interface chip, and a networking interface port of the feedback information providing system 100. It may be a module or a software module such as a network device driver or a networking program.

  The control unit 220 may manage components of the feedback information providing system 100, and may execute a program or an application used by the feedback information providing system 100. For example, the control unit 220 is used to analyze the first content received by the communication unit 210 based on the result of machine learning, and generate feedback information associated with the first content according to the analysis result. The program or application may be executed, and the operations necessary for execution of the program or application and processing of data may be processed. The controller 220 may be at least one processor of the feedback information providing system 100 or at least one core in the processors.

  The control unit 220 may include a content input detection unit 222, a content analysis unit 224, a feedback information generation unit 226, and a feedback information provision unit 228. Each of the configurations 222-228 of the controller 220 may be implemented with one or more software modules and / or hardware modules.

  The content input sensing unit 222 may sense an input of the first content from the user terminal 110. "Sensing" may be used synonymously with sensing or detection depending on context.

  The content analysis unit 224 uses, in real time, the first content input from the user terminal 110 based on the result of machine learning performed using the second content stored in the database 120 and the feedback information associated with the second content. You may analyze it.

  The machine learning performed using the data stored in the database 120 may be performed by the content analysis unit 224 or may be performed by another configuration of the feedback information providing system 100. Alternatively, the machine learning may be performed by a server / device other than the feedback information providing system 100, and the content analysis unit 224 may utilize the result of the corresponding machine learning for the first content analysis. The machine learning method may include, for example, a deep learning method, but any artificial intelligence based machine learning method may be used as the machine learning of the embodiment. A specific machine learning method and its pre-processing process will be described in more detail with reference to FIGS.

  The feedback information generation unit 226 may generate feedback information associated with the first content based on the analysis result by the content analysis unit 224. The feedback information generation unit 226 may generate feedback information associated with the first content in real time by real time analysis of the first content by the content analysis unit 224. In other words, feedback information associated with the first content changes according to the content of the first content input in real time.

  The feedback information provision unit 228 may provide the generated feedback information to the user terminal 110 via the communication unit 210.

  More specific functions and operations of the configurations 222 to 228 of the control unit 220 will be described in more detail with reference to FIGS. 3 to 9 described later.

  Because the configurations 222-228 of the control unit 220 described above may be implemented in at least one processor, the functions and operations of the configurations 222-228 may be performed by the at least one processor.

  The user terminal 110 inputs the first content, receives feedback information from the feedback information providing system 100, and registers the first content in the feedback information providing system 100 or another content registration server (not shown). It may be an apparatus. The user terminal 110 may output the input / registered first content and feedback information received from the feedback information providing system 100. For example, the user terminal 110 may be a smartphone, a personal computer (PC), a laptop computer, a laptop computer, a tablet, an Internet of Things (IoT) device, or a wearable computer (wearable computer). Etc.) may be a terminal used by the user.

  The user terminal 110 may include a communication unit 230 and a control unit 240. The communication unit 230 may be a device for the user terminal 110 to communicate with the feedback information providing system 100 or another user terminal. In other words, the communication unit 230 transmits / receives data and / or information to / from the feedback information providing system 100 or another user terminal, hardware such as a network interface card of the user terminal 110, a network interface chip, and a networking interface port. Or a software module such as a network device driver or a networking program.

  The control unit 240 may manage the components of the user terminal 110 and may execute a program or an application used by the user terminal 110. For example, the control unit 240 may execute a program or application required to input and register the first content, and may process operations required for execution of the program or application, processing of data, and the like. The control unit 240 may be at least one processor of the user terminal 110 or at least one core in the processors.

  Although not shown in the figure, the user terminal 110 may include a display unit for outputting the input first content and feedback information received from the feedback information providing system 100. The display unit may include, for example, a touch screen, and in this case, the display unit may be configured to include a function of an input unit for receiving an input of the first content from the user.

  Also, although not shown, the user terminal 110 may include a storage as an apparatus for storing data or information. The storage may include any memory or storage device. The storage unit may store a program or an application executed by the control unit 240 and information related thereto.

  Database 120 may store data used to perform the machine learning described above. The database 120 may include a second content and feedback information associated with the second content that are utilized for analysis of the first content and generation of feedback information associated with the first content. For example, the second content and the feedback information associated with the second content may be big data used to analyze the first content and generate feedback information associated with the first content.

  The manner in which the first content is analyzed and feedback information is generated and provided by the interaction between the user terminal 110, the feedback information providing system 100, and the database 120 will be described in more detail with reference to FIGS. explain.

  As described above, the description of the technical features described above with reference to FIG. 1 may be applied as it is to FIG.

  FIG. 3 is a flowchart illustrating a method of providing feedback information associated with a first content according to the first content input from a user terminal in an embodiment.

  In operation 310, the content input sensing unit 222 may sense an input of the first content from the user terminal 110. The first content may include text input from the user terminal 110, and the content input sensing unit 222 may sense input of the first content by sensing text input. For example, the content input sensing unit 222 may sense the input of the first content by determining whether the user terminal 100 has created a comment for evaluating a specific content or service.

  In operation 320, the content analysis unit 224 receives the first content from the user terminal 110 based on the result of machine learning performed using the second content stored in the database 120 and the feedback information associated with the second content. Content may be analyzed in real time. The second content and the feedback information associated with the second content may be, respectively, comments stored in advance in the database 120 for a specific (or predetermined) content or service and a score associated with the comment.

  In operation 330, the feedback information generator 226 may generate feedback information associated with the first content based on the analysis result in operation 320. The first content input from the user terminal 110 and the feedback information generated for provision to the user terminal 110 are, respectively, a comment from the user of the user terminal 110 for a predetermined content or service and a score associated with the corresponding comment. May correspond to

  The predetermined content or service may include content of a specific category and individual content belonging to the specific category. For example, if the user of the user terminal 110 inputs a comment for "movie A" when the predetermined content or service is "movie", the content analysis unit 224 stores the entire movie stored in the database 120 (or The user's comment on "Movie A" may be analyzed based on the comment on "Movie A)" and the result of machine learning performed using the score associated with the comment. Based on this analysis, the feedback information generator 226 may calculate a score associated with the comment input by the user.

  In operation 340, the feedback information provider 228 may provide the generated feedback information to the user terminal 110. The user terminal 110 may output the received feedback information together with the input first content. For example, the comment for “movie A” input from the user terminal 110 and the score for “movie A” provided from the feedback information provision unit 228 may both be output to the user terminal 110.

  The feedback information generation unit 226 may generate feedback information in real time by real time analysis of the first content by the content analysis unit 224, and the feedback information provision unit 228 provides the generated feedback information to the user terminal 110 in real time. You may Therefore, the feedback information output to the user terminal 110 changes in real time according to the input of the first content.

  As used herein, the term "real-time" is used in the sense of encompassing all substantially the same time (taking into consideration the analysis and processing time of the data), not just the actual time. is there.

  In operation 350, the communication unit 210 may receive correction information from the user terminal 110 in response to feedback information provided to the user terminal 110. The correction information may be information indicating correction of the feedback information provided to the user terminal 110 by the user of the user terminal 110. For example, the user may change the score if the score provided from the feedback information provision unit 228 does not meet his intention, and the correction information due to the change of the score may be transmitted from the user terminal 110 to the feedback information providing system 100. May be transmitted to

  The feedback information providing system 100 may use the feedback information in which the received correction information is reflected and the first content input by the user terminal 110 as data for performing the above-described machine learning. By such a process, provision of feedback information by feedback information providing system 100 by the subsequent first content input becomes more appropriate (that is, conforms to the intention of the user who has input the first content). . It goes without saying that the feedback information provided to the user terminal 110 and the first content input by the user terminal 110 are used as data for executing machine learning, regardless of the reception of the correction information.

  The real-time analysis method of the first content of step 320 described above will be described in more detail. Stage 320 may include stages 322 and 324 described below.

  In operation 322, the content analysis unit 224 may receive the text included in the first content input from the user terminal 110 through the communication unit 210 in predetermined units. The predetermined unit may be, for example, a character or sub-character unit. Letters may mean consonants, vowels, or special characters. Alternatively, the characters may mean any characters that can correspond to the ASCII code.

  At step 324, the content analysis unit 224 may analyze the text received in predetermined units in real time based on the result of machine learning. Thus, the feedback information associated with the text generated by the feedback information generation unit 226 changes in accordance with the result of the real time analysis by analyzing the text received in a predetermined unit in real time based on the result of machine learning It will be.

  For example, when a comment for a particular movie is input as the first content at the user terminal 110, a score may be calculated as feedback information in real time and provided to the user terminal 110 each time each character of the comment is input.

  More various types of feedback information associated with the first content will be described in more detail with reference to FIGS.

  As described above, the description of the technical features described above with reference to FIGS. 1 and 2 may be applied to FIG. 3 as it is, and thus redundant description will be omitted.

  FIG. 4 is a flowchart illustrating a method of analyzing first content including text input from a user terminal and generating feedback information associated with the first content, according to an embodiment.

  A method for analyzing text included in the first content by the content analysis unit 224 described above in real time based on the result of machine learning and generating feedback information associated with the text will be described in more detail with reference to FIG. 4. .

  In operation 410-1, the content analysis unit 224 may determine whether the text input from the user terminal 110 includes input restriction information or how much input restriction information is included. The input restriction information may be information whose input and / or registration is restricted when creating content including text. For example, the input restriction information may be a word or expression that includes or indicates a rant or a word or expression that includes or indicates sexual content.

  The content analysis unit 224 may determine whether the text input from the user terminal 110 includes the input restriction information based on the result of machine learning, and if the input restriction information is included, the input restriction information may be text You may decide how much it is included.

  The feedback information generation unit 226 may generate feedback information including information indicating whether the input restriction information is included or how much the input restriction information is included according to the determination of step 410-1.

  For example, the information indicating whether the input restriction information is included or how much the input restriction information is included may be information indicating that the registration or uploading of the text is restricted. And / or additionally, the information indicating whether the input restriction information is included or how much the input restriction information is included is visually changed according to whether the text includes the input restriction information or how much the input restriction information is included Indicator may be included.

  Specific examples of information and indicators indicating whether to include input restriction information or how much input restriction information is included will be described in more detail with reference to FIG. 5 described later.

  At step 410-2, the content analysis unit 224 may determine the nature of the text input from the user terminal 110. The nature of the text may be determined by how much the text contains positive or negative content. Alternatively, the content analysis unit 224 may determine the character of the text by determining whether the general content of the text is positive or negative. For example, malicious comments or text indicating negative content to a particular content or service (eg, a movie is not interesting) may be judged as text of negative character, high quality comments or specific text Text that indicates content that is positive for content or service (eg, a movie is interesting, etc.) may be determined as text of positive character.

  The feedback information generator 226 may generate feedback information including an indicator that changes visually according to the character of the text determined in step 410-2.

  An example in which visually changing feedback information is provided according to the character of the text will be described in more detail with reference to FIG. 6 described later.

  At step 410-3, the content analysis unit 224 may determine the category of the text input from the user terminal 110. The category of text may be determined by what purpose the text was created. For example, the category of text may be one of movie review, product review, message, news comment, SNS post, but is not limited thereto. At this time, the result of machine learning used by the content analysis unit 224 to analyze the text is a result of machine learning using data corresponding to movie reviews, product reviews, messages, news comments, or SNS posts. Good. In other words, the database 120 described above may include a database that stores data as described above.

  The feedback information generator 226 may generate feedback information including information indicating the category of the text determined in step 410-3.

  An example of providing feedback information including information indicating a category of text will be described in more detail with reference to FIG. 7 described later.

  At step 410-4, the content analysis unit 224 may determine a group of users who are predicted to be interested in the text input from the user terminal 110. The content analysis unit 224 may determine a group of users who are predicted to be interested in the corresponding content by analyzing the content included in the text based on the result of machine learning. For example, in the case where the content related to high school life is included in the text, the content analysis unit 224 may determine that the group of users who are interested in the text is “tenant student”.

  The feedback information generator 226 may generate feedback information including information indicating the group of users determined in step 410-4. In other words, a group of users expected to be interested in the text input from the user terminal 110 may be analyzed in real time and provided to the user terminal 110.

  An example of providing feedback information including information indicating a group of users who are predicted to be interested in the text will be described in more detail with reference to FIG. 8 described later.

  The embodiments described above with reference to steps 410-1 to 410-4 may be combined with one another.

  On the other hand, the feedback information may be a score displayed in a predetermined format based on the analysis result of the first content by the content analysis unit 224, a visual change on the input first content, and a background of the input first content. Visual changes, a visually changing slide bar, a visually changing meter needle, a user interface associated with a visually changing first content, a visually changing emoticon, and a visually changing It may include at least one of the changing icons, but is not limited thereto, and may include any user interface or image or other content.

  As described above, the description of the technical features described above with reference to FIGS. 1 to 3 may be applied as it is to FIG.

  5 to 8 illustrate an example of analyzing first content including text input from a user terminal, generating feedback information associated with the first content, and providing the feedback information to the user terminal.

  Specific embodiments of the above described steps 410-1 to 410-4 will be described in more detail with reference to FIGS.

  FIG. 5 shows an example where the text includes "rant (dog *! @)" As input restriction information. When a comment (rant 510) is input from the user terminal 110, the content analysis unit 224 may determine that the text includes the default 510.

  The feedback information generation unit 226 may generate feedback information that causes the background color of the text (that is, the background color of the first content) to be changed when the text includes the rant 510. Also, the feedback information generation unit 226 may generate feedback information that causes the background color of the text to be darker (for example, more red) as the text includes more violent words 510.

  Alternatively, the feedback information generation unit 226 may generate feedback information including information indicating that the corresponding text can not be registered when the text includes the spoofing message 510. For example, a message such as "This comment can not be registered" may be output to the user terminal 110, and such a message indicates that the text contains more than a certain level of abuse (or a text background) May be output).

  In addition, when the text includes a sloppy 510, the feedback information generation unit 226 may generate feedback information for performing a blow process on a portion corresponding to the sloppy text 510 or processing a special character.

  The feedback information providing system 100 may apply similar techniques to the content described above to limit the content / expression as well as sexual content / expression or other inappropriate content / expression. The explanation is omitted.

  FIG. 6 illustrates an example in which the character of the text is determined, and feedback information that is visually changed according to the character of the text is provided.

  i) The content analysis unit 224 may determine the text including the positive content as the text 610 as the text of the positive personality, but at this time, the feedback information generation unit 226 may be configured to display an icon or a face including smile. Feedback information may be generated that includes characters 620. ii) The content analysis unit 224 may determine that the text including negative content as the text 630 is a text of positive personality, but at this time, the feedback information generation unit 226 determines whether the text including the frown is an icon or Feedback information including emoticons 640 may be generated.

  In addition, when the content analysis unit 224 determines that the content of the input text gradually changes from positive content to negative content, the feedback information generation unit 226 generates an expression of an icon or emoticon. The feedback information may be generated including information to gradually change the face from smile to frown.

  FIG. 7 illustrates an example in which a category of text is determined, and feedback information including information indicating the category is provided.

  The content analysis unit 224 may determine which category the text belongs to according to the content of the input text. For example, the content analysis unit 224 may determine (based on the result of machine learning) that the example text ("Movies are ...") shown in the figure to be related to a movie review; May generate feedback information 710 including information about the movie review category ("<movie score>"). The content analysis unit 224 may determine the category of the text by analyzing the content of the input text in real time, and thus included in the feedback information generated and provided to the user terminal 110 according to the result of such determination. The information indicating the category to be changed will be changed.

  FIG. 8 illustrates an example in which a user group predicted to be interested in text is determined, and feedback information including information indicating the corresponding user group is provided.

  The content analysis unit 224 may determine (determine) a group of users who are predicted to be interested in the text according to the content of the input text. For example, the content analysis unit 224 may determine (based on the result of machine learning) that the text in the example shown in the figure (“the recent meal is ...”) relates to school life. Users who are interested in such text may be determined to be teenagers. Based on such a determination, the feedback information generation unit 226 may generate feedback information 810 including a message “It is a teenager who likes this sentence”. The content analysis unit 224 may determine the group of users who are predicted to be interested in the text by analyzing the content of the input text in real time, and thus, the generation and the user terminal 110 according to the result of such determination. The information indicating the interested user group included in the feedback information provided to the will be changed.

  As described above, the description of the technical features described above with reference to FIGS. 1 to 4 may be applied as it is to FIGS.

  FIG. 9 illustrates an example in which a message transmitted from a user terminal to another user terminal includes feedback information in one embodiment.

  The first content described with reference to FIGS. 1 to 8 may be a message for transmitting from the user terminal 110 to another user terminal. At this time, feedback information associated with the first content is information that enables the user of another user terminal that has received the message to recognize the nature or at least a part of the message without browsing the entire contents of the message. May be included. The message may be, for example, an email, a message of a messenger app, an instant message, or an SMS.

  As an example, as in the example shown in the figure, when a message such as “It seems that there is a problem with the product” is input at the sender user terminal 910, the content analysis unit 224 determines that the corresponding message is a claim. As it is applicable, the character of the text included in the message may be determined as negative. At this time, a message in which the background 940 color of the message has changed to red may be output to the receiver-side user terminal 920 as feedback information. Alternatively or additionally, the receiver user terminal 920 may output a message including a frown icon or an emoticon 930 as feedback information.

  By identifying the background 940 and / or the frowning icon or emoticon 930, the user of the recipient user terminal 920 can recognize that the corresponding message contains negative content, the entire content of the message It becomes possible to grasp even without browsing.

  In addition to the content shown in the figure, feedback information may be provided by symbolizing the content of the message and associating it with the title and background of the message.

  The feedback information may also be displayed on the originator terminal 910 that creates the message, so the author may consider how the message is output at the recipient user terminal 920 before delivering the message. become able to.

  As described above, the description of the technical features described above with reference to FIGS. 1 to 8 may be applied as it is to FIG.

  In addition to the embodiment described above, depending on the type of feedback information or the type of first content to be input, the performance of the user terminal 110 or the feedback information system 100 depending on the type of service to which the first content is input. Depending on or any other need, the feedback information may not be output or provided in real time to the user terminal 110 and the input of the first content is completed or the creation of the first content is completed or It may be applied only when completed.

  FIG. 10 is a diagram illustrating a data set used in machine learning for analyzing first content input from a user terminal and a preprocessing process according to an embodiment.

  The example shown in the figure shows machine learning in a method of predicting movie scores from Korean comment reviews using a Sub-Character Level Bi-directional Long Short-Term Memory Model. ing.

  Data set 1010 may be a data set used in machine learning. The data set 1010 may be data stored in the database 120 described above. In the example shown, the data set 1010 may be about 5 million Korean movie reviews collected from a neighbor movie. Each of the reviews may include the user's comment on a particular movie and the grade entered by the user. Also, each of the reviews may include a score as feedback information generated by the feedback information providing system 100 in association with a user's comment on a particular movie and the user's comment.

  Machine learning may be performed using data further comprising noise to the data set 1010, the information contained in the data set 1010 (eg, the second content and the second content described above with reference to FIGS. Some information in the feedback information associated with the second content may be performed using the deleted data. Deletion of partial data and / or addition of noise to the data set 1010 may be intentionally performed to improve the result of machine learning.

  Each sequence included in each review included in data set 1010 may have real-valued scores in the range of [1, 10], and have a length of up to 30 characters Good. The input sentence may include characters corresponding to ASCII code and special characters. For example, the total number of parsed characters, including Korean sub-characters, may be assumed to be 255.

  As an example, the number of movie review data sets corresponding to the data set 1010 is 4,989,205, and training using random shuffle, validation, and test set are 8: 1: It may be divided at a rate of 1.

  In FIG. 10, preprocessing in machine learning using data set 1010 will be described in more detail.

  The numbers displayed to the left of each comment in the data set 1010 may be the score associated with the corresponding comment. Here, the comment (X) may be separated into sub-character units.

  Each sub-character making up the comment (X) may be converted to an index of any one of 1-255. Table 1030 shows an example of converting the data set 1010 into an index. Each row of table 1030 may indicate one comment. In a table 1030 shown in the figure, indexes corresponding to three comments are shown. The leftmost column of each row of the table may indicate the comment associated with the corresponding row and the grade associated with that comment, and may indicate the result of parsing the comment into sub-characters from the next column.

  The data indexed as in Table 1030 may be used to machine learn the scores associated with each comment included in data set 1010.

  In other words, for each comment included in the data set 1010, the machine learning described above with reference to the embodiment described above separates each comment by Unicode parsing, and indexes each separated element of each comment. It may include a pre-treatment process including

  As described above, in the embodiment and the example described above, when processing a comment (text) input from the user terminal 110 or a comment (text) stored in the database 120, the comment is analyzed and processed on a morpheme basis Such encoded / encoded data that can be encoded / encoded after decomposition into sub-word levels that make up the comments (ie without morpheme analysis) Machine learning and analysis.

  FIG. 11 shows a methodology used in machine learning for analyzing the first content input from a user terminal according to an embodiment, and FIG. 12 shows the performance of machine learning according to the applied methodology. Shows an evaluation.

  FIG. 11 is a stochastically segmented chunk from within the sequence included in each comment when learning comments and associated scores for a movie using the sub-character level BiLSTM network described above A regularization technique is described that generates and processes noise associated with (Chunk). The use of such normalization techniques results in machine learning of robust, real-time score prediction models.

  A chunk may be composed of at least one item included in a sequence that composes a comment. The item may be, for example, a character (subcharacter) included in the comment or an index associated therewith. A chunk may refer to a sentence, for example, a bundle of consecutive items of a certain size in a sequence included in the sentence. The size and position of the chunk may be randomly set for each sequence, and the position may be random, and the size may be random from 0 to N.

  An example of the above-described normalization technique may be a technique using a Drop Chunk Regularization algorithm (see (1) in FIG. 11).

  The drop chunk normalization algorithm may perform machine learning by deleting at least part of data included in the chunk and processing it as if the data were absent. The drop chunk normalization algorithm may not be applied to the chunk when the size of the chunk is 0. The drop chunk normalization algorithm is a conventional method that deletes the gradient that corresponds to the masked data by masking instead of deleting the data so that the data is read but machine learning is not performed. Compared to the machine learning performance is even better. In addition, the machine learning performance is further improved compared to zero-padding (a technique in which 0 is substituted for data and processing), which is a more naive method than the masking method. Do.

  Also, an example of the above-described normalization technique may be a technique using a Permute Chunk normalization algorithm (see (2) in FIG. 11).

  The Permute Chunk normalization algorithm may be an algorithm that randomly places items in a chunk. The size of the chunks may be random, 1 to N, as described above. If N is 1, there is no point in applying the Permute Chunk normalization algorithm, so the algorithm may not be applied.

  Chunks to which the drop chunk normalization algorithm and / or the permuted chunk normalization algorithm described above are applied may be used in machine learning.

  We will now describe how to perform machine learning using the Discretized Regression technique. As shown in the following equation (1), Argmax output (see equation (1) (2)) or Exptected output technique (see equation (1)) may be used.

  In the above equation, y is a score associated with the comment, and may be, for example, a constant value of 0 to 10 if the score is a particular movie grade. x may be a comment associated with the score. θ may be a parameter of a neural network. y 'may be the result value.

  FIG. 12 is a diagram showing the results of experiments applying the above-described normalization algorithm and machine learning technique.

  (1) shows the performance of machine learning when the drop chunk normalization algorithm is applied. (2) shows the comparison of the machine learning performance when applying the drop chunk normalization algorithm and the permuting chunk normalization algorithm and the machine learning performance when using both of them. (3) shows a comparison of the machine learning performance of the discontinuous and continuous regression techniques. In the graph shown in the figure, the horizontal axis may increase by one for each of ten learning iterations.

  When referring to (1) and (2), if the drop chunk normalization algorithm is applied at machine learning, the machine learning performance is improved (121, 121, 1213, 1221), and both permuted chunk normalization algorithms When applied, it can be confirmed that the machine learning performance is improved (1223).

  When referring to (3), it is possible to confirm that the performance of machine learning is further improved when the continuous regression technique is applied by comparing the cases where the continuous regression technique and the discontinuous regression technique are applied.

  The above-described apparatus may be realized by hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments are, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), and a microprocessor. Or may be implemented utilizing one or more general purpose or special purpose computers, such as various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the OS. The processing device may also respond to software execution, access data, and store, manipulate, process, and generate data. Although one processor may be described as being used for the convenience of understanding, it is understood by those skilled in the art that the processor may include a plurality of processing elements and / or a plurality of types of processing elements. You will understand. For example, the processing device may include multiple processors or one processor and one controller. Other processing arrangements are also possible, such as parallel processors.

  The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to operate as desired, instructing the processing unit to operate independently or collectively. You may The software and / or data may be of any type of machine, component, physical device, virtual device, computer storage medium or device, or may be interpreted based on the processing device or providing instructions or data to the processing device. It may be embodied permanently or temporarily in the transmitted signal wave. The software may be distributed on computer systems connected by a network, and may be stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

  The method according to the embodiments may be realized in the form of program instructions executable by various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for an embodiment, or may be usable by computer software skilled in the art. Examples of computer readable recording media include hard disks, floppy disks, and magnetic media such as magnetic tapes, CD-ROMs, optical media such as DVDs, and floppy disks. Such magneto-optical media and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory etc are included. Examples of program instructions include not only machine language code, such as that generated by a compiler, but also high level language code that is executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

  Although the embodiments have been described based on the limited embodiments and the drawings as described above, various modifications and variations can be made by those skilled in the art from the above description. For example, components described in the systems, structures, devices, circuits, and the like may be performed in a different order from that of the methods described and / or different from those described in the methods described. Appropriate results can also be achieved if combined or combined, or counter-located or replaced by other components or equivalents.

100: Feedback information providing system 110: User terminal 120: Content and feedback information DB
210: Communication unit 220: Control unit 222: Content input detection unit 224: Content analysis unit 226: Feedback information generation unit 228: Feedback information provision unit 230: Communication unit 240: Control unit

Claims (18)

Sensing an input of the first content from the user terminal,
Analyzing the first content in real time based on a second content stored in a database and a result of machine learning performed using feedback information associated with the second content;
A method of providing feedback information, comprising: generating feedback information associated with the first content based on a result of the analysis; and providing the generated feedback information to the user terminal. The second content and feedback information associated with the second content are a comment from a user for a predetermined content or a predetermined service and a score associated with the comment, which are stored in advance in the database, respectively.
The first content and feedback information associated with the first content are respectively a comment from the user of the user terminal for the predetermined content or a predetermined service and a score associated with the comment.
The feedback information provision method according to claim 1. The first content includes text input from the user terminal,
The analyzing step is
Receiving the text input from the user terminal in a predetermined unit, and analyzing the text received in the predetermined unit in real time based on the result of the machine learning.
The method for providing feedback information according to claim 1, wherein the generated feedback information changes according to the result of the analysis in real time. The analysis in real time is
Determining whether the text includes input restriction information or how much the input restriction information is included,
The feedback information provision method according to claim 3, wherein the generated feedback information includes information indicating whether the input restriction information is included or how much the input restriction information is included.   The information indicating whether the input restriction information is included or how much the input restriction information is included is information indicating that registration or upload of the text is restricted, and the text includes the input restriction information, or The method for providing feedback information according to claim 4, further comprising at least one of visually changing indicators depending on how much the input restriction information is included. The analysis in real time is
Determining the nature of the text,
The method for providing feedback information according to claim 3, wherein the generated feedback information includes an indicator visually changing according to the determined character of the text. The analysis in real time is
Determining the category of the text,
The method for providing feedback information according to claim 3, wherein the generated feedback information includes information indicating the determined category of the text. The analysis in real time is
Determining the group of users who are predicted to be interested in the text;
The method for providing feedback information according to claim 3, wherein the generated feedback information includes information indicating the determined group of users. The feedback information is based on the result of the analysis,
Score displayed in a predetermined form, visual change of the input first content, visual change of the background of the input first content, visually changing slide bar, visual change The device according to claim 1, comprising at least one of a needle of a measuring instrument, a user interface associated with the first content that changes visually, a visually changing emoticon, and a visually changing icon. How to provide feedback information. The first content is a message for transmitting from the user terminal to another user terminal,
The feedback information includes information that enables the user of the other user terminal that has received the message to recognize the nature or at least a part of the content of the message without browsing the entire content of the message. The feedback information provision method according to claim 1.   The machine learning may be performed using data further including noise, or may be performed using data from which the second content and part of feedback information associated with the second content have been deleted. The method for providing feedback information according to claim 1. The machine learning is
The method for providing feedback information according to claim 2, further comprising the steps of: separating each comment by Unicode parsing for each of the comments; and indexing the separated elements of each comment. The machine learning is performed on a chunk composed of at least one item included in a sequence that composes the comment.
Drop Chunk normalization algorithm which deletes data of the chunk and processes as if there is no data and executes learning, and Permute Chunk which randomly arranges items in the chunk (Permute Chunk) 3. The method for providing feedback information according to claim 2, comprising the step of using at least one of the normalization algorithms. And receiving, from the feedback information provided to the user terminal, correction information from the user terminal.
The feedback information providing method according to claim 1, wherein the feedback information reflecting the correction information and the first content input by the user terminal are used in performing the machine learning.   A computer program which causes a computer of a feedback information providing system to execute the feedback information providing method according to any one of claims 1 to 14. A content input sensing unit that senses an input of the first content from the user terminal;
A content analysis unit that analyzes the first content in real time based on a second content stored in a database and a result of machine learning performed using feedback information associated with the second content;
A feedback information providing system comprising: a feedback information generation unit configured to generate feedback information associated with the first content based on a result of the analysis; and a feedback information provision unit configured to provide the generated feedback information to the user terminal. The second content and feedback information associated with the second content are a comment from a user for a predetermined content or a predetermined service and a score associated with the comment, which are stored in advance in the database, respectively.
The first content and the feedback information associated with the first content may be a comment from a user of the user terminal for the predetermined content or a predetermined service and a score associated with the comment, respectively. Feedback information system. The first content includes text input from the user terminal,
The content analysis unit
Receiving the text input from the user terminal in a predetermined unit, and analyzing the text received in the predetermined unit in real time based on the result of the machine learning,
The feedback information providing system according to claim 16, wherein the feedback information generation unit generates feedback information that changes in accordance with a result of the analysis in real time.

Images