HK40035387B - Method and device for determining text label, terminal and readable storage medium - Google Patents

Description

Text tag determination method, device, terminal and readable storage medium

Technical Field

This application relates to the field of tag mining, and in particular to a method, apparatus, terminal and readable storage medium for determining text tags.

Background Technology

Tags are defined as the most important keywords that can represent content. In the process of information flow content distribution, whether it is text and images or videos, tag information is very important. Once content has tags, it can be organized and displayed according to different tags, and more accurate content recommendations can be achieved by matching tags with user profiles.

In related technologies, methods for extracting tags from article content include determining tags for the current content based on TF-IDF statistical features. This method tends to filter out common words in the article and retain important words.

However, this statistical method does not consider the relationships between words in the article or between words and the document. As a result, the obtained tags deviate from the actual semantics of the content, and the accuracy of the obtained tags is not high.

Summary of the Invention

This application provides a text tag determination method, apparatus, terminal, and readable storage medium, which can improve the accuracy of tag determination. The technical solution is as follows:

On the one hand, a method for determining text labels is provided, the method comprising:

The target text is segmented to obtain a segmentation set, which includes the segmented words obtained from the segmentation of the target text. The target text is the text whose label is to be determined.

Based on the contextual relationships of the segmented words, determine the first candidate tag for the target text;

Based on the first frequency parameter of the segmented words in the target text and the second frequency parameter of the segmented words in the text set, a second candidate tag for the target text is determined;

The tags of the target text are determined based on the first candidate tag and the second candidate tag.

On the other hand, a text label determining device is provided, the device comprising:

The processing module is used to perform word segmentation on the target text to obtain a word segmentation set, wherein the word segmentation set includes the word segments obtained from the word segmentation of the target text, and the target text is the text whose label is to be determined;

The determination module is used to determine the first candidate tag of the target text based on the contextual relationship of the segmented words;

The determining module is further configured to determine a second candidate tag for the target text based on a first frequency parameter of the segmented words in the target text and a second frequency parameter of the segmented words in the text set;

The determining module is further configured to determine the tag of the target text based on the first candidate tag and the second candidate tag.

On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the text label determination method as described in any of the embodiments of this application above.

On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the text label determination method as described in any of the embodiments of this application above.

On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the text label determination methods described in the above embodiments.

The beneficial effects of the technical solutions provided in this application include at least the following:

By combining the contextual relationships of each word segment in the target text with the first frequency parameter of each word segment in the target text and the second frequency parameter of each word segment in the text set, the label of the target text is determined from both deep semantic and shallow frequency perspectives, thus improving the accuracy of target text label determination.

Attached Figure Description

To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

Figure 1 is a schematic diagram of the implementation environment provided by an exemplary embodiment of this application;

Figure 2 is a flowchart of a text label determination method provided in an exemplary embodiment of this application;

Figure 3 is a flowchart of a text label determination method provided in another exemplary embodiment of this application;

Figure 4 is a text label determination model based on a bidirectional long short-term memory neural network provided in an exemplary embodiment of this application;

Figure 5 is a schematic diagram of an attention calculation mechanism provided in an exemplary embodiment of this application;

Figure 6 is a flowchart of attention calculation provided in an exemplary embodiment of this application;

Figure 7 is a flowchart of a method for determining labels using a conditional random field according to an exemplary embodiment of this application;

Figure 8 is a schematic diagram of the system flow of a text label determination method provided in an exemplary embodiment of this application;

Figure 9 is a schematic diagram of the system flow for label determination provided in an exemplary embodiment of this application;

Figure 10 is a structural block diagram of a text label determining device provided in an exemplary embodiment of this application;

Figure 11 is a structural block diagram of a text label determining device provided in another exemplary embodiment of this application;

Figure 12 is a structural block diagram of a server provided in an exemplary embodiment of this application.

Detailed Implementation

To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

First, a brief introduction to the terms used in the embodiments of this application:

Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating/interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning/deep learning.

Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and further processes images to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content/behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, and simultaneous localization and mapping (SLAM).

Key technologies in speech technology include Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the future direction of human-computer interaction, with speech emerging as one of the most promising methods.

Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

Word embeddings are a collective term for a set of language modeling and feature learning techniques in embedded natural language processing, where words or phrases from the vocabulary are mapped to vectors of real numbers. Conceptually, it involves mathematical embeddings from a one-dimensional space for each word to a continuous vector space with lower dimensions. Methods for generating such mappings include neural networks, dimensionality reduction of word co-occurrence matrices, probabilistic models, interpretable knowledge base methods, and explicit representations of the context in which words occur. When used as underlying input representations, word and phrase embeddings have been shown to improve the performance of natural language processing tasks such as parsing and sentiment analysis.

In recommendation systems, tags are defined as the most important keywords that represent the semantics of an article. They are suitable for matching user profiles and content, offering a more granular semantic level compared to categories and topics. Tags exist in various stages of the recommendation system, including content profile dimensions, user profile dimensions, recall model features, ranking model features, and diversity dispersal.

The solutions provided in this application involve technologies such as natural language processing and machine learning in artificial intelligence. The text label determination method provided in this application can be applied to at least one of the following scenarios:

First, this text tagging method is applied to the recommendation system of an article reading platform. The articles include news articles, WeChat official account articles, original personal articles, and book content. The server receives article content uploaded by users or partners via their terminals. The recommendation system performs tag mining on the content and distributes it to relevant users based on the tags. For example, a news article reporting recent housing price trends and new government policies might be tagged as "housing prices, policies" by the server's recommendation system. This would then be pushed to the homepage or other pages of the applications of users who have recently followed housing price trends, increasing the likelihood of the news being clicked.

Second, this text tag determination method is applied to the recommendation system of a social platform, which includes blogs, microblogs, and public interaction modules of social software. The server receives content uploaded by users through their terminals, and the recommendation system performs tag mining on the content, classifying and targeting the content based on the tags. In one example, on the "square" of a social application, users can post dynamic content in the form of text, pictures, videos, etc. The server's recommendation system mines the tags. For example, if the content is posted as "I went to Beijing for tourism today, visited the Forbidden City and the Bird's Nest, and Peking duck was delicious," the recommendation system will determine the tags as "Beijing, tourism" and recommend it to users located in Beijing or to users who have also posted "tourism" related content, thus recommending more precise and vertical target users.

Third, this text tagging method is applied to the recommendation system of a video platform, which can provide video content such as regular videos and short videos. The server receives content uploaded by users through their terminals or video content provided by suppliers. It performs speech-to-text processing or optical character recognition (OCR) to obtain the corresponding text, mines tags from the text, and makes recommendations based on user profiles. For example, if the text extracted from a video is "Apple is about to launch the iPhone 12, and the phone supports…", the recommendation system identifies the tags as "Apple, phone". Based on the relationship between Apple and phones, it determines that the video belongs to the electronics product field and recommends it to users who have recently followed mobile phone sales, but not to users who frequently follow the agricultural sector.

It is worth noting that the above application scenarios are merely illustrative examples, and the text label determination provided in this application can be applied to other label determination application scenarios.

Based on the above introduction of terms and application scenarios, the implementation environment of the embodiments of this application will be described.

For illustration, please refer to Figure 1. Taking the application of this text label determination method in a server as an example, the implementation environment of this text label determination method includes: terminal 110, server 120 and communication network 130.

Terminal 110 has related applications installed. These applications 111 can be applications for article reading platforms, social media platforms, or video platforms. Users can upload and publish content through terminal 110, and also receive content from server 120 through terminal 110. That is, terminal 110 is both a content producer and a content consumer. Data transmission between terminal 110 and server 120 is conducted through communication network 130.

The server 120 includes a content recommendation module 140 and a storage module 150. The content recommendation module 140 includes a machine processing module 141, a content scheduling module 142, a manual review module 143, a deduplication module 144, and an interface module 145.

The content recommendation module 140 is used to extract valid entity information from the content as tags, and match the content tags with user profiles to achieve more targeted content distribution and recommendation.

The machine processing module 141 includes a label determination module and a label determination model, which are used to determine the labels of the input content and send the processing results to the content scheduling module 142.

The content scheduling module 142 is responsible for the entire scheduling process of content flow. It receives content from the interface module 145 and obtains the content's metadata from the content database module 153. The content scheduling module 142 is also used to interact with the content storage module 151 to store or retrieve content. The content scheduling module 142 is also used to schedule the manual review module 143 and the machine processing module 141, controlling the scheduling order and priority. The content scheduling module 142 is also used to obtain content that has passed the manual review module 143 and distribute it to the terminal 110 through the exit unit in the interface module 145.

The manual review module 143 is the carrier of human service capabilities, used to review and filter content that is pornographic, illegal, or otherwise cannot be determined by machines.

The deduplication module 144 communicates with the content scheduling module 142 for deduplication of titles, cover images, body text, video fingerprints, and audio fingerprints. In one example, the title and body text of the image and text content are vectorized, image vectors are deduplicated, and video and audio fingerprints are extracted to construct vectors for video content. Then, the distance between the vectors, such as Euclidean distance, is calculated to determine whether there is a duplicate.

The interface module 145 includes an input unit and an output unit. The input unit is used to receive content input from the terminal 110, and the output unit is used to output content to the terminal 110.

The storage module 150 includes a content database module 153, a corpus 152, a content storage module 151, and a crawler module 154.

The content database module 153 is used to store the metadata of the content, including at least one of the following: file size data, cover image link information, bitrate data, file format, title information, publication time data, author information, video file size data, and video format information. The content database module 153 is also used to store the review results and review status of the content by the manual review module 143. The content database module 153 is also used to store the processing results of the machine processing module 141 and the deduplication module 144 transmitted through the content scheduling module 142.

The content storage module 151 is used to store content entity information other than the metadata of the content, such as video source files and image source files of text and image content; the content storage module 151 is also used to provide video source files to the machine processing module 141 when the video content tags are determined, including the frame content extracted from the middle of the source file, the speech-to-text conversion of the video file and the input of the original material for optical character recognition.

The crawler module 154 is used to obtain external domain corpus via the Internet and store it in the corpus 152 as domain corpus information. The corpus 152 stores the correspondence between word segmentation sequences and word vectors. Optionally, the word vectors are obtained from large-scale corpus through unsupervised learning using a related model (word to vector, word2vec) used to generate word vectors.

Figure 2 is a flowchart of a text label determination method provided in an exemplary embodiment of this application. In this embodiment, the method is described using an application in a server as an example. Referring to Figure 2, the method includes:

Step 201: Perform word segmentation on the target text to obtain a word segmentation set.

In this embodiment, the server acquires text, image, or video resources uploaded by the terminal. For text resources, the target text is generated directly. For image resources, the server obtains the text information in the image resource through optical character recognition (OCR) and generates the target text. For video resources, the server extracts the audio from the video, performs audio-to-text conversion, and generates the target text; or, it extracts video frames, performs OCR on the video frames, and generates the target text. This target text is the text for which the tag to be determined is located.

The aforementioned word segmentation set includes word segments obtained from segmenting the target text. That is, the word segmentation set is a set obtained by dividing a text or sentence into several individual words. For example, the word segmentation of "From Wu Xiaobo to Luo Zhenyu, what are the vulnerabilities of knowledge payment IPs?" yields 11 word segments: "from", "Wu Xiaobo", "to", "Luo Zhenyu", ",", "knowledge payment", "IP", "have", "which", "vulnerabilities", and "?". These 11 word segments constitute the word segmentation set.

In the process of word segmentation, there are multiple segmentation methods. With the same number of characters, the fewer the total number of words, the fewer the semantic units, the greater the weight relative to a single semantic unit, and the higher the accuracy. For example, "knowledge payment" can be segmented into two words, "knowledge" and "payment," or into one word, "knowledge payment." However, in the semantic understanding of "From Wu Xiaobo to Luo Zhenyu, what are the vulnerabilities of knowledge payment IPs?", "knowledge payment" is extracted as a single word as a tag, which makes the tag more accurate.

Optionally, a dictionary-based segmentation method can be used during the word segmentation process. Given a maximum word length, the word is segmented according to this length. The resulting word is compared to see if it appears in the dictionary. If it does, the segmented word is the final word; otherwise, the word length is shortened. The segmentation method is divided into forward segmentation and reverse segmentation, yielding forward and reverse segmentation results respectively. The result with the shortest length is selected as the word segmentation vocabulary set.

Optionally, a word segmentation method based on a Hidden Markov Model (HMM) is used during the word segmentation process. The probability of transitions between each state and the probability of each state generating the word are obtained, along with the initial state, i.e., the probability of each state being at the beginning of the sentence. Starting from the initial state, the probability that each of the four states in the state sequence set {S, B, M, E} can generate the first string is calculated, and the probability of each state is recorded. To determine the second character, the state transitions from the previous state to the current state, and the state with the highest probability of obtaining the character at the current position is selected. The previous state is then recorded. Each character is processed sequentially. Finally, word segmentation is performed based on the state sequence to obtain a word segmentation vocabulary set.

Optionally, a segmentation method based on bigrams can be used during the segmentation process. This involves iterating through all words ending at the current position, where word length is limited. After finding the current word, the previous word is found, resulting in a segmentation between these two words. Finally, the maximum probability among all segmentation results is calculated, and the segmented vocabulary set is obtained based on this maximum probability.

Step 202: Determine the first candidate tag for the target text based on the contextual relationship of the segmented words.

In this embodiment of the application, the segmented words are pre-processed to obtain a representation of the segmented words based on contextual relationships. That is, the contextual information before the segmented word and the contextual information after the segmented word are considered at the same time. This makes a good fit to the fact that human users usually need to read the whole sentence to better determine the importance of words, thereby enabling more effective determination of the first candidate label.

Optionally, the segmented vocabulary is input into the machine learning model, and the output is the first candidate label of the target text. The machine learning model includes a bidirectional long short-term memory (Bi-LSTM) neural network, a conditional random field (CRF), and an attention mechanism.

Optionally, tags are extracted from the segmented vocabulary using a Text Rank (TR) model. This method is an improvement on Page Rank (PR), a method for determining a webpage's ranking based on hyperlinks on the internet. PR is a voting-based approach; to calculate the PR value of webpage A, it's necessary to know which webpages link to it. This means first obtaining the inbound links to webpage A, and then calculating its PR value based on the votes cast by these inbound links. Please refer to Formula 1:

In this formula, _Vi represents a webpage, _Vj represents webpages linking to _Vi (i.e., incoming links to _Vi ), S( _Vi ) represents the PageRank (PR) value of webpage _Vi , In( _Vi ) represents the set of all incoming links to webpage _Vi , Out( _Vj ) represents the webpage, and d represents the damping coefficient, which is used to overcome the inherent flaw in the part after "d*" in this formula: if there is only a summation part, the formula will not be able to handle the PR value of webpages without incoming links, because in this case, according to the formula, the PR value of these webpages is 0, but this is not the case in reality. Therefore, a damping coefficient is added to ensure that each webpage has a PR value greater than 0. In one example, with a damping coefficient of 0.85, the PR value can converge to a stable value in about 100 iterations. However, when the damping coefficient is close to 1, the number of iterations required will increase sharply, and the sorting will be unstable. The fraction before S( _Vj ) in the formula means that the PR value of _Vj should be equally distributed among all the webpages pointed to by all outgoing links of _Vj , so that the current vote is distributed to the currently linked webpage.

TR is an improvement on PR. In the calculation process, only one weight term W _ji is added to indicate that the edge connection between two nodes has different importance. The method includes: (1) dividing the target text into complete sentences; (2) for each sentence, performing word segmentation and part-of-speech tagging, filtering stop words, and retaining only words with specified parts of speech, such as nouns, verbs, and adjectives, that is, obtaining a set of sentences, where the elements in the set are the retained candidate labels; (3) constructing a candidate keyword graph G = (V, E), where V is a set of nodes composed of candidate keywords generated in (2), and then using co-occurrence relations to construct an edge between any two points. An edge exists between two nodes only if their corresponding words co-occur in a window of length m, where m represents the window size, that is, at most m words co-occur; (4) calculating the weight of each node iteratively propagating until convergence; (5) sorting the node weights in reverse order to obtain the most important T words as candidate labels; (6) marking the T words obtained in (5) in the original target text. If adjacent word groups are formed, they are combined into multi-word labels.

Step 203: Determine the second candidate label of the target text based on the first frequency parameter of the segmented words in the target text and the second frequency parameter of the segmented words in the text set.

In this embodiment, the word frequency tfidf _i,j corresponding to the segmented words is determined based on the first frequency parameter tf _i,j and the second frequency parameter idf _i . The calculation method for the first frequency parameter tf _i,j is given in Formula 2, and the calculation method for the second frequency parameter idf _i is given in Formula 3.

Where tf _i,j represents the first frequency parameter, idf _i represents the second frequency parameter, n _i,j represents the number of times the current segmented word appears in text set j, |D| represents the number of texts in the corpus, and |{j:t _i ∈d _j }| represents the number of text sets containing the segmented word.

The product of the first and second frequency parameters is then used as the word frequency tfidf _i,j corresponding to the segmented word. The formula for calculating this word frequency is shown in Formula 4.

Formula 4: tfidf _i,j = tf _i,j × idf _i

Words whose frequency meets the frequency requirements are identified as the second candidate tags for the target text.

Step 204: Determine the tags of the target text based on the first candidate tags and the second candidate tags.

In this embodiment, optionally, in response to the existence of an intersection between the first candidate tag and the second candidate tag, the intersection of the first candidate tag and the second candidate tag is taken to obtain the tag of the target text. In response to the existence of no intersection between the first candidate tag and the second candidate tag, the tag of the target text is determined from the first candidate tag and the second candidate tag according to a preset selection rule. In one example, the preset selection rule is to assign different weights to the first candidate tag and the second candidate tag according to the domain to which the target text belongs. When the weight of the first candidate tag is greater than the weight of the second candidate tag, the first candidate tag is selected as the tag of the target text; when the weight of the first candidate tag is less than the weight of the second candidate tag, the second candidate tag is selected as the tag of the target text.

In summary, the text label determination method provided in this embodiment determines the label of the target text by jointly considering the contextual relationship of each word segment in the target text, as well as the first frequency parameter of each word segment in the target text and the second frequency parameter in the text set. This method determines the label of the target text from both deep semantic and shallow frequency perspectives, thereby improving the accuracy of target text label determination. Furthermore, by effectively replacing manual feature recognition engineering with a machine learning model, the efficiency of target text label determination is improved.

In conjunction with the above embodiments, taking the first candidate label obtained through a machine learning model based on the contextual relationship of segmented words as an example, the text label determination method provided in this application embodiment will be described. Please refer to Figure 3, which shows a flowchart of a text label determination method provided in an exemplary embodiment. The method includes:

Step 301: Perform word segmentation on the target text to obtain a word segmentation set.

Please refer to step 201 above for details on the process of obtaining the word segmentation set; it will not be repeated here.

Step 302: Extract features from the segmented words to obtain word vectors.

In this embodiment of the application, a large-scale corpus of the Internet is obtained in real time by crawling, and the word vectors corresponding to the word segments are learned in an unsupervised manner. The word segmentation sequence is obtained from the word segmentation set obtained by decomposing the target text, and the word segmentation sequence is input into the corpus to query the corresponding word vectors.

Step 303: Perform feature analysis on the word vectors in combination with the context word vectors to obtain the entity probabilities corresponding to the segmented words.

In this embodiment, feature analysis is performed on the word vector combined with the context word vector to obtain the entity probability corresponding to the segmented word. In one example, the word vector is input into a bidirectional long short-term memory neural network. The entity probability includes a first probability, a second probability, and a third probability. The first probability represents the probability that the segmented word belongs to the labeled entity, the second probability represents the probability that the segmented word does not belong to the labeled entity, and the third probability represents the probability that the segmented word belongs to the corresponding entity within the labeled entity. For example, if the input segmented word is an entity, such as a person's name, place name, or organization name, its first probability is high; if the input segmented word is a non-entity, such as a preposition, verb, or adjective, its second probability is high; if the input segmented word is an entity and its preceding segmented word is also an entity, its third probability is high.

Then, the first probability, second probability, and third probability corresponding to the word vector are obtained. From the first probability, second probability, and third probability, the probability with the highest value is determined as the entity probability of the segmented word.

Step 304: Determine the first candidate tag of the target text from the segmented vocabulary based on the entity probability.

In this embodiment, word segmentation words whose entity probability corresponds to the second probability are filtered out. Optionally, the word segmentation words corresponding to the second probability do not belong to the tag entity, that is, they are word segmentation words that cannot be used as tags, such as word segmentation words whose part of speech is a preposition. Then, the word segmentation words corresponding to the entity probability are determined according to the first probability and the third probability to obtain the first candidate tag.

In one example, referring to Figure 4, the word vector 410 is input into a bidirectional long short-term memory neural network (i.e., BiLSTM in Figure 4) 420. The word segmentation words 411 correspond to the word vector 410. Each input word vector 410 will output three predicted scores, which represent the first probability B, the second probability O, and the third probability I 430, respectively. Optionally, the predicted scores are input into a conditional random field (i.e., CRF in Figure 4) 440 to obtain predicted labels (i.e., B labels, I labels, or O labels corresponding to each word segmentation word in Figure 4) 450. The first candidate label is obtained based on the predicted labels.

Optionally, after performing feature analysis on the word vectors combined with the context word vectors, self-attention calculation is added for the segmented words. The calculation mechanism is shown in Figure 5. The self-attention calculation method includes: Value501 in the figure is the output word vector of the bidirectional long short-term memory neural network, and Key502 is a parameter matrix different from Value501. The essence of the attention function can be described as a query element (i.e., Query in the figure, corresponding to the self-attention calculation layer) 503 obtaining a series of (key502-value501) pairs. That is, by calculating the similarity or relevance between Query503 and each Key502, the weight coefficient of Value501 corresponding to each Key502 is obtained. Then, the Value501 is weighted and summed to obtain the final attention value 504, which is expressed as Formula 5 below:

Where Attention(Qurey,Source) represents the attention value corresponding to the current sentence (Source) and the current element, Source represents the input sentence, Query represents the element, Key represents the parameter matrix, _Lx represents the length of the sentence, and Similarity(Qurey,Key _i ) is defined in Formula 7.

The calculation mechanism is shown in Figure 6, which can be summarized into two processes: the first process calculates the weight coefficients based on Query603 and Key602, and the second process performs a weighted summation of Value601 based on the weight coefficients. The first process can be further divided into two stages: the first stage 610 calculates the similarity or relevance between Query603 and Key602 to obtain an intermediate vector; the second stage 620 normalizes the original scores from the first stage 610. In the first stage 610, different functions and calculation mechanisms can be introduced to calculate the similarity or relevance between Query603 and a specific Key602. The most common method includes calculating the dot product of the two vectors to obtain the first intermediate vector (i.e., S1, S2, S3, S4 in Figure 6) 612. The calculation method is shown in Formula 6 611 (i.e., S(Q,K) in the figure).

Formula 6: Similarity(Qurey,Key _i )=Qurey·Key _i

Where Similarty represents the vector dot product, Query represents the element, and Key _i represents the i-th parameter matrix.

The first intermediate vector generated in the first stage (610) has a different value range depending on the specific generation method. The second stage (620) introduces a calculation method similar to SoftMax normalization (621) to transform the scores from the first stage into the second intermediate vector (i.e., a1, a2, a3, a4 in the diagram) (622). This normalization process can, on the one hand, organize the original calculated scores into a probability distribution where the sum of all element weights is 1; on the other hand, it can further highlight the weights of important elements through the inherent mechanism of SoftMax. For the calculation method, please refer to Formula 7.

Where a_{_i} is the second intermediate vector, Softmax() is the softmax function, Sim_{_i} is the first intermediate vector obtained in the first stage, and L _{_x} represents the length of the current sentence (Source).

The result of the third stage calculation of 630 is the corresponding weight coefficient. Then, a weighted sum is performed to obtain the attention value of 604. Please refer to Formula 8:

Where Attention(Qurey,Source) represents the attention value corresponding to the current sentence (Source) and the current element, _Lx represents the length of the current sentence (Source), _ai is the second intermediate vector, and _Valuei is the output vector of the bidirectional long short-term memory neural network.

Step 305: Determine the second candidate tag for the target text based on the first frequency parameter of the segmented words in the target text and the second frequency parameter of the segmented words in the text set.

Optionally, the second candidate label can also be obtained through lexical features or domain features. For example, domain features include lexical codes for specific vertical domains. For instance, Apple represents a brand in the field of electronic devices and a type of fruit in the field of agriculture.

For details on the method for determining the second candidate label, please refer to step 203 above; it will not be repeated here.

Step 306: Determine the tags of the target text based on the first candidate tags and the second candidate tags.

Optionally, the result of self-attention calculation from the data output by the bidirectional long short-term memory neural network can be input into the conditional random field model along with the second candidate label. For the method of determining the label using conditional random field, please refer to Figure 7. The features include: characters, parts of speech, word segmentation boundaries, feature words, and a lexicon. The lexicon includes lexicons of personal names, place names, institutions, movies, novels, music, medical terms, and popular online public opinion terms. After the features are collected, feature templates need to be configured. The feature templates need to be configured with different combinations of the same feature at different positions, different combinations of features at the same position, and different combinations of features at different positions. In one example, feature determination is as follows: (1) If the current position 701 is segmented into 702 (part of speech 703 is a noun) and (whether the lexicon is 704 = 1) and (label 705 = 1), then let t1 = 1 in the feature template; otherwise, t1 = 0. If the weight λ1 is larger, the model is more inclined to treat the words in the lexicon as labels. (2) If the part of speech 703 of the previous word segment is punctuation, the part of speech 703 of the next word segment is verb and the current word segment is noun, then let t2 = 1 in the feature template. If the weight λ2 is larger, the model is more inclined to treat the noun sandwiched between punctuation and verb as the title entity.

In summary, the text label determination method provided in this embodiment determines the label of the target text by jointly considering the contextual relationship of each word segment in the target text, as well as the first frequency parameter of each word segment in the target text and the second frequency parameter in the text set. This method determines the label of the target text from both deep semantic and shallow frequency perspectives, thereby improving the accuracy of target text label determination. Furthermore, by effectively replacing manual feature recognition engineering with a machine learning model, the efficiency of target text label determination is improved.

In an optional embodiment, the text tag determination method provided in this application embodiment is used as an example to illustrate the system process of the method. Please refer to Figure 8. The system process 800 includes four main processes, namely video acquisition 810, content organization 820, tag determination 830, and video distribution 840.

During the execution of video acquisition 810, video files uploaded by users through a client are received. Optionally, the application corresponding to this client can be the same as the application that receives the video during the execution of video distribution 840, or the two can be different.

Content organization 820 includes: corpus collection 821, video text conversion 822, word segmentation and extraction 823;

In the process of corpus collection 821, a large-scale corpus of the Internet is acquired in real time through web crawling, and the word vectors corresponding to the word segmentation are learned in an unsupervised manner; in the process of video text conversion 822, the audio in the video file is extracted and converted into text to obtain the target text, or the video frame is extracted and the text in the video is obtained through optical character recognition; in the process of word segmentation extraction 823, the target text obtained by video text conversion 822 is decomposed into multiple word segmentation vocabulary sets.

During the execution of tag determination 830, the word segmentation vocabulary obtained by content processing 820 is acquired, the word segmentation vocabulary is processed to determine the tags of the target text, and the correspondence between the tags and the target text is output to video distribution 840.

During the execution of the video distribution process, the video distribution 840 determines the correspondence between tags and videos based on the correspondence between tags and target text, classifies videos according to tags, and distributes videos to the corresponding video category interfaces in the application, or distributes videos to user clients who may be interested in the video based on user profiles.

Referring to Figure 9, the label determination 930 includes a word segmentation mapping vector layer 931, an intermediate vector layer 932, a self-attention calculation layer 933, a word segmentation mapping external feature layer 934, a fully connected intermediate layer 935, a fully connected layer 936, a multi-classification layer 937, a constraint layer 938, and a target text word segmentation vocabulary 900.

The segmented words are input into the segmentation mapping vector layer 931 and the segmentation mapping external feature layer 934, respectively. In the segmentation mapping vector layer 931, the segmented words are mapped into word vectors, which are then input into a bidirectional long short-term memory neural network, i.e., passing through a forward recurrent neural network (RNN) layer 9311 and a backward recurrent neural network layer 9312. The result is input into an intermediate vector layer 932, and the above recurrent neural network process is repeated. Multiple linear mapping relationships 9321 are output from the intermediate vector layer 932 and input into a self-attention calculation layer 933. After calculation, the result is input into a fully connected layer 936. In the segmentation mapping external feature layer 934, the segmented words are input into the segmentation mapping external feature layer 934, and the result is output to a fully connected intermediate layer 935 for connection. The result is then output to a fully connected layer 936, where it is fully connected with the result output from the self-attention calculation layer 933. The result is then output to a multi-classification layer 937, and the classification result is input into a constraint layer 938. Finally, the label of the target text is output.

Figure 10 is a structural block diagram of a text label determining device provided in an exemplary embodiment of this application. The device includes:

Processing module 1010 is used to perform word segmentation on target text to obtain a word segmentation set, wherein the word segmentation set includes word segments obtained from the word segmentation of the target text, and the target text is the text whose label is to be determined;

The determining module 1020 is used to determine the first candidate tag of the target text based on the contextual relationship of the segmented words;

The determining module 1020 is further configured to determine a second candidate tag for the target text based on a first frequency parameter of the segmented words in the target text and a second frequency parameter of the segmented words in the text set;

The determining module 1020 is further configured to determine the tag of the target text based on the first candidate tag and the second candidate tag.

In an optional embodiment, the determining module 1020 is further configured to, in response to the existence of an intersection between the first candidate tag and the second candidate tag, take the intersection of the first candidate tag and the second candidate tag to obtain the tag of the target text.

In an optional embodiment, the determining module is further configured to, in response to the case that there is no intersection between the first candidate tag and the second candidate tag, determine the tag of the target text from the first candidate tag and the second candidate tag according to a preset selection rule.

In an optional embodiment, referring to FIG11, the determining module 1020 further includes:

The first determining unit 1021 is used to determine the word frequency corresponding to the segmented word based on the first frequency parameter and the second frequency parameter;

The first determining unit 1021 is further configured to determine the word segmentation words whose word frequencies meet the frequency requirements as the second candidate tags of the target text.

In an optional embodiment, the determining module 1020 further includes:

Extraction unit 1022 is used to extract features from the segmented words to obtain word vectors of the segmented words;

Analysis unit 1023 is used to perform feature analysis on the word vector combined with the context word vector to obtain the entity probability corresponding to the segmented word;

The second determining unit 1024 is used to determine the first candidate tag of the target text from the segmented vocabulary based on the entity probability.

It should be noted that the text label determining device provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the text label determining device and the text label determining method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

This application also provides a server, which includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the text polarity recognition method provided in the above-described method embodiments. It should be noted that the server can be the one shown in Figure 12 below.

Please refer to Figure 12, which shows a schematic diagram of the structure of a server provided in an exemplary embodiment of this application. Specifically, the server 1200 includes a Central Processing Unit (CPU) 1201, a system memory 1204 including Random Access Memory (RAM) 1202 and Read Only Memory (ROM) 1203, and a system bus 1205 connecting the system memory 1204 and the CPU 1201. The server 1200 also includes a basic input/output system 1206 that facilitates the transfer of information between various devices within the computer, and a mass storage device 1207 for storing the operating system 1213, application programs 1214, and other program modules 1215.

The basic input/output system 1206 includes a display 1208 for displaying information and an input device 1209 for user input, such as a mouse or keyboard. Both the display 1208 and the input device 1209 are connected to the central processing unit 1201 via an input/output controller 1210 connected to the system bus 1205. The basic input/output system 1206 may also include the input/output controller 1210 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input/output controller 1210 also provides output to a display screen, printer, or other types of output devices.

The mass storage device 1207 is connected to the central processing unit 1201 via a mass storage controller (not shown) connected to the system bus 1205. The mass storage device 1207 and its associated computer-readable media provide non-volatile storage for the server 1200. That is, the mass storage device 1207 may include computer-readable media (not shown) such as a hard disk or a CD-ROM drive.

Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1204 and mass storage device 1207 described above can be collectively referred to as memory.

The memory stores one or more programs, which are configured to be executed by one or more central processing units 1201. The one or more programs contain instructions for implementing the text polarity recognition method described above. The central processing unit 1201 executes the one or more programs to implement the text polarity recognition method provided in the various method embodiments described above.

According to various embodiments of this application, the server 1200 can also be connected to a remote computer on a network, such as the Internet. That is, the server 1200 can be connected to the network 1212 via the network interface unit 1211 connected to the system bus 1205, or the network interface unit 1211 can be used to connect to other types of networks or remote computer systems (not shown).

The memory also includes one or more programs stored in the memory, and the one or more programs include steps executed by the server in the text polarity recognition method provided in the embodiments of this application.

This application also provides a computer device, which includes a memory and a processor. The memory stores at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded by the processor to implement any of the text label determination methods described in the above embodiments.

This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement any of the text label determination methods described in the above embodiments.

This application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the text label determination methods described in the above embodiments.

Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into the terminal. The computer-readable storage medium stores at least one instruction, at least one program segment, a code set, or an instruction set. The at least one instruction, the at least one program segment, the code set, or the instruction set is loaded and executed by the processor to implement any of the text tag determination methods described in the embodiments of this application.

Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments described above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims (12)

1. A method for determining text labels, characterized in that the method comprises: The target text is segmented to obtain a segmentation set, which includes the segmented words obtained from the segmentation of the target text. The target text is the text whose label is to be determined. Feature extraction is performed on the segmented words to obtain the word vectors of the segmented words; Feature analysis is performed on the word vectors in conjunction with the context word vectors to obtain a first probability, a second probability, and a third probability corresponding to the word vectors. The first probability represents the probability that the word segmentation belongs to the tag entity, the second probability represents the probability that the word segmentation does not belong to the tag entity, and the third probability represents the probability that the word segmentation belongs to the corresponding entity within the tag entity. Specifically, if the word segmentation is an entity, the first probability is higher; if the word segmentation is not an entity, the second probability is higher; if the word segmentation is an entity and its preceding word segmentation is also an entity, the third probability is higher. The first probability, the second probability, and the third probability are input into a conditional random field to obtain the predicted label corresponding to the segmented word, wherein the predicted label indicates one of the first probability, the second probability, and the third probability. Filter out word segments whose predicted labels indicate the second probability; Based on the word segmentation words with the first probability indicated by the predicted tags and the word segmentation words with the third probability indicated by the predicted tags, the first candidate tags of the target text are determined; Based on the first frequency parameter of the segmented words in the target text and the second frequency parameter of the segmented words in the text set, a second candidate tag for the target text is determined; The tags of the target text are determined based on the first candidate tag and the second candidate tag. 2. The method according to claim 1, wherein determining the tags of the target text based on the first candidate tags and the second candidate tags comprises: If there is an intersection between the first candidate tag and the second candidate tag, the intersection of the first candidate tag and the second candidate tag is taken to obtain the tag of the target text. 3. The method according to claim 1, wherein determining the tags of the target text based on the first candidate tags and the second candidate tags comprises: If there is no overlap between the first candidate tag and the second candidate tag, the tag of the target text is determined from the first candidate tag and the second candidate tag according to a preset selection rule. 4. The method according to any one of claims 1 to 3, characterized in that, determining the second candidate tag of the target text based on the first frequency parameter of the segmented words in the target text and the second frequency parameter of the segmented words in the text set includes: The word frequency corresponding to the segmented word is determined based on the first frequency parameter and the second frequency parameter; The word segments whose frequencies meet the frequency requirements are identified as the second candidate tags for the target text. 5. The method according to claim 4, wherein determining the word frequency corresponding to the segmented word based on the first frequency parameter and the second frequency parameter includes: The product of the first frequency parameter and the second frequency parameter is determined as the word frequency corresponding to the segmented word. 6. A text label determining device, characterized in that the device comprises: The processing module is used to perform word segmentation on the target text to obtain a word segmentation set, wherein the word segmentation set includes the word segments obtained from the word segmentation of the target text, and the target text is the text whose label is to be determined; A determination module is used to extract features from the segmented words to obtain word vectors for the segmented words; perform feature analysis on the word vectors in combination with the context word vectors to predict a first probability, a second probability, and a third probability corresponding to the segmented words. The first probability represents the probability that the segmented word belongs to a tag entity, the second probability represents the probability that the segmented word does not belong to the tag entity, and the third probability represents the probability that the segmented word belongs to the corresponding entity within the tag entity. Specifically, if the segmented word is an entity, the first probability is high; if the segmented word is not an entity, the second probability is high; if the segmented word is an entity and its preceding segmented word is an entity, the third probability is high. The first probability, the second probability, and the third probability are input into a conditional random field to obtain a predicted label corresponding to the segmented words. The predicted label indicates one of the first probability, the second probability, and the third probability. Segmented words whose predicted labels indicate the second probability are filtered out. Based on the segmented words whose predicted labels indicate the first probability and the segmented words whose predicted labels indicate the third probability, a first candidate label for the target text is determined. The determining module is further configured to determine a second candidate tag for the target text based on a first frequency parameter of the segmented words in the target text and a second frequency parameter of the segmented words in the text set; The determining module is further configured to determine the tag of the target text based on the first candidate tag and the second candidate tag. 7. The apparatus according to claim 6, wherein the determining module is further configured to, when there is an intersection between the first candidate tag and the second candidate tag, take the intersection of the first candidate tag and the second candidate tag to obtain the tag of the target text. 8. The apparatus according to claim 6, wherein the determining module is further configured to determine the tag of the target text from the first candidate tag and the second candidate tag according to a preset selection rule when there is no intersection between the first candidate tag and the second candidate tag. 9. The apparatus according to any one of claims 6 to 8, wherein the determining module further comprises: The first determining unit is configured to determine the word frequency corresponding to the segmented word based on the first frequency parameter and the second frequency parameter; The first determining unit is further configured to determine the word segmentation words whose word frequencies meet the frequency requirements as the second candidate tags of the target text. 10. A computer device, characterized in that the computer device includes a processor and a memory, the memory storing at least one program, the at least one program being loaded and executed by the processor to implement the text tag determination method as described in any one of claims 1 to 5. 11. A computer-readable storage medium, characterized in that the storage medium stores at least one program, said at least one program being loaded and executed by a processor to implement the text tag determination method as described in any one of claims 1 to 5. 12. A computer program product, characterized in that it comprises a computer program, which, when executed by a processor, implements the text label determination method as described in any one of claims 1 to 5.