CN115114393A - A Lucene-based Peer Customer Retrieval Method - Google Patents

Abstract

The invention discloses a method for retrieving peer customers based on Lucene, which mainly comprises the steps of constructing a feature index library of a first customer, identifying the features of the second customer, calculating the similarity between the features of the second customer and the features of the first customer, and recalling the results. Manual ranking. On the basis of the general Lucene-based full-text retrieval function, through feature engineering, the features that can represent the first customer and the features that can represent the second customer can be identified, and the similarity algorithm and manual ranking algorithm of the present invention can be extremely The effect of greatly improving the recall accuracy and retrieval efficiency of peer customers.

Description

A Lucene-based Peer Customer Retrieval Method

technical field

The invention relates to the field of information retrieval, in particular to a Lucene-based peer customer retrieval method.

Background technique

In the sales process, peer stimulation is a commonly used sales negotiation technique. Recommending well-known peer companies that have signed contracts to customers will help sales to impress customers in the negotiation process and achieve the purpose of cooperating with them.

In the existing public technology, there is no standardized solution for peer customer retrieval. Common peer customer retrieval methods include using public channels to retrieve peer customers and self-built peer customer retrieval platforms. These solutions have the following shortcomings:

1. Retrieving peer customers through open channels, relying on external channels and customer active behavior at the information level, requires secondary processing of information, and the efficiency of acquiring peer customers is extremely low.

2. The self-built peer-to-peer customer retrieval platform, through the construction of the information base of the cooperative customers, through the traditional relational database, to carry out keyword matching, has the problem of low retrieval efficiency, and cannot cope with the peer-to-peer customer retrieval in large-scale scenarios.

3. Based on the previous article, it is easy to think of upgrading and optimizing the self-built peer-to-peer customer retrieval platform solution, and improving the efficiency of word segmentation matching retrieval by using the full-text retrieval solution. However, after practice, especially in industries involving all categories and a large number of cross-region In the customer's platform, the accuracy of peer customers obtained simply by word segmentation matching cannot be guaranteed.

Therefore, a more efficient, convenient and high-accuracy peer-to-peer customer retrieval method based on a full-text retrieval engine is required.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a Lucene-based peer customer retrieval method that is efficient, convenient and highly accurate.

In order to solve the above-mentioned technical problems, the present invention provides a method for retrieving peer customers based on Lucene, which mainly includes constructing a feature index library of the first customer, identifying the features of the second customer, and comparing the features of the second customer with the features of the first customer. Degree calculation, recall results are manually ranked. It is characterized in that, comprises the following steps:

Step 1: constructing a feature index library of the first customer, specifically: using the company basic information, industry information and marketing information of the first customer to extract feature words and build a Lucene-based first customer feature index library;

Step 2: establishing the association between the industry feature word of the second customer and the associated catalog, including: identifying the industry feature word of the second customer, and obtaining the associated catalog and the corresponding association degree of the industry feature word of the second customer;

Step 3: Calculate the similarity score between the second customer and each first customer. The similarity score is calculated by comparing the industry feature words of the second customer with the company basic information, industry information and marketing information of the first customer. The sum of the three similarity scores obtained by similarity calculation, including: the similarity between the industry feature word of the second customer and the basic company information of the first customer, the industry feature word of the second customer and the industry of the first customer The similarity between the information, the similarity between the industry feature words of the second customer and the marketing information of the first customer;

Step 4: Re-rank the recall results. On the basis of the similarity score obtained in Step 3, optimize the ranking based on a preset ranking optimization rule; the ranking optimization scheme includes performing secondary sorting according to the attribute information of the first customer, The attribute information includes the customer level and the number of inquiries received by the customer. The customer level and the number of inquiries received by the customer are preset parameters in the database and are used as dictionary data in this step.

The step 1 specifically includes:

Step 11: Collect basic company information of the first client, where the basic company information includes company keywords;

Step 12: perform word segmentation on the basic information of the company, organize it into a text in the format of {word element,...}, and construct the index field of the first customer feature index library, wherein the word element is the smallest unit of the text word segmentation result;

Step 13: Collect industry information of the first customer, the industry information includes: main product name, main product keyword, main product catalog;

Step 14: According to the main product catalog as a statistical dimension, the industry information collected in step 13 is processed by word segmentation, and sorted into a text in the format of {main product catalog: {word element: word frequency},...} to construct the first An index field of a customer's feature index library, where the word element is the smallest unit of the text segmentation result, the word frequency is the number of times the word element appears in the statistical dimension of the corresponding main product catalog, and the word that appears multiple times in a main product The word frequency of the meta is only recorded once, and the maximum word frequency is the sum of the number of main products in the corresponding main product catalog;

Step 15: Collect marketing information of the first customer, where the marketing information includes the purchased marketing promotion keywords;

Step 16: The marketing information collected in step 15 is processed into a {marketing word,...} format without word segmentation, and an index field of the feature index library of the first customer is constructed;

Step 17: Complete the construction of the feature index library of the first customer based on the index field.

The step 2 specifically includes:

Step 21: Based on the preset industry feature word lexicon, for any main product name of the second customer, through the reverse maximum word segmentation algorithm, identify its industry feature word as the industry feature word of the second customer;

Step 22: Use the pre-trained language model Bert to build a classification model of the first customer's main product information and catalog;

Step 23: input the industry characteristic word of the second customer identified in step 22 to the model of step 23, return the three catalog codes with the highest predicted classification probability by the model, and use the classification probability as the degree of association between the industry characteristic word and the corresponding catalog;

Step 24: Output the prediction result and organize it into a text format of {industry characteristic word of the second customer: {association directory: association degree,...},...}.

The step 3 specifically includes:

其中i∈[1,n]，n为第二客户的行业特征词的关联目录与匹配上的第一客户的主营产品目录匹配的个数，catboost为第二客户的行业特征词与主营产品目录之间的关联度，sum(fre)为第二客户的行业特征词相匹配的主营产品词元对应的词频的累加值，totalProdNum为匹配上的第一客户的主营产品总数；Step 31: Calculation of the similarity between the industry characteristics of the second customer and the industry information of the first customer in the step 3, including the formula:

where i∈[1,n], n is the number of matches between the associated catalog of the second customer's industry feature word and the matching main product catalog of the first customer, catboost is the second customer's industry feature word and the main product catalog The degree of correlation between product catalogs, sum(fre) is the cumulative value of the word frequency corresponding to the main product word that matches the second customer's industry feature word, and totalProdNum is the total number of main products of the first customer that matches;

Step 32: The scoring formula for the similarity between the industry characteristics of the second customer and the basic company information of the first customer in the step 3 is: score_ck=n'*1000, where n' is the industry characteristic word of the second customer and the The number of matches between the basic information of the first client company;

Step 33: The scoring formula for the similarity between the industry characteristics of the second customer and the marketing information of the first customer in step 3 is: score_tr=n”*10000, where n” is the industry characteristic word of the second customer and the first customer’s marketing information. Number of matches between customer marketing messages;

Step 34: Combining step 31, step 32 and step 33, the total similarity score between the second customer feature and the first customer is: score_final=score_mpf+score_ck+score_tr.

The step 4 includes:

Step 41: The ranking optimization rule is: according to the score_final score in step 34, divide the score interval [0～1], [1～1000], [1000～], for the similarity score falls in [1～ 1000], [1000～] The matching results of the two intervals, according to the first customer level, the number of inquiries received by the first customer is secondarily sorted;

Step 42: The sorting result of Step 41 is the final peer search result of the second client.

Beneficial effects achieved by the present invention: On the basis of the general Lucene-based full-text retrieval function, through feature engineering, the features that can represent the first customer and the features that can represent the second customer are identified, and the similarity algorithm of the present invention is used. And artificial ranking algorithm, can greatly improve the recall accuracy and retrieval efficiency of peer customers.

Description of drawings

FIG. 1 is a schematic flowchart of a method of an exemplary embodiment of the present invention.

Detailed ways

The Lucene full-text retrieval engine is a commonly used text retrieval solution in the industry. Using the inverted index technology can significantly improve the efficiency of document retrieval. At the same time, using its custom scoring plug-in, it can achieve custom scoring and custom manual ranking, with better performance. flexibility.

Therefore, the present invention builds a customer feature index based on the index construction technology of the Lucene full-text search engine, and achieves the effect of efficiently and accurately retrieving peer customers through a self-defined scoring algorithm and manual ranking algorithm.

The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples. The described embodiments are only examples, and changes or equivalent changes made based on the technical essence of the present invention still fall within the protection scope of the present invention.

Referring to Fig. 1, the process flow of the peer customer retrieval method of the present invention, the specific steps are:

Step 1: Construct the feature index library of the first customer, specifically: using the company basic information, industry information and marketing information of the first customer to extract feature words and build a Lucene-based first customer feature index library, which is recorded as collection;

Step 11: Collect basic company information of the first client, where the basic company information includes company keywords and the like. Since company keywords are the information that can best represent the characteristics of customers, and it is the customers themselves who extract key features for themselves, it has better application effects to use customer keywords as customer characteristics.

Step 12: Perform word segmentation on the basic information of the company, organize it into a text in the format of {word element,...}, and construct an index field of the first customer feature index library, wherein the word element is the smallest unit of the text word segmentation result.

Step 13: Collect industry information of the first customer, the industry information includes: main product name, main product keyword, main product catalog; for example: {C11, C11, C22, C33}

Step 14: According to the main product catalog as a statistical dimension, the industry information collected in step 13 is processed by word segmentation, and sorted into a text in the format of {main product catalog: {word element: word frequency},...} to construct the first An index field of a customer's feature index library, where the word element is the smallest unit of the text segmentation result, the word frequency is the number of times the word element appears in the statistical dimension of the corresponding main product catalog, and the word that appears multiple times in a main product The word frequency of the meta is only recorded once, and the maximum word frequency is the sum of the number of main products in the corresponding main product catalog;

For example, the customer's main products have two categories, which are distributed in the 001 and 002 catalogs.

There are 10 products released in the 001 directory. After word segmentation, three tokens C11, C12, and C13 are obtained. The frequency distribution of the three tokens in the 10 products is as follows:

C11 C12 C13 Product 1 2 5 13 Product 2 10 18 8 Product 3 40 3 6 Product 4 0 2 15 Product 5 7 0 12 Product 6 0 30 7 Product 7 2 18 3 Product 8 0 9 4 Product 9 14 0 25 Product 10 0 5 19 Word frequency statistics 6 8 10

Note: In the above table, the horizontal axis is the word element, and the vertical axis is the catalog product. Each cell represents the number of times the word appears in the product. According to the above statistical results, the following information can be summarized: {001:{C11:6, C12:8, C13:10}}.

There are 15 products released in the 002 directory. After the word segmentation process, three tokens C11, C22, and C23 are obtained. The frequency distribution of the three tokens in the 15 products is as follows:

Note: In the above table, the horizontal axis is the word element, and the vertical axis is the catalog product. Each cell represents the number of times the word appears in the product. According to the above statistical results, the following information can be summarized: {002:{C11:15, C22:9, C23:11}}.

After processing in this step, it is finally sorted into the following information format:

{001:{C11:6,C12:8,C13:10},002:{C11:15,C22:9,C23:11}};

Step 15: Collect marketing information of the first customer, where the marketing information includes purchased marketing promotion keywords and the like. This information is currently the most recommended feature by customers and is the most representative of all information;

Step 16: The marketing information collected in Step 15 is not processed by word segmentation, and is organized into texts in the format of {marketing words,...}, and the index field of the feature index database of the first customer is constructed; for example: {C11};

Step 17: Complete the construction of the feature index library of the first customer based on the index field;

Step 2: establishing the association between the industry feature word of the second customer and the associated catalog, including: identifying the industry feature word of the second customer, and obtaining the associated catalog and the corresponding association degree of the industry feature word of the second customer;

Step 21: Based on the preset industry feature word lexicon, for any main product name of the second customer, through the reverse maximum word segmentation algorithm, identify its industry feature word as the industry feature word of the second customer;

Step 22: Use the pre-trained language model Bert to build a classification model of the first customer's main product information and catalog;

Step 23: input the industry characteristic word of the second customer identified in step 22 to the model of step 23, return the three catalog codes with the highest predicted classification probability by the model, and use the classification probability as the degree of association between the industry characteristic word and the corresponding catalog;

Step 24: Output the prediction result and organize it into a text format of {industry characteristic word of the second customer: {association directory: relevance,...},...}, for example: the characteristic word of the second customer is C11, after entering the above model, output {002:0.65,001:0.12};

Step 3: Calculate the similarity score between the second customer and each first customer. The similarity score is calculated by comparing the industry feature words of the second customer with the company basic information, industry information and marketing information of the first customer. The sum of the three similarity scores obtained from the similarity calculation, specifically: the similarity between the industry feature word of the second customer and the basic company information of the first customer on the B2B platform, the industry feature word of the second customer and the B2B platform. The similarity between the industry information of the first customer, the similarity between the industry feature words of the second customer and the marketing information of the first customer of the B2B platform;

其中i∈[1,n]，n为第二客户的行业特征词的关联目录与匹配上的第一客户的主营产品目录匹配的个数，catboost为第二客户的行业特征词与B2B平台的主营产品目录之间的关联度，sum(fre)为第二客户的行业特征词相匹配的主营产品词元对应的词频的累加值，totalProdNum为匹配上的第一客户的主营产品总数，整体分值*10是为了控制分值区间，作为第一档；例如:对于第二客户的C1特征词和关联目录信息{001:0.65,002:0.12}，此时会通过C1命中第一客户，将特征信息统计数据带入公式，得到：score_mpf＝(0.65*(15/15)+0.12*(6/10))*10＝7.22；Step 31: Calculation of the similarity between the industry characteristics of the second customer and the industry information of the first customer of the B2B platform in the step 3, including the formula:

where i∈[1,n], n is the number of matches between the associated catalog of the second customer's industry feature word and the matching first customer's main product catalog, catboost is the second customer's industry feature word and the B2B platform The degree of association between the main product catalogs, sum(fre) is the cumulative value of the word frequency corresponding to the main product word that matches the industry feature word of the second customer, and totalProdNum is the matching main product of the first customer The total number, the overall score*10 is used to control the score range, as the first grade; for example: for the second customer's C1 feature word and associated directory information {001:0.65,002:0.12}, at this time, the C1 will hit the first A customer, put the feature information statistics into the formula, and get: score_mpf=(0.65*(15/15)+0.12*(6/10))*10=7.22;

Step 32: The scoring formula for the similarity between the industry characteristics of the second customer and the company basic information of the first customer on the B2B platform in the step 3 is: score_ck=n'*1000, where n' is the industry characteristic of the second customer The number of matches between the word and the company's basic information of the first customer; for example, for the C1 feature word of the second customer, if the basic company information of the first customer is hit, score_ck=2*1000=2000;

Step 33: The scoring formula for the similarity between the industry characteristics of the second customer in step 3 and the marketing information of the first customer on the B2B platform is: score_tr=n”*10000, where n” is the industry characteristic word of the second customer and the The number of matches between the marketing information of the first customer; for example, for the C1 feature word of the second customer, if the marketing information of the first customer is hit, score_tr=1*10000=10000;

Step 34: Combined with step 31, step 32 and step 33, the total similarity score between the second customer feature and the first customer is: score_final=score_mpf+score_ck+score_tr; for example: for the second customer’s C1 feature word and Associate directory information {001:0.65,002:0.12}, hit the first customer, and the final total similarity score is:

score_final=7.22+2000+10000=12007.22

Step 4: Re-rank the recall results. On the basis of the similarity score obtained in Step 3, optimize the ranking based on a preset ranking optimization rule; the ranking optimization scheme includes performing secondary sorting according to the attribute information of the first customer, The attribute information includes the first customer level and the number of inquiries received by the first customer. The customer level and the number of inquiries received by the customer are the customer attribute information of the B2B platform database, which are preset parameters, and are used as dictionary data in this step.

Step 41: The ranking optimization rule is: according to the score_final score in step 34, divide the score interval [0～1], [1～1000], [1000～], for the similarity score falls in [1～ 1000], [1000～] The matching results of the two intervals, according to the first customer level, the number of inquiries received by the first customer is sorted secondarily; after practical comparison, the setting of this interval value is compared with other arbitrarily set values. The number of values can sort the search results more accurately and efficiently, and the sorting results are more in line with expectations. For example, for the C1 feature word of the second customer and the associated directory information {001:0.65,002:0.12}, multiple first customers are hit, and the distribution is as follows:

score_final Customer level Number of inquiries received first customer 1 12007.22 2 100 first client 2 10010.53 1 60 first customer 3 5022.85 1 80 first customer 4 2009.21 3 90 first customer 5 1008.42 2 46 first customer 6 55.36 1 14 first customer 7 12.15 2 twenty three

After re-ranking, the final ranking is as follows:

score_final Customer level Number of inquiries received first customer 1 2009.21 3 90 first client 2 12007.22 2 100 first customer 3 1008.42 2 46 first customer 4 5022.85 1 80 first customer 5 10010.53 1 60 first customer 6 12.15 2 twenty three first customer 7 55.36 1 14

Step 42: The sorting result of Step 41 is the final peer search result of the second client.

The present invention is mainly used to provide a customer retrieval method. Based on the basic function of the general Lucene-based full-text retrieval, through feature engineering, the features that can represent the first customer and the features that can represent the second customer are identified. The similarity algorithm and manual ranking algorithm can greatly improve the recall accuracy and retrieval efficiency of peer customers.

The above embodiments do not limit the present invention in any way, and all other improvements and applications made in the form of equivalent transformations to the above embodiments belong to the protection scope of the present invention.

Claims (5)

1. a kind of peer customer retrieval method based on Lucene, is characterized in that, comprises the following steps: Step 1: constructing a feature index library of the first customer, specifically: using the company basic information, industry information and marketing information of the first customer to extract feature words and build a Lucene-based first customer feature index library; Step 2: establishing the association between the industry feature word of the second customer and the associated catalog, including: identifying the industry feature word of the second customer, and obtaining the associated catalog and the corresponding association degree of the industry feature word of the second customer; Step 3: Calculate the similarity score between the second customer and each first customer. The similarity score is calculated by comparing the industry feature words of the second customer with the company basic information, industry information and marketing information of the first customer. The sum of the three similarity scores obtained by similarity calculation, including: the similarity between the industry feature word of the second customer and the basic company information of the first customer, the industry feature word of the second customer and the industry of the first customer The similarity between the information, the similarity between the industry feature word of the second customer and the marketing information of the first customer; Step 4: Re-rank the recall results. On the basis of the similarity score obtained in Step 3, optimize the ranking based on a preset ranking optimization rule; the ranking optimization scheme includes performing secondary sorting according to the attribute information of the first customer, The attribute information includes the customer level and the number of inquiries received by the customer. The customer level and the number of inquiries received by the customer are preset parameters in the database and are used as dictionary data in this step. 2. a kind of peer customer retrieval method based on Lucene as claimed in claim 1 is characterized in that: described step 1 specifically comprises: Step 11: Collect basic company information of the first client, where the basic company information includes company keywords; Step 12: perform word segmentation on the basic information of the company, organize it into a text in the format of {word element,...}, and construct the index field of the first customer feature index library, wherein the word element is the smallest unit of the text word segmentation result; Step 13: Collect industry information of the first customer, the industry information includes: main product name, main product keyword, main product catalog; Step 14: According to the main product catalog as a statistical dimension, the industry information collected in step 13 is processed by word segmentation, and sorted into a text in the format of {main product catalog: {word element: word frequency},...} to construct the first An index field of a customer's feature index library, where the word element is the smallest unit of the text segmentation result, the word frequency is the number of times the word element appears in the statistical dimension of the corresponding main product catalog, and the word that appears multiple times in a main product The word frequency of the meta is only recorded once, and the maximum word frequency is the sum of the number of main products in the corresponding main product catalog; Step 15: Collect marketing information of the first customer, where the marketing information includes the purchased marketing promotion keywords; Step 16: The marketing information collected in step 15 is processed into a {marketing word,...} format without word segmentation, and an index field of the feature index library of the first customer is constructed; Step 17: Complete the construction of the feature index library of the first customer based on the index field. 3. a kind of peer customer retrieval method based on Lucene as claimed in claim 2 is characterized in that: described step 2 specifically comprises: Step 21: Based on the preset industry feature word lexicon, for any main product name of the second customer, through the reverse maximum word segmentation algorithm, identify its industry feature word as the industry feature word of the second customer; Step 22: Use the pre-trained language model Bert to build a classification model of the first customer's main product information and catalog; Step 23: input the industry characteristic word of the second customer identified in step 22 to the model of step 23, return the three catalog codes with the highest predicted classification probability by the model, and use the classification probability as the degree of association between the industry characteristic word and the corresponding catalog; Step 24: Output the prediction result and organize it into a text format of {industry characteristic word of the second customer: {association directory: association degree,...},...}. 4. a kind of peer customer retrieval method based on Lucene as claimed in claim 3 is characterized in that: described step 3 specifically comprises: Step 31: Calculation of the similarity between the industry characteristics of the second customer and the industry information of the first customer in the step 3, including the formula:

where i∈[1,n], n is the number of matches between the associated catalog of the second customer's industry feature word and the matching main product catalog of the first customer, catboost is the second customer's industry feature word and the main product catalog The degree of correlation between product catalogs, sum(fre) is the cumulative value of the word frequency corresponding to the main product word that matches the second customer's industry feature word, and totalProdNum is the total number of main products of the first customer that matches; Step 32: The scoring formula for the similarity between the industry characteristics of the second customer and the basic company information of the first customer in the step 3 is: score_ck=n'*1000, where n' is the industry characteristic word of the second customer and the The number of matches between the basic information of the first client company; Step 33: The scoring formula for the similarity between the industry characteristics of the second customer and the marketing information of the first customer in step 3 is: score_tr=n”*10000, where n” is the industry characteristic word of the second customer and the first customer’s marketing information. Number of matches between customer marketing messages; Step 34: Combining step 31, step 32 and step 33, the total similarity score between the second customer feature and the first customer is: score_final=score_mpf+score_ck+score_tr. 5. a kind of peer customer retrieval method based on Lucene as claimed in claim 4 is characterized in that: described step 4 comprises: Step 41: The ranking optimization rule is: according to the score_final score in step 34, divide the score interval [0～1], [1～1000], [1000～], for the similarity score falls in [1～ 1000], [1000～] The matching results of the two intervals, according to the first customer level, the number of inquiries received by the first customer is secondarily sorted; Step 42: The sorting result of Step 41 is the final peer search result of the second client.

Images