CN116166873A - User portrait generation method and device, electronic equipment and storage medium - Google Patents

Abstract

The present disclosure provides a method and device for generating user portrait data, electronic equipment, and a storage medium. The method includes: obtaining target user data to be processed in a streaming manner, and the target user data includes user data generated in real time by business applications; obtaining pre-configured tag information and crowd package information, the tag information includes at least one tag information, each tag represents a user feature of the user, the crowd package information includes at least one crowd package information, and each crowd package includes multiple different tags; according to the tag information and crowd package information, profile the target user data, and obtain user profile data; in the process of profiling the target user data, obtain the running status of the back-end nodes of the database cluster, and run the back-end nodes according to the running status dimension management. According to the embodiments of the present disclosure, user data generated by business applications in real time can be profiled in a timely manner, so as to increase the speed of generating user profile data.

Description

User portrait generation method and device, electronic device, storage medium

technical field

The present disclosure relates to the field of computer technology, and in particular to a method and device for generating a user portrait, electronic equipment, and a storage medium.

Background technique

User portrait (Persona), also known as user role, as an effective tool to outline target users, contact user demands and design direction, user portrait has been widely used in various fields, through user portrait, can provide users with various convenient services.

At present, when performing user portraits, the online user data generated by online business applications within a certain period of time and the offline user data uploaded offline are generally obtained in batches according to preset time intervals, and the data obtained in batches is analyzed and processed. , to obtain user profile data.

It can be seen that the current method for generating user portraits may have a processing delay problem.

Contents of the invention

The disclosure provides a method and device for generating a user portrait, electronic equipment, and a storage medium.

In a first aspect, the present disclosure provides a method for generating a user portrait, which is applied to an electronic device deployed with a database cluster, and the method includes:

Streaming acquisition of target user data to be processed, wherein the target user data includes user data generated by business applications in real time;

Acquiring pre-configured tag information and group group information, wherein the tag information includes at least one tag information, each tag represents a user characteristic of the user, the group group information includes at least one group group information, and each group Packages include several different tags;

According to the tag information and the crowd package information, profile the target user data to obtain user profile data; and,

In the process of profiling the target user data, the running status of the back-end nodes of the database cluster is obtained, and the operation and maintenance management is performed on the back-end nodes according to the running status.

In a second aspect, the present disclosure provides a device for generating a user portrait, which is applied to an electronic device deployed with a database cluster, and the device includes:

A user data acquisition unit, configured to acquire target user data to be processed in a streaming manner, wherein the target user data includes user data generated in real time by business applications;

A configuration information acquisition unit, configured to acquire pre-configured label information and crowd package information, wherein the label information includes at least one label information, each label represents a user characteristic of the user, and the crowd package information includes at least one crowd Package information, each crowd package includes multiple different tags;

an image unit, configured to image the target user data according to the tag information and the crowd package information, and obtain user image data; and,

The guarding unit is configured to obtain the running status of the back-end nodes of the database cluster during the profiling of the target user data, and perform operation and maintenance management on the back-end nodes according to the running status.

In a third aspect, the present disclosure provides an electronic device, which includes: at least one processor; and a memory communicated with the at least one processor; wherein, the memory stores information that can be processed by the at least one processor. One or more computer programs executed by a processor, and one or more computer programs are executed by the at least one processor, so that the at least one processor can execute the above-mentioned user portrait generation method.

In a fourth aspect, the present disclosure provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned user portrait generation method when executed by a processor.

In the embodiment provided by the present disclosure, when the user authorization is obtained, the target user data to be processed is obtained through streaming, and the pre-configured tag information and crowd package information are obtained. Since the tag information includes at least one representing the user The tag information of the user characteristics, and the group package information includes at least one group package information composed of multiple different tags. Therefore, based on the tag information and the group package information, the target user data obtained by streaming is profiled, that is User portrait data can be generated in time; at the same time, in the process of profiling target user data, by obtaining the running status of the back-end node of the database cluster deployed in the electronic device, and operating and maintaining the back-end node according to the running status Management can also ensure the stable and reliable operation of the back-end node, thus providing guarantee for the healthy operation of image processing. Due to the continuous and non-interval characteristics of the stream processing method, obtaining the target user data to be processed based on the stream processing method can reduce the processing delay when performing user portraits; at the same time, based on the running status of the back-end nodes The operation and maintenance management of the back-end nodes can also ensure the stable and reliable operation of the back-end nodes and avoid the downtime of the back-end nodes, so as to ensure that the electronic equipment can provide real-time user portrait data for users to view in a timely manner.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present disclosure, and constitute a part of the specification, and are used together with the embodiments of the present disclosure to explain the present disclosure, and do not constitute a limitation to the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

FIG. 1 is a flow chart of a method for generating a user portrait provided by an embodiment of the present disclosure;

FIG. 2 is a flow chart of acquiring target user data provided by an embodiment of the present disclosure;

FIG. 3 is a flow chart of the rules for acquiring target portraits provided by an embodiment of the present disclosure;

FIG. 4 is a first framework processing diagram for detecting backend nodes provided by an embodiment of the present disclosure;

FIG. 5 is a second framework processing diagram for detecting backend nodes provided by an embodiment of the present disclosure;

FIG. 6 is a block diagram of a device for generating a user portrait provided by an embodiment of the present disclosure;

Fig. 7 is a block diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order for those skilled in the art to better understand the technical solution of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and they should be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the case of no conflict, various embodiments of the present disclosure and various features in the embodiments can be combined with each other.

As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The terminology used herein is for describing particular embodiments only and is not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that when the terms "comprising" and/or "consisting of" are used in this specification, the stated features, integers, steps, operations, elements and/or components are specified to be present but not excluded to be present or Add one or more other features, integers, steps, operations, elements, components and/or groups thereof. Words such as "connected" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

It should be noted that all actions to obtain signals, information or data in this application are carried out under the premise of complying with the corresponding data protection laws and policies of the country where the location is located, and with the authorization given by the corresponding device owner.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant art and the present disclosure, and will not be interpreted as having idealized or excessive formal meanings, Unless expressly so limited herein.

In related technologies, when an electronic device performs user profiling, it generally first obtains the information of tags and crowd packages for user profiling and the configuration information of the data source configured by the user based on the front-end configuration interface, wherein the data source can be It is a database for storing online data generated by business applications or offline data uploaded by users; after that, the electronic device obtains the latest generated data to be processed from the data source at 0:00 every day according to a preset time interval. User data; and conduct batch user portrait analysis on the data to obtain user portrait data. Since this user portrait generation method is based on a batch processing mechanism, the user portrait data generated by it often has delays and cannot reflect the latest developments and demands of users in business applications, causing other systems that rely on user portrait data, such as membership System, marketing system, telemarketing system and other systems may not be able to respond to user needs in a timely manner.

In order to solve the above problems, an embodiment of the present disclosure provides a method for generating a user portrait, please refer to FIG. 1 , which is a flowchart of a method for generating a user portrait provided by an embodiment of the present disclosure. This method can be applied to an electronic device deployed with a database cluster (such as a Starrocks cluster), the electronic device can be a server, the server can be a physical server, or a virtual server; of course, with the continuous advancement of technology, the The electronic device can also be a terminal device, that is, the method can also be applied to a terminal device alone, for example, it can be applied to an edge terminal device in an edge computing scenario, and there is no special limitation here.

As shown in FIG. 1 , the user portrait generation method provided by the embodiment of the present disclosure includes the following steps S101-S104, which will be described in detail below.

Step S101 , acquiring target user data to be processed in a streaming manner, wherein the target user data includes user data generated by a business application in real time.

Target user data can be user data generated by business applications in real time and subject to user portraits under the condition of user authorization; of course, the target user data can also be generated offline and uploaded to an online database, for example , the user data in the data warehouse Hive library. In addition, in the embodiments of the present disclosure, the user data can be either an individual user or an enterprise user, that is, the user data can be for an individual or for an enterprise, that is, for a certain customer, and no special description is made here. limited.

For example, in an e-commerce application, the target user data may be at least one of user data such as user attribute data, user browsing data, click data, order data, and commodity attention data generated in real time by the business application.

For another example, in a financial application, the target user data may be at least one of user attribute data, user loan data, user repayment data, and other data generated in real time by the business application.

In the embodiments of the present disclosure, the service application used to generate service data may be one or more applications in any application scenario, which is not specifically limited here.

In view of the processing delay problem that may exist when acquiring user data to be processed in a batch manner in the related art for user portrait, in the embodiment of the present disclosure, when acquiring user data, the target user data to be processed may be acquired in a streaming manner, That is, the user data generated by the business application in real time is obtained as the target user data by stream processing, and the user data generated by the business application in real time is profiled to generate the latest user portrait data, and then the latest user portrait data is provided in time for display Or it can be used by other business systems, so that the operators of business applications can respond to user needs in a timely manner.

Stream processing refers to the processing process in which the data producer writes the generated data records into the ordered data stream, and the data user continuously obtains the data records from the ordered data stream in the same order for use or processing. In the formal processing, there is usually no predetermined start or end, but a series of event nodes to perform real-time response processing on the data records generated by the data producer, where the data producer can be, for example, used to generate user For the business application of data, the data user may be, for example, a user profile application in an electronic device that executes the method described in the embodiments of the present disclosure.

Step S102, obtaining pre-configured tag information and group package information, wherein the tag information includes at least one tag information, each tag represents a user feature of the user, the group package information includes at least one group package information, and each group package Include multiple different tags.

Tag information is the information of at least one tag used to make user portraits, and each tag may correspond to a user feature of the user. For example, tag information can be tag information such as "gender", "age" and "hobbies"; among them, a tag (Tag) is an abstract classification and summary of a certain characteristic of a specific group or object, Tag Values usually have separable types.

For example, for "natural person", features such as "male" and "female" can be abstracted and summarized to obtain the label "gender".

Tag value refers to the specific content contained in a tag, and its characteristics are in line with the MECE (Mutually Exclusive Collectively Exhaustive) principle, that is, they are independent of each other and completely exhaustive.

For example, for the label "gender", its corresponding label values can be divided into "male", "female" and "unknown".

The crowd package information is the information of at least one crowd package used for user portrait; the crowd package is usually used to classify users. For example, for individual users, the corresponding group packages may include "student group package", "beauty group group package", "e-commerce group group package" and so on.

In the embodiment of the present disclosure, the label information and crowd package information may be pre-obtained by the user in a manner configured in the front-end interface. Of course, the tag information and group group information can also be automatically generated by the electronic device, that is, in some embodiments, the tag information and the group group information can also be obtained through the following steps: clustering the acquired target user data Analyze to obtain at least one cluster; respectively perform feature extraction processing on the target user data in each cluster to obtain at least one user feature corresponding to each cluster; according to the corresponding to each cluster Obtain at least one tag based on at least one user characteristic, and obtain at least one group package according to the at least one tag; obtain the tag information and the group group information according to the at least one tag and the at least one group package.

That is, in the actual implementation process, after obtaining the target user data, the electronic device may perform real-time cluster analysis on the target user data to be processed to obtain at least one cluster, and extract the user data in each cluster Corresponding to one or more user features, and then get the corresponding tags by performing semantic analysis on the extracted user features, and classify the obtained tags to get at least one crowd package, and then, according to the analyzed tags and Crowd package, automatically generate tag information and crowd package information.

Among them, when performing cluster analysis on target user data, any cluster analysis algorithm can be used, such as k-means algorithm, Clara algorithm, and Clarans algorithm, etc., which are not specifically limited here; When the user features of clusters are obtained, the user features can be extracted through a pre-trained feature extraction model. The feature extraction model can be, for example, a convolutional neural network model (CNN, Convolutional Neural Networks) or other network models. The model structure and model training method will not be described here; in addition, after the electronic device automatically generates tag information and crowd package information based on the above steps, it can also first display the tag information and crowd package information to the user for viewing; After confirmation, user portraits will be performed on the target user data based on the tag information and crowd package information after the user's supplementary confirmation, and there is no special limitation here.

Step S103, according to the label information and group group information, profile the target user data to obtain user profile data.

After obtaining the target user data to be processed according to the above step S101, and obtaining the pre-configured label information and crowd package information through step S102, the electronic device can target the target user based on the label information and the crowd package information The data is profiled to obtain the latest user profile data. For example, the latest user portrait data can be generated by filtering and matching the target user data according to the tag information and crowd package information. In the embodiment of the present disclosure, no specific limitation is imposed on the specific portrait processing.

Step S104, in the process of profiling target user data, acquire the running status of the back-end node of the database cluster, and perform operation and maintenance management on the back-end node according to the running status.

In the embodiment of the present disclosure, the database cluster may be a Starrocks cluster, wherein Starrocks is a full-scenario massively parallel processing (MPP, Massively Parallel Processing) database, which is dedicated to building an extremely fast unified analysis experience and supports a variety of data Models, such as detailed models, aggregation models, and update models, etc., and support multiple data import methods. Building a cluster based on Starrocks and using it for data analysis can greatly improve the processing speed; of course, the database cluster can also be other than the Starrocks cluster. The database cluster, for example, may also be a doris cluster; in the embodiments of the present disclosure, unless otherwise specified, the database cluster is taken as an example for illustration.

Considering that the back-end nodes (BE, BackEnd) in the Starrocks cluster, that is, BE may experience abnormal downtime or poor processing performance due to inappropriate resource parameter configuration during the process of executing user portraits, so it is necessary to Users can check the status of back-end nodes at any time, and manually restart or perform performance optimization when they find downtime or poor processing performance. On the one hand, this requires user operations, which is not convenient; Restarting or performing performance optimization may also cause the electronic device to fail to generate user portrait data in time. Therefore, in the embodiment of the present disclosure, the electronic device may profile the acquired target user data based on the deployed Starrocks cluster. Automatically detect the running status of its back-end nodes, and perform operation and maintenance management on the back-end nodes according to the running status to ensure the healthy operation of the back-end nodes.

According to the above description, it can be seen that the user portrait generation method provided by the embodiment of the present disclosure obtains the target user data in a streaming manner, and because the streaming processing method has the characteristics of continuous and no interval, therefore, based on the streaming processing method, the data to be processed is acquired. The processing method of target user data and portrait can reduce the processing delay in user portrait; at the same time, through the operation and maintenance management of back-end nodes based on the running status of back-end nodes, it can also ensure the stability and reliability of back-end nodes Running, avoiding the downtime of back-end nodes so as to ensure that electronic equipment can provide real-time user portrait data for users to view in a timely manner.

Please refer to FIG. 2 , which is a flow chart of acquiring target user data provided by an embodiment of the present disclosure. As shown in FIG. 2, in the above step S101, the streaming acquisition of target user data to be processed may include the following steps S201-S204.

Step S201 , detecting data change information of user data in a target database, wherein the target database is used to store user data generated by a service application.

The target database may be, for example, a mysql database or other relational database, which is not specifically limited here.

In this implementation, the target database can be detected by components such as the Canal component or the flink-cdc component to obtain the data change information. Components for data subscription and consumption; the flink-cdc component is used to detect and capture database change information, such as insertion, update, and deletion of data or data tables, and record these change information in the order in which they occur .

Wherein, the detecting the data change information of the user data in the target database may be: obtaining the data change information by detecting the change information of the preset log file corresponding to the target database.

The preset log file corresponding to the target database is a log file for recording data change information of the target database. For example, in the case that the target database is a mysql database, the preset log file may be a binlog log file of the mysql database.

Step S202, according to the data change information, obtain the changed data record in the target database.

Step S203, writing the changed data record into the target message queue.

The target message queue is used to cache the changed data records obtained based on the data change information, so that the electronic device can consume the changed data records in a streaming manner and process them one by one. The target message queue is, for example, It can be a kafka message queue.

Step S204, according to the changed data records in the target message queue, the target user data is obtained in a streaming manner.

That is, in view of the processing delay problem that may exist when obtaining user data to be processed in a batch manner in the related art for user portrait, in the embodiment of the present disclosure, the Canal component, the flink-cdc component, etc. can be used to detect the target database Incremental data changes in the target database, so that after the business application generates user data in real time and writes the user data into the target database, by detecting the incremental data changes in the target database, the changed data records in the target database are obtained, that is, the occurrence Changed user data, and write the changed data record to the target message queue, so that when profiling, the changed data record in the target message queue can be obtained through real-time, streaming consumption to obtain target user data.

In addition, in order to further increase the speed of generating user portraits, in this embodiment, the above-mentioned stream acquisition of target user data based on the changed data records in the target message queue can be: through stream data import (Routine Load) In this way, the changed data records in the target message queue are imported into the corresponding data tables of the database cluster to obtain the target user data.

In the embodiment of the present disclosure, in the above step S103, according to the tag information and the crowd package information, the portrait of the target user data is obtained to obtain the user portrait data, which may be: according to the tag information and the crowd package information, the target portrait rule is obtained , wherein the target portrait rule is a Structured Query Language (SQL, Structured Query Language) statement that supports the execution of the back-end node of the database cluster; the data imported into the Starrocks cluster is processed by the back-end node of the database cluster according to the target portrait rule The target user portrait data in the table is profiled to obtain the target user portrait data.

The target user data may include personal portrait data and customer group portrait data. The personal portrait data may be, for example, portrait data for individual users, and the customer group portrait data may be, for example, portrait data for corporate customers.

Please refer to FIG. 3 , which is a flow chart of a rule for obtaining a target portrait provided by an embodiment of the present disclosure. As shown in Figure 3, in the embodiment of the present disclosure, the target portrait rule can be obtained through the following steps: Step S301, generate rule conditions corresponding to the label information and the crowd package information, wherein the rule conditions are used to process user data Perform screening processing; step S302, convert the rule condition based on the preset rules of the database cluster to obtain target portrait rules.

That is, in the embodiment of the present disclosure, after the label information and crowd package information are provided to the electronic device, the electronic device can organize the label information and crowd package information into rule conditions in the form of json (JavaScript Object Notation), and then, In order to facilitate the execution of database clusters, such as Starrocks clusters, the rule conditions in the form of json are pre-converted into SQL statements to improve the image speed.

In the embodiment of the present disclosure, acquiring the running state of the back-end node of the database cluster in the above step S104, and performing operation and maintenance management on the back-end node according to the running state may include: detecting the database cluster at a preset time interval The running status of the backend node; if it is determined that the backend node is running abnormally, restart the backend node.

In this implementation, the daemon tool, for example, the supervisor daemon based on the python language can be used to detect each back-end node in the Starrocks cluster, and restart it when it detects that its running status is abnormal Abnormal back-end nodes to ensure the stable and reliable operation of back-end nodes and avoid delays in the timely generation of user portrait data due to back-end node downtime.

For ease of understanding, please refer to FIG. 4 , which is a first frame processing diagram for detecting backend nodes provided by an embodiment of the present disclosure. As shown in Figure 4, for the back-end nodes in the Starrocks cluster, namely BE1, BE2, and BE3, you can configure the corresponding configuration files supervisor.conf of the daemon process supervisor1, daemon process supervisor2, and daemon process supervisor3 respectively. A state detection of the back-end nodes, thereby ensuring that the back-end nodes are abnormal again, for example, they can be started in time after a downtime. Among them, FE shown in Figure 4 represents the front-end node of the Starrocks cluster, and Broker represents the implementation of Broker in the Starrocks cluster. The process of importing; in addition, the setting method of the supervisor is described in detail in related technologies, and will not be repeated here.

It should be noted that in the embodiment shown in Figure 4, the supervisor is not used to detect the front-end nodes and Broker processes of the Starrocks cluster. In actual implementation, the two processes can also be detected at the same time to further improve the system Stability is not specifically limited here.

It should also be noted that, in the embodiment of the present disclosure, the acquisition of the running status of the back-end node of the database cluster described in the above step S104, and the operation and maintenance management of the back-end node according to the running status may also be: according to Detect the running status of the back-end node of the database cluster at a preset time interval; when it is determined that the back-end node is running abnormally, obtain the corresponding running index of the database cluster; generate an abnormality analysis report according to the running index; analyze the abnormality The report is sent to a target terminal device, wherein the abnormality analysis report is used for a business user to optimize the performance of the database cluster, the business user corresponds to the database cluster, and the target terminal device includes a terminal device used by the business user.

In the embodiment of the present disclosure, the operation indicators corresponding to the database cluster, such as the Starrocks cluster, may be: central processing unit (CPU, Central Processing Unit) utilization rate, memory usage information, operating environment, such as JDK environment, and the operation of each node of the database cluster Logs, network environment information, stored read and write information, calls per second, whether there are slow SQL indicators, etc.

That is, considering that if the back-end node is abnormal, such as downtime, it may usually be due to abnormalities or improper settings in its task resources, operating environment, or internal operating status. Therefore, an abnormality occurs in a certain back-end node of the Starrocks Under certain circumstances, in order to facilitate business users corresponding to the Starrocks cluster, such as DBAs, operation and maintenance personnel, developers, or cluster manufacturers, to troubleshoot problems and improve cluster stability, this method can also be used in the process of user portrait When an abnormality is detected in the back-end node of the Starrocks cluster, by obtaining the operating indicators of the Starrocks cluster, an abnormal analysis report is generated based on the operating indicators, and the abnormal analysis report is pushed to the terminal device used by the above-mentioned business personnel , for example, in devices such as mobile phones and tablet computers, it is convenient for the above-mentioned business personnel to optimize the performance of the Starrocks cluster in a timely manner.

For ease of understanding, please refer to FIG. 5 , which is a second framework processing diagram for detecting backend nodes provided by an embodiment of the present disclosure. As shown in Figure 5, for any backend node in the Starrocks cluster, for example, BE1 shown in Figure 5, when it is detected that its downtime occurs, the backend node can be restarted based on the daemon process supervisor at the same time, Run the script used to obtain the operating indicators of the Starrocks cluster, such as auto_check.sh shown in Figure 5 to obtain the operating indicators of the Starrocks cluster, and generate an abnormal analysis report based on the operating indicators, and send the abnormal analysis report to Figure 5 by email Viewed by the business users shown, this type of business users can analyze the cause based on the exception analysis report and optimize the performance of the Starrocks cluster to improve its stability and ensure its processing speed when performing user portraits.

In addition, in some embodiments, the acquisition of the running status of the back-end nodes of the database cluster described in the above step S104, and performing operation and maintenance management on the back-end nodes according to the running status may also be: detecting The running state of the back-end node; when it is determined that the back-end node is running abnormally, obtain the running index corresponding to the database cluster and the running resource parameter information corresponding to the database cluster, wherein the running resource parameter information includes pre-set data for the database Information about the memory resources and CPU resources configured by the back-end nodes of the cluster; in the case of determining that the operating resource parameter information meets the preset conditions according to the operating index, adjust the operating resource parameters of the database cluster to optimize the performance of the database cluster deal with.

The operating index may include, for example, CPU usage, memory usage information, etc., and the preset condition may be that the CPU usage is greater than or equal to a first preset threshold within a preset time period, and/or, the memory usage information indicates The memory usage within the set time period is greater than or equal to the second preset threshold, wherein the first preset threshold and the second preset threshold can be set according to needs. That is, in this implementation, when an abnormality is detected in the database cluster, such as the Starrocks cluster, it can be determined whether the CPU usage rate, memory usage rate, etc. are high for a long time. If so, it may be because the current database If the CPU resources configured in the cluster, such as the number of CPU cores and memory resources are not appropriate, you can consider appropriately allocating more CPU resources and memory resources; of course, this is only an example. In the actual implementation process, in When adjusting the operating resource parameters of the database cluster, it can also be adjusted based on the pre-configured resource adjustment strategy, which is not specifically limited here.

It should be noted that one or more embodiments of the above-mentioned operation and maintenance management of the back-end nodes of the database cluster according to the running status of the back-end nodes can be used alone or in combination, or can also be combined with other operation and maintenance management methods Used in combination, there is no special limitation here.

After the user portrait data is obtained through the above method, for the convenience of the user to view, in the embodiment of the present disclosure, the method further includes: generating target display data corresponding to the user portrait data, wherein the target display data includes image data and/or tabular data; and, impression target display data.

That is, considering that the obtained user portrait data may only be some numbers, and the visibility is poor, for the convenience of users to view, after obtaining the user portrait data, visual analysis can be performed on the user portrait data based on visual analysis components, for example, FineBI To obtain target display data with better visibility composed of image data, table data, etc. for users to view, thereby improving user experience.

To sum up, the method provided by the embodiments of the present disclosure acquires the target user data to be processed by streaming, and makes a portrait of the target user data based on the acquired label information and crowd package information, so that the business application can be processed in real time. The generated user data is profiled to improve the timeliness of user profile data, so that users or business systems that rely on user profile data to make business decisions or perform business processing can respond to user needs in a timely manner based on the user profile data; in addition, through a new generation of Database clusters, such as Starrocks clusters, can further improve the generation speed of user portrait data. At the same time, in the process of obtaining user portrait data, by detecting the running status of the back-end nodes in the database cluster, the back-end nodes Operation and maintenance management can ensure the rapid generation of user portrait data; in addition, after obtaining user scribing data, it can also generate target display data including image data and/or table data for users to view, which can also improve user experience.

It can be understood that the above-mentioned method embodiments mentioned in this disclosure can all be combined with each other to form a combined embodiment without violating the principle and logic. Due to space limitations, this disclosure will not repeat them. Those skilled in the art can understand that, in the above method in the specific implementation manner, the specific execution order of each step should be determined according to its function and possible internal logic.

In addition, this disclosure also provides a user portrait generation device, electronic equipment, and a computer-readable storage medium, all of which can be used to implement any user portrait generation method provided by this disclosure, and refer to the corresponding technical solutions and descriptions in the method section. ,No longer.

Fig. 6 is a block diagram of an apparatus for generating a user portrait provided by an embodiment of the present disclosure.

Referring to FIG. 6 , an embodiment of the present disclosure provides a user portrait generation device, which includes: a user data acquisition unit 601 , a configuration information acquisition unit 602 , a portrait unit 603 and a guard unit 604 .

The user data obtaining unit 601 is configured to obtain target user data to be processed in a streaming manner, wherein the target user data includes user data generated by service applications in real time.

In some embodiments, the user data acquisition unit 601 may be used to: detect data change information of user data in the target database when acquiring the target user data to be processed in a streaming manner, wherein the target database is used to store the information generated by the business application User data; obtain the changed data records in the target database according to the data change information; write the changed data records to the target message queue; obtain the target user data in a streaming manner according to the changed data records in the target message queue.

In some embodiments, when detecting data change information of user data in the target database, the user data acquiring unit 601 may be configured to: obtain data change information by detecting change information of a preset log file corresponding to the target database.

In some embodiments, when the user data acquisition unit 601 obtains the target user data in a streaming manner according to the changed data records in the target message queue, it can be used to: import the changed data records in the target message queue The data records are imported into the corresponding data tables of the database cluster to obtain target user data.

The configuration information acquisition unit 602 is configured to acquire pre-configured tag information and group package information, wherein the tag information includes information about at least one tag, each tag represents a user feature of the user, and the group package information includes information about at least one group package. information, each crowd package includes a number of different labels.

The portrait unit 603 is used to draw a picture of the target user data according to the label information and group information, and obtain user portrait data.

The guarding unit 604 is configured to acquire the running status of the back-end nodes of the database cluster during the profiling of the target user data, and perform operation and maintenance management on the back-end nodes according to the running status.

In some embodiments, the portrait unit 603 can be used to obtain target portrait rules according to the label information and crowd package information when portraiting the target user data according to the tag information and crowd package information to obtain user portrait data, wherein , the target portrait rule is a structured query language statement that supports the execution of the back-end nodes of the database cluster; the back-end nodes of the database cluster profile the target user data in the imported data table according to the target portrait rules to obtain user portrait data, Wherein, the user portrait data includes personal portrait data and/or customer group portrait data.

In some embodiments, when the portrait unit 603 obtains target portrait rules according to the tag information and crowd package information, it can be used to: generate rule conditions corresponding to the tag information and crowd package information, wherein the rule conditions are used to The data is screened and processed; the rule conditions are converted and processed based on the preset rules of the database cluster to obtain the target portrait rules.

In some embodiments, when the guard unit 604 obtains the running status of the back-end nodes of the database cluster, and performs operation and maintenance management on the back-end nodes according to the running status, it can be used to: detect the back-end nodes of the database cluster according to a preset time interval The running status of the end node; if it is determined that the back-end node is running abnormally, restart the back-end node.

In some embodiments, when the guard unit 604 obtains the running status of the back-end nodes of the database cluster, and performs operation and maintenance management on the back-end nodes according to the running status, it can be used to: detect the back-end nodes of the database cluster according to a preset time interval The operating status of the end node; when the back-end node is determined to be abnormal according to the operating status, obtain the corresponding operating index of the database cluster; generate an abnormal analysis report according to the operating index; send the abnormal analysis report to the target terminal device, where the abnormal The analysis report is used for business users to optimize the performance of the database cluster, the business users correspond to the database cluster, and the target terminal devices include the terminal devices used by the business users.

In some embodiments, the user portrait generation device 600 further includes a presentation unit, configured to: generate target presentation data corresponding to the user portrait data after obtaining the user portrait data, wherein the target presentation data includes image data and/or tables data; impression target display data.

Fig. 7 is a block diagram of an electronic device provided by an embodiment of the present disclosure.

Referring to FIG. 7 , an embodiment of the present disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702, and one or more I/O interfaces 703 connected between the processor 701 and the memory 702 Among them; wherein, the memory 702 stores one or more computer programs that can be executed by at least one processor 701, and one or more computer programs are executed by at least one processor 701, so that at least one processor 701 can perform the above-mentioned User portrait generation method.

An embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned method for generating a user portrait when executed by a processor. Computer readable storage media may be volatile or nonvolatile computer readable storage media.

An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method for generating a user portrait.

Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, the functional modules/units in the system, and the device can be implemented as software, firmware, hardware, and an appropriate combination thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components. Components cooperate to execute. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit . Such software may be distributed on computer readable storage media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).

As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable program instructions, data structures, program modules, or other data. volatile, removable and non-removable media. Computer storage media include, but are not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable Compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disk storage, magnetic cartridge, magnetic tape, magnetic disk storage or other magnetic storage device, or any other device that can be used to store desired information and can be accessed by a computer any other medium. In addition, as is well known to those of ordinary skill in the art, communication media typically embodies computer-readable program instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery medium.

Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or Source or object code written in any combination, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as via the Internet using an Internet service provider). connect). In some embodiments, an electronic circuit, such as a programmable logic circuit, field programmable gate array (FPGA), or programmable logic array (PLA), can be customized by utilizing state information of computer-readable program instructions, which can Various aspects of the present disclosure are implemented by executing computer readable program instructions.

The computer program products described here can be specifically realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. .

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It should be understood that each block of the flowcharts and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable user portrait generating device, thereby producing a machine that makes these instructions When executed, means for implementing the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams are produced. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions make the computer, the programmable user portrait generating device and/or other devices work in a specific way, so that the computer-readable medium storing the instructions is then An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in the flowcharts and/or block diagrams is included.

Computer-readable program instructions can also be loaded onto computers, other programmable user portrait generating devices, or other devices, so that a series of operation steps are executed on the computer, other programmable user portrait generating devices, or other devices to generate computer-implemented process, so that instructions executed on computers, other programmable user portrait generating devices, or other devices implement the functions/actions specified in one or more blocks in the flowchart and/or block diagram.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

Example embodiments have been disclosed herein, and while specific terms have been employed, they are used and should be construed in a generic descriptive sense only and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that features, characteristics and/or elements described in connection with a particular embodiment may be used alone, or may be described in combination with other embodiments, unless explicitly stated otherwise. Combinations of features and/or elements. Accordingly, it will be understood by those of ordinary skill in the art that various changes in form and details may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims (10)

1. A user portrayal generation method, applied to an electronic device deployed with a database cluster, the method comprising:

the method comprises the steps of obtaining target user data to be processed in a streaming mode, wherein the target user data comprise user data generated by service application in real time;

obtaining preconfigured tag information and crowd-sourced information, wherein the tag information comprises information of at least one tag, each tag represents a user characteristic of a user, the crowd-sourced information comprises information of at least one crowd-sourced, and each crowd-sourced comprises a plurality of different tags;

According to the label information and the crowd pack information, portrait is carried out on the target user data, and user portrait data are obtained; the method comprises the steps of,

and in the process of portraying the target user data, acquiring the running state of the back-end node of the database cluster, and carrying out operation and maintenance management on the back-end node according to the running state.

2. The method of claim 1, wherein the streaming acquisition of target user data to be processed comprises:

detecting data change information of user data in a target database, wherein the target database is used for storing the user data generated by service application;

obtaining a changed data record in the target database according to the data change information;

writing the changed data record into a target message queue;

and obtaining the target user data in a streaming mode according to the changed data record in the target message queue.

3. The method of claim 2, wherein said streaming said target user data based on said changed data record in said target message queue comprises:

And importing the changed data record in the target message queue into a corresponding data table of the database cluster in a streaming data importing mode to obtain the target user data.

4. The method of claim 3, wherein the portraying the target user data based on the tag information and the crowd-sourced information to obtain user portrayal data comprises:

obtaining a target portrait rule according to the label information and the crowd-sourced information, wherein the target portrait rule is a structured query language statement which supports the back-end node execution of the database cluster;

and carrying out portrayal on target user data imported into the data table by a rear end node of the database cluster according to the target portrayal rule to obtain the user portrayal data, wherein the user portrayal data comprises personal portrayal data and/or guest group portrayal data.

5. The method of claim 4, wherein the obtaining the target representation rule based on the tag information and the crowd-sourced information comprises:

generating rule conditions corresponding to the tag information and the crowd pack information, wherein the rule conditions are used for screening user data;

And converting the rule conditions based on the preset rules of the database cluster to obtain the target portrait rule.

6. The method of claim 1, wherein the obtaining the operation state of the backend node of the database cluster and performing operation and maintenance management on the backend node according to the operation state comprises:

detecting the running state of the back-end node of the database cluster according to a preset time interval;

and restarting the back-end node under the condition that the back-end node is determined to be abnormally operated.

7. The method of claim 1, wherein the obtaining the operation state of the backend node of the database cluster and performing operation and maintenance management on the backend node according to the operation state comprises:

detecting the running state of the back-end node of the database cluster according to a preset time interval;

acquiring an operation index corresponding to the database cluster under the condition that the operation of the back-end node is abnormal;

generating an anomaly analysis report according to the operation index;

and sending the anomaly analysis report to target terminal equipment, wherein the anomaly analysis report is used for performing performance optimization processing on the database cluster by a service user, the service user corresponds to the database cluster, and the target terminal equipment comprises terminal equipment used by the service user.

8. A user portrayal generation apparatus for use in an electronic device having a database cluster deployed therein, the apparatus comprising:

the system comprises a user data acquisition unit, a service application and a processing unit, wherein the user data acquisition unit is used for acquiring target user data to be processed in a streaming mode, and the target user data comprises user data generated by the service application in real time;

a configuration information obtaining unit, configured to obtain preconfigured tag information and crowd-sourced information, where the tag information includes information of at least one tag, each tag represents a user feature of a user, the crowd-sourced information includes information of at least one crowd-sourced, and each crowd-sourced includes a plurality of different tags;

the portrait unit is used for portrait the target user data according to the label information and the crowd pack information to obtain user portrait data; the method comprises the steps of,

and the daemon unit is used for acquiring the running state of the back-end node of the database cluster in the process of portraying the target user data, and carrying out operation and maintenance management on the back-end node according to the running state.

9. An electronic device, comprising:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,,

the memory stores one or more computer programs executable by the at least one processor to enable the at least one processor to perform the user representation generation method of any one of claims 1-7.

10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the user portrayal generation method according to any one of claims 1-7.

Images