WO2022242678A1 - Short link generation method and apparatus, device, and storage medium - Google Patents

Abstract

The present application discloses a short link generation method. The method comprises: acquiring a target number segment, the minimum value of the target number segment being the same as the maximum value of a previous number segment of the target number segment, the maximum value of the target number segment being the same as the minimum value of a next number segment of the target number segment, and the target number segment comprising a first number of numbers; expanding the minimum value and the maximum value of the target number segment to obtain an extended target number segment, the extended target number segment comprising a second number of numbers, and the second number being greater than the first number; determining a target number from the second number of numbers; and converting the target number to determine a short link. In addition, the present application further discloses a short link generation apparatus, an electronic device, and a storage medium.

Description

Short link generation method, device, equipment and storage medium

Cross References to Related Applications

This application is based on a Chinese patent application with application number 202110548216.1 and a filing date of May 19, 2021, and claims the priority of this Chinese patent application. The entire content of this Chinese patent application is hereby incorporated into this application by reference.

technical field

The present application relates to the field of short link processing, and relates to but not limited to a method, device, device and storage medium for generating a short link.

Background technique

When the competition among e-commerce companies is becoming increasingly fierce, in order to simplify the content interface and improve user experience, e-commerce companies usually convert long links into short links.

At present, e-commerce companies usually use Message-Digest Algorithm (MD5) to segment encrypted strings, and then combine shift operations and modulo operations to convert long links into short links. The process of forming a short link is relatively long, therefore, the generation speed of the short link in the prior art is slow.

Contents of the invention

In order to solve at least one problem in the related art, the present application provides a method, device, device and storage medium for generating short links, which can increase the speed of generating short links.

The technical scheme of the present application is realized like this:

In the first aspect, the embodiment of the present application provides a method for generating a short link, the method comprising:

Obtain the target number segment; the minimum value of the target number segment is the same as the maximum value of the previous number segment of the target number segment, and the maximum value of the target number segment is the minimum value of the subsequent number segment of the target number segment The values are the same, and the target number segment includes the first number of numbers;

Extending the minimum value and the maximum value of the target number segment to obtain an extended target number segment, the expanded target number segment includes a second number of numbers, and the second number is greater than the first number;

determining a target number from said second number of numbers;

The target number is converted to determine the short link.

In the second aspect, the embodiment of the present application provides a short link generating device, the device comprising:

An acquisition unit configured to acquire a target number segment; the minimum value of the target number segment is the same as the maximum value of the previous number segment of the target number segment, and the maximum value of the target number segment is the same as the maximum value of the target number segment The minimum value of a number segment is the same, and the target number segment includes the first number of numbers;

The processing unit is configured to extend the minimum value and maximum value of the target number segment to obtain an extended target number segment, the extended target number segment includes a second number of numbers, and the second number is greater than said first quantity;

a determination unit configured to determine a target number from said second number of numbers;

The determining unit is further configured to convert the target number to determine a short link.

In the third aspect, the embodiment of the present application provides an electronic device, a memory, a processor, and a computer program stored on the memory and operable on the processor, and the above-mentioned short link generation method is realized when the processor executes the computer program .

In a fourth aspect, the embodiment of the present application provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method for generating a short link is realized.

Description of drawings

Fig. 1 is a kind of short link generating system provided by the embodiment of the present application;

FIG. 2 is an optional schematic flowchart of a method for generating a short link provided in an embodiment of the present application;

FIG. 3 is an optional schematic flowchart of a method for generating a short link provided in an embodiment of the present application;

FIG. 4 is a schematic flow diagram of an optional hot backup provided by the embodiment of the present application;

FIG. 5 is a schematic diagram of the principle of the signal generator provided by the embodiment of the present application;

Figure 6 is a schematic diagram of the principle of obtaining a number by the number issuer provided in the embodiment of the present application

FIG. 7 is a schematic flow diagram of an optional storage and hot backup short link provided by the embodiment of the present application;

FIG. 8 is a schematic structural diagram of an optional short link generation device provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an optional electronic device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the specific technical solutions of the application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following examples are used to illustrate the present application, but not to limit the scope of the present application.

Embodiments of the present application may provide a short link generation method and device, a device (such as an electronic device) and a storage medium (such as a computer-readable storage medium). In practical applications, the method for generating a short link can be realized by using a short link generating device.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used herein in the specification of the application are only for the purpose of describing specific embodiments, and are not intended to limit the application. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

Before the present invention is further described in detail, the nouns and terms involved in the embodiments of the present invention are described, and the nouns and terms involved in the embodiments of the present invention are applicable to the following explanations.

1) A link refers to a connection relationship from a webpage to a target, where the target can be another webpage, or a different location on the same webpage, or a picture, a file, etc.

2) Short link, a category of links, wherein a short link usually consists of 7 characters or less than 7 characters.

3) Message Queue (MQ) is a queue that can transmit data.

The short link generating method of the embodiment of the present application can be applied to the short link generating system 100 shown in FIG. 1 . As shown in FIG. Wherein, the server 10 and the client 20 communicate through the network 30 . After determining the short link, the server 10 sends the determined short link to the client 20 through the network 30 .

Below, with reference to the schematic diagram of the short link generation system shown in FIG. 1 , various embodiments of the short link generation method, device, device, and storage medium provided by the embodiments of the present application will be described.

An embodiment of the present application provides a method for generating a short link, and the method is applied to a device for generating a short link. The functions realized by the short link generation device can be realized by the processor in the server calling the program code, of course the program code can be stored in the memory, it can be seen that the server at least includes a processor and a memory.

The present application will be further described in detail through the accompanying drawings and specific embodiments below.

Fig. 2 is a schematic diagram of the implementation flow of a short link generation method provided in the embodiment of the present application. The method is applied to the server, as shown in Fig. 2, and the method may include the following steps:

S201. The server acquires a target number segment.

Wherein, the minimum value of the target number segment is the same as the maximum value of the previous number segment of the target number segment, the maximum value of the target number segment is the same as the minimum value of the subsequent number segment of the target number segment, and the The target number segment includes a first number of numbers.

Each number segment includes a segment of continuous positive integers. For example, the first number is 10, and the number segment [0, 10] includes a segment of continuous positive integers from 0 to 10.

In the embodiment of the present application, the server may include: a central number segment generator and a code generator, wherein the central number segment generator is used to generate multiple number segments, and the code generator is used to obtain a number segment from the multiple number segments As the target number segment, the target number segment is expanded, and the expanded target number segment is sent to the number sender.

After the central number segment generator sends out a certain number of number segments, it will continue to generate new number segments. In an example, after the central number segment generator sends out ten number segments, for example, [0,10], ..., [90,100], the central number segment generator will continue to generate new number segments [100 , 110].

In the embodiment of the present application, the above S201 includes: the central number segment generator generates multiple number segments, and the code generator takes one number segment from the multiple number segments as the target number segment.

In an example, the multiple number segments generated by the central number segment generator include: 5 number segments are [0,10], [10,20], [20,30], [30,40], [40 , 60] these 5 number segments; the code generator takes a number segment from these 5 number segments, for example, number segment [10, 20], and uses this number segment [10, 20] as the target number segment. In the target number section [10, 20], including the number of the first quantity, such as 10 numbers, the previous number section of the target number section is [0, 10], wherein the minimum number of the number section [0, 10] The value is 0, and the maximum value is 10; the next number segment of the target number segment is [20, 30], wherein, the minimum value of the number segment [20, 30] is 20, and the maximum value is 30; among them, the target number segment The minimum value of is 10, which is the same as the maximum value 10 of the previous segment [0, 10], and the maximum value of the target segment is 20, which is the same as the minimum value 20 of the subsequent segment [20, 30].

S202. The server extends the minimum value and maximum value of the target number segment to obtain an extended target number segment, and the extended target number segment includes a second number of numbers.

Wherein, the second quantity is greater than the first quantity.

Here, the code generator expands the minimum value and maximum value of the target number segment, which may be multiplied by a coefficient for the minimum value of the target number segment, and multiplied by a coefficient for the maximum value of the target number segment, thereby obtaining the extended The target number segment after the expansion, the target number segment after the expansion includes the number of the second quantity, and the target number segment after the expansion is sent to the number sender.

It should be noted that the embodiment of the present application does not limit the values of the coefficients. For example, the factor can be 10000 or 20000.

In an example, the target number segment is [10, 20], the minimum value of the target number segment is multiplied by 10000, and the minimum value of the extended target number segment is 100000, and the maximum value of the target number segment is multiplied by 20 With a coefficient of 10000, the maximum value of the expanded target number segment is 200000, thus, the expanded target number segment is [100000, 200000], and the expanded target number segment includes numbers with a second quantity of 100000.

S203. The server determines a target number from the second number of numbers.

Here, the server may further include: a number sender, through which the target number is determined from the second number of numbers.

In an example, the number issuer selects a number from 100,000 numbers ranging from 100,000 to 200,000 as the target number, for example, selects 100,000 as the target number.

In the embodiment of the present application, the server may include multiple number issuers, wherein, when multiple number issuers apply to obtain the number segments at the same time, the orthogonality of the number segments obtained by the number issuers can be ensured through distributed locks, thereby ensuring The target number segments obtained by each of the multiple signal issuers are orthogonal.

When the number in the target number segment has been issued, the code generator can take the next number segment that can be issued from the central number segment generator.

S204. The server converts the target number to determine the short link.

Here, the target number is converted by the sender to determine the short link. Wherein, the short link can be a character string.

In an example, the target number is 10000, and the number sender converts 10000 to determine the short link.

A short link generation method provided in an embodiment of the present application is to obtain a target number segment, the minimum value of the target number segment is the same as the maximum value of the previous number segment of the target number segment, and the maximum value of the target number segment The same as the minimum value of the next number segment of the target number segment, the target number segment includes a first number of numbers; expanding the minimum and maximum values of the target number segment to obtain a second number of numbers; Determining a target number from the second number of numbers; converting the target number to determine a short link. In this way, in the process of determining the short link, since the target number segment can be determined from a plurality of number segments first, and then a target number can be determined from the target number segment, the target number is converted to determine the short link, therefore, it is possible to directly Converting the target number to determine the short link does not need to go through tedious steps to convert the long link into a short link, so that the link generation process can be shortened, and the short link generation speed can be improved.

In some embodiments, the above S201 includes:

S201a. The server determines whether the number of numbers cached locally is smaller than a preset threshold.

It should be noted that, the embodiment of the present application does not limit the value of the preset threshold. For example, the preset threshold may be 10000 or 20000.

S201b. If the number of numbers cached locally is less than or equal to the preset threshold, acquire a target number segment.

In this embodiment of the application, before obtaining the target number segment, the server needs to first determine whether the number of locally cached numbers is less than the preset threshold. If the number of locally cached numbers is less than or equal to the preset threshold, it means that the original locally cached The number of numbers is small. At this time, it is necessary to obtain the target number segment, expand the obtained target number segment, and determine the target number from the expanded target number segment, so as to convert the target number and determine the short link.

In some embodiments, the method for generating a short link provided in the embodiment of the present application further includes: if the number of locally cached numbers is greater than a preset threshold, the server determines the target number from the locally cached numbers.

In this embodiment of the application, if the number of locally cached numbers is greater than the preset threshold, it means that the number of original locally cached numbers is large, and the server can directly determine the target number from the locally cached numbers. Obtain the target number section again, and expand the target number section, and then determine the target number from the expanded target number section.

In some embodiments, the above S204 includes:

S204a. The server performs a modulus on the target number to obtain a modulo result.

Wherein, the modulus result includes: at least one modulo parameter.

Here, the server performs modulo acquisition of the target number, which may be modulo acquisition of the target number to a modulo coefficient.

It should be noted that the embodiment of the present application does not limit the modulo coefficient. For example, the modulo factor can be 62 or other numbers.

In an example, the target number is 10000, and the target number 10000 is moduloed to 62 to obtain, wherein, the obtained modulus result includes: at least one modulo parameter, and the at least one modulo parameter includes: 18, 37, 2, 0, 0, 0.

S204b. The server determines a character corresponding to each of the modulo parameters in the at least one modulo parameter.

Wherein, at least one of the characters constitutes the short link.

Here, the character corresponding to each modulus parameter in the at least one modulus parameter can be determined according to the correspondence table formed by the correspondence between the modulus parameter and the character. Wherein, the correspondence table may be as shown in Table 1.

In the correspondence table shown in Table 1, there are a total of 62 characters including a-z, A-Z, and 0-9, and each character has a corresponding modulo parameter. Among them, the first line is a part of the modulus parameters, the second line is the character corresponding to the first line of the modulo parameter, the third line is a part of the modulus parameter, and the fourth line is the character corresponding to the third line of the modulo parameter, and so on.

Table 1. Example of correspondence table

0 1 2 3 4 5 6 7 8 9 N 1 4 Z C E. L f 0 c 10 11 12 13 14 15 16 17 18 19 Y V Q the s 7 t P o 2 8 20 twenty one twenty two twenty three twenty four 25 26 27 28 29 K j 9 u i x v S D. T 30 31 32 33 34 35 36 37 38 39

B m w r G k R f u a 40 41 42 43 44 45 46 47 48 49 3 d 5 B W l N I z x 50 51 52 53 54 55 56 57 58 59 A m Y q J G h h e o 60 61 the the the the the the the the 6 P the the the the the the the the

In an example, the target number is 10000, and 10000 is modulo 62, and the first modulus parameter obtained is 18. By looking up Table 1, the character corresponding to 18 can be obtained as 2; the second modulo parameter obtained is is 37, by looking up table 1, the character corresponding to 37 can be obtained as f; the obtained third modulus parameter is 2, and by looking up table 1, the character corresponding to 2 can be obtained as 4; the obtained fourth modulo parameter is 0, by looking up table 1, the character corresponding to 0 can be obtained as n; the obtained fifth modulus parameter is 0, and by looking up table 1, the character corresponding to 0 can be obtained as n; the obtained sixth modulo parameter is 0, by looking up table 1, the character corresponding to 0 can be obtained as n. Since the high bit is on the left, the character determined by the server corresponding to the 6 modulo parameters is nnn4f2.

It should be noted that the embodiments of the present application do not limit the characters included in the correspondence table, and may also be replaced by other characters according to business scenarios.

In the embodiment of the present application, the method of inserting visual interference bits and filtering strong regular character strings can be used, so that the character strings determined in the server are irregular, so that the terminal cannot obtain the generation method of the short link.

In an example, inserting a visual noise bit is inserting a visual noise bit in the determined character string. For example, the character "Q" is inserted into the second position of the determined character string "nnn4f2" to become "nQnn4f2".

In an example, the strong regular character string is filtered so that the character strings with consistent multiple digits such as "AAAAAA" and "AAAAAB" with strong regularity can be filtered and not sent to the terminal.

In the embodiment of this application, in order to make the terminal unable to obtain the short link generation method, in addition to the above-mentioned method of inserting visual interference bits and strong regular character string filtering, a distributed scenario can also be used, so that the acquired target numbers are different numbers In this way, since the obtained target numbers are numbers in different number segments, the difference between the obtained character strings will be relatively large, thereby making it difficult for the terminal to find the generation rules of the character strings.

In some embodiments, after the above S201, the method further includes:

S205. The server acquires a sorting policy.

Here, the sorting strategy can be a sorting strategy for numbers in random order, or a sorting strategy for numbers from large to small, or a sorting strategy for numbers from small to large, which is not limited in the embodiment of the present application .

S206. The server sorts the first number of numbers included in the target number segment based on the sorting policy.

In an example, if the sorting strategy is a sorting strategy for numbers in random order, the server may sort the first number of numbers included in the target number segment in random order.

In this way, sorting the numbers of the first quantity based on the sorting strategy can make the numbers after sorting different from the numbers before sorting, so that the sorting of the numbers of the first quantity included in the target number segment can be irregular, and then can make The terminal cannot obtain the generation method of the short link.

In some embodiments, after the above S204, the method further includes:

S207. The server stores the short link once, and stores it in the first storage area.

Wherein, the first storage area is used to realize high-speed access to the short link.

Here, the server stores the short link once, that is, the server stores the short link in the first storage area, wherein the first storage area can be a cache area, or a memory such as a remote dictionary service (Remote Dictionary Server, Redis) database, etc. The database is not limited in this embodiment of the present application.

In the embodiment of the present application, storing the generated short link, for example, in Redis, can make the generated short link persistent.

S208. The server performs secondary storage on the short link, and stores it in a second storage area.

Wherein, the second storage area is used to realize the backup of the short link, for example, hot backup.

Here, the secondary storage of the short link by the server is the hot backup of the short link by the server, and the hot backup of the short link to the second storage area, wherein the second storage area may be a storage area based on a distributed storage system, for example, It is a Hadoop-based database (Hadoop Database, HBase) database and a data warehouse tool (hive) database. Among them, the HBase database has the characteristics of low cost, excellent performance, and easy expansion. It also supports fast insertion and query operations, which solves the system bottleneck that the relational database (MYSQL database) cannot adapt to massive data access. The Hive database is used to store the full amount of data.

In this way, the short link is stored once, that is, the short link is stored in the cache, and high-speed access to the short link can be realized. After the short link is stored in the cache area, the full amount of short links can be stored twice, that is, Links are stored in disk, so that the security of full short links can be achieved.

When the terminal accesses the short link, it can first obtain the short link from the cache, and then obtain the short link from the disk if the short link is not obtained from the cache.

In the embodiment of the present application, the least recently used (Least Recently Used, LRU) algorithm may be used to clean up infrequently used short links in the cache.

Here, the basis for judging whether the short link in the cache is an infrequently used short link may be: the storage time of the short link, and/or the number of visits to the short link.

If the storage time of the short link is greater than the preset storage time, it is determined that the short link is an uncommon short link; if the storage time of the short link is less than or equal to the preset storage time, it is determined that the short link is a commonly used short link.

If the number of visits to the short link is greater than the preset number of visits, it is determined that the short link is a commonly used short link; if the number of visits to the short link is less than or equal to the preset number of visits, it is determined that the short link is an uncommon short link.

In some embodiments, after the above S207, the method further includes:

S209. The server sends the short link to the message queue MQ.

Here, the server may further include: a message queue client, which controls the sending of the short link to the message queue server, where sending the short link to the message queue server can be regarded as sending the short link to the message queue MQ.

After the server sends the short link to the message queue, the above S208 includes:

S208a. The server consumes the short link in the message queue, and stores the short link in the first database and the second database respectively.

Wherein, the first database and the second database are different. In an example, the first database may be an HBase database, and the second database may be a Hive database.

In the embodiment of this application, the short link can be sent to the message queue, and then the short link can be stored in the first database and the second database through the message queue, so that the asynchronous decoupling of short link storage and hot backup can be realized, thereby shortening the Short link storage and hot backup time.

In some embodiments, the above S209 includes:

S209a. The server judges whether the network of the message queue is abnormal.

Here, there is a maximum waiting time set in the configuration file of the message queue client. When the message queue client controls the short link to be sent to the message queue, if the message queue client has not received the message queue acknowledgment beyond the maximum waiting time If there is a reply to the short link, then the message queue client will judge that the network of the message queue is abnormal.

S209b. If the network of the message queue is abnormal, send the short link to the reissue message list through the reissue message thread, and resend the short link in the reissue message list to the message queue.

In the embodiment of this application, if the network of the message queue is abnormal, it means that the server cannot send the short link to the message queue. At this time, the short link needs to be sent to the reissue message list msgList through the reissue message thread, and then the reissue message list The short links in msgList are resent to the message queue.

Among them, the reissue message list msgList is a first-in-first-out queue implemented by Redis. It mainly stores short links that need to be reissued through the reissue message thread. The short link is saved by the LPUSH instruction, and the short link is obtained by the RPOP instruction. The sequence and efficiency of short link execution are guaranteed, and the time complexity of short link execution is O(1). Wherein, the time complexity O(1) indicates that the number of times of saving the short link through the LPUSH instruction, and the number of times of obtaining the short link through the RPOP instruction is a constant.

The resend message thread will regularly scan the resend message list msgList, and resend the short links that failed to be sent.

In an example, in the process of sending the first short link to the message queue, if the network of the message queue is abnormal, the first short link cannot be sent to the message queue. A short link is sent to the reissued message list, and then the first short link in the reissued message list is resent to the message queue.

In some embodiments, after the server sends the short link to the reissue message list through the reissue message thread, the method further includes:

S210. The server backs up the short links in the reissued message list to the backup message list through the backup message thread.

In the embodiment of this application, the backup message thread is used to back up the short link obtained by the reissue message thread, and the backup message thread will regularly scan the backup message list backUpList, and put the messages that have not been deleted for a long time into the reissue message list msgList , the backup message list backUpList is used to save the short link obtained through the backup message thread.

In an example, the server may atomically back up the short link in the reissue message list to the backup message list backUpList through the BRPOPLPUSH command of Redis. Wherein, atomically backing up the short link to the backup message list means backing up the integrity of the short link to the backup message list.

S211. The server judges whether the short link in the resent message list is resent to the message queue.

S212. If the short link in the reissue message list is resent to the message queue, delete the short link in the backup message list and the resend message queue in the reissue message list.

S213. If the short link in the reissued message list is not resent to the message queue, reserve the short link in the backup message list corresponding to the reissued message list, and when the reserved time is longer than the preset time, resend the short link in the backup message list to the reissue message list.

In an example, the reissued message list msgList includes the second short link and the third short link, and the server backs up the second short link and the third short link to the backup message list backUpList through the backup message thread. At this time, the backup message list backUpList includes: the second short link and the third short link. If the second short link in the reissued message list msgList is resent to the message queue, the second short link in the backup message list backUpList will be deleted; if the reissued message If the third short link in the list msgList is not resent to the message queue due to network abnormality, the third short link in the backup message list backUpList will be reserved. When the retention time is longer than the preset time length, the backup message list backUpList will be The third short link of is resent to the resend message list

msgList.

In the short link generation method provided by the embodiment of the present application, as shown in FIG. 3 , the terminal 20 sends a short link generation request to the server 10, and the server 10 sends the short link to the terminal 20 based on the short link generation request sent by the terminal. Wherein, the server can complete the generation of the short link and the storage and hot backup of the short link by performing step A: generating the short link, step B: storing the short link, and step C: hot backing up the short link. These three functions will be described in detail below.

Step A, generating a short link.

Wherein, step A may include:

Step A1, the code generator acquires the target number segment.

In the embodiment of the present application, the server includes: a central number segment generator, a code generator, and a number issuer. Wherein, the central number segment generator is used to generate a certain number of number segments, for example, 10 number segments, and each number segment is a number segment composed of a continuous positive integer.

In one example, the central number segment generator generates 10 number segments, for example, [0,10], [10,20], [20,30], ..., [m,k], [k,k+ 10], [k+10, n], ..., [90, 100], wherein, k, m and n are positive integers).

The code generator is used to take a number segment as a target number segment from a certain number of number segments generated by the central number segment generator, expand the target number segment, and send the expanded target number segment to the number sender.

In one example, the code generator selects [k, k+10] from the 10 number segments [0, 10], [10, 20], [20, 30], ..., [k, k+10] ] as the target number segment.

In an example, the target number segment is [k, k+10], and the target number segment is extended to obtain the extended target number segment as [k*10000, (k+10)*10000-1].

The number sender is used to select a number from the expanded target number segment as the target number.

In step A2, the number issuer generates a short link through a short link generation algorithm.

Here, for the description of generating the short link through the short link generation algorithm by the number issuer, please refer to the above description of S204a and S204b, which will not be repeated here.

In the embodiment of this application, the central number segment generator is used to generate number segments, and the code generator is used to issue number segments. Using the central number segment generator and code generator can decouple number segment generation and issuance, effectively sharing the central number segment generation The service pressure of the server, the code generator depends on the central number segment generator, and the terminal obtains the string through the number generator. In this way, the service can have better horizontal scalability. When the number of visits is large, only horizontal expansion of the number is required. The server can improve the throughput of the service. This design can support hundreds of millions of visits per day.

Step B, store the short link.

Here, for the explanation of step B, refer to the explanation of S207 and S208 in the above-mentioned embodiment for details, which will not be repeated here.

Because in the prior art, the data is stored in the cache first, and then the data is stored in the disk, so that one storage operation requires two storage time overheads, therefore, in the embodiment of this application, the message queue is used to resolve the two storage operations. Coupling, for example, after storing the data in the cache, put the data directly into the message queue, and then the downstream data processing system of the message queue receives the data and writes the data to the disk. In this way, the time overhead of 2 times can be shortened to 1 time .

Step C, performing hot backup on the short link.

In the embodiment of the present application, the realization principle of the hot backup short link is mainly realized through the message queue, as shown in Figure 4, in the process of hot backup of the short link, the following steps are included:

S401. The server sends the short link to the message queue MQ.

S402. The message queue MQ receives the short link sent by the server, and stores the short link in the HBase database and the Hive database respectively.

In some embodiments, FIG. 5 is a schematic diagram of the realization principle of the signal generator. As shown in FIG. 5 , the functions performed by the signal device include: a first function D, a second function E, and a third function F; wherein, the first Function D is the function of obtaining numbers; the second function E is the function of number loading; the third function F is the function of disaster recovery and fault recovery. The following will explain these functions in detail:

As shown in Figure 5, in the process of realizing the function of obtaining the number, the following steps are included:

S501. The number issuer judges whether the number of locally cached numbers is greater than a preset threshold;

S502. If it is greater than the preset threshold, obtain the target number from the locally cached numbers;

S503. If it is less than or equal to the preset threshold, submit the number loading task to the thread pool.

After submitting the number loading task to the linear pool, the number issuer can load the number asynchronously.

As shown in Figure 5, in the process of realizing the number loading function, the following steps are included:

S504. The signal transmitter acquires the minimum value X in the target number segment;

S505. The signal sender expands the minimum value and maximum value in the target number segment.

In an example, the target number segment after extending the minimum value and the maximum value in the target number segment may be: [X*100000,

(X+10)*100000];

S506, the number generator generates all numbers;

S507, the number generator disrupts the sequence of all numbers;

S508. The number issuing device saves all numbers to a local cache;

S509. The number issuer submits the disaster recovery task to the linear pool.

After the issuer submits the disaster recovery task to the linear pool, the issuer can perform asynchronous disaster recovery.

In the process of disaster recovery and failure recovery, the following steps are included:

S510, the backup thread obtains the latest number value from Redis;

S511. Obtain the latest number value from the disaster recovery database MySql;

S512, judging whether the latest number value obtained in Redis is smaller than the latest number value obtained in the disaster recovery database MySql;

S513. If the latest number value in Redis is greater than or equal to the latest number value in the disaster recovery database MySql, it means that the number value in Redis has not been deleted or tampered with;

In the case that the number value in Redis has not been deleted or tampered with, the number value in Redis can be updated to the disaster recovery database MySql.

S514. If the latest number value in Redis is smaller than the latest number value in the disaster recovery database MySql, it means that the number value in Redis has been deleted or tampered with;

S515. Monitor the Redis and send an alarm to relevant personnel, and recover the fault.

In an example, the fault can be recovered by performing a rollback operation, the rollback operation is setting the latest number value in Redis to the number value of the disaster recovery database MySql+1000.

By decoupling the four parts of number acquisition, number loading, disaster recovery, and fault recovery through asynchronous programming, automatic disaster recovery, fault detection, and fault recovery can be realized. The number sending performance is less than 1ms, and due to the The number pools between are completely isolated. Therefore, the number issuer provided by the embodiment of the present application has the advantages of high concurrency, high performance, and high reliability.

As shown in Figure 6, the central number segment generator can generate multiple number segments and send multiple number segments to the number issuer. The number issuer can obtain the number segment from the central number segment generator, and the number issuer can obtain the number After the segment, the digits of the number segment will be scrambled, and the sequence will be stored locally. Every time the number that can be used is taken out from the local storage as needed, after the number is taken out, it will be discarded and cannot be issued to the second terminal again. Out of order can make the terminal feel that the number segment is randomly generated when acquiring data. For example, the number segment obtained from the central number segment generator is [0, 9], then the local storage may be [7, 6, 0, 5, 2, 3, 4, 1, 8, 9] (random storage, also possibly another sequence).

The short link generation algorithm will be explained in detail below.

The number issuer ensures high-performance access to unique numbers, and what the short link generation algorithm needs to do is to ensure the unique mapping between the number and the short link. By ensuring the uniqueness of the short links obtained each time, we can reduce the number of operations on Redis and greatly reduce the time overhead caused by operating Redis, thereby improving the performance of short link generation.

The idea adopted by the short link generation algorithm is to convert decimal numbers into hexadecimal numbers, and use the modulus mapping method to generate a string with the same number of digits as the number of modulus, so as to realize the controllability of the length of the random string generation.

In the embodiment of the present application, the short link generation algorithm includes the above-mentioned S204a and S204b. For specific explanations, please refer to the explanations of S204a and S204b in the above-mentioned embodiments, which will not be repeated here.

The storage and hot backup short links are explained in detail below.

In the embodiment of the present application, the processes of storing and hot backup short chains are asynchronously decoupled, so that system problems such as excessively long existing processes and non-core processes affecting the performance of the main process can be solved.

After the short link is generated, the short link and related data are stored in the distributed memory database Redis, and then the data is put into middleware such as message queue MQ, and the subsequent data backup is performed by the corresponding hot backup process. By introducing MQ, the asynchronous decoupling of storage and hot backup of short links can be realized, so that the efficiency of short link generation can be realized.

As shown in Figure 7, the short link storage and hot backup mainly include two parts: asynchronous sending MQ message mechanism and failure message recovery mechanism.

Among them, in the asynchronous sending MQ message mechanism, including:

S701. The number generator generates a short link;

S702. Store the short link through Redis;

S703. The sender sends the short link to the message queue MQ;

S704, the signal generator judges whether the MQ network is abnormal;

S705. If abnormal, send the short link to the reissue message list msgList;

S706. If there is no abnormality, send the short link to the message queue MQ.

After sending the short link to the message queue MQ, the sender sends the short link to the HBase database and the Hive database respectively.

In the failure message recovery mechanism, the short link that has not been sent to the message queue due to MQ network abnormality can be resent by reissuing the message thread, and the short link in the reissued message thread can be backed up by backing up the message thread. Among them, the backup message thread includes:

S707. Back up the short link in the reissued message list by backing up the message thread;

S708. Determine whether the creation time exceeds a preset time;

S709. If the preset time is exceeded, resend the unexecuted short link exceeding the preset time to the reissue message list msgList;

In the embodiment of this application, the asynchronous sending MQ message mechanism is mainly realized by means of the client (Client) related to MQ, and the sending result will call the callback function: the short link sent successfully can be written into HBase and Hive asynchronously; The short link can be saved to the resend message list msgList. The failure message remediation mechanism is mainly used to solve the remediation when the MQ network is abnormal. The MQ network anomaly will have a catastrophic impact on the system and will cause the overall short-chain service to be unavailable for a period of time. Therefore, we use it when the MQ network is abnormal. Fail fast strategy. The fail-fast strategy can quickly give users feedback results without affecting the core process.

The reissued message list msgList and the backup message list backUpList are a first-in-first-out queue implemented using Redis. Messages are saved through the LPUSH instruction and obtained through the RPOP instruction, thereby ensuring the order and efficiency of message execution, and the execution time of messages is complex. The degree is O(1). The reissued message list msgList mainly stores which messages need to be reissued through the reissued message thread; the backup message list backUpList mainly stores which reissued messages have been obtained by the reissued message thread, that is, the messages in the reissued message list are backup.

The resend message thread regularly scans the resend message list msgList, and resends the failed message. For example, when the reissued message list msgList is found to have message message1, the reissued message thread will delete message1 from the reissued message list msgList through the Redis BRPOPLPUSH command and atomically save it to the backup message list backUpList. If the resend message thread successfully sends MQ, delete message1 from the backup message list again; if the resend message thread crashes or fails to send MQ, it will not be processed.

The backup message thread regularly scans the backup message list backUpList, and puts the messages that have not been deleted for a long time into the reissued message list msgList. For example, if the above message1 crashes after being retrieved by the reissued message thread, message1 will not be deleted from the backup message list. At this time, the backup message thread will obtain message1. If the message creation time of message1 is greater than the set time threshold, for example , 30 minutes, the backup message thread will delete message1 from the backup message list backUpList through the Redis BRPOPLPUSH command and atomically save it to the reissue message list msgList.

In the embodiment of this application, the performance can be improved from the previous 17ms to 3ms by introducing a signal generator, optimized generation algorithm, asynchronous decoupled storage, and hot backup.

FIG. 8 is a short link generation device provided by the embodiment of the present application. As shown in FIG. 8, the short link generation device 800 includes:

The acquiring unit 801 is configured to acquire a target number segment; the minimum value of the target number segment is the same as the maximum value of the previous number segment of the target number segment, and the maximum value of the target number segment is the same as the target number segment The minimum value of the latter number segment is the same, and the target number segment includes the first number of numbers;

The processing unit 802 is configured to extend the minimum value and the maximum value of the target number segment to obtain an extended target number segment, and the extended target number segment includes a second number of numbers, and the second number is greater than said first quantity;

A determining unit 803 configured to determine a target number from the second number of numbers;

The determining unit 803 is further configured to convert the target number to determine a short link.

In some embodiments, the acquiring unit 801 is further configured to determine whether the number of locally cached numbers is less than a preset threshold; if the number of locally cached numbers is less than or equal to the preset threshold, acquire the target number segment.

In some embodiments, the determining unit 803 is further configured to determine a target number from the locally cached numbers if the number of the locally cached numbers is greater than the preset threshold.

In some embodiments, the determining unit 803 is further configured to take a modulo on the target number to obtain a modulo result, the modulo result including: at least one modulo parameter; determining the at least one modulo parameter Each of the characters corresponding to the modulo parameter; at least one of the characters constitutes the short link.

In some embodiments, the device for generating short links further includes: a sorting unit; the acquiring unit 801 is also configured to acquire a sorting strategy; and the sorting unit is configured to sort the first number of the target number segments included in the target number segment based on the sorting strategy number to sort.

In some embodiments, the device for generating a short link further includes: a storage unit; the storage unit is configured to store the short link once, and store it in the first storage area; the first storage area is used to realize the storage of the short link The high-speed access of the short link; the storage unit is also configured to store the short link twice in a second storage area; the second storage area is used to realize the backup of the short link.

In some embodiments, the short link generation device further includes: a sending unit; the sending unit is configured to send the short link to a message queue; a storage unit is also configured to consume the short link in the message queue , respectively storing the short link in a first database and a second database, where the first database is different from the second database.

In some embodiments, the sending unit is further configured to determine whether the network of the message queue is abnormal; if the network of the message queue is abnormal, send the short link to the reissue message list through the reissue message thread, and Resend the short links in the resent message list to the message queue.

In some embodiments, the processing unit is further configured to back up the short link in the reissued message list to the backup message list through the backup message thread; determine whether the short link in the reissued message list is resent to the Message queue; if the short link in the reissued message list is resent to the message queue, delete the short link in the backup message list and the resent message queue in the reissued message list; if the If the short link in the reissued message list is not resent to the message queue, then the short link corresponding to the reissued message list in the backup message list is reserved, and when the retention time is longer than the preset time length, then Resend the short link in the backup message list to the reissue message list.

The embodiment of the present application also provides an electronic device, including a memory and a processor, the memory stores a computer program that can run on the processor, and the processor implements the short link provided in the above embodiment when executing the program generate method. Wherein, the electronic device may be a client or a server.

The embodiment of the present application also provides a storage medium, that is, a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for generating a short link provided in the above-mentioned embodiment is implemented.

It should be pointed out here that: the descriptions of the above storage medium and device embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects to those of the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

It should be noted that FIG. 9 is a schematic diagram of a hardware entity of an electronic device according to an embodiment of the present application. As shown in FIG. 9 , the electronic device 900 includes: a processor 901, at least one communication bus 902, and at least one external communication interface 904 and memory 905. Wherein, the communication bus 902 is configured to realize connection and communication among these components. In an example, the electronic device 900 further includes: a user interface 903, wherein the user interface 903 may include a display screen, and the external communication interface 904 may include a standard wired interface and a wireless interface.

The memory 905 is configured to store instructions and applications executable by the processor 901, and can also cache data to be processed or processed by the processor 901 and various modules in the electronic device (for example, image data, audio data, voice communication data and video data) Communication data), which can be realized by flash memory (FLASH) or random access memory (Random Access Memory, RAM).

It should be understood that reference throughout the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present application. Thus, appearances of "in one embodiment" or "in some embodiments" throughout this specification are not necessarily referring to the same embodiments. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, and should not be used in the embodiments of the present application. The implementation process constitutes any limitation. The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or May be integrated into another system, or some features may be ignored, or not implemented. In addition, the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms of.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed to multiple network units; Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be used as a single unit, or two or more units can be integrated into one unit; the above-mentioned integration The unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

Those of ordinary skill in the art can understand that all or part of the steps to realize the above method embodiments can be completed by hardware related to program instructions, and the aforementioned programs can be stored in computer-readable storage media. When the program is executed, the execution includes The steps of the foregoing method embodiments; and the foregoing storage media include: removable storage devices, read-only memory (Read Only Memory, ROM), magnetic disks or optical disks and other media that can store program codes.

Alternatively, if the above-mentioned integrated units of the present application are realized in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solutions of the embodiments of the present application or the part that contributes to related technologies can be embodied in the form of software products. The computer software products are stored in a storage medium and include several instructions to make A computer device (which may be a personal computer, a server, or a network device, etc.) executes all or part of the methods described in the various embodiments of the present application. The aforementioned storage medium includes various media capable of storing program codes such as removable storage devices, ROMs, magnetic disks or optical disks.

The above is only the embodiment of the present application, but the scope of protection of the present application is not limited thereto. Anyone familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, and should covered within the scope of protection of this application. Therefore, the protection scope of the present application should be determined by the protection scope of the claims.

The above description is only a preferred embodiment of the present invention, and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the within the protection scope of the present invention.

Claims (20)

A method for generating short links, the method comprising: Obtain the target number segment; the minimum value of the target number segment is the same as the maximum value of the previous number segment of the target number segment, and the maximum value of the target number segment is the minimum value of the subsequent number segment of the target number segment The values are the same, and the target number segment includes the first number of numbers; Extending the minimum value and the maximum value of the target number segment to obtain an extended target number segment, the expanded target number segment includes a second number of numbers, and the second number is greater than the first number; determining a target number from said second number of numbers; The target number is converted to determine the short link. The method according to claim 1, wherein said obtaining the target number segment comprises: Judging whether the number of locally cached numbers is less than a preset threshold; If the number of numbers cached locally is less than or equal to the preset threshold, the target number segment is acquired. The method according to claim 2, wherein the method further comprises: If the number of the locally cached numbers is greater than the preset threshold, then determine a target number from the locally cached numbers. The method according to claim 1, wherein said converting said target number and determining a short link comprises: Carrying out modulo taking on the target number to obtain a modulo taking result, the modulo taking result including: at least one modulo taking parameter; Determining characters corresponding to each of the modulo parameters in the at least one modulo parameter; at least one of the characters forms the short link. The method according to claim 1, wherein, after the acquisition of the target number segment, the method further comprises: Get sorting strategy; sorting the first number of numbers included in the target number segment based on the sorting strategy. The method according to claim 1, wherein, after converting the target number and determining the short link, the method further includes: Store the short link once, and store it in the first storage area; the first storage area is used to realize high-speed access to the short link; The short link is stored secondary to a second storage area; the second storage area is used to realize the backup of the short link. The method according to claim 6, wherein the method further comprises: Send the short link to the message queue; The said short link is stored once and stored in the second storage area, including: The short link in the message queue is consumed, and the short link is respectively stored in a first database and a second database, and the first database is different from the second database. The method according to claim 7, wherein said sending said short link to a message queue comprises: judging whether the network of the message queue is abnormal; If the network of the message queue is abnormal, the short link is sent to the reissue message list through the reissue message thread, and the short link in the reissue message list is resent to the message queue. The method according to claim 8, wherein, after sending the short link to the reissue message list through the reissue message thread, the method further comprises: Back up the short links in the reissued message list to the backup message list through the backup message thread; Judging whether the short link in the reissued message list is resent to the message queue; If the short link in the reissue message list is resent to the message queue, delete the short link in the backup message list and the resend message queue in the reissue message list; If the short link in the reissued message list is not resent to the message queue, then reserve the corresponding short link in the backup message list and the reissued message list, when the retention time is longer than the preset time length , then resend the short link in the backup message list to the reissue message list. A short link generation device, said device comprising: An acquisition unit configured to acquire a target number segment; the minimum value of the target number segment is the same as the maximum value of the previous number segment of the target number segment, and the maximum value of the target number segment is the same as the maximum value of the target number segment The minimum value of a number segment is the same, and the target number segment includes the first number of numbers; The processing unit is configured to extend the minimum value and the maximum value of the target number segment to obtain an extended target number segment, and the extended target number segment includes a second number of numbers, and the second number is greater than the specified number the first quantity; a determination unit configured to determine a target number from said second number of numbers; The determining unit is further configured to convert the target number to determine a short link. The short link generating device according to claim 10, wherein the acquiring unit is further configured to: Judging whether the number of locally cached numbers is less than a preset threshold; If the number of numbers cached locally is less than or equal to the preset threshold, the target number segment is acquired. The device for generating short links according to claim 11, wherein the determining unit is further configured to: If the number of the locally cached numbers is greater than the preset threshold, then determine a target number from the locally cached numbers. The device for generating short links according to claim 10, wherein the determining unit is further configured to: Carrying out modulo taking on the target number to obtain a modulo taking result, the modulo taking result including: at least one modulo taking parameter; Determining characters corresponding to each of the modulo parameters in the at least one modulo parameter; at least one of the characters forms the short link. The device for generating short links according to claim 10, wherein the obtaining unit is further configured to: obtain a sorting strategy; The short link generation device further includes: a sorting unit configured to sort the first number of numbers included in the target number segment based on the sorting strategy. The short link generating device according to claim 10, wherein the short link generating device further comprises: a storage unit configured to: Store the short link once, and store it in the first storage area; the first storage area is used to realize high-speed access to the short link; The short link is stored secondary to a second storage area; the second storage area is used to realize the backup of the short link. The short link generating device according to claim 15, wherein the short link generating device further comprises: a sending unit configured to: Send the short link to the message queue; The storage unit is also configured as: The short link in the message queue is consumed, and the short link is respectively stored in a first database and a second database, and the first database is different from the second database. The short link generation device according to claim 16, wherein the sending unit is further configured to: judging whether the network of the message queue is abnormal; If the network of the message queue is abnormal, the short link is sent to the reissue message list through the reissue message thread, and the short link in the reissue message list is resent to the message queue. The short link generation device according to claim 17, wherein the processing unit is further configured to: Back up the short links in the reissued message list to the backup message list through the backup message thread; Judging whether the short link in the reissued message list is resent to the message queue; If the short link in the reissue message list is resent to the message queue, delete the short link in the backup message list and the resend message queue in the reissue message list; If the short link in the reissued message list is not resent to the message queue, then reserve the corresponding short link in the backup message list and the reissued message list, when the retention time is longer than the preset time length , then resend the short link in the backup message list to the reissue message list. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, when the processor executes the computer program, the short-term device described in any one of claims 1 to 9 is realized Link generation method. A storage medium storing a computer program, when the computer program is executed by a processor, the method for generating a short link according to any one of claims 1 to 9 is realized.

Images