CN113626426B - A method and system for collecting and transmitting ecological grid data - Google Patents

Abstract

The invention provides a method and a system for acquiring and transmitting ecological grid data, wherein time weight and data volume weight of data requests of grid partitions of each ecological grid are calculated according to population density of the grid partitions, priority values of the data requests are obtained, tasks of sending the data requests to a cloud server are performed according to the sequence of sequencing results of the priority values, intelligent allocation and circulation and efficient cooperative treatment of data request events are achieved, and the improvement of processing efficiency of the data requests and the effective allocation of processing time of the data requests are achieved.

Description

A method and system for collecting and transmitting ecological grid data

technical field

The present disclosure belongs to the technical field of data processing, data collection and transmission, and in particular relates to a method and system for collection and transmission of ecological grid data.

Background technique

In social governance, grid governance is an important means of managing social event classification and classification standards. Based on the grid, various social governance environments and governance mechanisms form a close internal governance ecology, which is what we call a governance ecological grid. In today's information age, thousands of social affairs are processed every day. With the popularization of mobile client, WeChat applet and online business processing, the electronic automatic processing of social affairs is developing rapidly, greatly improving It improves the efficiency of social services and effectively facilitates the people's travel and errands, all of which depend on the intelligent and efficient processing of data flow in the social governance ecological grid. The big data system of the ecological grid forms a data system model for the intelligent distribution of social governance events through algorithms, and realizes the intelligent distribution and circulation of data events and efficient collaborative processing. With the increasingly rapid development of big data and artificial intelligence technologies, coordinating the balanced development of ecological grids in various regions and coordinating the electronic, intelligent, and efficient development of social affairs within each grid is a major category of social affairs management. In the prevention and control of infectious diseases, the intelligent data governance of the ecological grid helps to efficiently allocate resources in all aspects of society, to speed up the efficient processing of large-scale event requests from a wide range of users, and to improve the efficiency of online services. Handling efficiency and service quality, and improving the management and service level of social affairs. However, the current common data transmission technology is less focused on the time efficiency of data transmission and overall efficient coordination, which is not enough to cope with the increasingly large-scale data request events in the social governance ecological grid. It is urgent to propose new methods and technologies to solve this ecological problem. Grid governance requirements.

SUMMARY OF THE INVENTION

The purpose of the present invention is to propose a method and system for collecting and transmitting ecological grid data, so as to solve one or more technical problems existing in the prior art, and at least provide a beneficial choice or create conditions.

In today's automated big data social management, the popularization and application of big data, the Internet and artificial intelligence technology has made online handling of various affairs the normal state of the society. The population densities are different, and the time requirements for the processing of the corresponding data services are also different. The efficient and fast processing of large-scale data volume has become a major issue for the service data management system, which requires data governance based on ecological grids. This paper proposes a data transmission method and system that can effectively allocate and transfer and efficiently coordinate processing.

The present disclosure provides a method and system for collecting and transmitting ecological grid data. According to the population density of each grid partition, the time weight and data volume weight of data requests of each grid partition of each ecological grid are calculated, thereby obtaining each grid partition. The priority value of the data request, according to the order of the sorting result of the priority value, the task of sending data requests to the cloud server is performed in sequence, so as to realize the intelligent distribution and efficient co-processing of data request events, and achieve the processing of data requests. Increased efficiency and efficient allocation of processing time for data requests.

In order to achieve the above object, according to an aspect of the present disclosure, a method for collecting and transmitting ecological grid data is provided, and the method includes the following steps:

S100, dividing the electronic map and its corresponding population heat map according to the population density to obtain grid partitions (also referred to as partitions), and all the grid partitions are regarded as an ecological grid;

S200, collect a data request in each grid partition of the ecological grid, where the data request is a structure composed of date, number of bytes and character data input by a user at the client and transmitted to a distributed database for storage data;

S300, obtain the time weight by calculating the dates in all data requests;

S400, respectively calculating the overall number of data requests in each grid partition and the number of bytes and character data of each data request, thereby obtaining a data volume weight;

S500, calculating a priority value of each data request according to the time weight and the data volume weight;

S600, by calculating the priority value of each data request, and according to the order of the priority value of the data request as the time sequence, send the corresponding data request to the cloud server in sequence.

Further, in S100, the electronic map and its corresponding population heat map are divided into grid partitions according to the population density, and all grid partitions are regarded as an ecological grid, and the method is: according to the longitude and latitude coordinates and the longitude and latitude coordinates on the electronic map. The population density value on the corresponding population heat map, the electronic map and its corresponding population heat map correspond to each other on the actual geographic latitude and longitude coordinates, and the matrix of the frame of the population heat map contains the population density value of its corresponding latitude and longitude , using the edge line detection algorithm (the reference paper is [1]P Dollár, Zitnick C L.Fast Edge DetectionUsing Structured Forests[J].IEEE Transactions on Pattern Analysis and MachineIntelligence,2014,37(8):1558-1570. Or for [2] Xie S, Tu Z. Holistically-NestedEdge Detection. IEEE, 2015.) Perform edge detection on the population heat map to obtain edge lines on the population heat map, and use the edge lines to divide the population heat map into multiple parts, each The actual area on the corresponding electronic map is denoted as a grid partition as Dis. By calculating the arithmetic mean of the population density values on the population heat map of all the latitude and longitude coordinates included in the electronic map corresponding to the grid partition Dis, this is obtained. The average population density ρ of grid partitions, and the set of all grid partitions is an ecological grid denoted as Grid.

Further, in S200, a data request is collected in each grid partition of the ecological grid, and the data request is a user input on the client side and transmitted to the distributed database for storage by date, number of bytes and Structured data composed of character data, the method is as follows: in each grid partition of the ecological grid, encapsulate a piece of data input by the user located in it into a structured data, and the structured data contains Including the date of the data collection, the number of bytes of the data, and the character data of the data, the structured data is transmitted to the distributed database for storage as a data request and denoted as R. The data request is due to the grid partition of the collection. Different identifiers are provided in the distributed database, the number of bytes is the number of bytes of the structured data, and the character data is a character string input by the user at the client.

Further, in S300, the time weight is obtained by calculating the date in all data requests, and the method is: the date included in the data request included in each grid partition is recorded as the year, month, day, hour, minute, second An array T composed of six-digit values, T=[y,m,d,h,n,s], where y represents the value of the year, m represents the value of the month, d represents the value of the day, and h represents the value of the hour, n represents the value of minutes, and s represents the value of seconds. If the serial number of all data requests in a grid partition is represented by variable i, the data request with serial number i is recorded as R_i, and all data requests in one grid partition are recorded as As a set Rset, R_i∈Rset, let n1 be the number of all data requests in the set Rset, then i∈[1,n1], the date included in the data request R_i is recorded as T_i, T_i=[y_i,m_i,d_i,h_i ,n_i,s_i], where y_i represents the year value of R_i collection time, m_i represents the month value of R_i collection time, d_i represents the day value of R_i collection time, h_i represents the hour value of R_i collection time, and n_i represents R_i The value in minutes of the collection time, s_i represents the value in seconds of the R_i collection time; denote the date included in the data request extracted from a data request as the function Temp(), then T_i=Temp(R_i), denoted from a The function to filter out the date of the data request with the earliest date in all the data requests of the grid partition is Fir(). Let the date of the data request with the earliest date in Rset be T_fir, that is, T_fir=Fir(Rset), T_f= [y_f, m_f, d_f, h_f, n_f, s_f], where y_f represents the value of the year of T_f, m_f represents the value of the month of T_f, d_f represents the value of the day of T_f, h_f represents the value of the hour of T_f, and n_f represents the value of T_f The value of minutes, s_f represents the value of seconds of T_f; thus calculate the time weight of each data request of a grid partition, and record the time weight as Wet, then the time weight of the data request with serial number i in Rset can be recorded as Wet_i or Wet(Rset, R_i), Wet_i=Wet(Rset, R_i), define trs1 as the first intermediate variable, and the calculation formula of trs1 is:

trs1=y_f*y_i+m_f*m_i+d_f*d_i+h_f*h_i+n_f*n_i+s_f*s_i, define trs2 as the second intermediate variable, and the calculation formula of trs2 is:

The formula for calculating Wet(Rset, R_i) is:

The obtained Wet_i represents the time weight of the date of the data request R_i with the serial number i in the set Rset, and obtaining the time weight helps to measure the proportion of the date of a data request in the data request of the entire grid partition.

Further, in S400, the overall number of data requests in each grid partition and the number of bytes and character data of each data request are calculated respectively, so as to obtain the data volume weight, and the method is: The number of bytes included in the data request is denoted by By, and the character string included in the data request is denoted as Str. The sequence number of all data requests in a grid partition is represented by variable i, and the data request with sequence number i is denoted as R_i , all data requests of a grid partition are denoted as set Rset, R_i∈Rset, let n1 be the number of all data requests in set Rset, then i∈[1,n1], the number of bytes included in data request R_i is denoted It is By_i or By(R_i), and the included string is recorded as Str_i or Str(R_i). Let the function len() be the function to obtain the length of the string included in a data request, and the function ln() to calculate the natural number e as the base The logarithmic function of , let the data volume weight be Weg, then the data volume weight of the data request R_i with serial number i in Rset is recorded as Weg_i or Weg(Rset, R_i), Weg_i=Weg(Rset, R_i), based on R_i The formula for calculating the data volume weight Weg_i by including the number of bytes and strings is:

The obtained data volume weight Weg_i helps to measure the size of each data request in the population.

Further, in S500, according to the time weight and the data volume weight, the priority value of each data request is calculated and obtained, and the method is: the sequence number of all data requests of a grid partition is represented by variable i, then the sequence number is i. The data request is denoted as R_i, and all the data requests of a grid partition are denoted as the set Rset, R_i∈Rset, let n1 be the number of all data requests in the set Rset, then i∈[1,n1], for the grid partition For each data request in the data request, take the value of the date included in the data request and denote the value of the day as d, and let the value of the date included in the data request R_i with the serial number i in Rset be denoted as d_i or d(R_i) , d_i=d(R_i), let M represent the total number of days in the month of the month where the date of the data request is located, and let M_i represent the total number of days in the month of the month where the date of the data request R_i is located, according to S300 The method described in the method and the method described in S400, calculate the time weight Wet_i and the data volume weight Weg_i of the data request R_i in the set Rset of the grid partition where it is located, and obtain the priority value of the data request according to the time weight and the data volume weight Pr, the priority value of the data request R_i in the set Rset of the grid partition where it is located is recorded as Pr_i or Pr(Rset, R_i), Pr_i=Pr(Rset, R_i), then the calculation formula of the priority value Pr_i is:

The priority values of all data requests in the set Rset can be expressed as the set Pset,

Pset={Pr(Rset,R_i),R_i∈Rset}

There are a total of n1 data requests R_i in Rset. Correspondingly, there are n1 priority values in the set Pset. This method of calculating priority values helps to measure the priority of different data requests in a grid partition. .

Further, in S600, by calculating the priority value of each data request, according to the order of the priority value of the data request as the time sequence, the corresponding data request is sent to the cloud server in sequence, and the method is: The set of grid partitions is an ecological grid Grid, the number of all grid partitions contained in an ecological grid Grid is k, and the variable j represents the serial number of the grid partitions in the ecological grid Grid, j∈[ 1,k], the grid partition with the serial number j in the ecological grid Grid is recorded as Dis_j, the average population density of the grid partition Dis_j is recorded as ρ_j, according to the method described in S500, all data requests in the grid partition Dis_j The set of data is recorded as Rset_j, the data request with the serial number i in Rset_j is recorded as R_i_j, the data request with the serial number i in Rset_j The date included in R_i_j is recorded as T_i_j or Temp(R_i_j), and the number of bytes included in R_i_j is recorded as For By_i_j, the string included in the data request is recorded as Str_i_j, the date of the earliest data request in Rset_j is T_fir_j or Fir(Rset_j), the time weight of the data request with the serial number i in Rset_j can be recorded as Wet_i_j or Wet( Rset_j, R_i_j), the data volume weight of the data request with the serial number i in Rset_j is recorded as Weg_i_j or Weg(Rset_j, R_i_j).

S601, obtain a collection Grid from a distributed database;

S602, create an empty array Lset; in the set Grid, obtain the average population density record ρ_j corresponding to each grid partition Dis_j and add ρ_j to Lset;

S603, select the element with the largest value in Lset, and then obtain the grid partition Dis_j with the largest average population density corresponding to this element;

S604, obtain the set Rset_j of data requests of the grid partition Dis_j;

S605, by using the method described in S500, the priority values of all data requests in the set Rset_j are obtained, which can be expressed as the set Pset_j, to obtain Pset_j={Pr(Rset_j, R_i_j), R_i_j∈Rset_j};

S606, the function Max_sort() is used to calculate the function Max_sort(Pset_j), and the elements in Pset_j are sorted in descending order of value to obtain a sequence of sorting results as Seq, and the corresponding elements in the sorted result Seq are in descending order The order is the order in which each data request R_i_j in Rset_j is sent to the cloud processor, and each data request R_i_j in Rset_j is sent to the cloud processor according to this order.

The present disclosure also provides a collection and transmission system for ecological grid data, the system for collection and transmission of ecological grid data includes: a processor, a memory, and a system stored in the memory and operable on the processor The computer program, the processor implements the steps in the method for collecting and transmitting ecological grid data in claim 1 when the processor executes the computer program, and the system for collecting and transmitting ecological grid data can run on a desktop In computing devices such as upper computers, notebooks, mobile phones, mobile phones, tablet computers, PDAs, and cloud data centers, the operable systems may include, but are not limited to, processors, memories, and server clusters. The computer program runs in units of the following systems:

The data grid partition unit is used to divide the electronic map and its corresponding population heat map according to the population density to obtain grid partitions to form an ecological grid;

A data request collection unit for collecting data requests in each grid partition of the ecological grid;

The weight calculation unit is used to obtain the time weight by calculating the date in all data requests, and separately calculate the overall number of data requests in each grid partition and the number of bytes and character data of each data request to obtain the data volume weight;

a priority value calculation unit, used for calculating the priority value of each data request according to the time weight and the data volume weight;

The sorting and sending unit is used for sending the corresponding data requests to the cloud server in a sequential order according to the sorting of the priority values of the data requests as the time sequence.

The beneficial effects of the present disclosure are as follows: the present disclosure provides a method and system for collecting and transmitting ecological grid data. According to the population density of each grid partition, the time weight and the time weight of the data request of each grid partition of each ecological grid are calculated. The weight of the data volume is used to obtain the priority value of each data request, and the task of sending data requests to the cloud server is performed in order according to the order of the priority value sorting results, so as to realize the intelligent distribution and efficient coordination of data request events. Processing, to achieve the improvement of the processing efficiency of the data request and the effective allocation of the processing time of the data request.

Description of drawings

The above-mentioned and other features of the present disclosure will become more apparent from the detailed description of the embodiments shown in conjunction with the accompanying drawings, in which the same reference numerals refer to the same or similar elements of the present disclosure. The drawings are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative efforts. In the drawings:

Figure 1 is a flowchart of a method for collecting and transmitting ecological grid data;

Figure 2 shows a system structure diagram of an ecological grid data collection and transmission system.

Detailed ways

The concept, specific structure and technical effects of the present disclosure will be clearly and completely described below with reference to the embodiments and accompanying drawings, so as to fully understand the purpose, solutions and effects of the present disclosure. It should be noted that the embodiments in the present application and the features of the embodiments may be combined with each other in the case of no conflict.

In the description of the present invention, the meaning of several is one or more, the meaning of multiple is two or more, greater than, less than, exceeding, etc. are understood as not including this number, above, below, within, etc. are understood as including this number. If it is described that the first and the second are only for the purpose of distinguishing technical features, it cannot be understood as indicating or implying relative importance, or indicating the number of the indicated technical features or the order of the indicated technical features. relation.

FIG. 1 is a flowchart of a method for collecting and transmitting ecological grid data according to the present invention. The following describes a method and system for collecting and transmitting ecological grid data according to an embodiment of the present invention with reference to FIG. 1 .

The present disclosure proposes a method for collecting and transmitting ecological grid data, and the method specifically includes the following steps:

S100, divide the electronic map and its corresponding population heat map according to the population density to obtain grid partitions, and all grid partitions are regarded as an ecological grid;

S200, collect a data request in each grid partition of the ecological grid, where the data request is a structure composed of date, number of bytes and character data input by a user at the client and transmitted to a distributed database for storage data;

S300, obtain the time weight by calculating the dates in all data requests;

S400, respectively calculating the overall number of data requests in each grid partition and the number of bytes and character data of each data request, thereby obtaining a data volume weight;

S500, calculating a priority value of each data request according to the time weight and the data volume weight;

S600, by calculating the priority value of each data request, and according to the order of the priority value of the data request as the time sequence, send the corresponding data request to the cloud server in sequence.

Further, in S100, the electronic map and its corresponding population heat map are divided into grid partitions according to the population density, and all grid partitions are regarded as an ecological grid, and the method is: according to the longitude and latitude coordinates and the longitude and latitude coordinates on the electronic map. The population density value on the corresponding population heat map, the electronic map and its corresponding population heat map correspond to each other on the actual geographic latitude and longitude coordinates, and the matrix of the frame of the population heat map contains the population density value of its corresponding latitude and longitude , using the edge line detection algorithm (the reference paper is [1]P Dollár, Zitnick C L.Fast EdgeDetection Using Structured Forests[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014,37(8):1558-1570. Or For [2] Xie S, TuZ. Holistically-Nested Edge Detection. IEEE, 2015.), perform edge detection on the population heat map to obtain the edge line on the population heat map, and use the edge line to divide the population heat map into multiple parts. The actual area on the electronic map corresponding to each part is denoted as a grid partition as Dis, and it is obtained by calculating the arithmetic mean of the population density values on the population heat map of all the latitude and longitude coordinates included in the electronic map corresponding to the grid partition Dis. The average population density ρ of the grid partition, and the set of all grid partitions is an ecological grid, denoted as Grid.

Further, in S200, a data request is collected in each grid partition of the ecological grid, and the data request is a user input on the client side and transmitted to the distributed database for storage by date, number of bytes and Structured data composed of character data, the method is as follows: in each grid partition of the ecological grid, encapsulate a piece of data input by the user located in it into a structured data, and the structured data contains Including the date of the data collection, the number of bytes of the data, and the character data of the data, the structured data is transmitted to the distributed database for storage as a data request and denoted as R. The data request is due to the grid partition of the collection. Different identifiers are provided in the distributed database, the number of bytes is the number of bytes of the structured data, and the character data is a character string input by the user at the client.

Further, in S300, the time weight is obtained by calculating the date in all data requests, and the method is: the date included in the data request included in each grid partition is recorded as the year, month, day, hour, minute, second An array T composed of six-digit values, T=[y,m,d,h,n,s], where y represents the value of the year, m represents the value of the month, d represents the value of the day, and h represents the value of the hour, n represents the value of minutes, and s represents the value of seconds. If the serial number of all data requests in a grid partition is represented by variable i, the data request with serial number i is recorded as R_i, and all data requests in one grid partition are recorded as As a set Rset, R_i∈Rset, let n1 be the number of all data requests in the set Rset, then i∈[1,n1], the date included in the data request R_i is recorded as T_i, T_i=[y_i,m_i,d_i,h_i ,n_i,s_i], where y_i represents the year value of R_i collection time, m_i represents the month value of R_i collection time, d_i represents the day value of R_i collection time, h_i represents the hour value of R_i collection time, and n_i represents R_i The value of the minute of the collection time, s_i represents the value of the second of the R_i collection time; and the date included in the data request is extracted from a data request as the function Temp(), then there is T_i=Temp(R_i), recorded from The function to filter out the date of the data request with the earliest date in all the data requests of a grid partition is Fir(). Let the date of the data request with the earliest date in Rset be T_fir, that is, T_fir=Fir(Rset), T_f =[y_f,m_f,d_f,h_f,n_f,s_f], where y_f represents the value of the year of T_f, m_f represents the value of the month of T_f, d_f represents the value of the day of T_f, h_f represents the value of the hour of T_f, and n_f represents the value of the day of T_f The value of minutes in T_f, and s_f represents the value of seconds in T_f; thus, the time weight of each data request of a grid partition is calculated separately, and the time weight is recorded as Wet, then the time weight of the data request with the serial number i in Rset can be recorded As Wet_i or Wet(Rset, R_i), Wet_i=Wet(Rset, R_i), define trs1 as the first intermediate variable, and the calculation formula of trs1 is:

trs1=y_f*y_i+m_f*m_i+d_f*d_i+h_f*h_i+n_f*n_i+s_f*s_i, define trs2 as the second intermediate variable, and the calculation formula of trs2 is:

The formula for calculating Wet(Rset, R_i) is:

The obtained Wet_i represents the time weight of the date of the data request R_i with the serial number i in the set Rset, and obtaining the time weight helps to measure the proportion of the date of a data request in the data request of the entire grid partition.

Further, in S400, the overall number of data requests in each grid partition and the number of bytes and character data of each data request are calculated respectively, so as to obtain the data volume weight, and the method is: The number of bytes included in the data request is denoted by By, and the character string included in the data request is denoted as Str. The sequence number of all data requests in a grid partition is represented by variable i, and the data request with sequence number i is denoted as R_i , all data requests of a grid partition are denoted as set Rset, R_i∈Rset, let n1 be the number of all data requests in set Rset, then i∈[1,n1], the number of bytes included in data request R_i is denoted It is By_i or By(R_i), and the included string is recorded as Str_i or Str(R_i). Let the function len() be the function to obtain the length of the string included in a data request, and the function ln() to calculate the natural number e as the base The logarithmic function of , let the data volume weight be Weg, then the data volume weight of the data request R_i with serial number i in Rset is recorded as Weg_i or Weg(Rset, R_i), Weg_i=Weg(Rset, R_i), based on R_i The formula for calculating the data volume weight Weg_i by including the number of bytes and strings is:

The obtained data volume weight Weg_i helps to measure the size of each data request in the population.

Further, in S500, according to the time weight and the data volume weight, the priority value of each data request is calculated and obtained, and the method is: the sequence number of all data requests of a grid partition is represented by variable i, then the sequence number is i. The data request is denoted as R_i, and all the data requests of a grid partition are denoted as the set Rset, R_i∈Rset, let n1 be the number of all data requests in the set Rset, then i∈[1,n1], for the grid partition For each data request in the data request, take the value of the date included in the data request and denote the value of the day as d, and let the value of the date included in the data request R_i with the serial number i in Rset be denoted as d_i or d(R_i) , d_i=d(R_i), let M represent the total number of days in the month of the month in which the date of the data request is located, let M_i represent the total number of days in the month of the month where the date of the data request R_i is located, according to S300 The method and the method described in S400, calculate the time weight Wet_i and the data volume weight Weg_i of the data request R_i in the set Rset of the grid partition where it is located, and obtain the priority of the data request according to the time weight and the data volume weight The value Pr, the priority value of the data request R_i in the set Rset of the grid partition where it is located is recorded as Pr_i or Pr(Rset, R_i), Pr_i=Pr(Rset, R_i), then the calculation formula of the priority value Pr_i is :

The priority values of all data requests in the set Rset can be expressed as the set Pset,

Pset={Pr(Rset,R_i),R_i∈Rset}

There are a total of n1 data requests R_i in Rset. Correspondingly, there are n1 priority values in the set Pset. This method of calculating priority values helps to measure the priority of different data requests in a grid partition. .

Further, in S600, by calculating the priority value of each data request, according to the order of the priority value of the data request as the order of time, the corresponding data request is sent to the cloud server in the order, the method is as follows: using S100 The method described in the method and the method described in S200, make the set of all grid partitions constitute an ecological grid Grid, the number of all grid partitions contained in an ecological grid Grid is k, and the variable j represents the ecological grid The serial number of the grid partition in the grid, j∈[1,k], the grid partition with serial number j in the ecological grid Grid is recorded as Dis_j, and the average population density of the grid partition Dis_j is recorded as ρ_j, according to the S500 In the described method, the set of all data requests in the grid partition Dis_j is denoted as Rset_j, the data request with sequence number i in Rset_j is denoted as R_i_j, and the date included in the data request R_i_j with sequence number i in Rset_j is denoted as T_i_j or Temp(R_i_j), the number of bytes included in R_i_j is recorded as By_i_j, the string included in the data request is recorded as Str_i_j, the date of the earliest data request in Rset_j is T_fir_j or Fir(Rset_j), the serial number in Rset_j is i The time weight of the data request can be recorded as Wet_i_j or Wet(Rset_j, R_i_j), the data volume weight of the data request with serial number i in Rset_j is recorded as Weg_i_j or Weg(Rset_j, R_i_j), the function Max_sort() is based on the numerical value from the largest To the small sort function, perform the following steps:

S601, obtain a collection Grid from a distributed database;

S602, create an empty array Lset; in the set Grid, obtain the average population density record ρ_j corresponding to each grid partition Dis_j and add ρ_j to Lset;

S603, select the element with the largest value in Lset, and then obtain the grid partition Dis_j with the largest average population density corresponding to this element;

S604, obtain the set Rset_j of data requests of the grid partition Dis_j;

S605, by using the method described in S500, the priority values of all data requests in the set Rset_j are obtained, which can be expressed as the set Pset_j, to obtain Pset_j={Pr(Rset_j, R_i_j), R_i_j∈Rset_j};

S606, the function Max_sort() is used to calculate the function Max_sort(Pset_j), and the elements in Pset_j are sorted in descending order of value to obtain a sequence of sorting results as Seq, and the corresponding elements in the sorted result Seq are in descending order The order is the order in which each data request R_i_j in Rset_j is sent to the cloud processor, and each data request R_i_j in Rset_j is sent to the cloud processor according to this order.

The system for collecting and transmitting ecological grid data includes: a processor, a memory, and a computer program stored in the memory and running on the processor, and the processor implements the above when executing the computer program. The steps in an embodiment of a method for collecting and transmitting ecological grid data, the system for collecting and transmitting ecological grid data can run in computing devices such as desktop computers, notebooks, PDAs, and cloud data centers, etc. The operating system may include, but is not limited to, a cluster of processors, memory, and servers.

An embodiment of the present disclosure provides a collection and transmission system for ecological grid data. As shown in FIG. 2 , the collection and transmission system for ecological grid data in this embodiment includes: a processor, a memory, and a system stored in the memory. A computer program that can be run on the processor, when the processor executes the computer program, the steps in the above-mentioned embodiment of the method for collecting and transmitting ecological grid data are implemented, and the processor executes the computer program. Programs run in units of the following systems:

The data grid partition unit is used to divide the electronic map and its corresponding population heat map according to the population density to obtain grid partitions to form an ecological grid;

A data request collection unit for collecting data requests in each grid partition of the ecological grid;

The weight calculation unit is used to obtain the time weight by calculating the date in all data requests, and separately calculate the overall number of data requests in each grid partition and the number of bytes and character data of each data request to obtain the data volume weight;

a priority value calculation unit, used for calculating the priority value of each data request according to the time weight and the data volume weight;

The sorting and sending unit is used for sending the corresponding data requests to the cloud server in a sequential order according to the sorting of the priority values of the data requests as the time sequence.

The collection and transmission system for ecological grid data can be run in computing devices such as desktop computers, notebooks, palmtop computers, and cloud data centers. The system for collecting and transmitting ecological grid data includes, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above example is only an example of a method and system for collecting and transmitting ecological grid data, and does not constitute a limitation on a method and system for collecting and transmitting ecological grid data, which may include proportional More or less components, or a combination of some components, or different components, for example, the ecological grid data collection and transmission system may also include input and output devices, network access devices, buses, and the like.

The processor may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete component gate circuits or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor is the control center of the ecological grid data collection and transmission system, and is connected by various interfaces and lines. Each grid sub-area of the entire collection and transmission system of an ecological grid data.

The memory can be used to store the computer program and/or module, and the processor implements the one by running or executing the computer program and/or module stored in the memory and calling the data stored in the memory. A collection and transmission method of ecological grid data and various functions of the system. The memory may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may store Data (such as audio data, phonebook, etc.) created according to the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory such as hard disk, internal memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card , a flash memory card (Flash Card), at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

The present disclosure provides a method and system for collecting and transmitting ecological grid data. According to the population density of each grid partition, by calculating the time weight and data volume weight of the data request of each grid partition of each ecological grid, thereby obtaining The priority value of each data request, according to the order of the sorting result of the priority value, the task of sending data requests to the cloud server is carried out in sequence, so as to realize the intelligent distribution and efficient co-processing of data request events, so as to achieve the optimal response to data requests. Improved processing efficiency and efficient allocation of processing time to data requests.

Although the present disclosure has been described in considerable detail and with particular reference to a few of the described embodiments, it is not intended to be limited to any of these details or embodiments or to any particular embodiment so as to effectively encompass the intended scope of the present disclosure. Furthermore, the foregoing description of the present disclosure in terms of embodiments foreseen by the inventors is intended to provide a useful description, and those insubstantial modifications of the present disclosure that are not presently foreseen may nevertheless represent equivalent modifications of the present disclosure.

Claims (5)

1. A method for collecting and transmitting ecological grid data is characterized by comprising the following steps:

s100, dividing the electronic map and the population thermodynamic diagrams corresponding to the electronic map according to population density to obtain grid partitions, wherein all the grid partitions are used as an ecological grid;

s200, collecting a data request in each grid partition of the ecological grid, wherein the data request is a structured data which is input by a user at a client and transmitted to a distributed database for storage and consists of date, byte number and character data;

s300, obtaining time weight by calculating dates in all data requests;

s400, respectively calculating the whole number of the data requests in each grid partition, the byte number and the character data of each data request to obtain the data weight;

s500, calculating to obtain a priority value of each data request according to the time weight and the data volume weight;

s600, by calculating the priority value of each data request, taking the sequence of the priority values of the data requests as the sequence of time, and sending the corresponding data requests to a cloud server according to the sequence;

in S300, the time weight is obtained by calculating dates in all data requests, and the method includes: the date included in the data request included in each grid partition is recorded as an array T consisting of six-digit numerical values of year, month, day, hour, minute and second, where T is [ y, m, d, h, n, s ], where y represents a numerical value of year, m represents a numerical value of month, d represents a numerical value of day, h represents a numerical value of hour, n represents a numerical value of minute, s represents a numerical value of second, the sequence numbers of all the data requests of one grid partition are represented by a variable i, the data request with the sequence number i is recorded as R _ i, all the data requests of one grid partition are recorded as a set Rset, R _ i belongs to Rset, n1 is the number of all the data requests of the set Rset, i belongs to [1, n1], and the date included in the data request R _ i is recorded as T _ i, T _ i is [ y _ i, m _ i, d _ i, h _ i, n _ i, s _ i ], wherein y _ i represents a numerical value of year of R _ i acquisition time, m _ i represents a numerical value of month of R _ i acquisition time, d _ i represents a numerical value of day of R _ i acquisition time, h _ i represents a numerical value of hour of R _ i acquisition time, n _ i represents a numerical value of minute of R _ i acquisition time, and s _ i represents a numerical value of second of R _ i acquisition time; keeping track of the date included in the data request extracted from one data request as a function Temp (), if T _ i is Temp (R _ i), the function of the date of the data request with the earliest date screened from all data requests of one grid partition is Fir (), and the date of the data request with the earliest date in Rset is T _ Fir, i.e. T _ Fir is Fir (Rset), T _ f is [ y _ f, m _ f, d _ f, h _ f, n _ f, s _ f ], wherein y _ f represents the annual value of T _ f, m _ f represents the monthly value of T _ f, d _ f represents the daily value of T _ f, h _ f represents the hourly value of T _ f, n _ f represents the minute value of T _ f, and s _ f represents the second value of T _ f; thus, the time weight of each data request of one grid partition is calculated, if the time weight is let, the time weight of the data request with sequence number i in Rset is let _ i or let (Rset, R _ i), let _ i is let (Rset, R _ i), trs1 is defined as a first intermediate variable, and the calculation formula of trs1 is as follows:

the calculation formula of trs1 is y _ f _ y _ i + m _ f _ m _ i + d _ f _ d _ i + h _ f _ h _ i + n _ f _ n _ i + s _ f _ s _ i, and trs2 is defined as a second intermediate variable, and trs2 is as follows:

the formula in which Wet (Rset, R _ i) is calculated is:

the obtained Wet _ i represents the time weight of the date of the data request R _ i with the sequence number i in the set Rset, and the obtained time weight is helpful for measuring the precedence proportion of the date of one data request in the data request of the whole grid partition;

in S400, the total number of data requests in each grid partition, the byte number and the character data of each data request are respectively calculated, so as to obtain the data weight, and the method includes: if the number of bytes included in the data request included in each mesh partition is represented as By, the string included in the data request is represented as Str, the number of bytes included in the data request R _ i is represented as By _ i or By (R _ i), the string included in the data request is represented as Str _ i or Str (R _ i), and the function len () is a function for obtaining the length of the string included in one data request, the function ln () is a function for calculating the logarithm based on the natural number e, and the data amount weight is Weg, then the data amount weight of the data request R _ i with the sequence number i in Rset is represented as Weg _ i or Weg (Rset, R _ i), Weg _ i is represented as Weg (Rset, R _ i), and the formula for calculating the data amount weight Weg _ i based on the number of bytes included in R _ i and the string is:

the obtained data weight Weg _ i is helpful for measuring the size of each data request in the population;

in S500, a priority value of each data request is calculated according to the time weight and the data amount weight, and the method includes: for each data request in the grid partition, a value representing a day in a date included in the data request is represented as d, a value representing a day in a date included in a data request R _ i with a serial number i in Rset is represented as d _ i or d (R _ i), d _ i is represented as d (R _ i), M is assumed to represent the total number of days in a month in which the date included in the data request is located, M _ i is assumed to represent the total number of days in the month in a month in which the date of the data request R _ i is located, according to the method described in S300 and the method described in S400, a time weight Wet _ i and a data amount weight Weg _ i in a set Rset of the grid partition in which the data request R _ i is located are calculated, a priority value Pr of the data request is found from the time weight and the data amount weight, a priority value of the data request R _ i in a set Rset of the grid partition in which the data request R _ i is located is represented as Pr _ i or Rset (Rset _ i or Rset, r _ i), Pr _ i ═ Pr (Rset, R _ i), the formula for calculating the priority value Pr _ i is:

the priority values of all data requests in the set Rset are denoted as set Pset,

Pset＝{Pr(Rset,R_i),R_i∈Rset},

there are n1 total data requests R _ i in Rset, correspondingly, there are n1 priority values in set Pset.

2. The method for acquiring and transmitting the ecological grid data according to claim 1, wherein in S100, the electronic map and the population thermodynamic diagram corresponding to the electronic map are divided according to population density to obtain grid partitions, and all the grid partitions form an ecological grid, and the method comprises the following steps: according to the longitude and latitude coordinates on the electronic map and the population density value on the population thermodynamic diagram corresponding to the longitude and latitude coordinates, performing edge detection on the population thermodynamic diagrams to obtain edge lines on the population thermodynamic diagrams, wherein the electronic maps and the population thermodynamic diagrams corresponding to the electronic maps correspond to each other on the actual geographic longitude and latitude coordinates, the matrix of the frames of the population thermodynamic diagram comprises population density values of corresponding longitude and latitude, the population thermodynamic diagram is divided into a plurality of parts by using edge lines, an actual area on the electronic map corresponding to each part is taken as a grid partition and is marked as Dis, and calculating the arithmetic mean of the population density values of all longitude and latitude coordinates on the population thermodynamic diagram, which are included on the electronic map corresponding to the Grid partition Dis, to obtain the average population density rho of the Grid partition, wherein the set formed by all the Grid partitions is an ecological Grid and is marked as Grid.

3. The method for collecting and transmitting data of an ecological grid according to claim 1, wherein in S200, a data request is collected in each grid partition of the ecological grid, the data request is a piece of structured data composed of date, byte number and character data, which is input by a user at a client and transmitted to a distributed database for storage, and the method comprises: in each grid partition of the ecological grid, data input of a user at a client side is packaged into a piece of structured data, the structured data comprises data acquisition date, the number of bytes of the data and character data of the data, the structured data is transmitted to a distributed database to be stored as a primary data request denoted by R, the data request has different identifications in the distributed database due to different acquired grid partitions, the number of bytes is the number of bytes of the structured data, and the character data is a character string input by the user at the client side.

4. The method for acquiring and transmitting ecological grid data according to claim 1, wherein in S600, the priority values of the data requests are calculated, the priority values of the data requests are sorted to serve as the time sequence, and the corresponding data requests are sent to the cloud server according to the time sequence, and the method comprises the following steps: setting a set formed By all Grid partitions as an ecological Grid, setting the number of all Grid partitions contained in the ecological Grid as k, setting a variable j to represent the serial numbers of the Grid partitions in the ecological Grid, setting j to [1, k ], setting the Grid partition with the serial number of j in the ecological Grid as Disj, setting the average population density of the Grid partitions Disj as rho _ j, setting the set of all data requests in the Grid partitions Disj as Rset _ j, setting the data request with the serial number of i in the Rset _ j as R _ i _ j, setting the date included in the data request with the serial number of i in the Rset _ j as T _ i _ j or Temp (R _ i _ j), setting the number of bytes included in the R _ i _ j as By _ i _ j, setting the character string included in the data request as T _ Fir _ j or Strar (Rset) in the Rset _ j, the time weight of a data request with sequence number i in Rset _ j is recorded as Wet _ i _ j or Wet (Rset _ j, R _ i _ j), the data quantity weight of a data request with sequence number i in Rset _ j is recorded as Weg _ i _ j or Weg (Rset _ j, R _ i _ j), and the function Max _ sort () is a function sorted from large to small according to the numerical value, and the following steps are executed:

s601, acquiring a set Grid from a distributed database;

s602, creating a null array Lset; in the set Grid, obtaining an average population density mark rho _ j corresponding to each Grid partition Dis _ j and adding the rho _ j into Lset;

s603, selecting the element with the largest value in the Lset, and further acquiring the grid partition Dis _ j with the largest average population density corresponding to the element;

s604, acquiring a set Rset _ j of data requests of the grid partition Dis _ j;

s605, by the method described in S500, obtaining priority values of all data requests in the set Rset _ j, which are denoted as a set Pset _ j, and obtaining Pset _ j ═ { Pr (Rset _ j, R _ i _ j), R _ i _ j ∈ Rset _ j };

s606, calculating a function Max _ sort (Pset _ j) through the function Max _ sort (), namely, sequencing the elements in the Pset _ j from large to small according to the numerical values to obtain a sequence of sequencing results as Seq, taking the sequence of the sequencing results Seq from large to small corresponding to each element as the sequence of sending each data request R _ i _ j in Rset _ j to the cloud processor, and sending each data request R _ i _ j in Rset _ j to the cloud processor according to the sequence.

5. An ecological grid data collecting and transmitting system, comprising: the processor, the memory and the computer program stored in the memory and running on the processor, when the processor executes the computer program, the steps in the method for acquiring and transmitting the ecological grid data in claim 1 are realized, the system for acquiring and transmitting the ecological grid data runs in a desktop computer, a notebook computer, a mobile phone, a palm computer and a cloud data center, and the running system comprises the processor, the memory and a server cluster.

Images