[go: up one dir, main page]

CN109560965B - A method, device and device for obtaining traffic in a distributed system - Google Patents

A method, device and device for obtaining traffic in a distributed system Download PDF

Info

Publication number
CN109560965B
CN109560965B CN201811463382.6A CN201811463382A CN109560965B CN 109560965 B CN109560965 B CN 109560965B CN 201811463382 A CN201811463382 A CN 201811463382A CN 109560965 B CN109560965 B CN 109560965B
Authority
CN
China
Prior art keywords
information
log
access information
user
distributed system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811463382.6A
Other languages
Chinese (zh)
Other versions
CN109560965A (en
Inventor
白学余
杨富强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhengzhou Yunhai Information Technology Co Ltd
Original Assignee
Zhengzhou Yunhai Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhengzhou Yunhai Information Technology Co Ltd filed Critical Zhengzhou Yunhai Information Technology Co Ltd
Priority to CN201811463382.6A priority Critical patent/CN109560965B/en
Publication of CN109560965A publication Critical patent/CN109560965A/en
Application granted granted Critical
Publication of CN109560965B publication Critical patent/CN109560965B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/069Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876Network utilisation, e.g. volume of load or congestion level
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/21Flow control; Congestion control using leaky-bucket

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Environmental & Geological Engineering (AREA)
  • Information Transfer Between Computers (AREA)
  • Debugging And Monitoring (AREA)

Abstract

本发明公开了一种分布式系统的流量获取方法、装置、设备及计算机可读存储介质,该方法包括:确定任一访问分布式系统的用户为目的用户,并获取目的用户的访问信息,访问信息包括目的用户的用户信息、目的用户访问的桶的桶信息、目的用户访问的时间及目的用户上传或下载的字节数;将访问信息写入对应的每个日志中,日志包括分别用于存储分布式系统、分布式系统中各个桶、访问分布式系统的各个用户各自对应的访问信息的日志;接收目的用户发送的信息请求,由与信息请求对应的日志中获取对应的访问信息,并将获取的访问信息作为流量信息返回给目的用户。由此不仅实现了信息多样化,且便于对其进行分时段处理或者分用户处理等方式的二次处理。

Figure 201811463382

The invention discloses a traffic acquisition method, device, equipment and computer-readable storage medium of a distributed system. The method includes: determining any user accessing the distributed system as a target user, obtaining access information of the target user, and accessing The information includes the user information of the destination user, the bucket information of the bucket accessed by the destination user, the access time of the destination user, and the number of bytes uploaded or downloaded by the destination user; Store the logs of the access information corresponding to the distributed system, each bucket in the distributed system, and each user accessing the distributed system; receive the information request sent by the destination user, obtain the corresponding access information from the log corresponding to the information request, and Return the obtained access information to the destination user as traffic information. This not only realizes the diversification of information, but also facilitates secondary processing by time-segment processing or user-by-user processing.

Figure 201811463382

Description

Method, device and equipment for acquiring flow of distributed system
Technical Field
The present invention relates to the technical field of distributed object storage systems, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for acquiring traffic of a distributed system.
Background
In order to monitor the performance of a distributed object storage system (distributed system for short), it is generally required to acquire traffic information of the distributed system.
In the prior art, a counter is usually arranged at a server of a distributed system, and the total number of times that a user accesses the distributed system and the total data volume uploaded and downloaded can be obtained as flow information through the counting function of the counter, but this method can only count the total amount of each item of information, and the obtained data is single and is not easy to be processed again.
In summary, the manner for acquiring the traffic information of the distributed system in the prior art has the problems of single data acquisition and difficulty in secondary processing.
Disclosure of Invention
The invention aims to provide a traffic acquisition method, a traffic acquisition device, traffic acquisition equipment and a computer readable storage medium for a distributed system, which can solve the problems of single acquired data and difficulty in secondary processing existing in a mode for acquiring traffic information of the distributed system in the prior art.
In order to achieve the above purpose, the invention provides the following technical scheme:
a traffic acquisition method of a distributed system comprises the following steps:
determining any user accessing the distributed system as a target user, and acquiring access information of the target user, wherein the access information comprises user information of the target user, bucket information of a bucket accessed by the target user, time accessed by the target user and the number of bytes uploaded or downloaded by the target user;
writing the access information into each corresponding log, wherein the log comprises logs respectively used for storing the access information corresponding to the distributed system, each bucket in the distributed system and each user accessing the distributed system;
and receiving an information request sent by the target user, acquiring corresponding access information from a log corresponding to the information request, and returning the acquired access information serving as flow information to the target user.
Preferably, writing the access information into each corresponding log includes:
and writing the access information into each corresponding log, wherein the log comprises logs respectively used for storing the access information corresponding to the distributed system, each bucket in the distributed system and each user accessing the distributed system at different time periods.
Preferably, writing the access information into each corresponding log includes:
and writing the access information into each log with a corresponding name, wherein the log is named by the belonged time period and the identification of the belonged classification object, and the classification object comprises a distributed system, a bucket and a user.
Preferably, after obtaining the corresponding access information from the log corresponding to the information request, the method further includes:
and judging whether flow calculation is needed, if so, calculating corresponding flow information based on the acquired access information and returning the flow information to the target user, and if not, executing the step of returning the acquired access information to the target user as the flow information, wherein the flow information comprises the uploading or downloading bandwidth, the access times processed per second and the total time used for processing all accesses.
A traffic acquisition apparatus for a distributed system, comprising:
an acquisition module to: determining any user accessing the distributed system as a target user, and acquiring access information of the target user, wherein the access information comprises user information of the target user, bucket information of a bucket accessed by the target user, time accessed by the target user and the number of bytes uploaded or downloaded by the target user;
a write module to: writing the access information into each corresponding log, wherein the log comprises logs respectively used for storing the access information corresponding to the distributed system, each bucket in the distributed system and each user accessing the distributed system;
a return module to: and receiving an information request sent by the target user, acquiring corresponding access information from a log corresponding to the information request, and returning the acquired access information serving as flow information to the target user.
Preferably, the writing module includes:
a write unit to: and writing the access information into each corresponding log, wherein the log comprises logs respectively used for storing the access information corresponding to the distributed system, each bucket in the distributed system and each user accessing the distributed system at different time periods.
Preferably, the writing unit includes:
a write subunit to: and writing the access information into each log with a corresponding name, wherein the log is named by the belonged time period and the identification of the belonged classification object, and the classification object comprises a distributed system, a bucket and a user.
Preferably, the method further comprises the following steps:
a determination module configured to: and the return module is used for judging whether flow calculation is needed or not after acquiring corresponding access information from a log corresponding to the information request, if so, calculating corresponding flow information based on the acquired access information and returning the flow information to the target user, and if not, indicating the return module to execute the step of returning the acquired access information serving as the flow information to the target user, wherein the flow information comprises uploaded or downloaded bandwidth, access times processed per second and total time used for processing all accesses.
A traffic acquisition device of a distributed system, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the traffic acquisition method of the distributed system as described in any one of the above when executing the computer program.
A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of a traffic acquisition method of a distributed system as described in any one of the above.
The invention provides a method, a device, equipment and a computer readable storage medium for acquiring flow of a distributed system, wherein the method comprises the following steps: determining any user accessing the distributed system as a target user, and acquiring access information of the target user, wherein the access information comprises user information of the target user, bucket information of a bucket accessed by the target user, time accessed by the target user and the number of bytes uploaded or downloaded by the target user; writing the access information into each corresponding log, wherein the log comprises logs respectively used for storing the access information corresponding to the distributed system, each bucket in the distributed system and each user accessing the distributed system; and receiving an information request sent by the target user, acquiring corresponding access information from a log corresponding to the information request, and returning the acquired access information serving as flow information to the target user. According to the technical scheme, when a user accesses a distributed system, access information comprising user information, barrel information, access time and byte number uploaded or downloaded is obtained, the access information is stored in each corresponding log, and then the access information is obtained from the log and returned to the user when the user needs to obtain flow information. Different from the prior art that the total access times, the total uploaded and downloaded data volume can only be obtained through a counter, various information when a user accesses a distributed system can be obtained through the mode of refining, collecting and storing the corresponding information, so that the information diversification is realized, and secondary processing in modes of time-sharing processing or user-sharing processing and the like is facilitated.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a traffic acquisition method for a distributed system according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a traffic acquisition device of a distributed system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a flowchart of a traffic obtaining method of a distributed system according to an embodiment of the present invention is shown, where the method includes:
s11: determining any user accessing the distributed system as a target user, and acquiring access information of the target user, wherein the access information comprises user information of the target user, bucket information of a bucket accessed by the target user, time accessed by the target user and the number of bytes uploaded or downloaded by the target user.
The execution subject of the traffic acquisition method for the distributed system provided by the embodiment of the present invention may be a corresponding traffic acquisition device, and the device may be disposed in the distributed system. Any user accessing the distributed system can be used as a target user, that is, each user accessing the distributed system can adopt the technical scheme provided by the invention to realize the acquisition and storage of the corresponding access information. The user information may include a user identifier, a user attribute, and the like, the bucket information may include a bucket identifier, a bucket attribute, a bucket space size, and the like, the access time may include an access start time and an access end time, and may also include a time period used for the access, and the like, and the number of bytes uploaded or downloaded is the number of bytes uploaded to the distributed system or downloaded by the distributed system in the corresponding access; of course, the above information can be set according to actual needs, and all are within the protection scope of the present invention. In addition, it should be noted that a bucket is a storage place for implementing data storage in the distributed system, and is a logical concept of data storage, that is, the distributed system may include a plurality of buckets, and the time referred in this application may all be based on the same time standard of the distributed system.
S12: and writing the access information into each corresponding log, wherein the log comprises logs which are respectively used for storing the access information corresponding to each distributed system, each bucket in the distributed system and each user accessing the distributed system.
It should be noted that the log in the present application may be specifically created in advance in a server of the distributed system; the level of the log may include distributed system, bucket, user. Specifically, the log used for storing the access information corresponding to the distributed system is a log with the level of the distributed system, and the log is used for storing the access information corresponding to the distributed system and comprises the access information corresponding to each user accessing the distributed system each time; the log used for storing the access information corresponding to each bucket is a log with the bucket level, and the log is used for storing the access information corresponding to the corresponding bucket and comprises the access information corresponding to each user when each user accesses the corresponding bucket; the log used for storing the access information corresponding to each user is a log with the user level, and the log is used for storing the access information corresponding to the corresponding user, including the access information of the corresponding user when accessing the distributed system each time. If the access information when the user A accesses the bucket A needs to be respectively stored in the logs corresponding to the distributed system, the bucket A and the user A, the hierarchical storage of the logs is realized. When determining the log for storing the bucket and the access information corresponding to the access information, the corresponding log may be located based on the bucket information and the user information included in the access information.
S13: and receiving an information request sent by a target user, acquiring corresponding access information from a log corresponding to the information request, and returning the acquired access information serving as flow information to the target user.
It should be noted that each user may access the distributed system, and may also obtain the traffic information from the distributed system, so that when receiving an information request sent by a destination user, the traffic information corresponding to the information request needs to be returned to the destination user. The information request can include identification of a classified object corresponding to the access information to be acquired, access time and the like, so that the access information corresponding to the information request is positioned and returned to the target user; the classification object refers to an object corresponding to access information of the log storage, and comprises a distributed system, a bucket and a user, and the identifier of the classification object is the identifier of the object corresponding to the access information of the log storage. In addition, after the identification of the classification object corresponding to the information request is determined, only the required access information needs to be acquired from the log of the classification object corresponding to the identification, if the identification is barrel a, only the required access information needs to be acquired from the barrel a, and compared with the case that all the access information is stored in the same log, the hierarchical storage mode does not need to traverse the same log for acquiring the access information, but is acquired from the log corresponding to the classification object in a targeted manner, so that the data acquisition efficiency is obviously and greatly improved.
It should be noted that the access information that can be obtained by the target user may only include the access information corresponding to the target user, or may include the access information corresponding to all users, and the specific setting may be performed according to actual needs, which are all within the protection scope of the present invention.
According to the technical scheme, when a user accesses a distributed system, access information comprising user information, barrel information, access time and byte number uploaded or downloaded is obtained, the access information is stored in each corresponding log, and then the access information is obtained from the log and returned to the user when the user needs to obtain flow information. Different from the prior art that the total access times, the total uploaded and downloaded data volume can only be obtained through a counter, various information when a user accesses a distributed system can be obtained through the mode of refining, collecting and storing the corresponding information, so that the information diversification is realized, and secondary processing in modes of time-sharing processing or user-sharing processing and the like is facilitated.
In addition, in the technical scheme disclosed by the application, the acquired access information is stored in the logs of different levels, so that when the access information is required to be acquired, the classification object corresponding to the access information required to be acquired can be directly positioned, and then the acquisition of the access information is realized. The method and the device for the flow statistics achieve layering of flow statistics in a distributed system, a user level and a bucket level, are more flexible, and the obtained data can be in the running state of the distributed system.
The method for acquiring traffic of a distributed system according to an embodiment of the present invention writes access information into each corresponding log, and may include:
and writing the access information into each corresponding log, wherein the logs comprise logs respectively used for storing the access information corresponding to the distributed system, each bucket in the distributed system and each user accessing the distributed system at different time periods.
It should be noted that, in the present application, each log may not only correspond to different classification objects, but also correspond to different time periods, such as year, month, day, hour, minute, and the like. Therefore, the access information is written into the log corresponding to the time period to which the access starting moment belongs, and when the log information is acquired, the corresponding log can be positioned from two points of a classification object and the time period, so that the acquisition of the corresponding access information is realized, the information classification storage and acquisition are realized from two levels of time and level, and the data storage and acquisition efficiency is further improved.
The method for acquiring traffic of a distributed system according to an embodiment of the present invention writes access information into each corresponding log, and may include:
and writing the access information into each log with a corresponding name, wherein the log is named by the time period and the identification of the classified object, and the classified object comprises a distributed system, a bucket and a user.
It should be noted that each log may be named by integrating the time period and the classification object, so that the log name may be obtained in the following ways:
1) bucket level: time period + bucket id
2) User level: time period + user id
3) System level: time period + system flag (customizable);
and then the log needing to be positioned can be directly determined by the log name when the access information is written or acquired, and the method is simple and easy to implement.
The method for acquiring traffic of a distributed system according to the embodiment of the present invention may further include, after acquiring corresponding access information from a log corresponding to an information request:
and judging whether flow calculation is needed, if so, calculating corresponding flow information based on the acquired access information and returning the flow information to the target user, and if not, executing a step of returning the acquired access information to the target user as the flow information, wherein the flow information comprises uploading or downloading bandwidth, the access times per second and total time for processing all accesses.
It should be noted that the access information can be directly returned to the user, the user can perform processing such as calculation on the access information according to the actual needs of the user, the access information can also be directly calculated, and then the result obtained by the calculation is returned to the user, so that the user can conveniently obtain the data, and the user experience is improved. Specifically, the uploaded or downloaded bandwidth may be calculated based on the number of bytes uploaded or downloaded in the acquired traffic information and the corresponding time, the number of access times processed per second is a ratio of the total number of access times included in the acquired traffic information to the total time used for all accesses, and the total time used for all accesses is a sum of the times used for accesses included in the acquired traffic information. Therefore, the information statistics of different levels is realized by relying on the logs of the distributed system, and the method is accurate and effective.
An embodiment of the present invention further provides a device for acquiring traffic of a distributed system, as shown in fig. 2, where the device may include:
an obtaining module 11, configured to: determining any user accessing the distributed system as a target user, and acquiring access information of the target user, wherein the access information comprises user information of the target user, barrel information of a barrel accessed by the target user, time accessed by the target user and the number of bytes uploaded or downloaded by the target user;
a write module 12 for: writing the access information into each corresponding log, wherein the logs comprise logs respectively used for storing the access information corresponding to each distributed system, each bucket in the distributed system and each user accessing the distributed system;
a return module 13 for: and receiving an information request sent by a target user, acquiring corresponding access information from a log corresponding to the information request, and returning the acquired access information serving as flow information to the target user.
In the traffic acquiring apparatus of the distributed system provided in the embodiment of the present invention, the writing module may include:
a write unit to: and writing the access information into each corresponding log, wherein the logs comprise logs respectively used for storing the access information corresponding to the distributed system, each bucket in the distributed system and each user accessing the distributed system at different time periods.
In the traffic acquiring apparatus of the distributed system provided in the embodiment of the present invention, the writing unit may include:
a write subunit to: and writing the access information into each log with a corresponding name, wherein the log is named by the time period and the identification of the classified object, and the classified object comprises a distributed system, a bucket and a user.
The traffic acquiring apparatus for a distributed system provided in the embodiments of the present invention may further include:
a determination module configured to: and the return module judges whether flow calculation is needed or not after acquiring corresponding access information from a log corresponding to the information request, if so, calculates the corresponding flow information based on the acquired access information and returns the flow information to the target user, and if not, instructs the return module to execute the step of returning the acquired access information to the target user as the flow information, wherein the flow information comprises the uploading or downloading bandwidth, the access times per second and the total time for processing all accesses.
An embodiment of the present invention further provides a device for acquiring traffic of a distributed system, where the device may include:
a memory for storing a computer program;
a processor for implementing the steps of the traffic acquisition method of the distributed system as described above when executing the computer program.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the traffic obtaining method of any one of the above distributed systems are implemented.
It should be noted that, for the description of the relevant parts in the traffic acquiring device, the equipment and the computer readable storage medium of the distributed system provided in the embodiment of the present invention, reference is made to the detailed description of the corresponding parts in the traffic acquiring method of the distributed system provided in the embodiment of the present invention, and details are not described herein again. In addition, parts of the technical solutions provided in the embodiments of the present invention that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail, so as to avoid redundant description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1.一种分布式系统的流量获取方法,其特征在于,包括:1. a kind of flow acquisition method of distributed system, is characterized in that, comprises: 确定任一访问分布式系统的用户为目的用户,并获取所述目的用户的访问信息,所述访问信息包括所述目的用户的用户信息、所述目的用户访问的桶的桶信息、所述目的用户访问的时间及所述目的用户上传或下载的字节数;Determine any user accessing the distributed system as the target user, and obtain the access information of the target user, where the access information includes the user information of the target user, the bucket information of the bucket accessed by the target user, and the target user. The time of user access and the number of bytes uploaded or downloaded by the user for the stated purpose; 将所述访问信息写入对应的每个日志中,所述日志包括分别用于存储所述分布式系统、所述分布式系统中各个桶、访问所述分布式系统的各个用户各自对应的访问信息的日志;Write the access information into each corresponding log, where the log includes the corresponding access information for storing the distributed system, each bucket in the distributed system, and each user accessing the distributed system. log of information; 接收所述目的用户发送的信息请求,由与所述信息请求对应的日志中获取对应的访问信息,并将获取的访问信息作为流量信息返回给所述目的用户;Receive the information request sent by the target user, obtain the corresponding access information from the log corresponding to the information request, and return the obtained access information as traffic information to the target user; 其中,所述信息请求中包括需要获取的访问信息对应分类对象的标识,分类对象为日志存储的访问信息对应的对象,包括分布式系统、桶、用户,而分类对象的标识为日志存储的访问信息对应的对象的标识,相应的在由与信息请求对应的日志中获取对应的访问信息时,具体为由所述信息请求中携带的标识对应分类对象的日志中获取所需的访问信息。Wherein, the information request includes the identifier of the classification object corresponding to the access information to be acquired, the classification object is the object corresponding to the access information stored in the log, including distributed systems, buckets, and users, and the identifier of the classification object is the access information stored in the log. The identifier of the object corresponding to the information, when the corresponding access information is obtained from the log corresponding to the information request, specifically, the required access information is obtained from the log of the classified object corresponding to the identifier carried in the information request. 2.根据权利要求1所述的方法,其特征在于,将所述访问信息写入对应的每个日志中,包括:2. The method according to claim 1, wherein writing the access information into each corresponding log, comprising: 将所述访问信息写入对应的每个日志中,所述日志包括分别用于存储不同时间段下所述分布式系统、所述分布式系统中各个桶、访问所述分布式系统的各个用户各自对应的访问信息的日志。Write the access information into each corresponding log, where the log includes the distributed system, each bucket in the distributed system, and each user accessing the distributed system in different time periods respectively. Each corresponding log of access information. 3.根据权利要求2所述的方法,其特征在于,将所述访问信息写入对应的每个日志中,包括:3. The method according to claim 2, wherein writing the access information into each corresponding log, comprising: 将所述访问信息写入具有对应名称的每个日志中,所述日志以所属时间段及所属分类对象的标识命名,所述分类对象包括分布式系统、桶及用户。The access information is written into each log with a corresponding name, where the log is named with the time period to which it belongs and the identifier of the category object to which it belongs, and the category object includes distributed systems, buckets, and users. 4.根据权利要求3所述的方法,其特征在于,由与所述信息请求对应的日志中获取对应的访问信息之后,还包括:4. The method according to claim 3, wherein after obtaining the corresponding access information from the log corresponding to the information request, further comprising: 判断是否需要进行流量计算,如果是,则基于获取的访问信息计算对应的流量信息并返回给所述目的用户,如果否,则执行所述将获取的访问信息作为流量信息返回给所述目的用户的步骤,其中,所述流量信息包括上传或下载的带宽、每秒处理的访问次数及处理全部访问所用的总时间。Determine whether it is necessary to perform flow calculation, if so, calculate the corresponding flow information based on the obtained access information and return it to the target user, if not, execute the described return of the obtained access information as flow information to the target user , wherein the traffic information includes upload or download bandwidth, the number of visits processed per second, and the total time used to process all the visits. 5.一种分布式系统的流量获取装置,其特征在于,包括:5. A flow acquisition device of a distributed system, characterized in that, comprising: 获取模块,用于:确定任一访问分布式系统的用户为目的用户,并获取所述目的用户的访问信息,所述访问信息包括所述目的用户的用户信息、所述目的用户访问的桶的桶信息、所述目的用户访问的时间及所述目的用户上传或下载的字节数;The obtaining module is used to: determine any user who accesses the distributed system as the target user, and obtain the access information of the target user, where the access information includes the user information of the target user and the information of the bucket accessed by the target user. Bucket information, the access time of the destination user, and the number of bytes uploaded or downloaded by the destination user; 写入模块,用于:将所述访问信息写入对应的每个日志中,所述日志包括分别用于存储所述分布式系统、所述分布式系统中各个桶、访问所述分布式系统的各个用户各自对应的访问信息的日志;a writing module, configured to: write the access information into each log corresponding to the log, where the log includes a log for storing the distributed system, each bucket in the distributed system, and accessing the distributed system. The log of the corresponding access information of each user; 返回模块,用于:接收所述目的用户发送的信息请求,由与所述信息请求对应的日志中获取对应的访问信息,并将获取的访问信息作为流量信息返回给所述目的用户;The returning module is configured to: receive the information request sent by the target user, obtain the corresponding access information from the log corresponding to the information request, and return the obtained access information as traffic information to the target user; 其中,所述信息请求中包括需要获取的访问信息对应分类对象的标识,分类对象为日志存储的访问信息对应的对象,包括分布式系统、桶、用户,而分类对象的标识为日志存储的访问信息对应的对象的标识,相应的所述返回模块在由与信息请求对应的日志中获取对应的访问信息时,具体为由所述信息请求中携带的标识对应分类对象的日志中获取所需的访问信息。Wherein, the information request includes the identifier of the classification object corresponding to the access information to be obtained, the classification object is the object corresponding to the access information stored in the log, including distributed systems, buckets, and users, and the identifier of the classification object is the access information stored in the log. The identifier of the object corresponding to the information. When the corresponding return module obtains the corresponding access information from the log corresponding to the information request, it is specifically to obtain the required information from the log of the classified object corresponding to the identifier carried in the information request. access information. 6.根据权利要求5所述的装置,其特征在于,所述写入模块包括:6. The device according to claim 5, wherein the writing module comprises: 写入单元,用于:将所述访问信息写入对应的每个日志中,所述日志包括分别用于存储不同时间段下所述分布式系统、所述分布式系统中各个桶、访问所述分布式系统的各个用户各自对应的访问信息的日志。The writing unit is configured to: write the access information into each corresponding log, where the log includes the distributed system, each bucket in the distributed system, and the access data in different time periods respectively. The log of the access information corresponding to each user of the distributed system. 7.根据权利要求6所述的装置,其特征在于,所述写入单元包括:7. The apparatus according to claim 6, wherein the writing unit comprises: 写入子单元,用于:将所述访问信息写入具有对应名称的每个日志中,所述日志以所属时间段及所属分类对象的标识命名,所述分类对象包括分布式系统、桶及用户。The writing subunit is used to: write the access information into each log with a corresponding name, the log is named with the identification of the time period to which it belongs and the classification object to which it belongs, and the classification objects include distributed systems, buckets and user. 8.根据权利要求7所述的装置,其特征在于,还包括:8. The apparatus of claim 7, further comprising: 判断模块,用于:所述返回模块由与所述信息请求对应的日志中获取对应的访问信息之后,判断是否需要进行流量计算,如果是,则基于获取的访问信息计算对应的流量信息并返回给所述目的用户,如果否,则指示所述返回模块执行所述将获取的访问信息作为流量信息返回给所述目的用户的步骤,其中,所述流量信息包括上传或下载的带宽、每秒处理的访问次数及处理全部访问所用的总时间。The judgment module is used for: after the return module obtains the corresponding access information from the log corresponding to the information request, judge whether it is necessary to perform flow calculation, and if so, calculate the corresponding flow information based on the obtained access information and return To the destination user, if not, instruct the return module to execute the step of returning the acquired access information as traffic information to the destination user, wherein the traffic information includes upload or download bandwidth, The number of visits processed and the total time it took to process all visits. 9.一种分布式系统的流量获取设备,其特征在于,包括:9. A flow acquisition device of a distributed system, characterized in that, comprising: 存储器,用于存储计算机程序;memory for storing computer programs; 处理器,用于执行所述计算机程序时实现如权利要求1至4任一项所述分布式系统的流量获取方法的步骤。The processor is configured to implement the steps of the method for obtaining traffic in a distributed system according to any one of claims 1 to 4 when executing the computer program. 10.一种计算机可读存储介质,其特征在于,所述计算机可读存储介质上存储有计算机程序,所述计算机程序被处理器执行时实现如权利要求1至4任一项所述分布式系统的流量获取方法的步骤。10. A computer-readable storage medium, characterized in that, a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the distributed storage medium according to any one of claims 1 to 4 is implemented. The steps of the traffic acquisition method of the system.
CN201811463382.6A 2018-12-03 2018-12-03 A method, device and device for obtaining traffic in a distributed system Active CN109560965B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811463382.6A CN109560965B (en) 2018-12-03 2018-12-03 A method, device and device for obtaining traffic in a distributed system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811463382.6A CN109560965B (en) 2018-12-03 2018-12-03 A method, device and device for obtaining traffic in a distributed system

Publications (2)

Publication Number Publication Date
CN109560965A CN109560965A (en) 2019-04-02
CN109560965B true CN109560965B (en) 2022-03-04

Family

ID=65868557

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811463382.6A Active CN109560965B (en) 2018-12-03 2018-12-03 A method, device and device for obtaining traffic in a distributed system

Country Status (1)

Country Link
CN (1) CN109560965B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112328864B (en) * 2020-11-04 2023-06-23 浪潮云信息技术股份公司 Downloading amount statistical method for Ceph object storage

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070038889A1 (en) * 2005-08-11 2007-02-15 Wiggins Robert D Methods and systems to access process control log information associated with process control systems
US8121061B2 (en) * 2008-06-26 2012-02-21 Microsoft Corporation Efficient file management through granular opportunistic locking
US8949414B2 (en) * 2010-12-29 2015-02-03 Citrix Systems, Inc. Systems and methods for scalable N-core stats aggregation
CN105634845B (en) * 2014-10-30 2019-01-22 任子行网络技术股份有限公司 A kind of method and system for magnanimity DNS log progress multidimensional statistics analysis
CN105138592B (en) * 2015-07-31 2019-03-26 武汉虹信技术服务有限责任公司 A kind of daily record data storage and search method based on distributed structure/architecture
CN107612707B (en) * 2017-08-04 2021-04-09 深圳市其乐游戏科技有限公司 Preprocessing method and system for classified storage of homologous sample data in industry field
CN107577577A (en) * 2017-09-13 2018-01-12 郑州云海信息技术有限公司 Method, apparatus, system and the computer-readable recording medium of log-rank printing
CN108345684A (en) * 2018-03-06 2018-07-31 弘成科技发展有限公司 Intelligent analysis system and analysis method based on various dimensions multi objective system

Also Published As

Publication number Publication date
CN109560965A (en) 2019-04-02

Similar Documents

Publication Publication Date Title
CN111064808B (en) Load balancing method and device based on distributed storage system
CN109388550B (en) A cache hit rate determination method, apparatus, device and readable storage medium
CN108572795B (en) Capacity expansion method, device, device and storage medium based on building storage virtualization
TW202040387A (en) Method for constructing index of data record
CN112925783B (en) Business data processing method and device, electronic device and storage medium
US20080005271A1 (en) Data logging for resident applications within portable electronic devices
CN111090620B (en) A file storage method, apparatus, device and readable storage medium
CN105468989A (en) Cloud storage quota management method based on Linux kernel monitoring
CN107251523B (en) Data storage method based on cloud service, integrity detection method and device and terminal equipment
CN109246234A (en) A kind of image file method for down loading, device, electronic equipment and storage medium
US20170222886A1 (en) Metering storage usage
CN102999604B (en) The detection method and device of a kind of database performance
CN109560965B (en) A method, device and device for obtaining traffic in a distributed system
CN107092678A (en) A kind of method, device and equipment for obtaining application active degree
CN111427920B (en) Data acquisition method, device, system, computer equipment and storage medium
CN114429360B (en) Conversion rate determination method, conversion rate determination device, electronic device, and computer-readable storage medium
CN111488256B (en) Memory monitoring method, device, equipment and storage medium
CN111083010B (en) A speed measurement method, device and computer readable storage medium
CN106921536B (en) Data processing method and device based on client version information
CN104750717B (en) A kind of click volume recording method, server and client
CN113570411B (en) Popularization information display method and device, electronic equipment and storage medium
US20220156779A1 (en) Method, apparatus, and computer device for calculating website user credit point
CN119066105A (en) Interaction data volume determination method, device, equipment and readable storage medium
CN109117091A (en) A kind of SSD equipment mount point acquisition methods and relevant apparatus
US10614306B2 (en) Registering the harvest of a resource using image data and metadata

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant