CN115378804B - Log acquisition method, device, equipment and medium based on cloud edge cooperation - Google Patents
Log acquisition method, device, equipment and medium based on cloud edge cooperation Download PDFInfo
- Publication number
- CN115378804B CN115378804B CN202211000874.8A CN202211000874A CN115378804B CN 115378804 B CN115378804 B CN 115378804B CN 202211000874 A CN202211000874 A CN 202211000874A CN 115378804 B CN115378804 B CN 115378804B
- Authority
- CN
- China
- Prior art keywords
- log
- preset
- internet
- cloud platform
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000004590 computer program Methods 0.000 claims description 16
- 230000006855 networking Effects 0.000 claims description 4
- 238000004891 communication Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000295 complement effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/04—Network management architectures or arrangements
- H04L41/042—Network management architectures or arrangements comprising distributed management centres cooperatively managing the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application relates to the technical field of cloud edge coordination, and provides a log acquisition method, device, equipment and medium based on cloud edge coordination, wherein the method comprises the following steps: the method comprises the steps that an Internet of things cloud platform collects the full logs of each direct-connected device and a first preset log of each sub-device, writes a first path, and reads the logs from the first path to a configuration component according to a first configuration strategy; the edge gateway collects a second preset log of each piece of sub equipment, writes the second path, reads the log from the second path to the configuration component according to a second configuration strategy, and determines the log in the configuration component as a target log. According to the method, the device related logs can be cooperatively collected on the Internet of things cloud platform and the edge gateway, the Internet of things cloud platform collects the logs of the directly connected devices and part of logs of the sub-devices, the edge gateway collects the rest logs of the sub-devices, and the complete logs of all the devices are comprehensively obtained by combining the log information collected by the cloud edges, so that the problems of missing and imperfect log collection information of the Internet of things cloud platform are solved.
Description
Technical Field
The application relates to the technical field of cloud edge coordination, in particular to a log acquisition method, device, equipment and medium based on cloud edge coordination.
Background
In the prior art, the realized log acquisition scheme of the Internet of things basically has only one cloud layer, and the device logs are uniformly acquired at one side of the cloud platform of the Internet of things.
However, for the sub-equipment accessed through the edge gateway, because the log acquisition logic is realized by a third party, the log acquisition logic has no standardization, so that whether the related log information is acquired or not cannot be ensured, and the authenticity of the acquired information cannot be ensured.
Disclosure of Invention
The embodiment of the application provides a log acquisition method, a device, computer equipment and a storage medium based on cloud edge cooperation, which aim to solve the problems of log information deficiency and imperfect caused by unified acquisition of equipment logs only at one side of an Internet of things cloud platform.
In a first aspect, an embodiment of the present application provides a log collection method based on cloud edge collaboration, which is applied to a cloud edge collaboration system, where the cloud edge collaboration system includes an internet of things cloud platform and an edge gateway that are in communication with each other, and the method includes:
The Internet of things cloud platform collects the full log of each piece of directly connected equipment directly connected with the Internet of things cloud platform, and collects a first preset log of each piece of sub-equipment indirectly connected with the Internet of things cloud platform through the edge gateway;
The Internet of things cloud platform writes each full log and each first preset log into a first path corresponding to the Internet of things cloud platform;
The Internet of things cloud platform reads each full log from the first path according to a first configuration strategy and each first preset log to a configuration component of the Internet of things cloud platform;
the edge gateway collects a second preset log of each piece of sub-equipment connected with the edge gateway;
The edge gateway writes each second preset log into a second path corresponding to the edge gateway;
the edge gateway reads each second preset log from the second path to the configuration component according to a second configuration strategy;
and determining the log in the configuration component as a target log.
In a second aspect, an embodiment of the present application provides a log collection device based on cloud edge collaboration, which operates in a cloud edge collaboration system, where the cloud edge collaboration system includes an internet of things cloud platform and an edge gateway that are in communication with each other, and the device includes:
The acquisition unit is used for acquiring the full log of each piece of directly connected equipment directly connected with the internet of things cloud platform and acquiring a first preset log of each piece of sub equipment indirectly connected with the internet of things cloud platform through the edge gateway;
The writing unit is used for writing each full log and each first preset log into a first path corresponding to the cloud platform of the Internet of things;
the reading unit is used for reading each full log and each first preset log from the first path to the configuration component of the Internet of things cloud platform according to a first configuration strategy;
the acquisition unit is also used for acquiring a second preset log of each piece of sub-equipment connected with the acquisition unit;
The writing unit is further configured to write each second preset log into a second path corresponding to the edge gateway;
The reading unit is further configured to read each second preset log from the second path to the configuration component according to a second configuration policy;
and the determining unit is used for determining the log in the configuration component as a target log.
In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the log collection method based on cloud edge collaboration according to the first aspect when executing the computer program.
In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, where the computer program when executed by a processor causes the processor to execute the log collection method based on cloud edge collaboration according to the first aspect.
The embodiment of the application provides a log acquisition method, device, equipment and medium based on cloud edge collaboration, which can acquire equipment related logs on an Internet of things cloud platform and an edge gateway in a collaborative manner, wherein the Internet of things cloud platform acquires direct-connected equipment logs and partial logs of sub-equipment, the edge gateway acquires the rest logs of the sub-equipment, and the complete logs of all the equipment are comprehensively obtained by combining log information acquired by cloud edges, so that the problems of missing and imperfect log acquisition information of the traditional Internet of things cloud platform are solved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is an application scenario schematic diagram of a log collection method based on cloud edge collaboration provided by an embodiment of the present application;
Fig. 2 is a flow chart of a log collection method based on cloud edge collaboration according to an embodiment of the present application;
FIG. 3 is a schematic block diagram of a log collection device based on cloud edge collaboration according to an embodiment of the present application;
fig. 4 is a schematic block diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic diagram of an application scenario of a log collection method based on cloud edge collaboration according to an embodiment of the present application; fig. 2 is a flow chart of a log collection method based on cloud edge coordination, which is provided by the embodiment of the application, and is applied to a cloud edge coordination system, wherein the cloud edge coordination system comprises an internet of things cloud platform and an edge gateway which are communicated with each other, the internet of things cloud platform is connected with at least one direct-connection device, and at least one sub-device is communicated with the internet of things cloud platform through the edge gateway. The method is executed by application software installed in the cloud-edge cooperative system.
As shown in fig. 2, the method includes steps S101 to S107.
S101, the Internet of things cloud platform collects all logs of each piece of directly-connected equipment directly connected with the Internet of things cloud platform, and collects a first preset log of each piece of sub-equipment indirectly connected with the Internet of things cloud platform through the edge gateway.
It can be appreciated that, because each direct-connection device is in direct communication with the internet of things cloud platform, the direct collection of the full log of each direct-connection device on the internet of things cloud platform is more direct and rapid.
Specifically, each full log may include, but is not limited to, one or a combination of the following: the method comprises the steps of connecting or disconnecting logs of the direct-connected equipment with changed state, logs of success or failure of issuing a direct-connected equipment command written with a message, and alarm logs when the direct-connected equipment alarms.
In addition, since each piece of equipment is not in direct communication with the internet of things cloud platform, in order to ensure the efficiency of log collection, the internet of things cloud platform only needs to collect part of the log of each piece of equipment as the first preset log, specifically, according to the collection efficiency, the collection effect and the log attribute (such as whether a command issued by some pieces of equipment is successfully known only by the edge gateway, the log of the attribute can be determined according to the attribute feature to be unsuitable for being collected by the internet of things cloud platform), which part of the log of each piece of equipment is collected by the internet of things cloud platform is determined.
Wherein each first preset log may include: there is an alarm log when the child device is alerted.
In the embodiment, all logs of the directly connected equipment and part of logs of the sub-equipment are collected by the internet of things cloud platform.
In this embodiment, after the cloud platform of the internet of things reads each full log from the first path according to a first configuration policy, the method further includes:
and when the full log with the direct-connected equipment is detected to be updated, the Internet of things cloud platform acquires the updated full log.
By detecting the update condition of the log, the log is collected in time when the log is changed, and the comprehensiveness of the log collection can be ensured.
It should be noted that, the manner in which the internet of things cloud platform collects the first preset log of each piece of sub-equipment indirectly connected with the internet of things cloud platform through the edge gateway is similar to the manner in which the internet of things cloud platform collects the full log of each piece of directly connected equipment directly connected with the internet of things cloud platform, which is not described herein in detail.
In this embodiment, the internet of things cloud platform may perform log collection through a buried point technology, and the present invention does not limit the collection technology used when collecting logs.
S102, the Internet of things cloud platform writes each full log and each first preset log into a first path corresponding to the Internet of things cloud platform.
In this embodiment, the first path may be configured in a user-defined manner according to an actual storage requirement and a storage environment.
For example: the first path may be a designated path on a server of the internet of things cloud platform, such as/var/log/xxx.
In the above embodiment, the collected logs are written into the designated paths, so as to facilitate subsequent log reading.
S103, the Internet of things cloud platform reads each full log and each first preset log from the first path according to a first configuration strategy to a configuration component of the Internet of things cloud platform.
Wherein, the first configuration policy may be configured according to an execution environment and the like.
Specifically, the internet of things cloud platform reads each full log and each first preset log from the first path according to a first configuration policy to a configuration component of the internet of things cloud platform, which may include any one of the following modes:
reading each full log and each first preset log from the first path according to the first configuration strategy, and further:
(1) And circularly detecting the total amount of the full log and the first preset log read by the Internet of things cloud platform, and uploading the read full log and the read first preset log to the configuration component when the total amount reaches a preset value.
The preset value may be configured according to parameters such as an upper limit value of the actually used cache, and the invention is not limited.
The configuration component can be ES (Elasticsearch) components of the cloud platform of the Internet of things, and the ES component is a distributed document database, can rapidly expand, store and process PB level data, and can perform query, analysis and search of mass data in a short time.
(2) And uploading the read full log and the first preset log to the configuration component at preset time intervals.
The preset time interval can be configured in a self-defined mode so as to read the log according to actual requirements.
In this embodiment, the Fluent Bit component of the cloud platform of the internet of things may be used to read the log from the first path.
The Fluent Bit component is a log collector and a processor.
S104, the edge gateway collects a second preset log of each piece of sub-equipment connected with the edge gateway.
It will be appreciated that, in order to ensure the integrity of the collected log of each sub-device, the second preset log may be configured to be complementary to the first preset log.
Specifically, each second preset log may include, but is not limited to, one or a combination of the following: the method comprises the steps of connecting or disconnecting logs of the sub-equipment with changed states, logs of successful or failed issuing of commands of the sub-equipment with written information, alarm logs when alarms occur to the sub-equipment determined according to the information transmitted by the data processing module of the edge gateway, and failure logs when the sub-equipment fails to report collected data.
The data among each full log, each first preset log and each second preset log are not repeated.
Through the configuration, repeated collection of the logs can be avoided on the premise of ensuring that the collected logs are more comprehensive, so that the collection efficiency of the logs is improved, and meanwhile, the condition that the collected logs have data redundancy is avoided.
Specifically, the edge gateway can also collect logs through a buried point technology, and the invention does not limit the collection technology used in collecting logs.
S105, the edge gateway writes each second preset log into a second path corresponding to the edge gateway.
In this embodiment, the second path may also be configured in a user-defined manner according to an actual storage requirement and a storage environment.
For example: the second path may be a designated path on a server of the edge gateway, such as/var/log/yyy.
In the above embodiment, the collected logs are written into the designated paths, so as to facilitate subsequent log reading.
S106, the edge gateway reads each second preset log from the second path to the configuration component according to a second configuration strategy.
Wherein the second configuration policy may be configured according to an execution environment or the like.
Specifically, the edge gateway reads each second preset log from the second path to the configuration component according to a second configuration policy, and may include any one of the following manners:
(1) And circularly detecting the total amount of the second preset log read by the edge gateway, and uploading the read second preset log to the configuration component when the total amount reaches a configuration value.
The configuration value may be configured according to parameters such as an upper limit value of the actually used cache, which is not limited in the present invention.
(2) Each second preset log is read from the second path to the configuration component at intervals of configuration time.
The configuration time interval can be configured in a self-defined manner so as to read the log according to actual requirements.
In this embodiment, the Fluent Bit component of the edge gateway may be utilized to read the log from the second path.
The Fluent Bit component is a log collector and a processor.
S107, determining the log in the configuration component as a target log.
For example: the log record that thing networking cloud platform gathered is:
device1 2021-08-26:16:05:07 online
Device2 2021-08-27 16:05:07 online
And, the log record that the edge gateway gathered is:
Device3 2021-08-27:13:05:07 online
Device4 2021-08-22 11:05:07 online
The final determined target log record is:
device1 2021-08-26:16:05:07 online
Device2 2021-08-27 16:05:07 online
Device3 2021-08-27:13:05:07 online
Device4 2021-08-22:11:05:07 is online.
Through the embodiment, the complete log of all devices is comprehensively obtained by combining the log information acquired by the cloud edge, and the problems of missing and imperfect log acquisition information of the traditional internet of things cloud platform are solved.
In this embodiment, after the determining the log in the configuration component as the target log, the method further includes:
Calling an interface of the configuration component through an API (Application Program Interface, application programming interface) gateway of the Internet of things cloud platform to acquire the target log;
and displaying the target log.
Specifically, the target log may be displayed in the form of a device name, acquisition time, and status.
For example: device1 2021-08-26:05:07 is online.
By displaying the target log, related personnel can conveniently check the log.
In this embodiment, after the determining the log in the configuration component as the target log, the method further includes:
Responding to a log query instruction, and analyzing the log query instruction to obtain query condition information;
inquiring in the target log according to the inquiring condition information;
Determining the queried log as a log to be fed back;
And displaying the log to be fed back.
The log inquiry command can be triggered by related staff.
Wherein, the query condition information may include, but is not limited to: the name of the log, the time range of generation of the log, and the like.
In this embodiment, the log to be fed back may be displayed and sent to a terminal device of a relevant worker, so as to facilitate subsequent analysis.
According to the technical scheme, the device-related logs can be cooperatively collected on the Internet of things cloud platform and the edge gateway, the Internet of things cloud platform collects the logs of the directly connected devices and part of logs of the sub-devices, the edge gateway collects the rest logs of the sub-devices, and the whole logs of all devices are comprehensively obtained by combining the log information collected by the cloud edges, so that the problems of missing and imperfect log collection information of the traditional Internet of things cloud platform are solved.
The embodiment of the application also provides a log acquisition device based on cloud edge cooperation, which is used for executing any embodiment of the log acquisition method based on cloud edge cooperation. Referring to fig. 3, fig. 3 is a schematic block diagram of a log collection device 100 based on cloud-edge collaboration according to an embodiment of the present application.
As shown in fig. 3, the log collection device 100 based on cloud-edge collaboration includes a collection unit 101, a writing unit 102, a reading unit 103, and a determining unit 104.
The collection unit 101 is configured to collect a full log of each directly connected device directly connected to the internet of things cloud platform, and collect a first preset log of each sub-device indirectly connected to the internet of things cloud platform through the edge gateway.
It can be appreciated that, because each direct-connection device is in direct communication with the internet of things cloud platform, the direct collection of the full log of each direct-connection device on the internet of things cloud platform is more direct and rapid.
Specifically, each full log may include, but is not limited to, one or a combination of the following: the method comprises the steps of connecting or disconnecting logs of the direct-connected equipment with changed state, logs of success or failure of issuing a direct-connected equipment command written with a message, and alarm logs when the direct-connected equipment alarms.
In addition, since each piece of equipment is not in direct communication with the internet of things cloud platform, in order to ensure the efficiency of log collection, the internet of things cloud platform only needs to collect part of the log of each piece of equipment as the first preset log, specifically, according to the collection efficiency, the collection effect and the log attribute (such as whether a command issued by some pieces of equipment is successfully known only by the edge gateway, the log of the attribute can be determined according to the attribute feature to be unsuitable for being collected by the internet of things cloud platform), which part of the log of each piece of equipment is collected by the internet of things cloud platform is determined.
Wherein each first preset log may include: there is an alarm log when the child device is alerted.
In the embodiment, all logs of the directly connected equipment and part of logs of the sub-equipment are collected by the internet of things cloud platform.
In this embodiment, after the cloud platform of the internet of things reads each full log from the first path according to a first configuration policy, when detecting that the full log of the directly connected device is updated, the collecting unit 101 collects the updated full log.
By detecting the update condition of the log, the log is collected in time when the log is changed, and the comprehensiveness of the log collection can be ensured.
It should be noted that, the manner in which the collection unit 101 collects the first preset log of each piece of sub-equipment indirectly connected to the internet of things cloud platform through the edge gateway is similar to the manner in which the collection unit 101 collects the total log of each piece of directly connected equipment directly connected to the internet of things cloud platform, which is not described herein.
In this embodiment, the collecting unit 101 may collect logs by using a buried point technology, and the present invention does not limit the collecting technology used in collecting logs.
The writing unit 102 is configured to write each full log and each first preset log into a first path corresponding to the cloud platform of the internet of things.
In this embodiment, the first path may be configured in a user-defined manner according to an actual storage requirement and a storage environment.
For example: the first path may be a designated path on a server of the internet of things cloud platform, such as/var/log/xxx.
In the above embodiment, the collected logs are written into the designated paths, so as to facilitate subsequent log reading.
The reading unit 103 is configured to read each full log and each first preset log from the first path to a configuration component of the internet of things cloud platform according to a first configuration policy.
Wherein, the first configuration policy may be configured according to an execution environment and the like.
Specifically, the reading unit 103 reads each full log and each first preset log from the first path to the configuration component of the internet of things cloud platform according to a first configuration policy, which may include any one of the following manners:
reading each full log and each first preset log from the first path according to the first configuration strategy, and further:
(1) And circularly detecting the total amount of the total amount log and the first preset log read by the reading unit 103, and uploading the read total amount log and the first preset log to the configuration component when the total amount reaches a preset value.
The preset value may be configured according to parameters such as an upper limit value of the actually used cache, and the invention is not limited.
The configuration component can be ES (Elasticsearch) components of the cloud platform of the Internet of things, and the ES component is a distributed document database, can rapidly expand, store and process PB level data, and can perform query, analysis and search of mass data in a short time.
(2) And uploading the read full log and the first preset log to the configuration component at preset time intervals.
The preset time interval can be configured in a self-defined mode so as to read the log according to actual requirements.
In this embodiment, the Fluent Bit component of the cloud platform of the internet of things may be used to read the log from the first path.
The Fluent Bit component is a log collector and a processor.
The collection unit 101 is further configured to collect a second preset log of each piece of sub-equipment connected to the collection unit.
It will be appreciated that, in order to ensure the integrity of the collected log of each sub-device, the second preset log may be configured to be complementary to the first preset log.
Specifically, each second preset log may include, but is not limited to, one or a combination of the following: the method comprises the steps of connecting or disconnecting logs of the sub-equipment with changed states, logs of successful or failed issuing of commands of the sub-equipment with written information, alarm logs when alarms occur to the sub-equipment determined according to the information transmitted by the data processing module of the edge gateway, and failure logs when the sub-equipment fails to report collected data.
The data among each full log, each first preset log and each second preset log are not repeated.
Through the configuration, repeated collection of the logs can be avoided on the premise of ensuring that the collected logs are more comprehensive, so that the collection efficiency of the logs is improved, and meanwhile, the condition that the collected logs have data redundancy is avoided.
Specifically, the collecting unit 101 may also collect logs by using a buried point technology, and the present invention does not limit the collecting technology used in collecting logs.
The writing unit 102 is further configured to write each second preset log into a second path corresponding to the edge gateway.
In this embodiment, the second path may also be configured in a user-defined manner according to an actual storage requirement and a storage environment.
For example: the second path may be a designated path on a server of the edge gateway, such as/var/log/yyy.
In the above embodiment, the collected logs are written into the designated paths, so as to facilitate subsequent log reading.
The reading unit 103 is further configured to read each second preset log from the second path to the configuration component according to a second configuration policy.
Wherein the second configuration policy may be configured according to an execution environment or the like.
Specifically, the reading unit 103 reads each second preset log from the second path to the configuration component according to a second configuration policy, which may include any one of the following manners:
(1) And circularly detecting the total amount of the second preset log read by the reading unit 103, and uploading the read second preset log to the configuration component when the total amount reaches a configuration value.
The configuration value may be configured according to parameters such as an upper limit value of the actually used cache, which is not limited in the present invention.
(2) Each second preset log is read from the second path to the configuration component at intervals of configuration time.
The configuration time interval can be configured in a self-defined manner so as to read the log according to actual requirements.
In this embodiment, the Fluent Bit component of the edge gateway may be utilized to read the log from the second path.
The Fluent Bit component is a log collector and a processor.
The determining unit 104 is configured to determine a log in the configuration component as a target log.
For example: the log record that thing networking cloud platform gathered is:
device1 2021-08-26:16:05:07 online
Device2 2021-08-27 16:05:07 online
And, the log record that the edge gateway gathered is:
Device3 2021-08-27:13:05:07 online
Device4 2021-08-22 11:05:07 online
The target log finally determined by the determining unit 104 is recorded as:
device1 2021-08-26:16:05:07 online
Device2 2021-08-27 16:05:07 online
Device3 2021-08-27:13:05:07 online
Device4 2021-08-22:11:05:07 is online.
Through the embodiment, the complete log of all devices is comprehensively obtained by combining the log information acquired by the cloud edge, and the problems of missing and imperfect log acquisition information of the traditional internet of things cloud platform are solved.
In this embodiment, after the determining unit 104 determines the log in the configuration component as a target log, the determining unit invokes an interface of the configuration component through an API (Application Program Interface, application programming interface) gateway of the internet of things cloud platform to obtain the target log;
and displaying the target log.
Specifically, the target log may be displayed in the form of a device name, acquisition time, and status.
For example: device1 2021-08-26:05:07 is online.
By displaying the target log, related personnel can conveniently check the log.
In this embodiment, after the determining unit 104 determines the log in the configuration component as the target log, the log query instruction is parsed to obtain query condition information in response to the log query instruction;
inquiring in the target log according to the inquiring condition information;
Determining the queried log as a log to be fed back;
And displaying the log to be fed back.
The log inquiry command can be triggered by related staff.
Wherein, the query condition information may include, but is not limited to: the name of the log, the time range of generation of the log, and the like.
In this embodiment, the log to be fed back may be displayed and sent to a terminal device of a relevant worker, so as to facilitate subsequent analysis.
According to the technical scheme, the device-related logs can be cooperatively collected on the Internet of things cloud platform and the edge gateway, the Internet of things cloud platform collects the logs of the directly connected devices and part of logs of the sub-devices, the edge gateway collects the rest logs of the sub-devices, and the whole logs of all devices are comprehensively obtained by combining the log information collected by the cloud edges, so that the problems of missing and imperfect log collection information of the traditional Internet of things cloud platform are solved.
The log collection device based on cloud edge collaboration can be implemented in the form of a computer program, and the computer program can run on a computer device as shown in fig. 4.
Referring to fig. 4, fig. 4 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 is a server, or a cluster of servers. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.
Referring to fig. 4, the computer apparatus 500 includes a processor 502, a memory, and a network interface 505, which are connected by a device bus 501, wherein the memory may include a storage medium 503 and an internal memory 504.
The storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032, when executed, may cause the processor 502 to perform a cloud-edge collaboration based log collection method.
The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the execution of a computer program 5032 in the storage medium 503, which computer program 5032, when executed by the processor 502, may cause the processor 502 to perform a cloud-based collaborative log collection method.
The network interface 505 is used for network communication, such as providing for transmission of data information, etc. It will be appreciated by those skilled in the art that the architecture shown in fig. 4 is merely a block diagram of some of the architecture relevant to the present inventive arrangements and is not limiting of the computer device 500 to which the present inventive arrangements may be implemented, and that a particular computer device 500 may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
The processor 502 is configured to run a computer program 5032 stored in a memory, so as to implement the log collection method based on cloud edge collaboration disclosed in the embodiment of the present application.
Those skilled in the art will appreciate that the embodiment of the computer device shown in fig. 4 is not limiting of the specific construction of the computer device, and in other embodiments, the computer device may include more or less components than those shown, or certain components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device may include only a memory and a processor, and in such embodiments, the structure and function of the memory and the processor are consistent with the embodiment shown in fig. 4, and will not be described again.
It should be appreciated that in embodiments of the present application, the Processor 502 may be a central processing unit (Central Processing Unit, CPU), the Processor 502 may also be other general purpose processors, digital signal processors (DIGITAL SIGNAL processors, DSPs), application SPECIFIC INTEGRATED Circuits (ASICs), off-the-shelf Programmable gate arrays (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In another embodiment of the application, a computer-readable storage medium is provided. The computer readable storage medium may be a nonvolatile computer readable storage medium or a volatile computer readable storage medium. The computer readable storage medium stores a computer program, wherein the computer program realizes the log acquisition method based on cloud edge cooperation disclosed by the embodiment of the application when being executed by a processor.
The data in this case were obtained legally.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus, device and unit described above may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein. Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the units is merely a logical function division, there may be another division manner in actual implementation, or units having the same function may be integrated into one unit, for example, multiple units or components may be combined or may be integrated into another apparatus, or some features may be omitted, or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices, or elements, or may be an electrical, mechanical, or other form of connection.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present application.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units may be stored in a storage medium if implemented in the form of software functional units and sold or used as stand-alone products. Based on such understanding, the technical solution of the present application may be essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a background server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
While the application has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the application. Therefore, the protection scope of the application is subject to the protection scope of the claims.
Claims (10)
1. The utility model provides a log collection method based on cloud limit cooperation, is applied to cloud limit cooperation system, cloud limit cooperation system includes thing networking cloud platform and border gateway of intercommunication, its characterized in that, the method includes:
The Internet of things cloud platform collects the full log of each piece of directly connected equipment directly connected with the Internet of things cloud platform, and collects a first preset log of each piece of sub-equipment indirectly connected with the Internet of things cloud platform through the edge gateway;
The Internet of things cloud platform writes each full log and each first preset log into a first path corresponding to the Internet of things cloud platform;
The Internet of things cloud platform reads each full log from the first path according to a first configuration strategy and each first preset log to a configuration component of the Internet of things cloud platform;
the edge gateway collects a second preset log of each piece of sub-equipment connected with the edge gateway;
The edge gateway writes each second preset log into a second path corresponding to the edge gateway;
the edge gateway reads each second preset log from the second path to the configuration component according to a second configuration strategy;
and determining the log in the configuration component as a target log.
2. The log collection method based on cloud edge cooperation according to claim 1, wherein the log collection method is characterized in that:
Each full log includes a combination of one or more of the following: the connection or disconnection log of the direct-connected equipment with changed state, the log of success or failure of issuing the direct-connected equipment command written with the message, and the alarm log when the direct-connected equipment alarms;
each first preset log includes: an alarm log when the sub-equipment generates an alarm;
each second preset log includes one or more of the following combinations: the method comprises the steps of connecting or disconnecting logs of the sub-equipment with changed states, logs of successful or failed issuing of commands of the sub-equipment with written information, alarm logs when alarms occur to the sub-equipment determined according to the information transmitted by a data processing module of the edge gateway, and failure logs when the sub-equipment reports the acquired data;
the data among each full log, each first preset log and each second preset log are not repeated.
3. The cloud edge collaboration-based log collection method according to claim 1, wherein after the internet of things cloud platform reads each full log from the first path according to a first configuration policy, the method further comprises:
and when the full log with the direct-connected equipment is detected to be updated, the Internet of things cloud platform acquires the updated full log.
4. The cloud edge collaboration-based log collection method according to claim 1, wherein the internet of things cloud platform reads each full log and each first preset log from the first path to a configuration component of the internet of things cloud platform according to a first configuration policy, and the method comprises the following steps:
Reading each full log and each first preset log from the first path according to the first configuration strategy;
the total amount of logs and the first preset logs read by the Internet of things cloud platform is circularly detected, and when the total amount reaches a preset value, the read total amount of logs and the first preset logs are uploaded to the configuration component; or alternatively
And uploading the read full log and the first preset log to the configuration component at preset time intervals.
5. The cloud edge collaboration-based log collection method according to claim 1, wherein after determining the log in the configuration component as the target log, the method further comprises:
the interface of the configuration component is called through the API gateway of the Internet of things cloud platform to acquire the target log;
and displaying the target log.
6. The cloud edge collaboration-based log collection method according to claim 1, wherein after determining the log in the configuration component as the target log, the method further comprises:
Responding to a log query instruction, and analyzing the log query instruction to obtain query condition information;
inquiring in the target log according to the inquiring condition information;
Determining the queried log as a log to be fed back;
And displaying the log to be fed back.
7. Log collection system based on cloud limit is cooperated, operates in cloud limit cooperation system, cloud limit cooperation system includes thing networking cloud platform and border gateway of intercommunication, its characterized in that, the device includes:
The acquisition unit is used for acquiring the full log of each piece of directly connected equipment directly connected with the internet of things cloud platform and acquiring a first preset log of each piece of sub equipment indirectly connected with the internet of things cloud platform through the edge gateway;
The writing unit is used for writing each full log and each first preset log into a first path corresponding to the cloud platform of the Internet of things;
the reading unit is used for reading each full log and each first preset log from the first path to the configuration component of the Internet of things cloud platform according to a first configuration strategy;
the acquisition unit is also used for acquiring a second preset log of each piece of sub-equipment connected with the acquisition unit;
The writing unit is further configured to write each second preset log into a second path corresponding to the edge gateway;
The reading unit is further configured to read each second preset log from the second path to the configuration component according to a second configuration policy;
and the determining unit is used for determining the log in the configuration component as a target log.
8. The cloud-edge collaboration-based log collection device as claimed in claim 7, wherein:
Each full log includes a combination of one or more of the following: the connection or disconnection log of the direct-connected equipment with changed state, the log of success or failure of issuing the direct-connected equipment command written with the message, and the alarm log when the direct-connected equipment alarms;
Each first preset log includes one or more of the following combinations: an alarm log when the sub-equipment generates an alarm;
each second preset log includes one or more of the following combinations: the method comprises the steps of connecting or disconnecting logs of the sub-equipment with changed states, logs of successful or failed issuing of commands of the sub-equipment with written information, alarm logs when alarms occur to the sub-equipment determined according to the information transmitted by a data processing module of the edge gateway, and failure logs when the sub-equipment reports the acquired data;
the data among each full log, each first preset log and each second preset log are not repeated.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the cloud-based collaborative journal collection method of any one of claims 1-6 when the computer program is executed.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, which when executed by a processor causes the processor to perform the cloud-edge collaboration-based log collection method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211000874.8A CN115378804B (en) | 2022-08-19 | 2022-08-19 | Log acquisition method, device, equipment and medium based on cloud edge cooperation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211000874.8A CN115378804B (en) | 2022-08-19 | 2022-08-19 | Log acquisition method, device, equipment and medium based on cloud edge cooperation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115378804A CN115378804A (en) | 2022-11-22 |
| CN115378804B true CN115378804B (en) | 2024-06-14 |
Family
ID=84065493
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211000874.8A Active CN115378804B (en) | 2022-08-19 | 2022-08-19 | Log acquisition method, device, equipment and medium based on cloud edge cooperation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115378804B (en) |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103840953B (en) * | 2012-11-21 | 2017-10-31 | 中兴通讯股份有限公司 | Remote journal real time acquiring method and system |
| CN111835562B (en) * | 2020-07-01 | 2023-10-17 | 北京爱笔科技有限公司 | Log uploading method, log inquiring method and system |
| CN112231296B (en) * | 2020-09-30 | 2024-05-28 | 北京金山云网络技术有限公司 | Distributed log processing method, device, system, equipment and medium |
| CN113141393B (en) * | 2021-03-25 | 2023-04-07 | 杭州博联智能科技股份有限公司 | Dynamic edge gateway log collection and management method, system, device and medium |
-
2022
- 2022-08-19 CN CN202211000874.8A patent/CN115378804B/en active Active
Non-Patent Citations (2)
| Title |
|---|
| Mengyu Sun ; Zhangbing Zhou.IoT Services Configuration in Edge-Cloud Collaboration Networks.2020 IEEE International Conference on Web Services (ICWS).2020,全文. * |
| 分布式日志采集系统数据传输分析研究;齐剑雄;郭燕慧;;软件;20121015(第10期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115378804A (en) | 2022-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11036591B2 (en) | Restoring partitioned database tables from backup | |
| US9817703B1 (en) | Distributed lock management using conditional updates to a distributed key value data store | |
| US11544288B2 (en) | Systems and methods for managing distributed database deployments | |
| US10713280B2 (en) | Systems and methods for managing distributed database deployments | |
| US20210064476A1 (en) | Backup of partitioned database tables | |
| US11327949B2 (en) | Verification of database table partitions during backup | |
| US8122111B2 (en) | System and method for server configuration control and management | |
| CN109101364B (en) | Double-center double-activity data processing system and method | |
| CN105933163B (en) | A real-time distributed debugging and tracking method and system | |
| CN109902073A (en) | Log processing method, device, computer equipment and computer readable storage medium | |
| CN106100902A (en) | High in the clouds index monitoring method and apparatus | |
| CN115460251B (en) | Cloud edge cooperation-based equipment data acquisition method, device, equipment and medium | |
| CN103326896B (en) | The system and method for the information data that a kind of user of collection produces on the internet | |
| CN112737800A (en) | Service node fault positioning method, call chain generation method and server | |
| US20250130735A1 (en) | Data Processing Method and Related Device | |
| CN115378787B (en) | Cloud edge cooperation-based device management method, device, equipment and medium | |
| CN110990351B (en) | Unstructured data acquisition method, device and system and computer equipment | |
| CN115378804B (en) | Log acquisition method, device, equipment and medium based on cloud edge cooperation | |
| CN113434481B (en) | Method, device, equipment and storage medium for realizing data service | |
| CN114553682B (en) | Real-time alarm method, system, computer equipment and storage medium | |
| CN107920131A (en) | A kind of metadata management method and device of HDFS storage systems | |
| CN117806902B (en) | A state monitoring management system, method, device and storage medium | |
| CN117793164A (en) | Internet of things communication method, device, equipment and storage medium | |
| CN116112367B (en) | Traffic topology generation method and system | |
| CN113778795B (en) | Cross-version Oracle monitoring system based on Python language |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |