GB2638040A

GB2638040A - Monitoring operation of an industrial machine

Info

Publication number: GB2638040A
Application number: GB2412975.1A
Authority: GB
Inventors: TRIPATHI Prateek
Original assignee: Airs Ml Ltd
Current assignee: Airs Ml Ltd
Priority date: 2024-09-04
Filing date: 2024-09-04
Publication date: 2025-08-13
Also published as: GB202412975D0

Abstract

A machine monitoring system 10 for an industrial machine has an edge computing system 101, and receives data from sensors 12. The sensor data is input to a trained machine learning model (23, fig 2) to obtain an inferred operational state of the industrial machine. The system 10 determines whether the obtained inferred operational state is different from a reference operational state stored at the machine monitoring system 10. If the inferred operational state is different from the reference operational state, then the system determines the inferred operational state as occurrence of an inferred event, and initiates communication with a central server 16 via a network 14 to communicate the inferred event from the machine monitoring system 10 to the central server 16.

Description

MONITORING OPERATION OF AN INDUSTRIAL MACHINE

TECHNICAL FIELD

The invention relates to monitoring operation of an industrial machine. In particular, the invention relates to edge computing systems and methods for monitoring and processing machine operation, with only changes in machine operation being communicated across a network.

BACKGROUND

Many industries rely on the continued operation of highly complex machines and workflows. This includes industrial sectors ranging from manufacturing (e.g. aviation, automotive, wind turbine manufacture, etc.) to energy production (e.g. wind power production, oil and gas production, etc.), industrial processes (e.g. biomedical processes, etc.), construction, and many more.

Even relatively short periods, e.g. minutes, of downtime or malfunctioning of such industrial machines can result in significant production losses, as well as possibly serious accidents and/or an increased carbon footprint. It is common to employ various sensors to monitor various aspects/parameters of the performance of an industrial machine in order to perform predictive maintenance, i.e. with a view to identifying as quickly as possible when a machine is, or is likely to, malfunction or operate with reduced efficiency.

As even short periods of downtime can result in significant production losses, then substantially real-time sensor monitoring is needed. In addition, the complexity and scale of industrial machines being monitored means that many sensors, often of different types, are utilised to continuously monitor various different aspects of the performance of an industrial machine, thereby generating massive amounts of data to be processed and/or stored.

The generated sensor data can be transmitted for processing or storage across a (wireless) network to a central location/server, such as a Cloud-based location. However, this can result in high levels of network traffic, which is costly, results in high energy consumption, and has latency and data privacy issues. Indeed, transmitting raw sensor in this way can result in terabytes of data travelling through a network in real-time. Attempts to reduce network traffic by only transmitting a current state of a machine, such as the predicted remaining life of a machine, determined locally still results in significant amounts of data transmission.

It is against this background to which the present invention is set.

SUMMARY OF THE INVENTION

The invention provides systems and methods for real-time predictive maintenance of industrial machines but with low data transmission rates. The invention makes use of one or more edge/local processing devices, equipped with dedicated trained machine learning or artificial intelligence models, to make inferences about an operational state of an industrial machine based on sensor data indicative of machine operation received by the edge device. The invention beneficially provides for identifying locally when the inference/prediction about the operational state of a machine changes, and only initiates communication across a (wireless) network when such a change is identified. Indeed, the invention further benefits from communicating only information related to the inference change/event, rather than the raw sensor data or other parameters associated with the change/event. The invention can furthermore identify when the inferred change results in abnormal or anomalous machine operation. The invention is advantageous not only in that sensor data processing is performed locally / at the edge, but also in that only a relatively small selection of processed information is transmitted across a network (e.g. to the Cloud) and the processed information that is transmitted is in a low data size format/representation. Data transmission across a network can thereby be significantly reduced, e.g. to the order of kilobytes of data (rather than terabytes), while still performing effective, real-time monitoring and maintenance of an industrial machine. Further benefits associated with the invention will become apparent from the following description.

According to an aspect of the invention there is provided a machine monitoring system for monitoring operation of an industrial machine. The machine monitoring system is implemented as an edge computing system. The machine monitoring system is configured to: (a) receive sensor data, from one or more sensors, indicative of a measured operational parameter of the industrial machine; (b) input the received sensor data into a trained machine learning model, and execute the trained machine learning model to obtain an inferred operational state of the industrial machine; and (c) determine whether the obtained inferred operational state is different from a reference operational state stored at the machine monitoring system. If the inferred operational state is different from the reference operational state, then the machine monitoring system is configured to: (d) determine the inferred operational state as occurrence of an inferred event; and (e) initiate communication with a central server via a network to communicate the inferred event from the machine monitoring system to the central server.

If the inferred operational state is the same as the reference operational state, then the machine monitoring system may be configured to interrupt communication from the machine monitoring system to the central server.

The machine monitoring system may be configured to update the stored reference operational state to be the obtained inferred operational state.

The stored reference operational state may be the inferred operational state obtained at a time step immediately prior to a current time step of the machine monitoring system.

The machine monitoring system may comprise an interrupt management subsystem configured to initiate or interrupt communication via the network from the machine monitoring system to the central server. To initiate communication with the central server, the interrupt management subsystem may be configured to, in response to determination of the inferred event occurrence, change from a deactivated state in which communication to the central server is interrupted to an activated state in which communication to the central server is initiated.

The inferred event communicated from the machine monitoring system to the central server may include the obtained inferred operational state.

The machine monitoring system may be configured to encrypt the obtained inferred operational state, wherein the inferred event communication is the encrypted inferred operational state.

The inferred operational state may be an inferred classification from a plurality of defined classifications. Each of the defined classifications may correspond to a respective operational state of the industrial machine.

Each inferred operational state may correspond to one of a normal operational state and an anomalous operational state of the industrial machine.

The machine monitoring system may be configured to generate the inferred event notification and initiate communication with the central server to communicate the inferred event only if the inferred operational state corresponds to an anomalous operational state of the industrial machine.

Steps (a)-(e) may be performed substantially continuously.

The machine monitoring system may comprise a first edge processing device comprising one or more processors configured to perform steps (a)-(e).

The machine monitoring system may comprise a second edge processing device comprising one or more second processors. The second edge processing device may be configured to: receive sensor data, from one or more second sensors, indicative of a second measured operational parameter of the industrial machine; input the received sensor data into a second trained machine learning model, and execute the second trained machine learning model to obtain a second inferred operational state of the industrial machine; and determine whether the obtained second inferred operational state is different from a second reference operational state stored at the machine monitoring system. If the second inferred operational state is different from the second reference operational state, then the second edge processing device may be configured to: determine the second inferred operational state as occurrence of an inferred event; and initiate communication with the central server via the network to communicate the second inferred event from the machine monitoring system to the central server.

The first edge processing device and/or the second edge processing device may be implemented on one or more of: a microcontroller; a field programmable gate array; and an application specific integrated circuit.

The machine monitoring system may be implemented as an edge computing system. The machine monitoring system may comprise: a plurality of edge processing devices each comprising one or more processors; and a sensor hub device each comprising one or more processors. Each of the edge processing devices may be configured to: receive sensor data, from one or more sensors, indicative of a respective measured operational parameter of the industrial machine; input the received sensor data into a trained machine learning model of the respective edge processing device, and execute the trained machine learning model to obtain a respective inferred operational state of the industrial machine; and transmit the respective inferred operational state to the sensor hub device. The sensor hub device may be configured to: receive the inferred operational state from each respective edge processing device; determine an overall inferred operational state of the industrial machine based on the inferred operational states received from the edge processing devices; and communicate the overall inferred operational state to a central server via a network.

The sensor hub device may be configured to: determine whether the overall inferred operational state is different from a reference overall operational state stored at the sensor hub device; and communicate the overall inferred operational state to the central server only if the overall inferred operational state is different from the reference overall operational state.

Each of the edge processing devices may be configured to: determine whether the respective inferred operational state is different from a reference operational state stored at the respective edge processing device; and transmit the respective inferred operational state to the sensor hub device only if the respective inferred operational state is different from the reference operational state.

To determine the overall inferred operational state the sensor hub device may be configured to input the received inferred operational states from the edge processing devices into a further trained machine learning model, and may be configured to execute the further trained machine learning model to obtain the overall inferred operational state.

The overall inferred operational state may correspond to one of a normal operational state and an anomalous operational state of the industrial machine.

The sensor hub device may be configured to communicate the overall inferred operational state to the central server only if the overall inferred operational state corresponds to an anomalous operational state of the industrial machine.

Upon determination that the overall inferred operational state is an anomalous operational state, the sensor hub device may be configured to: record a first time as a time of the occurrence of the anomalous operational state; record a second time as a time when the overall inferred operational state changes to a different inferred operational state obtained from the further machine learning model; determine a time difference between the first time and the second time; if the time difference is greater than a defined threshold time difference, then generate an alarm notification and transmit the alarm notification, indicating anomalous operation, off-board the sensor hub device via the network.

If the time difference is less than the defined threshold time difference, then the sensor hub device may be configured to suppress generation of the alarm notification.

If the time difference is less than the defined threshold time difference, then the sensor hub device may be configured to: retrieve the inferred operational state data received from the edge processing devices that resulted in the inferred anomalous operational state being obtained from the further trained machine learning model; and update the further machine learning model, using the retrieved inferred operational state data, so that said inferred operational state data maps to an inferred normal operational state in the updated further machine learning model.

According to another aspect of the invention there is provided a machine monitoring system for monitoring operation of an industrial machine. The machine monitoring system is implemented as an edge computing system. The machine monitoring system comprises: a plurality of edge processing devices each comprising one or more processors; and a sensor hub device each comprising one or more processors. Each of the edge processing devices is configured to: receive sensor data, from one or more sensors, indicative of a respective measured operational parameter of the industrial machine; encode the received sensor data into a sensor data representation that has a lower dimension than the received sensor data; and transmit the encoded sensor data representation to the sensor hub device. Each of the plurality of encoded sensor data representations is incorporated into a defined latent space, wherein the sensor hub device is configured to: receive the respective encoded sensor data representation from each of the plurality of edge processing devices; decode the latent space including the plurality of encoded sensor data representations to obtain decoded sensor data; input the decoded sensor data into a trained machine learning model, and execute the trained machine learning model to obtain an inferred operational state of the industrial machine; determine whether the obtained inferred operational state is different from a reference operational state stored at the machine monitoring system; and if the inferred operational state is different from the reference operational state, then: determine the inferred operational state as occurrence of an inferred event; and initiate communication with a central server via a network to communicate the inferred event from the machine monitoring system to the central server.

According to another aspect of the invention there is provided a method for monitoring operation of an industrial machine. The method being performed by an edge computing system. The method comprises: (a) receiving sensor data, from one or more sensors, indicative of a measured operational parameter of the industrial machine; (b) inputting the received sensor data into a trained machine learning model, and executing the trained machine learning model to obtain an inferred operational state of the industrial machine; and (c) determining whether the obtained inferred operational state is different from a reference operational state stored at the machine monitoring system. If the inferred operational state is different from the reference operational state, then the method comprises: (d) determining the inferred operational state as occurrence of an inferred event; and (e) initiating communication with a central server via a network to communicate the inferred event from the machine monitoring system to the central server.

According to an aspect of the invention there is provided a non-transitory, computer readable storage medium storing instruction thereon that, when executed by one or more computer processors, cause the one or more computer processors to perform the method defined above.

BRIEF DESCRIPTION OF THE DRAWINGS

Examples of the invention will now be described with reference to the accompanying drawings, in which: Figure 1 schematically illustrates a system for monitoring an industrial machine, in accordance with an example of the invention; Figure 2 schematically illustrates an edge processing system/device of the system of Figure 1, in accordance with an example of the invention; Figure 3 schematically illustrates an example architecture of an interrupt management system of the edge processing system of Figure 2; Figure 4 schematically illustrates an example workflow implementing the edge processing system of Figure 2; Figure 5 schematically illustrates a plurality of edge processing devices and a sensor hub device of the system of Figure 1, in accordance with an example of the invention; Figure 6 schematically illustrates a plurality of edge processing devices and a sensor hub device of the system of Figure 1, in accordance with an example of the invention; Figure 7 schematically illustrates an example workflow implementing the system of Figure 1; Figure 8 schematically illustrates an example workflow implementing the edge processing system of Figure 2; and Figure 9 shows the steps of a method performed by the system of Figure 1, in accordance with an example of the invention.

DETAILED DESCRIPTION

Figure 1 schematically illustrates a monitoring system 10 for monitoring operation of an industrial machine (not shown) in accordance with an example of the invention. The industrial machine may be any suitable machine whose operation may be monitored using sensors configured to measure operational parameters of the machine. Illustrative examples of such industrial machines include engines for automotive (e.g. motor racing) or aviation use, manufacturing equipment (e.g. for manufacturing wind turbine components, vehicle components, etc.) in a manufacturing plant, biomedical engineering equipment, and satellites in orbit; however, it will be understood that the monitoring system described herein may be used with a wide variety of industrial machines/equipment.

The industrial machine being monitored may have a number of sensors 12 associated therewith that are configured to measure various operational parameters that are indicative of performance/health of the industrial machine. Figure 1 illustrates non-exhaustive examples sensors 12 that may be used. In particular, Figure 1 shows sensors in the form of an accelerometer 121, a camera (vision sensor) 122, a microphone (acoustic sensor) 123, an analogue-to-digital (ADC) sensor/device 124, an infrared (IR) sensor 125, a microelectromechanical system (MEMS) sensor 126, a current transformer (CT) sensor 127, an electrochemical sensor 128, a light dependent resistor (LDR) sensor 129, a vibrometer 130 and an acetometer 131. Other types of sensor may include temperature, pressure and humidity sensors, for instance.

The sensors 12 may typically be regarded as being separate from the monitoring system 10; however, in some examples the sensors may be regarded as being part of the monitoring system.

The monitoring system 10 is implemented as an edge computing system. That is, the monitoring system 10 includes one or more local devices or local servers that perform data processing at remote locations at the edge of a network 14. In this example, devices of the monitoring system 10 are typically located in a vicinity of the sensors 12 (and industrial machine), e.g. in a same room or building as the industrial machine.

The monitoring system 10 can transmit and receive data across the network 14, i.e. to/from one or more central servers/gateways 16 and beyond, e.g. one or more user devices or other processing devices located elsewhere across the network 14. The (wireless) network 14 may be a wide area network, e.g. the Internet.

Figure 1 illustrates that the monitoring system 10 includes a plurality of edge processing devices or systems (EPSs) 101. Each EPS 101 is configured to receive measured sensor data from one or more of the sensors 12. Typically, each EPS 101 receives sensor data from one or more sensors of a given type. For instance, the EPS 101a receives acoustic data from one or more acoustic sensors that are measuring an acoustic noise parameter associated with operation of the industrial machine. Each acoustic sensor may be located at/near a different part/component of the industrial machine and may measure acoustic noise associated with / emanating from the respective part/component.

Figure 1 illustrates that some of the EPSs 101 communicate directly over the network 14, e.g. with the central server 16. Figure 1 also shows that the monitoring system 10 includes a sensor hub device 102. The sensor hub device 102 is configured to receive data from (and transmit data to) one or more of the EPSs 101. The sensor hub device 102 can communicate over the network 14. The EPSs 101 feeding into the sensor hub device 102 communicate across the network 14 via the sensor hub device 102.

In the example of Figure 1, there is shown a plurality of EPSs that communicate directly across the network, e.g. with a central server. It will be understood that in different examples, any number of such EPSs that communicate directly to a central location may be present, such as one EPS or zero EPSs (if the system only includes EPSs that feed into a sensor hub).

Also in the example of Figure 1, there is shown a single sensor hub device that receives sensor data from a plurality of EPSs. It will be understood that in different examples, any number of EPSs may feed into the sensor hub device, e.g. one or more EPSs. Also, it will be understood that the monitoring system may include any suitable number of sensor hub devices that communicate over the network to a central location, each having any suitable number of EPSs feeding into them. Furthermore, it will be understood that in some examples the monitoring system may not include a sensor hub device (with each of the one or more EPSs communicating directly across the network to a central location).

The implementation and operation of the EPSs 101 and sensor hub device 102 is described in more detail below.

Figure 2 schematically illustrates functional modules included in one of the edge processing devices/systems (EPS) 101. In general, each EPS 101 stores and executes a (dedicated) model -specifically, a machine learning (ML) or artificial intelligence (Al) model -that is trained to convert sensor data into meaningful inferences about the operation of the industrial machine. Each EPS 101 then communicates with the rest of the network 14 via low data/memory representations of events occurring at/in the industrial machine as inferred by (the Al model of) the EPS 101.

Each EPS 101 typically has a memory and one or more computer processors for executing instructions stored in the memory. For instance, details of the Al model, such as architecture, model weights, and instructions for executing the model may be stored in the memory, and may be executed by the processor(s) to obtain inferences based on the sensor input. Each EPS 101 may be implemented as one, or a combination, of a microcontroller, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and Raspberry Pi (RTM). For instance, the EPS may be fabricated on a silicon integrated circuit through an SOC (system on a chip) design in the form of an ASIC.

The sensor hub may be implemented in a similar manner.

Figure 2 shows that the EPS 101 may have one or more functional modules for performing different tasks. As mentioned above, the EPS 101 is configured to receive sensor data from one or more sensors 12 via a suitable input (interface) 21. This is typically in the form of raw data output from the sensor, e.g. voltage levels, digital values, etc. The sensor data may typically be in the form of time series data. The sensor data may be received via a suitable connection, e.g. Ethernet. The received sensor data may in some examples undergo one or more data preprocessing steps in a data preprocessing module 22 of the EPS 101. This could include one or more (standard) techniques such as data clustering, data transformations, denoising, etc., as will be understood by the skilled person. Various transformation techniques may be used to improve Al model performance, such as conversion to spectrograms, scalograms, etc., meaning that the input layer/interface may differ based on the preprocessing steps.

The EPS 101 includes an inference module (or Al model module) 23 that executes the AI/ML model to obtain inferences. The Al model may in some examples be a neural network (NN) model comprising an input layer, one or more intermediate layers, and an output layer. Each intermediate layer typically includes a number of nodes and associated trained weights.

The Al model receives the (optionally pre-processed) sensor data as input at the input layer. The input layer may be a one-dimensional vector of any suitable dimension, with values representing different characteristics of the input sensor signal/data. The Al model processes the sensor data through the intermediate layers, and outputs an inference based on the sensor data at the output layer.

The output layer may include a plurality of output nodes each corresponding to a respective defined classification indicative of an operational state of the industrial machine. In an illustrative example, the output layer may comprise one or more output nodes each representing a respective normal operational state of the industrial machine and, a further output node representing an abnormal/anomalous operational state of the industrial machine. The output of the Al model may in some examples be in the form of a probability associated with each of the output nodes, with a classification being made based on which output node has the highest probability or probability above a defined threshold probability. An inference may for instance be represented by a value or classification.

Although an NN model is described here, in other examples different types of Al model may be implemented in the inference module 23, e.g. a support vector machine, a random forest, etc. In examples in which an NN model is used, the NN model may for instance be a deep learning model, such as an artificial NN, a spiking NN, a feed-forward NN, an autoencoder, a convolutional NN, a recurrent NN, etc. The Al model may be supervised or unsupervised.

Each EPS 101 is typically configured to receive sensor data, and obtain inferences based on the received sensor data, substantially continuously when the industrial machine is operating. That is, inferences are determined and updated in real time. For instance, the Al model may be executed at each time step of a processor implementing the Al model, or at defined intervals.

Figure 2 shows that the EPS 101 includes an inference event determination/communication module 24. As mentioned above, an inference obtained from / output by the Al model may be indicative of an operational state/mode of the industrial machine. An industrial machine may have one or more normal operational states and one or more abnormal/anomalous operational modes, for instance. The inference event module 24 receives the inference output by the inference (Al model) module 23. In one example, an inference event may be regarded as a change in inference obtained from the inference module 23, such as from one execution (e.g. time step) of the Al model to the next execution (e.g. next time step). This could be in the form of a change in the classification output obtained from the Al model.

To perform the inference event determination, a reference inference may be stored locally at the EPS 101, e.g. in the EPS memory and accessible by the inference event module 24. An initial inference corresponding to the respective sensor(s) of the EPS 101 may be provided (once) to the EPS 101, e.g. via the network 14 from a central location/repository, and set to be the reference inference in the memory/storage of the EPS 101. The inference module 23 determines a current inference by executing the Al model based on the (most recently) received sensor data.

The inference event module 24 then determines whether the obtained current inference (inferred operational state) is different from the reference inference (reference operational state). If the current inference is the same as the reference inference, then the inference event module 24 determines that no inference event has occurred. If the current inference is different from the reference inference, then the inference event module 24 determines that an inference event has occurred. In this case, the module 24 updates the reference inference to be the current inference, i.e. the current inference is the new reference. A change in inference may for instance be represented by an integer number, thereby providing an ultra-low power inference representation.

Figure 2 shows that the EPS 101 includes an inference encryption module 25. This is configured to encrypt inferences obtained from the inference module 23 or inference events determined by the inference event module 24. The inference encryption module 25 may be configured to encrypt only those inferences that represent a change in inference from a previous time step / iteration, i.e. only those inferences that result in an inference event being determined to have occurred in the inference event module 24. The inference encryption module 25 may receive an indication from the inference event module 24 when an inference event has occurred and perform an encryption of the current inference obtain from the Al model. The inference encryption module 25 may alternatively encrypt a representation of the inference event from the inference event module 24. The encrypted inference or inference event may be a low dimensional data representation/encoding of the inference or inference event. The encrypted inference or inference event therefore requires a low data file/packet size, and provides data security, for sending across the network 14.

Figure 2 shows that the EPS 101 includes an interrupt management system (IMS) module 26. The IMS module 26 is for communicating determined inference events across the network 14 to a central server/gateway. The IMS is configured to begin communication only when there is a change in inference, i.e. an inference event.

Figure 3 schematically illustrates an example architecture of the IMS module 26. The IMS is configured to interrupt processing performed by the processor(s) of the EPS 101 to communicate the determined inference event across the network 14. The IMS is configured to deactivate when no inference event is determined and activate only when an inference event occurs, thereby providing a low power approach for communication across the network 14.

Figure 4 schematically illustrates an example workflow implementing the EPS 101. In this example, the industrial machine is a vehicle engine, e.g. a car engine. In particular, the vehicle engine is being tested to detect acoustic vibrations occurring in the engine. A MEMS microphone sensor is used to detect the acoustic vibrations and the measured sensor data is input into the EPS 101. The Al model in the inference module 23 is trained to classify and provide and inference for the acoustic vibrations produced when the engine is operating in different gears, as well as an abnormal inference (that does not correspond to the acoustic vibrations of any of the gears).

When the engine is in a first gear, the EPS 101 may send the first/initial inference encoded as A' for 'Gear One' to the central server/gateway and will then switch off communication. As the engine shifts to 'Gear Two', which may be encoded as 'B', the change in inference will generate an event in the inference event module 24. This generated event will be communicated to the central server by the IMS module 26. The inferences 'A' and 'B' here can also be as a cryptographic value to help prevent spoofing of the engine performance by third parties.

As the EPS continuously monitors the operation/health of the engine/machine, terabytes of data may be processed at the edge by Al models, whereas only kilobytes of data are transmitted across the wider network. In another example, an EPS can be deployed to monitor the health of a satellite in orbit. By taking the above-described approach, only kilobytes of data need to be transmitted (e.g. back to Earth) in order to maintain a digital twin of the satellite, for instance.

The Al models deployed at the respective EPSs 101 may be trained at a central location/processing device. Prior to deployment of the monitoring system 10, the sensors 12 may be configured to collect measurement data for each of the operational parameters being monitored for the industrial machine under consideration. The (initially) collected sensor data may be transmitted across the network 14 to the central server 16. This sampling is performed only once in order to perform the Al model training. The Al models may be trained and tested centrally using the collected sensor data, and the trained Al models may then be transmitted across the network 14 to the respective EPSs 101 for deployment. The Al models may be trained using sensor data collected from the relevant sensors of the industrial machine under consideration, and may be trained additionally using sensor data obtained from monitoring other (equivalent) industrial machines. Specifically, each trained Al model may be deployed at a plurality of industrial machines. Then, as described above, during execution of the monitoring system 10 to monitor the health/performance of the industrial machine, the sensors 12 perform (substantially) continuous measurements of the operational parameters, but this sensor data is only shared and processed locally by the EPS 101. In particular, only changes to the inferred machine operation are communicated via the network 14, e.g. to an alert system.

Referring back to Figure 1, as mentioned above the sensor hub device 102 receives data from one or more EPSs 101. In some examples, the EPSs 101 feeding into the sensor hub 102 operate as described above with reference to Figure 2. In other examples, the sensor hub 102 performs an inference based on the sensor data from all of the EPSs 101 feeding into the sensor hub 102. In this way, instead of inferences being performed in relation to individual operational parameters (e.g. acoustic noise) by a single type of sensor, an inference is performed based on sensor data from different types of sensor. This also means that a single inference -covering a plurality of EPSs 101 -may be communicated across the network 14 from the sensor hub 102.

Figure 5 schematically illustrates how the sensor data from the sensors 12 may be processed by the EPSs 101 and the sensor hub 102. As described above, each EPS 101 receives sensor data from one or more sensors 12 of a given type at input 21. In the illustrated example, three different types of sensor 12, e.g. temperature, pressure, microphone, are attached to their respective EPS 101.

In the described example, each EPS 101 is provided with an encoder 51 for encoding the received sensor input. The encoder 51 may be in addition to, or as an alternative from, the other modules of the EPS 101 in the example illustrated in Figure 2. The encoder 51 encodes the respective sensor's value into a latent space 52, i.e. a low dimensional representation, that maintained by/in the (local) sensor hub 102. Dimensionality reduction can improve data analysis by reducing data complexity and noise. Dimensionality reduction techniques such as Principal Component Analysis (PCA), t-SNE, autoencoders, etc., can be applied to both spatial and temporal (time series) data to automatically choose the most salient features/properties of the sensor data. The data may be one-dimensional or multi-dimensional.

Encoding the sensor data at the EPS 101 prior to sending to the sensor hub 102 can reduce the amount of data that needs to be transmitted, particularly in single asset monitoring use cases, e.g. engine monitoring. This means ensuring that the location of failure is quickly shared for an active asset that is not transmitting data continuously. It also means that multiple inferences are not shared for a single asset in cases where just sharing the inference of the decoder can help maintain the health of the machine.

The sensor hub 102 includes a decoder 53 that takes the latent space (low dimensional representation) of sensor values and determines an inference based on this. In this way, the decoder 53 is a trained Al model that takes the low dimensional values stored in the latent space 52 and determines an inference as an output, i.e. one of a plurality of defined classification outputs each corresponding to a respective operational state of the machine. The sensor hub 102 has the functionality of the EPSs 101 described above, namely, that the sensor hub 102 determines whether the determined (current) inference is different from a stored reference inference and, if so, generates an inference event that may be encrypted and communicated to the server 16 through an interrupt management system (IMS) of the sensor hub 102.

As described, the sensor hub 102 determines when an inference event occurs. An inference event could be a machine changing from one normal operational state of the machine to another normal operational state. On the other hand, an inference event could be a machine from a normal operational state of the machine to an abnormal/anomalous operational state (or vice versa). When an abnormal operational state is inferred, a communication via the IMS (in either an EPS or sensor hub) may be triggered in the form of an alarm notification. The alarm notification may for instance be sent to an alert system through the network, such as to a user device, indicating that the machine is malfunctioning or operating in an abnormal manner. This may be in contrast to when an inference event represents a change between two normal operational states/modes, in which an update may be sent to the central server 16, but no alarm notification is generated.

Figure 5 shows that the sensor hub 102 also includes an asset status inference module 54. This is configured to determine whether an inference event indicating that the machine is in an abnormal operational state is correct or not. In particular, the asset status inference module 54 may be configured to note a time at which abnormal operation of the machine is first determined -indicated by a suitable inference event -and then record a time duration that the machine operates in said abnormal state (based on the determined inferences). If the time duration is less than a threshold time duration then the asset status inference module 54 may be configured to determine that the inference of abnormal operation was a false alarm. In this case, the alarm notification may be suppressed. This allows for the removal of noise from the data, which can cause a relatively short 'spike' in the data before it returns to normalcy. In this case, the change in inference is recorded; however, an alarm -indicating operation outside of a normal range of operation -is not needed. On the other hand, if the time duration is greater than the threshold time duration then the inference may be confirmed as anomalous, in particular as the machine operating outside of a normal range of operation, and so an alarm is generated.

Generation of the alarms may be in the sensor hub 102, as shown in Figure 5, or may be at the server side, based on the inference data received from the sensor hub 102 or EPS 101. Generation of the inference events, on the other hand, may be in the EPSs 101 or the sensor hub 102, as this determines what should be communicated.

The example illustrated in Figure 5 is for a specific approach in which a NN is split into encoders and decoders across the network for use cases that need multiple sensors, and correlation across those sensors, to determine/understand asset status inference.

Figure 6 schematically illustrates an example in which the sensor hub 102 performs asset status inference, and where the determination as to whether an alarm should be generated is used to retrain the Al models. The sensor hub 102 receives inferences (rather than raw sensor data) from a plurality of the EPSs 101 into a module 61 that sorts the EPS outputs (inferences) based on their address. These EPS outputs are then used as input to a further trained Al model in an alarm management system module of the sensor hub 102. This further Al model, e.g. an NN model, is trained to determine whether an alarm should be generated based on the received EPS outputs/inferences. When the further Al model determines that an alert/alarm should be generated, then it is checked whether this is a false alarm in a similar manner to as described above, i.e. an inference of anomalous operation that lasts only a short period of time may be caused by noise and so may not require an alarm notification to be generated (false alarm), whereas abnormal operation over an extended period may need to be notified to a user/operator by means of an alarm.

In the case that a false alarm is determined, an automatic update of the weights of the further Al model that output the false alarm inference may be triggered. This may be performed in an unsupervised weight update module 621 of the alarm management system 62, for instance. In particular, the model weights may be updated in a manner that means that the future occurrences of sensor data that led to the false alarm inference would not be determined to be anomalous according to the updated Al model, i.e. it would not be determined that alarm needs to be generated. These described features will help reduce the traffic required to maintain the digital twin of the machine/asset whilst also providing data security benefits and reducing false alarm instances.

Figure 7 schematically illustrates how the systems and methods described herein can be used to maintain a digital twin of an industrial machine of interest. Figure 7 shows a machine in the form of a jet engine 71. The engine 71 has a plurality of microphones 711 associated therewith, located at different positions relative to the engine 71 and configured to measure acoustic noise generated by the engine 71 during operation. The sensor measurements from each microphone are fed into a respective EPS 101. This can then be encoded at the respective EPS 101 to encode the sensor data into a latent space 72 (in a similar manner to as described above with reference to Figure 5). This latent space 72 therefore generates a spatial map for a digital twin 73 of the engine 71. In particular, this allows for spatial sound mapping based on an expected engine behaviour dataset.

This can then be used to identify anomalies in the spatial map. The digital twin 73 will allow for highlighting of specific locations 731 based on the location of anomalies from the respective EPSs 101.

Figure 8 schematically illustrates an example use case for the machine monitoring system 10. In particular, operation of a satellite 81 in orbit may be monitored by one or more of the EPSs 101 located at the satellite 81. As described above, each EPS 101 monitors sensor signals measuring a respective operational parameter of the satellite 81. An Al/ML model of the EPS 101 is executed substantially continuously using the monitored sensor data to continuously monitor the operational health/state of the satellite 81. While this may involve processing (an order of) terabytes of data at the edge, i.e. at the satellite 81, using the AI/ML model(s), only changes in the operational state of the satellite 81 are communicated across the network 14 back to a location on Earth 82. This means that only (an order of) kilobytes of data need to be transmitted across the network 14, e.g. to maintain a digital twin of the satellite 81. In some examples, the change in operational state is communicated across the network only when the Al/ML model(s) indicates that an anomaly has been detected at the satellite 81.

Figure 9 summarises the steps of a method 90 for monitoring operation of an industrial machine, performed by the system 10, in accordance with examples of the invention. At step 901, the method 90 involves receiving sensor data, from one or more sensors 121131, indicative of a measured operational parameter of the industrial machine. At step 902, the method 90 involves inputting the received sensor data into a trained machine learning model, and executing the trained machine learning model to obtain an inferred operational state of the industrial machine. At step 903, the method 90 involves determining whether the obtained inferred operational state is different from a reference operational state stored at the machine monitoring system 10. If the inferred operational state is different from the reference operational state, then at step 904, the method 90 involves determining the inferred operational state as occurrence of an inferred event. At step 905, the method 90 involves initiating communication with the central server 16 via the network 14 to communicate the inferred event from the machine monitoring system 10 to the central server 16.

Many modifications may be made to the described examples without departing from the scope of the appended claims.

Claims

CLAIMS1. A machine monitoring system for monitoring operation of an industrial machine, the machine monitoring system being implemented as an edge computing system, the machine monitoring system being configured to: (a) receive sensor data, from one or more sensors, indicative of a measured operational parameter of the industrial machine; (b) input the received sensor data into a trained machine learning model, and execute the trained machine learning model to obtain an inferred operational state of the industrial machine; (c) determine whether the obtained inferred operational state is different from a reference operational state stored at the machine monitoring system, and if the inferred operational state is different from the reference operational state, then: (d) determine the inferred operational state as occurrence of an inferred event; and (e) initiate communication with a central server via a network to communicate the inferred event from the machine monitoring system to the central server.
2. A machine monitoring system according to Claim 1, wherein if the inferred operational state is the same as the reference operational state, then the machine monitoring system is configured to interrupt communication from the machine monitoring system to the central server.
3. A machine monitoring system according to Claim 1 or Claim 2, configured to update the stored reference operational state to be the obtained inferred operational state.
4. A machine monitoring system according to any previous claim, wherein the stored reference operational state is the inferred operational state obtained at a time step immediately prior to a current time step of the machine monitoring system.
5. A machine monitoring system according to any previous claim, wherein the machine monitoring system comprises an interrupt management subsystem configured to initiate or interrupt communication via the network from the machine monitoring system to the central server, wherein to initiate communication with the central server, the interrupt management subsystem is configured to, in response to determination of the inferred event occurrence, change from a deactivated state in which communication to the central server is interrupted to an activated state in which communication to the central server is initiated.
6. A machine monitoring system according to any previous claim, wherein the inferred event communicated from the machine monitoring system to the central server includes the obtained inferred operational state.
7. A machine monitoring system according to Claim 6, configured to encrypt the obtained inferred operational state, wherein the inferred event communication is the encrypted inferred operational state.
8. A machine monitoring system according to any previous claim, wherein the inferred operational state is an inferred classification from a plurality of defined classifications, wherein each of the defined classifications corresponds to a respective operational state of the industrial machine.
9. A machine monitoring system according to any previous claim, wherein each inferred operational state corresponds to one of a normal operational state and an anomalous operational state of the industrial machine.
10. A machine monitoring system according to Claim 9, wherein the machine monitoring system is configured to generate the inferred event notification and initiate communication with the central server to communicate the inferred event only if the inferred operational state corresponds to an anomalous operational state of the industrial machine.
11. A machine monitoring system according to any previous claim, wherein steps (a)-(e) are performed substantially continuously.
12. A machine monitoring system according to any previous claim, comprising a first edge processing device comprising one or more processors configured to perform steps (a)-(e).
13. A machine monitoring system according to Claim 12, comprising a second edge processing device comprising one or more second processors configured to: receive sensor data, from one or more second sensors, indicative of a second measured operational parameter of the industrial machine; input the received sensor data into a second trained machine learning model, and execute the second trained machine learning model to obtain a second inferred operational state of the industrial machine; determine whether the obtained second inferred operational state is different from a second reference operational state stored at the machine monitoring system; and if the second inferred operational state is different from the second reference operational state, then: determine the second inferred operational state as occurrence of an inferred event; and initiate communication with the central server via the network to communicate the second inferred event from the machine monitoring system to the central server.
14. A machine monitoring system according to Claim 12 or Claim 13, wherein the first edge processing device and/or the second edge processing device is implemented on one or more of: a microcontroller;a field programmable gate array; andan application specific integrated circuit.
15. A machine monitoring system for monitoring operation of an industrial machine, the machine monitoring system being implemented as an edge computing system, the machine monitoring system comprising: a plurality of edge processing devices each comprising one or more processors; and a sensor hub device each comprising one or more processors, wherein each of the edge processing devices is configured to: receive sensor data, from one or more sensors, indicative of a respective measured operational parameter of the industrial machine; input the received sensor data into a trained machine learning model of the respective edge processing device, and execute the trained machine learning model to obtain a respective inferred operational state of the industrial machine; transmit the respective inferred operational state to the sensor hub device, wherein the sensor hub device is configured to: receive the inferred operational state from each respective edge processing device; determine an overall inferred operational state of the industrial machine based on the inferred operational states received from the edge processing devices; and communicate the overall inferred operational state to a central server via a network.
16. A machine monitoring system according to Claim 15, wherein the sensor hub device is configured to: determine whether the overall inferred operational state is different from a reference overall operational state stored at the sensor hub device; and communicate the overall inferred operational state to the central server only if the overall inferred operational state is different from the reference overall operational state.
17. A machine monitoring system according to Claim 15 or Claim 16, wherein each of the edge processing devices is configured to: determine whether the respective inferred operational state is different from a reference operational state stored at the respective edge processing device; and transmit the respective inferred operational state to the sensor hub device only if the respective inferred operational state is different from the reference operational state.
18. A machine monitoring system according to any of Claims 15 to 17, wherein to determine the overall inferred operational state the sensor hub device is configured to input the received inferred operational states from the edge processing devices into a further trained machine learning model, and execute the further trained machine learning model to obtain the overall inferred operational state.
19. A machine monitoring system according to Claim 18, wherein the overall inferred operational state corresponds to one of a normal operational state and an anomalous operational state of the industrial machine.
20. A machine monitoring system according to Claim 19, wherein the sensor hub device is configured to communicate the overall inferred operational state to the central server only if the overall inferred operational state corresponds to an anomalous operational state of the industrial machine.
21. A machine monitoring system according to Claim 19 or Claim 20, wherein upon determination that the overall inferred operational state is an anomalous operational state, the sensor hub device is configured to: record a first time as a time of the occurrence of the anomalous operational state; record a second time as a time when the overall inferred operational state changes to a different inferred operational state obtained from the further machine learning model; determine a time difference between the first time and the second time; if the time difference is greater than a defined threshold time difference, then generate an alarm notification and transmit the alarm notification, indicating anomalous operation, off-board the sensor hub device via the network.
22. A machine monitoring system according to Claim 21, wherein if the time difference is less than the defined threshold time difference, then the sensor hub device is configured to suppress generation of the alarm notification.
23. A machine monitoring system according to Claim 21 or Claim 22, wherein if the time difference is less than the defined threshold time difference, then the sensor hub device is configured to: retrieve the inferred operational state data received from the edge processing devices that resulted in the inferred anomalous operational state being obtained from the further trained machine learning model; and update the further machine learning model, using the retrieved inferred operational state data, so that said inferred operational state data maps to an inferred normal operational state in the updated further machine learning model.
24. A machine monitoring system for monitoring operation of an industrial machine, the machine monitoring system being implemented as an edge computing system, the machine monitoring system comprising: a plurality of edge processing devices each comprising one or more processors; and a sensor hub device each comprising one or more processors, wherein each of the edge processing devices is configured to: receive sensor data, from one or more sensors, indicative of a respective measured operational parameter of the industrial machine; encode the received sensor data into a sensor data representation that has a lower dimension than the received sensor data; and transmit the encoded sensor data representation to the sensor hub device, wherein each of the plurality of encoded sensor data representations is incorporated into a defined latent space, wherein the sensor hub device is configured to: receive the respective encoded sensor data representation from each of the plurality of edge processing devices; decode the latent space including the plurality of encoded sensor data representations to obtain decoded sensor data; input the decoded sensor data into a trained machine learning model, and execute the trained machine learning model to obtain an inferred operational state of the industrial machine; determine whether the obtained inferred operational state is different from a reference operational state stored at the machine monitoring system; and if the inferred operational state is different from the reference operational state, then: determine the inferred operational state as occurrence of an inferred event; and initiate communication with a central server via a network to communicate the inferred event from the machine monitoring system to the central server.
25. A method for monitoring operation of an industrial machine, the method being performed by an edge computing system, the method comprising: (a) receiving sensor data, from one or more sensors, indicative of a measured operational parameter of the industrial machine; (b) inputting the received sensor data into a trained machine learning model, and executing the trained machine learning model to obtain an inferred operational state of the industrial machine; (c) determining whether the obtained inferred operational state is different from a reference operational state stored at the machine monitoring system; and if the inferred operational state is different from the reference operational state, then: (d) determining the inferred operational state as occurrence of an inferred event; and (e) initiating communication with a central server via a network to communicate the inferred event from the machine monitoring system to the central server.