HK40075377B - Identifying device state changes using power data and network data - Google Patents

Description

Use power data and network data to identify changes in device status.

This application is a divisional application of patent application No. 2018800266529, filed on October 22, 2019, entitled "Using power data and network data to identify changes in device status".

Cross-references to related applications

This application claims priority to and is a continuation of U.S. Patent Application No. 15/439,788 (U.S. Patent No. 9,699,529) (SAGE-0003-U01), filed February 22, 2017; U.S. Patent Application No. 15/439,796 (U.S. Patent No. 9,800,958) (SAGE-0003-U02), filed February 22, 2017; and U.S. Patent Application No. 15/599,466 (SAGE-0003-U01-C01), filed May 19, 2017.

Background Technology

Reducing electricity or power usage offers, among other things, the benefits of saving money by lowering payments to electricity companies and protecting the environment by reducing the amount of resources required to generate electricity. Therefore, electricity users such as consumers, businesses, and other entities may expect to reduce their electricity usage to achieve these benefits. Users may be able to reduce their electricity usage more effectively if they have information in their homes and buildings about which appliances (e.g., refrigerators, ovens, dishwashers, stoves, and light bulbs) use the most electricity and what actions can be taken to reduce electricity consumption.

Power monitors for individual devices can be used to measure the power usage of a single device. For example, a device can be plugged into a power monitor, and the monitor can then be plugged into a wall outlet. These monitors can provide information about the power usage of a single device to which they are attached, but using these monitors to monitor all or even many devices in a house or building may be impractical, as it would require a large amount of equipment, which could be expensive and require significant manpower for installation.

Instead of power monitors for individual devices, power monitors can be mounted on a distribution board to obtain information about the power used simultaneously by many devices. Power monitors on distribution boards are more convenient because a single monitor can provide aggregated usage information for many devices. However, it is more difficult to extract more specific information about the power used by a single device because monitors typically measure signals reflecting the collective operation of many devices, which may overlap in complex ways. The process of obtaining information about the power usage of a single device from the electrical signals corresponding to the use by many devices can be called deaggregation.

In order to provide the greatest benefit to end users, there is a need for more accurate de-aggregation technology, enabling end users to receive accurate information about the electrical usage of individual devices.

Attached Figure Description

The following detailed embodiments of the present invention and some of its embodiments can be understood by referring to the following accompanying drawings:

Figure 1 is an example of a system for using power monitoring signals to identify devices and changes in device status.

Figure 2 shows an example of a power monitoring signal.

Figure 3 is an example of a system for using network data to identify devices and changes in device status.

Figures 4A, 4B, and 4C illustrate techniques for using network data to identify devices and changes in device status.

Figure 5 is an example of a system for identifying devices and changes in device status.

Figure 6 is an example list of devices.

Figure 7 is an example of a system for using power monitoring signals and network data to identify devices and changes in device status.

Figures 8A and 8B are example timelines of power monitoring signals and network data.

Figure 9 is a flowchart illustrating an example implementation of using power monitoring signals and network data to identify devices.

Figure 10 is a flowchart illustrating an example implementation of using power monitoring signals and network data to identify changes in device status.

Figure 11 is a flowchart illustrating an example implementation of training a power model using network data.

Figure 12 is an example of a device used to identify devices and changes in device status using power monitoring signals or network data.

Detailed Implementation

This document describes techniques for identifying devices and determining information about changes in the state of devices within a building. One data source for determining information about devices within a building is the electrical wiring that supplies power to the devices. Electrical sensors can be placed on the electrical wiring (or wires) that supply power to the building, and de-aggregation techniques can be used to determine information about individual devices within the building. For example, any techniques described in U.S. Patent No. 9,443,195 (which are hereby incorporated herein by reference in their entirety for all purposes) can be used to determine information about devices using electrical measurements.

Another data source for identifying devices or determining changes in the status of devices within a building is the building's computer network. A building may have a network (such as a local area network), and devices may connect to the network via cables (e.g., Ethernet cables) or wirelessly (e.g., Wi-Fi). A building may have multiple networks, such as a local area network coordinated by a wireless router, a mesh network created by other cooperating devices (e.g., Sonos speakers), and personal area networks (e.g., Bluetooth connections between devices). Information about devices on these networks can be obtained, for example, by receiving service broadcasts from the device after it is turned on or by polling the device for information.

Some devices may be powered by electrical wiring in a building but may not have network connectivity (e.g., a conventional refrigerator). Some devices may be powered by electrical wiring in a building and have network connectivity (e.g., a "smart" refrigerator). Some devices may have network connectivity but may not be powered by electrical wiring (e.g., mobile devices such as smartphones), or may only be powered by electrical wiring at certain times (e.g., while charging). The techniques described herein can provide more and/or more accurate information about the devices in a building by using a combination of information from the electrical wiring and information from the network within the building.

Information from power lines and the building's network can be used to provide services to users, informing them about the status of equipment within the building. Companies can offer power monitoring devices that can be installed in buildings and connected to both the building's power lines and computer networks. These devices can use both power line and network data to determine what equipment is present in the building and its status (e.g., on or off). This information can then be made available to users, such as by presenting it in a dedicated app or webpage (e.g., on a smartphone). Services can provide users with information about the equipment in the building, such as the equipment's status, real-time information about the power used by the equipment, and historical information about the equipment's activity.

For clarity, the technologies described herein use houses or homes as examples of buildings where the technologies can be applied; however, the technologies described herein can also be used in any environment where electricity is used, including but not limited to business and commercial buildings, government buildings, and other locations. The reference to homes throughout should be understood to encompass such other locations.

Power monitoring

Figure 1 illustrates an example of system 100, where information obtained from the wiring can be used to determine information about devices in the home. In Figure 1, electrical company 120 supplies electrical power to home 110. The electrical power is transmitted to distribution board 140, where it can then be distributed to different circuits throughout the house.

Distribution board 140 can be any distribution board found in a building. For example, distribution board 140 can implement phased power distribution, where a 240-volt AC signal is converted using a phase transformer into a three-wire distribution with a single grounding point each providing 120 volts and two main lines (or branches). Some equipment in the house can use one of the two main lines to obtain 120 volts; other equipment in the house can use the other main line to obtain 120 volts; and other equipment can use both main lines simultaneously to obtain 240 volts.

Any type of distribution board can be used, and the techniques described herein are not limited to phase distribution boards. For example, distribution board 140 can be single-phase, two-phase, or three-phase. The techniques are also not limited to the number of trunk lines provided by distribution board 140. In the following discussion, distribution board 140 will be described as having two trunk lines, but any number of trunk lines can be used, including a single trunk line. As will be understood by those skilled in the art, other voltage standards, such as those used in other countries or continents, are intended to be covered herein.

Figure 1 illustrates devices that consume power supplied by the distribution board 140. For example, a power monitor 150, a refrigerator 160, and a stove 165 may consume power supplied via the distribution board 140.

Power monitor 150 can be connected to sensor 130 to measure the electrical properties of wires connected to house 110. For example, sensor 130 can measure the voltage and/or current levels of wires supplying power to distribution panel 140. Measurement results can be obtained using any available sensor, and the technology is not limited to any particular sensor or any particular type of value that can be obtained from a sensor. Sensor 130 may include multiple sensors, such as one or more sensors for each main line.

Sensor 130 can provide one or more power monitoring signals to power monitor 150, such as measurements of current and/or voltage on each main line connected to distribution board 140. Power monitor 150 can process the power monitoring signals to de-aggregate them or to obtain information about individual devices in the home. For example, power monitor 150 can determine changes in the status of devices such as a television being turned on at 8:30 p.m., or a refrigerator compressor starting at 10:35 a.m. and 11:01 a.m.

The power monitor 150 may be a device obtained separately from the distribution board 140 and installed by a user or electrician to connect to the distribution board 140. The power monitor 150 may be part of the distribution board 140 and installed by the manufacturer of the distribution board 140. The power monitor 150 may also be part of an electricity meter (e.g., integrated with or incorporated into the meter), such as a meter supplied by an electrical company and sometimes referred to as a smart meter.

The power monitor 150 may use any suitable technique for performing deaggregation or determining information about the power usage or status of an individual device based on the power monitoring signal. For example, the power monitor 150 may use any technique described in U.S. Patent 9,443,195.

Figure 2 illustrates an example of a hypothetical power monitoring signal 200 that can be processed by the power monitor 150. Figure 2 illustrates changes in the power monitoring signal caused by changes in the states of two devices, a furnace and an incandescent light bulb. Each of these changes in the power signal can be referred to as a power event. In Figure 2, a power event labeled HE1 corresponds to the heating element of the furnace burner being turned on, and a power event labeled HE0 corresponds to the heating element of the burner being turned off (for an electric furnace, the burner may appear to be on to the user, but the furnace can periodically turn the heating element on and off to maintain a desired temperature). A power event labeled I1 corresponds to the incandescent light bulb being turned on, and a power event labeled I0 corresponds to the incandescent light bulb being turned off.

In power event 210, the furnace burner is turned on, and the burner's heating element consumes electricity. Therefore, power usage increases in the power monitoring signal. In power event 220, when the heating element is on, the incandescent bulb is turned on to further increase power usage. In power events 230, 240, 250, and 260, the heating element is turned off, then on, then off again, and then on again. In power event 270, the incandescent bulb is turned off, and in power event 280, the furnace burner is turned off, and the heating element stops consuming electricity.

The power monitor 150 can process the power monitoring signal 200 to identify the device and/or determine changes in the device's state. Some state changes may correspond to a person turning the device on or off, and some state changes may correspond to changes in the device's function (e.g., the circulation of a heating element or a washing machine changing from a washing mode to a spin mode). The power monitor 150 can use any suitable technology for identifying the device and determining changes in the device's state, such as any technology described in U.S. Patent 9,443,195.

In some implementations, the power monitor 150 may utilize a mathematical model to process power monitoring signals to identify devices and determine changes in device status. Such a model will be referred to as a power model. For example, the power monitor 150 may have power models for different types of devices (e.g., stoves, dishwashers, refrigerators, etc.), devices of different manufactures (e.g., Kenmore dishwashers, Maytag dishwashers, etc.), different versions of devices (e.g., Kenmore 1000 dishwashers), and specific devices (e.g., dishwashers at 100 Main Street). The power monitor 150 may also have power models for specific state changes, such as device on, device off, or device changing operation in another way (the dishwasher's water pump turns on or off). In common use, "1000" in Kenmore 1000 dishwasher may be referred to as the "model" of the dishwasher, but to avoid confusion with mathematical models, the "model" of the dishwasher will be referred to herein as the "version".

When power monitoring signals are processed using power models (such as any of the power models mentioned above), the power models can generate scores (e.g., probability, likelihood, confidence, etc.) indicating a match between the power model and the power monitoring signal. When a power event occurs in the power monitoring signal (such as any of the events in Figure 2), a portion of the power monitoring signal including the power event can be processed by multiple power models, and scores can be generated by each of the power models. The power model scores can be used to identify devices in the home and/or determine changes in the state of devices in the home.

In some implementations, power monitoring signal processing can be performed as follows: The power monitoring signal can be continuously processed to identify power events within the signal. In cases where the power monitoring signal has a stable value or slight fluctuations, there may be no change in the state of any equipment in the house. The power event detection component can detect changes in the electrical signal that may correspond to a change in the state of the equipment, and cause these portions of the power monitoring signal to be further processed. The power event detection component can be implemented using any suitable technique, such as a classifier.

After a power event is detected, a feature generation component can be used to generate a feature from the portion of the power monitoring signal that includes the power event. Any suitable feature can be calculated, such as any feature described in U.S. Patent 9,443,195.

The features can then be processed by one or more power models to identify devices or determine device state changes corresponding to power events. Any suitable power model can be used, such as the transition model, device model, watt-number model, and previous models described in U.S. Patent 9,443,195. For example, each power model can generate a score, and device state changes can be determined by selecting the power model with the highest score.

Techniques for using power models to identify devices in a home are now described. In some embodiments, the power monitor 150 may be equipped with an initial set of power models corresponding to devices that may be in the home. When processing the power monitoring signal, the power models can be used to generate scores for power events occurring in the power monitoring signal. These scores can be used to identify what devices are in the home.

In some implementations, the power monitor 150 may identify a device as being in the home if the score generated by the power model exceeds a threshold. For example, when the dishwasher model processes a portion of the power monitoring signal corresponding to the start of the dishwasher, it may generate a score exceeding the threshold. The threshold may be specific to the dishwasher model, or it may be the same for all power models. In some implementations, a confidence level may be calculated in addition to or instead of the score, and the device will be identified when the confidence level exceeds the threshold.

In some implementations, additional criteria may be considered before identifying the device. For example, the power monitor 150 may have multiple dishwasher power models corresponding to different types of dishwashers (e.g., conventional dishwashers and energy-efficient dishwashers), or power models may exist for dishwashers of different manufactures. When processing a power monitoring signal corresponding to the start of a dishwasher, the multiple dishwasher models may produce scores (or confidence levels) exceeding a threshold. Instead of identifying multiple dishwashers, a single dishwasher corresponding to the model with the highest score exceeding the threshold can be identified. For example, the power monitor 150 may have power models for Kenmore and Bosch dishwashers. Each model may generate scores exceeding the threshold, but the score of the Kenmore model may be higher than that of the Bosch model. Therefore, the Kenmore dishwasher can be identified.

In some implementations, the power monitor 150 can be updated using additional power models after a device has been identified. For example, after it has been determined that a house has a dishwasher, a power model can be added to the power monitor 150 corresponding to the most common manufacturer of the dishwasher. The power monitoring signal can then be processed using power models for dishwashers of different manufacturers, and the manufacturer of the dishwasher in the house can be identified using the highest-scoring power model. This process can be repeated to determine additional information, such as the version of the dishwasher (e.g., a Kenmore 1000 dishwasher).

After identifying the devices in the house, the power monitor 150 can process the power monitoring signal to determine changes in the state of the identified devices. In addition to having a power model for specific devices, the power monitor 150 may also have a model for specific state changes of the devices, and these models can be used to determine when the devices change state.

As described above, a power model for device state changes can be used to process power monitoring signals to generate a score indicating possible state changes. If the score (or confidence level) exceeds a threshold, the power monitor 150 can determine that a state change corresponding to the model has occurred. For example, the power monitor 150 may have power models for dishwasher start-up and end-up operations. When the power model for dishwasher start-up generates a score exceeding a threshold, the power monitor 150 can determine that the dishwasher has started. Similarly, when the power model for dishwasher end-up generates a score exceeding a threshold, the power monitor 150 can determine that the dishwasher has finished its cycle.

The power model used to determine state changes does not need to be the same as the power model used to identify devices, and the threshold used to determine state changes does not need to be the same as the threshold used to identify devices. The model and threshold can be adjusted to achieve a desired trade-off between accuracy and error rate.

The aspects of the operations described above for identifying the device and determining changes in the device's state can be performed by other computers instead of the power monitor 150 (e.g., a server computer operating in conjunction with the power monitor). For example, in some embodiments, the power monitor 150 may provide a power monitoring signal to another computer (e.g., a server), and this other computer may be able to identify the device and the state change. In some embodiments, the other computer may be able to identify the device, and the power monitor 150 may be able to determine the change in the device's state.

Network surveillance

In some implementations, the power monitor 150 may be connected to a computer network in the home. For example, the power monitor 150 may have a wired connection to a router in the home (e.g., LAN Ethernet), a wireless connection to a network (e.g., Wi-Fi), or a direct network connection to other devices (e.g., Bluetooth). In these implementations, the power monitor 150 is also a network monitor, but for clarity, the term power monitor will continue to be used in the following description.

Figure 3 illustrates an example system 300 in which a power monitor 150 is connected to a network in the home. In this example, the power monitor 150 has a network connection to a network device 310, which can be any device that facilitates the network in the home, such as a modem, router, or hub. Other devices in the home can also be connected to the network device 310. For example, a television 320 (e.g., a smart TV), a computer 330 (e.g., a personal computer), a smart switch 340 (e.g., a Phillips Hue or Belkin Wemo switch), and a telephone 360 (e.g., an Android phone or iPhone) can also be connected to the home network. The power monitor 150 can connect to other devices in the home using any suitable wired or wireless network configuration, such as LAN Ethernet or direct connection to other devices. In some implementations, the power monitor 150 can communicate on multiple networks simultaneously (e.g., a Wi-Fi connection to a home router and a Bluetooth connection to a specific device).

Power monitor 150 can use data transmitted over a computer network to understand the devices in the house. For example, power monitor 150 can listen for broadcast messages from other devices, poll other devices, or listen for network data generated by other devices. In some embodiments, power monitor 150 can indirectly understand other devices in the house via a network outside the house. For example, devices in the house can transmit information to a third-party server operating in conjunction with the device, and the third-party server can transmit information to a server operating in conjunction with power monitor 150. Each of these techniques can be used to identify devices in the house and determine changes in device status (e.g., device turned on or off).

In some implementations, devices in the home can transmit data that includes information about the device itself (e.g., broadcast messages or responses to polling). For example, network transmissions may include any of the following: status (e.g., a device that has just been turned on or is about to be turned off), services provided by the device, a user-assigned name (e.g., “John’s Mac”), manufacture, hardware version, software version, network address (e.g., IP address), identification number (e.g., MAC address, device serial number, universally unique identifier, globally unique identifier, or temporary identifier), or other information (e.g., protocol-specific identification (such as a ZeroConf service name) or a resource locator used with the SSDP protocol).

Figure 4A illustrates an example timeline of a power monitor 150 learning about another device by listening to broadcast messages from that device (broadcasting device). Some devices can be configured to broadcast information across a network to advertise the services or capabilities offered by the device to other devices on the network. For example, a television can provide a service that allows other devices (e.g., telephones or personal computers) to stream video content to the television for display, and the television can broadcast messages to the network so that other devices know that the service is available.

In Figure 4A, the broadcasting device transmits a message to announce the availability of the service, and later transmits a message to announce the cancellation of the service. In Figure 4A, time progresses from top to bottom, and at 410, the broadcasting device is turned on, such as by a person. Then, at 411, the broadcasting device can announce the services it provides by broadcasting one or more messages on the network, and these messages can be received by other devices on the network (including power monitor 150). Later, the broadcasting device can receive a power outage event at 412, such as input (e.g., from a person) to shut itself down. After receiving this input, the broadcasting device can broadcast a message that the service has been cancelled to other devices on the network at 413, and this message can be received by power monitor 150. After broadcasting the message, the broadcasting device can turn off at 414.

Broadcasting is not limited to what happens after a device is powered on and before it is powered off; it can be used at any time to announce any changes to a service or to alert other devices on the network that a service is available. For example, a device may broadcast messages before entering sleep mode or after waking from sleep mode to indicate the device's status, to indicate that the device is in use (e.g., playing a song or information about the song being played), or to indicate that the type of media being presented has changed. In some implementations, devices may use Simple Service Discovery Protocol (SSDP), zero-configuration networking (zeroconf), or NetBIOS to broadcast messages. For example, when using SSDP, a device may use multicast addressing to broadcast messages to all devices on the network.

Information from broadcast messages can be used to identify devices in the home. For example, the message may include data that can be used to identify the device, such as text or identifiers describing the device. Information from broadcast messages can also be used to determine the status of a device. For example, announcing service availability may indicate that the device is on, and announcing service withdrawal may indicate that the device is off. In some implementations, broadcast messages may provide some information about the device, and after receiving a broadcast message, power monitor 150 may poll the device to obtain additional information about the device's status. For example, a network speaker system may broadcast that the service is available, and power monitor 150 may then poll the network system to find out what music it is currently playing and information about the music being played (e.g., song title, volume, etc.).

Figure 4B illustrates an example timeline of a power monitor 150 using polling techniques to learn about another device. The power monitor 150 may poll other devices one at a time, or it may poll multiple devices simultaneously, but for clarity, only one polling device is shown.

At the start of the timeline in Figure 4B, the polling device is in a turned-off state. At 420, the power monitor 150 polls the polling device, but because the polling device is in a turned-off state, it does not respond to the polling request. The power monitor 150 can determine that the polling device is in a turned-off state when it does not receive a response to the polling request (or may not receive responses to multiple polling requests).

At 421, for example, the polling device is turned on by a person. At 422, the power monitor 150 polls the polling device again. Since the polling device is now on, the polling device responds at 423. After receiving a response from the polling device, the power monitor 150 can determine that the polling device has transitioned from a closed state to an open state. The polling response may include information that can be used to determine the state of the device, as described in more detail below. The polling response may include any information described above for broadcast messages.

At 424, power monitor 150 polls the polling device again, and at 425, the polling device responds to the polling request, allowing power monitor 150 to determine that the polling device is still on. At 426, the polling device is turned off. At 427, power monitor 150 polls the polling device again and receives no response, therefore power monitor 150 can determine that the polling device is now off.

Any suitable polling technique can be used to poll devices. In some implementations, the broadcast techniques described above can also allow polling. For example, SSDP can allow devices to poll other devices on the network to determine which devices provide what services, and which devices can respond with messages similar to the broadcast messages described above. In some implementations, lower-level protocols can be used for polling, such as Internet Control Message Protocol (ICMP) sending ping messages or Address Resolution Protocol (ARP) sending echo messages.

Information from polling responses can be used to identify devices in the house. For example, a polling response may include data that can be used to identify a device, such as text or identifiers describing the device. Polling responses can also be used to determine the status of a device. For example, a lack of response to a polling request may indicate that a device is off, a response may indicate that a device is on, and information in the response may provide additional information (e.g., the song currently playing).

Figure 4C illustrates an example timeline of how a power monitor 150 learns information about another device by monitoring network data sent from the device to other devices. In some implementations, individual device monitoring may be used only with explicit user selection, or only network packet headers (rather than packet bodies) may be used to avoid collecting too much information or to avoid collecting sensitive information.

In Figure 4C, the monitored device is turned on at 430. At 431, 432, 433, and 434, the monitored device transmits data over the network to another device, which may be different from the power monitor 150. At 435, the monitored device is turned off and it stops sending network transmissions. The power monitor 150 may passively receive network data over the network, for example, by passively receiving network packets.

The power monitor 150 can process the received data to determine information about the monitored device. Information in the received data can be used to identify the device, such as text or identifiers describing the device. The received data can also be used to determine the device's status. For example, the fact that the device is transmitting data indicates that the device is on, and if the device does not transmit any data for a period of time, it can be determined that the device is off.

In some implementations, the power monitor 150 may have information about publicly available APIs for third-party devices and use these APIs to determine the presence of such third-party devices in the house. For example, the power monitor 150 may periodically make requests using the Nest thermostat API to determine the presence of a Nest thermostat in the house. After determining that the Nest thermostat is in the house, it may be polled more frequently to determine the status of the Nest thermostat or the heating/cooling system. The API may allow, for example, other devices to determine the temperature of the house, a user-set desired temperature, or the status of the heating or cooling system (e.g., whether the furnace is currently active or the start and stop times of furnace activity). Utilizing a dedicated API query device can allow the power monitor 150 to determine the status of the device itself (e.g., the Nest thermostat) and other devices connected to it (e.g., the furnace).

In some implementations, the power monitor 150 can subscribe to notifications transmitted by a third-party device, referred to herein as a notification device. The notification device can be configured to (e.g., periodically or upon a change in state) transmit notifications to other devices and allow other devices to register for notifications using an API. The power monitor 150 can determine that the notification device is in the house and then subscribe to receive notifications from the device. The power monitor 150 can transmit a request to receive notifications from the notification device without knowing that the notification device exists, and subscribe to receive notifications if the notification device exists. Some notification devices may have a pairing process requiring user assistance, and the power monitor 150 can be configured to receive input from the user to assist the pairing process, such as receiving a username and password to pair with the notification device.

Other third-party devices may also provide information about the devices they are connected to, such as the smart switch 340 (e.g., Phillips Hue or Belkin Wemo). The smart switch may have an API that allows other devices to interact with it, and the power monitor 150 can use this API to determine the state of the switch, and accordingly, whether the device connected to the smart switch is consuming power (e.g., device on, device off, or dimmer-type switch at 40%).

In some implementations, the power monitor 150 may receive information about devices in the house via a server located outside the house (such as a third-party server operating in conjunction with third-party devices in the house). The power monitor 150 may operate in conjunction with a power monitor server, and third-party devices in the house may operate in conjunction with the third-party server. For example, a Nest thermostat may send information about the status of the thermostat or the house's heating/cooling system to a server operated by Nest. The third-party server may allow users to provide configuration information to transmit information to the power monitor server using, for example, an API of the power monitor server or a third-party server. For example, the Nest server may transmit information received from the Nest thermostat in the house to the power monitor server, such as on a periodic basis or when the status of the thermostat changes (or the status of the heating/cooling system changes). In some implementations, the power monitor server may send requests for information about third-party devices to the third-party server instead of receiving notifications from the third-party server. In some implementations, the third-party devices may communicate directly with the power monitor server, or the power monitor 150 may communicate directly with the third-party server.

In some implementations, information received in network transmissions from a device can be used to obtain further information about the device. For example, network transmissions may include unique identifiers, such as Media Access Control (MAC) addresses. Unique identifiers can be used to determine additional information about the device using data storage or information repositories about devices using unique identifiers. Data storage can provide information about the type, manufacture, and/or version of the device based on the unique identifier. For example, for MAC addresses, blocks of consecutive MAC addresses may be specific to the type of device (e.g., a television), manufacturer, or version of the device (e.g., a specific model of a television). In some implementations, the data storage for MAC address information may be available, such as through a third-party service, and information about the device can be obtained using the MAC address. Data storage for identifiers can be created, purchased, or (e.g., using a third-party server) accessed to obtain information about the device from the identifiers.

In some implementations, a user device (such as a smartphone) can be used to determine information about devices in the house, and the smartphone can relay this information to the power monitor 150. For example, a speaker 370 (e.g., a portable Bluetooth speaker) may not have a network connection to the network device 310 and may alternatively use another network (such as a Bluetooth network) to communicate with other devices. The speaker 370 may be too far from the power monitor 150 for the power monitor to detect the speaker 370's network. A user device such as a phone 360 can be brought into the same room as the speaker 370, and the phone 360 may be able to use a direct network connection to determine the presence and/or operational status of the speaker 370. The phone 360 can then relay information about the presence and/or operational status of the speaker 370 to the power monitor 150 or a server that works in conjunction with the power monitor 150.

In some implementations, the network model can be used to identify devices or determine the state of devices by processing network data. The network model can include any suitable mathematical model, such as classifiers, neural networks, self-organizing maps, support vector machines, decision trees, random forests, logistic regression, Bayesian models, linear and nonlinear regression, and Gaussian mixture models. The network model can receive input data obtained from network transmissions and output identification of devices or changes in device state with optional scores. For example, the network model can receive broadcast messages, information about polling responses received from a device (or their absence), or information about network data generated by the device. In some implementations, the network model may receive only the headers of network transmissions, or it may receive all data from network transmissions.

In some implementations, network models can be used to identify devices or to determine the state of a device using messages broadcast by the device. For example, a device may transmit a certain number and/or type of messages before shutting down, and a different number and/or type of messages when entering sleep mode. A first network model describing the expected messages can be created when the device is about to shut down, and a second network model describing the expected messages can be created when the device is about to enter sleep mode. When broadcast messages are received from the device, the messages can be processed using both network models to generate a score for each model, and the state transition can be determined by the network model with the highest score.

Network models can also be created to describe other aspects of the network transmissions described above. For example, a network model can be created to describe the frequency at which a device responds to polling requests and the expected time delay between transmitting a polling request and receiving a response. In another example, a network model can be created to describe the expected network transmissions of a device in a specific state, and this network model can be applied when passively monitoring the network transmissions of a device.

In some implementations, rule-based methods can be used to identify devices in a building or determine the status of devices. Power monitor 150 (or a server operating in conjunction with a power monitor) may have rules already created for identifying the type, manufacture, or version of a device; rules for determining changes in the status of a device; and rules for determining other aspects of the device. Rules can be created for any of the techniques described above, such as processing broadcast messages from devices, polling devices, or monitoring network data generated by devices.

Some rules can output Boolean values to indicate whether the rule's conditions are met. For example, a rule can be created to determine whether a device transmitting network data is a television, and if the conditions of that rule are met, then the device is determined to be a television. Some rules can output scores, for example, on a scale of 1 to 100, to indicate a match between the network data and the rule. For example, a rule can be created to determine whether a device transmitting network data is a television, and the score generated by the rule can be used to make that determination, such as by comparing the score to a threshold or by combining the score with other scores as described herein.

Multiple rules can exist for making a determination. For example, multiple rules can exist for determining whether a device transmitting network data is a television, and if any one of the rules is satisfied, then the device can be determined to be a television.

Any data in or related to network transmission can be used as input to a rule. For example, information extracted from network transmissions (such as network addresses, identifiers, headers, and strings) can be used as input to a rule. Information not in the network transmission but related to it, such as the time the network transmission was received, can also be used.

In some implementations, a rule may include one or more conditions that need to be satisfied for the rule to be satisfied. For example, conditions may include any comparison of data, such as not equal, equal, greater than, or less than. A rule may take any combination of conditions, including but not limited to combinations using Boolean algebra. Examples of rules include: a device broadcasting ZerConf services including AFP, HTTP, and SSH is an Apple macOS computer; structured data returned as part of a NetBIOS status polling request provides the host's user-visible name; a device with a previously unseen MAC address making a DHCP request is a new device joining the network for the first time; and the device's vendor is determined using a known MAC address vendor prefix.

In some implementations, device fingerprinting technology can be used to identify devices on a network. Device fingerprints can be created for known devices such as Mac computers, Windows computers, and Linux computers, and possibly different versions of each operating system. Fingerprints can be created by sending multiple requests to each device using, for example, different protocols and port numbers (e.g., using a program such as Nmap). The responses of each device can be recorded to create a fingerprint for each device. When creating device fingerprints, any appropriate data can be used, such as the number and type of network protocols broadcast by the device, broadcast behavior, the number or frequency of different types of broadcasts, or parameters used for network transmission (e.g., TCP window size).

For an unknown device in a building, a similar request can be sent to that device, and the response can be used to create a fingerprint of the unknown device. The fingerprint of the unknown device can be compared with the fingerprints of known devices to determine information about the unknown device. For example, if the fingerprint of the unknown device has the closest match to the fingerprint of a Mac laptop computer, the unknown device can be identified as a Mac laptop computer. In some implementations, the Address Resolution Protocol (ARP) can be used to identify devices on the network, and then the techniques described above can be used to fingerprint each of the identified devices.

Any of the techniques described above for using network data to identify devices or determine device status can generate scores indicating a match between the data and the identified device or status change. For example, applying a rule to a broadcast message could result in determining that the device is a Toshiba TV with a score of 80% or a Sony TV with a score of 60%. The scores generated by the network model can be combined with scores generated by the power model, as described in more detail below.

Combined power and network monitoring

Figure 5 illustrates a system 500 for providing a user with information about devices in their home using one or both power monitoring and network monitoring. In Figure 5, the power monitor 150 can have any of the aforementioned functionalities. For example, the power monitor 150 can identify the presence of devices in the home, determine the status of the devices, and determine the power consumption of the devices. The power monitor 150 can use any known networking technology to transmit information about the devices in the home to a server 510. For example, the power monitor can have a wireless connection to a router, which in turn connects to the server 510.

Server 510 can process information received from power monitor 150 and, for example, present information to a user via user equipment 540. Server 510 can maintain a device list for devices used in the home and update the device list using newly identified devices or updated device status. Server 510 can also log device status changes, record the power consumption history of home and individual devices, and perform any other operations described in U.S. Patent 9,443,195. To perform some operations, server 510 can access other resources, such as third-party server 530 and device information data storage 520.

A user can use user device 540 to obtain information about devices in the home. User device 540 can be any device that provides information to the user, including but not limited to telephones, tablets, desktop computers, and wearable devices. User device 540 can present the user with information such as changes in device status and real-time power usage. For example, user device 540 can present the user with a webpage, or a special-purpose application can be installed on user device 540. The information presented by user device 540 can include any information described in U.S. Patent 9,443,195.

To provide users with information about devices in their homes, a list of devices in the home can be maintained. Figure 6 illustrates an example device list 600. The list of devices may include any information related to the devices in the home, including but not limited to device IDs (which may be specific to the home or known to all devices by the company providing the service), names (e.g., names provided by the user), types, manufacturing, versions, one or more power models used to identify changes in the device's state, network IDs (e.g., network addresses or other identifiers such as MAC addresses), and one or more network models used to identify changes in the device's state and the device's status.

The device list 600 can be stored in one or more locations. For example, the device list 600 can be stored in one or more of the following locations: power monitor 150, server 510, device information data storage 520, or user equipment 540. Different versions of the device list can be stored in different locations corresponding to the processing at that location. For example, power monitor 150 may not store the name, type, manufacture, or version of the devices, because this information may not be needed to determine device status changes. User equipment 540 may not store information about the power model and network model, because user equipment 540 may not be aware of any device status changes.

Figure 7 illustrates a system 700 with a power monitor 150 that processes power monitoring signals and network data to identify devices in the home and to detect changes in the status of those devices. As described above, home 110 is supplied with power from electrical company 120. A distribution board 140 receives the power and distributes it to devices in the home. Some devices receive power from the distribution board 140 and are not connected to the network (e.g., refrigerator 160, stove 165, and light bulb 350). Some devices receive power from the distribution board and are connected to the network (e.g., power monitor 150, network device 310, smart TV 320, computer 330, and smart switch 340). Some devices may be connected to the network but do not receive power from the distribution board 140, or only sometimes receive power from the distribution board (e.g., telephone 360, which can be charged using telephone charger 710).

As described above, power monitor 150 can receive one or more power monitoring signals from sensor 130 that measure the electrical properties of the power supplied to house 110. Power monitor 150 is also connected to the network in house 110 via network device 310. Power monitor 150 can receive network data from other devices via network device 310 (as indicated by the dashed line), or it can receive network data directly from other devices (not shown in Figure 7). For some devices in a home, one or both of the power monitoring and network monitoring may be inaccurate or have a higher error rate than expected in identifying devices or changes in device status. By using both power monitoring and network monitoring collaboratively or simultaneously, the performance in identifying changes in device status can be improved. Simultaneous power monitoring and network monitoring can be achieved using any suitable technique, such as combined scoring, rule-based, or voting techniques, as described in more detail below.

In some implementations, the operation of the power monitor 150 may vary depending on whether a user is interacting with user equipment 540 to view information about devices in the home. When a user is using user equipment 540 to view information about devices in the home, it may be expected to identify new devices or changes in device status more quickly than when no user is viewing information about devices in the home.

When a user begins using user device 540 to view information about devices in the home, user device 540 can establish a network connection to server 510, and server 510 can transmit information about the devices to user device 540. Server 510 may also have a network connection (directly or indirectly) to power monitor 150. Therefore, server 510 can transmit information to power monitor 150 to change the operation of power monitor 150. Any changes to the operation of power monitor 150 can be made to improve the end-user's user experience. In some implementations, power monitor 150 can change its operation to more quickly identify new devices and changes in the status of new devices. For example, the power monitor can increase the polling frequency used to poll other devices in the house. A polling frequency of 5 minutes may be sufficient to determine updates when no user is viewing information about devices. When a user begins viewing information about devices, the polling frequency can be increased (e.g., 10 seconds) to determine updates more quickly.

The arrangement and specific functions of the components in Figures 5 and 7 provide only one example of how the techniques described herein can be implemented, but other configurations are possible. For example, power monitor 150 may perform some or all of the operations of server 510, and power monitor 150 may provide information directly to user equipment 540. In another example, power monitor 150 may include some or all of the functionality of user equipment 540, and the user may interact directly with power monitor 150 to obtain information about device events and power consumption.

Identification devices

When the power monitor 150 is first installed in a home, a device list can be created for the devices in the home. In some implementations, an empty device list can be created. In some implementations, the device list can be initialized using devices based on input from one or more people in the home. For example, a user can specify any type, manufacture, and/or version of the devices in the house.

Power monitor 150 may use one or both power monitoring and network monitoring to add or update devices in the device list using any of the techniques described herein. Devices may be added to the device list after it has been identified by the power monitor, and devices in the device list may be further updated as additional information about them is determined. For example, power monitor 150 may initially determine that the house has a dishwasher, and it may later determine that the house has a Kenmore 1000 dishwasher.

In some implementations, the power monitor 150 may add multiple devices to the device list in response to processing power events in the power monitoring signal. For example, if two power models produce scores above a threshold, the device corresponding to each power model may be added to the device list. In some implementations, scores produced by the power models may also be included in the device list.

The power monitor 150 can also process network data to identify devices in the house and add them to a device list. For example, the power monitor 150 can use any of the techniques described above to obtain information about devices connected to the network in the house, and then add the devices to the house's device list.

In some implementations, the power monitor 150 may add multiple devices to a device list (optionally with scores) in response to processing network data transmission. For example, in response to processing a broadcast message, a Toshiba TV may be added to a device list with a first score, and a Sony TV may be added to a device list with a second score.

The scores of devices in the device list can be updated over time. For example, processing a first power event can add a first device and a second device to the device list with corresponding scores. At a later time, the power monitor 150 can process a second power event to generate a second set of scores for the first and second devices. The second set of scores generated using the second power event can be used to update the overall scores of the first and second devices in the device list.

Similarly, later network data transmissions can be used to update the scores of devices in the device list. For example, based on processing the first network transmission, first and second devices can be added to the device list with corresponding scores. Later, the power monitor 150 can process a second network transmission and generate a second score for the first and second devices. As described above, the second set of scores generated using the second network transmission can be used to update the overall score of the first and second devices in the device list.

In some implementations, devices in the device list may have scores generated using both power monitoring and network monitoring. After processing a power event, a first device can be added to the device list with a corresponding score. Later, the score of the first device can be updated by processing a first network transmission. The overall score of the devices in the device list can be generated by processing any combination of power events and network transmission events.

Score combination can be achieved using any suitable technique. In some implementations, the overall score for a device can be the average of all individual scores generated for that device. In some implementations, variance can be determined for each score, and the scores can be normalized (e.g., Z-scored) before combination.

In some implementations, the power monitor can use both power monitor monitoring and network monitoring simultaneously to identify devices. For devices connected to a power line and having a network connection, the device can generate a power event in the power monitoring signal near the time of a state change and transmit data over the network (network event). The power monitor 150 can process both power events and network events simultaneously to identify devices.

Power events and network events can occur in any order. Figure 8A illustrates an example where a power event 810 occurs at a first time, followed by a network event 820 at a second time. For example, power event 810 could correspond to a television being turned on, and network event 820 could correspond to a broadcast message announcing the availability of a television service. Figure 8B illustrates an example where a network event 850 occurs at a first time, followed by a power event 860 at a second time. For example, a user could turn off the television, and network event 850 could be a broadcast message canceling a service, while power event 860 could correspond to the television being turned off.

When processing power events, the power monitor 150 can look for network events that are temporally close to the power event (e.g., within a time interval, such as within 1 second) and process them simultaneously. For example, the power monitor 150 can process network events where the difference between the time of the power event and the time of the network event is less than a threshold. Similarly, when processing network events, the power monitor 150 can look for power events that are temporally close to the network event.

In some implementations, the power monitor 150 can process power events and generate scores corresponding to power models. The power monitor 150 can then look for network events that are temporally close to the power events and use information from the network events to adjust the scores generated by the power models. For example, the first two power models could be used for Toshiba TVs and Sony TVs, and the scores could be close to each other. Network events can occur shortly after the power events, and these network events can correspond to Toshiba TVs. The power monitor 150 can use the network events to increase the score used to identify Toshiba TVs, and then identify and add the Toshiba TVs to the device list.

The time difference between power events and network events can also be used to identify devices. A device may emit a network event with a consistent delay after causing a power event, or vice versa. For example, a television turning on may consistently emit a network event (e.g., broadcast service) approximately 3.5 seconds after causing a power event. The timing of power events and network events can be a feature that can be used with any of the identification techniques described herein.

In some implementations, a device identification model can be used that receives both information about power events and information about network events as input, and then generates a score indicating a match between the input and the device. For example, a device identification model can be created based on the type of device, the manufacture of the device, or the version of the device. In some implementations, the power monitor 150 can calculate a score for each device identification model, select the device identification model with the highest score, and use the highest-scoring device identification model to identify the device. Any suitable classification technique can be used to create the device identification model, such as neural networks, self-organizing maps, support vector machines, decision trees, random forests, logistic regression, Bayesian models, linear and nonlinear regression, and Gaussian mixture models.

In some implementations, a confidence level can be calculated in addition to the score, and this confidence level can be used to determine whether to add a device to the device list or update information about devices in the device list. For example, the confidence level can be high if only one score is above a threshold and that score is significantly higher than all other scores. The confidence level may be low if multiple scores are above the threshold, none are above the threshold, or the highest score is close to the second-highest score. Any suitable technique can be used to determine the confidence level for a change in the status of the highest-scoring device. The device list may not be updated if the highest-scoring model (e.g., power model, network model, or device identification model) has a low confidence level.

Devices can also be removed from the device list. For example, a user can view the device list and remove devices that are not present in the home. Devices can also be removed automatically from the device list. For example, a device can be removed if it has not been identified for a period of time, if its overall score is below a threshold, or if enough data has been obtained to distinguish it from a replacement device (e.g., enough data has been collected to confidently determine that the TV is a Toshiba TV and not a Sony TV).

Figure 9 is a flowchart illustrating an example implementation of identifying equipment in a building. In Figure 9, the order of steps is exemplary, and other orders are possible. Not all steps are necessary, and in some implementations, some steps may be omitted or others may be added. The process described in the flowchart can be implemented, for example, by any computer or system described herein.

In step 910, a power monitoring signal is obtained. For example, a power monitoring signal can be obtained by receiving measurements from a sensor (such as a current or voltage sensor) that measures the electrical properties of power transmission lines in a building. Power transmission lines can power some or all of the equipment in the building. A power monitoring signal can also be obtained by receiving a power monitoring signal from another device. For example, a server can receive a power monitoring signal transmitted by a power monitor.

In step 920, the power monitoring signal is processed using multiple models to generate a score for each model. Each model may have been created to identify information about the device, such as the type of device, the manufacture of the device, or the version of the device. Any suitable model can be used, such as any model described herein. The score can be any value indicating a match between the power monitoring signal and the model, such as probability or logarithmic probability. Processing the power monitoring signal using models can be performed by extracting features from the power monitoring signal and then processing those features using models.

In step 930, the score is used to determine first information about a first device in the building. In some embodiments, a highest-scoring model can be selected, and the first information can be associated with the highest-scoring model. For example, the highest-scoring model could indicate that the first device is a dishwasher, and the first information could be: the device is a dishwasher. In some embodiments, information about network transmissions can be combined with a score from a power model to determine the first information about the first device, as described herein.

In step 940, the device list is updated using the first information about the first device. The device list can be stored in any suitable location, such as on a power monitor or on a server integrated with the power monitor. If the device list does not yet include any information about the first device, a new entry with the first information can be added to the device list. If the first device is already on the device list, the entry for the first device can be updated using the first information. For example, if the first information is: the first device is a Kenmore dishwasher, and the device list has an entry for the first device as a dishwasher, the device list can be updated to indicate that the dishwasher is manufactured by Kenmore. The power monitor can cause the device list to be updated locally, or it can cause the device list to be updated by transmitting the first information to the server, so that the server updates the device list.

In step 950, network data transmission is received from the second device via the network within the building. Network data transmission can be received directly from the second device (e.g., a point-to-point Bluetooth network) or indirectly from the second device (e.g., via a network router). The network data transmission can take any suitable format, such as network packets or Ethernet packets. Any of the techniques described herein can be used to receive network data transmission, such as receiving broadcast messages in broadcast network packets, receiving responses to polling requests, or monitoring network traffic on the network.

In step 960, network data transmission is used to determine second information about the second device. The second information can be determined using data or information in network packets or other information related to the network data transmission (such as the time the network transmission was received). In some embodiments, multiple network transmissions can be used to determine the second information. The second information about the second device can be determined using any of the techniques described herein, such as processing information related to network data transmission with rules or network models. In some embodiments, information in or about network data transmission can be combined with information obtained from power monitoring (e.g., a score from a power model) to determine the second information.

In step 970, similar to step 940, the device list is updated using the second information about the second device.

In step 980, a request for information about equipment in the building is received. For example, a user may use a user device (such as a smartphone or tablet) to view information about equipment in the building. Software running on the user device (e.g., a webpage viewed in a browser or special-purpose application) may request information about equipment in the building and, for example, send the request using a REST API. Requests that can be received may be associated with a server operating in conjunction with a power monitor.

In step 990, first information and second information are transmitted to the user equipment. For example, the first information may be: the first device is a dishwasher, and the second information may be: the second device is a Toshiba TV. The information may be transmitted by a server operating in conjunction with a power monitor, and the information may be presented by software running on the user equipment.

In some implementations, equipment in a building may be identified as described in the following clauses, any two or more of them, or in combination with other clauses presented herein.

1. A system for identifying devices in a building, the system comprising at least one computer including at least one processor and at least one memory, the at least one computer being configured to: obtain a power monitoring signal by measuring electrical properties of power transmission lines in the building, wherein the power transmission lines provide power to devices in the building; process a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, wherein the first power event corresponds to a state change of a first device; use the score to determine first information about the first device; cause the first device to be updated in a device list using the first information about the first device; receive a first broadcast network data packet, wherein the first broadcast network data packet is transmitted by a second device; use information in the first broadcast network data packet to determine second information about the second device; cause the second device to be updated in a device list using the second information about the second device; receive a request from a user equipment for information about devices in the building; and transmit the first information about the first device and the second information about the second device to the user equipment.

2. The system of Clause 1, wherein the first information about the first device includes the type of the first device, the manufacture of the first device, or the version of the first device.

3. The system of Clause 1, wherein at least one computer is configured to receive a second broadcast network packet, wherein the second broadcast network packet is sent by the first device; and to use a plurality of scores and information in the second broadcast network packet to determine first information about the first device.

4. The system of Clause 3, wherein multiple scores are used to determine the type of the first device and information in a second broadcast network data packet is used to determine the manufacture of the first device.

5. The system of Clause 3, wherein the first power event in the power monitoring signal occurs before the reception of the second broadcast network data packet.

6. The system of Clause 3, wherein at least one computer is configured to select a second broadcast network packet by comparing the time of a first power event and the time of a second broadcast network packet.

7. The system of Clause 1, wherein at least one computer comprises a power monitor installed in a building and a server computer having a network connection to the power monitor.

8. The system of Clause 1, wherein information in a first broadcast network packet includes an identifier of a second device, and wherein at least one computer is configured to use the identifier of the second device to obtain second information about the second device from a data storage.

9. The system of Clause 1, wherein at least one computer is configured to determine second information about a second device by processing information in a first broadcast network packet using multiple rules, wherein each of the multiple rules includes comparing the information in the network packet with at least one condition.

10. A computer-implemented method for identifying devices in a building, the method comprising: obtaining a power monitoring signal by measuring electrical properties of power transmission lines in the building, wherein the power transmission lines provide power to devices in the building; processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, wherein the first power event corresponds to a state change of a first device; using the score to determine first information about the first device; causing the first device to be updated in a device list using the first information about the first device; receiving a first broadcast network data packet, wherein the first broadcast network data packet is sent by a second device; using information in the first broadcast network data packet to determine second information about the second device; and causing the second device to be updated in a device list using the second information about the second device.

11. The computer-implemented method of Clause 10, wherein a power monitoring signal indicates the current or voltage of a transmission line.

12. The computer-implemented method of Clause 10 includes: receiving a plurality of broadcast network packets, each of the plurality of broadcast network packets being transmitted by a first device; and using a plurality of scores and information from the plurality of broadcast network packets to determine first information about the first device.

13. The computer-implemented method of Clause 10, wherein determining second information about the second device comprises: processing information in a first broadcast network data packet using a plurality of network models to generate a second score for each of the plurality of network models; and using the second scores of the plurality of network models to determine the second information about the second device.

14. The computer-implemented method of Clause 10, wherein determining second information about a second device includes generating a second score for each of a plurality of identification models, wherein each identification model processes information in a second power event in a power monitoring signal and in a first broadcast network data packet; and using the second scores of the plurality of identification models to determine second information about the second device.

15. The computer-implemented method of Clause 10 includes: receiving a plurality of broadcast network packets, wherein each of the plurality of broadcast network packets is transmitted by a second device; and using information from the plurality of broadcast network packets to determine second information about the second device.

16. The computer-implemented method of Clause 15, wherein determining the second information comprises: using a first time corresponding to a first broadcast network data packet and a second time corresponding to a second broadcast network data packet.

17. One or more non-transitory computer-readable media, comprising computer-executable instructions that, when executed, cause at least one processor to perform actions including: obtaining a power monitoring signal by measuring the electrical properties of power transmission lines in a building, wherein the power transmission lines provide power to devices in the building; processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, wherein the first power event corresponds to a state change of a first device; using the score to determine first information about the first device; using the first information about the first device to update the first device in a device list; receiving a first broadcast network data packet, wherein the first broadcast network data packet is transmitted by a second device; using information in the first broadcast network data packet to determine second information about the second device; and using the second information about the second device to update the second device in a device list.

18. One or more non-transitory computer-readable media of Clause 17, wherein determining the first information about the first device includes using a plurality of scores to select the highest-scoring power model, and wherein the first information corresponds to the highest-scoring power model.

19. One or more non-transitory computer-readable media of Clause 17, wherein the second information relating to the second device includes the type of the second device, the manufacture of the second device, or the version of the second device.

20. One or more non-transitory computer-readable media of Clause 17, wherein at least one computer is configured to determine second information about a second device by processing information in a first broadcast network data packet using multiple rules.

The techniques described above use a combination of power monitoring and network monitoring to provide improved identification of devices in buildings compared to technologies based solely on power monitoring. With power monitoring alone, some devices may not be identifiable at all, or may be identifiable with low accuracy. The combination of network and power monitoring allows for the identification of a greater number of devices (such as networked devices without mechanical parts) and also allows for device identification with higher accuracy. Using information from both power and network monitoring allows for the creation of models and/or classifiers with a lower error rate than models and/or classifiers that use information only from power monitoring.

Identify changes in device status

In addition to identifying devices in the home, the power monitor 150 can also use one or both power monitoring and network monitoring to determine the status of devices in the home. Some devices (e.g., light bulbs) may have only two states: on and off. Other devices may have multiple states. For example, a washing machine may have a wash cycle, a rinse cycle, and a spin cycle. A television may have an on state, an off state, and a sleep state.

Power monitoring can be used to identify state changes using any of the techniques described herein and in U.S. Patent 9,443,195. For example, power monitor 150 can process power monitoring signals, select the power model that produces the highest score, determine a device state change based on the model, and update the state of devices in the device list.

Network monitoring can also be used to identify state changes using any of the techniques described herein. For example, a power monitor can process broadcast messages, poll devices, and monitor network data to determine the state of devices based on their network data, as described above.

In some implementations, a power monitor can simultaneously use both power monitor monitoring and network monitoring to identify changes in device state. For devices connected to both a power cable and a network connection, the device may generate a power event in the power monitoring signal and transmit data over the network (a network event) near the time of the state change. The power monitor can process both power events and network events to identify the state change. Power events and network events can occur in any of the aforementioned orders.

As described above, when processing power events, the power monitor can look for network events that are temporally close to the power event and process them simultaneously. For example, the power monitor can process network events where the difference between the time of the power event and the time of the network event is less than a threshold. Similarly, when processing network events, the power monitor can look for power events that are temporally close to the network event.

In some implementations, the power monitor can process power events and generate scores corresponding to power models. The power monitor can then look for network events that are temporally close to the power events and use information from the network events to adjust the scores generated by the power models. For example, the first two power models could be Toshiba TV on and computer on, and the scores could be close to each other. The network event may occur shortly after the power event, and the network event may correspond to Toshiba TV on. The power monitor can use the network event to potentially increase the score for Toshiba TV on. For example, the score can be increased by a fixed amount, a percentage, or it can be combined with the score generated by processing the network event to generate an overall score for a change in device state. The power monitor can then identify the state change as corresponding to Toshiba TV on. The timing of the power events and network events can also be used as a feature of any techniques described herein for determining changes in device state.

For some devices, the timing between device state changes and network events may not be precisely known. For example, in some implementations, a device can be determined to be off when it has not transmitted any network data for a period of time. Therefore, it can be determined that the device was turned off sometime after its last network transmission, but the exact time of its shutdown may be unknown. In some implementations, a power monitor can process network events to determine time-varying scores. For example, if the device's last network transmission occurred at a specific time, the power monitor may output a higher score for a period shortly after that time, and a lower score, such as a score with exponential decay, at a later time.

Some devices may have a known rebroadcast window, where the device will rebroadcast the services it provides at set time intervals. If a device has already broadcast but has not yet broadcast another service within the rebroadcast window, it can be determined that the device is off.

In some implementations, a state change model can be used, which receives both information about power events and information about network events as input, and then generates a score indicating a match between the input and a device state change. For example, a state change model can be created for each state change of the device (e.g., a change in state based on the device type, device manufacturing, or device version). In some implementations, the power monitor can compute a score for each state change model, select the state change model with the highest score, and use the highest-scoring state change model to identify the state change. State change models can be created using any suitable classification technique, such as neural networks, self-organizing maps, support vector machines, decision trees, random forests, logistic regression, Bayesian models, linear and nonlinear regressions, and Gaussian mixture models.

As mentioned above, in addition to scores, confidence levels can also be calculated and used to determine whether a state change has occurred. State changes may not be identified if the highest-scoring model (e.g., a power model, a network model, or a state change model) has a low confidence level.

When determining a device state change, the power monitor 150 can use a list of known states of the device and the possible transitions between those states. For example, a television may have three states: on, off, and sleep. However, not all state transitions are possible depending on how the television operates. For the three states, there are six possible state transitions (on to off, on to sleep, off to on, off to sleep, sleep to on, and sleep to off). From the "off" state, it is only possible to transition to the "on" state (off to sleep is impossible). From the "sleep" state, it is only possible to transition to the "on" state (sleep to off is impossible). Therefore, the power monitor can implement techniques (e.g., models or rules) for identifying all possible state changes of the device. When determining a device state change, the power monitor can select a state change from the list of possible state changes of the device and can determine the current state from the ending state of the state change (e.g., when selecting the off to on state change, it is determined that the device is now on). When determining the state of the device, the power monitor can select a state from the list of possible states of the device. The power monitor may select a state change or state depending on the implementation and/or the device being monitored.

Any of the techniques described above for identifying devices can also be used to determine changes in the state of the device. Similarly, any techniques described for determining changes in the state of a device can also be used to identify the device.

Figure 10 is a flowchart illustrating an example implementation of determining the state of a device. In Figure 10, the order of steps is exemplary, and other orders are possible. Not all steps are necessary, and in some implementations, some steps may be omitted or other steps may be added. The process in the flowchart can be implemented, for example, by any computer or system described herein.

In step 1010, information about devices in the building is accessed from a data storage source. This information can be stored in any suitable manner and in any suitable location or multiple locations. For example, the power monitor 150 can obtain information about devices in the building from a locally stored device list, which includes information about techniques used to determine changes in device status (e.g., information about models or rules). The power monitor can also access information from a device list stored on a server. The stored information may include information about devices in the building that are powered by power lines and/or connected to networks within the building.

In step 1020, a power monitoring signal is obtained, similar to step 910.

In step 1030, the state of the first device is determined using a power monitoring signal. Any techniques described herein can be used to determine the state of the first device, such as using multiple power models to calculate multiple scores and selecting a state or state change corresponding to the highest-scoring power model. The state or state change of the first device can be selected from a list of possible states or state changes of the first device. In some embodiments, network monitoring techniques may not be used to determine the state (even if the first device has a network connection). In some embodiments, power monitoring techniques may be combined with network monitoring techniques to determine the state of the first device, for example, by using any techniques described herein. In some embodiments, the state of the first device can be determined by first identifying a state change and then determining the state of the first device from the state change.

In step 1040, the status of the first device is updated in the stored information. Any technique described in step 940 for updating the device list can also be used to update the status of the first device in the device list. For example, a power monitor can update the status in a locally stored device list, or transmit the information to a server so that the server uses the status of the first device to update the device list.

In step 1050, network data transmission is received. Any technique described in step 950 can be used to receive network transmission. In some embodiments, a power monitor server operating in conjunction with a power monitor can receive network data transmission from another server operating in conjunction with a device in the house (e.g., a Nest server operating in conjunction with a Nest thermostat), the power monitor can receive network data transmission directly from a server operating in conjunction with a device in the house, or the power monitor server can receive network transmission directly from a device in the house.

In step 1060, it is determined that the second device transmitted network data. Any suitable technique can be used to determine that the second device transmitted network data, such as using a network address or identifier in the network data to identify the second device as the sender of the network data. Even if the power monitor receives network data from the second device via another device (such as a router), the second device can still be considered the sender or transmitter of the network data. If network data transmission is received via another server, it can be determined that the network data transmission corresponds to the second device but was not transmitted by the second device.

In step 1070, the state of the second device is determined using network data. Any techniques described herein can be used to determine the state of the second device using network data. The state or state change of the second device can be selected from a list of possible states or state changes of the second device. In some embodiments, power monitoring techniques may not be used to determine the state (even if the second device receives power from power lines in a building). In some embodiments, network monitoring techniques may be combined with power monitoring techniques to determine the state of the second device, for example, by using any techniques described herein. In some embodiments, the state of the second device can be determined by first identifying a state change and then determining the state of the second device from the state change.

In step 1080, the status of the second device is updated in the stored information. Any technique described in steps 940 or 1040 for updating the device list can also be used to update the status of the second device in the device list.

In step 1090, a request for information about equipment in the building is received. Any of the techniques described in step 980 can be used to request information about equipment in the building.

In step 1095, information regarding the status of the first device and the second device is transmitted to the user equipment. For example, information indicating that the first device is off and the second device is in sleep mode may be transmitted to the user equipment.

In some implementations, the state of the device may be determined as described in the following clauses, any two or more of them, or in combination with other clauses presented herein.

1. A computer-implemented method for determining the state of equipment in a building, the method comprising: accessing stored information about the equipment in the building, wherein the equipment in the building includes a first device and a second device, and wherein the information about the equipment includes the state of the first device and the state of the second device; obtaining a power monitoring signal by measuring the electrical properties of a power transmission line in the building, wherein the power transmission line provides power to the equipment in the building; determining the state of the first device by: (1) processing a first power event in the power monitoring signal using a plurality of power models to generate a score for each of the plurality of power models, and (2) selecting the state of the first device from a plurality of possible states using the score; updating the state of the first device in stored information using the selected state; receiving a first broadcast network data packet, wherein the first broadcast network is transmitted by the second device; determining, using information in the first broadcast network data packet, that the first broadcast network data packet was sent by the second device; selecting the state of the second device from a plurality of possible states of the second device using the information in the first broadcast network data packet; and updating the state of the second device in stored information using the selected state.

2. The computer-implemented method of Clause 1, wherein the first broadcast network data packet includes a broadcast using a simple service discovery protocol, zero-configuration networking, or NetBIOS.

3. The computer-implemented method of Clause 1, the method comprising: transmitting a request for information to a second device; and wherein receiving a first broadcast network data packet comprises receiving a response to the request for information.

4. The computer-implemented method of Clause 3, wherein transmitting a request for information includes: polling the second device, requesting information using the API of the second device, or requesting information about services provided by the second device.

5. The computer-implemented method of Clause 1, wherein selecting the state of the second device comprises: (1) processing information in a first broadcast network data packet using multiple network models, and (2) selecting the highest-scoring network model, wherein the highest-scoring network model corresponds to the selected state.

6. The computer-implemented method of Clause 1, wherein selecting the state of the second device includes processing information in a first broadcast network packet using multiple rules, wherein each of the multiple rules includes comparing the information in the network packet with at least one condition.

7. The computer-implemented method of Clause 1, wherein selecting the state of the second device comprises: processing a second power event in a power monitoring signal using multiple power models to generate a second score for each of the multiple power models; processing information about a first broadcast network data packet using multiple network models to generate a third score for each of the multiple network models; and using the second score and the third score to select the state.

The computer-implemented method of Clause 8, paragraph 1, wherein selecting the state of a second device includes generating a second score for each of a plurality of state-changing models, wherein each state-changing model processes a second power event in a power monitoring signal and information in a first broadcast network data packet; and using the second score to select the state.

9. The computer-implemented method of Clause 1, comprising: receiving a plurality of broadcast network packets, wherein each of the plurality of broadcast network packets is transmitted by a second device; and using information in the plurality of broadcast network packets to select a state of the second device.

10. A system for determining the state of equipment in a building, the system comprising at least one computer including at least one processor and at least one memory, the at least one computer being configured to: store information about equipment in the building, wherein the equipment in the building includes a first device and a second device, and wherein the information about the equipment includes the state of the first device and the state of the second device; obtain a power monitoring signal by measuring the electrical properties of power transmission lines in the building, wherein the power transmission lines provide power to the equipment in the building; determine the state of the first device by the following steps: (1) processing the first power event power monitoring signal using multiple power models to generate a score for each of the multiple power models, and (2) selecting the state of the first device from multiple possible states using the score; updating the state of the first device in the stored information using the selected state; receiving first network data from a server computer, wherein the server computer transmits the network data based on information received from a second device; determining that the first network data corresponds to the second device using the first network data; selecting the state of the second device from multiple possible states of the second device using the first network data; and updating the state of the second device in the stored information using the selected state.

11. The system of Clause 10, wherein at least one computer is configured to update the state of the first device in stored information by transmitting the state of the first device to a server computer.

12. The system of Clause 10, wherein the power monitoring signal indicates the current or voltage of the transmission line.

13. The system of Clause 10, wherein at least one computer includes a power monitor installed in a building and a server computer having a network connection to the power monitor.

14. The system of Clause 13, wherein the server computer is configured to determine that the user equipment has a network connection with the server computer; the server computer is configured to send an indication to the power monitor that the user equipment has a network connection with the server computer; and the power monitor is configured to modify its processing in response to receiving the indication.

15. The system of Clause 14, wherein the power monitor is configured to modify its processing by changing the frequency of polling the device.

16. The system of Clause 10, wherein at least one computer is configured to select the state of a second device by processing a second power event in a power monitoring signal using a plurality of power models to generate a second score for each of the plurality of power models; and to select the state using the second score and information in first network data.

17. The system of Clause 10, wherein at least one computer is configured to obtain information about the power consumption of the second device from a device information data storage using the state of the second device; and to transmit the information about the power consumption of the second device to a user device.

18. The system of Clause 10, wherein at least one computer is configured to transmit the state of a first device and the state of a second device to a user device.

19. One or more non-transitory computer-readable media, comprising computer-executable instructions that, when executed, cause at least one processor to perform actions including: obtaining a power monitoring signal by measuring the electrical properties of power transmission lines in a building, wherein the power transmission lines provide power to devices in the building; determining the state of a first device by: (1) processing a first power event in the power monitoring signal using multiple power models to generate a score for each of the multiple power models, and (2) selecting the state of the first device from multiple possible states using the scores; updating the state of the first device in a device list using the selected state; receiving a first broadcast network data packet, wherein the first broadcast network data packet is transmitted by a second device; determining, using information in the first broadcast network data packet, that the first broadcast network data packet was transmitted by the second device; selecting the state of the second device from multiple possible states of the second device using information in the first network event; and updating the state of the second device in a device list using the selected state.

20. One or more non-transitory computer-readable media of Clause 19, wherein updating the state of the first device in the stored information includes changing the state from an off state to an on state.

21. One or more non-transitory computer-readable media of Clause 19, wherein the plurality of power models include one or more of a device model, a transformation model or a watt number model.

22. One or more non-transitory computer-readable media of Clause 19, wherein determining that the first broadcast network data packet was transmitted by the second device includes using a network address or identifier in the first broadcast network data packet.

23. One or more non-transitory computer-readable media of Clause 19, wherein processing a first power event in a power monitoring signal using multiple power models comprises: calculating multiple features using a portion of the power monitoring signal including the first power event; and processing the multiple features using multiple power models.

The techniques described above use a combination of power monitoring and network monitoring to provide improved identification of equipment state changes in buildings compared to techniques based solely on power monitoring. With power monitoring alone, some equipment state changes may not be identified at all, or may be identified with lower accuracy. The combination of network and power monitoring allows for the identification of a greater number of equipment state changes (such as state changes of networked devices without mechanical parts), and these changes can be identified with higher accuracy. Using information from both power and network monitoring allows for the creation of models and/or classifiers with lower error rates than models and/or classifiers using only information from power monitoring.

Report power usage

In some implementations, a power monitor can provide a user with information about the power consumption of individual devices in a home. For example, for any device that uses power monitoring and/or network monitoring to determine changes in device status, the techniques described in U.S. Patent 9,443,195 can be used to determine the device's power consumption over time and/or present the user with real-time information about the device's power consumption.

Some devices in a house may have relatively constant power usage or power usage falling within a range for each state of the device. For example, a television and a light bulb may consume a relatively fixed amount of power when they are on and no power when they are off. For such devices, the expected power consumption or power range can be determined by measuring the power consumption of several examples of the device in each state and calculating the average power consumption for that state. A list of expected power consumption or power ranges for the device can be created for each state, and this list can be available to a power monitor.

When a device with relatively constant power usage is determined to be in a special state (e.g., by processing network events), a power monitor (or a server operating in conjunction with a power monitor) can obtain the expected power usage or power range of the device state (e.g., from its own storage or a third-party server) and report to the user that the device is consuming the expected amount of power or within the expected power range. When the device is turned off, the power monitor can report to the user that the device has stopped consuming power.

Some devices in a home may be "always on," such as cable modems, wireless routers, smart thermostats, and IoT devices. For devices that are always on, the expected power usage or power range can be determined as described above. Once a device that is always on is identified as being in the home, a power monitor can report to the user that the device is always on and provide the expected power usage or power range.

The power monitor (or a server operating in conjunction with a power monitor) can also calculate the total amount of energy consumed by a device with relatively constant power usage or a device that is always on. For these devices, the total amount of energy consumed by the device can be calculated by (1) determining the amount of time the device has been in each of its possible states, (2) determining the expected power usage in each state, (3) determining the total amount of energy consumed in each state by multiplying the amount of time in the state by the expected power consumption of the state, and (4) determining the total amount of energy consumed by the device by adding the energy consumption of each state.

Training a power model using network events

Training power models for some devices can be more challenging than training them for others. Devices with mechanical components such as pumps and motors may have more easily identifiable power ratings due to the relationship between the operation of those mechanical components and power usage. Devices with few or no mechanical components may have more difficult-to-identify power ratings. For example, many electronic devices, such as computers and televisions, convert alternating current (AC) to direct current (DC) and then use that DC to power their internal electronics.

The power model can be trained using a large dataset of training data as described in U.S. Patent 9,443,195. For example, dozens, hundreds, or thousands of examples of power events in the power signal corresponding to a television being turned on can be used to train a power model for television turn-on. Because the power events corresponding to television turn-on can be similar to those for other electronic devices, collecting sufficient training data for all electronic devices in a house can be challenging.

Network events can be used to help collect training data for training power models of devices connected to both the power line and the network. Suppose we want to train a power model for when a first device (e.g., a television) is turned on. Instances of the first device being turned on can be identified by processing network events using any of the techniques described above. A portion of the power monitoring signal corresponding to the first device being turned on is then stored so that it can be used later for training purposes. This portion of the power monitoring signal may include parts that occur before and/or after the network event. By performing these steps over extended time periods and/or for multiple households, a large training dataset of training data can be collected for training the power model for when the first device is turned on.

Any part of the power monitoring signal may include multiple power events that are close to or even overlap with each other. Therefore, in addition to the power event for the first device to turn on, the collected training data may also include power events for changes in the state of other devices. Power events for other devices may be referred to as external power events.

Different methods can be used to handle extraneous power events in the training data. In some implementations, no special action can be taken, and the power model can be trained using all the data as is. Although the training data will include extraneous power events, if their number is small compared to the power events corresponding to the first device being turned on, they may not significantly affect model training. In some implementations, training data with more than one power event can be discarded, such that only the portion of the power monitoring signal containing a single power event is used to train the power model for the first device. In some implementations, extraneous power events can be identified, such that only power events that are likely to correspond to the first device being turned on are used to train the power model. For example, features can be computed for all power events in the training data, and clustering techniques (e.g., k-means) can be used to identify extraneous power events. In cases where a portion of the power monitoring signal contains both the power event of the first device being turned on at a first time and the extraneous power event at a second time, only the portion of the power monitoring signal including the power event at the first time can be used to train the power event for the first device being turned on.

The power model for turning on the first device can then be trained using the training data. Any suitable technique can be used to train the power model, such as any technique described in U.S. Patent 9,443,195. For example, the power model can include any of a transition model, a device model, or a watt-number model. Training techniques can include, but are not limited to, training neural networks, self-organizing maps, support vector machines, decision trees, random forests, logistic regression, Bayesian models, linear and nonlinear regressions, and Gaussian mixture models.

The power model can also be trained similarly for other state changes of the first device and other devices.

Figure 11 is a flowchart illustrating an example implementation of using network data to train a power model for a device. In Figure 11, the order of steps is exemplary, and other orders are possible. Not all steps are necessary, and in some implementations, some steps may be omitted or additional steps may be added. The flowchart process can be implemented, for example, by any computer or system described herein.

In step 1110, information about a first network event in the building is received. A network event can correspond to any data transmission, such as a network data packet transmitted from a device in the building to a power monitor in the building. In some implementations, the server can receive information about the network event from the power monitor, where the power monitor receives network data transmission from another device in the building. The information about the network event can include information about the transmitted network data or information about the transmitted network data, such as the time of reception. The network event can be associated with a time, such as the time when network data is received by the power monitor.

In step 1120, information about network events is used to determine that the first device has changed state. Any techniques described herein can be used to determine the state change of the first device using network events. The determination can be performed by a power monitor or by a server operating in conjunction with a power monitor. In some implementations, other steps in the process may depend on determining the state change of the first device. For example, the power monitor may process information about network events to determine the state change of the first device and transmit information about network events and/or power monitoring signals only after the state change of the first device has been determined (in step 1130).

In step 1130, a power monitoring signal is received, wherein the received power monitoring signal is a portion of the power monitoring signal used by the power monitor. For example, the server may receive the power monitoring signal from the power monitor. The received power monitoring signal may include measurements taken at a time corresponding to a network event (e.g., the time of transmission or reception).

Steps 1110, 1120, and 1130 can be repeated any number of times, and the corresponding network events and power monitoring signals can originate from the same building or from multiple buildings. The collected power monitoring signals may correspond to devices sharing common characteristics. For example, the power monitoring signals may all originate from televisions, all from Toshiba televisions, or from a specific version of a Toshiba television. Because the power monitoring signals share common characteristics, they can be used to train a power model to detect that common characteristic. In step 1140, it can be determined whether additional data is available for training the power model or is expected to be used for training the power model. If more data is available or expected, the process can proceed to step 1110. If more data is unavailable or not expected, the process can proceed to step 1150.

In step 1150, the power monitoring signals received in one or more instances of step 1130 are used to train the power model. The power model can be any power model described herein or in U.S. Patent 9,443,195, and can be trained using any techniques described herein or in U.S. Patent 9,443,195. In some embodiments, the power monitoring signals may be processed to identify and remove or ignore external power events corresponding to devices other than the first device, as described herein.

At step 1160, the trained power model can be deployed to power monitors in the building. For example, the power model can be used by the power monitors to identify devices in the building, or to identify changes in device state in the building using any of the techniques described herein.

In some implementations, the power model may be trained as described in the following clauses, any two or more of them, or in combination with other clauses presented herein.

1. A system for training a power model for recognizing device or device state changes, the system comprising at least one computer including at least one processor and at least one memory, the at least one computer being configured to: receive information about a first network event, wherein the first network event corresponds to a first device in a first building transmitting data at a first time; use the information about the first network event to determine a state change in the first device; obtain a first power monitoring signal, wherein the first power monitoring signal corresponds to a measurement of an electrical property of a transmission line in the first building, and wherein the first power monitoring signal includes the measurement result at the first time; receive information about a second network event, wherein the second network event corresponds to a second device in a second building transmitting data at a second time; use the information about the second network event to determine a state change in the second device; obtain a second power monitoring signal, wherein the second power monitoring signal corresponds to a measurement of an electrical property of a transmission line in the second building, and wherein the second power monitoring signal includes the measurement result at the second time; train a power model using the first power monitoring signal and the second power monitoring signal, wherein the power model is configured to use the power monitoring signal to recognize device or device state changes; and transmit the power model to a power monitor in a third building.

2. The system of Clause 1, wherein the first device and the second device are devices of the same type, of the same manufacturer, or of the same version.

3. The system of Clause 1, wherein at least one computer comprises a first power monitor in a first building, a second power monitor in a second building, and a server computer.

4. The system of Clause 1, wherein at least one computer is configured to acquire a third power monitoring signal, wherein the third power monitoring signal corresponds to a measurement of the electrical properties of a power transmission line in a third building; and to process the third power monitoring signal using a power model to determine a change in the state of a third device.

5. The system of Clause 1, wherein at least one computer is configured to transmit a request for information to a second device; and wherein network data is received in response to the request for information.

6. The system of Clause 1, wherein at least one computer is configured to receive information about a plurality of network events, wherein each network event corresponds to the transmission of data by a first device; and to use the information about the plurality of network events to determine a change in the state of the first device.

7. The system of Clause 1, wherein at least one computer is configured to determine a change in state of a first device by (1) processing information about a first network event using multiple network models and (2) selecting the highest-scoring network model.

8. A method for training a power model for recognizing a device or a change in device state, the method comprising: receiving information about a first network event, wherein the first network event corresponds to a first device transmitting data at a first time; obtaining a first power monitoring signal, wherein the first power monitoring signal corresponds to a measurement of an electrical property of a transmission line, and wherein the first power monitoring signal includes the measurement at the first time; receiving information about a second network event, wherein the second network event corresponds to the first device transmitting data at a second time; obtaining a second power monitoring signal, wherein the second power monitoring signal corresponds to a measurement of an electrical property of a transmission line, and wherein the second power monitoring signal includes the measurement at the second time; training a power model using the first power monitoring signal and the second power monitoring signal, wherein the power model is configured to use the power monitoring signal to recognize the first device or a change in the state of the first device.

9. The method of Clause 8 includes: using information about a first network event to determine a change in the state of a first device; and using information about a second network event to determine a change in the state of a second device.

10. The method of Clause 8 includes transmitting a power model to a power monitor in a building.

11. The method of Clause 8, wherein the first network event includes broadcasting network packets.

12. The method of Clause 8, wherein the information about the first network event includes the time of the first network event.

13. The method of Clause 8 includes: determining that a first power monitoring signal includes a first power event and a second power event; determining that the first power event corresponds to a change in state of a first device; and training a power model using the first power event.

14. The method of Clause 13, wherein determining that a first power event corresponds to a first device comprises: calculating a first feature of the first power event and a second feature of the second power event; and processing the first feature and the second feature using a mathematical model, or clustering the first feature and the second feature.

15. The method of Clause 13, wherein the first device is in a first building, and the method includes: receiving information about a third network event, wherein the third network event corresponds to a second device in a second building transmitting data at a third time; obtaining a third power monitoring signal, wherein the third power monitoring signal corresponds to a measurement of the electrical properties of a power transmission line in the second building, and wherein the second power monitoring signal includes the measurement at the third time; and training a power model using the third power monitoring signal.

16. One or more non-transitory computer-readable media, comprising computer-executable instructions that, when executed, cause at least one processor to perform actions including: receiving information about a first network event, wherein the first network event corresponds to a first device transmitting data at a first time; obtaining a first power monitoring signal, wherein the first power monitoring signal corresponds to a measurement of an electrical property of a transmission line, and wherein the first power monitoring signal includes the measurement result at the first time; receiving information about a second network event, wherein the second network event corresponds to a first device transmitting data at a second time; obtaining a second power monitoring signal, wherein the second power monitoring signal corresponds to a measurement of an electrical property of a transmission line, and wherein the second power monitoring signal includes the measurement result at the second time; training a power model using the first power monitoring signal and the second power monitoring signal, wherein the power model is configured to use the power monitoring signal to identify the first device or a change in the state of the first device.

17. One or more non-transitory computer-readable media of Clause 16, wherein a power monitoring signal indicates the current or voltage of a transmission line.

18. One or more non-transitory computer-readable media of Clause 16, wherein the power model includes a transformation model, a device model, or a watt number model.

19. One or more non-transitory computer-readable media of Clause 16, wherein the first network event includes broadcast network packets.

20. One or more non-transitory computer-readable media of Clause 16, including power monitors that transmit power models to buildings.

The techniques described above allow for the creation of power models for devices in which sufficient training data might otherwise be difficult to obtain. For devices with few or no mechanical parts (e.g., televisions and computers), there may not be significant variability in the device's power signature. To obtain an example power signature for such a device, one could manually change the device's state and store the power monitoring signal within a window around the manual state change; however, this process does not scale to obtain a large amount of training data or data for a large number of devices. By applying the techniques described herein, training data for the device can be obtained through network monitoring. Because this process may not require human intervention, it can therefore scale to obtain a large amount of training data for a large number of devices and/or a large number of devices. By using network monitoring to obtain training data, power models can be created more quickly (because training data is available more rapidly), and the power models can be more accurate (because more training data is available).

Implementation

Figure 12 illustrates components of some embodiments of a computing device 1200 that can be used in conjunction with the operation of any power monitor or server. In Figure 12, the components are shown as being on a single computing device, but the components may be distributed among multiple computing devices, such as in any of the devices described above or among several server computing devices.

Computing device 1200 may include any typical components of a computing device, such as one or more processors 1211, volatile or non-volatile memory 1210, and one or more network interfaces 1212 for connecting to a computer network. Computing device 1200 may also include any input and output components, such as a display, keyboard, and touchscreen. Computing device 1200 may also include various components or modules that provide specific functionality, and these components or modules may be implemented in software, hardware, or a combination thereof. Hereinafter, several instances of components are described with respect to one example embodiment, and other embodiments may include additional components or exclude some of the components described below.

The computing device 1200 may include a power event processing unit 1220, which can be used to process power monitoring signals and determine information about devices from the power monitoring signals, such as identifying devices and device status changes. The computing device 1200 may include a network event processing unit 1221, which can be used to process data from a computer network and determine information about devices from the network data, such as identifying devices and device status changes. The computing device 1200 may include a device identification unit, which can be used to identify information about devices (such as the type of device or the manufacture of the device) using information obtained from power monitoring and/or network monitoring. The computing device 1200 may include a device status change unit, which can be used to identify device status changes using information obtained from power monitoring and/or network monitoring. The computing device 1200 may have a device update unit 1224, which can be used to update the device list after a device has been identified or a device status change has been identified.

Computing device 1200 may include or have access to various data storage methods, such as data storage methods 1230, 1231, and 1232. Data storage may use any known storage technology, such as files or related or unrelated databases. For example, computing device 1200 may have a power model data storage method 1230 to store power models or information about power models. Computing device 1200 may have a network model data storage method 1231 to store network models or information about network models. Computing device 1200 may have a device information data storage method 1232, which can be used to store information about devices, such as a list of devices used in one or more buildings.

While only some embodiments of the invention have been shown and described, it will be apparent to those skilled in the art that many changes and modifications can be made thereto without departing from the spirit and scope of this disclosure as set forth in the appended claims. All patent applications and patents (both foreign and domestic) cited herein, as well as all other publications, are incorporated herein in their entirety to the full extent permitted by law.

The methods and systems described herein can be deployed, in part or in whole, by a machine that executes computer software, program code, and/or instructions on a processor. The processor can be part of a server, cloud server, client, network infrastructure, mobile computing platform, fixed computing platform, or other computing platform. The processor can be any type of computing or processing device capable of executing program instructions, code, binary instructions, etc. The processor can be or can include a signal processor, digital processor, embedded processor, microprocessor, or any variant such as a coprocessor (mathematical coprocessor, graphics coprocessor, communication coprocessor, etc.) that can directly or indirectly facilitate the execution of program code or program instructions stored thereon. Furthermore, the processor can implement the execution of multiple programs, threads, and code. Threads can be executed concurrently to enhance processor performance and facilitate simultaneous operation of applications. By implementation, the methods, program code, program instructions, etc., described herein can be implemented in one or more threads. Threads can spawn other threads that may have assigned and associated priorities; the processor can execute these threads based on priority or any other order of instructions provided in the program code. The processor may include memory storing the methods, code, instructions, and programs as described herein and elsewhere. A processor can access a storage medium through an interface that can store methods, code, and instructions as described herein and elsewhere. The storage medium associated with a processor for storing methods, programs, code, program instructions, or other types of instructions executable by a computing or processing device may include, but is not limited to, one or more of CD-ROMs, DVDs, memory, hard disks, flash drives, RAM, ROM, caches, etc.

The processor may include one or more cores that can enhance the speed and performance of the multiprocessor. In embodiments, the process may be a dual-core processor, a quad-core processor, or other chip-level multiprocessors that combine two or more independent cores (referred to as dies).

The methods and systems described herein can be deployed, in part or in whole, on a machine that executes computer software on a server, cloud server, client, firewall, gateway, hub, router, or other computer and/or networking hardware. The software program can be associated with a server, which may include a file server, print server, domain server, internet server, intranet server, and other variations such as a secondary server, host server, distributed server, etc. The server may include one or more of a memory, processor, computer-readable medium, storage medium, port (physical and virtual), communication device, and interface capable of accessing other servers, clients, machines, and devices via wired or wireless media. The methods, programs, or code described herein and elsewhere can be executed by the server. Additionally, other devices required for executing the methods as described in this application can be considered part of the infrastructure associated with the server.

The server can provide interfaces to other devices, including but not limited to clients, other servers, printers, database servers, print servers, file servers, communication servers, distributed servers, etc. Additionally, this coupling and/or connection can facilitate remote execution of programs across a network. Without departing from the scope of this disclosure, networking some or all of these devices can facilitate parallel processing of programs or methods at one or more locations. Furthermore, any device attached to the server via the interface can include at least one storage medium capable of storing methods, programs, code, and/or instructions. A central repository can provide program instructions to be executed on different devices. In this embodiment, a remote repository can act as a storage medium for program code, instructions, and programs.

The software program may be associated with a client, which may include file clients, print clients, domain clients, Internet clients, intranet clients, and other variations such as auxiliary clients, host clients, distributed clients, etc. The client may include one or more of the following: memory, processor, computer-readable medium, storage medium, port (physical and virtual), communication device, and interface capable of accessing other clients, servers, machines, and devices via wired or wireless media. The methods, programs, or code described herein and elsewhere may be executed by the client. Additionally, other devices required to execute the methods as described in this application may be considered part of the infrastructure associated with the client.

The client can provide interfaces to other devices, including but not limited to servers, other clients, printers, database servers, print servers, file servers, communication servers, distributed servers, etc. Additionally, this coupling and/or connection can facilitate remote execution of programs across a network. Without departing from the scope of this disclosure, networking some or all of these devices can facilitate parallel processing of programs or methods at one or more locations. Furthermore, any device attached to the client via an interface can include at least one storage medium capable of storing methods, programs, applications, code, and/or instructions. A central repository can provide program instructions to be executed on different devices. In this embodiment, a remote repository can act as a storage medium for program code, instructions, and programs.

The methods and systems described herein can be deployed, in part or in whole, through a network infrastructure. The network infrastructure may include elements such as computing devices, servers, routers, hubs, firewalls, clients, personal computers, communication equipment, routing devices, and other active and passive devices, modules, and/or components known in the art. One or more computing and/or non-computing devices associated with the network infrastructure may, among other components, include storage media such as flash memory, buffers, stacks, RAM, ROM, etc. The processes, methods, program code, and instructions described herein and elsewhere may be executed by one or more of the network infrastructure elements.

The methods, program code, and instructions described herein and elsewhere can be implemented on a cellular network with multiple cells. The cellular network can be a Frequency Division Multiple Access (FDMA) network or a Code Division Multiple Access (CDMA) network. The cellular network can include mobile devices, cell sites, base stations, repeaters, antennas, towers, etc. The cell network can be GSM, GPRS, 3G, EVDO, mesh, or other network types.

The methods, program code, and instructions described herein and elsewhere can be implemented on or through mobile devices. Mobile devices may include navigation devices, cellular phones, mobile phones, mobile personal digital assistants, laptops, handheld computers, netbooks, pagers, e-book readers, music players, etc. Among other components, these devices may include storage media such as flash memory, buffers, RAM, ROM, and one or more computing devices. The computing device associated with the mobile device can be enabled to execute the program code, methods, and instructions stored thereon. Alternatively, the mobile device may be configured to execute instructions in cooperation with other devices. The mobile device may communicate with a base station coupled to a server and configured to execute program code. The mobile device may communicate on end-to-end networks, mesh networks, or other communication networks. The program code may be stored on a storage medium associated with the server and executed by a computing device embedded within the server. The base station may include computing devices and storage media. The storage device may store program code and instructions executed by the computing device associated with the base station.

Computer software, program code, and/or instructions can be stored and/or accessed on machine-readable media, which may include: computer components, devices, and recording media that hold digital data for calculating a time interval; semiconductor storage known as random access memory (RAM); mass storage typically used for more persistent storage, such as optical discs, hard disks, tapes, magnetic drums, cards, and other types of magnetic storage; processor registers, cache memory, volatile memory, and non-volatile memory; optical storage, such as CDs and DVDs; removable media, such as flash memory (e.g., USB sticks or keys), floppy disks, magnetic tapes, paper tapes, punch cards, stand-alone RAM disks, Zip drives, removable mass storage, offline storage, etc.; and other computer storage, such as dynamic memory, static memory, read/write storage, variable storage, read-only, random access, sequential access, location-addressable, file-addressable, content-addressable, network-attached storage, storage area networks, barcodes, magnetic ink, etc.

The methods and systems described herein can transform physical and/or intangible items from one state to another. The methods and systems described herein can also transform data representing physical and/or intangible items from one state to another, such as transforming used data into a normalized used dataset.

The elements described and depicted herein (including those in the flowcharts and block diagrams throughout the accompanying drawings) imply logical boundaries between elements. However, in accordance with software or hardware engineering practice, the depicted elements and their functions may be implemented on a machine by a computer-executable medium having a processor capable of executing program instructions stored thereon, as a monolithic software architecture, as a standalone software module, or as a module employing external routines, code, services, etc., or any combination thereof, and all such implementations are within the scope of this disclosure. Examples of such machines may include, but are not limited to, personal digital assistants, laptop computers, personal computers, mobile phones, other handheld computing devices, medical devices, wired or wireless communication devices, transducers, chips, calculators, satellites, tablet PCs, e-books, gadgets, electronic devices, devices with artificial intelligence, computing devices, network equipment, servers, routers, etc. Furthermore, the elements or any other logical components depicted in the flowcharts and block diagrams may be implemented on a machine capable of executing program instructions. Therefore, while the foregoing figures and descriptions illustrate functional aspects of the disclosed systems, specific arrangements of the software used to implement these functional aspects should not be inferred from these descriptions unless expressly stated or otherwise understood from the context. Similarly, it will be understood that the various steps identified and described above are subject to variation, and the order of the steps may be adapted to the specific application of the techniques disclosed herein. All such variations and modifications are intended to fall within the scope of this disclosure. Therefore, the depiction and/or description of the order of the various steps should not be construed as requiring a particular order of execution of those steps, unless required by a particular application, or expressly stated or otherwise clear from the context.

The methods and/or processes and their steps described above may be implemented in hardware, software, or any combination of hardware and software suitable for a particular application. Hardware may include general-purpose computers and/or special-purpose computing devices or specific aspects or components of a particular computing device. Processes may be implemented in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors, or other programmable devices together with internal and/or external memory. Processes may also be embodied, or alternatively, in application-specific integrated circuits, programmable gate arrays, programmable array logic, or any other device or combination of devices that may be configured to process electronic signals. It should also be understood that one or more processes may be implemented as computer-executable code capable of executing on a machine-readable medium.

Computer executable code can be created using structured programming languages such as C, object-oriented programming languages such as C++, or any other high- or low-level programming languages (including assembly languages, hardware description languages, and database programming languages and techniques) that can be stored, compiled, or interpreted to run on one of the aforementioned devices and processors, heterogeneous combinations of processor architectures, or combinations of different hardware and software, or any other machine capable of executing program instructions.

Therefore, in one aspect, each method and combination thereof described above may be embodied in computer-executable code, which performs its steps when executed on one or more computing devices. In another aspect, the method may be embodied in a system that performs its steps and may be distributed across devices in various ways, or all functionality may be integrated into a dedicated, stand-alone device or other hardware. In yet another aspect, means for performing the steps associated with the processes described above may include any of the hardware and/or software described above. All such substitutions and combinations are intended to fall within the scope of this disclosure.

All documents cited in this article are hereby incorporated by reference.

Claims (62)

1. A system for determining the status of equipment in a building, the system comprising: At least one computer, comprising at least one processor and at least one memory, said at least one computer is configured to: A power monitoring signal is obtained by measuring the electrical properties of power transmission lines in the building, wherein the power transmission lines supply power to the equipment in the building, and the equipment in the building includes a first device and a second device. The state of the first device is determined by the following steps: (1) processing a first power event in the power monitoring signal using multiple power models to generate a score for each of the multiple power models, and (2) using the score to select the state of the first device from multiple possible states; Receive a first network data packet from a server computer, wherein the server computer transmits the first network data packet based on information received from the second device; Use the first network packet to determine that the first network packet corresponds to the second device; and The first network data packet is used to select the state of the second device from a plurality of possible states of the second device. 2. The system of claim 1, wherein the power monitoring signal indicates the current or voltage of the transmission line. 3. The system according to claim 1, wherein the first network data packet is a broadcast network data packet. 4. The system of claim 1, wherein the first network data packet includes a broadcast using Simple Service Discovery Protocol, Zero-Configuration Networking, or NetBIOS. 5. The system of claim 1, wherein the first network data packet is received in response to a request for information transmitted to the second device. 6. The system of claim 1, wherein the first network data packet is received in response to polling the second device, requesting information using the API of the second device, or requesting information about services provided by the second device. 7. The system of claim 1, wherein the state of the second device is selected by the following steps: (1) processing information in the first network data packet using multiple network models, and (2) selecting the highest-scoring network model, wherein the highest-scoring network model corresponds to the selected state. 8. The system of claim 1, wherein the state of the second device is selected by processing information in the first network data packet using a plurality of rules, wherein each of the plurality of rules includes comparing information in the first network data packet with at least one condition. 9. The system of claim 1, wherein the state of the second device is selected by the following steps: The second power event in the power monitoring signal is processed using multiple power models to generate a second score for each of the multiple power models; Information about the first network data packet is processed using multiple network models to generate a third score for each of the multiple network models; and The state is selected using the second score and the third score. 10. The system of claim 1, wherein the state of the second device is selected by generating a second score for each of a plurality of state change models, wherein each state change model processes a second power event in the power monitoring signal and information in the first network data packet; and the state is selected using the second score. 11. The system of claim 1, wherein the state of the second device is selected by the following steps: Receive multiple network data packets, wherein each of the multiple network data packets is transmitted by the second device; and The state of the second device is selected using information from the plurality of network packets. 12. A computer-implemented method for determining the state of equipment in a building, the method comprising: A power monitoring signal is obtained by measuring the electrical properties of power transmission lines in the building, wherein the power transmission lines supply power to the equipment in the building, and the equipment in the building includes a first device and a second device. The state of the first device is determined by the following steps: (1) processing a first power event in the power monitoring signal using multiple power models to generate a score for each of the multiple power models, and (2) using the score to select the state of the first device from multiple possible states; Receive a first network data packet, wherein the first network data packet is transmitted by the second device; Use the information in the first network data packet to determine that the first network data packet was sent by the second device; and The state of the second device is selected from multiple possible states of the second device using the information in the first network data packet. 13. The computer-implemented method of claim 12, wherein the first network data packet includes a broadcast using a simple service discovery protocol, zero-configuration networking, or NetBIOS. 14. The computer-implemented method according to claim 12, the method comprising: Transmit a request for information to the second device; Receiving the first network data packet includes receiving a response to the request for information. 15. The computer-implemented method of claim 14, wherein transmitting the request for information includes polling the second device, requesting information using the API of the second device, or requesting information about services provided by the second device. 16. The computer-implemented method of claim 12, wherein selecting the state of the second device comprises: (1) processing information in the first network data packet using multiple network models, and (2) selecting the highest-scoring network model, wherein the highest-scoring network model corresponds to the selected state. 17. The computer-implemented method of claim 12, wherein selecting the state of the second device includes processing information in the first network data packet using a plurality of rules, wherein each of the plurality of rules includes comparing the information in the first network data packet with at least one condition. 18. The computer-implemented method of claim 12, wherein selecting the state of the second device includes: The second power event in the power monitoring signal is processed using multiple power models to generate a second score for each of the multiple power models; and Information about the first network data packet is processed using multiple network models to generate a third score for each of the multiple network models; and The state is selected using the second score and the third score. 19. The computer-implemented method of claim 12, wherein selecting the state of the second device includes: A second score is generated for each of the multiple state-change models, wherein each state-change model processes a second power event in the power monitoring signal and information from the first network data packet; and The second score is used to select the state. 20. The computer-implemented method according to claim 12, comprising: Receive multiple broadcast network data packets, wherein each of the multiple broadcast network data packets is transmitted by the second device; and The state of the second device is selected using information from the plurality of broadcast network packets. 21. An apparatus for determining the state of equipment in a building, the apparatus comprising: At least one computer, comprising at least one processor and at least one memory, said at least one computer is configured to: A power monitoring signal is obtained by measuring the electrical properties of power transmission lines in the building, wherein the power transmission lines supply power to the equipment in the building, and the equipment in the building includes a first device and a second device. The state of the first device is determined by the following steps: (1) processing a first power event in the power monitoring signal using multiple power models to generate a score for each of the multiple power models, and (2) using the score to select the state of the first device from multiple possible states; Receive a first network data packet from a server computer, wherein the server computer transmits the first network data packet based on information received from the second device; Use the first network packet to determine that the first network packet corresponds to the second device; and The first network data packet is used to select the state of the second device from a plurality of possible states of the second device. 22. The device of claim 21, wherein the first network data packet includes a broadcast using a simple service discovery protocol, zero-configuration networking, or NetBIOS. 23. The apparatus of claim 21, wherein the at least one computer is further configured to: Transmit a request for information to the second device; Receiving the first network data packet includes receiving a response to the request for information. 24. The device of claim 23, wherein the at least one computer transmits the request for information by polling the second device, requesting information using the API of the second device, or requesting information about services provided by the second device. 25. The device of claim 21, wherein the at least one computer selects the state of the second device by means of the following steps: (1) processing information in the first network data packet using a plurality of network models, and (2) selecting the highest-scoring network model, wherein the highest-scoring network model corresponds to the selected state. 26. The device of claim 21, wherein selecting the state of the second device includes processing information in the first network data packet using a plurality of rules, wherein each of the plurality of rules includes comparing information in the first network data packet with at least one condition. 27. The device of claim 21, wherein selecting the state of the second device includes: The second power event in the power monitoring signal is processed using multiple power models to generate a second score for each of the multiple power models; Information about the first network data packet is processed using multiple network models to generate a third score for each of the multiple network models; and The state is selected using the second score and the third score. 28. The device of claim 21, wherein selecting the state of the second device includes: A second score is generated for each of the multiple state-change models, wherein each state-change model processes a second power event in the power monitoring signal and information from the first network data packet; and The second score is used to select the state. 29. The device according to claim 21, comprising: Receive multiple broadcast network data packets, wherein each of the multiple broadcast network data packets is transmitted by the second device; and The state of the second device is selected using information from the plurality of broadcast network packets. 30. A system for identifying equipment in a building, the system comprising: At least one computer, comprising at least one processor and at least one memory, said at least one computer is configured to: A power monitoring signal is obtained by measuring the electrical properties of power transmission lines in the building, wherein the power transmission lines supply power to the equipment in the building; A first power event in the power monitoring signal is processed using multiple power models to generate a score for each of the multiple power models, wherein the first power event corresponds to a state change of a first device; The score is used to determine first information about the first device; The first device is updated in the device list using the first information about the first device; Receive a first network data packet, wherein the first network data packet is sent by a second device; Use the information in the first network data packet to determine second information about the second device; and The second device is updated in the device list using the second information about the second device. 31. The system of claim 30, wherein the first information regarding the first device includes the type of the first device, the manufacture of the first device, or the version of the first device. 32. The system of claim 30, wherein the at least one computer is configured to: Receive a second network data packet, wherein the second network data packet is sent by the first device; and The first information about the first device is determined using multiple scores and information from the second network data packet. 33. The system of claim 32, wherein the plurality of scores are used to determine the type of the first device, and the information in the second network data packet is used to determine the manufacture of the first device. 34. The system of claim 32, wherein the first power event in the power monitoring signal occurs before the second network data packet is received. 35. The system of claim 32, wherein the at least one computer is configured to select the second network packet by comparing the time of the first power event with the time of the second network packet. 36. The system of claim 30, wherein the information in the first network data packet includes an identifier of the second device, and wherein the at least one computer is configured to use the identifier of the second device to obtain the second information about the second device from a data storage. 37. The system of claim 30, wherein the at least one computer is configured to determine second information about the second device by processing information in the first network packet using a plurality of rules, wherein each of the plurality of rules includes comparing the information in the first network packet with at least one condition. 38. The system of claim 30, wherein the at least one computer is configured to: Receive multiple network data packets, wherein each of the multiple network data packets is sent by the first device; and The first information about the first device is determined using multiple scores and information from the multiple network packets. 39. The system of claim 30, wherein the at least one computer is configured to: By processing information in the first network data packet using multiple network models to generate a second score for each of the multiple network models, the second information about the second device is determined; and The second score of the plurality of network models is used to determine the second information about the second device. 40. The system of claim 30, wherein the at least one computer is configured to: The second information about the second device is determined by generating a second score for each of a plurality of identification models, wherein each identification model processes a second power event in the power monitoring signal and the information in the first network data packet; and The second score of the plurality of recognition models is used to determine the second information about the second device. 41. The system of claim 30, wherein the at least one computer is configured to: Receive multiple network data packets, wherein each of the multiple network data packets is sent by the second device; and The information in the plurality of network packets is used to determine the second information about the second device. 42. The system of claim 41, wherein the at least one computer is configured to determine the second information by using a first time corresponding to a first network data packet and a second time corresponding to a second network data packet. 43. A computer-implemented method for identifying equipment in a building, the method comprising: A power monitoring signal is obtained by measuring the electrical properties of power transmission lines in the building, wherein the power transmission lines supply power to the equipment in the building; A first power event in the power monitoring signal is processed using multiple power models to generate a score for each of the multiple power models, wherein the first power event corresponds to a state change of a first device; The score is used to determine first information about the first device; The first device is updated in the device list using the first information about the first device; Receive a first network data packet, wherein the first network data packet is sent by a second device; Using the information in the first network data packet to determine second information about the second device; and The second device is updated in the device list using the second information about the second device. 44. The computer-implemented method of claim 43, wherein the power monitoring signal indicates the current or voltage of the transmission line. 45. The computer-implemented method of claim 43, comprising receiving a plurality of broadcast network packets, each of the plurality of broadcast network packets being transmitted by the first device; and using a plurality of scores and information in the plurality of broadcast network packets to determine the first information about the first device. 46. The computer-implemented method of claim 43, wherein determining the second information about the second device comprises processing information in the first network data packet using a plurality of network models to generate a second score for each of the plurality of network models; and using the second scores of the plurality of network models to determine the second information about the second device. 47. The computer-implemented method of claim 43, wherein determining the second information about the second device comprises generating a second score for each of a plurality of identification models, wherein each identification model processes a second power event in the power monitoring signal and information in the first network data packet; and using the second scores of the plurality of identification models to determine the second information about the second device. 48. The computer-implemented method of claim 43, comprising receiving a plurality of broadcast network packets, wherein each of the plurality of broadcast network packets is transmitted by the second device; and using information in the plurality of broadcast network packets to determine the second information about the second device. 49. The computer-implemented method of claim 48, wherein determining the second information includes using a first time corresponding to a first broadcast network data packet and a second time corresponding to a second broadcast network data packet. 50. An apparatus for identifying equipment in a building, the apparatus comprising: At least one computer, comprising at least one processor and at least one memory, said at least one computer is configured to: A power monitoring signal is obtained by measuring the electrical properties of power transmission lines in the building, wherein the power transmission lines supply power to the equipment in the building; A first power event in the power monitoring signal is processed using multiple power models to generate a score for each of the multiple power models, wherein the first power event corresponds to a state change of a first device; The score is used to determine first information about the first device; The first device is updated in the device list using the first information about the first device; Receive a first network data packet, wherein the first network data packet is sent by a second device; Using the information in the first network data packet to determine second information about the second device; and The second device is updated in the device list using the second information about the second device. 51. The device of claim 50, wherein the first information regarding the first device includes the type of the first device, the manufacture of the first device, or the version of the first device. 52. The apparatus of claim 50, wherein the at least one computer is configured to: Receive a second network data packet, wherein the second network data packet is sent by the first device; and The score and information from the second network data packet are used to determine the first information about the first device. 53. The device of claim 52, wherein the score is used to determine the type of the first device, and the information in the second network data packet is used to determine the manufacture of the first device. 54. The device of claim 52, wherein the first power event in the power monitoring signal occurs before the second network data packet is received. 55. The device of claim 52, wherein the at least one computer is configured to select the second network packet by comparing the time of the first power event and the time of the second network packet. 56. The device of claim 50, wherein the information in the first network data packet includes an identifier of the second device, and wherein the at least one computer is configured to use the identifier of the second device to obtain the second information about the second device from a data storage. 57. The device of claim 50, wherein the at least one computer is configured to determine second information about the second device by processing information in the first network packet using a plurality of rules, wherein each of the plurality of rules includes comparing the information in the first network packet with at least one condition. 58. The apparatus of claim 50, wherein the at least one computer is configured to: Receive multiple network data packets, each of which is sent by the first device; and The first information about the first device is determined using the score and information from the plurality of network packets. 59. The apparatus of claim 50, wherein the at least one computer is configured to: By processing information in the first network data packet using multiple network models to generate a second score for each of the multiple network models, the second information about the second device is determined; and The second score of the plurality of network models is used to determine the second information about the second device. 60. The apparatus of claim 50, wherein the at least one computer is configured to: The second information about the second device is determined by generating a second score for each of a plurality of identification models, wherein each identification model processes a second power event in the power monitoring signal and information in the first network data packet; and The second score of the plurality of recognition models is used to determine the second information about the second device. 61. The apparatus of claim 50, wherein the at least one computer is configured to: Receive multiple network data packets, wherein each of the multiple network data packets is sent by the second device; and The information in the plurality of network packets is used to determine the second information about the second device. 62. The apparatus of claim 61, wherein the at least one computer is configured to determine the second information including using a first time corresponding to a first network data packet and a second time corresponding to a second network data packet.