HK1231998B - Systems and methods for probabilistic semantic sensing in a sensory network - Google Patents

Description

System and method for probabilistic semantic sensing in sensor networks

Related applications

This application claims priority to U.S. Patent Application No. 14/639,901, filed on March 5, 2015, and to U.S. Provisional Application No. 61/948,960, filed on March 6, 2014, which are incorporated by reference in their entirety. This application is related to U.S. Non-Provisional Patent Application No. 14/024,561, filed on September 11, 2013, entitled “Networked Lighting Infrastructure for Sensing Applications,” and U.S. Provisional Application No. 61/699,968, filed on September 12, 2012, of the same title.

Technical Field

The present invention relates to the technical field of data communications. More particularly, the present invention relates to a system and method for probabilistic semantic sensing in a sensor network.

Background Art

A sensor network includes multiple sensors that can be used to sense and identify objects. The objects being sensed may include people, vehicles, or other entities. The entities may be stationary or in motion. Sometimes, sensors may not be positioned to fully sense the entire entity. Other times, obstructions may impair sensing of the entity. In both instances, real-world impairments can lead to unreliable results.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 illustrates a system for probabilistic semantic sensing in a sensor network according to one embodiment;

FIG2 further illustrates a system for probabilistic semantic sensing in a sensor network according to an embodiment;

3 is a block diagram illustrating a system for probabilistic semantic sensing in a sensor network according to an embodiment;

4A is a block diagram illustrating sensed event information according to an embodiment.

4B is a block diagram illustrating derived event information according to an embodiment.

FIG5 is a block diagram illustrating user input information according to one embodiment;

FIG6 is a block diagram illustrating a method for probabilistic semantic sensing in a sensor network according to an embodiment;

FIG7 illustrates a portion of the overall architecture of a lighting infrastructure application framework (LIAF) according to an embodiment;

FIG8 illustrates the architecture of a system at a higher level according to one embodiment;

FIG9 is a block diagram of a node platform according to an embodiment.

FIG10 is a block diagram of a gateway platform according to an embodiment.

FIG11 is a block diagram of a service platform according to an embodiment.

FIG12 is a diagram illustrating a revenue model for a lighting infrastructure application according to an embodiment;

FIG13 illustrates a parking garage application for a networked lighting system according to one embodiment;

FIG14 illustrates a lighting maintenance application for a networked lighting system according to one embodiment;

FIG15A illustrates a warehouse inventory application for a networked lighting system according to an embodiment;

FIG15B illustrates a warehouse inventory application for a networked lighting system according to an embodiment;

FIG16 illustrates an application of a networked lighting system for monitoring a loading dock, according to an embodiment;

FIG17 is a block diagram illustrating power monitoring and control circuitry at a node according to an embodiment;

FIG18 is a block diagram illustrating an application controller at a node according to an embodiment;

FIG19 is a block diagram illustrating an example of a software architecture that may be installed on a machine in accordance with some example embodiments; and

20 is a block diagram illustrating components of a machine capable of reading instructions from a machine-readable medium (eg, a machine-readable storage medium) and performing any one or more of the methodologies discussed herein, according to some example embodiments.

The headings provided herein are for convenience only and do not necessarily affect the scope or meaning of the terms used.

For purposes of explanation, in the following description, numerous specific details are set forth in order to provide a thorough understanding of some exemplary embodiments. However, it will be apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details.

DETAILED DESCRIPTION

The following description includes systems, methods, techniques, instruction sequences, and computer program products that embody illustrative embodiments of the present invention. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide an understanding of various embodiments of the inventive subject matter. However, it will be apparent to one skilled in the art that embodiments of the inventive subject matter can be practiced without these specific details. Generally, well-known instruction examples, protocols, structures, and techniques are not necessarily presented in detail.

The present invention is directed to probabilistic semantic sensing in sensor networks. The present invention addresses the problem of accurately sensing observable phenomena in the presence of real-world obstructions or damage. The present invention addresses this problem by sensing the same underlying physical phenomenon in parallel and generating a single probability associated with the meaning or semantics of the physical phenomenon. Specifically, the present invention addresses this problem by sensing the same underlying physical phenomenon in parallel to generate semantic data describing the physical phenomenon in the form of sensed events that each contain semantic data (e.g., "parking lot is empty"), associating each semantic data with a probability quantifying the reliability of the semantic data, correlating the sensed events based on classifiers to generate a logical aggregation of semantic data (e.g., "same parking lot"), analyzing the aggregation of semantic data with a probability engine to generate a single derived event for multiple sensed events (wherein the single derived event contains a single derived probability), and implementing one or more applications that utilize the derived event. Those skilled in the art will recognize that while the present invention is primarily discussed in the context of optical sensor networks, the present invention is also directed to sensor networks capable of sensing all types of physical phenomena (e.g., visual, acoustic, tactile, etc.).

The advent of optical sensor networks, or lighting infrastructure with embedded capabilities for application platforms, sensing, networking, and processing, creates the opportunity to distribute sensors at significant scale and spatial density and enable sensing-based applications. However, the success of applications enabled by optical sensor networks can be limited by the reliability of sensor data, which can be constrained in part by the location of sensor deployment or interference caused by real-world obstructions (foliage, cars, people, other objects, etc.). Furthermore, for any given sensor, various portions of the generated data may be more or less reliable—for example, due to limited resolution, a video sensor may be more reliable at detecting the occupancy status of a parking space directly in front of the sensor than one located further away. Furthermore, the impracticality of transmitting all data collected at every node in an optical sensor network strongly suggests drawing conclusions about the data at intermediate steps before the data or data outputs are combined. In other words, not all raw data may be accessible in the same location. To the extent that multiple sensors can produce relevant data for a particular calculation relevant to an application of the optical sensing network, or to the extent that external data input can influence such calculation, it is optimal to create a system in which these multiple data sources can be optimally combined to produce the most useful calculation results and therefore the most successful application.

This disclosure describes the creation of a probabilistic system and method that optimizes the usefulness of conclusions based on data with limited reliability from a light sensor network. The system and method described involve associating each semantic datum with an associated probability representing the certainty or confidence of the datum, using parameters of the semantic data to correlate different semantic data, and using a probabilistic engine to derive events using the derived probabilities. The enhanced reliability is described in the context of lighting management and monitoring, parking management, surveillance, traffic monitoring, retail monitoring, business intelligence monitoring, asset monitoring, and environmental monitoring.

A system for implementing the described method may include an optical sensor network (LSN). The LSN may include an integrated application platform, sensors, and network capacity, as described below. An LSN associated with the present invention may be constructed in a manner such that some processing of the raw sensor data occurs locally at each node within the network. The output of this processing may be semantic data, or in other words, metadata or derived data representing key features detected during processing. The purpose of generating semantic data is to reduce the size of the data so that it can then be passed on for further analysis. An LSN associated with the present invention may also be constructed in a manner such that the semantic data is passed beyond the origin node so that the semantic data is aggregated and correlated with other semantic data. The network connectivity of the LSN may adopt a variety of topologies, but the present invention is agnostic to the particular topology (hub-and-spoke, specialized, etc.), as long as the aggregation point within the network occurs where multiple sources of semantic data are combined.

FIG1 illustrates a system 101 for performing probabilistic semantic sensing in a sensor network according to one embodiment. System 101 may include a sensor network that includes a “light A” positioned on the left and a “light B” positioned on the right. “Light A” and “light B” may each include sensing nodes that communicate with each other and other sensing nodes (not shown) that are part of the sensor network. Each of the sensor nodes contains one or more sensors that sense raw sensor data of different portions of a parking lot and, more specifically, the occupancy status of parking spots in the parking lot. For example, “light A” is illustrated as receiving raw sensor data for a portion of the parking lot and generating semantic data including semantic data for parking spot _X1 and semantic data for parking spot _X2 . Also for example, “light B” is illustrated as receiving raw sensor data for a different portion of the parking lot and generating semantic data including semantic data for parking spot _X2 and semantic data for parking spot _X3 . More specifically, "Light A" captures semantic data in the following form: parking spot _X1 has an occupancy state of "empty" with a probability of 99% (e.g., P( _X1 ) = 0.99) (the "empty" state is accurate), and parking spot _X2 has an occupancy state of "empty" with a probability of 75% (e.g., P( _X2 ) = 0.75) (the "empty" state is accurate). The lower probability for parking spot _X2 may be due to the limited visibility of parking spot _X2 sensed by "Light A" (e.g., fewer pixels). Furthermore, "Light B" captures semantic data in the following form: parking spot _X2 has an occupancy state of "empty" with a probability of 25% and parking spot _X3 has an occupancy state of "empty" with a probability of 99%, where the lower percentage for parking spot _X2 is again due to limited visibility. That is, FIG1 illustrates semantic data including probabilities that vary depending on location.

FIG2 illustrates a system 103 for probabilistic semantic sensing in a sensor network, according to one embodiment. System 103 operates in a manner similar to system 101. System 103 is illustrated to show how real-world obstructions (e.g., trees, other vehicles, etc.) limit the probability of semantic data. Specifically, “Light A” is illustrated as capturing semantic data indicating that parking spot _X2 is empty with a probability of 10%, and “Light B” is illustrated as capturing semantic data indicating that parking spot _X3 is empty with a probability of 10%. The reduced confidence for parking spot _X2 is due to the tree, which prevents the sensor at “Light A” from fully sensing parking spot _X2 , and the reduced confidence for parking spot _X3 is due to a transport vehicle, which prevents the sensor at “Light B” from fully sensing parking spot _X3 . That is, FIG2 illustrates semantic data containing probabilities that vary depending on obstructions.

Regarding the process of determining semantic data based on raw sensor data, the present invention does not claim any details of this process, except for the following exceptions: (i) associating each semantic data with a probability of the semantic data, (ii) associating each semantic data with the spatial and temporal coordinates of the location of the sensor, and (iii) associating each semantic data with the spatial and temporal coordinates of an event remotely detected from the sensor.

The types of raw sensor data that can be analyzed to generate semantic data include, but are not limited to, environmental sensor data, gas data, accelerometer data, particle data, power data, RF signals, ambient light data, motion detection data, still images, video data, audio data, and the like. According to some embodiments, various sensor nodes in an LSN can employ processing of raw sensor data to generate probabilistic semantic data. Probabilistic semantic data can represent events, including the detection of people, vehicles, or other objects, through computer vision (video analytics) processing or other analysis of large data sets occurring locally at nodes in the network.

FIG3 is a block diagram illustrating a system 107 for performing probabilistic semantic sensing in a sensor network, according to one embodiment. The system 107 may include two or more sensing nodes 109, an aggregation node 125, and one or more probabilistic applications 117. Each of the sensing nodes 109 (e.g., machines) may include a sensing engine 111. Broadly speaking, each sensing node 109 includes one or more sensors 30 for sensing raw sensor data, which is passed as raw sensor data to the sensing engine 111. The sensing engine then processes the raw sensor data to generate semantic data 121. The semantic data 121 may include sensed event information 123 in the form of sensed events, each of which includes a classifier that categorizes the semantic data 121. The classifier may include semantic data (not shown) representing the meaning of the raw sensor data as a discrete event that includes a binary expression. For example, the binary expression may be for a parking space (e.g., occupied, unoccupied), a person (e.g., present, absent), or a vehicle (e.g., present, absent). Additional classifiers may be associated with semantic data, as discussed below.

The sensing nodes 109 may communicate the semantic data 121 to an aggregation node 125. The aggregation node 125 may include a correlation engine 113 and a probability engine 115. Other embodiments may include multiple aggregation nodes 125. Sensing nodes 109 that sense the same underlying phenomenon (e.g., parking spot #123) may communicate semantic data 121 (sensed events) representing the same underlying phenomenon to the same aggregation node 125. Thus, some sensing nodes 109 may communicate with two or more aggregation nodes 125 based on the underlying phenomenon being sensed and communicated by the sensing nodes 109. The aggregation node 125 may preferably be implemented in the cloud. Other embodiments may implement the correlation engine 113 and the probability engine 115 on any combination of the sensing nodes 109, another machine, or the like.

The correlation engine 113 receives sensed event information 123 in the form of sensed events via a network (e.g., a LAN, a WAN, the Internet, etc.) and correlates/aggregates semantic data 121 based on classifiers in each of the sensed events to generate aggregates of semantic data 127. The aggregates of semantic data 121 may be logically grouped. According to some embodiments, the correlation engine 113 may correlate and aggregate the semantic data 121 into aggregated semantic data 127 by constructing an abstract graph representing the correlation of two or more semantic data 121 received from one or more sensor nodes 109 in the sensor network. The correlation engine 113 may construct the abstract graph based on similarities of the semantic data 121 or based on other classifiers included in the sensed events. The correlation engine 113 may further construct the abstract graph based on relationships between the classifiers, including spatial and temporal coordinates associated with each piece of semantic data. By way of example only, the correlation engine 113 may correlate and aggregate all sensed events received from the sensing nodes 109 within a time period, each of which includes an assertion of the occupancy state (e.g., occupied, unoccupied) of a particular parking spot in a parking lot and a probability associated with the confidence level of the assertion. By way of further example, the correlation engine 113 may correlate and aggregate all sensed events received from the sensing nodes 109 within a time period, each of which indicates the presence of a person (e.g., present, absent) at a particular location in the parking lot and a probability associated with the confidence level of the assertion. The correlation engine 113 may construct the abstract graph using exact matches of the classifiers 141 ( FIG. 4A ), fuzzy matches of the classifiers 141, or a combination of both. According to some embodiments, the correlation engine 113 may correlate and aggregate the semantic data 121 into aggregated semantic data 127 based on a mathematical relationship between the locations of the sensors 30 and/or the locations of the semantic data (e.g., the locations of parking spots asserted as empty or occupied). For example, the correlation engine 113 may identify mathematical relationships between the locations of the sensor 30 (expressed in spatial and temporal coordinates) and/or the locations (eg, matches, near matches, etc.) of the semantic data (expressed in spatial and temporal coordinates).

The correlation engine 113 may pass the aggregated semantic data 127 to the probability engine 115, which then processes the aggregated semantic data 127 to generate derived event information 129 in the form of derived events. Those skilled in the art will appreciate that a sensor network may include multiple aggregation nodes 125 that pass derived event information 129 to the same sensor processing interface 131. The probability engine 115 processes the aggregated semantic data 127 by calculating derived events with derived probabilities using the relationship between the individual probabilities of each semantic data included in each sensed event, along with external data input. According to some embodiments, the external data input may include user input, application input, or other user-defined parameters with a desired accuracy, as further described below. The probability engine 115 processes a single aggregation of semantic data 127 to generate a single derived event. Thus, the probability engine 115 can intelligently reduce the amount of data that is then passed on for further analysis. By way of example only, the amount of data in a single aggregation of semantic data 127 is reduced by the probability engine 115 to a single derived event. Furthermore, the probability engine 115 can intelligently reduce the probabilities in the aggregation of semantic data 127 to produce a single derived event comprising a single derived probability. The probability engine 115 can further generate derived event information 129 based on thresholds 137 and weighting values 139. The probability engine 115 can use the thresholds 137 associated with each semantic data to determine the nature of the derived event. The initial determination of the thresholds 137 can be defined heuristically or generated by any other suboptimal process. The probability engine 115 can modify the thresholds 137 based on the ongoing analysis of the probabilities of the semantic data 121, as represented by the arrows illustrating the movement of the thresholds 137 both into and out of the probability engine 115. The probability engine 115 can modify the assignment of weights based on the ongoing analysis of the probabilities of the semantic data 121, as represented by the arrows illustrating the movement of the weighting values 139 both into and out of the probability engine 115. The probability engine 115 can assign higher weighting values 139 to semantic data 121 with higher probabilities or certainties. The probability engine 115 may receive initial weights defined heuristically or generated by any other suboptimal process. The probability engine 115 may change the assignment of weights based on ongoing analysis of the probabilities of the semantic data 121 in a stochastic process, as represented by the arrows illustrating the movement of the threshold 137 both into and out of the probability engine 115. According to some embodiments, the probability engine 115 may utilize the derived probabilities for additional processing, or may pass the derived probabilities in the derived events to the sensor processing interface 131 (e.g., an application processing interface). The sensor processing interface 131 may be read by one or more probabilistic applications 117 (e.g., "Application W," "Application X," "Application Y," "Application X"), which in turn process the derived events to enable the one or more applications to perform services.

4A is a block diagram illustrating sensed event information 123 according to one embodiment. Sensed event information 123 may be embodied as sensed events generated by sensing nodes 109 and communicated to aggregation nodes 125, where they are received by correlation engine 113. Sensed events may include classifiers 141 for characterizing semantic data 121. Classifiers 141 may include semantic data, application identifiers, probabilities, locations of sensors 30, locations of semantic data, and the like. Semantic data classifiers describe discrete events sensed by sensors 30 at sensing nodes 109 and may include semantic data expressing binary states, as previously described (e.g., parking lot occupied or unoccupied). Application identifier classifiers may identify one or more probabilistic applications 117 that receive derived event information 129 generated based on sensed events containing application identifiers. Probabilistic classifiers describe the certainty or reliability of a sensed event, as asserted in the associated semantic data. For example, a sensed event may include semantic data that the parking spot is empty with a probability of 99%, thereby indicating a 99% confidence that the parking spot is indeed empty. The sensor's location classifier describes the location of the sensor 30 that sensed the associated semantic data. The sensor's location classifier may be embodied as the spatial coordinates of the sensor 30 that sensed the associated semantic data and as a time coordinate indicating the date and time the associated semantic data was sensed by the sensor 30. The location classifier of the semantic data describes the location of the associated semantic data. The location classifier of the semantic data may be embodied as the spatial coordinates of the associated semantic data and as a time coordinate indicating the date and time the associated semantic data was sensed by the sensor 30.

Figure 4B is a block diagram illustrating the derived event information 129 according to one embodiment. The derived event information 129 may be embodied as derived events generated by the probability engine 115 and passed to the sensor processing interface 131. The derived event may include a classifier 143 for characterizing the derived event information 129. The meaning of the classifier 143 in the derived event corresponds to the meaning of the classifier 141 in the sensed event, as previously described. The semantic data classifier describes a discrete event sensed by one or more sensors 30 respectively located at a sensing node 109 and may include a descriptor expressing a binary state, as previously described (e.g., a parking space occupied or not occupied). In some instances, two or more sensors 30 may be located at the same sensing node 109. The application identifier classifier may identify one or more probabilistic applications 117 that receive the derived event information 129. The probabilistic classifier in the derived event describes the certainty or reliability of the derived event, as asserted in the associated semantic data. The probability classifier 143 may include a probability based on two or more sensed events as determined by the probability engine 115. For example, the derived event may include semantic data that the parking place is empty with a probability of 99% (based on the two or more sensed events). The location classifier of the sensor describes the location of the one or more sensors 30 that sensed the associated semantic data. The location classifier of the sensor may be embodied as the spatial coordinates of the one or more sensors 30 that sensed the associated semantic data and as corresponding time coordinates indicating the date and time of the associated semantic data sensed by the one or more corresponding sensors 30. The location classifier of the semantic data describes the location of the associated semantic data. The location classifier of the semantic data may be embodied as the spatial coordinates of the associated semantic data and as time coordinates indicating the date and time of the associated semantic data sensed by the corresponding one or more sensors 30.

FIG5 is a block diagram illustrating user input information 135 according to one embodiment. User input information 135 may include parameters or configuration values received by the probability engine 115 and used by the probability engine 115 to generate derived event information 129. User input information 135 may include desired accuracy information, application input information, and user preference information. Desired accuracy information may be received to identify a minimum level of raw sensor data required before the probability engine 115 generates derived events. Application input information may be received to configure the level for a particular probabilistic application 117. For example, the parking probability application 117 may utilize a configurable level for determining whether a parking spot is vacant. Configuring the level to a low level (e.g., 0) may force the probability engine 115 to make a parking spot vacant determination regardless of the amount of aggregated semantic data 127 available for making the determination. Configuring the level higher (e.g., 1 to X, where X>0) can enable the probability engine 115 to report insufficient information for amounts of aggregated semantic data 127 below the configured level and to report empty (or not empty) for amounts of aggregated semantic data 127 equal to or greater than the configured level. For example, a report (e.g., a sensed event) can include semantic data indicating that a parking space is "empty" (or not empty) or semantic data indicating "insufficient information."

6 is a block diagram illustrating a method 147 for performing probabilistic semantic sensing in a sensor network according to one embodiment. The method 147 may begin at operation 151 with the light sensing network receiving raw sensor data. For example, the light sensing network may include two sensing nodes 109 positioned at the tops of two light poles, each of which includes two lights illuminating a parking lot. The lights may be identified as "Light A" and "Light B". The sensing nodes 109 may each include a sensor 30 that receives the raw sensor data and a sensing engine 111 that processes the raw sensor data. The raw sensor data represents the occupancy status of multiple parking spaces in the parking lot. In one example, the raw sensor data collected at each of the sensor nodes 109 represents the same parking location.

At operation 153, the sensing engine 111 generates semantic data 121 based on the raw sensed data. For example, the sensing engine 111 at each of the sensing nodes 109 may process the raw sensed data to generate semantic data 121 in the form of sensed event information 123 in the form of two sensed events. The sensing engine 111 at "Light A" may generate a first sensed event that includes a classifier in the following form: semantic data asserting that the parking spot is empty, an application identifier identifying the parking spot application, a 95% probability that the asserted semantic data is true (e.g., the parking spot is indeed empty), coordinates (e.g., latitude, longitude/Global Positioning System (GPS) coordinates, and the like) identifying the location of the sensor 30 that sensed the asserted semantic data at "Light A," and coordinates identifying the location of the asserted semantic data (e.g., latitude, longitude/GPS coordinates, and the like identifying the location of the parking spot). A classifier specifying the date and time at which the sensor 30 was operated to acquire the semantic data is further associated with the location coordinates of the sensor 30. A classifier specifying the date and time at which the semantic data was sensed by the sensor 30 is further associated with the location coordinates of the semantic data.

The sensing engine 111 at "Light B" generates a second sensed event that includes a classifier 141 in the following form: semantic data asserting the same semantic data (e.g., the parking spot is empty), an application identifier identifying the parking spot application, an 85% probability that the asserted semantic data is true (e.g., the parking spot is indeed empty), coordinates (e.g., latitude, longitude/GPS coordinates, and the like) identifying the location of the sensor 30 that sensed the asserted semantic data at "Light B," and coordinates identifying the location of the asserted semantic data (e.g., latitude, longitude/GPS coordinates, and the like identifying the location of the parking spot). A classifier 141 specifying the date and time the sensor 30 was operated to acquire the semantic data is further associated with the location coordinates of the sensor 30. A classifier 141 specifying the date and time the semantic data was sensed by the sensor 30 is further associated with the location coordinates of the semantic data.

Finally, the sensing engine 111 at "Light A" transmits the first sensed event described above, generated at "Light A," to the aggregation node 125 via a network (e.g., a LAN, WAN, the Internet, etc.), where it is received by the correlation engine 113. Similarly, the sensing engine 111 at "Light B" transmits the second sensed event described above, generated at "Light B," to the same aggregation node 125 via a network (e.g., a LAN, WAN, the Internet, etc.), where it is received by the correlation engine 113. Those skilled in the art will appreciate that other embodiments may include additional sensing nodes 109 for sensing the same parking location. According to another embodiment, the aggregation node 125 including the correlation engine 113 may be located in the cloud. According to another embodiment, the correlation engine 113 may be located at the sensing node 109.

At operation 157, the correlation engine 113 may correlate the semantic data 121 based on the classifiers 141 in the semantic data 121 to generate an aggregate of the semantic data 121. The correlation engine 113 may continuously receive sensed events from the plurality of sensing nodes 109 in real time. The correlation engine 113 may correlate the sensed events based on the classifiers 141 in the sensed events to generate an aggregate 127 of semantic data. In the present example, the correlation engine 113 receives the first and second sensed events and correlates the two together based on selecting one or more classifiers 141 from the group of available classifiers. For example, the one or more classifiers 141 may include semantic data (e.g., parking lot is empty) and/or an application identifier and/or coordinates identifying the location of the semantic data being asserted. For example, the correlation engine 113 may correlate and aggregate the first sensed event with the second sensed event based on matching semantic data (e.g., parking lot is empty) and/or matching application identifiers (e.g., identifying the parking lot application) and/or matching coordinates identifying the location of the asserted semantic data (e.g., latitude, longitude/GPS coordinates, and the like, identifying the location of the parking lot). Other classifiers 141 for correlation and generation of aggregated semantic data 127 (e.g., aggregation of sensed events) may be selected from a group of available classifiers. Finally, at operation 157, the correlation engine 113 passes the aggregated semantic data 127 to the probability engine 115. According to one embodiment, the correlation engine 113 and the probability engine 115 execute in the aggregation node 125 in the cloud. In another embodiment, the correlation engine 113 and the probability engine 115 execute on different computing platforms and the correlation engine 113 passes the aggregated semantic data 127 to the probability engine 115 via a network (e.g., LAN, WAN, Internet, etc.).

At operation 159, the probability engine 115 may analyze each of the aggregates 127 of semantic data to generate derived event information 129 (e.g., derived events). For example, the probability engine 115 may analyze the aggregates 127 of semantic data including the first sensed event and the second sensed event to generate derived event information 129 in the form of a first derived event. The first derived event may include a probability of the semantic data classifier 143 (e.g., a 90% probability that the parking lot is empty) generated by the probability engine 115 based on the aggregates 127 of semantic data, the probability including the probability of the semantic data classifier 141 included in the first sensed event (e.g., an 85% probability that the parking lot is empty) and the probability of the semantic data classifier 141 included in the second sensed event (e.g., a 95% probability that the parking lot is empty). For example, the probability engine 115 may average the probabilities of the two semantic data 141 from the first and second sensed events (e.g., 85% and 95%) to produce a (single) probability (e.g., 90%) for the semantic data of the first derived event. Other examples may include additional probabilities for the semantic data classifiers 141 included in additional sensed events to produce a (single) probability of 90% for the semantic data of the first derived event. The probability engine 115 may utilize the user input information 135, threshold 137, and weighting value 139 as previously described to generate the derived event.

At operation 161, the probability engine 115 may pass the derived event information 129 to the sensor processing interface 131 to implement at least one probabilistic application 117. For example, the probabilistic application 117 may read the derived event information 129 (e.g., derived events) from the sensor processing interface 131 and utilize the classifiers 143 in the derived events to perform services and generate reports. The services may include controlling devices within or outside the sensor network and generating reports, as described more fully later in this document. For example, the probability engine 115 may pass the derived event information 129 in the form of a first derived event to the sensor processing interface 131 via a network (e.g., a LAN, a WAN, the Internet, etc.), where the derived event information is then read by one or more probabilistic applications 117 (e.g., Application X), which utilize the first derived event to perform services or generate reports. In one embodiment, the probability engine 115 and the sensor processing interface 131 may be on the same computing platform. In another embodiment, the probability engine 115 and the sensor processing interface 131 may be on different computing platforms.

According to some embodiments, applications of semantic data 121 and derived event information 129 may include management or monitoring of lighting capabilities. This type of probabilistic application 117 may involve classifiers 143 including presence events of people, vehicles, other entities, and associated illumination changes. This type of probabilistic application 117 may also include the determination of activity in areas beneath or around nodes of the lighting network, including beneath or around poles, walls, or other fixed objects. This type of probabilistic application 117 may also include the detection of tampering or theft associated with the lighting infrastructure. In each case, the probability associated with each semantic data may be limited by the location of the sensor 30, obstruction of the sensor field due to real-world obstructions, limitations on lighting illumination, availability of network bandwidth, or availability of computing power.

According to some embodiments, probabilistic applications 117 of semantic data 121 and derived event information 129 may include parking location and occupancy detection, monitoring, and reporting. This type of probabilistic application 117 may involve classifiers 141 and/or classifiers 143 for presence and motion events including people, cars, and other vehicles. This type of probabilistic application 117 may also use classifiers 143 for people, cars, and other vehicles based on parameters of the car or vehicle, including its make, model, type, and other aesthetic features. In each case, the probability associated with each semantic data may be limited by the location of the sensor 30, obstruction of the sensor field due to real-world obstructions (including the parking location of other vehicles or the location of other objects within the scope of the parking space), lighting limitations, availability of network bandwidth, or availability of computing power.

According to some embodiments, probabilistic applications 117 of semantic data 121 and derived event information 129 may include monitoring and reporting. This type of probabilistic application 117 may involve classifiers 141 and/or 143, including detection of people or objects or movement of people or objects. The goal of this type of probabilistic application 117 may be to increase public safety or regional security. In each case, the probability associated with each semantic data item may be limited by the location of the sensor 30, obstruction of the sensor field due to real-world obstructions, lighting limitations, network bandwidth availability, or computing power availability.

According to some embodiments, probabilistic applications 117 of semantic data 121 and derived event information 129 may include traffic monitoring and reporting. This type of probabilistic application 117 may involve classifiers 141 and/or 143 that include the presence and movement of people, cars, and other vehicles. This type of probabilistic application 117 may also classify people, cars, and other vehicles based on parameters of the car, including its make, model, type, and other aesthetic features. In each case, the probability associated with each semantic data may be limited by the location of the sensor 30, obstruction of the sensor field due to real-world obstacles, lighting limitations, availability of network bandwidth, or availability of computing power.

According to some embodiments, probabilistic applications 117 of semantic data 121 and derived event information 129 may include retail customer monitoring and reporting. This type of probabilistic application 117 may involve classifiers 141 and/or classifiers 143 that include the presence and movement of people, cars, and other vehicles in a manner that can be useful to retailers. This type of probabilistic application 117 may also include determining trends regarding the use of a retail location. In each case, the probability associated with each semantic data may be limited by the location of the sensor 30, obstruction of the sensor field due to real-world obstructions, lighting limitations, availability of network bandwidth, or availability of computing power.

According to some embodiments, probabilistic applications 117 of semantic data 121 and derived event information 129 may include business intelligence monitoring. This type of probabilistic application 117 involves classifiers 141 and/or classifiers 143 that include the status of systems used by an enterprise for operational, facility, or business purposes, including activity at points of sale (PoS) and other strategic locations. This type of probabilistic application 117 may also include determining trends related to business intelligence. In each case, the probability associated with each semantic data may be limited by the location of the sensor 30, obstruction of the sensor field due to real-world obstructions, lighting limitations, network bandwidth availability, or computing power availability.

According to some embodiments, probabilistic applications 117 of semantic data 121 and derived event information 129 may include asset monitoring. This type of probabilistic application 117 may involve classifiers 141 and/or classifiers 143 that include monitoring of high-value or other strategic assets, such as vehicles, inventory, valuables, industrial equipment, etc. In each case, the probability associated with each semantic data may be limited by the location of the sensor 30, obstruction of the sensor field due to real-world obstructions, lighting limitations, availability of network bandwidth, or availability of computing power.

According to some embodiments, probabilistic applications 117 of semantic data 121 and derived event information 129 may include environmental monitoring. This type of probabilistic application 117 may involve classifiers 141 and/or 143, including classifiers related to monitoring wind, temperature, pressure, gas concentration, airborne particulate matter concentration, or other environmental parameters. In each case, the probability associated with each semantic data may be limited by the location of the sensor 30, obstruction of the sensor field due to real-world obstructions, lighting limitations, network bandwidth availability, or computing power availability. In some embodiments, environmental monitoring may include seismic sensing.

Lighting Infrastructure Application Framework

The present invention further relates to the use of street or other lighting systems as the basis for a network of sensors 30, platforms, controllers, and software that implement functionality beyond lighting of outdoor or indoor spaces.

Industrialized countries around the world have extensive indoor and outdoor lighting networks. Streets, highways, parking lots, factories, office buildings, and all types of facilities typically feature extensive indoor and outdoor lighting. Until recently, virtually all of this lighting used incandescent or high-intensity discharge (HID) technology. However, incandescent and HID lighting are inefficient at converting electricity into light output. A significant portion of the electricity used for incandescent lighting is dissipated as heat. This not only wastes energy but also often leads to failure of the bulbs themselves and the lighting fixtures.

Because of these shortcomings and the operational and maintenance cost efficiencies of LED or other solid-state lighting technologies, many owners of large incandescent or HID lamps are converting them to solid-state lighting. Solid-state lighting not only provides longer-life bulbs, thereby reducing labor costs for replacement, but the resulting fixtures also operate at lower temperatures for longer periods of time, further reducing the need for maintenance. The attorney for this application provides lighting replacement services and devices to various municipalities, businesses, and private owners, enabling them to operate their facilities with reduced maintenance costs and reduced energy costs.

A networked sensor and application framework has been developed for deployment in street or other lighting systems. The architecture of this system allows for the deployment of a networked system within the lighting infrastructure, either already in place or at the time of its initial installation. While the system is typically deployed in outdoor street lighting, it can also be deployed indoors, for example, in factories or office buildings. Furthermore, when the system is deployed outdoors, it can be installed when streetlight bulbs are changing from incandescent lighting to more efficient lighting, such as using light-emitting diodes (LEDs). Replacing these incandescent bulbs is expensive, primarily due to labor costs and the need to use special equipment to reach each bulb in each streetlight. By installing the network described herein at the time described, the added cost is minimal compared to simply replacing the existing incandescent bulbs with LED bulbs.

Because this system enables many different uses, the deployed network, sensors 30, controllers, and software system described herein is referred to as the Lighting Infrastructure Application Framework (LIAF). The system uses the lighting infrastructure as a platform for business and customer applications implemented using a combination of hardware and software. The main components of the framework are node hardware and software, sensor hardware, site-specific or cloud-based server hardware, network hardware and software, and wide area network resources that enable data collection, analysis, action invocation, and communication with applications and users.

Those skilled in the art will appreciate that LIAF can be used to embody methods and systems for probabilistic semantic sensing, and more specifically probabilistic semantic sensing in light sensing networks, as previously described. Although the system described herein is in the context of street lighting, it will be apparent from the following description that the system also has applicability to other environments, for example, in parking garages or factory environments.

In one embodiment, this system provides a lighting system network that utilizes existing outdoor parking structures and indoor industrial lights. Each light can become a node in the network, and each node includes: a power control terminal for receiving power; a light source coupled to the power control terminal; a processor coupled to the power control terminal; a network interface coupled between the processor and the lighting system network; and a sensor 30 coupled to the processor for detecting conditions at the node. In some applications, as described below, the network is independent of the lighting system. In combination, this system allows each node to communicate information about its conditions to other nodes and a central location. Processing can thus be distributed among the nodes in the LIAF.

A gateway coupled to the network interfaces of some LIAF nodes is used to provide information from the sensors 30 at the nodes to a local or cloud-based service platform where application software stores, processes, distributes, and displays the information. This software performs the desired operations related to the conditions detected by the sensors 30 at the nodes. In addition, the gateway can receive information from the service platform and provide the information to each of the node platforms in its domain. The information can be used to facilitate maintenance of lights, control of lights, control cameras, locate unoccupied parking spaces, measure carbon monoxide levels, or numerous other applications, several typical of which are described herein. Sensors 30 arranged near or close to the nodes can be used with controllers to control light sources and provide control signals (e.g., locking or unlocking a parking area) to devices coupled to the nodes. Multiple gateways can be used to couple together multiple zones of a lighting system for the purposes of a single application.

Typically, each node will include an alternating current (AC)/direct current (DC) converter to convert the supplied AC power to DC for use by processors, sensors 30, etc. Gateways can communicate with each other via cellular phones, Wi-Fi, or other means to a service platform. Sensors 30 are typically devices that detect specific conditions, for example, audio from broken glass or car alarms, video cameras for security and parking-related sensing, motion sensors, light sensors, radio frequency identification detectors, weather sensors, or detectors for other conditions.

In another embodiment, a network of sensors 30 is provided for collecting information using an existing lighting system having fixtures with light sources. The method includes replacing the light source at each fixture with a module including power control terminals that are connected to the existing fixture, the replacement light source, a processor, a network interface coupled to the processor, and a power supply for the sensor 30 coupled to the processor. The sensor 30 detects conditions at and around the node and forwards information about the conditions to the processor. Preferably, the network interface of each module at each fixture is coupled together, typically using a broadband or cellular communication network. Information is collected from the sensors 30 using a communication network and provided via the network to an application running on a local server at the site or on a server in the cloud. The local or site-based application server is referred to as a site controller. Applications running on the site controller can manage data from one or more specific customer sites.

In one embodiment, each module at each of the fixtures includes a controller and a device coupled to the controller, and the controller is used to cause an action to be performed by the device. As mentioned above, a signal can be transmitted from a computing device to the module and thereby to the controller via a communication network to cause the device of the lighting system to perform an action.

The lighting infrastructure application framework described here is based on a node, gateway, and service architecture. The node architecture consists of a node platform deployed at various locations in the lighting infrastructure, such as at individual street lamps. At least some nodes include sensors 30 that collect and report data to other nodes and, in some cases, higher levels in the architecture. For example, at the individual node level, ambient light sensors can provide information about lighting conditions at the location of a lighting fixture. Cameras can provide information about events occurring at the node.

FIG7 illustrates a portion of the overall architecture of such a system. As shown, a lighting node includes a node platform 10 (e.g., "NP") (e.g., a sensing node 109) in addition to the light source itself. Depending on the specific application desired, the node platform 10 includes various types of sensors 30 selected by the owner of the lighting node. In the illustration, a daylight sensor 31 and an occupancy sensor 32 are depicted. The lighting node may also include a controller 40 for performing functions in response to the sensors 30 or in response to control signals received from other sources. Three exemplary controllers 40 are illustrated in the figure: an irrigation control 42 for controlling an irrigation system, a door control 45 for opening and closing a nearby door, and a light controller 48. The light controller 48 can be used to control the lighting sources in the node platform 10, for example, turning the lighting sources off or on at different times of the day, dimming the lighting sources, causing the lighting sources to flash, sensing the condition of the light sources themselves to determine whether maintenance is required, or providing other functionality. The sensor 30 , controller 40 , power supply, and other required components may be collectively assembled into a housing of the node platform 10 .

Other examples of these or similar control functions implemented by the controller 40 may include management of power distribution, power measurement and monitoring, and demand/response management. The controller 40 may activate and deactivate the sensors 30 and measure and monitor sensor outputs. Additionally, the controller 40 provides management of communication functions (such as gateway operations for software downloads and security management) and management of video and audio processing (for example, detection or monitoring of events).

In one embodiment, the architecture of this networked system enables "plug-and-play" deployment of sensors 30 at lighting nodes. The Lighting Infrastructure Application Framework (LIAF) provides the hardware and software to implement the sensor plug-and-play architecture. When a new sensor 30 is deployed, the software and hardware manage the sensor 30, but LIAF provides support for generic functionality associated with the sensor 30. This can reduce or eliminate the need for custom hardware and software support for the sensor 30. The sensor 30 may require power (typically a battery or wired low-voltage DC), and preferably generates an analog or digital signal as output.

LIAF allows for the deployment of sensors 30 at lighting nodes without additional hardware and software components. In one embodiment, LIAF provides DC power to the sensors 30. LIAF also monitors the analog or digital interfaces associated with the sensors 30, as well as all other activity at the node.

Node platforms 10 located at some of the lamps are coupled together to a gateway platform 50 (e.g., "GP") (e.g., aggregation node 125). The gateway platform 50 communicates with the node platform 10 using techniques as further described below, but may include wireless or wired connections. The gateway platform 50 will preferably communicate with the Internet 80 using well-known communication technologies 55 (e.g., cellular data, Wi-Fi, GPRS, or other means). Of course, the gateway platform 50 need not be a stand-alone implementation. It can be deployed at the node platform 10. In addition to the functionality provided by the node platform 10, the gateway platform 50 also provides wide area network (WAN) functionality and can provide complex data processing functionality.

The gateway platform 50 establishes communication with the service platform 90 (e.g., "SP"), enabling the node to provide data to various applications 100 (e.g., probabilistic application 117) or receive instructions from various applications 100. The service platform 90 is preferably implemented in the cloud to enable interaction with the applications 100 (e.g., probabilistic application 117). When the service platform 90 or a subset with such functionality is implemented locally at a site, then the service platform or subset with such functionality is referred to as a site controller. A variety of applications 100 (e.g., probabilistic application 117) that provide end-user accessible functionality are associated with the service platform 90. These applications 100 can be provided by owners, partners, customers, or other entities. For example, a typical application 100 provides a report on current weather conditions at a node. Applications 100 are typically developed by others and licensed to infrastructure owners, but applications can also be provided by node owners or otherwise made available for use on various nodes.

Typical lighting-related applications 100 include lighting control, lighting maintenance, and energy management. These applications 100 preferably run on a service platform 90 or a site controller. There may also be partner applications 100—applications 100 that can utilize confidential data and for which the lighting infrastructure owner grants privileges. These applications 100 may provide security management, parking management, traffic reporting, environmental reporting, asset management, logistics management, and retail data management, to name a few possible services. There are also client applications 100 that enable clients to utilize general data, where access to this data is, for example, authorized by the infrastructure owner. Another type of application 100 is owner-provided applications 100. These are applications 100 developed and used by the infrastructure owner (e.g., to control traffic flow in an area or along municipal streets). Of course, there may also be applications 100 that utilize custom data from the framework.

The main entities involved in the system illustrated in Figure 7 are lighting infrastructure owners, application framework providers, application 100 or application service owners, and end users. Typical infrastructure owners include municipalities, property owners, tenants, electric utilities, or other entities.

FIG8 is a diagram illustrating the architecture of this system at a higher level. As shown in FIG8 , groups of node platforms 10 communicate with each other and with a gateway platform 50. The gateway, in turn, communicates to the Internet 80 via a communication medium 55. In a typical embodiment as illustrated, there will be multiple groups of nodes 10, multiple gateway platforms 50, and multiple communication media 55, all of which are collectively coupled together to a service platform 90 available via the Internet 80. In this way, multiple applications 100 can provide a wide range of functionality to individual nodes through the gateways in the system.

FIG8 also illustrates a networking architecture for an array of nodes. In the left-hand portion 11 of the diagram, an array of nodes 10 is illustrated. The solid lines between the nodes represent the data plane, which connects selected nodes to facilitate high-bandwidth local services. For example, these connections can enable the exchange of local video or data between these nodes. The dashed lines in portion 11 represent the control plane, which connects all nodes to each other and provides transport for local and remote services, exchanging information about events, usage, and node status, and enabling control commands from and responses to the gateway.

FIG9 illustrates the node platform 10 in more detail. The node infrastructure includes a power module 12, which is typically implemented as an AC to DC converter. In one embodiment, where the node is deployed at outdoor streetlights, AC power is the primary power supply for these streetlights. Since most sensor 30 and controller 40 structures use semiconductor-based components, the power module 12 converts the available AC power to an appropriate DC power level for driving the node components.

As also shown in FIG9 , an array of sensors 30 and controllers 40 is connected to a power module 12, which may include an AC/DC converter and other well-known components. A processor running a processor module 15 coordinates the operation of the sensors 30 and controllers 40 to implement desired local functionality, including the operation of the sensing engine 111, as previously described. The processor module 15 also provides communication to other node platforms 10 via appropriate media. The application 100 may also drive the light source module 16, couple to appropriate third-party light source modules 18, and operate under the control of one of the controllers 40. Implementations may combine the functionality of the power module 12 and light controller 48 into a single module. As indicated by the diagram, wired connections 46 and 47 and wireless connections 44 and 49 may be provided as needed.

In FIG9 , the lighting infrastructure consists of light source modules 16 and 18, such as LED assemblies commercially available from Sensity Systems Inc. Of course, third-party manufacturers can provide third-party light source modules 18 and other components. Modules 16 can also be coupled to a controller 40. Sensors 30 associated with a node can be local to the node or remote. Controller 40 (except for the LED controller provided by Sensity Systems Inc.) is typically remote and uses wireless communication. Processor module 15 (also known as the node application controller) manages all functions within the node. Processor module 15 also implements management, data collection, and action instructions associated with application 100. These instructions are typically delivered to controller 40 as application scripts. In addition, software on the application controller provides activation, management, security (authentication and access control), and communication functions. Network module 14 provides radio frequency (RF)-based wireless communications to other nodes. These wireless communications can be based on neighborhood area networks (NANs), WiFi, 802.15.4, or other technologies. Sensor modules can be used to operate sensors 30. The processor module 15 is further illustrated as being communicatively coupled to the sensing engine 111 , which operates as previously described.

FIG10 is a block diagram of a gateway platform 50. As suggested by the diagram and as mentioned above, the gateway platform 50 can be located at the node or in its own housing separate from the node. In the diagram of FIG10 , the power module 12, processor module 15, LED light source module 16, and third-party light source module 18 components are again shown, along with the sensor module 30 and controller module 40. Further illustrated are the correlation engine 113 and the probability engine 115, both operating as previously described.

In addition to the functions supported by the node platform 10, the gateway platform 50 hardware and software components also implement high-bandwidth data processing and analysis using the media module 105 (e.g., at video rates) and the relay or WAN gateway 110. The gateway platform 50 can be considered the node platform 10, but with additional functionality. The high-bandwidth data processing media module 105 supports video and audio data processing functions that can analyze, detect, record, and report application-specific events. The relay or WAN gateway 110 can be based on GSM, Wi-Fi, LAN to Internet, or other wide area network technologies.

FIG11 is a block diagram of the service platform 90. The service platform 90 supports an application gateway 120 and a custom node application builder 130. The application gateway 120 manages interfaces to different types of applications (e.g., probabilistic applications 117) that are implemented using sensors and event data from lighting nodes. The service platform 90 with the application gateway 120 (e.g., sensor processing interface 131, according to one embodiment) can be deployed as a site controller at a customer lighting site. Thus, a site controller is an instance of the service platform 90 with only the functionality of the application gateway 120. The custom node application builder 130 allows for the development of custom node application scripts (e.g., probabilistic applications 117). These scripts specify to the processor module 15 (see FIG9 ) data collection instructions and operations to be performed at the node level. The scripts specify to the application gateway 120 how the results associated with the script are provided to the application (e.g., probabilistic application 117).

FIG11 also illustrates an owner application 140 (e.g., probability application 117), a sensitivity application 144 (e.g., probability application 117), a partner application 146 (e.g., probability application 117), and a client application 149 (e.g., probability application 117) utilizing an application gateway API 150 (e.g., sensor processing interface 131, according to one embodiment). Thus far, the agent has developed and implemented various types of applications (e.g., probability application 117) that are common to many uses of sensor 30. One such application 100 is lighting management. The lighting management application provides lighting status and control functionality for light sources at the node platform 10. Another application provided by the agent (e.g., probability application 117) provides lighting maintenance. The lighting maintenance application, for example, allows users to maintain their lighting network by enabling monitoring of the status of lights at each node. Energy management applications (e.g., probability application 117) allow users to monitor lighting infrastructure energy usage and thereby better control that usage.

The partner applications 146 shown in FIG11 are typically agency-approved applications and application service companies that have established a marketplace for various desired functions, such as those listed below. These applications 100 utilize the application gateway API 150. Typical partner applications 146 provide security management, parking management, traffic monitoring and reporting, environmental reporting, asset management, and logistics management.

Client applications 149 utilize an application gateway API 150 to provide client-related functionality. This API 150 provides access to publicly available, anonymous, and owner-approved data. Also shown are owner applications 140 that are developed and used by lighting infrastructure owners to meet their various specific needs.

Figure 12 illustrates the lighting infrastructure application revenue model for the system described above. This revenue model illustrates how revenue is generated and distributed among key stakeholders in the lighting infrastructure. Generally speaking, applications 100 and/or application service providers collect revenue A from application users. Application 100 owners or service providers pay fees B to the lighting infrastructure application framework service provider. The LIAF service provider pays fees C to the lighting infrastructure owner.

Key stakeholders in lighting infrastructure-based applications 100 include lighting infrastructure owners. These owners are the entities that own the light poles/fixtures and the property on which the lighting infrastructure is located. Another key party involved in the system is the LIAF service provider. These LIAF service providers are the entities that provide the hardware and software platforms deployed to provide data and services for applications 100. Agents in this context are LIAF service providers. Other important entities include application (e.g., probabilistic application 117) developers and owners. These entities sell applications 100 or application services. These applications 100 and service providers are based on data collected, processed, and distributed by LIAF.

Among the revenue sources used to fund the LIAF are applications, application services, and data. Revenue options exist for applications 100 or application service providers. Users of applications 100 or application services pay a license fee, typically based on time intervals or as a one-time payment. This fee is based on different usage tiers, for example, standard, professional, and manager. Usage fees can also depend on the type of data (e.g., raw or summarized, real-time versus non-real-time, etc.), access to historical data, data based on dynamic pricing on demand, and the location associated with the data.

Another application service includes advertisers. These advertisers are businesses that want to advertise their products or services to applications 100 and application service users. These advertisers pay advertising fees for each application 100 or service.

Regarding data, applications 100 and application service developers pay to access it. This data includes specific data for the entire light, such as energy usage at a node, on a per-light engine basis, per-light engine channel, or per sensor 30. Another type of data is light status (e.g., management status), such as temperature thresholds or energy costs to trigger dimming, dimming percentages, and reports on light status including settings for detection and reporting intervals. This data can also include operational status, such as the current state of the light (on or off, dimmed by and by, faulty, abnormal, etc.). Other types of data include environmental data (e.g., temperature, humidity, and atmospheric pressure at a node) or lighting data (e.g., ambient light and its color).

Nodes can also sense and provide numerous other types of data. For example, gases such as carbon dioxide, carbon monoxide, methane, natural gas, oxygen, propane, butane, ammonia, or hydrogen sulfide can be detected and reported. Other types of data include accelerometer status indicating seismic events, intrusion detector status, Bluetooth® RTM® sup.1 Media Access Control (MAC) addresses, active radio frequency identification (RFID) tag data, ISO-18000-7, and DASH 7 data. Some of these applications 100 and the data they can collect are described in more detail below.

Application-specific sensor data may include intrusion sensors for detecting intrusion at the base of a pole or light fixture, unauthorized opening of a cover at the base of a pole, or unauthorized opening of a light fixture; vibration sensors for detecting intrusion-related vibrations, earthquake-related vibrations, or pole damage-related vibrations. Motion sensors may detect motion, the direction of motion, and the type of motion detected.

Audio sensors can provide another type of collectible data. Audio sensors can detect glass breaking, gunshots, vehicle engine on/off events, tire noise, vehicle door closing, human communication events, or human distress noise events.

People detection sensors can detect a single person, multiple people, and count people. Vehicle detection can include a single vehicle, multiple vehicles, and the duration of sensor visibility. Vehicle detection can provide a vehicle count or identification information such as make, model, color, license plate, etc.

This system can also provide data about relevant events, typically by using data from multiple sensors 30. For example, sensor data from motion detectors and people detectors can be combined to trigger a lighting function to turn a light on, off, dim a light, or brighten a light. Counting people with motion detection can provide information about security, retail activity, or traffic-related events. Motion detection coupled with vehicle detection can be used to indicate a breach in a facility's security.

The use of combinations of sensors 30 (e.g., motion and vehicle counting or motion and audio) provides useful information for performing various actions. The timing of data collection can also be combined with data from sensors 30 (such as the data discussed above) to provide useful information, such as motion detection during the opening and closing hours of a facility. A light level sensor coupled to a motion detection sensor can provide useful information for lighting control. Motion detection can be combined with video to capture data only when an event occurs. Current and historical sensor data can be correlated and used to predict events or the need to adjust control signals, such as traffic flow patterns.

Another use for data collected at nodes is aggregation. This allows for the use of data events to generate representative values for groups using a variety of techniques. For example, aggregated data can be used to gather information about the type of luminaire at a location (e.g., pole-top vs. exterior wall luminaires); environmentally protected vs. unprotected luminaires; or luminaires located outside of exposed areas. Data can be collected based on lighting area (e.g., road, parking lot, driveway), facility type (e.g., manufacturing, R&D), company region (e.g., international vs. domestic), and more.

Power usage can be aggregated by appliance type, facility, facility type, or geographic region. Environmental sensing-related aggregations can be provided for geographic regions or facility types. Security applications include aggregations for geographic regions or facility types. Transportation applications include aggregations by time of day, week, month, year, or by geographic region (e.g., school zones versus retail zones). Retail applications include aggregations by time of day, week, month, etc., as well as by geographic region or facility type. Data can also be filtered or aggregated based on user-specified criteria, such as time of day.

Custom application development allows users to specify data to be collected and forwarded to custom applications 100 and services, actions to be performed based on the data at the lighting nodes, the format of the data to be forwarded to the application 100 or application services, and management of historical data.

This revenue sharing model allows revenue to be distributed among the lighting infrastructure owner, the application infrastructure owner, and the application 100 or application service owner. Today, lighting is a cost center for infrastructure owners, with capital investment, energy bills, and maintenance costs. Here, the broker provides the hardware, software, and network resources to enable the application 100 and application services on a daily basis, allowing the infrastructure owner to offset at least some of the capital, operating, and maintenance costs.

Figures 13 to 16 illustrate four sample applications 100 of the system described above. Figure 13 illustrates a parking management application 181 (e.g., a probability application 117). Each of a series of vehicle detection sensors 180 is positioned above each parking space in the parking garage, or a single multi-space occupancy detection sensor is positioned at each light. The sensors 180 can operate using any well-known technology that detects the presence or absence of a vehicle parked below it. When parking space-specific sensors 180 are deployed, each sensor 180 includes an LED that displays whether the space is open, occupied, or reserved. This enables drivers in the garage to locate open, available, and reserved spaces. It also allows garage owners to know when a space is available without having to visually inspect the entire garage. The sensors 180 are coupled to the node platform 10 using wired or wireless technology, such as described for the above system. The node platform 10 communicates to the site controller 200 via a local area network (LAN) 210 and/or communicates to the service platform 90 using a gateway platform 50. The gateway platform 50 is connected to the service platform 90 and the users 220 via the Internet 80. The site controller 200 can communicate with the service platform 90 or the parking management application 181. The parking management application 181 enables the users 220 to reserve a space by accessing the application 181 via the Internet 80.

FIG14 illustrates a lighting maintenance application 229 (e.g., the probability application 117). The lighting maintenance application 229 includes lighting nodes (e.g., the node platform 10) that are networked together using a system such as that described above and, in turn, coupled to a site controller 200. Information about the lighting nodes (e.g., power consumption, operating status, on-off activity, and sensor activity) is reported to the site controller 200 and/or the service platform 90 using the techniques described above. In addition, the site controller 200 and/or the service platform 90 may collect performance data (e.g., temperature or current) as well as status data (e.g., activity occurring at the node 10). The lighting maintenance application 229, which provides lighting maintenance-related functionality, accesses raw maintenance data from the service platform 90. Maintenance-related data (e.g., LED temperature, LED power consumption, LED faults, network faults, and power supply faults) can be accessed from the lighting maintenance application 229 by the lighting maintenance company 230 to determine when service is desired or when other attention is needed.

Figures 15A and 15B illustrate the inventory application 238 (e.g., probability application 117) and space utilization application 237 of the system described above. As illustrated above, a series of RFID tag readers 250 are positioned throughout the warehouse along the node platforms 10. These tag readers 250 detect RFID tags 260 on various items in the warehouse. Using the network of node platforms 10 described herein, the tag readers 250 can provide this information to the site controller 200 and/or the service platform 90. The tag readers 250 collect location and identification information and forward the data to the site controller 200 and/or the service platform 90 using the node platform 10. This data is then forwarded from the service platform 90 to the application 100 (e.g., inventory application 238). The location and identification data can be used to track the flow of goods within a protective structure (e.g., a warehouse). The same strategy can be used to monitor warehouse space utilization. Sensors 30 detect the presence of items in the warehouse and the space these items occupy. This space utilization data can be forwarded to the site controller 200 and/or the service platform 90. The application 100 that monitors and manages the space can utilize the space utilization application 237 (eg, the probability application 117 ) to access data describing the space from the service platform 90 .

FIG16 illustrates a logistics application 236 (e.g., probabilistic application 117) for monitoring loading docks and tracking cargo from source to destination. For example, RFID tags 260 can be positioned to track cargo from the source (e.g., loading port dock), to a transfer station (e.g., a weigh station or gas station), all the way to the destination (e.g., a warehouse) by utilizing the node platform 10. Similarly, RFID tags 260 can be positioned on cargo and vehicles transporting the cargo. RFID tags 260 transmit location information, identification information, and other sensor data information using the node platform 10, which in turn transmits the aforementioned information to the service platform 90. This can further be performed using the gateway platform 50 at each point (e.g., source, transfer station, and destination). The service platform 90 makes this data available to applications 100 (e.g., logistics application 236), enabling users 220 accessing the logistics application 236 to obtain accurate location and cargo status information.

FIG17 is a block diagram of electrical components used for power monitoring and control within a node. The illustrated power measurement and control module measures incoming AC power and controls the power provided to the AC/DC converter. The power measurement and control module also provides surge suppression and power to the node components.

This circuit is used to control power to light-emitting diodes at individual nodes. The actual number of inputs or outputs outlined below depends on customer application specifications. As shown in the diagram, AC power is provided via line 300 at a voltage range between 90 volts and 305 volts. Voltage and current are sensed by energy measurement integrated circuit 310. AC-to-DC transformer 320 provides 3.3 volts to circuit 310 to power integrated circuit 310. In Figure 17, the dashed lines represent the non-isolated portion of the high-voltage system. The dotted lines indicate the portion of the circuit protected at up to 10,000 volts.

Integrated circuit 310 is a complementary metal oxide semiconductor (CMOS) power measurement device that measures line voltage and current. The CMOS power measurement device is capable of calculating active, reactive, and apparent power, as well as RMS voltage and current. The CMOS power measurement device provides an output signal 315 to a universal asynchronous receiver/transmitter (UART) device 330. The UART device 330 converts data between a parallel interface and a serial interface. UART 330 is connected to provide a signal to a microcontroller 340, which controls the output voltage provided to a load 350, preferably an LED lighting system 350. This control is implemented using a switch 355.

Devices 360 and 365 are also coupled to microcontroller 340 and implement a controller area network bus system (commonly known as a CAN bus). The CAN bus allows multiple microcontrollers to communicate with each other without relying on a host computer. The CAN bus provides a message-based communication protocol. The CAN bus allows multiple nodes to be daisy-chained together for communication therebetween.

A power module 370 is optionally provided on the circuit board. The power module 370 accepts AC power through its input terminals and provides controlled DC power at its output terminals. If desired, the power module can provide input power for some of the devices illustrated in FIG. 18 , which is discussed next.

FIG18 is a block diagram of an application controller located at a node. The node provides wireless communication with application software. This application software controls power, lighting, and sensors 30 running on a microcontroller 400. It also provides power to the various modules illustrated in the figure and enables communication with sensors 30.

The application controller in FIG18 operates under the control of microcontroller 400, which is depicted in the center of the figure. Incoming power 405 (for example, supplied by module 370 in FIG17 ) is stepped down to 5 volts by transformer 410 to provide power for Wi-Fi communication and is also provided to 3.3 volt transformer 420, which powers microcontroller 400. Power supply 430 also receives input power and provides it to sensor 30 (not shown). 3.3 volt power is also provided to reference voltage generator 440.

The microcontroller 400 provides several input and output terminals for communicating with various devices. Specifically, in one embodiment, the microcontroller 400 is coupled to provide three 0-10 volt analog output signals 450 and receive two 0-10 volt analog input signals 460. These input and output signals 460 and 450 can be used to control and sense the conditions of the various sensors 30. Communication with the microcontroller 400 is achieved through a UART 470 using a CAN bus 480. As explained with respect to FIG. 17 , the CAN bus 480 enables communication among the microcontrollers without the need for a host computer.

To enable future applications 100 and provide flexibility, the microcontroller 400 also includes a number of general-purpose input/output pins 490. These general-purpose input/output pins accept or provide signals ranging from 0 volts to 36 volts. These general-purpose input/output pins are general-purpose pins whose behavior can be controlled or programmed through software. Having these additional control lines allows additional functionality to be implemented through software without requiring hardware replacement.

The microcontroller 400 is also coupled to a pair of I2C bus interfaces 500. These bus interfaces 500 can be used to connect to other components on the board or to connect to other components linked via cables. The I2C bus 500 does not require a predefined bandwidth, but still implements multi-mastering, arbitration, and collision detection. The microcontroller 400 is also connected to an SP1 interface 510 to provide surge protection. In addition, the microcontroller 400 is coupled to a USB interface 520 and a JTAG interface 530. Various input and output buses and control signals enable the application controller at the node interface (including a wide variety of sensors 30 and other devices) to provide, for example, lighting control and sensor management.

The foregoing is a detailed description of a networked lighting infrastructure for use with the sensing application 100. As described, the system provides unique capabilities for existing or future lighting infrastructures. While numerous details have been provided regarding a particular embodiment of the system, it will be understood that the scope of the invention is defined by the appended claims.

Machine and software architecture

1-18 are implemented in some embodiments in the context of multiple machines and associated software architectures. The following section describes representative software architectures and machine (e.g., hardware) architectures suitable for use with the disclosed embodiments.

Software architectures are used in conjunction with hardware architectures to create devices and machines tailored to specific applications. For example, a specific hardware architecture coupled with a specific software architecture will create a mobile device, such as a mobile phone, tablet device, or the like. Slightly different hardware and software architectures can create smart devices for use in the "Internet of Things." Yet another combination creates a server computer for use within a cloud computing architecture. Not all combinations of such software and hardware architectures are presented here, as those skilled in the art will readily understand how to implement the present invention in contexts other than the disclosure contained herein.

Software Architecture

FIG19 is a block diagram 2000 illustrating a representative software architecture 2002 that can be used in conjunction with the various hardware architectures described herein. FIG19 is merely a non-limiting example of the software architecture 2002, and it will be appreciated that many other architectures can be implemented to facilitate the functionality described herein. The software architecture 2002 can be executed on hardware, such as the machine 2100 of FIG20 , which includes, among other things, a processor 2110, a memory 2130, and I/O components 2150. Returning to FIG19 , a representative hardware layer 2004 is illustrated and can represent, for example, the machine 2100 of FIG20 . The representative hardware layer 2004 includes one or more processing units 2006 with associated executable instructions 2008. The executable instructions 2008 represent the executable instructions of the software architecture 2002, including implementations of the methods, engines, modules, and the like of FIG1 through FIG18 . Hardware layer 2004 also includes a memory and/or storage module 2010, which also has executable instructions 2008. Hardware layer 2004 may also include other hardware, as indicated by 2012, which represents any other hardware of hardware layer 2004, such as other hardware 2012 illustrated as part of machine 2100.

In the exemplary architecture of Figure 19, software 2002 can be conceptualized as a stack of layers, each of which provides specific functionality. For example, software architecture 2002 may include, for example, the following layers: operating system 2014, library 2016, framework/middleware 2018, application 2020 (e.g., probabilistic application 117), and presentation layer 2044. Operationally, application 2020 and/or other components within the layer may call application programming interface (API) calls 2024 through the software stack and receive responses, return values, etc., illustrated as messages 2026 in response to API calls 2024. The illustrated layers are representative in nature, and not all software architectures have all layers. For example, some mobile or dedicated operating systems 2014 may not provide framework/middleware layer 2018, while other operating systems may provide this layer. Other software architectures may include additional or different layers.

The operating system 2014 can manage hardware resources and provide common services. For example, the operating system 2014 can include a kernel 2028, services 2030, and drivers 2032. The kernel 2028 can serve as an abstraction layer between the hardware layer and other software layers. For example, the kernel 2028 can be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, and the like. Services 2030 can provide other common services for other software layers. Drivers 2032 can be responsible for controlling or interfacing with the underlying hardware. For example, depending on the hardware configuration, the drivers 2032 can include a display driver, a camera driver, a driver, a flash memory driver, a serial communication driver (e.g., a universal serial bus (USB) driver), a driver, an audio driver, a power management driver, and the like.

Libraries 2016 may provide a common infrastructure that may be utilized by applications 2020 and/or other components and/or layers. Libraries 2016 generally provide functionality that allows other software modules to perform tasks more easily than by directly interfacing with underlying operating system 2014 functionality (e.g., kernel 2028, services 2030, and/or drivers 2032). Libraries 2016 may include system 2034 libraries (e.g., C standard libraries) that may provide functionality such as memory allocation functions, string manipulation functions, mathematical functions, and the like. In addition, the libraries 2016 may include API libraries 2036, such as media libraries (e.g., libraries to support the rendering and manipulation of various media formats (e.g., Moving Picture Experts Group (MPEG) 4, H.264, MPEG-1 or MPEG-2 audio layers (MP3), AAC, AMR, Joint Photographic Experts Group (JPG), Portable Network Graphics (PNG)), graphics libraries (e.g., the Open Graphics Library (OpenGL) framework that can be used to render 2D and 3D graphics content on a display), database libraries (e.g., Structured Query Language (SQL) SQLite that can provide various relational database functions), website libraries (e.g., WebKit that can provide website browsing functionality), and the like. The libraries 2016 may also include a wide variety of other libraries 2038 to provide many other APIs 2036 to the applications 2020 and other software components/modules.

The framework 2018 (sometimes also referred to as middleware) can provide a higher-level common infrastructure that can be utilized by applications 2020 and/or other software components/modules. For example, the framework 2018 can provide various graphical user interface (GUI) functions, advanced resource management, advanced location services, etc. The framework 2018 can provide a wide range of other APIs 2036 that can be utilized by applications 2020 and/or other software components/modules, some of which may be specific to a particular operating system 2014 or platform.

Applications 2020 include built-in applications 2040 and/or third-party applications 2042. Examples of representative built-in applications 2040 may include, but are not limited to, a contacts application, a browser application, a book reader application, a location application, a media application, a messaging application, and/or a game application. Third-party applications 2042 may include any of the built-in applications as well as a broad category of other applications 2020. In a specific example, third-party applications 2042 (e.g., applications developed using an Android ^™ or iOS ^™ software development kit (SDK) by an entity other than the vendor of a particular platform) may be mobile software running on a mobile operating system 2014 (e.g., iOS ^™ , Android ^™ , phone) or other mobile operating system 2014. In this example, third-party applications 2042 may invoke API calls 2024 provided by the mobile operating system (e.g., operating system 2014) to facilitate the functionality described herein.

Applications 2020 may utilize built-in operating system functionality (e.g., kernel 2028, services 2030, and/or drivers 2032), libraries (e.g., system 2034, API 2036, and other libraries 2038), and framework/middleware 2018 to create a user interface for interacting with users 220 of the system. Alternatively or additionally, in some systems, interaction with users 220 may occur through a presentation layer (e.g., presentation layer 2044). In these systems, the application/module "logic" may be separated from the aspects of the application/module that interact with users 220.

Some software architectures 2002 utilize virtual machines. In the example of FIG. 19 , this is illustrated by virtual machine 2048. Virtual machine 2048 creates a software environment in which applications/modules can execute as if they were executed on a hardware machine (e.g., machine 2100 of FIG. 20 , for example). Virtual machine 2048 is hosted by a host operating system (operating system 2014 in FIG. 19 ) and typically, but not always, has a hypervisor 2046 that manages the operation of virtual machine 2048 and its interface with the host operating system (i.e., operating system 2014). Software architecture 2002 executes within virtual machine 2048, such as within operating system 2050, libraries 2052, framework/middleware 2054, applications 2056, and/or presentation layer 2058. These layers of software architecture 2002 executed within virtual machine 2048 may be the same as or different from the corresponding layers previously described.

Example machine architecture and machine-readable medium

FIG20 is a block diagram illustrating components of a machine 2100 capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and performing any one or more of the methodologies discussed herein, according to some example embodiments. Specifically, FIG20 shows a diagrammatic representation of a machine 2100 in the example form of a computer system, within which instructions 2116 (e.g., software, programs, applications, applet, app, or other executable code) may be executed for causing the machine 2100 to perform any one or more of the methodologies discussed herein. For example, the instructions 2116 may cause the machine 2100 to execute the flowchart of FIG6 . Additionally or alternatively, the instructions 2116 may implement the sensing engine 111, correlation engine 113, probability engine 115, and probability application 117 of FIG3 , among others, including implementing the modules, engines, and applications of FIG9-11 . The instructions 2116 transform a generic, unprogrammed machine 2100 into a specialized machine 2100 programmed to perform the described and illustrated functionality in the manner described. In alternative embodiments, the machine 2100 operates as a standalone device or may be coupled (e.g., using a network) to other machines 2100. In a networked deployment, the machine 2100 may operate as a server or a client machine in server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 2100 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook computer, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular phone, a smartphone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any other machine 2100 capable of executing instructions 2116, or otherwise, that specify actions to be taken by the machine 2100. Further, while a single machine 2100 is illustrated, the term "machine" shall also be taken to include any collection of machines 2100 that individually or jointly execute instructions 2116 to perform any one or more of the methodologies discussed herein.

The machine 2100 may include a processor 2110, memory 2130, and I/O components 2150 that may be configured to communicate with one another, for example, via a bus 2102. In an example embodiment, the processor 2110 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 2112 and a processor 2114 that may execute instructions 2116. The term "processor" is intended to include a multi-core processor 2112 that may include two or more independent processors 2112 (sometimes referred to as "cores") that may execute instructions 2116 concurrently. Although FIG. 20 shows multiple processors 2112, the machine 2100 may include a single processor 2112 with a single core, a single processor 2112 with multiple cores (e.g., a multi-core processor), multiple processors 2112 with a single core, multiple processors 2112 with multiple cores, or any combination thereof.

The memory/storage 2130 may include a memory 2132 (e.g., main memory or other memory storage device) and a storage unit 2136, both of which are accessible by the processor 2110, for example, via the bus 2102. The storage unit 2136 and the memory 2132 store instructions 2116, embodying any one or more of the methodologies or functions described herein. The instructions 2116 may also reside, completely or partially, within the memory 2132, within the storage unit 2136, within at least one of the processors 2110 (e.g., within a cache memory of the processor), or within any suitable combination thereof during instruction execution by the machine 2100. Thus, the memory 2132, the storage unit 2136, and the memory of the processor 2110 are examples of machine-readable media.

As used herein, a "machine-readable medium" means a device capable of temporarily or permanently storing instructions 2116 and data, and may include, but is not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage devices (e.g., erasable programmable read-only memory (EEPROM)), and/or any suitable combination thereof. The term "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database or associated cache memory and server) capable of storing instructions 2116. The term "machine-readable medium" should also be taken to include any medium or combination of media capable of storing instructions (e.g., instructions 2116) for execution by a machine (e.g., machine 2100), such that when executed by one or more processors (e.g., processor 2110) of machine 2100, the instructions 2116 cause the machine 2100 to perform any one or more of the methodologies described herein. Thus, a "machine-readable medium" refers to a single storage device or apparatus, as well as a "cloud-based" storage system or storage network comprising multiple storage devices or apparatuses. The term "machine-readable medium" itself excludes signals.

The I/O components 2150 may include a wide variety of components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, and the like. The specific I/O components 2150 included in a particular machine 2100 will depend on the type of machine. For example, a portable machine 2100 (e.g., a mobile phone) will likely include a touch input device or other such input mechanism, while a peripheralless server machine will likely not include such a touch input device. It will be appreciated that the I/O components 2150 may include many other components not shown in FIG. 20 . The I/O components 2150 are grouped according to functionality merely to simplify the following discussion and are not intended to be limiting. In various exemplary embodiments, the I/O components 2150 may include an output component 2152 and an input component 2154. Output components 2152 may include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), auditory components (e.g., a speaker), tactile components (e.g., a vibration motor, a resistive mechanism), other signal generators, etc. Input components 2154 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an opto-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, touch pad, trackball, joystick, motion sensor, or other pointing instrument), tactile input components (e.g., a physical button, a touch screen or other tactile input component that provides position and/or force or touch gestures of touch), audio input components (e.g., a microphone), and the like.

In other exemplary embodiments, the I/O components 2150 may include, among a wide array of other components, a biometric component 2156, a motion component 2158, an environmental component 2160, or a positioning component 2162. For example, the biometric component 2156 may include components for detecting expressions (e.g., hand gestures, facial expressions, vocal expressions, body postures, or eye tracking), measuring physiological signals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), and the like. The motion component 2158 may include an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), and the like. Environmental components 2160 may include, for example, an illumination sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an auditory sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor used for safety to detect concentrations of hazardous gases or to measure pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. Positioning components 2162 may include a position sensor component (e.g., a global positioning system (GPS) receiver component), an altitude sensor component (e.g., an altimeter or a barometer that detects air pressure, from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), and the like.

Communication can be implemented using a wide variety of technologies. I/O components 2150 may include communication components 2164 operable to couple machine 2100 to network 2180 or device 2170 via coupling 2182 and coupling 2172, respectively. For example, communication components 2164 may include network interface components or other suitable devices for interfacing with network 2180. In other examples, communication components 2164 may include wired communication components, wireless communication components, cellular communication components, near-field communication (NFC) components, components (e.g., low-energy), components, and other communication components for providing communication via other modalities. Device 2170 can be another machine 2100 or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a universal serial bus (USB)).

Furthermore, the communication component 2164 can detect an identifier or include a component operable to detect an identifier. For example, the communication component 2164 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes (e.g., Universal Product Code (UPC) barcodes), multi-dimensional barcodes (e.g., Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCCRSS-2D barcodes), and other optical codes), or an auditory detection component (e.g., a microphone for identifying tagged audio signals). Furthermore, various information can be derived via the communication component 2164, such as location derived via Internet Protocol (IP) geolocation, location derived via signal triangulation, location derived via detection of NFC beacon signals that can indicate a specific location, and the like.

Launch Media

In various exemplary embodiments, one or more portions of network 2180 may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet 80, a portion of the Internet 80, a portion of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a network, another type of network, or a combination of two or more such networks. For example, network 2180 or a portion of network 2180 may include a wireless or cellular network and coupling 2182 may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, coupling 2182 can implement any of a variety of types of data transmission technologies, such as single-carrier radio transmission technology (1xRTT), evolution-data optimized (EVDO) technology, general packet radio service (GPRS) technology, GSM enhanced data rates for evolution (EDGE) technology, the Third Generation Partnership Project (3GPP) (including 3G), fourth-generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), high-speed packet access (HSPA), worldwide interoperability for microwave access (WiMAX), long-term evolution (LTE) standards, other technologies defined by various standards-setting organizations, other long-range protocols, or other data transmission technologies.

Instructions 2116 may be transmitted or received over network 2180 using transmission media via a network interface device, such as that included in communication component 2164, and utilizing any of several well-known transfer protocols, such as the Hypertext Transfer Protocol (HTTP). Similarly, instructions 2116 may be transmitted or received to device 2170 using transmission media via coupling 2172, such as a peer-to-peer coupling. The term "transmission media" shall be taken to include any intangible medium capable of storing, encoding, or carrying instructions 2116 for execution by machine 2100, and includes digital or analog communication signals or other intangible media to facilitate communication of such software.

language

Throughout this specification, multiple examples may implement components, operations, or structures described as a single example. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed simultaneously, and the operations need not be performed in the order illustrated. Structures and functionality presented as separate components in the example configurations may be implemented as combined structures or components. Similarly, structures and functionality presented as single components may be implemented as separate components. These and other variations, modifications, additions, and improvements are within the scope of the subject matter herein.

Although an overview of the inventive subject matter has been described with reference to specific exemplary embodiments, various modifications and changes may be made to these embodiments without departing from the broader scope of the embodiments of the present invention. Such embodiments of the inventive subject matter may be referred to herein, individually or collectively, by the term "invention," which is merely for convenience and is not intended to automatically limit the scope of this application to any single invention or inventive concept when, in fact, more than one invention or inventive concept is disclosed.

The embodiments illustrated herein are described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived from the present invention, such that structural and logical substitutions and changes may be made without departing from the scope of the present invention. Therefore, the embodiments should not be construed in a limiting sense, and the scope of the various embodiments is defined solely by the appended claims, along with the full scope of equivalents to which such claims are entitled.

As used herein, the term "or" should be understood as inclusive or exclusive. In addition, multiple instances may be provided for resources, operations, or structures described herein as single instances. In addition, the boundaries between various resources, operations, modules, engines, and data stores are somewhat arbitrary, and specific operations are illustrated in the context of specific illustrative configurations. Other functional allocations are conceivable and may fall within the scope of various embodiments of the present invention. In general, structures and functionality presented as separate resources in an exemplary configuration may be implemented as combined structures or resources. Similarly, structures and functionality presented as single resources may be implemented as separate resources. These and other variations, modifications, additions, and improvements all fall within the scope of embodiments of the present invention as represented by the appended claims. Therefore, the description and drawings should be regarded as having illustrative rather than restrictive meanings.

Claims (20)

1. A method for generating rule elements for automatic probabilistic semantic sensing in a sensor network, the method comprising: receiving, by one or more processors of the machine, raw sensor data from a plurality of sensors; One or more processors of the machine generate a set of classifiers for the automatic probabilistic semantic sensing using semantic data based on the raw sensor data, the generated semantic data comprising a plurality of parallel sensed event records, each of the plurality of parallel sensed event records indicating a corresponding event sensed by a corresponding sensor among the plurality of sensors, each sensed event record comprising: a first classifier in the set of classifiers indicating an event detected by the corresponding sensor; a second classifier in the set of classifiers indicating a probability that the first classifier is true; a third classifier in the set of classifiers indicating a sensor location including spatial coordinates of the corresponding sensor; and a fourth classifier in the set of classifiers indicating an event location where the detected event occurred; grouping, by one or more processors of the machine, the parallel sensed event records into an intermediate group of sensed event records based on a subset of the first through fourth classifiers included in each of the sensed event records; generating, by one or more processors of the machine, a final derived event record based on a group of sensory event records selected from the intermediate group of sensory event records, the generated final derived event record comprising a fifth classifier comprising a time coordinate associated with the third classifier and indicating an event detected by a plurality of sensors of the plurality of sensors, and a sixth classifier indicating a probability that the fifth classifier is true; and One or more processors of the machine enable at least one application to perform a service or event sequence based on the generated final derived event record. 2 . The method of claim 1 , wherein the plurality of sensors comprises a first sensor located at a first node in an optical sensor network comprising a plurality of nodes. 3 . The method of claim 1 , wherein the raw sensor data comprises visual data, audio data, and environmental data, and wherein the events comprise detection of a person, detection of a vehicle, detection of other entities, and detection of an empty parking space. 4 . The method of claim 1 , wherein the third classifier comprises a spatial coordinate of a first sensor, and wherein the plurality of classifiers comprises a seventh classifier comprising a temporal coordinate associated with the third classifier. 5. The method of claim 4 , wherein the fourth classifier comprises spatial coordinates of a first event represented by the semantic data, and wherein the plurality of classifiers comprises an eighth classifier comprising a temporal coordinate describing a time at which the first sensor detected the first event represented by the semantic data, and wherein the first event is an empty occupancy state of a parking lot. 6. The method of claim 1, wherein the first plurality of classifiers includes a further classifier including an application identifier for identifying the at least one application from a plurality of applications, wherein each application from the plurality of applications is for performing a different service. 7. The method of claim 1, the grouping of the sensory event records comprising at least one of: correlation to generate an abstract graph based on matched classifiers, or correlation to generate an abstract graph based on fuzzy matched classifiers. 8. The method of claim 1 , wherein the grouping of the sensory event records comprises correlation based on a mathematical relationship between spatial coordinates and temporal coordinates, and wherein the spatial and temporal coordinates relate to at least one of a sensor itself, a detection of an event, and a combination of the two. 9. The method of claim 1 , wherein the plurality of sensed events include a first sensed event, a second sensed event, and a third sensed event, and wherein the first sensed event includes a corresponding first classifier describing an empty occupancy state of a first parking lot, the second sensed event includes a corresponding first classifier describing an empty occupancy state of the first parking lot, and the third sensed event includes a corresponding first classifier describing an empty occupancy state of the first parking lot. 10. A system for generating rule elements for automated probabilistic semantic sensing in a sensor network, the system comprising: a plurality of sensing engines implemented by one or more processors, the plurality of sensing engines being configured to generate a set of classifiers for the automatic probabilistic semantic sensing using raw sensor data received from a plurality of sensors, the plurality of sensing engines being further configured to generate semantic data based on the raw sensor data, the generated semantic data comprising a plurality of parallel sensed event records, each of the plurality of parallel sensed event records indicating a corresponding event sensed by a corresponding sensor of the plurality of sensors, each sensed event record comprising: a first classifier in the set of classifiers indicating an event detected by the corresponding sensor; a second classifier in the set of classifiers indicating a probability that the first classifier is true; a third classifier in the set of classifiers indicating a sensor location including spatial coordinates of the corresponding sensor; and a fourth classifier in the set of classifiers indicating an event location where the detected event occurred; and a correlation engine implemented by one or more processors, the correlation engine configured to group the parallel sensed event records into an intermediate group of sensed event records based on a subset of the first through fourth classifiers included in each of the sensed event records; and A probability engine implemented by one or more processors, the probability engine configured to generate a final derived event record based on a group of sensory event records selected from the group of sensory event records, the generated final derived event record including a fifth classifier including a time coordinate associated with the third classifier and indicating an event detected by a plurality of sensors among the plurality of sensors, and a sixth classifier indicating a probability that the fifth classifier is true, the probability engine further configured to enable at least one application to perform a service or event sequence based on the generated final derived event record. 11. The system of claim 10, wherein the plurality of sensors comprises a first sensor located at a first node in an optical sensor network comprising a plurality of nodes. 12. The system of claim 10, wherein the raw sensor data includes visual data, audio data, and environmental data, and wherein the events include detection of a person, detection of a vehicle, detection of other entities, and detection of an empty parking space. 13. The system of claim 10, wherein the third classifier comprises a spatial coordinate of a first sensor, and wherein the plurality of classifiers comprises a seventh classifier comprising a temporal coordinate associated with the third classifier. 14. The system of claim 13 , wherein the fourth classifier comprises spatial coordinates of a first event represented by the semantic data, and wherein the plurality of classifiers comprises an eighth classifier comprising a temporal coordinate describing a time at which the first sensor detected the first event represented by the semantic data, and wherein the first event is an empty occupancy state of a parking lot. 15. The system of claim 10, wherein the probability engine is configured to perform analysis based on user input, wherein the user input includes desired accuracy and user preferences. 16. The system of claim 10, wherein the probability engine is configured to perform analysis based on weights assigned to second classifiers, and wherein the probability engine changes the weights over time. 17. The system of claim 10, wherein the plurality of derived events includes a first derived event, and wherein the probability engine performs analysis based on a first threshold, and wherein the first threshold defines a minimum level of raw sensor data used to generate the first derived event. 18. The system of claim 17, wherein the probability engine changes the first threshold over time. 19. The system of claim 10, wherein the at least one application comprises a parking location application, a monitoring application, a traffic application, a retail customer application, a business intelligence application, an asset monitoring application, an environmental application, or a seismic sensing application. 20. A machine-readable medium having no transitory signals, the machine-readable medium storing a set of instructions that, when executed by a processor, causes the machine to perform operations in a method for generating rule elements for automated probabilistic semantic sensing in a sensor network, the operations comprising at least: receiving, by one or more processors of the machine, raw sensor data from a plurality of sensors; One or more processors of the machine generate a set of classifiers for the automatic probabilistic semantic sensing using semantic data based on the raw sensor data, the generated semantic data comprising a plurality of parallel sensed event records, each of the plurality of parallel sensed event records indicating a corresponding event sensed by a corresponding sensor among the plurality of sensors, each sensed event record comprising: a first classifier in the set of classifiers indicating an event detected by the corresponding sensor; a second classifier in the set of classifiers indicating a probability that the first classifier is true; a third classifier in the set of classifiers indicating a sensor location including spatial coordinates of the corresponding sensor; and a fourth classifier in the set of classifiers indicating an event location where the detected event occurred; grouping, by one or more processors of the machine, the parallel sensed event records into an intermediate group of sensed event records based on a subset of the first through fourth classifiers included in each of the sensed event records; generating, by one or more processors of the machine, a final derived event record based on a group of sensory event records selected from the intermediate group of sensory event records, the generated final derived event record comprising a fifth classifier comprising a time coordinate associated with the third classifier and indicating an event detected by a plurality of sensors of the plurality of sensors, and a sixth classifier indicating a probability that the fifth classifier is true; and One or more processors of the machine enable at least one application to perform a service or event sequence based on the generated final derived event.