WO2025064775A1 - Visual credential verification - Google Patents

Abstract

Systems and methods for adaptive visual credential generation and verification are provided. The system and methods herein can enhance access control and security measures by utilizing illuminated credential devices to identify individuals with the required access credentials to pass through credential verification checkpoints. The system and methods herein employ a combination of embedded electronics, a means of remotely controlling the illuminated credential devices, and identification technology as ascertain an individual's access status to the checkpoint. The system and methods herein can streamline access authorization, thereby improving overall security posture in various institutional and organizational settings where large numbers of people are entering a facility or attending an event and the individuals carry credentials, which are visually verified by a guard or greeter. Related apparatus, systems, techniques, and articles are also described.

Description

VISUAL CREDENTIAL VERIFICATION

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 63/584,545, filed on September 22, 2023, entitled “Visual Credential Verification,” the entirety of which is incorporated by reference herein.

TECHNICAL FIELD

[0002] The current subject matter is generally related to credential verification and access control. The subject matter described herein also relates to a personnel inspection system, which in some example implementations, can be capable of performing threat detection and discrimination without personal item divestment.

BACKGROUND

[0003] Credential verification is performed to verify credentials associated with an individual. Credential verification can be performed to ascertain a status of the individual in regard to accessing a location, an event, or a resource.

[0004] Credential verification can be performed in conjunction with personnel inspection systems configured to perform inspection of people and objects passing through inspection locations. For example, airport security attempts to prevent any threats or potentially dangerous situations from arising or entering the country. Some existing radio frequency (RF) imaging systems (such as those utilized by airport security for passenger screening) are large, expensive, and require individuals to remain stationary while an antenna rotates around the stationary individual to capture an image. In addition, these existing RF imaging systems can require divestment of personal items such as cell phones, keys, wallets, and the like, by the individual under inspection. Such divestment requirement can reduce throughput and usability of the imaging systems.

[0005] Some existing inspection systems, such as walkthrough metal detectors, can include coils to generate and measure changes in a magnetic field caused by magnetic or conductive materials (e.g., metallic) passing through the magnetic field. These existing inspection systems can be capable of measuring for metallic objects passing through a threshold but can lack any ability to distinguish personal items such as a cell phone, laptop, keys, belt buckle, and the like from threats, such as firearms or improvised explosive devices. Accordingly, these example existing inspection systems require divestment of personal items thereby limiting their throughput and usability.

SUMMARY

[0006] In one aspect, a method is provided and in one embodiment, the method can include determining, by a data processor, first data characterizing a display state of a credential device associated with an individual. The display state can identify an access status of the individual to access a credential verification checkpoint. The method can also include providing, by the data processor, second data to a transmitter communicably coupled to the data processor. The second data can characterize control signals controlling the display state of the credential device. The method can further include transmitting, by the transmitter, the second data to the credential device. Responsive to receiving the second data the credential device can display the display state on the credential device.

[0007] In some embodiments, the method further include receiving, by the data processor, sensor data corresponding to the display state displayed on the credential device as the individual accesses the credential verification checkpoint. The method can further include determining, by the data processor, the access status of the individual based on the display state. The method can also include providing, by the data processor, third data for display at the credential verification checkpoint, the third data identifying the access status of the individual.

[0008] In other embodiments, the sensor data can be acquired by an image sensor including at least one of a video camera, an electro-optical camera, a surface map camera, or a depth map camera. In some embodiments, the access status of the individual can be determined based on at least one of a computer vision algorithm and a manual input received from a personnel of the credential verification checkpoint. In some embodiments, the transmitter can be configured to transmit the second data to the credential device using radio frequency communication protocols or infrared communication protocols. In other embodiments, the display state can include a color selected from a plurality of colors. In some embodiments, the credential device can include at least one light emitting diode configured to illuminate based on the received second data. In other embodiments, the first data can be determined based on a user profile associated with the individual. The user profile can include at least one of a unique identifier of the credential device, one or more display states associated with the individual, and one or more access statuses associated with one or more credential verification checkpoints.

[0009] In some embodiments, the first data can be dynamically determined based on a predetermined schedule and/or in a trigger event. In other embodiments, the second data can be transmitted to the credential device based on the predetermined schedule and/or the trigger event, which once received by the credential device can cause the credential device to change the display state to a second display state.

[0010] In another aspect, a system is provided and in one embodiment, the system can include a computing device including at least one data processor and a memory containing non-transitory computer-executable instructions. The system can also include at least one image sensor communicatively coupled to the computing device. The system can also include at least one transmitter coupled to the at least one data processor. The at least one data processor can be configured to execute the instructions stored in the memory to perform operations including determining first data characterizing a display state of a credential device associated with an individual. The display state can identify an access status of the individual to access a credential verification checkpoint. The operations can also include providing second data to the at least one transmitter. The second data can characterize control signals controlling the display state of the credential device. Responsive to receiving the second data the transmitter can be configured to transmit the second data to the credential device, which responsive to receiving the second data can cause the credential device to display the display state on the credential device.

[0011] In some embodiments, the operations can further include receiving sensor data from the at least one sensor. The sensor data can correspond to the display state displayed on the credential device as the individual accesses the credential verification checkpoint. The operations can also include determining the access status of the individual based on the display state. The operations can further include providing third data for display at the credential verification checkpoint. The third data can identify the access status of the individual.

[0012] In some embodiments, the at least one image sensor can include a video camera, an electro-optical camera, a surface map camera, or a depth map camera. In some embodiments, the access status of the individual can be determined based on at least one of a computer vision algorithm and a manual input received from a personnel of the credential verification checkpoint. In other embodiments, the at least one transmitter can be configured to transmit the second data to the credential device using radio frequency communication protocols or infrared communication protocols. In some embodiments, the display state can include a color selected from a plurality of colors. In other embodiments, the credential device can include at least one light emitting diode configured to illuminate based on the received second data. In some embodiments, the first data can be determined based on a user profile associated with the individual. The user profile can include at least one of a unique identifier of the credential device, one or more display states associated with the individual, and one or more access statuses associated with one or more credential verification checkpoints.

[0013] In other embodiments, the first data can be dynamically determined based on a predetermined schedule and/or in a trigger event. In some embodiments, the second data can be transmitted to the credential device based on the predetermined schedule and/or the trigger event, which once received by the credential device can cause the credential device to change the display state to a second display state.

DESCRIPTION OF DRAWINGS

[0014] FIG. 1 is a system block diagram of an embodiment of a system for visual credential verification;

[0015] FIG. 2 is a process block diagram of an embodiment of a process for generating visual credentials for verification using the system of FIG. 1;

[0016] FIG. 3 is a process block diagram of an embodiment of a process for determining and providing an access status of an individual using the system of FIG. 1;

[0017] FIG. 4 is a system block diagram of an example inspection system that can be capable of performing threat detection and object discrimination without personal item divestment for use with the visual credential verification system of FIG. 1;

[0018] FIG. 5 is a diagram illustrating four example plots of transmitter spatial arrangements;

[0019] FIG. 6 is a diagram illustrating an arrangement of an example personnel inspection system according to some implementations; [0020] FIG. 7 is a process block diagram illustrating an example process for an example inspection system according to some aspects of the current subject matter;

[0021] FIG. 8 is a process block diagram illustrating an example process for determining a polarizability index of an object in an example inspection system according to some aspects of the current subject matter;

[0022] FIG. 9 is a process block diagram illustrating an example process for detecting objects on individuals within large groups of persons in an example inspection system according to some aspects of the current subject matter;

[0023] FIG. 10 is a diagram illustrating an exemplary implementation of a personnel inspection system;

[0024] FIG. 11 is a diagram illustrating an exemplary configuration of a personnel inspection system including a plurality of sensors and transmitters at different locations; and

[0025] FIG 12 is a diagram illustrating an exemplary configuration of a personnel inspection system including a plurality of cameras.

DETAILED DESCRIPTION

[0026] Credentialed events with large crowds, such as trade shows, conferences, sporting events, performing arts events, museums, or the like struggle to verify that all participants entering the event have the appropriate credentials, such as a ticket for the event. Forging credentials by taking a picture of an official credential, or a screenshot of a phone-based credential has become commonplace. These types of events frequently have a large influx of visitors passing through a security checkpoint upon opening, or after break periods.

Verifying that all participants have paid for, have verified access to, or are otherwise qualified to attend the event has multiple benefits, such as enhanced event security, threat detection, improved attendee experience, and efficient operation of security checkpoints by event personnel.

[0027] Traditional access control systems rely on various authentication methods, including biometrics, key cards, radio frequency identification (RFID), or personal identification number (PIN)s. However, these methods often suffer from inefficiencies, impersonation risks, and delays in granting access. Techniques using short range radio signals, such as RFID, near-field communications (NFC), Bluetooth, or the like, can remotely interrogate credentials, but can fail in a high flow environments where many attendees are attempting to access an event and need to perform credential verification. In the case of a large crowd or high visitor flows, RFID or other wireless ID systems can be limited in their inability to isolate individuals attempting to enter the venue without appropriate credentials. If a group of 10 individuals enters the venue, but the RFID reader sees only 9 badges or credentials, it is then incumbent upon the greeter or guard of the credential verification system to stop the entire group of 10 and interrogate each of their badges individually. The credential verification system described herein can address these limitations by providing a visually intuitive and efficient approach to access control through the implementation of interactive color-coded ID badges.

[0028] The credential verification system and method of use described herein can improve access to and security of events or resources by utilizing colored lights on ID badges to identify individuals with the required access credentials. The system can employ a combination of embedded electronics, a means of remotely controlling colored lights on visitor credentials, and individual identification techniques. A credential device, such as a badge, name tag, or the like, carried by an individual approaching a security or access checkpoint can be configured to display a particular color via a light emitting device of the credential device. The color displayed on the credential device can change periodically and each color can be associated with a unique credential status or permission level. Personnel operating the checkpoint can visualize the color on the credential device and based on the color, determine that the individual possesses a verified credential or appropriate security status to pass through the checkpoint based on authorization data associating the displayed color with a credentialed or authorized status. In some embodiments, a sensor can be configured to acquire sensor data associated with the color displayed by the credential device and the sensor data can be compared to the authorization data. Based on matching the displayed color of the credential device with a color identified in the authorization data, the system can determine that the individual possesses a credentialed or authorized status to pass through the checkpoint.

[0029] The system and methods herein can streamline access authorization, thereby improving overall security posture in various institutional and organizational settings where large numbers of people are entering a facility or event and need to carry appropriate credentials which are visually verified by a guard or greeter. The use of colored lights on ID badges can provide an additional layer of security by providing a quick and easy visual verification of access credentials. The color-coding system can simplify the access control process for both users and security personnel, reducing the chances of human error during authentication. The system herein can utilize an authorization database to allow for instant updates to access privileges, ensuring immediate changes in an individual's permissions are reflected across all access control readers. Compared to complex biometric systems or key card technologies, the credential verification system described herein can provide a cost- effective solution for credential verification and can enhance access control and security in various settings. By providing a visually intuitive and efficient means of verifying access credentials, the system herein can reduce unauthorized access, streamline security checkpoint operations, and increase overall security measures within organizations.

[0030] FIG. 1 illustrates an embodiment of a visual credential verification system (VCVS) 100 configured to perform visual credential verification according to methods described herein. As shown in FIG. 1, an individual can approach a credential verification checkpoint 108. The checkpoint 108 can be an access point through which the individual can pass after the individual’s credentials are verified by checkpoint personnel or automated verification systems present at the checkpoint 108. The individual 105 can have a credential device 106 on their person, such as an ID badge, or the like. The credential device 106 can include an embedded electronic module with one or more color (e.g., RGB) lights, such as one or more light-emitting diodes (LED). The displayed color of the LEDs on the device 106 can change on a semi-regular basis to avoid spoofing and can also represent different access levels or permissions. The device 106 can be tamper-resistant and equipped with a unique identifier to ensure authenticity and prevent duplication.

[0031] The device 106 can include a small, lightweight form factor, such as an ID badge, fob, or the like. Device 106 can include a power source, such as a replaceable or rechargeable battery, a data processor, and a memory storing computer-readable instructions, which when executed by the data processor cause the LEDs to illuminate a color associated with a control signal received by the device 106. The device 106 can also include one or more LEDs configured to illuminate in a plurality of colors, and a communication transceiver configured receive control signals causing the device 106 to illuminate different colors via the LEDs. The communication transceiver can be configured to communicate or exchange data via radio frequency (RF), infrared (IR), Bluetooth®, or NFC communication protocols with the computing device 120, for example with the device control module 145.

[0032] The checkpoint 108 can include a transmitter 110, a sensor 112 and a computing device 120. The transmitter 110 can generate control signals 111 that can be provided to the credential device 106 to cause the credential device 106 to display one or more colors. The colors can correspond to a display state of the device 106 that can be associated with an access status of an individual accessing the checkpoint 108. For example, the transmitter 110 can transmit control signals 111 causing the device 106 to illuminate a red color. A red color or display state can be associated with an unauthorized access state of an individual 105 attempting to pass through the checkpoint 108 and the individual 105 may not be permitted to pass through the checkpoint 108. The transmitter 110 can transmit control signals 111 causing the device 106 to illuminate a green color. A green color or display state can be associated with an authorized access state of the individual 105 attempting to pass through the checkpoint 108 and the individual 105 may be permitted to pass through the checkpoint 108. The display states can be provided by the device 106 as a consistently-illumination pattern, a variable illumination pattern (e.g., blinking or flashing periodically), or a combination of thereof.

[0033] The credential device 106 can allow access control based on display states that are associated with a “color of the moment”. As visitors proceed toward the checkpoint 108, the device 106 can receive control signals 111 that instruct the device 106 to display a specific color. The checkpoint personnel or guard has a display 150 that can show them the color of the moment. The role of the checkpoint personnel is to visually verify that every individual 105 passing through the checkpoint 108 has a device 106 that displays the appropriate color as a display state. In some embodiments, an automated timer can change the “color of the moment” randomly. When the color is changed, the instructions 111 are broadcast to all of the approaching individuals 105, at which time all of their device 106 will change to the new color of the moment. Transmitters 110 can be strategically positioned at checkpoints 108 as well as entry points, restricted areas, or the like and can be integrated into the security screening technology being used at that entrance.

[0034] The sensor 112 can be configured to capture sensor data 113 associated with the individual 105 or the credential device 106. For example, the sensor data 113 can include image data of the credential device 106, such as a color being displayed on the credential device 106. In some embodiments, additional sensors 112 can be configured to confirm the identity of the individual 105 or to perform multi-factor authentication of the individual 105. The sensors 112 can include, but are not limited to, an infrared (IR) camera, thermal camera, ultrasonic distance sensor, video camera, electro-optical (EO) camera, surface/depth map camera, and/or a radio frequency identification (RFID) reader. In some implementations, sensor 112 transmits the sensor data 113 to processing system 120 for further analysis.

[0035] The VCVS 100 can also include a processing system 120 configured to generate the control signals 111 and process the sensor data 113 in order to verify the credentials of the individual 105. The processing system 120 can include a processor 125, a memory 130, a verification module 135, a communications module 140, a device control module 145, a display 150, an input device 155, and a speaker 160. The processing system 120 can be communicatively coupled to a database 165. In some embodiments, the database 165 can be configured within the computing device 120, such as within the memory 130. The database 165 can be a secure and centralized authorization database that stores the necessary information about authorized individuals 105, their respective access levels, and the corresponding color codes associated with their device 106. The database 165 can be regularly updated to accommodate changes in access privileges for individuals 105. In some embodiments, the database 165 can store data associated with an access status of the individual 105, which can include but are not limited to an individual’s ticket status (e.g., paid, VIP, handicap, or associated with particular locations in a venue). The access status of the individual can also include a security clearance, an employment status, a visitor status, or the like.

[0036] The processor 125 can be configured to execute instructions stored in the memory 120 (and/or the database 165) verify the credentials of one or more individuals 105. Responsive to executing the instructions, the processor 125 can cause the verification module 135 to determine and provide display states or color of devices 106 that are associated with authorized and unauthorized access statuses of individuals. The verification module 135 can query the database 165 and/or the memory 130 to determine the display state or color to be provided to the devices 106 via the communication module 140. The verification module 135 can also provide past, current, and upcoming display states for display via the display 150 so that checkpoint personnels can ascertain the current display state or color when individuals present themselves at the checkpoint 108. In some embodiments, the verification module 135 can process sensor data 113 to evaluate a display state of a device 106 of an individual 105 passing through the checkpoint with respect to the current, authorized color or display state. In this way, the verification module 135 can facilitate credential verification manually by checkpoint personnel or programmatically. Advantageously, this can allow large numbers of individuals 105 to have credentials verified more easily when passing through a checkpoint 108, without requiring screening on an individual basis, which can slow progression through checkpoints.

[0037] The processing system 120 can also include a communication module 140, a device control module 145, a display 150, an input device 155, and a speaker 160. The communication module 140 can be configured to cause the transmitter 110 to transmit control signals 111 to the device 106 to cause the device 106 to change display states from one color to a different color. The device control module 145 can be configured to determine an illumination state of the device 106 and to cause the communication module 140 to transmit control signals 111 to the transmitter 110 for provision to the device 106. The device control module 145 can also be configured to control aspects of the illumination of the device 106, such as patterns of illumination (e.g., flashing, pulsing, steady-state, color changes, etc.). The device control module 145 can also determine and provide control signals 111 associated with a frequency, timing, or duration of the illumination of the device 106. The device control module 145 can be further configured to provide control signals to the verification module 135 (and the display 150) associated with a current verified access color. For example, the verification module 135 can receive control signals from the device control module 145 to cause the display 150 to display the color displayed on devices 106 for which access through the checkpoint 108 is currently permitted. The verification module 135 can also cause the display 150 to generate visible alerts or notifications responsive to determining a device 106 displaying a particular color has successfully passed through the checkpoint 108 (e.g., successful credential verification) or that the device 106 displaying a particular color is not permitted to pass through the checkpoint 108 (e.g., unsuccessful credential verification). In some embodiments, the computing device 120 can include a speaker 160 and the verification module 135 can cause the speaker 160 to generate audible alerts or notifications responsive to determining a device 106 displaying a particular color has successfully passed through the checkpoint 108 (e.g., successful credential verification) or that the device 106 displaying a particular color is not permitted to pass through the checkpoint 108 (e.g., unsuccessful credential verification). [0038] Users can interact with the processing system 120 via input device 120 to provide various inputs associated with credential verification performed via the system 100. For example, checkpoint personnel may provide inputs via device 120 to cause the processing system 120 to determine a new color to be displayed on device 106 for accessing the checkpoint 108. Such inputs can be ad hoc trigger events by which the processing system 120 can generate control signals to be transmitted by the transmitter 112 for provision to the devices 106, which upon receiving the control signals 111 can cause the device 106 to change display states from a first color to a second color.

[0039] In some embodiments, the transmitter 110, sensor 112, and the database 165 can be coupled to the processing system 120 via a network. The network can include, for example, any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. Further, the network can include, but is not limited to, any one or more of the following network topologies, including a bus network, a star network, a ring network, a mesh network, a star-bus network, tree or hierarchical network, and the like.

[0040] In some embodiments, the VC VS 100 can be incorporated into or otherwise communicably coupled to an inspection system configured to inspection objects and personnel such as embodiments of the inspection system 400 described in relation to FIGS. 4- 12. For example, a guard or an inspection system personnel monitoring the inspection system 400 can view individuals 105 and their device 106 as they pass through the inspection system 400. The VCVS 100 can be configured within or operatively coupled to the inspection system 400 so that a color indicator, for example a multi-color LED can be viewable to the guard (but not to the individual 105). Thus, the guard can view the “reference” color on the multi-color LED so that individuals 105 are screened for proper credential verification as well as object and personnel inspection. All individuals 105 should be displaying the same reference color on their device 106. As the guard watches for potential weapons alerts via the inspection system 400, the color of devices 106 can also be monitored via the VCVS 100 to ensure the devices 106 are displaying the determined color or display state associated with a verified access status.

[0041] FIG. 2 illustrates an embodiment of a process 200 for performing credential verification using the VCVS 100 of FIG. 1. At 210, the data processor 125 can cause the verification module 135 to determine first data characterizing a display state of a credential device 106 associated with an individual 105. The display state can identify an access status (e.g., an authorized or unauthorized access status) of the individual to access the credential verification checkpoint 108. The verification module 135 can determine the first data by querying the database 165 and/or the memory 130 periodically or responsive to an input to ascertain a display state of the devices 106. At 220, the data processor 125 can cause the communication module 140 to provide second data 111 to a transmitter 110 communicably coupled to the data processor 125. The second data 111 can include control signals controlling the display state of the credential device 106. In some embodiments, the second data 111 can be broadcast on a periodic basis or schedule and in other embodiments, the second data 111 can be broadcast in an ad hoc manner, such as when triggered or otherwise instructed by checkpoint personnel. At 230, the transmitter 110 can transmit the second data to the credential device 106. Upon receiving the second data 111, the credential device 106 can display the display state or color identified in the second data 111 via one or more LEDs configured within the credential device 106. As the individuals 105 approaches the credential verification checkpoint 108, the credential device 106 can display the display state or color associated with the second data and the checkpoint personnel (or the sensor 112) can visually identify the display state and determine an access status of the individual 105 as being authorized or unauthorized to pass through the credential verification checkpoint 108.

[0042] FIG. 3 illustrates an embodiment of a process 300 for determining and providing an access status of an individual using the VCVS 100 of FIG. 1. At 310, the data processor 125 can receive sensor data 113 from sensor 112. The sensor data 113 can correspond to a display state displayed on the credential device 106 as an individual 105 accesses the credential verification checkpoint 108.

[0043] At 320, the data processor 125 can determine an access status of the individual 105 based on the display state of the device 106. For example, the data processor 125 can determine the access status of the individual 105 via a computer vision algorithm configured within the verification module 135. The verification module 135 can determine a display color of the device 106 by processing the sensor data 113 using the computer vision algorithm and can compare the determined color of the device 106 in the sensor data to a color corresponding to an authorized access status, which can be stored in the database 165 and/or the memory 130. Based on the comparing, the data processor 125 can determine whether or not the color displayed on the device 106 is associated with an authorized or unauthorized access status. If the color displayed on the device 106 as captured in the sensor data 113 matches a color associated with an authorized access status, the data processor can determine that the individual is authorized to pass through the credential verification checkpoint 108. If the color displayed on the device 106 as captured in the sensor data 113 does not match a color associated with an authorized access status, the data processor can determine that the individual is not authorized to pass through the credential verification checkpoint 108.

[0044] In some embodiments, the data processor 125 can additionally, or alternatively, receive an input via the input device 155 from a guard or personnel of the credential verification checkpoint 108. For example, in some embodiments, the guard or personnel of the credential verification checkpoint 108 can visually observe the color of the device 106 in the sensor data 113 (or without viewing the sensor data 113, but instead by viewing the device 106 themselves) and can provide a manual input to the input device 155 indicating a match between the color of the device 106 and a color associated with an authorized access status.

[0045] At 330, the data processor can provide third data for display, via a display 150 at the credential verification checkpoint 108. The third data can identify the access status of the individual 105. For example, responsive to matching the color displayed on the device 106 as captured in the sensor data 113 to a color associated with an authorized access status, the data processor can cause third data to be displayed indicating that the individual is permitted to pass through the credential verification checkpoint 108. The third data can be a displayed color, a textual alert, or a combination thereof. For example, the third data can include a green icon or the like to indicate the individual has an authorized access status. A red icon or the like can indicate the individual has an unauthorized access status. In some embodiments, the third data can further include an audible tone emitted from the speaker 160.

[0046] Exemplary technical effects of the subject matter described herein include the ability to adaptively modify credentials required for verification of an individual at a security checkpoint. The credentials can be adapted with easily visible colors to improve credential determination and verification by personnel and/or sensors at the security checkpoint. The credentials can be adapted in an ad hoc or predetermined manner to provide increased customization of credential verification in high-flow environments. In this way, personnel and/or credential verification systems can more readily review and determine credentials of individuals in large groups, which can facilitate more rapid verification of individuals and faster access for a larger number of individuals compared to traditional systems with static credential management.

[0047] In some embodiments, the VCVS 100 can be configured within or communicably coupled to a personnel inspection system configured to perform object discrimination and person identification as will be described further herein. Personnel inspection systems are used to detect threats which can be introduced to particular area the inspection system seeks to protect. A common personnel inspection system can include, for example, a metal detector configured at an entrance to a courthouse or a stadium, or a body scanner at airport. These inspection systems are configured to generate data that can be processed to determine the presence or absence of a threat. A threat can be considered any individual, object or element passing through the system, which if allowed to enter the protected area can cause damage, introduce security concerns, and/or disrupt events or activities occurring in the protected area. For example, a firearm is a threat which personnel inspection systems seek to detect at airports or stadiums. Typically, personnel inspection systems are configured to detect threats which include metallic objects.

[0048] Traditional personnel inspection systems have a number of drawbacks. Personnel inspections systems are commonly configured to scan or evaluate data associated with a single individual at a time, such as a queue of individuals at a security area of the airport. Each individual must be scanned or processed before another individual can be scanned or processed for threat detection, which can result in delays and long wait times to enter the protected area. Individuals commonly experience elevated levels of anxiety and distrust when being evaluated for the presence of potential threats in traditional personnel inspection systems.

[0049] Traditional personnel inspection systems also require individuals to remove all potential clutter objects, such as any metal objects, prior to entering the area at which the inspection system is deployed. In this way, traditional inspection systems can broadly identify a potential threat as any detected metal object that may remain on an individual passing through the inspection system without discriminating for the size, type, or composition of the object. Using this type of broad, binary discrimination threshold can result in large rates of false alarms and require individuals to undergo subsequent inspection processing to clear objects that were inaccurately identified as threat objects. For example, traditional inspection systems cannot typically discern the object and material properties of a belt buckle uniquely from those of a firearm. In traditional inspection systems, both objects are equally detected and characterized as potential threats, yet the belt buckle poses much less of a threat, or even no threat, compared to the firearm. Individuals typically have a number of metal objects on their body which may be falsely identified as threats in traditional inspection systems. Shoe or boot grommets, belt buckles, glasses, as well as cell phones, laptops, hearing aids, and pacemakers all include metal which traditional inspection systems may falsely identify as detected threats. As a result, personnel inspection systems require all potential threat objects to be removed or divested from the individual passing through the inspection system.

[0050] Additionally, traditional inspection systems are not configured to utilize spectrum optimization for threat detection using extremely low frequency (ELF) radio waves. Low frequency radio waves, e.g. 3-10 kHz, have been used to detect concealed metal objects and, to a certain extent, can be used to discriminate between different metal types and shapes, ostensibly for the goal of detecting potential weapons like firearms and knives. However, eddy currents tend to dominate in this part of the spectrum for typical objects of concern like firearms and cellphones. Thus, while discrimination is theoretically possible, it can be very challenging when the highly conductive components of consumer electronics can give signals of much greater magnitude than the less conductive and often more magnetic components of weapons such as firearms and knives. However, there still exists a large degree of freedom in selecting operating frequencies in this band, with opportunities to further optimize for particular objects of interest.

[0051] The current subject matter can include an improved personnel inspection system, which in some example implementations, can be capable of performing threat detection and discrimination in high clutter environments in which individuals may be carrying personal items such as cell phones and laptops and without personal item divestment. In some implementations, a personnel inspection system can perform threat detection and discrimination with high throughput that allows individuals to pass through the detector at normal walking speeds such that individuals are not required to slow down for inspection and, in some implementations, the inspection threshold can allow for multiple individuals to pass through the threshold side-by-side (e.g., two or more abreast). [0052] The current subject matter can also enable threat detection and discrimination using a spectrum that is optimized for a lower part of the spectrum, sub-1 kHz, to de-emphasize the contribution from conduction relative to magnetism, which could be discriminated through the characterization of objects’ effective magnetic polarizability tensor.

[0053] Advantages of this improved personnel inspection system can include higher throughput of individuals being evaluated, reduced incidence of false alarms due to more accurate discrimination of metal objects as threats or non-threats, and reduced stress levels and improved emotional response for individuals being evaluated using the improved personnel inspection system. In addition, the improved personnel inspection system can more accurately distinguish metal objects present on an individual passing through the improved inspection system as threats or non-threats without requiring the individual to remove the metal object from their body. The data that is collected, processed, and generated by the improved inspection system can also be used within the context of other security- focused operations such as notification to system operators of individuals who are in possession of a detected threat object, training exercises for inspection system operators or supervisors, as well overall process improvement of security procedures which may occur prior to or after individuals are screened or evaluated using the improved inspection system.

[0054] Some example implementations of the improved inspection system disclosed herein can include a continuous-wave magnetic detection system of high sensitivity, capable of detecting disturbances in its transmitted field of up to one part in 10,000. To facilitate this sensitivity, the system can be configured to transmit a stable magnetic field and to measure the transmitted magnetic field using a low-noise method, as magnetic disturbances caused by unintentional system noise can be very difficult to distinguish from magnetic disturbances caused by metallic objects. In this context, system noise can encompass a number of signal interferences, including traditional electronic noise, amplitude variations in the transmitted magnetic field, and/or digital error, which can be introduced by harmonic mismatches between intentional signals and sampling rates associated with analog to digital conversion.

[0055] Some example implementations can include an active magnetic system that can acquire a series of magnetic field measurements of an observational domain; determine in- phase and quadrature components of the magnetic field measurements; determine a measure of polarizability (e.g., a polarizability tensor, polarizability index) of an object in the observational domain; localize the object including determining speed, position, and time offset of the object; and perform threat detection and/or discrimination of the object in the presence of clutter using the magnetic field measurements, the polarizability, and/or the localization information. In some implementations, the system can be configured to detect for firearms and/or improvised explosive devices (lEDs).

[0056] In some implementations, the system can determine a polarizability of objects under inspection and can perform threat detection and discrimination (e.g., classification) using the polarizability of the objects. By determining and utilizing the polarizability of objects, certain threats, such as firearms and improvised explosive devices, can be more accurately detected, resulting in improved personnel inspection systems.

[0057] In personnel inspection systems configured with magnetic-field sensing to detect illegal or threat objects, such as firearms, the frequency band that is used to optimize the signal-to-noise (SNR) ratio is not the same as the frequency band that is used to optimize discrimination between threat and non-threat objects. In addition, one or both of the optimized frequency bands can be too low to be effectively detected by the receivers of the system. A challenge in personnel inspection systems configured with magnetic-field sensing can include determining how best to measure for detected threat objects at the appropriate frequency band while retaining the benefit provided by both optimized frequency bands.

[0058] The improved inspection system described herein can be configured to interrogate an object in such a way that it gets information in a first frequency band with good SNR properties, as well as a second frequency band with good discrimination properties, the first frequency band being distinct from the second frequency band. The use of fluxgates, either in addition to or instead of induction-coil receivers, can allow the system disclosed herein to measure very low frequency (e.g., sub 1 kHz) magnetic fields. By using frequency band measurements from the first frequency band with good SNR properties to determine certain properties of the object (such as its location, speed, orientation, or the like), before recovering additional properties of the object via the second frequency band with good discrimination properties. Some example implementations of the magnetic sensing algorithm and the system described herein can exploit the benefits of both frequency bands simultaneously.

[0059] For example, many consumer electronics contain non-ferrous metals like aluminum and copper, while many firearms contain ferrous metals like steel. Due to the fact that eddy currents scale with frequency, the maximum distinction between ferrous and non-ferrous metals can be achieved by exciting an object with magnetic fields at very low frequencies (sub 1 kHz). Such low frequencies can be efficiently measured using fluxgate receivers as compared to induction-coils, which are simpler and easier to build at higher frequencies. However, fluxgate receivers (and their digitizing electronics) often have noise characteristics that get better at higher frequencies. The system and magnetic sensing algorithm described herein utilize frequency measurements at low frequencies (e.g., frequencies ~30 Hz) and at higher frequencies (e.g., -230 Hz). For example, the system and magnetic sensing algorithm described herein can utilize frequency measurements between about 1-5 Hz, 1-50 Hz, 50-100 Hz, and between and between about 100 and 1000 Hz to simultaneously exploit enhanced discrimination characteristics of low frequencies and the superior SNR of the high frequencies to improve metal object detection and discrimination. Accordingly, in some implementations, the system can operate at frequencies below 1 kilo hertz (Hz) in order to improve performance of detecting and discriminating firearms in the presence of common personal items such as cell phones. At relatively lower frequencies (under 1 kHz, for example), magnetic contributions to the magnitude of polarizability can dominate over conductive contributions to the magnitude of polarizability. As a result, the signal magnitude may be driven less by the total metallic content of a threat than by the material that is unique to the characteristics of many threats and absent from typical consumer electronics.

[0060] The above description of the processing performed by the inspection system and magnetic sensing algorithm described herein can be further considered with regard to a single object passing through the inspection system. The single object will have some properties that vary with frequency, such as material properties and/or magnetic dipole moments and/or polarizability tensor elements and will have some properties that are shared across the frequency bands such as location, orientation, and speed. The inspection system and magnetic sensing algorithm described herein provide enhanced detection capabilities by using measurements for determining the frequency-invariant properties of the object, and using the good discrimination properties primarily for classification.

[0061] In some implementations, the magnetic sensing algorithm described herein is configured to initially perform a retrieval operation at a frequency band with favorable SNR characteristics to solve for location, orientation, and speed, and subsequently retrieves an object signature, such as material properties, magnetic dipole moments, and/or a polarizability tensor or index of the object at the other frequency band. Information determined from the first retrieval operation can be used to constrain the second retrieval and improve the overall accuracy of detection.

[0062] In other implementations of the magnetic sensing approach described herein, the magnetic sensing algorithm can retrieve all object properties in a single step by using a weighted cost function that evaluates the higher frequencies more closely for the frequencyinvariant properties. In both implementations, the properties of the object at the more discriminating frequency band will be recovered with greater fidelity than if they had been recovered independently. Subsequently, these properties can be used in a classification step that decides if the object belongs to a particular category, such as firearm or consumer electronic device.

[0063] FIG. 4 is a system block diagram of an example inspection system 400 that can be capable of performing threat detection and discrimination without personal item divestment and can be configured for use with the visual credential verification system (VCVS) 100 described in relation to FIGS. 1-3.

[0064] As shown in FIG. 4, the system 400 includes magnetic receivers 405 coupled to a data acquisition base station 415. The data acquisition base station 415 can be configured to filter, demodulate, and digitize the magnetic field measurement data received from the receivers 405. The transmitters 406 and magnetic receivers 405 can be arranged to probe an observational domain (OD) 407 through which an individual 105 may pass. In some embodiments, the individual 105 can be carrying an object 109, such as a computing device or a concealed firearm. The OD 407 can be sometimes referred to as a “scene”, such as a threshold or other defined region. The OD 407 can be considered to include voxels defining a volume of space through which the individual 105 and/or the object 109 traverses. The OD 407 can be a single continuous region or multiple separate regions. The system 400 also include transmitters 406 coupled to a transmission driver 460. The transmission driver 460 can be configured to generate a signal to drive transmitters 406. The system 400 also includes a processing system 420 configured to analyze the received magnetic field measurements. The processing system 420 includes a data acquisition module 430, a calibration module 435, a reconstruction module 440, an automatic threat recognition module 445, a rendering module 450, and a memory 455. The system 400 can also include a display 465 for providing output; and a sensor 425 to provide additional inputs to the system 400. [0065] In some implementations, the system can be configured to operate as a distributed lock-in amplifier, utilizing a synchronous homodyne digital dual-phase demodulation technique to accurately extract in-phase (I) and quadrature (Q) information from the system’s specific transmitted frequencies. Demodulation can be achieved by digitally mixing or multiplying the desired signal with a reference signal and subsequently filtering the result using a low-pass filter. The reference signal can be a directly measured signal related to or derived from the driving signal used in the transmitter, or can be a synthetic analogue. By utilizing two versions of the reference signal, one phase shifted or time-delayed from the other, the amplitude, phase, and/or I and Q of the measured signal can be reconstructed.

[0066] In some implementations, the transmission driver 460 can include a combined set of digitally-controlled high-accuracy direct digital synthesis (DDS) waveform generators, a digitally-controlled summing programmable-gain amplifier (PGA) circuit, and a closed-loop class-D power amplifier with enhanced power supply rejection ratio (PSRR). Such a system provides flexibility in the frequency and amplitude of the transmitted waveforms while achieving high stability in the transmitted magnetic fields required to meet necessary signal- to-noise ratios in the measured data. The system 400 can digitally control an amplitude and a frequency of the transmitted magnetic fields. Digitally controlling the amplitude and the frequency of the transmitted magnetic field can be performed in a dynamic manner and in an arbitrary or ad-hoc manner. In some implementations, the system can include a closed-loop microcontroller-based feedback system configured to measure and dynamically adjust the per-frequency amplitude of the transmitted field thereby increasing the stability and predictability of the system.

[0067] Transmitters 406 can include at least two wire-loop transmitters capable of generating a magnetic field according to a driving signal having an operating (e.g., characteristic) frequency (e.g., a modulation frequency). The transmitters 406 can operate at 30 Hz and 130 Hz, for example. In other embodiments, the transmitters 406 can operate at higher or lower frequencies, such as frequencies less than 50 Hz, frequencies between 100 Hz and 200 Hz, and frequencies above 200 Hz, such as frequencies between 200 Hz and 1000 Hz. In general, a wire-loop can be considered to reside within a primary plane. In some implementations, the system 400 can include transmitters arranged to deliver fields with sufficient diversity to probe all cardinal directions (e.g., cartesian coordinates) throughout the OD 407. In a static system where objects under inspection are stationary, at least three transmitters can be included that are either oriented orthogonally (e.g., the primary plane of each of the three transmitters can be oriented orthogonal to one another), or else offset in space. If the object is undergoing motion in a particular direction, as in an object passing through the inspection system 400, two transmitters can be used if they are oriented orthogonal to the direction of motion or spatially offset transverse to both the direction of motion and their shared orientation. This configuration represents a reasonable constraint on object motion (e.g., in one direction) and can further represent the fewest number of transmitter coils capable of achieving sufficient field diversity to fully probe a given object.

[0068] As shown in FIG. 4, the transmitter driver 460 can generate one or more signals for driving the transmitters 406. In some implementations, the transmitters 406 can be driven by cycling through the transmitters in time, driving one, then another, until all desired measurements are captured. A benefit of such approach can include that the drive electronics can be shared across all of the transmitters 406. However, this approach can impose a dutycycle on each transmitter 406, reducing its signal-to-noise ratio. In such a configuration, the transmitters 406 may not be measured at the same instant in time, which, if the object is in motion, may introduce motion-induced artifacts.

[0069] In some implementations, the transmitters 406 can be driven simultaneously, but at slightly (e.g., 10 Hertz (Hz)) offset frequencies. The frequencies can be offset enough such that they can be distinctly demodulated in post-processing, which can be set by the bandwidth necessary to resolve the object’s motion, which can be about 5-10 Hz for objects moving at typical walking speeds of 1.3 meters per second (m/s). At the same time, the frequencies can be chosen to be similar enough that dispersion in the polarizability is negligible. In some implementations, the offset can be 10 Hz, which can be considered a negligible difference at all but vanishing frequencies. In this example frequency multiplexing approach, the transmit driver 460 can include separate drive electronics to drive each transmitter separately, which can enable improved signal-to-noise ratios without (and/or reducing) the risk of motion blur.

[0070] In some implementations, the transmission driver 460 can be capable of generating driving signals that can be distributed to transmitters 406, which can establish a fully phase coherent measurement system across all receive-transmit pairs. In addition, the driving signal can be provided as a reference signal routed from the transmitter driver 460 to the data acquisition base station 415, which can be utilized for demodulation, as described more fully below. [0071] The magnetic receivers 405 can include flux gate sensors, which can directly measure the magnetic field (e.g., magnitude and phase) as compared to wire coils, which measure a rate of change of magnetic field. In some embodiments, one or more of the receivers 405 can include 3-axis flux gate magnetometers. In some embodiments, one or more of the receivers 405 can include 2-axis flux gate magnetometers. Flux gate magnetometers can be advantageous in that they can operate with high sensitivity, high linearity and a low noise floor as compared to coil receivers. The receivers 405 can provide accurate magnetic measurement at frequencies too low for traditional methods.

[0072] A flux gate sensor can measure the amplitude of a magnetic field in three axis (e.g., x, y, and z) at the location of the flux gate sensor. A flux gate sensor can include a sense coil surrounding an inner drive coil that is closely wound around a highly permeable core material, such as mu-metal. An alternating current can be applied to the drive winding, which can drive the core in a continuous repeating cycle of saturation and unsaturation. In the presence of an external magnetic field, with the core in a highly permeable state, such a field is locally attracted or gated through the sense winding. This continuous gating of the external field in and out of the sense winding induces a signal in the sense winding, whose principal frequency is twice that of the drive frequency, and whose strength and phase orientation vary directly with the external field magnitude and polarity.

[0073] In some implementations, flux gate sensors can be utilized with operating frequencies below 1 kHz, such as 130 Hz and 30 Hz. At these relatively low operating frequencies, flux gate sensors can operate with improved noise-floors, for example, some flux-gates can achieve a volt-to-field ratio on an order of 20 micro- Volts / nano-Tesla.

[0074] Data acquisition base station 415 can demodulate, filter, and digitize data received from receivers 405. The data acquisition base station 415 can aggregate the received data, determine in-phase and quadrature data (I and Q data, respectively) from the received and aggregated digitized data, and transmit the aggregated data as in-phase and quadrature data to processing system 420. Filtration and amplification of the raw magnetometer signals provided to the data acquisition module 430 allows the system to achieve high dynamic range in frequencies of interest, e.g., frequencies below 1 kHz, such as 130 Hz and 30 Hz, by rejecting large ambient direct current (DC) magnetic signals. The bandwidth and design of the filters used in the hardware and/or the software of the system 400 can be selected to reject unwanted signals in the environment, such as 50 and 60 Hz signals generated by alternating current (AC) lines, while maintaining sufficient bandwidth in the demodulated signal to recover the motion of the object.

[0075] Sensor 425 can include an infrared (IR) camera, thermal camera, ultrasonic distance sensor, video camera, electro-optical (EO) camera, and/or surface/depth map camera. Sensor 425 creates an additional information image or video, such as an optical image, of at least the OD 407. In some implementations, sensor 425 transmits images or video to processing system 420 for further analysis. System 400 can include multiple sensors 425. Sensor 425 can also be used to detect for the presence of a target in the OD 407. Detecting the presence of a target in the OD 407 can be used to trigger scanning by the system 400. In some implementations, sensor 425 can include a radio frequency identification (RFID) reader.

[0076] The system can also present an image to an operator via display 465 in which the visible portion of the visitor and/or their belongings most likely to contain the object(s) is segmented, highlighted, or otherwise made to provide notice to an operator and aid in the operator’s response. In addition, aspects of the object can be determined based on the images obtained from the depth camera. The obtained aspects can be associated with classification of the object. For example, if the object is in plain view, the magnetic sensing algorithm can determine the object class, such as determining that the object is a laptop or an umbrella. If the object is concealed, the magnetic sensing algorithm can determine a part of the person’s body or a location on the person where the object is concealed, such as a pocket of the person’s clothing, an ankle or wrist of the person, or a bag that the person may be carrying. Data associated with these locations can be combined with information derived from the magnetic field data in a classification step that uses all available information to achieve greater predictive accuracy during threat detection.

[0077] Processing system 420 includes a number of modules for processing magnetic field data and additional information images from sensor 425 of the OD 407 including data acquisition module 430, calibration module 435, reconstruction module 440, automatic threat recognition module 445, rendering module 450, and a memory 455.

[0078] Data acquisition module 430 acquires a time-series of voltage measurements which represent magnetic field measurements from the DAS base station 415 and additional information images from the sensor 425. In some implementations, the sampling rate of the data acquisition module 430 is derived from the same master clock used to generate the transmitted fields via the transmitters 406. For each receiver 405, data acquisition module 430 derives I and Q data from this time-series in post-processing via demodulation with an accompanying reference signal. Timing of the I and Q data can be synchronized across receivers 405 and data acquisition module 430 can publish the synchronized data as frames (e.g., time slices) for further analysis by system 400.

[0079] In some implementations, the master clock of the system 400 can be distributed across multiple meters of space in the system, using an internal network of low-jitter low- skew clock fanouts and low voltage differential signaling (LVDS) converters. This configuration can enable a sampling rate to be an integer harmonic of every transmitted frequency, eliminating digitization errors which otherwise damage the sensitivity of the system. By configuring each device in the data acquisition process 430 on the same clock domain, receivers 405, which can be located meters apart, can be correctly assumed to be receiving samples at the same time intervals, with no drift due to frequency mismatching. Thus, for a given frame, data acquisition module 430 publishes a set of data for each receiver 405 and sensor 425. In some implementations, data can be acquired and frames can be published at a rate sufficient to resolve the carrier frequencies.

[0080] In some implementations, data acquisition module 430 removes the static background signal (e.g., the primary field). In some implementations, the data acquisition base station 415 can remove the static background signal (e.g., the primary field) such that the I and Q data characterizes the secondary field and not the primary field.

[0081] Calibration module 435 applies calibration correction to the published data. Calibration corrections can include compensating the published data for serial time- sampling. In addition, calibration module 435 can compare measured primary fields to one or more field model predictions, and compensate for any differences. In some implementations, calibration can account for amplitude and phase changes of the transmitters that occur due to normal wear and tear, manufacturing variations, or temperature changes.

[0082] Reconstruction module 440 transforms the calibrated data into images and/or feature maps. An image can be created for each receiver 405, and/or based on a composite of measurements obtained by multiple receivers 405. The reconstruction module 440 can include determining the polarizability measure (e.g., tensor) and localization of an object. [0083] Polarizability can be characterized as a proportionality constant relating an object’s far-field response to a primary filed that induced it. It can have units of volume, and can depend on the shape, permeability, and conductivity of the object, as well as the frequency of the applied field. In order to determine the polarizability, in some implementations, a best-fit algorithm can be utilized to implement a minimum residual matched filter.

[0084] The transmitter fields can be calculated from models of rectangular coils. The receiver fields can be calculated from dipole fields along the particular axis of the sensor, such that a 3-axis receiver node is treated like 3 independent and orthogonal dipoles.

[0085] In some implementations, image data from the sensor 425 can be used to further enforce the sparsity constraint beyond that supplied by a-priori knowledge of items or subjects that may occupy the OD 407. Specifically, an image of the OD 407 acquired by sensor 425 can be used to determine a spatial location of the target (e.g., which voxels of the OD 407 the target resides in and which voxels of the OD 407 are empty). Empty voxels contain no objects and therefore can be considered zero for compressed sensing (e.g., enabling better and/or quicker estimations of the solution to the underdetermined linear system).

[0086] In addition, an appropriately sized OD 407 can result in a scene that is sufficiently sparse for compressed sensing. For example, if an OD 407 is a volume that is 2 meters by 1 meter by 0.5 meters, and is divided into 8,000,000 voxels of 5 mm, a typical human located within this OD 407 would occupy only about 10% of the voxels at any moment (e.g., approximately 800,000 voxels). A retrieved set of polarizable objects from a sensor 425 can be used to determine three-dimensional surfaces within the OD 407 volume and consequently which voxels the individual resides in. The empty voxels can be forced to zeros when retrieving the set of polarizable objects while non-zeroed voxels can be altered during reconstruction (e.g., can be considered variables to find an optimal reconstructed solution to the underdetermined linear system).

[0087] Reconstruction module 440 can reconstruct one or more magnetic retrieved set of polarizable objects. In addition, reconstruction module 440 can create aggregate retrieved set of polarizable objects by combining multiple independent retrieved sets of polarizable objects. In some implementations, reconstruction module 440 can treat all receivers 405 as one large sparse aperture and reconstruct a single retrieved set of polarizable objects using the information acquired from all receivers 405 in the single aperture.

[0088] Reconstruction module 440 can perform localization of the object using multiple time-slices. Such an approach can use a single model-fitting approach that solves for the object’s location (e.g., x, y, and t-crossing), speed, and polarizability tensor. An example localization approach is described more fully below.

[0089] Reconstruction module 440 can generate feature maps from the reconstructed images. Feature maps can include characterizations or features of the magnetic measurements. Statistical analysis can be performed across multiple images. Some example features include field magnitude, field phase, and polarizability tensor properties (discussed further below). Other features are possible.

[0090] Automatic threat recognition module 445 analyzes the images and/or feature maps for presence of threat objects. Threat objects can include dangerous items that an individual may conceal on their person, for example, firearms and explosives. Automatic threat recognition module 445 may identify threats using, for example, a classifier that assesses the feature maps generated by reconstruction module 440. The classifier may train on known threat features. In some implementations, the threat recognition process can compare the determined images to a library of predetermined polarizability signatures.

[0091] In some implementations, features (e.g., classification variables) can include field magnitude, phase, and polarizability tensor properties at one or more operating frequencies.

[0092] Rendering module 450 generates or renders an image characterizing the outcome of the threat recognition analysis performed by the threat recognition module 445. The image can be rendered on display 465. For example, rendering module 450 can illustrate an avatar of a scanned person and any identified threats. Rendering module 450 can illustrate a characterization that automatic threat recognition module 445 did not detect any threats.

[0093] FIG. 5 is a diagram illustrating four example plots of transmitter spatial arrangements. In plot 505, three transmitters are arranged such that they are oriented orthogonally to each other and thus the configuration of 505 is capable of measuring all dimensions of a static (e.g., stationary) object. In plot 510, four transmitters are arranged such that they are oriented in the same plane but are offset in space and thus capable of measuring all dimensions of a static object. In plot 515, two transmitters are arranged orthogonal to one another and thus capable of measuring all dimensions of an object in motion. Similarly, in plot5220, two transmitters are arranged such that they are oriented in the same plane but are offset in space and thus capable of measuring all dimensions of an object in motion. As discussed in more detail below, in some implementations, system 400 can include transmitters 406 arranged according to the configuration illustrated in plot 520. Such a configuration can provide, in some implementations, a desirable form factor and reduced cost.

[0094] FIG. 6 is a diagram illustrating an arrangement of an example personnel inspection system 600 according to some implementations. Two transmitters 406 are arranged such that they are oriented in the same plane but are offset in space and thus capable of measuring all dimensions of an object in motion. In the example, the transmitters 406 are coils capable of transmitting between 1 and 1,000 Hz, and can be configured to transmit multiple signals offset in frequency so as to operate simultaneously. For example, the transmitters 406 can operate at 30 Hz and 130 Hz. Other offset frequencies are possible, for example, 5-10 Hz. In some implementations, the transmitters 406 can operate at (e.g., be driven at) a first frequency less than 50 Hz, a second frequency between 100 Hz and 200 Hz, and a third frequency between 200 Hz and 1000 Hz.

[0095] The use of low frequencies, e.g., frequencies less than 50 Hz, can provide the greatest SNR as the primary discriminator between objects of ferrous materials (e.g., steel) and objects of non-ferrous materials (e.g., aluminum or copper). For example, at such a low frequency, a firearm composed of a steel slide/barrel can produce a much greater overall signal strength than a laptop containing an aluminum plate, despite the laptop being much larger in size than the firearm. This would not be the case at 3-10 kHz, where it would be difficult to create an algorithm that could reliably differentiate between a laptop in isolation and a laptop with a firearm placed near it. For these reasons, this tone can be relied on for determining the location/speed/orientation of an object, as it will be most likely to accurately retrieve the critical steel components for further characterization.

[0096] Mid-range frequencies (e.g., frequencies between 100 Hz and 200 Hz) and high- range frequencies (e.g., frequencies between 200 Hz and 1000 Hz) can be used to estimate relative conductivities of objects which can have a similar size and magnetization. For example, an eyeglass case be made of thin steel. A sub-compact firearm can be made of 1 thicker steel. Both can be magnetic with similar outer dimensions, and thus may look similar at very low frequencies where magnetism dominates the response. By including higher frequencies, the greater conduction of the thick steel in the firearm will become measurable relative to the eyeglass case as a distinct change in phase in the magnetic polarizability components. Multiple tones are helpful so that similar comparisons can be made for objects of various sizes, since object size is a key factor in controlling the frequency at which the conductive properties become significant.

[0097] Considerations for selecting frequencies can include that lower frequencies increase the relative magnitude of ferrous compared to non-ferrous metals, higher frequencies can have better noise floors for certain fluxgate sensors, and particular frequencies (e.g., 50 and 60 Hz, harmonics) can be avoided entirely due to interference in typical environments. Receivers 405 are arranged vertically on posts (e.g., vertical poles) on either side of the transmitter to provide for dual lanes to allow individuals under inspection to pass through the system.

[0098] In order to recover a magnetic signature of a detected object independent of the location and orientation of the object, it can be desirable to collect measurements of the object while it is being exposed to magnetic fields which are transmitted into the object from orthogonal directions. By arranging the configuration of the transmitters 406 shown in FIG.

6 in an orthogonal manner, the total number of coils and overall complexity of the inspection system can be reduced, which can reduce overall operating and maintenance costs as well as the size of the physical footprint of the inspections system.

[0099] The configuration of transmitters 406 shown in FIG. 6, can enable some example systems disclosed herein to collect measurements of an object while it is interrogated by magnetic fields in largely orthogonal directions so that a magnetic signature can be determined that is largely independent of location and orientation of the object. At the same time, the total number of coils and complexity of the system can be minimized for the sake of cost and physical footprint. Field diversity can be achieved in the system with a minimal configuration of transmitters, for example two transmitters, by exploiting the motion of the object past the transmitters. In addition, the system complexity can be further reduced by powering the two transmitters simultaneously via orthogonal time-varying patterns, such as two sinusoids with slightly offset frequencies. The frequencies can be similar enough that dispersion in the polarizability index is negligible. [0100] In some embodiments, the configuration of the transmitters 406 can provide simultaneous passive magnetic field detection to assist object discrimination based on a manufacturing process by which the object was manufactured. Passive magnetic field detection can be used to retrieve an effective magnetic moment, but passive magnetic field detection can perform worse than active magnetic field detection. Active magnetic field detection can be used to retrieve an effective magnetic polarizability tensor as a primary feature for threat/benign object discrimination. Passive magnetic field detection can aid object detection as a secondary feature for objects with similar signal strengths and signatures characterized by active magnetic field detection, but which are manufactured using different manufacturing processes.

[0101] For example, consider a steel plate or frame used in some cellphone designs compared to the steel blade of some knives. These can have similar overall dimensions and materials, and therefore appear similar in their magnetic polarizability tensors. However, knives are more likely to be constructed to have specific mechanical properties, which implies greater force and uniformity during construction, which can lead to more magnetic domains aligning during manufacturing and a larger overall magnetic moment. Passive magnetic field detection can recognize the greater remnant magnetization and differentiate the knife from the cellphone on this basis.

[0102] To estimate the passive magnetic moment of the object (e.g., a “hard” magnetic component) in the presence of earth’s magnetic field, which can induce an additional magnetic moment (e.g., a “soft” magnetic component), the recovered magnetic polarizability tensor at a low frequency could be extrapolated to 0 Hz and can be used, in combination with the known earth’s magnetic field at a location in which the system is deployed, to estimate the induced component (e.g., the “soft” magnetic component) of the magnetic moment and to subtract this from the overall retrieved magnetic moment to recover the desired (e.g., the “hard”) magnetic component.

[0103] The relative orientations between the passive magnetic moment and magnetic polarizability tensor may also prove to be a helpful signature. For example, if the magnetic moment is coming from the same object as the polarizability tensor, then alignment is likely, whereas if the magnetic moment is dominated by some small permanent magnet, like the clasp on some tablet covers, it’s orientation may not be strictly related to the simultaneously recovered magnetic polarizability tensor. [0104] FIG. 7 is a process block diagram illustrating an example process 700 for an example inspection system 400 according to some aspects of the current subject matter.

[0105] At 705, the processing system 420 can receive data characterizing samples obtained by a plurality of magnetic field receivers 405. For example, the magnetic field samples can be acquired by receivers 405 and images (e.g., video) can be acquired by a camera or other sensor 425. An event (e.g., identifying that a person is approaching and/or entered the observational domain) can be identified. For example, an event can be identified based on event data signal received from a photocell testing occupancy of the system or motion in a field of view of the camera feeds. The received event data can cause the system to initiate object detection, or alternatively the system can be configured to search for an object in the absence of event data.

[0106] At 710, the data acquisition module 430 aggregates an amount of magnetic field data for processing by subsequent components. For example, the data acquisition module 430 can transmit a copy of a circular buffer containing magnetic field time samples from a plurality of receivers in the system. The calibration module 435 can be configured to account for deviations in the magnetic field data, as compared to a pre-prepared model including amplitude, phase, or environmental characteristics. The reconstruction module 440 can be configured to determine a best-fit object or objects which can be defined by a set of attributes including position, speed, time-offset, and polarizability index. Best fit can be determined by a cost function that measures the difference between the actual measurements and those predicted by a model (e.g., residual), and can also include other attributes that may suggest the plausibility of a solution such as the isotropy of the polarizability index. The polarizability index can include a complex tensor including 1-6 unique elements characterizing directional polarizability components of the object at one or more frequencies transmitted by the transmitters 406. These attributes, including the polarizability index, can be determined simultaneously or in series using an optimization routine, such as a gradient descent algorithm or a nested parameter search. The model can be, for example, an isotropic or an anisotropic model, with uniform or non-uniform motion through the scan zone. In some embodiments, features derived from the polarizability index of the object can characterize a shape, a permeability, and a conductivity of the object within a unit of volume.

[0107] At 715, the reconstruction module 440 can localize the object within a volume under inspection. For example, the volume under inspection can include a volume of 3D space arranged relative to the plurality of magnetic field receivers 405 and/or the transmitters 406. The reconstruction module 440 can be configured to perform the localization, which can include determining an object speed, an object position, and an object time-offset relative to a predetermined plane. The predetermined plane can be configured relative to the plurality of magnetic field receivers 405 and/or the transmitters 406, such as a plane associated with a threshold through which a user and an object must pass for personnel inspection and threat detection according to the subject matter described herein.

[0108] At 720, the automatic threat recognition module 445 can classify the object as a threat or non-threat. A threat/non-threat decision can be based on physical attributes, such as matching the polarizability index or a subset of its components against known polarizability index examples from previously determined threat objects, such as a gun barrel or a knife. Alternatively, a threat/non-threat decision can use a classifier trained by in a machine learning process, in which many labeled examples of threats and non-threats are used to train the magnetic sensing algorithm to determine if a newly detected object should be characterized as a threat or non-threat. The classification can include making threat / nonthreat determinations using the frequency and the shape information.

[0109] Finding multiple objects in the same domain (e.g., either spatially or temporally) can be indicative of a detection event associated with the visitor passing through the inspection system disclosed herein. The magnetic sensing algorithm can combine these objects for classification by an aggregate classifier that is capable of making additional use of the combined information, as compared to classifiers that run on each object independently. For example, the classifier can add some properties of the neighboring objects together, as if they constitute an underlying large and/or distributed object that is best considered as a whole.

[0110] At 725, the classification can be provided. The classification can be provided, for example, via a display such as display 465. In some implementations, the classification can be provided to a backend security management system that can coordinate multiple assets (e.g., screening or inspection devices). In some implementations, the classification can be stored in memory 455 of the processing system 420. In some implementations, the classification can be stored in a database configured within the inspection system to store the polarizability index associated with determined threats and determined non-threats. [0111] At 730, the system 400 can repeat the steps of process 700 in an iterative manner.

[0112] FIG. 8 is a process block diagram illustrating an example process 800 for determining a polarizability index of an object in an example system 400 according to some aspects of the current subject matter. In some implementations, a set of magnetic field samples can be provided as an input to the process 800. The set of magnetic field samples can be collected simultaneously while probing an object with a set of transmitted fields. The process 800 can include introducing a set of trial solutions that can be solved independently via pseudo-inverse, from which the trial solution with the smallest residual can be selected. The process 800 can be performed to determine isotropic and anisotropic polarizability indexes. The process 800 can be extended to include samples collected over time while an object experiences motion relative to the system 400.

[0113] At step 805, the automatic threat recognition module 445 can define a set of trial solutions. A series of trial solutions can be indexed to include magnetic field samples associated with the possible locations in which the object may be present. For each location, the automatic threat recognition module 445 can determine a corresponding polarizability index based on computing a transfer matrix and a pseudo-inverse for each of the magnetic field samples to determine the set of trial solutions. In some implementations, the transfer matrixes and the pseudo-inverses can be pre-determined and stored in memory 455.

[0114] At step 810, the automatic threat recognition module 445 can calculate an associated polarizability index and an associated residual for each trial solution. For example, isotropic polarizability indexes can be determined based on selecting the trial solution which best fits the magnetic field samples. When determining an anisotropic polarizability index, the polarizability index can be considered as a complex symmetric tensor defined by 6 unique polarizability elements. The pseudo-inverse can be applied and the polarizability index associated with each of the 6 unique polarizability elements can be computed.

[0115] At step 815, the automatic threat recognition module 445 can select a final trial solution. The final trial solution can be selected as the trial solution with the minimum or lowest residual. During operation, the automatic threat recognition module 445 can capture the magnetic field samples in real-time. The set of pseudo-inverses can be applied via matrix multiplication. Subsequently, the transfer matrixes can be applied to compute the residuals. The trial solution with the lowest or minimum residual can then be selected. [0116] At 820, the automatic threat recognition module 445 can repeat the steps of process

800 in an iterative manner.

[0117] In some embodiments, the threat recognition module 445 can utilize a predictive model, trained in a machine learning process, to estimate a confidence score or a goodness- of-fit of trial solutions as part of determining the most likely object parameter set (i.e., location, speed, timing, and polarizability tensor). The machine learning process can be configured to make use of the information in each frequency band in accordance with real objects in real-world settings. Thus, the machine learning process can be configured to train the predictive model for each frequency band.

[0118] In some embodiments, the machine learning process can be configured to generate a predictive model configured to generate a residual function for the trial solutions based on inputs associated with observed object properties and a location, speed, and a time-shift of a trial solution. In some embodiments, the predictive model can be a convolutional neural network (CNN). The predictive model can generate an output characterizing a distance between the observed object properties and a location, speed, and a time-shift of a trial solution. The distance can correlate to the confidence score or a goodness-of-fit of the trial solution.

[0119] For example, assume a known object traveling through the system with known properties y_a, z_a, t_a, v_a, a_a (y coordinate, z coordinate, timepoint, and instantaneous speed when breaking the x=0 plane, and polarizability tensor). The set of all of these actual properties can be p_a. Since the system 400 described herein is known, we can calculate the time-series signal contributed by this object at the z transmitter 406, j receiver 405, .s' frequency, and call it g_tjS, which includes all post-processing, such as demodulation, calibration, and filtering. The data recorded by the system 400 also consists of other contributions, such as noise, interference, and clutter, which can be denoted by some timeseries term <pij_S, such that the full measurement is g j _s = g ij_s + <pij_S.

[0120] The confidence measure or goodness-of-fit of a trial solution can be characterized by y_t, z_t, t_t, v_t, from which an a_t can be retrieved. The set of trial properties can be p_t, and a contribution 9^ _tj-s can be calculated. During inference, the confidence measure or goodness-of-fit estimate will know the measurement g^ and the calculated contribution g^ ij_s, but won’t know gtj_s or <ptj_S. Therefore, a residual function that takes the recorded measurement (g^s) and trial measurement (g^ .._s) as input can be generated that has the following properties:

some reasonable definition of “distance” d between two property sets.

[0124] Supervised regression training can be used to create a residual function from a convolutional neural network. A labeled dataset can be constructed such that for each entry in the database, such as an object with properties drawn from a distribution for each property, a model of noise and/or clutter drawn from a distribution of noise and/or clutter models, and a trial solution can be drawn from a neighborhood around the actual object. g_tjS and

_tj-s can be determined as the model inputs and a unitless distance d(p_a, p_t) can be the output of the regression model. The inputs can be high-dimensional data structures (e.g., [g] = [Samples, Transmitters, Nodes, Towers, Axes, Frequencies]) and a similarly high-dimensional convolutional neural network can be trained to exploit the relationships between neighboring entries along each dimension. Useful information can be observed in the relationship between the data from neighboring time samples, neighboring nodes in a tower, neighboring frequencies, etc. In one embodiment, the distance can be defined as:

[0125] some

characteristic length selected to appropriately weigh the spatial and polarizability contributions to distance.

[0126] A number of alternative embodiments corresponding to augmentations of the data can be envisioned. For example, real-world scans can be used to provide the noise/clutter <pij_S, and the simulated object can be superimposed on the real scan data. Real- world collected weapon data can be used. However, since values of y/z/t/speed may not be known, an alternate definition of distance can be utilized that only uses the polarizability tensor. Real- world collected weapon data can be used in isolation and the recovered values can be treated as ground truth (including values of y/z/t/speed) and the real- world collected weapon data can be subsequently superimposed on field data to randomize the noise/clutter. Advantageously, in this embodiment, where both the object and noise/clutter data are real- world collected data, the combinatorics are large and controllable, and the ground-truth can be reasonably estimated.

[0127] Additionally, a number of alternative embodiments corresponding to augmentations of the predictive model and/or the machine learning process can be envisioned. For example, in addition to a residual, the CNN regression can estimate Ay, Az, At, Av, as an error in each of the independent trial solution parameters. In this way, a much faster nested retrieval algorithm can be considered. For example, starting with an initial best-guess (which could be seeded by a total search of a low-resolution parameter grid), the estimated error could be used to seed the next guess iteratively until a residual minimum is found.

[0128] In other embodiments, the definition of distance d could be altered. For example, certain elements of the polarizability tensor can be weighted as more important than other parts. Alternatively, the distance d can include the difference in the outputs of the Automated Threat Detection (ATD) model(s) run on both parameter sets. In this way, certain variations in the polarizability tensor might be less impactful on ATD’s behavior than others, and therefore less critical when considering the goodness-of-fit of a solution.

[0129] A residual function trained in this way can advantageously outperform a naive residual function (such as root-mean-squared-error) in a number of areas. For example, the residual function trained in the aforementioned manner can learn to recognize and suppress typical noise/clutter contributions in relation to real signals. Additionally, the residual function trained in the aforementioned manner can exploit the relational information in neighboring timepoints, nodes, axes, and frequencies. While many different machine learning architectures can accomplish the former, the latter is a motivating factor to use a CNN. As a result, the system 400 described herein can more accurately distinguish between two objects that are in moderate proximity to one another, whereas a naive residual function will prefer a solution that attempts to explain both objects simultaneously. The residual function of the CNN described herein can explicitly penalize such solutions in favor of those that fit first one object and then the next. [0130] FIG. 9 is a process block diagram illustrating an example process 900 for detecting objects on individuals within large groups of persons using an example inspection system 400 according to some aspects of the current subject matter. The inspection system described herein can also enable accurate detection of metal objects on persons walking together in groups and can be configured to detect metal objects on large numbers of persons at a time as compared to traditional systems which are limited to scanning one person passing through the inspection system at a time. Because large groups of people assembled in an unorganized manner can make it difficult for an inspection system to determine when to start and/or stop a scan, the magnetic sensing algorithm used in the system described herein can dynamically adapt to conditions where many objects may be sensed simultaneously and the objects travel through the system in an overlapping fashion in time which can result in conditions where there is not a clear scan start or stop.

[0131] At step 905, the processing system 420 can process sensor data received from sensor 425 in a streaming manner via the magnetic sensing algorithm, such that the magnetic sensing algorithm can find one or more objects whenever the object(s) happened to pass through the system and recognizes when insufficient data has been collected in regard to an object(s). The processing system 420 can determine that a sufficient amount of data has been collected based on the goodness of fit of the model to the real data, or by the proximity of the found object to the end of the data buffer (e.g., present time). If the processing system 420 determines that insufficient data is found, nothing is done, and the magnetic sensing algorithm simply executes again at the next available moment or after an allotted amount of additional data has been collected.

[0132] At step 910, the processing system 420 can then store and account for found objects to avoid detecting the same object over and over again. For example, the modeled signals associated with the found object(s) can be recomputed and can be subtracted from the measured data before searching for the next object. The magnetic sensing algorithm can also be biased to search for objects which are located away from the previously found objects, for example, by modifying the cost function in the reconstruction module 440 to penalize objects found in close vicinity with already stored objects

[0133] The processing system 420 and the magnetic sensing algorithm can be configured to execute at regular intervals, such that an amount of elapsed time from when sufficient data has been collected on an object and when the magnetic sensing algorithm is executed to find that object can be minimized. The magnetic sensing algorithm can detect the object to be found within a time period that extends slightly into the future, e.g., beyond the time point at which the object data was collected and processed so far (if this best fits the available data), and uses this scenario as criteria for knowing when to wait for more data.

[0134] At steps 915, the magnetic sensing algorithm of the processing system 420 can re- estimate background noise in subsequent executions by opportunistically processing a retained buffer to determine “quiet times” which can be defined as time-periods where the processed signals vary little relative to neighboring time-periods. The “quiet time” data can be used as a proxy for a condition in which no object is present near the sensors. A different optimal set of “quiet time” samples can be found for every sensor, accounting for the conditions when an object(s) may be near some sensors but not near other sensors at different points in time. When an object is found, it can be stored in a buffer or a memory 455 of processing system 420. This buffer data can be processed in the next iterative execution of the magnetic sensing algorithm, which subtracts all previously found objects from the available data so as to avoid contamination/distortion in the present object search.

[0135] FIG. 10 is a diagram illustrating an exemplary implementation of a personnel inspection system 1000 according to the subject matter disclosed herein. The system 1000 includes similar components performing similar functionality as the components of the system 400 described in relation to the discussion of FIGS. 4-6.

[0136] Traditional personnel inspection systems commonly include an archway through which an individual must pass for threat evaluation and detection. In these traditional inspection systems the archway or some other overhead member can carry wiring conveying power and data signals between one or more components of the inspection system, ensure alignment of sensors configured on or within components of the inspection system, and to add stability to the overall structure of the inspection system with respect to environmental conditions such as wind, or uneven mounting surfaces. However, traditional inspection systems that include archways or other overhead elements are often aesthetically unappealing and can create a sense of distrust, anxiety, and claustrophobia for individuals passing through the archway.

[0137] The improved inspection system described in relation to FIG. 10 provides the advantages of conveying power and data signals to various inspection system components, ensuring alignment of the inspection system components and reducing emotional response for individuals without requiring them to pass through an archway or overhead element of an inspection system.

[0138] A further consideration that the improved personnel inspection system disclosed herein addresses is the non-uniformity of the operating environments in which the system may be located. Operating environments can include hard, yet somewhat smooth surfaces, such as a tile floor, an asphalt surface, or a concrete floor, as well as softer surfaces which can be non-uniform, such as a sand covered surface at a beach entrance or the surface of a grassy field at an entrance to an outdoor music festival.

[0139] The system 1000 disclosed herein can be deployed in a variety of different locations, indoor and outdoor, and can be configurable based on different kinds of surfaces on which the system may be positioned. For example, the base plate 1010 can be configured as a universal base plate to which a variety of modular mounting systems can be attached depending on the venue at which the system is located and/or the surface upon which the system 1000 is located. The modular mounting systems can enable positioning, leveling, and coupling or adherence to the surface of the location at which the system is deployed for operation. For example, the base plate 1010 can be configured with suction cups on the bottom side (the side facing the surface of the location at which the system is deployed) to secure the base plate 1010 to hard surfaces, such as tile or marble. In some implementations, the base plate 1010 can include gripping or piercing mechanisms capable of securing the base plate 1010 to soft surfaces, such as carpet. In some implementations, the base plate 1010 can include screw or auger- like mechanisms to couple the base plate 1010 to dirt or terrestrial surfaces. In some implantations, the base plate 1010 can be configured with a base plate frame that is hidden within the base plate 1010 and provide for permanent installation of the system.

[0140] Conventional security systems require visitors to pass through a portal for detection where an archway is used to carry data signals and power to/from each part of the system, to ensure proper alignment of the various sensors, and to add stability to the overall structure. Improper alignment can generate unwanted biases in the system’s performance, and motion of the archway structure, for example in environments which may include high wind. Improper alignments can cause unwanted distortions to be generated which are difficult to separate from the desired signals. Archways, however, are visually naesthetic and can create a sense of distrust and unpleasantness for visitors passing through them.

[0141] As shown in FIG. 10, the system 1000 disclosed herein can be configured to provide proper location of the posts 1005 which can comprise the receivers 405 described in relation to FIGS. 4 and 6, and to provide proper orientation/leveling of the posts 1005. In some implementations, a base plate 1010 can be configured to receive the posts 1005 and ensure proper alignment of the posts at appropriate and predetermined locations. In some implementations, the system 1000 shown in FIG. 10 can include inclinometers configured to instruct an operator in a leveling procedure. In some implementations, the inclinometers can be used by the magnetic sensing algorithm to compensate for a known misalignment. In some implementations, the system 1000 can include accelerometers to compensate for structural instability in the system and to track motion in of system components. The accelerometer data can be provided to the magnetic sensing algorithm. In some implementations, signals generated by the inclinometers and/or accelerometers as well as power to the inclinometers and/or accelerometers the can be routed through the base plate 1010. In some implementations, the signals generated by the inclinometers and/or accelerometers can be transmitted wirelessly to the system 400. In some implementations, the base plate 1010 can include a plurality of slots to receive the posts 1005 or other structures suitable for mounting one or more sensors 425.

[0142] The base plate 1010 can be a semi-rigid low-profile mat configured to accurately position the location and orientation of the bases of the Rx and Tx posts 1005. An inclinometer 1015 can determine relative tilt angles between the Rx and Tx posts 1005 at installation. When the system 1000 determines the relative tilt angles are above some threshold value, the operator can be alerted and asked to improve or remedy the leveling of the posts 1005.

[0143] For example, in some implementations, the tilt angles can be used to revise the location and orientation of the sensors. The location and orientation of the sensors can be further used to create the sensor model utilized by the reconstruction module 440

[0144] In some implementations, accelerometers 1020 can be configured to track the tilt angles dynamically, which enables the system to track motion of various sensors in the system. For example, when combined with known field gradients which can be determined with respect to motion for the various sensors, the calibration module 435 can predict distortions to be expected for measured motions. This prediction can then be removed from the measured signal to recover a more accurate representation of the signal measured as if there was no motion. In this example, the fields to be modeled and removed can be the measured motion, such as the displacement or the tilt of the sensors multiplied by the field gradients for each direction/type of motion.

[0145] Magnetic-field-based personnel inspection systems can detect or receive magnetic fields which have been transmitted by the system, such as by transceivers 406, and an objects secondary fields which can impacted by the presence of metal in the environment in which the inspection system is deployed. Environmental magnetic fields, such as those which may be reflected by metal which is nearby the inspection system, such as in a metal floor on which the inspection system located, can negatively impact the inspection systems performance to accurately discriminate and detect threat objects. Determining and characterizing the amount of metal in every environment in which the inspection system is located can be difficult, expensive, and often impossible.

[0146] As further shown in FIG. 10, the system 1000 can be configured to account for physical, metallic structures which may be present in locations where the system is deployed, such as a metal floor. Receivers 405 can detect the transmitted fields 1030 generated by the transmitters 106, as well as secondary fields 1035 of an object being detected. The secondary fields can be impacted by the presence of metal in the environment which may be present in proximity of system, such as in the floor 1025 underneath the system. The presence of metal in the floor can negatively impact the system’s detection performance and accuracy. Locating and characterizing metal in each environment can be difficult and expensive. To account for the presence of metal in the environment, the system 1000 can fit an appropriately parameterized model of metal present in the environment based on determining how the magnetic fields 1035 transmitted from the metal floor 1025 are distorted relative to a known model. The system 1000 can include an image model, whereby the metal in the floor 1025 is accounted for as a complex-weighted mirror image of the entire system including the transmitters 406 and the receivers 405. For example, the image model can model a perfect electric -conductor, which is known to modify magnetic fields in a very predictable and analytically describable way. The image model can then be parameterized by a depth with respect to the sensor coordinate system, and an overall complex weight of magnitude less than or equal to 1. Measurements of transmitted field 1030 can be used to perform an optimization routine by fitting the complex weight of the mirror image and the depth of plane of the mirror image. Such an optimization routine can search (for example, by trying various pre-defined solutions or via gradient descent) for the model parameters that best fit the measured data. For example, the optimization routine can perform the search by trying previously defined solutions or using a gradient descent optimization. These model parameters can subsequently be used in the operation of the system during threat/non-threat detection. As a result, the system 1000 can automatically detect objects more accurately in deployed environments which may or may not include metal in proximity to the system.

[0147] FIG. 11 is a diagram illustrating an exemplary configuration of a personnel inspection system 1100 as disclosed herein including a sensor and transmitters 406 at different locations according to some implementations. In the illustrated example, the system 1100 includes two transmitters 406, two vertical posts, 1105A and 1105B, each including a plurality of three-axis fluxgate receivers 1110. Each of the three-axis fluxgate receivers 1110 are illustrated as an “x” and configured on or within posts 1105A and 1105B. The system 1100 further includes a vertically oriented post 1105C located adjacent to the two transmitters 406. The post 1105C includes a plurality of two-axis fluxgate receivers 1115 illustrated as an “x”. The number of each type of component, such as the transmitters 406 and/or the fluxgate receivers 1110, 1115, can be determined by balancing the cost and complexity of additional components with the marginal gain that that additional component add to the fidelity of the magnetic sensing algorithm.

[0148] The system 1100 can further include a camera 1120 integrated with the system 1100. In some implementations the camera 1120 can be configured on or within one or more of the posts 1105A, 1105B, and 1105C, as well as configured on or within the transmitters 406. The camera 1120 can generate images and video, in a streamed or recorded manner. The integrated camera 1120 can be configured with an appropriate viewing angle. For example, in some implementations, the camera 1120 can be a rear-facing camera.

[0149] When an alarm is generated in response to the personnel inspection system detecting a threat, the alarm must be resolved by an inspection system operator or other security team member. Resolving the alarm requires the operator or security team member to interact with the individual and to follow security protocols to search the individual. Security protocols may not always be followed properly by the operator or security team member. A manager of the security team or inspection system operator may be unable to ascertain whether or not appropriate security and searching protocols are being followed. The improved personnel inspection system described in relation to FIG. 11 addresses these issues.

[0150] For example, the camera 1120 can provide images or video with a sufficient field of view so that supervisors of inspection system operators could evaluate an inspection system operator’s response to an alarm or detected object. In this way, the camera 1120 integrated within the system 1100 can enable a supervisor or manager of inspection system operators to assess an operator’s adherence to screening procedures or policies, provide training feedback, provide images or video for evidentiary purposes, record alarm resolution actions taken by an operator. In some implementations, the image or video from the camera 1120 can be combined with an image of a detected object generated by the system and used to refute or support allegations of improper treatment by inspection system operators.

[0151] A personnel inspection system employing magnetic sensing is limited to sensing metal on an individual passing through the inspection system and is unable to detect the body or physical characteristics of an individual passing through the inspection system. In this way, the personnel inspection system lacks the concept of a person passing through the inspection system and can thus generate limited information about the individual to an inspection system operator.

[0152] The improved inspection system described herein and shown in FIG. 11 can include a camera 1120 to detect persons passing through the inspection system 1100. In some implementations, the camera 1120 can be a depth camera. A depth camera can include a camera configured to use stereo vision to calculate depth in images acquired by the depth camera. Depth cameras can include depth sensors, and infrared projectors. In some implementations, the depth camera can include a color sensor configured to detect light in the red, green, blue (RGB) scale, also known as RGB sensors. The outputs of the depth cameras can be used to determine a location, orientation, or disposition of a detected object, as well as the speed or gait of the object passing through the inspection system. The outputs of the depth cameras can allow the inspection system to count a number of objects or individuals passing through the inspection system and to track one or more individuals passing through the inspection system. The inspection system disclosed herein, when configured to include a depth camera, can provide a number of advantages compared to inspection systems which may not include a depth camera. Such advantages can include more rapid identification of threat objects or individuals and more robust notification or alarm data provided to identify a threat object or individual.

[0153] In such implementations, the depth camera 1120 can register its coordinate system with the automatic threat recognition sub-system 445 configured within the inspection system 400 as shown in FIG. 4. In some embodiments, the camera 1120 can be communicably coupled to the VC VS 100 described in relation to FIGS. 1-3.

[0154] In this way, the system can have simultaneous knowledge of a visitor’s location/disposition and any metal objects on them in a common coordinate system. Such implementations enable a magnetic sensing algorithm to be directed towards those voxels which are occupied by the visitor and/or specific areas on the visitor (such as pockets, bags, and ankles). For example, the voxels of the OD 407 can be compared to the pixels obtained via the depth camera to determine which voxels are occupied (e.g., there is a pixel with coordinates sufficiently close to this voxel), unoccupied (e.g., there is no pixel with coordinates sufficiently close to this voxel and no chance of occlusion), or unknown (e.g., a pixel has been found that may obscure the OD 407 whether or not an object resides in this voxel). Then, the magnetic sensing algorithm can be restricted to only search for objects in the occupied and/or unknown voxels.

[0155] In addition, using knowledge the speed and gait of the visitor in the magnetic sensing algorithm can improve its accuracy, as well as count and track visitors into and out of the system for statistical reporting. Accuracy of an optimization routine is usually improved when the number of variables it must solve for can be reduced or constrained. Knowledge of the speed and location of a visitor would allow constraining or imposition of these attributes on the object that the optimization routine is solving for.

[0156] In such implementations, the depth camera 1120 (or any sensor 425 or combination of sensors capable of providing both RGB and depth values at various pixels, such as a structured light camera or stereo cameras) can be integrated directly into the system 400 of FIG. 4 and/or the VCVS 100 of FIGS. 1-3. In this way, the image data obtained by the depth camera can be available to the magnetic sensing algorithm. Given knowledge of the location, orientation, and lens properties of the depth camera, the pixels obtained via the depth camera can be registered to the coordinate system of the automatic threat recognition sub-system 445 of FIG. 4, such that each pixel, given a returned depth value, can be translated into the 3D coordinate system of the magnetic sub-system by a series of transformations.

[0157] As such, the depth camera(s) 1120 can identify which voxels in the magnetic subsystem’s scan zone are occupied and/or unoccupied at a given moment in time, and the magnetic sensing algorithm can be directed accordingly. The speed and gait of the subject can also be estimated and used in the magnetic sensing algorithm to identify concealed objects more accurately. An object which may be discovered via the magnetic sensing algorithm using particular 3D coordinate(s). The object can then be associated with a subset of pixels in the depth camera image(s) based on a simple distance threshold.

[0158] Furthermore, this subset of pixels can be associated with a larger contiguous object identified and segmented from the depth camera 1120, either by its depth values or by its RGB values, or both. For example, the system can identify the outline of a person most likely to be carrying the found object. Properties of, either the neighborhood around the object or the person identified to be holding the object can be used in its classification. For example, the person’s face can be compared to a watch list via a facial recognition algorithm, and past history or knowledge of this person can be used in classifying the object. At the same time, a threat overlay image made by applying a threat overlay atop of the RGB image can be enhanced to aid operator recognition by highlighting either the visible container of the object or the person holding the object, or both. This can help improve an operator’s reaction time and accuracy in resolving an alarm triggered by the system.

[0159] FIG. 12 is a diagram illustrating an exemplary configuration of a personnel inspection system 1200 as disclosed herein including a plurality of cameras according to some implementations. By configuring the inspection system 900 with a plurality of cameras, the inspection system can better determine which particular individual and where on the particular individual to search. Personnel inspection system which lack threat localization using multiple cameras to generate multiple viewing angles allow large, unorganized crowds of individuals to be screened without forming queues for individual screening and can identify potential threats localized in three dimensions. Additionally, when an alarm associated with a detected threat is generated, the use of multiple cameras generating multiple view angles enables the system to visually identify an individual associated with the detected threat using the captured images to facilitate informing an inspection system operator or other security personnel where to search the identified individual for the detected threat. Alternate solutions, which may use lights or a solely still image to identify an individual associated with a detected threat, may only provide an occluded viewing angle from a single camera. The single view angle may be insufficient to safely and efficiently identify a detected threat on an individual among a crowd of individuals passing through the inspection system.

[0160] As shown in FIG. 12, the system 1200 includes posts 1205A and 1205B. The posts 1205A and 1205B include a plurality of three-axis fluxgate receivers 1210 illustrated individually as an “x”. Post 1205C includes a plurality of two-axis fluxgate receivers 1215 illustrated individually as an “x”. Post 1205C also includes transmitters 406.

[0161] In some implementations, multiple individuals can pass between the posts 1205A, 1205B, and 1205C at any one time. For example, two people may pass through posts 1205A and 1205B at the same time which can result in obscuring the field of view of a single sensor, such as sensor 425 described in relation to FIG. 4. As a result, in implementations including a single sensor, the field of view of the sensor can become occluded. In some implementations, multiple sensors can be included in the inspection system to avoid an occluding field of view at a single sensor and to provide multiple viewpoints.

[0162] As shown in FIG. 12, multiple cameras 1220, such as cameras 1220A and 1220B, can be configured within the personnel inspection system 1200. The cameras 1220A and 1220B can be registered to the system’s coordinate system representing the coordinates of the OD 407 to perform threat localization from multiple viewpoints. As objects are tracked through time, each camera can determine the location of the threat across or within multiple time-slices. Thus, even if a viewing angle from a single camera 1220A is occluded at one moment in time, the likelihood that the viewing angle of a second camera 1220B is also occluded at all available moments in time is very low. The system can provide the inspection system operator images and/or videos from all viewing angles and can utilize the images and/or videos from one or more cameras to determine the actual threat location. In some implementations, the rendering module 450 can be further configured to automatically select optimal viewing angles and/or time- slices for presentation to aid the operator’s recognition and response to a detected threat. For example, the image that has the greatest number of pixels occupied in the vicinity of the found object is likely to give a reasonable and unoccluded view of the object. [0163] For example, as shown in FIG. 12, the system 1200 includes two fisheye cameras, 1220A and 1220B. Each camera is mounted on opposing posts 1205A and 1205C in order to capture a scan zone from two very distinct viewing angles. Each pixel in the images and/or videos generated by both cameras is mapped to the coordinate system of system 1200 at multiple planes along the direction of an object or person passing through a lane formed between posts 1205A and 1205C. The system 1200 can be configured to generate an alarm localized in a particular plane of view associated with camera 1205A or 1205B. Visual representations or threat indicators can be overlaid on images and/or video generated by each camera and centered on the appropriately registered pixels corresponding to the detected threat. The system 1200 can be configured to repeat these operations for multiple planes of view as the object or person passes through the system so as to produce a threat overlay image or video from each camera.

[0164] The videos and/or images from one or all cameras can be shown to the operator via a computing device communicatively coupled to the system 1200, such as a laptop, tablet or other mobile computing device configured with a display. In some implementations, the system can utilize a threshold criterion for determining if a viewing angle of one of the cameras is occluded. The system 1200 can determine occlusion, for example, by determining a depth estimate of the pixels in question by evaluating motion in the video stream, or by incorporation of a depth camera. If the system 1200 determines that the depth estimate is not consistent with the 3D coordinate returned by the system , then the viewing angle should be considered occluded, and the system can determine that the image will not be useful in guiding the response of an operator.

[0165] In some implementations, the inspection system 1200 can process the additional sensor data, such as a video or images, and can relay an image, a video, or a video frame of a subject alongside or overlaid with classification results. For example, the inspection system 1200 can overlay a graphical indicator atop an image and the graphical indicator can identify the detected threat or object. In some implementations, the image overlaid with the graphical indicator can be provided with additional metadata about the individual, detected object, or system parameters. In some implementations, the image overlaid with the graphical indicator can be provided to an individual, such as an inspection system operator or security guard who can be located further downstream in the sequence of objects or individuals being inspected via the inspection system. In this way, the inspection system can provide the image overlaid with the graphical indicator to the inspection system operator or security guard for additional monitoring and/or interception of the detected object, threat, or individual.

[0166] Although a few variations have been described in detail above, other modifications or additions are possible. For example, the number of receivers is not limited and some implementations may include any number of receivers. The transmitters are not limited to a particular frequency, for example, coils with different properties (operating frequencies, locations, and the like) can be used. Different reconstruction algorithms may be used and different features may be used for threat detection.

[0167] Without in any way limiting the scope, interpretation, or application of the claims appearing below, a technical effect of one or more of the example implementations disclosed herein may include one or more of the following, for example, some example implementations of the current subject matter can perform threat detection and discrimination in high clutter environments in which individuals may be carrying personal items such as cell phones and laptops and without personal item divestment. In some implementations, a personnel inspection system can perform threat detection and discrimination with high throughput that allows individuals to pass through the metal detector at normal walking speeds such that individuals are not required to slow down for inspection and, in some implementations, the inspection threshold can allow for multiple individuals to pass through the threshold side-by-side (e.g., two or more abreast). In some configurations, individuals walking in near proximity can be screened, thereby eliminating the need for screened individuals to remain stationary during the screening process.

[0168] One or more aspects or features of the subject matter described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) computer hardware, firmware, software, and/or combinations thereof. These various aspects or features can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device. The programmable system or computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0169] These computer programs, which can also be referred to as programs, software, software applications, applications, components, or code, include machine instructions for a programmable processor, and can be implemented in a high-level procedural language, an object-oriented programming language, a functional programming language, a logical programming language, and/or in assembly /machine language. As used herein, the term "machine-readable medium" refers to any computer program product, apparatus and/or device, such as for example magnetic discs, optical disks, memory, and Programmable Logic Devices (PLDs), used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine -readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor. The machine- readable medium can store such machine instructions non-transitorily, such as for example as would a non-transient solid-state memory or a magnetic hard drive or any equivalent storage medium. The machine-readable medium can alternatively or additionally store such machine instructions in a transient manner, such as for example as would a processor cache or other random access memory associated with one or more physical processor cores.

[0170] To provide for interaction with a user, one or more aspects or features of the subject matter described herein can be implemented on a computer having a display device, such as for example a cathode ray tube (CRT) or a liquid crystal display (LCD) or a light emitting diode (LED) monitor for displaying information to the user and a keyboard and a pointing device, such as for example a mouse or a trackball, by which the user may provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well. For example, feedback provided to the user can be any form of sensory feedback, such as for example visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, speech, or tactile input. Other possible input devices include touch screens or other touch- sensitive devices such as single or multipoint resistive or capacitive trackpads, voice recognition hardware and software, optical scanners, optical pointers, digital image capture devices and associated interpretation software, and the like. [0171] In the descriptions above and in the claims, phrases such as "at least one of" or "one or more of" may occur followed by a conjunctive list of elements or features. The term "and/or" may also occur in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradicted by the context in which it is used, such a phrase is intended to mean any of the listed elements or features individually or any of the recited elements or features in combination with any of the other recited elements or features. For example, the phrases "at least one of A and B;" "one or more of A and B;" and "A and/or B" are each intended to mean "A alone, B alone, or A and B together." A similar interpretation is also intended for lists including three or more items. For example, the phrases "at least one of A, B, and C;" "one or more of A, B, and C;" and "A, B, and/or C" are each intended to mean "A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together." In addition, use of the term "based on," above and in the claims is intended to mean, "based at least in part on," such that an unrecited feature or element is also permissible.

[0172] The subject matter described herein can be embodied in systems, apparatus, methods, and/or articles depending on the desired configuration. The implementations set forth in the foregoing description do not represent all implementations consistent with the subject matter described herein. Instead, they are merely some examples consistent with aspects related to the described subject matter. Although a few variations have been described in detail above, other modifications or additions are possible. In particular, further features and/or variations can be provided in addition to those set forth herein. For example, the implementations described above can be directed to various combinations and subcombinations of the disclosed features and/or combinations and subcombinations of several further features disclosed above. In addition, the logic flows depicted in the accompanying figures and/or described herein do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Other implementations may be within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:

1. A method comprising: determining, by a data processor, first data characterizing a display state of a credential device associated with an individual, the display state identifying an access status of the individual to access a credential verification checkpoint; providing, by the data processor, second data to a transmitter communicably coupled to the data processor, the second data characterizing control signals controlling the display state of the credential device; and transmitting, by the transmitter, the second data to the credential device, which responsive to receiving the second data causes the credential device to display the display state on the credential device.

2. The method of claim 1, further comprising receiving, by the data processor, sensor data corresponding to the display state displayed on the credential device as the individual accesses the credential verification checkpoint; determining, by the data processor, the access status of the individual based on the display state; and providing, by the data processor, third data for display at the credential verification checkpoint, the third data identifying the access status of the individual.

3. The method of claim 2, wherein the sensor data is acquired by an image sensor including at least one of a video camera, an electro-optical camera, a surface map camera, or a depth map camera.

4. The method of claim 2, wherein the access status of the individual is determined based on at least one of a computer vision algorithm and a manual input received from a personnel of the credential verification checkpoint.

5. The method of any one of the preceding claims, wherein the transmitter is configured to transmit the second data to the credential device using radio frequency communication protocols or infrared communication protocols.

6. The method of any one of the preceding claims, wherein the display state includes a color selected from a plurality of colors.

7. The method of any one of the preceding claims, wherein the credential device includes at least one light emitting diode configured to illuminate based on the received second data.

8. The method of any one of the preceding claims, wherein the first data is determined based on a user profile associated with the individual, the user profile comprising at least one of a unique identifier of the credential device, one or more display states associated with the individual, and one or more access statuses associated with one or more credential verification checkpoints.

9. The method of any one of the preceding claims, wherein the first data is dynamically determined based on a predetermined schedule and/or in a trigger event.

10. The method of claim 9, wherein the second data is transmitted to the credential device based on the predetermined schedule and/or the trigger event, which once received by the credential device causes the credential device to change the display state to a second display state.

11. A system comprising: a computing device comprising at least one data processor and a memory containing non-transitory computer-executable instructions; at least one image sensor communicatively coupled to the computing device; and at least one transmitter coupled to the at least one data processor, wherein the at least one data processor is configured to execute the instructions stored in the memory to perform operations comprising determining first data characterizing a display state of a credential device associated with an individual, the display state identifying an access status of the individual to access a credential verification checkpoint, and providing second data to the at least one transmitter, the second data characterizing control signals controlling the display state of the credential device, wherein responsive to receiving the second data the transmitter is configured to transmit the second data to the credential device, which responsive to receiving the second data causes the credential device to display the display state on the credential device.

12. The system of claim 11, wherein the operations further comprise receiving sensor data from the at least one sensor, the sensor data corresponding to the display state displayed on the credential device as the individual accesses the credential verification checkpoint; determining the access status of the individual based on the display state; and providing third data for display at the credential verification checkpoint, the third data identifying the access status of the individual.

13. The system of claim 12, wherein the at least one image sensor includes a video camera, an electro-optical camera, a surface map camera, or a depth map camera.

14. The system of claim 12, wherein the access status of the individual is determined based on at least one of a computer vision algorithm and a manual input received from a personnel of the credential verification checkpoint.

15. The system of any one of claims 11-14, wherein the at least one transmitter is configured to transmit the second data to the credential device using radio frequency communication protocols or infrared communication protocols.

16. The system of any one of claims 11-15, wherein the display state includes a color selected from a plurality of colors.

17. The system of any one of claims 11-16, wherein the credential device includes at least one light emitting diode configured to illuminate based on the received second data.

18. The system of any one of claims 11-17, wherein the first data is determined based on a user profile associated with the individual, the user profile comprising at least one of a unique identifier of the credential device, one or more display states associated with the individual, and one or more access statuses associated with one or more credential verification checkpoints.

19. The system of any one of claims 11-18, wherein the first data is dynamically determined based on a predetermined schedule and/or in a trigger event.

20. The system of claim 19, wherein the second data is transmitted to the credential device based on the predetermined schedule and/or the trigger event, which once received by the credential device causes the credential device to change the display state to a second display state.