WO2019032073A2

WO2019032073A2 - A system to detect direction and location of a shooter

Info

Publication number: WO2019032073A2
Application number: PCT/TR2018/050262
Authority: WO
Inventors: Ozgur OZUGUR; Tolga SONMEZ; Adil Burak BASLI; Caglar AKMAN; Bahattin KOCAMAN; Mahir CIHAN
Original assignee: Havelsan Hava Elektronik Sanayi ve Ticaret AS
Current assignee: Havelsan Hava Elektronik Sanayi ve Ticaret AS
Priority date: 2017-06-05
Filing date: 2018-05-23
Publication date: 2019-02-14
Anticipated expiration: 2019-12-05
Also published as: WO2019032073A3

Abstract

Invention relates to find direction of arrival of bullets and/or location of firing source or sources detection system and method on acoustic basis (100) by means of detecting sound waves generated by the bullet, which is fired from weapon systems, when leaving the muzzle blast and/or the shock waves generated by bullets when traveling to the target.

Description

A SYSTEM TO DETECT DIRECTION AND LOCATION OF A SHOOTER The Related Art

Invention relates to system to find direction and/or location of firing source or sources on acoustic basis with high accuracy by means of detecting sound waves generated by the bullets fired from supersonic weapon systems such as pistol, machine gun, mortar and rocket launcher, when leaving the muzzle blast and the shock waves generated by bullets when traveling to the target, in order to detect the position and/or direction of the person firing by supersonic weapons in a secured area.

The invention particularly uses optimal fusion of measurements obtained from multiple acoustic sensors to find a best estimate of shooter's location/direction.

Background of the Invention

Acoustic based shooter detection systems find direction of shot and positions of firing source or sources by means of detecting sound waves generated by the bullet. There are two types of wave that is created by supersonic weapon systems such as pistol, machine gun, mortar, rocket launcher: the first one is the muzzle blast and the other is the shock waves generated by the bullet when traveling to the target. In most of the existing similar systems, acoustic sensors operate standalone. A single sensor capable to detect both muzzle blast sound and shock wave can identify the location of the firearm. However, making such estimation can be difficult due to physical restrictions. Detecting muzzle blast sound depends on distance between the sensor and the firearm. In order to be able to detect the shock wave, the bullet should pass close to sensor.

A single sensor may detect only muzzle blast and another one may detect only Shockwaves. This invention can deal with all possible combinations. As the acoustic sensors form a network and share the measurements made by each sensor, fusion of information and a better estimate of location is possible. Besides, the existing systems generally have 4 microphones as high quality microphones are expensive. For that reason, not only errors are large but also there is no redundancy for faulty microphones.. The applications encountered during technical investigations relate to algorithms developed for acoustic detectors that works standalone. The algorithm of the system disclosed under this invention works completely in a different way. Muzzle blast sound and shock wave decomposition is processed in two ways in this algorithm. It has activity windows for signal processing and decision is made based on energy features and spectrum properties. In addition, said applications do not disclose a mechanic design for reducing errors and providing a redundancy against faults. The application numbered US201 1069585A1 does not use multiple microphones for TOA (Time of Arrival). In addition, data that is obtained from sensor network are not included in fusion algorithm in the mentioned patent. As a result, due to described disadvantages above and inadequacy of existing solutions, it has been necessary to make an innovation in the related art.

Purpose of the Invention The invention has been developed with inspiration from existing problem and aims to eliminate the above mentioned disadvantages.

Primary purpose of the invention is to identify the location and direction of the firearms by using multiple acoustic sensors.

More than one acoustic sensor is operated simultaneously. Even though each sensor detects either or both of muzzle blast sound of bullet leaving firearm and shock sound generated in the air by bullet, all data are sent to central unit and processed together and all measurements are integrated through optimization to find the best estimate. In other words, even a single sensor detects only muzzle blast sound or only shock wave, all undergo an integration at central unit. Thus, sensors which receive partial data are also contribute to final estimation .

In addition, since MEMS (Microelectromechanical systems ) and new low cost microphones (electret, electret condenser) are used in the invention, number of microphones can be increased. For that reason, both estimation errors are lowered and backup against failures is provided. A special mechanical design for 8 microphones, optimization algorithm for integration of measurements from multiple sensors and a method to detect weak signals have been developed. The system disclosed under the invention can also be used for remote identification of UAVs (Unmanned Air Vehicle) acoustically.

The structural and characteristic features of the invention and advantages are shown in the figures and block diagrams given below; a detailed explanation of the invention will be made in reference to these figures and diagrams.

Brief Description of the Drawings

Figure 1 shows structure of the acoustic sensor in detection of shooter direction/position used in the invention.

Figure 2 shows structure of central management unit in shooter direction and location identification system of the invention.

Figure 3 shows flow chart of algorithm in the signal processing module.

Figure 4 shows structure of flow chart of shooter direction and location identification system of the invention.

Figure 5 shows prototype of 8-microphone array of acoustic sensor.

Figure 6 shows that the microphones in the acoustic sensor can also be distributed in different alternative ways on the given hemisphere.

Figure 7 shows shooter direction and location identification system of the invention.

The drawings are not necessarily scaled and the details that are not necessary for understanding the present invention might have been neglected. In addition, the components which are equivalent to a great extent or have equivalent functions have been assigned the same number.

Description of Part References

100- A system to identify direction and location of shooter

10- Acoustic sensor

1 1 - Microphone (MEMs/Electret /Electret Condenser)

12- GPS receiver 13- Temperature sensor

14- 3-dimension inertial sensor (Orientation sensor)

15- Embedded hardware

16- Communication module

17- Signal processing module

18- Digital sampling module

20- Central management unit

21 - Sensor fusion filters

22- Terrain data

23- User interface

24- Acoustic library

25- Data record system

26- Communication unit

Detailed Description of the Invention

In this detailed description, the preferred embodiments of the invention have been described in a manner not forming any restrictive effect and only for the purpose of better understanding of the matter.

Bullets from pistols, rifles, etc. generates an explosion sound when leaving muzzle. This is called muzzle blast sound. Bullets travelling faster than sound, generate waves in the air. Such waves are called shock waves. Coming directions of such waves (muzzle blast sound and shock waves) are detected by means of microphone (1 1 ) array called acoustic sensor (10) and the collective information is used to detect shooter.

The invention relates to shooter detection system based on acoustic sensing (100), identifying direction of arrival and location of firing source or sources with a high accuracy by detecting sound waves generated by the bullets fired from supersonic weapon systems such as pistol, machine gun, mortar, rocket launcher. These sound waves are formed when the bullet is leaving the muzzle and/or the shock waves generated by bullets when travelling to the target. Since performance of direction and location detection system (100) may vary subject to terrain structure, it is also important to place the acoustic sensors (10) or in other terms, acoustic sensors in the best manner. Firearm direction and location detection system (100) disclosed under this invention consists of hardware and software. The invention (100) contains at least an acoustic sensor (10) (see Figure 1 ). Single acoustic sensor (10) may estimate location by using both muzzle blast sound and shock wave. With single acoustic sensor (10), direction can be estimated by using only muzzle blast sound. For fusion, to find the location one of at least two acoustic sensors (10) should receive muzzle blast sound and the other should receieve Shockwave.

Each acoustic sensor (10) contains one array having at least 4 MEMS/Electret/Electret/Condenser microphones (1 1 ), and signal processing embedded hardware (15). The mechanical design made for microphone (1 1 ) array is shown in Figures 5 and 6. Total 8 microphones (1 1 ) are located at each end of the mechanical unit of two different crosses (+) with the same centre. The end of the arms are located on the surface of a hemisphere for optimal distribution. Thus, directional optimization is provided. In addition, even if the four of the microphones in the array (1 1 ) fail, system continues to work, (see Figure 6)

Embedded hardware (15) provided on each acoustic sensor (10), is constantly running the signal processing module (17) at which acoustic signal processing algorithms are operating, to process sounds in the environment. Microphones (1 1 ) located in acoustic sensors (10), are sensitive to detect sound waves from muzzle blast and/or bullet shock waves. Analogue signals detected by microphones (1 1 ) are converted into digital signals by digital sampling module (18) before processing at signal processing module (17). Each acoustic sensor (10) contains GPS receiver (12) for sensor localization and 3-dimensional inertial sensors (14) (acceleration meter, rotation meter and magnetometer) to find attitude determination at the same time. Optionally, temperature sensor (13) may also be included to estimate sound speed more accurately.

Wireless communication module (16) and acoustic sensors (10) are constantly in contact with central management unit (20). Thus, commands and configuration messages received from central management unit (20) are processed by embedded hardware (15). Detected firearm sounds are also transmitted to central management unit (20).

Central management unit (20) has sensor fusion filters (21 ) receiving and combining only muzzle blast sound data from acoustic sensor (10) or only shock wave data or both muzzle blast sound and shock wave data, (see Figure 2). The sensor fusion filter (21 ) enables identification of shooting source more accurately and tracks them in time. Thanks to sensor fusion filters (21), even though each acoustic sensor (10) does not detect both muzzle blast sound and shock wave at the same time, central management unit (20) is capable to integrate all possible data from acoustic sensor (10). Sensor fusion filters (21) will be enabled to identify the shooter location in the best way even in enviroments with echo by use of terrain data (22) (height, DTED map, meteorological data etc.).

Fusion algorithm running in sensor fusion filters (21) are solved by useing the following equations: c := (t_ni,t_si,u_nl,u_sl,t_n2,t_S2,u_n2,u_S2,L ) = / (t₀,x₀,w₀ ) (1) min (c-f)^T(c-f)

(2)

f0

Where ^x° indicates location of the shooter at time *° ,and ^u° shows travelling direction of the bullet. Muzzle blast sound reaches ¹ acoustic sensor (10) at time ^f"^] . Shock wave sound reaches ¹ acoustic sensor (10) at ^isl .

) in Equation 1 are measured by acoustic sensors (10). The measurements are sent to central management unit (20). f{^to^,xo^,uo) j_S found by analytic modelling. These two data is optimized with equation 2 and solution (^fo' o'"o ) j_S obtained. Each measurement contains error to some extent. As more measurements are used, they becaome statistically independent and these errror average out. System solution achieved with fusion has less errors.

Additionally, although acoustic sensors (10) collect raw data, proceseed data such as time, direction and wave types are sent to the central processing unit for fusion. Thus, a low volume of data is sent in the data network. Terrain data (22) is also used in finding shot direction and placement of acoustic sensors (10). Aforementioned sensor fusion filters (21), compare bullet sound frequency, duration and distribution features by using acoustic library (24) and identify the bullet type and type of firearm that is shooting the bullet. Central management unit (20) uses a geographical information system capable to show digital data and maps. Locations and/or directions of firearms are displayed on user interface (23). Sounds of all identified important activities are recorded in the data record system (25) for subsequent examination. Central management unit (20) contains a communication unit (26) capable to contact with acoustic sensors (10) continuously, exchange data, send commands and configuration messages. Flow chart of shot direction and location detection system (100) is given in Figure 4. Process steps are:

• installation of central management unit (20),

· installation of acoustic sensors (10),

❖ initiation of microphone (1 1 ) array

❖ location identification by the GPS receiver (12),

❖ identification of attitude by 3-dimensional inertial sensor (14)

❖ initiation of wireless communication by communication module (16), · identification of status, location and orientation of acoustic sensors (10) by central management unit (20) by using data transmitted by communication module (16) and received by communication unit (26),

• initiation of geographical information system capable to show digital data and maps at central management unit (20) at the user interface (23),

❖ transmission of terrain data (22) to acoustic sensors (10) via communication unit (26) by central management unit (20),

• identification of enemy and friendly zones by users from user interface (23),

• optimal placement of the acoustic sensors (10) in the terrain to find out shooting from threat zones with the least uncertainty by distributing in neighbourhood of the identified threat zone, if possible, (in order to make better estimation with lower uncertainty) and recommending to the user via user interface (23) by central management unit (20),

❖ placement of the acoustic sensors (10) by user,

• providing connection of wired and wireless communication units to external systems (sensors or communication systems with day/night vision cameras),

• initiation of work in normal mode;

❖ evaluation of the sounds detected by microphones (1 1 ), then by signal processing module (17) provided in embedded hardware (15),

❖ transmission of muzzle blast sound data or shock wave data or both muzzle blast sound and shock wave data to central management unit (20) by means of communication module (16),

❖ evaluation of measurment data from each acoustic sensor (10) by sensor fusion filters (21 ). Flow chart of algorithm, that is run in the signal processing module (17), is shown in detail in Figure 3. Muzzle blast and shock wave sounds' arrival time instant is used to detect muzzle blast sound and shock wave direction vectors and also feed time of arrival algorithm as input data for the calculations in signal processing module (17). In addition, other data (wave length, magnitude etc.) of the waves needed by shooter direction and location detection system (100) are also found in the signal processing module (17).

In general, shock waves have signals of N-like shape and short duration (200-300 μεη.) having fast rising/descending edges in range of 1 kHz - 10 kHz. Muzzle blast sound is a low frequency signal having rising/descending edges slower in comparison to shock waves and having second section longer than first section and under 500 Hz.

Process steps of algorithm run in the signal processing module (17) are:

• Digital signals received from microphone (1 1 ) and sampled by digital sampling module (18) (by A/D converter) are divided into predefined number of processing windows (being at least two times of the largest possible shock wave and muzzle blast sound durations),

• estimation of dynamic threshold values by using the values in former and current window (median method)

• dividing process window into small sub-windows up to the length of shock wave at most for activity detection, taking sum of squares of digital values thereof and marking the signal zones exceeding threshold values calculated in the previous step, as activity zone,

• determining the bandwidth of activity zone using the peak power point and the half peak power points where the energy is halved.

• activity zone bandwidth is compared with the maximum shock wave bandwidth which depends on the sampling rate of the hardware, nominal shock wave duration and sub-window length parameters, and if it is less than this bandwidth this activity zone is sent to the shock wave processing block

❖ in the shock wave processing block, zero crossing (ZC) coding, that is frequently used speech recognition, is applied to this activity zone,

❖ activity is subdivided into sections in ZC coding process up to next zero crossing,

❖ calculation of start time, rise time, peak point and peak amplitude of each part, ❖ then determining whether these parts a shock wave or not in the shock wave analysis block, rise time is compared to a predefined threshold, duration length of first and second parts and their ratio are checked,

❖ elimination of activity zone parts not passing from the checks mentioned in previous step,

❖ then a frequency check is made for the non-eliminated parts, the ratio of the energy of high frequency components to the energy of low frequency components are calculated

❖ if the ratio calculated in previous step is higher than a threshold that is determined with a real shock wave, then this part is reported as a confirmed

Shockwave

• if bandwidth of activity zone is larger than a predefined maximum shock wave bandwidth but smaller than muzzle blast sound maximum bandwidth, then the activity zone is sent to the muzzle blast sound processing block ,

❖ in the muzzle blast processing block, zero crossing (ZC) coding, that is frequently used speech recognition, is applied to this activity zone,

❖ calculation of start time, rise time, peak amplitude, peak point and peak amplitude of each part,

❖ then determining whether these parts a muzzle blast or not in the muzzle blast analysis block, rise time is compared to a predefined threshold, duration length of first and second parts and their ratio are checked

❖ if the ratio calculated in previous step is higher than a threshold that is determined with a real muzzle blast, then this part is reported as a confirmed muzzle blast

• if bandwidth of activity zone is larger than a predefined maximum shock wave bandwidth but not smaller than muzzle blast sound maximum bandwidth, activity zone is discarded.

Claims

The invention is a system to find direction of arrival of bullet's , and/or, location of firing source or sources on acoustic basis (100) by means of detecting sound waves generated by the bullets which is fired from weapon systems, when leaving the muzzle blast and/or the shock waves generated by bullets when traveling to the target, characterized by comprising; at least an acoustic sensor (10) consisting of

❖ An array with at least four microphones (1 1 ) sensitive to detect sound waves from muzzle blast and/or bullet shock waves of bullet,

❖ A continuously working embedded hardware (15), that is running acoustic signal processing algorithms on the sounds in the environment at its signal processing module (17) and running the commands and configuration messages coming from central management unit (20),

❖ wireless communication module (16) that is connected to the central management unit (20) to exchange data, to receive commands and configuration messages, providing transmission of firearm sounds detected by embedded hardware (15) to the central management unit (20),

and a central management unit (20) consisting of

❖ sensor fusion filters (21 ) receiving and combining only muzzle blast sound data or only shock wave data or both muzzle blast sound and shock wave data from acoustic sensors (10),

❖ a communication unit (26) capable to be continuously in contact with acoustic sensors (10), exchange data, commands and configuration messages,

❖ central management unit (20) that include user interface (23) where shooter positions are displayed to user.

A shooter direction and location detection system (100) according to claim 1 characterized by comprising; total 8 microphones (1 1 ) located at each end of mechanical unit in two different crosses (+) with the same centre, where the end of the arms are located on the surface of a hemisphere in order to provide directional optimization of said acoustic sensors (10).

A shooter direction and location detection system (100) according to claim 1 characterized by comprising; said acoustic sensors (10) consist of a GPS receiver (12) to identify sensor location.

4. A shooter direction and location detection system (100) according to claim 1 characterized by comprising; said acoustic sensors (10) consist of 3-dimension inertial sensors (14) to find attitude. 5. A shooter direction and location detection system (100) according to claim 1 characterized by comprising; said acoustic sensors (10) consist of a temperature sensor (13) providing more accurate calculation of sound speed.

6. A shooter direction and location detection system (100) according to claim 1 characterized by comprising; said acoustic sensors (10) consist of digital sampling module (18) converting the analogue signals detected by microphones (1 1 ) into digital signals before processing at signal processing module (17).

7. A shooter direction and location detection system (100) according to claim 1 characterized by comprising; said central management unit (20) consists of said sensor fusion filters (21 ) that utilizes terrain data (22) to identify shot source location even in ambient echo.

8. A shooter direction and location detection system (100) according to claim 7 characterized by comprising; said terrain data (22) are height, DTED map and/or meteorological data.

9. A shooter direction and location detection system (100) according to claim 1 characterized by comprising; said central management unit (20) consists of acoustic library (24) providing identification of bullet type and weapons firing such a bullet by means of comparison of bullet sound frequency, propagation and duration features in order to be used by said sensor fusion filters (21 ).

10. A shooter direction and location detection system (100) according to claim 1 characterized by comprising; said central management unit (20) consists of data record system (25) where sounds of detected important activities are recorded for subsequent examination.

11. A method to detect shooter direction and location on acoustic basis (100) identifying bullet direction of arrival and positions of shot source or sources with a high accuracy by means of detecting sound waves generated by the bullets fired from weapon systems when leaving the muzzle and/or the shock waves generated by bullets when travelling to the target, and with prior installation of central management unit (20) and at least three acoustic sensors (10), characterized by comprising; process steps of

• Activation of microphone array (1 1 ) to detect sound of muzzle blast and/or shock waves of bullet,

• Activation of the GPS receiver (12), which provides location of each acoustic sensor (10),

· Calculation of attitude of acoustic sensors (10) by 3-dimension inertial sensors (14)

• Initiation of wireless communication by communication module (16),

• Determination of status, location and attitude of acoustic sensors (10) by central management unit (20) by using data transmitted by communication module (16) and received by communication unit (26),

· Starting geographical information system at user interface (23) at central management unit (20),

• Transmission of terrain data (22) to acoustic sensors (10) via communication unit (26) by central management unit (20),

• Providing connection to external systems by means of wired and wireless communication units,

• Evaluation of sounds detected by microphone array (1 1 ) by signal processing module (17) located in embedded hardware (15) receiving and processing the commands and configuration messages sent by central management unit (20),

• Transmission of muzzle blast sound data or shock wave data or both muzzle blast sound and shock wave data to central management unit (20) by means of communication module (16),

• Receiving and evaluation of only muzzle blast sound data from acoustic sensor (10) or only shock wave data or both muzzle blast sound and shock wave data by sensor fusion filters (21 ) together with terrain data (22).

12. A method to detect shooter direction and location according to claim 1 and characterized by comprising; said signal processing module (17) performs process steps of • Digital signals received from microphone (1 1 ) and sampled by digital sampling module (18) are divided into processing windows which is longer than at least two times of the largest possible shock wave or muzzle blast sound duration lengths,

• Estimation of dynamic threshold values by using the values in former and current window,

· Determining the bandwidth of activity zone using the peak power point and the half peak power point where the energy is halved.

❖ in the shock wave processing block, zero crossing (ZC) coding, that is frequently used in speech recognition, is applied to this activity zone,

❖ calculation of start time, rise time, peak point and peak amplitude of each part,

❖ then determining whether these parts a shock wave or not in the shock wave analysis block, rise time is compared to a predefined threshold, duration length of first and second parts and their ratio are checked,

Shockwave

❖ in the muzzle blast processing block, zero crossing (ZC) coding, that is frequently used speech recognition, is applied to this activity zone, ❖ activity is subdivided into sections in ZC coding process up to next zero crossing,

❖ then in the analysis of these parts, rising time is checked if it fits in a acceptable range for muzzle duration length of first and second parts, their ratio are checked to decide a muzzle blast exists or not,

• if bandwidth of activity zone is larger than a predefined maximum shock wave bandwidth but not smaller than muzzle blast sound maximum bandwidth, activity zone is discarded. 13. A method to detect shooter direction and location according to claim 1 1 characterized by comprising; threat and friendly zones are determined by the user interface (23).

14. A method to detect shooter direction and location according to claim 1 1 characterized by comprising; central management unit (20) performs the most convenient placement of acoustic sensors (10) in the terrain to find out shooting from threat areas with the least uncertainty by distributing in neighbourhood of the identified threat zone and recommends the user to arrange location of acoustic sensors (10) via user interface (23).