CN111800732A - Virtual and real information integration spatial positioning system - Google Patents

Abstract

A virtual and real information integration space positioning system comprises an image output device, a processing unit, at least one network gateway, a plurality of Mesh routers, a millimeter wave vibration sensing module, a sound sensing module, a wireless positioning module and an image capturing module. The signal characteristic value of the object or the environment state acquired by the millimeter wave vibration sensing module or the sound sensing module or the wireless positioning module or the image acquisition module is sent to the processing unit through each Mesh router and the network gateway, the processing unit processes the signal characteristic value and enables the image construction module to generate at least one piece of positioning information of the object or the environment state to be displayed on the image output device, and the image post-molding module is used for post-molding and generating a simulation image simulating the object or the environment state according to the signal characteristic value of the object or the environment state and a real image to be displayed on the image output device so as to form integration of virtual and real information display and space positioning.

Description

Virtual and real information integration spatial positioning system

technical field

The present invention is a space positioning system, especially a space positioning system integrating virtual and real information.

Background technique

Among the existing positioning technologies, for example, Bluetooth (Bluetooth) is a wireless technology standard that can realize short-distance data exchange between fixed and mobile devices and personal network domains in a specific space. In addition, iBeacon, a device using the Bluetooth function, can be combined with a portable smart mobile device (such as a mobile phone) to implement the user's mobile device to achieve the effect of indoor positioning. The positioning is to determine the distance between the signal point and the receiving point through the signal strength (Received Signal Strength Indication, RSSI) received by the mobile phone, so that the mobile phone can determine the position. It is a kind of positioning calculation based on the corresponding data. positioning technology.

In this technical solution, a Bluetooth positioning system (such as the aforementioned iBeacon) can be used to monitor the approach or distance of an item (mobile phone). When the mobile phone receives the bluetooth broadcast signal of the target, the mobile phone can judge whether the target (iBeacon) is approaching or far from the mobile phone by the intensity change of the bluetooth broadcast signal, and then generate a message to notify the background monitoring personnel. However, this iBeacon Bluetooth positioning system cannot provide an effective judgment on whether the Bluetooth receiver (mobile phone) has left the detection and accurate position (only judges the change of signal strength, no sense of space), which leads to the situation that the mobile phone holder has a wrong positioning. , resulting in invalid positioning, the invalid positioning cannot accurately know the position of the person to be positioned, such as the precise position of the horizontal floor or the precise position of the height of the floor.

In addition, in the existing positioning systems, there is also a lack of real-time and learning information transmission virtual and real positioning systems. For example, the aforementioned Bluetooth system can produce positioning effects, but it is impossible to know the person to be positioned (mobile phone holder) or the environment in real time. The status of living and non-living objects or environments in space, such as comprehensive virtual and real information maps including sound (positioning) information, more precise positioning, and image positioning information. That is the technical difficulty that the applicant of this case intends to solve.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve and improve the above-mentioned deficiencies in the prior art.

In order to achieve the above purpose, the present invention provides a spatial positioning system for integrating virtual and real information, which includes: an image output device; a processing unit communicatively connected to the image output device, and the processing unit is provided with a map data construction module and an image A post-production module, the processing unit is connected to a database in communication; at least one network gateway is connected to the processing unit via network communication; a plurality of Mesh routers are connected to the network gateway in communication; a millimeter wave vibration sensing module is provided with a first processor , a millimeter-wave signal generator, a transmitting unit and a receiving unit, the millimeter-wave signal generator is electrically connected to the first processor, the transmitting unit and the receiving unit are respectively electrically connected to the millimeter-wave signal generator, The first processor is communicatively connected to one of the Mesh routers; a sound sensing module is provided with a sound transmitting element and/or a sound receiving element and a second processor, and the second processor is provided with an analog-to-digital converter, And the second processor is electrically connected to the sound transmitting element and/or the sound receiving element respectively, and the second processor is communicatively connected to the other one of the Mesh routers; a wireless positioning module is provided with a third processor, the first The three processors are communicatively connected to the other one of the Mesh routers; an image capture module is provided with an image capture device and a fourth processor that are electrically connected to each other, and the fourth processor is communicatively connected to the other one of the Mesh routers.

Preferably, the image output device is a display screen, a projector or a wearable smart device.

Preferably, the wireless positioning module is a Bluetooth locator or a WiFi device.

Preferably, it further includes a portable intelligent device that communicates with one of the plurality of Mesh routers.

Preferably, the database is provided with a feature comparison module.

Preferably, one of the Mesh routers is set as a Mesh coordinator.

Therefore, the signal characteristic value of the object or environment state obtained by the millimeter wave vibration sensing module or the sound sensing module or the wireless positioning module or the image capturing module is sent to the processing through each of the Mesh routers and the network gateway a unit, which is processed by the processing unit and causes the image construction module to generate at least one positioning information of the object or the environment state, and present it on the image output device, and the image post-processing module is used for signal characteristics according to the object or environment state The value and the real image are post-processed to generate a simulated image simulating the state of the object or the environment, which is presented on the image output device, thereby forming the integration of virtual and real information presentation and spatial positioning.

Description of drawings

FIG. 1 is a schematic block diagram of a preferred embodiment of the present invention.

1A is a schematic block diagram of a detailed block structure of the millimeter-wave vibration sensing module of FIG. 1 according to a preferred embodiment of the present invention.

FIG. 1B is a schematic block diagram of a detailed block structure of the sound sensing module of FIG. 1 according to a preferred embodiment of the present invention.

1C is a schematic block diagram of a detailed block structure of the wireless positioning module of FIG. 1 according to a preferred embodiment of the present invention.

FIG. 1D is a schematic block diagram of a detailed block structure of the image capture module of FIG. 1 according to a preferred embodiment of the present invention.

2 is a millimeter wave vibration sensing module according to a preferred embodiment of the present invention, which stereoscopically scans and locates each object or environmental state in a localized area or a specific space, and predicts and generates the localization information or sound sensing module of each object or environmental state Receive/transmit and locate each object or environmental state in the regional environment or specific space by sound, and predict and generate the positioning information of each object or environmental state or the wireless positioning module is set in the regional environment or specific space, and generates and communicates with The positioning information or image capture module between devices (such as smart phones) obtains the status image of each object or environment state in the regional environment or a specific space for (real-time) monitoring to obtain the positioning information, which is a schematic diagram of its use.

FIG. 3 is a schematic block diagram of a block structure in which a Mesh router is set as a Mesh coordinator according to a preferred embodiment of the present invention.

Description of reference numerals: 1-image output device; 2-processing unit; 21-image construction module; 22-image post-production module; 23-database; 24-feature comparison module; 3-network gateway; 4-Mesh router; 41-Mesh coordinator; 5-millimeter wave vibration sensing module; 51-first processor; 52-millimeter wave signal generator; 53-transmitting unit; 54-receiving unit; 6-sound sensing module;61 -Sound sending element; 62-Radio element; 63-Second processor; 631-Analog to digital converter; 7-Wireless positioning module; 71-Third processor; 8-Image capture module; 81-Image capture equipment; 82-fourth processor; 9-mobile source; 100-network; 200-Mesh network; 300-regional environment or specific space; 400-communication device; 42-portable intelligent device.

Detailed ways

Please refer to FIG. 1 , the present invention provides a spatial positioning system integrating virtual and real information, which includes: an image output device 1, a processing unit 2, a plurality of network gateways 3, a plurality of mesh routers 4, a millimeter wave vibration sensor A detection module 5 , a sound sensing module 6 , a wireless positioning module 7 and an image capture module 8 .

The processing unit 2 is communicatively connected to the image output device 1. The image output device 1 can be a display screen, a projector or a wearable smart device. A preferred wearable smart device is, for example, a smart glasses. The processing unit 2 There is an image data building module 21 and an image post-processing module 22 , and the processing unit 2 is connected to a database 23 in communication. In this embodiment, the processing unit 2 can be specifically formed by an electronic device, such as a mainframe computer (PC), a notebook computer (NB), a smart phone or other devices capable of processing signals. The map asset building module 21 may specifically be a software application (such as mainframe computer (PC) software or an app), of course, may actually be a map making software, but not limited thereto. Similarly, the image post-processing module 22 can be a software application program. In addition, the database 23 is provided with a feature comparison module 24 for comparing the data features of the database 23 .

The plurality of network gateways (Gateways) are communicably connected to the processing unit 2 via the network 100 . The plurality of Mesh routers (Routers) 4 constitute the Mesh network 200, and the network gateways 3 are connected to each other by wired or wireless communication. Pass the information smoothly. In this embodiment, in accordance with FIG. 3 , one of the Mesh routers 4 is set as a Mesh coordinator 41 (Coordinator) to facilitate network coordination.

Please refer to FIG. 1A , wherein the millimeter-wave vibration sensing module 5 is provided with a first processor 51 , a millimeter-wave signal generator 52 , a transmitting unit 53 and a receiving unit 54 , and the millimeter-wave signal generator 52 is electrically connected The first processor 51 , the transmitting unit 53 and the receiving unit 54 are respectively electrically connected to the millimeter wave signal generator 52 , and the first processor 51 is communicatively connected to one of the mesh routers 4 .

Please refer to FIG. 1B , wherein the sound sensing module 6 is provided with a sound transmitting element 61 and/or a sound receiving element 62 and a second processor 63 . The second processor 63 is provided with an analog-to-digital converter 631 , which can Convert the signal. The second processor 63 is electrically connected to the sound transmitting element 61 and/or the sound receiving element 62 respectively, and the second processor 63 is communicatively connected to the other Mesh router 4 .

Please refer to FIG. 1C and FIG. 1D , wherein the wireless positioning module 7 is provided with a third processor 71 , and the third processor 71 is communicatively connected to the other Mesh router 4 . The image capturing module 8 is provided with an image capturing device 81 and a fourth processor 82 that are electrically connected to each other, and the fourth processor 82 is communicatively connected to the other Mesh router 4 . As mentioned above, in this embodiment, each of the Mesh routers 4 can be individually mounted on the millimeter wave vibration sensing module 5 , the sound sensing module 6 , the wireless positioning module 7 and the image capturing module 8 to directly establish a mesh network.

Therefore, please continue to refer to FIG. 1 , FIG. 1A and FIG. 2 , in which the background design/monitoring personnel can preferentially build the millimeter-wave vibration sensing module 5 in the regional environment or in the specific space 300 , and place the location information according to the drawings. The asset building module 21 builds on the original image data of the regional environment or the specific space 300 to generate a (original) positioning point. Therefore, the millimeter-wave vibration sensing module 5 can be used (for example, fixed at the eaves or the corner of the wall), so that the millimeter-wave signal generator 52 can use the transmitting unit 53 to conduct multiple pre-determined sampling frequencies to the regional environment or specific space. The object (obstacle or moving source 9) and the environment within 300 transmit a millimeter-wave test signal, and then receive each transmitted millimeter-wave test signal through the receiving unit 54 and touch the object (obstacle or moving source 9 (vibration)) and The millimeter wave reflection signal formed by the reflection of the environment, the signal characteristic value is obtained by the ratio of the millimeter wave test signal and the millimeter wave reflection signal, so that the processing unit 2 can measure the distance according to the radar principle (ie, the millimeter wave vibration sensing module 5 and ( (each) obstacle, (each) moving source 9, the distance between the environment (terrain)), and make the map resource building module 21 calculate (real-time) to generate at least one of the object (obstacle or moving source 9) or environment In this way, the positioning positions of (each) obstacle, (each) moving source 9 and the environment can be calculated, and then merged with the original image data.

Furthermore, the image post-processing module 22 can post-process and generate an image that can accurately simulate the object (obstacle) according to the signal characteristic value of the object (obstacle or moving source 9), the environment (terrain) and the (real) real image. or the simulation image of the moving source 9) and the environment (terrain), so that the original image and the (original) positioning point information of the millimeter wave vibration sensing module 5 and (each) obstacle, (each) moving source 9, and the environment are mutually The positioning position information and the simulated image information between them are superimposed and sent to the image output device 1 for viewing by background monitoring personnel or users. In this embodiment, the object is a non-living time (obstacle), such as a wall, a pillar, an eaves, furniture or other objects with metal (including metal) in the indoor space, so as to facilitate millimeter wave sensing, and can be a light switch, Lamps, door curtains, door panels, etc., the object is alive (movement source 9), such as a person or an animal (with its own vibration source or vibration generated by heartbeat or walking and moving).

1, in addition to being able to be sensed by the millimeter wave vibration sensing module 5, processed by the processing unit 2 and made the image resource building module 21 generate the object (obstacle or moving source) 9), at least one positioning information of the environment, wherein, the aforementioned signal characteristic value data is stored in the database 23 for analysis and comparison, so that the next millimeter wave vibration sensing module 5 (after being set at different positions) senses the acquired signal characteristics The value is sent and stored in the database 23, and the feature comparison module 24 is used for feature comparison. According to the signal feature value stored in the first record and its comparison with the subsequently received signal feature value, it is determined that each signal feature value has The corresponding object is processed by the processing unit 2 and information is sent to the image output device 1 to display the corresponding image information, so that the image created by the image construction module 21 has learning properties.

In this embodiment, in addition to the foregoing, please refer to FIG. 1 , FIG. 1D and FIG. 2 , wherein, in the same regional environment or specific space 300 , the (original) positioning point of the millimeter-wave vibration sensing module 5 is constructed at The image capture module 8 can be set at the same positioning point to be used together with the millimeter wave vibration sensing module 5. In this embodiment, the image capture device 81 can be a plurality of multi-directional cameras, cameras or image sensors (CCD or CMOS). device). Therefore, the image signal of the object (obstacle or moving source 9) and the environment (terrain) can be obtained (in real time) by the image capture module 8. The image signal is, for example, different angles, different angles of obstacles or terrain. The colorimetric analysis or optical analysis of the images captured multiple times in the direction, as well as the displacement positioning and image analysis of the images captured multiple times after the moving source 9 is moved, are sent through each of the Mesh routers 4 and each of the network gateways 3 To the processing unit 2, the processing unit 2 processes and causes the image resource building module 21 to generate the object (obstacle or moving source 9), the location (image) information of the environment, and superimpose the original image data, and It can be displayed on the image output device 1 to improve the integrity and correctness of the overall image data.

Therefore, by using the image output device 1 (such as smart glasses), the user can directly view the regional environment or the object (obstacle or moving source 9 ) in the specific space 300 and the comprehensive image information of the environment through operation selection. , (vibration) simulation image or video real image (selected by operation), of course, users can also observe real-life images and objects (obstacles or moving sources 9), environment (vibration) simulation virtual images through smart glasses The relative positional relationship between the objects (obstacles), the map resources of the environment, and the (vibration) simulated virtual image of the moving source 9, so as to form the integration of virtual and real information presentation and spatial positioning.

In addition, as shown in FIG. 1, in addition to presenting information on the image output device 1 (such as smart glasses), this embodiment further includes a portable smart device 42, such as a smart phone, a notebook computer, a large mainframe A computer (PC) or smart glasses, especially when it is a smart glasses, can also directly communicate with one of the plurality of Mesh routers 4, and can also (real-time) obtain the same image information obtained by the image output device 1 and images, without being limited by distance (for example, the portable intelligent device 42 and the user are abroad), and remote monitoring can be shared.

Preferably, in conjunction with FIG. 1 , FIG. 1B , FIG. 1C and FIG. 2 , in the same regional environment or specific space 300 , except that the millimeter-wave vibration sensing module 5 and the image capture are constructed at the (original) positioning point In addition to the module 8 , the sound sensing module 6 or the wireless positioning module 7 can be installed in the regional environment or the specific space 300 at the same (original) positioning point. Therefore, through the sound receiving element 62 or the sound transmitting element 61 of the sound sensing module 6, it is possible to know (in real time) the sound localization information (single direction) of the object (especially when the moving source 9 is a human or an animal) or the environment. Or make a sound notification, or even when the sound sending element 61 and the sound receiving element 62 are provided at the same time, the mobile source 9 or the environment can be positioned more accurately by the sending and receiving of sound, also through the Mesh router. 4 and each of the network gateways 3 are sent to the processing unit 2, which is processed by the processing unit 2 and causes the image resource building module 21 to generate the (real) sound localization information of the mobile source 9 or the environment, which is superimposed on the original image. The data can be presented and transmitted on the image output device 1 (which can be selected by operation) for broadcast, so as to improve the integrity and correctness of the overall picture data.

And, further, in conjunction with FIG. 1 , FIG. 1C and FIG. 2 , wherein the wireless positioning module 7 can also be deployed in the same regional environment or the (original) positioning point on its map in the specific space 300 . In this embodiment, the wireless positioning module 7 can be a Bluetooth locator (iBeacon) or a WiFi device, and the local environment or the object in the specific space 300 can be a communication device 400 (for example, a communication device 400 , in addition to the mobile source 9 ). smartphone, smart glasses). When the communication device 400 held by the user enters the local environment or the specific space 300, and operates or does not operate the corresponding Beacon application service (the corresponding mobile application (App)) on the communication device 400, the communication Between the device 400 and the wireless positioning module 7, in addition to generating a specific identification ID (identity) broadcast (pushing) by the wireless positioning module 7 to the communication device 400, the communication device 400 can also use the signal strength (RSSI) The measured value of the signal generates the signal characteristic value, which is also sent to the processing unit 2 by the Mesh router 4 and each of the network gateways 3, processed by the processing unit 2, and causes the image data construction module 21 to generate the communication device 400. The Bluetooth positioning map data of the positioning information is superimposed on the original map data, so that all the aforementioned map data can be combined and used to improve the integrity and correctness of the overall map data.

In addition, as shown in FIG. 1 and FIG. 2 , in the implementation of the present invention, the millimeter wave vibration sensing module 5, the sound sensing module 6, the wireless positioning module 7 and the image capturing module 8 can also be installed on obstacles or moving sources. 9 (for example, people or animals), the integration of virtual and real information and spatial positioning are formed.

The above discussion is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; therefore, equivalent modifications, combinations, substitutions or conversions are made without departing from the spirit and scope of the present invention. , all should be covered within the protection scope of the present invention.

Claims (6)

1. A virtual-real information integration space positioning system is characterized by comprising:

an image output device;

the processing unit is in communication connection with the image output device, is provided with a figure information building module and an image post-molding module, and is in communication connection with a database;

at least one network gateway, which is connected with the processing unit through network communication;

the Mesh routers are in communication connection with the network gateway;

the millimeter wave vibration sensing module is provided with a first processor, a millimeter wave signal generator, a transmitting unit and a receiving unit, the millimeter wave signal generator is electrically connected with the first processor, the transmitting unit and the receiving unit are respectively electrically connected with the millimeter wave signal generator, and the first processor is in communication connection with one Mesh router;

the second processor is provided with an analog-to-digital converter and is respectively and electrically connected with the sound sending element and/or the sound receiving element, and the second processor is in communication connection with the other Mesh router;

the wireless positioning module is provided with a third processor which is in communication connection with the other Mesh router;

the image acquisition module is provided with an image acquisition device and a fourth processor and is electrically connected with each other, and the fourth processor is in communication connection with the other Mesh router; the signal characteristic value of the object or the environment state acquired by the millimeter wave vibration sensing module or the sound sensing module or the wireless positioning module or the image acquisition module is sent to the processing unit through each Mesh router and the network gateway, the processing unit processes the signal characteristic value and enables the image construction module to generate at least one piece of positioning information of the object or the environment state to be displayed on the image output device, and the image post-molding module is used for post-molding and generating a simulation image simulating the object or the environment state according to the signal characteristic value of the object or the environment state and a real image to be displayed on the image output device so as to form integration of virtual and real information display and space positioning.

2. The virtual-real information integration space positioning system of claim 1, wherein the image output device is a display screen, a projector, or a wearable intelligent device.

3. The virtual-real information integrated spatial location system of claim 1, wherein the wireless location module is a bluetooth locator or a WiFi device.

4. The virtual-real information integrated spatial locality system of claim 1, further comprising a portable intelligent device communicatively coupled to one of the plurality of Mesh routers.

5. The virtual-real information integration space positioning system of claim 1, wherein the database is provided with a feature comparison module.

6. The virtual-real information integration space positioning system of claim 1, wherein one of the Mesh routers is configured as a Mesh coordinator.

Images