TWI891318B - A system for automated road safety and a method thereof - Google Patents

Abstract

The present invention discloses a system for automated road safety and a method thereof. The system comprises an image recognition module for recognizing a road image so as to produce a first recognition result; an object tracking module connected to the image recognition module for tracking at least one object information based on the first recognition result so as to produce a corresponding trajectory; a computation processing module connected to the image recognition module and the object tracking module for performing computation analysis based on the at least one object information and the trajectory so as to produce a corresponding alert information; and an output device connected to the computation processing module for outputting the alert information. The method is executed with the system so as to achieve automated guidance and alerts to road users, and foster a good road safety awareness.

Description

Automated road safety system and method thereof

A safety system and method thereof, in particular, relates to an automated road safety system and method thereof.

According to statistics, the number of people killed or injured in traffic accidents in Taiwan each year is quite alarming. In 2022, there were more than 370,000 traffic accidents, with an annual growth rate of 4.9%. At the same time, the overall number of deaths and injuries increased by 4.7% annually. The growth rate can be said to be quite rapid. According to analysis, Taiwan’s traffic planning has long been focused on improving traffic efficiency, but lacks a people-oriented traffic concept, resulting in continuous traffic in urban areas. As roads become more crowded, pedestrian space becomes less available. This is not only due to drivers' bad habits of not stopping for pedestrians, but also due to vehicles constantly yielding to each other, and even to large vehicles running into blind spots. Therefore, in order to implement traffic safety improvements, in addition to the importance of road engineering planning and road traffic education, the application of technology to guide and remind road users to pay attention to road conditions is also a crucial issue.

To this end, the present invention provides an automated road safety system and method. These systems can perform identification and analysis based on existing intersection surveillance cameras to assist in guiding and alerting road users. For example, when a pedestrian is on a zebra crossing and the intersection light is green, a warning light is displayed to alert drivers to the presence of pedestrians ahead and to yield to them. Alternatively, when a large vehicle is about to turn, the system warns oncoming vehicles to move out of the vehicle's blind spot. Furthermore, when an ambulance, fire truck, or police car is on duty and passing through the road section, the system can guide vehicles ahead of the ambulance, fire truck, or police car one by one to move in a designated direction or to stop at a designated location based on the detected distance and direction of the vehicle, significantly improving road safety.

The primary objective of this invention is to provide an automated road safety system that uses an image recognition module and an object tracking module to obtain object information and trajectory, thereby analyzing its safety level and guiding road users to stop or move in a designated direction, thereby improving road quality.

Another objective of the present invention is to provide an automated road safety method that uses road images to identify and track object information and their trajectory to determine whether an abnormal situation is currently occurring. The method then provides warning information to guide road users based on the actual situation, thereby reducing the occurrence of traffic accidents.

To achieve the aforementioned objectives, one embodiment of the present invention discloses an automated road safety system comprising: an image recognition module for recognizing a road image and generating a first recognition result; an object tracking module signal-connected to the image recognition module and tracking at least one object information therein based on the first recognition result to generate at least one corresponding radial track; an operation processing module signal-connected to the image recognition module and the object tracking module, performing operation analysis based on the at least one object information and the at least one radial track to generate a corresponding warning message; and an output device signal-connected to the operation processing module and outputting the warning message.

In a preferred embodiment, the output device includes a warning device for outputting the warning information, wherein the warning device presents the warning information in the form of light, voice, text, image, or a combination thereof.

In a preferred embodiment, the output device includes an electronic device for receiving and displaying the warning information, and the electronic device includes a positioning module for providing location information of the electronic device.

In a preferred embodiment, the system includes a sound recognition module, signal-connected to the computational processing module, configured to recognize sound information and generate a second recognition result. When the second recognition result is a specific on-duty vehicle, the computational processing module performs computational analysis based on the first and second recognition results to generate the corresponding warning information. The specific on-duty vehicle is selected from a police car, a fire truck, an ambulance, or a combination thereof.

In a preferred embodiment, it includes: a capture device, which is signal-connected to the image recognition module and the sound recognition module respectively, for obtaining the road image and the sound information respectively, wherein the capture device is selected from a road intersection monitoring system, a driving recorder, or a combination thereof.

To achieve another of the aforementioned objectives, one embodiment of the present invention discloses an automated road safety method, comprising: acquiring a road image; recognizing the road image to generate a first recognition result; tracking at least one object information in the first recognition result to generate at least one corresponding radial track; and computing the at least one object information and the at least one radial track to output a warning message.

In a preferred embodiment, the warning information is presented in the form of light, voice, text, image or a combination thereof.

In a preferred embodiment, the method includes the steps of: obtaining location information of an electronic device; and calculating a first distance value between the location information and the first recognition result. When the first distance value is not greater than a first preset distance value, outputting the warning information to the electronic device.

In a preferred embodiment, the method includes the steps of: obtaining sound information from an environment; identifying the sound information to generate a second recognition result; and when the second recognition result is a specific on-duty vehicle, calculating the first recognition result and the second recognition result to output the corresponding warning information, wherein the specific on-duty vehicle is selected from a police car, a fire truck, an ambulance, or a combination thereof.

In a preferred embodiment, the warning information is determined based on the second recognition result to determine a path direction of the specific on-duty vehicle, and based on the first recognition result of the road image obtained in the path direction, the corresponding at least one object information and the at least one path track are tracked. When a second distance value between the specific on-duty vehicle and the at least one object information is no greater than a second preset distance, the warning information is output to guide a vehicle and/or a road user to move in a specified direction.

The beneficial effect of the present invention lies in the automated identification and analysis, which can warn and guide road users to use the road correctly, greatly improving traffic safety and awareness of correct road use.

In order to make the above and/or other purposes, effects, and features of the present invention more clearly understood, the following is a detailed description of the preferred embodiments:

Please refer to Figure 1A, which is a schematic diagram of a system according to an embodiment of the present invention. As shown in the figure, the automated road safety system according to an embodiment of the present invention includes: an image recognition module 1, an object tracking module 2, a computational processing module 3, an output device 4, a sound recognition module 5, and a capture device 6. The computational processing module 3 is signal-connected to the image recognition module 1, the object tracking module 2, the output device 4, and the sound recognition module 5, respectively; the capture device 6 is signal-connected to the image recognition module 1 and the sound recognition module 5, respectively; and the image recognition module 1 is signal-connected to the object tracking module 2 and the capture device 6, respectively. The detailed operation of the system is as follows:

The image recognition module 1 is used to identify whether there are any abnormal situations in the road image. For example, the abnormal situation can be a car accident, a pedestrian trying to cross the road, heavy fog, heavy rain, backlight, and other safety hazards, and generate a corresponding first recognition result.

The object tracking module 2 tracks at least one radial trajectory of at least one object information in the first recognition result. For example, the object information can be a person, a vehicle, an object, or an animal, but is not limited to this. The object tracking module 2 tracks the movement of people, vehicles, objects, or animals on the road in real time, such as the direction of pedestrians walking, the direction of vehicles driving, the direction of objects falling, or the direction of animals moving, etc., but is not limited to this.

The calculation processing module 3 is used to perform calculations on the object information and its path track obtained above to output corresponding warning information. In one embodiment, the object information can be multiple objects on the road, which covers all people, vehicles, objects or animals on the road. Therefore, each object will have a corresponding path track, such as: lights, falling objects, animals crossing the road, people crossing the road, etc. Pedestrians or moving vehicles, so that the computing processing module 3 can generate corresponding warning information based on the current light and road conditions. The warning information can be lights, voice, text, images, or a combination of the above. In addition, the warning information can include guidance corresponding to the current situation, such as: suggestions for switching lanes, guiding other routes to avoid abnormal road sections, paying attention to the distance to the vehicle ahead or pedestrians, etc.

The output device 4 is used to output warning information. In one embodiment, the output device 4 includes a warning device 41. For example, when the warning information is presented in the form of light, it can be presented in the form of text or a specific light signal projected to any conspicuous place in front of the road user's line of sight, but it is not limited to this. It can also be presented in the form of voice to remind the road user at the same time.

In one embodiment, please refer to FIG. 1B, which is a schematic diagram of a system according to one embodiment of the present invention. As shown in the figure, the output device 4 may further include an electronic device 42, wherein the electronic device 42 is used to receive and display warning information. The electronic device 42 may also include a positioning module 421, which is used to obtain the location information of the vehicle and/or road users. The electronic device 42 may be a multimedia device within the vehicle or a mobile device installed in the user, such as a smartphone, tablet computer, or other mobile device with positioning capabilities, but is not limited to this.

The sound recognition module 5 is used to recognize sound information to generate a corresponding second recognition result. For example, when a police car, fire truck or ambulance is on duty, the alarm bell will be turned on while driving. Therefore, when a specific alarm bell is recognized, the sound recognition module 5 can identify the corresponding specific vehicle on duty, so that the calculation processing module 3 can perform a second recognition based on the first recognition result and the second recognition result. The system performs computational analysis and outputs corresponding warning information. That is, if a specific on-duty vehicle appears, the system will capture the road image near the specific on-duty vehicle based on the second recognition result for recognition, and perform computational analysis based on the first recognition result to generate the corresponding warning information. This warning information can be used to guide the specific on-duty vehicle to use the road first, but is not limited to this.

The capture device 6 is used to obtain road image and sound information respectively. In one embodiment, the capture device 6 can be a road intersection monitoring system, a driving recorder, or a combination thereof, but is not limited thereto.

To more clearly illustrate the operation of one embodiment of the present invention, please refer to Figure 2, which is a flow chart of the method of one embodiment of the present invention. As shown in the figure, the automated road safety method of one embodiment of the present invention has the following steps:

Step S1: Obtain a road image;

Step S2: Recognize the road image to generate a first recognition result;

Step S3: Tracking at least one object information according to the first recognition result to generate at least one corresponding radial track; and

Step S4: Calculate the at least one object information and the at least one line path, and output a warning message.

As shown in step S1, the road image can be obtained through a road intersection monitoring system or a driving recorder, but is not limited thereto.

In one embodiment, environmental sound information can be obtained, which can also be obtained from a road intersection monitoring system, but is not limited thereto.

As shown in step S2, the acquired road image is continuously recognized to generate a first recognition result. The first recognition result can be a variety of preset abnormal situations. For example, the abnormal situation can be a car accident, a pedestrian trying to cross the road, heavy fog, heavy rain, backlight, and other safety hazards, but is not limited to these.

In one embodiment, recognition can also be performed based on sound information to generate a corresponding second recognition result, thereby identifying a specific vehicle, but the present invention is not limited to this.

As shown in step S3, based on the first recognition result obtained in the previous step, at least one object information is tracked to generate at least one corresponding radial track, wherein the object information can be a pedestrian, a vehicle, an object, or an animal, and the amount of object information is not limited.

As shown in step S4, an operation analysis is performed based on the object information and the path track to output corresponding warning information based on the current situation. For example, when a pedestrian wants to cross the road, the pedestrian's path track and the light signal in the direction of his or her path, as well as the path track of the vehicle approaching the pedestrian's location, can be used to warn the pedestrian to pay attention to the vehicle on the left or right side, and warn the vehicle to stop and look, and then drive according to the light signal, but this is not limited to this.

In one embodiment, when the first distance between the location information of the electronic device and the abnormal road section is not greater than a first preset distance, corresponding warning information can be output to the corresponding electronic device. The electronic device can be installed inside the vehicle or a mobile device carried by a passerby, but is not limited to this.

In one embodiment, when the second recognition result is a specific on-duty vehicle, the first recognition result is calculated to output corresponding warning information. For example, the specific on-duty vehicle can be a police car, a fire truck, or an ambulance. Therefore, when a specific on-duty vehicle is identified, the direction of the specific on-duty vehicle (towards a certain intersection or the opposite direction) and its position can be calculated based on the sound information, and the road of the location can be captured based on this location. The system uses road images to identify vehicles and/or pedestrians near the location of a specific duty vehicle and tracks the paths of surrounding vehicles and/or pedestrians. When the paths of surrounding vehicles and/or pedestrians overlap with the path of the specific duty vehicle, it can output corresponding warning information to vehicles and/or pedestrians near the location of the specific duty vehicle to guide or warn road users to avoid the specific duty vehicle.

In one embodiment, when a specific duty vehicle is identified, the specific duty vehicle and the information of objects near the specific duty vehicle and their trajectory can be confirmed through image recognition, and a second distance value between the trajectory of surrounding vehicles and/or pedestrians and the trajectory of the specific duty vehicle is further calculated. If the second distance value is no greater than a second preset distance value, it means that the specific duty vehicle is about to pass a location with surrounding vehicles and/or pedestrians, and a warning message can be output to the electronic device of the nearby road user to warn the road user (vehicle and/or pedestrian) to move in a specified direction, wherein the second preset distance value can be 80-100 meters or set according to needs, but is not limited thereto.

In one embodiment, the warning information may include different warning countermeasures based on different abnormal situations. The abnormal situation may be a state with potential safety hazards, such as the aforementioned situations of pedestrians crossing the road, falling objects, and car accidents, or it may also be a traffic jam, landslide, road construction, or even heavy rain, heavy fog, or backlight, but is not limited to these.

The warning information may be presented in the form of lights, voice, images, text, or a combination thereof. In one embodiment, when a vehicle's green light indicates a right turn and a pedestrian is about to cross the road, text may be projected in the form of lights, such as "Pedestrian crossing on the right (left), please yield." Alternatively, a graphic indicating "please yield" may be projected directly in the form of lights, or even combined with voice to output the warning information to the user's mobile device, as long as the purpose of the reminder is achieved.

To more clearly illustrate the operation of this embodiment, the following is a detailed description:

In one embodiment, when a road image identifies a vehicle turning at an intersection where pedestrians are about to cross or are currently crossing, the system automatically warns the vehicle's driver and pedestrian based on their trajectory, urging them to pay attention and yield to the party with the right of way. For example, if the traffic light for the direction the pedestrian is about to cross is green, the system instructs the driver to yield. The system also warns the pedestrian of vehicles to be careful of vehicles on their right (or left) side. This reduces the risk of vehicles failing to notice pedestrians due to blind spots when turning at an intersection. Furthermore, it can prevent traffic accidents at night due to insufficient light at dimly lit intersections.

In one embodiment, when a road image identifies a police car, fire truck, or ambulance on duty, a warning message is triggered in advance when the police car, fire truck, or ambulance approaches nearby vehicles and/or pedestrians. This may be by projecting route-guiding lights, playing voice reminders, or even on the pedestrian's electronic device, to prompt the pedestrian to quickly avoid the vehicle or pedestrian.

Furthermore, guidance routes can be provided based on the actual situation at the intersection, allowing drivers and pedestrians in different lanes to quickly clear the road according to the guidance routes. For example: When an ambulance is identified in the inside lane and is 80-100 meters away from the vehicle in front, a warning light with a directional guidance arrow will be projected in front of the vehicles in the inside lane to guide drivers to move their vehicles to the left side of the intersection in order to prevent vehicles from blocking the inside lane. At the same time, warning lights will be projected in other lanes to indicate that an ambulance is about to pass and to stop, facilitating the ambulance's passage.

Even better, when a police car, fire truck, or ambulance is on duty, the vehicle's location information can be used to more accurately track the vehicle's trajectory and provide advance warnings or guidance when approaching intersections, significantly reducing the risk of pedestrians and drivers not noticing the sirens and colliding with specific on-duty vehicles.

In one embodiment, when a road accident is detected by road imaging, corresponding warning information is output to the electronic devices of vehicles and/or pedestrians near the accident scene based on the trajectory of the vehicles and/or pedestrians near the accident scene. The warning information may include guidance to an alternative road or switching to another safe lane, and/or reminding the driver to reduce speed and pay attention to the traffic ahead.

In one embodiment, when road images identify road maintenance safety measures or fences, corresponding warning information is output to the electronic devices of vehicles and/or pedestrians near the road maintenance safety measures or fences based on the track of the vehicles and/or pedestrians near the accident scene. The warning information may include guidance to an alternative road or switching to another safe lane, and/or reminding the driver to reduce speed and pay attention to the traffic ahead.

In one embodiment, when a road image identifies a landslide or a road section that affects vehicle traffic, corresponding warning information is output to the electronic devices of vehicles and/or pedestrians near the landslide or the location that affects vehicle traffic based on the trajectory of the vehicles and/or pedestrians near the location. The warning information may include guiding to an alternative road or switching to another safe lane, and/or reminding the driver to reduce speed and pay attention to the traffic ahead.

In one embodiment, when road images identify a road section with heavy fog, heavy rain, or backlight, corresponding warning information is output to the electronic devices of vehicles and/or pedestrians near the fog, heavy rain, or backlight based on the track of the vehicles and/or pedestrians near the fog, heavy rain, or backlight. The warning information may include reminding the driver/pedestrian of the distance to the road section, slowing down to pay attention to the vehicle ahead, and/or warning the driver/pedestrian of the distance between the vehicle ahead and the driver, so as to help the driver and/or pedestrian increase their vigilance.

In summary, the present invention provides an automated road safety system and method that utilizes image recognition technology to monitor the real-time movements of people, vehicles, and objects. Based on the actual road conditions, the system provides warnings and/or guidance, helping pedestrians and drivers develop good road safety habits. Furthermore, the system can reduce the incidence of traffic accidents caused by blind spots, thereby achieving the purpose of the present invention.

However, the above description is only a preferred embodiment of the present invention and does not limit the scope of implementation of the present invention. Therefore, any simple equivalent changes and modifications made according to the scope of the patent application and the content of the specification of the present invention are still within the scope of the patent of the present invention.

1. Image Recognition Module 2. Object Tracking Module 3. Processing Module 4. Output Device 4. Alarm Device 4. Electronic Device 4. Positioning Module 5. Voice Recognition Module 6. Capture Device S1. Step S2. Step S3. Step S4. Step

Figure 1A: A schematic diagram of a system according to one embodiment of the present invention; Figure 1B: A schematic diagram of a system according to one embodiment of the present invention; and Figure 2: A flow chart of a method according to one embodiment of the present invention.

1: Image Recognition Module

2: Object Tracking Module

3: Computational processing module

4:Output device

41: Warning device

5: Voice Recognition Module

6: Capture device

Claims (8)

An automated road safety system comprises: A capture device for capturing a road image and sound information; An image recognition module, signal-connected to the capture device, for recognizing the road image and generating a first recognition result; A sound recognition module, signal-connected to the capture device, for recognizing the sound information and generating a second recognition result; An object tracking module, signal-connected to the image recognition module, for tracking at least one object information in the first recognition result and generating at least one corresponding radial track; A calculation processing module is connected to the image recognition module, the sound recognition module and the object tracking module respectively, and performs calculation analysis based on the at least one object information and the at least one radial track to generate a corresponding warning message. When the second recognition result is a specific on-duty vehicle, the calculation processing module performs calculation analysis based on the first recognition result and the second recognition result to generate the corresponding warning message. At this time, the warning The information includes determining a travel direction of the specific on-duty vehicle based on the second recognition result, tracking the corresponding at least one object information and the at least one radial track based on the first recognition result of the road image obtained in the travel direction, and outputting the warning information when a second distance value between the specific on-duty vehicle and the at least one object information is no greater than a second preset distance to guide a vehicle and/or a road user to move in a designated direction; and an output device, signal-connected to the computing processing module, outputting the warning information. According to the automated road safety system described in claim 1, the output device includes a warning device for outputting the warning information, wherein the warning device presents the warning information in the form of light, voice, text, image, or a combination of the above. The automated road safety system according to claim 2, wherein the output device includes an electronic device for receiving and displaying the warning information, the electronic device includes a positioning module for providing location information of the electronic device, and the electronic device is selected from a multimedia device, a mobile device, a tablet computer, a laptop computer, or a combination thereof. The automated road safety system according to claim 1, wherein the specific on-duty vehicle is selected from a police car, a fire truck, an ambulance, or a combination thereof. The automated road safety system according to claim 1, wherein the capture device is selected from an intersection monitoring system, a dashcam, or a combination thereof. An automated road safety method comprises the following steps: Acquiring a road image; Recognizing the road image to generate a first recognition result; Tracking at least one object information in the first recognition result to generate at least one corresponding radial track; Calculating the at least one object information and the at least one radial track to output a warning message; Acquiring sound information from the environment; Recognizing the sound information to generate a second recognition result; and When the second recognition result indicates a specific on-duty vehicle, the first and second recognition results are calculated and the corresponding warning information is output. The specific on-duty vehicle is selected from a police car, a fire truck, an ambulance, or a combination thereof. The warning information determines a travel direction of the specific on-duty vehicle based on the second recognition result, tracks the corresponding at least one piece of object information and at least one radial track based on the first recognition result of the road image acquired in the travel direction, and outputs the warning information when a second distance between the specific on-duty vehicle and the at least one piece of object information is no greater than a second preset distance to guide a vehicle and/or a road user to move in a designated direction. The automated road safety method according to claim 6, wherein the warning information is presented in the form of lights, voice, text, images, or a combination thereof. The automated road safety method according to claim 6 comprises the steps of: obtaining location information of an electronic device; and calculating a first distance between the location information and the first recognition result, and outputting a warning message to the electronic device when the first distance is not greater than a first preset distance. The electronic device includes a positioning module configured to provide the location information of the electronic device, and the electronic device is selected from a multimedia device, a mobile device, a tablet computer, a laptop computer, or a combination thereof.