[go: up one dir, main page]

WO2020167110A1 - System and method based on the correlation of interior and exterior visual information in a vehicle for improving safety in manual or semi-automatic driving - Google Patents

System and method based on the correlation of interior and exterior visual information in a vehicle for improving safety in manual or semi-automatic driving Download PDF

Info

Publication number
WO2020167110A1
WO2020167110A1 PCT/MX2019/000016 MX2019000016W WO2020167110A1 WO 2020167110 A1 WO2020167110 A1 WO 2020167110A1 MX 2019000016 W MX2019000016 W MX 2019000016W WO 2020167110 A1 WO2020167110 A1 WO 2020167110A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
driver
controllers
visual
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/MX2019/000016
Other languages
Spanish (es)
French (fr)
Inventor
Álvaro IZAGUIRRE SERRANO
Luis Enrique GONZÁLEZ JIMÉNEZ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Instituto Tecnologico Y De Estudios Superiores De Occidente AC
Original Assignee
Instituto Tecnologico Y De Estudios Superiores De Occidente AC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Instituto Tecnologico Y De Estudios Superiores De Occidente AC filed Critical Instituto Tecnologico Y De Estudios Superiores De Occidente AC
Publication of WO2020167110A1 publication Critical patent/WO2020167110A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B7/00—Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00
    • G08B7/06—Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00 using electric transmission, e.g. involving audible and visible signalling through the use of sound and light sources

Definitions

  • the present invention is related to electronic systems in general, in particular it is related to electronic monitoring and control systems applied to motor vehicles and more specifically it refers to a system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving.
  • Our invention is directed to a system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving.
  • a search was conducted to determine the closest state of the art and the following documents were found:
  • Document D1 was located consisting of the patent application US20160297365A1 of Axel Nix of June 13, 2016, which reveals a driver assistance system suitable for use in a vehicle includes a forward vision camera mounted on the front windshield of the vehicle. vehicle and an event recorder that has an image processor and memory. Based at least in part on the image processor's processing of the image data captured by the camera, a lane departure warning is generated if, when the vehicle's turn signal is not activated, at least one of ( i) the vehicle is heading towards a lane marking and (ii) the vehicle is at a certain distance from an edge of a lane line for a specified time. Based at least in part on Image Processor processing of the image data captured by the camera, a forward collision alert is generated if the vehicle is heading towards a given object with the gap between the vehicle and the given object. which is less than a threshold.
  • the document does not include an internal camera so no allows monitoring of the driver's status and therefore its efficiency is low.
  • Document D1 does not reveal or suggest the association of a pair of cameras, one internal and one external, an Information processing device that includes a processor that will analyze the data from these cameras and will also be able to communicate with the different electronic control units ( ECUs for its acronym in English) in the car system to be able to alert the driver or take a specific action to avoid harm to the user through a Human Machine Interface (HMI).
  • ECUs electronice control units
  • HMI Human Machine Interface
  • D2 EP3161507A1 by Dooley Damien et. to the. of April 29, 2015 which reveals a method to track a target vehicle (9) approaching a motor vehicle (1) by means of a camera system (2) of the motor vehicle (1).
  • a temporal sequence of Images (10) of an environmental region of the motor vehicle (1) is provided by means of at least one camera (3) of the camera system (5).
  • the target vehicle (9) is detected in an image (10) of the sequence by means of an Image processing device (5) of the camera system (5) based on a characteristic of a front (11) or a part rear of the target vehicle (9) and then the target vehicle (9) followed by subsequent images (10) of the sequence based on the detected characteristic.
  • the image processing device (5) detects at least one predetermined characteristic of a side flank (14) of the target vehicle (9) in one of the Later images (10) of the sequence and after detecting the characteristic of the lateral flank (14) the target vehicle (9)
  • the images (10) of the sequence are tracked by the characteristic of the lateral edge (14).
  • a temporal sequence of images of an environmental region of the motor vehicle is provided by means of at least one camera of the camera system.
  • the target vehicle is detected in a sequence image by means of an image processing device of the camera system based on a characteristic of a front or rear of the target vehicle, and then the target vehicle is tracked through subsequent images of the sequence based on the detected characteristic.
  • the invention also relates to a camera system for a motor vehicle, which is designed to carry out said method, as well as to a motor vehicle with said camera system.
  • Document D3 US9308914 by Sun Bo et. to the. of January 23, 2015, which reveals an advanced driver assistance system of a vehicle to perform one or more functions of Driver assistance includes an environmental sensor, a driving status monitor, an Activation Switch and a driver assistance controller.
  • the environmental sensor detects an environment around the vehicle.
  • the controller health monitor monitors the controller and determines a controller abnormal condition.
  • the activation switch is operable by the controller to activate or deactivate the driver assistance functions.
  • the driver assistance controller performs one or more driver assistance functions to control the vehicle according to a target behavior, where the target behavior is defined according to the environment surrounding the vehicle.
  • the driver assistance controller activates the driver assistance functions when the driver status monitor detects the abnormal situation with the activation switch set to deactivate.
  • the D3 document does not reveal or suggest the association of a pair of cameras, one internal and one external, an information processing device that comprises a processor that will analyze the data from these cameras and will also be able to communicate with the different ECUs in the system of the car to be able to warn the driver or take a specific action to avoid harm to the user through a human-machine interface (HMI).
  • HMI human-machine interface
  • the main objective of the invention is to make available a system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving that allows increasing driver safety, through monitoring visual of the exterior (other vehicles, signs, people, buildings, etc.) and the interior (loss of driver attention, state of driver, number and position of occupants, etc.) of the vehicle and correlate these two sources of information to carry out a preventive action (avoid ignition or acceleration, alert the passenger with light or noise, turn on the turn signals, etc.) or corrective (decelerate, activate automatic driving, maintain safety distances, etc.).
  • a preventive action avoid ignition or acceleration, alert the passenger with light or noise, turn on the turn signals, etc.
  • corrective decelerate, activate automatic driving, maintain safety distances, etc.
  • Another objective of the invention is to make available said system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, where the proposed visual monitoring includes a wide visual field that normally it is not within the driver's reach.
  • Another objective of the invention is to make available said system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, where the output of the proposed system allows preventive and corrective actions , can be used to reduce traffic accidents, make the use of traffic more time and energy efficient, and improve the driving experience of a vehicle.
  • Another objective of the invention is to make available said system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, which also confers on the vehicle automatic reaction capacity in the event of risk situations detected by it.
  • the system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving allows the correlation of images obtained from the interior and exterior of a car and is composed by at least one external camera arranged in a front area of the vehicle looking towards the driver's field of vision in the front direction of the vehicle and at least one internal camera arranged inside the vehicle cabin focused towards the driver's area; a main information processing and control device that comprises a processor that will analyze the data (images-video) from said cameras and also comprises communication means to communicate with the plurality of engine control units (ECU's) of the electronic system of control of the vehicle and also configured to receive information from the plurality of sensors of the vehicle, to be able to notify the driver or take a specific action to avoid a damage to the user via a human machine interface (HMI) with visual or audible warning and warning alarms.
  • ECU's engine control units
  • HMI human machine interface
  • the system allows through said at least one interior camera to determine where the driver is looking and will analyze the maximum visual field based on the position of the driver's eyes and head; said at least one second exterior camera mounted externally preferably on the windshield of the vehicle or other area that allows viewing the front area of the vehicle will be recording the image in front of the automobile and the objects within visual field.
  • the system will correlate the objects in the frontal image with the visual field detected by the position of the eyes and will analyze the objects to detect hazards based on a classification and a detection sensitivity adjustment, as well as information from other sensors, such as by example the speed of the car, among other sensors.
  • the system When the system detects a pedestrian in front and sees that it is not within the driver's field of vision, then a serious alert will be issued to the driver, instead if a bird is detected it should be ignored. In addition, it will provide the ability to detect traffic signs and see if the driver was able to observe them, otherwise an alarm of medium severity will be issued.
  • the system will decide if it is necessary to take any control primitive (i .e. Secure profiles of speed, automatic driving support, task activation in other vehicle ECUs, etc.) or if it is only necessary to notify the driver by means of a visual or audible alarm from the human-machine interface (HM I).
  • any control primitive i .e. Secure profiles of speed, automatic driving support, task activation in other vehicle ECUs, etc.
  • HM I human-machine interface
  • the invention also involves a method to improve safety in manual or semi-automatic driving by a driver, by means of the system of correlation of interior and exterior visual information in a vehicle, as claimed in claim 1, characterized in that it consists in:
  • ECU's engine control units
  • actuators for example, speed controllers, injection controllers, light controllers, ABS brake controllers
  • H M I human-machine interface
  • Figure 1 shows a schematic block diagram of the system based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving.
  • Figure 2 illustrates a schematic diagram of the interior of a vehicle showing the arrangement of the internal and external cameras of the system based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving.
  • Figure 3 shows a conventional perspective of a vehicle where the components of the system based on the correlation of interior and exterior visual information in a vehicle are schematically illustrated to improve safety in manual or semi-automatic driving.
  • Figure 4 shows a flow diagram of the operation of the system based on the correlation of visual information inside and outside in a vehicle to improve driving safety manual or semi-automatic.
  • the system based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, consists of at least one external camera (1) arranged in a zone front of the vehicle (2) looking towards the visual field of the driver's area (3) in the front direction of the vehicle (2) and at least one internal camera (4) arranged inside the cabin (5) of the vehicle ( 2) focused towards the visual field of the driver's area (3); a main information processing and control device (6) comprising a processor that will analyze the data (images-video) from said cameras external and internal (1, 4) and also comprises communication means to communicate with the plurality of engine control units (ECU's) and actuators (7) [for example speed controllers, injection controllers, light controllers, ABS brakes] from the vehicle's electronic control system and also configured to receive information from the plurality of sensors (8) [such as RPM speed sensor, ABS brake sensor, among others] of the vehicle (2), in order to notify the driver (3) or take a specific action to avoid harm to the user through a
  • ECU's engine
  • the system allows through said at least one interior camera (4) to determine where the driver (3) is looking and will analyze the maximum visual field based on the position of the driver's eyes and head (3); said at least one second external camera (1) mounted externally preferably on the windshield of the vehicle (2) or another area that allows viewing the front area of the vehicle (2) will be recording the image in front of the car and the objects within visual field .
  • the system will correlate the objects in the frontal image with the visual field detected by the position of the driver's eyes (3) and will analyze the objects for hazards based on a classification and a detection sensitivity setting, as well as information from a plurality of sensors (8), such as the speed of the car, brake sensors, among others sensors.
  • the system When the system detects a pedestrian in front and sees that it is not within the driver's field of vision, then a serious alert will be issued to the driver (3), on the other hand, if a bird is detected, it must be ignored. In addition, it will provide the ability to detect traffic signs and see if the driver (3) was able to observe it, otherwise an alarm of medium severity will be issued. Based on this information, the system will decide if it is necessary to take any control primitive (ie safe speed profiles, automatic driving support, activation of tasks in other vehicle ECUs, etc.), or if it is only necessary to notify the driver (3) via a visual or audible alarm from the human-machine interface (HMI) (9).
  • HMI human-machine interface
  • the proposed logic system follows the following directives to execute the algorithm: a. It reads the status of the HMI man-machine interface, this reading indicates if the system functionality is active (at 1), since the user is allowed to deactivate it from the graphical interface. In addition, the system allows modifying the sensitivity (a2) for the detection of objects in the Images. By reading the status of the HMI these values can be known.
  • the algorithm executes the reading of the external camera (c1), and the reading of the internal camera (c2), to obtain an external image (c a) from outside the vehicle (front) (c1 b) and to obtain an interior image (c2a) from inside the vehicle (c2b), to be able to transmit this data to the main controller.
  • This is represented on the main controller as the internal and external images.
  • the algorithm will read the sensors to know the state of the vehicle, that is, if the vehicle is moving or if it is on or not, using the signals from the sensors (d 1) to detect the speed (d2) or the revolutions per minute (RPM) (d3).
  • the main controller will analyze the inner image looking for a face within the inner image (e1). Since the interior image (e1) will be captured with a camera from the car dashboard, the location of a face within this image will allow the presence / absence of a driver inside the vehicle to be detected.
  • the algorithm does not detect a driver, but detects that the vehicle is moving (g) through the speed (g 1) and RPM (g 2) sensors, it will send an alert through the HM I (g 3), it will also send control directives (g4) to the injection and ABS controllers to stop the vehicle, because if the car is in motion, but a driver could not be located, this indicates that the driver is in a position that does not allow you to be watching the road, which can cause an accident.
  • the algorithm detects a driver, then the interior image (h 1) will be analyzed again, since it is known in which part of the image the face is located, this face will be analyzed to detect the position of the eyes (h2), thus as the direction in which the driver is looking (h3).
  • the algorithm will calculate the visual range (i 1) of the driver. That is, as far as it is possible for the driver to interpret the image in front of him, through the position of the eyes (i2) and the direction of the gaze (i3).
  • the calculated visual range will be correlated in the image of the external camera. This will allow to define an image containing the image within the driver's visual range (j 3), as well as an image with the part outside the visual range (j 4).
  • the algorithm will analyze the image that is outside the visual range (k 1) in search of objects, in this analysis all objects within the image outside the visual range (k2) will be detected.
  • the algorithm If the algorithm does not detect objects in the image out of visual range, the execution of the algorithm ends. In case it is detect one or more objects, the algorithm will check if the car is moving (L1) through the speed (L2) and RPM (L3) sensors, in case the algorithm is stopped it will finish its execution.
  • the algorithm will perform a categorization of the objects within the image outside the visual range of the driver (m 1). This categorization will take into account the value for the sensitivity (m2) in the detection of objects defined by the user.
  • the algorithm will separate objects like pedestrians, cyclists, or other cars from objects like dogs, birds, fire hydrants, etc.
  • the algorithm will create two categories (m3 warnings and m4 actions):
  • Control directives (fi) will be sent as actions ( ⁇ 1) to prevent accidents in addition to displaying warnings in the HMI (eg detection of pedestrians, cyclists, other cars, etc.).

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)

Abstract

The invention relates to a system based on the correlation of interior and exterior visual information in a vehicle, for improving safety in manual or semi-automatic driving, which is characterised in that it comprises at least one external camera disposed in a front area of the vehicle and facing the driver's visual field in the front direction of the vehicle, and at least one internal camera disposed in the passenger compartment of the vehicle and focused on the area of the driver; a main control and information-processing device comprising a processor that analyses data (images/video) from the external and internal cameras. The system also comprises communication means for communicating with a plurality of engine control units (ECUs) and actuators (for example, speed controllers, injection controllers, light controllers and ABS brake controllers) of the electronic control system of the vehicle and is configured to receive information from a plurality of sensors (for example, RPM speed sensor, ABS brake sensor, among others) of the vehicle, the main control and information-processing device being in communication with a human–machine interface (HMI) with visual or audio warning or notification alarms and configured to alert the driver to take a particular action to avoid harm to the user.

Description

SISTEMA Y MÉTODO BASADO EN LA CORRELACIÓN DE INFORMACIÓN VISUAL INTERIOR Y EXTERIOR EN UN VEHÍCULO PARA MEJORAR LA SEGURIDAD EN CONDUCCIÓN MANUAL O SYSTEM AND METHOD BASED ON THE CORRELATION OF INTERIOR AND EXTERIOR VISUAL INFORMATION IN A VEHICLE TO IMPROVE SAFETY IN MANUAL DRIVING OR

SEMI-AUTOMÁTICA SEMI-AUTOMATIC

CAMPO DE LA INVENCIÓN FIELD OF THE INVENTION

La presente Invención se relaciona con los sistemas electrónicos en lo general, en lo particular se relaciona con sistemas electrónicos de monitoreo y control aplicados a vehículos automotores y más específicamente se refiere a un sistema y método basado en la correlación de información visual interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi-automática. The present invention is related to electronic systems in general, in particular it is related to electronic monitoring and control systems applied to motor vehicles and more specifically it refers to a system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving.

ANTECEDENTES DE LA INVENCIÓN BACKGROUND OF THE INVENTION

En la actualidad se han venido desarrollando diversos sistemas electrónicos de control y monitoreo de ciertas acciones, factores o eventos en múltiples campos tecnológicos, En el ámbito automotriz el desarrollo de tecnología ha venido evolucionando a pasos agigantados en lo referente a las nuevas tecnolog ías y sistemas de comunicación , asi como los sistemas de control y manejo autónomo de vehículos automotores. También ya es muy común el uso de cámaras de video para un monitoreo visual por parte del conductor para la conducción de reversa cuando es necesario estacionarse en combinación con sensores de proximidad que se dispone en diversas zonas del chasis de los vehículos y mediante alarma sonora se avisan de la proximidad de obstácu los u otros vehículos para evitar colisiones durante el manejo y maniobras de estacionamiento o durante la cond ucción . At present, various electronic control and monitoring systems of certain actions, factors or events in multiple technological fields have been developed.In the automotive field, the development of technology has been evolving by leaps and bounds in relation to new technologies and systems. communication systems, as well as control systems and autonomous management of motor vehicles. It is also very common to use video cameras for visual monitoring by the driver for reverse driving when it is It is necessary to park in combination with proximity sensors that are available in various areas of the vehicle chassis and by means of an audible alarm, they are warned of the proximity of obstacles or other vehicles to avoid collisions during handling and parking maneuvers or while driving.

Actualmente en el mercado automotriz se ofertan sistemas de monitoreo de punto ciego, sin embargo, estos sistemas se basan en radares para detección de objetos en los laterales del automóvil, se detecta cualquier objeto sin distinción y solo informa al conductor con un led con los espejos o bien una advertencia en el tablero del automóvil. El enfoque principal de estos sistemas es la detección de objetos en los laterales del automóvil en puntos donde el espejo no puede reflejar estos objetos . Otros sistemas permiten la detección o ausencia del conductor, asi como una evaluación biométrica para ver el estado del conductor y determinar si éste está apto para conducir y generar una advertencia o evitar que el automóvil se encienda o acelere. Currently in the automotive market blind spot monitoring systems are offered, however, these systems are based on radars to detect objects on the sides of the car, any object is detected without distinction and only informs the driver with a led with the mirrors or a warning on the dashboard of the car. The main focus of these systems is the detection of objects on the sides of the car at points where the mirror cannot reflect these objects. Other systems allow the detection or absence of the driver, as well as a biometric evaluation to see the state of the driver and determine if he is fit to drive and generate a warning or prevent the car from starting or accelerating.

Sin embargo, en el ámbito de monitoreo y control de las condiciones de manejo, no existe muchas alternativas que brinden seguridad en la conducción y eviten colisiones cuando los conductores se distaren o manejan cansados con sueño, entre otros factores que comprometen la conducción . However, in the field of monitoring and control of driving conditions, there are not many alternatives that provide driving safety and avoid collisions when drivers are distancing themselves or driving tired while drowsy, among other factors that compromise driving.

Nuestra invención está dirigida a un sistema y método basado en la correlación de información visual interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi- automática. Se realizó una búsqueda para determinar el estado de la técnica más cercano y se encontraron los siguientes documentos: Our invention is directed to a system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving. A search was conducted to determine the closest state of the art and the following documents were found:

Se ubicó el documento D1 que consiste en la solicitud de patente US20160297365A1 de Axel Nix del 13 de junio de 2016, que revela un sistema de asistencia al conductor adecuado para su uso en un vehículo incluye una cámara de visión delantera montada en el parabrisas delantero del vehículo y una grabadora de eventos que tiene un procesador de imágenes y una memoria. Basado al menos en parte en el procesamiento por el procesador de imágenes de los datos de imagen capturados por la cámara, se genera una advertencia de cambio de carril si, cuando no está activada la señal de giro del veh ículo, al menos uno de (i) el vehículo se dirige hacia una marca de carril y (ii) el vehículo se encuentra a cierta distancia de un borde de una linea de carril durante un tiempo determinado. Basado al menos en parte en el procesamiento por el procesador de Imágenes de los datos de imagen capturados por la cámara, se genera una alerta de colisión hacia delante si el vehículo se dirige hacia un objeto dado con la separación entre el vehículo y el objeto dado que es menor que un umbral . Document D1 was located consisting of the patent application US20160297365A1 of Axel Nix of June 13, 2016, which reveals a driver assistance system suitable for use in a vehicle includes a forward vision camera mounted on the front windshield of the vehicle. vehicle and an event recorder that has an image processor and memory. Based at least in part on the image processor's processing of the image data captured by the camera, a lane departure warning is generated if, when the vehicle's turn signal is not activated, at least one of ( i) the vehicle is heading towards a lane marking and (ii) the vehicle is at a certain distance from an edge of a lane line for a specified time. Based at least in part on Image Processor processing of the image data captured by the camera, a forward collision alert is generated if the vehicle is heading towards a given object with the gap between the vehicle and the given object. which is less than a threshold.

El documento no incluye una cámara interna de modo que no permite monitorear el estado del conductor y por lo tanto su eficiencia es baja. The document does not include an internal camera so no allows monitoring of the driver's status and therefore its efficiency is low.

El documento D1 no revela ni sugiere la asociación de un par de cámaras , una interna y una externa, un dispositivo de procesamiento de Información que comprende un procesador que analizará los datos de estas cámaras y además podrá comunicarse con las distintas unidades de control electrónico (ECUs por sus siglas en Inglés) en el sistema del automóvil para poder avisar al conductor o bien tomar una acción determinada para evitar un daño al usuario a través de una Interfaz hombre-máquina (HMI) . Document D1 does not reveal or suggest the association of a pair of cameras, one internal and one external, an Information processing device that includes a processor that will analyze the data from these cameras and will also be able to communicate with the different electronic control units ( ECUs for its acronym in English) in the car system to be able to alert the driver or take a specific action to avoid harm to the user through a Human Machine Interface (HMI).

Se u bicó el documento D2 EP3161507A1 de Dooley Damien et. al. del 29 de abril de 2015, que revela un método para rastrear un vehículo objetivo (9) que se aproxima a un vehículo a motor (1 ) por medio de un sistema de cámara (2) del vehículo a motor (1 ). Se proporciona una secuencia temporal de Imágenes (10) de una región ambiental del vehícu lo de motor (1 ) por medio de al menos una cámara (3) del sistema de cámara (5) . El veh ículo objetivo (9) se detecta en una imagen ( 10) de la secuencia por medio de un dispositivo de procesamiento de Imagen (5) del sistema de cámara (5) basándose en una característica de un frente (11 ) o una parte trasera del vehículo objetivo (9) y luego el vehículo objetivo (9) seguido de Imágenes subsecuentes (10) de la secuencia basada en la característica detectada. El dispositivo de procesamiento de imágenes (5) detecta al menos una característica predeterminada de un flanco lateral ( 14) del vehículo objetivo (9) en una de las imágenes posteriores (10) de la secuencia y después de detectar la característica del flanco lateral (14) el vehículo objetivo (9) Las imágenes (10) de la secuencia son rastreadas por la característica del borde lateral (14) . D2 EP3161507A1 by Dooley Damien et. to the. of April 29, 2015, which reveals a method to track a target vehicle (9) approaching a motor vehicle (1) by means of a camera system (2) of the motor vehicle (1). A temporal sequence of Images (10) of an environmental region of the motor vehicle (1) is provided by means of at least one camera (3) of the camera system (5). The target vehicle (9) is detected in an image (10) of the sequence by means of an Image processing device (5) of the camera system (5) based on a characteristic of a front (11) or a part rear of the target vehicle (9) and then the target vehicle (9) followed by subsequent images (10) of the sequence based on the detected characteristic. The image processing device (5) detects at least one predetermined characteristic of a side flank (14) of the target vehicle (9) in one of the Later images (10) of the sequence and after detecting the characteristic of the lateral flank (14) the target vehicle (9) The images (10) of the sequence are tracked by the characteristic of the lateral edge (14).

Con el método se proporciona una secuencia temporal de imágenes de una región ambiental del vehículo a motor por medio de al menos una cámara del sistema de cámara. El vehículo objetivo se detecta en una imagen de la secuencia por medio de un dispositivo de procesamiento de imágenes del sistema de cámara basado en una característica de una parte delantera o trasera del vehículo objetivo y luego se rastrea el vehículo objetivo a través de imágenes subsiguientes de la secuencia en función de la característica detectada. La Invención también se refiere a un sistema de cámaras para un vehículo a motor, que está diseñado para llevar a cabo dicho método, asi como a un vehículo a motor con dicho sistema de cámara. With the method, a temporal sequence of images of an environmental region of the motor vehicle is provided by means of at least one camera of the camera system. The target vehicle is detected in a sequence image by means of an image processing device of the camera system based on a characteristic of a front or rear of the target vehicle, and then the target vehicle is tracked through subsequent images of the sequence based on the detected characteristic. The invention also relates to a camera system for a motor vehicle, which is designed to carry out said method, as well as to a motor vehicle with said camera system.

Puede notarse que la invención del documento D2 no tiene nada que ver con nuestra invención y solo permite rastrear e identificar vehículos que se aproximen a otro vehículo por medio de sistemas de cámaras basándose en una característica particular del vehículo. Se ubicó el documento D3 US9308914 de Sun Bo et. al. del 23 de enero del 2015, que revela un sistema avanzado de asistencia al conductor de un vehículo para realizar una o más funciones de asistencia al conductor incluye un sensor ambiental, un monitor de estado de manejo, un Interruptor de activación y un controlador de asistencia al conductor. El sensor ambiental detecta un entorno que rodea el veh ículo. El monitor de estado del controlador supervisa el controlador y determina una situación anormal del controlador. El interruptor de activación es operable por el controlador para activar o desactivar las funciones de asistencia al conductor. El controlador de asistencia al conductor realiza una o más funciones de asistencia al conductor para controlar el vehículo según un comportamiento objetivo, donde el comportamiento objetivo se define según el entorno que rodea al vehículo. El controlador de asistencia al conductor activa las funciones de asistencia al conductor cuando el monitor de estado del conductor detecta la situación anormal con el interruptor de activación configurado para desactivarse. It can be noted that the invention of document D2 has nothing to do with our invention and only allows tracking and identifying vehicles that are approaching another vehicle by means of camera systems based on a particular characteristic of the vehicle. Document D3 US9308914 by Sun Bo et. to the. of January 23, 2015, which reveals an advanced driver assistance system of a vehicle to perform one or more functions of Driver assistance includes an environmental sensor, a driving status monitor, an Activation Switch and a driver assistance controller. The environmental sensor detects an environment around the vehicle. The controller health monitor monitors the controller and determines a controller abnormal condition. The activation switch is operable by the controller to activate or deactivate the driver assistance functions. The driver assistance controller performs one or more driver assistance functions to control the vehicle according to a target behavior, where the target behavior is defined according to the environment surrounding the vehicle. The driver assistance controller activates the driver assistance functions when the driver status monitor detects the abnormal situation with the activation switch set to deactivate.

El documento D3 no revela ni sugiere la asociación de un par de cámaras, una interna y una externa, un dispositivo de procesamiento de información que comprende un procesador que analizará los datos de estas cámaras y además podrá comunicarse con los distintos ECUs en el sistema del automóvil para poder avisar al conductor o bien tomar una acción determinada para evitar un daño al usuario a través de una interfaz hombre-máquina (H M I) . The D3 document does not reveal or suggest the association of a pair of cameras, one internal and one external, an information processing device that comprises a processor that will analyze the data from these cameras and will also be able to communicate with the different ECUs in the system of the car to be able to warn the driver or take a specific action to avoid harm to the user through a human-machine interface (HMI).

Otros documentos que se citan como referencia son los documentos EP2656336B 1 ; JP3757684B2 y US20100222976A1. En ninguno de los sistemas existentes se considera el relacionar información del interior del vehículo con el exterior. Other documents that are cited for reference are documents EP2656336B 1; JP3757684B2 and US20100222976A1. In none of the existing systems is it considered to relate information from the interior of the vehicle to the exterior.

Los sistemas actuales detectan la presencia de un objeto, pero no lo identifican/caracterizan para poder reaccionar a el mismo. Current systems detect the presence of an object, but do not identify / characterize it in order to react to it.

Hacia el interior del vehículo, los sistemas actuales solo detectan presencia del conductor o si este está somnoliento. Inside the vehicle, current systems only detect the presence of the driver or if he is drowsy.

Ninguno de los sistemas propuestos considera la generación de primitivas de control para el vehículo que incremente la segu ridad durante la conducción . None of the proposed systems considers the generation of control primitives for the vehicle that increases safety while driving.

Ante la necesidad de contar con un sistema y método basado en la correlación de información visual Interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi- automática, fue que se desarrolló la presente invención Faced with the need for a system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, the present invention was developed

OBJETIVOS DE LA INVENCIÓN OBJECTIVES OF THE INVENTION

La invención tiene como objetivo principal hacer disponible un sistema y método basado en la correlación de información visual interior y exterior en un veh ículo para mejorar la seguridad en conducción manual o semi-automática que permita incrementar la seguridad del conductor, a través de un monitoreo visual del exterior (otros vehículos, señales, personas, construcciones, etc.) y el Interior (pérdida de atención del conductor, estado de emergencia en el conductor, número y posición de ocupantes, etc.) del vehículo y correlacionar estas dos fuentes de información para realizar una acción preventiva (evitar encendido o acelerado, alertar al pasajero con luz o ruido, prender las intermitentes, etc.) o correctiva (desacelerar, activar conducción automática, mantener distancias de seguridad, etc.) . The main objective of the invention is to make available a system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving that allows increasing driver safety, through monitoring visual of the exterior (other vehicles, signs, people, buildings, etc.) and the interior (loss of driver attention, state of driver, number and position of occupants, etc.) of the vehicle and correlate these two sources of information to carry out a preventive action (avoid ignition or acceleration, alert the passenger with light or noise, turn on the turn signals, etc.) or corrective (decelerate, activate automatic driving, maintain safety distances, etc.).

Otro objetivo de la invención es hacer disponible dicho sistema y método basado en la correlación de información visual interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi-automática, en donde el monitoreo visual propuesto incluye un amplio campo visual que normalmente no está al alcance del conductor. Otro objetivo de la invención es hacer disponible dicho sistema y método basado en la correlación de información visual Interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi-automática, en donde la salida del sistema propuesto, permita acciones preventivas y correctivas, se pueden utilizar para reducir accidentes de tránsito, hacer más eficiente en tiempo y energía el uso del mismo, y mejorar la experiencia de manejo de un vehículo . Another objective of the invention is to make available said system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, where the proposed visual monitoring includes a wide visual field that normally it is not within the driver's reach. Another objective of the invention is to make available said system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, where the output of the proposed system allows preventive and corrective actions , can be used to reduce traffic accidents, make the use of traffic more time and energy efficient, and improve the driving experience of a vehicle.

Otro objetivo de la invención es hacer disponible dicho sistema y método basado en la correlación de información visual Interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi-automática, que además confiera al vehículo capacidad de reacción automática ante situaciones de riesgo detectadas por el mismo. Another objective of the invention is to make available said system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, which also confers on the vehicle automatic reaction capacity in the event of risk situations detected by it.

Y todas aquellas cualidades y objetivos que se harán aparentes al realizar una descripción general y detallada de la presente invención apoyados en las modalidades ilustradas. And all those qualities and objectives that will become apparent when making a general and detailed description of the present invention supported by the illustrated modalities.

BREVE DESCRIPCIÓN DEL INVENTO BRIEF DESCRIPTION OF THE INVENTION

De manera general, el sistema y método basado en la correlación de información visual interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi-automática, permite la correlación de imágenes obtenidas del interior y del exterior de un automóvil y está compuesto por al menos una cámara externa dispuesta en una zona frontal del vehículo viendo hacia el campo visual del conductor en la dirección frontal del vehículo y al menos una cámara interna dispuesta en el interior de la cabina del veh ículo enfocada hacia la zona del conductor; un dispositivo principal de control y procesamiento de Información q ue comprende un procesador que analizará los datos (imágenes- video) de dichas cámaras y además comprende medios de comunicación para comunicarse con la pluralidad de unidades de control del motor (ECU's) del sistema electrónico de control del vehículo y configurado además para recibir información de la pluralidad de sensores del veh ículo, para poder avisar al conductor o bien tomar una acción determinada para evitar un daño al usuario a través de una interfaz hombre-máquina (H M I) con alarmas visuales o sonoras de aviso y advertencia. In general, the system and method based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, allows the correlation of images obtained from the interior and exterior of a car and is composed by at least one external camera arranged in a front area of the vehicle looking towards the driver's field of vision in the front direction of the vehicle and at least one internal camera arranged inside the vehicle cabin focused towards the driver's area; a main information processing and control device that comprises a processor that will analyze the data (images-video) from said cameras and also comprises communication means to communicate with the plurality of engine control units (ECU's) of the electronic system of control of the vehicle and also configured to receive information from the plurality of sensors of the vehicle, to be able to notify the driver or take a specific action to avoid a damage to the user via a human machine interface (HMI) with visual or audible warning and warning alarms.

El sistema permite a través de dicha al menos una cámara interior determinar hacia donde está mirando el conductor y analizará el campo visual máximo en base a la posición de los ojos y la cabeza del conductor; dicha al menos una segunda cámara exterior montada exteriormente preferentemente en el parabrisas del veh ículo u otra zona que permita visualizar el área frontal del vehículo estará registrando la imagen al frente del automóvil y los objetos dentro de campo visual. El sistema correlacionara los objetos en la imagen frontal con el campo visual detectado por la posición de los ojos y analizara los objetos para detectar peligros en base a una clasificación y un ajuste de sensibilidad de detección , así como la información de otros sensores, como por ejemplo la velocidad del automóvil, entre otros sensores. The system allows through said at least one interior camera to determine where the driver is looking and will analyze the maximum visual field based on the position of the driver's eyes and head; said at least one second exterior camera mounted externally preferably on the windshield of the vehicle or other area that allows viewing the front area of the vehicle will be recording the image in front of the automobile and the objects within visual field. The system will correlate the objects in the frontal image with the visual field detected by the position of the eyes and will analyze the objects to detect hazards based on a classification and a detection sensitivity adjustment, as well as information from other sensors, such as by example the speed of the car, among other sensors.

Cuando el sistema detecta un peatón en el frente y ve que no se encuentra dentro del campo visual del conductor, entonces se emitirá una alerta grave para el conductor, en cambio si se detecta un ave se deberá de ignorar. Además, proveerá la capacidad para la detección de señales de tráfico y ver si el conductor fue capaz de observarla, de otro modo se emitirá una alarma de severidad med ia. When the system detects a pedestrian in front and sees that it is not within the driver's field of vision, then a serious alert will be issued to the driver, instead if a bird is detected it should be ignored. In addition, it will provide the ability to detect traffic signs and see if the driver was able to observe them, otherwise an alarm of medium severity will be issued.

En base a esta información el sistema decidirá si es necesario tomar alguna primitiva de control (i .e. perfiles seguros de velocidad, soporte de conducción automática, activación de tareas en otras ECUs del veh ícu lo, etc.) o bien si solo es necesario el avisar al conductor mediante una alarma visual o sonora de la interfaz hombre-máquina (HM I). Based on this information the system will decide if it is necessary to take any control primitive (i .e. Secure profiles of speed, automatic driving support, task activation in other vehicle ECUs, etc.) or if it is only necessary to notify the driver by means of a visual or audible alarm from the human-machine interface (HM I).

La invención también implica un método para mejorar la seguridad en conducción manual o semi-automática por parte de un conductor, mediante el sistema la correlación de Información visual interior y exterior en un vehículo, como reclamado en la reivindicación 1 , que se caracteriza porque consiste en: The invention also involves a method to improve safety in manual or semi-automatic driving by a driver, by means of the system of correlation of interior and exterior visual information in a vehicle, as claimed in claim 1, characterized in that it consists in:

Determinar el estado de una interfaz hombre-máqu ina H MI, que está en comunicación con un dispositivo principal de control y procesamiento de información; Determine the state of a human-machine interface H MI, which is in communication with a main information processing and control device;

Ejecutar un algoritmo en el dispositivo principal de control y procesamiento de información, para Execute an algorithm in the main information processing and control device, to

Tomar imágenes (imágenes y/o video) del exterior e interior de un vehícu lo mediante al menos una cámara exterior d ispuesta en una zona frontal del vehículo viendo hacia el campo visual del conductor en la dirección del vehículo y al menos una cámara interior dispuesta en el interior de la cabina del vehículo enfocada hacia la zona del conductor; Take images (images and / or video) of the exterior and interior of a vehicle using at least one exterior camera arranged in a front area of the vehicle looking towards the driver's field of vision in the direction of the vehicle and at least one interior camera arranged inside the vehicle cabin focused towards the driver's area;

tomar la lectura de los sensores del vehículo para conocer el estado del vehículo, si está en movimiento o no y si el conductor está abordo o no; take the reading of the vehicle sensors to know the status of the vehicle, if it is moving or not and if the driver is on board or not;

procesar las imágenes del exterior y del interior para detectar objetos externos al veh ículo en las imágenes y el estado del rostro del conductor a través del dispositivo principal de control y procesamien to de información; Process the exterior and interior images to detect objects external to the vehicle in the images and the driver's face status through the main device of information control and processing;

Alertar a través del HMI que enviará directivas de control hacía los controladores del vehículo mediante el dispositivo principal de control y procesamiento de información que está en comunicación con la pluralidad de unidades de control del motor (ECU’s) y actuadores [por ejemplo controladores de velocidad, controladores de inyección, controladores de luces, controladores de frenos ABS] del sistema electrónico de control del vehículo para detener el vehículo, cuando no se detecta presencia del conductor, o Alert through the HMI that it will send control directives to the vehicle's controllers through the main control and information processing device that is in communication with the plurality of engine control units (ECU's) and actuators [for example, speed controllers, injection controllers, light controllers, ABS brake controllers] of the vehicle's electronic control system to stop the vehicle, when the driver's presence is not detected, or

correlacionar las imágenes externas de dicha al menos una cámara externa en base a las lecturas del rostro y ojos del conductor derivadas de las imágenes de dicha al menos una cámara interna y la determinación del rango visual del conductor; en base a ello determinar las imágenes fuera del rango visual del conductor, tomando en consideración la lectura del estado del vehículo; correlating the external images of said at least one external camera based on the readings of the driver's face and eyes derived from the images of said at least one internal camera and determining the visual range of the driver; Based on this, determine the images outside the visual range of the driver, taking into consideration the reading of the vehicle's condition;

categorizar los objetos dentro de la imagen fuera del rango visual del conductor para en base a ello catalogar avisos y acciones; categorize the objects within the image outside the visual range of the driver to catalog warnings and actions based on this;

desplegar avisos o advertencias en el HMI en la pantalla del tablero de instrumentos del vehículo o avisar al conductor mediante una alarma visual o sonora de la interfaz hombre- máquina (H M I) ; display warnings or warnings in the HMI on the vehicle's dashboard screen or warn the driver by means of a visual or audible alarm from the human-machine interface (H M I);

enviar directivas de control como acciones sobre los instrumentos de control del vehículo para evitar accidentes; además de desplegar avisos en el HMI. Para comprender mejor las características de la presente invención se acompaña a la presente descripción , como parte integrante de la misma, los dibujos con carácter ilustrativo más no limitativo, que se describen a continuación. send control directives as actions on vehicle control instruments to avoid accidents; in addition to displaying notifications in the HMI. In order to better understand the characteristics of the present invention, the present description is accompanied by, as an integral part thereof, the drawings with an illustrative but non-limiting character, which are described below.

BREVE DESCRIPCIÓN DE LAS FIGURAS BRIEF DESCRIPTION OF THE FIGURES

La figura 1 muestra un diagrama esquemático de bloques del sistema basado en la correlación de información visual interior y exterior en un veh ículo para mejorar la seguridad en conducción manual o semi-automática. Figure 1 shows a schematic block diagram of the system based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving.

La figura 2 ilustra un diagrama esquemático del interior de un vehículo mostrando la disposición de las cámaras interna y externa del sistema basado en la correlación de información visual interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi-automática. Figure 2 illustrates a schematic diagram of the interior of a vehicle showing the arrangement of the internal and external cameras of the system based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving.

La figura 3 muestra una perspectiva convencional de un veh ículo donde se ilustran esquemáticamente los componentes del sistema basado en la correlación de información visual interior y exterior en un vehículo para mejorar la segu ridad en conducción manual o semi-automática. La figura 4 muestra un diagrama de flujo de la operación del sistema basado en la correlación de información visual interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi-automática. Figure 3 shows a conventional perspective of a vehicle where the components of the system based on the correlation of interior and exterior visual information in a vehicle are schematically illustrated to improve safety in manual or semi-automatic driving. Figure 4 shows a flow diagram of the operation of the system based on the correlation of visual information inside and outside in a vehicle to improve driving safety manual or semi-automatic.

Para una mejor comprensión del invento, se pasará a hacer la descripción detallada de alguna de las modalidades del mismo, mostrada en los dibujas que con fines ilustrativos mas no limitativos se anexan a la presente descripción. For a better understanding of the invention, a detailed description of some of the modalities of the invention will be made, shown in the drawings that for illustrative but non-limiting purposes are attached to the present description.

DESCRIPCIÓN DETALLADA DEL INVENTO Los detalles característicos del sistema basado en la correlación de Información visual Interior y exterior en un vehícu lo para mejorar la seguridad en conducción manual o semi-automática, se muestran claramente en la siguiente descripción y en los d ibujos ilustrativos que se anexan, sirviendo los mismos signos de referencia para señalar las mismas partes. DETAILED DESCRIPTION OF THE INVENTION The characteristic details of the system based on the correlation of Interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, are clearly shown in the following description and in the illustrative drawings that are append, serving the same reference signs to indicate the same parts.

Haciendo referencia a las figuras 1 a 3, el sistema basado en la correlación de información visual interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi- automática, consta de al menos una cámara externa (1 ) dispuesta en una zona frontal del vehículo (2) viendo hacia el campo visual de la zona del conductor (3) en la dirección frontal del vehículo (2) y al menos una cámara interna (4) dispuesta en el interior de la cabina (5) del vehículo (2) enfocada hacia el campo visual de la zona del conductor (3); un dispositivo principal de control y procesamiento de información (6) que comprende un procesador que analizará los datos (imágenes-video) de dichas cámaras externa e interna (1 , 4) y además comprende medios de comunicación para comunicarse con la pluralidad de unidades de control del motor (ECU's) y actuadores (7) [por ejemplo controladores de velocidad , controladores de inyección, controladores de luces, controladores de frenos ABS] del sistema electrónico de control del vehículo y configurado además para recibir información de la pluralidad de sensores (8) [como por ejemplo sensor de velocidad RPM, sensor de frenos ABS, entre otros] del veh ículo (2), para poder avisar al conductor (3) o bien tomar una acción determinada para evitar un daño al usuario a través de una interfaz hombre-máquina (HMI) (9) con alarmas visuales o sonoras de aviso y advertencia. With reference to Figures 1 to 3, the system based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, consists of at least one external camera (1) arranged in a zone front of the vehicle (2) looking towards the visual field of the driver's area (3) in the front direction of the vehicle (2) and at least one internal camera (4) arranged inside the cabin (5) of the vehicle ( 2) focused towards the visual field of the driver's area (3); a main information processing and control device (6) comprising a processor that will analyze the data (images-video) from said cameras external and internal (1, 4) and also comprises communication means to communicate with the plurality of engine control units (ECU's) and actuators (7) [for example speed controllers, injection controllers, light controllers, ABS brakes] from the vehicle's electronic control system and also configured to receive information from the plurality of sensors (8) [such as RPM speed sensor, ABS brake sensor, among others] of the vehicle (2), in order to notify the driver (3) or take a specific action to avoid harm to the user through a human-machine interface (HMI) (9) with visual or audible warning and warning alarms.

El sistema permite a través de dicha al menos una cámara interior (4) determinar hacia donde está mirando el conductor (3) y analizará el campo visual máximo en base a la posición de los ojos y la cabeza del conductor (3); dicha al menos una segunda cámara externa (1 ) montada exteriormente preferentemente en el parabrisas del vehículo (2) u otra zona que permita visualizar el área frontal del vehículo (2) estará registrando la imagen al frente del automóvil y los objetos dentro de campo visual . El sistema correlacionará los objetos en la imagen frontal con el campo visual detectado por la posición de los ojos del conductor (3) y analizará los objetos para detectar peligros en base a una clasificación y un ajuste de sensibilidad de detección, así como la información de una pluralidad de sensores (8), como por ejemplo la velocidad del automóvil, sensores de frenos, entre otros sensores. The system allows through said at least one interior camera (4) to determine where the driver (3) is looking and will analyze the maximum visual field based on the position of the driver's eyes and head (3); said at least one second external camera (1) mounted externally preferably on the windshield of the vehicle (2) or another area that allows viewing the front area of the vehicle (2) will be recording the image in front of the car and the objects within visual field . The system will correlate the objects in the frontal image with the visual field detected by the position of the driver's eyes (3) and will analyze the objects for hazards based on a classification and a detection sensitivity setting, as well as information from a plurality of sensors (8), such as the speed of the car, brake sensors, among others sensors.

Cuando el sistema detecta un peatón en el frente y ve que no se encuentra dentro del campo visual del conductor, entonces se emitirá una alerta grave para el conductor (3), en cambio si se detecta un ave se deberá de ignorar. Además, proveerá la capacidad para la detección de señales de tráfico y ver si el conductor (3) fue capaz de observarla, de otro modo se emitirá una alarma de severidad media. En base a esta información el sistema decidirá si es necesario tomar alguna primitiva de control (i.e. perfiles seguros de velocidad, soporte de conducción automática, activación de tareas en otras ECUs del vehículo, etc.), o bien si solo es necesario el avisar al conductor (3) mediante una alarma visual o sonora de la interfaz hombre-máquina (H M I) (9). When the system detects a pedestrian in front and sees that it is not within the driver's field of vision, then a serious alert will be issued to the driver (3), on the other hand, if a bird is detected, it must be ignored. In addition, it will provide the ability to detect traffic signs and see if the driver (3) was able to observe it, otherwise an alarm of medium severity will be issued. Based on this information, the system will decide if it is necessary to take any control primitive (ie safe speed profiles, automatic driving support, activation of tasks in other vehicle ECUs, etc.), or if it is only necessary to notify the driver (3) via a visual or audible alarm from the human-machine interface (HMI) (9).

La integración de los componentes del sistema propuesto en el interior de la cabina (5) del vehículo (2) se presenta en la figura 3, en la misma podemos observar la localización de los componentes desde el punto de vista del usuario [conductor (3)]. The integration of the components of the proposed system inside the cabin (5) of the vehicle (2) is presented in Figure 3, in which we can see the location of the components from the point of view of the user [driver (3 )].

De acuerdo con la figura 4, el sistema lógico propuesto sigue las siguientes directivas para ejecutar el algoritmo: a. Lee el estado de la interfaz hombre-máquina HMI , esta lectu ra indica si la funcionalidad del sistema está activa (a 1 ), ya que se permite al usuario desactivarla desde la interfaz gráfica. Además, el sistema permite modificar la sensibilidad (a2) para la detección de objetos en las Imágenes. Al leer el estado del HMI se pueden conocer estos valores. According to figure 4, the proposed logic system follows the following directives to execute the algorithm: a. It reads the status of the HMI man-machine interface, this reading indicates if the system functionality is active (at 1), since the user is allowed to deactivate it from the graphical interface. In addition, the system allows modifying the sensitivity (a2) for the detection of objects in the Images. By reading the status of the HMI these values can be known.

b. La determinación de si la funcionalidad del sistema está activa o no, si la función está activa, el algoritmo ejecuta la lectura de la cámara externa (c1 ), y la lectura de la cámara interna (c2), para obtener una Imagen exterior (cí a) desde el exterior del vehículo (frente) (c1 b) y para obtener una imagen interior (c2a) desde el Interior del vehículo (c2b), para poder transmitir estos datos hacia el controlador principal. Esto se representa en el controlador principal como las imágenes interna y externa. b. Determining whether the system functionality is active or not, if the function is active, the algorithm executes the reading of the external camera (c1), and the reading of the internal camera (c2), to obtain an external image (c a) from outside the vehicle (front) (c1 b) and to obtain an interior image (c2a) from inside the vehicle (c2b), to be able to transmit this data to the main controller. This is represented on the main controller as the internal and external images.

d . Después el algoritmo leerá los sensores para conocer el estado del vehículo, es decir si el vehículo está en movimiento o si está encendido o no, mediante las señales de los sensores (d 1) para detectar la velocidad (d2) o las revoluciones por minuto (RPM) (d3) . d. Then the algorithm will read the sensors to know the state of the vehicle, that is, if the vehicle is moving or if it is on or not, using the signals from the sensors (d 1) to detect the speed (d2) or the revolutions per minute (RPM) (d3).

e. Ya que se cuenta con la captura de los datos, imágenes y lecturas de sensores, el controlador principal analizará la imagen interior en busca de un rostro dentro de la imagen interior (e1 ). Ya que la imagen interior (e1 ) será capturada con una cámara desde el tablero del automóvil, la localización de un rostro dentro de esta imagen permitirá detectar la presencia/ausencia de un conductor dentro del vehículo. and. Since the data, images and sensor readings are captured, the main controller will analyze the inner image looking for a face within the inner image (e1). Since the interior image (e1) will be captured with a camera from the car dashboard, the location of a face within this image will allow the presence / absence of a driver inside the vehicle to be detected.

f) . Si el algoritmo no detecta a un conductor, pero detecta que el veh ículo está en movimiento (g) a través de los sensores de velocidad (g 1 ) y RPM (g 2 ) , enviará una alerta a través del HM I ( g 3 ) , además enviará directivas de control (g4) hacia los controladores de inyección y de ABS para detener el vehículo, ya que si el automóvil está en movimiento, pero no se pudo localizar a un conductor, esto indica que el conductor está en una posición que no permite que esté observando el camino, lo cual puede provocar un accidente. f). If the algorithm does not detect a driver, but detects that the vehicle is moving (g) through the speed (g 1) and RPM (g 2) sensors, it will send an alert through the HM I (g 3), it will also send control directives (g4) to the injection and ABS controllers to stop the vehicle, because if the car is in motion, but a driver could not be located, this indicates that the driver is in a position that does not allow you to be watching the road, which can cause an accident.

h) . Si el algoritmo detecta un conductor, entonces se analizará de nuevo la imagen interior (h 1 ), ya que se conoce en que parte de la imagen se encuentra el rostro se analizará este rostro para detectar la posición de los ojos (h2), asi como la dirección hacia la cual el conductor está mirando (h3). h). If the algorithm detects a driver, then the interior image (h 1) will be analyzed again, since it is known in which part of the image the face is located, this face will be analyzed to detect the position of the eyes (h2), thus as the direction in which the driver is looking (h3).

i ) . Ya que se conoce la posición de los ojos y la dirección de éstos, el algoritmo calculará el rango visual (i 1 ) del conductor. Es decir, hasta donde es posible que el conductor pueda interpretar la imagen que tiene enfrente, mediante la posición de los ojos (i2) y la dirección de la mirada (i3) . i). Since the position of the eyes and the direction of the eyes are known, the algorithm will calculate the visual range (i 1) of the driver. That is, as far as it is possible for the driver to interpret the image in front of him, through the position of the eyes (i2) and the direction of the gaze (i3).

j) . Con el cálculo del rango visual (j 1 ) se analizará la imagen exterior (j2), el rango visual calculado se correlacionará en la imagen de la cámara externa. Esto permitirá definir una imagen q ue contiene la imagen dentro del rango visual del conductor (j 3) , asi como una imagen con la parte fuera del rango visual (j 4 ) . j). With the calculation of the visual range (j 1) the external image (j2) will be analyzed, the calculated visual range will be correlated in the image of the external camera. This will allow to define an image containing the image within the driver's visual range (j 3), as well as an image with the part outside the visual range (j 4).

k). Enseguida, el algoritmo analizará la imagen que se encuentra fuera del rango visual (k 1 ) en busca de objetos, en este análisis se detectarán todos los objetos dentro de la imagen fuera del rango visual (k2) . k). Next, the algorithm will analyze the image that is outside the visual range (k 1) in search of objects, in this analysis all objects within the image outside the visual range (k2) will be detected.

L ) . Si el algoritmo no detecta objetos en la imagen fuera de rango visual la ejecución del algoritmo finaliza. En caso de que se detecten uno o más objetos, el algoritmo revisará si el automóvil se encuentra en movimiento (L1 ) a través de los sensores de velocidad (L2) y RPM (L3), en caso de que esté detenido el algoritmo terminará su ejecución. L). If the algorithm does not detect objects in the image out of visual range, the execution of the algorithm ends. In case it is detect one or more objects, the algorithm will check if the car is moving (L1) through the speed (L2) and RPM (L3) sensors, in case the algorithm is stopped it will finish its execution.

m . En caso de que se detecten objetos y el vehículo se encuentre en movimiento el algoritmo ejecutará una categorlzación de los objetos dentro de la imagen fuera del rango visual del conductor (m 1 ). Esta categorización tomará en cuenta el valor para la sensibilidad (m2) en la detección de objetos definida por el usuario . El algoritmo separará objetos como peatones, ciclistas u otros automóviles de objetos como perros, aves, hidrantes, etc. m. In the event that objects are detected and the vehicle is in motion, the algorithm will perform a categorization of the objects within the image outside the visual range of the driver (m 1). This categorization will take into account the value for the sensitivity (m2) in the detection of objects defined by the user. The algorithm will separate objects like pedestrians, cyclists, or other cars from objects like dogs, birds, fire hydrants, etc.

En base a la categorización de los objetos, el algoritmo creará dos categorías (avisos m3 y acciones m4) : Based on the categorization of the objects, the algorithm will create two categories (m3 warnings and m4 actions):

Se desplegarán avisos o advertencias en el HMI (n) en la pantalla del tablero de instrumentos (avisos n 1 como por ejemplo detección de señales de tránsito, mascotas, etc.); Notices or warnings will be displayed on the HMI (n) on the dashboard screen (notices n 1 such as detection of traffic signs, pets, etc.);

Se enviarán directivas de control (fi) como acciones (ñ 1 ) para evitar accidentes además de desplegar avisos en el HMI (por ejemplo, detección de peatones, ciclistas , otros automóviles, etc.) . Control directives (fi) will be sent as actions (ñ 1) to prevent accidents in addition to displaying warnings in the HMI (eg detection of pedestrians, cyclists, other cars, etc.).

El invento ha sido descrito suficientemente como para que una persona con conocimientos medios en la materia pueda reproducir y obtener los resultados que mencionamos en la presente Invención . Sin embargo, cualquier persona hábil en el campo de la técnica que compete el presente invento puede ser capaz de hacer modificaciones no descritas en la presente solicitud, sin embargo, si para la aplicación de estas modificaciones en una estructura determinada o en el proceso de manufactura del mismo, se requiere de la materia reclamada en las siguientes reivindicaciones, dichas estructuras deberán ser comprendidas dentro del alcance de la invención. The invention has been sufficiently described so that a person of ordinary skill in the art can reproduce and obtain the results mentioned in the present invention. However, any person skilled in the field of art that is concerned with the present invention may be able to make modifications not described in the present application, however, if for the application of these modifications in a determined structure or in the manufacturing process of the same, the matter claimed in the following claims is required, said structures must be included within the scope of the invention.

Claims

REIVINDICACIONES Habiendo descrito suficientemente la invención, se reclama como propiedad lo contenido en las siguientes cláusulas reivindicatorías. Having sufficiently described the invention, the content of the following claims clauses is claimed as property. 1 U n sistema basado en la correlación de información visual interior y exterior en un vehículo para mejorar la seguridad en conducción manual o semi-automática, caracterizado por comprender al menos una cámara externa dispuesta en una zona frontal del veh ículo viendo hacia el campo visual del conductor en la dirección frontal del vehículo y al menos una cámara interna dispuesta en el interior de la cabina del vehículo enfocada hacia la zona del conductor; un dispositivo principal de control y procesamiento de información que comprende un procesador que analizará los datos (imágenes-video) de dichas cámaras externa e interna y además comprende medios de comunicación para comunicarse con la pluralidad de unidades de control del motor (ECU’s) y actuadores [por ejemplo controladores de velocidad, controladores de inyección, controladores de luces, controladores de frenos ABS] del sistema electrónico de control del veh ículo y configurado además para recibir información de la pluralidad de sensores [como por ejemplo sensor de velocidad RPM , sensor de frenos ABS, entre otros] del vehículo; dicho dispositivo principal de control y procesamiento de información estando en comunicación con una interfaz hombre-máquina (HMI) con alarmas visuales o sonoras de aviso y advertencia, configurado para avisar al conductor o bien tomar una acción determinada para evitar un dafio al usuario. 1 A system based on the correlation of interior and exterior visual information in a vehicle to improve safety in manual or semi-automatic driving, characterized by comprising at least one external camera arranged in a front area of the vehicle looking towards the visual field of the driver in the front direction of the vehicle and at least one internal camera arranged inside the cabin of the vehicle focused towards the driver's area; a main information processing and control device comprising a processor that will analyze the data (video-images) from said external and internal cameras and also comprises communication means to communicate with the plurality of engine control units (ECU's) and actuators [for example speed controllers, injection controllers, light controllers, ABS brake controllers] of the electronic control system of the vehicle and further configured to receive information from the plurality of sensors [such as RPM speed sensor, speed sensor ABS brakes, among others] of the vehicle; said main control and information processing device being in communication with a human-machine interface (HMI) with visual or audible warning and warning alarms, configured to notify the driver or take a specific action to prevent injury to the user. 2.- Un método para mejorar la seguridad en conducción manual o semi-automática por parte de un conductor, mediante el sistema la correlación de información visual interior y exterior en un veh ículo, como reclamado en la reivindicación 1 , caracterizado porque consiste en: 2. A method to improve safety in manual or semi-automatic driving by a driver, through the system of correlation of visual information inside and outside in a vehicle, as claimed in claim 1, characterized in that it consists of: Determinar el estado de una interfaz hombre-máquina HMI, que está en comunicación con un dispositivo principal de control y procesamiento de información; Determine the state of a human-machine interface HMI, which is in communication with a main control and information processing device; Ejecutar un algoritmo en el dispositivo principal de control y procesamiento de información, para Execute an algorithm in the main information processing and control device, to Tomar imágenes (imágenes y/o video) del exterior e interior de un vehículo mediante al menos una cámara exterior dispuesta en una zona frontal del vehículo viendo hacia el campo visual del conductor en la dirección frontal del vehículo y al menos una cámara interior dispuesta en el interior de la cabina del vehículo enfocada hacia la zona del conductor; Take images (images and / or video) of the exterior and interior of a vehicle using at least one exterior camera arranged in a front area of the vehicle looking towards the driver's visual field in the front direction of the vehicle and at least one interior camera arranged in the interior of the vehicle cabin focused towards the driver's area; Tomar la lectura de los sensores del vehículo para conocer el estado del veh ículo, si está en movimiento o no y si el conductor está abordo o no; Take the reading of the vehicle sensors to know the state of the vehicle, if it is moving or not and if the driver is on board or not; Procesar las imágenes del exterior y del interior para detectar objetos externos al veh ículo en las imágenes y el estado del rostro del conductor a través del dispositivo principal de control y procesamiento de información: Process the exterior and interior images to detect objects external to the vehicle in the images and the state of the driver's face through the main information processing and control device: Alertar a través del HMI que enviará directivas de control hacía los controladores del vehículo mediante el dispositivo principal de control y procesamiento de información que está en comunicación con la pluralidad de unidades de control del motor (ECU’s) y actuadores [por ejemplo controladores de velocidad, controladores de inyección, controladores de luces, controladores de frenos ABS] del sistema electrónico de control del vehículo para detener el vehículo, cuando no se detecta presencia del conductor, o Alert through the HMI that will send control directives made the vehicle controllers through the main information processing and control device that is in communication with the plurality of engine control units (ECU's) and actuators [for example speed controllers, injection controllers, light controllers, ABS brakes] of the vehicle's electronic control system to stop the vehicle, when no driver presence is detected, or Correlacionar las Imágenes externas de dicha al menos una cámara externa en base a las lecturas del rostro y ojos del conductor derivadas de las imágenes de dicha al menos una cámara interna y la determinación del rango visual del conductor; en base a ello determinar las imágenes fuera del rango visual del conductor, tomando en consideración la lectura del estado del vehículo; Correlating the external Images of said at least one external camera based on the readings of the driver's face and eyes derived from the images of said at least one internal camera and the determination of the visual range of the driver; Based on this, determine the images outside the visual range of the driver, taking into consideration the reading of the vehicle's condition; Categorizar los objetos dentro de la imagen fuera del rango visual del conductor para en base a ello catalogar avisos y acciones; Categorize the objects within the image outside the visual range of the driver in order to catalog warnings and actions based on this; Desplegar avisos o advertencias en el HMI en la pantalla del tablero de instrumentos del vehículo o avisar al conductor mediante una alarma visual o sonora de la Interfaz hombre- máquina (HM I); Display notices or warnings in the HMI on the vehicle's dashboard screen or warn the driver by means of a visual or audible alarm from the Human Machine Interface (HM I); Enviar directivas de control como acciones sobre los Send control directives as actions on the Instrumentos de control del vehículo para evitar accidentes; además de desplegar avisos en el H MI . Vehicle control instruments to prevent accidents; in addition to displaying notices in the H MI.
PCT/MX2019/000016 2019-02-11 2019-02-15 System and method based on the correlation of interior and exterior visual information in a vehicle for improving safety in manual or semi-automatic driving Ceased WO2020167110A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
MXMX/A/2019/001712 2019-02-11
MX2019001712A MX2019001712A (en) 2019-02-11 2019-02-11 System and method based on the correlation of interior and exterior visual information in a vehicle for improving safety in manual or semi-automatic driving.

Publications (1)

Publication Number Publication Date
WO2020167110A1 true WO2020167110A1 (en) 2020-08-20

Family

ID=72045415

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/MX2019/000016 Ceased WO2020167110A1 (en) 2019-02-11 2019-02-15 System and method based on the correlation of interior and exterior visual information in a vehicle for improving safety in manual or semi-automatic driving

Country Status (2)

Country Link
MX (1) MX2019001712A (en)
WO (1) WO2020167110A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090051516A1 (en) * 2006-02-23 2009-02-26 Continental Automotive Gmbh Assistance System for Assisting a Driver
US20180065452A1 (en) * 2016-09-07 2018-03-08 Panasonic Automotive Systems Company Of America, Division Of Panasonic Corporation Of North America Using vision devices and techniques to control sunlight blocking devices for vehicles
EP3372465A1 (en) * 2017-03-10 2018-09-12 The Hi-Tech Robotic Systemz Ltd Method and system for vehicle status based advanced driver assistance
US20180304814A1 (en) * 2017-04-24 2018-10-25 Panasonic Automotive Systems Company Of America, Division Of Panasonic Corporation Of North America Rear view mirror-like perspective change system and method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090051516A1 (en) * 2006-02-23 2009-02-26 Continental Automotive Gmbh Assistance System for Assisting a Driver
US20180065452A1 (en) * 2016-09-07 2018-03-08 Panasonic Automotive Systems Company Of America, Division Of Panasonic Corporation Of North America Using vision devices and techniques to control sunlight blocking devices for vehicles
EP3372465A1 (en) * 2017-03-10 2018-09-12 The Hi-Tech Robotic Systemz Ltd Method and system for vehicle status based advanced driver assistance
US20180304814A1 (en) * 2017-04-24 2018-10-25 Panasonic Automotive Systems Company Of America, Division Of Panasonic Corporation Of North America Rear view mirror-like perspective change system and method

Also Published As

Publication number Publication date
MX2019001712A (en) 2020-08-12

Similar Documents

Publication Publication Date Title
US9352683B2 (en) Traffic density sensitivity selector
US10166918B2 (en) Drive assisting apparatus
CN102656613B (en) A predictive human-machine interface using eye gaze technology, blind spot indicators and driver experience
US6496117B2 (en) System for monitoring a driver's attention to driving
JP7792717B2 (en) Systems and programs
KR102051142B1 (en) System for managing dangerous driving index for vehicle and method therof
CN108068698A (en) vehicle systems and vehicles
JP7345114B2 (en) Vehicle display control device, display control method, and program
CN104175954A (en) Vehicle blind area monitoring and alarming system
JP4415856B2 (en) Method for detecting the forward perimeter of a road vehicle by a perimeter sensing system
CN111038499B (en) A blind spot monitoring system and method capable of autonomously controlling a vehicle according to the driver's state
JP7760190B2 (en) Systems and programs
CN102712317A (en) Combining driver and environment sensing for vehicular safety systems
US20180339713A1 (en) Method and device for supporting a vehicle occupant in a vehicle
CN204547919U (en) Vehicles with Safe Lane Change Assist
CN108297879A (en) A kind of Car warning system
CN114572232B (en) Attention assistance for dynamic blind spots that follow the driver's state while driving
CN207773004U (en) Turn to early warning system and vehicle
JP4751894B2 (en) A system to detect obstacles in front of a car
WO2020167110A1 (en) System and method based on the correlation of interior and exterior visual information in a vehicle for improving safety in manual or semi-automatic driving
WO2014090957A1 (en) Method for switching a camera system to a supporting mode, camera system and motor vehicle
KR20180039838A (en) Alarm controlling device of vehicle and method thereof
JP2009113770A (en) VEHICLE DISPLAY METHOD AND VEHICLE DISPLAY DEVICE
CN112550282A (en) Warning method and warning system for motor vehicle
JP7194890B2 (en) car mirror device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19914857

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19914857

Country of ref document: EP

Kind code of ref document: A1