WO2013004864A1 - Viewing system for controlling occupancy of vehicles at a toll station - Google Patents

Abstract

The invention is used to automatically control occupancy of vehicles at a toll station. Said automatic system comprises means for acquiring images of a vehicle to be controlled, means for processing the acquired images, and means for detecting faces inside the vehicle and counting said faces. A discount is then applied to the toll according to the number of validated occupants.

Description

 VISION SYSTEM FOR VEHICLE OCCUPATION CONTROL IN A

 TOLL STATION

Technical field

 The present invention relates to a system and method, based on vision, which allow automatic control of vehicle occupancy at a toll station.

Background

 In recent years there has been an increase in awareness of energy savings, which has led to the promotion of the use of public transport, and in the field of private transport, to encourage the shared use of cars.

 However, on toll roads, it has been found that 85% of vehicles are occupied by a single person at peak times, which means inefficient use of transport. In order to improve this fact, several policies for the promotion of car sharing have already been implemented, for example by applying a differentiation of tariffs, to discount the toll on vehicles with 3 or more people, considered as high occupancy vehicles, VAO. The application of a toll discount motivates users of public roads to share their vehicles.

 This test can be carried out on the manual tracks by the same operator, but on automatic roads, automatic verification systems must be used to distinguish the occupants of the vehicles in all lighting conditions. The systems that have been proposed so far operate in two very different ways: some are based on detection systems with sensors located around the vehicle, that is, external systems, and others with on-board systems, that is internal systems, which transmit the collected data. inside the vehicle as it passes through the tollbooth.

 The systems installed inside the vehicle of the prior art allow to detect the number of occupants directly by internal sensors, and transmit this information, either by microwave or radio frequency, as it passes through each toll station. In some systems the ultrasonic frequencies emitted from the heartbeat of the occupants are differentiated. Other systems have proposed a camera that allows the detection and differentiation of eyes, or a fingerprint reader. The disadvantage of these systems is that they require an installed internal equipment, which regardless of the cost that it may represent, requires an adaptation of each user's vehicle, which implies an impediment that has not allowed to be applied. In addition, systems based on internal detection equipment have not achieved sufficiently good levels of reliability.

Therefore, external systems are preferred to determine the number of occupants in a vehicle. Currently as an external system, computer vision is applied to the external detection of the human figure, but the reliability necessary to be able to be used in the operation of toll roads. In addition, these state-of-the-art systems present the difficulties derived from the effects of the reflections produced by the glasses, the lack of light inside the vehicles, the concealments that are produced by the arrangement of the seats and headrests, or from the point of view of the camera used and also, the restrictions of the use of a sunshield on the windshield, or tinted glass on the back.

 For example, there are external systems to control the correct use of lanes for VAOs. There is a camera that is positioned on the side of the lane for VAOs to control vehicle occupancy. However, its position on one side only prevents the camera from capturing all occupants inside the vehicle since there are always concealments by seats or passengers with a single camera.

 For example, there are external systems comprising a camera located in a position above a lane of the road to control occupancy by capturing a frontal image of the vehicle. However, the camera must be mounted in a position high enough for them to pass all type of vehicles, also tall vehicles, such as trucks or buses. This causes the camera to have a very inclined viewing angle that does not reveal the entire interior of the vehicle.

 Other systems use infrared sensors to detect sources of infrared radiation inside the vehicle. However, the glasses of the vehicles nowadays usually contain elements that reflect the infrared to avoid the heating of the vehicle. This means that infrared sensors represent a barrier to infrared that does not allow the safe and reliable determination of the number of occupants.

 Therefore, it would be desirable to provide a system and procedure for the control of vehicle occupancy at toll stations that allows detecting the number of occupants with high reliability and in real time. This system and procedure is aimed at eliminating both the effects of reflections on the windshields that complicate the detection of occupants inside the vehicle and the concealments that can result in a fictional number of occupants less than the real number. Summary of the Invention

 The present invention provides embodiments in which the problems of the prior art are solved. An object of the present invention is an automatic system for controlling vehicle occupancy at a toll station. Said automatic system comprises means for the acquisition of images of a vehicle to be controlled, means for the treatment of acquired images and the detection of faces within the vehicle and their counting. The means for acquiring images comprise at least two cameras that form a matrix of cameras and are in corresponding positions on both sides of a lane in which a vehicle to be controlled circulates.

Another object of the present invention is to provide a method that allows to safely and reliably detect the number of occupants in a vehicle and apply, if the Occupancy meets the minimum requirements, a discount to the toll rate. Said procedure comprises the steps of acquiring at least two images of a vehicle with at least two cameras, eliminating the effects of the reflections on the windshields and determining the number of occupants in the vehicle.

 Another object of the present invention is to provide a computer program containing instructions for carrying out the steps of the described procedures, and reproducing the effects of the various media that are part of the described system.

 Another object of the present invention is to provide a computer-readable medium containing instructions for carrying out the steps of the described procedures, and reproducing the effects of the various media that are part of the described system.

 With the system, procedure, and device of the present invention a very high level of reliability is obtained based on the analysis of at least two images obtained by at least two cameras installed in a toll station. The safe detection of all the occupants of the vehicle is achieved, bypassing the concealments and the effects of the reflections on the windshields, operating in all lighting conditions and in all types of vehicles.

Drawings

 FIG. 1 illustrates a system for the control of occupancy of vehicles of the prior art. FIGs. 2A-2B illustrate a vehicle occupancy control system according to different configurations comprising at least two cameras.

 FIG. 3 illustrates a control interface for an example vehicle occupancy control system.

 FIGs. 4A-4D illustrate flow charts describing individual steps of the process according to the invention.

Detailed description of the invention

 The system and procedure according to the present invention make it possible to determine, automatically, if a vehicle carries a minimum number of passengers, in order to apply discounts to vehicles considered high occupancy (VAO), in order to promote the collective use of private transport.

 To be able to make discounts to vehicles that have a minimum number of passengers, considered high occupancy, on the automatic roads of a toll station, an automatic system for verifying the number of occupants of the vehicle has been developed. Among the problems of the state of the art that has been solved, are: a) The variable lighting conditions that occur inside the vehicles according to the time of day and night.

b) The effects of light reflections or the presence of images reflected on the windshield, or glass, of cars, which distort the acquired images, or introduce parasitic elements that make it difficult to detect the faces of the occupants. The difficulties of recognizing face patterns derived from the very varied types of faces (hair, mustaches or beards, glasses, caps, etc.).

 The variation of sizes projected on the image of the faces of the occupants, depending on their position in the vehicle in the front or rear seats.

 Possibilities of concealment in the visualization of the occupants in function of their relative position, with respect to the point of view of the camera.

 Restrictions on the size margins, facial centering with the optical axis of the camera, or partial concealments. Therefore, these systems are not considered suitable for use at a toll station. For example, the implementation for a facial centering is very difficult and expensive and favorable uniform lighting cannot be guaranteed throughout the day.

 The present invention provides a system and method for obviating the problems mentioned above. It is possible to detect faces with high reliability and to safely determine the number of passengers inside the vehicle. The system and procedure can also be used in unfavorable lighting conditions in which the state of the art vision systems do not function properly.

 Here, the term "reflections" represents the images reflected on the windshield, or glass, of cars, or the reflections of the light sources on the windshield, or glass, that may exist and that can be recorded by a camera.

 Here, the term "second row" represents the rear of a vehicle, that is, the rear seats. Correspondingly, the term "first row" represents the front of the vehicle where the pilot and co-pilot are located.

 Here, instead of "combining" an image, the terms "complement", "add" or "compose a valid image" can also be used.

 The present invention has embodiments in hardware, in software, in firmware, or in a combination thereof. Any stage of the procedure is carried out by a hardware device, or by software that is executed. In the same way, the described system contains devices that interact and produce data that is managed through stages of the procedure.

 Therefore, the invention is composed of both devices and devices in a system, as well as computer programs and software that, when executed, perform the steps of the described method. The system can be configured with these programs, or software, using computer-readable media, which contain the computer-executable instructions.

Therefore any mention of a technical characteristic has its equivalent in hardware or software, and can be referred to as a particular device (a camera), or a programmable device (means of image acquisition), or a procedural step (to acquire images). In the following it is understood that this nomenclature is interchangeable, and any characteristic described in terms of procedure, device, system, computer program, or computer-readable medium can be claimed. The automatic system of the present invention comprises means for the acquisition of at least two images of the vehicle to be controlled using at least two cameras, one image per camera, and means for the treatment of the acquired images. Additionally, the system may comprise means to manage and store the data obtained. It can also comprise a lighting system, an automatic exposure time control and at least one control interface.

 FIG. 1 illustrates a prior art system. This system controls the occupation of a vehicle 101 and uses a single camera 102. Said camera is located in a lateral position of the track, and therefore does not allow to see the entire interior of a vehicle. This is due to concealments (101a, 101b, 101c, 101d, 101e) caused by seats, passengers, particularly on the side of the camera, stained side glass or other possible opaque elements in said side glasses that make it difficult to view vehicle interior It is also difficult to detect small people, especially children or babies. In particular, the detection of people in the second row, that is to say in the back of the vehicle, is negatively affected.

 Even if the camera can be rotated automatically and thus change its angle of view with respect to the vehicle during occupancy control, there would be at least a blind spot that partially covers a rear part due to a seat and / or an occupant in the front row. Therefore, it is very likely that there are people in the second row that this single camera cannot see. Thus, a safe and reliable detection of the occupation of a vehicle cannot be guaranteed.

 However, without a secure detection, a system for occupancy control cannot be implemented. It is very important that the result of the control is safe and reliable if discounts are then applied to vehicles with high occupancy.

 The same goes for a camera that is placed in a frontal position with respect to the vehicle. Said front position requires that all vehicles be possible. Since vehicles vary greatly in height, a sufficiently high position for the camera must be chosen. However, such height can make vision difficult due to the unfavorable angle to the vehicle that allows only the safe detection of the pilot and co-pilot, but not of the people in the back.

 As demonstrated in the previous part, a single camera is not enough to safely determine the number of occupants in a vehicle. Therefore, a system is needed to clearly see the entire interior of the vehicle, especially if there are or are not occupants in the second row, that is to say on the back of the vehicle.

The present invention solves these problems by a system that uses at least two paired cameras. With at least two paired cameras there are at least two different views. With at least two different points of view it is possible to fully examine the interior of a vehicle and thus determine the total number of passengers traveling in that vehicle. Preferably, the at least two different points of view are chosen such that there is at least one point of view on each side of the track, that is in lateral positions, using at least one image per camera. The view from a position side usually has only one major obstacle that is a front seat and the person, if any, in said seat on the side of the camera while a front view normally has two obstacles that are the two front seats.

 FIG. 2A illustrates an exemplary configuration in which two paired cameras are used to control the occupation of a vehicle 101. A first camera 102 is located on a first side of the road in a lateral position. The second chamber 201 is placed in a corresponding position on the other side of the track. Preferably, the two cameras 101, 201 are located at the height of the waiting position of the vehicle 101.

 In this way an array of paired cameras is installed in the toll station. In the case of two cameras, a 1 + 1 matrix is formed, while four cameras form a 2 + 2 matrix and eight cameras form a 4 + 4 matrix. In any case there is always a pair of cameras in corresponding positions of the two sides of the road which ensures that the vision of at least two points of view can be carried out.

 Each pair of cameras is not only in corresponding positions but also has the same height. In this way, the system can acquire images of a vehicle to be controlled from two different sides that are complementary to each other and allows the second row of the vehicle to be seen without significant concealment. There are as many views as cameras that are going to be used. Thus, with two cameras there are two points of view, with four cameras there are four points of view and with eight cameras there are eight points of view.

 In the configuration of FIG. 2A exactly two cameras are used. The advantage is its low cost of implementation and the amount of recorded data to be processed, managed and stored is also lower. Two chambers, one on each side of the road, in corresponding lateral positions with respect to the vehicle, can also be easily installed in existing toll stations as they can be easily mounted. Also, the time required to process the acquired images and have the result of the occupancy number is short due to the low amount of data and the occupants of the vehicle do not have to wait long at the toll station.

 Positions in the vicinity of the toll station should be chosen in such a way that they have a complete view through the windshield of the vehicle. Preferably, it is based on the fact that said complete vision must exist at least to acquire at least two images of a vehicle 101 in a passing position at the toll station. The time that the vehicle is passing or is stopped can be used to carry out the acquisition of at least two images, treat said at least two images to eliminate the effects of the reflections on the windshields, detect the faces of the occupants and count them and in the end apply or not the discount depending on the occupation detected. Preferably, multiple images are acquired to obtain more information and compensate for occupant movements that could result in partial or complete concealments of an occupant.

However, it is also possible to ensure that the cameras acquire the images of the vehicle in approximation. In this way, images can be acquired even before the vehicle stops and image processing can begin sooner. This may result in that the system carry out its control procedure and tolerance to concealments more efficient and faster since the effects of the reflections on the windshield of an approaching vehicle change constantly due to the movement of the vehicle. Thus, with multiple images acquired in approximation, a sequence of images can be formed that makes it possible to eliminate these effects from reflections and more reliably detect the occupation of the vehicle.

 From FIG. 2A it can be seen that each camera has a point of view that differs from the points of view of the other cameras. Due to their corresponding lateral positions on the road, the cameras can examine the interior of the vehicle completely and acquire images so that the number of occupants is to be determined through the windshield. The two cameras in FIG. 2A are suitable for one type of vehicle. In case of very different types of vehicle, especially different in height, it may be necessary to complement the two cameras of FIG. 2A with cameras at another height level to get a complete view inside each type of vehicle.

 FIG. 2B illustrates an embodiment of a four-chamber system, ie a 2 + 2 matrix. This 2 + 2 matrix is formed from the 1 + 1 matrix shown in FIG. 2A. For better illustration purposes, the four cameras appear with a horizontal offset. The 2 + 2 matrix can have two cameras 102, 201 at a first height level and the other two cameras 202, 203 at a second height level but in the same lateral position as a respective camera at a first height level. A system with two levels of height has the advantage that it works with both sports or small vehicles that are usually rather low as with large or off-road vehicles that usually have a higher chassis.

 FIG. 2B can also represent a configuration of an eight camera system forming a 4 + 4 matrix in which only four cameras in a first height level are listed while the corresponding four cameras in a second height level are not illustrated.

 The at least two cameras used in any configuration described above are digital cameras with a high dynamic range and which are capable of capturing many images per second, for example up to 25 images per second. The high dynamic range can be used to adapt the cameras to environmental and meteorological conditions such as a radiant day, night or rain with a luminosity that can vary between very low and very high. Depending on the brightness you have to change the contrast or other functions of the cameras to better acquire high quality images of the interior of the vehicle to be controlled. For this, the cameras contain an automatic and programmable exposure time control system that serves to better compensate for the great variation in brightness that occurs between daytime images in full sun and at night.

In addition, since the dynamic range is not high enough, two effects can occur before the image processing that could render the captured images unusable. In the case of a high brightness that exceeds the dynamic range of the camera, image saturation can occur resulting in white areas and in the worst case in an image completely white In the opposite case of a low luminosity there may not be enough light to pass a minimum level discounting the reading noise resulting in black areas or in the worst case in a completely black image. That is why the system adjusts the dynamic range of the camera over the area inside the vehicle, previously locating the windshield situation.

 In some cases it may be necessary to have additional light for good vision. Therefore the configuration of the cameras, whether two, four or eight, to be located in each toll station, is complemented by a specific lighting system, which ensures a correct vision both during the day and at night if necessary. To achieve optimum lighting without disturbing drivers at night or during the day when there is little light, you can use a large luminous panel located on each side of the vehicle, which produces a diffused light that also conveniently illuminates the area of passage of the vehicle, or infrared spotlights invisible to users can be used. Such infrared spotlights represent a source of lighting that can improve visibility for digital cameras.

 The image acquisition means includes a system for transmitting the at least two images to the image processing medium and / or to a data management and / or storage medium and / or at least one control interface. The images obtained are acquired through a single high-speed field bus to facilitate the wiring of each station. However, fiber optic cables that allow high data rate can also be used. High transmission speed is very important to ensure real-time operation.

 The system can also comprise image processing means, data management means, data storage means and at least one control interface. The image processing means processes the images acquired by the at least two cameras. In some configurations the images are acquired by two, four or eight cameras. The procedure that performs the image processing is described below. Said treatment serves to safely and reliably detect the number of occupants in a vehicle.

 The result of this treatment is a valid image that is then managed together with other data by the management means for the purpose of applying or not discounts to high occupancy vehicles. Said valid image and all acquired images plus information on the date, time, place and others can be stored on storage media. The valid image can also be displayed on a control interface to monitor system operation locally, to carry out arbitrary checks or subsequent quality controls, for example, in case there are claims for discounts not applied.

FIG. 3 illustrates the screen of such an interface by way of example. The interface indicates on a screen the selected images of each side of the road with the visualization of the locations made, the valid images archived with the best detection made, for subsequent queries or complaints, and the best images obtained from the occupants of the vehicle, with its location in the first or second row. This interface can send the occupancy data of each vehicle and image file to a central control unit of the toll station, to proceed with the corresponding billing. Remote access to the control interface is also possible.

 The image processing means, the data management means, the data storage means and the control interface can be in the same toll station, in a central building of the toll station or also in a remote location of the toll station, for example the headquarters of the company that manages the toll station or its customer service department. Preferably, the image processing means and at least one control interface are at the toll station. It is also possible that there is duplication of the image processing means, the data management means, the data storage means and the at least one control interface and are located both at the toll station and at a remote location.

 The system, in all its configurations described above, operates in real time, since the total processing time is less than the average passage time between vehicle and vehicle, even at peak times, and allows validating the discount to be applied to each vehicle if It meets the minimum occupancy required, without data loss.

 The system according to any configuration described above can be used in an automatic toll station, that is, it is not necessary to have an operator manually controlling the vehicles.

 The system, in all its configurations described above, also comprises means for carrying out a procedure for the control of vehicle occupancy. Said procedure will be described below.

 FIG. 4a generally illustrates a vehicle occupation control system 400 that implements a suitable procedure for said control. System 400 comprises at least two cameras 102, 201 as image acquisition means. In some configurations these are two cameras, in others four cameras and in others more than eight cameras. The illustrated cameras 202, 203 of lines in lines can represent said four or eight cameras of the exemplary embodiments. The system 400 further comprises means for carrying out the occupancy control procedure and a control unit that may include a control interface described above.

The procedure according to FIG. 4A carries out the steps of acquiring at least two images 401, eliminating the effects of the reflections 402 on the windshields and determining the occupation 403 of the vehicle to be controlled. The procedure also comprises a control step 404 that can be carried out during the procedure to monitor its proper functioning or at the end of the procedure, for example to review subsequent claims. The acquisition 401 of at least two images and the subsequent elimination of the effects of the reflections 402 on the windshield allows the system a first view of the interior of the vehicle thus detecting in a safe and reliable manner the occupation of the vehicle to be controlled. These at least two images can already allow a fairly clear view of the interior of the vehicle and in Particular of the second row. This is because the at least two images have been captured from two different positions but complementary to each other as described above. In this way, an image that can contain concealments with a second image that does not have such concealments can be complemented by being captured from another angle. Thus, the safety and reliability of the system and procedure is increased with respect to a system and procedure that only a single camera uses.

 Image acquisition 401 is carried out when the vehicle is located in an area of the toll station that is located in front of the at least two cameras 102, 201.

 To increase the security and reliability of the system even further, a larger number of different images of the scene can be acquired, that is, the at least two cameras 102, 201 capture multiple images of the vehicle when it is located in an area of the station. tolls that are located in front of the at least two cameras 102, 201, or they can also start capturing images when the vehicle to be controlled is still in proximity to said area. As mentioned above, even with two cameras there may be concealments, for example by movements of people inside the vehicle or when the driver stops the vehicle in a waiting position in which a camera has an insufficient angle to control the vehicle interior Normally, these concealments are punctual, that is, they exist at one time and in the next they are gone. Therefore, the capture and processing of multiple images can solve these problems. In one configuration the multiple images captured represent more than 50 images. The acquisition of multiple and / or more than 50 images facilitates better detection and increases the safety and reliability of the system and the image processing procedure.

 In the embodiment of acquiring the images of a vehicle in approximation, the more than 50 images compose a sequence of images comparable to a sequence of a film. Said sequence improves the subsequent elimination of the effects of the windshield reflections 402 as described below.

 However, there is the problem that the brightness changes from day to night and even during the day and depending on the weather conditions. This can worsen the quality of the acquired images and hinder the following steps of the procedure such as the elimination of the effects of the glare 402 on the windshield and / or the determination of the occupation 403.

 Therefore, the system according to FIG. 4A can be combined with an automatic and programmable exposure time control described above. Said control comprises a cyclic and continuous process that, after acquiring an image, analyzes it to determine the levels of illumination present at this time and adjust the brightness. This is described in FIG. 4B.

The procedure according to FIG. 4B comprises a continuous acquisition 401 to capture the at least two images 405. From the at least two images 405, the respective histograms are obtained which allow the current lighting levels to be analyzed 406 and thus adjust the brightness according to the variable lighting conditions of the outside. Since the images are acquired continuously also the brightness adjustment can be carry out continuously. The brightness adjustment is produced through a histogram analysis that allows, on the one hand, to dynamically and continuously recalculate the exposure times of each camera, to stabilize the scanning range in the different lighting conditions. Knowing the lighting conditions, cameras can be adapted with their high dynamic range to improve image acquisition and thus the quality of the images obtained. On the other hand, histograms also provide information about the brightness in certain areas of an image as will be described below. Therefore, continuous acquisition allows the system to be adapted to the current exterior lighting to improve the quality of the captured images and, at the same time, facilitate the elimination of the effects of the reflections 402 and improve the subsequent reliability of the detection of faces and the occupation of the vehicle, that is to say the occupation determination 403.

 In addition, there may be another problem in the execution of the procedure. Windshields and also the side windows can reflect the light that is more or less strong depending on the environmental and weather conditions. In particular, the effects of the reflections that can be produced on the glass affect the faces of the occupants and vary in position from one image to another. This is understood as a concealment outside the vehicle and can make it difficult to safely detect occupants.

 Therefore, these effects of the reflections on the windshields must be eliminated. According to the present invention, a method is provided which includes a step 402 in which said effects of the reflections can be eliminated. This step 402 has to be carried out with each image acquired. In a configuration it is at least two images. In another configuration it can be multiple images, for example multiple images representing more than 50 images. In yet another configuration, preferably for the acquisition of images of an approaching vehicle, the multiple images represent a sequence of images.

 The step of eliminating the effects of reflections 402 according to FIG. 4A or 4B includes an image treatment described in FIG. 4C. Preferably, the luminosity analysis stage 406 is carried out before stage 402 is produced. Said stage 406 allows to obtain from each image captured a histogram that provides not only information on the lighting conditions at the time of acquisition, but also on the individual luminosities in specific areas of each image.

 In one configuration, step 402 of eliminating the effects of reflections includes an iterative process that aims to identify the windshield, detect the luminosities of both the exterior and the interior of the vehicle and extract the relevant information inside.

To reduce the processing time of each image, we proceed to detect the contours 407 in each image and identify 408 the perimeters of the windshield of the vehicle to generate a mask 409 that allows reducing the area of the image to be analyzed. This reduced area then allows you to adjust the brightness optimally inside the vehicle and make the occupancy determination 403. In one configuration, a comparison can be made with windshield patterns for the identification of windshields 408 thus allowing greater security in windshield identification, and reducing data processing time.

 In the images, the windshield detected determines the area to which the image is to be limited by putting the mask 409. The advantage is that it is concentrated only in an area where there may actually be vehicle occupants. People outside the limit do not matter since it is considered that these people are not part of the occupation of the vehicle and should not be counted. In addition, a small area does not carry as much information to process and can thus speed up the treatment. The mask 409 that overlays the images also serves to facilitate the elimination of the effects of the glare 402 on the windshields that may make detection difficult.

 The mask 409 also helps to adjust the brightness inside the vehicle. Therefore, the histogram achieved in steps 401, 405 or 406 is analyzed only for the area defined by the mask. From the gain levels obtained from this histogram the cameras can be adjusted to the brightness of the interior of the vehicle in an optimal way. This means that there is both a continuous brightness adjustment to the exterior lighting conditions and a continuous brightness adjustment to focus on the interior of the vehicle.

 For the elimination of the effects of the reflections 402, this approach to the interior is taken advantage of by the fact that the reflections on the windshields have a luminosity different from the luminosity inside the vehicle and therefore also of the occupants' faces. This is based on the fact that a histogram of an image gives information about the different levels of gray present in an image. Each gray level corresponds to a certain luminosity and can be related to an object or a specific area within an image. Thus, it can be said that the luminosities are obtained through the histogram of each image. Therefore, it is necessary to analyze and then process the histogram to know what brightness corresponds to the reflections and then eliminate said luminosity from the images.

 The elimination of the effects of the glare 402 on the windshield then begins with obtaining the histogram of the area corresponding to the windshield in each image. Each gray level in the histogram corresponds to a specific area with a certain luminosity and can be related in the present invention to a luminosity inside or outside the vehicle. That is, the analysis of the histogram allows to discriminate between the interior light, where subsequently the faces of the occupants must be detected and located, and the exterior light that includes the effects of the reflections to be eliminated, as shown in FIG. 5A. Then we proceed to separate the two different luminosities and focus on the brightness of the interior of the vehicle in each image through a process of adjusting the gain.

FIG. 5A illustrates an example of histogram of the entire image and the limited image portion of the windshield area. In this example you can see two main areas FF1 and FF2 of brightness, which exist in both day and night images, due to the good lighting in the toll areas, and that the image cut on the interior of the vehicle allows to obtain a better histogram equalizer than the one obtained on the globality, in which the global histogram shows an accumulation of pixels in the area of high brightness. The FF1 zone represents that there is a high degree of ambient lighting. The FF2 zone more faithfully represents the differences in brightness of the interior of the vehicle.

 FIG. 5B represents the histogram limited to a single line of the image.

 The histograms obtained can be compared with histogram patterns, as shown in FIG. 6. The histogram model in Figure 6A shows a theoretical model with three zones: the FFa zone is a dark area inside the vehicle. The FFb zone is a more bright and extensive area, corresponding to the exterior of the vehicle. And the FFc zone represents the reflections. In FIG. 6B shows an equalized histogram model of a uniformly illuminated area, such as that obtained inside the vehicle. In this way, it is possible to determine which zone in the histogram corresponds to the brightness of the interior and which zone corresponds to the effects of the reflections.

 In another configuration, a brightness threshold can be predetermined. Any zone with a light intensity greater than the threshold can be classified as high intensity, and any zone with a light intensity less than or equal to the threshold can be classified as low intensity. Once the area of luminosity inside the vehicle is specified, a gain adjustment process is carried out. It is about adjusting the interior brightness zone in the histogram and thus separating the interior brightness from the exterior luminosity that includes the effects of reflections. Then, said adjusted area is widened which results in saturation in the area of the luminosity corresponding to the exterior. Thus, the gain adjustment process converts the effects of the reflections into white spots not computable in the image.

 Once this histogram has been generated, the windshield limits are first readjusted to improve the adjustment of the histogram according to the predefined models. In subsequent iterations, a gain adjustment process is applied in which it is concentrated only in this area, as can be seen in FIG. 5A. The different variations in brightness of subsequent iterations are therefore amplified. The effects of annoying reflections and other parasitic effects of the resulting information have been eliminated.

 Thus, the term "spot" is used to describe a reflection that has gone through a gain adjustment process and has become saturated with respect to its luminosity as described above.

 Saturation converting the effects of the reflections into non-computable white spots can result in a partial concealment of a face and thus hinder vision inside the vehicle. However, since at least two cameras are used for image acquisition which includes at least two different views, such partial concealments are tolerated because the at least two images that have been captured complement each other. In this way it can be achieved that a partial concealment in an image is eliminated by the information of at least one other complementary image.

This is explained in more detail with the configuration of two cameras according to FIG. 2A by way of example in which each camera captures an image. Each image has been captured since a different point of view, which means that the effects of reflections on the windshield may exist in one image yes and in the other no. The reason is that the presence of effects of the reflexes and / or their observation depends a lot on the viewing angle of the observer. Therefore, a change of position of the observer normally changes the effect of a reflex.

 The two images are treated 401, 405, 406 to obtain the histograms and identify the windshield 408 to then be able to carry out the gain adjustment process focusing on the area corresponding to the interior of the vehicle delimited by the mask 409. As has been described above, the gain adjustment process converts the effects of the reflections into non-computable white spots. After the gain adjustment process, this spot can be seen in a first image but in the second one it is not present because there were not the same effects of the reflections. Yes there may be other spots in different places of the second image caused by other effects of the reflections that were not in the first image. Therefore, what cannot be seen in a first image can be seen in the second. The solution then to eliminate the effects of the reflections 402 is to combine the two images and complement the hidden places in a first image with the visible of the second. In this way the effects of the reflections are eliminated and do not hinder the vision and determination of the occupation 403 of the vehicle.

 Preferably, in the embodiment of acquiring the at least two images of a stationary vehicle, multiple images are acquired to obtain more information and compensate for occupant movements that could result in partial or complete concealments of an occupant. The procedure of eliminating the effects of the glare on the windshield described above for two images can also be carried out with multiple images.

 In one embodiment, multiple images of an approximate vehicle are acquired that can represent more than 50 images in sequence. The advantage of the multiple sequence images acquired from an approaching vehicle is that each image contains effects of the reflections in different positions due to the movement of the vehicle. Therefore, there are several images that may contain a spot resulting from the process of adjusting the gain in a first position, several images containing a spot in a second position, several images containing a spot in a third position, and so on. There may also be images that have no stains. Therefore each image can provide some information about the existence and location of an occupant's face to complement the information of another image where a reflection hides the vision at least partially.

Within a sequence of images, you can see how the spots move along a path to one of the sides of the image. This means that if the multiple images after the acquisition process are put in a row in the order of their acquisition, a sequence of images can be obtained in which the spots are moved from an initial place to a different place, and preferably they leave the image, that is they are completely eliminated. If there are only two images, each from a different point of view, those two images are added to give a valid image. Vision difficulties in one image do not appear in the other. Therefore, the two images are complementary to each other.

 The same is done with a sequence of images obtained from multiple acquired images. In this case, the process of composing the valid image is carried out in several stages. First, correspondence is sought in complementary images throughout the sequence of images. From these complementary images, recomposed images are composed with the result that a second sequence of images can be obtained comprising fewer images than before. From this second sequence, complementary images are sought again to compose them obtaining a third sequence. Depending on the number of images in the sequence and the correspondence between the individual images there may be only one sequence, two sequences, three sequences or more sequences until finally a valid image is composed that represents the sum of all the images acquired with the effects of the reflections on the windshield removed. Thus, the occupation of a vehicle to be controlled can be determined in a safe and reliable way to apply or not discounts at the toll station.

 The aforementioned safe determination results on the one hand because of the change of view that occurs with at least two cameras, and on the other hand the elimination of the effects of the glare 402 on the windshield.

 The next stage consists in determining the occupation 403, or what is the same, counting the number of validly identified occupants. It is carried out on the one hand through a facial recognition 410 as seen in FIG. 4D. Facial recognition detects the faces of the occupants through typical features such as the nose, mouth, ears and eyes. The occupancy in a vehicle can then be determined by the number of faces detected.

 However, with a control system comprising at least two cameras, there may be the problem of duplication of people in the vehicle. For example, in the case of exactly two cameras that each capture exactly one image you can see the profile of an occupant for example both from the right and from the left. It may also be that, because of a movement, a large portion of an occupant's face is seen through a camera and only a small portion of the back through the other. So it is very likely that the two profiles captured by the two cameras look like two different people. Therefore, the system must determine whether these two profiles really correspond to two people or if perhaps it is only one person. The same can also happen with at least two, four or eight cameras that capture at least two and / or multiple images.

 The solution to determine if it is one or two people is to carry out an additional process of locating 415 of the occupants inside the vehicle to avoid duplication.

FIG. 4D illustrates an exemplary system 400 and method comprising the steps of identifying faces 410 and locating them 415 later. Said identification 410 and location 415 is carried out with each image that has been captured 401 cyclically. Once the field of vision is delimited inside the vehicle 408, a continuous process is carried out which includes a facial recognition 410 and a segmentation of faces 411. Said segmentation works through windows 412 for searching faces. A possible face is detected by means of operators that allow identifying the most relevant facial elements, which are the eyes, nose, ears and mouth, that is, a facial recognition 410 is executed. A window is opened within the delimited field of vision, facial elements are sought and the degree of certainty 413, 414 of the recognized facial elements is assessed.

 Subsequently, the reliability of the detections made and the probability of false vehicle occupancy detections are evaluated. This is carried out with a system of coherence analysis of the results. At this point the use of information on duplication of faces is also used as a decision factor. It is more likely that there is a person if both sides coincide in space. The result of the reliability assessment is reflected in a three-dimensional probability map. The reliability assessment exceeds a predetermined value, that is, the default value represents a reliability threshold. If the reliability exceeds the aforementioned predetermined value, the detection performed represents a person. A detection with reliability that does not exceed a predetermined value will not count as an occupant.

 For example, the default value can be predetermined as 70%. The reliability threshold can also be predetermined by a value of 90%, since the system of the present invention achieves an average reliability greater than 90%.

 By having the location of the windows whose content is a face of high probability of detection, FIG. 7, its position in a three-dimensional space is calculated, that is to say the step of locating the occupants 415 is carried out. The calculation in the space is carried out by means of the stereovision techniques and allows to locate the detected faces in the interior space of the vehicle without duplication. Location 415 includes the application of a floating threshold of the multivariable similarity function of each facial element, so that partial concealments are tolerated. Once the faces identified 410 in the vehicle are located 415 in each image obtained from each camera, a geometric reference system of its location within the vehicle proceeds to correlate 416 the locations made to group all the locations made of each occupant by the different cameras and avoid possible repeated accounting of an occupant.

 Once the identification of faces 410 and its location 415 is finished, the number of occupants in the vehicle to be controlled is known. This number decides whether or not to apply a discount for high occupancy. All the data obtained so far are managed by means of management that can comprise software and hardware.

The management of data obtained includes the transmission of the number of occupants to a central unit at the toll station and / or at least one control interface to monitor the procedure and its correct application. The central unit is programmed to decide if the detected number meets the requirements to be considered high occupancy and, in case Yes, to apply a discount to occupants traveling in this high occupancy vehicle.

 The method of the present invention may also comprise a step of storing the data obtained such as all acquired images, the date and time of acquisition, the place and the like for subsequent verifications, controls or claims.

 It is understood that the embodiments described herein can be implemented by hardware, software, firmware, middleware, or any combination thereof. If the system or procedure is implemented by software, firmware, middleware, code program or code segments, a computer program can be stored in a computer-readable medium as a storage component. The term "computer readable medium" may include, without limitation, various means capable of storing, containing and / or carrying instructions and / or data. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g. hard disk, floppy disk, magnetic strips, etc.), optical discs (e.g. compact discs (CD), digital versatile disc (DVD) ), etc.), smart cards and flash memory devices (for example, EPROM, card, USB stick, etc.).

 A computer program, or code segment, can represent a procedure, a function, a sub-program, a program, a routine, a sub-routine, a module, a software package, a class or any combination of instructions, structures of data or program report. A code segment can be coupled to another code segment or a hardware circuit through passing and / or receiving information, data, arguments, parameters or memory contents. Information, arguments, parameters, data and so on can be passed, forwarded or transmitted using any suitable means including a common memory, message pass, key pass, network transmission, and so on.

 For a software implementation, the techniques described herein can be implemented by modules (eg procedures, functions, etc.) that perform the functions described herein. Software codes can be stored in memory units and executed by processors. The memory unit can be implemented inside the processor or outside the processor, in which case it can be connected to the processor communicatively by several means known in the field. In addition, at least one processor may include one or more operable modules to perform the functions described herein.

Claims

Claims 1. Automatic system for vehicle occupancy control at a toll station comprising  means to acquire images;  means to eliminate the effects of glare on the windshield; Y  means for determining the number of occupants in the vehicle;  in which the means for acquiring images comprise at least two cameras, in which each camera acquires at least one image.  2. System according to claim 1, wherein the cameras have a lateral position with respect to the vehicle to be controlled, and there is at least one camera on each side of the road through which the vehicle is traveling.  3. System according to claim 2, wherein said means for acquiring images includes paired cameras, for example, two, four or eight cameras, and said paired cameras are at least two different levels in height.  4. System according to claim 2, further comprising a lighting system, an automatic or programmable control of exposure time, means for data management, means for data storage and at least one control interface. 5. System according to claim 2, wherein the means for acquiring images acquire multiple images representing more than 50 images in sequence. 6. System according to claim 2, wherein the means for eliminating the effects of the reflections on the windshield include means for iterative image processing, and comprises  means for detecting contours in each image;  means to identify the windshield in each image from the detected contours; Y  means to determine the brightness inside the vehicle.  7. System according to claim 6, wherein the means for determining the brightness comprises  means for generating a mask around the identified windshield;  means for generating a brightness histogram based on the mask area; means for determining the values that represent the outside environment and the values that represent the interior of the vehicle based on a certain threshold; and means for carrying out a gain adjustment process in which the effects of glare on the windshield are excluded.  8. System according to claim 7, wherein the means for carrying out a gain adjustment process comprises  means for focusing the area on the histogram that belongs to the brightness of the interior of the vehicle; Y means for widening said focused area. 9. System according to claim 2, wherein the means for determining the number of occupants comprises  means for detecting facial elements in each image;  means for assessing the reliability of facial recognition based on at least one threshold;  means to locate and count the faces of the occupants of the vehicle in each image. 10. System according to claim 9, wherein the at least one reliability threshold is greater than a predetermined value.  11. System according to any of the preceding claims, wherein the means for acquiring images acquire images when the vehicle is stopped at the toll station.  12. System according to claim 1, wherein the means for acquiring images also acquires images during the approach of the vehicle to the toll station. 13. Procedure for controlling vehicle occupancy at a toll station, which includes the stages of  acquire at least two images of a vehicle with at least two cameras, one image per camera;  eliminate the effects of glare on the windshield; Y  Determine the number of occupants in the vehicle.  14. A method according to claim 13, wherein the cameras have a lateral position with respect to the vehicle to be controlled, and there is at least one camera on each side of the road through which the vehicle circulates.  15. A method according to claim 14, wherein the cameras are paired cameras, for example, two, four or eight cameras, and said paired cameras are at least two different height levels.  16. The method according to claim 14, further comprising the steps of  manage the data obtained; Y  Store the data obtained. 17. The method of claim 14, wherein the step of acquiring at least two images includes acquiring multiple images representing more than 50 images in sequence.  18. The method of claim 14, wherein the step of eliminating the effects of the glare on the windshield includes an iterative treatment of images, and comprises detecting contours in each image;  identify the windshield in each image from the detected contours; and adjust the acquisition brightness to that inside the vehicle.  19. The method according to claim 18, wherein the step of determining the brightness comprises generate a mask around the identified windshield; generate a brightness histogram based on the mask area; determine the values that represent the outside environment and the values that represent the interior of the vehicle based on a certain threshold; Y  carry out a gain adjustment process in which the effects of glare on the windshield are excluded.  20. The method of claim 19, wherein the step of carrying out a gain adjustment process comprises  focus the area on the histogram that belongs to the brightness of the interior of the vehicle; Y  widen said focused area.  21. The method of claim 14, wherein the step of determining the number of occupants comprises  detect facial elements in each image;  assess the reliability of facial recognition based on at least one threshold;  Locate and count the faces of the occupants of the vehicle in each image.  22. Method according to any of the preceding claims, wherein the step of acquiring images is carried out when the vehicle is stopped at the toll station.  23. The method of claim 22, wherein the step of acquiring images is further carried out during the approach of the vehicle to the toll station.  24. Computer-readable medium comprising instructions, which once executed by a computer, carry out the procedure of any of claims 13 to 23.  25. Computer program for carrying out the steps of any of claims 13 to 23.