JP2019205078A - System and program - Google Patents

Abstract

To utilize images acquired by a camera attached to a vehicle to further identify a cause of an accident than before and make more effective use without being limited to identification of the cause of the accident.SOLUTION: Processing means of the system includes a function of recording an image captured by a camera attached to a vehicle and a function of performing image analysis of the image captured by the camera and executing processing corresponding to an analysis result on the basis of the analysis result.SELECTED DRAWING: Figure 2

Description

  The present invention relates to, for example, a system and program for processing an image captured by a camera or the like attached to a vehicle or the like.

  Japanese Patent Application Laid-Open Publication No. 2004-228561 discloses a system applied to a drive recorder or the like. This system repeats a cycle of recording video data in a recording area. When a predetermined cycle end condition is satisfied, the currently ongoing cycle is ended, and a new cycle is started in which the video data recorded in the completed cycle is overwritten with new video data in the old order. Outputs information about overwritten positions based on at least one of the volume of video data recorded in the current cycle and the volume of video data that was not overwritten among the video data recorded in the previous cycle. Let the means output.

JP 2018-6969 A

  The image recorded by the drive recorder is effectively used to identify the cause of an accident or the like. It is desired to use an image acquired by a camera attached to a vehicle or the like more effectively for identifying the cause of the accident than in the past, and more effectively using the image without being limited to identifying the cause of the accident.

  The object of the invention of the present application is not limited to this, and there is an intention to acquire the right by the divisional application, amendment, etc. for the configuration aiming to obtain the effect obtained from the configuration portion disclosed in the present specification and drawings, etc. . For example, the present specification discloses a problem in which a part described as “can be” in the present specification is read as “to be a problem”. The issues are described as being independent of each other, and there is an intention to acquire the right by a divisional application, amendment, etc. independently for the configuration for solving this issue. Even if the problem is implicitly grasped from the description of the specification, the applicant intends to make a part of the configuration described in the present specification as a claim in the amendment or divisional application. A problem combining these independent problems is also disclosed.

(1) A function of recording an image captured by a camera attached to a vehicle, and a function of performing image analysis of an image captured by the camera and executing processing corresponding to the analysis result based on the analysis result It is preferable that the system has processing means.

  In addition to recording an image captured by a camera attached to a vehicle, the image can be used more effectively.

  Examples of vehicles include passenger cars, freight cars, special cars, and the like. For example, the camera may image at least one of the inside and outside of the vehicle. Next, “inside the vehicle” and “outside the vehicle” may be interpreted as follows, for example.

  When the interior space and the exterior space are clearly separated by the body of the vehicle, the interior of the body may be referred to as “inside the vehicle” and the exterior of the body may be referred to as “outside the vehicle”. For example, in the case of a van in which an occupant space and a load space are continuous, the continuous occupant space and the load space may be referred to as the interior of the vehicle. In the case of a sedan in which the passenger space and the cargo space are partitioned, both the passenger space and the cargo space may be referred to as the interior of the vehicle. In a covered lorry with a closed cargo bed, the passenger space and the inside of the sealed cargo bed may be referred to as the interior of the vehicle. In open trucks (flat-body vehicles) with an open cargo bed, it is difficult to clearly distinguish the space on the cargo bed that accommodates the load from the surrounding space. The space to be accommodated may be called the inside of the vehicle. In addition, it is good to call the passenger space of an open cargo truck the inside of a car. In some special vehicles such as forklifts, both the passenger space and the load space are open, and it may be difficult to clearly distinguish the passenger space and the load space from the surrounding space. Even in such a case, both the passenger space and the cargo space may be called the inside of the vehicle. For example, a space above a seat on which an occupant sits and a space above a fork that lifts and lowers a load may be referred to as a vehicle interior. In this way, a space that can be considered to move with the vehicle as the vehicle moves is referred to as the inside of the vehicle, and a space around the space is referred to as the outside of the vehicle.

  As the “camera attached to the vehicle”, a camera whose relative position and posture with respect to the vehicle are fixed in a normal use state may be used. Further, as the “camera attached to the vehicle”, a hemispherical or omnidirectional 360-degree camera may be used. When a hemispherical or omnidirectional 360-degree camera is used, a wider range of images can be acquired than when a normal camera is used. For example, images inside and outside the vehicle can be acquired with a single camera. In place of the 360 degree camera, a plurality of ordinary cameras may be attached to the vehicle.

  An image captured by the camera may be a moving image. If the image captured by the camera is a moving image, the processing means can easily determine the movement of the subject, the presence or absence of the subject before and after a certain point in time by performing image analysis.

  The function of recording the image captured by the camera may be realized by a server that communicates with the in-vehicle device of the vehicle on which the camera is mounted, but it is even better when performed by the in-vehicle device. A removable medium such as an SD card may be used as the recording medium. When a removable medium is used as the recording medium, the removable medium on which the image is recorded can be removed from the in-vehicle device and displayed on another terminal such as a personal computer.

  The function of analyzing the image captured by the camera may be realized by a server that communicates with the in-vehicle device on which the camera is mounted, but it is better when performed by the in-vehicle device. When the in-vehicle device analyzes the image, processing corresponding to the analysis result can be quickly executed in the vehicle.

  The processing means may be composed of one device or a plurality of devices. For example, the drive recorder has a function of recording an image captured by a camera. The function of the processing means of this system may be realized by a microcomputer built in the drive recorder and an application program stored in the memory. Further, for example, the processing means may be constituted by two devices, an in-vehicle device and a server, and the in-vehicle device may record an image and the server may perform image analysis.

  In addition, the function which records the image imaged with the camera attached to things other than a vehicle, and the image analysis of the image imaged with the said camera are performed, and the process corresponding to an analysis result is performed based on an analysis result It is good also as a system which has a processing means with a function. However, it is particularly preferable to use (1).

  (2) The processing means has a function of determining whether or not the movement of the person in the image corresponds to a specific movement registered in advance as the function of the image analysis. When applicable, processing corresponding to the analysis results includes recording an image of a specific movement of a person, sending an image of the specific movement to a server, and reporting that a person has made a specific movement And at least one process of issuing an alarm may be performed.

  For example, a recorded image or an image transmitted to a server can be used as evidence that a person has made a specific movement. For example, by notifying that a person has made a specific movement, it is possible to notify an external person that there is a person making a specific movement. For example, a person who has made a specific movement due to the issued alarm is motivated to stop the specific movement by noticing that the alarm has been issued to him.

  For example, the person who performs a specific movement may be a vehicle driver. The specific movement may be, for example, an action other than the driver's driving. For example, an operation of viewing a mobile terminal such as a smartphone, an operation of eating food, an operation of looking aside or aside, and a doze may be used. For example, the presence or absence of a sidewalk operation may be determined based on the detection result of the driver's eye position. The recorded image can be used for improving driver awareness, analyzing the cause of an accident, and the like. When the processing means detects that the driver is operating a mobile terminal such as a smartphone, the processing means may issue an alarm that calls attention so as not to hold the mobile terminal while driving. If the driver notices an alarm and stops a specific movement, the danger can be avoided in advance. The server for data transmission may be a server of a vehicle manager or a police server. The manager of the vehicle can evaluate the driving attitude of the driver based on the transmitted data. The police can evaluate the driving attitude of the driver, analyze the cause of the accident, and the like based on the transmitted image.

  As a person who makes a specific movement, for example, a suspicious person outside the car, and as a specific movement, for example, an action to try to break into the car, for example, an action to break a glass window with a hammer, an action to look into the car, etc. It is good to do. By issuing a warning, it is possible to discourage the suspicious person who intends to illegally enter the vehicle from entering the vehicle. The recorded image or the image transmitted to the server can be used as useful information for arresting the thief after an on-vehicle crime occurs. By notifying the owner or driver of a vehicle that a person has made a specific movement, the owner or driver may immediately return to the vehicle and prevent damage from aiming on the vehicle. The driver or owner who has received the notification can also report to the police. Thus, this function can be used for crime prevention.

  As a person who makes a specific movement, for example, a suspicious person in the car, and as a specific movement, various ways of aiming on the car, such as illegal entry into the parked car, movement of the suspicious person in the parked car, It is recommended that the device be removed. If a thief rushes away by issuing a warning, the effect of reducing the damage on the vehicle is obtained. The recorded image or the image transmitted to the server can be used as useful information for arresting the thief after an on-vehicle crime occurs. By notifying the vehicle owner or driver that a person has made a specific movement, the owner or driver can immediately return to the vehicle or report to the police to reduce damage to the vehicle. It becomes possible to make it. As an example of notification to the outside, transmission of an e-mail attached with an image showing a suspicious person can be cited. The destination of the e-mail may be a vehicle manager, driver, owner, police, or the like.

  As a person who makes a specific movement, for example, a passenger of a business vehicle on which an unspecified person rides, and as a specific movement, various actions of robbery, such as an action of flickering a blade, an action of holding a pistol, an action of shaking violence, etc. It is good to do. When the processing means executes a process of issuing an alarm, damage can be prevented before the burglar stops thinking of a crime. The recorded image or the image transmitted to the server can be used as useful information for arresting the robber. By notifying the vehicle administrator that a person has made a specific movement, the administrator can notice the occurrence of a burglary and notify the police. The processing means may have a function of notifying the police using a communication function.

  For example, the person who makes a specific movement may be an infant in a car carrying the same, and the specific movement may be, for example, a movement when waking up from sleep. When detecting the awakening of the infant by image analysis, the processing means may notify the guardian by e-mail or the like. There are cases where an infant sleeps in the car while driving and wants to go shopping or do some work with the engine running. At this time, if the infant is forced to wake up, the mood will be worse, and there is a demand to leave it as it is. However, parents care about whether the infant is awake when they are away from the vehicle. When this system is used, an e-mail is sent to the guardian's mobile terminal when the infant wakes up, so the parent can immediately rush to the vehicle to deal with the infant's awakening.

  For example, the person who needs a specific movement may be a care recipient outside the vehicle or a rescuer, and the specific movement may be an action peculiar to the care recipient or an action corresponding to a situation where rescue is considered necessary. For example, as an example of a specific action of a person who needs assistance, for example, an action of walking with a cane, bending a waist, and walking well. As an example of an operation corresponding to a situation where rescue is deemed necessary, an operation corresponding to a situation where the user falls and blows bubbles from the mouth and falls.

  The function to detect a person making a specific movement includes the function to select at least one of the image in front of the vehicle, the image in the rear of the vehicle, and the image in the vehicle. It is good to do. The selection of at least one image may be specified by the user. As a result, the processing unit does not analyze an unnecessary or less important image, so that it is not necessary to give the processing unit excessive processing capability.

  The function of detecting a person who performs a specific movement may include a function of detecting a driver's sleepiness. For example, the pulse may be measured from the degree of change in the redness of the driver's skin, and the presence or absence of the driver's sleepiness may be determined based on the change in the pulse. It is recommended to improve the drowsiness detection accuracy by linking body temperature measurement with a thermo camera, pulse measurement with a microwave, and the like. When the processing means of this system detects drowsiness, it is preferable to lower the set temperature of the air conditioner or issue a warning by voice or sound.

  (3) The processing means has a function of determining whether or not a person in the image has a specific characteristic registered in advance as a function of the image analysis, and as a process corresponding to the analysis result, A system that executes processing according to specific characteristics of a person may be used.

  Appropriate processing can be performed according to the characteristics of the person in the image.

  For example, a face authentication function may be included in the function of determining whether or not a person in the image has a specific feature. For example, face authentication of a person seated in the driver's seat is performed, and when the person seated in the driver's seat does not correspond to a person registered in advance, the processing unit may access the vehicle so that the engine cannot be started. For example, a signal for prohibiting engine start may be transmitted from the processing means to the engine control unit (ECU). Thereby, a person who can drive can be registered in advance, and driving of the vehicle by an unregistered person can be prohibited. As a result, the crime prevention function of the vehicle can be enhanced. It is also effective as a countermeasure against child mischief. In addition to the process of prohibiting starting the engine, the accelerator cannot be stepped on, the side brake (parking brake) cannot be released, the shift lever does not enter the drive position, an alarm is sounded, and registered Processing such as sending an e-mail with an image to a mail address may be performed.

  In addition, for example, face authentication of a person outside the vehicle is performed, and when the person outside the vehicle corresponds to a person registered in advance, the processing unit may control the vehicle to release the door lock. For example, when a vehicle is provided for a car sharing service, registered members of the car sharing service may be registered in advance. The registered member can use the vehicle even if he / she does not have a card key or the like by moving his / her face within the angle of view of the camera of the vehicle by approaching the vehicle. Thereby, the usability of the vehicle of the car sharing service is improved.

  The vehicle may be a business vehicle on which an unspecified person gets in, and the processing means may determine whether or not the passenger is a criminal being arranged by the face authentication function. The photo of the criminal's face being arranged may be transmitted from the police database to the processing means of this system by communication via LTE, wireless LAN, etc., and registered in advance in the processing means. When it is determined that the passenger is a criminal being arranged, the processing means may perform notification to the vehicle manager, notification to the police, and the like via the communication device and the communication network. Furthermore, the processing means may notify the driver in a way that is not understood by the passengers. The vehicle manager, police, driver, etc. can recognize that the criminal being arranged has boarded the vehicle and can take appropriate action promptly.

  For example, taxis are in a closed room where only crew members and passengers get on, so there is a great demand for this system in terms of ensuring the safety of crew members. When the processing means detects that the criminal has boarded a taxi, the taxi company or the police may be notified of the criminal boarding. Further, according to the present system, it is possible to detect the criminal's boarding when the criminal gets into the vehicle before the criminal actually commits the criminal act in the vehicle. This makes it possible to avoid crimes.

  A function for determining whether or not a person in an image has a specific feature, for example, a function for comparing the appearance of a person in an image with a specific appearance registered in advance and determining whether or not they match. (Hereinafter referred to as appearance authentication function) may be included. As specific appearances, it is advisable to register in advance an appearance peculiar to a deaf person, an appearance peculiar to a criminal, or the like. Examples of appearances specific to criminals include, for example, having a knife on the road, having a dangerous object, and the like. In addition, it is preferable that a person carrying an illegal drug can be detected by combining an image of an infrared camera, an ultraviolet camera, or the like with an image of a normal camera. When the processing means detects a person having a specific appearance in the image, the processing means may notify the police or vehicle manager via the communication device and the communication network.

  When it is desired to find a specific missing person or a deceased person, for example, the appearance at the time of missing may be individually registered as search information in the processing means. For example, it is preferable to register individual search information such as gender, height, type of clothes, color of clothes, whether or not a hat is worn, whether or not you have a cane, and whether or not your waist is bent. Furthermore, it is good to register the area (search area) that seems to be hesitant. The processing means may execute a detection process of a person matching the individual search information when the vehicle is located in the search target area. Such individual search information may be transmitted from a database (server) such as the police or a local support center to the processing means of this system. A function for determining whether or not a person in the image has a specific feature is performed by the in-vehicle device, and when the vehicle enters the search area, the search information may be downloaded from the server to the processing means of the vehicle.

  A function for determining whether or not a person in the image has a specific feature, for example, a function for determining whether or not a person sitting in a passenger seat or a rear seat is wearing a seat belt (hereinafter referred to as an appearance authentication function) )). When the processing means of this system detects a person who is not wearing a seat belt, it may issue a message prompting the user to wear the seat belt.

  In particular, when a person in the back seat approaches the entrance to the expressway with no seat belt attached, a message prompting the user to attach the seat belt may be issued. For example, when the distance from the current position of the vehicle to the entrance of the expressway is equal to or less than a reference distance, it may be determined that the vehicle has approached the entrance of the expressway. In addition, if a signal for checking forgetting to insert an ETC card is received, it may be determined that the vehicle has approached the entrance of the expressway. Further, when an image of the front of the vehicle is analyzed and the entrance of the expressway is detected in the image, it may be determined that the vehicle is approaching the entrance of the expressway.

  (4) The processing means has a function of counting the number of people in the image as the function of the image analysis, and is a system that executes a process of recording or transmitting the number of people data in the image as a process corresponding to the analysis result. Good.

  The system can count the number of people without using a special device for counting the number of people.

  For example, a camera is attached to a bus, and an in-vehicle image may be adopted as an image from which the number of people is counted. The processing means may display the count result and inform the crew. The crew can easily grasp the number of passengers (passengers) on the vehicle. This function can be used in place of a tour call. When all passengers should get off, when the result of counting the number of passengers does not become zero, the crew can know that there are people left in the car. For example, it is possible to detect the remnants of a person who cannot express his / her intention such as a baby or a person who cannot get off himself / herself such as a disabled person.

  For example, the vehicle to which the camera is attached is a type of vehicle in which the number of passengers is predetermined, such as a sightseeing bus, a long-distance high-speed bus, etc., and the predetermined number of passengers is registered in the processing means of this system. Good. The processing means may compare the counting result of the number of persons with the registered number of persons, and if there is an excess or deficiency, warn the crew about the excess or deficiency. Thereby, it is prevented that it departs with the number of passengers shortage. In addition, it is possible to easily grasp that a person who is not qualified to get on the vehicle has boarded, and to prompt the user to get off.

  For example, a camera (for example, a hemisphere camera on the ceiling) is installed in the bus of the route bus, and the processing means performs image analysis to count the number of passengers, the number of passengers standing, and the number of passengers sitting. It is good to have. Furthermore, the processing means obtains the current position information of the bus, the signal for opening / closing the bus door, the buzzer for getting off the bus, the touch of the IC card when getting on and off the bus, and the signal of the toll box, and recording each of them as a trigger. It is good to have. This information is accumulated in the data lake, and it becomes possible to analyze big data on where the person who got on and where the passenger got off and how the seating situation in the car was during that time. Such data can be valuable information for consulting firms that perform city planning and the like. Furthermore, it is good to acquire together the information transmitted through the in-vehicle network such as CAN and the vital information of the driver. Such information can be used to specify a causal relationship such as whether the abnormality is a driving factor, a passenger factor, a driver's own factor, or a vehicle factor when some abnormality occurs in the vehicle.

  For example, an image outside the vehicle may be adopted as an image from which the number of people is counted. The processing means of this system may record the counting result as the number data. Based on the recorded number data, it is possible to easily extract places where there are many people. It is preferable to record the image that is the basis of counting, using the counting result as a trigger. For example, an image may be recorded when the number of persons is equal to or greater than a predetermined threshold value.

  (5) The processing means has a function of detecting the display content of the display device of the vehicle or the operation status of the operation device of the vehicle from the image in the vehicle as the function of the image analysis, and as the processing corresponding to the analysis result, The system may be a system that executes processing according to the display content of the display device or the operation status of the operation device.

  These data can be acquired by this system without retrofitting a device that acquires data that is the basis of the display contents of the display device or a device that acquires data according to the operation status of the operating device. For example, these data can be acquired without connecting a retrofit device to the OBD connector. In particular, even when the vehicle manufacturer prohibits connecting a third-party product to the OBD connector, these data can be acquired by the present system. The display device may include various instruments, a warning light (warning lamp) in the vehicle, a head-up display that displays an image on a windshield, and the like.

  The display content of the display device to be detected may be, for example, the vehicle speed displayed on the speedometer. The processing means may preferably recognize the position of the needle of the analog speedometer and recognize the numbers on the dial to obtain the vehicle speed data. In addition, vehicle speed data may be obtained by character recognition of numbers displayed on a digital speedometer. The acquired vehicle speed data can be used for various processes. Hereinafter, an example of the usage form of the vehicle speed data will be described.

  The processing means may have a function of automatically detecting the speedometer by image recognition. With this function, the processing means can detect the speedometer in various vehicles without depending on the vehicle type. In addition, the position and shape in the image of the speedometer of the vehicle on which the processing unit is mounted may be registered in the processing unit in advance. Thereby, the processing means can easily detect the speedometer.

  The stand-alone type car navigation device uses position information by GPS to acquire the current position. When a GPS signal cannot be received such as in a tunnel, the current position of the vehicle must be determined by another method. As an example, the current position of the vehicle in the map is moved for 5 minutes at a constant speed of about 80% of the speed immediately before the GPS signal lost. A mode for performing such processing is referred to as a tunnel mode. The tunnel mode is effective not only when entering a tunnel but also when traveling under an elevated road. However, if a traffic jam occurs when processing in the tunnel mode, the method of assuming that the vehicle is traveling at a constant speed results in an excessively large error in the current position, which is substantially useless. If the acceleration sensor is attached to the vehicle and the acceleration data from the acceleration sensor is used, the accuracy of the current position information in the tunnel mode can be increased. However, it is difficult for the user to attach the acceleration sensor, which is expensive and expensive.

  It is preferable to read the own vehicle speed from the display content of the speedometer and update the current position information of the own vehicle using the map matching function implemented in the car navigation system based on the read speed. Thereby, the accuracy of the current position information of the host vehicle can be increased.

  In a place where the road in the tunnel and the ground road are running side by side, or in a place where the elevated road and the ground road under the overhead road are running side by side, matching is made to which road based on whether or not the tunnel mode is activated. It is good to choose. For example, when the tunnel mode is activated, the vehicle position is mapped to the road in the tunnel or the ground road under the overhead, and when the tunnel mode is not activated, the vehicle position is aligned with the road within the tunnel. You can map it to a running ground or elevated road.

  The display content of the display device to be detected may be, for example, the remaining amount of fuel displayed on the fuel gauge. The processing means may detect the remaining amount of fuel by performing image recognition of the fuel gauge. When the processing means detects that the remaining amount of fuel has fallen below a predetermined reference value, the processing means may notify the driver of a message prompting refueling. For example, if the remaining fuel level is less than ¼ of the reference value, for example, when the tank is full when returning from home or near the vehicle storage location of the company in conjunction with the current position information of the vehicle The processing means may issue a warning and prompt refueling. In addition, the car navigation device and this system are linked, and the processing means compares the distance to the destination and the remaining amount of fuel, and refuels on the way to the destination or on the way to the destination based on the comparison result. Prompt. For example, a message prompting refueling may be output at a gas station near the current location near a gas station on the way.

  The function of extracting the speedometer and fuel gauge from the image may be realized by artificial intelligence (AI) installed in the processing means of this system. If it is difficult to extract a speedometer or fuel gauge from an image by artificial intelligence, the position of the speedometer or fuel gauge in the image and image information may be registered in advance in the processing means.

  When the processing means reads the fuel meter of the vehicle provided for the car sharing service and detects that the remaining amount is less than a predetermined reference value, the processing means that the benefit of the car sharing service can be obtained when refueling is performed. You should notify the driver. For example, a message such as “If you refuel now, you can receive a discount of an amount equivalent to 15 minutes” may be issued.

  The display content of the display device to be detected may be, for example, whether or not a warning lamp in the instrument panel is lit. It is preferable to include a headlight lighting indicator, a blinker indicator, a high beam indicator (high beam warning light), a perking brake indicator, a half-door warning light, a fuel warning light, and the like as warning lights that are subject to determination of lighting. The processing means of this system can detect the state of the vehicle corresponding to the warning light by determining whether or not these warning lights are lit.

  The display content of the display device to be detected may be, for example, warning information displayed by a head-up display. The processing means may detect a warning content by detecting a change in the image displayed by the head-up display, and execute a predetermined process according to the warning content. For example, a warning may be issued by sound or voice. By issuing a warning based on visual information instead of auditory information, the driver can easily recognize the warning.

  For example, in conjunction with brightness information outside the vehicle or current position information of the vehicle, the processing means of this system is in a situation where the headlight should be turned on, but the headlight lighting indicator is not lit. When this is detected, the driver may be notified to turn on the headlight. The situation where the headlight should be turned on includes, for example, a situation where the outside of the vehicle is darker than a certain brightness, a situation where the current position of the vehicle is in a tunnel or a culvert, and the like. When the lighting indicator of the headlight remains lit despite the situation where the headlight should be turned on, the processing means of the present system may notify the driver to turn off the headlight. Because there are tunnels that are bright inside due to lighting, it is determined whether or not the headlights should be turned on only from the brightness information outside the vehicle. Otherwise, a situation in which it is erroneously determined may occur. Such an erroneous determination can be avoided by determining whether or not the vehicle is in the tunnel based on the current position information of the vehicle. Moreover, when it is detected that not only the headlight but also the vehicle width light (small light) should be turned on, the vehicle width light is turned on when it is detected that the light indicator of the vehicle width light is not turned on. The driver should be notified.

  The processing means of this system determines the presence or absence of an oncoming vehicle from the image in front of the vehicle, and if it detects that the high beam indicator is lit despite the presence of an oncoming vehicle, it alerts the driver to change to the low beam. Good. Since the beam high / low switching operation differs depending on the vehicle type, the driver is likely to experience an event that the intention to change to the low beam is still the high beam when driving a vehicle of a vehicle type different from the regular vehicle type. In such a case, the function of issuing this warning is particularly effective.

  The processing means of this system may detect that the blinker is blinking from an image outside the vehicle when the outside of the vehicle is dark, and recognize the indicator blinking in the vehicle in synchronization with the blinking period of the blinker as the blinker indicator. . By doing in this way, a processing means can find out the turn signal indicator in an instrument panel easily.

  When the processing means of this system detects that no one is in the vehicle from the image inside the vehicle, it detects that the headlights or the width lights are on, and then the driver or vehicle manager outside the vehicle. It is better to notify that the lamp is forgotten to be turned off. This notification may be performed, for example, by sending an e-mail to a pre-registered mail address. In addition, the processing means may notify the hazard lamp forgetting to turn off and the room light forgetting to turn off by e-mail or the like. The lighting state of the hazard lamp may be detected from the blinking state of the blinker indicator. The lighting state of the room light may be detected from the brightness of the room light reflected in the image.

  There are generally three types of room light settings: a constantly lit mode, a normally unlit mode, and a door open / closed state interlocking mode (hereinafter simply referred to as interlocking mode). In many vehicles, these three modes are cyclically updated each time the room light on / off switch is pressed once. Some vehicles have the function of turning on the room light when the engine is stopped in the dark (at night) and automatically turning off after a while (for example, after a few seconds). You may want to turn on the room light for a longer period of time to prepare for getting off after the engine has stopped. For example, after the engine is stopped, the room light should be turned on during the period when the purchased items are sorted and taken out, the portable terminal is taken out, and the like. In such a case, the driver switches the setting of the room light to the interlock mode or the always-on mode so that the room light does not automatically turn off when a certain period of time has elapsed since the engine stopped. However, when the door is open, it is difficult to determine whether the current room light setting is the constant lighting mode or the interlocking mode. Even if the current setting of the room light is in the always-on mode, if the user gets off the vehicle without thinking that it is in the interlock mode, the room light remains on. By using this system, the driver or the administrator notices that the room light has been forgotten to turn off, and turning off the room light can prevent the vehicle battery from running out due to the room light being turned on for a long time. Just looking at the operation switch, it does not know which mode the current setting of the room light is in, for example, a vehicle with an operation switch with a structure that the setting of the room light changes cyclically each time the push button is pressed This feature of the system is particularly useful.

  When the room light is set to the interlock mode, the room light remains lit if any door is open or half-door. A driver or manager who is notified that the room light is lit can recognize that the door may be in a half-door state or the door on the side that is difficult to see may be open.

  When the setting of the room light is in the interlocking mode, there is a vehicle having a function of keeping the room light on until a certain time (light-out delay time) elapses after the door is closed. In such a vehicle, it is preferable that the processing means of this system does not determine whether or not to forget to turn off the light until the turn-off delay time elapses after the door is closed. The turn-off delay time may be set by the system based on input from the user.

  Further, the processing means may transmit a signal for turning off the room light to the controller of the vehicle when detecting that the room light remains lit even after a certain period of time has passed since no one is in the vehicle. .

  The processing means of this system keeps one of the lights on when it detects that the vehicle door has been opened with the headlights, vehicle width lights, hazard lights, room lights, etc. lit or flashing. It is advisable to have a function to issue an alarm to notify that. This makes it possible for the driver to notice that he / she has forgotten to turn off before going out of the vehicle and to prevent the driver from forgetting to turn off.

  When the processing means of this system detects that the parking brake is applied based on the lighting of the parking brake indicator, even if it detects that the driver is falling asleep from the driver's image, It is better not to issue. In addition, when the parking brake is applied, an alarm is issued even if the driver sees a movement that is considered dangerous if driving, such as looking at a mobile device. Do not do it.

When the processing means of this system detects that the warning light indicating that it is necessary to perform inspection or repair at an early stage remains on for a certain period of time, it notifies the dealer of the vehicle or the registered repair shop. It is desirable to have a function of notifying the vehicle state. For example, a user who is not interested in an automobile often leaves the vehicle without being aware of the lighting of the warning light when the user can travel even if the warning light is on. For example, when the warning light of the engine itself is lit, it is normally impossible to run, but the vehicle can run as it is even if the warning light notifying the abnormality of the engine oil is lit. Further, when the O ₂ sensor is abnormal, the fuel consumption is deteriorated, but the traveling itself is possible. When traveling is possible, if the vehicle is continued without being inspected and repaired, a minor failure may become serious. For a dealer or a repairer, notification of lighting of a warning lamp is useful information for conducting an inspection / repair business for the owner of the vehicle whose warning lamp is lit. Further, the owner of the vehicle can prevent the failure from becoming serious by performing inspection and maintenance at an early stage.

  The display content of the display device to be detected may be, for example, the temperature displayed in the temperature display area in the instrument panel that displays the temperature measurement value measured by the temperature sensor that measures the temperature outside the vehicle. The processing means may have a function of reading the temperature by character recognition from the image displayed in the temperature display area of the instrument panel as a function of image analysis. The processing means may execute a process of transmitting temperature information to an external server via a communication device and a communication network as a process corresponding to the analysis result.

  For example, in order to acquire data related to pollen scattering, a service that installs a sensor in a general user's house, collects measurement data from the sensor, predicts detailed pollen scattering status, and distributes pollen scattering information Provided by Weathernews. In addition, outdoor photos posted by general users and images collected from outdoor live cameras installed in homes of ordinary households are incorporated into the weather forecast. In addition, a service for distributing these images (for example, Weather News SOLiVE 24) is provided. A system that provides these weather information services can use many vehicles as temperature measurement probes by collecting outside air temperature information measured by many vehicles. If many vehicles that transmit temperature information are distributed over a wide range, temperature information collected from many vehicles can be used as spot information. For example, it becomes possible to issue a spot-like heat stroke alarm that cannot be expressed by general weather information.

  For a company that provides such a weather information service, it is possible to collect temperature information over a wide range without installing a temperature sensor in-house. The company that has received the provision of temperature information may provide free or discounted provision of a paid weather information service provided by the company in return for the owner of the vehicle that provided the temperature information.

  By recognizing not only the temperature information read from the temperature display area in the instrument panel but also the number displayed on the temperature display sign beside the road image captured by the camera, Get it. The temperature indication marker may be detected by recognizing characters such as “current temperature” in the marker, for example. The acquired outside air temperature information may be transmitted by the processing means of this system to an external server, similarly to the temperature information acquired from the temperature display area in the instrument panel.

  The function of detecting the operation state of the operating device by image analysis may include a function of detecting the position of the driver's foot with respect to the accelerator pedal and the brake pedal. As a process according to the operation status of the operating device, a process of notifying the driver so that the driver can easily recognize whether the driver's foot is at the accelerator pedal position or the brake pedal position. It is good to execute. For example, the driver may be notified by changing the color of light emitted by the light emitting element when the foot is at the accelerator pedal position and when the foot is at the brake pedal position. This system is installed in a vehicle that has a function to detect whether the accelerator pedal or the brake pedal is on and display the detection result as visual information on the display. The contents displayed on the display may be image-analyzed to detect the position of the driver's foot with respect to the accelerator pedal and the brake pedal.

  Further, when an obstacle is detected forward by image analysis, an alarm may be issued with a sound or the like when the foot is at the position of the accelerator pedal. When the driver notices the alarm, it is possible to suppress the occurrence of a mistake in stepping between the accelerator pedal and the brake pedal.

  The function of detecting the operation state of the operating device by image analysis may include a function of detecting the rotation angle of the steering wheel (handle). As a process according to the operation state of the operating device, a process for obtaining the traveling direction and the travel distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed may be executed. For example, the processing result may be given to a stand-alone car navigation device. The stand-alone type car navigation device can calculate the travel route of the vehicle even in a situation where the GPS signal cannot be received based on the information on the traveling direction and the travel distance given from the system.

  The function of performing image analysis may include a function of detecting whether the vehicle is in automatic driving (including automatic driving levels 1 to 5) or in manual driving (automatic driving level 0). This detection may be performed, for example, by recognizing from the image the display state of an indicator indicating whether or not various functions of automatic driving displayed on the instrument panel are operating. As a process corresponding to the analysis result, for example, when it is detected that the automatic driving function is released, a process of notifying the driver that the automatic driving function is released may be executed. In addition, when it is detected that the driver has not touched the steering wheel even though the automatic driving function is released, the processing means issues a warning to alert the driver to operate the steering wheel. Good. Further, the processing means may record an image captured by the camera, triggered by the cancellation of the automatic driving function and the fact that the driver has not touched the steering wheel. The recorded image can be used as evidence that the driver's operation was inappropriate.

  In a vehicle equipped with a radar detector, the processing means may invalidate the radar detector function when the automatic driving function is effective. As a result, it is possible to eliminate generation of unnecessary alarms from the radar detector. When the automatic driving state is invalid, the processing means may validate the function of the radar detector.

  Whether the vehicle is in automatic driving or manual driving, and the level (mode) of automatic driving is the level (mode) of the indicator, steering wheel movement, the position of the driver's hand, the direction of the driver's head It is good to judge from the direction of being.

  (6) The processing means has a function of recognizing characters displayed on a sticker affixed to the vehicle from an image in the vehicle as a function of the image analysis, and the recognized character as processing corresponding to the analysis result It is good to set it as the system which performs the process which alerts according to the information shown by.

  By being alerted, the driver, manager or owner of the vehicle can pay attention to the character information on the sticker attached to the vehicle and can perform an appropriate process according to the character information.

  For example, an inspection mark (vehicle inspection sticker) may be included in the sticker to be analyzed. The date on which the validity period of the vehicle inspection expires is displayed on the inner surface of the inspection mark. The processing means reads the information on the date when the valid period of the vehicle inspection expires by character recognition, and if the period from the present time to the expiry date of the effective period of vehicle inspection (the vehicle inspection deadline) becomes shorter than the predetermined period, It is good to call attention to receive. For example, the processing means may call attention by issuing a voice saying “The expiry date of the vehicle inspection validity period is approaching”. When the period until the expiry date of the valid period of the vehicle inspection is further shortened, for example, the processing means may issue a warning with a voice “Please have a vehicle inspection”.

  For example, a maintenance / inspection sticker may be included in the sticker to be analyzed. The time limit for the next periodic inspection is displayed on the maintenance / inspection sticker in the date. The processing means may read the date information indicating the time limit of the periodic inspection by character recognition, and if the period from the current time to the time limit of the periodic inspection becomes shorter than a predetermined period, the processing means may alert the user to receive the periodic inspection. For example, the processing means may call attention by issuing a voice saying “The deadline for periodic inspection is approaching”. When the period until the periodical inspection is further shortened, for example, the processing means may issue a warning with a voice “Please perform periodic inspection”.

  The image analysis function may include a function for determining whether or not an inspection mark is included in an image captured by the camera. As a process corresponding to the analysis result, when the inspection mark is not reflected in the image, the processing means executes a process for calling attention to adjust the mounting position and posture of the camera so that the inspection mark is reflected. Good. For example, the processing means may emit a voice “The inspection mark is not shown. Adjust the position and posture of the camera so that the inspection mark is shown.” Thereby, at the time of installation of a camera, the position and attitude | position can be adjusted easily. More preferably, the processing means has a function of reading contents displayed on a display device in the vehicle, such as a speedometer, a fuel meter, and the like, and has a function of calling attention so that the display device in the vehicle is reflected in the image.

  The image analysis function may include a function of determining whether or not a sticker other than a sticker approved by law is attached to the windshield, the side windows of the driver's seat and the passenger seat. As a process corresponding to the analysis result, when it is determined that a sticker other than a sticker permitted by law is attached, a process of notifying the driver or the like to remove the sticker may be executed.

  (7) The processing means includes a function of analyzing an image of a region in which the passenger seat is reflected as the function of the image analysis, and a system that executes processing corresponding to the situation of the passenger seat as processing corresponding to the analysis result It is good to do.

  For example, appropriate processing can be executed depending on whether a person is on the passenger seat or whether an object is placed on the passenger seat.

  The processing means may be mounted on a vehicle having a function of issuing a warning when a person sitting in the passenger seat does not wear a seat belt. The processing means may have a function of determining whether or not a person is sitting on the passenger seat from an image of a region where the passenger seat is reflected as a function of image analysis. As processing corresponding to the analysis result, the processing means may disable the function of issuing a warning that the passenger seat is not worn on the passenger seat when detecting that no person is sitting in the passenger seat. The processing means may have a function of determining whether an object is placed on the passenger seat as a function of image analysis. As processing corresponding to the analysis result, the processing means may execute processing for invalidating a function of issuing a warning that the passenger seat is not worn on the passenger seat when an object is placed on the passenger seat.

  The vehicle may erroneously determine that a person is on the vehicle when an object equivalent to the human weight is placed on the passenger seat, and may issue a warning that the seat belt is not worn. When this system is attached to a vehicle, the vehicle will not issue a warning that the seat belt is not worn when an object is placed in the passenger seat. For this reason, the driver is not confused by the warning. In particular, this function is effective in a vehicle having a specification in which a warning buzzer continues to sound when a heavy object is placed on the passenger seat. In order not to issue a warning that the seat belt is not worn to the vehicle, for example, the processing means may transmit a signal for invalidating the warning to the vehicle controller through the in-vehicle network (CAN).

  Regarding the rear seat, in a vehicle having a similar function, when it is detected that an object is placed on the rear seat, the processing means may cancel the seat belt non-wearing warning function.

  (8) The processing means has a function of comparing a current in-car image and a past in-car image as the function of the image analysis, and as a process corresponding to the analysis result, The system may be configured to execute a process for notifying when the degree of inconsistency with a past in-vehicle image exceeds a predetermined level.

  By receiving notification of the owner or manager of the vehicle, it is possible to know that the situation inside the vehicle has changed beyond a certain level. As an image in the past of the vehicle, an image in a state where no objects other than those previously attached to the vehicle are brought into the vehicle may be used. By comparing this image with the current in-vehicle image, it is possible to determine whether there is anything brought into the vehicle.

  By comparing the images at a plurality of times in the past with the images at the present time, it is preferable to calculate the time during which an object brought into the vehicle remains in the vehicle. The processing means may have a function of notifying an object that has been left in the vehicle for a long time.

  For example, in a state where no load is loaded on the vehicle, and preferably only a driver is in the vehicle, the initial setting button of the present system is pressed to drive the vehicle. The processing means recognizes a portion in the image having an optical flow corresponding to the vehicle speed as a window. Any other movement is recognized as a driver. The part other than the part recognized as the window and the driver is recognized as an area where an object can be placed, and is recorded as an unloaded state image (unloaded state image) in which no load is loaded on the vehicle. The initial setting process is completed by the process so far. Thereafter, when the engine is started and when the engine is stopped, the difference between the no-load state image and the current image is taken to identify the presence of an object brought into the vehicle. A warning is issued when there is no change in the position of the object identified as being brought into the vehicle. This makes it possible for the driver or the like to recognize an object that has been left in the vehicle for a long time. This function is particularly effective for people who are quickly filled with garbage.

  (9) The processing means has a function of determining whether or not a pre-registered object is reflected in an in-vehicle image as a function of the image analysis, and a pre-registered object as a process corresponding to the analysis result It may be a system that executes a process of issuing an alarm when is shown in an image in the vehicle.

  The driver or the like can notice that there is something pre-registered in the vehicle by the warning. For example, an image of an item (a specific bag, wallet, etc.) that the user wants to check for lost items may be registered in the processing means. When it is detected that the registered object is reflected in the image in a state where no one is in the vehicle, the processing means may issue an alarm or notify the forgotten object. As an example of issuing an alarm, for example, a buzzer attached in the vehicle may be sounded. As an example of notification of forgotten things, it may be notified by e-mail or the like to a smartphone or a mobile phone such as a driver wirelessly.

  For example, the processing means may issue an alarm when detecting forgotten things in the vehicle and detecting a movement to open the door. The warning allows drivers and others to notice forgotten things before leaving the vehicle. For example, when the processing means detects forgotten things in the vehicle and detects a movement of closing the door, the processing means may notify the driver or the like that there are forgotten things by e-mail or the like. The driver can notice that there is something left behind after leaving the vehicle. The movement of opening and closing the door may be detected by detecting an optical flow, for example.

  The processing means may have a function of detecting other movements in the vehicle by detecting the optical flow. For example, the processing means may detect a door opening / closing movement, a window opening / closing movement, a winker operation, a steering wheel operation, a sun visor raising / lowering operation or the like as an image analysis function. This system can acquire such information without connecting to the connector (for example, OBD connector) of the in-vehicle network of the vehicle.

  When the sun visor is down, especially when the driver side is imaged with a 360-degree camera or the like attached to the windshield, the sun visor gets in the way and images in that direction cannot be obtained at all. Therefore, a warning may be issued when the processing means detects that the sun visor is getting out, even though the front of the vehicle is bright enough to determine that the sun visor is unnecessary. The processing means may determine whether or not the sun visor is descended based on the ratio of the sun visor area to the image.

  (10) The processing means has a function of recognizing an image by capturing an image of a user approaching the camera as a function of the image analysis, and an object of the user approaching the camera as a process corresponding to the analysis result It is good to set it as the system which performs the process according to the information described in.

  The user can execute processing corresponding to information written on the object by bringing the object he / she has close to the camera.

  For example, the driver's driver's license may be used as an approach to the camera. The processing means recognizes the effective date described in the driver's license, and when the period from the current time to the effective date is shorter than the predetermined period, the user is informed that the driver's license is due for renewal. It is good to inform.

  For example, a receipt at a gas station may be used as a camera close to the camera. The processing means may recognize the date, the amount of fuel supplied, and the amount of money displayed on the receipt, and record the information to form a database.

  For example, an identification card (for example, a driver's license, an employee ID card, a Juki card, or the like) may be used as a camera close to the camera. The processing means may recognize a character or the like written on the identification card and determine whether or not the person is registered in advance. When the person of the identification card is not registered, a process for prohibiting the engine start may be performed. Thereby, the person who can drive a vehicle can be limited to the person registered previously. Furthermore, the processing means may record the name of the person on the identification card. The recorded information can be used to identify the driver when managing the operation of the vehicle.

  The processing means preferably has a function of recognizing what the user has brought close to the camera outside the vehicle as a function of recognizing the image, and more preferably has a function of recognizing what the user has brought close to the camera inside the vehicle. For example, the processing means recognizes that there is something close to the camera when something that is not in the normal image occupies a part wider than a predetermined range within the angle of view of the camera compared to the normal image. Good. In particular, when the processing unit recognizes that the user occupies a part wider than a predetermined range within the angle of view of the camera, the processing unit may determine that the user has approached the camera.

  (11) The processing means has a function of performing an image analysis of an object reflected in the front image as a function of the image analysis, and drives information to be noted regarding the running of the vehicle as a process corresponding to the analysis result. It may be a system that executes a process provided to a user.

  By alerting the driver according to the situation ahead, the occurrence of an accident that may occur in the future due to the vehicle traveling forward can be suppressed.

  The image analysis function may include a function of calculating at least one of the inter-vehicle distance and the relative speed with the vehicle ahead. Based on the inter-vehicle distance or relative speed, a situation where the risk of collision is high may be detected, and the driver may be notified of the high risk of collision by issuing an alarm or the like. The inter-vehicle distance may be calculated from information such as the size of the preceding vehicle in the image, for example, the size of the license plate, the size of the tread (tire interval), and the width of the tail lamp. The relative speed may be calculated from the rate of change in the inter-vehicle distance.

  The image analysis function may include a function for detecting the color of the traffic light ahead. The processing means may notify the driver to urge the stop when the traffic light is lit red.

  The function of image analysis may include a function of recognizing a road sign ahead. The processing means may urge the driver to pay attention according to information indicated by the road sign. For example, when the road sign indicates a speed limit, the processing means may acquire the speed limit at the current location by character recognition and notify the driver of the speed limit. If the road sign indicates prohibition of entry, the processing means may issue a warning and notify the driver when entering the prohibited road. For example, when it is detected that the vehicle is going to run against the traffic rule indicated by the road sign or is running against the traffic rule, the driver may be notified of the violation of the rule. Thereby, generation | occurrence | production of traffic rule violation can be prevented or suppressed. For example, when the processing means detects a sign indicating that the currently traveling section is an automatic speed regulator installation section by recognizing a character string of a road sign, the processing means warns the driver of excessive speed. It is good to notify as follows.

  The image analysis function may include a function of detecting a traffic jam in a lane adjacent to the vehicle lane. For example, it may be determined based on the average traveling speed of the vehicles in the adjacent lanes and the inter-vehicle distance whether or not there is a traffic jam. For example, it may be determined that a traffic jam has occurred when the traveling speed of vehicles in adjacent lanes is equal to or less than a predetermined reference value, the inter-vehicle distance is equal to or less than a predetermined reference value, and a predetermined number of vehicles are lined up. . As a process corresponding to the analysis result, when detecting that a traffic jam has occurred in the lane adjacent to the host vehicle lane, a process to alert the driver to be careful of jumping out from between the vehicles in the traffic jam It is good to execute. The lane adjacent to the host vehicle lane may include an oncoming lane and an adjacent lane that travels in the same direction as the host vehicle. Particularly in places where people gather, such as shopping malls, theme parks, stations, and places where vehicles enter and exit such as gas stations, there is a high possibility that traffic in the same direction lane adjacent to the host vehicle lane will also be congested. In such a case, it is effective to detect a traffic jam in an adjacent lane that runs in the same direction as the host vehicle. Thereby, the occurrence of an accident due to jumping out can be suppressed. For example, when the traveling speed of the host vehicle is less than a predetermined speed, for example, less than 30 km / h, it is preferable not to emit a warning sound for jumping out. Thereby, generation | occurrence | production of an unnecessary warning sound can be avoided. In addition, it is preferable to alert the user by detecting a situation with a high risk of popping out from the front image. For example, when there is a parked vehicle next to the road, or when there is a parked vehicle in a parking lot in which a parking meter is installed, it is preferable to alert the pop-up.

  For example, some car navigation devices are provided with a function for urging the driver to drive based on traffic jam probe information, general road high-speed driving information, and peripheral facility information (such as residential areas and schools). By using this system, it is possible to call attention due to traffic jams even in locations where traffic jam probe information cannot be obtained.

  The function of image analysis may include a function of detecting animals (dogs, cats, raccoon dogs, wild boars, etc.) and people around the vehicle before starting. When the processing means detects an animal or a person around the vehicle, the processing means may inform the driver that there is an animal or the like around the vehicle. Thereby, the occurrence of an accident immediately after starting can be suppressed.

  The image analysis function may include a function of detecting that the vehicle is moving slowly from the surrounding scenery flow. The processing means may issue an alarm when detecting that the vehicle is moving forward slowly. For example, when the vehicle has stopped temporarily, and the vehicle gradually moves due to the creep phenomenon due to the looseness of the brake pedal, the driver is informed that the vehicle is moving and that there is a risk of a rear-end collision. be able to. For example, it may be determined that the vehicle is moving forward slowly when the vehicle speed is moving at a speed that allows the vehicle to move forward due to a creep phenomenon. The speed of the flow of the surrounding scenery is preferably detected by detecting an optical flow.

  The function of image analysis may include a function of determining whether the preceding vehicle is a large vehicle, for example, a bus, and the loading platform is an aluminum box truck or the like. The processing means may notify the driver of the presence of the traffic light when detecting that the preceding vehicle is a large vehicle and has approached the traffic light. For example, a sound such as “signal attention” may be emitted. The presence / absence of a traffic signal may be registered in advance in the processing means as POI information. For example, when a large vehicle is traveling ahead, it is difficult to see a traffic light in the traveling direction. It is preferable to look at the traffic signal on the opposite lane and the traffic signal for pedestrians to predict the presence of the traffic signal, or to take a sufficient inter-vehicle distance so that the traffic signal in the traveling direction can be easily seen. However, if such attention is neglected, there is a risk of overlooking traffic lights in the direction of travel. In the countryside, there may be no traffic signal in the opposite lane or pedestrian traffic light. By notifying the presence of a traffic light when the preceding vehicle is a large vehicle, the risk of the driver overlooking the traffic light can be reduced. Further, instead of directly notifying the presence of the traffic light, the processing means may alert the user to travel such that the risk of overlooking the traffic light is reduced. For example, a voice such as “Please take a sufficient distance between vehicles” may be emitted. For example, alerting may be performed when the distance to the intersection where the traffic signal is installed is longer than the safe stop distance of the host vehicle. The safe stop distance of the host vehicle varies depending on the host vehicle speed and the weather (whether it is raining or not). As the safe stop distance, an appropriate distance may be set based on the vehicle speed and the weather (whether it is raining or not). For example, when the safe stop distance in fine weather is 50 m, the processing means is currently in fine weather, the distance from the preceding large vehicle is less than 50 m, and outside the urban area in the car navigation device. When there is an intersection where a traffic light is installed within 100 meters in the direction of travel, it is better to call attention.

  The image analysis function may include a function of determining whether or not the road ahead is a slope and, if it is a slope, a function of detecting the degree of inclination. The processing means may urge attention such as excessive speed, low speed, and setting to the hill climb mode depending on the slope of the slope.

  The vehicle is one that handles luggage, such as forklifts, trucks, etc., and the processing means recognizes signs such as character strings and barcodes displayed on the luggage from the captured images and determines whether the vehicle should be handled by the vehicle. It is desirable to have a function for determining whether or not. Information represented by a sign attached to the luggage to be handled by the host vehicle may be registered in the processing means in advance. The processing means may execute a process for issuing an alarm when detecting that the host vehicle is about to load a baggage that is not to be handled.

  The processing means may have a function of detecting a window area in the image from the optical flow. Further, the processing means may have a function of determining from the optical flow whether the vehicle has entered the tunnel, whether the vehicle has left the tunnel, or whether the vehicle is traveling in the tunnel. In addition, the processing means may have a function of determining whether or not the vehicle is traveling under an overpass (under a vertical parallel road) from the optical flow. Further, the processing means may have a function of determining the presence / absence of a guardrail and the necessity of attention to a side groove from the optical flow.

  (12) The processing means has a function of recognizing characters written on a license plate of another vehicle reflected in an image outside the vehicle as a function of the image analysis, and character recognition as a process corresponding to the analysis result The system may execute at least one of the process of recording the information of the license plate specified by, and the process of transmitting through the network.

  It is possible to specify the information of the license plate of another vehicle that is running near the vehicle equipped with the vehicle-mounted device having the function of the processing means of this system or that is stopped nearby. The processing means may transmit the current position information of the vehicle equipped with the vehicle-mounted device and the date / time information when the license plate is imaged in association with the information of the specified license plate. The license plate information acquired together with the position information and date / time information is used to know the travel routes of many vehicles together with the information of the license plate acquired by the automobile number automatic reading device (N system) installed on the road. Useful information. In this way, by associating the vehicle number and the GPS information, a system that can be called a wide-area mobile N system can be constructed.

  It is preferable to construct a center system that collects positional information, date / time information, and license plate information in association with each other via a network from this system mounted on each of a plurality of vehicles. The center system may have a function of extracting position information and date / time information of a vehicle having specific license plate information and specifying a travel route of the vehicle.

  (13) The processing means has a function of determining whether or not an edge of the window glass exists in the window portion of the vehicle as a function of the image analysis, and a window in the window portion as a process corresponding to the analysis result. If a glass edge is detected, a system that executes notification processing may be used.

  The presence of the window glass edge in the window means that the window glass has been forgotten to be closed. The driver can notice that the window glass has been forgotten to be closed by receiving a notification from the system. The detection of the edge of the window glass may be performed, for example, by detecting image discontinuity due to a difference in refractive index between the window glass and the atmosphere. As the notification processing, for example, processing such as notification to the driver's portable terminal or the like by wireless or notification by e-mail may be performed. In addition, the notification process may be performed by sounding a buzzer attached to the vehicle. A smoke film is affixed to the window of the vehicle on which the processing means is mounted, and the processing means determines whether the window is based on the difference in image brightness between the area where the window is open and the smoke area of the window glass. It is good to detect that it is open.

  (14) The processing means has a function of determining whether or not it is raining by detecting the presence or absence of the operation of the wiper or the increase or decrease of raindrops adhering to the windshield as a function of the image analysis, As a process corresponding to the analysis result, when it is determined that it is raining, a system that executes a process of transmitting information for notifying that it is raining from the communication device may be used.

  By receiving the information transmitted from this system, it is possible to know whether or not it is raining in the area where the vehicle is traveling. The processing means may notify the external device such as a server that it is raining and notify the current position information of the vehicle. The external device can create a map of a raining region by collecting information from a plurality of vehicles. Further, the processing means may notify that it is raining and notify the current time information. As a result, the external device can acquire information related to temporal changes in the raining area.

  As an external device, for example, a server of an organization that provides information related to weather may be used. The processing means may upload an image at a timing when the wiper is not reflected, for example, a timing when the wiper is lowered, to the server. From the image uploaded to the server, it is possible to obtain information related to rainfall, such as rainfall. The server can utilize data such as rainfall collected from the vehicle as big data in various application fields. For example, it can be used for prediction of local guerrilla heavy rain. The processing means may determine that rain has stopped when raindrops are attached to the windshield, but there is no increase or decrease in the number (number) of the raindrops. If it is determined that the rain has stopped, the processing means may notify the server that the rain has stopped. The server can obtain more detailed information regarding rainfall.

  (15) The processing means has a function of determining the weather from the sky situation reflected in the image as a function of the image analysis, and a process of transmitting information on the weather from the communication device as a process corresponding to the analysis result A system that executes

  Information on weather collected from a plurality of vehicles can be used as big data on weather information. For example, the weather such as clear, clear, and cloudy may be determined based on the color of the sky reflected in the image, the presence or absence of clouds, and the like. Furthermore, it is preferable to detect the intensity of the morning sun or the western day from the height of the sun and the brightness of the sky and transmit information that the morning sun or the western day is strong. The current position information of the vehicle may be transmitted together with information on the weather. The external device that collects this information may provide information that the morning sun and the western sun are strong, and the current position information of the vehicle to drivers of other vehicles that are heading to that point. A driver who has received this information can be aware that attention needs to be paid to the morning sun or the west.

  (16) The processing means has a function of detecting sound information from a microphone mounted on the vehicle, associates a detection result of the sound information with an analysis result of the image analysis function, and responds to the result of the association A system that executes processing may be used.

  It is possible to perform more appropriate processing by associating both, as compared with the case where processing according to only the sound information detection result or the image analysis result is executed.

  The processing means has a function of detecting a change in the state of the display unit of the device having the display unit mounted in the vehicle as a function of image analysis, and the detected sound information and display as a process corresponding to the analysis result When the change in the state of the unit is synchronized, it is preferable to execute a process of storing the detected sound information in association with the device whose state of the display unit has changed.

  When detecting the sound information from the microphone, the processing means can identify the device that has emitted the sound based on the information stored in association with the sound information and the change in the state of the display unit. For example, the processing means has a voice recognition function, and determines whether or not the sound information from the microphone is issued from a specific device. If the information is issued from the specific device, the voice recognition is performed. Cancel the function. Thereby, execution of unnecessary voice recognition processing can be avoided. In addition, when the processing means determines that the sound information from the microphone is emitted from a specific device, the processing means adds information that can distinguish the region of the device that generated the sound from the other regions in the in-vehicle image. It is good to have a recording function. For example, it is preferable to record an image of a rectangle or a closed curve surrounding an area where a device that generates sound is superimposed on the original image. Alternatively, a semi-transparent geometric pattern may be overlaid and recorded in an area where the device that generated the sound is reflected.

  The processing means may have a function of determining whether an emergency vehicle such as an ambulance, a fire engine, a police car or the like is reflected in the image as a function of image analysis. When the processing means detects the siren sound of the emergency vehicle from the microphone and detects the emergency vehicle in the image, the processing means may determine that there is an emergency vehicle in the vicinity of the host vehicle by associating both. When it is determined that there is an emergency vehicle, the processing means may notify the driver of a message that calls attention.

(17) There are multiple cameras mounted in the car,
The processing means has a function of detecting the position of another camera in the image as a function of the image analysis, and executes a process of specifying a positional relationship among a plurality of cameras as a process corresponding to the analysis result. A system is good.

  It is possible to obtain an image of a wide area in the vehicle that is difficult to capture with one camera. When the positional relationship of a plurality of cameras is specified, the positional relationship of images captured by each camera can be specified.

  The processing means may have a function of specifying the model of another camera reflected in the image and specifying the angle of view from the model. The processing means may have a function of combining images captured by a plurality of cameras. By synthesizing the images, the situation inside the vehicle can be grasped intuitively as compared with the case of displaying individual images of the camera.

  For example, at least a part of the functions of the processing means may be realized by a viewer such as a personal computer. A sign such as a camera icon may be displayed in the displayed image so that the position of the camera can be seen. It is difficult to specify the position of the camera in the vehicle from the image captured by the camera (self). The position of the camera in the vehicle may be detected from the image of another camera in which the camera is reflected. For example, when images captured by three cameras are combined and displayed, the positions of the three cameras may be displayed with icons or the like in the combined image. The icons may be different depending on the camera model or the angle of view.

  The processing means may have a function of displaying an image captured by each camera and displaying a position of the camera and an angle of view of the camera (an area captured by the camera) in the image. For example, the angle of view of the camera may be displayed in the image according to the functions of the omnidirectional 360 degree camera, the hemispherical 360 degree camera, the wide angle camera, and the like. The user who has seen the image can intuitively grasp whether each camera is capturing the area.

(18) The camera is a 360 degree camera having an image circle smaller than the diagonal length of the imaging surface,
The processing means may be a system having a function of deforming a basic pattern for calculating a feature amount of an image according to a position in the image circle as a function of the image analysis.

  The feature amount of the image may be calculated using the basic pattern without correcting the distortion of the image in the image circle. Thereby, the load of the processing means can be reduced. The basic pattern is defined for an image obtained by imaging using a standard lens (a general wide-angle lens). Projection methods for obtaining an image circle include, for example, projection methods such as orthographic projection, equidistant projection, stereoscopic projection, and uniform solid angle projection, and the basic pattern may be modified according to these projection methods. In order to obtain an image circle, a circumferential fisheye lens may be used. The imaging surface of the camera is rectangular, and the diameter of the image circle is preferably shorter than the long side, and more preferably equal to the short side or shorter than the short side.

  For example, a template pattern of an image to be detected may be prepared in advance as a basic pattern. In this case, the template pattern may be deformed according to the position in the image circle. In addition, for example, when recognizing an image using a Harlike feature, each of a plurality of rectangular patterns for obtaining a Harlike feature may be deformed according to the position in the image circle.

  In the case of performing image recognition, it is preferable to limit an area where a specific object to be detected is reflected among the areas in the image circle and perform image analysis within the limited area. For example, the function of the in-vehicle device of this system may be realized by a drive recorder. In general, the recommended installation position of the drive recorder is determined in advance, so the area in the image circle where the driver is reflected, the area where the traffic lights are reflected, the area where the people in the rear seat are reflected, etc. are almost fixed. . For this reason, according to the person or thing which should be detected, it is good to limit the area | region which performs image analysis in order to detect the person or thing to the area | region in which the person or thing is reflected. For example, when analyzing an image in front of the vehicle, it is preferable to analyze a region where the windshield is reflected.

  In addition, for example, a specific region of the image circle may be converted into a planar image (parallel projection image), and the converted planar image may be matched using a normal basic pattern.

  Further, it is preferable to map the image in the image circle to a spherical surface using a predetermined conversion function, zoom to a part of the image mapped to the spherical surface, and perform image analysis as a planar image ignoring distortion. When it is desired to detect a specific object such as a road sign, it is preferable to perform image analysis by zooming to a region where the specific object is reflected. When it is desired to detect a road sign, it is preferable to perform image analysis by zooming to a region where the road sign appears in the front image.

  (19) The processing means has a function of specifying an area that should not be disclosed in the image as a privacy protection area from the viewpoint of privacy protection as the function of the image analysis. It is preferable that the system executes a process of generating an image in which a person reflected in the privacy protection area cannot be specified by appearance from an image captured by the camera.

  By making it impossible to specify a person, the privacy of that person can be protected. As an area that should not be disclosed, for example, an area in which the interior of the vehicle is reflected may be used. In particular, when an image is acquired using an omnidirectional camera, the inside of the vehicle is picked up, but there may be a case where it is not desired to leave an image in the vehicle. In such a case, for example, the area inside the vehicle is painted black, or a mosaic is applied to the area inside the vehicle, and the image outside the vehicle is recorded as it is. The processing means may distinguish an area where the inside of the vehicle is shown from an area where the outside of the vehicle is shown by performing image analysis, for example. For example, a region where the optical flow is generated at a speed corresponding to the vehicle speed may be outside the vehicle. The processing means may have a function that allows the user to set a privacy protection area in advance.

  The user can select a normal mode in which an image that can identify a person in the privacy protection area is recorded, and a privacy mode in which an image in which a person reflected in the privacy protection area cannot be identified by appearance is recorded. It is good to do so. The user can switch between the normal mode and the privacy mode in accordance with the purpose of use and usage situation of the vehicle.

  The privacy area may be an area where a person is shown. For example, the processing means may detect a person by performing image analysis, and set a region surrounded by the detected outline of the person as a privacy region. Alternatively, only the detected human face portion may be set as the privacy region. At this time, it is preferable to display the person as a silhouette. Or it is good to display only the outline of a person and to display the image when there is no person inside the outline. At this time, the person looks transparent. As an outline of a person, for example, an edge with movement may be displayed.

  The processing means may display an avatar instead of the person in the area where the person is shown. The avatar to be displayed should be similar to the original person. For example, an avatar reflecting the characteristics may be displayed according to gender, body shape, face outline, and the like. A person and an avatar may be associated in advance, and the processing means may specify a person shown in the image and display an avatar associated with the specified person. An avatar may be created in advance for each person. Move your avatar to match the movement of the person. For example, when the person raises his hand, the avatar's hand is raised, and when the person turns sideways, the avatar may turn sideways. The avatar's facial expression should be changed according to the facial expression of the person. For example, the processing means may change the expression of the avatar to a laughing expression, a sad expression, or an angry expression when detecting a facial expression such as laughing, sad, or angry. The processing means may read the emotion of the person and change the color of the avatar according to the emotion of the person. For example, when it is detected that the person is angry, the color of the avatar may be changed to a color representing angry, for example, red.

  The processing means may generate an image in which the person appearing in the privacy protection area cannot be identified from the appearance captured by the camera before recording the image, and record the generated image. In addition, when an image captured by the camera is played back, it is preferable that the person appearing in the privacy protection area is converted into an image that cannot be specified by appearance and displayed.

  (20) The processing means may be a system that executes processing for recording the result of image analysis as text data or coding as processing corresponding to the analysis result.

  By converting to text data or encoding, a desired image can be easily found from the recorded image. For example, “person detection” may be generated as text data when a person is detected in the image, and “red signal” may be generated as text data when a red signal is detected in the image. The generated text data may be recorded in association with the image from which the text data is generated. Instead of text data, the image analysis result and the code may be associated with each other and the image analysis result may be coded (binary data).

  (21) The processing means has a function of detecting a black-blurred area in the image as a function of the image analysis, and brightens a black-blurred area of the image as a process corresponding to the analysis result. A system that executes a process of restoring and displaying on the display screen is preferable.

  A user who has seen the image can easily recognize objects and people in the area where the blackening is blurred. In particular, when an image is acquired using an omnidirectional camera, a scene outside the vehicle and a dark interior are imaged simultaneously in the image, and thus the area in which the interior is reflected is likely to be blurred. The process of detecting a black-blurred area and brightly restoring and displaying it is particularly effective when an image is acquired using an omnidirectional camera. For example, gamma correction, color correction, conversion to a black-and-white image, contrast enhancement, and the like may be performed as a process for restoring brightly.

  (22) The processing means has a function of allowing a user to divide into a plurality of sections from an imaging range captured by the camera, and a function of weighting each section, and the brightness of each section and the section It is preferable that an imaging condition is determined based on the weight attached to the, and the system has a function of causing the camera to perform imaging under the determined imaging condition.

  The user can acquire an image under an imaging condition suitable for the area to be viewed by increasing the weight of the section to be viewed in the image and reducing the weight of the section that does not need to be viewed. The user can divide the imaging range into an arbitrary shape according to the shape of an area that the user wants to see or does not need to see. For this reason, for example, it becomes possible to perform weighting more appropriate for the user as compared with the case where weighting is performed uniformly on a section divided into a square lattice pattern regardless of the content of the image.

  In particular, when imaging is performed using a 360-degree camera, the difference in position within the imaging range in which the vehicle portion is captured is greater for each vehicle type than when a general camera is used. For this reason, there is a high possibility that the general-purpose weighting table set in the firmware of the processing means in advance does not match the relationship between the actually anticipated imaging range sections. The function of enabling the weighting to be determined using PC browser software or the like instead of using a general-purpose weighting table has a particularly remarkable effect when using a 360 degree camera. For example, an image captured by a camera in an unweighted state is analyzed using PC AI software or the like to estimate what portion is captured in the imaging range. For example, a weight is applied to a road portion. It is preferable to attach “5”, weight “3” to the face portion, weight “1” to the instrument panel portion, and weight “0” outside the range of the round image.

  The processing means may determine, for example, gain, exposure time, etc. as the imaging conditions. It is preferable to configure the omnidirectional camera with the back surfaces of the two hemispherical cameras facing each other and to make the imaging conditions of the two hemispherical cameras the same. As a result, when the images captured by the two hemispherical cameras are combined, the boundary between the two images becomes difficult to understand.

  The processing means may divide a part of the imaging range by a closed curve and let an inner area surrounded by the closed curve be one partition when the user classifies the image. When the user divides the inner region surrounded by the closed curve into two by the curve, each of the divided regions may be set as one section. The processing means may allow the user to input a weight with a numerical value for each set section. The numerical value to be input may be selected from a pull-down or pop-up list.

  The processing means may have a function of inputting the type of the section instead of the function of inputting the weight numerically or in addition to the function of inputting the weight numerically. A weight is associated with the partition type in advance, and when the partition type is input, the processing unit may associate the weight associated with the type with the partition. As types of sections, for example, types such as a driver, a meter panel, a car navigation screen, a windshield, a rear seat, and a passenger seat may be provided. The processing means may have a function that allows the user to change the correspondence between the partition type and the weight. The list of partition types may be registered in advance by the user.

  The processing means may have a function of setting a section by causing the user to specify a region to be viewed within the imaging range, for example, as a function for weighting each section. The weight of the section that the user wants to see may be greater than the weight of other areas. This function corresponds to an example in which a section is divided into two types, that is, a section that the user wants to see and another section in the function of allowing the user to input a section type.

  The processing means may allow the user to divide the imaging range in an interactive format (wizard format). For example, the processing means outputs a message “Please surround the unnecessary area” or “Please fill in the unnecessary area”, and the area surrounded by the user or the filled area is displayed as “Unnecessary area”. " In addition, the processing means outputs the message “please fill the driver's area” and “please fill the road area”, and the filled area is the section or road that has the driver type. It may be recognized as a section having the following type. The operation of enclosing with a closed curve or the operation of painting may be performed using a pointing device such as a touch panel or a mouse.

  The processing means may realize a function of dividing the imaging range into a plurality of sections based on the captured image by artificial intelligence (AI). For example, the inside and outside of the vehicle, the seat, the dashboard, the instrument panel, and the like may be automatically determined from the image, and the imaging range may be classified for each component that is automatically determined. The processing means may display the classified result on the display screen and let the user determine whether there is no problem in this classification. If the user determines that the classification performed by the AI is wrong, the processing means may have a function for allowing the user to correct the classification. If the user determines that the section and the weight or type corresponding to the section are incorrect, the processing means may cause the user to select the correct type from a pull-down or pop-up list.

  (23) The processing means has a function of allowing the user to divide into a plurality of sections from an imaging range captured by the camera, and a function of weighting each section. It is preferable to use a system that performs compression processing by making a block at the time of compression of a relatively small partition larger than a block at the time of compression of a relatively large partition.

  The user can increase the degree of compression of the less important section by increasing the weight of the higher importance section and reducing the weight of the lower importance section. As a result, compared to the case where the entire image is uniformly compressed, the volume of image data can be reduced while maintaining the high quality of the image of the highly important section. For example, the processing means may compress 8 × 8 pixels as one block in the weighted partition, and 16 × 16 pixels as one block in the lighter partition.

  There is a case where an image of the lens barrel portion of the lens is reflected in a region outside the round image formed on the rectangular imaging surface of the camera. Since an image outside the circle image is unnecessary, it is preferable to paint this portion black. Thereby, the capacity of the image data including the pixels outside the round image can be reduced. Further, it is preferable to obtain image data by raster scanning only the pixels inside the round image except for the pixels outside the round image. Thereby, the capacity of the image data can be reduced.

  (24) The processing means has a function of detecting the following vehicle and a function of detecting the distance to the following vehicle as the function of the image analysis, and the processing corresponding to the analysis result is based on the distance to the following vehicle. It is good to set it as the system which performs the process which detects tilting operation of a following vehicle.

  The processing means can take an appropriate measure against the tilting operation by executing a process according to the determination result. For example, even if the speed of the vehicle is equal to or higher than a predetermined reference value, when the same subsequent vehicle is continuously present within a predetermined determination distance for a predetermined determination time, the subsequent vehicle is in a driving state. It is good to judge.

  A camera having a function of measuring the distance to the object for each pixel may be used. For example, a back-illuminated time-of-flight distance image sensor may be used as the camera image sensor.

  When the processing means determines that the following vehicle is driving and is driving, for example, notification to the driver, recording of the rear image, advice to the driver, traffic control organization (for example, It is better to handle such as reporting to the police. When reporting to a traffic control agency, it is advisable to attach images of the vehicle that is driving and its driver. The processing means notifies the driver that there is a possibility of being harassed by a sudden stop when it detects that a vehicle driving in a tilted manner overtakes its own vehicle and turns in front of the own vehicle through image analysis. Good.

  The processing means may detect a person and a vehicle around the vehicle and control the vehicle according to the distance to the person and the vehicle. For example, the processing means has a high possibility of contact with a person based on the speed of the own vehicle, the distance to the person ahead, and the determination result of whether or not the person is on the planned traveling route of the own vehicle. If it determines, it is good to perform control which stops a vehicle.

  (25) A program for causing a computer to realize the function of the processing means according to any one of (1) to (24) above may be used.

  The above-described inventions shown in (1) to (24) can be arbitrarily combined. For example, a configuration in which at least a part of at least one of the inventions after (2) is added to all or part of the configuration of the invention shown in (1) may be adopted. In particular, the invention shown in (1) may be an invention in which at least a part of at least one of the inventions after (2) is added. Further, an arbitrary configuration may be extracted from the inventions shown in (1) to (24), and the extracted configurations may be combined. The applicant of the present application has the intention to acquire the right to the invention including these configurations.

  In addition to recording an image captured by a camera attached to a vehicle, the image can be used more effectively.

  The effect of the invention of the present application is not limited to this, and the effect obtained from the part of the configuration disclosed in this specification and the drawings is also disclosed, and the right for the configuration that exhibits the effect is also obtained through divisional application, amendment, etc. Have the will to do. For example, in the present specification, a portion described as “to be able to” is a description that clearly shows the effect to be achieved, and there is a portion showing an effect even if “to be able to” is not described. Even if there is no such description, there is an effect grasped by the configuration.

FIG. 1 is a diagram illustrating a windshield, a dashboard, and the like in a vehicle in a state where a drive recorder as a system according to the first embodiment is mounted on the vehicle. FIG. 2 is a block diagram of a drive recorder as a system according to the first embodiment. FIG. 3 is a block diagram of a system according to a modified example in which the function of the processing unit is realized by two devices, an in-vehicle device and a server. FIG. 4 is a block diagram of a system according to a modification in which the function of the processing unit is realized by an in-vehicle device and an external device such as a personal computer. FIG. 5 is a diagram of an imaging surface for explaining a method of dividing an imaging range into a plurality of sections by a method according to a comparative example. FIG. 6 is a diagram of an imaging surface for explaining a method of dividing the imaging range into a plurality of sections in the system according to the twenty-second embodiment.

[First embodiment]
A system according to a first embodiment of the present invention will be described with reference to FIGS. The function of this system is realized by a drive recorder attached to the vehicle.

  FIG. 1 is a view showing a windshield, a dashboard, and the like in a vehicle with the drive recorder 1 mounted on the vehicle. The vehicle on which the drive recorder 1 is mounted is a passenger car. The drive recorder 1 is attached to a position on the passenger seat side adjacent to the rear-view mirror 3 near the center in the left-right direction above the windshield 2 of the vehicle. The drive recorder 1 is fixed by being attached to the inner surface of the windshield 2 with an attachment member such as a double-sided tape. The drive recorder 1 is connected to a cigar socket 5 of the vehicle via a power cable 4. When the accessory power supply of the vehicle is turned on, power is supplied from the cigar socket 5 to the drive recorder 1. The drive recorder 1 includes a main body 10 and a 360-degree omnidirectional camera 11. The camera 11 is fixed in relative position and posture with respect to the vehicle in a normal use state.

  FIG. 2 is a block diagram of the drive recorder 1 as a system according to the first embodiment. The drive recorder 1 includes a processing unit 20, a camera 11, a GPS receiver 12, an SD card reader 13, an acceleration sensor 14, a speaker 15, a display 16, and operation buttons 17. Although the SD card reader 13 should be strictly called “SD card reader / writer”, it is simply called “SD card reader” in this specification.

  The speaker 15 and the display 16 function as output means for notifying the user of various information. The speaker 15 issues an alarm or various information by sound or voice based on control by the processing unit 20. The display 16 displays various information as images based on control by the processing unit 20. The operation buttons 17 function as input means for giving various commands to the drive recorder 1 by the user.

  The camera 11 is a 360 degree omnidirectional camera, and images the whole area of the vehicle in the front-rear direction, the up-down direction, and the left-right direction as a moving image. Therefore, the camera 11 images the outside of the vehicle through the windshield, the rear glass, and the side window, and also images the interior of the vehicle including the driver's seat, the passenger seat, the rear seat, the instrument panel, the center console, and the like. A moving image captured by the camera 11 is taken into the processing unit 20 as image data. The processing unit 20 records an image on the SD card connected to the SD card reader 13 based on the image data input from the camera 11.

  The GPS receiver 12 detects the position information of the host vehicle at the current time based on an instruction from the processing unit 20. The position information includes information such as time obtained based on a signal from a GPS satellite, the speed of the host vehicle, longitude, latitude, altitude, and the like. The processing unit 20 performs processing for recording the history of the position information.

  The SD card reader 13 reads data from the SD card inserted into the SD card insertion slot or records data on the SD card based on control by the processing unit 20.

  As the acceleration sensor 14, a three-axis type sensor that detects acceleration and inclination in directions of three axes (x axis, y axis, and z axis) is used. The acceleration sensor 14 always outputs the detection value to the processing unit 20. The processing unit 20 acquires triaxial acceleration information, for example, every 10 ms.

  The processing unit 20 includes a well-known memory such as a CPU 20a, a ROM 20b, and a RAM 20c, a timer 20d, and other peripheral circuits. Various programs are stored in the ROM 20b of the processing unit 20. The processing unit 20 implements various functions by executing these programs. The various programs include an operating system (OS), a GPS information processing program, an image analysis program, and the like.

  The processing unit 20 stores the GPS information received by the GPS receiver 12 in an SD card connected to the SD card reader 13 by executing a GPS information processing program. By executing the image processing program, the processing unit 20 constantly records the moving image captured by the camera 11 in the predetermined recording area of the SD card with the image data associated with the time information and the position information.

  The processing unit 20 performs image analysis of an image captured by the camera 11 by executing an image analysis program. Furthermore, the processing unit 20 executes various processes corresponding to the analysis results.

  By mounting the system according to the first embodiment on a vehicle, it is possible not only to record an image captured by a camera attached to the vehicle, but also to use the image more effectively. When an image captured by the camera 11 is converted into a moving image, it is possible to easily determine the movement of the subject, the presence / absence of the subject before and after a certain point in time, and the like by performing image analysis of the moving image.

Next, a modification of the first embodiment will be described.
In the first embodiment, the drive recorder 1 is attached to a passenger car. However, in a modified example, the drive recorder 1 is attached to another type of vehicle, such as a truck or special vehicle. In the first embodiment, the camera 11 images both inside and outside the vehicle. However, in a modified example, the camera 11 images at least one inside and outside the vehicle. For example, “inside the vehicle” and “outside the vehicle” may be interpreted as follows.

  When the interior space and the exterior space are clearly separated by the body of the vehicle, the interior of the body may be referred to as “inside the vehicle” and the exterior of the body may be referred to as “outside the vehicle”. For example, in the case of a van in which an occupant space and a load space are continuous, the continuous occupant space and the load space may be referred to as the interior of the vehicle. In the case of a sedan in which the passenger space and the cargo space are partitioned, both the passenger space and the cargo space may be referred to as the interior of the vehicle. In a covered lorry with a closed cargo bed, the passenger space and the inside of the sealed cargo bed may be referred to as the interior of the vehicle. In open trucks (flat-body vehicles) with an open cargo bed, it is difficult to clearly distinguish the space on the cargo bed that accommodates the load from the surrounding space. The space to be accommodated may be called the inside of the vehicle. In addition, it is good to call the passenger space of an open cargo truck the inside of a car. In some special vehicles such as forklifts, both the passenger space and the load space are open, and it may be difficult to clearly distinguish the passenger space and the load space from the surrounding space. Even in such a case, both the passenger space and the cargo space may be called the inside of the vehicle. For example, a space above a seat on which an occupant sits and a space above a fork that lifts and lowers a load may be referred to as a vehicle interior. In this way, a space that can be considered to move with the vehicle as the vehicle moves is referred to as the inside of the vehicle, and a space around the space is referred to as the outside of the vehicle.

  In the first embodiment, a 360-degree omnidirectional camera is used as the camera 11, but a 360-degree hemispherical camera may be used, or a normal wide-angle camera may be used. When a wide-angle camera is used, a plurality of cameras may be attached to the vehicle so that a wide area can be imaged.

  In the first embodiment, the function of recording an image captured by the camera 11 is realized by the in-vehicle device, but may be realized by a server that communicates with the in-vehicle device. In the first embodiment, the function of analyzing the image captured by the camera is realized by the in-vehicle device, but may be realized by a server that communicates with the in-vehicle device.

  As described above, the processing unit 20 of the present system may be configured by a single in-vehicle device or may be configured by a plurality of devices. As in the first embodiment, the function of the processing unit 20 may be realized by a microcomputer built in the drive recorder 1 and an application program stored in the memory. In addition, for example, the function of the processing unit 20 may be realized by two devices, an in-vehicle device and a server, and the in-vehicle device may record an image and the server may perform image analysis. The function of the processing unit 20 may be realized by an in-vehicle device and an external device such as a personal computer, the image recording is performed by the in-vehicle device, and the analysis and display of the image is performed by the external device.

  FIG. 3 is a block diagram of a system according to a modified example in which the function of the processing unit is realized by two devices, an in-vehicle device and a server. The in-vehicle drive recorder 1 includes a communication device 21. The server 30 includes a processing unit 31, a communication device 32, an input device 33, and a display 34. The communication device 21 of the drive recorder 1 and the communication device 32 of the server 30 are both connected to the communication network 40 and transmit / receive data to / from other devices via the communication network 40. The processing unit 20 of the drive recorder 1 and the processing unit 31 of the server 30 perform data communication with each other via the communication network 40.

  The input device 33 inputs various commands and data to the processing unit 31 when operated by the user. The processing unit 31 causes the display 34 to display processing results and various messages to the user.

  FIG. 4 is a block diagram of a system according to a modification in which the function of the processing unit is realized by an in-vehicle device and an external device 50 such as a personal computer. The external device 50 includes a processing unit 51, an SD card reader 52, an input device 53, and a display 54. The processing unit 51 of the external device 50 is recorded on the SD card 60 by attaching the SD card 60 on which the image data is recorded by the SD card reader 13 of the drive recorder 1 to the SD card reader 52 of the external device 50. The image data can be accessed.

  The processing unit 51 reads image data recorded on the SD card 60 based on a command input from the input device 53 and performs image analysis or image processing. The processing unit 51 displays the analysis result or the processing result on the display 54.

  Next, a second embodiment to a twenty-fourth embodiment will be described. Hereinafter, redundant description of the configuration common to the system according to the first embodiment will be omitted. In addition, the system according to various embodiments described below is realized as one example in any of the block diagrams shown in FIGS. 2, 3, and 4.

[Second Embodiment]
Next, a second embodiment will be described. In the second embodiment, the processing unit of the system has a function of determining whether or not a person's movement in the image corresponds to a specific movement registered in advance as an image analysis function. When the movement of the person corresponds to the specific movement, the processing unit records an image obtained by capturing the specific movement of the person, and an image server 30 (for capturing the specific movement) as processing corresponding to the analysis result. At least one process of transmission to FIG. 3), notification that a person has made a specific movement, and issuing an alarm is executed.

  The recorded image or the image transmitted to the server can be used as evidence that the person has made a specific movement. By notifying that a person has made a specific movement, it is possible to notify an external person that there is a person making a specific movement. A person who has made a specific movement by the issued alarm is motivated to stop the specific movement by noticing that the alarm has been issued to him.

Next, an example of “a person who performs a specific movement” will be described.
As a person who performs a specific movement, for example, a vehicle driver is used. As the specific movement, for example, an operation other than the driving of the driver is performed. For example, an operation of looking at a portable terminal such as a smartphone, an operation of eating food, an operation of looking away or aside, a doze, and the like. For example, the presence or absence of a sidewalk operation is determined based on the detection result of the driver's eye position. The recorded image can be used for improving driver awareness, analyzing the cause of an accident, and the like. When the processing unit detects that the driver is operating a mobile terminal such as a smartphone, the processing unit issues an alarm that calls attention so as not to hold the mobile terminal while driving. If the driver notices an alarm and stops a specific movement, the danger can be avoided in advance. The data transmission destination server 30 is, for example, a vehicle manager's server or a police server. The manager of the vehicle can evaluate the driving attitude of the driver based on the transmitted data. The police can evaluate the driving attitude of the driver, analyze the cause of the accident, and the like based on the transmitted image.

  As a person who performs a specific movement, for example, a suspicious person outside the vehicle is assumed. The specific movement is, for example, an action of trying to illegally enter the car, for example, an action of trying to break a glass window with a hammer, an action of looking into the car, or the like. By issuing an alarm, a suspicious person trying to illegally enter can be prevented from entering the vehicle. The recorded image or the image transmitted to the server 30 can be used as useful information for arresting a thief after an on-vehicle crime occurs. By notifying the owner or driver of the vehicle that the person has made a specific movement, the owner or the driver can immediately return to the vehicle and prevent damage from aiming on the vehicle. The driver or owner who has received the notification can also report to the police. Thus, this function can be used for crime prevention.

  As a person who makes a specific movement, for example, a suspicious person in a car is assumed. Specific movements may include various tricks on the vehicle, such as illegal entry into a parked vehicle, movement of a suspicious person in a parked vehicle, and removal of a device in the vehicle. If a thief rushes away by issuing a warning, the effect of reducing the damage on the vehicle is obtained. The recorded image or the image transmitted to the server 30 can be used as useful information for arresting a thief after an on-vehicle crime occurs. By notifying the vehicle owner or driver that a person has made a specific movement, the owner or driver can immediately return to the vehicle or report to the police to reduce damage to the vehicle. It becomes possible to make it. As an example of notification to the outside, transmission of an e-mail attached with an image showing a suspicious person can be cited. The transmission destination of the e-mail is, for example, a vehicle manager, a driver, an owner, a police, or the like.

  As a person who performs a specific movement, for example, it is assumed that a passenger of a business vehicle on which an unspecified person gets on. Specific movements include various actions of robbery, such as an action of flickering a blade, an action of holding a handgun, and an action of violence. By executing a process in which the processing unit issues an alarm, it is possible to prevent damage if a burglar stops thinking of a crime. The recorded image or the image transmitted to the server can be used as useful information for arresting the robber. By notifying the vehicle administrator that a person has made a specific movement, the administrator can notice the occurrence of a burglary and notify the police. In addition, the processing unit has a function of notifying the police of the occurrence of a robbery case using a communication function.

  As a person who performs a specific movement, for example, an infant in a car carrying the passenger is assumed. The specific movement is, for example, a movement when waking up from sleep. When detecting the awakening of the infant by image analysis, the processing unit notifies the guardian by e-mail or the like. There are cases where an infant sleeps in the car while driving and wants to go shopping or do some work with the engine running. At this time, if the infant is forced to wake up, the mood will be worse, and there is a demand to leave it as it is. However, parents care about whether the infant is awake when they are away from the vehicle. When this system is used, an e-mail is sent to the guardian's mobile terminal when the infant wakes up, so the parent can immediately rush to the vehicle to deal with the infant's awakening.

  As a person who performs a specific movement, for example, a care recipient outside the vehicle and a rescuer are required. As the specific movement, for example, an action specific to a care recipient and an action corresponding to a situation where rescue is deemed necessary are assumed. As an example of the action of a person who needs assistance, for example, an action of walking with a cane, bending a waist and walking well. As an example of an operation corresponding to a situation where rescue is deemed necessary, an operation corresponding to a situation where the user falls and blows bubbles from the mouth and falls.

  The processing unit has a function of detecting at least one of a front image of the vehicle, a rear image of the vehicle, and an in-vehicle image as a function of detecting a person making a specific movement, and the corresponding image from the selected image. It has a function to detect a person. Further, the processing unit has a function of causing the user to designate at least one image. As a result, the processing unit does not analyze unnecessary or less important images, so that it is not necessary to give the processing unit excessive processing capability.

  The processing unit has a function of detecting a driver's sleepiness as a function of detecting a person making a specific movement. For example, the processing unit measures the pulse from the degree of change in the redness of the driver's skin, and determines the presence or absence of the driver's sleepiness based on the change in the pulse. Further, the processing unit has a function of improving the drowsiness detection accuracy by linking the measurement of the body temperature by the thermo camera, the measurement of the pulse by the microwave, and the like. When the processing unit of this system detects drowsiness, a process of lowering the set temperature of the air conditioner or a process of issuing a warning by voice or sound is executed.

[Third embodiment]
Next, a third embodiment will be described. The processing unit of the system according to the third embodiment has a function of determining whether or not a person in an image has a specific feature registered in advance as a function of image analysis. The process according to the specific characteristic which the person in has has is performed.

  The system according to the third embodiment can perform an appropriate process according to the characteristics of the person reflected in the image.

  The processing unit has a face authentication function as a function of determining whether or not a person in the image has a specific feature. The processing unit performs face authentication of a person seated in the driver's seat, and if the person seated in the driver's seat does not correspond to a person registered in advance, the processing unit accesses the vehicle so that the engine cannot be started. For example, the processing unit transmits a signal prohibiting engine start to an engine control unit (ECU) via an in-vehicle network, for example, CAN. Thereby, a person who can drive can be registered in advance, and driving of the vehicle by an unregistered person can be prohibited. As a result, the crime prevention function of the vehicle can be enhanced. It is also effective as a countermeasure against child mischief.

  According to the modification of the third embodiment, the processing unit prevents the accelerator from being stepped on, prevents the side brake (parking brake) from being released, prevents the shift lever from entering the drive position, and sounds an alarm. It has a function of performing at least one of processes such as sending an e-mail together with an image to a registered mail address.

  According to another modification of the third embodiment, the processing unit performs face authentication of a person outside the vehicle and controls the vehicle to release the door lock when the person outside the vehicle corresponds to a person registered in advance. have. For example, when the vehicle is provided for a car sharing service, registered members of the car sharing service are registered in advance in the processing unit. The registered member can use the vehicle even if he / she does not have a card key or the like by moving his / her face within the angle of view of the camera of the vehicle by approaching the vehicle. Thereby, the usability of the vehicle of the car sharing service is improved.

  According to still another modification of the third embodiment, the vehicle is a business vehicle on which an unspecified person gets in, and the processing unit determines whether or not the passenger is a criminal being arranged by the face authentication function. Has function. The photograph of the criminal's face being arranged is transmitted from the police database to the processing unit of this system by communication via LTE, wireless LAN, or the like. The processing unit has a function of registering the received face photo. When the processing unit determines that the passenger is a criminal who is arranging, the processing unit 21 notifies the vehicle administrator, the police, etc., the communication device 21 (FIG. 3) and the communication network (FIG. 3). Do through. Furthermore, the processing unit has a function of notifying the driver in a way that is not understood by the passengers. The vehicle manager, police, driver, etc. can recognize that the criminal being arranged has boarded the vehicle and can take appropriate action promptly.

  For example, a taxi is in a closed room where only crew members and passengers get in, so there is a great demand for this system in terms of ensuring the safety of crew members. When the processing unit detects that the criminal gets on the taxi, it notifies the taxi company and the police of the criminal getting on. Further, according to the present system, it is possible to detect the criminal's boarding when the criminal gets into the vehicle before the criminal actually commits the criminal act in the vehicle. This makes it possible to avoid crimes.

  According to still another modification of the third embodiment, the processing unit determines whether or not a person in the image has a specific feature by using a predetermined registered appearance of the person in the image as a function. Compared with the appearance, it has a function (hereinafter referred to as an appearance authentication function) for determining whether or not they match. As specific appearances, appearances specific to deaf persons, appearances specific to criminals, and the like are registered in advance in the processing unit. Examples of appearances specific to criminals include, for example, having a knife on the road, having a dangerous object, and the like. Further, the processing unit has a function of detecting a person possessing an illegal drug by combining an image of an infrared camera, an ultraviolet camera, or the like with an image of a normal camera. When the processing unit detects a person having a specific appearance in the image, the processing unit notifies the police or vehicle manager via the communication device 21 (FIG. 3) and the communication network 40 (FIG. 3).

  According to still another modification of the third embodiment, the processing unit has a function of individually registering the appearance at the time when the missing person such as a specific missing person or a deceased person becomes missing as search information. For example, it has a function of registering individual search information such as sex and gender, height, type of clothes, color of clothes, wearing a hat, whether or not a cane is held, and whether or not a hip is bent. Furthermore, the processing unit has a function of registering a region (search target region) that seems to be hesitant. When the vehicle is located in the search target area, the processing unit executes a process of detecting a person that matches the individual search information. Such individual search information is received by the processing unit of the system from a database (server) such as the police or a local support center. In this system, the processing unit of the in-vehicle device performs a function of determining whether or not a person in the image has a specific feature. When the vehicle enters the search area, the processing unit of the vehicle is transmitted from the server 30 (FIG. 3). Download search information to.

  According to still another modification of the third embodiment, the processing unit determines whether or not the person in the image has a specific feature, so that the person sitting in the passenger seat or the rear seat can use the seat belt. It has a function for determining whether or not it is worn (hereinafter referred to as an appearance authentication function). When the processing unit of this system detects a person who is not wearing a seat belt, it issues a message prompting the user to wear the seat belt.

  Further, the processing unit has a function of issuing a message prompting the user to wear the seat belt when the person in the back seat approaches the entrance to the expressway without wearing the seat belt. When the distance from the current position of the vehicle to the entrance of the expressway is equal to or less than the reference distance, the processing unit determines that the entrance of the expressway has been approached. As another example, when the processing unit receives a signal for checking forgetting to insert an ETC card, the processing unit determines that it has approached the entrance of the expressway. As yet another example, the processing unit analyzes an image in front of the vehicle and detects an entrance of an expressway in the image, and determines that the entrance of the expressway is approached.

[Fourth embodiment]
Next, a fourth embodiment will be described. The processing unit of the system according to the fourth embodiment has a function of counting the number of people in the image as a function of image analysis, and executes a process of recording or transmitting the number of people data in the image as a process corresponding to the analysis result.

  The system according to the fourth embodiment makes it possible to count the number of people without using a special device for counting the number of people.

  According to the modification of the fourth embodiment, the camera is attached to the bus, and the processing unit adopts an image in the vehicle as an image from which the number of people is counted. A processing part displays a count result on the display 16 (FIG. 2), and notifies a crew member. The crew can easily grasp the number of passengers (passengers) on the vehicle. This function can be used in place of a tour call. When all passengers should get off, when the result of counting the number of passengers does not become zero, the crew can know that there are people left in the car. For example, it is possible to detect the remnants of a person who cannot express his / her intention such as a baby or a person who cannot get off himself / herself such as a disabled person.

  According to another modification of the fourth embodiment, the vehicle to which the camera is attached is a vehicle of a type in which the number of passengers is determined in advance, such as a sightseeing bus, a long-distance high-speed bus, etc. It has a function to register a fixed number of passengers. The processing unit compares the result of counting the number of persons with the registered number of persons, and if there is an excess or deficiency, warns the crew about the excess or deficiency. This warning is performed by generating a warning sound of the speaker 15 (FIG. 2) or displaying a warning message on the display 16 (FIG. 2). Thereby, it is prevented that it departs with the number of passengers shortage. In addition, it is possible to easily grasp that a person who is not qualified to get on the vehicle has boarded, and to prompt the user to get off.

  According to still another modification of the fourth embodiment, a camera (for example, a hemisphere camera on the ceiling) is installed in the bus of the route bus, the processing unit performs image analysis, and the number of passengers, the number of standing passengers, It has a function to count the number of passengers sitting. In addition, the processing unit acquires the current location information of the bus, the signal for opening / closing the bus door, the buzzer for getting off the bus, the touch of the IC card when getting on and off the bus, and the signal of the toll box, And a function of recording an image associated with an event when it occurs. The server 30 (FIG. 3) accumulates such information in the data lake. Using this data lake, it is possible to analyze big data on where the passengers got off, where they were, and how they sat in the car. Such data can be valuable information for consulting firms that perform city planning and the like. Further, the processing unit has a function of acquiring information transmitted through an in-vehicle network such as CAN and vital information of the driver together. Such information can be used to specify a causal relationship such as whether the abnormality is a driving factor, a passenger factor, a driver's own factor, or a vehicle factor when some abnormality occurs in the vehicle.

  According to still another modification of the fourth embodiment, the processing unit employs an image outside the vehicle as an image from which the number of people is counted. The processing unit of this system has a function of recording the counting result as the number data. Based on the recorded number data, it is possible to easily extract places where there are many people. The processing unit has a function of recording the image that is the basis of counting using the counting result as a trigger. For example, it has a function of recording an image when the number of persons is equal to or greater than a predetermined threshold.

[Fifth embodiment]
Next, a fifth embodiment will be described. The processing unit of the system according to the fifth embodiment has a function of detecting the display content of the display device of the vehicle or the operation status of the operation device of the vehicle from the image in the vehicle as a function of image analysis, and as a process corresponding to the analysis result Then, processing corresponding to the display content of the display device or the operation status of the operation device is executed.

  These data can be acquired by this system without retrofitting a device that acquires data that is the basis of the display contents of the display device or a device that acquires data according to the operation status of the operating device. For example, these data can be acquired without connecting a retrofit device to the OBD connector. In particular, even when the vehicle manufacturer prohibits connecting a third-party product to the OBD connector, these data can be acquired by the present system. Examples of the vehicle display device include various instruments, a warning light (warning lamp) in the vehicle, a head-up display that projects an image on a windshield, and the like.

  In a modification of the fifth embodiment, the processing unit of the system has a function of detecting the vehicle speed displayed on the speedometer as the display content of the display device. The processing unit has a function of recognizing the position of the needle of the analog speedometer and acquiring vehicle speed data by recognizing the numbers on the dial. Further, the processing unit has a function of acquiring vehicle speed data by recognizing characters displayed on the digital speedometer. The acquired vehicle speed data can be used for various processes. Hereinafter, an example of the usage form of the vehicle speed data will be described.

  The processing unit has a function of automatically detecting the speedometer by image recognition. With this function, the processing unit can detect the speedometer in various vehicles without depending on the vehicle type. In addition, the processing unit has a function of registering in advance the position and shape in the image of the speedometer of the vehicle on which the processing unit is mounted. Thereby, the processing unit can easily detect the speedometer.

  A general stand-alone car navigation device uses position information by GPS in order to acquire a current position. When a GPS signal cannot be received such as in a tunnel, the current position of the vehicle must be determined by another method. As an example, the current position of the vehicle in the map is moved for 5 minutes at a constant speed of about 80% of the speed immediately before the GPS signal lost. A mode for performing such processing is referred to as a tunnel mode. The tunnel mode is effective not only when entering a tunnel but also when traveling under an elevated road. However, if a traffic jam occurs when processing in the tunnel mode, the method of assuming that the vehicle is traveling at a constant speed results in an excessively large error in the current position, which is substantially useless. If the acceleration sensor is attached to the vehicle and the acceleration data from the acceleration sensor is used, the accuracy of the current position information in the tunnel mode can be increased. However, it is difficult for the user to attach the acceleration sensor, which is expensive and expensive.

  This system has a stand-alone car navigation function. The processing unit has a function of reading the own vehicle speed from the display content of the speedometer and updating the current position information of the own vehicle using the map matching function based on the read speed information. Thereby, the accuracy of the current position information of the host vehicle can be increased.

  The processing section will determine which of the tunnel mode is activated at the place where the road in the tunnel and the ground road are parallel, and where the elevated road and the ground road under the overhead are parallel. Has a function to select whether to match the road. For example, when the tunnel mode is activated, the vehicle position is mapped to the road in the tunnel or the ground road under the overhead, and when the tunnel mode is not activated, the vehicle position is aligned with the road within the tunnel. Has the function of mapping to a running ground road or elevated road.

  In another modification of the fifth embodiment, the processing unit has a function of detecting the remaining amount of fuel displayed on the fuel gauge as the display content of the display device to be detected. The processing unit detects the remaining amount of fuel by performing image recognition of the fuel gauge. When detecting that the remaining amount of fuel has fallen below a predetermined reference value, the processing unit notifies the driver of a message prompting refueling. For example, if the remaining fuel level is less than ¼ of the reference value, for example, when the tank is full when returning from home or near the vehicle storage location of the company in conjunction with the current position information of the vehicle The processing unit issues a warning and prompts refueling. In addition, the car navigation device and this system are linked together, and the processing unit compares the distance to the destination and the remaining amount of fuel, and based on the comparison result, refuels during departure or on the way to the destination. Prompt. For example, a message prompting refueling is output at a gas station near the current location near a gas station on the way.

  The function of extracting the speedometer and fuel gauge from the image is realized by artificial intelligence (AI) installed in the processing unit of this system. The processing unit further has a function of registering the position of the speedometer and fuel gauge in the image and image information in advance. When it is difficult to extract a speedometer or a fuel gauge from an image by artificial intelligence, the processing unit extracts the speedometer and the fuel gauge from the image using registered image information.

  When the processing unit reads the fuel meter of the vehicle provided for the car sharing service and detects that the remaining amount is less than a predetermined reference value, it can receive the benefit of the car sharing service when refueling. Has a function to notify the driver. For example, a message such as “If you refuel now, you can receive a discount of an amount equivalent to 15 minutes” is issued.

  In still another modified example of the fifth embodiment, the processing unit has a function of determining whether or not a warning lamp in the instrument panel is turned on. The warning light that is the target for determining whether or not it is lit includes a headlight lighting indicator, a blinker indicator, a high beam indicator (high beam warning light), a perking brake indicator, a half-door warning light, a fuel warning light, and the like. The processing means of this system can detect the state of the vehicle corresponding to the warning light by determining whether or not these warning lights are lit.

  In still another modification of the fifth embodiment, the processing unit has a function of detecting warning information displayed by the head-up display. The processing unit detects a warning content by detecting a change in the image displayed by the head-up display, and executes a predetermined process according to the warning content. For example, a warning is issued by sound or voice. By issuing a warning based on visual information instead of auditory information, the driver can easily recognize the warning.

  In yet another modification of the fifth embodiment, the processing unit is connected to the brightness information outside the vehicle and the current position information of the vehicle. When it is detected that the lighting indicator is not lit, it has a function of notifying the driver to turn on the headlight. The situation where the headlight should be turned on includes, for example, a situation where the outside of the vehicle is darker than a certain brightness, a situation where the current position of the vehicle is in a tunnel or a culvert. The processing unit has a function of notifying the driver to turn off the headlight when the lighting indicator of the headlight remains on despite the situation where the headlight should be turned on has been eliminated. Because there are tunnels that are bright inside due to lighting, it is determined whether or not the headlights should be turned on only from the brightness information outside the vehicle. Otherwise, a situation in which it is erroneously determined may occur. The processing unit can avoid such an erroneous determination by determining whether or not the vehicle is in the tunnel based on the current position information of the vehicle. Further, when the processing unit detects that not only the headlight but also the vehicle width light (small light) should be turned on, the vehicle width light is not turned on. It has a function to notify the driver to light up.

  In yet another modification of the fifth embodiment, the processing unit determines the presence or absence of an oncoming vehicle from an image in front of the vehicle, and detects that the high beam indicator is lit even though there is an oncoming vehicle. It has a function to warn the driver to change to. Since the beam high / low switching operation differs depending on the vehicle type, the driver is likely to experience an event that the intention to change to the low beam is still the high beam when driving a vehicle of a vehicle type different from the regular vehicle type. In such a case, the function of issuing this warning is particularly effective.

  In yet another modification of the fifth embodiment, the processing unit detects that the blinker is blinking from an image outside the vehicle when the outside of the vehicle is dark, and blinks in the vehicle in synchronization with the blinking cycle of the blinker. It has a function of recognizing the indicator as a blinker indicator. By doing in this way, the process part can find out the turn signal indicator in an instrument panel easily.

  In yet another modification of the fifth embodiment, when the processing unit detects that no one is in the vehicle from the image in the vehicle, and detects that the headlight or the width lamp is lit, It has a function of notifying the driver outside the vehicle and the vehicle manager that they have forgotten to turn off the lights. This notification is performed, for example, by sending an e-mail to a pre-registered mail address. In addition, the processing unit has a function of notifying the user of forgetting to turn off the hazard lamp and forgetting to turn off the room light by e-mail or the like. The lighting state of the hazard lamp is detected from the blinking state of the blinker indicator. The lighting state of the room light is detected from the brightness of the room light reflected in the image.

  There are generally three types of room light settings: a constantly lit mode, a normally unlit mode, and a door open / closed state interlocking mode (hereinafter simply referred to as interlocking mode). In many vehicles, these three modes are cyclically updated each time the room light on / off switch is pressed once. Some vehicles have the function of turning on the room light when the engine is stopped in the dark (at night) and automatically turning off after a while (for example, after a few seconds). You may want to turn on the room light for a longer period of time to prepare for getting off after the engine has stopped. For example, after the engine is stopped, the room light should be turned on during the period when the purchased items are sorted and taken out, the portable terminal is taken out, and the like. In such a case, the driver switches the setting of the room light to the interlock mode or the always-on mode so that the room light does not automatically turn off when a certain period of time has elapsed since the engine stopped. However, when the door is open, it is difficult to determine whether the current room light setting is the constant lighting mode or the interlocking mode. Even if the current setting of the room light is in the always-on mode, if the user gets off the vehicle without thinking that it is in the interlock mode, the room light remains on. By using this system, the driver or the administrator notices that the room light has been forgotten to turn off, and turning off the room light can prevent the vehicle battery from running out due to the room light being turned on for a long time. Just looking at the operation switch, it does not know which mode the current setting of the room light is in, for example, a vehicle with an operation switch with a structure that the setting of the room light changes cyclically each time the push button is pressed This feature of the system is particularly useful.

  When the room light is set to the interlock mode, the room light remains lit if any door is open or half-door. A driver or manager who is notified that the room light is lit can recognize that the door may be in a half-door state or the door on the side that is difficult to see may be open.

  When the setting of the room light is in the interlocking mode, there is a vehicle having a function of keeping the room light on until a certain time (light-out delay time) elapses after the door is closed. In such a vehicle, the processing unit of the present system has a function of not determining whether or not to forget to turn off the light until the turn-off delay time elapses after the door is closed. The processing unit has a function of setting a turn-off delay time based on an input from the user.

  In addition, the processing unit transmits a signal to turn off the room light to the controller of the vehicle when detecting that the room light remains lit even after a certain time has passed since no one is in the vehicle. have.

  In yet another modification of the fifth embodiment, when the processing unit detects that the vehicle door is opened while the headlight, the vehicle width light, the hazard light, the room light, etc. are lit or blinking, It has a function to issue an alarm notifying that the light remains on. This makes it possible for the driver to notice that he / she has forgotten to turn off before going out of the vehicle and to prevent the driver from forgetting to turn off.

  In yet another modification of the fifth embodiment, when the processing unit detects that the parking brake is applied based on the lighting of the parking brake indicator, the driver falls asleep from the driver image. It has a function that does not issue an alarm even if it is detected. In addition, when the parking brake is applied, an alarm is issued even if the driver sees a movement that is considered dangerous if driving, such as looking at a mobile device. Has no function.

In yet another modification of the fifth embodiment, when the processing unit detects that a warning lamp that informs that inspection or repair needs to be performed at an early stage remains on for a certain period of time, It has a function to notify the dealer and registered repairers of the vehicle status. For example, a user who is not interested in an automobile often leaves the vehicle without being aware of the lighting of the warning light when the user can travel even if the warning light is on. For example, when the warning light of the engine itself is lit, it is normally impossible to run, but the vehicle can run as it is even if the warning light notifying the abnormality of the engine oil is lit. Further, when the O ₂ sensor is abnormal, the fuel consumption is deteriorated, but the traveling itself is possible. When traveling is possible, if the vehicle is continued without inspection and repair, a minor failure may become serious. For a dealer or a repairer, notification of lighting of a warning lamp is useful information for conducting an inspection / repair business for the owner of the vehicle whose warning lamp is lit. Further, the owner of the vehicle can prevent the failure from becoming serious by performing inspection and maintenance at an early stage.

  In still another modification of the fifth embodiment, the processing unit detects the temperature displayed in the temperature display area in the instrument panel that displays the temperature measurement value measured by the temperature sensor that measures the outside temperature. have. The processing unit has a function of reading the temperature by character recognition from the image displayed in the temperature display area of the instrument panel as a function of image analysis. The processing unit executes a process of transmitting temperature information to an external server via the communication device and the communication network as a process corresponding to the analysis result.

  For example, in order to acquire data related to pollen scattering, a service that installs a sensor in a general user's house, collects measurement data from the sensor, predicts detailed pollen scattering status, and distributes pollen scattering information Provided by Weathernews. In addition, outdoor photos posted by general users and images collected from outdoor live cameras installed in homes of ordinary households are incorporated into the weather forecast. In addition, a service for distributing these images (for example, Weather News SOLiVE 24) is provided. A system that provides these weather information services can use many vehicles as temperature measurement probes by collecting outside air temperature information measured by many vehicles. If many vehicles that transmit temperature information are distributed over a wide range, temperature information collected from many vehicles can be used as spot information. For example, it becomes possible to issue a spot-like heat stroke alarm that cannot be expressed by general weather information.

  For a company that provides such a weather information service, it is possible to collect temperature information over a wide range without installing a temperature sensor in-house. The company that has received the provision of temperature information may provide free or discounted provision of a paid weather information service provided by the company in return for the owner of the vehicle that provided the temperature information.

  The processing unit further recognizes not only the temperature information read from the temperature display area in the instrument panel but also the numbers displayed on the roadside temperature display signs that are transferred to the image acquired by the camera. Thus, it has a function of acquiring outside air temperature information. The temperature indication sign is detected by recognizing characters such as “current temperature” in the sign. The acquired outside air temperature information is transmitted from the processing unit to an external server in the same manner as the temperature information acquired from the temperature display area in the instrument panel.

  In still another modification of the fifth embodiment, the processing unit has a function of detecting the position of the driver's foot with respect to the accelerator pedal and the brake pedal as a function of detecting the operation state of the operating device by image analysis. As a process according to the operation status of the operating device, the processing unit notifies the driver so that the driver can easily recognize whether the driver's foot is at the accelerator pedal position or the brake pedal position. Execute the process. For example, the driver is notified by changing the color of light emitted from the light emitting element when the foot is at the accelerator pedal position and when the foot is at the brake pedal position.

  In still another modified example of the fifth embodiment, a vehicle having a function of detecting whether the accelerator pedal is applied or whether the brake pedal is applied and displaying the detection result as visual information on a display is provided. This system is installed. The processing unit of this system has a function of analyzing the image displayed on the display and detecting the position of the driver's foot with respect to the accelerator pedal and the brake pedal.

  In yet another modification of the fifth embodiment, the processing unit issues an alarm with a sound or the like when the foot is at the position of the accelerator pedal even though the obstacle is detected forward by image analysis. Has function. When the driver notices the alarm, it is possible to suppress the occurrence of a mistake in stepping between the accelerator pedal and the brake pedal.

  In still another modification of the fifth embodiment, the processing unit has a function of detecting the rotation angle of the steering wheel (handle) as a function of detecting the operation state of the operating device by image analysis. The processing unit executes a process for obtaining a traveling direction and a travel distance of the vehicle based on the rotation angle of the steering wheel and the vehicle speed as a process according to the operation state of the operating device. The processing unit gives the processing result to the stand-alone type car navigation device. The stand-alone type car navigation device can calculate the travel route of the vehicle even in a situation where the GPS signal cannot be received based on the information on the traveling direction and the travel distance given from the system.

  In still another modified example of the fifth embodiment, the processing unit determines whether the vehicle is in automatic operation (including automatic operation levels 1 to 5) or manual operation (automatic operation level 0) as a function of performing image analysis. It has a function to detect. This detection is performed by the processing unit recognizing from the image the display state of an indicator indicating whether or not various functions of automatic driving displayed on the instrument panel are operating. When detecting that the automatic driving function is released as processing corresponding to the analysis result, the processing unit executes processing for notifying the driver that the automatic driving function is released. In addition, if the driver detects that the driver has not touched the steering wheel even though the automatic driving function has been released, the processing unit issues a warning to alert the driver to operate the steering wheel. To do. Furthermore, with the automatic driving function being released and the fact that the driver has not touched the steering wheel as a trigger, the processing unit records the image captured by the camera as an event record separately from the constant record. Has function. The recorded image can be used as evidence that the driver's operation was inappropriate.

  The processing unit has a function of invalidating the function of the radar detector when the automatic driving function is valid in a vehicle equipped with the radar detector. As a result, it is possible to eliminate generation of unnecessary alarms from the radar detector. When the automatic driving state is invalid, the processing unit validates the function of the radar detector.

  Whether the vehicle is in automatic driving or manual driving, and the level (mode) of automatic driving, the processing unit determines the movement of the indicator and steering wheel, the position of the driver's hand, Judgment is based on the direction of the head.

[Sixth embodiment]
Next, a sixth embodiment will be described. The processing unit of the system according to the sixth embodiment has a function of recognizing characters displayed on a sticker attached to a vehicle from an in-vehicle image as a function of image analysis, and recognizes as a process corresponding to the analysis result. A process for calling attention is executed according to the information indicated by the characters.

  By being alerted, the driver, manager or owner of the vehicle can pay attention to the character information on the sticker attached to the vehicle and can perform an appropriate process according to the character information.

  In a modification of the sixth embodiment, the processing unit has a function of recognizing characters displayed on an inspection mark (vehicle inspection sticker). The date on which the validity period of the vehicle inspection expires is displayed on the inner surface of the inspection mark. The processing unit reads information on the date when the valid period of the vehicle inspection expires by character recognition, and if the period from the current time to the expiry date of the effective vehicle inspection (the deadline of the vehicle inspection) is shorter than the predetermined period, Call attention to receive. For example, the processing unit issues an alert by issuing a voice saying “The expiry date of the vehicle inspection validity period is approaching”. When the period until the expiration date of the valid period of the vehicle inspection is further shortened, for example, the processing unit issues a warning with a voice “Please have a vehicle inspection”.

  In another modification of the sixth embodiment, the processing unit has a function of recognizing characters displayed on the maintenance / inspection sticker. The time limit for the next periodic inspection is displayed on the maintenance / inspection sticker in the date. The processing unit reads the date information indicating the deadline of the periodic inspection by character recognition, and alerts the user to receive the periodic inspection when the period from the current time to the deadline of the periodic inspection becomes shorter than a predetermined period. For example, the processing unit issues an alert by issuing a voice saying “The deadline for periodic inspection is approaching”. When the period until the periodical inspection is further shortened, for example, the processing unit issues a warning with a voice “Please perform periodic inspection”.

  In still another modified example of the sixth embodiment, the processing unit has a function of determining whether or not an inspection mark is included in an image captured by a camera as a function of image analysis. As a process corresponding to the analysis result, when the inspection mark is not reflected in the image, the processing unit executes a process for calling attention to adjust the camera mounting position and posture so that the inspection mark is reflected. To do. For example, the processing unit may emit a voice “The inspection mark is not shown. Adjust the position and orientation of the camera so that the inspection mark is shown.” Thereby, at the time of installation of a camera, the position and attitude | position can be adjusted easily. The processing unit has a function of reading contents displayed on a display device in the vehicle, such as a speedometer and a fuel meter, and has a function of calling attention so that the display device in the vehicle is reflected in the image.

  In yet another modification of the sixth embodiment, the processing unit has a sticker other than a sticker approved by law as a function of image analysis, on the side window of the windshield, the driver's seat and the passenger seat. Has a function to determine whether or not. As a process corresponding to the analysis result, the processing unit executes a process of notifying the driver or the like to remove the sticker when it is determined that a sticker other than a sticker permitted by law is attached.

[Seventh embodiment]
Next, a seventh embodiment will be described. The processing unit of the system according to the seventh embodiment has a function of analyzing an image of an area in which a passenger seat is reflected as an image analysis function, and executes processing corresponding to the situation of the passenger seat as processing corresponding to the analysis result. To do.

  For example, appropriate processing can be executed depending on whether a person is on the passenger seat or whether an object is placed on the passenger seat.

  In a modification of the seventh embodiment, the drive recorder 1 (FIG. 1) is mounted on a vehicle having a function of issuing a warning when a person sitting in the passenger seat is not wearing a seat belt. As a function of image analysis, the processing unit has a function of determining whether or not a person is sitting on the passenger seat from an image of an area where the passenger seat is reflected. As processing corresponding to the analysis result, when the processing unit detects that no person is sitting in the passenger seat, the processing unit invalidates the function of issuing a warning that the passenger seat is not worn. The processing unit further has a function of determining whether an object is placed on the passenger seat as an image analysis function. As a process corresponding to the analysis result, when an object is placed on the passenger seat, the processing unit executes a process of invalidating a function of issuing a warning that the passenger seat is not worn on the passenger seat.

  The vehicle may erroneously determine that a person is on the vehicle when an object equivalent to the human weight is placed on the passenger seat, and may issue a warning that the seat belt is not worn. When this system is attached to a vehicle, the vehicle will not issue a warning that the seat belt is not worn when an object is placed in the passenger seat. For this reason, the driver is not confused by the warning. In particular, this function is effective in a vehicle having a specification in which a warning buzzer continues to sound when a heavy object is placed on the passenger seat. In order not to issue a warning that the seat belt is not attached to the vehicle, the processing unit transmits a signal for invalidating the warning to the vehicle controller through the in-vehicle network (CAN).

  In a vehicle having the same function for the rear seat, when the processing unit detects that an object is placed on the rear seat, the processing unit cancels the seat belt non-wearing warning function.

[Eighth embodiment]
Next, an eighth embodiment will be described. The processing unit of the system according to the eighth embodiment has a function of comparing a current in-vehicle image with a past in-vehicle image as a function of image analysis, and as a process corresponding to the analysis result, a current in-vehicle image When the degree of mismatch with the past in-vehicle image exceeds a predetermined level, a notification process is executed.

  By receiving notification of the owner or manager of the vehicle, it is possible to know that the situation inside the vehicle has changed beyond a certain level.

  In the modified example of the eighth embodiment, the processing unit uses an image in a state where no objects other than those previously attached to the vehicle are brought into the vehicle as the past image in the vehicle. It has a function to determine whether there is something brought into the vehicle by comparing the image in the vehicle.

  The processing unit further has a function of calculating the time during which an object brought into the vehicle is present in the vehicle as it is by comparing the images at a plurality of past points in time with the current image. The processing unit has a function of notifying an object when it has been left in the vehicle for a long time.

  Next, specific processing of the processing unit will be described. First, in a state where no load is loaded on the vehicle, and preferably only a driver is in the vehicle, the initial setting button of the present system is pressed to drive the vehicle. The processing unit recognizes a portion in the image having an optical flow corresponding to the vehicle speed as a window. Any other movement is recognized as a driver. The part other than the part recognized as the window and the driver is recognized as an area where an object can be placed, and is recorded as an unloaded state image (unloaded state image) in which no load is loaded on the vehicle. The initial setting process is completed by the process so far.

  Thereafter, when the engine is started, when the engine is stopped, the processing unit takes the difference between the no-load state image and the current image, and identifies the presence of an object brought into the vehicle. The processing unit issues a warning when there is no change in the position of the object identified as being brought into the vehicle for a certain period of time. This makes it possible for the driver or the like to recognize an object that has been left in the vehicle for a long time. This function is particularly effective for people who are quickly filled with garbage.

[Ninth embodiment]
Next, a ninth embodiment will be described. The processing unit of the system according to the ninth embodiment has a function of determining whether or not a pre-registered object is reflected in an image in the vehicle as an image analysis function, and is registered in advance as a process corresponding to the analysis result. A process for issuing an alarm when the object is reflected in the image inside the vehicle is executed.

  The driver or the like can notice that there is something pre-registered in the vehicle by the warning.

  In the modification of the ninth embodiment, the processing unit has a function of registering an image of an object (a specific bag, wallet, etc.) that the user wants to check for a lost item. When the processing unit detects that a registered object is reflected in the image in a state where no one is in the vehicle, the processing unit issues an alarm or notifies a forgotten object. As an example of issuing an alarm, for example, a buzzer attached in the vehicle is sounded. As an example of notification of forgotten things, for example, a wireless notification is sent to a smartphone such as a driver or a mobile phone by e-mail or the like.

  In another modification of the ninth embodiment, the processing unit has a function of issuing an alarm when detecting forgotten things in the vehicle and detecting a movement of opening the door. The warning allows drivers and others to notice forgotten things before leaving the vehicle. The processing unit further has a function of notifying the driver or the like that there is a forgotten item by e-mail or the like when detecting the forgotten item in the vehicle and detecting the movement of closing the door. The driver can notice that there is something left behind after leaving the vehicle. The movement for opening and closing the door is detected by detecting an optical flow, for example.

  In still another modification of the ninth embodiment, the processing unit has a function of detecting other movements in the vehicle by detecting an optical flow. For example, the processing unit has a function of detecting a door opening / closing movement, a window opening / closing movement, a winker operation, a steering wheel operation, a sun visor raising / lowering operation or the like as an image analysis function. This system can acquire such information without connecting to the connector (for example, OBD connector) of the in-vehicle network of the vehicle.

  When the sun visor is down, especially when the driver side is imaged with a 360-degree camera or the like attached to the windshield, the sun visor gets in the way and images in that direction cannot be obtained at all. Therefore, a warning is issued when the processing unit detects that the sun visor is getting down, even though the front of the vehicle is bright enough to determine that the sun visor is unnecessary. Whether or not the sun visor is descending is determined by the processing unit based on the ratio of the sun visor area to the image.

[Tenth embodiment]
Next, a tenth embodiment will be described. The processing unit of the system according to the tenth embodiment has a function of recognizing an image by capturing an object close to the camera as a function of image analysis, and an object close to the camera as a process corresponding to the analysis result The process according to the information described in is executed.

  The user can execute processing corresponding to information written on the object by bringing the object he / she has close to the camera.

  In a modification of the tenth embodiment, the processing unit has a function of performing image recognition of the driver's license. The processing unit recognizes the effective date written on the driver's license, and when the period from the current time to the effective date is shorter than the predetermined period, the user confirms that the renewal date of the driver's license is approaching. To inform.

  In another modification of the tenth embodiment, the processing unit has a function of performing image recognition of a receipt at a gas station. The processing unit character-recognizes the date, the amount of fuel, and the amount of money displayed on the receipt, records these information, and creates a database.

  In yet another modification of the tenth embodiment, the processing unit has a function of recognizing an image of an identification card (for example, a driver's license, an employee ID card, a Juki card, etc.). The processing unit recognizes the name and the like written on the identification card, and determines whether or not the person is registered in advance. The processing unit performs a process for prohibiting engine start when the person of the identification card is not registered. Thereby, the person who can drive a vehicle can be limited to the person registered previously. Further, the processing unit records the name of the person on the identification card. The recorded information can be used to identify the driver when managing the operation of the vehicle.

  In yet another modification of the tenth embodiment, the processing unit recognizes, as an image recognition function, a function for recognizing what the user has brought close to the camera outside the vehicle, and a function for recognizing what the user has brought close to the camera inside the car. Has function. For example, the processing unit recognizes that there is something close to the camera when something that is not in the normal image occupies a part wider than a predetermined range within the angle of view of the camera compared to the normal image To do. In particular, when the processing unit recognizes that the user occupies a part wider than a predetermined range within the angle of view of the camera, the processing unit determines that the user has approached the camera.

[Eleventh embodiment]
Next, an eleventh embodiment will be described. The processing unit of the system according to the eleventh embodiment has a function of performing image analysis of an object reflected in the front image as a function of image analysis, and information to be noted regarding vehicle travel as a process corresponding to the analysis result. To provide the driver with the information.

  By alerting the driver according to the situation ahead, the occurrence of an accident that may occur in the future due to the vehicle traveling forward can be suppressed.

  In the modification of the eleventh embodiment, the processing unit has a function of calculating at least one of an inter-vehicle distance and a relative speed with the vehicle ahead as an image analysis function. The processing unit detects a situation where the risk of collision is high based on the inter-vehicle distance or the relative speed, and notifies the driver that the risk of collision is high by issuing an alarm or the like. The inter-vehicle distance is calculated from information such as the size of the preceding vehicle in the image, for example, the size of the license plate, the size of the tread (tire interval), and the width of the tail lamp. The relative speed is calculated from the rate of change in the inter-vehicle distance.

  In another modification of the eleventh embodiment, the processing unit has a function of detecting the color of the traffic light ahead as a function of image analysis. When the traffic light is lit red, the processing unit notifies the driver to urge the stop.

  In yet another modification of the eleventh embodiment, the processing unit has a function of recognizing a road sign ahead as an image analysis function. The processing unit alerts the driver according to the information indicated by the road sign. For example, when the road sign indicates a speed limit, the processing unit acquires the speed limit at the current point by character recognition and notifies the driver of the speed limit. When the road sign indicates prohibition of entry, the processing unit issues a warning and notifies the driver when entering the prohibited road. When the processing unit detects that the vehicle is going to run against the traffic rule indicated by the road sign or is running against the traffic rule, the processing unit notifies the driver that it is a violation of the rule. Thereby, generation | occurrence | production of traffic rule violation can be prevented or suppressed. When the processor recognizes the character string of the road sign and detects a sign indicating that the current traveling section is an automatic speed regulator installation section, the processing section should be careful to give the driver too much speed. Notice.

  In still another modified example of the eleventh embodiment, the processing unit has a function of detecting a traffic jam in a lane adjacent to the host vehicle lane as a function of image analysis. Whether the traffic jam has occurred or not is determined by the processing unit based on the average traveling speed of the vehicles in the adjacent lanes and the inter-vehicle distance. When the traveling speed of vehicles in adjacent lanes is equal to or less than a predetermined reference value, the distance between the vehicles is equal to or less than the predetermined reference value, and a predetermined number of vehicles are lined up, the processing unit determines that traffic congestion has occurred. To do. When the processing unit detects that a traffic jam has occurred in the lane adjacent to the host vehicle lane as a process corresponding to the analysis result, the processing unit prompts the driver to be careful of jumping out between vehicles in the traffic jam. Execute the alert process. The lane adjacent to the host vehicle lane includes the oncoming lane and the adjacent lane that travels in the same direction as the host vehicle. Particularly in places where people gather, such as shopping malls, theme parks, stations, and places where vehicles enter and exit such as gas stations, there is a high possibility that traffic in the same direction lane adjacent to the host vehicle lane will also be congested. In such a case, it is effective to detect a traffic jam in an adjacent lane that runs in the same direction as the host vehicle. Thereby, the occurrence of an accident due to jumping out can be suppressed. When the traveling speed of the host vehicle is less than a predetermined speed, for example, less than 30 km / h, the processing unit executes a process that does not emit a warning sound for jumping out. Thereby, generation | occurrence | production of an unnecessary warning sound can be avoided. In addition, the processing unit has a function of detecting a situation where there is a high risk of popping out from a front image and calling attention. For example, the processing unit alerts popping out when there are parked vehicles continuously along the road, or when there is a parked vehicle in a parking lot where a parking meter is installed.

  For example, some car navigation devices are provided with a function for urging the driver to drive based on traffic jam probe information, general road high-speed driving information, and peripheral facility information (such as residential areas and schools). By using this system, it is possible to call attention due to traffic jams even in locations where traffic jam probe information cannot be obtained.

  In still another modification of the eleventh embodiment, the processing unit has a function of detecting an animal (dog, cat, raccoon dog, wild boar, etc.) or a person around the vehicle before starting as an image analysis function. Have. When the processing unit detects an animal or a person around the vehicle, the processing unit notifies the driver that there is an animal or the like around the vehicle. Thereby, the occurrence of an accident immediately after starting can be suppressed.

  In still another modification of the eleventh embodiment, the processing unit has a function of detecting that the vehicle is slowly moving forward from the flow of surrounding scenery as a function of image analysis. When the processing unit detects that the vehicle is moving forward slowly, it issues an alarm. For example, when the vehicle has stopped temporarily, and the vehicle gradually moves due to the creep phenomenon due to the looseness of the brake pedal, the driver is informed that the vehicle is moving and that there is a risk of a rear-end collision. be able to. The processing unit determines that the vehicle is moving forward slowly when the vehicle speed is moving at a speed at which the vehicle moves forward due to a creep phenomenon. The speed of the surrounding landscape flow is determined by the processing unit detecting the optical flow.

  In yet another modification of the eleventh embodiment, the processing unit determines whether the preceding vehicle is a large vehicle, for example, a bus, or a loading platform is an aluminum box truck, or the like, as a function of image analysis. Has function. When the processing unit detects that the preceding vehicle is a large vehicle and has approached the traffic signal, the processing unit notifies the driver of the presence of the traffic signal. For example, a voice such as “Signal attention” is issued. The presence / absence of a traffic signal is registered in advance in the processing unit as POI information. For example, when a large vehicle is traveling ahead, it is difficult to see a traffic light in the traveling direction. It is preferable to look at the traffic signal on the opposite lane and the traffic signal for pedestrians to predict the presence of the traffic signal, or to take a sufficient inter-vehicle distance so that the traffic signal in the traveling direction can be easily seen. However, if such attention is neglected, there is a risk of overlooking traffic lights in the direction of travel. In the countryside, there may be no traffic signal in the opposite lane or pedestrian traffic light. By notifying the presence of a traffic light when the preceding vehicle is a large vehicle, the risk of the driver overlooking the traffic light can be reduced. In addition to the function of notifying the presence of a traffic light directly, the processing unit has a function of alerting the user to travel so as to reduce the risk of overlooking the traffic light. For example, the processing unit emits a voice such as “Please keep a sufficient distance between vehicles”. For example, alerting is performed when the distance to the intersection where the traffic signal is installed is longer than the safe stop distance of the vehicle. The safe stop distance of the host vehicle varies depending on the host vehicle speed and the weather (whether it is raining or not). The processing unit sets an appropriate distance as the safe stop distance based on the host vehicle speed and the weather (whether it is raining or not). For example, if the safe stop distance on a clear day is 50m, the processing unit is located outside the city area in the car navigation device when the current weather is fine and the distance to the large vehicle in front is less than 50m. When there is an intersection where a traffic light is installed within 100m in the direction of travel, a warning is given.

  In yet another modification of the eleventh embodiment, the processing unit determines whether or not the road ahead is a slope as a function of image analysis. It has a function to detect. The processing unit urges attention such as excessive speed, low speed, and setting to hill climb mode depending on the slope of the slope.

  In still another modified example of the eleventh embodiment, the vehicle is one handling a load, such as a forklift, a truck, etc., and the processing unit reads a character string, a barcode, etc. displayed on the load from the captured image. The vehicle has a function of recognizing the sign and determining whether the vehicle should be handled. Information represented by a sign attached to the luggage to be handled by the host vehicle is registered in advance in the processing unit. When the processing unit detects that the vehicle is about to load a baggage that is not to be handled, the processing unit executes a process for issuing an alarm.

  In yet another modification of the eleventh embodiment, the processing unit has a function of detecting a window area in the image from the optical flow. Further, the processing unit has a function of determining from the optical flow whether the vehicle has entered the tunnel, whether the vehicle has left the tunnel, or whether the vehicle is traveling in the tunnel. In addition, the processing unit has a function of determining from the optical flow whether the vehicle is traveling on an underpass (below a vertical parallel road). Further, the processing unit has a function of determining the presence / absence of a guardrail and the necessity of attention to a side groove from the optical flow.

[Twelfth embodiment]
Next, a twelfth embodiment will be described. The processing unit of the system according to the twelfth embodiment has a function of recognizing characters written on a license plate of another vehicle reflected in an image outside the vehicle as a function of image analysis, and as a process corresponding to the analysis result, At least one of the process of recording the information of the license plate specified by the character recognition and the process of transmitting through the network is executed.

  It is possible to specify the information of the license plate of another vehicle that is running near the vehicle equipped with the vehicle-mounted device having the function of the processing unit of this system or that is stopped nearby. The processing unit transmits the current position information of the vehicle on which the vehicle-mounted device is mounted and the date / time information when the license plate is imaged in association with the information of the specified license plate. The license plate information acquired together with the position information and date / time information is used to know the travel routes of many vehicles together with the information of the license plate acquired by the automobile number automatic reading device (N system) installed on the road. Useful information. In this way, by associating the vehicle number and the GPS information, a system that can be called a wide-area mobile N system can be constructed.

  In the modified example of the twelfth embodiment, a center system is constructed that collects positional information, date / time information, and license plate information in association with each other via a network from this system mounted on each of a plurality of vehicles. The center system has a function of extracting the position information and date / time information of a vehicle having specific license plate information and specifying the travel route of the vehicle.

[Thirteenth embodiment]
Next, a thirteenth embodiment will be described. The processing unit of the system according to the thirteenth embodiment has a function of determining whether or not a window glass edge exists in the window portion of the vehicle as a function of image analysis. When the edge of the window glass is detected, a notification process is executed.

  The presence of the window glass edge in the window means that the window glass has been forgotten to be closed. The driver can notice that the window glass has been forgotten to be closed by receiving a notification from the system. The processing unit detects the edge of the window glass by detecting image discontinuity due to the difference in refractive index between the window glass and the atmosphere. As the notification process, the processing unit performs a process such as a wireless notification to the driver's portable terminal or the like, or a notification by e-mail. In addition, the processing unit performs notification processing by sounding a buzzer attached to the vehicle. When the smoke film is pasted on the window of the vehicle equipped with the processing unit, the processing unit determines the window based on the difference in image brightness between the area where the window is open and the smoke area of the window glass. Detect that is open.

[14th embodiment]
Next, a fourteenth embodiment will be described. The processing unit of the system according to the fourteenth embodiment has a function of determining whether or not it is raining by detecting whether or not the wiper operates or whether or not raindrops attached to the windshield increase or decrease as an image analysis function. As a process corresponding to the analysis result, if it is determined that it is raining, a process of transmitting information notifying that it is raining from the communication device is executed.

  By receiving the information transmitted from this system, it is possible to know whether or not it is raining in the area where the vehicle is traveling. The processing unit has a function of notifying an external device such as a server that it is raining and also notifying current position information of the vehicle. The external device can create a map of a raining region by collecting information from a plurality of vehicles. Further, the processing unit has a function of notifying that it is raining and notifying current time information. As a result, the external device can acquire information related to temporal changes in the raining area.

  In a modification of the fourteenth embodiment, the external device is a group server that provides information related to the weather. The processing unit has a function of uploading to the server an image when the wiper is not reflected, for example, when the wiper is lowered. The server has a function of acquiring information related to rainfall, such as rainfall, from the uploaded image. The server has a function of utilizing data such as rainfall collected from vehicles as big data in various application fields. For example, the server is used for predicting local guerrilla heavy rain. The processing unit determines that the rain has stopped when raindrops are attached to the windshield, but the number (number) of the raindrops does not increase or decrease. When it is determined that the rain has stopped, the processing unit notifies the server that the rain has stopped. The server can obtain more detailed information regarding rainfall.

[Fifteenth embodiment]
Next, a fifteenth embodiment is described. The processing unit of the system according to the fifteenth embodiment has a function of determining the weather from the sky situation reflected in the image as a function of image analysis, and transmits information on the weather from the communication device as a process corresponding to the analysis result. Execute the process.

  Information on weather collected from a plurality of vehicles can be used as big data on weather information. The server that collects the information determines weather such as clear, clear, and cloudy from the color of the sky reflected in the image, the presence or absence of clouds, and the like. Further, the processing unit has a function of detecting the strength of the morning sun and the western day from the height of the sun and the brightness of the sky, and transmitting information that the morning sun and the western day are strong. The processing unit transmits the current position information of the vehicle together with information related to the weather. The server that collects this information has a function of providing information that the Asahi and the West are strong, and the current position information of the vehicle to drivers of other vehicles heading to that point. A driver who has received this information can be aware that attention needs to be paid to the morning sun or the west.

[Sixteenth embodiment]
Next, a sixteenth embodiment will be described. The processing unit of the system according to the sixteenth embodiment has a function of detecting sound information from a microphone mounted on a vehicle, and associates a detection result of sound information with an analysis result of the image analysis function, and results of association The process according to is executed.

  It is possible to perform more appropriate processing by associating both, as compared with the case where processing according to only the sound information detection result or the image analysis result is executed.

  In the modification of the sixteenth embodiment, the processing unit has a function of detecting a change in the state of a display unit of a device having a display unit mounted in a vehicle as a function of image analysis, and a process corresponding to the analysis result. As described above, when the detected sound information and the change in the state of the display unit are synchronized, a process of storing the detected sound information and the device in which the state of the display unit is changed in association is executed.

  In another modification of the sixteenth embodiment, when the processing unit detects sound information from the microphone, the processing unit generates a sound based on information stored in association with the sound information and the change in the state of the display unit. Has a function to identify the device that issued. For example, the processing unit has a voice recognition function, determines whether or not the sound information from the microphone is issued from a specific device, and if the sound information is issued from the specific device, the voice recognition is performed. Cancel the function. Thereby, execution of unnecessary voice recognition processing can be avoided. In addition, when the processing unit determines that the sound information from the microphone is emitted from a specific device, the processing unit adds information that can distinguish the region of the device that generated the sound from the other image in the vehicle image. And has a function to record. For example, an image of a closed curve having a rectangular shape or an arbitrary shape surrounding an area where a device that generates sound is reflected is recorded on the original image. Alternatively, a semi-transparent geometric pattern is overlaid on the area where the device that generated the sound is shown.

  In yet another modification of the sixteenth embodiment, the processing unit has a function of determining whether or not an emergency vehicle such as an ambulance, a fire engine, or a police car is reflected in the image as a function of image analysis. When the processing unit detects the siren sound of the emergency vehicle from the microphone and detects the emergency vehicle in the image, the processing unit associates the two and determines that the emergency vehicle is in the vicinity of the host vehicle. When determining that there is an emergency vehicle, the processing unit notifies the driver of a message that calls attention.

[Seventeenth embodiment]
Next, a seventeenth embodiment will be described. In the seventeenth embodiment, there are a plurality of cameras mounted in the vehicle, and the processing unit has a function of detecting the position of another camera reflected in the image as a function of image analysis, and processing corresponding to the analysis result As described above, a process for specifying the positional relationship among a plurality of cameras is executed.

  It is possible to obtain an image of a wide area in the vehicle that is difficult to capture with one camera. When the positional relationship of a plurality of cameras is specified, the positional relationship of images captured by each camera can be specified.

  According to a modification of the seventeenth embodiment, the processing unit has a function of specifying the model of another camera that is reflected in the image and specifying the angle of view from the model. The processing unit further has a function of combining images captured by a plurality of cameras. By synthesizing the images, the situation inside the vehicle can be grasped intuitively as compared with the case of displaying individual images of the camera.

  According to another modification of the seventeenth embodiment, at least a part of the functions of the processing unit is realized by a viewer such as a personal computer. The processing unit displays a sign such as a camera icon so that the position of the camera can be seen in the displayed image. It is difficult to specify the position of the camera in the vehicle from the image captured by the camera (self). The processing unit detects the position of the camera in the vehicle from the image of another camera in which the camera is reflected. For example, when the images captured by the three cameras are combined and displayed, the processing unit displays the positions of the three cameras with icons or the like in the combined image. Further, the icons are changed according to the camera model or the angle of view.

  According to still another modification of the seventeenth embodiment, the processing unit displays an image captured by each camera, and also includes the position of the camera and the angle of view of the camera (the camera captures the image). Display area). For example, the angle of view of the camera is displayed in the image according to the functions of the 360-degree camera, the hemisphere 360-degree camera, the wide-angle camera, and the like. The user who has seen the image can intuitively grasp whether each camera is capturing the area.

[Eighteenth embodiment]
Next, an eighteenth embodiment will be described. In the eighteenth embodiment, the camera mounted on the vehicle is a 360-degree camera having an image circle smaller than the diagonal length of the imaging surface, and the processing unit calculates an image feature amount as a function of image analysis. Therefore, the basic pattern has a function of deforming according to the position in the image circle.

  The processing unit has a function of calculating the feature amount of the image using the basic pattern without correcting the distortion of the image in the image circle. Thereby, the load on the processing unit can be reduced. The basic pattern is defined for an image obtained by imaging using a standard lens (a general wide-angle lens). Projection methods for obtaining image circles include, for example, projection methods such as orthographic projection, equidistant projection, stereoscopic projection, and uniform solid angle projection, and the processing unit deforms the basic pattern in accordance with these projection methods. As a camera lens, a circumferential fisheye lens is used to obtain an image circle. The imaging surface of the camera is rectangular, and the diameter of the image circle is shorter than the long side and equal to the short side or shorter than the short side.

  As a basic pattern, a template pattern of an image to be detected is registered in advance in the processing unit. The processing unit deforms the template pattern according to the position in the image circle. In addition, the processing unit has a function of recognizing an image using, for example, a Harrij feature, and a function of deforming each of a plurality of rectangular patterns for obtaining a Harrij feature amount according to a position in the image circle. .

  In a modified example of the eighteenth embodiment, the processing unit has a function of limiting a region in which a specific object to be detected is reflected among regions in the image circle when performing image recognition. Image recognition is performed on the inside. This function is realized by a drive recorder. In general, the recommended installation position of the drive recorder is determined in advance, so the area in the image circle where the driver is reflected, the area where the traffic lights are reflected, the area where the people in the rear seat are reflected, etc. are almost fixed. . For this reason, according to the person or thing which should be detected, the area | region which performs image analysis in order to detect the person or thing is limited to the area | region where the person or thing is reflected. For example, when analyzing an image in front of the vehicle, the processing unit analyzes a region where the windshield is reflected.

  In another modification of the eighteenth embodiment, the processing unit converts a specific region of the image circle into a planar image (parallel projection image), and performs matching using the normal basic pattern for the converted planar image. .

In yet another modification of the eighteenth embodiment, the processing unit maps the image in the image circle to a sphere using a predetermined conversion function, zooms to a part of the image mapped to the sphere, and ignores the distortion. Then, image analysis is performed as a planar image. When it is desired to detect a specific object, such as a road sign, the image analysis is performed by zooming to a region where the specific object is reflected. When it is desired to detect a road sign, image analysis is performed by zooming to an area of the front image where the road sign appears.
[Nineteenth embodiment]
Next, a nineteenth embodiment will be described. In the nineteenth embodiment, the processing unit has a function of specifying an area that should not be disclosed in the image as a privacy protection area from the viewpoint of privacy protection as a function of image analysis. A process of generating an image in which a person reflected in the privacy protection area cannot be specified by appearance from an image captured by the camera is executed.

  By making it impossible to specify a person, the privacy of that person can be protected. The processing unit has a function of setting an area in which the vehicle is reflected and an area that should not be disclosed. In particular, when an image is acquired using an omnidirectional camera, the inside of the vehicle is picked up, but there may be a case where it is not desired to leave an image in the vehicle. In such a case, the processing unit has a function of blackening an area inside the vehicle, or a function of applying a mosaic to the area inside the vehicle and recording an image outside the vehicle as it is. The processing unit has a function of discriminating between a region where the inside of the vehicle is shown and a region where the outside of the vehicle is shown by performing image analysis. As an example, the processing unit has a function of determining a region where an optical flow is generated at a speed corresponding to the vehicle speed as outside the vehicle. The processing unit has a function that allows the user to set a privacy protection area in advance.

  In a modification of the nineteenth embodiment, the processing unit specifies a normal mode for recording an image that can specify a person in the privacy protection area and a person reflected in the privacy protection area by appearance. A function that allows the user to select a privacy mode for recording images that cannot be recorded. The user can switch between the normal mode and the privacy mode in accordance with the purpose of use and usage situation of the vehicle.

  According to another modification of the nineteenth embodiment, the processing unit has a function of setting an area in which a person is shown as a privacy area. The processing unit has a function of detecting a person by performing image analysis and setting a region surrounded by the detected outline of the person as a privacy region. Further, the processing unit has a function of setting only the detected human face portion as a privacy area. At this time, the processing unit has a function of displaying a person as a silhouette. Alternatively, the processing unit has a function of displaying only an outline of a person and displaying an image when there is no person inside the outline. At this time, the person looks transparent. The processing unit has a function of displaying an edge of a person having movement as the outline of the person.

  As yet another modification of the nineteenth embodiment, the processing unit has a function of displaying an avatar instead of a person in an area where the person is shown. The avatar to be displayed is similar to the original person. For example, an avatar reflecting the characteristics is displayed according to gender, body shape, face outline, and the like. A person and an avatar are associated with each other in advance, and the processing unit has a function of specifying a person shown in the image and displaying the avatar associated with the specified person. The avatar is created in advance for each person and registered in the processing unit. The processing unit has a function of moving the avatar according to the movement of the person. For example, when a person raises his / her hand, the avatar's hand is raised, and when the person turns sideways, the avatar also turns sideways. The processing unit has a function of changing the expression of the avatar according to the expression of the person. For example, when the processing unit detects a facial expression such as a person smiling, sad, or angry, it changes the avatar's facial expression into a smiling expression, sad expression, or angry expression. . The processing unit has a function of reading a person's emotion and changing the color of the avatar according to the person's emotion. For example, when it is detected that the person is angry, the color of the avatar is changed to a color representing angry, for example, red.

  As yet another modification of the nineteenth embodiment, the processing unit, before recording an image, detects an image in which a person appearing in the privacy protection area cannot be identified from the image captured by the camera. It has a function to generate and record the generated image. Furthermore, the processing unit has a function of converting and displaying a person reflected in the privacy protection area into an image that cannot be specified by appearance when reproducing an image captured by the camera.

[20th embodiment]
Next, a twentieth embodiment will be described. In the twentieth embodiment, the processing unit executes processing for recording the result of image analysis as text data or coding as processing corresponding to the analysis result.

  By converting to text data or encoding, a desired image can be easily found from the recorded image. For example, the processing unit generates “person detection” as text data when a person is detected in the image, and “red signal” as text data when a red signal is detected in the image. The processing unit has a function of recording the generated text data in association with the image from which the text data is generated. In still another modification, the processing unit has a function of associating an image analysis result with a code instead of text data and coding the image analysis result (binary data).

[21st embodiment]
Next, a twenty-first embodiment will be described. In the twenty-first embodiment, the processing unit has a function of detecting a black-blurred area in the image as a function of image analysis, and a black-blurred area of the image is processed as a process corresponding to the analysis result. Execute processing to restore brightly and display on the display screen.

  A user who has seen the image can easily recognize objects and people in the area where the blackening is blurred. In particular, when an image is acquired using an omnidirectional camera, a scene outside the vehicle and a dark interior are imaged simultaneously in the image, and thus the area in which the interior is reflected is likely to be blurred. The process of detecting a black-blurred area and brightly restoring and displaying it is particularly effective when an image is acquired using an omnidirectional camera. As the processing for bright restoration, the processing unit performs, for example, gamma correction, color correction, conversion to a monochrome image, contrast enhancement, and the like.

[Twenty-second embodiment]
Next, a twenty-second embodiment will be described. In the twenty-second embodiment, the processing unit has a function of allowing the user to divide the imaging range captured by the camera into a plurality of sections, and a function of weighting each section. An imaging condition is determined based on the attached weight, and the camera has a function of imaging with the determined imaging condition.

  The user can acquire an image under an imaging condition suitable for the area to be viewed by increasing the weight of the section to be viewed in the image and reducing the weight of the section that does not need to be viewed. The user can divide the imaging range into an arbitrary shape according to the shape of an area that the user wants to see or does not need to see.

  FIG. 5 is a diagram of an imaging surface for explaining a method of dividing the imaging range into a plurality of sections by the method according to the comparative example. An image circle 73 is formed on the rectangular or square imaging surface 71 of the forward 360-degree camera, and an image circle 74 is formed on the rectangular or square imaging surface 72 of the rear 360-degree camera. In the image circle 73 that images the front, a front view is reflected through the front window, and a dashboard, an instrument panel, a display screen of the car navigation system, and the like are reflected in the vehicle. A driver, a passenger seat, a rear seat, and a ceiling in the vehicle are reflected in an image circle 74 that captures the rear, and scenery of the side and the rear of the vehicle is reflected through the side window and the rear window.

  The imaging surfaces 71 and 72 are divided into a plurality of sections by a square lattice. The user can weight each partition.

  FIG. 6 is a diagram of an imaging surface for explaining a method of dividing the imaging range into a plurality of sections in the system according to the twenty-second embodiment. Similar to the example shown in FIG. 5, an image circle 73 is formed on the imaging surface 71 of the forward 360 degree camera, and an image circle 74 is formed on the imaging surface 72 of the rear 360 degree camera.

  When the processing unit 51 (FIG. 4) causes the user to divide, a part of the imaging range is surrounded by a closed curve 75, and an inner area surrounded by the closed curve 75 is set as one section. The user sets a plurality of sections by drawing a plurality of closed curves 75. When the user divides the inner region surrounded by the closed curve into two by the curve, each of the divided regions is set as one section.

  As in the comparative example shown in FIG. 5, it is possible to perform weighting more appropriate for the user as compared with the case of weighting the sections uniformly divided into a square lattice pattern regardless of the content of the image.

  In particular, when imaging is performed using a 360-degree camera, the difference in position within the imaging range in which the vehicle portion is captured is greater for each vehicle type than when a general camera is used. For this reason, there is a high possibility that the general-purpose weighting table set in the firmware of the processing unit in advance does not match the relationship between the actually anticipated imaging range sections. The function of enabling the weighting to be determined using PC browser software or the like instead of using a general-purpose weighting table has a particularly remarkable effect when using a 360 degree camera.

  The processing unit determines, for example, gain, exposure time, and the like as imaging conditions. In the configuration in which the back surfaces of the two hemispherical cameras are opposed to each other to form a spherical camera, the processing unit makes the imaging conditions of the two hemispherical cameras the same. As a result, when the images captured by the two hemispherical cameras are combined, the boundary between the two images becomes difficult to understand.

  In a modification of the twenty-second embodiment, the processing unit (for example, the AI software of the PC) analyzes the image captured by the camera without weighting, and estimates what is reflected in which part of the imaging range. It has a function to do. For example, the processing unit assigns a weight “5” to the road portion, a weight “3” to the face portion, a weight “1” to the instrument panel portion, and falls outside the range of the round image. A weight “0” is assigned.

  The processing means may divide a part of the imaging range by a closed curve and let an inner area surrounded by the closed curve be one partition when the user classifies the image. The processing means may allow the user to input a weight with a numerical value for each set section. The numerical value to be input may be selected from a pull-down or pop-up list.

  In another modification of the twenty-second embodiment, the processing unit has a function of inputting a partition type instead of a function of inputting a weight as a numerical value or in addition to a function of inputting a numerical value. A weight is associated with the partition type in advance, and when the partition type is input, the processing unit associates the weight associated with the type with the partition. As types of sections, for example, types such as a driver, a meter panel, a car navigation screen, a windshield, a rear seat, and a passenger seat are provided. The processing unit has a function that allows the user to change the correspondence between the type of partition and the weight. A list of partition types can be registered in advance in the processing unit by the user.

  In still another modified example of the twenty-second embodiment, the processing unit has a function of setting a section by causing the user to specify a region to be viewed in the imaging range as a function for weighting each section. The weight of the section that the user wants to see is set larger than the weights of other areas. This function corresponds to an example in which a section is divided into two types, that is, a section that the user wants to see and another section in the function of allowing the user to input a section type.

  In still another modification of the twenty-second embodiment, the processing unit has a function of allowing the user to divide the imaging range in an interactive format (wizard format). The processing unit outputs the message "Please surround the unnecessary area" or "Please fill in the unnecessary area", and the area surrounded or filled by the user as "Unnecessary area" recognize. Further, the processing unit outputs a message “please fill the driver's area” and “please fill the road part”, and the filled area is divided into a section having a driver type or a road. Recognized as a section having a type. The processing unit has a function of allowing the user to perform an operation of enclosing with a closed curve or an operation of painting using a pointing device such as a touch panel or a mouse.

  In yet another modification of the twenty-second embodiment, the processing unit realizes a function of dividing the imaging range into a plurality of sections based on the captured image using artificial intelligence (AI). The processing unit has a function of automatically discriminating the inside and outside of the vehicle, the seat, the dashboard, the instrument panel, and the like from the image and classifying the imaging range for each component that is automatically discriminated. The processing unit displays the classified result on the display screen, and allows the user to determine whether there is no problem in this classification. The processing unit has a function of causing the user to correct the classification. If the user determines that the classification performed by the AI is incorrect, the user can modify the classification. The processing unit has a function of allowing the user to select the correct type from a pull-down or pop-up list. If the user determines that the weight or type corresponding to the partition is wrong, the user can easily correct the weight or type.

[23rd embodiment]
Next, a twenty-third embodiment will be described. In the twenty-third embodiment, the processing unit has a function of allowing the user to divide into a plurality of sections from the imaging range captured by the camera, and a function of weighting each section. A compression process is executed by making a block at the time of compression of a relatively small section larger than a block at the time of compression of a section having a relatively large weight.

  The user can increase the degree of compression of the less important section by increasing the weight of the higher importance section and reducing the weight of the lower importance section. As a result, compared to the case where the entire image is uniformly compressed, the volume of image data can be reduced while maintaining the high quality of the image of the highly important section. For example, the processing unit compresses 8 × 8 pixels as one block in the weighted partition, and 16 × 16 pixels as one block in the lighter partition.

  In a modification of the twenty-third embodiment, the processing unit has a function of painting the area outside the image circle black. In some cases, an image of the lens barrel portion of the lens is reflected in a region outside the image circle formed on the rectangular imaging surface of the camera. By painting an unnecessary image in a region outside the image circle in black, the capacity of image data including pixels outside the image circle can be reduced. In another modification, the processing unit has a function of acquiring image data by raster scanning only the pixels inside the image circle except for the pixels outside the image circle. Thereby, the capacity of the image data can be reduced.

[Twenty-fourth embodiment]
Next, a twenty-fourth embodiment will be described. In the twenty-fourth embodiment, the processing unit has a function of detecting the following vehicle and a function of detecting the distance to the following vehicle as image analysis functions, and the processing corresponding to the analysis result is based on the distance to the following vehicle. To detect the tilting operation of the following vehicle.

  The processing unit can appropriately handle the tilting operation by executing the process according to the determination result. For example, when the speed of the vehicle is equal to or higher than a predetermined reference value and the same subsequent vehicle is continuously present within a predetermined determination distance for a predetermined determination time or longer, the processing unit is driven and driven. It is determined that

  In the modification of the twenty-fourth embodiment, a camera having a function of measuring the distance to the object for each pixel is used. For example, a back-illuminated time-of-flight distance image sensor is used as the image sensor of the camera.

  In another modification of the twenty-fourth embodiment, when the processing unit determines that there is a following vehicle and is driving, for example, notification to the driver, recording of a rear image is performed as processing according to the determination result. , Provide advice to drivers, report to traffic enforcement agencies (eg police), etc. When notifying the traffic control organization, the processing unit attaches an image of the vehicle that is driving in a tilted manner and the driver. The processing unit notifies the driver that there is a possibility of being harassed by a sudden stop when it detects that a vehicle driving in a tilted manner overtakes its own vehicle and turns in front of the own vehicle through image analysis. To do.

  In yet another modification of the twenty-fourth embodiment, the processing unit has a function of detecting people and vehicles around the vehicle and controlling the vehicle according to the distance to the people and the vehicle. For example, the processing unit has a high possibility of contact with a person based on the speed of the own vehicle, the distance to the person ahead, and the determination result of whether or not the person is on the planned traveling route of the own vehicle. If it determines with it, control which stops a vehicle is performed.

  The scope of the present invention is not limited to the configurations explicitly described in the specification, and includes combinations of various aspects of the present invention disclosed in the present specification. Of the present invention, the configuration for which a patent is sought is specified in the appended claims. However, the present invention is disclosed in this specification even if the configuration is not specified in the claims. I intend to make this configuration a claim in the future.

  The present invention is not limited to the configuration described in the above embodiment. The components of the above-described embodiments and modifications may be arbitrarily selected and combined. In addition, any component of each embodiment or modification, and any component described in the means for solving the invention or any component described in the means for solving the invention And may be combined arbitrarily. We also intend to acquire rights in these amendments or divisional applications.

  In addition, the applicant intends to acquire rights for the entire design or partial design by filing an application for conversion to a design application. The drawing shows the entire apparatus as a solid line, but includes not only the entire design but also a partial design that is claimed for a part of the apparatus. For example, it is a drawing that includes a part of the apparatus as a partial design as well as a part of the apparatus as a partial design. A part of the device may be a part of the device or a part of the member. In addition to the overall design, we have the intention to make a partial design with any part of the solid line part of the drawing as a broken line part.

DESCRIPTION OF SYMBOLS 1 Drive recorder 2 Windshield 3 Room mirror 4 Power cable 5 Cigar socket 10 Drive recorder main body 11 Camera 12 GPS receiver 13 SD card reader 14 Acceleration sensor 15 Speaker 16 Display 17 Operation button 20 Processing unit 21 Communication device 30 Server 31 Processing Unit 32 Communication device 33 Input device 34 Display 40 Communication network 50 External device 52 SD card reader 53 Input device 54 Display 60 SD card 71, 72 Imaging surface 73, 74 Image circle 75 Closed curve

Claims (25)

A processing unit having a function of recording an image captured by a camera attached to a vehicle, and a function of performing image analysis of the image captured by the camera and executing processing corresponding to the analysis result based on the analysis result Having a system.   The processing means has a function of determining whether or not the movement of a person in an image corresponds to a specific movement registered in advance as the function of the image analysis, and the movement of the person corresponds to a specific movement When processing is performed, as processing corresponding to the analysis result, recording of an image capturing a specific motion of the person, transmission of an image capturing the specific motion to the server, notification of the specific motion of the person, and an alarm The system of claim 1, wherein at least one process of issuing is performed.   The processing means has a function of determining whether or not a person in the image has a specific characteristic registered in advance as the image analysis function, and the person in the image has a process corresponding to the analysis result. The system according to claim 1, wherein processing according to a specific feature is executed.   The processing means has a function of counting the number of persons in the image as the function of the image analysis, and executes a process of recording or transmitting the number of persons data in the image as a process corresponding to the analysis result. The system according to any one of the above.   The processing means has a function of detecting the display content of the display device of the vehicle or the operation status of the operation device of the vehicle from the image inside the vehicle as the function of the image analysis, and as a process corresponding to the analysis result, The system according to any one of claims 1 to 4, wherein a process according to display contents or an operation state of the operation device is executed.   The processing means has a function of recognizing characters displayed on a sticker affixed to the vehicle from an image in the vehicle as a function of the image analysis, and is indicated by the recognized characters as processing corresponding to the analysis result The system according to claim 1, wherein processing according to information is executed.   The processing means includes, as the image analysis function, a function of analyzing an image of an area in which a passenger seat is reflected, and executes processing corresponding to the situation of the passenger seat as processing corresponding to the analysis result. 7. The system according to any one of items 6.   The processing means has a function of comparing a current in-vehicle image with a past in-vehicle image as a function of the image analysis, and as a process corresponding to the analysis result, in a current in-vehicle image and a past in-vehicle The system of any one of Claims 1 thru | or 7 which performs the process which notifies, when the mismatching degree with the image of above exceeds a predetermined level.   The processing means has a function of determining whether or not a pre-registered object is reflected in an image in the vehicle as a function of the image analysis, and as a process corresponding to the analysis result, the pre-registered object is in the vehicle The system according to any one of claims 1 to 8, wherein a process for issuing an alarm when an image is captured is executed.   The processing means has a function of recognizing an image by capturing an object close to the camera as a function of the image analysis, and is described in an object close to the camera as a process corresponding to the analysis result. The system according to any one of claims 1 to 9, wherein a process according to the information is executed.   The processing means has a function of performing an image analysis of an object reflected in a front image as a function of the image analysis, and provides a driver with information to be careful about traveling of the vehicle as a process corresponding to the analysis result. The system of any one of Claims 1 thru | or 10 which performs the process to perform.   The processing means has a function of recognizing characters written on a license plate of another vehicle reflected in an image outside the vehicle as a function of the image analysis, and is specified by character recognition as a process corresponding to the analysis result. The system according to any one of claims 1 to 11, wherein at least one of a process of recording information on the license plate and a process of transmitting the information through a network is executed.   The processing means has a function of determining whether or not an edge of the window glass exists in the window portion of the vehicle as the function of the image analysis, and as a process corresponding to the analysis result, the edge of the window glass in the window portion. The system according to any one of claims 1 to 12, wherein a notification process is executed when an error is detected.   The processing means has a function of detecting whether or not it is raining by detecting the presence or absence of the operation of the wiper or the increase or decrease of raindrops adhering to the windshield as a function of the image analysis. The system according to any one of claims 1 to 13, wherein as a corresponding process, when it is determined that it is raining, a process of transmitting information notifying that it is raining from a communication device is executed. .   The processing means has a function of determining the weather from the sky situation reflected in the image as a function of the image analysis, and executes a process of transmitting information related to the weather from the communication device as a process corresponding to the analysis result 15. The system according to any one of claims 1 to 14.   The processing means has a function of detecting sound information from a microphone mounted on a vehicle, associates a detection result of sound information with an analysis result of the image analysis function, and executes a process according to the result of the association The system according to any one of claims 1 to 15. There are several cameras installed in the car,
The processing means has a function of detecting the position of another camera in the image as a function of the image analysis, and executes a process of specifying a positional relationship among a plurality of cameras as a process corresponding to the analysis result. The system according to any one of claims 1 to 16. The camera is a 360 degree camera having an image circle smaller than the diagonal length of the imaging surface,
The said processing means has a function which deform | transforms the basic pattern for calculating the feature-value of an image according to the position in the said image circle as a function of the said image analysis. The system described.   The processing means has a function of specifying an area that should not be disclosed in the image as a privacy protection area from the viewpoint of privacy protection as the function of the image analysis, and the privacy protection area as a process corresponding to the analysis result The system according to any one of claims 1 to 18, wherein a process for generating an image in which a person reflected in the image cannot be specified by appearance from an image captured by the camera is executed.   The system according to any one of claims 1 to 19, wherein the processing means executes a process of recording the result of the image analysis as text data or coding as a process corresponding to the analysis result.   The processing means has a function of detecting a black blurred area in the image as a function of the image analysis, and as a process corresponding to the analysis result, brightly restores the black blurred area of the image. The system according to any one of claims 1 to 20, wherein a process to be displayed on a display screen is executed.   The processing means has a function of allowing a user to divide into a plurality of sections from an imaging range captured by the camera, and a function of weighting each section, and the brightness of each section is assigned to the section. The system according to any one of claims 1 to 21, wherein the system has a function of determining an imaging condition based on the determined weight and causing the camera to perform imaging under the determined imaging condition.   The processing means has a function of allowing a user to divide the imaging range captured by the camera into a plurality of sections, and a function of weighting each section, and the process corresponding to the analysis result has a relatively small weight. The system according to any one of claims 1 to 22, wherein a compression process is performed by making a block at the time of compression of a partition larger than a block at the time of compression of a partition having a relatively large weight.   The processing means has a function of detecting the following vehicle and a function of detecting a distance to the following vehicle as the functions of the image analysis, and as a process corresponding to the analysis result, the processing means of the following vehicle is based on the distance to the following vehicle. The system according to any one of claims 1 to 23, wherein processing for detecting tilting operation is executed.   A program for causing a computer to realize the functions of the processing means according to any one of claims 1 to 24.