JP2014145672A - Travel guide system, travel guide method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a travel guide system, a travel guide method, and a computer program with which an installation position of a marked object can be accurately identified in detecting the marked object based on a photographed image.SOLUTION: When there is a guide intersection in front of a direction of travel of a vehicle, a travel guide system photographs the vicinity of the guide intersection included in a guide route with a front camera 19 mounted on the vehicle, identifies an installation position of a marked object included in the photographed images from line segments along a photographing direction, and combines the line segments identified from the plurality of photographed images photographed from different directions, so as to determine the installation position of the marked object.

Description

  The present invention relates to a travel guidance system, a travel guidance method, and a computer program that guide a vehicle using a mark.

  2. Description of the Related Art In recent years, a navigation device is often mounted on a vehicle that provides vehicle travel guidance so that a driver can easily arrive at a desired destination. Here, the navigation device detects the current position of the vehicle by a GPS receiver or the like, acquires map data corresponding to the current position through a recording medium such as a DVD-ROM or HDD or a network, and displays it on a liquid crystal monitor. It is a device that can do. Further, such a navigation device has a route search function for searching for an optimum route from the departure place to the destination when a desired destination is input. Then, the guidance route set based on the search result is displayed on the display screen, and when approaching an intersection (hereinafter referred to as a guidance intersection) for guidance such as a right or left turn, a voice or a display screen is used. By performing the guidance, the user is surely guided to a desired destination. In recent years, some mobile phones, smartphones, PDAs (Personal Digital Assistants), personal computers, and the like have functions similar to those of the navigation device. Furthermore, it is also possible to perform the above guidance for pedestrians and two-wheeled vehicles as well as vehicles.

  Here, when performing guidance such as turning left or right at the guidance intersection, it is important to allow the user to accurately identify the guidance intersection. Therefore, conventionally, guidance has been performed using landmark objects (for example, store signboards) that serve as landmarks of guidance intersections. Further, when performing guidance using a landmark object, it is important that the landmark object is visible to the user when performing guidance. Therefore, for example, in Japanese Patent No. 3399506, when a vehicle approaches a guidance intersection, the direction of the guidance intersection is captured by a camera mounted on the vehicle, and image processing is performed on the captured image so that the user can visually recognize the intersection. A technique for detecting a landmark object and guiding a guidance intersection using the detected landmark object is described.

Japanese Patent No. 3399506 (page 6-7, FIG. 11)

  Here, when performing guidance of the guidance intersection by the landmark object, it is important to accurately specify the installation position of the landmark object. However, in the configuration in which the installation position of the landmark object ahead of the traveling direction of the vehicle is specified by performing image processing on the image captured by the camera as in the technique described in Patent Document 1, the depth direction ( The detection error in the direction of the optical axis of the camera is large, and there is a problem that the installation position of the mark object cannot be accurately specified. As a result, there has been a situation in which guidance at a guidance intersection is performed using a mark object that is invisible to the user.

  The present invention has been made in order to solve the conventional problems, and a travel guidance system that can accurately specify the installation position of the mark object when detecting the mark object from the captured image. An object is to provide a travel guidance method and a computer program.

In order to achieve the above object, the travel guidance system (1) according to claim 1 of the present application includes a guide route setting means (13) for setting a guide route and an imaging device (19) mounted on a vehicle. The first picked-up image acquisition means (13) for acquiring a first picked-up image obtained by picking up the included guide intersection (52) from the first direction, and the guide intersection included in the guide route by the image pickup device mounted on the vehicle. Based on the second captured image acquisition means (13) for acquiring a second captured image captured from a second direction different from the first direction, and the first captured image acquired by the first captured image acquisition means, The first mark position that specifies the installation position of the mark object (54) as a mark when guiding the guidance intersection included in the first captured image by the line segment (61) along the first direction. Specific means ( 3) and, based on the second captured image acquired by the second captured image acquisition means, an installation position of the landmark object included in the second captured image is a line segment along the second direction. Based on the second mark position specifying means (13) specified by (62), the line segment specified by the first mark position specifying means and the line segment specified by the second mark position specifying means, Mark position determining means (13) for determining the installation position of the mark object.
Note that the “guidance intersection” corresponds to an intersection that is a target for performing guidance such as a right / left turn instruction when performing guidance of movement of the moving object according to the guidance route.
Further, the “marking object” is a structure installed on the ground surface that can serve as a user's mark, and corresponds to, for example, a store signboard.

  The travel guidance system (1) according to claim 2 is the travel guidance system according to claim 1, wherein the mark position determining means (13) is specified by the first mark position specifying means (13). An intersection (62) between the line segment (61) thus determined and the line segment specified by the second mark position specifying means (13) is determined as an installation position of the mark object (54). .

  Further, the travel guidance system (1) according to claim 3 is the travel guidance system according to claim 1 or 2, wherein the first mark position specifying means (13) is added to the first captured image. A line segment (with a width based on a detection error in the front-rear direction along the first direction from the origin, where the installation position of the mark object (54) detected by performing image processing on the origin is set as the origin ( 61), the second mark position specifying means (13) uses the installation position of the mark object detected by performing image processing on the second captured image as an origin, and from the origin to the A line segment (62) having a width based on a detection error in the front-rear direction along the second direction is specified.

  A travel guidance system (1) according to claim 4 is the travel guidance system according to any one of claims 1 to 3, wherein the landmark object determined by the landmark position determining means (13) is provided. Information registration means (13) for registering the installation position of the object (54) in association with the landmark object, and intersection guidance means for guiding the guidance intersection based on the information registered by the information registration means ( And 13).

  Further, the travel guidance system (1) according to claim 5 is the travel guidance system according to claim 4, wherein the intersection guidance means (13) is visually recognized by a passenger of the vehicle at the guidance intersection (52). When it is determined that there is a possible mark object (54), the guide object is guided using the mark object, and the mark object that is visible to a passenger of the vehicle at the guide intersection When it is determined that there is not, the guidance intersection is guided using the distance to the guidance intersection.

  According to a sixth aspect of the present invention, there is provided a travel guidance method for setting a guidance route (52) included in the guidance route from a first direction by a guidance route setting step for setting a guidance route and an imaging device (19) mounted on the vehicle. A first captured image acquisition step of acquiring a captured first captured image; and a second imaging in which a guidance intersection included in the guidance route is captured from a second direction different from the first direction by an imaging device mounted on the vehicle. Marking when guiding the guidance intersection included in the first captured image based on the second captured image acquisition step of acquiring an image and the first captured image acquired by the first captured image acquisition step The first mark position specifying step for specifying the installation position of the mark target object (54) by the line segment (61) along the first direction and the second captured image acquisition step Based on the obtained second captured image, a second marker position specifying step of specifying the installation position of the marker object included in the second captured image by a line segment (62) along the second direction. And a mark position determining step for determining an installation position of the mark object based on the line segment specified by the first mark position specifying step and the line segment specified by the second mark position specifying step; It is characterized by having.

  Further, the computer program according to claim 7 sets a guide route setting function for setting a guide route in the computer and a guide intersection (52) included in the guide route by the imaging device (19) mounted on the vehicle. A first captured image acquisition function for acquiring a first captured image captured from a direction, and a first intersection image captured from a second direction different from the first direction by an imaging device mounted on the vehicle. When guiding the guidance intersection included in the first captured image based on the second captured image acquisition function for acquiring two captured images and the first captured image acquired by the first captured image acquisition function The first mark position specifying function for specifying the installation position of the mark object (54) as a mark by the line segment (61) along the first direction and the second captured image acquisition function. Based on the acquired second captured image, a second mark position specifying function for specifying an installation position of the mark object included in the second captured image by a line segment (62) along the second direction. A mark position determining function for determining an installation position of the mark object based on the line segment specified by the first mark position specifying function and the line segment specified by the second mark position specifying function; Is executed.

  According to the travel guidance system according to claim 1, having the above-described configuration, when a landmark object is detected from a captured image, the landmark object is installed by combining detection results of a plurality of captured images captured from different directions. By determining the position, it is possible to accurately specify the installation position of the landmark object. As a result, it is possible to prevent the guidance intersection from being guided using a mark object that cannot be visually recognized by the user, and to perform more appropriate guidance at the guidance intersection.

  According to the travel guidance system of claim 2, the intersection of the line segment of the installation position of the mark object specified in the plurality of captured images taken from different directions is determined as the installation position of the mark object. Therefore, even if there is a detection error in the depth direction (the optical axis direction of the camera), the installation position of the mark object can be accurately specified.

  According to the travel guidance system of the third aspect, the installation position of the mark object detected by performing image processing on the captured image is set as the origin, and from the origin to the front-rear direction along the imaging direction. A line segment with a width based on the detection error is specified as the installation position of the mark object, so even if there is a detection error in the depth direction (the optical axis direction of the camera), It becomes possible to specify the installation position of the object.

  Further, according to the travel guidance system of the fourth aspect, the installation position of the determined landmark object is registered in association with the landmark object, and guidance of the guidance intersection is performed based on the registered information. By using the determined installation position of the mark object, it is possible to specify the visual aspect of the mark object from the vehicle occupant, and the guide intersection can be guided using the mark object visible from the vehicle occupant. Can be done.

  Further, according to the travel guidance system of claim 5, when it is determined that there is a mark object visible from the vehicle occupant, the guide object is guided using the mark object, and the vehicle When it is determined that there is no landmark object visible from the passenger, the guidance intersection is guided using the distance to the guidance intersection, so the visual aspect of the landmark object from the vehicle passenger is considered. Thus, it is possible to appropriately guide the vehicle traveling at the guidance intersection.

  According to the travel guidance method of the sixth aspect, when detecting the landmark object from the captured image, the installation position of the landmark object is determined by combining the detection results of a plurality of captured images captured from different directions. By determining, it becomes possible to accurately specify the installation position of the mark object. As a result, it is possible to prevent the guidance intersection from being guided using a mark object that cannot be visually recognized by the user, and to perform more appropriate guidance at the guidance intersection.

  Furthermore, according to the computer program according to claim 7, when detecting the mark object from the picked-up image, the installation position of the mark object is determined by combining the detection results of a plurality of picked-up images picked up from different directions. By doing so, it is possible to accurately specify the installation position of the mark object. As a result, it is possible to prevent the guidance intersection from being guided using a mark object that cannot be visually recognized by the user, and to perform more appropriate guidance at the guidance intersection.

It is the block diagram which showed the navigation apparatus which concerns on this embodiment. It is the figure which showed an example of the landmark information memorize | stored in map information DB. It is the figure which showed an example of the memory area of image recognition result DB. It is a flowchart of the intersection guidance processing program concerning this embodiment. It is a flowchart of the intersection guidance processing program concerning this embodiment. It is the figure which showed the picked-up image which imaged the periphery of the guidance intersection with the front camera. It is the schematic diagram which showed the positional relationship of a vehicle and a landmark object candidate. It is a figure explaining the detection method of the installation position of a marker object candidate. It is a figure explaining the case where each line segment of the installation position of the landmark object candidate detected based on the some captured image imaged from a different direction crosses. It is a figure explaining the case where each line segment of the installation position of the marker object candidate detected based on the some captured image imaged from a different direction does not cross | intersect.

  DETAILED DESCRIPTION Hereinafter, a travel guidance system according to the present invention will be described in detail with reference to the drawings based on an embodiment in which the navigation apparatus is embodied. First, a schematic configuration of the navigation device 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a navigation device 1 according to this embodiment.

  As shown in FIG. 1, the navigation device 1 according to the present embodiment includes a current position detection unit 11 that detects a current position of a vehicle on which the navigation device 1 is mounted, a data recording unit 12 that records various data, A navigation ECU 13 that performs various arithmetic processes based on the input information, an operation unit 14 that receives operations from the user, and a liquid crystal display 15 that displays a map around the vehicle and facility information related to the facility to the user. Communicating between a speaker 16 that outputs voice guidance regarding route guidance, a DVD drive 17 that reads a DVD as a storage medium, and an information center such as a probe center or a VICS (registered trademark: Vehicle Information and Communication System) center And a communication module 18 for performing. In addition, a front camera 19 is connected to the navigation device 1.

Below, each component which comprises the navigation apparatus 1 is demonstrated in order.
The current position detection unit 11 includes a GPS 21, a vehicle speed sensor 22, a steering sensor 23, a gyro sensor 24, and the like, and can detect the current vehicle position, direction, vehicle traveling speed, current time, and the like. . Here, in particular, the vehicle speed sensor 22 is a sensor for detecting a moving distance and a vehicle speed of the vehicle, generates a pulse according to the rotation of the driving wheel of the vehicle, and outputs a pulse signal to the navigation ECU 13. And navigation ECU13 calculates the rotational speed and moving distance of a driving wheel by counting the generated pulse. Note that the navigation device 1 does not have to include all the four types of sensors, and the navigation device 1 may include only one or more types of sensors.

  Further, the data recording unit 12 reads out a hard disk (not shown) as an external storage device and a recording medium, a map information DB 31, an image recognition result DB 32, a predetermined program, and the like recorded in the hard disk, and also stores predetermined data in the hard disk And a recording head (not shown) which is a driver for writing. The data recording unit 12 may be configured by a memory card, an optical disk such as a CD or a DVD, instead of the hard disk. Further, the map information DB 31 and the image recognition result DB 32 may be stored in an external server, and the navigation device 1 may be acquired by communication.

  Here, the map information DB 31 is, for example, link data 33 relating to roads (links), node data 34 relating to node points, intersection data 35 relating to each intersection, point data relating to points such as facilities, and map display data for displaying a map. The storage means stores search data for searching for routes, search data for searching for points, and the like.

  Here, as the link data 33, for example, a link ID for identifying the link, end node information for specifying a node located at the end of the link, the road type of the road constituting the link, the number of lanes, the road The width, travel classification for each lane, and the like are stored. The node data 34 stores a node ID for identifying the node, position coordinates of the node, connection destination node information for specifying a connection destination node to which the node is connected via a link, and the like. Further, as the intersection data 35, relevant node information for identifying the node forming the intersection, connection link information for identifying a link connected to the intersection (hereinafter referred to as a connection link), and a landmark object located around the intersection The landmark information 36 regarding the candidate object is stored.

  The landmark object candidate is a candidate for an object (such as a store signboard) that can be a landmark when guiding a guidance intersection. Here, as a landmark object candidate, for example, a specific facility (for example, a convenience store, a gas station, a fast food) that is located within a predetermined range (for example, within a radius of 100 m) from an intersection and that can provide a sign that can be identified from a distance. Corresponding to the store sign. The landmark information 36 stores the types (names) of landmark object candidates for each intersection in the whole country. In addition, when a plurality of landmark object candidates exist for one intersection, the type (name) is stored for each of the plurality of landmark object candidates. In addition, for the landmark object candidate whose installation position is determined in an intersection guidance processing program (FIGS. 4 and 5) described later, the coordinates of the installation position are also stored.

Here, FIG. 2 is a diagram showing an example of the mark information 36 stored in the map information DB 31. As shown in FIG. 2, the landmark information 36 includes an intersection ID for identifying an intersection, a type (name) of a landmark object candidate located around the intersection ID, and a position coordinate (a landmark object candidate whose installation position is determined). Only) are stored in association with each other.
For example, at the intersection with the intersection ID “10001”, there are four objects as landmark object candidates, and each landmark object candidate is “Convenience Store XX Sign”, “Gas Station XX Sign”, and “ “Fast food XX signboard” and “XX bookstore sign”. Furthermore, it is shown that “Convenience Store XX Sign” is installed at (X1, Y1), and “Fast Food XX Shop Sign” is installed at (X2, Y2). Similarly, information related to landmark object candidates at other intersections is also stored.

  The landmark information 36 stores, for each type of landmark object candidate, a voice phrase for allowing the user to identify the landmark object candidate among voice phrases used for voice guidance at the guidance intersection. For example, “Convenience Store XX” is stored as a speech phrase for allowing the user to identify the landmark target candidate of “Convenience Store XX Signboard”, and the landmark target candidate of “Gas Station XX Signboard” is selected by the user. “Gasoline station xx” is stored as a voice phrase to be specified. Similarly, other landmark object candidates are also stored. The mark information 36 may be stored in an external server and acquired by the navigation device 1 through communication.

  On the other hand, the image recognition result DB 32 is a storage unit that stores a history of image recognition results performed based on a captured image captured by the front camera 19 when the vehicle approaches the guidance intersection. Here, the navigation ECU 13 images the vicinity of the guidance intersection when the vehicle approaches the guidance intersection as will be described later. Then, by performing image processing on the captured image, the type and installation position of the landmark object candidate included in the captured image are detected. Note that the installation position of the landmark object candidate is detected by a line segment as described later. In the image recognition result DB 32, the installation position of the detected landmark object candidate and the approach direction of the vehicle with respect to the guidance intersection are stored in association with the detected landmark object candidate.

Here, FIG. 3 is a diagram showing an example of the image recognition result stored in the image recognition result DB 32. As shown in FIG. 3, the image recognition result DB 32 includes an intersection ID for identifying a guided intersection where a vehicle has entered, an approach direction (entrance link) of the vehicle, and landmark object candidates detected when entering the guided intersection. The type and the installation position of the mark object candidate (the coordinates of the start point and end point of the line segment) are stored in association with each other.
For example, in FIG. 3, when a vehicle has entered an intersection with an intersection ID “10001” from the direction of the link “34561” in the past, “Convenience Store XX Sign” and “Gas Station XX Sign” are detected, and The installation position of “Convenience Store XX Sign” is detected by the line segment from (x1, y1) to (x2, y2), and the installation position of “Gas Station XX Sign” is from (x3, y3) to (x4 , Y4). Then, the navigation ECU 13 finally determines the position of the mark object candidate from the line segments of the installation positions of the mark object candidates detected in different approach directions as will be described later.

  On the other hand, the navigation ECU (Electronic Control Unit) 13 is an electronic control unit that controls the entire navigation device 1. The CPU 41 as an arithmetic device and a control device, and a working memory when the CPU 41 performs various arithmetic processes. ROM 43 and ROM 43 in which a RAM 42 that stores route data when a route is searched, a control program, an intersection guidance processing program (FIGS. 4 and 5) described later, and the like are recorded. And an internal storage device such as a flash memory 44 for storing the program read from. The navigation ECU 13 constitutes various means as processing algorithms. For example, the guide route setting means sets a guide route. The first captured image acquisition means acquires a first captured image obtained by capturing the guidance intersection included in the guidance route from the first direction by the front camera 19. The second captured image acquisition means acquires a second captured image obtained by capturing the guidance intersection included in the guidance route from the second direction different from the first direction by the front camera 19. The first mark position specifying means, based on the first picked-up image, sets the position of the mark target object used as a mark when guiding the guidance intersection included in the first picked-up image as a line segment along the first direction. Specified. The second mark position specifying means specifies the installation position of the mark object included in the second captured image based on the second captured image by a line segment along the second direction. The mark position determining means determines the installation position of the mark object based on the line segment specified by the first mark position specifying means and the line segment specified by the second mark position specifying means. The information registration means registers the installation position of the mark object determined by the mark position determination means in association with the mark object. The intersection guidance means guides the guidance intersection based on the information registered by the information registration means.

  The operation unit 14 is operated when inputting a departure point as a travel start point and a destination as a travel end point, and includes a plurality of operation switches (not shown) such as various keys and buttons. Then, the navigation ECU 13 performs control to execute various corresponding operations based on switch signals output by pressing the switches. The operation unit 14 can also be configured by a touch panel provided on the front surface of the liquid crystal display 15. Moreover, it can also be comprised with a microphone and a speech recognition apparatus.

  The liquid crystal display 15 also includes a map image including a road, traffic information, operation guidance, operation menu, key guidance, a planned travel route from the departure point to the destination, guidance information along the planned travel route, news, weather Forecast, time, mail, TV program, etc. are displayed. In particular, in this embodiment, when the guidance intersection approaches within a predetermined distance (for example, 300 m) ahead of the traveling direction of the vehicle, an enlarged view near the guidance intersection and the traveling direction at the guidance intersection of the vehicle are displayed.

  Further, the speaker 16 outputs voice guidance for guiding traveling along the planned traveling route and traffic information guidance based on an instruction from the navigation ECU 13. In particular, in this embodiment, at the timing when the guidance intersection reaches a predetermined distance ahead of the vehicle in the traveling direction (for example, three times of 700 m, 300 m, and 100 m), output of voice guidance for guiding traveling along the guidance route is started. To do. Further, in the navigation device 1 according to the present embodiment, when there is an appropriate target object (for example, a store signboard) at the guidance intersection in front of the traveling direction of the host vehicle, the target mark object is used. Output voice guidance. For example, “Soon after XX (store name) is going to the right” is output.

  The DVD drive 17 is a drive that can read data recorded on a recording medium such as a DVD or a CD. Based on the read data, music and video are reproduced, the map information DB 31 is updated, and the like.

  The communication module 18 is a communication device for receiving traffic information composed of information such as traffic jam information, regulation information, and traffic accident information transmitted from a traffic information center, for example, a VICS center or a probe center. For example, a mobile phone or DCM is applicable.

The front camera 19 is constituted by a camera using a solid-state image sensor such as a CCD, for example, and is installed above the front bumper of the vehicle and installed with the optical axis direction downward from the horizontal by a predetermined angle. And the front camera 19 images the surrounding environment around a guidance intersection, when a vehicle approaches a guidance intersection. Moreover, navigation ECU13 detects the kind and position of the marker object candidate located in the periphery of a guidance intersection by performing image processing with respect to the captured image.
Then, the navigation ECU 13 selects a mark object for guiding the guidance intersection based on the detected mark object candidate.

  Next, an intersection guidance processing program executed by the navigation ECU 13 in the navigation device 1 having the above configuration will be described with reference to FIGS. 4 and 5. 4 and 5 are flowcharts of the intersection guidance processing program according to the present embodiment. Here, the intersection guidance processing program is a program that is executed at predetermined intervals after the ACC of the vehicle is turned on, and guides the guidance intersection in front of the traveling direction of the vehicle. The programs shown in the flowcharts of FIGS. 4 and 5 below are stored in the RAM 42 and the ROM 43 provided in the navigation device 1 and are executed by the CPU 41.

  First, in the intersection guidance processing program, in step (hereinafter abbreviated as S) 1, the CPU 41 determines whether route guidance based on the guidance route set in the navigation device 1 is being performed. Here, the guidance route is a recommended route from the departure point (for example, the current position of the host vehicle) to the destination selected by the user, and is set based on the result of the route search process. Further, the route search process is performed by a known Dijkstra method or the like using link data 33, node data 34, traffic information acquired from the VICS center, etc. stored in the map information DB 31.

  And when it determines with the route guidance based on the guidance route set in the navigation apparatus 1 being performed (S1: YES), it transfers to S2. On the other hand, when it is determined that route guidance based on the guidance route set in the navigation device 1 is not performed (S1: NO), the intersection guidance processing program is terminated.

  In S <b> 2, the CPU 41 acquires the current position and direction of the vehicle based on the detection result of the current position detection unit 11. A map matching process for specifying the current position of the vehicle on the map data is also performed. Furthermore, it is desirable to specify the current position of the vehicle in detail using high-precision location technology. Here, the high-accuracy location technology detects the white line and road surface paint information captured from the camera behind the vehicle by image recognition, and further compares the white line and road surface paint information with a previously stored map information DB, thereby driving the vehicle. This is a technology that makes it possible to detect lanes and highly accurate vehicle positions. The details of the high-accuracy location technology are already known and will be omitted.

  Next, in S <b> 3, the CPU 41 acquires a guide route set in the navigation device 1.

  Subsequently, in S4, the CPU 41 determines whether or not the vehicle is positioned a predetermined distance before the guidance utterance point at the guidance intersection based on the current position and direction of the vehicle obtained in S1 and the guidance route obtained in S2. judge. Note that the guidance intersection is an intersection to which guidance such as a right / left turn instruction is given when the navigation device 1 provides guidance for traveling according to the guidance route set in the navigation device 1 as described above. The guidance utterance point at the guidance intersection is a point at which the guidance voice is uttered at the guidance intersection. For example, the guidance utterance point is 700 m before, 300 m before, and 100 m before the guidance intersection. Further, the predetermined distance of S4 is the vehicle during the time required for detection and selection of the mark object used for guiding the guidance intersection (specifically, the time required for performing the processing of S5 to S17 described later). The distance is longer than the distance traveled. Although it depends on the current vehicle speed of the vehicle, it is set to 50 m, for example.

  If it is determined that the vehicle is positioned a predetermined distance before the guidance utterance point at the guidance intersection (S4: YES), the process proceeds to S5. On the other hand, when it is determined that the vehicle is not located a predetermined distance before the guidance utterance point of the guidance intersection (S4: NO), the intersection guidance processing program is terminated.

  In S <b> 5, the CPU 41 acquires from the map information DB 31 information on landmark object candidates in the vicinity of the guidance intersection that is ahead of the vehicle in the traveling direction. Specifically, the CPU 41 first specifies an intersection ID of a guidance intersection that is ahead in the vehicle traveling direction. And it acquires by reading the kind (name) of the marker object candidate matched with identified intersection ID from the marker information 36 (FIG. 2) memorize | stored in map information DB31.

  After that, in S6, the CPU 41 determines whether or not the landmark object used for guidance at the guidance intersection ahead of the traveling direction of the vehicle is determined. In addition, the mark object used for guidance of the guidance intersection ahead of the advancing direction of a vehicle is decided in below-mentioned S15.

  And when it determines with the mark target object used for guidance of the guidance intersection ahead of the advancing direction of a vehicle not having been decided (S6: YES), it transfers to S7. On the other hand, when it is determined that the mark object used for guidance at the guidance intersection ahead of the traveling direction of the vehicle has already been determined (S6: NO), the process proceeds to S18.

  In S <b> 7, the CPU 41 detects mark target candidates included in the captured image by performing image processing on the captured image obtained by capturing the vicinity of the guidance intersection ahead of the traveling direction with the front camera 19. In particular, when a landmark object candidate is included in the captured image, the installation position of the landmark object candidate is also detected. However, when a landmark object candidate whose installation position has already been determined (that is, a landmark object candidate whose installation position information is registered in the landmark information 36) is detected from the captured image, the landmark object candidate. There is no need to detect the installation position. Details of the process of S7 will be described below.

  First, the CPU 41 records, as a data record, an image most recently captured by the front camera 19 and a template for image recognition corresponding to the landmark object candidate to be detected (the landmark object candidate for which information has been acquired in S5). Read from a storage medium such as the unit 12. Note that a template for image recognition is prepared in advance for each type of landmark object (for example, “Convenience Store XX Sign”, “Gas Station xx Sign”, etc.). Next, the CPU 41 executes image recognition processing by template matching on the captured image based on the captured image captured by the front camera 19 and the read template. And the mark target object candidate of the process target contained in a captured image is detected. Since image recognition processing by template matching is already known, detailed description is omitted.

  Subsequently, the CPU 41 detects the relative position of the landmark object candidate with respect to the host vehicle with respect to the landmark object candidate detected in the captured image. When a plurality of landmark object candidates are detected in the captured image, the relative position is detected for each of the detected landmark object candidates. Below, the detail of the detection method of the relative position of the marker object candidate with respect to the own vehicle is demonstrated. FIG. 6 is a captured image 53 obtained by capturing the vicinity of the guidance intersection 52 in the traveling direction of the vehicle 51 with the front camera 19 of the vehicle 51. A signboard 54 is installed at the guidance intersection 52 as a landmark object candidate. FIG. 7 is a schematic diagram showing the positional relationship between the host vehicle and the signboard 54.

  As shown in FIG. 6, the captured image 53 includes a signboard 54 installed around the guidance intersection 52. Accordingly, the signboard 54 is detected as a landmark object candidate. Then, the CPU 41 compares the shape data of the signboard 54 acquired in S5 with the shape of the signboard 54 in the captured image, and further uses the angle of view, lens characteristics, etc. of the front camera 19 to thereby signboard 54 for the host vehicle. The relative position of is detected.

  Specifically, as shown in FIG. 6, when the length of the signboard 54 in the captured image is h and the length of the actual signboard 54 is H, h and H From this ratio, the distance L to the signboard 54 with respect to the traveling direction of the vehicle is calculated. If the front camera 19 is a stereo camera, the distance L can be calculated only from the captured image. Further, the distance d is calculated from the ratio of the distance W / 2 from the center axis of the captured image (that is, the optical axis of the front camera 19) to the image end and the distance d from the center axis of the captured image to the signboard 54. The Note that W is a distance equivalent to the screen width of the captured image, and is uniquely specified from the angle of view of the front camera 19, lens characteristics, and the like. Then, the relative position of the signboard 54 with respect to the host vehicle is detected by the combination of the distance L and the distance d. Further, the installation position P of the landmark object candidate detected in the captured image is detected based on the absolute position based on the current position of the vehicle detected in S2.

  Further, particularly in the present embodiment, as shown in FIG. 7, the installation position P of the landmark object candidate detected by the image processing is set as the origin, and the imaging direction by the front camera 19 (camera optical axis direction γ) from the origin. A line segment 55 having a width α based on the detection error in the front-rear direction is specified. Note that the value of α varies depending on the accuracy of the camera and the like, but is 10 m, for example. When a plurality of landmark object candidates are detected in the captured image, the line segment 55 is specified for each of the detected landmark object candidates.

  Next, in S8, the CPU 41 determines whether or not at least one landmark object candidate is detected as a result of the image processing in S7.

  If it is determined that at least one landmark object candidate has been detected (S8: YES), the process proceeds to S9. On the other hand, when it is determined that the landmark object candidate cannot be detected (S8: NO), the process proceeds to S18.

  In S9, the CPU 41 stores the image recognition result of the landmark object candidate in S7 in the image recognition result DB 32. The stored image recognition results include an intersection ID for identifying a guided intersection where the vehicle has entered as shown in FIG. 3, an approach direction (entrance link) of the vehicle, a type of detected landmark object candidate, This is the installation position (line segment) of the landmark object candidate. In addition, when the image recognition result in the same direction at the same guidance intersection is already stored, it may be overwritten, or a predetermined number may be accumulated and stored.

  Next, in S10, the CPU 41 acquires a past image recognition result corresponding to the current guidance intersection from the image recognition result DB 32 (FIG. 3). Specifically, the CPU 41 extracts an image recognition result corresponding to the intersection ID of the guidance intersection ahead of the traveling direction of the vehicle from the image recognition results stored in the image recognition result DB 32. The acquired image recognition results are the approach direction (entrance link) of the vehicle, the type of landmark object candidate, and the installation position (line segment). For example, in the state where the image recognition result shown in FIG. 3 is stored in the image recognition result DB 32, when the vehicle enters the intersection with the intersection ID “10001”, the entry directions of the links “34561” to “34564” Image recognition results ("Convenience Store XX Signboard" and installation positions (x1, y1) to (x2, y2), etc.) are respectively acquired.

  Subsequently, in S11, the CPU 41 detects the landmark object detected from the current captured image based on the current image recognition result performed in S7 and the past image recognition result read from the image recognition result DB 32 in S10. It is determined whether or not there is a recognition result of a different approach direction in the past for the candidate.

  If it is determined that there is a recognition result of a different approach direction in the past for the landmark object candidate detected from the captured image this time (S11: YES), the process proceeds to S12. On the other hand, when it is determined that there is no recognition result of a different approach direction in the past for the landmark object candidate detected from the current captured image (S11: NO), the process proceeds to S16.

  In S <b> 12, the CPU 41 determines whether or not the line segments of the respective installation positions specified in a plurality of approach directions including the current approach direction intersect with respect to the landmark object candidate detected from the current captured image.

  And when it determines with the line segment of each installation position crossing (S12: YES), it transfers to S13. On the other hand, when it is determined that the line segments of the respective installation positions do not intersect (S12: NO), the process proceeds to S18 without determining the installation position of the landmark object candidate.

  In S13, the CPU 41 determines the installation position of the landmark object candidate at the intersection where the line segments of the installation positions intersect. For example, as shown in FIG. 9, the line segment 61 of the installation position of the landmark object candidate is specified by the captured image captured from the current imaging direction, and the landmark object is captured by the captured image captured in the past from a direction different from the current imaging direction. When the line segment 62 of the object candidate installation position is specified and the line segments 61 and 62 intersect, the intersection Q is determined as the installation position of the landmark object candidate. On the other hand, as shown in FIG. 10, when the line segments 61 and 62 do not intersect, it is estimated that different landmark object candidates are recognized from different directions, not the same landmark object, The installation position is not determined.

  In the example shown in FIG. 9, the case where the line segment of the installation position of the landmark object candidate is specified for each of the two approach directions is described. When the line segment of the installation position of the landmark object candidate is specified, when all the line segments intersect at one intersection, the intersection may be determined as the installation position of the landmark object candidate. Alternatively, the intersection of two arbitrarily selected line segments may be determined as the installation position of the landmark object candidate.

  Next, in S14, the CPU 41 registers the installation position of the landmark object candidate determined in S13 in association with the landmark object candidate. Specifically, the information on the installation position determined this time for the corresponding landmark object candidate among the landmark information 36 (FIG. 2) stored in the map information DB 31 is stored. As a result, it is possible to accurately specify the installation position of the landmark object candidate based on the landmark information 36. For example, it is determined whether or not the landmark object candidate can be visually recognized by a vehicle occupant without performing image recognition processing. It becomes possible to do. Therefore, it is possible to perform more appropriate guidance at the guidance intersection using the mark object that is visible to the vehicle occupant.

  Thereafter, in S15, the CPU 41 detects the landmark object candidate detected from the captured image and whose installation position is determined in S13, which is actually located in the vicinity of the guidance intersection 52 and is also visible to the passenger. And the landmark object candidate is determined as a landmark object to be used for guidance of the guidance intersection. If there are a plurality of landmark object candidates that meet the conditions, the most suitable landmark object candidate among the plurality of landmark object candidates is determined as the landmark object to be used for guidance at the guidance intersection. For example, a landmark object candidate that is on the near side of the guidance intersection and is on the exit direction side of the vehicle at the guidance intersection or a landmark object candidate that is closest to the intersection center is preferentially selected. It should be noted that the selection priority may be set in advance for each candidate mark object in the mark information 36 (FIG. 2). Thereafter, the process proceeds to S18.

  On the other hand, in S16, the CPU 41 detects the landmark object candidate detected from the current captured image based on the current image recognition result performed in S7 and the past image recognition result read from the image recognition result DB 32 in S10. It is determined whether there is a recognition result of the same approach direction in the past.

  If it is determined that there is a recognition result in the same approach direction in the past for the landmark object candidate detected from the captured image this time (S16: YES), the process proceeds to S17. On the other hand, if it is determined that there is no recognition result in the same approach direction in the past for the landmark object candidate detected from the captured image this time (S16: NO), the landmark object used for guidance is determined. Without proceeding to S18.

  Next, in S <b> 17, the CPU 41 detects the landmark object candidate detected from the current captured image even if it has traveled in the same approach direction as the current approach direction in the past and is detected at the same installation position. It is determined whether or not it has been detected. It should be noted that the same determination of the installation position is performed based on the line segment specified in S7. In addition to the case where the line segments are completely matched, the case where only a part is matched is also detected at the same installation position. Is determined.

  Then, it is determined that the landmark object candidate detected from the current captured image is detected from the captured image when traveling in the same approach direction as the current approach direction in the past and is detected at the same installation position. If it is done (S17: YES), although the accurate installation position cannot be determined, it can be determined that the landmark object candidate is a landmark object that is located around the guidance intersection 52 and is also visible to the passenger. The landmark object candidate is determined as a landmark object to be used for guidance of the guidance intersection (S15). On the other hand, when it is determined that the same installation position is not detected (S17: NO), the process proceeds to S18 without determining the mark object used for guidance.

  Thereafter, in S18, the CPU 41 determines whether or not the vehicle has reached the guidance utterance point of the guidance intersection based on the current position of the vehicle detected by the GPS 21 or the like. The guidance utterance points at the guidance intersection are, for example, 700 m before, 300 m before, and 100 m before the guidance intersection.

  And when it determines with the vehicle having arrived at the guidance utterance point of a guidance intersection (S18: YES), it transfers to S20. On the other hand, when it is determined that the vehicle has not reached the guidance utterance point of the guidance intersection (S18: NO), the process proceeds to S19.

  In S19, the CPU 41 determines whether or not a new captured image captured by the front camera 19 has been input.

  If it is determined that a new captured image captured by the front camera 19 has been input (S19: YES), the process returns to S6, and the landmark object used for guidance at the guidance intersection is not confirmed. Performs the processes after S7 on the newly input captured image. On the other hand, when it is determined that there is no input of a new captured image captured by the front camera 19 (S19: NO), the process returns to S18.

  On the other hand, in S20, the CPU 41 determines whether or not a landmark object to be used for guidance of a guidance intersection in front of the vehicle traveling direction has been determined. In addition, the mark object used for the guidance of the guidance intersection ahead of the advancing direction of a vehicle is decided in said S15.

  And when it determines with the mark target object used for the guidance of the guidance intersection ahead of the advancing direction of a vehicle having been decided (S20: YES), it transfers to S21. On the other hand, when it is determined that the landmark object used for guidance at the guidance intersection ahead of the traveling direction of the vehicle is not confirmed (S20: NO), the process proceeds to S22.

In S21, the CPU 41 performs guidance related to the guidance intersection using the landmark object candidate determined as the landmark object at the guidance intersection in S15. Specifically, guidance for specifying the exit direction of the guidance intersection of the vehicle (that is, guidance for identifying the exit road from which the vehicle exits the guidance intersection) is performed. For example, if the vehicle is approaching a guidance intersection where a right turn guidance is provided and “Convenience Store ○ Signboard” is confirmed as the landmark of the guidance intersection, the user will be marked with “Convenience Store ○ Signboard”. Instructs to turn right at the intersection. Specifically, voice guidance “300m ahead, convenience store ○ is right” is output from the speaker 16. Further, when the guidance intersection approaches within a predetermined distance (for example, 300 m) of the vehicle, an enlarged view of the vicinity of the guidance intersection including the mark object and the traveling direction at the guidance intersection of the vehicle are displayed on the liquid crystal display 15. If the mark target object is displayed at the installation position determined in S13 in the enlarged intersection view, more accurate guidance is possible.
As a result, it becomes possible for the user to accurately specify the guidance intersection and the road from which the vehicle exits from the guidance intersection.

  On the other hand, in S22, the CPU 41 guides the guidance intersection using the distance to the guidance intersection. For example, voice guidance “300 m ahead is in the right direction” is output from the speaker 16.

As described above in detail, according to the navigation device 1 according to the present embodiment, the travel guidance method using the navigation device 1 and the computer program executed by the navigation device 1, there is a guidance intersection ahead of the traveling direction of the vehicle. In addition, the vicinity of the guidance intersection included in the guidance route is imaged by the front camera 19 mounted on the vehicle, and the installation position of the landmark object included in the captured image is specified by the line segment along the imaging direction. Along with (S7), the installation position of the mark object is determined by combining each line segment specified from a plurality of captured images taken from different directions (S13), so the installation position of the mark object is accurately specified. Is possible. As a result, it is possible to prevent the guidance intersection from being guided using a mark object that cannot be visually recognized by the user, and to perform more appropriate guidance at the guidance intersection.
Further, since the intersection of the line positions of the installation positions of the mark objects identified in the plurality of captured images taken from different directions is determined as the installation position of the mark object, the depth direction (the optical axis direction of the camera) Even if there is a detection error, the installation position of the mark object can be accurately specified.
In addition, a line segment having a width based on a detection error in the front-rear direction along the imaging direction from the origin is set as the mark, with the installation position of the mark object detected by performing image processing on the captured image as a mark. Since it is specified as the installation position of the target object, even if there is a detection error in the depth direction (the optical axis direction of the camera), the installation position of the mark target object can be specified by a line segment in consideration of the detection error. .
In addition, since the installation position of the determined landmark object is registered in association with the landmark object and guidance of the guidance intersection is performed based on the registered information, by using the determined installation position of the landmark object It becomes possible to specify the visual recognition mode of the mark object from the vehicle occupant, and it is possible to guide the guidance intersection using the mark object visible from the vehicle occupant.
In addition, when it is determined that there is a mark object visible from the vehicle occupant, the guide object is guided using the mark object, and there is no mark object visible from the vehicle occupant. When the determination is made, the guidance intersection is guided using the distance to the guidance intersection, so that the vehicle traveling at the guidance intersection is appropriately guided in consideration of the visual recognition of the landmark object from the vehicle occupant. It becomes possible.

Note that the present invention is not limited to the above-described embodiment, and various improvements and modifications can be made without departing from the scope of the present invention.
For example, in the present embodiment, as shown in FIG. 8, the line segment of the installation position of the mark object is specified along the imaging direction of the front camera 19 (the optical axis direction of the front camera 19). The line segment may be specified along the direction of the target landmark object candidate (that is, a straight line connecting the current position of the vehicle and the installation position P).

  Further, the processing of S16 and S17 of the intersection guidance processing program (FIG. 4) is omitted, and when it is determined that there is no recognition result of a different approach direction (S11: NO), the process proceeds to S18. Also good.

  Further, in the present embodiment, the shape data of the mark object candidate is also stored as the mark information 36. However, if the front camera 19 is a stereo camera, the mark object candidate with respect to the subject vehicle can be used without using the shape data. Since the relative position can be detected, the shape data of the landmark object candidate is not necessary.

  In this embodiment, image recognition processing by template matching is performed as image processing for detecting a landmark object candidate from a captured image. However, the landmark object candidate is detected by other image processing (for example, feature point detection). It is good also as a structure to detect.

  In addition to the navigation device, the present invention can be applied to a device having a function of performing route guidance based on a guidance route. For example, the present invention can be applied to a mobile terminal such as a mobile phone or a smartphone, a personal computer, a tablet terminal (hereinafter referred to as a mobile terminal or the like). However, it is set as the structure connected to onboard equipment so that the captured image of the front camera 19 can be acquired. Further, the present invention can be applied to a system including a server and a mobile terminal. In that case, each step of the above-described intersection guidance processing program (FIGS. 4 and 5) may be configured to be implemented by either a server or a mobile terminal.

1 Navigation device 13 Navigation ECU
36 Placemark information 41 CPU
42 RAM
43 ROM
52 Guided intersection 53 Captured image 54 Signboard

Claims (7)

A guide route setting means for setting a guide route;
First captured image acquisition means for acquiring a first captured image obtained by capturing the guidance intersection included in the guidance route from a first direction by an imaging device mounted on a vehicle;
Second captured image acquisition means for acquiring a second captured image obtained by imaging a guidance intersection included in the guidance route from a second direction different from the first direction by an imaging device mounted on the vehicle;
Based on the first captured image acquired by the first captured image acquisition means, the installation position of the mark object to be used as a marker when guiding the guidance intersection included in the first captured image is First mark position specifying means for specifying by a line segment along one direction;
Based on the second captured image acquired by the second captured image acquisition unit, the installation position of the landmark object included in the second captured image is specified by a line segment along the second direction. 2 mark position specifying means;
Mark position determining means for determining an installation position of the mark object based on the line segment specified by the first mark position specifying means and the line segment specified by the second mark position specifying means. A travel guidance system characterized by that.   The mark position determining means determines an intersection of the line segment specified by the first mark position specifying means and the line segment specified by the second mark position specifying means as an installation position of the mark object. The travel guidance system according to claim 1. The first mark position specifying means uses an installation position of the mark object detected by performing image processing on the first captured image as an origin, and extends in the front-rear direction along the first direction from the origin. Identify the line with a width based on the detection error,
The second mark position specifying means uses an installation position of the mark object detected by performing image processing on the second captured image as an origin, and extends in the front-rear direction along the second direction from the origin. The travel guidance system according to claim 1 or 2, wherein a line segment having a width based on a detection error is specified. Information registration means for registering the installation position of the mark object determined by the mark position determining means in association with the mark object;
The travel guidance system according to any one of claims 1 to 3, further comprising: an intersection guidance unit that guides the guidance intersection based on information registered by the information registration unit. The intersection guide means
When it is determined that the landmark object visible from the vehicle occupant is present at the guidance intersection, the guidance intersection is guided using the landmark object,
The guidance intersection is guided using a distance to the guidance intersection when it is determined that the landmark object visible from a passenger of the vehicle does not exist at the guidance intersection. 4. Driving guide system according to 4 A guide route setting step for setting a guide route;
A first captured image acquisition step of acquiring a first captured image obtained by imaging a guidance intersection included in the guidance route from a first direction by an imaging device mounted on a vehicle;
A second captured image acquisition step of acquiring a second captured image obtained by imaging a guidance intersection included in the guidance route from a second direction different from the first direction by an imaging device mounted on the vehicle;
Based on the first captured image acquired in the first captured image acquisition step, an installation position of a mark object to be used as a marker when guiding the guidance intersection included in the first captured image is A first mark position specifying step that is specified by a line segment along one direction;
Based on the second captured image acquired in the second captured image acquisition step, the installation position of the landmark object included in the second captured image is specified by a line segment along the second direction. 2-marker position specifying step;
A mark position determining step for determining an installation position of the mark object based on the line segment specified by the first mark position specifying step and the line segment specified by the second mark position specifying step. A travel guide method characterized by the above. On the computer,
A guide route setting function for setting a guide route,
A first captured image acquisition function for acquiring a first captured image obtained by imaging a guidance intersection included in the guidance route from a first direction by an imaging device mounted on a vehicle;
A second captured image acquisition function for acquiring a second captured image obtained by imaging a guidance intersection included in the guidance route from a second direction different from the first direction by an imaging device mounted on the vehicle;
Based on the first captured image acquired by the first captured image acquisition function, the installation position of the mark object used as a marker when guiding the guidance intersection included in the first captured image is A first mark position specifying function that specifies a line segment along one direction;
Based on the second captured image acquired by the second captured image acquisition function, the installation position of the landmark object included in the second captured image is specified by a line segment along the second direction. 2 mark position identification function,
A mark position determining function for determining an installation position of the mark object based on the line segment specified by the first mark position specifying function and the line segment specified by the second mark position specifying function;
A computer program for executing

Images