JP2002008019A - Track recognition device and railway vehicle using the track recognition device - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To provide a track recognizing device capable of accurately recognizing information on a vehicle track. SOLUTION: A point detecting section 60 receives information on a distance image from a distance image generating section 30, information on a direction image from a rail candidate point extracting section 40, and information on left and right own vehicle running rails detected by a rail detecting section 50. Input, detects the lead portion as a straight line based on the rail candidate points in the search window set in order from the near side between the left and right vehicle running rails, and finds a point based on the intersection of this lead portion and the vehicle running rail. At the same time, the distance to the point is detected based on the distance image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a track recognizing apparatus for recognizing a track of an own vehicle using an image of a traveling direction captured by an image pickup means mounted on a vehicle running on a track, and maintenance using the track recognizing apparatus. For vehicles.

[0002]

2. Description of the Related Art Generally, in a railway vehicle, when a maintenance vehicle or the like travels on a business main line after the end of commercial operation, a "train central control (ATC / ATS)" device used for business is not used. It is necessary for the passenger to proceed while checking the path of the vehicle.

As a technique for reducing the burden on the passenger and improving safety, for example, Japanese Patent Application Laid-Open No. 2000-030054 by the present applicant discloses a technique which differs from the average value of the surrounding luminance in an image. The trajectory of the track (rail) is tracked by extracting the center point of the area or the point having the most different luminance as a track candidate point, and a point at which the adjacent rail joins or branches with the own vehicle travel rail in the plurality of extracted rails. A technology is disclosed in which such a track recognition device is applied to a railway vehicle such as a maintenance vehicle to perform vehicle control, display control of track information for passengers, and the like.

In Japanese Patent Application Laid-Open No. 2000-030055, a small area of a predetermined range is set at a point position on the own-vehicle track (own-vehicle running rail). A technology has been disclosed that detects a vertical edge and detects the open / closed state of a point by comparing these, and such a track recognition device is applied to a railway vehicle such as a maintenance vehicle to control the vehicle and to board the vehicle. There is disclosed a technique for controlling display of orbit information for a person.

[0005]

However, as disclosed in the above-mentioned Japanese Patent Application Laid-Open No. 2000-030054, the adjacent rail is tracked together with the own vehicle running rail, and the adjacent rail joins or branches with the own vehicle running rail. When recognizing a point as a point, it may be difficult to accurately recognize the point. For example, it may be difficult to capture the adjacent rails in a certain image frame in a section where the adjacent rails are relatively spaced from the own vehicle traveling rail such as near a station. In such a case, it is difficult to recognize the points with high accuracy. Also, a rail that diverges from the running rail at a distance (adjacent rail)
Since it is difficult to recognize points, it is difficult to accurately recognize points even in such a case. Furthermore, when tracking the rail based on the absolute value evaluation of the brightness with respect to the surroundings as described above, when the absolute brightness on the rail changes due to the influence of external light or the like, it may be difficult to track the rail, Even in such a case, it is difficult to accurately recognize the points.

The above-mentioned Japanese Patent Application Laid-Open No. 2000-030055
As disclosed in Japanese Unexamined Patent Application Publication No. H11-107, even when the open / close identification of a point is performed based on the edge detection of the luminance distribution, when the absolute luminance on the rail changes due to the influence of external light or the like, the edge detection of the rail is performed. It is difficult to sort the rail and the laying surface), and the accuracy of the open / close identification may be reduced.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a track recognizing device capable of accurately recognizing information on a vehicle's track and a railway vehicle using the track recognizing device.

[0008]

According to a first aspect of the present invention, there is provided a trajectory recognizing apparatus for recognizing a trajectory of an own vehicle using an image of a traveling direction of the own vehicle captured by an image pickup means. A recognition device, rail detection means for detecting a pair of left and right own vehicle running rail based on the image,
A point for searching between the left and right vehicle running rails on the image to detect a lead portion of an adjacent rail, and detecting a point where the vehicle running rail joins or branches with the adjacent rail based on the lead portion. And a detecting means.

A trajectory recognition device according to a second aspect of the present invention is the trajectory recognition device according to the first aspect, wherein rail candidate points are extracted by evaluating continuity of luminance at each pixel on the image to extract rail candidate points. Means, wherein the point detecting means detects the lead portion based on the rail candidate points.

According to a third aspect of the present invention, in the trajectory recognizing apparatus according to the second aspect, the point detecting means sets a plurality of search windows between the left and right traveling rails of the own vehicle, and It is characterized in that a straight line as a candidate for the lead portion is detected for each of the search windows based on the candidate points.

According to a fourth aspect of the present invention, in the trajectory recognition apparatus according to the third aspect, the search window is
It is characterized in that it is set at a position where it does not hang on the vehicle running rail.

The trajectory recognition device according to the fifth aspect of the present invention is the trajectory recognition device according to the third or fourth aspect,
When the detected straight line does not cross the left and right vehicle running rails, the point detecting means excludes the straight line from the lead portion.

According to a sixth aspect of the present invention, in the trajectory recognizing apparatus according to any one of the third to fifth aspects, the point detecting means may be configured such that the detected straight line is a left half of the search window. Or the straight line has an inclination close to the inclination of the own vehicle running rail on the left side, or the straight line is located in the right half of the search window and has an inclination close to the inclination of the own vehicle running rail on the right side. Is characterized in that this straight line is excluded from the lead portion.

According to a seventh aspect of the present invention, in the trajectory recognizing device according to any one of the first to seventh aspects, the point detecting means detects the self-vehicle running rail of the lead portion detected. The position of an intersection near the own vehicle among a pair of left and right intersections with respect to is set as the above point.

The trajectory recognition device according to the invention of claim 8 is the invention according to any one of claims 1 to 7, wherein an evaluation area based on the point is set on the image. An open / close discriminating means for evaluating the continuity of luminance of each pixel in the evaluation area and discriminating the open / close of the point is provided.

A trajectory recognition device according to a ninth aspect of the present invention is a trajectory recognition device for recognizing a trajectory of an own vehicle using an image of a traveling direction of the own vehicle captured by an image pickup means. Rail detecting means for detecting a pair of left and right own vehicle running rails, point detecting means for detecting a point at which the own vehicle running rail joins or branches with an adjacent rail,
Open / close identification means for setting an evaluation area based on the point on the image, evaluating the continuity of luminance in each pixel in the evaluation area, and identifying the open / close state of the point. Features.

The trajectory recognition device according to claim 10 is
The trajectory recognition device according to claim 8 or 9,
The open / close identification unit extracts a pixel representing a rail based on the evaluation of the continuity of luminance at each pixel in the evaluation area, and the pixel representing the rail is equal to or larger than a rail width in a direction crossing the own vehicle travel rail. When the points are continuous, it is determined that the point is closed.

According to another aspect of the present invention, there is provided a railway vehicle using the track recognizing apparatus according to any one of claims 1 to 10, wherein Control means for performing at least one of vehicle control and display control for passengers based on information on points recognized by the device is provided.

[0019]

Embodiments of the present invention will be described below with reference to the drawings. The drawings relate to an embodiment of the present invention. FIG. 1 is a functional block diagram showing a track recognition device mounted on a tracked vehicle, FIG. 2 is a flowchart of rail candidate point extraction processing, and FIG. Illustrated illustration, FIG.
Is an explanatory diagram showing a small area when evaluating the continuity of luminance in each search direction, FIG. 5 is a chart showing a relationship between a search direction of luminance continuity and its evaluation value, and FIG. FIG. 7 is a flowchart of a reference point detection process, FIG. 8 is an explanatory diagram showing a reference point search area in the direction image, FIGS. 9 and 10 are flowcharts of a rail tracking process, and FIG. Explanatory diagram showing a search line, FIG.
Is a flowchart of a point detection process, FIG. 13 is an explanatory diagram showing a search window, and FIG. 14 is a flowchart of an open / close identification process.

In FIG. 1, reference numeral 1 denotes a vehicle traveling on a track such as a railway, and in this embodiment, a vehicle for maintenance and inspection around the track. The vehicle 1 is equipped with a track recognizing device 2 for detecting a track on which the vehicle travels and recognizing a track on which the vehicle travels (hereinafter, referred to as a vehicle track). The trajectory recognition device 2 includes a stereo camera 10 as imaging means for imaging an object in the traveling direction of the vehicle 1 from different viewpoints,
An image capturing unit 20 that generates a pair of images (also referred to as original images) based on a signal from the stereo camera 10, and a distance image that generates a distance distribution (distance image) over the entire image by performing stereo processing on the pair of original images. A generation unit 30; a three-dimensional object detection unit 35 that detects a three-dimensional object based on a distance image; and a rail candidate point extraction unit that extracts a rail candidate point by evaluating the continuity of luminance in each pixel of one image. A rail candidate point extracting unit 40, a vehicle running rail detecting unit 50 as a rail detecting means for detecting the left and right vehicle running rails based on the rail candidate points, and an adjacent rail existing between the left and right vehicle running rails. Point detecting means for detecting a lead portion based on a rail candidate point and detecting, as a point, a point where the own vehicle traveling rail joins or branches with an adjacent rail based on the lead portion. An open / close identification for setting an evaluation area on an image based on the points and evaluating the continuity of the luminance of each pixel in the evaluation area to identify the open / closed state of the lead portion at the point And an open / close identification unit 70 as a means.

The own-vehicle running-rail detecting section 50 is sequentially connected to a reference point detecting section 51 for detecting a pair of rail reference points, which are the reference for detecting the own-vehicle running rail, based on the rail candidate points. And a rail tracking unit 52 for sequentially detecting the rail representative points and recognizing the own vehicle trajectory.

Here, the information on the own vehicle trajectory recognized by the rail tracking unit 52, the point detection unit 60, and the open / close identification unit 70 is transmitted to the control unit 100 as control means of the vehicle 1.
The control unit 100 performs various vehicle controls based on the trajectory information, displays the trajectory information to passengers, and the like.

Two CCs constituting the stereo camera 10
The D cameras 10a and 10b are monochrome or color CCD cameras synchronized with each other.
The camera 10a is a main camera that captures a reference image during stereo processing, and the other CCD camera 10b is a sub-camera that captures a comparison image during stereo processing. It is arranged to become.

The image capturing section 20 includes a CCD camera 10a,
The two-system analog signals from 10b are converted into digital image data of a predetermined luminance gradation to generate a pair of images (original images). In this case, it is assumed that the upper, lower, left and right of the image are equal to the upper, lower, left and right of the image, and the image coordinate system is such that the upper left of the image is the origin, the horizontal scanning direction is the X axis, and the vertical scanning direction is the Y axis.

The distance image generation unit 30 specifies a small area corresponding to the original image by calculating a city block distance for each small area of the reference image and the comparison image and obtaining a correlation between them. Stereo matching is performed, and a distance image is generated using distance (= parallax) of pixels generated according to the distance to the object as distance data.

The three-dimensional object detection unit 35 uses the distance image
A three-dimensional object is detected by generating a histogram in the distance direction for the strip-shaped region. The above-described generation of a distance image and detection of a three-dimensional object are described in, for example, Japanese Patent Application Laid-Open No. Hei 6-266 by the present applicant.
No. 828 discloses this in detail.

Next, the rail candidate point extracting process by the rail candidate point extracting unit 40 will be described with reference to the flowchart shown in FIG. Here, the rail candidate point extracting unit 40
Is inputted with one of a pair of images (reference image) stored in an image memory in the image capturing unit 20, and sequentially enters each pixel within a designated image range (see FIG. 3) on the image. Attention is paid to determine whether or not the pixel of interest is a rail candidate point.

In the rail candidate point extraction process, first, in step S101, a certain pixel of each pixel within the designated image range on the image is focused on, and a predetermined number of pixels continuous in the left-right direction with the current focused pixel as a center. It is checked whether or not the amount of change in the luminance of the pixel is equal to or greater than a set value.

Here, when the traveling direction of the vehicle 1 is imaged, the own vehicle traveling rail always extends substantially upward starting from the lower part of the image due to the characteristics of the rail. Further, since the curvature when the rail curves is generally small, it is unlikely that the own vehicle running rail continues in the left and right direction of the image even when the own vehicle running rail is curved. Further, a rail that merges or branches with the own vehicle travel rail gently merges or branches with the own vehicle travel rail. Therefore, it is unlikely that such a rail continues in the left-right direction of the image. Therefore, it is difficult to imagine that the rails related to the own vehicle trajectory are continuous in the left and right direction of the image. When the pixel of interest represents the rail related to the own vehicle trajectory, a pixel row that is continuous in the left and right direction of the pixel of interest crosses the rail. . In such a case, since the brightness of the imaged rail surface is clearly different from the brightness of the laying surface or the like, the change in the brightness of each pixel in the pixel row becomes large.

Therefore, in step S101, if it is determined that the amount of change in the luminance of the pixel row continuing in the left-right direction of the pixel of interest is smaller than the set value, the pixel of interest this time represents a rail related to the own vehicle trajectory. It is determined that the pixel is not a pixel, the process proceeds to step S102, and the current pixel of interest is excluded from the rail candidate points, and then the process proceeds to step S108.

On the other hand, if it is determined in step S101 that the amount of change in the luminance of a pixel row that is continuous in the left-right direction around the pixel of interest is equal to or greater than a predetermined value, the process proceeds to step S101.
Go to 103. In step S103, the continuity of luminance in a plurality of directions for the pixel of interest is evaluated. in this case,
For example, eight directions (−90 ° (90
°), -68 °, -45 °, -23 °, 0 °, 23 °,
(45 °, 68 °).

More specifically, in step S103, as shown in FIGS. 4 (a) to 4 (h), first, for example, 3 pixels continuous in each search direction around the pixel of interest (displayed in a bold frame in the figure) A small area of × 5 pixels is set. The length of the small area in the longitudinal direction is set to be larger than the rail width shown on the image.

Then, for each small area, the average value (difference value Diff) of the luminance difference from the adjacent pixels and the variance value De of the luminance are calculated.
v is calculated, and the larger one of them is set as the evaluation value of the continuity of the luminance in the search direction of the pixel of interest.

Here, assuming that the luminance of each pixel in the small area is Pm, n (m = 0 to 5, n = 0 to 2) as shown in FIG. 4, the difference value Diff is, for example,

(Equation 1) And the variance Dev is, for example,

(Equation 2) However,

(Equation 3) It is calculated by In addition, as is clear from each equation,
The evaluation value becomes smaller as the luminance continuity becomes higher.

Next, in step S104, the maximum value and the minimum value are extracted from the evaluation values of the continuity of the luminance of the target pixel in the respective search directions.

In the following steps S105 and S106, it is determined whether or not the target pixel is a rail candidate point based on the extracted maximum value and minimum value of the evaluation value.

Here, when the pixel of interest represents a rail, the continuity of luminance is increased only in the direction in which the rails are continuous, and the continuity in other directions across the rail is reduced. Therefore, when the pixel of interest is a rail candidate point, FIG.
As shown in (1), the evaluation value of the luminance continuity shows a low value only in a certain search direction, and the evaluation value in another search direction shows a high value.

When the pixel of interest represents a laid surface such as gravel, the luminance continuity is low in all directions. Therefore, as shown in FIG. The evaluation value shows a high value.

When the pixel of interest represents a laying surface such as concrete, the luminance continuity is high in all directions. Therefore, as shown in FIG. The evaluation value shows a low value.

Therefore, first, in step S105, it is checked whether or not the minimum value of the evaluation value is larger than the specified value a1, thereby checking whether or not the luminance of the target pixel has a predetermined continuity. Is determined to be larger than the specified value a1, the target pixel is determined to be a pixel indicating a gravel or the like having low continuity of luminance in all directions, and the process proceeds to step S102.

On the other hand, if it is determined in step S105 that the minimum value of the evaluation value is equal to or smaller than the specified value a1, the process proceeds to step S106, and the difference between the maximum value and the minimum value of the evaluation value is smaller than the specified value a2. Check whether or not. That is,
In step S106, it is checked whether or not the luminance continuity is specialized in a predetermined direction in the target pixel having the luminance continuity.

If it is determined in step S106 that the difference between the maximum value and the minimum value of the evaluation value is smaller than the specified value a2, the pixel of interest is a concrete or the like having high luminance continuity in all directions. And the process proceeds to step S102.

On the other hand, if it is determined in step S106 that the difference between the maximum value and the minimum value of the evaluation value is equal to or larger than the specified value a2, the pixel of interest has luminance continuity specialized in a certain direction. It is determined that it is a rail candidate point, and the process proceeds to step S107.

In the following step S107, a continuous direction of luminance at the target pixel determined to be a rail candidate point is detected. This continuous direction detection of the luminance is performed based on the evaluation value of the luminance continuity in a plurality of directions of the pixel of interest, which is a rail candidate point, and the direction in which the evaluation value is the minimum is set as the luminance continuous direction. You.

When the process proceeds from step S102 or step S107 to step S108, the process proceeds to step S1.
At 08, it is checked whether or not the next pixel of interest is within the specified image range, that is, whether or not there is a pixel that has not been determined as a rail candidate point within the specified image range. If it is determined that the target pixel is the next pixel, the process returns to step S101. On the other hand, if it is determined that there is no undetermined pixel in the designated image range, the routine exits from the routine.

By such processing, as shown in FIG. 6, the rail candidate point extracting section 40 has rail candidate point distribution information (such as an image) composed of rail candidate points having information on the continuous direction of luminance. Direction image).

Next, reference point detection processing of the own vehicle traveling rail by the reference point detection section 51 will be described with reference to the flowchart shown in FIG. The reference point detector 51 receives information about the three-dimensional object from the three-dimensional object detector 35 and information about the directional image from the rail candidate point extractor 40, and sets the information below the directional image (at the foot of the vehicle 1). The coordinates of the left and right vehicle running rails on the reference point search line thus detected are detected, and the detected pair of coordinates is used as a rail representative point (hereinafter referred to as a rail reference point) as a reference when tracking the vehicle running rail. ). Here, the detection of the rail reference point is performed based on the rail candidate points in the reference point search area as shown in FIG. 8, and the reference point search area is set along the reference point search line. It is.

When the reference point detection process starts, first,
In step S201, a rail candidate point corresponding to a three-dimensional object having a predetermined height (predetermined height) is removed from the rail candidate points in the reference point search area. That is, a three-dimensional object having a predetermined height with respect to the laying surface has a high possibility of a wall surface, a pillar, a traffic light, a side groove, and the like, and has a low possibility of being a rail. Therefore,
In step S201, first, a three-dimensional object having a predetermined height is extracted from the three-dimensional objects detected by the three-dimensional object detection unit 35,
By excluding the rail candidate points corresponding to the extracted three-dimensional object from the target of the reference point detection processing, the processing speed is increased and the detection accuracy is improved.

In the following step S202, it is checked whether or not the current reference point detection process is a process immediately after the system startup. If it is determined that the current process is a process immediately after the system startup, the process proceeds to step S205.

On the other hand, if it is determined in step S202 that the current process is not the process immediately after the system startup, the process proceeds to step S203, and it is determined whether or not the rail reference point has been detected in the previous process. If it is determined that the rail reference point has been detected in the previous processing, the process proceeds from step S203 to step S204. If it is determined that the rail reference point has not been detected in the previous processing, the processing proceeds from step S203 to step S203. Proceed to S205.

In step S204, after the previously detected rail reference point is set as the predicted point of the current rail reference point detection, the process proceeds to step S206.

On the other hand, in step S205, the theoretical rail coordinates on the reference point search line when the vehicle 1 is on the straight rail are set as the predicted points of the current rail reference point detection, and then the process proceeds to step S206. That is, if the current process is a process immediately after system startup or if no rail reference point is detected in the previous process, the vehicle 1
Is assumed to be on a straight rail, and the rail coordinates on the reference point search line uniquely determined from the mounting position and mounting direction of the camera are substituted as the predicted points for the current rail reference point detection.

In step S206, Hough transform processing (straight line detection processing) is performed on rail candidate points having the same continuous luminance direction and distributed linearly in the reference point search area.

In the following step S207, step S2
Based on the data of the straight line group detected in step 06, a coordinate pair as a candidate for the left and right rail reference points is obtained. Specifically, focusing on the fact that the distance between the left and right rails is constant and the left and right rails are parallel to each other in the real space, step S2
From the straight line group detected at step 06, a pair of straight lines whose distance from each other theoretically corresponds to the actual rail-to-rail distance on the image and which are parallel to each other are selected, and the selected straight line pair intersects the reference point search line. The coordinate pair is set as a candidate for the left and right rail reference points.

In the following step S208, from the coordinate pairs of the extracted rail reference point candidates, the one closest to the predicted point set in step S204 or step S205 is set as the current rail reference point, and the routine exits. .

Next, the rail tracking processing by the rail tracking section 52 will be described with reference to the flowchart shown in FIG. The rail tracking unit 51 extracts one of a pair of images (reference image) stored in an image memory in the image capturing unit 20, information on a three-dimensional object from the three-dimensional object detection unit 35, and extraction of rail candidate points. Information on the direction image from the unit 40 and information on the rail reference point from the reference point detection unit 51 are input, and based on the reference image and the direction image, the representative points of the own vehicle traveling rail are sequentially set in the distance from the rail reference point to the starting point. A rail tracking process for performing a search is performed.

Specifically, for each search line (see FIG. 11) set on the image, a rail representative point on the search line is predicted on the basis of the previously determined rail representative point on the near side. A final rail representative point is determined from a predetermined range on the search line centered on the predicted position based on three evaluation indices described later. Here, in the present embodiment, the intervals between the search lines are set to be narrower as the distance from the vehicle 1 increases.

When the rail tracking process starts, first,
In step S301, in order to increase the speed and accuracy of the rail tracking process, pixels corresponding to a three-dimensional object having a predetermined height (a predetermined height) are removed from the pixels on the image, and a rail candidate point extraction unit is used. A rail corresponding to a three-dimensional object having a predetermined height (predetermined height) is removed from the rail candidate points on the direction image extracted at 40.

In the following step S302, it is checked whether or not the search line for performing the current rail representative point search is a search line located near the vehicle 1, and it is determined that the current search line is near the vehicle 1. In this case, step S303
Proceed to.

In step S303, a rail representative point (coordinates) on the current search line is predicted based on the previously determined rail representative point immediately before. This predicted position is determined by detecting the intersection of a straight line extending in the continuous direction of the luminance of the rail representative point from the preceding rail representative point and the current search line.

On the other hand, if it is determined in step S302 that the current search line is a search line located a predetermined distance from the vehicle 1, the process proceeds to step S304. In step S304, a rail representative point on the current search line is predicted based on a few rail representative points before. The prediction of the rail representative point is performed by linearly predicting the rail position on the current search line from the rail representative points in the foreground.

Subsequent steps S305 to S309
The processing up to step S303 or step S30
A final rail representative point is evaluated and determined based on the evaluation index from a pair of left and right search areas respectively set on the search line with the predicted position of the left and right rail representative points predicted in 4 as a center. is there. As the evaluation index, an evaluation value of the continuity of the position from the near side (step S305),
Evaluation value of the continuity of the luminance pattern from the front (step S
306) and the evaluation value of the continuity of the directivity from the near side (step S307).

In step S 305, first, for each rail candidate point (hereinafter referred to as a rail search point) existing in the search area, an evaluation indicating the continuity of the position from the previously determined rail representative point on the front side is determined. As the value E0 (x), the degree of coincidence (deviation) of the predicted position with respect to the rail representative point is obtained.

Here, the evaluation value E at each rail search point
0 (x) is calculated by the following equation, for example.

[0065]

(Equation 4) Here, X: predicted position of the rail representative point on the search line x: position of the rail search point on the search line That is, the evaluation value E0 (x) is such that each rail search point is shifted from the predicted position of the rail representative point. The larger the value, the larger the value.

In the following step S306, as an evaluation value E1 (x) indicating the continuity of the luminance pattern from the front, the luminance pattern in the set area centered on the previously determined rail representative point on the front is determined. The matching degree (deviation) of the luminance pattern in the set area centered on each rail search point on the current search line is obtained.

That is, paying attention to the fact that the luminance pattern around the rail is similar at any position in the longitudinal direction of the rail, the luminance pattern in the set area centered on the preceding rail representative point and each rail search point are determined. By evaluating the degree of coincidence with the luminance pattern in the center set area, it is attempted to evaluate the possibility that each rail search point is a rail representative point. Here, in the present embodiment, in consideration of the rail width on the image,
For example, the region is set to be a 5 × 3 pixel region that is slender on both sides.

Evaluation value E 1 (x) at each rail search point
Is calculated by the following equation, for example.

[0069]

(Equation 5) Here, xr, yr: the coordinates of the rail representative point on the previous search line x, y: the coordinates of each rail search point Px, y: the luminance at each rail search point In the next step S307, the continuity of the direction from the near side Is determined by comparing the direction of each rail search point with respect to the preceding rail representative point and the continuous direction of luminance at the preceding rail representative point as the evaluation value E2 (x).

Here, the evaluation value E at each rail search point
2 (x) is calculated by the following equation, for example.

[0071]

(Equation 6) In the following step S308, comprehensive evaluation is performed based on the evaluation values E0 (x), E1 (x), and E2 (x) at each rail search point, and a final rail representative point is determined. That is, the evaluation values E0 (x), E1 (x),
An evaluation value E (x) obtained by predetermined summation of E2 (x) is obtained, and a rail search point having the smallest evaluation value E (x) is set as a final rail representative point (coordinate). However, when the evaluation value E (x) does not satisfy the condition of the predetermined threshold value, it is assumed that the rail representative point has not been detected.

Then, in step S309, it is checked whether or not there is a pixel whose luminance is lower than the pixel representing the rail representative point in an area within one pixel on the left and right of the rail representative point obtained in step S308, and represents the rail representative point. If it is determined that there is a pixel whose luminance is lower than that of the pixel, the coordinates of the rail representative point are corrected to the coordinates of the pixel having the lowest luminance, and the process proceeds to step S310.

The correction in step S309 is caused by, for example, the fact that the mirror surface of the rail is generally darker than the laying surface when the rail is imaged at night, but depending on the imaging conditions, etc. The mirror surface of the rail may appear brighter than the installation surface. Therefore, under the imaging conditions in which the mirror portion of the rail is brightly displayed on the laying surface or the like, the coordinates of the rail representative point obtained in step S308 are changed to the coordinates of the brightest pixel in the area within one pixel on either side. It may be corrected.

In step S310, step S308
In, it is checked whether only one of the left and right rail representative points has been detected. If it is determined that only one of the rail representative points has been detected, the process proceeds to step S311.

In step S311, from one of the left and right rail representative points, which has been detected, the other rail representative point is complementarily set using the left and right rail intervals and the relative difference in the rail direction. Proceed to S312.

In the following step S312, it is checked whether or not the state in which the rail representative point on the same side cannot be detected has continued for a prescribed number of times. If it is determined that the error has occurred, the process exits the routine. That is, if the rail representative point on the same side cannot be detected continuously for the specified number of times, the reliability of the subsequently detected rail representative point decreases, and the rail tracking processing ends.

On the other hand, if it is determined in step S310 that the detection of the rail representative point is not the detection of only one, the process proceeds to step S313. In step S313, both the left and right rail representative points cannot be detected in step S308. Check if there was.

If it is determined in step S313 that both the left and right rail representative points cannot be detected, the flow advances to step S314 to proceed to step S30.
3 or after performing the rail representative point complementing process using the predicted position of the rail representative point obtained in step S304, the process proceeds to step S315.

In the next step S315, it is checked whether or not the state where the left and right representative points of both rails cannot be detected has continued for a specified number of times. If it is determined that the specified number of times has not been reached, the process proceeds to step S318. If it is determined that the error has occurred, the process exits the routine. That is, if the left and right rail representative points cannot be detected continuously for the specified number of times, the reliability of the subsequently detected rail representative points decreases, and the rail tracking processing ends.

On the other hand, in step S313, it is determined that both the left and right rail representative points have been detected, and the process proceeds to step S31.
In step S316, it is determined whether or not the detected distance between the left and right rail representative points is an appropriate value for the rail distance on the image based on the rail distance in the real space. When it is determined that the interval between the representative points is an appropriate value, the process proceeds to step S318, and when it is determined that the interval between the rail representative points is not an appropriate value, the process proceeds to step S317.

In step S317, the rail representative point having the larger evaluation value E (x) among the left and right rail representative points is invalidated, and the process advances to step S311 to proceed to the rail representative point that has not been invalidated (one of the rail representative points). From the representative point), the other rail representative point is complemented by using the left and right rail intervals and the relative difference in the rail direction.

When the process proceeds from step S312, step S314, or step S315 to step S318,
In step S318, it is checked whether or not the search processing of the rail representative point has been performed up to the specified search line located far from the own vehicle. If it is determined that the search of the rail representative point has been performed up to the specified search line, , And exit the routine.

On the other hand, if it is determined in step S 318 that the search process of the rail representative point has not been performed up to the specified search line, the process returns to step S 302, and the same processing is performed for a search line one distance further from the current search line. Repeat the search process.

Next, the point detection processing by the point detection section 60 will be described with reference to the flowchart shown in FIG. The distance image stored in the image memory of the distance image generation unit 30, the information on the direction image from the rail candidate point extraction unit 40, and the information on the rail representative point from the rail tracking unit 52 are input to the point detection unit 60. Is done. Then, based on the input information, a rail candidate point group existing between the left and right vehicle running rails is extracted,
Further, this is approximated by a straight line to detect a rail lead portion of an adjacent track that branches or merges with the own vehicle track. Then, a point on the own vehicle running rail is specified based on the detected lead portion, and a distance to the point is detected based on the distance image.

When the point detection process is started, first, in step S401, a rectangular search window is set between the left and right traveling rails of the own vehicle. Here, FIGS. 13 (a) and 13 (b)
As shown in (1), the depth of the search window is set, for example, in correspondence with three rail representative points connected in front and back. Further, the width of the search window is set to be smaller than the width between the left and right rail representative points respectively located on the innermost side on the image plane among the corresponding rail representative points. As described above, the search window is set at a position where the search window does not run on the own vehicle traveling rail.

That is, even when the length of the lead portion is short, it is about 30 m (for one vehicle), so that the depth of the search window must be set to be 30 m or less when projected in real space. Therefore, in the present embodiment, 7
Since the distance between the three rail representative points connected forward and backward at a position corresponding to 0 m ahead represents a length of about 20 m in the real space, the depth of the search window corresponds to the three rail representative points connected forward and backward. Set. In addition, in order to prevent the search window from being erroneously detected as a lead portion by setting the search window at a position where the search window does not hang on the own vehicle travel rail, the width of the search window is set on the image plane among the corresponding rail representative points. Is set to be smaller than the width between the left and right rail representative points respectively located at the innermost sides.

In step S402, Hough transform processing (straight line detection processing) is performed on rail candidate points having the same luminance continuous direction and distributed linearly in the set search window, and read is performed. A straight line that is a candidate for a part is detected.

In the following step S403, step S4
It is checked whether or not a straight line is detected in the search window by the process of 02. If no straight line is detected, the process jumps to step S408.

On the other hand, it is determined that a straight line has been detected in the search window by the processing of step S402, and step S40
When the process proceeds from step 3 to step S404, the process proceeds to step S405 after calculating an equation representing the detected straight line.

In step S405, the tangent equation of the left and right vehicle running rails is calculated from the curve equation of the own vehicle running rail based on the rail representative point near the search window, and then the process proceeds to step S406.

In step S406, it is determined whether or not the detected straight line is a lead portion. Specifically, in step S406, first, the inclination of the straight line is compared with the inclination of the left and right vehicle running rails (tangents to the left and right vehicle running rails). Check if it crosses with. If the straight line does not cross the traveling rail of the own vehicle, it is determined that the straight line is not a lead portion. Further, in step S406, it is determined whether or not this straight line is located in the left half of the search window and has an inclination close to the inclination of the left own vehicle traveling rail with respect to the straight line crossing the own vehicle traveling rail, and Is located in the right half of the search window and has an inclination close to the inclination of the right vehicle running rail. And, when the straight line is located in the left half of the search window and has an inclination close to the inclination of the left own vehicle traveling rail, or the straight line is located in the right half of the search window and the inclination of the right own vehicle traveling rail. If the straight line has a close slope, it is determined that this straight line is not a lead portion.

If it is determined in step S406 that the detected straight line is not a lead portion, the process jumps to step S408. If it is determined that the detected straight line is a lead portion, the process proceeds to step S407.

In step S407, step S40
4. After the intersection of the lead portion and the own-vehicle running rail is obtained from the equation obtained in S405, the process proceeds to step S408.

Step S406 or S407
From step S408, it is checked in step S408 whether there is a next rail representative point farther than the current search window. If it is determined that there is a next rail representative point, the process returns to step S401. Here, step S40
Returning from step 8 to step S401, in step S401, the detection window is shifted farther than the previous search window as shown by the dashed line in FIG. A search window is set corresponding to the rail representative point of. In this case, the newly set search window partially overlaps the previously set search window.

On the other hand, if it is determined in step S408 that there is no next rail representative point, step S409 is performed.
In step S409, first, among the lead portions detected for each search window, the lead portions continuously detected in at least three or more different search windows as the same lead portion are extracted. Statistical processing is performed on the intersection of the extracted lead part and the own vehicle running rail calculated for each search window, and the position of the statistically processed intersection between the left and right intersections that is closer to the own vehicle is set as the point position. .

In the following step S410, the actual distance to each point position is calculated based on the distance image input from the distance image generator 30, and then the routine exits.

Next, the open / close identification processing of the lead portion at the point position by the open / close identification section 70 will be described with reference to the flowchart shown in FIG. Open / close identification unit 70
The information on one of the pair of images (reference image) stored in the image memory in the image capturing unit 20 and the information on the point from the point detection unit 60 is input, and based on the position of the point. Then, an evaluation area for performing the open / close identification of the point is set on the image. Then, the continuity of luminance at each pixel in the evaluation area is evaluated to identify the open / closed state of the point.

The open / close identification of the lead portion is sequentially performed from a point on the front side of the own vehicle. When the open / close identification process starts, first, in step S501, there is a point where the open / close state of the lead portion is not identified. Check whether or not.
If it is determined in step S501 that there is no unidentified point, the process exits the routine.

On the other hand, if it is determined in step S501 that there is an unidentified point, the process proceeds to step S502,
In order to perform open / close identification for the foremost point among unidentified points, an evaluation area centering on the above point is set.

In the following step S503, the continuity of luminance is evaluated for each pixel in the set evaluation area.
A pixel representing a rail (hereinafter, referred to as a rail display point) is extracted from each of the pixels.

Here, the extraction of the rail display points is performed by a process substantially similar to the above-described rail candidate point extraction process. That is, in the rail display point extraction processing, first, attention is paid to the pixels in the evaluation area sequentially, and the continuity of luminance in a plurality of directions with respect to the attention pixel is evaluated. At this time, in order to improve the detection accuracy of the rail display point, for example, a small area of 12 × 2 pixels is set in 36 directions with respect to the target pixel, and the evaluation value of the continuity of luminance is set for each small area in each search direction. Ask. Then, based on the minimum value and the maximum value of the evaluation value of the luminance continuity, it is checked whether or not the pixel of interest is a pixel having luminance continuity specialized in a certain direction. If the pixel has specialized brightness continuity, the pixel of interest is extracted as a rail display point.

In the following step S504, it is checked whether or not the rail display points in the evaluation area are continuous over the rail width in the direction crossing the own vehicle travel rail. And step S
If it is determined in 504 that the rail display points in the evaluation area are continuous over the rail width in the direction crossing the own-vehicle running rail, the process proceeds to step S505, at which point the lead portion is connected to the own-vehicle running rail. Is determined (the point is closed), and then step S501 is performed.
Return to

On the other hand, if it is determined in step S504 that the rail display point in the evaluation area is not continuous beyond the rail width in the direction crossing the own vehicle travel rail, the process proceeds to step S506, at which point the lead portion is moved. After it is determined that the vehicle is not connected to the vehicle running rail (the point is open), the process returns to step S501.

According to such an embodiment, the point is detected by searching between the left and right running rails and detecting the lead of the adjacent rail, so that the point detection accuracy is improved. Can be. That is, since the point detection can be performed without detecting the adjacent rail, which is difficult to detect depending on the traveling environment, imaging conditions, and the like, the point detection accuracy can be improved. Further, since it is not necessary to perform the tracking processing of the adjacent rail, the load at the time of the point detection processing can be reduced.

Further, since the detection of the lead portion is performed using the rail candidate points extracted by evaluating the continuity of the luminance in each pixel, the influence of external light or the like can be reduced, and the point detection accuracy can be improved. Can be improved.

Since the lead portion is detected on the basis of a straight line which is a candidate for a lead portion detected within a search window set at a position between the own vehicle running rails and a position not to be applied to the own vehicle running rail, the own vehicle running It is possible to prevent erroneous detection of the rail as a lead portion, and to improve the point detection accuracy.

When a straight line detected as a candidate for a lead does not cross the left and right vehicle running rails, the straight line is excluded from the lead, thereby improving the accuracy of detecting the lead. Point detection accuracy can be improved.

When the straight line detected as a candidate for the lead portion is located in the left half of the search window and has an inclination close to the inclination of the left running rail of the vehicle, or the straight line is located in the right half of the search window. In addition, when the inclination is close to the inclination of the traveling rail on the right side of the vehicle, by removing this straight line from the lead portion, the detection accuracy of the lead portion can be improved, and the detection accuracy of the point can be improved. Can be.

[0109] In addition, the open / close determination of a point is performed by setting an evaluation area based on the point and evaluating the continuity of luminance of each pixel in the evaluation area. Can be.

[0110] The open / close determination of the point is performed based on whether or not the pixel representing the rail extracted based on the evaluation of the continuity of the luminance of each pixel is continuous in the direction crossing the running rail of the vehicle or more. Therefore, it can be performed easily and in a short time.

The track recognition device 2 constructed as described above is mounted on a railway vehicle 1 such as a maintenance vehicle or the like, which runs on a business main line or the like after the end of commercial operation, thereby providing information on the recognized points. Can be used effectively and effectively.

Further, in the present embodiment, the brightness data and the distance data in the scenery in the traveling direction of the own vehicle are acquired by using the stereo camera. However, the present invention is not limited to this. The luminance data and the distance data may be respectively obtained by the combination of the radars. In this case, the detection of the three-dimensional object and the distance to the point in the description of the present embodiment are detected by the radar.

[0113]

As described above, according to the present invention, it is possible to provide a track recognizing device capable of accurately recognizing information relating to the own vehicle track and a railway vehicle using the track recognizing device.

[Brief description of the drawings]

FIG. 1 is a functional block diagram showing a track recognition device mounted on a track vehicle.

FIG. 2 is a flowchart of rail candidate point extraction processing;

FIG. 3 is an explanatory diagram showing a search area on an image.

FIG. 4 is an explanatory diagram showing a small area when evaluating the continuity of luminance in each search direction.

FIG. 5 is a table showing a relationship between a search direction of luminance continuity and its evaluation value.

FIG. 6 is an explanatory diagram showing a direction image of a rail candidate point.

FIG. 7 is a flowchart of a reference point detection process.

FIG. 8 is an explanatory diagram showing a reference point search area in a direction image.

FIG. 9 is a flowchart of a rail tracking process.

FIG. 10 is a flowchart of a rail tracking process.

FIG. 11 is an explanatory diagram showing a search line for a rail representative point.

FIG. 12 is a flowchart of a point detection process.

FIG. 13 is an explanatory diagram showing a search window.

FIG. 14 is a flowchart of an open / close identification process.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Vehicle (railroad vehicle) 2 Track recognition device 10 Stereo camera (imaging means) 40 Rail candidate point extraction part 50 Own vehicle traveling rail detection part (rail detection means) 60 Point detection part (point detection means) 70 Open / close identification part (open / close) Identification means) 100 control unit (control means)

Claims (11)

[Claims] 1. A track recognizing device for recognizing a track of a host vehicle using an image of a traveling direction of the host vehicle captured by an image pickup means, wherein a rail detection detecting a pair of right and left running rails of the host vehicle based on the image. Means for searching between the left and right host vehicle running rails on the image to detect a lead portion of an adjacent rail, and determining a point at which the host vehicle running rail joins or branches with the adjacent rail based on the lead portion. A trajectory recognizing device comprising: point detecting means for detecting. 2. A rail candidate point extracting means for evaluating a continuity of luminance in each pixel on the image to extract a rail candidate point, wherein the point detecting means comprises a lead section based on the rail candidate point. The trajectory recognition device according to claim 1, wherein the trajectory is detected. 3. The point detecting means sets a plurality of search windows between the left and right vehicle running rails, and detects a straight line which is a candidate for the lead portion for each of the search windows based on the rail candidate points. The trajectory recognition device according to claim 2, wherein the trajectory recognition is performed. 4. The trajectory recognition device according to claim 3, wherein the search window is set at a position where the search window does not hang on the own vehicle running rail. 5. The point detecting means, when the detected straight line does not cross the left and right vehicle running rails, excludes the straight line from the lead portion. An orbit recognition device according to item 1. 6. The point detection means, when the detected straight line is located in the left half of the search window and has a slope close to the slope of the left vehicle running rail, or when the straight line is in the search window. The straight line is excluded from the lead portion when the vehicle is located on the right half of the vehicle and has a gradient close to the gradient of the right vehicle running rail on the right side.
An orbit recognition device according to any one of claims 1 to 5. 7. The point detecting means sets, as the point, a position of an intersection, which is closer to the host vehicle, among a pair of left and right intersections of the lead portion with respect to the host vehicle traveling rail. The trajectory recognition device according to claim 6. 8. An open / close identification means for setting an evaluation area based on the point on the image, evaluating the continuity of luminance of each pixel in the evaluation area, and identifying the open / close of the point. The trajectory recognition device according to any one of claims 1 to 7, wherein: 9. A track recognition device for recognizing a track of a host vehicle using an image of a traveling direction of the host vehicle captured by an imaging unit, wherein a rail detection detecting a pair of left and right running rails of the host vehicle based on the image. Means, a point detecting means for detecting a point at which the own vehicle running rail joins or branches with an adjacent rail, and an evaluation area is set on the image based on the point, and luminance at each pixel in the evaluation area is set. An open / close identification means for evaluating the continuity of the points to identify the open / closed state of the point. 10. The open / close identification means extracts a pixel representing a rail based on an evaluation of the continuity of luminance at each pixel in the evaluation area, and the pixel representing the rail crosses the own vehicle travel rail. The trajectory recognition device according to claim 8 or 9, wherein the point is determined to be closed when the rails are continuous in the direction at least the rail width. 11. A railway vehicle using the track recognition device according to any one of claims 1 to 10, wherein a vehicle is controlled or a passenger is controlled based on information on points recognized by the track recognition device. A railway vehicle using a track recognition device, comprising: control means for performing at least one of display control.