JP2010071768A - Vehicle inspection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle inspection apparatus capable of inspecting by taking a photograph of the side face of a vehicle. <P>SOLUTION: The vehicle inspection apparatus 1 is configured to include an image taking device 16 disposed along a railway track 4 for taking a wheel image of the side face of the wheel 20 of the vehicle 6 and an image storing device 30 for storing the image taken by the image taking device 16, wherein the image taking device 30 takes the wheel image based on a detection signal detecting the wheel 20 by a wheel detection sensor 26 which is installed corresponding to the image taking device 30. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle inspection apparatus that images a wheel of a train vehicle and inspects the state of the wheel.

  It is important for inspecting the wheels of a train vehicle to record the progress of wheel wear and deterioration according to the distance traveled by the vehicle in order to know when to replace the wheel parts. The inspection of the wheel is performed by stopping the vehicle in the pit. Generally, the wheel is shielded by various parts in the peripheral portion, and the entire wheel cannot be inspected by one stop. Under such circumstances, as a method for inspecting the wheel, a technique for photographing the wheel with a camera or the like while the vehicle is traveling is disclosed. For example, the wheel tread defect inspection device described in Patent Document 1 continuously captures a wheel tread with a plurality of cameras installed beside the track, and measures a defect on the wheel tread from the continuous image. At this time, dirt and defects on the wheel are detected as foreign matters and are classified based on the aspect ratio. In addition, the wheel inspection device described in Patent Document 2 is a camera installed on the side of a track, and after continuously shooting the approaching wheel tread in a slit shape in the horizontal direction, this is stacked in the vertical direction, A wheel tread for one wheel is formed and stored.

JP-A-7-174672 JP-A-5-126686

  By the way, since not only the wheel tread but also the wheel side face are scratched or broken, it is necessary to inspect the wheel side face. Conventionally, only the wheel tread has been performed for the technique of photographing and inspecting the wheel, and the side of the wheel has not been considered.

  The problem to be solved by the present invention is to provide a vehicle inspection device capable of photographing and inspecting a wheel side surface.

  In order to solve the above problems, a vehicle inspection apparatus according to the present invention is installed along a track, acquires an image of a wheel side surface of a vehicle passing through the track, and an image acquired by the image acquisition device. The image acquisition device acquires a wheel image based on a detection signal obtained by detecting a wheel by a wheel detection sensor provided in association with the image acquisition device.

  According to the present invention, the wheel side surface detected by the wheel detection sensor can be captured by the image acquisition device to acquire the wheel image, and the feature point of the wheel side surface, for example, the scratch on the wheel side surface can be observed using the wheel image. . The image acquisition device preferably uses a camera or the like, and is preferably installed on both sides of the track in order to acquire images on both sides of the wheel. The field of view of the camera or the like is set so that, for example, the field where the wheel side surface is in contact with the track is at least a third of the wheel, that is, the field of view is 120 degrees or more. Furthermore, it is preferable to provide a lighting device such as a flash. Although it is necessary to release the shutter of the camera in accordance with the approach timing of the wheel, photographing in synchronism with a wheel detection sensor that detects the approach of the wheel is described in, for example, Japanese Patent Application Laid-Open No. 64-405. Has been.

  By the way, as described above, the wheel is shielded by various parts in the peripheral portion, and an entire image of the wheel cannot be acquired by one photographing. In order to solve this problem, the wheel inspection device includes a plurality of image acquisition devices, a plurality of wheel detection sensors, a wheel counting device, and a timer for recording the passing time of the vehicle, and each of the image acquisition devices has an identification number. Is set, the wheel counting device generates a wheel number indicating the order of the wheels in the vehicle based on the detection signal, and the image acquisition device corresponds to the acquired wheel image, the identification number, the wheel number, and the passage time. In addition, it is desirable to configure so as to store in the image storage device.

  Thereby, the whole image of a wheel is acquirable by imaging a part of wheel side by each image acquisition device. Further, from the identification number, it is possible to identify which image acquisition device the wheel image was captured, and from the wheel number, it is possible to identify what number wheel of the vehicle is captured. Furthermore, if the passage time is stored, for example, a vehicle including photographed wheels can be specified with reference to a train schedule. The image acquisition devices are preferably installed at appropriate intervals along the track. The appropriate interval is determined by the number of installed image acquisition apparatuses. For example, when four image acquisition devices are installed, the interval is a quarter of the outer circumference of the wheel. Further, a panorama forming apparatus that synthesizes the entire wheel image from the acquired plurality of wheel images may be provided, and the entire wheel image and the wheel number may be stored in association with each other.

  Here, as described above, the wheel is shielded by various parts in the peripheral portion, and if the image acquisition device is installed so as to face the wheel, the wheel image cannot be obtained, so the installation angle of the image acquisition device is changed. There is a need. Normally, since the image acquisition device is installed so as to look up the wheels, the wheel image acquired by the image acquisition device is not a circle but an ellipse, and the shape of the feature point is different from the original shape, which makes it difficult to discriminate. In order to solve this problem, the image acquisition device performs projective transformation on the wheel image stored in the image storage device to generate a directly-facing image that faces the wheel, and stores the facing image in the image storage device. It is desirable to have an image processing apparatus.

  Thereby, the wheel image acquired with the image acquisition apparatus which is not facing the wheel can be made into the facing image which faces a wheel. If the parameters used for the projective transformation are obtained in advance from the positional relationship between the reference wheel and the image acquisition device, a facing image can be generated from the wheel images acquired using the parameters thereafter. The projective transformation is described in, for example, digital image processing (CG-ARTS Association). Further, parameters relating to projective transformation can be obtained by camera calibration described in three-dimensional image processing (written by Shojido and Seiji Iguchi).

  Further, the image processing device processes the facing image, extracts the contour of the wheel, estimates the wheel axle position based on the contour, performs polar coordinate conversion with the estimated axle position as the origin, and polar coordinates While generating an image, it can also comprise so that a polar coordinate image may be preserve | saved at an image preservation | save apparatus.

  For example, the distance from the axle position in the facing image to the feature point is taken in the vertical direction of the polar coordinate image, and the rotation angle around the axle position in the facing image is taken in the horizontal direction of the polar coordinate image. By this polar coordinate conversion, the increase / decrease in the distance from the axle position to the feature point is reflected in the vertical displacement, and the increase / decrease in the rotation angle around the axle position is reflected in the horizontal displacement. It becomes easy to identify. For extracting the contour of the wheel, contour enhancement processing such as a Sobel filter is performed on the facing image.

  Further, the image processing device generates a corrected image by correcting the polar coordinate image based on the magnitude of the deviation between the axle position of the reference wheel image stored in advance in the image storage device and the estimated axle position. At the same time, the corrected image can be stored in the image storage device.

  The axle position estimated from the above-mentioned facing image may differ in the vertical direction from the axle position of a reference wheel, for example, a non-wearing wheel. That is, the worn wheel has a smaller wheel radius. In addition, a difference may occur in the horizontal direction due to a slight difference in photographing timing of the image acquisition device. Therefore, if the axle position is obtained and stored from a wheel image of a wheel that has not been worn in advance, the polar coordinate image can be corrected based on the estimated deviation from the axle position. It is also possible to correct the facing image.

  Further, the vehicle inspection device extracts and extracts a wheel image of the same wheel as the wheel of the wheel image acquired by the image acquisition device by searching the image storage device based on the identification number, the wheel number, and the passage time, and the extraction The brightness pattern template of the feature point displayed in the polar coordinate image corresponding to the wheel image is created, and on the polar coordinate image generated by the image processing device is searched from the wheel image acquired by the image acquisition device based on the template. It is also possible to have a function of specifying feature points and comparing changes of feature points.

  Thereby, it becomes possible to observe the secular change of the feature point on the wheel side surface. That is, an image of the same wheel as the wheel of the wheel image newly acquired by the image acquisition device is extracted from the image storage device based on the wheel number or the like, and a feature point template associated with the extracted wheel image is used. Thus, by searching on the polar coordinate image of the newly acquired wheel image to find the same feature point and observing the size and length of the feature point, it is possible to track the aging of the feature point and the wheel deterioration status. In order to search the polar coordinate image and find the same feature point, it is preferable to use an existing pattern matching such as a normalized correlation method. The pattern matching can be performed, for example, by providing the vehicle inspection device with an information processing device. Further, it may be performed by an image processing apparatus.

  The image processing apparatus can also be configured to have a feature point detection function that binarizes with a predetermined threshold value after emphasizing the feature point region on the wheel image by filter processing.

  ADVANTAGE OF THE INVENTION According to this invention, the vehicle inspection apparatus which can image | photograph and test | inspect a wheel side surface can be provided.

  Hereinafter, the Example of the vehicle inspection apparatus 2 of this invention is described.

  FIG. 1A is a layout diagram of an inspection unit 2 constituting the vehicle inspection apparatus 1 of this embodiment. As shown in FIG. 1A, a plurality of inspection units 2 (four in this embodiment) are installed on both sides of the rails 3 and 4, and the outside and inside of the left and right wheels of the vehicle 6 are independently inspected. It is supposed to be. The arrows in the figure indicate the traveling direction of the vehicle 6. Each inspection unit 2 has a unit number. In general, since the vehicle inspection is performed before the vehicle 6 enters the garage, the inspection unit 2 is installed in front of the garage.

  FIG. 1B is a configuration diagram of the vehicle inspection device 1. The vehicle inspection apparatus 1 includes an inspection unit 2, a database 8, a panorama forming apparatus 10, and a personal computer 12, and can communicate with each other via a communication line 14. The inspection unit 2 is composed of a plurality (four in this embodiment) of image acquisition devices 16, 17, 18, and 19. Each image acquisition device has the same function, but operates independently. ing. The arrow in the figure is the direction in which the wheel 20 of the vehicle 6 rolls.

  The inspection unit 2 transmits to the database 8 a plurality of wheel images obtained by dividing and shooting the wheels 20 acquired by the image acquisition devices 16, 17, 18, and 19, and the database 8 stores them. The panorama forming apparatus 10 generates an entire wheel image from a plurality of wheel images stored in the database 8 and transmits it to the personal computer 12. The personal computer 12 includes a command input device such as a keyboard and a mouse, and a display device. The personal computer 12 accesses the database 8 and performs a pattern matching process of feature points of the wheels, which will be described later, with reference to images and measurement data related to the wheels. Further, the entire wheel image transmitted from the panorama forming apparatus 10 is displayed on the display device. The operations of the database 8, the panorama forming apparatus 10, and the personal computer 12 will be described in detail later.

  FIG. 2A is a configuration diagram of the image acquisition device 16. The image acquisition device 16 includes a camera 22, a flash 24, a wheel detection sensor 26, a wheel counting device 28, a timer 29, an image storage device 30, and an image processing device 32. The cameras 22 are installed along the rails 4 at intervals of one-fourth of the circumference of the wheel 20, and the rotation of the wheel 20 is advanced by 90 degrees with one rotation of the wheel 20 being 360 degrees. It is supposed to shoot the side. Since the perimeter differs for each wheel, the perimeter of the reference wheel 21 is arranged at a quarter interval. The reference wheel 21 has the same structure as the wheel 20 and is not worn.

  FIG. 2B is an imaging field of view of the camera 22 provided in the image acquisition device 16. As shown in FIG. 2B, the imaging field of view includes an axle 34 at the center of the wheel 20 and bolts 36 to 43 that are concentric with the axle 34 as the center. The bolts 36 to 43 have the same shape. The photographing field of view of the camera 22 has a field of view of around 60 degrees on both sides of the contact point between the wheel 20 and the rail 4. Accordingly, the camera 22 captures a range of about 120 degrees below the wheel 20 and is a region 48 indicated by a horizontal line in the rectangle 46 in FIG. Further, a quarter of the wheel 20, that is, 90 degrees is shown in the center of the screen, and is an area 49 indicated by a grid in the rectangle 46. Further, since the camera 22 is various parts attached to the wheel 20, the camera 22 does not face the wheel 20, and has an angle of view that looks up slightly from below the wheel 20. Therefore, the wheel 20 that is essentially a perfect circle is photographed in an elliptical shape that contracts in the vertical direction. The camera 22 is a digital camera that can capture images with a short exposure of 20 nanoseconds or less. The flash 24 needs a strong light quantity that allows the camera 22 to sufficiently shoot even with an exposure of 20 nanoseconds or less, and can use a flash-controlled LED. The wheel detection sensor 26 can be composed of, for example, a magnetic sensor mounted on the rail 4 or a photoelectric sensor installed in the vicinity of the rail 4, and detects that the wheel 20 has reached a predetermined position with a delay of several tens of nanoseconds. Then, a synchronization signal is output. Note that a “flow” occurs in the image because the moving wheel 20 is photographed by the camera 22, and the size of the “flow” is less than half the width of the feature point to be detected. Adjust the speed to high speed. In this embodiment, it is assumed that the wheel detection sensor 24 and the camera 20 are in the same place with respect to the rail 4.

  Here, the configuration of other devices of the image acquisition device 16 will be described together with the operation of the image acquisition device 16. First, the wheel detection sensor 26 detects that the wheel 20 has come to a predetermined position, and outputs a synchronization signal to the camera 22, the flash 24, the wheel counting device 28, and the timer 29. The camera 22 captures the wheel 20 in synchronization with the synchronization signal, and outputs the captured wheel image to the image storage device 30. The flash 24 emits light in synchronization with the synchronization signal and illuminates a part of the wheel 20. The timer 29 outputs the passage time of the vehicle 6 to the image storage device 30 in synchronization with the synchronization signal. The wheel counting device 28 can know what number of wheels 20 of the vehicle 6 was photographed by counting the number of rising times of the synchronization signal of the wheel detection sensor 26, and this order is stored in the image storage device 30 as a wheel number. Output. The image storage device 30 stores the wheel ID assigned with the unit number of the inspection unit 2 described above as a wheel ID, and stores the wheel ID, the wheel image, and the passage time in association with each other. The unit number of the inspection unit 2 can identify which wheel 20 of the vehicle 6 was shot from. In addition, the wheel counting device 28 counts the number of rises of the synchronization signal to a predetermined count value, thereby knowing the timing when the passing of one train of the vehicle 6 is completed and notifying the image processing device 32 of this. The count value is reset to prepare for the arrival of a new vehicle. The predetermined count value is the number of wheels 20 on one side of the vehicle 6. The image processing device 32 processes the wheel image stored in the image storage device 30 after receiving the notification of the completion of the passage of the vehicle 6 by the wheel counting device 28.

  Here, the processing flow of the image processing apparatus 32 will be described with reference to FIG. In step 1, a wheel image is input from the image storage device 30. Wheel ID is used as the file name of the wheel image.

  Step 2 will be described with reference to FIGS. FIG. 4A shows the positional relationship between the wheel 20 and the camera 22, and the plane 50 shows the plane of the side surface of the wheel 20 to be inspected. FIG. 4B is a diagram for explaining camera calibration to be described later, and FIG. 5 is a diagram for explaining a directly-facing image matrix to be described later. As shown in FIG. 4A, since the camera 22 does not face the wheel 20, projection conversion is performed on the wheel image photographed by the camera 22 to generate a facing image that faces the wheel 20. Since the positional relationship between the camera 22 and the wheel 20 is fixed, the parameters relating to the projective transformation can be obtained by camera calibration.

  Here, camera calibration will be described. If the camera calibration is performed at least once when the camera 22 is fixed, then projective transformation can be performed using the parameters obtained at that time. However, since the radius of the wheel changes due to wear, there is a slight difference in the vertical position of the wheel in the captured image. Further, due to the difference in the photographing timing of the camera 22, a minute difference is generated even in the horizontal position. Therefore, the camera calibration is performed by setting the reference wheel 21 that is not worn as a reference position. The structures of the wheel 20 and the reference wheel 21 are the same.

  Here, in order to obtain the camera parameters, two coordinate systems of an on-screen image matrix and wheel world coordinates are defined. The on-screen image matrix is represented by [x, y], and can be expressed in the coordinate system of a digital image obtained by photographing the wheel 20 or the reference wheel 21 with the camera 22, and the pixels are specified by the number of rows and the number of columns. To do. A rectangle 46 in FIG. 4B represents the range of the on-screen image matrix, and the upper left corner is the base point [0, 0]. The size per pixel is millimeter / pixel.

  The wheel world coordinates are represented by (X, Y), which is a coordinate system of the actual wheel 20 or the reference wheel 21 and takes a real value. The position of the axle 34 at the reference position when the wheel detection sensor 26 detects the wheel 20 or the reference wheel 21 is set as the origin. A line segment connecting the wheel 20 or the reference wheel 21 and the contact point between the rail 4 and the axle 34 is defined as a Y axis, and an axis passing through the axle 34 and parallel to the rail 4 is defined as an X axis. The unit can be determined in an MKSA unit system such as millimeters.

The camera parameters are obtained by defining the above two coordinate systems. FIG. 4B is an on-screen image matrix of the reference wheel 21. The reference wheel 21 has a reference axle 35 at the center. The marks 52, 53, 54, and 55 are marks for obtaining camera parameters, and patterns originally stamped on the reference wheel 21 may be used. The on-screen image matrices of the marks 52 to 55 are respectively [x ₁ , y ₁ ], [x ₂ , y ₂ ], [x ₃ , y ₃ ], [x ₄ , y ₄ ], and the corresponding wheel world coordinates are Let (X ₁ , Y ₁ ), (X ₂ , Y ₂ ), (X ₃ , Y ₃ ), and (X ₄ , Y ₄ ), respectively. These on-screen image matrices and wheel world coordinate values are known. Here, it is assumed that unknown camera parameters to be obtained are c ₁₁ , c ₁₂ , c ₁₃ , c ₂₁ , c ₂₂ , c ₂₃ , c ₃₁ , and c ₃₂ . However, although the wheel world coordinate is a real number, the on-screen image matrix is an integer value, so it does not exactly match, but the integer value closest to the corresponding real value is selected. In this way, the relationship of (Equation 1) is established between the on-screen image matrix [x, y] and the wheel world coordinates (X, Y).

ここで、定数Ｈを消去して整理すると（数２）が得られる。

Here, when the constant H is deleted and arranged, (Equation 2) is obtained.

これを行列で表現すると（数３）となる。

When this is expressed in a matrix, (Equation 3) is obtained.

（数３）の（Ｘ，Ｙ）と［ｘ，ｙ］は、マーク５２乃至５５の車輪ワールド座標と画面上画像マトリクスを代入して成立すべきであるから、これらを代入すると（数４）を得る。

Since (X, Y) and [x, y] in (Equation 3) should be established by substituting the wheel world coordinates of the marks 52 to 55 and the image matrix on the screen, substituting these (Equation 4) Get.

（数４）の左辺の８×８行列の逆行列を両辺に乗じることにより、カメラパラメータｃ_１１、ｃ_１２、ｃ_１３、ｃ_２１、ｃ_２２、ｃ_２３、ｃ_３１、ｃ_３２を求めることができる。また、マークを５つ以上定めた場合は、（数４）の左辺の行列は行数が列数よりも多くなる。この場合でも、左辺の行列の転置行列を両辺に乗じることで、左辺の行列を８×８行列とし、カメラパラメータｃ_１１、ｃ_１２、ｃ_１３、ｃ_２１、ｃ_２２、ｃ_２３、ｃ_３１、ｃ_３２について解くことができる。

The camera parameters c ₁₁ , c ₁₂ , c ₁₃ , c ₂₁ , c ₂₂ , c ₂₃ , c ₃₁ , c ₃₂ can be obtained by multiplying both sides by the inverse matrix of the 8 × 8 matrix on the left side of (Equation 4). it can. When five or more marks are defined, the number of rows in the matrix on the left side of (Expression 4) is greater than the number of columns. Even in this case, by multiplying both sides by the transposed matrix of the matrix on the left side, the matrix on the left side becomes an 8 × 8 matrix, and the camera parameters c ₁₁ , c ₁₂ , c ₁₃ , c ₂₁ , c ₂₂ , c ₂₃ , c ₃₁ , it can be solved for c _32.

  If the camera parameters are known by the above-described method, (Equation 2) is compiled from (Equation 5) for x and y, and the wheel world coordinates (X, Y) are converted to the on-screen image matrix [x, y]. Can be converted.

また、（数５）を車輪ワールド座標（Ｘ，Ｙ）についてまとめると、（数６）を得る。

Further, when (Equation 5) is summarized with respect to the wheel world coordinates (X, Y), (Equation 6) is obtained.

これを行列表現で表すと（数７）となる。

This can be expressed in matrix form (Equation 7).

（数７）を変形して（数８）を得る。

(Equation 7) is transformed to obtain (Equation 8).

（数８）の右辺行列の右肩の−１は逆行列を表す。逆行列はガウスの掃きだし法をはじめとして様々な解法があり、容易に求めることができる。（数８）により、画面上画像マトリクス［ｘ，ｙ］から車輪ワールド座標（Ｘ，Ｙ）への変換ができる。

-1 on the right side of the right-side matrix of (Expression 8) represents an inverse matrix. The inverse matrix has various solutions, including Gaussian sweeping, and can be easily obtained. According to (Equation 8), the on-screen image matrix [x, y] can be converted into the wheel world coordinates (X, Y).

Subsequently, with reference to FIG. 5, a facing image matrix is defined as a digital array of facing images. The face-to-face image matrix is represented as [j, i], which is a digital image of the wheel 20 and can be expressed in matrix, and the pixels are specified by the number of rows and the number of columns. The size per pixel is expressed in millimeters / pixel, and has the luminance of the j-th pixel in the vertical direction and the i-th pixel in the horizontal direction as its value. FIG. 5 is a diagram of the wheel 20 that has been directly opposed, and a rectangle 56 is a region of a directly-facing image matrix, and the upper left corner of the rectangle is a base point [0, 0]. Here, the wheel world coordinates of the four corners are respectively A (−W / 2, H ₀ ), B (W / 2, H ₀ ), C (W / 2, H ₀ + H), D (−W / 2, H ₀ + H). W, H ₀ , and H are real numbers. Furthermore, dx and dy are defined such that dx = W / M representing the horizontal size of one pixel of the directly-facing image matrix [j, i], and dy = H / N representing the vertical size. M and N are natural numbers. Both units are millimeters / pixel. In this case, the facing image matrix is a matrix of N rows and M columns, and the wheel facing image matrices at the four corners are A [0, 0], B [M-1, 0], and C [0, N-1], respectively. , D (M-1, N-1). From the above, the wheel world coordinates (X, Y) can be expressed by (Equation 9), and conversion from the directly-facing image matrix [j, i] to the wheel world coordinates (X, Y) becomes possible.

また、（数９）を書き換えた（数１０）により、車輪ワールド座標（Ｘ，Ｙ）から正対画像マトリクス［ｊ，ｉ］への変換が可能となる。

Also, by rewriting (Equation 9) from (Equation 9), it is possible to convert the wheel world coordinates (X, Y) into the directly-facing image matrix [j, i].

（数１０）において、ｊとｉは必ずしも整数にならないが、それぞれ、左辺の値に最も近い整数を選択するものとする。（数９）により、正対画像マトリクス［ｊ，ｉ］の点を車輪ワールド座標（Ｘ，Ｙ）の点に変換する。続いて車輪ワールド座標（Ｘ，Ｙ）の点を、（数５）により画面上画像マトリクス［ｘ，ｙ］の点に変換する。画面上画像マトリクス［ｘ，ｙ］は撮影された正対補正前の画像であるから対応する点の輝度を、正対画像マトリクス［ｊ，ｉ］の輝度として割り当てる。以上の処理を正対画像マトリクス［ｊ，ｉ］の全域に亘り行う。すなわち、ｊ＝０，．．．，Ｍ−１とｉ＝０，．．．，Ｎ−１に亘って行う。なお、対応する点が画面上画像マトリクス［ｘ，ｙ］の領域である矩形４６からはみ出す場合は０などの固定値を割り当てることとする。これにより正対画像マトリクス［ｊ，ｉ］は正対した車輪の部分画像となる。

In (Expression 10), j and i are not necessarily integers, but an integer closest to the value on the left side is selected. By using (Equation 9), the points of the directly-facing image matrix [j, i] are converted into the points of the wheel world coordinates (X, Y). Subsequently, the points of the wheel world coordinates (X, Y) are converted into the points of the on-screen image matrix [x, y] by (Equation 5). Since the on-screen image matrix [x, y] is a photographed image before the frontal correction, the luminance of the corresponding point is assigned as the luminance of the frontal image matrix [j, i]. The above processing is performed over the entire area of the directly-facing image matrix [j, i]. That is, j = 0,. . . , M−1 and i = 0,. . . , N−1. Note that when the corresponding point protrudes from the rectangle 46 which is the area of the on-screen image matrix [x, y], a fixed value such as 0 is assigned. As a result, the directly-facing image matrix [j, i] is a partial image of the facing wheel.

  The above is step 2 of the processing flow. That is, in step 2, camera calibration is performed with the reference wheel 21 to calculate camera parameters, the image matrix on the screen of the wheel 20 is converted into wheel world coordinates using the camera parameters, and the wheel world coordinates and the facing image matrix are converted. Is used to generate a face-to-face image matrix from the on-screen image of the wheel 20.

Next, the axle coordinate detection process in step 3 will be described with reference to FIG. FIG. 6 is a diagram illustrating a method of calculating the wheel world coordinates (g _x , g _y ) of the axle 34. As described above, the directly-facing image matrix [j, i] is a digital image of the area of the rectangle 56 in FIG. An image obtained by performing contour enhancement processing such as a Sobel filter and binarizing the digital image with a predetermined threshold is shown as a rectangle 58. Here, the larger value as the contour edge is shown in black. The horizontal line segment 60 appears as a part of the outline of the rail 4. Since this always appears at the same place by fixing the camera 22, it can be excluded from the processing target in advance. On the other hand, the contour appearing in the range 62 of the rectangle 58 is the contour of the wheel 20 and is a part of a circle. Here, let arbitrary two points on the outline which are a part of a circle be E and F, respectively. The coordinate value can be known from a rectangle 58 which is a binarized image. However, since these are the directly-facing image matrix [j, i], it is necessary to convert them into wheel world coordinates according to (Equation 9). As the wheel world coordinates converted by (Equation 9), the coordinates of E and F are set to E (e _x , e _y ) and F (f _x , f _y ), respectively. A vertical bisector G of E and F passes through the axle 34. The vertical bisector G is expressed by (Equation 11) where an arbitrary point on the vertical bisector G is (x, y).

ここで（数１１）のパラメータをα、β、γで置き換えて（数１２）とする。

Here, the parameter of (Equation 11) is replaced with α, β, and γ to obtain (Equation 12).

α、β、γはＥとＦの座標により確定するパラメータである。ここで、範囲６２の車輪２０の輪郭上の任意の２点をｋ_０組（ｋ_０は自然数）とって、これらの垂直二等分線をｋ_０本引く。このために添え字ｋを付して（数１２）を（数１３）に書き換える。ただし、ｋ＝１，２，．．，ｋ_０である。

α, β, and γ are parameters determined by the coordinates of E and F. Here, any two points on the contour of the wheel 20 in the range 62 k ₀ set (k ₀ is a natural number) taking, subtract these perpendicular bisectors k ₀ present. For this purpose, the subscript k is added and (Equation 12) is rewritten to (Equation 13). However, k = 1, 2,. . , K ₀ .

ｋ_０本の垂直二等分線は全て車軸３４で交わるので（数１３）の連立方程式の解が、車軸２０の車輪ワールド座標になる。この連立方程式を解くために（数１３）を（数１４）で表現する。

Since all the k ₀ bisectors intersect at the axle 34, the solution of the simultaneous equations of (Equation 13) becomes the wheel world coordinates of the axle 20. In order to solve these simultaneous equations, (Equation 13) is expressed by (Equation 14).

さらに、簡単化のために行列をＵで表して（数１５）とする。

Further, for simplification, the matrix is represented by U (Equation 15).

（数１５）の両辺にＵの転置行列を乗じて（数１６）とする。

(Equation 15) is obtained by multiplying both sides of (Equation 15) by the transpose matrix of U.

Ｕ^ＴＵは２×２の行列になって、通常は逆行列が存在して（ｘ，ｙ）について解くことができる。これが車軸３４の車輪ワールド座標（ｇ_ｘ，ｇ_ｙ）である。（数１６）の解き方は逆行列を両辺に乗じてもよいし、ガウスの掃きだし法で解いてもよい。垂直二等分線が３本以上になると、車輪２０の輪郭の検出位置のずれの影響で必ずしも車軸３４で交わらないが、上述の方法により二乗誤差が最小の最適解として車軸３４が求まる。なお、車輪ワールド座標（ｇ_ｘ，ｇ_ｙ）を（数１０）によって正対画像マトリクスに変換すると、垂直方向及び水平方向に配列の順番が負の値になり得る。これは車軸３４が、矩形５６（矩形５８も同様）の範囲内にないことを示しているが問題はない。以上が処理フローのステップ３である。

U ^T U is a 2 × 2 matrix, and usually an inverse matrix exists and can be solved for (x, y). This is the wheel world coordinates (g _x , g _y ) of the axle 34. The equation (16) may be solved by multiplying both sides by an inverse matrix or by Gaussian sweeping. When there are three or more vertical bisectors, the axle 34 is not necessarily intersected by the influence of the deviation of the detected position of the contour of the wheel 20, but the axle 34 is obtained as the optimum solution with the least square error by the above method. Note that when the wheel world coordinates (g _x , g _y ) are converted into a directly-facing image matrix by (Equation 10), the arrangement order in the vertical direction and the horizontal direction can be negative values. This indicates that the axle 34 is not within the range of the rectangle 56 (the same applies to the rectangle 58), but there is no problem. The above is step 3 of the processing flow.

Next, with reference to FIG. 7, the polar coordinate image creation in step 4 will be described. FIG. 7 is a polar coordinate image obtained by converting the facing image matrix. The image processing apparatus 32 has a polar coordinate conversion table in advance, and first, polar coordinate conversion table generation will be described. The polar coordinate (t, r) is a coordinate system that takes real values, the distance r from the origin of the wheel world coordinates is the vertical axis, and the declination t from the line segment that connects the contact point between the wheel 20 and the rail 4 and the axle 34 is horizontal. Axis. The unit of the r axis is an MKSA unit system such as millimeters, and the t axis is radians or degrees. The range of the horizontal axis is −π / 3 radians to π / 3 radians, that is, −60 degrees to 60 degrees. Further, the range of the vertical axis, a range of up to _{R 0} and _{R W} that includes contacts the wheel 20 and rail 4 _{(R 0} and _{R W} is a real number). Here, the wheel world coordinates (X, Y) are converted into polar coordinates (t, r) by (Equation 17).

また、（数１８）により極座標（ｔ，ｒ）から車輪ワールド座標（Ｘ，Ｙ）へ変換される。

Also, the polar coordinate (t, r) is converted to the wheel world coordinate (X, Y) by (Equation 18).

次に、極座標画像のディジタル配列として、図８に示すような極座標画像マトリクス［ｑ，ｐ］を定義する。極座標画像マトリクス［ｑ，ｐ］は極座標系によるディジタル画像であって、行列表現が可能であり、行数と列数で画素を特定する。ここで、極座標画像マトリクス［ｑ，ｐ］の１画素の水平方向の大きさを表すｄｑ＝２π／（３Ｔ）、垂直方向の大きさを表すｄｐ＝（Ｒ_Ｗ−Ｒ_０）／Ｒなるｄｑとｄｐを定義する（Ｔ、Ｒは自然数）。単位は水平方向が“ラディアン／画素”で、垂直方向が“ミリメートル／画素”となる。極座標画像マトリクス［ｑ，ｐ］は矩形６４の左上隅を基点（０，０）とし、横方向ｑ番目で縦方向ｐ番目の画素の輝度値を有する。また、極座標画像マトリクス［ｑ，ｐ］はＲ行Ｔ列の行列となり、矩形６４の四隅の極座標画像マトリクスはそれぞれ、Ｏ[０，０]、Ｐ[Ｔ−１，０]、Ｑ[０，Ｒ−１]、Ｒ（Ｔ−１，Ｒ−１）となる。（数１９）により極座標画像マトリクス［ｑ，ｐ］から車輪極座標（ｔ，ｒ）への変換が可能である。

Next, a polar coordinate image matrix [q, p] as shown in FIG. 8 is defined as a digital array of polar coordinate images. The polar image matrix [q, p] is a digital image based on a polar coordinate system and can be expressed in a matrix, and the pixels are specified by the number of rows and the number of columns. Here, dq which represents dq = 2π / (3T) representing the size in the horizontal direction of one pixel of the polar coordinate image matrix [q, p] and dp = (R _W −R ₀ ) / R representing the size in the vertical direction. And dp are defined (T and R are natural numbers). The unit is “radian / pixel” in the horizontal direction and “millimeter / pixel” in the vertical direction. The polar coordinate image matrix [q, p] has the upper left corner of the rectangle 64 as a base point (0, 0), and has the luminance value of the pixel in the horizontal direction q and the vertical direction p. The polar coordinate image matrix [q, p] is a matrix of R rows and T columns, and the polar coordinate image matrices at the four corners of the rectangle 64 are O [0, 0], P [T-1, 0], Q [0, R-1], R (T-1, R-1). Conversion from the polar coordinate image matrix [q, p] to the wheel polar coordinate (t, r) is possible by (Equation 19).

また、（数２０）により車輪極座標（ｔ，ｒ）から極座標画像マトリクス［ｑ，ｐ］への変換が可能である。

Further, conversion from the wheel polar coordinates (t, r) to the polar coordinate image matrix [q, p] is possible by (Equation 20).

また、（数１９）と（数１７）の順序で変換を進めることで、極座標画像マトリクス［ｑ，ｐ］の矩形６４内の画素が車輪ワールド座標（Ｘ，Ｙ）に変換される。さらに、（数５）により車輪ワールド座標（Ｘ，Ｙ）から画面上画像マトリクス［ｘ，ｙ］へ変換できることから、極座標画像マトリクス［ｑ，ｐ］の画素を、画面上画像マトリクス［ｘ，ｙ］と対応付けることができる。画面上画像マトリクス［ｘ，ｙ］は撮影された車輪画像の輝度値を持ち、この輝度値が極座標画像マトリクス［ｑ，ｐ］に割り当てられる。以上の処理を矩形６４全域に亘って繰り返すことにより、極座標画像マトリクス［ｑ，ｐ］の各画素に対応する画面上画像マトリクス［ｘ，ｙ］の画素が決まり、この対応関係を極座標変換テーブルとする。なお、変換の途中で実数から整数への対応付けが発生する場合には、実数に対して最も差異の少ない整数を選ぶものとする。また、極座標画像マトリクスの所定の画素が画面上画像マトリクスの矩形６４の範囲内にない場合は０などの固定値を割り当てるものとする。極座標変換テーブルの生成は１度行って保存していればよく、以降は生成した極座標変換テーブルを用いて極座標変換すればよい。

Further, by proceeding with the conversion in the order of (Equation 19) and (Equation 17), the pixels in the rectangle 64 of the polar coordinate image matrix [q, p] are converted into wheel world coordinates (X, Y). Furthermore, since the wheel world coordinates (X, Y) can be converted from the wheel world coordinates (X, Y) to the on-screen image matrix [x, y] by (Equation 5), the pixels of the polar coordinate image matrix [q, p] are converted into the on-screen image matrix [x, y]. ] Can be associated. The on-screen image matrix [x, y] has the luminance value of the captured wheel image, and this luminance value is assigned to the polar coordinate image matrix [q, p]. By repeating the above processing over the entire area of the rectangle 64, the pixel of the on-screen image matrix [x, y] corresponding to each pixel of the polar coordinate image matrix [q, p] is determined, and this correspondence relationship is determined with the polar coordinate conversion table. To do. Note that when real number-to-integer mapping occurs during conversion, an integer having the smallest difference with respect to the real number is selected. If a predetermined pixel of the polar coordinate image matrix is not within the range of the rectangle 64 of the on-screen image matrix, a fixed value such as 0 is assigned. The polar coordinate conversion table may be generated once and stored, and thereafter, polar coordinate conversion may be performed using the generated polar coordinate conversion table.

By the way, the world coordinates of the axle 34 of the wheel 20 calculated in step 3 are (g _x , g _y ), and the corresponding number of columns in the image matrix corresponding to this is the correct number in the case of the reference wheel 21. Different from the image matrix [j ₀ , i ₀ ]. In order to express this difference, the position of the axle 34 of the wheel 20 in the facing image matrix is represented as (j ₀ + j _d , i ₀ + i _d ). When this is converted into wheel world coordinates in (Equation 9), (Equation 21) is obtained.

さらに、ｊ_０＝Ｍ／２の関係から、Ｘ＝ｊ_ｄ・ｄｘである。これをさらに（数５）で画面上画像マトリクスに変換し、［ｘ_ｄ，ｙ_ｄ］とする。一方、車軸３４の正対画像マトリクスの位置［ｊ_０，ｉ_０］を、（数５）で画面上画像マトリクスに変換し、さらに、（数９）で車輪ワールド座標に変換した結果を［ｘ_０，ｙ_０］と表す。両者を比較することにより、車輪２０と基準車輪２１との間で、画面上画像マトリクスにおいて［ｘ_ｄ−ｘ_０，ｙ_ｄ−ｙ_０］の差異が生じることが分かる。これらは一般的には非整数であるが、最も近い整数を選び［ｚ_ｘ，ｚ_ｙ］とする。そして、極座標変換テーブルで得られた画面上画像マトリクス［ｘ，ｙ］に対して［ｘ＋ｚ_ｘ，ｙ＋ｚ_ｙ］とし、この値を極座標変換テーブルの出力として用いる。以上に示した補正付極座標変換テーブルを用いて、矩形領域６４の全域に亘り対応する画面上画像マトリクス［ｘ＋ｚ_ｘ，ｙ＋ｚ_ｙ］の輝度を割り当てる。このとき、画面上画像マトリクス［ｘ＋ｚ_ｘ，ｙ＋ｚ_ｙ］が矩形４６から外れる場合には対応する輝度はないので、ゼロなどの固定値を割り当てる。以上の処理により分割撮影された車輪画像は極座標画像すなわち極座標画像マトリクスに変換される。例えば、車軸３４を中心とする同心円上に存在する特徴点６５や６６は、それぞれ、図８（ｂ）のように水平方向の特徴点６５と６６に変換される。ボルト３９，４０，４１もそれぞれ変換される。なお、特徴点６５，６６は例えば車輪３４についた傷である。また、車輪２０の輪郭は水平線分６８に変換される。以上が処理フローにおけるステップ４の処理である。

Furthermore, X = j _d · dx from the relationship of j ₀ = M / 2. This is further converted into an on-screen image matrix by (Equation 5), and set as [x _d , y _d ]. On the other hand, the position [j ₀ , i ₀ ] of the directly-facing image matrix of the axle 34 is converted into an on-screen image matrix in (Equation 5), and further converted into wheel world coordinates in (Equation 9). ₀ , y ₀ ]. By comparing the two, it can be seen that a difference of [x _d −x ₀ , y _d −y ₀ ] occurs in the on-screen image matrix between the wheel 20 and the reference wheel 21. These are generally non-integer numbers, but the nearest integer is selected as [z _x , z _y ]. Then, [x + z _x , y + z _y ] is used for the on-screen image matrix [x, y] obtained from the polar coordinate conversion table, and this value is used as the output of the polar coordinate conversion table. Using the polar coordinate conversion table with correction shown above, the brightness of the corresponding on-screen image matrix [x + z _x , y + z _y ] is assigned over the entire rectangular area 64. At this time, if the on-screen image matrix [x + z _x , y + z _y ] deviates from the rectangle 46, there is no corresponding luminance, so a fixed value such as zero is assigned. The wheel images divided and photographed by the above processing are converted into polar coordinate images, that is, polar coordinate image matrices. For example, feature points 65 and 66 existing on concentric circles centered on the axle 34 are converted into feature points 65 and 66 in the horizontal direction as shown in FIG. Bolts 39, 40 and 41 are also converted. The feature points 65 and 66 are scratches on the wheel 34, for example. Further, the contour of the wheel 20 is converted into a horizontal line segment 68. The above is the process of step 4 in the process flow.

  Next, the bolt detection process in step 5 will be described with reference to FIG. Since the polar coordinate image matrix is a digital image, it can be detected by pattern matching using normalized correlation. The bolt shape is known, and the shape of the bolt after the polar coordinate conversion is also known. As shown in FIG. 9A, the template can be subjected to contour enhancement processing such as a Sobel filter. For example, FIG. 9B is an example of a polar coordinate image matrix, and shows bolts 39, 40, and 41 that have undergone polar coordinate conversion. Although the phase of the wheel, that is, the rotation angle varies depending on the shooting timing, the bolts 39, 40, and 41 move circularly at equidistant positions from the axle, and thus move on the horizontal line after the polar coordinate conversion. Therefore, if the template of FIG. 9A is searched in the vicinity of the horizontal line, the bolts 39, 40, and 41 can be detected. However, since there is no difference in shape between the bolts, the bolt on the wheel cannot be specified. For example, when there are k bolts (k is a natural number of 2 or more) on the wheel, the specification is made with k ambiguities. Here, only a specific bolt has a different shape from the others, and if it is detected with a template corresponding to this, the bolt can be specified, so the rotation angle of the wheel is also specified uniquely. Is done. The above is the bolt detection process in step 5.

Next, the feature point detection process in step 6 will be described with reference to FIGS. FIG. 9B is an example of a polar coordinate matrix of the wheel 20 having the feature points 70. For regions other than the bolt region detected in step 5, the polar coordinate image matrix is binarized with a predetermined threshold value after the Sobel filter processing, and the presence / absence of a feature point and the position of the end point if there is a feature point Can be detected. Further, the size of the feature point can be measured from the positions of both end points. At the same time, the displacement in the horizontal direction from the position ( _qbolt , _pbolt ) of the bolt nearest to the feature point 70, in the example of FIG. 9B, the bolt 39 in the example of FIG. To do. In this way, the position of the feature point 70 on the wheel is specified based on the positional relationship with the nearest bolt. FIG. 10 is a diagram showing the location and ambiguity of feature points when the number of bolts is eight. If the shapes of the eight bolts are the same and it is not possible to determine which of the bolts 39 in FIG. 10, the feature point 70 is recorded with eight ambiguities at positions 70 to 77. However, as described above, if any one of the eight bolts has a special shape and the position of the special bolt can be specified, the position of the feature point can be uniquely determined from the order of the positions of the other bolts. Further, it is possible to eliminate ambiguity when specifying the position by imprinting a predetermined mark on a predetermined area of the wheel, detecting the mark, and using the mark as a reference. The feature point detection process in step 6 has been described above.

  In step 7, the image processing device 32 outputs various images and data obtained up to step 6 to the database 8. The image and data output to the database 8 are the wheel image, the wheel ID, the passing time, the facing image generated in Step 2, the axle position determined in Step 3, the polar image generated in Step 4, and the bolt determined in Step 5. , And the position and length of the feature point obtained in step 6. Thus, the processing of the image processing apparatus 32 shown in FIG.

  Here, FIG. 11 summarizes the five coordinate systems that have come out in the description of the processing flow of FIG. As described above, these can be converted into each other, and the conversion from the wheel world coordinates to the on-screen image matrix is performed by (Equation 5), and the inverse conversion is performed in (8). According to (Equation 9), the conversion from the directly-facing image matrix to the wheel world coordinates is performed, and the inverse transformation is performed by (Equation 10). According to (Equation 17), conversion from wheel polar coordinates to wheel world coordinates is performed, and the inverse transformation is performed by (Equation 18). The transformation from the polar coordinate image matrix to the wheel polar coordinate is performed by (Equation 19), and the inverse transformation is performed by (Equation 20). The on-screen image matrix [x, y], the directly-facing image matrix [j, i], and the polar coordinate image matrix [q, t] that can be expressed in a discrete and matrix manner are indicated by solid lines. Further, the world coordinates on the wheel (X, Y) and the wheel polar coordinates (t, r) are coordinate systems taking real values, and are indicated by broken lines. The on-screen image matrix [x, y], the directly-facing image matrix [j, i], and the polar coordinate image matrix [q, p] are digital images and can be filtered by the image processing device 32.

  From this, devices other than the image acquisition device 16 will be described. The database 8 summarizes the images and measured data from the image acquisition devices 16 to 19 for each wheel ID, and stores the passage time as an index.

  Here, the feature point pattern matching processing performed by the personal computer 12 will be described with reference to FIGS. FIG. 9C shows a template pattern 72 that is a luminance pattern of feature points cut out from the polar coordinate matrix stored in the database 8.

  First, it is assumed that the polar coordinate matrix of the wheel 20 generated by the image processing device 32 in the past is stored in the database 8. Therefore, a new wheel image of the wheel 20 is acquired, and a polar coordinate matrix FIG. 9B is generated and displayed on the personal computer 12. Then, the personal computer 12 searches the database 8 based on the wheel ID of the wheel 20 and the passage time, and extracts the polar coordinate matrix of the wheel 20 stored in the past. A template pattern 72 that is a luminance pattern of the feature point is cut out from the extracted polar matrix based on the position information of the feature point. The template pattern 72 is a circumscribed rectangle of the feature point, and is represented by Tmp [q, p], where the horizontal direction is the t axis, the vertical direction is the r axis, and the upper left corner of the circumscribed rectangle is the base point (0, 0). At this time, the height of the template, that is, the number of pixels in the vertical direction is h, and the width of the template, that is, the number of pixels in the horizontal direction is w. On the other hand, it is assumed that the luminance pattern of the polar matrix in FIG. 9B is represented by Trg [q, p]. At this time, similarity Sim [q, p] related to pattern matching based on the normalized correlation between them is expressed by (Equation 22).

過去に取得した極座標マトリクスから作成したテンプレートパタン７２と、新たに取得した極座標マトリクス上の特徴点７０の形状が極端に変化していなければ両者はｑ軸とｐ軸にシフト探査するのみで該当する位置を検出できる。また、撮影のタイミングの差異により回転角であるｑは変化するものの、車軸からの距離は不変であるからパタンマッチング処理の際にも、テンプレートパタン７２を取得した際の極座標マトリクスの配列順番［ｑ_９０，ｐ_９０］も一緒に保存しておき、ｐ_９０を固定して、類似度Ｓｉｍ［ｑ，ｐ_９０］をｑ軸方向のみに探査すればよい。これによって、過去に取得した極座標マトリクス上の特徴点と、新たに取得した極座標マトリクス上の特徴点とが、車輪２０上の同じ箇所の特徴点であるかを特定し、特徴点の長さ、形状の経年変化を観察する。

If the template pattern 72 created from the previously acquired polar coordinate matrix and the shape of the feature point 70 on the newly acquired polar coordinate matrix have not changed extremely, both are applicable only by shifting search to the q-axis and p-axis. The position can be detected. In addition, although the rotation angle q changes due to a difference in photographing timing, the distance from the axle does not change. Therefore, in the pattern matching process, the polar matrix matrix arrangement order [q ₉₀ , p ₉₀ ] may be stored together, p ₉₀ may be fixed, and the similarity Sim [q, p ₉₀ ] may be searched only in the q-axis direction. Thereby, it is specified whether the feature points on the polar coordinate matrix acquired in the past and the feature points on the newly acquired polar coordinate matrix are the same feature points on the wheel 20, the length of the feature points, Observe the aging of the shape.

  Next, processing of the panorama forming apparatus 10 will be described with reference to FIG. FIG. 12A shows a virtual display memory used for forming a panorama, and the substance is a memory in an external processing apparatus connected to the personal computer 12. FIGS. 12B to 12E are front-facing image matrices obtained by the image processing apparatuses 16 to 19, respectively. The panorama forming apparatus 10 superimposes a series of wheel partial images stored in the database 8 at a predetermined position to form an image of one entire wheel and displays it on the personal computer 12. In the panorama forming apparatus 12, the facing image matrixes shown in FIGS. 12B to 12E are respectively set in the regions 78 to 81 surrounded by the broken line and the arc of the virtual display memory in FIG. Rotate by an angle and stick together. For example, the directly-facing image matrix in FIG. 12B has a right rotation of 45 degrees, and (c) to (e) are bonded together with a rotation angle increased by 90 degrees. The rotation conversion is easily realized by a 4 × 4 matrix operation. Further, when pasting, the arcs of (b) to (e) are overlapped with the arcs of the regions 78 to 81. Since regions 78 to 81 in FIG. 12 are set based on the position of the axle 35 of the reference wheel 21 and the length of the radius, the inspection wheel 20 coincides with the radius of the reference wheel 21 and there is a slight difference in photographing timing. If the rotation angle and the arc are not aligned, the rotation angle and the amount of translational movement in the vertical and horizontal directions can be determined optimally if they are determined in advance. However, since there are actually rotational deviations Δq, vertical deviations Δj, and horizontal deviations Δi for each wheel, these deviations must be corrected and pasted together. These deviations, Δq, Δj, and Δi will be described.

  As described above, the database 8 holds a series of polar coordinate matrices indicated by the rectangle 64. These are images for one rotation of the wheel 20 and are polar coordinate matrices of four wheel images taken at intervals of π / 2 in the q-axis direction. However, in reality, a positive and negative rotational deviation Δq occurs around π / 2 due to a difference in photographing timing of the camera 20 or a slip of the wheel 20. Since the rotational deviation Δq occurs between the cameras 22, it is necessary to measure and hold them individually. However, this rotational deviation is pattern matching based on the normalized correlation shown in (Equation 22) for the overlapping regions of adjacent polar coordinate matrices, and the deviation Δq in the t-axis direction from the texture coincidence point including the feature point or the bolt position. Can be measured.

The position of the axle 34 of the wheel 20 is closer to the rail 4 than the reference axle 35 of the reference wheel 21 in the facing image matrix shown in FIG. In the directly-facing image matrix [j, i], it is displaced downward in the image, that is, in the increasing direction in the i direction. The displacement Δi in step 4 of the process flow of the image processing apparatus 32, a deviation i _d from the reference axle 34 obtained on the correction of the polar coordinate conversion table. Therefore, the Δi = _{i d.}

Further, the position of the axle 34 of the photographed wheel 20 can be displaced to the left and right in the horizontal direction compared to the position of the reference axle 34 of the reference wheel 20 due to a slight timing difference at each photographing. In the directly-facing image matrix [j, i], the image is displaced in the horizontal direction, that is, in the increasing direction or decreasing direction in the j direction. This displacement amount Δj is the deviation j _d from the axle reference obtained at the time of correction of the polar coordinate conversion table in step 4 of the processing flow of the image processing device 32. Therefore, the Δj = _{j d.} As described above, each image can be found and corrected to be pasted together.

  Through the processing described above, a panoramic image of the entire circumference of the facing wheel is created. However, there is a problem that the panoramic image obtained by the above processing does not have the same rotation phase every time it is taken, and therefore the same feature point is not displayed at the same position. Therefore, it is effective to perform rotation conversion after panorama processing so that the predetermined marker or bolt detected in step 5 of the processing flow of the image processing device 16 is always positioned at the top or bottom of the virtual display memory. . It is also possible to perform rotation conversion so that the predetermined feature point detected in step 6 of the processing flow of the image processing device 16 becomes a predetermined position.

  In the panorama forming process, the facing image matrices in FIGS. 12B to 12E each have a range of about 120 degrees, and therefore there is an overlapping area when they are pasted together. In the case of overlapping, an area of 90 ° at the center of each facing image matrix, that is, a portion displayed by a horizontal line pattern is selected. Further, since the feature points are irregularities generated on the side surfaces of the wheels, the contrast of the feature points may be better photographed in the peripheral portion than in the central portion of the facing image matrix screen due to the positional relationship between the wheel side surfaces and the flash 24. In this case, it is better to select an image with good contrast of the feature points in the peripheral part. For this reason, with respect to the overlapping part of the facing image matrix, in step 6 of the processing flow of the image processing device 16, the Sobel filter applied for feature point detection and the binary value with a predetermined threshold value are used. A face-to-face image matrix having a larger feature point area detected as a result of the conversion processing is selected. When a panorama is formed in this way, an image that makes it easier to visually recognize feature points is obtained.

  The panoramic image of the virtual display memory is displayed on the personal computer 12 in part or in whole as necessary. Depending on the display resolution, when the entire wheel is displayed, a reduced screen display such as a thinning process is performed. Further, when an arbitrary feature point being displayed is instructed by a pointing device of the personal computer 12, a directly-facing image matrix of that portion, and further an on-screen image matrix can be displayed.

  As described above in the present embodiment, the vehicle inspection apparatus 1 is installed along the track 4 and is acquired by the image acquisition apparatus 16 that acquires the wheel image of the side surface of the wheel 20 of the vehicle 6 and the image acquisition apparatus 16. An image storage device 30 for storing the image, and the image acquisition device 30 acquires a wheel image based on a detection signal detected by the wheel detection sensor 26 provided in association with the image acquisition device 30. It was configured as follows.

  Thereby, the side surface of the wheel 20 detected by the wheel detection sensor 26 can be captured by the image acquisition device 16 to acquire the wheel image, and the feature point on the side surface of the wheel 20 can be observed using the wheel image.

  Each of the inspection units 2 provided with the image acquisition device 16 includes a plurality of image acquisition devices 16 to 19, a plurality of wheel detection sensors 26, a wheel counting device 28, and a timer 29 that records the passing time of the vehicle. The wheel counting device 28 generates the wheel number indicating the order of the wheels 20 in the vehicle based on the detection signal, attaches the unit number to the wheel ID, and acquires the image acquisition device 16. The wheel image, the wheel ID, and the passage time are associated with each other and stored in the image storage device 30.

  Thereby, the whole image of the wheel 20 can be acquired by photographing a part of the wheel side surface with each image acquisition device. In addition, it is possible to identify which image acquisition device has captured the wheel image from the unit number, and it is possible to identify from which number the wheel of the vehicle 6 is captured by the wheel image. Furthermore, the passing time is preserve | saved and the vehicle 6 provided with the image | photographed wheel 20 can be specified with reference to a train schedule.

  Further, the image acquisition device 16 performs projective transformation on the wheel image stored in the image storage device 30 to generate a directly-facing image facing the wheel 20, and stores the facing image in the database 8. 32.

  Thereby, the wheel image acquired by the image acquisition device 30 that is not directly facing the wheel 20 can be made a directly facing image that faces the wheel 20. If the parameters used for the projective transformation are obtained in advance from the positional relationship between the reference wheel 21 and the camera 22 provided in the image acquisition device 16, a facing image can be generated from the wheel image acquired using the parameters thereafter. it can.

  Further, the image processing device 32 processes the facing image to extract the contour of the wheel 20, estimates the axle position of the wheel 20 based on the contour, and performs polar coordinate conversion with the estimated axle position as the origin. The polar coordinate image is generated and the polar coordinate image is stored in the database 8.

  In the present embodiment, the distance from the axle position in the facing image to the feature point is taken in the vertical direction of the polar coordinate image, and the rotation angle around the axle position in the facing image is taken in the horizontal direction of the polar coordinate image. By this polar coordinate conversion, the increase / decrease in the distance from the axle position to the feature point is reflected in the vertical displacement, and the increase / decrease in the rotation angle around the axle position is reflected in the horizontal displacement. It becomes easy to identify.

  Further, the image processing device 32 generates a corrected image by correcting the polar coordinate image based on the magnitude of the deviation between the axle position of the reference wheel image stored in the image storage device 30 in advance and the estimated axle position. In addition, the corrected image may be stored in the database 8.

  The axle position estimated from the above-described facing image may be different from the axle position of the reference wheel 21 in the vertical direction. In addition, a difference may occur in the horizontal direction due to a slight difference in photographing timing of the image acquisition device 16. Therefore, if the axle position is obtained and stored in advance from the wheel image of the reference wheel 21 or the like, the polar coordinate image can be corrected based on the estimated deviation from the axle position.

  In addition, the vehicle inspection device 1 extracts the wheel image of the same wheel 20 as the wheel 20 of the wheel image acquired by the image acquisition device 16 by searching the image storage device 30 based on the wheel ID and the passage time, A template pattern 72 of the brightness pattern of the feature point displayed in the polar coordinate image corresponding to the extracted wheel image is created, and the image processing device 32 generates the wheel image acquired by the image acquisition device 16 based on the template pattern 72. The feature point is identified by searching on the polar coordinate image, and the feature point comparison function is compared.

  Thereby, the secular change of the feature point of the side surface of the wheel 20 can be observed. That is, the image of the wheel 20 that is the same as the wheel 20 of the wheel image newly acquired by the image acquisition device 16 is extracted from the image storage device based on the wheel ID or the like, and the feature points associated with the extracted wheel image are extracted. By using a template to search on the polar coordinate image of the newly acquired wheel image, find the same feature point, and observe the size and length of the feature point, so that the aging of the feature point and the wheel deterioration status can be detected. Can track.

  The vehicle inspection apparatus 1 according to the present embodiment has been described above. However, the present invention is not limited to this embodiment, and can be applied by appropriately changing the configuration. For example, the number of image acquisition devices 16 may be changed as appropriate. The interval between the image acquisition devices 16 is preferably set according to the number.

  Moreover, although the passage time was preserve | saved when acquiring a wheel image, if the vehicle 6 can be specified, it does not need to preserve | save a passage time. In that case, you may provide vehicle number reading apparatuses, such as a manufacture number.

  Further, although the wheel image and data are stored in the database 8, they may be stored in an image storage device.

  Further, although the feature point pattern matching is performed by the personal computer 12, it may be configured to be performed by the image processing apparatus 32. Furthermore, although pattern matching is performed with feature points on the polar coordinate matrix, it can also be performed with feature points on the directly-facing image matrix.

  In addition, the feature point pattern is used as a template when forming the panorama, but the rotational deviation Δq can also be measured by collating the position of the same bolt between adjacent images using the bolt pattern. The panorama may be formed by the personal computer 12.

(A) is an arrangement | positioning place of an inspection unit, (b) is a block diagram of a vehicle inspection apparatus. (A) is a block diagram of a test | inspection unit, (b) is the on-screen image matrix of the wheel image | photographed with the camera. It is a processing flow of an image processing apparatus. (A) is a figure which shows the positional relationship of a wheel and a camera, (b) is an on-screen image matrix of the reference | standard wheel image | photographed with the camera. It is a confrontation image matrix of a wheel. It is the front image matrix of the wheel which added the outline emphasis process. It is a polar coordinate image of a wheel. (A) is a facing image matrix of the wheel which has a feature point, (b) is a polar coordinate matrix which converted the facing image matrix. (A) is a template pattern of a polar coordinate matrix of a bolt, (b) is a template pattern of a polar coordinate matrix of a feature point, and (c) is a polar coordinate matrix of a wheel having a feature point. It is a figure explaining the ambiguity at the time of specifying a feature point. It is a figure which shows the relationship between five coordinate systems and numerical formula. It is a figure explaining the process of a panorama formation apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle inspection apparatus 2 Inspection unit 4 Rail 6 Vehicle 8 Database 10 Panorama formation apparatus 12 Personal computer 16 Image acquisition device 20 Wheel 21 Reference wheel 22 Camera 24 Flash 26 Vehicle detection sensor 28 Wheel counting device 30 Image storage device 32 Image processing device 34 Axle 35 Reference axle 36, 37, 38, 39, 40, 41, 42, 43 Bolt 52, 53, 54, 55 Mark 72 Template pattern of feature points

Claims (7)

  An image acquisition device that is installed along a track and acquires a wheel image of a wheel side surface of a vehicle passing through the track; and an image storage device that stores an image acquired by the image acquisition device, A vehicle inspection device that acquires the wheel image based on a detection signal obtained by detecting the wheel by a wheel detection sensor provided in association with the image acquisition device.   The vehicle inspection device includes a plurality of the image acquisition devices, a plurality of wheel detection sensors, a wheel counting device, and a timer that records a passing time of the vehicle, and an identification number is set for each of the image acquisition devices. The wheel counting device generates a wheel number indicating the order of the wheels in the vehicle based on the detection signal, and the image acquisition device includes the acquired wheel image, the identification number, and the wheel number. The vehicle inspection device according to claim 1, wherein the image storage device stores the passage time in association with each other.   The image acquisition device performs projective transformation on a wheel image stored in the image storage device to generate a directly facing image facing the wheel, and stores the facing image in the image storing device. The vehicle inspection device according to claim 1, further comprising:   The image processing device processes the facing image to extract a contour of the wheel, estimates an axle position of the wheel based on the contour, and performs polar coordinate conversion using the estimated axle position as an origin. The vehicle inspection apparatus according to claim 3, wherein a polar coordinate image is generated and the polar coordinate image is stored in the image storage device.   The image processing device generates a corrected image by correcting the polar coordinate image based on a deviation amount between a wheel position of a reference wheel image stored in advance in the image storage device and the estimated axle position. The vehicle inspection device according to claim 4, wherein the corrected image is stored in the image storage device.   The vehicle inspection device searches and extracts the image storage device based on the identification number, the wheel number, and the passage time, based on the identification number, the wheel number, and the passage time, of the wheel image of the wheel image acquired by the image acquisition device. Then, the brightness pattern template of the feature point displayed in the polar coordinate image corresponding to the extracted wheel image is created, and the image processing device generates the wheel image acquired by the image acquisition device based on the template. The vehicle inspection apparatus according to claim 5, wherein the vehicle inspection apparatus has a function of searching the polar coordinate image to identify the feature point and comparing a change in the feature point.   The vehicle inspection according to claim 6, wherein the image processing device has a feature point detection function of binarizing with a predetermined threshold after emphasizing a feature point region on the wheel image by filter processing. apparatus.

Images