JP2740265B2 - Image processing method for harvester steering control - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention relates to a method of cutting and threshing by running a traveling machine along a row of rice, wheat, soybean, etc., planted in a row in a field. A harvester such as a so-called general-purpose combine or a normal combine, or a harvester that harvests by running a traveling machine along a row of vegetables or the like planted in a field, along the row of the detection target such as the rice. The present invention relates to a method for performing image processing on image data obtained by capturing an image of an object to be traveled in accordance with the above.

[Conventional technology]

Detects the boundary between uncut and already-cut areas when the combine is automatically steered so that the combine is moved forward along the row of rice planted in the field and the row of rice is cut in multiple rows. For example, Japanese Patent Application Laid-Open No. Sho 62-12
In the publication No. 2507, the video camera mounted on the combine is disposed downward and forward so as to be able to capture images over both the uncut area and the already-cut area in the forward direction of the combine, and the image taken by the video camera A technique for extracting the boundary line by binarizing data has been proposed.

[Problems to be solved by the invention]

In this prior art, since a feature extraction process of performing a binarization process on all image data included in the entire imaging screen imaged by the video camera and then detecting a boundary line is performed, one imaging screen image is obtained. (Called one frame)
It takes a lot of time to perform the feature extraction process, and there is a problem that automatic steering cannot be performed quickly and accurately.

In addition, the boundary between the uncut area and the already-cut area cannot be detected unless an image is taken over a wide range on the right and left sides of the field to some extent.

Therefore, when capturing a wide range on the left and right sides of the field using a lens system having a predetermined field of view and capturing it on the screen, an extra target such as a ridge in a field is captured in the obtained captured screen, An extra noise component in the background portion enters, and when performing feature extraction processing such as binarization from image data, such an extra target is discriminated and removed, or the noise component in the background portion is removed. Pre-processing is complicated.

Furthermore, since the imaging method of the video camera also changes to the left and right as the combine is steered, the position at which the boundary between the uncut area and the already-cut area is also changed to the left and right. If the area to be image-processed is fixedly cut, the boundary may be lost.

When detecting this boundary line, it is necessary to determine a binarization threshold value in so-called binarization processing for discriminating the state of the uncut area and the state of the already-cut area, and a sampling area for that purpose. Is fixedly set in advance at a location that is expected to be an uncut area in the imaging screen, as the boundary line fluctuates, the sampling area is shifted to the already-cut area or the extra background portion as described above. There is a possibility that the threshold value may be digged in, and if the binarization threshold value is determined from such a sampling region, there is a problem that the subsequent determination of the boundary line cannot be performed accurately.

An object of the present invention is to solve these problems.

[Means for solving the problem]

In order to achieve this object, the present invention provides a two-dimensional imaging unit mounted on a harvester such as a combiner, at an appropriate time interval or at an appropriate forward distance between an uncut area and an already-cut area in the forward direction of travel by the harvester. The image processing method according to claim 1, wherein the image data obtained by the imaging unit is stored in an image memory, and then binarized to determine a boundary between the uncut area and the already-cut area. The area to be binarized in the imaged screen by the means is sequentially updated so as to become a window image area corresponding to a width of a predetermined distance on both the left and right sides with reference to the previously determined boundary line. The sampling area for binarization threshold value determination for judging is set to a predetermined width from the previously determined boundary line to the uncut area on the uncut area side of the window image area. How to successively updated so that the range excluding the image area equivalent to that which was adopted.

〔Example〕

Next, a description will be given of an embodiment in which the present invention is applied to a general-purpose combine. A combine 1 that is a traveling work machine includes a pair of right and left crawler-type traveling devices 3, 3 on a lower surface of a traveling machine body 2,
A steering unit 4 with a seat is disposed at a front right portion of the traveling body 2 in the forward direction of the traveling body 2, and an engine (not shown) and a tank 5 for accumulating grains are provided at the rear thereof. On the left side, a threshing unit 6 with a built-in front and rear longitudinal handling cylinder 6a and the like in which a double-pitch screw plate and an appropriate number of teeth are implanted on the outer peripheral surface, a receiving net 8 below, a sheave, etc. The apparatus includes an oscillating sorting device 9 and a wind sorting device based on the wind of Karamino Juan 10.

On the other hand, a rectangular tubular feeder house 11 communicating with the front opening of the threshing unit 6 is mounted on the front surface of the traveling machine 2 so as to be vertically movable with respect to the traveling machine via an elevating extraction cylinder 12. Inside the feeder house 11, a chain conveyor 13 in which left and right transport plates are attached to a pair of left and right chains at appropriate intervals is disposed, and the thresh conveyor 13 is used for threshing a harvested grain culm from a cutting unit 15 described later. It is transported to the unit 6.

The reference numeral 7a designates a grain from a second gutter (not shown) for the threshing unit 6
Reference numeral 7b denotes a discharge cylinder for discharging the grains in the tank 5 to the outside.

The reaping section 15 includes a bucket-shaped platform 17 extending left and right over the entire width of the traveling body 2,
17 comprises a reel 19 with a tine bar 18 provided over the entire width of the traveling body 2, and has a clipper-shaped cutting blade 20 which is also left and right on the lower surface side of the platform 17, and an arrow above the bottom plate of the platform 17. The auger 22 has a horizontally long scraping auger 22 that rotates in the direction, and drives the reel 19 and the auger 22 to rotate and drives the cutting blade 20 via a transmission unit.

Reference numeral 23 denotes a weeding body projecting forward from both the left and right ends of the reaper 15, and the reel 19 is configured to be capable of adjusting the forward and backward movement and the up and down swing according to the falling state of the grain culm and the like. I have.

Reference numeral 25 denotes a grain culm of rice, which is an object to be detected, which is planted vertically and horizontally at appropriate intervals in the field H, as shown in FIG. Combine 1
To move them forward and reap them.

Reference numeral 24 denotes imaging means such as a video camera attached to the upper end of the front part of the cabin 4a of the control unit 4, and the imaging means 24 is configured such that the optical axis 35a of the lens system 35 has a depression angle θ with respect to the horizontal plane (in the embodiment, approximately (20 degrees) and installed forward.

FIG. 4 is a block diagram of the control means of the present invention, wherein reference numeral 27 denotes an A / D converter for converting an analog input signal of the imaging means 24 into a digital signal, reference numeral 28 denotes an image memory,
29 is an image processor, 30 is a central control unit (CPU), 31 is a read-only memory (ROM), 32
Is a read / write memory (RAM), 33 is an I / 0 unit,
Reference numeral 34 indicates a bus.

The traveling body 2 can be steered left and right by a steering actuator typified by an electromagnetic solenoid such as a hydraulic switching valve driven according to a command signal from the central control device 30.

The image pickup means 24 such as a video camera has a two-dimensional image pickup device such as a CCD solid-state image pickup device arranged on a plane (image pickup screen 36) on which an image obtained through a lens system is formed. For the purpose of discriminating the boundary line between the already-cut and un-cut areas in the combine harvesting operation, the RGB color system [red (R), green (G), The primary color light of blue (B) is used, and white is obtained by adding color.]
It is preferable to execute a binarization process for determining a boundary between the already-cut area and the un-cut area from a combination of threshold values of signal outputs of color components.

When detecting the boundary Ko, the imaging means
If the entire imaging screen 36 obtained by 24 is subjected to the binarization processing, an extra target such as a ridge in the field is captured in the imaging screen 36, or an uncut area on one side at the time of the middle splitting operation. When performing feature extraction processing such as binarization of image data, such extra target components are discriminated and excluded, or preprocessing for removing noise components is performed. Is also complicated.

On the other hand, when the steering of the combine is controlled, the imaging screen 36
Since the position of the middle boundary line also changes from moment to moment, fixing the region to be binarized in the imaged screen may cause the boundary line to be lost.

Therefore, as the image processing of the present invention, a window that excludes image data corresponding to an imaging range of an appropriate width on both the left and right sides along the traveling direction of the traveling body on the detected plane 37 among the stored image data is used. In executing the processing, a window image area corresponding to the width of a predetermined distance is provided on both the left and right sides of the previously determined boundary line, which is the latest result, based on the boundary line determined for each time interval or for each advance distance as appropriate. Is sequentially updated so that

Further, since the color of the spike portion when the image of the uncut area M is normally picked up and the color of the straw after cutting in the already cut area K are different, the difference is represented by a histogram of color values of three primary colors. In the binarization process, a binarization threshold for determining that the land is the uncut area M is determined. Therefore, the sampling area for determining the binarized threshold value must be set at a location where the uncut area M exists without fail.

In other words, in sampling the image data of the uncut area M, the uncut area M near the boundary Ko may be the already-cut area K, and the obtained binary The threshold may itself be incorrect.

Therefore, in the present invention, in the window image area to be binarized, an area other than the uncut value M on the side near the boundary line Ko is set as the sampling area.

Hereinafter, an example of image processing such as window processing when the inspection plane 37 in the uncut area M of the grain culm is imaged from obliquely above will be described.

In this case, when the image obtained by the imaging unit 24 is formed on the imaging screen 36 of the two-dimensional imaging device,
Objects far from 24 appear small, objects near it appear large.

That is, as shown in FIG. 2 and FIG.
The rectangular imaging area (Ao, Bo, Co, Do) in (at a vertical lower distance h from the imaging means 24) is longer on the imaging screen 36 on the lower side thereof on the imaging screen 36 as shown in FIG. ,
It becomes a trapezoidal image (A, B, C, D) that is short on the left and right on the upper side.

Further, in a normal lens system, an image formed on the imaging screen 36 is an inverted image. However, in the following description, an upright image will be described in order to facilitate understanding.

FIG. 9 shows a flowchart of the image processing of the present invention. In step S1 following the start, the following coordinate conversion calculation is executed.

First, the coordinate relationship between the detected plane 37 and the imaging screen 36 will be described. In FIG. 8, the origin O in the orthogonal (X, Y, Z) coordinate system is set to the lens system of the focal length f in the imaging means 24. The center of the virtual lens of 35, the X axis is the imaging means
Along the left-right direction of the image pickup device 24, the Y axis is set along the vertical direction of the imaging means 24, while the Z axis is set to coincide with the optical axis 35a of the lens system 35.

Imaging screen 36 of a two-dimensional imaging device in imaging means 24
Is located at a focal length f along the Z axis from the point O in order to clearly form an image of the subject. Therefore, the imaging screen 36 is a plane parallel to the XY plane in the orthogonal (X, YZ) coordinate system.

On the other hand, the horizontal plane including the tip of the grain culm 25 is defined as a detected plane 37, and the angle of intersection between the detected plane 37 and the Z axis (corresponding to the depression angle of the imaging means) is defined as θ.

When the detected point To on the detected plane 37 is imaged by the imaging means 24, the imaging point T is a point where a straight line V connecting the To point and the O point intersects the imaged screen.

In the orthogonal (X, YZ) coordinate system, the coordinates of point T are represented by (x
o, yo, zo), the equation of the straight line V is (X-xo) / l = (Y-yo) / m = (Z-zo) / n (1) where l, m , n is the direction cosine, l = xo / L, m = yo / L, n = zo / L L = (xo) + (yo) + (zo) In these equations, from the object to be detected to the lens system Since the focal length f is extremely small as compared with the above distance, it may be set as zo ≒ f (constant).

On the other hand, when the detected plane 37 is located at a position h below the center O of the virtual lens and the depression angle of the imaging unit is θ,
In the orthogonal (X, YZ) coordinate system, the equation of the detected plane 37 can be expressed by cos θ · Y−sin θ · Z + h = 0 (2)

The coordinates (xf, yf, zf) of the To point, which is the intersection of the straight line V and the plane to be detected 37, are obtained from the above equations (1) and (2). Becomes

Next, the detected plane 37 is connected to another rectangular coordinate system (X ″, Y ″, Z ″).
In this coordinate system, the coordinates of the To point on the detected plane 37 are (xf ″, yf ″,
zf "), the coordinate transformation (by translation and rotation) Becomes

That is, the To point [coordinates (xf ″, yf ″)] of the detection target (subject) on the detection plane 37 is determined by the imaging screen 3 of the imaging unit 24.
The relationship is such that an image is formed on the point T of the coordinates (xo, yo) on 6.

According to this conversion formula (4), the coordinates (xo, y
From o), it is possible to calculate the coordinates (xf ″, yf ″) on the detected plane 37 (or vice versa).

Note that when the imaging unit 24 captures an image of the field from directly above, the above-described coordinate conversion is simplified.

As described above, since a one-to-one relationship between the coordinates on the detected plane 37 and the coordinates on the imaging screen 36 can be known, the range from the uncut area M to the already-cut area K captured on the imaging screen 36 is obtained. Among them, as a step before setting the area to be binarized, that is, the window image area W, the following calculation is executed in step S2.

That is, in the present invention, as shown in the relationship between FIG. 6 and FIG. 7, the uncut land M side with respect to the boundary Ko between the uncut ground M and the already cut ground K on the detected plane 37. To a predetermined width Ho
Up to a predetermined width Ho on the side of the already cut area K (the area surrounded by Ao, Bo, Co, Do, Eo, Fo in FIG. 6). Area (A, B, C, D,
The area surrounded by E and F) is set as the window image area W.

Accordingly, the coordinates of the left boundary of the window image area on the imaging screen 36 corresponding to the coordinates of the boundary on the uncut area M side and the imaging screen 36 corresponding to the coordinates of the boundary on the already-cut area K side.
It is necessary to smoothly calculate the coordinates of the right boundary of the window image area W above.

Therefore, when an inspection area having a predetermined width Ho (meter) (2.5 meters in the present embodiment) is set on both left and right sides of the boundary line Ko on the detected plane 37, the corresponding window image area W of the imaging screen 36 is set. Since the horizontal width is H, the straight line Ao-Bo and the straight line Fo-Eo on the detection plane 37 shown in FIG.
This corresponds to the straight lines AB and FE on the imaging screen 36 in the figure.

Therefore, the X coordinate (mask-w) as a position laterally shifted by the width H to the left and right sides from the boundary line Ko 'on the imaging screen 36.
[Y]) is previously defined as an array concept (having the same characteristics as an array,
The calculation is set and stored in a grammar unit consisting of a set of elements arranged according to a predetermined rule.

Similarly, a predetermined width So (meter) (1.0 meter in the embodiment) is set from the boundary line Ko on the detected plane 37 to the uncut area M side, and this area (Ko line and Go-Jo in FIG. 6) is set. Is excluded from the area enclosed by Ao, Bo, Co, Do, Eo, and Fo in FIG. 6, or the area is used as a sampling area for determining a binarization threshold. Imaging screen corresponding to
An X coordinate (sample-w [y]) as a position shifted leftward by a width S corresponding to the predetermined width So on 36 is calculated and set in advance based on the array concept and stored.

Next, in step S3, a window left boundary array (window-left [y] = edge-line [y] -mask-w [y] window right boundary array (window -Right [y]) = edge-line [y] + mask-w [y] and the sample right boundary array (wi
ndow-sample [y]) = sets the initial value of edge-line [y] -sample-w [y].

Here, the edge array (edge-line [y]) refers to an array of X coordinates with respect to the Y coordinate of the boundary line Ko 'on the imaging screen 36.

In this case, since the boundary Ko has not been determined at first,
There is no corresponding boundary line Ko 'on the imaging screen 36. Therefore, in the first calculation, the left and right center lines on the imaging screen 36 are set to the temporary boundary line Ko 'or the temporary edge array (edge-
line [y]) is set by the operator, and the initial value is set. In this way, since the window image area and the sampling area to be binarized are determined temporarily, the binarization processing is executed in step S4 based on the window image area. The so-called edge processing for detecting the boundary Ko with K is executed.

In this case, it is assumed that the boundary Ko 'on the image capturing screen 36 is a straight line, and the boundary Ko' is virtually determined by the so-called Hough transformation, and the edge arrangement (edge-
line [y]).

The edge array (edge-line) obtained in step S6
[Y]), in step S7, a window left border array (window-left [y] = edge-line [y] -mask-w [y] window right border array (window-right [y]) = Edge-line [y] + mask-w [y] Sample right boundary array (window-sample [y]) = edge-line [y] -sample-w [y] is updated, and the next 2 The process of performing the value conversion process and the determination of the sampling area to determine the boundary line is repeated.

[Functions and effects of the invention]

As described above, binarization processing is performed only on a part of the imaging screen, that is, a so-called window image area, as compared with binarization processing of image data obtained by imaging means mounted on the traveling body over the entire imaging screen. Therefore, it is possible to shorten the processing time, and furthermore, the processing time required to determine the boundary between the uncut area and the already-cut area.

Then, instead of fixing the window image area as a position in the imaging screen, the window image area is updated and set so as to take a predetermined width to the left and right with respect to the previously determined boundary line as the window image area. There is an effect that there is no possibility that the line will be lost even if the line is displaced in the left or right direction of the imaging screen.

Further, the sampling area for determining the binarization threshold value necessary for the binarization processing is obtained by removing an image area corresponding to a predetermined width from the boundary line toward the uncut area from the window image area. In other words, the window image area on the side near the boundary is removed from the uncut area, and the window image area on the unprovisioned area far from the boundary is used as the sampling area. ,
It is possible to obtain data for determining a binarization threshold value from image data of a location that is definitely present as an uncut area,
The boundary line can be obtained accurately.

As a result, there is an effect that the automatic steering of the agricultural work machine can be executed quickly and accurately.

[Brief description of the drawings]

1 is a plan view of the combine, FIG. 2 is a side view, FIG. 3 is a plan view showing an imaging range, FIG. 4 is a block diagram of a control device, and FIG. 5 is a window image for the imaging range in FIG. FIG. 6 is a plan view of an imaging range in the embodiment, FIG. 7 is an explanatory diagram of a window image area and a sampling area with respect to the imaging range of FIG. 6, and FIG. And FIG. 9 is a flowchart. 1 ... combine, 2 ... traveling body, 15 ... reaper, 24
... Imaging means 25 cereal stem 29 image processing processor 30 central control device 36 imaging screen 37 detected plane.

Claims (1)

(57) [Claims] 1. A two-dimensional image pickup means mounted on a harvester such as a combiner takes an image at an appropriate time interval or an appropriate advance distance over an uncut area and an already-cut area ahead of the harvester in the traveling direction. In an image processing method for storing image data obtained by an imaging unit in an image memory and then performing a binarization process to determine a boundary between the uncut area and the already-cut area, The area to be binarized is sequentially updated so as to be a window image area corresponding to a width of a predetermined distance on both the left and right sides with reference to the previously determined boundary line. The sampling region for determining the threshold value threshold is excluded from the window image area excluding the image area corresponding to a predetermined width from the previously determined boundary line to the uncut area on the uncut area. The image processing method in the steering control of the harvester, characterized in that successively updates so that enclose.