JP2010054461A - Device for generating distance image data for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for generating distance image data for a vehicle which enables continuous acquisition of a situation in front of a one's own vehicle. <P>SOLUTION: The device for generating distance image data for a vehicle comprises: a projector 5; an image intensifier 7b and a high speed camera 8; a timing controller 9; and an image processing part 10 for generating distance image data indicating the distance to an object for each pixel based on the luminance of the same pixel in a plurality of images of a different target distance obtained by the image intensifier 7b and the high speed camera 8. The image processing part 10 comprises means for performing steps S4-S6 of excluding distance image data expect the distance image data of the object based on the luminance and frequency on each of the plurality of images of a different target distance. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention belongs to the technical field of a vehicle distance image data generation device.

In Patent Literature 1, whether or not an object such as an obstacle exists at the target distance is based on an image that is projected in accordance with the timing of reflected light that is projected in front of the host vehicle and returned from the target distance. Techniques for detection are disclosed.
US Pat. No. 6,732,123

  However, in the above prior art, objects other than the target distance cannot be detected. That is, there is a problem that the situation is intermittently grasped and the situation ahead of the host vehicle cannot be grasped continuously.

  An object of the present invention is to provide a vehicular distance image data generation apparatus that can continuously grasp the situation ahead of the host vehicle.

  In order to achieve the above object, in the vehicle distance image data generation device of the present invention, a light projecting unit that projects pulsed light in front of the host vehicle at a predetermined cycle, and an imaging timing set according to the target distance. Imaging means for imaging reflected light returning from the distance, timing control means for controlling the imaging timing so that the target distance changes continuously, and a plurality of imagings with different target distances obtained by the imaging means Distance image data generation means for generating distance image data representing the distance to the object for each pixel based on the luminance of the same pixel in the image, wherein the distance image data generation means is a plurality of imaging with different target distances Filter means for excluding data other than objects from the generation of distance image data from the brightness and frequency for each image It is characterized in.

  Therefore, in this invention, the situation ahead of the own vehicle can be grasped | ascertained continuously.

  DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The best mode for realizing a vehicular distance image data generating apparatus according to the present invention will be described below with reference to the drawings.

First, the configuration will be described.
FIG. 1 is a block diagram illustrating a configuration of an obstacle detection apparatus 1 according to a first embodiment to which the vehicle distance image data generation apparatus according to the present invention is applied. A device 2, an object recognition processing unit 3, and a determination unit 4 are provided.

The distance image data generation device 2 includes a projector (projecting unit) 5, an objective lens 6, a light doubling unit 7, a high-speed camera (imaging unit) 8, a timing controller (timing control unit) 9, and image processing. Unit (distance image data generating means) 10.
The projector 5 is a near-infrared LED disposed at the front end of the vehicle, and outputs pulsed light for a predetermined light projecting time tL (for example, 5 ns) in accordance with the pulse signal output from the timing controller 9. The cycle of the pulse signal is the projection cycle tP of the projector 5, and the projection cycle tP is, for example, an interval of 1/100 s or less.
The objective lens 6 is for receiving reflected light from an object, and is disposed adjacent to the projector 5. For example, the optical system is set to have an angle of view capable of capturing a predetermined range in front of the host vehicle.

The light multiplication unit 7 includes a gate 7a and an image intensifier 7b.
The gate 7a opens and closes in response to an opening / closing command signal from the timing controller 9. Here, in Example 1, the opening time (gate time) tG of the gate 7a is set to 5 ns, which is the same as the light projection time tL. Here, the gate time tG corresponds to the imaging target width of the imaging area (target distance). The longer the gate time tG, the longer the imaging target width of the imaging area. In the first embodiment, since the gate time tG = 5 ns, the imaging target width is 1.5 m from the speed of light (about 3 × 10 ⁸ m / s) × gate time (5 ns).

The image intensifier 7b temporarily converts extremely weak light (reflected light from an object, etc.) into electrons and then electrically amplifies it, and returns it to a fluorescent image again to double the amount of light and to produce an image with contrast. It is a viewing device. Photoelectrons launched from the photocathode of the image intensifier 7b by a photoelectric phenomenon are accelerated at a high voltage of the order of kV, and are emitted into the phosphor screen on the anode side, thereby emitting fluorescence having a photon number of 100 times or more. The fluorescence generated on the phosphor screen is guided to the image sensor of the high-speed camera 8 by the fiber optic plate without being scattered while maintaining the same positional relationship.
The high-speed camera 8 captures an image emitted from the light doubling unit 7 in response to a command signal from the timing controller 9 and outputs a captured image (color image) to the image processing unit 10. In the first embodiment, a camera having a resolution of 640 × 480 (horizontal: vertical), a luminance value of 1 to 255 (256 levels), and 100 fps or more is used.

The timing controller 9 is the time from the light projection start time of the projector 5 until the gate 7a is opened so that the captured image captured by the high-speed camera 8 is the timing of the reflected light returning from the target imaging area. The imaging timing is controlled by setting a certain delay time tD and outputting an open / close command signal corresponding to the delay time. That is, the delay time tD is a value that determines the distance from the host vehicle to the imaging area (imaging target distance), and the relationship between the delay time tD and the imaging target distance is as follows.
Imaging distance = speed of light (approx. 3 x 10 ⁸ m / s) x delay time tD / 2
FIG. 2 shows a temporal relationship between the operation of the projector 5 (light projection operation) and the operation of the gate 7a (camera gate operation) when imaging one imaging area.

  The timing controller 9 increases the imaging range of the high-speed camera 8 by increasing the delay time tD by a predetermined interval (for example, 10 ns) so that the imaging area continuously moves from the front side of the vehicle to the front side. Change to the side. The timing controller 9 starts the imaging operation of the high-speed camera 8 immediately before the gate 7a is opened, and ends the imaging operation after the gate 7a is completely closed. However, in the first embodiment, the timing controller 9 sets the number of times of multiple exposure in advance according to the imaging target distance, reads this data, and performs a light projection operation with a predetermined number of times of imaging at one imaging target distance. And an opening / closing operation of the gate 7a. Therefore, when the predetermined number of times set is a plurality of times, the imaging operation is terminated after a plurality of light projection operations and the opening / closing operation of the gate 7a.

  Further, in the first embodiment, as illustrated in FIG. 3, when the imaging target distance is continuously changed from B1 → B2 → B3 →..., The imaging target distance is increased more than the imaging target width A of the imaging area. By increasing the amount (B2-B1), the increase amount of the imaging target distance is set so that a part of the imaging area changes while overlapping.

  FIG. 4 is a schematic diagram showing temporal luminance changes when the amount of increase in the imaging target distance is reduced to the limit, in other words, when imaging is performed with an infinite increase in the imaging area. By overlapping each other, the luminance value of the same pixel in a plurality of consecutive captured images gradually increases, and after the peak, the luminance value gradually decreases. In practice, the number of imaging areas is limited (1 to n), but by overlapping a part of continuous imaging areas, the temporal luminance change becomes close to the characteristics of FIG.

  The timing controller 9 outputs an image processing command signal to the image processing unit 10 when all the captured images for one frame, that is, the set predetermined range (area 1, area 2,..., Area n) are captured. To do.

  The image processing unit 10 generates distance image data representing distance information by color, luminance, and the like from one frame of captured images (imaging pixels) captured by the high-speed camera 8, and object recognition is performed on the generated distance image data. Output to the processing unit 3.

The object recognition processing unit 3 identifies an object included in the distance image data by performing image processing, for example, labeling, pattern matching, or the like on the object included in the distance image data.
Based on the relationship (distance, relative speed, etc.) between the object (person, car, sign, etc.) identified by the object recognition processing unit 3 and the host vehicle, the determination unit 4 presents information to the driver by an alarm, Determine whether vehicle control such as automatic braking is necessary.

[Distance image data generation control processing]
FIG. 5 is a flowchart illustrating the flow of the distance image data generation control process executed by the image processing unit 10 according to the first embodiment. Each step will be described below. This process is repeatedly executed at a predetermined calculation cycle.

  In step S1, the image processing unit 10 stores the luminance value data having the lowest luminance of the captured image obtained by imaging the front of the host vehicle without performing the light projection by the projector 5 for later luminance determination, and proceeds to step S2. To do. This data is used when generating the distance image data.

  In step S2, the image processing unit 10 inputs a captured image, and proceeds to step S3. In the first embodiment, the input captured image is an image captured a plurality of times with the number of multiple exposures set in advance according to the distance range to be captured.

  In step S3, the image processing unit 10 determines whether image input for one frame (area 1, area 2,..., Area n) has been completed. If YES, the process proceeds to step S4. If NO, the process proceeds to step S2.

  In step S4, the image processing unit 10 performs a luminance frequency calculation process for the nth image (for example, starting from 1 or 0). In this process, the number of pixels having the same luminance value, that is, the frequency is calculated with respect to the luminance value of each pixel of the image, for example, a 0-256 bit value. This is a frequency calculation process.

  In step S5, with respect to the relationship between the luminance and frequency of one image obtained in the process of step S4, the total frequency, that is, the luminance value corresponding to the predetermined P% of the higher luminance of the total number of pixels is obtained. Calculate and use this as a threshold. Details will be described later.

  In step S6, an exclusion process is performed so that data having a lower luminance value than the threshold value determined in the process of step S5 is not used for subsequent distance image generation.

  In step S7, a process of increasing n is performed in order to process the next image after the nth image. For example, n = n + 1.

  In step S8, it is determined whether or not the processing for one frame, that is, the processing of the n-th image for one frame has been completed. If completed, the process proceeds to step S9. If not completed, the process returns to step S4. .

  In step S9, distance image data is generated as an image with distance information depending on color and brightness, and the process proceeds to return.

Next, the operation will be described.
[Distance image data generation function]
The timing controller 9 controls the imaging timing of the high-speed camera 8 by setting the delay time tD so that the captured image captured by the high-speed camera 8 becomes the timing of the reflected light returning from the target imaging area. To do. When an object is present in the target imaging area, the time for the light emitted from the projector 5 to return from the imaging area is the time for the light to reciprocate the distance between the host vehicle and the imaging area (imaging target distance). Therefore, the delay time tD can be obtained from the imaging target distance and the speed of light.

  In the captured image of the high-speed camera 8 obtained by the above method, when an object exists in the imaging area, the luminance value data of the pixel corresponding to the position of the object is affected by the reflected light, and the luminance of other pixels Indicates a value higher than the value data. Thereby, based on the luminance value data of each pixel, the distance from the object existing in the targeted imaging area can be obtained.

  Furthermore, in the first embodiment, captured images of the imaging areas 1 to n are acquired while changing the delay time tD. Subsequently, the brightness value data of the distance before and after the same pixel position is compared, the highest brightness value data is set as the distance of the object detected by the pixel, and the data (distance with distance information of the imaging range (640 × 480)) Image data).

  Conventional distance detection methods using laser radars and stereo cameras are susceptible to rain, fog, snow, etc., and the noise level with respect to the signal level is large (the SN ratio is small), so the reliability in bad weather is low. . When using a millimeter wave radar that is not easily affected by bad weather, the reliability of distance detection is high, but it is difficult to perform object recognition (object identification) from the millimeter wave radar signal. Is required. Further, since the camera image becomes unclear during bad weather, it is difficult to perform accurate object recognition.

On the other hand, in the first embodiment, only reflected waves returning from the target imaging area are reflected in the captured image, so that the light bent due to the influence of rain, fog, snow, or the like, that is, the noise mixing level is kept low. High signal-to-noise ratio can be obtained. That is, high distance detection accuracy can be obtained regardless of bad weather or night.
And since the distance of the object detected from an image is known from the produced | generated distance image data, when performing object recognition using methods, such as pattern matching after that, the distance with an object can be grasped | ascertained instantaneously.

  Furthermore, in the first embodiment, the captured area is continuously changed to acquire a plurality of captured images, and the captured images are compared to detect the distance of each pixel. In addition, it can be grasped over a wide range. For example, even in a situation where a pedestrian has jumped between the host vehicle and the preceding vehicle, the distance between the preceding vehicle and the pedestrian can be grasped at the same time. Control can be performed.

FIG. 6 shows a situation in which four pedestrians A to D exist at different positions in front of the host vehicle, and the relationship between the distance between the host vehicle and each pedestrian is A <B <C <D. .
At this time, in the first embodiment, a part of the imaging area is overlapped so that the reflected light from one object is reflected on the pixels constituting the object of the captured image in the plurality of imaging areas in which the reflected light is continuous. For this reason, the temporal luminance change of the pixels constituting the object corresponding to each pedestrian shows a triangular characteristic that takes a peak at the position of the pedestrian as shown in FIG.

  Since the distance image data is data used for alarms and vehicle control, it is impossible to set an imaging area infinitely finely as long as a certain calculation speed is required. By making the reflected light from the plurality of captured images included, as shown in FIG. 8, the temporal luminance change of the pixel is approximated to the above characteristics, and the imaging area corresponding to the peak of the triangular portion is By setting the distance of the object in the pixel, detection accuracy can be increased.

[Brightness correction by multiple exposure]
In the distance image data generation device 2 according to the first embodiment, a plurality of captured images are acquired while gradually increasing the imaging target distance. Therefore, reflected light from an object existing in a far imaging area is reflected in a nearby imaging area. It becomes weaker than the reflected light of the object existing in the. For this reason, when the imaging time (gate time tG) is constant in each imaging area, the captured image becomes darker as the imaging area is farther.

  On the other hand, the image processing unit 10 generates the distance image data representing the distance to the object for each pixel based on the luminance of the same pixel in the plurality of captured images. If a luminance difference due to a difference occurs, an accurate comparison cannot be performed when comparing the luminance of the same pixel.

  On the other hand, in the first embodiment, the number of multiple exposures is set and stored for the imaging target distance so that the luminance peak value (maximum luminance value) of the captured image in the imaging area where the object exists becomes the target luminance. . Then, the timing controller 9 performs a plurality of sets of a light projecting operation and an opening / closing operation of the gate 7a on one image according to the set number of multiple exposures. In other words, the number of multiple exposures in the first embodiment refers to the light projecting operation of the light projector 5 performed by the control of the timing controller 9 for one image captured by the high-speed camera 8, and the light projection as described above. This means the number of repetitions of the opening / closing operation of the gate 7a (by the delay time tD) paired with the operation.

  For this reason, the luminance range width (difference between the minimum luminance value and the maximum luminance value) of each captured image with different imaging target distances is made uniform regardless of the imaging area, and the target luminance is set in advance so that good processing can be performed. (See FIG. 9).

[Improvement of distance detection accuracy]
FIG. 10 is an explanatory diagram illustrating a state in which the image processing unit of the distance image data generation device according to the first embodiment performs processing using the luminance frequency.
In the first embodiment, the distance detection accuracy in bad weather is further improved. Details will be described below.
In the distance image data generation device 2 according to the first embodiment, for example, an image obtained by reflected light from fog or the like (may be rainy) in front of an object to be detected under bad weather conditions in the presence of fog or the like. May be higher than the brightness of the image obtained from the reflected light from the object. Then, when the peak value of the luminance of the image is calculated, fog is detected as an object.

On the other hand, in the distance image data generation device 2 according to the first embodiment, the object is easily detected as a luminance peak by equalizing the luminance at the distance by the multiple exposure.
Further, a luminance frequency calculation process is performed on the captured image (step S4).
That is, the frequency with respect to the brightness is calculated for each of a plurality of images constituting one frame. Then, for example, a graph with luminance on the horizontal axis and frequency on the vertical axis as shown in FIG. 10B can be obtained. In FIG. 10B, the contour line of the daily strogram when there is no fog or the like is indicated by reference numeral 101, and the contour line of the histogram when there is fog or the like is indicated by reference numeral 102.

Then, the luminance and frequency characteristics are expressed in terms of area by the horizontal axis of FIG. 10B and the contour lines 101 and 102 of the histogram. With respect to the luminance and frequency characteristics expressed in this area, the area ratio (%) of the higher luminance is divided by a luminance value as a predetermined ratio, for example, 5% to 15%. FIG. 10B shows the area portions 101a and 102a. Then, the luminance value that is the boundary between the divided boundaries is determined as the threshold values 101b and 102b (step S5).
Then, the pixel data of a portion having a lower luminance than the threshold values 101b and 102b is excluded so as not to be used for the distance image data generation in step S9 (step S6).

  This process is performed for each image captured at each imaging target distance (steps S7 and S8). As shown in FIG. 10B, the luminance and frequency characteristics (reference numeral 101) in the absence of fog or the like are high in the low luminance portion and low in the vicinity of the high luminance peak value. In this characteristic, for example, even if the area other than the area with high luminance (reference numeral 101a) shown in FIG. 10B is excluded, the luminance peak value of the image due to reflection from the object to be detected in this area with high luminance is obtained. Therefore, the detection of the object by the luminance peak value is performed satisfactorily.

  On the other hand, the characteristic of luminance and frequency (reference numeral 102) when fog is present and the reflection is strong is such that the frequency is increased to a certain level due to reflection of fog or the like, as shown in FIG. 10 (b). Get higher. With this characteristic, for example, an area other than the high luminance area range (reference numeral 102a) shown in FIG. 10B is excluded (step S6). The luminance peak value of the image due to reflection from the object to be detected in the area with high luminance is raised to the vicinity of the reflection luminance peak value such as fog by the luminance correction by multiple exposure. Therefore, the luminance peak value of the image due to reflection from the object to be detected is included in the area range with high luminance.

  Then, by this exclusion process, a portion such as fog is largely excluded from the distance image, and an object that becomes a far imaging target distance can be seen more clearly in the distance image. Thereby, even if the detection performance in the distance image of the object is lowered due to bad weather and strong reflection of fog or the like at a short distance, the object detection performance is improved.

Further explanation will be added to clarify the operation of the distance image data generation device 2 of the first embodiment.
As shown in FIG. 10 (a), when the brightness of a fog or the like in the image at a distance closer to the object is higher than the image at a certain distance, the distance image is generated as shown in FIG. 10 (d). As indicated by reference numeral 201, fog or the like is strongly detected and the object is hidden.
In the distance image data generated by the image processing unit 10, for example, as the distance increases from near to far, it is expressed for each pixel in colors such as red → yellow → green → blue. For this reason, particularly when the object is blue and viewed from a plurality of pixels in this way, the object appears to be hidden when the front fog or the like appears yellow in a large area. This hidden sensation shows the difficulty of subsequent pattern matching or the like as it is.

On the other hand, with the brightness correction by multiple exposure and the process of excluding a predetermined area portion having a high brightness from the characteristics of brightness and frequency, fog and the like are processed as shown by reference numeral 202 in FIG. The area indicated by reference numeral 201 in FIG. 10 (d) that is not performed is greatly reduced in area such as fog so that the object indicated by reference numeral 301 can be clearly seen.
This improves the object detection performance by subsequent pattern matching or the like.

Next, the effect will be described.
The distance image data generation device 2 according to the first embodiment has the following effects.

  (1) A projector 5 that projects pulsed light in front of the host vehicle at a predetermined period, an image intensifier 7b that captures reflected light returning from the target distance at an imaging timing set according to the target distance, and a high speed The same pixel in a plurality of captured images with different target distances obtained by the camera 8, the timing controller 9 that controls the imaging timing so that the target distance changes continuously, the image intensifier 7b, and the high-speed camera 8 The image processing unit 10 generates distance image data representing the distance to the object for each pixel based on the luminance, and the image processing unit 10 determines the object from the luminance and the frequency for each of a plurality of captured images having different target distances. Because the processing of steps S4 to S6 for excluding data other than that from the generation of distance image data is provided, the host vehicle You can keep track of the situation ahead. In addition, it is possible to generate a distance image in which an object is detected with high accuracy while suppressing the influence under bad weather conditions.

  (2) In the above (1), the processing of steps S4 to S6 executed by the image processing unit 10 includes the frequency calculation processing of step S4 for calculating the luminance and frequency characteristics of each of a plurality of captured images having different target distances, The threshold determination process of step S5 that determines a predetermined threshold on the high luminance side with respect to the entire area in the luminance and frequency characteristics, and the luminance of pixels having a luminance lower than the threshold are excluded from the generation of the distance image data. In order to provide the exclusion process in step S6, when fog or the like reflects light with high brightness in front of the object in bad weather, the area of the fog or the like is greatly reduced to make the object clearer and deeper than the fog or the like. This makes it easier to detect the object on the side, thereby generating distance image data with improved accuracy.

  (3) In the above (1) or (2), the timing controller 9 determines that the captured image of one target distance is projected multiple times by the projector 5, and the imaging timing by the image intensifier 7b and the high-speed camera 8. In order to control to be generated by multiple imaging at, multiple exposure allows the brightness peak of an object located far away to be a value close to the brightness peak of an object located nearby. The distance image data in which the object is reliably detected can be generated. Also, in bad weather, when fog or the like reflects light with high brightness in front of the object, the object can be made clearer and the object behind the fog can be detected more easily, thereby improving the distance image Data can be generated.

  The best mode for carrying out the present invention has been described above based on the embodiment. However, the specific configuration of the present invention is not limited to the configuration of the embodiment, and it relates to each claim of the claims. Design changes and additions are allowed without departing from the spirit of the invention.

  For example, the light projection period, the light projection time, the gate time, the imaging target width, the amount of change in the imaging target distance, and the number of imaging areas in one frame are appropriately set according to the performance of the imaging means and the performance of the distance image data generation means can do.

It is a block diagram which shows the structure of the obstruction detection apparatus 1 of Example 1. FIG. It is a figure which shows the temporal relationship between the operation | movement (light projection operation | movement) of the light projector 5, and the operation | movement (camera gate operation | movement) of the gate 7a at the time of imaging one imaging area. It is a figure which shows the state which a part of imaging area overlaps. It is a schematic diagram which shows a temporal luminance change at the time of imaging by increasing an imaging area infinitely. 3 is a flowchart illustrating a flow of distance image data generation control processing executed by the image processing unit 10 according to the first embodiment. It is a figure which shows the condition where four pedestrians AD exist in the different position ahead of the own vehicle. It is a schematic diagram which shows the temporal luminance change of the pixel corresponding to each pedestrian A-B. It is a figure which shows the distance image data generation effect | action of Example 1. FIG. It is a figure which shows the distance image data production | generation effect | action by the multiple exposure of Example 1. FIG. It is explanatory drawing of the state which the image process part of the distance image data generation apparatus of Example 1 performs the process using the frequency of a brightness | luminance.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Obstacle detection apparatus 2 Distance image data generation apparatus 3 Object recognition process part 4 Judgment part 5 Light projector (light projection means)
6 Objective lens 7 Photomultiplier 7a Gate 7b Image intensifier 8 High-speed camera (imaging means)
9 Timing controller (timing control means)
10 Image processing unit (distance image data generating means)
101 Contour line 101a (of the histogram when there is no fog, etc.) Area portion 101b (on the higher luminance side of a predetermined area ratio relative to the whole when there is no fog) Threshold value 102 (dividing into a predetermined area ratio) Contour line 102a (of a histogram when there is fog or the like) Area portion 102b (on the side where the luminance of the predetermined area ratio is higher than the whole when there is fog or the like) Threshold value 201 (in the distance image data) 202 such as fog (in the distance image data) 301 portion such as fog (in the distance image data)

Claims (3)

Light projecting means for projecting pulsed light in front of the host vehicle at a predetermined period;
Imaging means for imaging reflected light returning from the target distance at an imaging timing set according to the target distance;
Timing control means for controlling the imaging timing so that the target distance changes continuously;
Distance image data generation means for generating distance image data representing a distance to an object for each pixel based on the luminance of the same pixel in a plurality of captured images with different target distances obtained by the imaging means;
With
The distance image data generation means includes
Filter means for excluding data other than objects from the generation of distance image data from the brightness and frequency for each of a plurality of captured images with different target distances,
A vehicular distance image data generating apparatus characterized by the above. The vehicle distance image data generation device according to claim 1,
The filter means includes
A frequency characteristic calculating means for calculating the luminance and frequency characteristics of each of a plurality of captured images having different target distances;
A threshold value determining means for determining a threshold value of luminance that partitions a predetermined ratio on the high luminance side with respect to the total area in the luminance and frequency characteristics;
Exclusion means for excluding the luminance of pixels having a luminance lower than the threshold from generation of distance image data;
Comprising
A vehicular distance image data generating apparatus characterized by the above. In the vehicular distance image data generating device according to claim 1 or 2,
The timing control means includes
Control is performed so that a captured image of one target distance is generated by a plurality of times of light projection by the light projecting unit and a plurality of times of imaging at the imaging timing by the image capturing unit.
A vehicular distance image data generating apparatus characterized by the above.