HK40006044B - User detection system - Google Patents

Description

User Detection System

This application is based on and claims the benefit of priority from Japanese Patent Application No. 2017-240799 (filing date: December 15, 2017), the entire contents of which are incorporated herein by reference.

Technical Field

An embodiment of the present invention relates to a user detection system.

Background Art

In recent years, technologies have been developed to prevent people or objects from being trapped in elevator doors or automatic doors. For example, a technology has been developed that uses a camera to capture images near the door, detects people or objects at risk of being trapped based on the captured images (captured images), and reflects these detection results in the door's opening and closing control.

In this technology, an image of a person or object without risk of being caught by a door is first prepared as a reference image. This reference image is then compared with the image captured by the camera to detect people or objects at risk of being caught by the door.

Summary of the Invention

However, with this technique, the lighting conditions when the reference image is captured may differ from those when the captured image is captured. In such cases, the system cannot adapt to changes in lighting conditions, potentially preventing proper detection of people or objects at risk of being trapped by the door. Furthermore, this technique requires the preparation of the reference image in advance, which can be time-consuming.

The problem to be solved by the present invention is to provide a user detection system that can detect a person or object that is likely to be caught by a door without being affected by the lighting conditions when the camera captures the image.

According to one embodiment, a user detection system comprises: a camera, which is arranged near a door and can capture the walking area and the periphery of the door when the door is opened and closed; a boundary detection unit, which detects the boundary between a first structure and a second structure located around the door based on the image captured by the camera; a user detection unit, which detects the presence or absence of a user in the walking area based on the detection result of the boundary detection unit; and a control unit, which controls the opening and closing action of the door based on the detection result of the user detection unit.

According to the user detection system configured as described above, a person or object that is likely to be caught by a door can be detected without being affected by the lighting conditions when the camera captures the image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 shows the configuration of a user detection system according to a first embodiment.

FIG2 is a flowchart showing an example of a procedure for setting a detection area corresponding to a car door sill according to the embodiment.

FIG. 3 is a diagram for explaining three-dimensional coordinates according to this embodiment.

FIG. 4 shows an example of a detection area corresponding to a car door sill according to this embodiment.

FIG5 is a flowchart showing an example of a procedure for setting a detection area corresponding to an elevator hall threshold according to the embodiment.

FIG6 shows an example of a detection area corresponding to the elevator hall threshold according to this embodiment.

FIG7 shows an example of a detection area corresponding to the space between the car threshold and the hall threshold according to this embodiment.

FIG. 8 shows an example of a binary image of a captured image according to this embodiment.

FIG. 9 is a flowchart showing an example of the procedure of the user detection process according to this embodiment.

FIG. 10 is a flowchart for explaining in detail a part of the processing in the flowchart shown in FIG. 9 .

FIG. 11 is a diagram for supplementing the explanation of the flowchart shown in FIG. 10 .

FIG. 12 shows an example of changes in the cumulative pixel value when it is determined that there is no user who is likely to be caught by the door.

FIG. 13 shows an example of changes in the cumulative pixel value when determining that there is a user who is at risk of being caught by a door.

FIG. 14 is a flowchart showing an example of a procedure for setting a detection area corresponding to the distal end of a car door according to the second embodiment.

FIG. 15 shows an example of a detection area corresponding to the top end of a door according to this embodiment.

FIG. 16 shows another example of the detection area corresponding to the top end of the door according to this embodiment.

FIG. 17 shows an example of changes in the cumulative pixel value when it is determined that there is no user who is likely to be caught by the door.

FIG. 18 shows an example of changes in the cumulative pixel value when determining that there is a user who is at risk of being caught by a door.

DETAILED DESCRIPTION

Below, the embodiment is described with reference to the accompanying drawings. What is disclosed is only an example, and the invention is not limited by the contents described in the following embodiment. Deformations that can be easily thought of by those skilled in the art are of course included in the scope of the disclosure. In order to make the description clearer, the dimensions, shapes, etc. of each part are sometimes changed according to the actual embodiment and are schematically represented in the accompanying drawings. In multiple drawings, corresponding elements are sometimes marked with the same reference numerals and detailed descriptions are omitted.

In addition, as a door that may trap people or objects, the various treatments will be described below using an elevator door as an example. However, the various treatments described below can be applied not only to elevator doors but also to automatic doors, etc.

In addition, the elevator is referred to as a "car" in the following description.

<First embodiment>

Fig. 1 shows the configuration of a user detection system according to the first embodiment. Note that, although one car is used as an example here, a plurality of cars can also have the same configuration.

In the user detection system of this embodiment, a camera 12 is installed above the entrance and exit of the car 11. Specifically, the camera 12 is installed in the lintel 11a covering the upper portion of the entrance and exit of the car 11 in an orientation such that the entire door sill of the car 11 is included in the imaging range. In other words, the camera 12 is installed in an orientation such that the area where the door moves when opening and closing and the surrounding area of the door are included in the imaging range. The camera 12 is a small surveillance camera such as an in-vehicle camera, equipped with a wide-angle lens, and can continuously capture images at a rate of several frames per second (e.g., 30 frames per second).

The camera 12 is activated when the speed of the car 11 falls below a specified value. Specifically, the camera 12 is activated and begins recording when the car 11 begins to decelerate in order to stop at a specified floor and its speed falls below the specified value. In other words, the camera 12 continuously records images from the moment the car 11 begins to decelerate in order to stop at a specified floor and its speed falls below the specified value, including while the car 11 is stopped at a specified floor, until the car 11 begins to accelerate in order to travel from that specified floor to another floor and its speed exceeds the specified value.

In the elevator lobby 15 on each floor, a hall door 14 is provided at the entrance to the car 11 so that it can be opened and closed freely. When the car 11 arrives, the hall door 14 engages with the car door 13 and opens and closes. The power source (door motor) is on the car 11 side, and the hall door 14 opens and closes simply following the car door 13. In the following description, it is assumed that the hall door 14 opens when the car door 13 opens, and closes when the car door 13 closes.

The images (videos) continuously captured by the camera 12 are analyzed and processed in real time by the image processing device 20 (user detection device). In FIG1 , the image processing device 20 is shown removed from the car 11 for convenience, but in reality, the image processing device 20 is housed in the lintel 11a along with the camera 12.

The image processing device 20 includes a storage unit 21 and a detection unit 22. The image processing device 20 acquires images captured by the camera 12. The storage unit 21 sequentially stores images captured by the camera 12 and includes a buffer for temporarily storing data required for processing by the detection unit 22. Furthermore, the storage unit 21 may store images that have undergone pre-processing such as distortion correction, enlargement and reduction, and partial cropping as pre-processing of the captured images. Furthermore, the storage unit 21 may also store programs executed by a CPU (not shown).

The detection unit 22 detects the outline (edge) of a structure of a predetermined shape that is always located near the door of the car 11, and detects the presence (or the existence) of a user who is at risk of being pinched by the door based on the shape of the detected edge. If the detection unit 22 detects the presence of a user who is at risk of being pinched by the door, it outputs a signal (indication) associated with the detection result to the car control device 30.

Furthermore, a door sill is one example of a structure that is always located near the door of the car 11. The door sill is a groove for opening and closing the door. Usually, a car door sill 13a for the car door 13 and a hall door sill 14a for the hall door 14 are respectively provided near the door.

The car control device 30 controls the operation of various devices (such as destination floor buttons and lighting) installed inside the car 11. Furthermore, the car control device 30 controls operations in accordance with signals output from the detection unit 22. For example, the car control device 30 controls the operation of a notification unit (not shown) installed inside the car 11 based on signals output from the detection unit 22, thereby providing a warning to users who are in danger of being trapped by the door.

The car control device 30 is also equipped with a door opening and closing control unit 31, which controls the door opening and closing operation of the car door 13. In addition to the normal door opening and closing control, the door opening and closing control unit 31 controls the door opening and closing operation (maintaining the door open, reversing the door open) of the car door 13 in response to the signal output from the detection unit 22.

As shown in Figure 1, the detection unit 22 includes a boundary detection unit 22a and a user detection unit 22b. These functional units are described in detail below. Furthermore, the boundary detection unit 22a and the user detection unit 22b can be implemented by a CPU (not shown) within the image processing device 20 executing a program (i.e., software) stored in the storage unit 21, or by hardware, or by a combination of software and hardware.

The boundary detection unit 22a obtains the most recent image from the captured images stored in the storage unit 21 and, based on this image, sets a detection area for the user detection unit 22b to detect users at risk of being trapped by the door. This detection area is set at the locations where the car threshold 13a and the hall threshold 14a are estimated to be reflected in the captured image. The locations where the car threshold 13a and the hall threshold 14a are estimated to be reflected are calculated based on the dimensions of the car 11 and the values inherent to the camera 12, specifically, the following conditions (a) to (e).

(a) Width of the car threshold 13a and the elevator lobby threshold 14a

(b) Height of car door 13 and hall door 14

(c) Relative position of the camera 12 in three-dimensional space relative to the car threshold 13a and the lobby threshold 14a

(d) Angles of the three axes of the camera 12

(e) Field of view and focal length of camera 12

Here, an example of a procedure for setting a detection area corresponding to the car door sill 13a, which is executed by the boundary detection section 22a, will be described with reference to the flowchart of FIG. 2 and the schematic diagrams of FIG. 3 and FIG. 4 .

First, the boundary detection unit 22a calculates the three-dimensional coordinates of the two ends of the car door sill 13a on the ground based on the width of the car door sill 13a (condition a) and the relative position of the camera 12 relative to the car door sill 13a in three-dimensional space (condition c) (step S1).

As shown in FIG3 , the three-dimensional coordinates are defined by setting the X-axis horizontal to the car door 13, the Y-axis from the center of the car door 13 to the elevator lobby 15 (a direction perpendicular to the car door 13), and the Z-axis at the height of the car 11. In other words, the ground is at a height of 0 in three-dimensional space (in other words, the Z coordinate is 0).

Then, the boundary detection unit 22a projects the three-dimensional coordinates of the two ends of the car door sill 13a calculated in step S1 onto the two-dimensional coordinates on the captured image, thereby calculating the two-dimensional coordinates corresponding to the two ends of the car door sill 13a. Specifically, the boundary detection unit 22a calculates the two-dimensional coordinates of points PA1 and PA2, as shown in FIG4 , with the horizontal direction of the captured image being the X-axis and the vertical direction being the Y-axis (step S2).

Next, the boundary detection unit 22a calculates (determines) two points PA3 and PA4, respectively, which are spaced a predetermined number of pixels (e.g., 5 pixels) vertically from the pixels at which the two points PA1 and PA2, determined by the two-dimensional coordinates calculated in step S2, are located, as shown in FIG4 (step S3). The vertical direction corresponds to the negative direction of the Y axis of the two-dimensional coordinates. Furthermore, here, points PA3 and PA4 are calculated (determined) as being spaced a predetermined number of pixels from points PA1 and PA2. However, points PA3 and PA4 may also be calculated (determined) as being spaced 50 mm vertically from points PA1 and PA2 in the three-dimensional coordinates. Furthermore, the vertical direction in this case corresponds to the negative direction of the Y axis of the three-dimensional coordinates.

Thereafter, the boundary detection unit 22a sets the area enclosed by the line segment LA1 formed by connecting point PA1 and point PA2, the line segment LA2 formed by connecting point PA1 and point PA3, the line segment LA3 formed by connecting point PA2 and point PA4, and the line segment LA4 formed by connecting point PA3 and point PA4 as shown in Figure 4 as the detection area E1 corresponding to the car door sill 13a (step S4), and ends the setting process.

Next, an example of a procedure for setting a detection area corresponding to the hall threshold 14a, which is executed by the boundary detection unit 22a, will be described with reference to the flowchart of FIG. 5 and the schematic diagram of FIG. 6 .

First, the boundary detection unit 22a calculates the three-dimensional coordinates of the two ends of the elevator lobby threshold 14a on the ground based on the width of the elevator lobby threshold 14a (condition a) and the relative position of the camera 12 relative to the elevator lobby threshold 14a in three-dimensional space (condition c) (step S11).

The boundary detection unit 22a then projects the three-dimensional coordinates of the two ends of the elevator hall threshold 14a calculated in step S11 onto the two-dimensional coordinates on the captured image, thereby calculating the two-dimensional coordinates corresponding to the two ends of the elevator hall threshold 14a. Specifically, the boundary detection unit 22a calculates the two-dimensional coordinates of points PB1 and PB2, as shown in FIG6 , with the horizontal direction of the captured image being the X-axis and the vertical direction being the Y-axis (step S12).

Next, the boundary detection unit 22a calculates two points PB3 and PB4, each of which is spaced a predetermined number of pixels (e.g., 5 pixels) vertically from the pixels where the two points PB1 and PB2, identified by the two-dimensional coordinates calculated in step S12, are located, as shown in FIG6 (step S13). The vertical direction corresponds to the positive Y-axis direction of the two-dimensional coordinates.

Thereafter, the boundary detection unit 22a sets the area enclosed by the line segment LB1 formed by connecting point PB1 and point PB2, the line segment LB2 formed by connecting point PB1 and point PB3, the line segment LB3 formed by connecting point PB2 and point PB4, and the line segment LB4 formed by connecting point PB3 and point PB4 as shown in Figure 6 as the detection area E2 corresponding to the elevator hall threshold 14a (step S14), and ends the setting process.

7, the setting of the detection area corresponding to (the gap between) the car threshold 13a and the hall threshold 14a performed by the boundary detection unit 22a will be described.

The boundary detection unit 22a sets the area enclosed by the line segment LA1 formed by connecting the point PA1 and the point PA2 calculated in the above step S2, the line segment LB1 formed by connecting the point PB1 and the point PB2 calculated in the above step S12, the line segment LC1 formed by connecting the point PA1 and the point PB1, and the line segment LC2 formed by connecting the point PA2 and the point PB2 as shown in Figure 7 as the detection area E3 corresponding to (the gap between) the car threshold 13a and the elevator hall threshold 14a.

Furthermore, as shown in Figures 4, 6, and 7, examples are shown in which one detection area is set for each of the car threshold 13a, the hall threshold 14a, and the gap. However, if the car threshold 13a and the hall threshold 14a are divided into multiple parts, a detection area may be set for each of the divided thresholds. In other words, multiple detection areas may be set for each of the car threshold 13a, multiple detection areas for each of the hall threshold 14a, and multiple detection areas for each of the gaps.

By setting the detection areas E1 to E3 as described above, the area for detecting a boundary described later can be limited to only the detection areas E1 to E3.

Let's return to the description of the function of the boundary detection unit 22a.

The boundary detection unit 22a detects (extracts) the outline (edge) of a person or object from the captured image and binarizes the captured image ("0 (no edge)" or "1 (edge)") as shown in FIG8. This makes it possible to detect the boundary between the car threshold 13a and the floor, the boundary between the elevator hall threshold 14a and the floor, and the boundary between the car threshold 13a and the elevator hall threshold 14a and the aforementioned gap.

The white portions in FIG8 correspond to detected edges, and the black portions in FIG8 correspond to portions where no edges are detected. Furthermore, FIG8 illustrates a case where all edges of people or objects are detected from the captured image. However, the boundary detection unit 22a only needs to detect at least the edges of people or objects located near the detection areas E1 to E3 from the captured image.

As a method for detecting an edge, for example, a method of comparing the pixel values of a predetermined pixel with those of pixels adjacent to the predetermined pixel is used. That is, the boundary detection unit 22a compares the pixel values of a predetermined pixel with those of pixels adjacent to the predetermined pixel. If the difference between these pixel values is greater than a preset threshold, an edge is detected. If the difference is less than the preset threshold, no edge is detected.

Alternatively, the boundary detection unit 22a may detect the presence of an edge by comparing the pixel values of a predetermined pixel with the pixel values of a pixel that is a predetermined pixel (predetermined width) away from the predetermined pixel. Alternatively, the boundary detection unit 22a may compare the average brightness value of a pixel group consisting of a plurality of pixels with the average brightness value of another pixel group that is different from (adjacent to) the pixel group. If the difference between these average brightness values is greater than a predetermined threshold, an edge is detected. If the difference is less than the predetermined threshold, an edge is not detected.

In addition, as a method of detecting an edge, in addition to the various methods described above, any known method can be used.

Furthermore, in the above description, the boundary detection unit 22a detects the edges of people or objects from the captured image and binarizes the captured image to detect the boundaries between the car threshold 13a and the floor, the elevator lobby threshold 14a and the floor, and the boundaries between the car threshold 13a and the elevator lobby threshold 14a and the aforementioned gaps. However, the boundary detection unit 22a may also detect the aforementioned various boundaries by detecting the intensity of the brightness gradient of each pixel (in other words, the intensity of the edge) using any known method. Alternatively, the boundary detection unit 22a may segment the captured image into a large number of small areas, analyze the texture, and detect pixels divided by different textures to detect the aforementioned various boundaries.

Next, the function of the user detection unit 22b will be described.

The user detection unit 22b determines whether there is a user who may be caught by the door based on the captured image binarized by the boundary detection unit 22a (hereinafter referred to as "binary image") and various boundaries detected by the boundary detection unit 22a.

Here, an example of the procedure of the user detection process executed by the user detection unit 22b will be described with reference to the flowchart of FIG. 9 .

First, the user detection unit 22b acquires image data representing a binary image from the boundary detection unit 22a (step S21). Alternatively, the image data representing a binary image can be acquired from the storage unit 21. The user detection unit 22b then extracts detection areas E1 to E3 from the binary image represented by the acquired image data (step S22).

Next, the user detection unit 22b refers to the extracted images of the detection areas E1 to E3 (hereinafter referred to as "detection area images") to determine whether any of the various boundaries detected by the boundary detection unit 22a is interrupted (step S23). In detail, the user detection unit 22b refers to the extracted detection area image corresponding to the detection area E1 to determine whether the boundary between the car door sill 13a and the floor is interrupted. In addition, the user detection unit 22b refers to the extracted detection area image corresponding to the detection area E2 to determine whether the boundary between the elevator hall door sill 14a and the floor is interrupted. Furthermore, the user detection unit 22b refers to the extracted detection area image corresponding to the detection area E3 to determine whether the boundary between the car door sill 13a and the elevator hall door sill 14a and the above-mentioned gap is interrupted.

Furthermore, when it is determined that none of the boundaries are interrupted (No in step S23), the user detection unit 22b determines that there is no user who is likely to be caught by the door, and ends the user detection process.

On the other hand, when it is determined that an interruption has occurred at any one of the various boundaries (yes in step S23), the user detection unit 22b determines that there is a user who is in danger of being pinched by the door (step S24), and outputs a signal associated with the result of the judgment to the car control device 30 (step S25), thereby ending the user detection processing.

Next, the processing of step S23 will be described in more detail with reference to the flowchart of FIG10 and the schematic diagram of FIG11 .

First, the user detection unit 22 b focuses on a plurality of pixels located in the x-th column among a large number of pixels included in the detection area image extracted in step S22 (step S31 ).

The detection area image contains a large number of pixels, for example, m×n pixels. Specifically, as shown in Figure 11, the detection area image has m pixels (m columns) arranged horizontally, and n pixels (n rows) arranged vertically. In other words, values 1 to m are substituted into x in step S31 above.

Then, the user detection unit 22 b calculates the total value (hereinafter referred to as “accumulated pixel value”) of the values of n pixels (pixel group) arranged on the x-th column of interest (step S32 ).

The detection area image is an image obtained by extracting the detection area portion from the binary image. Therefore, the pixel values contained in the detection area image are either 0 or 1. As described above, in a binary image, the value of pixels in white areas is 1, and the value of pixels in black areas is 0. Therefore, the user detection unit 22b calculates the cumulative pixel value of the n pixels arranged in the x-th column based on this.

Next, the user detection unit 22b determines whether the calculated cumulative pixel value is 0 (step S33). If the cumulative pixel value is 0 (yes in step S33), the user detection unit 22b determines that a boundary interruption has occurred (step S34), and then executes the process of step S24 described above.

On the other hand, when it is determined that the cumulative pixel value is not 0 (No in step S33 ), the user detection unit 22 b determines whether attention has been paid to all the columns (step S35 ).

If it is determined that all columns have been considered, that is, if x = m (YES in step S35), the user detection unit 22b determines that no boundary interruption has occurred (step S36). In other words, the user detection unit 22b determines that there is no user who is likely to be trapped by the door, and the user detection process ends.

On the other hand, if it is determined that not all columns have been considered, that is, if it is determined that x≠m (No in step S35), the user detection unit 22b adds 1 to x and sets this value as a new x (step S37). Thereafter, the process returns to step S31 and the various processes described above are repeated.

12 and 13 each show an example of a binary image after binarization by the boundary detection unit 22a, a detection area image corresponding to the detection area E2 extracted from the binary image, and a result of applying the series of processing shown in FIG. 10 to the detection area image.

When the series of processes shown in FIG10 are applied to the detection area image ( FIG12(b) ) extracted from the binary image ( FIG12(a) ), the cumulative pixel values for each column are shown in FIG12(c) In FIG12(c) , the vertical axis represents the cumulative pixel values, and the horizontal axis represents the number (or horizontal coordinate in the image) used to identify each column.

In this case, as shown in FIG12(c), there are no columns with cumulative pixel values of 0. Therefore, the user detection unit 22b determines that the boundary corresponding to detection area E2, that is, the boundary between the hall threshold 14a and the floor, is not interrupted. Therefore, the user detection unit 22b determines that there are no users in detection area E2 who are at risk of being trapped by the door.

On the other hand, when the series of processes shown in FIG10 are applied to the detection area image ( FIG13(b) ) extracted from the binary image ( FIG13(a) ), the cumulative pixel values for each column are as shown in FIG13(c) . In this case, the cumulative pixel values in the column enclosed by the dotted line in FIG13(c) are 0. Therefore, the user detection unit 22b determines that the boundary corresponding to detection area E2, namely, the boundary between the hall threshold 14a and the floor, has been interrupted. Therefore, the user detection unit 22b determines that a user who is at risk of being trapped by the door is present within detection area E2.

Note that, although the description here is made by taking the detection area image corresponding to the detection area E2 as an example, the same applies to the detection area images corresponding to the detection areas E1 and E3.

Here, the presence of columns with cumulative pixel values of 0 is used to determine whether various boundaries are discontinuous. However, for example, the ratio of columns with cumulative pixel values of 1 or greater (= [number of columns with cumulative pixel values of 1 or greater]/m) can be calculated, and then the ratio can be determined to be less than a predetermined threshold (e.g., less than 95%) to determine whether various boundaries are discontinuous. Furthermore, the method for determining the presence of columns with cumulative pixel values of 0 is the same as when the threshold is set to 100%.

Alternatively, as a method for determining whether various boundaries are interrupted, known methods such as labeling and Hough transform can be used to detect straight lines from the detection area image and determine whether the detected straight lines are interrupted, thereby determining whether various boundaries are interrupted.

Furthermore, when the intensity of the brightness gradient of each pixel is used to detect various boundaries, the user detection unit 22b can calculate the cumulative pixel value for each column of the detection area image in the same manner as described above. If there are columns whose calculated cumulative pixel value is less than a predetermined threshold, the various boundaries can be determined to be interrupted. Alternatively, if the proportion of columns whose cumulative pixel values are less than the predetermined threshold is greater than a predetermined threshold (e.g., 5% or more), the various boundaries can be determined to be interrupted. Furthermore, this method using a threshold can also be applied to detecting the edges of people or objects and binarizing the captured image to detect various boundaries.

In addition, the user detection unit 22a can also maintain the detection results of the user detection processing for one captured image between several frames (for example, 5 frames from the captured frame of the captured image). When the proportion of the detection results that indicate that there is a risk of users being pinched by the door obtained between the 5 frames is above a predetermined threshold (for example, 50%), it is formally judged that there is a risk of users being pinched by the door, and a signal associated with the judgment is output to the car control device 30.

Furthermore, in this embodiment, the detection object is set as a user who is at risk of being pinched by the door, but the detection object can be any user who is at risk of causing an accident when the door is opened or closed. For example, a user who is at risk of colliding with the door can also be set as the detection object.

Furthermore, while this embodiment illustrates the case where the detection area is set and user detection is performed when the door is fully open, the user detection process can also be performed while the door is closing or opening. In this case, the boundary detection unit 22a can simply adjust the width of the detection area (the horizontal length of the image) based on the door opening and then set the detection area. Furthermore, the boundary detection unit 22a can also adjust the width of the detection area based on the door opening for each captured frame. In other words, the boundary detection unit 22a can also dynamically change the detection area.

According to the first embodiment described above, the image processing device 20 includes a detection unit 22 capable of detecting the presence of a user at risk of being trapped by a door based on a single captured image. This eliminates the need for preparing a reference image and enables user detection processing that is not affected by lighting conditions. This provides a highly secure user detection system.

Furthermore, since only one captured image is used in the user detection process according to this embodiment, the time required to detect the presence of a user who may be caught by the door can be shortened. In other words, the time required for the user detection process can be shortened.

<Second embodiment>

Next, the second embodiment will be described. In the above-mentioned first embodiment, the detection area is set at the position where the car door sill 13a and the elevator hall door sill 14a are estimated to be reflected on the captured image. In the present embodiment, the detection area is set at the position where the top ends of the car door 13 and the elevator hall door 14 are estimated to be reflected on the captured image. Furthermore, the top ends of the car door 13 and the elevator hall door 14 can also be referred to as the end of the car door 13 and the elevator hall door 14 that is located in the door closing direction. Alternatively, it can also be referred to as the end of the car door 13 and the elevator hall door 14 that is located on the door stop side. Hereinafter, the functional parts having the same functions as those in the first embodiment are marked with the same symbols and their detailed descriptions are omitted. Hereinafter, the parts that are different from the first embodiment will be mainly described.

Furthermore, in this embodiment, the camera 12 is installed in a single-swing elevator. Therefore, there is one camera located at the top of each of the car doors 13 and the hall door 14, and the number of detection areas set is also one. However, if the camera 12 is installed in a center-swing elevator, there are two cameras located at the top of each of the car doors 13 and the hall door 14, and the number of detection areas set is also two. However, the method for setting detection areas is the same for both types of elevators, so the description of setting detection areas for the case of the camera 12 installed in a center-swing elevator will be omitted here.

As described above, the boundary detection unit 22a sets the detection area by estimating the positions where the top ends of the car door 13 and the hall door 14 are reflected on the captured image. The positions where the top ends of the car door 13 and the hall door 14 are estimated are calculated based on the above-mentioned conditions (a) to (e) and the current opening amount of the car door 13.

Furthermore, the boundary detection unit 22 a can acquire the information indicating the current opening amount of the car door 13 by communicating with the car control device 30 .

Alternatively, the boundary detection unit 22a may first communicate with the car control device 30 to obtain a signal indicating that the car door 13 has opened from a fully closed state, or a signal indicating that the car door 13 has closed from a fully open state. The boundary detection unit 22a then calculates the current opening amount of the car door 13 based on the elapsed time from the time these signals were obtained and the door opening and closing speed of the car door 13.

Alternatively, the boundary detection unit 22a may calculate the current opening amount of the car door 13 by performing a known image analysis process on the acquired captured image.

Here, an example of a procedure for setting a detection area corresponding to the top end of the car door 13 performed by the boundary detection unit 22a will be described with reference to the flowchart of FIG. 14 and the schematic diagram of FIG. 15 .

First, the intersection detection unit 22a calculates the three-dimensional coordinates of the lower top portion and the three-dimensional coordinates of the position extending vertically from the lower top portion to the height of the camera 12 (or the height of the car door 13) (hereinafter referred to as the "upper top portion") based on the current opening of the car door 13, the relative position of the camera 12 in three-dimensional space relative to the position where the top end of the car door 13 contacts (abuts) the ground (hereinafter referred to as the "lower top portion"), and the height from the lower top portion to the camera 12 (or the height of the car door 13) (step S41).

The boundary detection unit 22a then projects the three-dimensional coordinates of the top lower portion and the top upper portion calculated in step S41 onto the two-dimensional coordinates on the captured image to calculate the two-dimensional coordinates corresponding to the top lower portion and the top upper portion. Specifically, the boundary detection unit 22a calculates the two-dimensional coordinates of points PD1 and PD2 in the captured image shown in FIG15 (step S42).

Thereafter, the boundary detection unit 22a sets the following area as shown in FIG. 15 as the detection area E4 corresponding to the top end of the car door 13, including the portion located a predetermined number of pixels (e.g., 5 pixels) from the line segment LD1 formed by connecting the two points PD1 and PD2 identified by the two-dimensional coordinates calculated in step S42 (each pixel located on the line segment LD1) (step S43), and the setting process ends. Furthermore, this method can also be applied to the setting of the detection area in the first embodiment described above. In other words, instead of determining four points, only two points can be determined, and then the detection areas E1 to E3 can be set using this method.

Furthermore, Figure 15 illustrates the setting of the detection area when the car door 13 is fully open (the current opening of the car door 13 is the largest), and the setting of the detection area when the car door 13 is closed is the same. In this case, the detection area E4 shown in Figure 16 is set.

As described above, when the camera 12 is installed in a center-opening elevator, the series of processing shown in FIG. 14 is performed for each of the left and right doors, thereby setting a detection area for each door. Details are omitted.

The process of detecting various boundaries executed after setting the detection area E4 is the same as that of the first embodiment. That is, the boundary detection unit 22a detects at least the edge of a person or object located near the detection area E4 to detect the boundary.

Furthermore, the boundary detected by the boundary detection unit 22a varies depending on the current door opening/closing state (current opening amount of the car door 13) of the car door 13. For example, when the car door 13 is fully open (the current opening amount of the car door 13 is at its maximum), the boundary between the top end of the car door 13 and the entrance/exit pillar provided near the car door 13 is detected. Alternatively, when the car door 13 is closed, the boundary between the top end of the car door 13 and the background (the floor of the elevator lobby 15) is detected.

The user detection process performed by the user detection unit 22b is also the same as that in the first embodiment described above. Specifically, the user detection unit 22b refers to the detection area image corresponding to the detection area E4 to determine whether the boundary detected by the boundary detection unit 22a is interrupted, thereby detecting the presence of a user who is at risk of being trapped by the door.

17 and 18 both show a binary image after binarization by the boundary detection unit 22a, a detection area image corresponding to the detection area E4 extracted from the binary image, and the result of applying the series of processing shown in FIG. 10 to the detection area image.

When the series of processes shown in FIG. 10 described in the first embodiment are applied to the detection area image ( FIG. 17(b) ) extracted from the binary image ( FIG. 17(a) ), the cumulative pixel values of each column are as shown in FIG. 17(c) . In this case, as shown in FIG. 17(c) , there are no columns with cumulative pixel values of 0. Therefore, the user detection unit 22b determines that the boundary corresponding to the detection area E4 is not interrupted. In this case, since the car door 13 is fully open, the boundary between the top end of the car door 13 and the entrance post located near the car door 13 is not interrupted. Therefore, the user detection unit 22b determines that there is no user in the detection area E4 who is at risk of being pinched by the door.

On the other hand, when the series of processes shown in FIG. 10 described in the first embodiment are applied to the detection area image ( FIG. 18(b) ) extracted from the binary image ( FIG. 18(a) ), the cumulative pixel values of each column are as shown in FIG. 18(c) . In this case, as shown in FIG. 18(a) , due to the presence of the user's hand, the cumulative pixel values in the column enclosed by the dotted line in FIG. 18(c) are 0. Therefore, the user detection unit 22b determines that the boundary corresponding to the detection area E4 has been interrupted. In this case, since the car door 13 is fully open, the boundary between the top end of the car door 13 and the entrance and exit pillars located near the car door 13 has been interrupted. Therefore, the user detection unit 22b determines that a user who is at risk of being pinched by the door is present within the detection area E4.

Furthermore, similar to the first embodiment, the proportion of columns with cumulative pixel values of 1 or greater may be calculated, and then a determination may be made as to whether the calculated proportion is less than a preset threshold value, thereby determining whether the boundary is interrupted. Alternatively, a known method such as a labeling method or a Hough transform may be used to detect straight lines from the detection area image, and then a determination may be made as to whether the detected straight lines are interrupted, thereby determining whether the boundary is interrupted.

Furthermore, the intensity of the brightness gradient of each pixel can be used instead of the edge to detect the boundary. In this case, as in the first embodiment described above, the cumulative pixel value of each column of the detection area image can be calculated, and if there are columns whose calculated cumulative pixel value is less than a predetermined threshold, the boundary is determined to be interrupted. Alternatively, the boundary can be determined to be interrupted if the proportion of columns whose cumulative pixel values are less than the predetermined threshold is greater than or equal to the predetermined threshold.

In addition, as in the first embodiment described above, the user detection unit 22a can also maintain the detection results of the user detection processing for one captured image between several frames (for example, 5 frames from the captured frame of the captured image). When the proportion of the detection results that indicate that there is a risk of users being pinched by the door obtained between the 5 frames is above a predetermined threshold value (for example, 50%), it is formally determined that there is a risk of users being pinched by the door, and a signal associated with the determination is output to the car control device 30.

According to the second embodiment described above, the image processing device 20 includes a detection unit 22. This detection unit 22 is configured not only with detection areas E1 to E3 corresponding to the door sills, but also with a detection area E4 corresponding to the top end of the car door 13. This allows detection of users at risk of being trapped by the door even within detection area E4. This allows detection of users at risk of being trapped by the door over a wider range. This provides a safer user detection system.

According to at least one embodiment described above, a user detection system can be provided that can detect a person or object that is likely to be caught by a door without being affected by lighting conditions when the camera captures an image.

Furthermore, although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments may be implemented in various other forms and may be omitted, replaced, or modified without departing from the spirit of the invention. These embodiments and their variations are intended to be within the scope and spirit of the invention and are within the scope of the invention set forth in the claims and their equivalents.

Claims (10)

1. A user detection system, characterized in that it comprises: A camera is positioned near the door such that the threshold of the car is included in the shooting range, and is capable of shooting the walking area of the door when it is opened and closed and the perimeter of the door including the threshold. The boundary detection unit detects the boundary between the first structure and the second structure located around the door based on the image captured by the camera. The user detection unit detects the presence or absence of a user within the walking area based on the detection results of the boundary detection unit; and The control unit controls the opening and closing of the door based on the detection results from the user detection unit. The boundary detection unit sets the detection area on the captured image at the location where the threshold is predicted. It is speculated that the position of the threshold is calculated based on the dimensions of the car and the inherent value of the camera. 2. The user detection system according to claim 1, characterized in that, The boundary detection unit detects the straight-line boundary between the first structure and the second structure. The user detection unit detects the presence or absence of a user within the walking area based on whether the straight boundary detected by the boundary detection unit is interrupted. 3. The user detection system according to claim 2, characterized in that, When the straight-line intersection is interrupted, the user detection unit determines that a user exists in the walking area. 4. The user detection system according to claim 1, characterized in that, The boundary detection unit sets the detection area based on the inherent parameters of the gate, thus narrowing the area for detecting the boundary to this detection area. 5. The user detection system according to claim 1, characterized in that, The boundary detection unit detects the boundary based on an image captured by the camera. 6. The user detection system according to claim 1, characterized in that, The boundary detection unit uses the door opening and closing groove, i.e., the threshold, as the first structure and the floor as the second structure to detect the boundary. 7. The user detection system according to claim 1, characterized in that, The boundary detection unit uses the top of the door as the first structure and the entrance/exit column located near the door as the second structure to detect the boundary. 8. The user detection system according to claim 1, characterized in that, The boundary detection unit detects the boundary by using the top of the door as the first structure and the floor as the second structure. 9. The user detection system according to claim 1, characterized in that, The boundary detection unit binarizes the image captured by the camera in order to detect the boundary between the first structure and the second structure. The user detection unit focuses on the pixel group located in the xth column among the large number of pixels constituting the binarized image, calculates the total value of the pixel values of the focused pixel group, and detects the presence or absence of a user in the walking area based on whether the calculated total value is 0. 10. The user detection system according to claim 1, characterized in that, The boundary detection unit binarizes the image captured by the camera in order to detect the boundary between the first structure and the second structure. The user detection unit sequentially focuses on a large number of pixels arranged in the horizontal and vertical directions of the binarized image, and calculates the total pixel value of the focused pixel column in sequence. The presence or absence of a user in the walking area is detected based on whether the proportion of pixel columns with a total value of 1 or higher is less than a threshold.