TWI820341B - Method for image tracking and display - Google Patents

Abstract

A method for image tracking and display includes the following procedure. A variation detection process, which determines whether there is a target increase or decrease in an image corresponding to a variable trigger area. A display screen change process, which adjusts the camera to capture the image corresponding to a trigger bounding frame and outputs the image to a monitor when the result of the variation detecting procedure is “YES”. A stable detection process, which determines whether a change parameter of the image corresponding to the trigger bounding frame is less than or equal to a predetermined change value. A target detection process, which performs a target detecting on the image corresponding to the triggered bounding frame and generates an image display frame. An image adjusting process, which outputs the image corresponding to the range of the image display frame to the monitor. Accordingly, the method can reduce the variation of the display screen.

Description

Image tracking and display methods

The present invention relates to an image processing method, and in particular to an image tracking and display method.

Due to the maturity of network technology, the way meetings are held has also changed from the traditional face-to-face format to the long-distance video format. In addition to having sufficient and stable network bandwidth to provide stable audio and video signals, video conferencing allows the camera to fully present participants in the display screen, which is also an effect required for general video conferencing. In this way, participants on both sides can know who is participating in the meeting.

In addition, due to the advancement of camera technology, cameras with electric pan-tilt-zoom (PTZ) functions are also commonly used in video conferencing to capture various images in video conferencing by adjusting the camera pan angle, tilt angle and focal length.

In a conventional image tracking technology, the system uses face detection technology to determine the distribution of people in the meeting space, and then transmits the images of all participants to the monitor and displays them. The specific method is to detect the increase, movement or decrease of faces within the viewing range of the camera lens, and then immediately adjust the image displayed on the monitor to allow participants to obtain the number and dynamics of participants.

However, since the above-mentioned conventional method is for face detection, the actions of the participants, such as swinging, turning their heads, turning their faces sideways, or moving their positions, etc., may trigger the detection and tracking mechanism. This will cause the displayed screen to keep changing.

Therefore, how to solve the above-mentioned frequent changes in the display screen caused by continuous tracking of human faces is indeed one of the important issues at present.

In view of the above problems, one purpose of the present invention is to provide an image tracking and display method that can make the real-time image displayed on the display have a more stable effect.

To achieve the above object, the present invention provides an image tracking and display method, which is used in conjunction with a camera lens and a display. The camera lens continuously captures an image, and sets a trigger bounding box and a variable trigger area corresponding to the image. The image tracking and display method includes a change detection procedure, a display screen change procedure, a stability detection procedure, a target detection procedure and an image adjustment procedure. The change detection program determines whether there is an increase or decrease in the number of a target in the image corresponding to the variable trigger area. The display screen changing process is when the result of the change detection process is "yes", the camera lens is adjusted to capture the image corresponding to the trigger bounding box range, and the image is output to the display. The stability detection program determines whether the change parameter of the image corresponding to the trigger bounding box is less than or equal to a preset change value. The target detection program performs target detection on the image corresponding to the trigger bounding box and generates an image display frame. The image adjustment program outputs the image corresponding to the image display frame range to the monitor.

In one embodiment, the change detection process is re-executed after the image adjustment process is completed.

In one embodiment, the image adjustment process further includes adjusting a zoom ratio, a translation angle, a tilt angle and combinations thereof of the camera lens according to the image display frame.

In one embodiment, an initial target detection process and an initial image adjustment process are further included before the change detection process. The initial target detection program performs an initial target detection on the image corresponding to the trigger bounding box and generates an initial image display frame. The initial image adjustment program outputs the image corresponding to the initial image display frame range to the monitor.

In one embodiment, the trigger bounding box corresponds to one of the widest focal lengths of the camera lens, or is a self-defined range smaller than the widest focal length of the camera lens.

In one embodiment, the target detection process is executed through a humanoid detection technology.

As mentioned above, an image tracking and display method of the present invention is to detect changes in the number of targets in the image, and then convert the image on the display into an image corresponding to the trigger bounding box, and move the target within the image. After stabilizing, gradually adjust to the image corresponding to the appropriate image display frame. Accordingly, it shows The image displayed by the monitor will only change the size of the output image when it detects changes in the number of targets in the variable trigger area. In other stable states, the image displayed by the monitor will not always be adjusted or changed. Therefore, when the image tracking and display method is applied to a video conference, the images observed by participants during the conference can be more stable.

11:Camera lens

12:Display

20:Conference room

F01: Image display frame

F02: Trigger bounding box

A01: Variable trigger area

P01~P07: Program

[Fig. 1] is a schematic diagram showing an environment used in conjunction with the image tracking and display method of the present invention.

[Fig. 2] is a schematic diagram showing the image tracking and display method of the present invention for image frame selection.

[Fig. 3] is a schematic flowchart showing an image tracking and display method according to a preferred embodiment of the present invention.

[Fig. 4A] to [Fig. 4C] are schematic diagrams showing image frame selection in each program corresponding to the image tracking and display method of the present invention.

In order to enable those with ordinary knowledge in the technical field to understand the contents of the present invention and implement the contents of the present invention, the following description is provided with appropriate embodiments and drawings, in which the same components will be described with the same component symbols. .

Referring to FIG. 1 , an image tracking and display method of the present invention is used in conjunction with a camera lens 11 and a display 12 . This embodiment takes an image tracking and display method applied in a video conference as an example. Therefore, the camera lens 11 and the display 12 are also part of the video conference system. The camera lens 11 is installed in front of a conference room 20, and faces the inside of the conference room 20 to continuously capture an image. The display 12 can display the image captured by the camera lens 11, and can also display the image output from the remote end. . In this embodiment, the camera lens 11 is a camera lens with an electric pan-tilt-zoom (PTZ) function, which can adjust the zoom magnification, translation angle and tilt angle of the lens.

Please refer to Figure 1 and Figure 2 at the same time. The video conference system sets an image display frame F01, a trigger bounding frame F02 and a variable trigger according to the image correspondence. Area A01. Among them, the image in the image display frame F01 is used to be displayed on the display 12. The camera lens 11 can use the optical zoom magnification and the motor to adjust the translation or tilt angle to capture the corresponding image. It can also be selected through digital PTZ technology. Of. The trigger bounding box F02 can be the widest field of view boundary of the camera lens 11, or can be a region defined by the user through the video conferencing system. To further illustrate, when the focal length of the camera lens 11 is too wide and the conference room includes When areas such as the ceiling and where no one will appear are within the camera's field of view, the user can define the coverage of the trigger bounding box F02. The trigger bounding box F02 in this embodiment takes the widest angle field of view boundary of the camera lens 11 as an example.

Below, please refer to FIG. 3 in conjunction with the above description to describe an image tracking and display method of the present invention. The image tracking and display method includes an initial target detection procedure P01, an initial image adjustment procedure P02, a change detection procedure P03, a display screen change procedure P04, a stable detection procedure P05, an object detection procedure P06 and a Image adjustment program P07.

After starting the video conferencing system, the initial target detection procedure P01 will be executed first. Please refer to FIG. 3 and FIG. 4A at the same time. The initial target detection program P01 performs an initial target detection on the image corresponding to the trigger bounding box F02 and generates an initial image display frame F01. Among them, the initial target detection is to detect whether there are people in the image corresponding to the trigger bounding box F02, and to identify their distribution location. After recognition, an initial image display frame F01 is generated corresponding to the person and the distribution position.

Here, the so-called target can be preset by the system or set by the user. This embodiment uses video conference as an example, so the target is the "humanoid" defaulted by the system. It should be noted that in other applications, the target can also be a vehicle or an animal, and is not limited to humans. In this embodiment, humanoid detection is implemented by artificial intelligence (AI) hardware, which is implemented by, for example, using modules such as R-FCN (Region-based Fully Convolutional Network), SSD (single shot multibox detector) or YOLOv2. .

In addition, it is worth mentioning that after the initial image display frame F01 is generated, if the proportion of the initial image display frame F01 does not match the display proportion of the monitor, it also includes compensation processing for the deformed image.

The initial image adjustment program P02 outputs the image corresponding to the range of the initial image display frame F01 to the monitor. Here, the image corresponding to the range of the initial image display frame F01 can be captured by physically adjusting the mechanism of the camera lens, or can be captured by digital calculation. In this embodiment, the image corresponding to the range of the initial image display frame F01 is obtained by adjusting the zoom magnification, translation angle and tilt angle of the camera lens, and the captured image is continuously output to the display. In this embodiment, it also includes confirming whether the zoom magnification, translation angle and tilt angle of the camera lens are adjusted to the position. When the adjustment is in a continuous state, the confirmation action is continued, and when it is determined that the adjustment is completed, the next procedure is performed.

It should be noted here that the above-mentioned initial image display frame F01 is equivalent to the image display frame that will be mentioned later. The word initial only means that it is the first action performed when the system is turned on.

Please refer to FIG. 3 and FIG. 4B at the same time. The change detection program P03 determines whether there is an increase or decrease in the number of a target in the image corresponding to the variable trigger area A01. Here, the variable trigger area A01 is the area range between the image display frame F01 and the trigger boundary frame F02, and the change detection program P03 determines whether there is a person entering or leaving the variable trigger area A01. In other words, the judgment result is generated based on the change in the number of human figures in the variable trigger area A01. Since the judgment is based on the "human form", the area of change must be greater than a critical value. In this way, misjudgments caused by smaller movements such as turning the head or swinging can be avoided. In this embodiment, when there is no change in the target quantity in the variable trigger area A01, the change detection procedure P03 is continuously executed.

The display screen change program P04 is when the result of the change detection program P03 is "yes", the camera lens is adjusted to capture the image corresponding to the range of the trigger bounding box F02, and the image is output to the display. Here, the result of the so-called change detection program P03 is "yes", which means that there are people entering and exiting the conference room, resulting in an increase or decrease in the number of people. At this time, the camera lens will be adjusted to capture the range corresponding to the trigger bounding box F02 image. In this embodiment, this adjustment process can be instantaneous switching. By adjusting the display screen to a wide-angle state, participants can be informed of the changes taking place in the conference room. People entering the conference room can know the field of view of the camera lens through the display screen, and can choose an appropriate position. Take a seat.

The stability detection program P05 determines whether a change parameter of the image corresponding to the trigger bounding box F02 is less than or equal to a preset change value. Here, the system can choose to use the inter-frame difference algorithm (Frame difference), erosion algorithm (erison) or dilation algorithm (dilation) to obtain whether the changes in personnel tend to be stable. In this embodiment, the inter-frame difference algorithm is taken as an example, which uses the preceding and following frames of the continuous image to perform pixel difference analysis to obtain the variation parameters. When the change parameter is greater than the preset change value, it means that the person is still moving, and when the change parameter is less than the preset change value, it means that the person has left the conference room (disappeared from the screen) or has been seated stably. And after confirming that the person is in stable condition, proceed with the following procedures.

Next, please refer to FIG. 3 and FIG. 4C simultaneously. The target detection program P06 performs a target detection on the image corresponding to the trigger bounding box F02 and generates an image display frame F01. Here, the target detection program P06 is similar to the aforementioned initial target detection program P01. The difference may be that due to the different number of people in the image, the size of the image display frame F01 will be changed, which in turn will change the image displayed on the monitor. image. It should be noted that since the size range of the image display frame F01 will change according to the number of targets, in other words, the range of the variable trigger area A01 will also change accordingly.

The image adjustment program P07 outputs the image corresponding to the range of the image display frame F01 to the monitor. Here, the image adjustment procedure P07 is similar to the aforementioned initial image adjustment procedure P02, and will not be described again here. In addition, after the image adjustment program P07 is executed, the change detection program P03 will be executed again to form a loop.

To sum up, an image tracking and display method of the present invention is to convert the image on the display into an image corresponding to the trigger bounding box when it is found that the number of targets in the image changes, and the target action in the image After stabilizing, gradually adjust to the image corresponding to the appropriate image display frame. According to this, the image displayed on the monitor will only change the size of the output image when a change in the number of targets in the variable trigger area is detected. In other stable states, the image displayed on the monitor will not always be adjusted or changed. . Therefore, when the image tracking and display method is applied to a video conference, the images observed by participants during the conference can be more stable.

The above is only illustrative and not restrictive. Nothing left to take off Equivalent modifications or changes that depart from the spirit and scope of the present invention shall be included in the appended patent scope.

P01~P07: Program

Claims (8)

An image tracking and display method is used in conjunction with a camera lens and a display. The camera lens continuously captures an image. The image is correspondingly set with a trigger bounding box and a variable trigger area, including: A change detection program that determines whether there is an increase or decrease in the number of a target in the image corresponding to the variable trigger area; A display screen change procedure, when the result of the change detection procedure is "yes", adjust the camera lens to capture the image corresponding to the trigger bounding box range, and output the image to the display; A stable detection procedure to determine whether the variation parameter of the image corresponding to the triggering bounding box is less than or equal to a preset variation value; A target detection program performs target detection on the image corresponding to the trigger bounding box and generates an image display frame; and An image adjustment program outputs the image corresponding to the image display frame range to the display. The image tracking and display method as described in claim 1, wherein the change detection process is re-executed after the image adjustment process is completed. The image tracking and display method as described in claim 1, wherein the image adjustment procedure further includes adjusting the zoom magnification, a translation angle, a tilt angle and combinations thereof of the camera lens according to the image display frame. The image tracking and display method as described in claim 1, which further includes before the change detection process: An initial target detection process, performs an initial target detection on the image corresponding to the trigger bounding box, and generates an initial image display frame; and An initial image adjustment procedure is to output the image corresponding to the initial image display frame range to the display. The image tracking and display method of claim 1, wherein the trigger bounding box corresponds to one of the widest focal lengths of the camera lens. The image tracking and display method of claim 1, wherein the size of the trigger bounding box is smaller than one of the widest focal lengths of the camera lens. The image tracking and display method as described in claim 1, wherein the target detection procedure is performed by a humanoid detection technology. The image tracking and display method as described in claim 1, wherein the variable trigger area is an area between the trigger bounding box and the image display frame.

Images