TWI888137B - Obstacle avoidance camera system and method - Google Patents

Abstract

An obstacle avoidance shooting method includes: shooting a scene to generate a video according to a first parameter or a second parameter by a camera, wherein the video is configured to present a frame including a display area and a detection area; receiving the video and detecting at least one object in the detection area by a processor; adjusting the first parameter to the second parameter according to the movement direction of the at least one object by the processor, wherein the second parameter is configured to prevent the at least one object from showing in the display area; and receiving the video and presenting the display area by a display device.

Description

Obstacle avoidance shooting system and method

The present invention relates to a photography method, and more particularly to an obstacle avoidance photography system and method.

Stage performances are often videotaped or broadcast live.

However, unexpectedly moving audience members may block the target when passing by the camera, which may cause the picture to be blurred or out of focus, which will affect the visual experience of the audience watching the video. Even if a cameraman is arranged, there is no guarantee that all non-shooting targets that enter the picture can be avoided.

In view of this, the present invention proposes an obstacle avoidance shooting system and method to solve the problem of image occlusion and out-of-focus caused by unexpected movement of the audience during recording or live broadcasting.

According to an embodiment of the present invention, an obstacle avoidance shooting method includes: a camera shoots a scene according to a first parameter or a second parameter to generate a video, wherein the video is used to present a picture including a display area and a detection area; a processor receives the video and detects at least one object in the detection area; the processor adjusts the first parameter to a second parameter according to the moving direction of the at least one object, wherein the second parameter is used to avoid the at least one object from entering the display area; and a display device receives the video and presents the display area.

According to an embodiment of the present invention, an obstacle avoidance shooting system includes a camera, a processor, and a display device. The camera shoots a scene according to a first parameter or a second parameter to generate a video, and the video presentation includes a picture of a display area and a detection area. The processor is electrically connected to the camera to receive the video. The processor is used to detect at least one object in the detection area, and adjusts the first parameter to a second parameter according to the moving direction of the at least one object, wherein the second parameter is used to avoid the at least one object from entering the display area. The display device is electrically connected to the processor to receive the video. The display device is used to present the display area.

In summary, the obstacle avoidance shooting system and method proposed by the present invention can not only avoid non-shooting targets, but also take into account the naturalness of the picture to ensure the viewing quality during playback.

The above description of the disclosed content and the following description of the implementation methods are used to demonstrate and explain the spirit and principle of the present invention, and provide a further explanation of the scope of the patent application of the present invention.

The following detailed description of the features and advantages of the present invention is provided in the implementation mode, and the content is sufficient to enable any person skilled in the relevant art to understand the technical content of the present invention and implement it accordingly. Moreover, according to the content disclosed in this specification, the scope of the patent application and the drawings, any person skilled in the relevant art can easily understand the relevant purposes and advantages of the present invention. The following embodiments are to further explain the viewpoints of the present invention in detail, but are not to limit the scope of the present invention by any viewpoint.

FIG1 is a block diagram of an obstacle avoidance photography system according to an embodiment of the present invention. As shown in FIG1 , the obstacle avoidance photography system 10 comprises: a camera 1, a processor 3 and a display device 5.

Camera 1 shoots a scene according to the parameters to generate a video. FIG2 is a schematic diagram of an example of a camera shooting a scene. As shown in FIG2, the scene includes a target P1 on the stage, an audience P2 under the stage, and an object P3 that affects the stage photography. In one embodiment, camera 1 is a pan-tilt-zoom camera (PTZ camera), and its parameters include pan, tilt, and zoom.

The processor 3 is electrically connected to the camera 1 to receive video. Based on the example of FIG. 2 , FIG. 3 is a schematic diagram of an example of a video received by the processor 3 . As shown in FIG. 3 , the video screen A0 is divided into a display area A1 and a detection area A2 . In one embodiment, the display area A1 is located in the center of the screen A0 , and the detection area A2 is located below and on both sides of the screen A0 . In other embodiments, the detection area A2 includes at least one of the top, bottom, left and right sides of the screen A0 , and the display area A1 is the portion outside the detection area A2 . The processor 3 is used to detect an object P3 (such as a live audience standing up and walking) in the detection area A2 , and adjust the parameters during shooting according to the moving direction of the object P3 to prevent the object P3 from entering the display area A1 .

In one embodiment, the focusing operation of the camera 1 is limited to the target P1 in the display area, and will not focus on the object in the detection area A2.

The processor 3 may be a hardware device independent of the camera 1, such as a personal computer, a server or a smart phone. The processor 3 may also be a hardware device built into the camera 1, such as a microprocessor, an application processor (AP) or an application-specific integrated circuit (ASIC), but is not limited to the above examples.

The display device 5 is electrically connected to the processor 3 to receive and present the video. Since the video transmitted by the processor 3 to the display device 5 does not include the detection area A2, the image presented by the display device 5 only includes the display area A1. The display device 5 is, for example, a liquid crystal display or a light-emitting diode display, but is not limited to the above examples.

Fig. 4 is a flow chart of an obstacle avoidance photography method according to an embodiment of the present invention. The method includes steps S1 to S9, and can be implemented using the obstacle avoidance photography system 10 described above.

In step S1, the camera 1 captures a scene according to a first parameter to generate a video, wherein the first parameter includes a translation amount, a tilt amount, and a zoom amount.

In step S2, the processor 3 receives the video and determines whether there is an object P3 in the detection area A2. If the determination in step S2 is yes, the process proceeds to step S3. If the determination in step S2 is no, the process returns to step S1. In one embodiment, the video includes a plurality of frames, and the processor 3 subtracts a temporally preceding frame from a temporally succeeding frame, and determines whether there is an object P3 in the detection area A2 based on whether the difference between the two frames exceeds a threshold. In another embodiment, the processor 3 inputs a plurality of consecutive frames into an object detection algorithm, and determines whether there is an object P3 in the detection area A2 based on the output of the algorithm.

In step S3, the processor 3 determines the movement mode of the object P3. The movement modes include: moving away from the shooting target P1, moving left and right, and moving forward and backward. These three modes are not independent of each other. For example, the judgment result may meet the conditions of moving left and right and moving forward and backward at the same time. In one embodiment, after detecting the object P3, the processor 3 marks the bounding box containing the object P3 in the picture A0, and then calculates the distance between the bounding box and the boundary of the display area A1 in each frame in the next multiple frames. If the distance is getting larger and larger, it means that the object P3 is gradually moving away from the shooting target P1, and step S4 is executed at this time.

In step S4, since the object P3 will not enter the display area to cover the target P1, the processor 3 restores the first parameter to the initial parameter, that is, the parameter used when the camera 1 initially shoots the target P1.

FIG5 is a top view based on the example of FIG2. The decreasing distance between the bounding box and the center line of the screen A0 indicates that the object P3 may enter the display area A1, and precautions need to be taken. Specifically, the processor 3 calculates the moving direction D3 of the object P3 and decomposes the moving direction D3 into a first direction D31 and a second direction D32. The first direction D31 is perpendicular to the shooting direction D1 of the camera 1, indicating that the object P3 moves left and right in the screen A0. The second direction D32 is parallel to the shooting direction D1 of the camera 1, indicating that the object P3 moves forward toward the camera 1 or moves backward away from the camera in the screen A0.

In step S5, if the object P3 moves horizontally in the picture A0, in order to prevent the object P3 from entering the display area A1, the processor 3 increases the tilt amount (e.g., from 0 degrees to 10 degrees) and/or the zoom amount (e.g., from 1 times to 1.5 times) in the first parameter, so that the camera 1 is directed toward the upper center of the picture A0 (i.e., mainly the display area A1) to zoom in and shoot. In one embodiment, the processor 3 calculates the distance between the object P3 and the display area A1, adjusts the tilt amount when the distance is greater than a threshold, and adjusts the zoom amount when the distance is less than or equal to a threshold. In other embodiments, if the camera 1 is located higher than the stage, that is, shooting from above, then in step S5, the processor reduces the tilt to prevent the object P3 from entering the display area A1.

Before executing step S6, two points need to be considered: first, the object P3 moving forward and backward in the picture A0 may enter the display area A1; second, in order to avoid the above situation, the shooting posture of the camera 1 is adjusted, and whether the picture taken later will be cut to the shooting target P1. To this end, the processor 3 first performs the judgment of step S6, and then adjusts at least one of the translation amount and the zoom amount in the first parameter according to the result.

In step S6, the processor 3 detects whether the picture A0 is about to deviate excessively.

Referring back to FIG. 3 , in one embodiment, when the camera 1 uses the initial parameters to shoot the target P1 (before step S1), the processor 3 sets the core area A3 in the picture A0. The core area A3 is located in the display area A1 and is used to cover the shooting target P1. In one embodiment, the processor 3 inputs the video to the artificial intelligence model, and the artificial intelligence model determines which part of the picture A0 belongs to the core area A3. In other embodiments, the core area A3 can be set manually by a human.

Considering that the target P1 to be photographed by the camera 1 will move, the position of the core area A3 in the picture will change due to the translation or tilt of the camera 1, so the processor 3 needs to continuously detect the position of the core area A3 and make dynamic adjustments. On the other hand, if the camera 1 translates (or tilts) excessively in order to avoid the object P3, it will cause excessive deviation. Figure 6 is an example diagram of excessive deviation. As shown in Figure 6, because the camera 1 translates too much to the right, a part of the core area A3 enters the detection area A2, resulting in the audience seeing the left half of the target P1 cut off when the display device 5 presents the display area A1.

Based on the above example, the method proposed by the present invention is: when the camera 1 continues to pan until the edge of the core area A3 is about to approach the edge of the detection area A2 (i.e., the boundary between the display area A1 and the detection area A2), it will perform a small pan in the opposite direction (or directly stop panning) and zoom in to avoid the shooting target P1 from entering the detection area A2. Specifically, in an embodiment of step S6, the processor determines whether the edge of the core area A3 and the edge of the detection area A2 are less than a threshold. If the judgment of step S6 is no, then continue to step S7. If the judgment of step S6 is yes, then continue to step S8.

In step S7, the processor 3 adaptively adjusts the translation amount in the first parameter according to the moving direction of the object P3 in the picture A0. For example, if the object P3 moves to the left, the camera 1 should translate to the left to prevent the object P3 from appearing in the display area A1.

In an embodiment of step S8, the processor 3 restores the last translation adjustment and increases the zoom amount. For example, if the last time the image was translated to the right by 10 degrees in step S7, the image is translated to the left by 5 degrees in step S8, and zooms in after the image is pulled back, so as to maintain the target P1 in the display area A1.

In another embodiment of step S8, the processor 3 directly increases the zoom amount. For example, if the image was shifted 10 degrees to the right when entering step S7 last time, the image is not shifted to the right when entering step S8 this time, and zoom-in is directly performed to maintain the target P1 in the display area A1.

In step S9, that is, after steps S4, S5, S7 and S8 are completed, the first parameter is adjusted to the second parameter by the processor 3. The adjustment method includes: restoring to the initial parameter, adjusting at least one of the translation, tilt and zoom in the first parameter. Then, the processor 3 sends the second parameter to the camera 1 to adjust its shooting posture, and continues to shoot the scene to generate video of different pictures. In addition, the display device 5 receives the processed video from the processor 3 and presents the display area A1 in the picture shot according to the second parameter.

The obstacle avoidance shooting method proposed in the above embodiment is applicable to one or more objects. In other words, when the camera 1 avoids the first object P3, it will also avoid other objects according to the same rule.

In summary, the obstacle avoidance shooting system and method proposed by the present invention can not only avoid non-shooting targets, but also take into account the naturalness of the picture to ensure the viewing quality during playback.

Although the present invention is disclosed as above with the aforementioned embodiments, it is not intended to limit the present invention. Any changes and modifications made without departing from the spirit and scope of the present invention are within the scope of patent protection of the present invention. Please refer to the attached patent application for the scope of protection defined by the present invention.

10: Obstacle avoidance shooting system 1: Camera 3: Processor 5: Display device P1: Target P2: Audience P3: Object A0: Screen A1: Display area A2: Detection area A3: Core area D3: Moving direction D31: First direction D32: Second direction D1: Shooting direction S1~S9: Steps

FIG1 is a block diagram of an obstacle avoidance shooting system according to an embodiment of the present invention; FIG2 is a schematic diagram of an example of a camera shooting a scene; FIG3 is a schematic diagram of an example of a video signal received by a processor; FIG4 is a flow chart of an obstacle avoidance shooting method according to an embodiment of the present invention; FIG5 is a top view based on the example of FIG2; and FIG6 is a schematic diagram of an example of excessive deviation.

S1~S9: Steps

Claims (10)

A method for obstacle avoidance photography, comprising: Using a camera to shoot a scene according to a first parameter or a second parameter to generate a video, wherein the video is used to present a picture including a display area and a detection area; Using a processor to receive the video and detect at least one object in the detection area; Using the processor to adjust the first parameter to the second parameter according to the moving direction of the at least one object, wherein the second parameter is used to avoid the at least one object from entering the display area; and Using a display device to receive the video and present the display area. As for the obstacle avoidance shooting method of claim 1, each of the first parameter and the second parameter includes a translation amount, a tilt amount, and a zoom amount. As in the obstacle avoidance shooting method of claim 2, when the moving direction is perpendicular to the shooting direction of the camera, the processor adjusts the tilt amount and the zoom amount in the first parameter to generate the second parameter. As in the obstacle avoidance shooting method of claim 2, when the moving direction is parallel to the shooting direction of the camera, the processor adjusts at least one of the translation amount and the zoom amount in the first parameter to generate the second parameter. The obstacle avoidance shooting method of claim 4 further includes: Using the camera to shoot the scene according to an initial parameter to generate the video; Using the processor to set a core area in the display area; When it is detected that the distance between the core area and the detection area is less than a threshold, the processor adjusts the translation amount and the zoom amount in the first parameter; and When it is detected that the distance between the core area and the detection area is not less than the threshold, the processor adjusts the translation amount in the first parameter. An obstacle avoidance shooting system includes: A camera, shooting a scene according to a first parameter or a second parameter to generate a video, wherein the video is used to present a picture including a display area and a detection area; A processor, electrically connected to the camera to receive the video, the processor is used to detect at least one object in the detection area, and adjusts the first parameter to a second parameter according to the moving direction of the at least one object, wherein the second parameter is used to avoid the at least one object from entering the display area; and A display device, electrically connected to the processor to receive the video, the display device is used to present the display area. As for the obstacle avoidance shooting system of claim 6, each of the first parameter and the second parameter includes a translation amount, a tilt amount, and a zoom amount. As in claim 7, the obstacle avoidance shooting system, wherein the processor is further used to adjust the tilt amount and the zoom amount in the first parameter to generate the second parameter when the moving direction is perpendicular to the shooting direction of the camera. As in claim 7, the processor is further used to adjust at least one of the translation amount and the zoom amount in the first parameter to generate the second parameter when the moving direction is parallel to the shooting direction of the camera. As in claim 9, the camera is further used to capture the scene based on an initial parameter to generate the video, and the processor is further used to set a core area in the display area, adjust the translation and zoom in the first parameter when it is detected that the distance between the core area and the detection area is less than a threshold, and adjust the translation in the first parameter when it is detected that the distance between the core area and the detection area is not less than the threshold.

Images