TW201926982A - A panoramic camera augmented reality system - Google Patents

Abstract

The invention relates to a panoramic camera augmented reality system, comprising: a casing, a circuit board disposed in the casing; a first image acquisition device and a second image acquisition device fixed on the casing and electrically connected to the circuit board The parameters of the first image capturing device are different from the parameters of the second image capturing device, and the first image capturing device and the second image capturing device are respectively used for capturing a main scene image and An auxiliary scene image; and an image receiving unit and an image processing unit disposed in the casing and electrically connected to the circuit board; the image receiving unit is configured to receive the first image capturing device and the a main scene image and an auxiliary scene image acquired by the second image capturing device at the same time; the image processing unit is configured to the first image capturing device and the second image capturing device at the same time The collected main scene image and the auxiliary scene image are subjected to correction stitching processing to obtain a panoramic image.

Description

Panoramic camera augmented reality system

The present invention relates to the field of cameras, and more particularly to a panoramic camera augmented reality system.

The panoramic camera augmented reality system is widely used in banks, government agencies, prisons, etc., or for filming, because it can provide users with panoramic images of omnidirectional viewing angles and achieve seamless monitoring. Especially when used in movie scenes or self-portraits, it is often necessary to highlight a scene through background support. However, the existing panoramic camera augmented reality system can only realize a panoramic image of 360 degrees or 720 degrees, and can not reflect the primary and secondary angles of the image, and cannot simultaneously realize the part of the image that needs to be clearly displayed and the image with only the display. Like, it is not conducive to improving the user experience.

Therefore, it is necessary to provide a panoramic camera augmented reality system capable of solving the above problems.

A panoramic camera augmented reality system, comprising:

cabinet;

An image capture device fixed to the casing, the image capture device comprising a first image capture device and a second image capture device, the first image capture device and the second image The acquiring device is independently controlled, and the first image capturing device and the second image capturing device are different in at least one of a viewing angle, a focal length, a zooming magnification, and a resolution, so that the first image capturing device and The second image capturing device is respectively configured to capture images of different specifications;

An image receiving unit and an image processing unit disposed in the casing and electrically connected to the first image capturing device and the second image capturing device; the image receiving unit is configured to receive An image acquired by the first image capturing device and the second image capturing device at the same time;

The image processing unit is configured to perform a correction stitching process on the images acquired by the first image capturing device and the second image capturing device at the same time to obtain a panoramic image.

In a preferred embodiment, the panoramic camera augmented reality system further includes a storage module, the storage module stores an image and/or image of an external source, and the image processing unit is further configured to use the storage module. An image or image of an external source is extracted from the group and attached to a predetermined position of the panoramic image.

In a preferred embodiment, the panoramic camera augmented reality system further includes a communication module disposed in the casing and a controller, the communication module is configured to receive a remote control command sent by a mobile terminal, the remote control The instruction includes an image acquisition device to be controlled and an operation type to be executed by the image acquisition device to be controlled and/or an operation type to be performed by the image processing unit, the controller is configured to parse the remote control instruction, and according to the analysis As a result, the corresponding image acquisition device is controlled to perform the operation.

In a preferred embodiment, the types of operations that the image capture device need to perform include advancing the image capture device, pulling in the image capture device, adjusting the focus, and adjusting the aperture; the image processing unit needs to perform Types of operations include image stitching or image fit.

In a preferred embodiment, the parameter determining the photographing specification is any one or more of a viewing angle, a focal length, an optical zoom factor, a size of the image sensor, and a resolution.

In a preferred embodiment, a tripod is connected under the casing, and the tripod is provided with a rotating mechanism.

In a preferred embodiment, the stand is made of a material having a high thermal conductivity.

In a preferred embodiment, the second image acquisition device is disposed on the stand.

In a preferred embodiment, when the panoramic camera augmented reality system switches to a manual mode, the image capture device is manually controlled to perform the operation.

In a preferred embodiment, the image capture device is coupled to a power cord that extends away from the housing and connects to a battery away from the end of the image capture device.

In a preferred embodiment, the casing is a symmetrical structure or an asymmetric structure, and a solar panel is fixed on the casing, and the solar panel is used for absorbing sunlight and converting the absorbed sunlight into electric energy. And then outputting the electrical energy to the battery to charge the battery.

In a preferred embodiment, the casing is further provided with one or more of an HDMI output interface, a USB interface, and a TF card slot.

Compared with the prior art, since the panoramic camera augmented reality system includes a first image capturing device and a second image capturing device, the first image capturing device and the second image capturing device have a viewing angle, At least one of a focal length, a zoom factor, and a resolution is different, so that the first image capturing device and the second image capturing device are respectively used to capture images of different specifications, so that the obtained panoramic image can realize the main image. The sub-angles are beneficial to improve the user experience.

First embodiment

Referring to FIG. 1 to FIG. 2, a panoramic camera augmented reality system 1 is provided. The panoramic camera augmented reality system 1 includes a casing 10, a circuit board 20 disposed in the casing, and a first image capturing device 30 fixed to the casing 10 and electrically connected to the circuit board 20. And a second image capture device 40.

In the present embodiment, the casing 10 is substantially ellipsoidal. In another embodiment, the casing 10 may also have other shapes, such as square, rectangular, spherical, and the like, or various asymmetric structures or asymmetric structures.

The number of first image capture devices 30 is one or more. The number of the second image capturing devices 40 is one or more. In the embodiment, a first image capturing device 30 and a second image capturing device 40 are included as an example for description. The first image capturing device 30 and the second image capturing device 40 are oppositely disposed on the casing 10.

The first image capture device 30 and the second image capture device 40 are all independently controlled. The first image capturing device 30 determines that the parameter of the photographing specification is different from the parameter that the second image capturing device 40 determines the photographing specification, and the first image capturing device 30 and the second image capturing device 40 are different. They are used to capture the main scene image and the auxiliary scene image respectively, and the auxiliary scene image can be used to highlight the main scene image. The parameters determining the photographing specifications are any one or more of a viewing angle, a focal length, an optical zoom magnification, an image sensor size, and a resolution, and these parameters can determine the specifications of the captured image.

The first image capture device 30 includes a first optical lens 31 and a first image sensor 32 located behind the first optical lens 31. The second image capturing device 40 includes a second optical lens 41 and a second image sensor 42 located behind the second optical lens 41.

The first optical lens 31 and the second optical lens 41 are respectively used to collect optical signals of a scene in a shooting range thereof. For example, the first optical lens 31 is a wide-angle lens, and the first optical lens 31 has a viewing angle of 205 degrees; the second optical lens 41 is also a wide-angle lens, and the second optical lens 41 has a viewing angle of 205 degrees. And the binocular lens symmetrical structure is disposed on the casing 10, and the first optical lens 31, that is, the second optical lens 41 is responsible for the double hemisphere, that is, the 360 degree shooting of the scene is realized on the radial circumferential surface. The axial circumferential surface of the sphere also achieves a 360 degree vertical shooting of the scene, so that the two lenses are combined into a sphere, so that the panoramic camera augmented reality system 1 can cover a 360 degree or 720 degree panoramic field of view (field of view) , FOV).

It can be understood that the first image capturing device 30 and the second image capturing device 40 can also be asymmetrically disposed on the casing 10 by providing first image capturing devices 30 and second with different viewing angle ranges. The image capture device 40 is such that they can take pictures of different viewing angle ranges to obtain a 360 degree or 720 degree panoramic field of view image. The shooting angle of view of the first image capturing device 30 may be 8°, 28°, 45°, 94°, 120°, 152°, etc., in general, when the shooting angle or zoom factor of the first image capturing device 30 is changed. The shooting angle of view or the zoom factor of the second image capturing device 40 may be selected to change or remain unchanged.

The first optical lens 31 can also be externally connected with a fisheye lens to increase its angle of view. The resolution of the first optical lens 31 may be 7680×4320 (90-120p), and the zoom magnification may be 14, 22, 40, 76, 101 times.

The first image sensor 32 and the second image sensor 42 are respectively configured to receive optical signals collected by the corresponding first optical lens 31 and the second optical lens 41 and form an image.

The panoramic camera augmented reality system 1 further includes a storage module 50 that stores an image or image of an external source. Subsequently, the image processing unit 80 is further configured to be used from the storage module 50. Extract an image or image from an external source and attach it to a predetermined location on the panoramic image. The predetermined position may be the position where the image is stitched. For example, when the original picture stitching position is incomplete, or the image or picture of the stitching position is unclear, an external picture or image may be attached thereto to cover the unclear portion of the stitching position. The attached external image or image can be a still image or a moving image. In another embodiment, the panoramic camera enhanced display system 1 may further include a display module (not shown), and the display module is connected to the output display terminal. The output display terminal includes an LED display, a projection display, a mobile phone mobile display, a virtual reality glasses display, and the like.

Of course, it can be understood that the bonding can be performed before the image is stitched, the image is stitched, and after the image is stitched.

An image receiving unit 60 electrically connected to the circuit board 20 is further disposed in the casing 10. The image receiving unit 60 is configured to receive a main scene image and an auxiliary scene image collected from the first image sensor 32 and the second image sensor 42 at the same time.

One or more image processing units 70 (only one is shown) are disposed in the casing 10, and the image processing unit 70 is electrically connected to the circuit board 20. The image processing unit 70 is configured to acquire, from the image receiving unit 60, a main scene image and an auxiliary scene image respectively captured by the first image sensor 32 and the second image sensor 42 The main scene image and the auxiliary scene image are subjected to correction stitching to form a panoramic image.

The panoramic camera augmented reality system 1 is capable of switching between a remote control mode and a manual mode. When the panoramic camera augmented reality system 1 is in the remote control mode, the first optical lens 31 and the second optical lens 41 can be independently controlled by remote control commands to perform corresponding operations.

The user can send the remote control command to the panoramic camera augmented reality system 1 through a mobile terminal 2 via a wireless transmission mode such as WIFI or Bluetooth, thereby controlling the first optical lens 31 to perform a corresponding operation. The mobile terminal 2 can be a remote controller, a smart phone or a tablet computer or the like.

A communication module 80 is disposed on the casing 10, and the communication module 80 is configured to receive a remote control command sent by the mobile terminal 2. The remote control command includes a first optical lens 31 to be controlled and/or a second optical lens 41 and a type of operation performed by the optical lens to be controlled and/or a type of operation that the image processing unit 60 needs to perform.

The types of operations that the first and second image capturing devices 30, 40 need to perform include one of a ZOOM IN, a ZOOM OUT, a focal length adjustment, and an adjustment aperture. The types of operations that the image processing unit 60 need to perform include image stitching or image stitching.

A controller 90 is further disposed in the casing 10, and the controller 90 is electrically connected to the communication module 80. The controller 90 is configured to parse the remote control instruction, and control the corresponding first optical lens 31 or second optical lens 41 to perform the operation according to the analysis result. That is, if the user wants to control the first optical lens 311 or the second optical lens 41 to adjust the focal length, the corresponding button of the mobile terminal 2 can be pressed to transmit the remote control command. As such, the controller 90 can control the first optical lens 311 or the second optical lens 41 to adjust the focal length.

After receiving the remote control command, the panoramic camera augmented reality system 1 can cause the first optical lens 31 and the second optical lens 41 to be controlled to perform corresponding operations, thereby avoiding the problem of inconvenience of manual control in some occasions, Time and effort, which is conducive to improving the user experience.

When the first optical lens 31 and the second optical lens 41 are in the manual mode, the first optical lens 31 and the second optical lens 41 may further be manually controlled, that is, the first optical lens 31 and the first The advancement, zooming, zooming, adjusting the aperture, image stitching, and image fitting of the two optical lenses 41 can be achieved by manual control.

Specifically, the casing 10 may be provided with a plurality of mechanical buttons, and the user may not manipulate the panoramic camera augmented reality system 1 through the mobile terminal 2, but control the panoramic camera enhancement by pressing a specific mechanical button. The reality system 1 performs a specific operation.

Alternatively, the casing 10 is a touch screen, so that the user can directly operate the panoramic camera augmented reality system 1 by gesture sensing.

Of course, the first optical lens 31 and the second optical lens 41 can be further controlled by voice, and will not be described herein.

The material of the casing 10 may be a metal with a high thermal conductivity such as aluminum or copper to improve the heat dissipation effect. A heat sink (such as graphene) may be attached to the casing 10 to further improve the heat dissipation effect.

In another embodiment, as shown in FIG. 1 , the first image capturing device 30 and the second image capturing device 40 can still be connected to the power line 200 , and the power line 200 is away from the first image capturing device 30 . And the end of the second image capture device 40 extends out of the casing 10. Each end of each power cord 200 is provided with a USB interface 201, and each USB interface 201 is used to connect a battery 110. The battery 110 is configured to provide power to the panoramic camera augmented reality system 1 when the panoramic camera augmented reality system 1 is in operation. In the present embodiment, the power lines 200 of the different image capture devices 20 are intertwined to form a wound structure 202. Since the battery 110 is disposed outside the casing 10, it is convenient to disassemble or replace the battery. More specifically, the battery 110 can be a rechargeable battery.

A solar panel 120 is fixed on the casing 10, and the solar panel 120 is configured to absorb sunlight and convert the absorbed sunlight into electrical energy, and then output the electrical energy to the battery 110 to The battery 110 is charged.

In the above embodiment, the first image capturing device 30 and the second image capturing device 40 each have a battery 110, or the first image capturing device 30 and the second image capturing device 40 share a battery 110.

The first optical lens 31 and the second optical lens 41 are also switchable between a normal mode and a night vision mode. Specifically, an infrared filter is disposed on the first optical lens 31 and the second optical lens 41. When the first optical lens 31 and the second optical lens 41 are in the normal mode, the filter is opened to block infrared rays or far infrared rays from entering the first and second image sensors 32, 42, that is, The first and second image sensors 32, 42 can only sense visible light. When the first optical lens 31 and the second optical lens 41 are in the night vision mode, the infrared filter stops working, so that infrared rays or far infrared rays can enter the first and second image sensors 32. , 42 and imaged.

A tripod 130 is connected under the casing 10, and the tripod 130 is used to support the casing 10, and the height of the casing 10 can be adjusted. The stand 130 can be made of a material having a high thermal conductivity to retract heat generated by the various components in the casing 10 for heat dissipation. Alternatively, the tripod 130 has a heat pipe inside, for example, the heat pipe has a thermal fluid inside to remit heat generated by the image capturing device 20, the image processing unit 40 or the controller 90 to achieve heat dissipation. .

The casing 10 is provided with an HDMI output interface (not shown), a TF card slot (not shown), and the like; the HDMI output interface, the USB interface 201, and the TF card slot are electrically connected to the circuit board 10. Set the HDMI output interface, USB interface 201, TF card slot and other convenient cameras to connect with the host computer, mobile terminal and other devices to achieve data transmission and exchange. It can be understood that the external interface on the casing 10 is not limited to the above-mentioned interfaces, and other interfaces that can realize data transmission with the upper computer and the mobile terminal can also meet the requirements of the present invention.

The above control method of the panoramic camera augmented reality system 1 includes the following steps:

Step S1: The panoramic camera augmented reality system 1 is activated to control the panoramic camera augmented reality system 1 to enter a remote control mode.

Step S2: Select at least one first optical lens 31 and/or second optical lens 41 to be controlled by the mobile terminal 2.

Step S3: selecting, by the mobile terminal 2, the type of operation that the first optical lens 31 and the second optical lens 41 to be controlled need to perform and/or the type of operation that the image processing unit 60 needs to perform, thereby triggering the The mobile terminal 2 generates a remote control command and transmits the remote control command to the panoramic camera augmented reality system 1. The remote control command includes the first and/or second image capture devices 30, 40 to be controlled, and the types of operations and/or operations to be performed by the first and/or second image capture devices to be controlled. The type of operation that the image processing unit 60 needs to perform. The types of operations that the image capture device needs to perform include advancing the image capture device, pulling in the image capture device, adjusting the focus, and adjusting the aperture; the types of operations that the image processing unit need to perform include image stitching or Image fit. That is, operations such as photographing, image stitching, image sticking, and the like can be performed by the mobile terminal 2 transmitting a remote command.

Step S4: The controller 90 acquires the remote control command through the communication module 80, parses the remote control command, and controls the corresponding image capturing device and/or the image processing unit to perform the operation according to the analysis result.

Of course, the order of the above steps may be changed according to different requirements, and some steps may be omitted or merged.

Second embodiment

The panoramic camera augmented reality system 3 provided by the second embodiment is substantially the same as the panoramic camera augmented reality system 1 provided by the first embodiment, and the difference is that the panoramic camera augmented reality system 3 provided in this embodiment includes two first The image capturing device 30 and the two second image capturing devices 40, the two first image capturing devices 30 and the two second image capturing devices 40 are respectively disposed at the long-axis end positions of the ellipsoidal casing 10. And the position of the short-axis end portion. In other embodiments, the positions of the two first image capturing devices 30 and the two second image capturing devices 40 are not limited in the position of the casing 10, and the shooting scene can be taken according to actual needs. Decide.

The viewing angle, focal length, zoom magnification, and resolution parameters of the plurality of first image capturing devices 30 are the same.

The viewing angle, focal length, zoom magnification, and resolution parameters of the plurality of second image capturing devices 40 may be the same or different.

The number of the first image capturing device 30 and the second image capturing device 40 is multiple, and the remote camera can be sent to the panoramic camera augmented reality system 1 by the mobile terminal 2 according to actual needs or can be operated by manual operation. The panoramic camera augmented reality system 1 is configured to change the angle of view, the focal length, and the zoom magnification of the first image capturing device 30; or change the angle of view, the focal length, and the zoom magnification of the second image capturing device 40 to obtain photos of different specifications.

Third embodiment

The panoramic camera augmented reality system 4 provided by the third embodiment is substantially the same as the panoramic camera augmented reality system 1 provided by the first embodiment, and is different in the casing 10 of the panoramic camera augmented reality system 4 provided by the embodiment. Two first image acquisition devices 30 and six second image acquisition devices 40 are provided. Two first image capturing devices 30 are disposed at end positions of the long axis of the ellipsoid, and four second collecting devices 40 are respectively disposed on both sides of the first image capturing device 30, and the ellipsoidal casing 10 is Two second image acquisition devices are symmetrically arranged at the top position, and only one is shown in the figure, but the understanding is not affected.

And the tripod 130 is provided with a rotating mechanism 140, and the rotating mechanism 140 can be any one of an axle, a hinge structure or a ball bearing, so that the stand 130 can be relatively rotated for adjusting the The angle between the casing 10 and the stand 130, so that the casing 10 and the stand 140 can be folded to open or close. Of course, the housing 10 and the stand 130 can also be disposed in a detachable assembly structure, and details are not described herein.

A second image processing device 40 is further disposed on the stand 130 to perform framing for the scene below the panoramic camera 3.

In other embodiments, the number of the first image capturing device 30 and the second image capturing device 40 is determined according to actual needs, and details are not described herein.

In summary, the panoramic camera augmented reality system 1, 3, 4 includes at least one first image capturing device 30 and at least one second image capturing device 40, the first image capturing device 30 and At least one of the angle of view, the focal length, the zoom factor, and the resolution of the second image capturing device 40 is different, so that the remote camera can be sent to the panoramic camera augmented reality system 1 by the mobile terminal 2 or can be manually operated. Operating the panoramic camera augmented reality system 1 to change the angle of view, the focal length, and the zoom magnification of the first image capture device 30; or changing the angle of view, the focal length, and the zoom magnification of the second image capture device 40 to capture images of different specifications, The resulting panoramic image can achieve the primary and secondary angles, which is beneficial to improve the user's Augmented Reality experience.

In summary, the present invention has indeed met the requirements of the invention patent, and has filed a patent application according to law. However, the above description is only a preferred embodiment of the present invention, and the scope of the patent application of the present invention is not limited thereto. Equivalent modifications or variations made by persons skilled in the art in light of the spirit of the invention are intended to be included within the scope of the following claims.

1, 3, 4‧‧‧ panoramic camera augmented reality system

2‧‧‧Mobile terminal

10‧‧‧Chassis

20‧‧‧ boards

30‧‧‧First image acquisition device

40‧‧‧Second image acquisition device

50‧‧‧ memory module

60‧‧‧Image receiving unit

31‧‧‧First optical lens

41‧‧‧Second optical lens

42‧‧‧Second image sensor

32‧‧‧First image sensor

70‧‧‧Image Processing Unit

80‧‧‧Communication module

90‧‧‧ Controller

110‧‧‧Battery

120‧‧‧ solar panels

130‧‧ ‧ tripod

200‧‧‧Power cord

201‧‧‧USB interface

202‧‧‧ Winding structure

140‧‧‧Rotating mechanism

FIG. 1 is a schematic structural diagram of a panoramic camera augmented reality system according to a first embodiment of the present invention.

2 is a block diagram of a module of the panoramic camera augmented reality system shown in FIG. 1.

FIG. 3 is a schematic structural diagram of a panoramic camera augmented reality system according to a second embodiment of the present invention.

FIG. 4 is a schematic structural diagram of a panoramic camera augmented reality system according to a third embodiment of the present invention.

no

Claims (12)

A panoramic camera augmented reality system, comprising: a casing, a circuit board disposed in the casing; an image acquisition device fixed to the casing and electrically connected to the circuit board, the image acquisition device includes a first image capturing device and a second image capturing device, wherein the first image capturing device and the second image capturing device are independently controlled, and the first image capturing device determines parameters of the camera specification and The parameters of the second image capturing device that determine the photographing specifications are different, and the first image capturing device and the second image capturing device are respectively configured to capture a main scene image and an auxiliary scene image; An image receiving unit and an image processing unit in the casing and electrically connected to the circuit board; the image receiving unit is configured to receive the first image capturing device and the second image capturing a main scene image and an auxiliary scene image acquired by the device at the same time; the image processing unit is configured to collect the main scene collected by the first image capturing device and the second image capturing device at the same time The image and the auxiliary scene image are subjected to correction stitching processing to obtain a panoramic image. The panoramic camera augmented reality system of claim 1, wherein the panoramic camera augmented reality system further comprises a storage module, wherein the storage module stores images and/or images of an external source, the image processing unit It is also used to extract an image or image of an external source from the storage module and attach it to a predetermined position of the panoramic image. The panoramic camera augmented reality system of claim 2, wherein the panoramic camera augmented reality system further comprises a communication module and a controller disposed in the casing, wherein the communication module is configured to receive a mobile terminal Sending a remote control command, the remote control command comprising an image capture device to be controlled and an operation type to be executed by the image capture device to be controlled and/or an operation type to be performed by the image processing unit, the controller for parsing The remote control command controls the corresponding image acquisition device to perform the operation according to the analysis result. The panoramic camera augmented reality system of claim 3, wherein the type of operations that the image capturing device needs to perform includes advancing the image capturing device, pulling in the image capturing device, adjusting the focal length, and adjusting one of the apertures; The types of operations that the image processing unit needs to perform include image stitching or image stitching. The panoramic camera augmented reality system of claim 1, wherein the parameter determining the photographing specification is any one or more of a viewing angle, a focal length, an optical zoom factor, a size of the image sensor, and a resolution. The panoramic camera augmented reality system of claim 1, wherein a tripod is connected under the casing, and the tripod is provided with a rotating mechanism, and the panoramic camera augmented reality system is disposed on the rotating mechanism. The panoramic camera augmented reality system of claim 6, wherein the stand is made of a material having a high thermal conductivity. The panoramic camera augmented reality system of claim 6, wherein the second image acquisition device is disposed on the tripod. The panoramic camera augmented reality system of claim 4, wherein the image capture device is manually controlled to perform the operation when the panoramic camera augmented reality system switches to a manual mode. The panoramic camera augmented reality system of claim 1, wherein the image capture device is connected to a power cord that extends away from the housing and connects to a battery away from an end of the image capture device. The panoramic camera augmented reality system of claim 10, wherein the casing is a symmetrical structure or an asymmetric structure, and a solar panel is fixed on the casing, the solar panel is used for absorbing sunlight and The absorbed sunlight is converted into electrical energy, which is then output to the battery to charge the battery. The panoramic camera augmented reality system of claim 1, wherein the casing is further provided with one or more of an HDMI output interface, a USB interface, and a TF card slot.