CN109819236A - A kind of multi-user's VR live broadcast system based on unmanned plane binocular video - Google Patents
A kind of multi-user's VR live broadcast system based on unmanned plane binocular video Download PDFInfo
- Publication number
- CN109819236A CN109819236A CN201910115589.2A CN201910115589A CN109819236A CN 109819236 A CN109819236 A CN 109819236A CN 201910115589 A CN201910115589 A CN 201910115589A CN 109819236 A CN109819236 A CN 109819236A
- Authority
- CN
- China
- Prior art keywords
- video
- module
- data
- server
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 claims abstract description 15
- 230000002159 abnormal effect Effects 0.000 claims abstract description 6
- 238000009877 rendering Methods 0.000 claims abstract description 6
- 238000000034 method Methods 0.000 claims description 6
- 230000003139 buffering effect Effects 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 claims 2
- 230000000644 propagated effect Effects 0.000 claims 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000010899 nucleation Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
A kind of multi-user's VR live broadcast system based on unmanned plane binocular video, including camera drive module, video encoding module, wireless image transmission module, stream media distribution server, VR video playback module, VR control server；Camera drive module provides data source for video encoding module；Video encoding module is sent the video data of the coding output buffer by Real-time Transport Protocol in real time；Wireless image transmission module provides point-to-point LAN environment for the transmission of video data；Stream media distribution server, receiving unit and retransmission unit including realizing video data；VR video playback module, main reception, decoding and the rendering for realizing video data；VR control server establishes long connection with heart pattern and server, and determining server connect the warning message abnormal, push network connection disconnects with the device network, releases when receiving data packet abnormal；Have the characteristics that convenience, interactivity, authenticity and multi-user.
Description
Technical field
The invention belongs to VR direct seeding technique field field more particularly to a kind of multi-user VR based on unmanned plane binocular video
Live broadcast system.
Background technique
VR (VirtualReality, i.e. virtual reality, abbreviation VR) technology is that one kind can create and the experiencing virtual world
Computer simulation system, it generates a kind of simulated environment using computer, is a kind of Multi-source Information Fusion, interactive three
The system emulation for tieing up dynamic vision and entity behavior, is immersed to user in the environment.
Current VR experiences generally existing following characteristics.First, content-form is single, is substantially and wears the aobvious equipment body of head
The video that recorded clip is good in advance is tested, the feeling of immersion is lacked.Second, point-to-point single service does not support multi-user simultaneously
Experience.Third, charge method tradition need artificial on duty.
Summary of the invention
For could above-mentioned the deficiencies in the prior art, the purpose of the present invention is to provide a kind of based on unmanned plane binocular video
Multi-user's VR live broadcast system, the system can provide a kind of convenience, interactivity, the extremely strong multi-user virtual of authenticity for user
Real live-experience.
To achieve the above object, the technical solution adopted by the present invention is that:
A kind of multi-user's VR live broadcast system based on unmanned plane binocular video, including camera drive module, Video coding mould
Block, wireless image transmission module, stream media distribution server, VR video playback module, VR control server；
The camera drive module includes binocular camera mould group, and binocular camera mould group connects TX2 core board, at the end TX2
Webcam driver is realized using V4L2, and the MJPEG video flowing after exporting splicing provides data source for video encoding module；
MJPEG stream is decoded as by the video encoding module, main realize using CUDA high-performance decoder module
YUV420P is stored to video buffering queue to be encoded, then uses CUDA high-performance code module by the view of buffer area to be encoded
Frequency is output to coding output buffer according to H264/H265 format is encoded into, then passes through Real-time Transport Protocol for the coding output buffer
Video data send in real time；
The wireless image transmission module, mainly includes sending module and receiving module two parts, and sending module connects the end TX2
It is mounted on unmanned plane, receiving module is embedded in the unmanned aerial vehicle (UAV) control platform of earth station, provides a little pair for the transmission of video data
The LAN environment of point；
The stream media distribution server, main includes the receiving unit and retransmission unit for realizing video data；
Receiving unit is based on Real-time Transport Protocol, receives the H264/H265 data that the end TX2 sends over and is stored in reception buffer area；
Retransmission unit is realized based on the RTSP server of Live555, is responsible for that the data conversion of buffer area will be received into H264/
The video flowing of H265 format distributes propagation by RTSP agreement in a network in real time, while multiple RTSP clients being supported to draw stream.
The Live555 is the cross-platform c++ open source Streaming Media frame based on RTSP agreement, mainly includes
RTSP server and client；Its core function is that the video file of several format or audio file are converted into video
Stream or audio stream pass through the distribution of RTSP agreement in a network and propagate；
The VR video playback module, using VR equipment as carrier, main reception, decoding and the wash with watercolours for realizing video data
Dye；
It receives, is based primarily upon the RTSP client of Live555, that realizes video data receives and stores reception buffering
Area.
Decoding, using the hard encoding and decoding technique MediaCodec of Android, to H264/H265 format in reception buffer area
Data are decoded processing, and by way of shared out-pile memory with native layers of progress data exchange.
Rendering is mainly realized using OpenGL technology, and the video data that application layer is shared is split as a left side at native layers
Right eye reuses OpenVR SDK driving VR equipment, video data is sent into VR device plays.
The VR equipment selects HTC Vive Focus all-in-one machine.
The VR control server is based on Transmission Control Protocol, and each VR equipment establishes long connection with heart pattern and server,
If continuous 3 heart beat cycles of server do not receive device network data packet, determining server connect exception with VR device network,
The warning message that push network connection disconnects releases abnormal when being received again by data packet；
User generates order and pays success by the two dimensional code in scanning VR equipment；Server receives Third-party payment
The valid order mark of platform feedback issues the instruction for playing video to corresponding VR equipment；User is under the assistance of staff
It is correct to wear VR equipment, open VR live-experience.
The beneficial effects of the present invention are:
Compared with prior art, since camera drive module uses binocular camera mould group, without software mode splicing left and right
Eye video image, so having the advantages that firmly synchronous；Since video encoding module uses CUDA (Compute Unified
Device Architecture unifiedly calculates equipment framework) it realizes, so having the advantages that encoding-decoding efficiency is high；Since VR is regarded
The carrier (VR equipment) of frequency playing module selects HTC Vive Focus all-in-one machine, so having leisure to immerse, comfortable wearing, take
With convenient, abundant in content advantage；Since retransmission unit is that the RTSP server based on Live555 is realized, so having point pair
The broadcast advantage of multiple spot；Since the means of payment uses unattended self-service barcode scanning mode, so having energy saving, efficient excellent
Gesture.
Detailed description of the invention
Fig. 1 is system structure diagram of the invention.
Specific embodiment
Structural principle and working principle of the invention are described in further detail with reference to the accompanying drawings and examples.
Referring to Fig. 1, a kind of multi-user's VR live broadcast system based on unmanned plane binocular video, including camera drive module, view
Frequency coding module, wireless image transmission module, stream media distribution server, VR video playback module, VR control server；
The camera drive module includes binocular camera mould group, and binocular camera mould group connects TX2 core board, in TX2
Webcam driver is realized using V4L2 in end, and the MJPEG video flowing after exporting splicing provides data for video encoding module
Source；
The video encoding module, main realize use CUDA (Compute Unified Device
Architecture unifiedly calculates equipment framework) MJPEG stream is decoded as YUV420P storage to video by high-performance decoder module
Then buffering queue to be encoded uses CUDA high-performance code module by the video data encoding of buffer area to be encoded at H264/
H265 format is output to coding output buffer, then is sent out the video data of the coding output buffer in real time by Real-time Transport Protocol
It sees off.
The wireless image transmission module, mainly includes sending module and receiving module two parts, and sending module connects the end TX2
It is mounted on unmanned plane, receiving module is embedded in the unmanned aerial vehicle (UAV) control platform of earth station, provides a little pair for the transmission of video data
The LAN environment of point；
The stream media distribution server, main includes the receiving unit and retransmission unit for realizing video data；
Receiving unit is based on Real-time Transport Protocol, receives the H264/H265 data that the end TX2 sends over and is stored in reception buffer area；
Retransmission unit is realized based on the RTSP server of Live555, is responsible for that the data conversion of buffer area will be received into H264/
The video flowing of H265 format distributes propagation by RTSP agreement in a network in real time, while multiple RTSP clients being supported to draw stream.
Note: Live555 is the cross-platform c++ open source Streaming Media frame based on RTSP agreement, mainly includes RTSP
Server and client side.Its core function be the video file of several format or audio file are converted into video flowing or
Audio stream passes through the distribution of RTSP agreement in a network and propagates, the broadcast advantage with point-to-multipoint.
The VR video playback module, main reception, decoding and the rendering for realizing video data；
It receives, is based primarily upon the RTSP client of Live555, that realizes video data receives and stores reception buffering
Area.
Decoding, using the hard encoding and decoding technique MediaCodec of Android, to H264/H265 format in reception buffer area
Data are decoded processing, and by way of shared out-pile memory with native layers of progress data exchange.
Rendering is mainly realized using OpenGL technology.The video data that application layer is shared is split as a left side at native layers
Right eye reuses OpenVR SDK driving VR equipment, video data is sent into VR device plays.
The VR equipment selects HTC Vive Focus all-in-one machine, there is leisure to immerse, comfortable wearing, it is easy to carry,
Abundant in content advantage.
The VR control server is based on Transmission Control Protocol, and each VR equipment establishes long connection with heart pattern and server,
If continuous 3 heart beat cycles of server do not receive device network data packet, determining server connect exception with the device network,
The warning message that push network connection disconnects releases abnormal when being received again by data packet；
User generates order and pays success by the two dimensional code in scanning VR equipment；Server receives Third-party payment
The valid order mark of platform feedback issues the instruction for playing video to corresponding VR equipment；User is under the assistance of staff
It is correct to wear VR equipment, open VR live-experience.
Claims (5)
1. a kind of multi-user's VR live broadcast system based on unmanned plane binocular video, which is characterized in that including camera drive module, view
Frequency coding module, wireless image transmission module, stream media distribution server, VR video playback module, VR control server；
The camera drive module includes binocular camera mould group, and binocular camera mould group connects TX2 core board, is utilized at the end TX2
V4L2 realizes webcam driver, and the MJPEG video flowing after exporting splicing provides data source for video encoding module；
The video encoding module, it is main to realize that MJPEG stream, which is decoded as YUV420P, using CUDA high-performance decoder module deposits
Video buffering queue to be encoded is stored up, then uses CUDA high-performance code module by the video data encoding of buffer area to be encoded
It is output to coding output buffer at H264/H265 format, then passes through Real-time Transport Protocol for the video data of the coding output buffer
It sends in real time；
The wireless image transmission module, mainly includes sending module and receiving module two parts, and sending module connects the end TX2 and carries
On unmanned plane, receiving module is embedded in the unmanned aerial vehicle (UAV) control platform of earth station, for video data transmission provide it is point-to-point
LAN environment；
The stream media distribution server, main includes the receiving unit and retransmission unit for realizing video data；
The VR video playback module, using VR equipment as carrier, main reception, decoding and the rendering for realizing video data；
The VR control server is based on Transmission Control Protocol, and each VR equipment establishes long connection with heart pattern and server, if clothes
Business continuous 3 heart beat cycles of device do not receive device network data packet, then determining server connect abnormal, push with VR device network
It is connected to the network the warning message disconnected, is released when being received again by data packet abnormal.
2. a kind of multi-user's VR live broadcast system based on unmanned plane binocular video according to claim 1, which is characterized in that
The receiving unit is based on Real-time Transport Protocol, receives the H264/H265 data that the end TX2 sends over and is stored in reception buffer area；
The retransmission unit realized based on the RTSP server of Live555, be responsible for receiving the data conversion of buffer area at
The video flowing of H264/H265 format distributes propagation by RTSP agreement in a network in real time, while supporting multiple RTSP clients
Draw stream.
3. a kind of multi-user's VR live broadcast system based on unmanned plane binocular video according to claim 2, which is characterized in that
The Live555 is the cross-platform c++ open source Streaming Media frame based on RTSP agreement, mainly includes RTSP server
And client；Its core function is that the video file of several format or audio file are converted into video flowing or audio stream
It is propagated in a network by the distribution of RTSP agreement.
4. a kind of multi-user's VR live broadcast system based on unmanned plane binocular video according to claim 1, which is characterized in that
VR equipment selects HTC Vive Focus all-in-one machine.
5. a kind of multi-user's VR live broadcast system based on unmanned plane binocular video according to claim 1, which is characterized in that
The VR video playback module receives, is based primarily upon the RTSP client of Live555, realizes the reception of video data simultaneously
It stores and receives buffer area；It is decoded, using the hard encoding and decoding technique MediaCodec of Android, to H264/ in reception buffer area
The data of H265 format are decoded processing, and by way of shared out-pile memory with native layers of progress data exchange；Its
Rendering is mainly realized using OpenGL technology, the video data that application layer is shared is split as right and left eyes at native layers, then make
VR equipment is driven with OpenVR SDK, video data is sent into VR device plays.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910115589.2A CN109819236A (en) | 2019-02-15 | 2019-02-15 | A kind of multi-user's VR live broadcast system based on unmanned plane binocular video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910115589.2A CN109819236A (en) | 2019-02-15 | 2019-02-15 | A kind of multi-user's VR live broadcast system based on unmanned plane binocular video |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109819236A true CN109819236A (en) | 2019-05-28 |
Family
ID=66606592
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910115589.2A Pending CN109819236A (en) | 2019-02-15 | 2019-02-15 | A kind of multi-user's VR live broadcast system based on unmanned plane binocular video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109819236A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110164233A (en) * | 2019-06-06 | 2019-08-23 | 西安勺子智能科技有限公司 | A kind of Portable individual weapon simulated training system based on MR |
| CN111343425A (en) * | 2020-02-28 | 2020-06-26 | 深圳市凯达尔科技实业有限公司 | Virtual reality live broadcast control method based on unmanned aerial vehicle |
| CN111918383A (en) * | 2020-08-17 | 2020-11-10 | 大连交通大学 | Self-checking message synchronization method based on Bluetooth device and wireless local area network |
| CN112714281A (en) * | 2020-12-19 | 2021-04-27 | 西南交通大学 | Unmanned aerial vehicle carries VR video acquisition transmission device based on 5G network |
| CN112770046A (en) * | 2020-12-21 | 2021-05-07 | 深圳市瑞立视多媒体科技有限公司 | Generation method of control SDK of binocular USB camera and control method of binocular USB camera |
| CN113848986A (en) * | 2021-11-03 | 2021-12-28 | 广州港集团有限公司 | Unmanned aerial vehicle safety inspection method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102404561A (en) * | 2010-09-14 | 2012-04-04 | 盛乐信息技术（上海）有限公司 | Method for achieving moving picture experts group (MPEG) 4I frame encoding on compute unified device architecture (CUDA) |
| US20130027568A1 (en) * | 2011-07-29 | 2013-01-31 | Dekun Zou | Support vector regression based video quality prediction |
| CN105007465A (en) * | 2015-07-21 | 2015-10-28 | 江苏物联网研究发展中心 | Cipher-text H264 video live-broadcasting and on-demand platform based on live555 frame and ffmpeg library |
| CN105704501A (en) * | 2016-02-06 | 2016-06-22 | 普宙飞行器科技（深圳）有限公司 | Unmanned plane panorama video-based virtual reality live broadcast system |
| CN106412598A (en) * | 2016-09-13 | 2017-02-15 | 中山大学 | Wireless video transmission system based on CUDA display card coding and transmission method thereof |
| CN108347436A (en) * | 2018-01-31 | 2018-07-31 | 南京奇蛙智能科技有限公司 | A kind of unmanned plane long-distance video method for pushing based on high in the clouds |
-
2019
- 2019-02-15 CN CN201910115589.2A patent/CN109819236A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102404561A (en) * | 2010-09-14 | 2012-04-04 | 盛乐信息技术（上海）有限公司 | Method for achieving moving picture experts group (MPEG) 4I frame encoding on compute unified device architecture (CUDA) |
| US20130027568A1 (en) * | 2011-07-29 | 2013-01-31 | Dekun Zou | Support vector regression based video quality prediction |
| CN105007465A (en) * | 2015-07-21 | 2015-10-28 | 江苏物联网研究发展中心 | Cipher-text H264 video live-broadcasting and on-demand platform based on live555 frame and ffmpeg library |
| CN105704501A (en) * | 2016-02-06 | 2016-06-22 | 普宙飞行器科技（深圳）有限公司 | Unmanned plane panorama video-based virtual reality live broadcast system |
| CN106412598A (en) * | 2016-09-13 | 2017-02-15 | 中山大学 | Wireless video transmission system based on CUDA display card coding and transmission method thereof |
| CN108347436A (en) * | 2018-01-31 | 2018-07-31 | 南京奇蛙智能科技有限公司 | A kind of unmanned plane long-distance video method for pushing based on high in the clouds |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110164233A (en) * | 2019-06-06 | 2019-08-23 | 西安勺子智能科技有限公司 | A kind of Portable individual weapon simulated training system based on MR |
| CN111343425A (en) * | 2020-02-28 | 2020-06-26 | 深圳市凯达尔科技实业有限公司 | Virtual reality live broadcast control method based on unmanned aerial vehicle |
| CN111918383A (en) * | 2020-08-17 | 2020-11-10 | 大连交通大学 | Self-checking message synchronization method based on Bluetooth device and wireless local area network |
| CN111918383B (en) * | 2020-08-17 | 2023-10-27 | 大连交通大学 | Bluetooth device and wireless local area network-based self-checking message synchronization method |
| CN112714281A (en) * | 2020-12-19 | 2021-04-27 | 西南交通大学 | Unmanned aerial vehicle carries VR video acquisition transmission device based on 5G network |
| CN112770046A (en) * | 2020-12-21 | 2021-05-07 | 深圳市瑞立视多媒体科技有限公司 | Generation method of control SDK of binocular USB camera and control method of binocular USB camera |
| CN113848986A (en) * | 2021-11-03 | 2021-12-28 | 广州港集团有限公司 | Unmanned aerial vehicle safety inspection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109819236A (en) | A kind of multi-user's VR live broadcast system based on unmanned plane binocular video | |
| Liu et al. | Cutting the cord: Designing a high-quality untethered vr system with low latency remote rendering | |
| Xia et al. | WiserVR: Semantic communication enabled wireless virtual reality delivery | |
| CN107370714A (en) | The high efficiency communication method that facing cloud renders | |
| EP2674916B1 (en) | Moving image distribution server, moving image playback device, control method, program, and recording medium | |
| WO2019057194A1 (en) | Linked microphone-based live streaming method and system | |
| CN113099204A (en) | Remote live-action augmented reality method based on VR head-mounted display equipment | |
| CN107529069A (en) | A kind of video stream transmission method and device | |
| CN108235053B (en) | Interactive rendering method, device, terminal and system | |
| CN102611873A (en) | Method and system for realizing 2D/3D (two dimension/3 dimension) video communication and transmission optimization | |
| CN111773660A (en) | A cloud game processing system, method and device | |
| CN107979763A (en) | A kind of virtual reality device generation video, playback method, apparatus and system | |
| CN102595164A (en) | Method, device and system for sending video image | |
| CN105611220A (en) | Communication system based on holographic projection technology | |
| CN116912385A (en) | Video frame adaptive rendering processing method, computer device and storage medium | |
| CN102195894B (en) | The system and method for three-dimensional video-frequency communication is realized in instant messaging | |
| CN117692704A (en) | Virtual scene display method and system | |
| CN113141346B (en) | VR one-to-multiple system and method based on series flow | |
| CN120318893B (en) | Multi-mode face semantic communication method, device and medium | |
| CN116582686A (en) | An AI-based video coding method, device, device and storage medium | |
| CN102883213B (en) | Subtitle extraction method and device | |
| CN103533295A (en) | Full high-definition video meeting system | |
| Zeng et al. | A new architecture of 8k vr fov video end-to-end technology | |
| Pitkänen et al. | Remote VR gaming on mobile devices | |
| CN111901628A (en) | Cloud rendering method based on zSpace desktop VR all-in-one machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190528 |
|
| RJ01 | Rejection of invention patent application after publication |