TWI626846B - Electronic device and method of sending video streaming - Google Patents

Abstract

The present invention provides a video streaming method for an electronic device, the method comprising: collecting an audio signal; encoding the collected audio signal to obtain digital audio data; controlling the first camera of the electronic device to capture the first static image and controlling The second camera captures the second still image; at least one of the first still image and the second still image is trimmed according to a specified clipping manner; and the first still image and the The second still image is synthesized; the synthesized image is encoded into an image of a specified format; the image of the specified format and the digital audio data are mixed into a video stream; and the video stream is transmitted to a remote device. The present invention can realize streaming live broadcast of the electronic device.

Description

Electronic device and method for transmitting video stream

The present invention relates to the field of electronic device management technologies, and in particular, to an electronic device and a method for transmitting a video stream.

Currently, mobile phones on the market cannot be streamed live with images taken by the front camera and the rear camera.

In view of the above, it is necessary to provide an electronic device that can be streamed and streamed in combination with images taken by the front camera and the rear camera.

In view of the above, it is also necessary to provide a method of transmitting a video stream, which can be streamed and streamed in combination with images taken by the front camera and the rear camera.

The electronic device includes: a first camera for capturing a first still image; a second camera for capturing a second still image; a microphone for collecting an audio signal; and an audio transcoder for encoding the microphone Acquiring the sound signal to obtain the digital audio data; the image processing module includes: an acquisition module, configured to control the first camera to capture the first still image, and control the second camera to capture the second still image a trimming module, configured to trim at least one of the first still image and the second still image according to a specified trimming manner; and a synthesizing module, configured to perform the first a still image and the second still image are synthesized, Thereby obtaining a synthesized image; a video transcoder for encoding the synthesized image into an image of a specified format; a mixer for mixing the image of the specified format and the digital audio data into a video stream; a communication module for transmitting the video stream to a remote device.

The video streaming method is applied to an electronic device, and the method includes: an acquiring step of acquiring an audio signal; a first encoding step of encoding the collected audio signal to obtain digital audio data; and the image processing step includes: controlling the electronic The first camera of the device captures the first still image and the second camera that controls the electronic device captures the second still image; at least the first still image and the second still image are in accordance with a specified clipping manner Performing a pruning; and synthesizing the first still image and the second still image according to a specified synthesis manner to obtain a synthesized image; and a second encoding step of encoding the synthesized image into a specified format And a mixing step of mixing the image of the specified format and the digital audio data into a video stream; and transmitting a step of transmitting the video stream to a remote device.

Compared with the prior art, the electronic device and the method for transmitting the video stream provided by the present invention can be combined with the images captured by the front camera and the rear camera for live streaming.

1‧‧‧Electronic equipment

10‧‧‧ processor

20‧‧‧Storage

30‧‧‧Microphone

40‧‧‧First camera

50‧‧‧second camera

60‧‧‧Input device

70‧‧‧Input controller

80‧‧‧Communication module

110‧‧‧Image Processing Module

1101‧‧‧Getting module

1102‧‧‧Cutting module

1103‧‧‧Synthesis module

120‧‧‧Video Transcoder

130‧‧‧Audio Transcoder

140‧‧‧mixer

2‧‧‧Server

3‧‧‧ Remote equipment

1 is a block diagram of a preferred embodiment of an electronic device of the present invention.

2 is a functional block diagram of a preferred embodiment of the image processing module of the present invention.

3 is a flow chart of a preferred embodiment of a method of transmitting a video stream in accordance with the present invention.

FIG. 4A illustrates 30 still images taken by the first camera and the second camera, respectively.

FIG. 4B illustrates adjusting the width of 30 still images captured by the first camera.

FIG. 4C illustrates the synthesis of still images taken by the first camera and the second camera.

FIG. 5 exemplifies a change of the combined position of the still image captured by the first camera and the second camera of FIG. 4C.

Referring to Figure 1, there is shown an architectural diagram of a preferred embodiment of the electronic device of the present invention. In this embodiment, the electronic device 1 includes, but is not limited to, a processor 10, a storage 20, a microphone 30, a first camera 40, a second camera 50, an input device 60, an input controller 70, and a communication module 80. . The above various components are electrically connected. In this embodiment, the electronic device 1 may be a device such as a mobile phone or a tablet computer.

The processor 10 can be a central processing unit, a microprocessor or other wafer with data processing functions.

The storage device 20 may be a storage device of the electronic device 1 itself, or may be an external storage device, such as a smart media card (Smart Media Card), a secure digital card (Secure Digital Card), or a flash memory card (Flash Card). Wait.

The microphone 30 can be used to collect sound signals. The first camera 40 and the second camera 50 may be a rear camera and a front camera of the electronic device 1, respectively.

In one embodiment, the first camera 40 and the second camera 50 are two cameras having different primitives. The primitive of the first camera 40 may be larger than the primitive of the second camera 50.

The input device 60 is configured to receive an input signal of a user. In this embodiment, the input device 60 can be a touch screen for receiving a touch input signal or a button for receiving a press input signal.

The input controller 70 can respond to the input signal of the input device 60 to generate a corresponding control signal. The communication module 80 can be a wireless communication module, for example, a Wifi device or a Bluetooth device. The electronic device 1 can use the communication module 80 to send a request signal, for example, requesting the server 2 to establish a transmission channel between the electronic device 1 and the remote device 3. In one embodiment, the server 2 can be a server such as Facebook. The remote device 3 can be a device such as a mobile phone or a computer.

In this embodiment, the processor 10 includes at least an image processing module 110 and a video. Transcoder 120, audio transcoder 130 and mixer 140.

In the embodiment, the image processing module 110 can control the first camera 40 and the second camera 50 to continuously capture a plurality of still images, perform size cropping on the captured still images, and synthesize the cropped images. The still image and the plurality of synthesized still images are successively sent to the video transcoder 120.

In one embodiment, the image processing module 110 can be a graphics processor integrated in the processor 10 or a software module executed by the processor 10. The image processing module 110 includes an acquisition module 1101, a trimming module 1102, and a synthesis module 1103. The modules 1101-1103 are programmable software instructions that can be executed by the processor 10 or the graphics processor. It is to be understood that in other embodiments, the modules 1101-1103 may also be program instructions or firmware that are solidified in the processor 10. The function of the modules 1101-1103 will be described with reference to FIG.

The video transcoder 120 may encode each still image synthesized by the image processing module 110 into an image of a specified format (for example, MPEG4 format), and transmit the image of the specified format to the mixer 140.

In this embodiment, the microphone 30 transmits the collected audio signal to the audio transcoder 130. The audio transcoder 130 can encode the audio signal into digital audio material and transmit it to the mixer 140.

In this embodiment, the mixer 140 may mix the image of the specified format and the digital audio data into a video stream. In one embodiment, the mixer 140 mixes the image of the specified format and the digital audio material into a video stream in response to a user input and stores the video stream in the storage 20. The mixer 140 can also transmit the video stream to the communication module 80, and then the communication module 80 is established between the electronic device 1 and the remote device 3 through the server 2. The transmission channel immediately transmits the video stream to the remote device 3, thereby implementing a live stream, and sharing the video stream to the remote device 3.

Referring to Figure 3, there is shown a flow chart of a preferred embodiment of the video transmission method of the present invention. The order of the steps in the flowchart may be changed according to different requirements, and some steps may be omitted or combined.

Step 301: The electronic device 1 sends a request signal to the server 2 by using the communication module 80 to request the server 2 to establish a transmission channel between the electronic device 1 and the remote device 3.

In an embodiment, the electronic device 1 can send the request signal in response to the first control signal generated by the input controller 70.

For example, when the input device 60 is a touch screen, the touch screen can be corresponding to the user touching a specified object displayed on the touch screen (for example, a streaming live icon). The first touch input signal, the input controller 70 can respond to the first touch input signal of the touch screen to generate a corresponding first control signal.

Step 302: The electronic device 1 determines whether the communication module 80 receives the feedback signal of the server 2, and the feedback signal is used to confirm that the server 2 has completed establishing the electronic device 1 and the remote device. A transmission channel between 3 and including related information of the transmission channel, such as a network address of the transmission channel. When the communication module 80 receives the feedback signal, steps S3031 and S3041 are simultaneously performed. When the communication module 80 does not receive the feedback signal, the process returns to step S301.

Step 3031: When the communication module 80 receives the feedback signal, the microphone 30 collects an audio signal.

Step 3032, the audio transcoder 130 encodes the audio signal collected by the microphone 30, thereby obtaining digital audio data. The audio transcoder 130 also transmits the digital audio material to the mixer 140.

In this embodiment, the audio transcoder 130 also embeds the acquisition time corresponding to the audio signal into the digital audio data when encoding the audio signal collected by the microphone 30, thereby causing the digital digit to be The audio data includes the corresponding acquisition time.

Step 3041: When the communication module 80 receives the feedback signal, the acquiring module 1101 controls the first camera 40 to capture a first still image and the second camera 50 to capture a second still image.

For example, referring to FIG. 4A, 30 first still images, that is, A1, A2, A3, ..., A30, are taken by the first camera 40 in the first second, and 30 second still images are B1. B2, B3, ..., B30 are taken by the second camera 50 in the first second. The shooting time of the first still image A1 is the same as the shooting time of the second still image B1. The shooting time of the first still image A2 is the same as the shooting time of the second still image B2. The shooting time of the first still image A3 is the same as the shooting time of the second still image B3. And so on, the shooting time of the first still image A30 is the same as the shooting time of the second still image B30. In this embodiment, the primitive of the first camera 40 is larger than the primitive of the second camera 50, so the first width “W1” of each first still image is greater than that of each second static image. The second width is "W2".

Step 3042, the trimming module 1102 trims one or both of the first still image and the second still image according to a specified trimming manner, so that the first still image and the second still image can be The specified synthesis method is used for synthesis.

In this embodiment, the specified pruning mode may be any one of the following first pruning mode, second pruning mode, and third pruning mode.

The first trimming manner may be to trim the widths of the first still image and the second still image to be consistent.

For example, the first width of the first still image is greater than the second width of the second still image. In one embodiment, the trimming module 1102 can be centered on a center point of the first still image. And reducing the first still image until the first width of the first still image is equal to the second width of the second still image.

For example, as shown in FIG. 4A, the trimming module 1102 reduces the first still image A1 and A2 centering on the center point of the first still image A1, A2, A3, ..., A30, respectively. , A3 ... A30 such that the first width of the first still image A1, A2, A3 ... A30 and the second still image B1, B2, B3 ... B30 The second width is equal, as shown in Figure 4B.

For example, the first width of the first still image is greater than the second width of the second still image. In other embodiments, the trimming module 1102 can be the second static image shown in FIG. 4A. The center point of the state image is centered, and the second still image is enlarged until the second width of the second still image is equal to the first width of the first still image, as shown in FIG. 4B.

For example, the first width of the first still image is greater than the second width of the second still image. In other embodiments, the trimming module 1102 can also directly cut the image shown in FIG. 4A. The left and right sides of the first still image are such that the first width of the first still image is equal to the second width of the second still image.

The second trimming manner may be to trim the heights of the first still image and the second still image to be consistent.

For example, the first height of the first still image is greater than the second height of the second still image. In one embodiment, the trimming module 1102 can be at the center of the first still image. The point is centered, and the first still image is reduced until the first height of the first still image is equal to the second height of the second still image.

Similarly, the trimming module 1102 may also center the center point of the second still image to enlarge the second still image until the second height of the second still image and the first of the first still image. The height is equal.

Similarly, the trimming module 1102 can also make the first height of the first still image equal to the second height of the second still image by directly cutting the upper and lower sides of the first still image.

The third trimming manner may be that the length and the width of the first still image and the second still image are respectively trimmed to be consistent.

Similarly, the first width of the first still image is greater than the second width of the second still image, and the first height of the first still image is greater than the second height of the second still image as an example, in one In an embodiment, the trimming module 1102 can make the first width of the first still image equal to the second width of the second still image by cutting the left and right sides of the first still image, and cutting through the first The upper and lower sides of the still image are such that the first height of the first still image is equal to the second height of the second still image.

In an embodiment, when the trimming module 1102 adopts the first trimming mode, the specified synthesizing manner refers to a first synthesizing manner, and the first synthesizing manner refers to the first static manner. The image and the second still image are synthesized from the vertical direction.

When the trimming module 1102 adopts the second trimming mode, the specified synthesizing mode refers to the second synthesizing mode, and the second synthesizing mode refers to the first still image and the second still image. Synthesize from the horizontal direction.

When the trimming module 1102 adopts the third trimming mode, the specified synthesizing manner refers to the first synthesizing mode or the second synthesizing mode.

In step 3043, the synthesizing module 1103 synthesizes the first still image and the second still image according to the specified synthesizing manner, and transmits the synthesized image to the video transcoder 120.

In this embodiment, the synthesizing module 1103 combines the first still image and the second still image corresponding to the same shooting time.

For example, the first trimming mode is adopted by the trimming module 1102, and the specified synthesizing manner refers to synthesizing the first still image and the second still image from a vertical direction as an example. Referring to FIG. 4C As shown, the synthesizing module 1103 combines the first still image A1 and the second still image B1 corresponding to the same shooting time into one image A1B1 from the vertical direction, and the first static corresponding to the same shooting time from the vertical direction. The image A2 and the second still image B2 are combined into one image A2B2, and the first still image A3 and the second still image B3 corresponding to the same shooting time are combined into one image A3B3 from the vertical direction. Similarly, the first still image A30 and the second still image B30 corresponding to the same shooting time are combined into one image A30B30 from the vertical direction.

Step S3044, the video transcoder 120 separately encodes the synthesized image into an image of a specified format, and transmits the encoded image to the mixer 140.

In one embodiment, the specified format may be in the MPEG4 format.

For example, the video transcoder 120 encodes the synthesized 30 images A1B1, A2B2, A3B3, ..., A30B30 into MPEG4 format images, and encodes the 30 MPEG4 images. The formatted image is transmitted to the mixer 140.

Step S305, the mixer 140 mixes the image of the specified format and the digital audio data into a video stream.

In this embodiment, the mixer 140 mixes the image of the specified format and the digital audio data according to the shooting time corresponding to the image of the specified format and the acquisition time included in the digital audio data. Video stream. That is, the mixer 140 mixes the image of the specified format and the digital audio data with the same shooting time and acquisition time.

For example, the mixer 140 mixes the 30 MPEG4 format images with the digital audio data corresponding to the first second into a video stream.

In an embodiment, the mixer 140 may store the video stream in the storage 20 when receiving the second control signal sent by the input controller 70.

For example, when the input device 60 is a touch screen, the touch screen can generate a corresponding number for a user to touch a specified object (for example, a storage icon) displayed on the touch screen. The touch controller can respond to the second touch input signal of the touch screen to generate a corresponding second control signal.

In step S306, the mixer 140 transmits the video stream to the communication module 80.

In step S307, the communication module 80 transmits the video stream to the remote device 3 through the transmission channel established by the server 2, so that the electronic device 1 performs streaming live broadcast. The video stream is shared to the remote device 3.

In an embodiment, the image processing module 110 may change the first static image and the first module when the third control signal sent by the input controller 70 is received. The method of synthesizing two static images.

For example, referring to FIG. 4C, when the input device 60 is a touch screen, the touch screen can be touched by the user on the synthesized image A1B1 displayed on the touch screen to generate a corresponding third. Touching the input signal, the input controller 70 can respond to the third touch input signal of the touch screen to generate a corresponding third control signal. Synthetic mode of the image processing module 110 The group 1103 can change the first still image and the second still image on the composite image A1B1, A2B2, A3B3, ..., A30B30 up and down according to the third control signal, as shown in FIG. 5.

It should be noted that the above embodiments are only intended to illustrate the technical solutions of the present invention and are not intended to be limiting, and the present invention will be described in detail with reference to the preferred embodiments thereof Modifications or equivalents of the technical solutions are not to be construed as a departure from the spirit and scope of the invention.

Claims (13)

An electronic device, comprising: a first camera for capturing a first still image; a second camera for capturing a second still image; a microphone for collecting an audio signal; and an audio transcoder for encoding The audio signal collected by the microphone, thereby obtaining digital audio data; the image processing module includes: an acquisition module, configured to control the first camera to capture the first still image and control the second camera to capture the a second static image; a trimming module, configured to trim at least one of the first still image and the second still image according to a specified trimming manner, so that the first still image and the second still image The synthesis can be performed in a specified synthesis manner; and the synthesis module is configured to synthesize the first still image and the second still image corresponding to the same shooting time according to the specified synthesis manner, thereby obtaining a synthesized An image transcoder for encoding the synthesized image into an image of a specified format; a mixer for specifying the image The type of digital video and audio data into a mixed video stream; and a communication module for the video stream to the remote device. The electronic device of claim 1, wherein the electronic device further includes a processor and a storage, wherein the acquisition module, the trimming module, and the synthesizing module are stored in the storage device And executed by the processor. The electronic device of claim 1, wherein the communication module sends a request signal to the server to request the server to establish a transmission channel between the electronic device and the remote device; The communication module transmits the video stream to the remote device in real time through the transmission channel. The electronic device of claim 1, wherein the audio transcoder embeds an acquisition time corresponding to the audio signal into the digital audio data when encoding the audio signal collected by the microphone And wherein the digital audio data includes a corresponding acquisition time; and the mixer mixes the image of the specified format and the digital audio data with the same shooting time and acquisition time. The electronic device according to claim 1, wherein the trimming method is to trim at least one of a width and a height of the first still image and the second still image to be identical. The electronic device according to claim 5, wherein the synthesizing means synthesizing the first still image and the second still image from a vertical direction or from a horizontal direction. The electronic device of claim 1, wherein the electronic device further comprises: an input device, configured to receive an input signal; and an input controller, configured to generate a corresponding control signal in response to the input signal, The synthesizing module changes the synthesizing manner according to the control signal. A video stream transmission method is applied to an electronic device, wherein the method includes: an acquisition step of acquiring an audio signal; a first encoding step of encoding the collected audio signal to obtain digital audio data; and the image processing step includes: controlling the The first camera of the electronic device captures the first still image and the second camera that controls the electronic device captures the second still image; at least the first still image and the second still image are in accordance with a specified clipping manner One of the trimming is performed such that the first still image and the second still image can be synthesized in a specified composite manner; and the first still image corresponding to the same shooting time is to be performed according to the specified composition manner Synthesizing with the second still image to obtain a synthesized image; a second encoding step of encoding the synthesized image into an image of a specified format; and a mixing step of capturing the image of the specified format and the digital audio data Mixing into a video stream; and transmitting a step of transmitting the video stream to a remote device. The video streaming method of claim 8, wherein the method further comprises: a requesting step of transmitting a request signal to the server to request the server to establish the electronic device and the remote device And a transmission channel; and in the transmitting step, transmitting the video stream to the remote device through the transmission channel. The video stream transmission method of claim 8, wherein: in the first encoding step, embedding an acquisition time corresponding to the audio signal into the digital audio material, thereby causing the digit The audio data includes a corresponding acquisition time; and in the mixing step, when the image of the specified format and the digital audio data are mixed into a video stream, the image of the specified format with the same shooting time and acquisition time is used. Mix with digital audio data. The video stream transmission method according to claim 8, wherein the trimming method is to trim at least one of a width and a height of the first still image and the second still image to be identical. The video streaming method according to claim 11, wherein the synthesizing means synthesizing the first still image and the second still image from a vertical direction or from a horizontal direction. The video streaming method of claim 8, wherein the method further comprises the steps of: receiving an input signal; and generating a corresponding control signal in response to the input signal, wherein in the image processing step, The said synthesis mode is changed according to the control signal.