KR102842494B1 - The method for streaming a video and the electronic device supporting same - Google Patents

Abstract

An electronic device according to one embodiment disclosed in the present document transmits an image to an external device, and includes a communication circuit, a memory for storing a plurality of images corresponding to a plurality of viewpoints, and a processor, wherein each of the plurality of images is composed of a plurality of sub-tracks, each of the plurality of sub-tracks is composed of consecutive frames, each of the frames has an attribute of one of a first type frame or a second type frame, and the plurality of sub-tracks have different proportions of the first type frames among the entire frames, and the processor transmits one of the plurality of sub-tracks of a first image for a first viewpoint among the plurality of images to the external device, receives a request for a viewpoint change from the external device through the communication circuit, and transmits, at a first time, a first sub-track including the first type frame among the plurality of sub-tracks of a second image for a second viewpoint corresponding to the request to the external device. In addition to these, various embodiments that can be understood through the specification are possible.

Description

Method for streaming video and electronic device supporting the same

Various embodiments of this document relate to a method for streaming video and an electronic device supporting the same.

A video service is being used that provides a user's terminal device with multi-view video captured simultaneously using multiple cameras. When streaming video corresponding to a single viewpoint from a server to a terminal device, if a request to switch viewpoints occurs on the terminal device, the server can stream video from a different viewpoint in response to the request. For example, at a baseball stadium, videos are captured using multiple cameras, and one video (e.g., video captured from the first base side) can be streamed to the user's terminal device (e.g., smartphone, tablet PC, laptop PC). If the user generates an input to switch viewpoints, a video captured from a different angle (e.g., video captured from the third base side) can be streamed.

Conventional methods for transmitting multi-viewpoint images include: i) a method of simultaneously receiving images from different viewpoints, ii) a method of additionally generating switching frames, iii) a method of transmitting two streams in a hierarchical structure, and iv) a method of transmitting additional images for viewpoint switching. The above methods may require an increase in network bandwidth or, due to the generation of additional frames, may increase encoding resources and costs.

A server device according to various embodiments of this document can reduce network bandwidth required for streaming by using sub-tracks having different frame structures for one point in time.

An electronic device transmits an image to an external device, and includes a communication circuit, a memory for storing a plurality of images corresponding to a plurality of viewpoints, and a processor, wherein each of the plurality of images is composed of a plurality of sub-tracks, each of the plurality of sub-tracks is composed of consecutive frames, each of the frames has an attribute of one of a first type frame or a second type frame, and the plurality of sub-tracks have different ratios of the first type frames among the entire frames, and the processor transmits one of the plurality of sub-tracks of a first image for a first viewpoint among the plurality of images to the external device, receives a request for a viewpoint change from the external device through the communication circuit, and transmits, at a first time, a first sub-track including the first type frame among the plurality of sub-tracks of a second image for a second viewpoint corresponding to the request to the external device.

An electronic device according to various embodiments disclosed in this document can reduce network bandwidth used for video playback in a replay video service by using sub-tracks having different frame structures.

An electronic device according to various embodiments disclosed in this document can reduce a response time to a user's viewpoint change request by using sub-tracks having different frame structures.

Figure 1 illustrates a video streaming system according to various embodiments.
Figure 2 illustrates a streaming server according to one embodiment.
FIG. 3 illustrates the storage and selection operations of multiple sub-tracks according to various embodiments.
FIG. 4 illustrates a video streaming method performed in a streaming server according to various embodiments.
Figure 5 illustrates dynamic changes in frame structure over time according to various embodiments.
FIG. 6 illustrates a video streaming method through transmission of frame structure information according to various embodiments.
FIG. 7 is a block diagram of an electronic device within a network environment according to various embodiments.
In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

Hereinafter, various embodiments of this document will be described with reference to the attached drawings. However, this is not intended to limit the technology described in this document to specific embodiments, and it should be understood that various modifications, equivalents, and/or alternatives of the embodiments of this document are included. In connection with the description of the drawings, similar reference numerals may be used for similar components.

Figure 1 illustrates a video streaming system according to various embodiments.

Referring to FIG. 1, a video streaming system (100) may include a source device (110), a streaming server (120), and a terminal device (130). The video streaming system (100) may include other devices between the components of FIG. 1. For example, a separate link server may be included between the source device (110) and the streaming server (120). As another example, a separate relay device (e.g., a desktop PC) may be included between the streaming server (120) and the terminal device (130).

According to various embodiments, the source device (110) can collect or store multi-view images. For example, the source device (110) may be a camera device capable of capturing images, a storage device capable of storing images captured by the camera device, or multiple image processing devices (e.g., multiple smartphones). In one embodiment, the multi-view images may be images captured from various angles of the same object, or images captured from various directions within the same or adjacent space. For example, the multi-view images may be images captured by installing multiple cameras at various angles within a baseball field. The images captured by each camera may be linked and stored according to the capture time, or may be streamed through a network.

According to various embodiments, a streaming server (120) (e.g., server (708) of FIG. 7) may receive video from a source device (110). The streaming server (120) may stream the received video to a terminal device (130). The streaming server (120) may receive real-time video from the video source device (110) or may receive stored video (e.g., video on demand (VOD) video).

In one embodiment, the streaming server (120) can receive multi-view images from the source device (110). Multi-view images can be images of the same object captured from various angles, or images captured from various directions within the same or adjacent space. For example, the streaming server (120) can receive images (or image sources) corresponding to N viewpoints by mapping them into N tracks.

According to various embodiments, the streaming server (120) may convert the received multi-view video into a plurality of sub-tracks (or a plurality of sub-videos) corresponding to each viewpoint (hereinafter, “viewpoint video”) having different GOP structures. For example, the first viewpoint video for the first viewpoint may be converted into a plurality of sub-tracks having different GOP structures. When each sub-track is decoded, it may be restored into an identical video as the first viewpoint video.

According to various embodiments, the streaming server (120) may transmit the received multi-view video to the terminal device (130). The streaming server (120) may stream the video of a point in time requested by the terminal device (130). For example, the streaming server (120) may stream the fifth view video (or fifth track) corresponding to the fifth view point to the terminal device (130) upon the request of the terminal device (130).

According to various embodiments, when the streaming server (120) receives a request to stream a different viewpoint video (hereinafter, a viewpoint change request) from the terminal device (130) while streaming a viewpoint video, the streaming server (120) may stream a viewpoint video corresponding to the viewpoint change request. For example, the streaming server (120) may receive a viewpoint change request from the terminal device (130) to change to a fifth viewpoint while streaming a 17th viewpoint video to the terminal device (130). The streaming server (120) may stream the fifth viewpoint video to the terminal device (130) in response to the viewpoint change request.

According to various embodiments, when there are multiple terminal devices (130), the streaming server (120) can provide streaming video to multiple terminal devices (130). In this case, since the streaming service must be provided using limited network resources (e.g., bandwidth), the streaming server (120) needs to efficiently transmit streaming data. In addition, in order to satisfy the user's service quality requirements, the streaming server (120) needs to quickly respond to a viewpoint change request requested by the terminal device (130) and quickly transmit the viewpoint video requested by each terminal device (130). The streaming server (120) can efficiently operate network resources (e.g., bandwidth) and quickly respond to a viewpoint change request from the terminal device (130) by using multiple sub-tracks having different frame structures for one viewpoint video.

According to various embodiments, the streaming server (120) may include a processor (150), memory (160), and communication circuitry (190).

The processor (150) can perform various operations required for the operation of the streaming server (120). The processor (150) can receive a multi-view image from the source device (110) via the communication circuit (190). The processor (150) can transmit a viewpoint image requested by the terminal device (130) among the multiple viewpoint images constituting the received multi-viewpoint image.

According to various embodiments, the processor (150) may transmit one point-of-view image among the replay images to the terminal device (130) via the communication circuit (190). For example, the streaming server (120) may stream a fifth point-of-view image (or a fifth track) corresponding to a fifth point-of-view to the terminal device (130) at the request of the terminal device (130).

According to various embodiments, the processor (150) may change the streamed viewpoint video in response to a viewpoint change request from the terminal device (130). For example, while streaming a 17th viewpoint video to the terminal device (130), the processor (150) may receive a viewpoint change request from the terminal device (130) to change to a 5th viewpoint. In response to the viewpoint change request, the processor (150) may stream the 5th viewpoint video to the terminal device (130).

According to various embodiments, the processor (150) can store a plurality of sub-tracks (or a plurality of sub-images) having different frame structures for one viewpoint image.

In one embodiment, multiple sub-tracks for a single viewpoint image may have different GOP (group of pictures) structures. Multiple sub-tracks constituting a single viewpoint image may have different I-frame ratios. For example, if a single viewpoint image is composed of first to third sub-tracks, the first sub-track may have all frames composed of I-frames. The second sub-track may have an I-frame ratio of 1/4 of the total frames and a P-frame ratio of 3/4. The third sub-track may have an I-frame ratio of 1/8 of the total frames and a P-frame ratio of 7/8.

According to various embodiments, the processor (150) may check various conditions such as network conditions and the number of terminal devices (130) to select one sub-track among a plurality of sub-tracks and transmit it to the terminal device (130).

According to various embodiments, the memory (160) can store various information required for the operation of the streaming server (120). According to one embodiment, the memory (160) can include an input buffer (170) and an output buffer (180). The input buffer (170) and the output buffer (180) can be configured to be distinguished by physical or logical addresses, or can be implemented as different storage devices. The input buffer (170) can store multi-view images. The input buffer (170) can store a plurality of sub-tracks for each view image. The plurality of sub-tracks can have different frame structures, and can be converted into images having the same quality when decoded. The output buffer (180) can store images to be transmitted to the terminal device (130). The output buffer (180) can temporarily store a view image (or a sub-track selected from among a plurality of sub-tracks constituting a view image) by switching of the processor (150).

According to various embodiments, the communication circuit (190) may transmit and receive data with an external device (e.g., a source device (110), a terminal device (130)) via wireless or wired communication. The communication circuit (190) may receive a point-of-view image from the source device (110) and transmit it to be stored in the input buffer (170). Alternatively, the communication circuit (190) may transmit an image stored in the output buffer (180) to the terminal device (130).

According to various embodiments, a terminal device (130) (e.g., an electronic device (701) of FIG. 7) may receive data regarding a single viewpoint image through a network and display the data through a display. There may be a plurality of terminal devices (130), and each terminal device may receive and output a viewpoint image desired by a user from a streaming server (120). The terminal devices (130) may be outputting images of different viewpoints. For example, when the terminal devices (130) include first to fifth terminal devices, the first to fifth terminal devices may each output different viewpoint images. Alternatively, the first to third terminal devices may output a first viewpoint image, the fourth terminal device may output a second viewpoint image, and the fifth terminal device may output a fourth viewpoint image. Viewpoint images selected by each user of the terminal devices (130) may be output in various ways.

Figure 2 illustrates a streaming server according to one embodiment.

Referring to FIG. 2, a streaming server (120) according to various embodiments may include an input buffer (170), a track selector (220), and an output buffer (180). In FIG. 2, the streaming server (120) is classified according to its function, but is not limited thereto. Some components may be added or omitted, or components may be integrated or separated. The streaming server (120) may not be a single device or a single server device, but may be a form in which multiple devices or multiple server devices are combined. Alternatively, at least some functions of the streaming server (120) may be performed by another device or terminal device (130).

The input buffer (170) according to various embodiments can store multi-view images. The input buffer (170) can store multiple sub-tracks for view images (250-1, 250-2, ... 250-N) corresponding to each view point. The multiple sub-tracks can have different frame structures and, when decoded, can be converted into images having the same quality. For example, the first view image (250-1) can be composed of three sub-tracks, and each sub-track can have a different I-frame rate.

Although FIG. 2 illustrates an example where a single viewpoint image is composed of three sub-tracks, the present invention is not limited thereto. Various embodiments regarding multiple sub-tracks stored in the input buffer (170) can be further provided through FIG. 3.

A track selection unit (220) according to various embodiments may include a first selection unit (or first switch) (221) that selects a viewpoint image (or track) in response to a viewpoint change request of a terminal device (130) and a second selection unit (or second switch) (222) that selects one of a plurality of sub-tracks (or sub-tracks) for transmission efficiency of the image.

The switching operation of the track selection unit (220) according to various embodiments may be an operation of a processor (e.g., processor (150) of FIG. 1) within the streaming server (120) or an operation under the control of the processor (150). Hereinafter, the operation of the track selection unit (220) (first selection unit (221) and second selection unit (222)) may be an operation under the control of the processor (150) of FIG. 1.

The switching operation in Fig. 2 is exemplary and is not limited thereto. The switching operation of the track selection unit (220) may be switching by software or switching by hardware elements.

According to one embodiment, the switching of the first selection unit (221) and the second selection unit (222) may be controlled by the same computational element or by different computational elements. The track selection unit (220) may also operate through communication with each other in different containers/virtualization servers/physical servers, depending on how the streaming server (120) is installed.

The first selection unit (221) according to various embodiments may switch (or select) a track to be streamed in response to a viewpoint change request from the terminal device (130). For example, when a viewpoint change request to switch to an Nth viewpoint is received from the terminal device (130) while streaming a first viewpoint image (250-1), the first selection unit (221) may switch the streamed track from the first track where the first viewpoint image (250-1) is transmitted to the Nth track where the Nth viewpoint image (201-N) is transmitted.

The second selection unit (222) according to various embodiments may select one of a plurality of sub-tracks according to specified conditions. For example, when a track is changed by the first selection unit (221), the sub-track having the highest I-frame ratio among the plurality of sub-tracks constituting the switched point image may be selected. Thereafter, the second selection unit (222) may select a sub-track having a gradually decreasing I-frame ratio among the plurality of sub-tracks (e.g., see FIG. 3).

The output buffer (180) according to various embodiments can store images to be transmitted to the terminal device (130). The output buffer (180) can temporarily store images transmitted through tracks selected by the first selection unit (221) and the second selection unit (222).

FIG. 3 illustrates the storage and selection operations of multiple sub-tracks according to various embodiments.

Referring to FIG. 3, the input buffer (170) according to various embodiments can store re-viewpoint images. The input buffer (170) can store multiple sub-tracks for each viewpoint image (250-1, 250-2, ... 250-N). The multiple sub-tracks can have different GOP (group of pictures) structures.

For example, the first sub-track for the first viewpoint image (250-1) )(250-1-1) can be composed of all I-frames (I-frame ratio (RI) = 1). The second sub-track for the first view image (250-1) )(250-1-2) contains one I-frame per four frames, and the rest can be composed of P-frames (I-frame ratio (RI)=1/4). The third sub-track for the first view image (250-1) )(250-1-3) contains one I-frame per eight frames, and the rest can be composed of P-frames (I-frame ratio (RI)=1/8).

For another example, the first sub-track for the Nth viewpoint image (250-N) )(250-N-1) can be composed of all I-frames (I-frame ratio (RI)=1). The second sub-track for the N-th viewpoint image (250-N) )(250-N-2) contains 4 I frames per 4 frames, and the rest can be composed of P frames (I-frame ratio (RI) = 1/4). The third sub-track for the N-th view image (250-N) ( )(250-N-3) contains one I-frame per eight frames, and the rest can be composed of P-frames (I-frame ratio (RI)=1/8).

According to various embodiments, a processor (e.g., processor (150) of FIG. 1) of a streaming server (120) may convert a source video received from a source device (e.g., source device (110) of FIG. 1) into a plurality of sub-tracks having different GOP structures and store the converted source video. Alternatively, the processor (150) may receive a source video converted into a plurality of sub-tracks for each viewpoint video from the source device (110) and store the converted source video.

According to various embodiments, multiple sub-tracks for one viewpoint image can all provide the same image quality, and due to the characteristics of inter-frame compression, the bitrate of each sub-track can increase in the order of the third sub-track (250-N-3) < the second sub-track (250-N-2) < the first sub-track (250-N-1). Accordingly, the network bandwidth required to play back images with the same image quality can increase in the order of the third sub-track (250-N-3) < the second sub-track (250-N-2) < the first sub-track (250-N-1). In order to provide the same streaming service to multiple users, when efficiently utilizing limited network bandwidth, a sub-image with a lower I-frame ratio can have a higher network priority. For example, the network priority may be higher in the order of 1st sub-track (250-N-1) < 2nd sub-track (250-N-2) < 3rd sub-track (250-N-3).

According to various embodiments, each of the plurality of sub-tracks constituting a viewpoint image may be composed of consecutive video frames. When a viewpoint switching request is received from a terminal device (130) while the image is being streamed, the processor (150) (e.g., the first selection unit (221) of FIG. 2) may switch from the first viewpoint image (250-1) being streamed to an Nth viewpoint image (or track) (250-N) corresponding to the viewpoint switching request. The processor (150) (e.g., the second selection unit (222) of FIG. 2) may select one of the plurality of sub-tracks (or sub-images) according to a specified condition.

According to various embodiments, at the first time when the viewpoint image is switched (or, the time when the processor (150) performs switching of the viewpoint image), the types of frames that each sub-track has may be different. For example, when switching from the first viewpoint image (250-1) being streamed at time t0 to the Nth viewpoint image (250-N), the frames that the first to third sub-tracks (250-N-1, 250-N-2, 250-N-3) that constitute the Nth viewpoint image (250-N) have may be I-frames, P-frames, and P-frames, respectively.

According to various embodiments, at a first time (e.g., t0) when the viewpoint image is switched, the processor (150) (e.g., the second selection unit (222) of FIG. 2) may determine the sub-track to be streamed as the first sub-track (250-N-1) having an I-frame and start streaming. In a comparative example, when a terminal device (e.g., the terminal device (130) of FIG. 1) receives a second sub-track (250-N-2) or a third sub-track (250-N-3) having a P-frame from the streaming server (120), the terminal device (130) may not be able to output a normal screen because it does not have a reference frame capable of decoding the received frame. On the other hand, in one embodiment of the present invention, when the first sub-track (250-N-1) having an I-frame is transmitted primarily, a reference frame may not be required and the terminal device (130) may provide a normal screen to the user.

According to various embodiments, when the second sub-track (250-N-2) maintains a viewpoint image at a second time (e.g., t1) having an I-frame (when no separate viewpoint switching request is received), the processor (150) (e.g., the second selection unit (222) of FIG. 2 ) may select and stream the second sub-track (250-N-2). Similarly, when the third sub-track (250-N-3) maintains a viewpoint image at a third time (e.g., t5) having an I-frame, the processor (150) (e.g., the second selection unit (222)) may select and stream the third sub-track (250-N-3). By sequential switching of the processor (150) (e.g., the second selection unit (222)), the ratio of I-frames of the streaming sub-tracks may be gradually reduced, and network efficiency may be increased.

According to various embodiments, the processor (150) can quickly set the change time between sub-tracks depending on the network conditions. For example, when the communication quality of the network is below a specified value, the processor (150) (e.g., the second selection unit (222) of FIG. 2) can select and stream the second sub-track (250-N-2) immediately at a second time (e.g., t1) when the second sub-track (250-N-2) has an I-frame. For another example, when the communication quality of the network is above a specified value, the processor (150) (e.g., the second selection unit (222) of FIG. 2) can select and stream the second sub-track (250-N-2) at a time other than the second time (e.g., t1) that is later.

According to various embodiments, the processor (150) may provide related information to the terminal device (130) when the frame structure before and after the viewpoint change is changed. For example, the third sub-track ( for the first viewpoint image (250-1) )(250-1-3)(I-frame rate (RI)=1/8) while streaming, the first sub-track ( for the Nth point-of-view video (250-N) is requested for a point-of-view change. )(250-N-1)(I-frame rate (RI)=1), the processor (150) may provide a message to the terminal device (130) indicating that a change in image quality may occur.

According to various embodiments, the processor (150) may set a period that restricts changes between sub-tracks based on specified conditions. The conditions may be based on information regarding the user's viewpoint switching history or pre-analyzed information regarding viewpoint switching.

FIG. 4 illustrates a video streaming method performed in a streaming server according to various embodiments.

Referring to FIG. 4, according to various embodiments, in operation 410, a processor of a streaming server (120) (e.g., processor (150) of FIG. 1) may stream to a terminal device (130) one of a plurality of sub-tracks constituting a first viewpoint image from a multi-viewpoint image. For example, the processor may be streaming using a sub-track having the lowest I-frame ratio among the plurality of sub-tracks.

According to various embodiments, the processor (150) may perform streaming by dequeuing the frame from the input buffer (170) and then enqueuing the frame into the output buffer (180).

According to various embodiments, in operation 420, the processor (150) of the streaming server (120) may determine whether a viewpoint change request is received from the terminal device (130). If the processor (150) does not receive a viewpoint change request (No in operation 420), the processor (150) may maintain the streaming state or continue streaming by changing to a sub-track with a higher network priority.

According to various embodiments, in operation 430, when the processor of the streaming server (120) receives a viewpoint change request from the terminal device (Yes in operation 420), at a first time for switching to a second viewpoint image corresponding to the request, the processor may stream a sub-track including a first type frame (e.g., an I-frame) among the second viewpoint images.

For example, in the example of FIG. 3, when a streaming point-in-time video is switched to an Nth point-in-time video (250-N), and at the time of switching, among the first to third sub-tracks (250-N-1, 250-N-2, 250-N-3) constituting the Nth point-in-time video (250-N), if only the first sub-track (250-N-1) includes an I-frame, the processor (150) can stream the first sub-track (250-N-1).

For another example, when a streaming point-in-time video is switched to an N-th point-in-time video (250-N), and at the time of switching, among the first to third sub-tracks (250-N-1, 250-N-2, 250-N-3) constituting the N-th point-in-time video (250-N), if both the first sub-track (250-N-1) and the second sub-track (250-N-2) include I-frames, the processor (150) can stream the second sub-track (250-N-2) having a higher network priority.

For another example, when a streaming point-in-time video is switched to an Nth point-in-time video (250-N), and at the time of switching, all of the first to third sub-tracks (250-N-1, 250-N-2, 250-N-3) constituting the Nth point-in-time video (250-N) include I-frames, the processor (150) can stream the third sub-track (250-N-3) having the highest network priority.

According to various embodiments, the processor (150) may select a sub-track based on the network conditions at the first time point, when switching from a first-time view image to a second-time view image. For example, if the number of terminal devices (130) receiving streaming data decreases rapidly or the bandwidth allocated to the network expands, a sub-track with a relatively high proportion of I-frames may be selected and streamed.

FIG. 5 illustrates dynamic changes in frame structure over time according to various embodiments.

Referring to FIG. 5, an input buffer (e.g., input buffer (170) of FIG. 5) according to various embodiments may store a multi-point image. The input buffer (170) may store a plurality of sub-tracks (550-1-1 to 550-1-3) for a point-in-time image (550-1) corresponding to one point-in-time.

According to various embodiments, the plurality of sub-tracks (550-1-1 to 550-1-3) may have different group of pictures (GOP) structures. For example, the plurality of sub-tracks (550-1-1 to 550-1-3) may be images having different I-frame rates.

According to various embodiments, a processor of a streaming server (120) (e.g., processor (150) of FIG. 1) can dynamically change the I-frame rate (RI) of a plurality of sub-tracks (550-1-1 to 550-1-3) according to specified conditions. For example, the processor (150) can increase or decrease the I-frame rate (RI) according to the relationship between computing resources or network bandwidth and the number of users using the service. Alternatively, the processor (150) can change the I-frame rate (RI) according to a user's track switching pattern or the properties (or characteristics) of the video to be transmitted.

According to various embodiments, in a first time interval (T1), a first sub-track ( )(550-1-1) can be composed entirely of I-frames (I-frame ratio (RI) = 1). The second sub-track for the first view image (550-1) )(550-1-2) contains two I frames per eight frames, and the rest can be composed of P frames (I-frame ratio (RI) = 1/4). The third sub-track for the first view image (550-1) )(550-1-3) contains one I-frame per eight frames, and the rest can be composed of P-frames (I-frame ratio (RI)=1/8).

According to various embodiments, in a second time interval (T2) after a first time interval (T1), a first sub-track ( )(550-1-1) can be continuously composed entirely of I-frames (I-frame ratio (RI) = 1). The second sub-track for the first view image (550-1) )(550-1-2) contains 4 I frames per 8 frames, and the rest can be composed of P frames (I-frame ratio (RI)=1/2). The third sub-track for the first view image (550-1) )(550-1-3) may include two I-frames per eight frames, and the remainder may be composed of P-frames (I-frame ratio (RI)=1/4). For example, if it is predicted that the user's track switching requests will be frequent in the second time interval (T2), the processor (150) may increase the I-frame ratio for a sub-track with a relatively high network priority, thereby quickly responding to the user's continuous track switching requests.

According to various embodiments, in the third time interval (T3) following the second time interval (T2), the same I-frame rate as in the first time interval (T1) can be maintained. For example, if the frequency of track change requests is expected to decrease again, the processor (150) can return the I-frame rate of the plurality of sub-tracks (550-1-1 to 550-1-3) to the state in the previous first time interval (T1).

According to various embodiments, the processor (150) can dynamically change the I-frame rate according to at least one condition among a network status, the number of terminal devices (130) receiving streaming services, whether the user frequently makes requests for viewpoint switching, and the characteristics of the viewpoint video provided.

FIG. 6 illustrates a video streaming method through transmission of frame structure information according to various embodiments. Referring to FIG. 6, a processor (e.g., processor (150) of FIG. 1) of a streaming server (120) according to various embodiments may transmit information (610) regarding the frame structure of a multi-view video (hereinafter, referred to as frame structure information) of the multi-view video to a terminal device (130). The frame structure information (610) regarding the multi-view video may be provided through the same channel as the channel that provides the streaming service to the terminal device (130) or through a separate channel.

According to various embodiments, the frame structure information (610) may include information about the entire multi-viewpoint image, or may include information about a portion of the multi-viewpoint image. For example, while receiving a first-viewpoint image, the terminal device (130) may receive frame structure information (610) for a second-viewpoint image with a high possibility of transition.

According to various embodiments, the processor of the terminal device (130) (e.g., the processor (720) of FIG. 7) can effectively respond to a user's request for a viewpoint change by using the received frame structure information (610). For example, the time taken for the actual track to change from the time the user generates an input related to a viewpoint change (e.g., a touch input or a voice input) (hereinafter, the reaction time (d_react)) may be an important quality factor of a streaming service. The reaction time (d_react) may be managed to remain below a specified value, and the reaction time (d_react) may be changed depending on the type of service or content.

According to various embodiments, the processor of the terminal device (130) (e.g., the processor (720) of FIG. 7) may change the timing at which the track is actually changed depending on the value of the reaction time (d_react). For example, if a timing change request occurs at time t0 and the reaction time (d_react) is set to 1, the terminal device (130) may request that the track be switched at time t1. For another example, if a timing change request occurs at time t0 and the reaction time (d_react) is set to 2, the terminal device (130) may request that the track be switched at time t2. In this case, the second sub-track may also include an I-frame, so that the processor of the terminal device (130) (e.g., the processor (720) of FIG. 7) may switch directly to the second sub-track without switching to the first sub-track. For another example, if a viewpoint change request occurs at time t0 and the reaction time (d_react) is set to 4, the terminal device (130) may request a track change at time t4. In this case, the third sub-track may also include an I-frame, so that the processor of the terminal device (130) (e.g., the processor 720 of FIG. 7) may switch directly to the third sub-track without switching to the first or second sub-track. For another example, if a viewpoint change request occurs at time t0 and the reaction time (d_react) is set to 8, the terminal device (130) may request a track change at time t8. In this case, the fourth sub-track may also include an I-frame, so that the processor of the terminal device (130) (e.g., the processor 720 of FIG. 7) may switch directly to the fourth sub-track without switching to the first, second, or third sub-track. The processor of the terminal device (130) (e.g., the processor (720) of FIG. 7) can select a sub-track with a high network priority, thereby improving communication efficiency.

According to various embodiments, when an input for switching a user's viewpoint occurs, the processor of the terminal device (130) (e.g., the processor (720) of FIG. 7) may transmit a viewpoint switching request to the streaming server (120) after waiting for a specified time corresponding to the set value of the reaction time (d_react).

According to various embodiments, the streaming server (120) can transmit replay video using various transmission protocols, such as real-time streaming protocols (e.g., RTSP, RTMP) or adaptive streaming protocols (e.g., HLS, DASH).

According to various embodiments, in the case of real-time streaming protocols (e.g., RTSP, RTMP), the streaming server (120) can store the status of the terminal device (130) and transmit the video frame by frame. The streaming server (120) can manage information on the track that the terminal device (130) connected to the streaming service is playing. When a viewpoint change request occurs, the streaming server (120) can select and transmit a track to be transmitted according to the characteristics of each frame of multiple tracks. The viewpoint change request can be transmitted in the form of a message of a protocol for video transmission or can be transmitted in another form. When the terminal device (130) uses the frame structure information (610), the streaming server (120) can postpone the viewpoint change request or attempt to directly switch to an appropriate track.

According to various embodiments, in the case of an adaptive streaming protocol (e.g., HLS, DASH), a streaming server (120) may transmit a manifest file, which is frame structure information (610) regarding a multi-view video, to a terminal device (130). For example, in the case of DASH, the address of the entire multi-track may be transmitted in XML format in a Media Presentation Description (MPD). A terminal device (130) that receives the manifest file may request and play the current segment of a decodable track.

According to various embodiments, the terminal device (130) can directly control track selection. When a request for a track change occurs, the terminal device (130) can select multiple tracks to be played during the track change and request segments of those tracks. If the user requests rapid, continuous track changes, the terminal device (130) can select an appropriate speed to select discontinuous tracks.

According to various embodiments, the streaming server (120) may store information regarding the response time (d_react) of the terminal device (130) in advance. In this case, the streaming server (120) may, taking into account the response time (d_react) of the terminal device (130), switch the viewpoint image after waiting for a specified time when a viewpoint switching request occurs.

According to various embodiments, the streaming server (120) may receive viewpoint images from different source devices (120). For example, the streaming server (120) may receive viewpoint images from the first source device, and viewpoint images from the second source device, among the first to fifteenth viewpoint images constituting the multi-viewpoint image.

According to various embodiments, the streaming server (120) or the terminal device (130) may request that some of the video in a section where viewpoint changes frequently occur or are likely to occur during streaming video be stored in an MEC (mobile edge computing) server within a base station supporting the network. When a viewpoint change request occurs, the terminal device (130) may first receive and display some of the video stored in the MEC server from the MEC server. The terminal device (130) may receive and display the video following the video stored in the MEC server from the streaming server (120). The streaming server (120) may transmit the video following the video stored in the MEC server to the terminal device (130).

According to various embodiments, the streaming server (120) or the terminal device (130) may request each source device (130) to prefetch data for a section in which switching occurs or is expected. In the above example, if there is a high possibility of switching from a 5th viewpoint image received from a 1st source device to a 10th viewpoint image received from a 2nd source device, the streaming server (120) or the terminal device (130) may request the 2nd source device to increase the prefetched data of the 10th viewpoint image while the 5th viewpoint image is being streamed.

FIG. 7 is a block diagram of an electronic device (701) within a network environment (700) according to various embodiments. The electronic device according to various embodiments disclosed in this document may be a device of various forms. The electronic device may include at least one of, for example, a portable communication device (e.g., a smart phone), a computing device (e.g., a personal digital assistant (PDA), a tablet PC, a laptop PC (desktop PC, workstation, or server), a portable multimedia device (e.g., an e-book reader or an MP3 player), a portable medical device (e.g., a heart rate, blood sugar, blood pressure, or body temperature monitor), a camera, or a wearable device. The wearable device may include at least one of an accessory type (e.g., a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD)), a fabric or clothing-integrated type (e.g., an electronic garment), a body-attached type (e.g., a skin pad or tattoo), or a bio-implantable circuit. In some embodiments, the electronic device may include, for example, a television, a digital video disk (DVD) player, an audio device, an audio accessory device (e.g., a speaker, a headphone, or a headset), a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a set-top box, a home appliance, or the like. It may include at least one of an automation control panel, a security control panel, a game console, an electronic dictionary, an electronic key, a camcorder, or an electronic picture frame.

In another embodiment, the electronic device may include at least one of a navigation device, a global navigation satellite system (GNSS), an event data recorder (EDR) (e.g., a black box for a vehicle/ship/airplane), an automotive infotainment device (e.g., a head-up display for a vehicle), an industrial or domestic robot, a drone, an automated teller machine (ATM), a point of sales (POS) device, a measuring device (e.g., a water, electricity, or gas meter), or an Internet of Things device (e.g., a light bulb, a sprinkler device, a fire alarm, a thermostat, or a streetlight). The electronic device according to the embodiment of the present document is not limited to the above-described devices, and may also provide functions of multiple devices in combination, such as, for example, a smartphone equipped with a function of measuring an individual's biometric information (e.g., heart rate or blood sugar). In this document, the term "user" may refer to a person using an electronic device or a device (e.g., an artificial intelligence electronic device) using the electronic device.

Referring to FIG. 7, in a network environment (700), an electronic device (701) may communicate with an electronic device (702) via a first network (798) (e.g., a short-range wireless communication network), or may communicate with an electronic device (704) or a server (708) via a second network (799) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (701) may communicate with the electronic device (704) via the server (708). According to one embodiment, the electronic device (701) may include a processor (720), a memory (730), an input device (750), an audio output device (755), a display device (760), an audio module (770), a sensor module (776), an interface (777), a haptic module (779), a camera module (780), a power management module (788), a battery (789), a communication module (790), a subscriber identification module (796), or an antenna module (797). In some embodiments, the electronic device (701) may omit at least one of these components (e.g., the display device (760) or the camera module (780)), or may have one or more other components added. In some embodiments, some of these components may be implemented as a single integrated circuit. For example, a sensor module (776) (e.g., a fingerprint sensor, an iris sensor, or an ambient light sensor) may be implemented embedded in a display device (760) (e.g., a display).

The processor (720) may control at least one other component (e.g., a hardware or software component) of the electronic device (701) connected to the processor (720) by executing, for example, software (e.g., a program (740)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (720) may load a command or data received from another component (e.g., a sensor module (776) or a communication module (790)) into a volatile memory (732), process the command or data stored in the volatile memory (732), and store the resulting data in a non-volatile memory (734). According to one embodiment, the processor (720) may include a main processor (721) (e.g., a central processing unit or an application processor), and a secondary processor (723) (e.g., a graphics processing unit, an image signal processor, a sensor hub processor, or a communication processor) that may operate independently or together therewith. Additionally or alternatively, the auxiliary processor (723) may be configured to use less power than the main processor (721) or to be specialized for a given function. The auxiliary processor (723) may be implemented separately from the main processor (721) or as part of it.

The auxiliary processor (723) may control at least a portion of functions or states associated with at least one component of the electronic device (701) (e.g., a display device (760), a sensor module (776), or a communication module (790)), for example, on behalf of the main processor (721) while the main processor (721) is in an inactive (e.g., sleep) state, or together with the main processor (721) while the main processor (721) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (723) (e.g., an image signal processor or a communication processor) may be implemented as part of another functionally related component (e.g., a camera module (780) or a communication module (790)).

The memory (730) can store various data used by at least one component (e.g., the processor (720) or the sensor module (776)) of the electronic device (701). The data can include, for example, software (e.g., the program (740)) and input data or output data for commands related thereto. The memory (730) can include a volatile memory (732) or a non-volatile memory (734).

The program (740) may be stored as software in the memory (730) and may include, for example, an operating system (742), middleware (744), or an application (746).

The input device (750) can receive commands or data to be used in a component of the electronic device (701) (e.g., a processor (720)) from an external source (e.g., a user) of the electronic device (701). The input device (750) can include, for example, a microphone, a mouse, a keyboard, or a digital pen (e.g., a stylus pen).

The audio output device (755) can output audio signals to the outside of the electronic device (701). The audio output device (755) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback, and the receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

A display device (760) can visually provide information to an external party (e.g., a user) of the electronic device (701). The display device (760) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display device (760) may include touch circuitry configured to detect a touch, or a sensor circuit (e.g., a pressure sensor) configured to measure the intensity of a force generated by the touch.

The audio module (770) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (770) can acquire sound through an input device (750), output sound through an audio output device (755), or an external electronic device (e.g., an electronic device (702)) (e.g., a speaker or headphones) directly or wirelessly connected to the electronic device (701).

The sensor module (776) can detect the operating status (e.g., power or temperature) of the electronic device (701) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (776) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface (777) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (701) with an external electronic device (e.g., the electronic device (702)). In one embodiment, the interface (777) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal (778) may include a connector through which the electronic device (701) may be physically connected to an external electronic device (e.g., the electronic device (702)). In one embodiment, the connection terminal (778) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

The haptic module (779) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (779) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module (780) can capture still images and videos. According to one embodiment, the camera module (780) may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module (788) can manage the power supplied to the electronic device (701). According to one embodiment, the power management module (788) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

A battery (789) may power at least one component of the electronic device (701). In one embodiment, the battery (789) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

The communication module (790) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (701) and an external electronic device (e.g., electronic device (702), electronic device (704), or server (708)), and the performance of communication through the established communication channel. The communication module (790) may operate independently from the processor (720) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (790) may include a wireless communication module (792) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (794) (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device (704) via a first network (798) (e.g., a short-range communication network such as Bluetooth, WiFi direct, or infrared data association (IrDA)) or a second network (799) (e.g., a long-range communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (792) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (796) to identify and authenticate the electronic device (701) within a communication network such as the first network (798) or the second network (799).

The antenna module (797) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (797) may include one antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (797) may include a plurality of antennas. In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (798) or the second network (799), may be selected from the plurality of antennas by, for example, the communication module (790). A signal or power may be transmitted or received between the communication module (790) and the external electronic device via the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., an RFIC) may be additionally formed as a part of the antenna module (797).

At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

According to one embodiment, commands or data may be transmitted or received between the electronic device (701) and an external electronic device (704) via a server (708) connected to a second network (799). Each of the external electronic devices (702, 1204) may be the same or a different type of device as the electronic device (701). According to one embodiment, all or part of the operations executed in the electronic device (701) may be executed in one or more of the external electronic devices (702, 704, or 708). For example, when the electronic device (701) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (701) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (701). The electronic device (701) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, or client-server computing technologies may be utilized, for example.

According to various embodiments, an electronic device (e.g., a streaming server (120) of FIG. 1, a server (708) of FIG. 7) transmits an image to an external device (e.g., a terminal device (130) of FIG. 1, an electronic device (701) of FIG. 7), and includes a communication circuit (e.g., a communication circuit (190) of FIG. 1), a memory (e.g., a memory (160) of FIG. 1) for storing a plurality of images corresponding to a plurality of points of view, and a processor (e.g., a processor (150) of FIG. 1), wherein each of the plurality of images is composed of a plurality of sub-tracks, each of the plurality of sub-tracks is composed of consecutive frames, each of the frames has an attribute of one of a first type frame or a second type frame, and the plurality of sub-tracks have different ratios of the first type frames among the entire frames, and the processor (e.g., the processor (150) of FIG. 1) transmits one of the plurality of sub-tracks of a first image for a first point of view among the plurality of images to the external device (e.g., a terminal device (130) of FIG. 1), The electronic device (701) of FIG. 7) transmits a request for a viewpoint change from the external device (e.g., the terminal device (130) of FIG. 1, the electronic device (701) of FIG. 7) through the communication circuit (e.g., the communication circuit (190) of FIG. 1), and, at a first time, transmits a first sub-track including the first type frame among a plurality of sub-tracks of a second image for a second viewpoint corresponding to the request to the external device (e.g., the terminal device (130) of FIG. 1, the electronic device (701) of FIG. 7).

According to various embodiments, the first sub-track may be a sub-track having the highest inclusion ratio of the first type frame among the plurality of sub-tracks of the second image.

According to various embodiments, the first sub-track may be a video in which all frames are composed of the first type frames.

According to various embodiments, the first time is a time for switching to the second image in response to the request, and the processor (e.g., the processor (150) of FIG. 1) can change to a second sub-track including the first type frame among the plurality of sub-tracks of the second image and transmit the change at a second time after the first time.

According to various embodiments, the second sub-track may have a lower inclusion ratio of the first type frame than the first sub-track.

According to various embodiments, the second sub-track may have a higher compression ratio of the image than the first sub-track.

According to various embodiments, the second sub-track may have a higher network priority than the first sub-track.

According to various embodiments, the processor (e.g., the processor (150) of FIG. 1) may sequentially change and transmit, after the first time, a sub-track having a lower ratio of the first type frame among the plurality of sub-tracks of the second image.

According to various embodiments, the processor (e.g., the processor (150) of FIG. 1) may transmit information about the frame configuration for the first image or the second image to the external device (e.g., the terminal device (130) of FIG. 1, the electronic device (701) of FIG. 7) before the first time.

According to various embodiments, the processor (e.g., processor (150) of FIG. 1) may change the frame configuration for the plurality of sub-tracks of the second image over time.

According to various embodiments, the processor (e.g., the processor (150) of FIG. 1) may change the frame configuration based on one of a network condition, the number of external devices (e.g., the terminal device (130) of FIG. 1, the electronic device (701) of FIG. 7), and an attribute of the first image or the second image.

According to various embodiments, the first image may have the same frame configuration as the second image.

According to various embodiments, the processor (e.g., processor (150) of FIG. 1) may receive the request, wait for a specified time, and then switch the image in response to the request.

According to various embodiments, the first type frame may be a frame that can be decoded alone, and the second type frame may be a frame that can be decoded using the second type frame.

According to various embodiments, the processor (e.g., processor (150) of FIG. 1) may control a first switch for switching corresponding to the request, and a second switch for selecting one of the plurality of sub-tracks of the second image.

According to various embodiments, the processor (e.g., processor (150) of FIG. 1) may wait until at least one of the plurality of sub-tracks of the second image is changed to the first type frame at the first time, when all of the plurality of sub-tracks of the second image are second type frames.

According to various embodiments, a method for transmitting an image is performed in an electronic device (e.g., a streaming server (120) of FIG. 1, a server (708) of FIG. 7) that transmits one of a plurality of images to an external device (e.g., a terminal device (130) of FIG. 1, an electronic device (701) of FIG. 7), and includes an operation of transmitting one of a plurality of sub-tracks of a first image for a first time point among the plurality of images to the external device (e.g., a terminal device (130) of FIG. 1, an electronic device (701) of FIG. 7), each of the plurality of sub-tracks being composed of consecutive frames, each of the frames having an attribute of one of a first type frame or a second type frame, and the plurality of sub-tracks having different ratios of the first type frames among the entire frames, and an operation of receiving a request for a time point change from the external device (e.g., a terminal device (130) of FIG. 1, an electronic device (701) of FIG. 7), an operation of switching to a second image for a second time point corresponding to the request, and a first image switched to the second image. At the time, the operation may include transmitting a first sub-track including the first type frame among the plurality of sub-tracks of the second image to the external device (e.g., the terminal device (130) of FIG. 1, the electronic device (701) of FIG. 7).

According to various embodiments, the operation of transmitting the first sub-track may include an operation of transmitting a sub-track having the highest inclusion ratio of the first type frame among the plurality of sub-tracks of the second image.

According to various embodiments, the act of transmitting the first sub-track may include the act of transmitting the first sub-track in which all frames are composed of the first type frames.

According to various embodiments, the image transmission method may further include an operation of changing and transmitting a second sub-track including the first type frame among the plurality of sub-tracks of the second image.

Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

The term "module" as used herein may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or portion of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document may be implemented as software (e.g., a program (740)) including one or more instructions stored in a storage medium (e.g., an internal memory (736) or an external memory (738)) readable by a machine (e.g., an electronic device (701)). For example, a processor (e.g., a processor (720)) of the machine (e.g., an electronic device (701)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, ‘non-transitory’ simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store ^™ ) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

According to various embodiments, each component (e.g., a module or a program) of the above-described components may include a single or multiple entities. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims (20)

In an electronic device that transmits images to an external device,
communication circuit;
A memory for storing multiple images corresponding to multiple points in time; and
Processor; including;
Each of the above multiple images is composed of multiple sub-tracks,
Each of the plurality of sub-tracks is composed of consecutive frames, and each of the frames has an attribute of one of a first type frame or a second type frame,
The above plurality of sub-tracks have different ratios of the first type frame among the entire frame,
The above multiple sub-tracks correspond to the same point in time,
The above processor
Transmitting one of the plurality of sub-tracks of the first image for the first time point among the plurality of images to the external device,
Through the above communication circuit, a request for a viewpoint change is received from the external device,
At a first time, the first sub-track having the highest inclusion ratio of the first type frame among the plurality of sub-tracks of the second image for the second time corresponding to the request is transmitted to the external device,
An electronic device that transmits, to the external device, a second sub-track, among a plurality of sub-tracks of a second image for the second time point, having a lower ratio of the first type frame than the first sub-track, at a second time point after the first time point. delete In the first paragraph, the first sub-track
An electronic device in which all frames are composed of the first type frames. In the first paragraph, the first time is,
An electronic device that switches to the second image in response to the above request. delete delete delete delete delete In the first paragraph, the processor
An electronic device for changing the frame configuration of the plurality of sub-tracks of the second image over time. delete delete delete In the first paragraph,
The above first type frame is an I-frame,
An electronic device wherein the above second type frame is a P-frame. In the first paragraph, the processor
A first switch that performs switching corresponding to the above request;
An electronic device controlling a second switch for selecting one of the plurality of sub-tracks of the second image. In the first paragraph, the processor
An electronic device that waits until at least one of the plurality of sub-tracks changes to the first type frame when all of the plurality of sub-tracks of the second image are second type frames at the first time. In a method of transmitting an image performed in an electronic device that transmits a plurality of images corresponding to a plurality of points of view to an external device,
An operation of transmitting one of a plurality of sub-tracks of a first image for a first point in time among a plurality of images to the external device, wherein each of the plurality of sub-tracks is composed of consecutive frames, each of the frames has an attribute of one of a first type frame or a second type frame, the plurality of sub-tracks have different ratios of the first type frames among the entire frames, and the plurality of sub-tracks correspond to the same point in time;
An action of receiving a request for a viewpoint change from the external device;
At a first time, an operation of transmitting, to the external device, a first sub-track having the highest inclusion ratio of the first type frame among a plurality of sub-tracks of a second image for a second time corresponding to the request; and
A method comprising: transmitting, at a second time after the first time, to the external device a second sub-track, among a plurality of sub-tracks of a second image for the second time, the second sub-track having a lower ratio of the first type frame than the first sub-track. delete delete delete