JP2022116368A - Video processing device, display device, and video processing method - Google Patents

Abstract

An object of the present invention is to provide an output image having a display quality superior to that of the conventional art. Kind Code: A1 In a video processing device (1), a first receiving section (81) receives an input video, and a second receiving section (82) different from the first receiving section (81) corresponds to the input video. Receive display information. The display information includes metadata synchronized with the input video. The metadata includes attention information indicating an attention area of the input video. The video processing device (1) operates in either (i) a first mode in which the input video is processed using the display information, or (ii) a second mode in which the input video is processed without using the display information. mode. [Selection drawing] Fig. 1

Description

The following disclosure relates to video processing devices.

In recent years, various techniques have been proposed for further enhancement of video content. For example, technical development of a hybridcast broadcasting system has been made (see Non-Patent Document 1).

In a hybridcast broadcasting system, input video and display information corresponding to the input video (eg, metadata of the input video) are simultaneously delivered to the video processing device through separate paths. By processing the input video based on the display information in the video processing device, it is possible to provide a more attractive video (output video) to the user (viewer).

“Expanding Functions of Broadcasting and Telecommunications Linked Systems - Toward Advanced Hybridcast -”, Toshiyuki Oya, NHK STRL R&D/No.146/2014.8

However, as will be described later, in a broadcasting system in which input video and display information are distributed at the same time, there is a concern that the display information may not be properly received by the video processing device. In such a case, the display quality of the output video may deteriorate. However, in the prior art (eg, Non-Patent Document 1), no particular consideration is given to such a problem.

One disclosure of the present invention has been made in view of the above problems, and an object of the present invention is to provide an output image having a display quality superior to that of the conventional art.

In order to solve the above problems, a video processing device according to an aspect of the present disclosure is a video processing device that generates an output video by processing an input video, a first receiving unit that receives the input video and a second receiving section for receiving display information corresponding to the input video, wherein the second receiving section is a receiving section different from the first receiving section, and the display information is the Metadata synchronized with the input image is included, the metadata includes attention information indicating an attention area of the input image, and the image processing device is configured to: (i) process the input image using the display information; and (ii) a second mode in which the input image is processed without using the display information.

To solve the above problems, a video processing method according to an aspect of the present disclosure is a video processing method for generating an output video by processing an input video, wherein the input video is received by a first receiving unit. and a second receiving step of receiving display information corresponding to the input video by a second receiving unit different from the first receiving unit, wherein the display information is the input including metadata synchronized with video, the metadata including attention information indicating an attention area of the input image, the image processing method comprising: (i) processing the input image using the display information; and (ii) a second mode in which the input image is processed without using the display information.

According to one aspect of the present disclosure, it is possible to provide an output image with better display quality than conventional.

2 is a block diagram showing the configuration of the main part of the display device of Embodiment 1; FIG. It is a figure explaining the expansion process in ZOOM mode. FIG. 11 is another diagram illustrating enlargement processing in ZOOM mode; FIG. 4 is a diagram illustrating the relationship between an input image and an output image in ZOOM mode; FIG. 10 is a block diagram showing the configuration of a main part of a display device of a comparative example; It is a figure explaining an example of the process in a comparative example. 4 is a sequence diagram illustrating the flow of processing of the video processing device according to the first embodiment; FIG. 4 is a flowchart illustrating the overall flow of processing of the video processing device of Embodiment 1; 4 is a flowchart illustrating the flow of video processing when the ZOOM mode is preselected; 10 is a flowchart illustrating the flow of video processing when the TO_ORIGINAL mode is preselected; It is a figure explaining the expansion process in TO_ORIGINAL mode. FIG. 10 is a diagram showing an example of a zoom-out area change according to an increase in PROCESS_CNT; 7 is a flowchart illustrating the flow of video processing when the ORIGINAL mode has been selected in advance; 4 is a diagram illustrating temporal change of α in Embodiment 1. FIG. It is a figure explaining the effect of Embodiment 1 by comparison with a comparative example. It is a figure explaining the effect of Embodiment 1 by comparison with a reference example. 10 is a flowchart illustrating the flow of image processing when the TO_ZOOM mode is preselected in the second embodiment; FIG. 10 is a diagram illustrating temporal change of α in Embodiment 2; FIG. FIG. 11 is a diagram illustrating temporal changes of α in Embodiment 3; 10 is a flowchart illustrating the flow of video processing when the ORIGINAL mode is preselected in Embodiment 3; FIG. 12 is a diagram illustrating temporal changes of α in Embodiment 4; FIG. 12 is a diagram illustrating temporal change of α in Embodiment 5; It is a figure explaining the effect of Embodiment 5 by comparison with Embodiment 1. FIG.

[Embodiment 1]
The display device 100 of Embodiment 1 will be described below. For convenience, members having the same functions as the members described in the first embodiment are denoted by the same reference numerals in the subsequent embodiments, and description thereof will not be repeated. Descriptions of matters similar to those of known technology are also omitted as appropriate.

The device configuration shown in each figure is merely an example for convenience of explanation. Further, each numerical value described below in the specification is also a mere example. As used herein, the description "A to B" for two numbers A and B shall mean "greater than or equal to A and less than or equal to B" unless otherwise specified.

(Overview of display device 100)
FIG. 1 is a block diagram showing the configuration of the essential parts of the display device 100. As shown in FIG. The display device 100 includes the video processing device 1 and a display section 90 . The video processing device 1 acquires various data including the input video MOV1 from the broadcasting system 1000 . Hereinafter, the input video MOV1 is also simply abbreviated as MOV1. Other symbols are appropriately abbreviated in the same way. The video processing device 1 generates an output video MOV2 by processing MOV1. The video processing device 1 causes the display unit 90 to display the MOV2. Note that MOV1 is also called an original video.

Broadcast system 1000 simultaneously distributes MOV1 and display information (information corresponding to MOV1). The broadcasting system 1000 comprises (i) a first network NT1 for distributing MOV1 and (ii) a second network NT2 for distributing display information. It is assumed that the display information includes metadata synchronized with MOV1 (more specifically, video information metadata). The metadata indicates attention information indicating an attention area of MOV1 (more strictly, an attention area of each frame of MOV1) (eg, an ATT described later).

Broadcast system 1000 in the example of FIG. 1 is a hybridcast broadcast system. Therefore, NT1 and NT2 in Embodiment 1 are assumed to be a broadcasting network and an IP (Internet Protocol) communication network, respectively.

However, the broadcasting system 1000 is not limited to a hybridcast broadcasting system. The broadcasting system 1000 may be a video distribution system that distributes the MOV1 and the display information synchronously through separate paths. Therefore, NT1 and NT2 may be different communication networks. Similarly, the first receiver 81 and the second receiver 82 described below may be different receivers (communication interfaces).

NT1 transmits broadcast waves to video processing apparatus 1 . A broadcast wave is an example of a carrier wave carrying MOV1. The NT 2 transmits display information to the video processing device 1 . NT2 transmits display information to video processing apparatus 1 in synchronization with transmission of MOV by NT1.

The video processing device 1 includes a control section 10 , a first reception section 81 and a second reception section 82 . The control unit 10 comprehensively controls each unit of the video processing device 1 (and the display device 100). The control unit 10 includes a video processing unit 110 , a display information reception determination unit 120 , a scene change detection unit 130 and a selection unit 140 . The operation of each section of the control section 10 will be described later. The video processing device 1 also includes various storage devices (eg, frame memory) not shown.

The first receiver 81 acquires (receives) MOV1 via NT1. In the example of FIG. 1, first receiver 81 includes broadcast tuner 810 and video decoder 820 . Broadcast tuner 810 receives broadcast waves. The video decoder 820 acquires (i) MOV1 and (ii) the time stamp of the MOV1 by decoding the broadcast wave. In the following description, unless otherwise specified, "time stamp" refers to "time stamp of MOV1". The first receiving unit 81 supplies the acquired MOV1 and time stamp to the control unit 10 .

The second receiver 82 acquires display information via NT2. The second receiver 82 may be a known IP communication interface. The second receiver 82 supplies the acquired display information to the controller 10 .

FIG. 2 is a diagram for explaining enlargement processing in a ZOOM mode (described later). In MOV1 of FIG. 2, a scene (one scene of the main content described later) showing the whole bodies of two people is drawn. The attention area ATT of MOV1 is a partial area of MOV1 and is an area targeted for zoom-in display (focus display). In other words, ATT can be said to be an area intended to attract the user's (viewer's) attention in the area of MOV1.

In FIG. 2, for convenience of explanation, ATT is shown superimposed on MOV1. In this specification, the shape of ATT shall be similar to the shape of MOV1. That is, the aspect ratio of ATT is assumed to be equal to the aspect ratio (constant value) of MOV1. In the example of FIG. 2, ATT is set with the intention of focusing on the faces of two people in the above scene of MOV1.

In the example of FIG. 2, the video processing unit 110 generates MOV1 and MOV2 based on attention information. Specifically, the video processing unit 110 generates MOV2 as a video in which the ATT of MOV1 is enlarged. Thus, the video processing unit 110 generates MOV2 by enlarging MOV1 so as to zoom in on ATT. Note that the resolution of MOV2 may be the same as that of MOV1, or may be different from that of MOV1.

Hereinafter, the enlargement ratio of MOV2 with respect to MOV1 is represented as α. In the example of FIG. 2, α=P. P is the magnification in ZOOM mode and is any number greater than one. The ZOOM mode is a mode in which an image having the same scale as ATT is displayed as MOV2 among the first modes described below.

FIG. 3 is another diagram illustrating enlargement processing in the ZOOM mode. The relationship between MOV1 and ATT will be described below with reference to FIG. As shown in FIG. 3, the horizontal direction (X direction) and vertical direction (Y direction) of MOV1 are set in advance. The positive directions of the X direction and the Y direction are assumed to be the right direction and the downward direction on the page of FIG. A similar orientation shall be applied to each of the other figures unless otherwise specified.

In the following description, it is assumed that the resolution (number of pixels) of MOV1 is "SIZEX×SIZEY". SIZEX and SIZEY are the horizontal and vertical resolutions of MOV1, respectively. As an example, when MOV1 is a Full HD (High Definition) video, SIZEX=1920 and SIZEY=1080.

As shown in FIG. 3, the upper left vertex of the four vertices of MOV1 is set as the origin O(0,0). Then, of the four vertices of MOV1, the lower right vertex (that is, the opposite vertex of origin O) is referred to as vertex E. The coordinates of vertex E are represented as E(SIZEX-1, SIZEY-1). The origin O and the vertex E can also be expressed as the start point and end point of MOV1 on the XY plane.

In the example of FIG. 3, the upper left vertex among the four vertices of ATT is assumed to be vertex AS0 (xs0, ys0). The vertex AS0 corresponds to the origin O. Also, let the lower right vertex of the four vertices of ATT be vertex AE0 (xe0, ye0). Vertex AE corresponds to vertex E. In the first embodiment, attention information includes information indicating coordinates of vertices AS0 and AE0. In other words, attention information includes information indicating each value of xs0, ys0, xe0, and ye0.

Note that since the magnification in the ZOOM mode is P, for each of the X and Y directions,
P×(xe0−xs0)=SIZEX−1 (1A)
P×(ye0-ys0)=SIZEY-1 (1B)
relationship is established.

In the ZOOM mode, the video processing unit 110 calculates P based on either formula (1A) or (1B) above. Then, the video processing unit 110 expands MOV1 using the P to generate MOV2.

As an example, the ATT is preferably set so that the image of the subject (main subject) to which the user should pay particular attention in MOV1 is positioned in the central portion of the ATT. Also, in many MOV1s, the image of the main subject moves as time progresses, so it is preferable to set ATT so as to follow the movement of the image of the main subject. Therefore, in each frame of MOV1, the ATT can be different.

FIG. 4 is a diagram illustrating the relationship between MOV1 and MOV2 in ZOOM mode. FIG. 4 illustrates a case where metadata (display information) is appropriately acquired in the ZOOM mode. In this case, the same processing as in FIG. 4 is performed in a comparative example to be described later. FIG. 4 illustrates a case where MOV1 switches from a sub-content video (hereinafter referred to as sub-content) to a main content video (hereinafter referred to as main content). The sub-contents and main content in the example of FIG. 4 are CM (Commercial) video (hereinafter referred to as CM) and TV drama program video (hereinafter referred to as drama), respectively.

In the example of FIG. 4, the time stamp of frame i (i-th frame) of MOV1 is denoted as time stamp i. The same applies to other time stamps. The description for frame i also applies to other frames unless otherwise stated. In the following description, "frame" refers to "frame of MOV1" unless otherwise specified.

In the example of FIG. 4, MOV1 switches from CM to drama at frame i+2. It is assumed that the second receiving unit 82 can acquire the metadata of a certain frame at the beginning of the certain frame of MOV1 (the display start timing of the certain frame).

In frames i to i+1 (commercial display period), the attention area is set at the center of the screen in order to draw the user's attention to the main part of the commercial displayed in the center of MOV1. Hereinafter, the attention area set in this way is also abbreviated as "attention area: center". On the other hand, from frame i+2 onwards (drama display period), in order to draw the user's attention to the faces of the two persons positioned on the upper side of the screen of MOV1, the region of interest is set to the faces. . Hereinafter, the attention area set in this way is also abbreviated as "attention area: face".

The video processing unit 110 enlarges the frame i using metadata corresponding to the frame i (corresponding to the time stamp i). Then, the video processing unit 110 outputs the enlarged frame i as the frame i−1 of MOV2. In this way, MOV2 is generated as a video that is delayed by one frame from MOV1 (see also FIG. 15 described later).

Therefore, in the example of FIG. 4, the video processing unit 110 generates each corresponding frame of MOV2 by enlarging the attention area of frames i to i+1. After that, the video processing unit 110 generates each corresponding frame of MOV2 by enlarging the attention area after the frame i+2. According to MOV2 generated in this way, it is possible to draw the user's attention to different areas of MOV2 during the CM display period and the drama display period.

(Comparative example)
Prior to further explanation of the display device 100 (more specifically, the video processing device 1), a display device 100r as a comparative example will be described in order to explain the problems of the conventional technology. FIG. 5 is a block diagram showing the configuration of the main part of the display device 100r. For the sake of convenience, the image processing device of the display device 100r will be referred to as an image processing device 1r. Also, the control section of the video processing device 1r is referred to as a control section 10r.

FIG. 6 is a diagram illustrating an example of processing in a comparative example. FIG. 6 is a diagram paired with FIG. Also in the comparative example, the second receiving unit 82 receives metadata via NT2. However, in general, the stability of the communication state of the second receiving section 82 (or NT2) is lower than the stability of the communication state of the first receiving section 81 (or NT1). Therefore, it is conceivable that the second receiving unit 82 may be unable to properly receive the metadata.

As an example, consider the case where metadata for frames i+1 to i+3 cannot be received, as shown in FIG. In such a case, a certain frame cannot be enlarged using the metadata (appropriate metadata) corresponding to the same frame, so the image of MOV2 may be disturbed (the display quality of MOV2 may be degraded). . Therefore, it is preferable to take measures to cope with poor reception of metadata.

For example, as shown in countermeasure 1, if the metadata of a certain frame (eg, frame i+1) cannot be received, a blank image (eg, black screen) is added to the frame of MOV2 corresponding to the frame (eg, frame i of MOV2). It is also possible to insert In countermeasure 1, the video processing device 1r inserts blank images in frames i to i+2 of MOV2.

According to measure 1, it is possible to prevent the portion of MOV1 that is not originally intended as the attention area from being enlarged. However, in MOV2 generated by Countermeasure 1, the user cannot view some of the scenes that were originally drawn in MOV1. This brings dissatisfaction to users who want to watch MOV1.

In order to avoid such user dissatisfaction, it is preferable not to insert a blank image into MOV2. Therefore, for example, as shown in countermeasure 2, it is conceivable to apply the latest metadata successfully received to each subsequent frame as it is. In countermeasure 2, the video processing device 1r generates frames i to i+2 of MOV2 by enlarging frames i+1 to i+3 using the metadata of frame i. Therefore, in measure 2, the same enlargement ratio (P) as frame i is maintained for frames i+1 to i+3.

As described above, in the metadata of frame i, the region of interest is set in the center. Therefore, frames i to i+2 of MOV2 are images obtained by enlarging the central portion of frames i+1 to i+3. However, as described above, in the metadata of frames i+2 to i+3, the region of interest is set in the center. That is, frames i+2 to i+3 are images originally intended to focus attention on the central portion.

Therefore, when MOV2 is generated by countermeasure 2, in frames i+1 to i+2 of MOV2, a region (non-attention region) that is originally not intended to be focused on (eg, the torso portions of two people) is out of focus. It will be done. As described above, even with countermeasure 2, the display quality of MOV1 may be lowered, and the dissatisfaction of users who want to view MOV1 cannot be fully resolved.

(Example of processing of video processing device 1)
The video processing device 1 was newly created by the inventor of the present application in consideration of the problems of the comparative example described above. An example of the processing of the video processing device 1 will be described below. Unlike the video processing device 1r, the video processing device 1 can operate by switching between the first mode and the second mode.

The first mode is a mode for processing MOV1 using display information. The first mode includes the ZOOM mode described above. In ZOOM mode, α=P is maintained. On the other hand, the second mode is a mode in which MOV1 is processed without using display information. The second mode includes ORIGINAL mode.

The ORIGINAL mode is a mode for generating MOV2 that has the same scale as MOV1 and expresses the same scene as MOV1. As an example, in the ORIGINAL mode, the video processing device 1 assigns frame i of MOV1 as frame i−1 of MOV2. Thus, in the ORIGINAL mode, α is maintained at 1 regardless of the passage of time. That is, the original scale of MOV1 (original scale) is kept constant.

Furthermore, the first mode includes TO_ORIGINAL mode in addition to the ZOOM mode described above. The TO_ORIGINAL mode is a mode in which α is gradually (gradually) decreased from P to 1. The TO_ORIGINAL mode can be said to be an intermediate mode between the ZOOM mode and the ORIGINAL mode. By providing the TO_ORIGINAL mode, it is possible to effectively prevent deterioration in display quality when transitioning from the ZOOM mode to the ORIGINAL mode, as described below.

Hereinafter, a state in which the second receiving section 82 can receive display information is referred to as a receivable state. On the other hand, a state in which the second receiving section 82 cannot receive display information is referred to as a reception disabled state. The display information reception determination unit 120 determines whether the state (communication state) of the second reception unit 82 is a receivable state or a non-receivable state. The display information reception determination unit 120 may be realized using a known communication state diagnosis technique. Information (communication state information) indicating the determination result of the display information reception determination unit 120 is generated, and the communication state information is supplied to the selection unit 140 .

The scene change detection unit 130 detects whether or not there is a scene change in MOV1 by analyzing MOV1. The scene change detection unit 130 generates information (scene change information) indicating the determination result of the scene change detection unit 130 and supplies the scene change information to the selection unit 140 . Scene change detection may be performed by a known technique.

As an example, the scene change detection unit 130 derives feature amounts for each frame of MOV1. Then, the scene change detection unit 130 detects a scene change between a certain frame (eg, frame i) and the next frame (eg, frame i+1) based on the change in the feature amount between the frame (eg, frame i) and the next frame (eg, frame i+1). has occurred. As an example, the feature amount may be the sum of luminance of all pixels in one frame. Also, the feature amount may be calculated based on a histogram (brightness histogram) showing the distribution of brightness of each pixel in one frame.

The selection unit 140 selects the mode of the video processing device 1 . That is, the selection unit 140 selects one of the first mode and the second mode. Specifically, the selection unit 140 selects one mode from the ZOOM mode, the TO_ORIGINAL mode, and the ORIGINAL mode. The video processing device 1 processes MOV1 (that is, generates MOV2) according to the mode selected by the selection unit 140 .

As described above, whether metadata can be received (that is, the state of the second receiving unit 82) greatly affects the display quality of MOV2. Therefore, it is preferable that the selection unit 140 selects the mode based on the communication state information. Furthermore, the presence or absence of a scene change in MOV1 also greatly affects the display quality of MOV2. Therefore, it is preferable that the selection unit 140 selects the mode further based on the scene change information.

Below, the case where the selection unit 140 selects the mode based on both the communication state information and the scene change information will be mainly exemplified. Below, the flag value indicating the mode of the video processing device 1 is expressed as state. As an example, state=ZOOM indicates that the video processing device 1 is operating in ZOOM mode. The same applies to other notations. The selection unit 140 updates the state according to the mode selected by itself.

FIG. 7 is a sequence diagram illustrating the processing flow of the video processing device 1. As shown in FIG. As for FIG. 7, please also refer to each processing of FIGS. 8 to 13 described later. In the example of FIG. 7, the scene of MOV1 is changed as "scene 1→scene 2→scene 3" as time passes. “Metadata reception state” in FIG. 7 indicates the state of the second reception unit 82 . "Metadata reception status: Enabled" indicates a receivable status. On the other hand, "metadata reception status: not possible" indicates a reception disabled status.

First, scene 1 will be described. In the example of FIG. 7, it is assumed that the display information is received by the second receiving unit 82 at the beginning of scene 1 . Therefore, it is assumed that the ZOOM mode (an example of the first mode) has been selected in advance by the selection unit 140 . In scene 1, the ZOOM mode is maintained while the second receiving unit 82 is receiving display information. In the ZOOM mode, MOV2 with magnification P is generated.

In the example of FIG. 7, the state of the second receiver 82 transitions from the receivable state to the unreceivable state in the middle of scene 1 . If the ZOOM mode is maintained in such a case, MOV2 with low display quality can be generated as in the comparative example described above. Therefore, when the state of the second receiver 82 transitions from the receivable state to the unreceivable state, the selector 140 selects the TO_ORIGINAL mode instead of the ZOOM mode. In the TO_ORIGINAL mode, α is set to gradually decrease from P to 1. Therefore, each frame of MOV2 is generated such that the region of interest gradually zooms out to the entirety of MOV1.

In the example of FIG. 7 , the unreceivable state continues until the middle of scene 2 . A scene change from scene 1 to scene 2 occurs while this unreceivable state continues. As shown in FIG. 7, the selector 140 selects the OGIGINAL mode (second mode) when a scene change occurs in MOV1 in the unreceivable state. In ZOOM mode, MOV2 with an enlargement factor of 1 (also referred to as MOV2 with original size) is generated.

In the example of FIG. 7 , the unreceivable state is canceled in the middle of scene 2 . A scene change from scene 2 to scene 3 occurs after the unreceivable state is resolved. As shown in FIG. 7, the selector 140 selects the ZOOM mode (an example of the first mode) when a scene change occurs in MOV1 in the receivable state.

FIG. 8 is a flowchart illustrating the overall flow of processing of the video processing device 1 (more specifically, the control unit 10). As described above, the control unit 10 executes processing for each frame of MOV1. Therefore, the control unit 10 starts processing when the vertical synchronization signal (Vsync) of MOV1 is turned ON.

First, the control unit 10 acquires the time stamp of MOV1 (S1). Then, the scene change detection unit 130 generates scene change information by analyzing MOV1 (S2). Below, a flag value indicating whether or not there is a scene change in MOV1 is represented as SCD_Flag. SCD_Flag is an example of scene change information.

As an example, when a scene change occurs in MOV1 (when there is a scene change), the scene change detection unit 130 sets SCD_Flag as SCD_Flag=True. On the other hand, the scene change detection unit 130 sets SCD_Flag as SCD_Flag=False when no scene change occurs in MOV1 (when there is no scene change).

Subsequently, the control unit 10 processes MOV1 according to the mode of the video processing device 1 (in other words, according to the state) (S3). Examples of video processing according to modes are described below.

(Example of ZOOM mode)
FIG. 9 is a flowchart illustrating the flow of image processing when the ZOOM mode is preselected. First, the display information reception determining unit 120 determines whether or not the metadata corresponding to the time stamp (that is, the metadata of frame i) has been acquired (S11).

If the metadata of the frame i can be obtained (YES in S11), the video processing unit 110 obtains attention information of the frame i included in the metadata. That is, the video processing unit 110 identifies the attention area of the frame i. Then, the video processing unit 110 enlarges the frame i so as to zoom in on the attention area (S12). That is, as described above, the image obtained by enlarging the frame i by the enlargement ratio P is generated as the frame i−1 of MOV2.

If the metadata of frame i could not be acquired (NO in S11), the scene change detection unit 130 determines whether a scene change has occurred in MOV1 (more specifically, in frame i) (that is, SCD_Flag=True). (S13).

When SCD_Flag=True (YES in S13), the selection unit 140 selects the ORIGINAL mode. That is, the selection unit 140 switches the mode from the ZOOM mode to the ORIGINAL mode. Triggered by the mode switching, the video processing unit 110 sets α to 1 (S14). The selection unit 140 sets state=ORIGINAL and updates the state (S15).

On the other hand, when SCD_Flag is not True (that is, when SCD_Flag is False) (NO in S13), selection unit 140 selects TO_ORIGINAL mode. That is, the selection unit 140 performs mode switching from the ZOOM mode to the TO_ORIGINAL mode. Triggered by the mode switching, the video processing unit 110 sets PROCESS_CNT to 0 (that is, initializes PROCESS_CNT) (S16).

As will be described later, PROCESS_CNT is a count value used for setting α in TO_ORIGINAL mode (or TO_ZOOM mode described in Embodiment 2). The selection unit 140 sets state=TO_ORIGINAL and updates the state (S17).

(Example of TO_ORIGINAL mode)
FIG. 10 is a flowchart illustrating the flow of video processing when the TO_ORIGINAL mode is preselected. First, the scene change detection unit 130 determines whether SCD_Flag=True (S21).

If SCD_Flag=True (YES in S21), the selection unit 140 selects the ORIGINAL mode. That is, the selection unit 140 performs mode switching from the TO_ORIGINAL mode to the ORIGINAL mode. Triggered by the mode switching, processes similar to those of S14 and S15 described above are executed (S22 and S23).

On the other hand, if SCD_Flag=True (NO in S21), the display information reception determining unit 120 determines whether metadata corresponding to the time stamp has been acquired (S24). If YES in S24, the selector 140 selects the TO_ZOOM mode. That is, the selection unit 140 performs mode switching from the TO_ORIGINAL mode to the TO_ZOOM mode. The selection unit 140 sets state=TO_ZOOM and updates the state (S25).

In the case of NO in S24, the selector 140 determines whether or not PROCESS_CNT<MAX_CNT (S26). MAX_CNT is the maximum value of PROCESS_CNT preset in TO_ORIGINAL mode. MAX_CNT can also be expressed as the number of frames until the expansion rate returns from P to 1. As an example, in Embodiment 1, it is set as MAX_CNT=600.

If PROCESS_CNT<MAX_CNT (YES in S26), the video processing unit 110 sets α according to the current PROCESS_CNT. In other words, the video processing unit 110 gradually decreases α (S27). Then, the video processing unit 110 generates an image obtained by enlarging the frame i by the enlargement factor α as the frame i−1 of MOV2. Subsequently, the video processing unit 110 increments (counts up) PROCESS_CNT by 1 (S28).

As described above, in the TO_ORIGINAL mode, PROCESS_CNT is incremented by one from the initial value (0). Finally, PROCESS_CNT=MAX_CNT. That is, the condition PROCESS_CNT<MAX_CNT is no longer satisfied (NO in S26).

If NO in S26, proceed to S22. That is, the selector 140 selects the ORIGINAL mode when PROCESS_CNT increases to MAX_CNT (maximum value). In this way, the selection unit 140 also switches from the TO_ORIGINAL mode to the ORIGINAL mode when PROCESS_CNT increases to MAX_CNT (that is, when α decreases to 1).

FIG. 11 is a diagram illustrating enlargement processing in the TO_ORIGINAL mode. In TO_ORIGINAL mode, α is gradually decreased from P. That is, in TO_ORIGINAL mode, ATT is reduced to the corresponding zoom area ZTT. In other words, the TO_ORIGINAL mode is a mode in which MOV1 is expanded to ZTT by an expansion factor of P or less.

In the example of FIG. 11, let the upper left vertex of the four vertices of ZTT be the vertex ZS (xs, ys). Vertex ZS corresponds to origin O and vertex AS0. Also, let the lower right vertex of the four vertices of ZTT be vertex ZE (xe, ye). Vertex ZE corresponds to vertex E and vertex AE0.

In the TO_ORIGINAL mode, the video processing unit 110 identifies ATT (that is, identifies xs0, ys0, ys0, and ye0) using attention information included in the latest successfully received metadata. Then, the video processing unit 110 calculates the following formulas (2A) to (2D),

xs, ys, ys, and ye are calculated based on

Also, regarding the enlargement factor α in the TO_ORIGINAL mode,
α×(xe-xs)=SIZEX-1 (3A)
α×(ye-ys)=SIZEY-1 (3B)
relationship is established.

Therefore, in the TO_ORIGINAL mode, the video processing unit 110 calculates α based on either formula (3A) or (3B) above. Then, the video processing unit 110 generates MOV2 by enlarging MOV1 using the α.

As shown in equations (2A)-(2D) above, when PROCESS_CNT=0, ZTT is the same region as ATT. Therefore, the video processing unit 110 sets α to P (the maximum value in the TO_ORIGINAL mode).

Also, when PROCESS_CNT=MAX_CNT, ZTT is the same area as MOV1. Therefore, the video processing unit 110 sets α to 1 (the maximum value in the TO_ORIGINAL mode).

Furthermore, as can be seen from equations (3A)-(3B) above, α decreases as PROCESS_CNT increases. That is, ZTT is set such that ATT is gradually zoomed out. FIG. 12 is a diagram showing an example of changes in ZTT according to an increase in PROCESS_CNT. In the example of FIG. 12, PROCESS_CNT=0 to 4 are illustrated. Also from FIG. 12, it can be understood that α decreases as PROCESS_CNT increases.

As described above, in the TO_ORIGINAL mode, α is set as a monotonically decreasing function of PROCESS_CNT with P as the maximum value and 1 as the minimum value. In the example of Embodiment 1, α is set to decrease linearly as PROCESS_CNT increases (in other words, as time t increases).

As can be understood from the above description, in the TO_ORIGINAL mode, when MAX_CNT is set large, zooming out is performed at a low speed (gradually). On the other hand, when MAX_CNT is set large, zooming out is performed at high speed.

In general video content (eg movies or dramas), the interval between scene changes is often about 5 to 10 seconds. Therefore, if the duration of the TO_ORIGINAL mode is set to about 5 to 10 seconds, it is expected that the possibility of causing discomfort to the user watching MOV2 can be reduced.

Based on the above points, as an example, when the frame rate of MOV1 is 60 Hz, MAX_CNT is preferably set to about 300-600. MAX_CNT may be appropriately set in consideration of the specifications of MOV1.

(Example of ORIGINAL mode)
FIG. 13 is a flowchart illustrating the flow of video processing when the ORIGINAL mode has been selected in advance.

First, the display information reception determination unit 120 determines whether metadata corresponding to the time stamp has been acquired. Also, the scene change detection unit 130 determines whether or not there is a scene change in MOV1. In the ORIGINAL mode, the control unit 10 determines "whether metadata corresponding to the time stamp has been acquired and SCD_Flag=True" (S31).

In the case of NO in S31, that is, in the case of at least one of (i) ``the metadata corresponding to the time stamp could not be obtained'' and (ii) ``the SCD_Flag=False'', ORIGINAL mode remains unchanged.

On the other hand, if S31 is YES, the selection unit 140 selects the ZOOM mode. That is, the selection unit 140 switches the mode from the ORIGINAL mode to the ZOOM mode. Triggered by the mode switching, a process similar to that of S12 described above is executed (S32). Then, the selection unit 140 sets state=ZOOM and updates the state (S33).

(effect)
As described above, the prior art does not mention any measures to be taken when metadata reception fails. That is, in the prior art, the technical concept of "switching between the first mode and the second mode" is not considered at all. On the other hand, according to the video processing device 1, by switching between the first mode and the second mode, appropriate video processing can be performed according to the situation (for example, according to the state of the second receiving unit). can be done. Therefore, according to the video processing device 1, MOV2 with higher display quality than the conventional one can be provided.

Next, further effects of the video processing device 1 will be described with reference to FIGS. 14 and 15. FIG. 14A and 14B are diagrams illustrating temporal changes of α in Embodiment 1. FIG. The horizontal axis of the graph in FIG. 14 represents t (time), and the vertical axis represents α (magnification rate). In the example of FIG. 14, for the sake of simplicity, it is assumed that no scene change occurs in MOV1. The example of FIG. 14 is also referred to as case 1 below.

In Case 1, the receivable state continues from the initial time (t=0) to t11. Therefore, the ZOOM mode is selected from the initial time to t11. In ZOOM mode, α is kept at P.

In Case 1, at t11, the second receiver 82 transitions from the receivable state to the unreceivable state. It is assumed that this unreceivable state continues without recovery. At t11, the TO_ORIGINAL mode is selected triggered by the fact that the second receiving unit 82 has reached the unreceivable state (metadata reception has failed). Therefore, after t11, α linearly decreases as t increases. However, the mode of decreasing α in the TO_ORIGINAL mode is not limited to the example of FIG. For example, α may be set to decrease non-linearly as t increases.

In case 1, the decrease of α in TO_ORIGINAL mode resulted in α decreasing to 1 (minimum value) at t12. Therefore, the ORIGINAL mode is selected when α becomes 1 at t12. In ORIGINAL mode, α is kept at 1.

As can be understood from FIG. 14, when the ZOOM mode is directly switched to the ORIGINAL mode (in the case of the reference example described later), α changes from P to 1 instantaneously. Therefore, the display quality of MOV2 may deteriorate when switching from the ZOOM mode to the ORIGINAL mode. For example, at the time of switching, the user viewing MOV2 may feel uncomfortable.

On the other hand, by interposing the TO_ORIGINAL mode between the ZOOM mode and the ORIGINAL mode, α can be smoothly decreased over time. In this way, by switching from the ZOOM mode to the ORIGINAL mode via the TO_ORIGINAL mode, it is possible to provide MOV2 with higher display quality (video with less discomfort) when switching to the ORIGINAL mode.

FIG. 15 is a diagram for explaining the effect of the video processing device 1 in comparison with FIG. 6 (comparative example). As shown in FIG. 15, unlike the video processing device 1r, the video processing device 1 switches from the ZOOM mode to the TO_ORIGINAL mode when it fails to receive the metadata of the frame i+1.

Also, in the video processing apparatus 1, mode switching from the TO_ORIGINAL mode to the ORIGINAL mode is performed when a scene change of MOV1 occurs in frame i+2 (one frame in the unreceivable state). Therefore, as can be understood from FIG. 14 described above, the display quality of the MOV2 can be improved even compared to the video processing device 1r.

(Comparison with reference example)
FIG. 16 is a diagram for explaining the effect of the video processing device 1 by comparison with the reference example. In the reference example, unlike the first embodiment, the TO_ORIGINAL mode is not included in the first mode. Therefore, in the reference example, only switching between the ZOOM mode and the ORIGINAL mode is performed.

In the example of FIG. 16, similarly to the example of FIG. 15, the scene change of MOV1 occurs under the unreceivable state. In the reference example, mode switching from the ZOOM mode to the ORIGINAL mode is performed with the scene change as a trigger. Therefore, in the reference example, flashing (flickering) of MOV2 may occur due to the momentary change of α from P to 1 at the time of switching.

In contrast, according to the video processing apparatus 1 of Embodiment 1, as described above, mode switching from the ZOOM mode to the TO_ORIGINAL mode is performed with the scene change as a trigger. By providing the TO_ORIGINAL mode, unlike the reference example, it is possible to prevent an instantaneous change in α from occurring. That is, flushing can be effectively prevented.

In general, it is difficult to predict in advance the timing at which the second receiving section will reach the unreceivable state. Also, the timing at which a scene change occurs in MOV1 differs depending on the content of MOV1. However, according to the first embodiment, by providing the TO_ORIGINAL mode, it is possible to prevent flushing without requiring the above-described prediction of each timing.

[Embodiment 2]
Embodiment 2 describes the TO_ZOOM mode. The TO_ZOOM mode is a mode paired with the TO_ORIGINAL mode. In the TO_ZOOM mode, unlike the TO_ORIGINAL mode, α is set to gradually increase. TO_ZOOM mode is also included in the first mode.

(Example of TO_ZOOM mode)
FIG. 17 is a flowchart illustrating the flow of video processing when the TO_ZOOM mode is selected in advance in the second embodiment. As described below, in the example of FIG. 17, α is set by decrementing (counting down) PROCESS_CNT.

At the start of the TO_ZOOM mode, PROCESS_CNT is set to an initial value (predetermined value corresponding to α at the start of the TO_ZOOM mode). The initial value can take any integer value in the range 0 to CNT_MAX. The following description of FIG. 17 illustrates the case where the initial value is greater than zero.

First, the selection unit 140 determines whether or not 0<PROCESS_CNT (S41). If YES in S41, the video processing unit 110 sets α according to the current PROCESS_CNT.

Also in the TO_ZOOM mode, the video processing unit 110 identifies ATT (that is, identifies xs0, ys0, ys0, and ye0) using attention information included in the latest successfully received metadata. Then, video processing section 110 calculates xs, ys, ys, and ye based on the above equations (2A) to (2D).

Also in the TO_ZOOM mode, the above-described relationships (3A) to (3B) are established with respect to the enlargement factor α. Therefore, even in the TO_ZOOM mode, the video processing unit 110 may calculate α based on either formula (3A) or (3B) above. As noted above, α increases as PROCESS_CNT decreases. Therefore, α can be gradually increased by decrementing PROCESS_CNT as in the process of FIG. 17 (S42). For example, α can be decreased linearly as PROCESS_CNT decreases (in other words, as time t increases).

Then, the video processing unit 110 generates an image obtained by enlarging the frame i by the enlargement factor α as the frame i−1 of MOV2. Subsequently, the video processing unit 110 decrements PROCESS_CNT by 1 (S43). Thus, in TO_ZOOM mode, unlike TO_ORIGINAL mode, ZTT is set such that ATT is gradually zoomed in. FIG.

As described above, in TO_ZOOM mode, PROCESS_CNT is decremented by one from the initial value (maximum value in TO_ZOOM mode). Finally, PROCESS_CNT=0. That is, the condition 0<PROCESS_CNT is no longer satisfied (NO in S41).

In the case of NO in S41, since PROCESS_CNT=0, the video processing unit 110 sets α=P (S44). Then, the selection unit 140 selects the ZOOM mode. As described above, the selector 140 switches from the TO_ZOOM mode to the ZOOM mode when PROCESS_CNT decreases to 0 (minimum value). The selection unit 140 sets state=ZOOM and updates the state (S45).

(Case 2)
FIG. 18 is a diagram illustrating temporal change of α in the second embodiment. The example of FIG. 18 is also referred to as case 2 below. In Case 2, the receivable state continues (that is, the ZOOM mode is maintained) from the initial time to t21. At t21, the second receiver 82 transitions from the receivable state to the unreceivable state. After that, at t22, this unreceivable state is resolved.

Therefore, in case 2, the TO_ORIGINAL mode is selected during the period from t21 to t22 (see also FIGS. 9 and 10). Here, α at t22 is referred to as α22. In Case 2, it is assumed that α22>1. That is, in Case 2, it is assumed that the unreceivable state is resolved at a predetermined time (t22) before α decreases to 1.

In the second embodiment, at t22, the selection unit 140 switches the mode of the video processing device 1 from the TO_ORIGINAL mode to the TO_ZOOM mode (see also FIG. 10). That is, the selector 140 selects the TO_ZOOM mode when the unreceivable state is resolved at a predetermined point in time before α decreases to 1. α22 is an example of the initial value of α in TO_ZOOM mode.

In the example of FIG. 18, in the TO_ZOOM mode, the video processing device 1 (more specifically, the video processing unit 110) linearly increases α as t increases. However, the mode of increasing α in the TO_ZOOM mode is not limited to the example of FIG. For example, α may be set to increase non-linearly as t increases.

In case 2, α increased to P at t23 as a result of increasing α in TO_ZOOM mode. The selector 140 selects the ZOOM mode when α becomes P at t23 (see also FIG. 17).

As in Case 1, if the ORIGINAL mode is directly switched to the ZOOM mode, α changes from 1 to P instantaneously. Therefore, the display quality of MOV2 may be degraded when switching to the ZOOM mode. For example, at the time of switching, the user viewing MOV2 may feel uncomfortable. Therefore, in the second embodiment, a TO_ZOOM mode is further provided as an intermediate mode between the ZOOM mode and the ORIGINAL mode.

By interposing the TO_ZOOM mode between the ORIGINAL mode and the ZOOM mode, α can be smoothly increased over time. In this way, by switching from the ORIGINAL mode to the ZOOM mode via the TO_ZOOM mode, it is possible to provide MOV2 with higher display quality (video with less discomfort) when switching to the ZOOM mode.

[Embodiment 3]
FIG. 19 is a diagram illustrating temporal change of α in the third embodiment. The example of FIG. 19 is also referred to as case 3 below. In Case 3, the receivable state continues (that is, the ZOOM mode is maintained) from the initial time to t31. Then, at t31, the second receiver 82 transitioned from the receivable state to the unreceivable state. Therefore, at t31, the TO_ORIGINAL mode is selected. After that, until t33, the state in which the receivable state and the unreceivable state were intermittently switched continued.

In case 3, unlike case 2, the selection unit 140 maintains the current mode of the video processing device 1 unless the receivable state is maintained for a predetermined time (hereinafter referred to as tss) (see FIG. 20 below). see also). As an example, tss is about 10 to 20 seconds. This is because, in a situation where the receivable state and the unreceivable state are intermittently switched, if the mode is changed for each switching between the receivable state and the unreceivable state, there is a concern that the visibility of MOV2 will rather deteriorate. . Therefore, in Case 3, the TO_ORIGINAL mode is maintained even after t31.

In Case 3, the reduction of α in the TO_ORIGINAL mode resulted in a reduction of α to 1 at t32. It is assumed that t32 is before t33. At t32, Triggered by α becoming 1, the ORIGINAL mode is selected.

Then, at t33, the intermittent switching between the receivable state and the unreceivable state is resolved. As shown in FIG. 19, the receivable state is maintained after t33. In case 3, the selection unit 140 selects the TO_ZOOM mode at t34 (see also FIG. 20). t34 is the time point tss after t33, and is expressed as t34=t33+tss.

In this way, when the second mode (eg, ORIGINAL mode) is selected in advance, the selection unit 140 selects the first A mode (eg, TO_ZOOM mode) may be selected. In Case 3, the video processing device 1 increases from α1 in the TO_ZOOM mode.

The increase in α in TO_ZOOM mode resulted in α increasing to P at t35. The selector 140 selects the ZOOM mode when α becomes P at t35.

According to the processing of Case 3, unlike the processing of Case 2, the mode can be selected ignoring the temporary switching between the receivable state and the unreceivable state. Therefore, when the stability of the communication state of the second receiving unit 82 (or NT2) is considered to be low, it is preferable to adopt the processing of Case 3 instead of Case 2.

(another example of ORIGINAL mode)
FIG. 20 is a flowchart illustrating the flow of video processing when the ORIGINAL mode is preselected in the third embodiment. In Embodiment 3, when the ORIGINAL mode is selected in advance, the flowchart of FIG. 20 is applied instead of the flowchart of FIG. t34 in the following description represents the current time (however, the time after t33).

First, the display information reception determination unit 120 determines whether metadata corresponding to the time stamp has been acquired. In addition, the control unit 10 determines whether or not the condition "t34-t33>tss" is satisfied. That is, in the ORIGINAL mode of the third embodiment, the control unit 10 determines "whether metadata corresponding to the time stamp has been acquired and whether t34-t33>tss" (S51).

In the case of NO in S51, that is, in at least one of (i) ``the metadata corresponding to the time stamp could not be obtained'' and (ii) ``t34-t33≤tss''. , the ORIGINAL mode is maintained as it is.

On the other hand, if S51 is YES, the selection unit 140 selects the TO_ZOOM mode. That is, the selection unit 140 performs mode switching from the ORIGINAL mode to the TO_ZOOM mode.

Triggered by the mode switching, the video processing unit 110 sets PROCESS_CNT=MAX_CNT (S52). By setting PROCESS_CNT in this way, the initial value of α can be set to 1 in the TO_ZOOM mode. Then, the selection unit 140 sets state=TO_ZOOM and updates the state (S53).

[Embodiment 4]
FIG. 21 is a diagram illustrating temporal change of α in the fourth embodiment. The example of FIG. 21 is also referred to as case 4 below. The temporal change of the metadata reception state in Case 4 is the same as in Case 3. In case 4, unlike case 3, mode switching is performed further considering the presence or absence of a scene change in MOV1. Note that in the fourth embodiment, the flowchart of FIG. 13 is applied when the ORIGINAL mode is selected in advance.

The mode transition (change of α) from the initial time to t43 in Case 4 is the same as the mode transition from the initial time to t33 in Case 3. In case 4, t43 is the point in time when the intermittent switching between the receivable state and the unreceivable state is resolved.

As shown in FIG. 21, in case 4, a scene change of MOV1 occurs at time t44 following t43. As described in the first embodiment, when the second mode (eg, the ORIGINAL mode) has been selected in advance, the selection unit 140 selects the first 1 mode (eg, ZOOM mode) may be selected. In case 4, the selection unit 140 switches the mode from the ORIGINAL mode to the ZOOM mode at t44.

When a scene change occurs, it is conceivable that the setting of the attention area (for example, at least one of the position and size of the attention area) is significantly different between the frame immediately before the scene change and the frame immediately after the scene change. In such a case, if the attention information in the frame immediately before the scene change is used to execute the operation of the TO_ZOOM mode, there is a concern that the user viewing MOV2 may feel uncomfortable. This is because, in MOV2 in the TO_ZOOM mode, there is a possibility that the area that should be focused on (the focused area of the frame immediately after the scene change) will not be sufficiently emphasized.

Therefore, as described above, when the second receiving section 82 is in the receivable state, switching from the ORIGINAL mode to the ZOOM mode may be performed without intervening the TO_ZOOM mode at the scene change. This is because, in some cases, increasing α from 1 to P instantaneously at the time of a scene change can provide an image that gives less discomfort to the user.

[Embodiment 5]
FIG. 22 is a diagram illustrating temporal change of α in the fifth embodiment. The example of FIG. 22 is also referred to as case 5 below. In case 5, as in case 4, mode switching is performed in consideration of the presence or absence of a scene change in MOV1.

In case 5, the receivable state continues from the initial time to t51. In case 5, at t51, the second receiving unit 82 transitions from the receivable state to the unreceivable state, and this unreceivable state continues without recovery. Therefore, in case 5, the TO_ORIGINAL mode is selected at t51. Therefore, after t51, α decreases as t increases.

In case 5, a scene change of MOV1 occurs at time t52 following t51. Here, α at t52 is referred to as α52. In case 5, it is assumed that α52>1. That is, in Case 5, it is assumed that a scene change occurs at a predetermined time (t52) before α decreases to one.

If the first mode (eg, TO_ORIGINAL mode) has been selected in advance, the selection unit 140 selects the second mode (ORIGINAL mode) in response to the scene change of MOV1 in the unreceivable state. good. In case 5, the selection unit 140 switches the mode from the TO_ORIGINAL mode to the ORIGINAL mode at t52.

As described above, when the second receiving section 82 is in the unreceivable state, α may be instantaneously decreased from α52 to 1 at the scene change. This is because, for the same reason as Case 4, it is conceivable that an image that is less uncomfortable for the user can be provided by instantly decreasing α at the time of a scene change.

FIG. 23 is a diagram for explaining the effect of the fifth embodiment by comparing with the first embodiment. FIG. 23 is a diagram paired with FIG. As shown in FIG. 23, in the fifth embodiment, unlike the first embodiment, even before α decreases to 1 in the TO_ORIGINAL mode, when a scene change occurs in MOV1 under the unreceivable state, , the mode is switched from the TO_ORIGINAL mode to the ORIGINAL mode.

It is preferable to adopt the mode switching method of the fifth embodiment when it is desired to reduce the discomfort of MOV2 caused by scene changes. On the other hand, when it is desired to eliminate the flashing of MOV2, it is preferable to adopt the mode switching method of the first embodiment. A designer of the video processing apparatus 1 may appropriately select which of the mode switching methods of the first embodiment and the fifth embodiment is to be adopted according to the specifications of the MOV 1 and the like.

[Modification]
(1) In the above description, by analyzing MOV1, it is determined whether or not there is a scene change in MOV1. However, in some broadcasting systems 1000, the display information may further include scene change information. In this case, the scene change information can be acquired from the NT2 by the second receiving section 82 .

However, if the second receiving section 82 falls into a reception disabled state, the scene change information cannot be obtained from the NT2. Therefore, it is preferable that the controller 10 is provided with a scene change detector 130 . For example, the processing of case 5 is based on the premise that the control unit 10 is provided with the scene change detection unit 130 .

(2) In the above description, the case of determining whether or not there is a scene change in MOV1 based on the feature amount of each frame of MOV1 has been exemplified. However, the scene change detection unit 130 may derive the feature amount by analyzing broadcast waves. In this case as well, it is possible to determine whether or not there is a scene change in MOV1 based on the feature amount.

(3) In the above description, attention information includes information indicating the coordinates of two paired vertices (origin O and vertex E) of ATT (attention area). However, the information of interest is not limited to the above examples as long as it is information that can uniquely specify the position of the ATT on the XY plane.

As an example, the attention information may include information indicating (i) the coordinates of the center of the ATT and (ii) the width (hereinafter W0) and height (H0) of the ATT. . This is because the position of the ATT can be uniquely specified also by these pieces of information. Hereinafter, the center of ATT is represented as point C (xc0, yc0).

For example, in the case of the example in FIG.
W0=xe0-xs0 (4A)
H0=ye0-ys0 (4B)
xc=(xs0+xe0)/2 (4C)
yc=(xs0+xe0)/2 (4D)
relationship is established. Therefore, if xc, yc, W0, and H0 are known, the positions of the two paired vertices of ATT (that is, xs0, ys0, xe0, and ye0) can be specified.

Alternatively, the attention information may include (i) the coordinates of the point C and (ii) information indicating either W0 or H0. That is, if one of W0 and H0 (eg W) is known, the other (eg H) can be calculated based on the aspect ratio of MOV1.

For example, consider a case where the aspect ratio of MOV1 is horizontal:vertical=16:9. In this case, H0=(9/16)*W0. In this way, the attention information may include (i) information indicating the coordinates of the point C and (ii) information indicating at least one of W0 and H0.

(4) The mode selection method of the selection unit 140 is not limited to the examples described above. As an example, the selection unit 140 may perform mode selection based on a user's operation. In this case, the user can select a desired mode regardless of the state of the second receiving section 82 (or whether there is a scene change in MOV1).

For example, it is conceivable that some users wish to view the original scale video regardless of the reception state of the second receiver 82 . Therefore, the selection unit 140 may select the second mode according to the user's operation even when the second reception unit 82 is in the receivable state. In this way, by allowing the selection unit 140 to perform mode selection in accordance with the user's operation, it is possible to further improve the user's convenience.

[Example of realization by software]
A control block (particularly, the control unit 10) of the display device 100 may be implemented by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be implemented by software.

In the latter case, the display device 100 is provided with a computer that executes instructions of a program, which is software that implements each function. This computer includes, for example, at least one processor (control device) and at least one computer-readable recording medium storing the program. In the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-temporary tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. In addition, a RAM (Random Access Memory) for developing the above program may be further provided. Also, the program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be implemented in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

[Additional notes]
The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. is also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.

1 video processing device 10 control unit 81 first reception unit 82 second reception unit 90 display unit 100 display device 110 video processing unit 120 display information reception determination unit 130 scene change detection unit 140 selection unit 1000 broadcasting system NT1 first network NT2 2 network MOV1 Input image MOV2 Output image ATT Area of interest ZTT Zoom out area α Magnification ratio

Claims (13)

A video processing device that generates an output video by processing an input video,
a first receiving unit that receives the input video;
a second receiving unit that receives display information corresponding to the input video,
The second receiving unit is a receiving unit different from the first receiving unit,
the display information includes metadata synchronized with the input video;
the metadata includes attention information indicating an attention area of the input video;
The video processing device has one of (i) a first mode in which the display information is used to process the input video, and (ii) a second mode in which the input video is processed without using the display information. A video processing device that selects one mode of 2. The video processing apparatus according to claim 1, wherein said first mode is selected when said second receiving section is capable of receiving said display information. 3. The video processing device according to claim 2, wherein in the first mode, the video processing device generates the output video by enlarging the attention area of the input video by a predetermined magnification. Assuming that the enlargement ratio of the output image with respect to the input image is α,
If the first mode has been previously selected,
The video processing device is
Gradually decreasing α when the second receiving unit transitions to a reception disabled state in which the display information cannot be received,
4. The video processing apparatus according to claim 3, wherein the second mode is selected when .alpha. has decreased to 1. The video processing device is
Triggered by the cancellation of the unreceivable state at a predetermined time until α decreases to 1 in the first mode,
5. The video processing device according to claim 4, wherein α is gradually increased from the value at said predetermined time. If the second mode has been previously selected,
The video processing device is
Selecting the first mode when the state in which the unreceivable state of the second receiving unit is resolved is maintained for a predetermined period of time,
5. The image processing apparatus according to claim 4, wherein .alpha. is gradually increased from 1 in the first mode. The video processing device is
generating scene change information indicating whether or not there is a scene change in the input video by analyzing the input video;
7. The image according to any one of claims 1 to 6, wherein one of the first mode and the second mode is selected based on the communication state of the second receiving section and the scene change information. processing equipment. The display information further includes scene change information indicating whether or not there is a scene change in the input video,
7. The video processing device according to any one of claims 1 to 6, wherein the video processing device selects one of the first mode and the second mode based on the communication state of the second receiving section and the scene change information. The video processing device according to item 1. If the second mode has been previously selected,
9. The video processing device according to claim 7 or 8, wherein the video processing device selects the first mode when a scene change occurs in the input video when the second receiving unit is capable of receiving the display information. The video processing device described. If the first mode has been previously selected,
3. The video processing device selects the second mode when a scene change occurs in the input video when the second receiving unit is in a reception disabled state in which the display information cannot be received. 8. The video processing device according to 7. The first receiving unit receives a broadcast wave and acquires the input video by decoding the broadcast wave,
11. The video processing device according to any one of claims 1 to 10, wherein said second receiving section receives said display information via an IP (Internet Protocol) communication network. A video processing device according to any one of claims 1 to 11;
and a display unit that displays the output video. A video processing method for generating an output video by processing an input video,
a first receiving step of receiving the input image by a first receiving unit;
a second receiving step of receiving display information corresponding to the input image by a second receiving unit different from the first receiving unit;
the display information includes metadata synchronized with the input video;
the metadata includes attention information indicating an attention area of the input video;
The image processing method includes (i) a first mode of processing the input image using the display information, or (ii) a second mode of processing the input image without using the display information. , further comprising the step of selecting one mode of .

Images