KR102838008B1 - Electronic device and the method for controlling the same - Google Patents

Abstract

An electronic device for displaying a plurality of images by dividing them is disclosed. The electronic device includes an input signal processing unit which receives a plurality of image signals, extracts image data of the plurality of image signals respectively, and generates a single integrated image signal having integrated image data in which the plurality of images based on the extracted plurality of image data are respectively assigned to a plurality of areas of a screen, and an image processing unit which generates an image frame to be displayed on the screen based on the integrated image data of the integrated image signal.

Description

ELECTRONIC DEVICE AND THE METHOD FOR CONTROLLING THE SAME

The present invention relates to an electronic device for processing and displaying a multi-view image and a control method thereof.

As display devices with large screens become more common, there is a demand to view multiple images on a single screen. Conventional display devices require multiple video processors to process each of the multiple video signals received through multiple HDMI receivers for multi-view display.

In this way, conventional display devices have the problem that the data throughput on the transmission line increases significantly due to the increase in the amount of data that must be processed by performing image processing on each of the multiple image signals received through multiple HDMI receivers for multi-view image display, and that the system configuration cost increases.

Accordingly, an object of the present invention is to provide an electronic device and a control method thereof capable of reducing data throughput and reducing system configuration costs when displaying multiple images on a single screen.

An electronic device is provided to achieve the above-described purpose. The electronic device includes an input signal processing unit which receives a plurality of image signals, extracts image data of the plurality of image signals, and generates one integrated image signal having integrated image data in which a plurality of images based on the extracted plurality of image data are respectively allocated to a plurality of areas of a screen, and an image processing unit which generates an image frame to be displayed on the screen based on the integrated image data of the integrated image signal.

The above multiple video signals can be received via HDMI (High-Definition Multimedia Interface).

The input signal processing unit may include a decoder that decodes the plurality of received image signals; and an image integration unit that extracts a plurality of image data from each of the plurality of decoded image signals to generate the integrated image data.

The above image integration unit can generate the integrated image signal based on information regarding at least one of the positions or sizes of the multiple image signals.

The above image integration unit can generate a synchronization signal of the integrated image signal corresponding to the integrated image data based on the synchronization signals of the plurality of image signals.

The above integrated video signal may include information regarding the difference in position or size between the plurality of video signals before and after integration.

The device may further include a display unit that displays the video frame based on the integrated video signal.

The above display unit may include a timing controller that adjusts the time of an image frame generated by the image processing unit.

The electronic device may further include a second interface unit that receives a second image signal and a second image processing unit that generates a second image frame for display on a screen based on the second image signal.

The above display unit can display the integrated video signal and the second video signal by synthesizing them.

The above image processing unit can perform at least one of decoding corresponding to the image format of the image data, de-interlacing for converting the image data in an interlace format into a progressive format, scaling for adjusting the image data to a preset resolution, noise reduction for improving image quality, detail enhancement, or frame refresh rate conversion.

A method for controlling an electronic device according to an embodiment of the present invention is provided. The method for controlling an electronic device includes the steps of receiving a plurality of image signals, extracting image data of each of the plurality of image signals, generating a single integrated image signal having integrated image data that allocates a plurality of images based on the extracted plurality of image data to a plurality of areas of a screen, and generating an image frame to be displayed on the screen based on the integrated image data of the integrated image signal.

The step of generating the above integrated video signal may include a step of decoding the plurality of video signals.

It can be generated based on at least one piece of information on the position or size of the above multiple video signals.

The step of generating the integrated image signal may include a step of generating a synchronization signal of the integrated image signal corresponding to the integrated image data based on the synchronization signals of the plurality of image signals.

The control method of the above electronic device may further include a step of receiving a second image signal from a second interface unit.

The control method of the above electronic device may further include a step of displaying the integrated image signal by synthesizing it with the second image signal.

The above video frame generation step may perform at least one of decoding corresponding to the video format of the video data, de-interlacing for converting the video data in an interlace format into a progressive format, scaling for adjusting the video data to a preset resolution, noise reduction for improving the video quality, detail enhancement, or frame refresh rate conversion.

The electronic device according to the present invention integrates multiple images displayed on multiple areas of a screen into one integrated image signal at an input stage, and then processes them into an image frame for display on a screen in one image processing unit, thereby reducing data throughput on a transmission line for image processing and saving system configuration costs.

FIG. 1 is a drawing showing a screen of an electronic device according to a first embodiment of the present invention.
Figure 2 is a block diagram showing the configuration of an electronic device according to the first embodiment of the present invention.
FIG. 3 is a flowchart showing a method for controlling an electronic device for displaying a split image according to a first embodiment of the present invention.
Figure 4 is a diagram showing video signals 1 to 4 in TMDS format.
Figure 5 is a diagram showing an integrated video signal (MVS).
Figure 6 is a block diagram showing the configuration of an electronic device according to a second embodiment of the present invention.
Figure 7 is a block diagram showing the configuration of an electronic device according to a third embodiment of the present invention.
FIG. 8 is a diagram showing a scenario for processing and displaying multiple image signals of an electronic device according to a fourth embodiment of the present invention.
FIG. 9 is a diagram showing a scenario for processing and displaying multiple image signals of an electronic device according to a fifth embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numbers or symbols refer to components that perform substantially the same functions, and the size of each component in the drawings may be exaggerated for clarity and convenience of explanation. However, the technical idea of the present invention and its core configuration and operation are not limited to the configuration or operation described in the following embodiments. In describing the present invention, if it is determined that a specific description of a known technology or configuration related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

In this document, the expressions "have," "can have," "include," or "may include" indicate the presence of a given feature (e.g., a numerical value, function, operation, or component such as a part), but do not exclude the presence of additional features.

In this document, the expressions "A or B," "at least one of A and/or B," or "one or more of A or/and B" can include all possible combinations of the listed items. For example, "A or B," "at least one of A and B," or "at least one of A or B" can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

In the embodiments of the present invention, terms including ordinal numbers such as first, second, etc. are used only for the purpose of distinguishing one component from another, and singular expressions include plural expressions unless the context clearly indicates otherwise.

In addition, in the embodiments of the present invention, terms such as 'upper', 'lower', 'left', 'right', 'inner', 'outer', 'inner surface', 'outer surface', 'front', and 'rear' are defined based on the drawings, and the shape or position of each component is not limited thereby.

The expression "configured to", as used herein, can be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" does not necessarily mean something is "specifically designed to" in terms of hardware. Instead, in some contexts, the expression "a device configured to" can mean that the device, in conjunction with other devices or components, is "capable of." For example, the phrase "a subprocessor configured to perform A, B, and C" can mean a dedicated processor (e.g., an embedded processor) for performing the operations, or a generic-purpose processor (e.g., a CPU or application processor) that can perform the operations by executing one or more software programs stored in a memory device.

An electronic device (1) according to various embodiments of the present document may include at least one of, for example, a television, a smart phone, a tablet PC, a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a PDA, a portable multimedia player (PMP), an MP3 player, a medical device, a camera, or a wearable device that processes images. In some embodiments, the electronic device (1) may include at least one of, for example, a Blu-ray player, a digital video disk (DVD) player, a set-top box, a home automation control panel, a security control panel, a media box, a game console, an electronic dictionary, a camcorder, or an electronic picture frame.

In another embodiment, the electronic device (1) may include at least one of a navigation device, a global navigation satellite system (GNSS), an event data recorder (EDR), a flight data recorder (FDR), an automobile infotainment device, electronic equipment for ships (e.g., a navigation device for ships, a gyro compass, etc.), avionics, a security device, and a head unit for a vehicle.

In this document, the term user may refer to a person using an electronic device (1) or a device (e.g., an artificial intelligence electronic device) using an electronic device (1).

FIG. 1 is a drawing showing a screen of an electronic device (1) according to a first embodiment of the present invention.

As shown in Fig. 1, the electronic device (1) divides the screen into four areas and displays four images (Video 1 to 4). The electronic device (1) is not limited to dividing and displaying four images on one screen, and may also divide and display 1 to 3 images or 5 or more images.

Figure 2 is a block diagram showing the configuration of an electronic device (1) according to the first embodiment of the present invention.

As shown in Fig. 2, the electronic device (1) may include an interface unit (10) capable of receiving, for example, four image signals from a source device (2), an input signal processing unit (11), an image processing unit (12), a display unit (13), an audio processing unit (14), a voice output unit (15), and a processor (16).

The source device (2) may include a device capable of transmitting video content to an electronic device (1), such as a set-top box, a server, a relay device, a computer, a mobile device, etc.

The interface unit (10) may include, for example, four HDMI (High-Definition Multimedia Interface) Rx1 to Rx4 for receiving four video signals in a TMDS (Transition Minimized Differential Signal) format. The interface (10) shown in Fig. 2 is shown as an example of four HDMI Rx1 to Rx4 for convenience of explanation, and is not limited thereto.

The interface section (10) may include a wired interface section and a wireless interface section.

The wired interface section may include a terrestrial/satellite broadcasting antenna connection tuner for receiving broadcast signals, a cable broadcasting cable connection interface, etc.

The wired interface section can include HDMI, DP, DVI, Component, S-Video, composite (RCA terminal) for connecting video devices.

The wired interface section may include a USB interface for connecting general-purpose electronic devices.

The wired interface section may include a connection interface for an optical cable device.

The wired interface section may include an interface for connecting audio devices such as a headset, earphones, and external speakers.

The wired interface section may include a connection interface for wired network devices such as Ethernet.

The wireless interface section may include connection interfaces for wireless network devices such as Wi-Fi, Bluetooth, ZigBee, Z-wave, RFID, WiGig, WirelessHD, UWB (Ultra-Wide Band), Wireless USB, and NFC (Near Field Communication).

The wireless interface unit may include an IR transceiver module for transmitting and/or receiving remote control signals.

The wireless interface section may include a mobile communication device connection interface such as 2G to 5G.

The interface unit (10) may include a dedicated communication module that performs dedicated communication for each of the various source devices (2).

The interface unit (10) may include a common communication module that performs common communication with various source devices (2), for example, a Wi-Fi module.

The interface unit (10) may include an input interface unit and an output interface unit. In this case, the input interface unit and the output interface unit may be integrated into one module or implemented as separate modules.

The input signal processing unit (11) can generate one integrated video signal (MVS) by integrating four 4K video signals 1 to 4 received through HDMI Rx1 to Rx4.

The input signal processing unit (11) may include a decoder (111) that decodes a video signal, an image integration unit (112) that integrates multiple video signals, and an audio switching unit (113).

The decoder (111) can decode video signals 1 to 4 in TMDS format, respectively. The decoder (111) may include, for example, an FEC (Forward Error Correction) decoder or a DSC (Display Stream Compression) decoder.

The video integration unit (112) can extract active video data 1 to 4 from video signals 1 to 4 in TMDS format, for example. The video integration unit (112) can generate integrated video data (MVD; merged video data) by assigning the four extracted active video data 1 to 4 to four areas of the screen at set positions and sizes.

The image integration unit (112) can select an image signal to be displayed on the screen from among multiple image signals received from the interface unit (10). The image integration unit (112) can generate integrated image data (MVD) based on information about at least one of the position of the selected image signal and/or the size of the selected image signal.

The video integration unit (112) can generate a synchronization signal corresponding to integrated video data (MVD) based on the synchronization signals of four video signals 1 to 4.

The video integration unit (112) can generate integrated information (MVI; merged video information), such as information on the four integrated video signals 1 to 4, information on the difference between before and after integration, and information on the difference in the size of the image changed during integration.

The video integration unit (112) can generate a merged video signal (MVS) based on the integrated video data (MVD), synchronization signal, and integrated information (MVI) generated during integration and transmit it to the video processing unit (12).

The audio switching unit (113) can extract four audio signals (Audio1 to 4) included in each of four video signals 1 to 4 and transmit them to the audio processing units 1 to 4 (14). The audio switching unit (113) can switch the audio signals (Audio1 to 4) to be assigned to the audio processing units 1 to 4 based on information about output devices, for example, speakers 1 to 4, that will output the four audio signals (Audio1 to 4) corresponding to the four videos (Video1 to 4). Information about speakers 1 to 4 that will output the audio signals (Audio1 to 4) can be received from the processor (16) through user input.

The input signal processing unit (11) can be implemented as a form included in a main SoC (Main SoC) mounted on a PCB embedded in an electronic device (1).

The input signal processing unit (11) may be implemented as a processor (16), for example, a CPU (Central Processing Unit), an AP (Application Processor), or a microprocessor, that loads at least a part of a control program including instructions from a nonvolatile memory in which the control program is installed into a volatile memory and executes the instructions of the loaded control program.

The image processing unit (12) performs various image processing processes for generating image frames to be displayed on the display unit (13) for the integrated image signal (MVS) received from the input signal processing unit (11). The types of image processing processes performed by the image processing unit (12) are diverse. The image processing processes may include, for example, decoding corresponding to the image format of the integrated image data (MVS), de-interlacing for converting the integrated image data (MVS) in an interlace format into a progressive format, scaling for adjusting the integrated image data (MVS) to a preset resolution, noise reduction for improving image quality, detail enhancement, frame refresh rate conversion, etc.

The image processing unit (12) can transmit the image frame resulting from performing this process to the display unit (13) built into the electronic device (1).

The display unit (13) can display an image frame processed in the image processing unit (12).

The method of implementing the display unit (13) is not limited, and can be implemented with various display panels such as liquid crystal, plasma, light-emitting diode, organic light-emitting diode, surface-conduction electron-emitter, carbon nano-tube, nano-crystal, etc.

The display unit (13) may additionally include additional components depending on the implementation method. For example, the display unit (13) may include a timing controller (131) that adjusts the time of an image frame generated by the image processing unit (12) and a panel (132) that constitutes a screen for displaying an image. The display unit (13) may additionally include a panel driving unit that drives the panel (132).

Audio processing units 1 to 4 (14) can process four received audio signals (Audio 1 to 4). Audio processing units 1 to 4 (14) can perform conversion, amplification, mixing, etc. from digital audio signals (Audio 1 to 4) received from audio switching unit (113) to analog audio signals (Audio 1 to 4). Audio processing units 1 to 4 (14) can output the mixed analog audio signals (Audio 1 to 4) to the audio output unit (15).

The audio output unit (15) may include four speakers 1 to 4 that play back each audio signal (Audio1 to 4) included in four video signals 1 to 4. For example, speakers 1 to 3 may be built into the electronic device (1), and speaker 4 may be provided externally and connected through a second interface unit, for example, a Bluetooth communication module (19). Of course, speakers 1 to 4 may all be built in, or may all be provided externally.

The processor (16) can control each component of the electronic device (1), for example, the interface (10), the input signal processing unit (11), the image processing unit (12), the display unit (13), the audio processing unit (14), and the voice output unit (15).

The processor (16) can transmit information about the selection of images to be displayed in multiple areas of the screen, the location where the selected images are displayed, and the size of the displayed images to the input signal processing unit (11). This information can be obtained, for example, based on screen setting information input by the user through the OSD.

When a screen setting change input is received from a user, the processor (16) can transmit it to the input signal processing unit (11) and reflect it in the generation of an integrated video signal (MVS). The screen setting change input can include cases such as changing an image displayed in a specific area to a new image, exchanging displayed images between areas, or changing the size of a displayed image.

The processor (16) can transmit the finally set screen setting information to the input signal processing unit (11) to reflect it in the generation of a navigation video signal (MVS) when the electronic device (1) is powered on. Of course, the input signal processing unit (11) can also directly refer to the most recently set screen setting information stored in the memory.

The processor (16) can also transmit screen setting information set for each user ID to the input signal processing unit (11).

The processor (16) can store information about speakers 1 to 4 that will play audio signals (Audio 1 to 4) corresponding to four videos (Video 1 to 4) selected or set by the user, or transmit the information to the input signal processing unit (11).

The processor (16) can collect data, analyze, process, and generate result information using at least one of a machine learning, neural network, or deep learning algorithm as a rule-based or artificial intelligence algorithm.

For example, the processor (16) can perform the functions of a learning unit and a recognition unit. The learning unit can perform a function of generating a learned neural network, for example, and the recognition unit can perform a function of recognizing (or inferring, predicting, estimating, or judging) data using the learned neural network. The learning unit can generate or update a neural network. The learning unit can obtain learning data to generate the neural network. For example, the learning unit can obtain learning data from a memory or from the outside. The learning data can be data used for learning the neural network.

Before training a neural network using training data, the learning unit may perform a preprocessing operation on the acquired training data, or select data to be used for training from among a plurality of training data. For example, the learning unit may process the training data into a preset format, filter it, or add/remove noise to process it into a form of data suitable for training. The trained neural network may be composed of a plurality of neural network networks (or layers). Nodes of the plurality of neural network networks have weights, and the plurality of neural network networks may be connected to each other so that the output value of one neural network is used as the input value of another neural network. Examples of the neural network may include models such as a CNN (Convolutional Neural Network), a DNN (Deep Neural Network), an RNN (Recurrent Neural Network), an RBM (Restricted Boltzmann Machine), a DBN (Deep Belief Network), a BRDNN (Bidirectional Recurrent Deep Neural Network), and Deep Q-Networks.

Meanwhile, the recognition unit can obtain target data. The target data may be obtained from memory or from the outside. The target data may be data that is to be recognized by the neural network. The recognition unit may perform a preprocessing operation on the obtained target data before applying the target data to the trained neural network, or select data to be used for recognition from among a plurality of target data. For example, the recognition unit may process the target data into a preset format, filter it, or add/remove noise to process it into a form of data suitable for recognition. The recognition unit may obtain an output value output from the neural network by applying the preprocessed target data to the neural network. According to various embodiments, the recognition unit may obtain a learning rate value (or a reliability value) together with the output value.

The processor (16) includes at least one general-purpose processor that loads at least a part of a control program including instructions from a non-volatile memory in which the control program is installed into a volatile memory and executes the instructions of the loaded control program, and may be implemented as, for example, a CPU (Central Processing Unit), an AP (application processor), or a microprocessor.

The processor (16) may include a single core, a dual core, a triple core, a quad core, and a multiple thereof. A plurality of processors (16) may be provided. The processor (16) may include, for example, a main processor and a sub processor that operates in a sleep mode (e.g., a mode in which only standby power is supplied). In addition, the processor, ROM, and RAM are interconnected through an internal bus.

The processor (16) can be implemented as a form included in a main SoC (Main SoC) mounted on a PCB embedded in the electronic device (1). In another embodiment, the main SoC may further include an image processing unit.

The control program may include program(s) implemented in the form of at least one of BIOS, device driver, operating system, firmware, platform, and application program (application). The application program may be installed or stored in advance when the electronic device (1) is manufactured, or may be installed based on the received data by receiving application program data from the outside when used later. The application program data may also be downloaded to the electronic device (1) from an external server, such as an application market, for example. Such control programs, external servers, etc. are examples of computer program products, but are not limited thereto.

The electronic device (1) may further include a memory (17). The memory (17) is a computer-readable recording medium in which unlimited data is stored. The memory (17) is accessed by the processor (16), and data is read, recorded, modified, deleted, updated, etc. by these.

Memory (17) can store screen setting information, for example, image information allocated to display in multiple areas of the screen, image position information, image size information, etc.

Data stored in the memory (17) may include various image/audio contents received through the interface unit (10) and multiple frame data sequentially displayed by processing the received images. The memory (17) may include a voice recognition module (voice recognition engine) for voice recognition.

Memory (17) may include an operating system, various applications executable on the operating system, image data, additional data, etc.

Memory (17) includes non-volatile memory in which a control program is installed and volatile memory in which at least a portion of the installed control program is loaded.

The memory (17) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

The electronic device (1) may include a voice recognition unit (18).

The voice recognition unit (18) can recognize voice input or transmitted from a microphone built into the electronic device (1) or an external device, such as a microphone built into a mobile device or remote control, by executing a voice recognition module stored in the memory (17).

When receiving voice through the microphone of a mobile device or remote control, the mobile device or remote control can digitize the analog voice signal and transmit it to the electronic device (1) side, for example, via Bluetooth.

When the electronic device (1) itself receives a microphone voice signal, the received analog voice signal can be digitized and transmitted to the processor (16) of the electronic device (1).

The electronic device (1) can transmit the received voice signal to the server. At this time, the server may be an STT server that converts voice signal-related data into appropriate text or a main server that also performs STT server functions.

Data processed by the STT server can be received again by the electronic device (1) or transmitted directly to another server.

Of course, the electronic device (1) may process the received voice signal within the electronic device (1) without transmitting it to the STT server. In other words, the electronic device (1) may perform the role of an STT server on its own.

The electronic device (1) can perform a specific function using text transmitted from a server or text converted by itself. At this time, the function may be performed by a processor (16) within the electronic device (1) or a separate server (a server other than the STT server or a server that also functions as an STT server) to which the converted text is transmitted.

FIG. 3 is a flowchart showing a control method of an electronic device (1) for displaying a split image according to an embodiment of the present invention.

The user can set Video1, Video2, Video3, and Video4 to be displayed in the 1/4 quadrant, 2/4 quadrant, 3/4 quadrant, and 4/4 quadrant of the screen, respectively. At this time, the user can also adjust the size of Video1, Video2, Video3, and Video4 displayed in each area. The processor (16) can transmit such screen setting input information, i.e., the position and/or size of the image, to the input signal processing unit (11).

In step S11, the interface unit (10) can receive four video signals 1 to 4. At this time, the interface unit (10) can be HDMI as shown in Fig. 2.

In step S12, the input signal processing unit (11) can select four 4K video signals 1 to 4 to be displayed on the screen among a plurality of video signals and perform decoding.

Video signals 1 to 4 can be composed of four TMDS formats as shown in Fig. 4.

TMDS (Transition Minimized Differential Signal) transmits video, audio, and other data using one of three modes: 'video data period', 'data island period', and 'control period'. Video data can be transmitted during the 'video data period'. The 'data island period' occurs during the 'horizontal/vertical retracement period', so that audio and other data can be sent divided into multiple packets. The 'control period' can occur between the 'video data period' and the 'data island period'.

HDMI can transmit 10-bit video data using 8b/10b encoding in the 'video data period' and 2b/10b encoding in the 'control period' using TMDS. In addition, HDMI can transmit voice and other data using 4b/10b encoding in the 'data island period'. At this time, data equivalent to 32 pixels can be transmitted in one 'data island period', and a 32-bit packet header describing the contents of the packet can be included. The packet header can include 8-bit BCH ECC (Error Correction Code) parity data for error correction function.

Each packet can have four subpackets. Each subpacket can be 64 bits. This packet can also contain 8 bits of BCH ECC parity data. Also, each 'data island period' can transmit up to 18 packets. Of the 15 packet types in the HDMI 1.3a specification, 7 can be allocated for audio, and the remaining 8 can be allocated for other data. Among these, there are standard control packets and Gamut metadata packets. The standard control packet can have a function to mute when noise occurs in the audio (AVMUTE) and information about color depth. The Gamut metadata packet can contain information about the color space for the video stream being played, which is necessary for using xvYCC.

Figure 4 is a diagram showing video signals 1 to 4 (Video1 to 4) in TMDS format.

Video signal 1 may include active video data 1 (Active Video 1), audio data 1 (Audio 1), and other data 1 displayed on the entire screen. Video signal 1 may include vertical and horizontal synchronization signals 1 (VSync 1, HSync 1) for displaying active video data 1 on the entire screen.

Video signal 2 may include active video data 2 (Active Video2), audio data 2 (Audio2), and other data 2 displayed on the entire screen. Video signal 2 may include vertical and horizontal synchronization signals 2 (VSync2, HSync2) for displaying active video data 2 on the entire screen.

Video signal 3 may include active video data 3 (Active Video3), audio data 3 (Audio3), and other data 3 displayed on the entire screen. Video signal 3 may include vertical and horizontal synchronization signals 3 (VSync3, HSync3) for displaying active video data 3 on the entire screen.

Video signal 4 may include active video data 4 (Active Video 4), audio data 4 (Audio 4), and other data 4 displayed on the entire screen. Video signal 4 may include vertical and horizontal synchronization signals 4 (VSync 4, HSync 4) for displaying active video data 4 on the entire screen.

In step S13, the input signal processing unit (11) can extract active video data 1 to 4 (Active Video 1 to 4) from video signals 1 to 4.

In step S14, the input signal processing unit (11) can generate integrated video data (MVD) that assigns active video data 1 to 4 (Active Video 1 to 4) to multiple areas of the screen based on position and size information to be displayed among multiple areas of the screen, as shown in FIG. 5.

For example, integrated video data (MVD) may have Active Video data 1 (Active Video1) of a predetermined size placed in the 1/4 quadrant of the screen, Active Video data 2 (Active Video2) of a predetermined size placed in the 2/4 quadrant of the screen, Active Video data 3 (Active Video3) of a predetermined size placed in the 3/4 quadrant of the screen, and Active Video data 4 (Active Video4) of a predetermined size placed in the 4/4 quadrant of the screen.

Figure 5 is a diagram showing an integrated video signal (MVS).

The input signal processing unit (11) can generate vertical and horizontal synchronization signals M (MVSync, MHSync) of integrated video data (MVD) by referring to the position and/or size of active video data 1 to 4 (Active Video1 to 4) and vertical and horizontal synchronization signals 1 to 4 (VSync1 to 4, HSync1 to 4) of active video data 1 to 4 (Active Video1 to 4).

The input signal processing unit (11) can generate integrated information (MVI), such as information on integrated video data (MVD) and information on the changed position and/or size difference of active video data 1 to 4 (Active Video 1 to 4).

The input signal processing unit (11) can generate an integrated video signal (MVS) including generated integrated video data (MVD), vertical and horizontal synchronization signals M (MVSync, MHSync), and integrated information (MVI).

In step S15, the image processing unit (12) extracts active integrated image data based on the integrated image signal (MVS), and performs various processing for displaying the extracted integrated image data on the screen to generate an image frame. The processing for generating the image frame may include, for example, decoding corresponding to the image format of the integrated image data (MVS), de-interlacing for converting the integrated image data (MVS) in an interlace format into a progressive format, scaling for adjusting the integrated image data (MVS) to a preset resolution, noise reduction for improving image quality, detail enhancement, frame refresh rate conversion, etc.

In step S16, the display unit (14) can display an integrated image corresponding to the integrated image data (MVD) on the entire screen based on the generated image frame.

As described above, the electronic device (1) of the present invention can process multiple image signals to be displayed in multiple areas of the screen with only one image processing unit (12) by adjusting and integrating the active image data of each of the multiple image signals to fit the position and/or size to be displayed among multiple areas of the screen before the image processing unit (12) processes the received multiple image signals.

In addition, the electronic device (1) of the present invention can display an 8k image as a whole by dividing the screen into four and then allocating four 4k image signals 1 to 4 to each divided area and displaying them.

Figure 6 is a block diagram showing the configuration of an electronic device (1) according to a second embodiment of the present invention.

An electronic device (1) according to a second embodiment excludes a display unit that displays an image on its own, and integrates four image signals 1 to 4 received via an interface unit (10), for example, HDMI, to generate an integrated image signal (MVS), and generates an image frame based on the integrated image signal (MVS) to output it to an externally provided display device (3), for example, a television or monitor. In addition, audio signals 1 to 4 included in the four image signals 1 to 4 can be processed and output to an externally provided audio output device (4) via a cable (C) or a Bluetooth communication module (19). Of course, the electronic device (1) according to the second embodiment may also include a display unit for displaying simple notifications, control menus, etc.

Figure 7 is a block diagram showing the configuration of an electronic device (1) according to a third embodiment of the present invention.

An electronic device (1) according to a third embodiment receives information about an image to be integrated, the image's layout on the screen when integrated, the size of the image to be displayed, and information about a speaker that reproduces an audio signal of the image to be displayed, from a display device (3), and integrates multiple image signals 1 to 4 to generate a single integrated image signal (MVS). An explanation of the generation of the integrated image signal (MVS) is omitted because it is similar to the input signal processing unit (11) according to the first embodiment shown in Fig. 2.

An electronic device (1) according to the third embodiment can generate an integrated video signal (MVS) by integrating, for example, four video signals 1 to 4 to be displayed by assigning them to multiple areas of a screen, and then transmit the integrated video signal to a display device (3). At this time, the video signals 1 to 4 can be received from an external source device (2) through an interface unit.

The display device (3) receives an integrated video signal (MVS) in which video signals 1 to 4 to be displayed in multiple areas of the screen are combined from the electronic device (1), generates a video frame for screen display, and can display the video frame on the screen of the display unit (33).

FIG. 8 is a diagram showing a scenario for processing and displaying multiple image signals of an electronic device (1) according to a fourth embodiment of the present invention.

The user can change Video1, Video2, Video3, and Video4, which were displayed in the 1/4, 2/4, 3/4, and 4/4 quadrants of the existing screen, respectively, to a USB Movie in the 4/4 quadrant. At this time, the processor (16) can transmit this screen setting input information to the input signal processing unit (11).

The interface unit (10) may include HDMI1 to 4, which receive four first video signals 1 to 4 respectively, and a USB interface connected to a USB in which a second video signal (USB Movie) is stored.

The input signal processing unit (11) selects the first video signals 1 to 3 among the four first video signals 1 to 4, extracts active video data 1 to 3 (Active Video 1 to 3), and allocates the active video data 1 to 3 (Active Video 1 to 3) to the 1/4 quadrant, 2/4 quadrant, and 3/4 quadrant areas of the screen, respectively, to generate integrated video data (MVD), and then generates an integrated video signal (MVS) based on this. In this way, the integrated video data (MVD) can be generated with the 4/4 quadrant area of the screen empty. The integrated video signal (MVS) can be transmitted to the video processing unit 1 (121).

The second video signal (USB Movie) received via the USB interface can be transmitted to the video processing unit 2 (122).

The input signal processing unit (11) can extract audio signals 1 to 2 corresponding to video signals 1 to 2 and then transmit them to audio processing units 1 to 2 (141, 142).

The video processing unit 1 (121) can generate a first video frame for displaying the integrated video signal (MVS) in the 1/4 quadrant, 2/4 quadrant, and 3/4 quadrant of the screen.

The video processing unit 2 (122) can generate a second video frame for displaying the second video signal (USB Movie) on the 4/4 quadrant of the screen.

The display unit (13) can display the first video 1 to 3 (Video 1 to 3) and the second video (USB Movie) on the screen by synthesizing the first video frame and the second video frame.

The user can specify that audio signal 1 of Video 1 of HDMI1 be played through TV speakers (151) and audio signal 2 of Video 2 of HDMI2 be played through Bluetooth speakers (152).

Audio processing units 1 to 2 (141, 142) can process audio signals 1 to 2 respectively and transmit them to the TV speaker (151) and Bluetooth speaker (152) of the audio output unit (15) for playback.

FIG. 9 is a diagram showing a scenario for processing and displaying multiple image signals of an electronic device (1) according to a fifth embodiment of the present invention.

The user can change Video1, Video2, Video3, and Video4, which were displayed in the 1/4, 2/4, 3/4, and 4/4 quadrants of the existing screen, to Video3, Video2, Video1, and Video4. At this time, the processor (16) can transmit this screen setting input information to the input signal processing unit (11).

The input signal processing unit (11) selects all of the first video signals 1 to 4 among the four first video signals 1 to 4, extracts active video data 1 to 4 (Active Video 1 to 4), and allocates the active video data 1 to 4 (Active Video 1 to 4) to the areas of the 3/4 quadrant, 2/4 quadrant, 1/4 quadrant, and 4/4 quadrant of the screen, respectively, to generate integrated video data (MVD), and then generates an integrated video signal (MVS) based on this. In this way, when the order of the displayed images is changed according to the screen settings, the input signal processing unit (11) can reflect this and generate the integrated video data (MVD) so that the display order of the images is changed, and then the integrated video data can be transmitted to the video processing unit (12).

The input signal processing unit (11) can extract audio signals 1 to 2 corresponding to video signals 1 to 2 and then transmit them to audio processing units 1 to 2 (141, 142).

The video processing unit (12) can generate video frames for displaying Video 3, Video 2, Video 1, and Video 4 on the 1/4, 2/4, 3/4, and 4/4 quadrants of the screen, respectively, based on the integrated video signal (MVS).

The display unit (13) can display the image frame transmitted from the image processing unit (12) on the screen.

The user can specify that audio signal 1 of Video 1 of HDMI1 be played through TV speakers (151) and audio signal 2 of Video 2 of HDMI2 be played through Bluetooth speakers (152).

Audio processing units 1 to 2 (141, 142) can process audio signals 1 to 2 respectively and transmit them to the TV speaker (151) and Bluetooth speaker (152) of the audio output unit (15) for playback.

As a modified embodiment, the electronic device (1) can be applied to PIP (Picture in Picture) that can simultaneously display a small additional image separately within the main image on the screen. At this time, the image integration unit (112) can receive PIP setting information and generate an integrated image signal (MVS) by reflecting the position and size of the additional image on the main image of the entire screen. As a result, the electronic device (1) can process the main image and the additional image with one image processing unit and display the PIP image.

According to an embodiment of the present invention, an input signal processing module that displays multiple images by allocating them to multiple areas of a screen may be implemented as a computer program product stored in a memory (17) as a computer-readable recording medium or as a computer program product transmitted and received via network communication. In addition, the above-described input signal processing module may be implemented as a computer program alone or in combination.

Although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications may be made by those skilled in the art without departing from the gist of the present invention as claimed in the claims. In addition, such modified embodiments should not be understood individually from the technical idea or prospect of the present invention.

1: Electronic devices
10: Interface section
11: Input signal processing unit
111: Decoder
112: Video Integration Department
113: Audio switching section
12, 121,122: Image processing unit
13: Display section
14,141,142: Audio Processing Unit
15: Voice output section
151: TV Speaker
152: Bluetooth Speaker
16: Processor
17: Memory
18: Voice recognition unit
19: Bluetooth module
2: Source device
3: Display device
4: Voice output device

Claims (20)

A first interface unit for receiving n first video signals and a second interface unit for receiving a second video signal of a different type from the first video signals;
An input signal processing unit which extracts n-1 image data from among the n first image signals and generates one integrated image signal having integrated image data in which n-1 images based on the extracted n-1 image data are each assigned to n-1 areas among n areas of the screen;
A first image processing unit that generates a first image frame for display on the screen based on the integrated image data of the integrated image signal;
A second image processing unit that generates a second image frame for display in an area remaining from among n areas of the screen, excluding n-1 areas, based on the second image signal; and
An electronic device comprising a timing controller that controls the timing of the first image frame and the second image frame generated by the first image processing unit and the second image processing unit, and a panel that constitutes the screen for displaying images, and a display unit that synthesizes the first image frame and the second image frame to display the first image of the first image signal and the second image of the second image signal in n areas of the screen. In the first paragraph,
An electronic device in which the n first video signals are received via an HDMI (High-Definition Multimedia Interface) and the second video signals are received via a USB interface. In the first paragraph,
An electronic device comprising an input signal processing unit, a decoder for decoding the received first image signal; and an image integration unit for extracting image data from the decoded first image signal and generating the integrated image data. In the third paragraph,
The above image integration unit is an electronic device that generates the integrated image signal based on information regarding at least one of the position and size of the first image signal. ◈Claim 5 was waived upon payment of the registration fee.◈ In the third paragraph,
The above image integration unit is an electronic device that generates a synchronization signal of the integrated image signal corresponding to the integrated image data based on the synchronization signal of the first image signal. ◈Claim 6 was waived upon payment of the registration fee.◈ In the third paragraph,
An electronic device in which the integrated video signal includes information about the difference in position or size between before and after integration of the first video signal. delete delete delete delete In the first paragraph,
The above first image processing unit,
Decoding corresponding to the video format of the above video data;
De-interlacing, which converts the above image data in interlace mode into progressive mode;
Scaling, which adjusts the above image data to a preset resolution;
Noise reduction to improve image quality;
Detail enhancement; or
An electronic device performing at least one of frame refresh rate conversions. In a method for controlling an electronic device,
A step of receiving n first image signals and a second image signal of a different type from the first image signals;
A step of extracting n-1 image data from among the n first image signals, and generating one integrated image signal having integrated image data in which n-1 images based on the extracted n-1 image data are each assigned to n-1 areas among n areas of the screen;
A step of generating a first image frame for display on the screen based on the integrated image data of the integrated image signal;
A step of generating a second image frame for display in an area remaining from among n areas of the screen, excluding n-1 areas, based on the second image signal; and
A control method for an electronic device, comprising the step of synthesizing the first image frame and the second image frame by controlling the time of the generated first image frame and the second image frame, thereby displaying the first image of the first image signal and the second image of the second image signal in n areas of the screen. In Article 12,
A method for controlling an electronic device, wherein the n first video signals are received via HDMI and the n second video signals are received via a USB interface. In Article 12,
A method for controlling an electronic device, wherein the step of generating the above integrated video signal includes a step of decoding the first video signal. In Article 14,
A control method for an electronic device in which the integrated video signal is generated based on at least one of the position and size information of the first video signal. ◈Claim 16 was abandoned upon payment of the registration fee.◈ In Article 14,
A method for controlling an electronic device, wherein the step of generating the integrated image signal includes a step of generating a synchronization signal of the integrated image signal corresponding to the integrated image data based on the synchronization signal of the first image signal. ◈Claim 17 was abandoned upon payment of the registration fee.◈ In Article 14,
A control method for an electronic device, wherein the integrated video signal includes information about the difference in position or size before and after integration of the first video signal. delete delete In Article 12,
The above first video frame generation step is:
Decoding corresponding to the video format of the above video data;
De-interlacing, which converts the above image data in interlace mode into progressive mode;
Scaling, which adjusts the above image data to a preset resolution;
Noise reduction to improve image quality;
Detail enhancement; or
A method of controlling an electronic device that performs at least one of frame refresh rate conversions.