KR101240274B1 - Mobile terminal and contents mixing method thereof - Google Patents

Abstract

The present invention relates to a mobile terminal for performing content mixing of a device connected through a DLNA (Digital Living Network Alliance) and a content mixing method thereof. According to the present invention, when at least one device is connected to a DLNA, a preview screen of the connected device is displayed, and the user conveniently controls the device by performing mixing between contents being played in each device according to the user's mixing gesture on the displayed preview screen. And content mixing.

Description

MOBILE TERMINAL AND CONTENTS MIXING METHOD THEREOF

The present invention relates to a mobile terminal capable of effectively controlling the operation of a device connected by a DLNA (Digital Living Network Alliance) and content mixing between devices, and a content mixing method thereof.

The mobile terminal may be configured to perform various functions. Examples of such various functions include a data and voice communication function, a function of photographing a video or a moving image through a camera, a voice storage function, a music file playback function through a speaker system, and an image or video display function. Some mobile terminals include additional functionality to play games, while others are implemented as multimedia devices. Moreover, recent mobile terminals can receive broadcast or multicast signals to watch video or television programs.

Further, efforts for supporting and increasing the functions of the mobile terminal continue. The foregoing efforts include not only changes and improvements in the structural components forming the mobile terminal, but also improvements in software or hardware. Among them, the touch function of the mobile terminal allows a user who is unfamiliar with the button / key input to conveniently perform the operation of the terminal by using the touch screen. In recent years, not only simple input but also user interface (UI) Is becoming an important function of Accordingly, as the touch function is applied to a mobile terminal in various forms, development of a user interface (UI) corresponding thereto is further demanded.

DLNA stands for Digital Living Network Alliance, a standard that allows at least one device to send and receive multimedia content (music, photos, videos, etc.) through a home network. The DLNA technology is applied to PCs, TVs, Blu-ray players, printers, knit work storage, and recently, it is being actively applied to mobile devices such as mobile phones.

Therefore, if the PC, TV, PS3, and mobile device in the home electronics have DLNA function, the images, music, and photos stored on the mobile device through Wi-Fi without physically connecting each other (cable connection) Can be shared with each other. For example, you can send movies, music, and photos stored on your PC to your TV, Blu-ray player, and home theater via Wi-Fi, or transfer video from your mobile device to your PC or TV for larger screen, picture quality, and sound quality. Can be.

In particular, as the wireless N screen service for sharing contents of a smartphone, a tablet PC, a smart TV, and a PC is recently launched, the DLNA is becoming an essential function of a smartphone. Therefore, development of various user interfaces (UI) using the DLNA is more urgently needed than ever.

It is an object of the present invention to provide a mobile terminal capable of selectively mixing contents of a device connected by DLNA and a content mixing method thereof.

Another object of the present invention is to provide a mobile terminal and a content mixing method thereof, which provide a user interface (UI) for controlling the operation of a device connected by DLNA and controlling content mixing between devices.

Of the mobile terminal according to an embodiment of the present invention for realizing the above object

The method for controlling content mixing includes connecting at least one device to a DLNA; Displaying a preview screen of the connected device; And performing mixing between contents being played on the device by a mixing gesture in the preview screen.

The preview screen indicates a state of each device currently operating.

The number of devices displayed on the preview screen varies according to the location of the user in the home.

The device displayed on the preview screen may be activated or deactivated according to a user's selection, and the deactivated device is displayed in a size changed to a translucent state.

The device includes a mobile terminal, an audio device, a video device and a digital picture frame, a PC, and a printer, and the content includes images, music, pictures, and text.

The mixing gesture may be touch & drag.

The mixing of the contents may include determining a type of the mixing gesture; Determining the type of the touch & drag when the mixing gesture is touch &drag; If the touch & drag is touch and drag between devices, mixing contents of a corresponding device; And if the touch and drag is touch and drag in the device, controlling an operation of the corresponding device.

The mixing of the contents may further include displaying a list of contents that can be played on the corresponding device when the mixing gesture is touch & hold.

The mixing of the contents may further include displaying a detailed setting menu or a controller for performing detailed settings for the corresponding device when the mixing gesture is a long touch.

According to an aspect of the present invention, there is provided a mobile terminal including: a display unit configured to display preview screens of devices connected by DLNA; And a controller configured to control mixing between contents being played in the device according to the mixing gesture detected in the preview screen.

The preview screen indicates a state of each device currently operating.

The preview screen is displayed by varying the number of devices displayed according to the user's location at home.

The controller activates or deactivates a specific device displayed on the preview screen according to a user's selection, and the deactivated device is resized to a translucent state.

The device includes a mobile terminal, an audio device, a video device and a digital picture frame, a PC, and a printer, and the content includes images, music, pictures, and text.

The mixing gesture is touch & drag.

The controller determines the type of the mixing gesture to determine the type of the touch & drag when the mixing gesture is touch & drag, and performs mixing on the contents of the device when the touch & drag is touch & drag between devices. And touch & drag in the device to control the operation of the device.

If the mixing gesture is touch & hold, the controller displays a list of contents that can be played on the corresponding device.

If the mixing gesture is a long touch, the controller displays a detailed setting menu or a controller for performing detailed settings for the corresponding device.

According to an embodiment of the present invention, a preview screen of a plurality of devices connected through DLNA is displayed on a mobile terminal, and each device is operated through a mixing gesture or the contents of each device are mixed and displayed / played on the screen. As a result, the user can conveniently perform operation control and content mixing of each device.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention;
2 is a block diagram of a wireless communication system in which a mobile terminal may operate in accordance with an embodiment of the present invention.
3 is a connection diagram of at least one device connected via DLNA.
4 is an example of a preview screen of a device connected by DLNA.
FIG. 5 illustrates an example of performing inter-device mixing by a mixing gesture in a preview screen of a DLNA connected device; FIG.
6A and 6B illustrate another example of performing a device-to-device mixing operation by a mixing gesture in a preview screen of a DLNA connected device.
7A and 7B illustrate examples of mixable lists on a preview screen.
8 is an example of mixing by selecting a specific list from a mixable list.
9 is a diagram illustrating an example of controlling an operation of a device in a preview screen of a DLNA connected device;
10 is a diagram illustrating an example of performing detailed setting of a device in a preview screen of a DLNA connected device;
11A and 11B illustrate an example of controlling a computer through a mobile terminal during DLNA connection.
12 is a flowchart illustrating a content mixing method of a mobile terminal according to an embodiment of the present invention.

Hereinafter, a mobile terminal related to the present invention will be described in detail with reference to the drawings. The suffix "module" and " part "for components used in the following description are given merely for convenience of description, and do not give special significance or role in themselves. Therefore, it should be noted that the "module" and "unit" may be used interchangeably with each other.

The terminal may be implemented in various forms. For example, a terminal described herein includes a mobile terminal such as a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, and the like. And fixed terminals such as digital TVs, desktop computers, and the like. In the following description, it is assumed that the terminal is a mobile terminal. However, it will be readily apparent to those skilled in the art that the configuration according to the following description may be applied to the fixed terminal, except for components specifically configured for mobile use.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, A controller 170, a controller 180, a power supply 190, and the like. 1 shows a mobile terminal having various components. However, not all illustrated components are required. A mobile terminal may be implemented by more components than the illustrated components, or a mobile terminal may be implemented by fewer components.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more components that allow wireless communication between the mobile terminal 100 and the wireless communication system or wireless communication between the mobile terminal 100 and a network on which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

Meanwhile, the broadcast related information may be provided through a mobile communication network, and in this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcasting signals using various broadcasting systems. In particular, the broadcasting receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S) Only Digital Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 is configured to be suitable for all broadcasting systems that provide broadcasting signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

In addition, the mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data depending on a voice call signal, a video call signal, or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and the wireless Internet module 113 can be built in or externally. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short range communication module 114 refers to a module for short range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, and the like can be used as the short distance communication technology.

The location information module 115 is a module for confirming or obtaining the location of the mobile terminal. A typical example of the position information module 115 is a Global Position System (GPS) module. According to the current technology, the GPS module calculates distance information and accurate time information from three or more satellites, and then applies trigonometry to the calculated information to obtain three-dimensional current position information according to latitude, longitude, and altitude It can be calculated accurately. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module can calculate speed information by continuously calculating the current position in real time.

An audio / video (A / V) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes an image frame such as a still image or a moving image obtained by the image sensor in the video communication mode or the photographing mode. Then, the processed image frame can be displayed on the display module 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. The microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a key pad, a dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like. Particularly, when the touch pad has a mutual layer structure with the display module 151 described later, it can be called a touch screen.

The sensing unit 140 is a string of the mobile terminal 100 such as an open / closed state of the mobile terminal 100, a position of the mobile terminal 100, presence or absence of a user contact (touch), orientation of the mobile terminal, acceleration / deceleration of the mobile terminal, and the like. The sensing state generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. Also, it is responsible for a sensing function related to whether or not the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor 141. The user's touch includes touch, touch & drag, multi-touch, long touch as well as proximity touch. The sensing unit 140 includes a geomagnetic sensor for calculating a moving direction when a user moves, a gyro sensor for calculating a rotating direction, and an acceleration sensor.

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, Video input / output (I / O) ports, earphone ports, and the like.

Here, the identification module is a chip that stores various information for authenticating the use right of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identity module (SIM) ), A Universal Subscriber Identity Module (" USIM "), and the like. In addition, an apparatus having an identification module (hereinafter referred to as 'identification device') can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port. The interface unit 170 receives data from an external device or receives power from the external device to transfer the data to each component in the mobile terminal 100 or to transmit data in the mobile terminal 100 to an external device.

The interface unit 170 may be a path through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, Various command signals may be transmitted to the mobile terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 may include a display module 151, an audio output module 152, and an alarm unit 153.

The display module 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed.

Meanwhile, as described above, when the display module 151 and the touch pad have a mutual layer structure to constitute a touch screen, the display module 151 can be used as an input device in addition to the output device. The display module 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display 3D display). Some of these displays can be configured to be transparent so that they can be viewed from outside. This may be referred to as a transparent display. A typical example of the transparent display is a transparent organic light emitting diode (TOLED) or the like. In addition, there may be two or more display modules 151 according to the embodiment of the mobile terminal 100. For example, the mobile terminal 100 may be provided with an external display module (not shown) and an internal display module (not shown) at the same time. The touch screen may be configured to detect not only the touch input position and area but also the touch input pressure.

The audio output module 152 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 outputs an acoustic signal related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The sound output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The video signal or the audio signal may be output through the display unit 151 or the audio output module 152 so that they may be classified as a part of the alarm unit 153.

The haptic module 154 generates various tactile effects that the user can feel. Vibration is a representative example of the haptic effect generated by the haptic module 154. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to vibration, the haptic module 154 may be configured to provide a pin array that vertically moves with respect to the contact skin surface, a jetting force or suction force of air through an injection or inlet port, grazing to the skin surface, contact of an electrode, electrostatic force, and the like. Various tactile effects can be generated, such as effects by the endothermic and the reproduction of a sense of cold using the elements capable of endotherm or heat generation.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sensation of the finger or arm. At least two haptic modules 154 may be provided according to the configuration of the mobile terminal 100.

The memory 160 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 160 may store data on vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), RAM random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic It may include a storage medium of at least one type of disk, optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device, receives power, transfers the power to each component inside the mobile terminal 100, or transmits data inside the mobile terminal 100 to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the usage right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module a universal subscriber identity module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

The interface unit may be a passage through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, or various command signals input from the cradle by a user may be transferred. It may be a passage that is delivered to the terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases, And may be implemented by the control unit 180.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. In addition, the software codes may be stored in the memory 160 and executed by the control unit 180. [

In the above, the mobile terminal related to the present invention has been described in terms of components according to functions. The present invention is not limited to a slider type mobile terminal, but may be applied to various types of mobile terminals such as a folder type, a bar type, a swing type, a slider type, and the like.

The terminal 100 shown in Fig. 1 is configured to be operable in a communication system capable of transmitting data through a frame or packet, including a wired / wireless communication system and a satellite-based communication system .

Hereinafter, a communication system capable of operating a terminal according to the present invention will be described with reference to FIG.

The communication system may use different air interfaces and / or physical layers. For example, wireless interfaces that can be used by communication systems include, but are not limited to, Frequency Division Multiple Access ('FDMA'), Time Division Multiple Access ('TDMA'), Code Division Multiple Access (CDMA), Universal Mobile Telecommunications Systems (UMTS) (especially Long Term Evolution (LTE)), Global System for Mobile Communications (GSM) . Hereinafter, for convenience of description, the description will be limited to CDMA. However, the present invention is applicable to all communication systems including CDMA wireless communication systems.

2, the CDMA wireless communication system includes a plurality of terminals 100, a plurality of base stations (BSs) 270, and base station controllers (BSCs) 275 , And a mobile switching center ('MSC') 280. The MSC 280 is configured to be connected to a public switched telephone network (PSTN) 290 and is also configured to be connected to BSCs 275. BSCs 275 may be coupled in pairs with BS 270 through a backhaul line. The backhaul line may be provided according to at least one of E1 / T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL, or xDSL. Thus, a plurality of BSCs 275 may be included in the system shown in FIG.

Each BS 270 may comprise at least one sector, and each sector may comprise an omnidirectional antenna or an antenna pointing to a particular direction of radial from BS 270. [ In addition, each sector may include two or more antennas of various types. Each BS 270 may be configured to support a plurality of frequency assignments, each of which has a specific spectrum (eg, 1.25 MHz, 5 MHz, etc.).

The intersection of sectors and frequency assignments may be called a CDMA channel. BS 270 may be referred to as a Base Station Transceiver Subsystem (BTSs). In this case, the word "base station" may be referred to as a combination of one BSC 275 and at least one BS 270. [ The base station may also indicate “cell site”. Alternatively, each of the plurality of sectors for a particular BS 270 may be referred to as a plurality of cell sites.

2, a broadcasting transmitter (BT) 295 transmits a broadcasting signal to the terminals 100 operating in the system. The broadcast receiving module 111 shown in FIG. 1 is provided in the terminal 100 to receive a broadcast signal transmitted by the BT 295.

In addition, FIG. 2 illustrates several Global Positioning System ('GPS') satellites 300. The satellites 300 help to locate at least one of the plurality of terminals 100. Although two satellites are shown in FIG. 2, useful location information may be obtained by two or more satellites. The location information module 115 shown in FIG. 1 cooperates with satellites 300 to obtain desired location information. Here, the location can be tracked using not only the GPS tracking technology but also all the technologies that can track the location. Also, at least one of the GPS satellites 300 may optionally or additionally be responsible for satellite DMB transmission.

Among the typical operations of a wireless communication system, the BS 270 receives a reverse link signal from the various terminals 100. At this time, the terminals 100 are connecting a call, transmitting or receiving a message, or performing another communication operation. Each of the reverse link signals received by the particular base station 270 is processed by the particular base station 270. The data resulting from the processing is transmitted to the connected BSC 275. The BSC 275 provides call resource allocation and mobility management functions, including the organization of soft handoffs between the base stations 270. The BSC 275 also transmits the received data to the MSC 280 and the MSC 280 provides additional transmission services for connection with the PSTN 290. [ Similarly, the PSTN 290 connects to the MSC 280, the MSC 280 connects to the BSCs 275, and the BSCs 275 communicate with the BSs 270 ).

Digital Living Network Alliance (DLNA) is a home network technology standard that allows digital content to be freely transferred and shared among a variety of devices, including PCs, smartphones, smart pads, and TVs. The DLNA is attracting attention as a major technology for implementing an N screen service while being applied to mobile devices in earnest beyond a simple home network. The DLNA can share content directly between devices using a mobile communication network or a Wi-Fi network without uploading content to a central server.

According to the present invention, when at least one device is connected through DLNA, the function and content provided by each device in the preview screen of the mobile terminal are mixed (synthesized) by a predetermined mixing gesture, and thus the mixed function and We propose a set of user interfaces for running or displaying content.

DLNA connection

3 is a connection diagram of at least one device connected via DLNA.

As shown in FIG. 3, home electronic devices (mobile terminals, audio devices, video devices, digital photo frames, PCs, printers, and the like) are connected to a Wi-Fi router (not shown). If the electronic devices are designed according to the DLNA standard, DLNA connection is possible even if the manufacturer and product name are different.

Therefore, when a predetermined icon (eg, all share or smart share) displayed on the standby screen (desktop) of the mobile terminal is selected, the mobile terminal operates as a DLNA controller to wirelessly connect to the screen of the mobile terminal in the current home in a DLNA manner. All devices present are displayed in a preview form (hereinafter referred to as a preview screen). In this case, the status of each device currently operating (e.g., TV viewing screen, electronic picture display screen, etc.) is displayed on the screen of the mobile terminal as it is.

Preview screen display when connecting to DLNA

4 is an example of a preview screen of a device connected by DLNA.

The display of devices connected by DLNA in FIG. 4 is based on the user's convenience of operation. That is, the devices displayed on the preview screen may be displayed at a predetermined position of the screen according to the size of the device, the DLNA connection order, the most frequent and the latest usage order as well as the user's setting. In particular, the devices displayed on the preview screen may be arbitrarily changed or moved by touch and drag.

Available devices may be limited according to the location of the user in the home. For example, if the TV is in the living room and the electronic frame is in the room, two devices may not be used at the same time. In this case, the user may deactivate a user by selecting a specific device (menu or touch input) on the preview screen, and the controller 180 may display only devices that are automatically available according to the user's location.

In addition, the deactivated device may be displayed in a translucent state or resized in a translucent state. If the size is reduced in the translucent state, the size of the currently active device may be relatively large.

In the preview screen shown in FIG. 4, a TV 101 is displayed at the center of the screen, an audio device (eg, a speaker) 102 at both sides of the TV 101, a video device 104, and the lower side thereof. It is displayed on the digital picture frame 103 around the video device 104.

Accordingly, the user may touch-manipulate the devices displayed on the free unit screen of the mobile terminal 100 to mix the functions and the contents provided by each device to execute or display the mixed functions and the contents.

Perform mixing between DLNA connected devices

5 is an example of performing a device-to-device mixing operation by a mixing gesture in a preview screen of a DLNA connected device.

In FIG. 5, it is assumed that a video of a predetermined program is displayed on the screen of the TV 101, predetermined music is reproduced in the audio device 102, and a photo slideshow of a specific person is in progress in the digital frame 103. do. The state of each device is displayed on the preview screen of the mobile terminal.

In the preview screen, when the user touches the audio device 102 on which the music is currently playing and drags it to the TV screen on which the current video is displayed, the video of the TV 101 and the music of the audio device 102 are mixed ( Synthesized), and the mixed image is reproduced through the display unit 151 of the mobile terminal serving as a DLNA controller. In this case, when the recording function for the TV image is set, the mixed image is stored.

In addition, the present invention may display a message asking the user whether to mix and a subsequent operation (e.g., save, delete, etc.) once the mixing gesture is detected in order to prevent mixing due to an erroneous mixing gesture.

6A and 6B illustrate another example of performing a device-to-device mixing operation by a mixing gesture in a preview screen of a DLNA connected device.

As shown in FIG. 6A, when a user touches an audio device 102 on which music is currently playing and drags it to an electronic picture frame 103 on which a photo slide show is currently being performed, the user may select a picture of the digital picture frame 103. The music of the audio device 102 is mixed (synthesized) and played through the display unit 151 of the mobile terminal serving as a DLNA controller.

And, as shown in Figure 6b, when the size of the electronic picture frame is small, for accurate mixing, when the finger touching the electronic device is close to the size of the electronic picture frame 103 and the slide show is to be paused Can be.

Accordingly, the present invention can be reproduced or stored on the screen of the mobile terminal by mixing a specific image and music being played on another device by a simple mixing gesture, as shown in Figures 5 and 6a. That is, the present invention mixes multimedia content between all devices connected to the DLNA, such as a file of a computer, an image of another device, an image of a mobile terminal, and a voice of a mobile terminal, and the mixed content through a mobile terminal operated by a DLNA controller. Play and save.

Although the content mixing illustrated in FIGS. 5, 6A, and 6B is for content currently being played, the present invention may apply content mixing to all playlists (lists) that are not currently being played but are playable. .

7A and 7B illustrate examples of a mixable list in the preview screen.

As shown in FIG. 7A, when a specific device is touched and held on the preview screen of the mobile terminal, information that can be mixed in the corresponding device is displayed. The information is a list (list) representing titles that can be played or displayed on each device, and differs depending on the type of device. For example, as shown in FIG. 7A, touching and holding the audio device 102 displays a list of playable music, touching and holding the TV 101 displays a list of playable images, and digitally. Touching and holding the frame 103 displays a list of playable pictures.

In addition, as illustrated in FIG. 7B, the controller 180 of the mobile terminal may display detailed information rather than a simple list. For example, playable music may be graphically displayed or actual pictures may be displayed.

Accordingly, the user may view information playable on a specific device on the preview screen and select which contents (music, video, photo, and text) are mixed.

8B illustrates an example in which mixing is performed by selecting a specific title from the mixable list.

The user selects a specific music (follow me) from the list of playable music and then decides the music to be mixed, for example, "music video" from the list displayed on the digital frame 103, and selects the selected music as the music. Drag it to the video list. As a result, the music video mixed with the selected music is played through the screen of the mobile terminal.

Meanwhile, the present invention may control the operation of each device as well as the content mixing between the devices displayed on the preview screen. The user may control the volume, channel, and playback order of the content currently being played by touching a portion of each device on the preview screen and dragging it in a predetermined direction.

9 illustrates an example of controlling an operation of a device in a preview screen of a DLNA connected device.

As shown in FIG. 9, when the user touches the audio device 102 and drags upwards (upward), the controller 180 increases the volume (volume) of the music currently being played by the audio device 102. Drag downwards to decrease the volume.

When the TV 101 is touched and dragged to the left, the video of the previous channel is displayed. When touched and dragged to the right, the video of the next channel is displayed. Also, after touching the left side of the video device 104 and touching and dragging to the left, the previous video is played.If touching the right side and then dragging to the left, the video of the previous track is displayed. Display.

The control operation of the device in FIG. 9 is a limited control relating to the volume and reproduction order, and does not mean a detailed setting (sound quality, image quality, screen size and the like).

10 illustrates an example of performing detailed setting of a device in a preview screen of a DLNA connected device.

As shown in FIG. 10, when a long device is touched, the controller 180 displays the detailed setting controller 70 that can control the setting of the corresponding device. For example, when the user long touches the audio device 102, the controller 180 displays the detailed setting controller 70 including the equalizer and the balance at the bottom of the preview screen. Accordingly, the user may adjust the sound quality and balance of the audio device 102 by using the detailed setting controller 70.

11A and 11B illustrate an example of controlling a computer through a mobile terminal during DLNA connection.

When the computer and the mobile terminal are DLNA connected, the computer transmits S / W identification information to the mobile terminal, and the controller 180 of the mobile terminal checks whether there is information for controlling the corresponding S / W. If there is no information to control the S / W, the S / W identification information is transmitted to the network. The network receiving the identification information transmits a plug-in to the mobile terminal so as to control the corresponding software.

Accordingly, the controller 180 of the mobile terminal can perform music production by interworking with the computer screen wirelessly as shown in FIG. 11B by executing the transmitted plug-in.

As described above, when a plurality of devices are connected through the DLNA, a preview screen is displayed on the screen of the mobile terminal operating as the DLNA controller, and the contents played on each device as well as the operation of each device through the touch operation on the preview screen. You can mix and display them on the screen or save them.

12 is a flowchart illustrating a content mixing method of a mobile terminal according to an embodiment of the present invention.

First, at least one device is connected to the DLNA, and the preview screen of the DLNA connected devices is displayed on the display unit 151 of the DLNA controller, that is, the mobile terminal (S10 and S11). The controller 180 of the mobile terminal detects the mixing gesture on the preview screen and determines the type of the detected mixing gesture (S12, S13).

If it is determined that the corresponding mixing gesture is touch & drag (S14), it is determined whether the corresponding touch & drag is made between devices (touch and drag between devices) or within the device (S15). The touch and drag between devices corresponds to FIGS. 5, 6A and 6B, and the touch and drag within a device correspond to FIG. 9.

As a result of the determination, when the mixing gesture is touch and drag between devices, the controller 180 performs mixing on the contents of two devices selected by the user (S16) and displays them on the display unit 151 or the memory 160 according to a user's selection. Store in (S17). On the other hand, if the mixing gesture is touch and drag in the device, the operation (volume, channel, playback order, etc.) for the corresponding device is controlled (S18).

If the mixing gesture is touch and hold instead of touch and drag, the controller 180 displays the playable list of the corresponding device as shown in FIGS. 7A and 7B, and then performs steps S16 and S17. If the mixing gesture is a long touch, as shown in FIG. 10, the detailed setting controller 70 or the menu of the device is displayed (S19), and the user's settings set on the corresponding detailed setting controller 70 are stored.

As described above, the present invention displays a preview screen of a plurality of devices connected through DLNA on a mobile terminal, controls the operation of each device through a mixing gesture on the preview screen, and displays or plays back the contents by mixing the contents of each device. This allows the user to conveniently perform device control and content mixing.

Further, according to an embodiment of the present invention, the above-described method can be implemented as a code that can be read by a processor on a medium on which the program is recorded. Examples of the medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and may be implemented in the form of a carrier wave (e.g., transmission over the Internet) .

The above-described mobile terminal is not limited to the configuration and method of the above-described embodiments, but the embodiments are configured by selectively combining all or some of the embodiments so that various modifications can be made. May be

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible.

110: wireless communication unit 111: broadcast receiving module
112: mobile communication module 113; Wireless Internet Module
140; Sensing unit 150: output unit
151: display unit 160: memory
180:

Claims (20)

Connecting at least one or more external devices to the DLNA;
Displaying an external device connected to the DLNA and an operation state thereof on a preview screen of a mobile terminal;
Mixing content being played on different external devices according to the mixing gesture detected on the preview screen; And
Outputting the mixed content through a mobile terminal;
The content mixing control method of the mobile terminal, characterized in that the available device is automatically activated and displayed according to the position of the user. delete delete The device of claim 1, wherein the device displayed on the preview screen is
The content mixing control method of the mobile terminal can be activated or deactivated according to the user's selection, the size of the deactivated device is changed to a translucent state. The device of claim 1, wherein the device is
A mobile terminal, an audio device, a video device, and a digital photo frame, a PC, and a printer, wherein the content includes image, music, photo, and text. The method of claim 1, wherein the mixing gesture is
Content mixing control method of the mobile terminal, characterized in that the touch and drag. The method of claim 1, wherein the output of the mixed content includes visual and audio output. The method of claim 1, wherein the mixing of the content comprises:
Determining a type of the mixing gesture;
Determining the type of the touch & drag when the mixing gesture is touch &drag;
Mixing the content being played on the first and second devices when the touch and drag is touch and drag between devices; And
And controlling the operation of a corresponding device if the touch and drag is touch and drag within a device. The method of claim 8, wherein mixing the content comprises:
Displaying a list of contents playable on a corresponding device when the mixing gesture is touch &hold; And
And mixing and playing the contents corresponding to the two lists according to the touch and drag between the displayed lists. The method of claim 8, wherein mixing the content comprises:
And displaying a detailed setting menu or a controller for performing detailed settings for a corresponding device when the mixing gesture is a long touch. A display unit configured to display an external device connected by DLNA and an operation state thereof on a preview screen; And
And a controller configured to output the mixed contents by mixing the contents being played on different external devices according to the mixing gesture detected in the preview screen.
And the controller activates and displays a device available on the preview screen according to the position of the user. delete delete 12. The apparatus of claim 11, wherein the control unit
The mobile terminal activates or deactivates a specific device displayed on the preview screen according to a user's selection, and the deactivated device is resized to a translucent state. The device of claim 11, wherein the device is
A mobile terminal, an audio device, a video device, and a digital picture frame, a PC, and a printer, wherein the content includes images, music, photos, and text. The method of claim 11, wherein the mixing gesture is
Mobile terminal, characterized in that the touch and drag. 12. The mobile terminal of claim 11, further comprising a memory for storing the mixed content. 12. The apparatus of claim 11, wherein the control unit
Determine the type of the mixing gesture to determine the type of the touch and drag when the mixing gesture is touch & drag, and mixing the content being played on the first and second devices if the touch and drag is touch and drag between devices And if the touch & drag is touch and drag in a device, controlling the operation of the corresponding device. 12. The apparatus of claim 11, wherein the control unit
And displaying a list of contents that can be played on a corresponding device when the mixing gesture is touch & hold, and mixing and playing contents corresponding to two lists according to touch and drag between the displayed lists. The method of claim 18, wherein the control unit
And a detailed setting menu or a controller for performing detailed settings for the corresponding device when the mixing gesture is a long touch.

Images