WO2015006961A1

WO2015006961A1 - User interface for an electronic device and associated methods

Info

Publication number: WO2015006961A1
Application number: PCT/CN2013/079622
Authority: WO
Inventors: Yingfei Liu; Kongqiao Wang
Original assignee: Nokia China Investment Co Ltd; Nokia Inc
Current assignee: Nokia China Investment Co Ltd; Nokia Inc
Priority date: 2013-07-18
Filing date: 2013-07-18
Publication date: 2015-01-22
Anticipated expiration: 2016-01-18
Also published as: CN105579948B; CN105579948A

Abstract

An apparatus comprising: a processor; and a memory including computer program code, the memory and the computer program code configured, with the processor, to cause the apparatus to perform at least the following: receive control element data associated with user interface control elements of a user interface display screen of a controlled electronic device; determine a multi-level hierarchical menu structure of the controlled electronic device user interface control elements using the received control element data; and generate an adapted menu, for use by a controller electronic device to control the controlled electronic device, the adapted menu comprising a subset of levels of the multilevel hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the controller electronic device.

Description

USER INTERFACE FOR AN ELECTRONIC DEVICE AND ASSOCIATED

METHODS

Technical Field

The present disclosure relates to the field of user interfaces, associated methods, computer programs and apparatus. Certain disclosed aspects/examples relate to portable electronic devices, in particular, so-called hand-portable electronic devices which may be hand-held in use (although they may be placed in a cradle in use). Such hand-portable electronic devices include so-called

Personal Digital Assistants (PDAs), 10 mobile telephones, smartphones and other smart devices, and tablet PCs.

The portable electronic devices/apparatus according to one or more disclosed aspects/example embodiments may provide one or more audio/text/video communication functions (e.g. tele-communication, video-communication, and/or text

transmission (Short Message Service (SMS)/Multimedia Message Service (MMS)/e-mailing) functions), interactive/non-interactive viewing functions (e.g. web-browsing, navigation, TV/program viewing functions), music recording/playing functions (e.g. MP3 or other format and/or (FM/AM) radio broadcast recording/playing), downloading/sending of data functions, image capture functions (e.g. using a (e.g. in-built) digital camera), and

gaming functions. Background

Generally electronic devices are configured enable a user to control them using a user 25 interface. Some electronic devices can be controlled remotely. For example, a television may be controlled by a remote controller (e.g. the remote controller being physically separated from the controlled electronic device and configured to transmit control signalling to the controlled electronic device via wireless or wired channels).

The listing or discussion of a prior-published document or any background in this specification should not necessarily be taken as an acknowledgement that the document or background is part of the state of the art or is common general knowledge. One or more aspects/examples of the present disclosure may or may not address one or more of the background issues.

Summary

In a first example aspect there is provided an apparatus comprising: a

processor; and a memory including computer program code, the memory and the computer program code configured, with the processor, to

cause the apparatus to perform at least the following:

receive control element data associated with user interface control elements of a user interface display screen of a controlled electronic device;

determine a multi-level hierarchical menu structure of the controlled electronic 10 device user interface control elements using the received control element data; and

generate an adapted menu, for use by a controller electronic device to control the controlled electronic device, the adapted menu comprising a subset of levels of the multilevel hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the controller electronic device.

It will be appreciated that the subset of levels may be considered to comprise some but not all of the hierarchical levels.

The control element data may be received from one of the controlled electronic device 20 and remote server. For example, the server may be remote with respect to the controller device and the controlled device. The control element data may be received via one or more of wireless and wired communication channels. The apparatus may be configured to receive the control element data from another 25 device and to transmit the received control element data to the controlled electronic device. For example, the controlled device may receive the control element data from the another device via the apparatus (which may be of form part of the controller device).

The display of the controller electronic device may be smaller than the display of the controlled electronic device.

A display screen may be a home screen or an application screen configured to be displayed on a display.

The home screen may be considered to be a first screen of user actuate-able control information displayed when the electronic device is started. A home screen may comprise one or more icon user interface control elements corresponding to applications (email application, telephone application), files, folders, and/or functions (e.g. settings menus). That is, a home screen may be dedicated to providing the user with access to an array of user interface control elements corresponding to applications, files, folders and/or functions. A home screen may be configured to allow the user to open 5 applications. A home screen may be considered to be a basic/primary screen displayed in the entire foreground when no user applications are running (e.g. when the electronic device is in a fully active mode), or when any running applications are minimized. That is, a home screen may be visible (e.g. partially or fully visible) when an application or window is not filling up the entire screen. A home screen may be an application 10 launching screen.

An application screen may be a screen/window dedicated to provide a user interface with a particular application (e.g. an email application, a word processing application, a web browser). A user interface control element may be associated with a corresponding function performable on one or more of the controller and controlled electronic device. A user interface control element may be one of an icon, and a menu item.

The controller electronic device user interface control elements may be adapted for the display of the controller electronic device by being one or more of being smaller than or being an abbreviated version of the corresponding controlled electronic device user interface control elements. An abbreviated version may comprise less information than the original unabbreviated version (for example, an unabbreviated icon may comprise the

text 'next page ' and the corresponding abbreviated icon may comprise the abbreviated text 'next '.

The apparatus may be configured to transmit adapted menu signalling to the controlled device to provide for actuation of corresponding controlled electronic device user 30 interface control elements.

It will be appreciated that the controlled electronic device may or may not be configured to allow the user to interact directly (e.g. via a touch screen or peripheral device such as a mouse or pointer) with the user interface control elements of the user interface display 35 screen of a controlled electronic device (e.g. in addition to allowing the user to interact with the user interface control elements of a user interface display screen of a controlled electronic device via the controller electronic device).

The multi-layer hierarchical structure may comprise multiple branches. Each adapted menu may comprise a subset of levels of the multi-level hierarchical structure with a particular branch.

The apparatus may be configured to enable display of the controller electronic device user interface control elements in an arrangement corresponding to the arrangement of displayed controlled electronic device user interface control elements. The corresponding arrangement may be considered to be a corresponding spatial 10 arrangement. The apparatus may be configured to update the display of the controller electronic device in response to the controlled electronic device user interface control elements being updated. This would be done, for example, by the reception of appropriate updated

control element data. In other example embodiments, after the determination of the multi-level hierarchical menu structure using the received control element data, the apparatus may be configured to autonomously update the display of the controller electronic device without the reception of further control element data. In such cases a sufficient amount of the menu structure would have had to be received for the

autonomous update of the controller electronic device adapted menu (e.g. the control element data may have had enough information for the generation of multiple adapted menus).

In response to a controlled electronic device user interface control element being 25 selected, the apparatus may be configured to: generate a further adapted menu comprising controller electronic device user interface control elements corresponding to a further subset of levels, wherein the further subset of levels are lower in the hierarchy than the level of the selected user interface control element.

The controlled device may comprise a television, a desktop computer, a tablet computer, a home cinema system, a projector, or a laptop computer.

The controller device may comprise a portable electronic device, a laptop computer, a mobile phone, a Smartphone, a tablet computer, a personal digital assistant, a digital 35 camera, a watch, a server, a non-portable electronic device, a desktop computer, a monitor, a server, a wand, a pointing stick, a touchpad, a touch-screen, a mouse, a joystick or a module/circuitry for one or more of the same.

The apparatus may comprise the controller device, the controlled device or a server, the server being remote to the controlled and controller devices.

In a further aspect there is provided a method, the method comprising:

receiving control element data corresponding to user interface control elements of a user interface display screen of a controlled electronic device;

determining a multi-level hierarchical menu structure of the controlled electronic

device user interface control elements using the received control element data; and generating an adapted menu, for use by a controller electronic device to control

the controlled electronic device, the adapted menu comprising a subset of levels of the multi-level hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the controller electronic device. In a further aspect there is provided an apparatus, the apparatus comprising:

means for receiving configured to receive control element data corresponding to user interface control elements of a user interface display screen of a controlled electronic device; means for determining configured to determine a multi-level hierarchical menu 20 structure of the controlled electronic device user interface control elements using the received control element data; and

means for generating configured to generate an adapted menu, for use by a controller electronic device to control the controlled electronic device, the adapted menu comprising a subset of levels of the multi-level hierarchical menu structure with 25 corresponding controller electronic device user interface control elements adapted for the display of the controller electronic device.

In a further aspect there is provided a computer program comprising computer program code, the computer program code being configured to perform at least the following:

receive control element data corresponding to user interface control elements of a user interface display screen of a controlled electronic device;

determine a multi-level hierarchical menu structure of the controlled electronic device user interface control elements using the received control element data; and

In a further aspect there is provided a system comprising a controller electronic device and a controlled electronic device wherein the controller electronic device, the controlled electronic device or an apparatus in communication with the controller and controlled 5 electronic devices is configured to:

determine a multi-level hierarchical menu structure of the controlled electronic

device user interface control elements using the received control element data; and

generate an adapted menu, for use by the controller electronic device to control the controlled electronic device, the adapted menu comprising a subset of levels of the multi-level hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the controller electronic device.

The present disclosure includes one or more corresponding aspects, examples or features in isolation or in various combinations whether or not specifically stated (including claimed) in that combination or in isolation. Corresponding means and corresponding functional units (e.g. comparator, provider) for performing one or more of the discussed functions are also within the present disclosure.

Corresponding computer programs for implementing one or more of the methods disclosed are also within the present disclosure and encompassed by one or more of the described examples.

The above summary is intended to be merely exemplary and non-limiting. Brief Description of the Figures

A description is now given, by way of example only, with reference to the accompanying 30 drawings, in which:

figure 1 illustrates an example apparatus embodiment comprising a number of electronic components, including memory and a processor, according to one example embodiment of the present disclosure;

figure 2 illustrates an example apparatus embodiment comprising a number of electronic 35 components, including memory, a processor and a communication unit, according to another example embodiment of the present disclosure;

figure 3 illustrates an example apparatus embodiment comprising a number of electronic components, including memory and a processor, according to another example embodiment of the present disclosure;

figures 4a-4f illustrate an example embodiment configured to allow a mobile phone is to 5 control a television;

figures 5a-5e illustrate an example embodiment configured to allow a PDA is to control a desktop computer;

figures 6a-6b illustrate an example apparatus according to the present disclosure in

communication with a remote server/cloud; figure 7 illustrates a flowchart according to a method of the present disclosure; and figure 8 illustrates schematically a computer readable medium providing a program.

Description of Example Aspects It is common for certain electronic devices to be controlled by a remote controller. For example, in a conventional TV remote controller, to select a channel, the user looks at the TV screen (the controlled electronic device) to see which channel is currently being displayed. Then he looks at the remote control to find the button corresponding to the desired channel. After the desired button is pressed he may look back to the screen to see if he selected the correct button. This requires the user to alternate their focus between the controller device and the controlled device.

Examples included in the present disclosure relate to providing an adapted menu for use in a controller electronic device to control a controlled electronic device, the adapted

menu being generated based on received control element data associated with user interface control elements of a user interface display screen of a controlled electronic device. By providing for controller electronic device user interface control elements corresponding to those displayed on the controlled display, an intuitive corresponding user interface may be provided to the user by the controller device thereby mitigating the

need to look back and forth between the differently structured user interfaces of the controller and controlled devices.

Furthermore, because the adapted menu is generated based on received control element data associated with user interface control elements of a user interface display 35 screen of a controlled electronic device, the user interface of the controller electronic device may be defined dynamically (i.e. on-the-fly). This may allow user interface adapted menus to be generated for multiple different applications (e.g. including newly installed applications) as and when they are required and/or for the controller electronic device to be used for control of different controlled electronic devices.

Other example embodiments depicted in the figures have been provided with reference 5 numerals that correspond to similar features of earlier described example embodiments. For example, feature number 1 can also correspond to numbers 101 , 201 , 301 etc. These numbered features may appear in the figures but may not have been directly referred to within the description of these particular example embodiments. These have still been provided in the figures to aid understanding of the further example 10 embodiments, particularly in relation to the features of similar earlier described example embodiments.

Figure 1 shows an apparatus 101 comprising memory 107, a processor 108, input I and output O. In this example embodiment only one processor and one memory are shown 15 but it will be appreciated that other example embodiments may utilise more than one processor and/or more than one memory (e.g. same or different processor/memory types).

In this example embodiment the apparatus 101 is an Application Specific Integrated 20 Circuit (ASIC) for a portable electronic device with a touch sensitive display. In other example embodiments the apparatus 101 can be a module for such a device, or may be the device itself, wherein the processor 108 is a general purpose CPU of the device and the memory 107 is general purpose memory comprised by the device.

The input I allows for receipt of signalling to the apparatus 101 from further components, such as components of a portable electronic device (like a touch-sensitive or hover-sensitive display) or the like. The output O allows for onward provision of signalling from within the apparatus 101 to further components such as a display screen, speaker, or other output device. In this example embodiment the input I and output O are part of a

connection bus that allows for connection of the apparatus 101 to further components.

The processor 108 is a general purpose processor dedicated to executing/processing information received via the input I in accordance with instructions stored in the form of computer program code on the memory 107. The output signalling generated by such 35 operations from the processor 108 is provided onwards to further components via the output O.

The memory 107 (not necessarily a single memory unit) is a computer readable medium (solid state memory in this example, but may be other types of memory such as a hard drive, ROM, RAM, Flash or the like) that stores computer program code. This computer program code stores instructions that are executable by the processor 108, when the 5 program code is run on the processor 108. The internal connections between the memory 107 and the processor 108 can be understood to provide, in one or more example embodiments, an active coupling between the processor 108 and the memory 107 to allow the processor 108 to access the computer program code stored on the memory 107.

In this example the input I, output O, processor 108 and memory 107 are all electrically connected to one another internally to allow for electrical communication between the respective components

I, O, 107, 108. In this example the components are all located proximate to one another so as to be formed together as an ASIC, in other words, so as 15 to be integrated together as a single chip/circuit that can be installed into an electronic device. In other examples one or more or all of the components may be located separately from one another.

Figure 2 depicts an apparatus 201 of a further example embodiment, such as a mobile 20 phone. In other example embodiments, the apparatus 201 may comprise a module for a mobile phone (or PDA or audio/video player), and may just comprise a suitably configured memory 207 and processor 208.

The example embodiment of figure 2 comprises a display device 204 such as, for 25 example, a liquid crystal display (LCD), e-lnk or touch-screen user interface. The apparatus 201 of figure 2 is configured such that it may receive, include, and/or otherwise access data. For example, this example embodiment 201 comprises a communications unit 203, such as a receiver, transmitter, and/or transceiver, in communication with an antenna 202 for connecting to a wireless network and/or a port 30 (not shown) for accepting a physical connection to a network, such that data may be received via one or more types of networks. This example embodiment comprises a memory 207 that stores data, possibly after being received via antenna 202 or port or after being generated at the user interface 205. The processor 208 may receive data from the user interface 205, from the memory 207, or from the communication unit 203.

It will be appreciated that, in certain example embodiments, the display device 204 may incorporate the user interface 205. Regardless of the origin of the data, these data may be outputted to a user of apparatus 201 via the display device 204, and/or any other

output devices provided with apparatus. The processor 208 may also store the data for later use in the memory 207. The memory 207 may store computer program code and/or applications which may be used to instruct/enable the processor 208 to perform functions (e.g. read, write, delete, edit or process data).

Figure 3 depicts a further example embodiment of an electronic device 301 , such as a tablet personal computer, a portable electronic device, a portable telecommunications device, a server or a module for such a device, the device comprising the apparatus 101 of figure 1. The apparatus 101 can be provided as a module for device 301 , or even as a

processor/memory for the device 301 or a processor/memory for a module for such a device 301 .

The device 301 comprises a processor 308 and a storage medium 307, which are connected (e.g. electrically and/or wirelessly) by a data bus 380. This data bus 380 can provide an active coupling between the processor 308 and the storage medium 307 to allow the processor 308 to access the computer program code. It will be

appreciated that the components (e.g. memory, processor) of the device/apparatus may be linked via cloud computing architecture. For example, the storage device may be a remote server accessed via the internet by the processor.

The apparatus 101 in figure 3 is connected (e.g. electrically and/or wirelessly) to an 20 input/output interface 370 that receives the output from the apparatus 101 and transmits this to the device 301 via data bus 380. Interface 370 can be connected via the data bus 380 to a display 304 (touch-sensitive or otherwise) that provides information from the apparatus 101 to a user. Display 304 can be part of the device 301 or can be separate. The device 301 also comprises a processor 308 configured for general control of the 25 apparatus 101 as well as the device 301 by providing signalling to, and receiving signalling from, other device components to manage their operation. The storage medium 307 is configured to store computer code configured to perform, control or enable the operation of the apparatus 101. The storage medium 307 may be

configured to store settings for the other device components. The processor 308 may access the storage medium 307 to retrieve the component settings in order to manage the operation of the other device components. The storage medium 307 may be a temporary storage medium such as a volatile random access memory. The storage medium 307 may also be a permanent storage medium such as a hard disk drive, a flash

memory, a remote server (such as cloud storage) or a non-volatile random access memory. The storage medium 307 could be composed of different combinations of the same or different memory types.

Figure 4a shows a person interacting with his mobile phone 401 (controller electronic device) to control internet content displayed on his television 412 (controlled electronic device). In this case, the user wishes to navigate to a news story about the London 5 Marathon. In this case, the controller (mobile phone) electronic device 401 is configured to receive from another device (e.g. a wireless hub) control element data from the internet. The control element data, in this example, comprises a markup language version (e.g. HTML)

of a display screen having information relating to a number of user interface control elements for display on the display screen of the controlled electronic device. In this case, the controlled electronic device 412 user interface control elements comprise a number of hyperlinks which, when selected by the user, allow various pages and screens of the user interface to be navigated. In this case, the controller electronic device 401 is

configured to transmit a copy of the received control data to the controlled electronic device to enable the controlled electronic device user interface to be displayed by the controlled electronic device.

In this case, the controlled electronic device uses the received control element data to 20 display a display screen having two hierarchical levels of user interface control elements. The higher level comprises user interface elements relating to categories of information. In this case, the information categories include: news stories 422; social networking events 421 ; meetings 423; and stock market values 424. Also displayed on the display screen of the controlled device 412 are specific hyperlinks (corresponding to a lower 25 level of the hierarchy) relating to particular pages of information. For example, in the stock market value branch 424 of the hierarchy, there are a number of lower level hyperlink user interface control elements corresponding to particular stock market indices (e.g. such as the Dow Jones 424c, the FTSE 100 424b or the Hang Seng index 424a). Because the user interface control elements are of a multi-layer hierarchy, if a higher level user interface control element is selected directly on the controlled electronic device, the resulting screen includes any available lower level user interface control elements within that branch which were visible in the original screen. For example, if the user were to select the 'social' higher-level controlled electronic device user interface

control element 421 , the resulting screen would include the lower-level user interface control elements corresponding to the 'Tom likes your post' 421 a and 'Dave: fancy a coffee' 421 b messages. It will be appreciated that, if the user wished to open either of these messages directly using the controlled electronic device, they could directly select the appropriate lower-level controlled electronic device user interface control element from the original controlled electronic device display screen (as shown in figure 4b).

The controller electronic device is configured to determine a multi-level hierarchical menu structure of the controlled electronic device user interface using the received control element data. That is, the controller electronic device is configured to determine that, in this case, the hierarchical relationship of the controlled electronic device 412 user interface control elements (421-424, 421 a-b, 422a-b, 423a-b, and 424a-c) comprise two

levels: a first level corresponding to categories of information 421-424; and a second lower level corresponding to particular pages 421 a-b, 422a-b, 423a-b, and 424a-c. In this case, the multi-level hierarchical menu is determined using the markup tags present in the received control element data (e.g. with hyperlinks marked as titles being assigned to a higher level than hyperlinks included in the body text). It will be appreciated that

other hierarchies may comprise more than one level. In this example, the controller device 401 is configured to generate, based on the determined multi-level hierarchical menu structure, a first adapted menu comprising a subset of levels of the multi-level hierarchical menu structure with corresponding 20 controller electronic device user interface control elements adapted for the display of the controller electronic device. This is shown in figure 4c where the controller (mobile phone) electronic device 401 is shown initially displaying the higher level of the hierarchical multi-layer structure 421 '- 424'. That is, initially, the controller device 401 is configured to display hyperlinks corresponding to the 'news' 422', 'social' 421 ', 'meetings' 423' and 'stock markets' 424' categories of information, but not the lower level user interface control elements associated with specific pages of information. In this case the controller device is configured to display the controller electronic device user interface control elements 42 I '

l l 424' in a spatial arrangement corresponding to the arrangements of displayed controlled electronic device user interface control elements 421-424. For example, as the 'news' 422 category hyperlink control elements is shown in the top right of the controlled (television) electronic device, the corresponding 'news' category hyperlink controller electronic device user interface control element 422' is shown in the top right of the

controller (mobile phone) electronic device.

In this case, the user wishes to read the news story of relating to the Herschel Space Observatory so he selects the 'news' category hyperlink user interface control element 422' on the controller device. By selecting the news category user interface control element 422' the controller electronic device is configured to display the user interface 5 elements in the hierarchical level below the selected user interface element. That is, the device/apparatus is configured to generate and display a second adapted menu comprising a subset of levels of the multi-level hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the controller electronic device. In this case, the user interface control 10 elements 422a'-422b' correspond to the news stories listed on the display of the controlled electronic device. This is shown in figure 4d.

In this case, the user goes on to select the desired news story 422a' from the lower-level controller electronic device user interface control elements 422a'-b'. Because a subset

of the hierarchy levels is visible at one time, the area which can be devoted to each user interface control element is greater, and so the user interface may be more intuitive and easier to use. In addition, because the user interface control elements of the controller device correspond to that of the controlled device user interface, the user can navigate to the desired page without having to look at the controlled electronic device user interface

display screen.

When the user has selected the desired news story by selecting the corresponding controller electronic device user interface control element 422a', the screen of the controller electronic device is configured to display the desired page 431 '. In addition, 25 the controller electronic device is configured to transmit adapted menu signalling to the controller device to provide for actuation of corresponding user interface control elements. That is, in this case, the adapted menu signalling, in this case, provides for actuation of the corresponding news story hyperlink.

In response to the controlled electronic device news story user interface control element being actuated, the controlled electronic device is configured to display the selected news story 431.

Then the user can use the controlled electronic device to read the desired news story (this is shown in figure 4f). It will be appreciated that, in other example embodiments, the controller user interface may not be configured to display content other than user interface control elements. For example, in other example embodiments, when the user selects a user interface control element corresponding to the function of opening a page, the page content may only be displayed on the controlled electronic device and not on the controller electronic device.

If will be appreciated that other example hierarchies may comprise more than two levels. 5 For example embodiment, a hierarchy may comprise ten levels. In such an example embodiment, the generated adapted menu may comprise between one and nine of the ten levels. Not necessarily all elements in the sub-levels or all sub-levels will be available for selection at the same time using the controlled electronic device. In certain cases, scrolling may be required through the various elements of a sub-level or between 10 sub-levels particularly if the display of the controller device is small. It will be appreciated that in other example embodiments, the receiving of control element data, the determining of a multi-level hierarchy, and the generating of one or more adapted menus may be carried out by the controlled device or a device remote 15 from the controller and controlled device (e.g. a remote server). Figure 5a shows the screen of a controller electronic device 501 such as a personal digital assistant (PDA). Figure 5b shows the display of a corresponding controlled electronic device 512 such as a desktop computer.

In this example, the user is using the controller electronic device 501 to compose a text message on the controlled electronic device 501. In the situation shown in figures 5a and 5a he has already entered the text string 'Mikes birthday is tomorrow!!' In this example, he wishes to emphasise the word 'tomorrow' by underlining it. He has already 25 selected the desired text that he wishes to underline. The selected text is distinguished from the unselected text by having a dark background.

In this case, the controlled (desktop computer) electronic device 512 is configured to generate control element data using, in this case, a word processing application. The

control data, in this case, comprises information relating to a number of user interface control elements for display on the display screen of the controlled electronic device. In this case, the controlled electronic device user interface control elements comprise a number of icons 521 a-e,

522a-e which, when selected by the user, allow various functions to be performed within the word processing application. Unlike the previous case, where the control element data was transferred from the controller electronic device to the controlled electronic device, in this case, the controlled electronic device 512 is configured to transmit a copy of the generated control element data to the controller electronic device 501 to enable the controller electronic device user interface to be displayed on the display of the controller electronic device.

In this case, the controlled electronic device 512 uses the generated control element 5 data to display a display screen having a number of user interface control elements 521 a-e, 522a-e. In this case, the controlled electronic device 512 user interface element control icons correspond to the functions of: emboldening text 521 a; italicising text 521 c; underlining text 521 c; making text subscript 521d; making text superscript 521 e; drawing a line 522a; drawing an arrow 522b; drawing a square 522c; drawing a circle 522d; and 10 drawing a curved line 522e. In this case, the functions relating to text 521 a-e are shown in a text toolbar 521 ; and the functions relating to drawing 522a-e are shown in a drawing tool bar 522. In this case, the tool bars 521 , 522 are not independently selectable using the controlled electronic device user interface.

The controller electronic device 501 is configured to determine a multi-level hierarchical menu structure of the controlled electronic device user interface using the received control element data. In this case, the controller electronic device recognises that each of the displayed controlled electronic device user interface control elements relate to one of two tool bars. Therefore, the controller electronic device determines a two-level

hierarchical menu structure comprising: a first level relating to the tool bars; and a second level relating to the specific functions. It will be appreciated that, in other example embodiments, determining the multi-level hierarchy may be performed by the controlled electronic device or by a device remote from the controlled and the controller electronic device (e.g. a remote server).

In this example, the controller electronic device 501 is configured to generate, based on the determined multi-level hierarchical menu structure, an adapted menu comprising a subset of levels of the multi-level hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the 30 controller electronic device.

This is shown in figure 5a where the controller (PDA) electronic device is shown initially displaying an adapted menu corresponding to the higher level of the hierarchical multilayer structure. That is, initially, the controller device is configured to display user 35 interface control elements corresponding to the text tool bar 521 ' and to the drawing tool bar 522' respectively, but not the lower level user interface control elements associated with specific functions. In this case the controller device is configured to display the controller electronic device user interface control elements 521 ' 522' in an arrangement corresponding to the arrangements of displayed controlled electronic device user interface control elements. For example, as the 'drawing' toolbar 521 is shown in the top right of the controlled (television) electronic device, the corresponding 'drawing' toolbar 5 controller electronic device user interface control element 521 ' is shown in the top right of the controller (PDA) electronic device.

In this case, the user wishes to underline the selected text. By selecting the text toolbar user interface control element 521 ' the controller electronic device is configured to

generate and display a further adapted menu comprising the user interface elements 521 a'-b' in the hierarchical level below the selected user interface element. That is, the device/apparatus is configured to display an adapted menu comprising a subset of levels of the multi-level hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the controller electronic

device. In this case, the user interface elements correspond to the text functions on the display of the controlled electronic device (i.e. bold 521 a', italics 521 b', underline 521 c', subscript 521d' and superscript 521 e'). This is shown in figure 5c.

In this case, the user goes on to select the desired underline text function user interface 20 control element 521 c' from the lower-level controller electronic device user interface control elements. Because, only a subset of the hierarchy levels is visible at one time, the area which can be devoted to each user interface control element is greater, resulting in a user interface which is easier to use. In addition, because the user interface control elements of the controller device correspond to that of the controlled device user 25 interface, the user can navigate to the desired function without having to look at the controlled electronic device user interface display screen.

When the user has selected the desired underline function by selecting the corresponding controller electronic device user interface control element 521c', the

screen of the controller electronic device 501 is configured to display the desired change. In addition, the controller electronic device 501 is configured to transmit adapted menu signalling to the controlled device 512 (e.g. via Bluetooth or radio frequency wireless signalling) to provide for actuation of corresponding controlled electronic device user interface control elements. That is, in this case, the adapted menu signalling provides for

actuation of the corresponding underline function icon user interface control element

521 c.

In response to the controlled electronic device underline function user interface control element being actuated, the controlled electronic device is configured to display the updated text. Then the user can use the controlled electronic device to view the changes (this is shown in figure 5e). Figure 6a shows an example of an apparatus 601 in communication with a remote server. Figure 6b shows an example of an apparatus 601 in communication with a "cloud" for cloud computing. In figures 6a and 6b, apparatus 601 (which may be apparatus 101 , 201 or 301 ) is also in communication with a further apparatus 604. The 10 apparatus 602 may be display screen, mobile phone, laptop or personal digital assistant, for example. In other examples, the apparatus 601 and further apparatus 602 may both be comprised within a device such as a portable communications device or PDA. Communication may be via a communications unit, for example. Figure 6a shows the remote computing element to be a remote server 695, with which the apparatus 601 may be in wired or wireless communication (e.g. via the internet, Bluetooth, NFC, a USB connection, or any other suitable connection as known to one skilled in the art). In figure 6b, the apparatus 601 is in communication with a remote cloud 696 (which may, for example, be the Internet, or a system of remote computers

configured for cloud computing). For example, the apparatus performing the receiving of control element data; the determining the multi-level hierarchical menu structure and/or the generating of the adapted menu may be located at a remote server 695 or cloud 696 and accessible by the apparatus 601. In other examples the apparatus may also be in direct communication with the remote server 695 or cloud 696.

Figure 7 shows a flow diagram illustrating the steps of receiving 781 control element data corresponding to user interface control elements of a user interface display screen of a controlled electronic device; determining 782 a multi-level hierarchical menu structure of the controlled electronic device user interface control elements using the received control 30 element data; and generating 783 an adapted menu, for use by a controller electronic device to control the controlled electronic device, the adapted menu comprising a subset of levels of the multi-level hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the controller electronic device. Figure 8 illustrates schematically a computer/processor readable medium 800 providing a program according to an example. In this example, the computer/processor readable medium is a disc such as a digital versatile disc (DVD) or a compact disc (CD). In other examples, the computer readable medium may be any medium that has been programmed in such a way as to carry out an inventive function. The computer program code may be distributed between the multiple memories of the same type, or multiple 5 memories of a different type, such as ROM, RAM, flash, hard disk, solid state, etc. It will be appreciated that other example embodiments may be a portable electronic device, a laptop computer, a mobile phone, a Smartphone, a tablet computer, a personal digital assistant, a digital camera, a watch, a non-portable electronic device, a desktop 10 computer, a monitor, a server, or a module/circuitry for one or more of the same.

Any mentioned apparatus/device/server and/or other features of particular mentioned apparatus/device/server may be provided by apparatus arranged such that they become configured to carry out the desired operations only when enabled, e.g. switched on, or the like. In such cases, they may not necessarily have the appropriate software loaded into the active memory in the non-enabled (e.g. switched off state) and only load the appropriate software in the enabled (e.g. on state). The apparatus may comprise hardware circuitry and/or firmware.

The apparatus may comprise software loaded onto memory. Such software/computer programs may be recorded on the same

memory/processor/functional units and/or on one or more memories/processors/ functional units.

In some examples, a particular mentioned apparatus/device/server may be preprogrammed with the appropriate software to carry out desired operations, and wherein

the appropriate software can be enabled for use by a user downloading a "key", for example, to unlock/enable the software and its associated functionality. Advantages associated with such examples can include a reduced requirement to download data when further functionality is required for a device, and this can be useful in examples where a device is perceived to have sufficient capacity to store such pre-programmed

software for functionality that may not be enabled by a user. Any mentioned apparatus/circuitry/elements/processor may have other functions in addition to the mentioned functions, and that these functions may be performed by the same apparatus/circuitry/elements/processor. One or more disclosed aspects may 35 encompass the electronic distribution of associated computer programs and computer programs (which may be source/transport encoded) recorded on an appropriate carrier (e.g. memory, signal).

Any "computer" described herein can comprise a collection of one or more individual processors/processing elements that may or may not be located on the same circuit board, or the same region/position of a circuit board or even the same device. In some 5 examples one or more of any mentioned processors may be distributed over a plurality of devices. The same or different processor/processing elements may perform one or more functions described herein.

The term "signalling" may refer to one or more signals transmitted as a series of 10 transmitted and/or received electrical/optical signals. The series of signals may comprise one, two, three, four or even more individual signal components or distinct signals to make up said signalling. Some or all of these individual signals may be transmitted/received by wireless or wired communication simultaneously, in sequence, and/or such that they temporally overlap one another. With reference to any discussion of any mentioned computer and/or processor and memory (e.g. including ROM, CD-ROM etc), these may comprise a computer processor, Application Specific Integrated Circuit (ASIC), field-programmable gate array (FPGA), and/or other hardware components that have been programmed in such a way to carry 20 out the inventive function. The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a

whole, in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The applicant indicates that the disclosed aspects/examples may consist of any such individual feature or combination of features. In view of the foregoing description it will be evident to a

person skilled in the art that various modifications may be made within the scope of the disclosure. While there have been shown and described and pointed out fundamental novel features as applied to examples thereof, it will be understood that various omissions and 35 substitutions and changes in the form and details of the devices and methods described may be made by those skilled in the art without departing from the scope of the disclosure. For example, it is expressly intended that all combinations of those elements and/or method steps which perform substantially the same function in substantially the same way to achieve the same results are within the scope of the disclosure. Moreover, it should be recognized that structures and/or elements and/or method steps shown and/or described in connection with any disclosed form or examples may be incorporated

in any other disclosed or described or suggested form or example as a general matter of design choice. Furthermore, in the claims means-plus-function clauses are intended to cover the structures described herein as performing the recited function and not only structural equivalents, but also equivalent structures. Thus although a nail and a screw may not be structural equivalents in that a nail employs a cylindrical surface to secure

wooden parts together, whereas a screw employs a helical surface, in the environment of fastening wooden parts, a nail and a screw may be equivalent structures.

Claims

WHAT IS CLAIMED IS:

1. An apparatus comprising:

a processor; and

a memory including computer program code,

the memory and the computer program code configured, with the processor, to cause the apparatus to perform at least the following:

determine a multi-level hierarchical menu structure of the controlled electronic device user interface control elements using the received control element data; and generate an adapted menu, for use by a controller electronic device to control the controlled electronic device, the adapted menu comprising a subset of levels of the multilevel hierarchical menu structure with corresponding controller electronic device user 15 interface control elements adapted for the display of the controller electronic device.

2. The apparatus of claim 1 , wherein control element data is received from one of the controlled electronic device and a remote server.

3. The apparatus of claim 3, wherein the apparatus is configured to receive the control element data from another device and to transmit the received control element data to the controlled electronic device.

4. The apparatus of claim 1 , wherein the display of the controller electronic device is smaller than the display of the controlled electronic device.

5. The apparatus of claim 1 , wherein the controller electronic device user interface control elements are adapted for the display of the controller electronic device by being one of smaller than or an abbreviated version of the corresponding controlled electronic

device user interface control elements.

6. The apparatus of claim 1 , wherein the apparatus is configured to transmit adapted menu signalling to the controlled device to provide for actuation of corresponding controlled electronic device user interface control elements.

7. The apparatus of claim 1 , wherein the multi-layer hierarchical structure comprises multiple branches.

8. The apparatus of claim 1 , wherein the apparatus is configured to enable display of the controller electronic device user interface control elements in an arrangement corresponding to the arrangement of displayed controlled electronic device user interface

control elements.

9. The apparatus of claim 1 , wherein the apparatus is configured to update the display of the controller electronic device in response to the controlled electronic device user interface control elements being updated.

10. The apparatus of claim 1 , wherein the controlled device comprises a television, a desktop computer, a tablet computer, a home cinema system, a projector, or a laptop computer.

11. The apparatus of claim 1 , wherein the controller device comprises a portable electronic device, a laptop computer, a mobile phone, a Smartphone, a tablet computer, a personal digital assistant, a digital camera, a watch, a server, a non-portable electronic device, a desktop computer, a monitor, a server, a wand, a pointing stick, a touchpad, a touch-screen, a mouse, a joystick or a module/circuitry for one or more of the same.

12. The apparatus of claim 1 , wherein the apparatus comprises the controller device, the controlled device or a server, the server being remote to the controlled and controller devices.

13. A method, the method comprising:

the controlled electronic device, the adapted menu comprising a subset of levels of the multi-level hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the controller electronic device.

14. A computer program comprising computer program code, the computer program code being configured to perform at least the following:

receive control element data corresponding to user interface control elements of a user interface display screen of a controlled electronic device; determine a multi-level hierarchical menu structure of the controlled electronic

device user interface control elements using the received control element data; and generate an adapted menu, for use by a controller electronic device to control the

controlled electronic device, the adapted menu comprising a subset of levels of the multilevel hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the controller electronic device.

15. A system comprising a controller electronic device and a controlled electronic device wherein the controller electronic device, the controlled electronic device or an apparatus in communication with the controller and controlled electronic devices is configured to:

receive control element data corresponding to user interface control elements of 15 a user interface display screen of a controlled electronic device;

generate an adapted menu, for use by the controller electronic device to control the controlled electronic device, the adapted menu comprising a subset of levels of the 20 multi-level hierarchical menu structure with corresponding controller electronic device user interface control elements adapted for the display of the controller electronic device.