WO2019244279A1 - User interface device, user interface design device, control method for user interface device, and screen design method of user interface device - Google Patents

Abstract

This user interface device (1) includes: a first platform (10) which is a master platform; and a second platform (20) which is a slave platform. A UI screen executed by the first platform (10) includes a proxy component (103) arranged at a display position of a UI screen executed by the second platform (20). The first platform (10) gives a focus on a UI component that a user is selecting, and transmits, to the second platform (20), an operation event generated by an operation of the user, while the focus is given on the proxy component (103). When the operation event is consumed in the second platform (20), the first platform (10) maintains the giving of the focus on the proxy component (103), and when the operation event is not consumed, the first platform (10) executes a process according to the operation event.

Description

User interface device, user interface design device, user interface device control method, and user interface device screen design method

The present invention relates to a user interface (UI) device, and more particularly, to a user interface device that simultaneously displays a plurality of UI screens executed by a plurality of platforms (PF).

A user interface device that displays a plurality of UI screens simultaneously is known. For example, Patent Document 1 below discloses a technique for switching a window to be brought to the front according to an operation event generated by a user operation in a window system that simultaneously displays a plurality of UI screens (windows).

JP 2014-167711 A

制 御 There is a control program called hypervisor that can build multiple virtual machines on a computer and run different platforms on each virtual machine. By using the hypervisor, for example, a complex UI screen (hereinafter, “composite UI”) including a plurality of UI screens executed by a plurality of platforms, such as Linux (registered trademark) and a real-time operating system (RTOS), is used. A user interface device that provides a “UI screen” can be realized.

However, in the conventional user interface device, information on the operation state is not shared between different platforms. Therefore, for example, when switching the UI screen to be operated by the user from the UI screen executed by the first platform to the UI screen executed by the second platform, a special operation for performing the switching is required. .

SUMMARY An advantage of some aspects of the invention is to solve the above-described problem, and a user can switch a UI screen to be operated between different platforms without a special operation for switching a platform. It is an object to provide a simple user interface device.

A user interface device according to the present invention is a user interface device for displaying a composite UI screen including a plurality of UI screens executed by a plurality of platforms, wherein the plurality of platforms include a main platform and a sub-platform; Executes a UI screen including a plurality of UI parts, the plurality of UI parts have no display element, include a proxy part arranged at a display position of the UI screen executed by the slave platform, and the main platform includes a plurality of UI parts. A focus management unit that gives focus to the UI component selected by the user among the UI components, and a UI component control unit that gives a display effect indicating that focus is given to the UI component to which focus is given. , While focus is on the proxy part, An operation event transmitting unit for transmitting an operation event generated by a user operation to a slave platform; and receiving operation event consumption information from the slave platform for receiving operation event consumption information indicating whether the operation event has been consumed by the slave platform. A sub-platform, the sub-platform includes an operation event receiving unit that receives an operation event from the main platform, an operation event consumption determination unit that determines whether the operation event is consumed by the sub-platform, An operation event consumption information transmission unit for transmitting a determination result as to whether or not the platform has been consumed to the main platform as operation event consumption information, and when the operation event is consumed by the sub platform, focus management of the main platform. Department is a proxy Maintaining the focus of the application to the component, if the operation event is not consumed by the slave platform executes processing main platform according to the operation event.

In the user interface device according to the present invention, while the focus is given to the proxy component of the main platform, the operation event generated by the user operation is transmitted to the sub platform, so that the user does not need to perform any special operation. The operation target can be switched from the UI screen of the main platform to the UI screen of the sub platform. Further, when the operation event transmitted to the slave platform is not consumed by the slave platform, the operation event is processed by the master platform, so that the operation target of the user is returned from the UI screen of the slave platform to the UI screen of the master platform. It is also possible without special operations.

The objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description and the accompanying drawings.

FIG. 2 is a diagram illustrating a configuration of a user interface device according to the first embodiment. FIG. 5 is a diagram illustrating an example of a composite UI screen according to the first embodiment. FIG. 3 is a diagram illustrating a configuration of a composite UI screen illustrated in FIG. 2. FIG. 3 is a diagram illustrating a configuration of a composite UI screen illustrated in FIG. 2. 5 is a flowchart showing an operation of the first platform in the user interface device according to the first embodiment. 5 is a flowchart showing an operation of a second platform in the user interface device according to the first embodiment. FIG. 5 is a diagram illustrating an example of an operation of the user interface device according to the first embodiment. FIG. 5 is a diagram illustrating an example of an operation of the user interface device according to the first embodiment. FIG. 5 is a diagram illustrating an example of an operation of the user interface device according to the first embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a user interface device. FIG. 9 is a diagram showing a configuration of a user interface device according to a second embodiment. FIG. 14 is a diagram showing an example of a composite UI screen according to the second embodiment. FIG. 13 is a diagram illustrating a configuration of a composite UI screen illustrated in FIG. 12. FIG. 13 is a diagram illustrating a configuration of a composite UI screen illustrated in FIG. 12. FIG. 14 is a diagram illustrating an example of an operation of the user interface device according to the second embodiment. FIG. 14 is a diagram illustrating an example of an operation of the user interface device according to the second embodiment. FIG. 14 is a diagram illustrating an example of an operation of the user interface device according to the second embodiment. FIG. 14 is a diagram illustrating an example of an operation of the user interface device according to the second embodiment. FIG. 17 is a diagram illustrating an example of a composite UI screen according to the third embodiment. FIG. 20 is a diagram illustrating a configuration of a composite UI screen illustrated in FIG. 19. FIG. 20 is a diagram illustrating a configuration of a composite UI screen illustrated in FIG. 19. FIG. 20 is a diagram illustrating a configuration of a composite UI screen illustrated in FIG. 19. FIG. 17 is a diagram illustrating an example of a composite UI screen according to the third embodiment. FIG. 24 is a diagram illustrating a configuration of a composite UI screen illustrated in FIG. 23. FIG. 24 is a diagram illustrating a configuration of a composite UI screen illustrated in FIG. 23. FIG. 24 is a diagram illustrating a configuration of a composite UI screen illustrated in FIG. 23. FIG. 17 is a diagram illustrating an example of a composite UI screen according to the third embodiment. FIG. 28 is a diagram for describing a configuration of a composite UI screen illustrated in FIG. 27. FIG. 28 is a diagram for describing a configuration of a composite UI screen illustrated in FIG. 27. FIG. 28 is a diagram for describing a configuration of a composite UI screen illustrated in FIG. 27. It is a figure showing the modification of a compound UI screen. FIG. 14 is a diagram illustrating a configuration of a user interface design device according to a fourth embodiment. FIG. 9 is a diagram illustrating an example of a UI design screen. FIG. 7 is a diagram illustrating an example of execution data of a UI screen. 9 is a flowchart illustrating a procedure for creating a UI screen. FIG. 9 is a diagram illustrating an example of a UI design screen. FIG. 7 is a diagram illustrating an example of execution data of a UI screen.

<Embodiment 1>
FIG. 1 is a diagram showing a configuration of a user interface device 1 according to the first embodiment. As shown in FIG. 1, the user interface device 1 is connected to the operation input device 2 and the display device 3.

The operation input device 2 is an input unit for a user to input an operation to the user interface device 1. The operation input device 2 includes at least a “direction key” for the user to select an operation target and a “decision key” for confirming the operation target selected by the user using the direction key. Instead of the direction key, any input means capable of inputting a direction, such as an operation lever, may be used. The display device 3 is display means for the user interface device 1 to display a UI screen, and is, for example, a liquid crystal display device.

For example, when the user interface device 1 is applied to a vehicle-mounted device such as a car navigation device or an audio display device, the operation input device 2 can be easily operated by a driver such as a center cluster (head unit of the vehicle-mounted device) or a center console of the vehicle. The display device 3 is arranged at a position where the driver can easily view it, such as a center cluster or an instrument panel (meter cluster) of the vehicle.

The user interface device 1 includes a plurality of platforms such as Linux and RTOS, and the display device 3 can display a composite UI screen including a plurality of UI screens executed by the plurality of platforms. Further, a plurality of platforms that execute a plurality of UI screens constituting the composite UI screen have a master-slave relationship with each other. In the present embodiment, as shown in FIG. 1, the user interface device 1 includes a first platform 10 and a second platform 20, and the first platform 10 is a main platform and the second platform 20 is a sub platform. It shall be specified as a platform.

FIG. 2 shows an example of a composite UI screen displayed on the display device 3 by the user interface device 1. The composite UI screen of FIG. 2 includes a first UI screen 100 executed by the first platform 10 and a second UI screen 200 executed by the second platform 20. The first UI screen 100 has two buttons 101 and 102, and the second UI screen 200 is arranged below the buttons 101 and 102 in the first UI screen 100.

The first UI screen 100 and the second UI screen 200 each include one or more UI components. In particular, the first UI screen 100 executed by the first platform 10, which is the main platform, has no display element as a UI component, and the "proxy component arranged at the display position of the UI screen executed by the slave platform" "Is included. That is, the first UI screen 100 actually includes buttons 101 and 102 as UI parts and a proxy part 103 corresponding to the second UI screen 200 as shown in FIG. Then, the second UI screen 200 (FIG. 4) executed by the second platform 20 as the sub-platform is displayed in the portion of the proxy component 103 of the first UI screen 100, so that it looks like FIG. The composite UI screen shown in FIG. Note that the second UI screen 200 is also composed of one or more UI components, but is not shown here.

Returning to FIG. 1, the first platform 10 includes a UI screen execution unit 11 for displaying the first UI screen 100 on the display device 3 and an interface unit 12 for communicating with the second platform 20. .

The UI screen execution unit 11 includes a focus management unit 111 and a UI component control unit 112. The focus management unit 111 gives a focus to a UI component selected by the user as an operation target among a plurality of UI components constituting the first UI screen 100. The UI component control unit 112 gives a display effect indicating that the focus is given to the UI component to which the focus is given. The UI component control unit 112 may provide any display effect on the UI component to which the focus is given. In the present embodiment, the UI part control unit 112 causes the UI part with focus to be drawn with a thick line. For example, FIG. 2 shows a state where the button 101 is given focus.

(4) By operating the operation input device 2, the user can change the UI component selected as the operation target, that is, can move the focus. Further, although details will be described later, the proxy component 103 can also be selected as an operation target (focus grant target).

The interface unit 12 includes an operation event transmitting unit 121 and an operation event consumption information receiving unit 122. The operation event transmission unit 121 transmits an operation event generated by the user operating the operation input device 2 to the second platform 20, which is a slave platform, while the focus is given to the proxy component. The operation event consumption information receiving unit 122 receives, from the second platform 20, "operation event consumption information" indicating whether or not the operation event has been consumed by the second platform 20.

The second platform 20 includes a UI screen execution unit 21 for displaying the second UI screen 200 on the display device 3 and an interface unit 22 for communicating with the first platform 10.

The UI screen execution unit 21 includes an operation event consumption determination unit 210. The interface unit 22 includes an operation event receiving unit 221 and an operation event consumption information transmitting unit 222.

The operation event receiving unit 221 of the interface unit 22 receives the operation event transmitted from the operation event transmitting unit 121 of the first platform 10. The UI screen execution unit 21 performs a process according to the operation event received by the operation event reception unit 221.

The operation event consumption determination unit 210 of the UI screen execution unit 21 determines whether the operation event received by the operation event reception unit 221 has been consumed by the UI screen execution unit 21. The operation event consumption information transmission unit 222 transmits the result of the determination by the operation event reception unit 221 to the first platform 10 as operation event consumption information.

The UI screen execution unit 11 of the first platform 10 refers to the operation event consumption information received by the operation event consumption information reception unit 122 and transmits the operation event transmission unit 121 while the focus is given to the proxy component. It is determined whether or not the operation event has been consumed by the second platform 20. When the operation event is consumed by the second platform 20, the focus management unit 111 maintains the focus on the proxy component. If the operation event has not been consumed by the second platform 20, the UI screen execution unit 11 executes a process corresponding to the operation event.

5 and 6 are flowcharts showing the operation of the user interface device 1 according to the first embodiment. FIG. 5 shows the operation of the first platform 10, and FIG. 6 shows the operation of the second platform 20, respectively. I have.

Hereinafter, the user interface device 1 according to the first embodiment will be described. Since the flow of FIG. 5 and the flow of FIG. 6 are executed in parallel, the description of those flows will also be performed in parallel. Steps S101 to S108 quoted below are included in the flow of FIG. 5, and steps S201 to S205 are included in the flow of FIG.

(4) When the user interface device 1 starts, the UI screen execution unit 11 of the first platform 10 causes the display device 3 to display the first UI screen 100 (Step S101). Further, the UI screen execution unit 21 of the second platform 20 displays the second UI screen 200 at the position of the proxy component 103 on the first UI screen 100 displayed on the display device 3 (Step S201). As a result, the display device 3 displays a composite UI screen including the first UI screen 100 and the second UI screen 200 as shown in FIG.

Next, the UI screen execution unit 11 confirms whether or not the user has operated the operation input device 2, that is, whether or not an operation event has occurred (step S102). While no operation event occurs (NO in step S102), the UI screen execution unit 11 repeats step S102.

When an operation event occurs (NO in step S102), the focus management unit 111 of the first platform 10 sends the focus position in the first UI screen 100, that is, to which UI part of the first UI screen 100 It is checked whether the information has been given (step S103). At this time, if the focus is given to a UI component other than the proxy component 103 (NO in step S104), the UI screen execution unit 11 performs a process according to the operation event (step S108), and proceeds to step S102. Return to

For example, as shown in FIG. 7, when an operation event occurs by pressing the down key of the operation input device 2 in a state where the focus is given to the button 101, the focus management unit 111 of the UI screen execution unit 11 changes the focus. By moving the button from the button 101 to the button 102 located therebelow, the UI component control unit 112 gives a display effect indicating that the button 102 is focused. That is, the button 102 is drawn with a thick line.

Further, for example, as shown in FIG. 8, when an operation event occurs by pressing the down key of the operation input device 2 in a state where the focus is given to the button 102, the focus management unit 111 of the UI screen execution unit 11 The focus is moved from the button 102 to the proxy component 103 located below the button 102. In FIG. 8, the proxy component 103 is indicated by a thick line for convenience, but since the proxy component 103 has no display element, even if the proxy component 103 is focused on an actual composite UI screen, the proxy component 103 is displayed. No display effect is given to 103 itself. However, as a result of the focus being given to the proxy component 103 and the operation event being transmitted to the second platform 20, a change may occur in the second UI screen 200 displayed at the position of the proxy component 103.

When the operation event occurs, if the focus is given to the proxy component 103 (YES in step S104), the operation event transmission unit 121 of the first platform 10 transmits the operation event to the second platform 20. (Step S105).

On the other hand, after displaying the second UI screen 200 on the display device 3, the second platform 20 waits for an operation event to be transmitted from the operation event transmission unit 121 of the first platform 10 (FIG. 6 Step S202).

When the operation event is transmitted from the operation event transmission unit 121 of the first platform 10 in step S105 of FIG. 7, the operation event is received by the operation event reception unit 221 of the second UI screen 200 (step S105). (YES in S202). Then, the UI screen execution unit 21 of the second platform 20 performs a process according to the operation event received by the operation event reception unit 221 (Step S203). Therefore, while the focus is given to the proxy component 103, the operation performed by the user on the operation input device 2 is input to the second platform 20. Therefore, during that time, the user's operation target is switched from the first UI screen 100 to the second UI screen 200.

As described above, the user can switch the operation target from the first UI screen 100 to the second UI screen 200 by the same operation as moving the focus on the first UI screen 100. That is, the user can switch the operation target from the first UI screen 100 to the second UI screen 200 by an easy operation.

After the UI screen execution unit 21 performs the process corresponding to the operation event in step S203, the operation event consumption determination unit 210 determines whether the operation event has been consumed by the UI screen execution unit 21 (step S204). . For example, when a user operation is an invalid operation on the second UI screen 200 such as an operation of a direction key for specifying a direction in which the focus cannot be moved, an operation event generated by the operation is transmitted to the UI screen execution unit. 21 can not be consumed. The operation event consumption information transmission unit 222 transmits the determination result by the operation event consumption determination unit 210 to the first platform 10 as operation event consumption information (step S205). Thereafter, the second platform 20 returns to the processing of Step S202.

The operation event consumption information transmitted by the operation event consumption information transmission unit 222 is received by the operation event consumption information reception unit 122 of the first platform 10 (Step S106 in FIG. 5).

When the operation event consumption information is received by the operation event consumption information receiving unit 122, the UI screen execution unit 11 of the first platform 10 refers to the operation event consumption information and consumes the operation event on the second platform 20. It is confirmed whether or not it has been performed (step S107). If the operation event has been consumed by the second platform 20 (YES in step S107), the process returns to step S102. At this time, the focus management unit 111 keeps giving the focus to the proxy component 103.

If the operation event has not been consumed by the second platform 20 (NO in step S107), the UI screen execution unit 11 executes a process corresponding to the operation event (step S108).

For example, as shown in FIG. 9, in a state where the focus is given to the proxy component 103, an operation event occurs by pressing the up key of the operation input device 2, and the operation event cannot be consumed by the second platform 20. In this case, the focus management unit 111 moves the focus from the proxy component 103 to the button 102 located above the proxy component 103, and the UI component control unit 112 draws the button 102 with a thick line.

As described above, the user can switch the operation target from the second UI screen 200 to the first UI screen 100 by the same operation as moving the focus on the first UI screen 100. Therefore, the user can switch the operation target between the first UI screen 100 and the second UI screen 200 by an easy operation.

FIG. 10 is a diagram illustrating an example of a hardware configuration of the user interface device 1. The first platform 10 and the interface unit 12 of the user interface device 1 shown in FIG. 1 include, for example, a processor 501 (Central Processing Unit (CPU), a processing device, an arithmetic device, a microprocessor, a microprocessor, A computer or a DSP (Digital Signal Processor) is realized by executing software stored in the memory 502.

The software is described as a program and stored in the memory 502, and the processor 501 reads out and executes the program stored in the memory 502 to realize the functions of the first platform 10 and the second platform 20. That is, when the user interface device 1 is executed by the processor 501, the first platform 10 gives focus to the UI component selected by the user among the plurality of UI components, and the focus is given. A process of giving a display effect indicating that focus is given to the UI component, and transmitting an operation event generated by a user operation to the second platform 20 while the focus is given to the proxy component; The second platform 20 receives the operation event from the first platform 10, determines whether the operation event has been consumed by the second platform 20, and determines whether the operation event has been consumed by the second platform 20. The first platform as the operation event consumption information 0, and the first platform 10 receives the operation event consumption information. If the operation event is consumed by the second platform 20, the focus is maintained on the proxy component. If the is not consumed by the second platform 20, a process for executing a process corresponding to the operation event and a memory 502 for storing a program to be executed as a result are provided. In other words, it can be said that this program causes a computer to execute the procedure and method of the operation of the components of the user interface device 1.

Here, the memory 502 is a non-volatile or non-volatile memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), and an EEPROM (Electrically Erasable Programmable Read Only Memory). Volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc) and its drive device, or any storage medium to be used in the future. You may.

<Embodiment 2>
In the second embodiment, a configuration of the user interface device 1 suitable for a case where the second UI screen 200 includes a button or the like to be operated (focused) as a UI component will be described.

FIG. 11 is a diagram illustrating a configuration of the user interface device 1 according to the second embodiment. The configuration of the user interface device 1 of FIG. 11 is different from the configuration of FIG. 1 in that a proxy component status information transmission unit 123 is added to the interface unit 12 of the first platform 10 and a UI screen execution unit 21 of the second platform 20 is added. In addition, a focus management unit 211 and a UI component control unit 212 are added to the configuration, and a proxy component status information receiving unit 223 is added to the interface unit 22 of the second platform 20.

The proxy component status information transmitting unit 123 of the first platform 10 transmits to the second platform 20 “proxy component status information” indicating whether or not the proxy component on the first UI screen 100 is focused. . The proxy component status information receiving unit 223 of the second platform 20 receives the proxy component status information transmitted by the proxy component status information transmitting unit 123.

The basic operations of the focus management unit 211 and the UI component control unit 212 of the second platform 20 are the same as those of the focus management unit 111 and the UI component control unit 112 of the first platform 10. That is, the focus management unit 211 gives the focus to the UI component currently selected by the user as the operation target on the second UI screen 200. The UI component control unit 212 gives a display effect indicating that focus is given to the UI component to which focus is given.

However, the UI component control unit 212 refers to the proxy component status information received by the proxy component status information receiving unit 223 and changes its operation depending on whether or not the proxy component on the first UI screen 100 is focused. . That is, while the focus is given to the proxy component of the first UI screen 100, the UI component control unit 212 normally gives focus to any UI component of the second UI screen 200. Give a display effect indicating that. However, while the focus is not given to the proxy component on the first UI screen 100, the UI component control unit 212 displays a message indicating that no UI component on the second UI screen 200 is given focus. The effect is given to the second UI screen 200.

{For example, it is assumed that the user interface device 1 causes the display device 3 to display the composite UI screen of FIG. The composite UI screen of FIG. 12 includes a first UI screen 100 executed by the first platform 10 and a second UI screen 200 executed by the second platform 20. The first UI screen 100 has two buttons 101 and 102, and the second UI screen 200 is arranged below the buttons 101 and 102 in the first UI screen 100. The second UI screen 200 has two buttons 201 and 202.

The first UI screen 100 actually includes buttons 101 and 102 as UI parts and a proxy part 103 corresponding to the second UI screen 200 as shown in FIG. Then, the second UI screen 200 (FIG. 14) executed by the second platform 20, which is the sub-platform, is displayed in the portion of the proxy component 103 in FIG. 13, so that the composite UI shown in FIG. A UI screen is configured.

For example, as illustrated in FIG. 15, when an operation event occurs due to pressing of the down key of the operation input device 2 in a state where the focus is given to the button 101, the focus management unit 111 of the UI screen execution unit 11 changes the focus. By moving the button from the button 101 to the button 102 located below the button 101, the UI component control unit 112 gives a display effect indicating that the button 102 is focused. That is, the button 102 is drawn with a thick line.

As shown in FIG. 15, when the focus is given to the button 101 or the button 102, the proxy component status information transmitting unit 123 of the first platform 10 sends the proxy component indicating that the focus is not given to the proxy component 103. The status information is transmitted to the second platform 20.

The proxy component status information transmitted by the proxy component status information transmitting unit 123 is received by the proxy component status information receiving unit 223 of the second platform 20. When the UI component control unit 212 confirms that the focus is not assigned to the proxy component 103 with reference to the proxy component status information, the UI component control unit 212 focuses on any of the buttons 201 and 202 on the second UI screen 200. Is given to the second UI screen 200, indicating that no is provided. Therefore, in FIG. 15, both the buttons 201 and 202 on the second UI screen 200 are drawn with thin lines.

Also, for example, as shown in FIG. 16, when an operation event occurs by pressing the down key of the operation input device 2 in a state where the focus is given to the button 102, the focus management unit 111 of the UI screen execution unit 11 The focus is moved from the button 102 to the proxy component 103 located below the button 102.

In this case, the proxy component status information transmitting unit 123 of the first platform 10 transmits to the second platform 20 proxy component status information indicating that the focus has been given to the proxy component 103.

When the UI component control unit 212 of the second platform 20 confirms that the focus has been given to the proxy component 103 with reference to the proxy component status information received by the proxy component status information receiving unit 223, the second UI A display effect indicating that the focus is given to one of the buttons 201 and 202 (the button 201 in FIG. 15) on the screen 200 is given.

As described above, the user switches the operation target from the UI component of the first UI screen 100 to the UI component of the second UI screen 200 by the same operation as moving the focus on the first UI screen 100. Can be. In addition, when the operation target is switched to the UI component on the second UI screen 200, a display effect indicating that the UI component is given focus is given. It can be easily recognized that the switching from the 100 UI components to the UI components on the second UI screen 200 is performed.

Also, as shown in FIG. 17, when an operation event occurs by pressing the down key of the operation input device 2 in a state where the focus is given to the button 201 of the second UI screen 200, the focus management unit 211 Is moved from the button 201 to the button 202 located below the button 201, and the UI component control unit 212 gives a display effect indicating that the button 202 is focused. That is, the button 202 is drawn with a thick line.

On the other hand, as shown in FIG. 18, when an operation event occurs by pressing the up key of the operation input device 2 in a state where the focus is given to the button 201 of the second UI screen 200, Has no UI component to be focused above the button 201, the operation event is not consumed by the second platform 20. In this case, as in the first embodiment, the focus management unit 111 of the first platform 10 shifts the focus from the proxy component 103 to the button 102 located above the proxy component 103, and the UI component control unit 112 focuses on the button 102. Is drawn with a thick line.

{Furthermore, the proxy component status information transmission unit 123 of the first platform 10 transmits proxy component status information indicating that the focus has been taken from the proxy component 103 to the second platform 20. Upon confirming that the focus has been taken from the proxy component 103 by referring to the proxy component status information received by the proxy component status information receiving unit 223, the UI component control unit 212 executes the button 201 of the second UI screen 200. , 202 is given to the second UI screen 200 a display effect indicating that no focus is given. Therefore, both the buttons 201 and 202 on the second UI screen 200 are drawn with thin lines.

As described above, the user switches the operation target from the UI component on the second UI screen 200 to the UI component on the first UI screen 100 by the same operation as moving the focus on the first UI screen 100. Can be. Further, when the operation target is switched to the UI component on the first UI screen 100, a display effect indicating that no UI component on the second UI screen 200 has been given a focus is given. It can be easily recognized that the operation target is not a UI component of the second UI screen 200.

<Embodiment 3>
In the first and second embodiments, the user interface device 1 has two platforms. However, the user interface device 1 may have three or more platforms.

For example, when the user interface device 1 includes three platforms, the composite UI screen provided by the user interface device 1 includes a first UI screen 100 executed by the first platform and a second UI screen, as illustrated in FIG. It may include a second UI screen 200 executed by the platform and a third UI screen 300 executed by the third platform. In the composite UI screen of FIG. 19, a second UI screen 200 is arranged in the first UI screen 100, and a third UI screen 300 is arranged in the second UI screen 200.

The first UI screen 100 constituting the composite UI screen of FIG. 19 is actually composed of buttons 101 and 102 as UI parts and a proxy part 103 corresponding to the second UI screen 200 as shown in FIG. Contains. Similarly, the second UI screen 200 includes buttons 201 and 202 as UI components and a proxy component 203 corresponding to the third UI screen 300, as shown in FIG. Then, a second UI screen 200 is displayed on the proxy component 103 portion of the first UI screen 100, and a third UI screen 300 (FIG. 22) is further displayed on the proxy component 203 portion of the second UI screen 200. When displayed, the composite UI screen shown in FIG. 19 is apparently configured.

As in the composite UI screen of FIG. 19, a first UI screen 100 executed by the first platform, a second UI screen 200 executed by the second platform, and a third UI executed by the third platform When the UI screen 300 is nested, the first platform is defined as a main platform, the second platform is defined as a slave platform, and the second platform is defined as a relationship between the first platform and the second platform. Regarding the relationship between the platform and the third platform, the first and second embodiments can be applied by defining the second platform as the main platform and the third platform as the slave platform.

In addition, for example, as shown in FIG. 23, in the composite UI screen, a first UI screen 100 executed by the first platform includes a second UI screen 200 executed by the second platform, and a third platform May be arranged.

The first UI screen 100 constituting the composite UI screen of FIG. 23 is actually composed of buttons 101 and 102 as UI parts and a proxy part 103 corresponding to the second UI screen 200 as shown in FIG. , A proxy component 104 corresponding to the third UI screen 300. Then, the second UI screen 200 (FIG. 25) is displayed in the portion of the proxy component 103 of the first UI screen 100, and the third UI screen 300 (FIG. 26) is displayed in the portion of the proxy component 104. In appearance, the composite UI screen shown in FIG. 23 is configured.

As shown in the composite UI screen of FIG. 23, a first UI screen 100 executed by the first platform includes a second UI screen 200 executed by the second platform and a second UI screen 200 executed by the third platform. In the case where the third UI screen 300 is arranged, the first and second embodiments can be applied by defining the first platform as the main platform and the second platform and the third platform as the sub-platforms. . However, it is necessary for the UI screen execution unit 11 of the first platform 10 to hold information indicating a slave platform targeted by the proxy components 103 and 104 (hereinafter, referred to as “target platform”).

For example, as shown in FIG. 27, in the composite UI screen, the first UI screen 100 executed by the first platform includes the second UI screen 200 executed by the second platform, and the third platform May be selectively switched and displayed on the third UI screen 300 executed by the user. In this case, as shown in FIG. 28, the proxy component arranged in the first UI screen 100 is replaced with the proxy component 103 corresponding to the second UI screen 200, or the proxy component corresponding to the third UI screen 300. The user can switch between the components 104. Then, when the proxy component 103 is arranged in the first UI screen 100, the second UI screen 200 (FIG. 29) is displayed on the part of the proxy component 103, and the proxy component is displayed in the first UI screen 100. When the 104 is arranged, the third UI screen 300 (FIG. 30) is displayed in the portion of the proxy component 104. As a result, as shown in FIG. 27, the second UI screen 200 and the third UI screen 300 are selectively switched and displayed in the composite UI screen.

In the example of FIG. 27 as well, the first and second embodiments can be applied by defining the first platform as the main platform and the second and third platforms as the sub-platforms. Also, the UI screen execution unit 11 of the first platform 10 needs to hold information on the target platform of each of the proxy components 103 and 104.

[Modification]
In the first to third embodiments, it has been assumed that a plurality of UI screens constituting a composite UI screen are displayed on a screen of one display device 3. However, for example, as shown in FIG. And the second UI screen 200 may be displayed on different display devices 3a and 3b, respectively.

In this case, the configuration of the composite UI screen and the operation of the user interface device 1 may be basically the same as those in the first and second embodiments. For example, if the user interface device 1 performs a process of trimming the upper part of the composite UI screen of FIG. 12 and displaying it on the display device 3a and trimming the lower part of the composite UI screen and displaying it on the display device 3b, FIG. Such a display is possible.

Although there is no restriction on the arrangement of the display devices 3a and 3b, as shown in the example of FIG. 31, when the upper part of the composite UI screen of FIG. 12 is displayed on the display device 3a and the lower part is displayed on the display device 3b, the button 101 It is preferable that the display device 3b is disposed at a position lower than the display device 3a so that the user can intuitively grasp the positional relationship between the display device 102 and the second UI screen 200.

In the case where the composite UI screen includes three or more UI screens as in the third embodiment, each UI component may be divided and displayed on three or more display devices.

<Embodiment 4>
In the fourth embodiment, a description will be given of a user interface design apparatus capable of designing a UI screen constituting a composite UI screen.

FIG. 32 is a diagram showing a configuration of a user interface design device 60 according to the fourth embodiment. As shown in FIG. 32, the user interface design device 60 is connected to the operation input device 71 and the display device 72.

The operation input device 71 is an input unit for a user who is a designer of the UI screen to input an operation to the user interface design device 60, and includes, for example, a keyboard and a mouse. The display device 72 is a display unit for displaying a screen used by the user interface design device 60 for designing a UI screen (hereinafter, referred to as a “UI design screen”), and is, for example, a liquid crystal display device.

32, the user interface design device 60 includes a UI component storage unit 61, a UI component placement unit 62, a UI component attribute value setting unit 63, and an execution data generation unit 64.

The UI component storage unit 61 is a storage medium that stores a plurality of UI components that can be arranged on the UI screen. The UI components stored in the UI component storage unit 61 include proxy components. The UI component placement unit 62 configures the UI screen by arranging a plurality of UI components stored in the UI component storage unit 61 on the UI design screen according to a user operation. The UI component attribute value setting unit 63 sets an attribute value (for example, a position and a size in the UI screen) of the UI component arranged on the UI screen being designed according to a user operation. The execution data generation unit 64 converts the UI screen created using the UI component placement unit 62 and the UI component attribute value setting unit 63 into execution data to be executed by a specific platform.

FIG. 33 shows an example of a UI design screen provided by the user interface design device 60. The UI design screen 600 in FIG. 33 includes a UI component storage area 601, a UI component placement area 602, and a UI component attribute value setting window 603.

A list of UI components stored in the UI component storage unit 61 is displayed in the UI component storage area 601. The UI component placement area 602 is an area where the user places a UI component to configure a UI screen using the UI component placement unit 62. The user can design the layout of the UI screen by arranging the UI components displayed in the UI component storage area 601 in the UI component storage area 601 by, for example, dragging and dropping a mouse.

The UI component attribute value setting window 603 allows the user to use the UI component attribute value setting unit 63 to display the UI components arranged on the UI screen being designed, that is, the UI components arranged in the UI component storage area 601. This is a window for inputting attribute values. The UI component attribute value setting window 603 does not need to be always displayed, but may be displayed when the user inputs an attribute value. For example, the user may right-click the mouse on a UI component for which the user wants to input an attribute value, so that the UI component attribute value setting window 603 of the UI component may be displayed.

The UI screen designed using the 設計 UI design screen 600 is converted into execution data by the execution data generation unit 64 as shown in FIG. The platform of the user interface device 1 can display the UI screen on the display device 3 by executing the execution data generated by the execution data generation unit 64. The format of the execution data is not limited to that shown in FIG. 34, and may be any format.

The user interface design device 60 can also be used for designing a general UI screen that does not include a proxy component.

FIG. 35 is a flowchart showing a procedure for creating a UI screen using the user interface design device 60. A user who is a designer first places a UI component (including a proxy component) displayed in the UI component storage area 601 of the UI design screen 600 in the UI component storage area 601 by dragging and dropping a mouse. Then, a UI screen is configured (step S301). Next, the user displays a UI component attribute value setting window 603 of a UI component (UI component arranged in the UI component storage area 601) arranged on the UI screen being designed, and displays the position and the position of the UI component. An attribute value such as a size is input (step S302). Then, the user converts the designed UI screen into execution data using the execution data generation unit 64 (step S303).

As described in the third embodiment, when the user interface device 1 has three or more platforms, it is necessary to determine the target platform of the proxy component arranged on the UI screen. Therefore, in the user interface design device 60 that designs the UI screen of the user interface device 1 including three or more platforms, the UI component attribute value setting unit 63 sets the proxy component arranged on the UI screen as the attribute value of the target platform as an attribute value. To indicate the information.

FIG. 36 shows an example of a UI design screen 600 provided by the user interface design device 60 for designing a UI screen of the user interface device 1 having three or more platforms. In the UI design screen 600 of FIG. 36, an input box for the target platform of the proxy component is provided in a UI component attribute value setting window 603 for inputting the attribute value of the proxy component. Also, as shown in FIG. 37, the information of the target platform is described in the execution data. Thereby, the platform that executes the execution data can recognize the target platform of each proxy component.

As in the first and second embodiments, when the user interface device 1 has only two platforms, it is clear that the platform that executes the UI screen including the proxy component is the main platform and the other platform is the sub-platform. Therefore, it is not necessary to include information on the target platform in the attribute value of the proxy component. However, even when the user interface device 1 includes only two platforms, information on the target platform may be included in the attribute value of the proxy component.

According to the user interface design device 60 of the present embodiment, since the proxy component can be included in the UI component that can be arranged on the UI screen, the composite UI screen as described in the first to third embodiments is designed. be able to.

The user interface design device 60 can also be realized by the hardware configuration shown in FIG. That is, the user interface design device 60 can be realized by the processor 501 and the memory 502 storing the program executed by the processor 501. A plurality of UI components (including a proxy component arranged at a display position of a UI screen executed by the second platform) are arranged in the memory 502 according to a user operation when executed by the processor 501. A process of configuring a UI screen executed by the first platform, a process of setting an attribute value to each of UI components arranged on the UI screen according to a user operation, and a process of generating the UI screen by the first platform A process for converting the data into execution data to be executed and a program that is to be executed as a result are stored.

In the present invention, it is possible to freely combine the embodiments or to modify or omit the embodiments as appropriate within the scope of the invention.

Although the present invention has been described in detail, the above description is illustrative in all aspects, and the present invention is not limited thereto. It is understood that innumerable modifications that are not illustrated can be assumed without departing from the scope of the present invention.

Reference Signs List 1 user interface device, 2 operation input device, 3 display device, 10 first platform, 11 UI screen execution unit, 111 focus management unit, 112 UI component control unit, 12 interface unit, 121 operation event transmission unit, 122 operation event Consumption information receiving unit, 123 {proxy component status information transmitting unit, 20} second platform, 21 UI screen execution unit, 210 operation event consumption determination unit, 211 focus management unit, 212 UI component control unit, 22 interface unit, 221 operation event Receiving unit, 222 {operation event consumption information transmitting unit, 223} proxy component state information receiving unit, 100 {first UI screen, 200} second UI screen, 300 {third UI screen, 101, 102, 201, 202, 301} button 103, 104, 203 {proxy component, 501 processor, 502 memory, 60 user interface design device, 61 UI component storage unit, 62 UI component placement unit, 63 UI component attribute value setting unit, 64 execution data generation unit, 71 operation input Device, 72 display device, 600 UI design screen, 601 UI component storage area, 602 UI component placement area, 603 UI component attribute value setting window.

Claims (8)

A user interface device for displaying a composite UI screen including a plurality of UI screens executed by a plurality of platforms,
The plurality of platforms includes a primary platform and a secondary platform,
The UI screen executed by the main platform includes a plurality of UI components,
The plurality of UI parts have no display element, and include a proxy part arranged at a display position of a UI screen executed by the slave platform,
The main platform is:
A focus management unit that gives focus to a UI component selected by a user among the plurality of UI components;
A UI component control unit that gives a display effect indicating that the focus is given to the UI component to which the focus is given;
While the focus is given to the proxy component, an operation event transmission unit that transmits an operation event generated by the operation of the user to the slave platform,
From the slave platform, an operation event consumption information receiving unit that receives operation event consumption information indicating whether the operation event has been consumed by the slave platform,
With
The slave platform,
An operation event receiving unit that receives the operation event from the main platform;
An operation event consumption determination unit that determines whether the operation event has been consumed by the slave platform;
An operation event consumption information transmission unit that transmits a determination result as to whether or not the operation event has been consumed by the slave platform to the master platform as the operation event consumption information,
With
When the operation event is consumed by the slave platform, the focus management unit of the master platform maintains the focus given to the proxy component,
If the operation event has not been consumed by the slave platform, the main platform executes a process corresponding to the operation event,
User interface device. The main platform is:
A proxy component status information transmitting unit that transmits proxy component status information indicating whether or not the focus has been given to the proxy component to the slave platform,
The slave platform,
A proxy component status information receiving unit that receives the proxy component status information;
While the focus is given to the proxy component, a display effect indicating that the focus is given to any UI component of a UI screen executed by the slave platform is given, and the focus is given to the proxy component. A UI component control unit that gives a display effect to the UI screen executed by the sub-platform to indicate that the focus is not applied to any UI component of the UI screen executed by the sub-platform while the sub-platform is executed;
Further comprising,
The user interface device according to claim 1. The plurality of platforms includes a plurality of the slave platforms,
The plurality of UI components include a plurality of the proxy components associated with each of the plurality of slave platforms.
The user interface device according to claim 1. A user interface design apparatus for designing a UI screen of a user interface apparatus that displays a composite UI screen including a plurality of UI screens executed by a plurality of platforms,
A UI component storage unit that stores a plurality of UI components that can be arranged on a UI screen executed by the first platform;
A UI component placement unit that configures a UI screen by arranging a plurality of UI components stored in the UI component storage unit according to a user operation;
A UI component attribute value setting unit that sets an attribute value for each of the UI components arranged on the UI screen in accordance with an operation of the user;
An execution data generation unit that converts a UI screen created using the UI component placement unit and the UI component attribute value setting unit into execution data for execution by the first platform;
With
The plurality of UI components include a proxy component having no display element and arranged at a display position of a UI screen executed by the second platform.
User interface design device. The UI component attribute value setting unit can set which platform is the second platform as an attribute value of the proxy component.
The user interface design device according to claim 4. When a plurality of the proxy components are arranged on the UI screen, the UI component attribute value setting unit can set which platform becomes the second platform for each of the proxy components.
The user interface design device according to claim 5. A method for controlling a user interface device that displays a composite UI screen including a plurality of UI screens executed by a plurality of platforms,
The plurality of platforms includes a primary platform and a secondary platform,
The UI screen executed by the main platform includes a plurality of UI components,
The plurality of UI parts have no display element, and include a proxy part arranged at a display position of a UI screen executed by the slave platform,
The main platform is
Giving focus to the UI component selected by the user among the plurality of UI components,
Giving the UI component to which the focus is given a display effect indicating that the focus is given;
While the focus is given to the proxy component, transmit an operation event generated by the operation of the user to the slave platform,
Said slave platform,
Receiving the operation event from the main platform;
Determining whether the operation event has been consumed on the slave platform,
A determination result as to whether or not the operation event has been consumed by the slave platform is transmitted to the master platform as operation event consumption information,
The main platform is
Receiving the operation event consumption information;
If the operation event is consumed on the slave platform, maintain the focus given to the proxy component;
If the operation event is not consumed by the slave platform, execute a process corresponding to the operation event,
A method for controlling a user interface device. A screen design method for a user interface device that displays a composite UI screen including a plurality of UI screens executed by a plurality of platforms,
A UI component placement unit of the user interface design device configures a UI screen executed by the first platform by arranging a plurality of UI components according to a user operation;
A UI component attribute value setting unit of the user interface design device that sets an attribute value for each of the UI components arranged on the UI screen according to an operation of the user;
An execution data generation unit of the user interface design device that converts a UI screen created by the UI component placement unit and the UI component attribute value setting unit into execution data for execution by the first platform;
The plurality of UI components include a proxy component having no display element and arranged at a display position of a UI screen executed by the second platform.
A screen design method for a user interface device.

Images