HK1177011B - Markup based extensibility for user interfaces - Google Patents

Description

Markup-based extensibility for user interfaces

The application is as follows: 2006.08.29, application No. 200680030421.2 (international application No. PCT/US2006/033809), filed as a divisional application of the application entitled "tag-based extensibility for user interfaces".

Technical Field

The present application relates to user interfaces, and more particularly to markup-based extensibility for user interfaces.

Background

With the advent of the computer age, users of computers and software have grown accustomed to user-friendly software applications that help them write, calculate, organize, prepare presentations, send and receive electronic mail, create music, and the like. For example, modern electronic word processing applications allow users to prepare a variety of useful documents. Modern spreadsheet applications allow users to enter, manipulate, and organize data. Modern electronic slide presentation applications allow users to create a variety of slide presentations containing text, pictures, data, or other useful objects. Modern database applications allow users to store, organize, and exchange large amounts of data.

Most software applications provide one or more graphical user interfaces through which a user can enter and edit data and access and use various functions of the associated software application. A typical user interface includes a workspace where data can be entered, edited, and reviewed. Additionally, the user interface typically includes one or more buttons and/or controls for selecting functionality provided by the associated software application. For example, buttons or controls may be provided for printing or saving a document, buttons or controls may be provided for applying formatting properties to aspects of a document, and so forth.

Typically, third party software developers create software plug-ins that are added to existing applications to provide functionality that is not available from the existing applications. For example, a plug-in application may provide a feature for a word processing application to add special footnotes and endnotes to a document. In addition to providing additional functionality, the add-in application typically provides one or more new user interface components to an existing application user interface, such as new toolbars, buttons, or other controls, for accessing the additional functionality.

According to previous approaches, third party developers were given access to object models associated with application existing user interfaces to allow customization of the existing user interfaces according to the needs of third party plug-in software. Unfortunately, these previous approaches have drawbacks because the object model for a given user interface is typically not designed around a common user across a variety of different applications, including: such as word processing applications, spreadsheet applications, slide presentation applications, etc., and these applications often exhibit different and possibly undesirable behavior than that associated with customized user interface components.

In view of these and other considerations, the present invention has been made.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended as an aid in determining the scope of the claimed subject matter.

Embodiments of the present invention allow modification of existing application user interfaces to include new or modified user interfaces associated with add-in applications by providing methods, systems, and computer products for exposing software application user interface programming to third party application add-in developers. According to aspects of the invention, an extensible markup language (XML) schema is exposed to third party developers that governs XML programming that may be used to modify a given user interface to allow the developers to specify changes to the user interface programming according to the associated XML schema. For example, if a third party developer wishes to add a new button or control to an existing user interface that can be linked to a functionality of a plug-in application, the third party developer can modify the existing programming of the user interface according to the grammatical and syntactic rules specified by the associated XML schema. According to an aspect of the invention, the XML or other suitable representation of the user interface modification does not need to follow the same programming language as the original user interface. Moreover, the original built-in user interface programming may be complex, and the XML schema exposed in accordance with the present invention may be only a subset of the overall programming of the original user interface. When the modified programming is executed by the host software application, the user interface is presented with changes made by the third party developer. For example, if the programming of the user interface is modified to add a new button, the new button will be presented in the user interface in response to the modification to the programming of the user interface, as long as the modification is made according to the associated XML schema.

According to aspects of the invention, new interface components may be added to an existing user interface and may be linked to associated plug-in functionality. The size of the new user interface component may be automatically scaled to fit the available display space as a window in which the displayed user interface is reduced or enlarged. In addition, according to aspects of the present invention, an end user of the modified user interface may remove the added user interface components as needed. If a particular software plug-in is deleted, the added or modified user interface components associated with the deleted plug-in will not be displayed in the user interface that is subsequently presented.

According to other aspects of the invention, existing user interface components, such as buttons and controls, may be disabled or removed by third parties. Additionally, existing user interface components may be repurposed such that when selected the repurposed component exhibits different behavior.

According to other aspects of the invention, third party contextual user interfaces and contextual user buttons or controls may be added to existing user interfaces presented in existing user interfaces when a document object associated with the added contextual user interface is selected by accessing a schema presented for modifying the existing user interfaces. In addition, a control (which when selected causes one or more interpolation functions to be applied to the selected object) may be added to a gallery or collection of controls deployed in an existing user interface to apply one or more functions of the application to the selected object.

According to other aspects of the invention, XML schema may be used to construct a ab initio user interface that is customized according to the third party plug-in requirements of the associated software application. When such a start-from-scratch user interface solution is presented, the resulting user interface has little resemblance to the application user interface that is typically exposed to the user of the associated software application. That is, when an associated document is run, a customized user interface for providing user functionality to the run document is presented in accordance with the customized user interface provided to the document.

These and other features and advantages, which characterize the present invention, will be apparent from a reading of the following detailed description and a review of the associated drawings. It is to be understood that both the foregoing general description and the following detailed description are explanatory only and are not restrictive of the invention as claimed.

Drawings

FIG. 1 illustrates an exemplary computing operating environment for embodiments of the present invention.

FIG. 2 is a computer screen display showing an example user interface that may be modified according to embodiments of the present invention.

FIG. 3 is a simplified block diagram illustrating a relationship between an example user interface and an XML representation of the example user interface that may be refined for modifying the example user interface according to embodiments of the present invention.

FIG. 4 is a computer screen display of an example contextual user interface that may be modified according to embodiments of the present invention.

FIG. 5 is a computer screen display of an example gallery and collection of selectable controls user interfaces that may be modified according to embodiments of the present invention.

Detailed Description

As briefly described above, embodiments of the present invention are directed to methods, systems, and computer products for exposing the programming of an application user interface to allow modification of the associated user interface to include adding, removing, disabling, enabling, and repurposing new or existing user interfaces. In the following detailed description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific embodiments and examples. These embodiments may be combined, other embodiments may be utilized, and structural changes may be made without departing from the spirit or scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims and their equivalents.

Referring now to the drawings, in which like numerals refer to like elements, aspects of the present invention and an exemplary computing operating environment will be described. FIG. 1 and the following discussion are intended to provide a brief, general description of a suitable computing environment in which the invention may be implemented. While the invention will be described in the general context of program modules that execute in conjunction with an application program that runs on an operating system on a personal computer, those skilled in the art will recognize that the invention may also be implemented in combination with other program modules.

Generally, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the invention may be practiced with other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Embodiments of the invention may be implemented as a computer process (method), a computing system, or as an article of manufacture, such as a computer program product or computer readable media. The computer program product may be a computer storage media readable by a computer system and encoding a computer program of instructions for executing a computer process. The computer program product may also be a propagated signal on a carrier readable by a computing system and encoding a computer program of instructions for executing a computer process.

With reference to FIG. 1, an exemplary system for implementing the invention includes a computing device, such as computing device 100. In a basic configuration, computing device 100 typically includes at least one processing unit 102 and system memory 104. Depending on the exact configuration and type of computing device, system memory 104 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.) or some combination of the two. System memory 104 typically includes an operating system 105 suitable for controlling the operation of a networked personal computer, such as Microsoft corporation of Redmond, WashingtonAnd (4) operating the system. The system memory 104 may also include one or more software applications 106 and may include program data 107. Those components in dashed line 108 in fig. 1 illustrate this basic configuration.

According to embodiments of the invention, the application programs 106 may include various types of programs, such as email programs, calendar programs, Internet browsing programs, and the like. An example of such a program is available from Microsoft corporationThe application 106 may also include multi-functionality software for providing a variety of other types of functionality. Such a multi-functionality application may include a number of program modules, such as a word processing program, a spreadsheet program, a slide presentation program, a database program, and so forth. An example of such a multi-functionality application is OFFICE, available from Microsoft corporation^TM. The plug-in software application 120 includes any software application that may be added to the application 106, as described herein, to enhance or provide additional functionality to the application 106. Additionally, as described herein, the plug-in software application may include a document-based software solverFor example, a spreadsheet document that includes an attached toolbar, or a word processing document that includes a macro or other code that adds a toolbar with buttons or controls.

Computing device 100 may have additional features or functionality. For example, computing device 100 may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in FIG. 1 by removable storage 109 and non-removable storage 110. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. System memory 104, removable storage 109 and non-removable storage 110 are all examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by computing device 100. Any such computer storage media may be part of device 100. Computing device 100 may also have input device(s) 112 such as keyboard, mouse, pen, voice input device, touch input device, etc. Output device(s) 114 such as a display, speakers, printer, etc. may also be included. These devices are well known in the art and need not be discussed in detail herein.

The computer device 100 may also contain communication connections 116 that allow the device to communicate with other computing devices 118, such as over a network in a distributed computing environment, for example, an intranet or the Internet. Communication connection(s) 116 is one example of communication media. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. The term "computer readable media" as used herein includes both storage media and communication media.

FIG. 2 is a computer screen display showing an example user interface that may be modified according to embodiments of the present invention. Those skilled in the art will appreciate that the example user interface illustrated in FIG. 2 is for purposes of example and illustration only. That is, embodiments of the present invention may be applied to a variety of different user interface arrangements having different user interface components and different user interface layouts. Thus, the description of the present invention in terms of the exemplary user interface illustrated in FIG. 2 should not be taken as limiting or restrictive of the claimed invention.

The user interface in fig. 2 includes a ribbon-shaped user interface for displaying selectable controls associated with task-based functionality available under a given software application, such as software application 106 shown in fig. 1. A first area 210 of the user interface 200 includes generic selectable controls for functions not associated with a particular task (e.g., word processing versus spreadsheet data analysis). For example, area 210 includes selectable controls for common file commands (e.g., "open file," "save file," and "print"). According to one embodiment of the invention, the selectable controls included in the first area 210 are controls that may be used by various software applications (e.g., a word processing application, a spreadsheet application, a slide presentation application, etc.). That is, the selectable controls included in the first region 210 are controls that are typically found and used across multiple different software applications.

Adjacent to the first area 210 of the user interface 200 is a task-based tab area. The tab area includes selectable tabs associated with task-based functionality provided by a given software application. For purposes of example, the task-based tab shown in FIG. 2 is associated with a task performed using a word processing application. For example, a "Writing (tab 215 is associated with a function used to perform a Writing task," Insert "tab 220 is associated with a function to perform an Insert operation or task," Page layout "tab 230 is associated with a function provided by an associated application that performs or edits Page layout properties of a given document.

It should be noted that many other task-based tabs or selectable controls may also be added to the tab area in the user interface to invoke functionality associated with other tasks. For example, task tabs may be added for text effects, document styles, reviews and comments, and so forth. Also, as described above, the user interface 200 may be used with a variety of different software applications. For example, if the user interface 200 is used in a slide presentation application, the tabs contained in the tab area may include tabs associated with various tasks performed by the slide presentation application, such as "create slide," "insert," "format," "drawing effect," and so forth. Similarly, tabs that may be used in the tab area of the user interface 200 of the spreadsheet application 140 include tabs associated with tasks performed using the spreadsheet application, such as "data" or "data entry," "list," "pivot table," "parse," "formula," "page and print," and the like.

Immediately below the generic control area 210 and the task-based tab area is a selectable functionality control area for displaying selectable functionality controls associated with the selected tabs 215, 220, 230 in the task-based tab area. In accordance with embodiments of the present invention, when a particular tab, such as the "compose" tab 215, is selected, selectable functions available to the associated software application for performing the selected task (e.g., composing the task) are displayed in the logical group. For example, referring to fig. 2, a first logical group 240 is shown under the title "Clipboard". According to embodiments of the present invention, clipboard area 240 includes selectable functionality controls that are logically grouped together and associated with clipboard actions underlying the general task of writing.

Selectable controls displayed in the format area 250 include selectable controls such as text alignment, text type, font size, line spacing, boldface, italics, underline, and the like. Thus, the functions associated with the format operations are logically grouped together and located under the entire "composition" task. A third logical grouping 260 is shown under the heading "Writing Tools". The composition tools area 260 includes composition tools such as find/replace, auto-correct, etc. According to embodiments of the present invention, when a different task-based tab is selected from the tab area, a different set of selectable functionality controls in different logical groupings associated with the selected task-based tab is displayed in the user interface 200. For example, if the "insert" task tab 220 is selected, the selectable functionality controls displayed in the user interface 200 change from those shown in FIG. 2 to include selectable functionality controls associated with the insert task.

As described herein, third party application or solution developers typically provide software applications or modules that can be added to existing applications to add functionality to the existing applications. For example, a software developer may generate a software application or module that is added to a word processing application in order to add additional formatting properties or other functionality that is not available in a host application (e.g., a word processing application, a spreadsheet application, a slide presentation application, etc.), or a third party add-in application or module that may modify or enhance functionality provided by the host application. As described herein, embodiments of the present invention provide access to programming associated with one or more user interface modifications of a host application to allow third party application developers to modify the user interface of the host application to provide the user with interface components for added or modified functionality provided by a given plug-in or module.

It should be noted that embodiments of the present invention are not intended to be limited to use in association with plug-in applications. For example, a third party developer may use aspects of the invention to modify components of an application user interface without regard to added functionality. For example, as described herein, a third party may desire to change the behavior of a user interface component in addition to any added functionality.

FIG. 3 is a simplified block diagram illustrating a relationship between an example user interface and an XML representation of the example user interface that may be modified for modifying the example user interface according to embodiments of the present invention. Referring to fig. 3, a user interface 305 of a host application 106 (e.g., a word processing application, a spreadsheet application, a slide presentation application, etc.) is shown with a file command button 310, a tab1 button 315, and a tab2 button 320. As described above with reference to FIG. 2, the example user interface 305 shown in FIG. 3 is for illustrative purposes only and is not intended to limit the various buttons, controls, or other selectable functionality that may be provided with a given user interface in accordance with embodiments of the present invention. The example user interface 305 includes a first logical group of buttons or controls 325 and a second logical group of buttons or controls 330. According to the exemplary user interface 305, the tab2 button 320 is selected to provide the first and second logical groups of buttons or controls 325, 330.

According to an embodiment of the present invention, the user interface 305 is programmed and structured according to a markup language, such as the extensible markup language (XML). It should be understood that other languages suitable for programming and constructing a user interface 305 as described herein can be used. As shown in FIG. 3, an XML file 340 is shown to provide the XML programming and structure of the example user interface 305. For example, a < UI > root tag is provided with three sub-tags < FILE >, < TAB1>, and < TAB2 >. The < TAB2> tag includes two sub-tags: an < LG1> tag (a first logical group of buttons or controls) and an < LG2> tag (a second logical group of buttons or controls). It should be understood that the XML file 340 is not intended to illustrate a well-formatted XML file, but is merely for illustrative purposes.

The developer of the host application provides an XML schema document or file (not shown) that provides XML syntax and syntax rules that are applied to the XML used to program and build the user interface 305, such as adding, removing, or modifying new or existing tags to the XML file 340. For example, the XML schema may specify the type and name of the XML markup to be added to the XML file 340. For example, an XML schema associated with user interface 305 may require that each tab tag must include at least one logical group child tag. In yet another example, the XML schema associated with the user interface 305 may indicate that some of the XML in the XML file 340 cannot be modified, e.g., that the logical group markup under a given tab markup of the original user interface 305 cannot be modified by a third party.

In another example, the XML schema may specify that certain format attributes applied to the user interface 305 may or may not be changed by applying changes to the XML file 340. The complete XML schema file 340 and the complete associated XML schema file may be exposed to third party developers. Alternatively, only a subset of the XML files 340 associated with the host application user interface 305 and a subset of the associated XML schema files are exposed to third party developers to allow them a limited number and type of modifications to the host application user interface 305.

The XML file 340 and the associated XML schema file of the host application user interface 305 are exposed to the third party 340 to allow the third party to change the XML file 340 to modify the user interface 305 according to the grammatical and syntactic rules provided by the associated XML schema file, as described herein. For example, the third party 350 may provide a software application plug-in that provides additional functionality not currently available in the host application. For example, a third party may need to add a third-party tab button in the user interface, and when that button is selected, an additional logical grouping of buttons or controls may be provided under the added third-party tab button to provide buttons and/or controls for selecting functionality provided by the third-party application add-in. For example, a third party application plug-in may incorporate functionality for providing specialized footnotes and endnotes to a word processing application. In order to expose the functionality provided by the plug-in to the user, the third party developer needs to provide an additional tab in the user interface 305, such as a "footnote/endnote" tab. In addition to adding the "footnote/endnote" tab, a third party developer may wish to add additional logical groups of buttons or controls that are displayed in the user interface 305 when the "footnote/endnote" tab is selected to provide buttons and/or controls for selecting aspects of the added footnote and endnote functionality. It will be appreciated that existing buttons or controls may similarly be removed from the user interface 305 by modifying the modified XML file 340.

According to one embodiment, the markup exposed to the third party developer via the XML file 340 is of an incremental nature, thereby providing the third party developer with limited modification capabilities to the host application user interface 305. Thus, according to this embodiment, third party developers do not receive XML files 340 that allow them to insert their own user interface components at arbitrary locations. According to this embodiment, instead of the XML file 340 and associated schema being exposed to the third party, allowing the third party developer to specify the desired location of the new user interface component, such as inserting the new component after the existing component ABC, the host application then integrates the modified XML file with the entire XML file representing the host application interface 305 to render the modified user interface 307.

According to one embodiment, the amended XML file 355 provided by each third party developer is identified by a unique XML namespace to prevent one third party application add-in from modifying user interface components provided or modified by another third party add-in. Alternatively, the third party add-in file 355 may be identified by other means, such as by a Globally Unique Identifier (GUID) or a local file name.

Once the XML representation 340 of the host application user interface 305 is accessed by the third party 350, the third party 350 may refine the XML applied to the user interface 305 according to the associated XML schema file to add the desired buttons or controls to the user interface 305. The resulting refined XML file 355 illustrates the modifications made to the XML file 355 to the user interface 305. For example, referring to the refined XML file 355, the third party developer 350 has added < TAB3> TAB 360 and < LG4> TAB (fourth logical group of buttons and/or controls) 365. The XML file 355 illustrates the incremental nature of modifications that may be made to an existing user interface according to embodiments of the present invention. That is, because the example modification is directed to the addition of a new user interface TAB (TAB3) and associated logical grouping of buttons or controls (LG4), the XML file 355 need only make incremental changes that add the new TAB and logical grouping to the existing user interface, as opposed to making global changes to the XML exposed by the user interface 305.

As shown in FIG. 3, once the required changes made to the XML file 340 are applied to the host application 305, a modified user interface 307 may be presented. Referring to the improved user interface 307, a logical grouping of added third indicia 370 and added fourth keys and/or controls 375 is shown. Once the added logical groups of indicia and buttons or controls are linked to the associated functionality of the third party application add-in, selection of these added indicia, buttons or controls causes execution of the associated added functionality. Thus, by gaining access to the XML file and associated XML schema exposed by the host application user interface, a third party application developer can modify the host application user interface 305 to provide the desired buttons and/or controls for selecting functionality of the associated add-in application.

The following is an excerpt of an example XML file that has been modified or provided by a third party for the final addition of a "data analysis" button or control to be inserted after the logical group of buttons or controls labeled "data tools". As shown in example XML, such XML syntax (e.g., < insert after >) is indicated by an associated XML schema file for allowing third party application developers to insert a key labeled "data analytics" after a logical group of keys or controls labeled "packet data tools". It will be appreciated by those skilled in the art that the following exemplary XML is for illustrative purposes only and is not limiting of the claimed invention.

The changes to the XML file 340 may be made by a third party developer according to different methods. According to one embodiment, the XML file 340 and associated schema may be accessed by third party application developers at the application layer. For example, buttons or controls are provided in the user interface 305 to allow third party application developers to access the XML file 340 and associated schema used to program and build the user interface. It should be appreciated that the XML file 340 and associated schema file may be provided by other means, such as by a developer tool or developer application that a third party application developer may utilize to prepare modifications to the XML file 340 based on the associated XML schema file for application to the host application when loading a third party application add-in onto the host application.

According to another embodiment of the invention, the user interface 305 associated with a given host application may be customized at the document level. That is, the document-based user interface 305 may include customizations made to the user interface that are specific to a given document. For example, a user may create a customized document, such as a sales template document used by hundreds or thousands of sales on behalf of a large company. Because documents are intended to be used by many people, it is desirable for third party developers to be able to customize the user interface provided by a host application (e.g., a spreadsheet application) so that customized buttons or controls are presented in the user interface when the particular document is run.

For such a document-based solution, a file format may be used in which various functions and attributes associated with the document exist as related components held in a container file (container file). For example, the container file may include a component that represents user input data; another component representing a document structure (e.g., a template structure); another component that represents a format attribute applied to the document; another component that represents a document-based user interface customized for a document, and so on. A third party application developer may enter such a document container and access the XML representation file and associated XML schema of the user interface component applied to the document. Once the developer has accessed the XML file and associated XML schema of the user interface 305, the developer can customize the user interface by modifying the XML according to the modified XML schema, as described above.

According to another embodiment, the modified XML file 355 and associated XML schema file may be attached as a resource to a Component Object Model (COM) plugin provided by a third party application developer to provide functionality of the plugin application. When the COM plug-in is applied to a host application, e.g., a word processing application, the modified XML file 355 is spent by the host application to present the resulting modified user interface 307.

Referring back to FIG. 3, as described above, the third party software application plug-in is used to add a new user interface component to the existing user interface of a host application (e.g., a word processing application, a spreadsheet application, a slide presentation application, etc.). According to an embodiment of the present invention, a ab initio schema is provided in which the XML file 340 exposed to third party application developers removes all but the basic user interface components from the associated host application user interface. The associated XML schema then provides the third party application developer the ability to add back to the original user interface components (e.g., first tab 315, second tab 320, and logical group controls 325 and 330). In addition, third party application developers can then add new user interface components according to the associated schema file.

According to this embodiment, shortcuts are provided to allow all but the necessary user interface components to be removed from the host application user interface and to allow the add-in developer to have more controls in the customization of the user interface, since the add-in developer can start on the user interface of a virtual blank (blank slate) from which to build the customized user interface. According to one embodiment, such shortcuts are attributes of XML that, when set to "true," cause the removal of most or all of the original user interface components.

The add-in developer is restricted access to some areas of a given host application user interface 305 and some components of the user interface. That is, some areas and some components of the host application user interface are specified so that third party add-in developers cannot remove or modify these areas and components. For example, referring to the host application user interface 305 in FIG. 3, a file button 310 intended to provide some functionality across multiple applications (e.g., a word processing application, a spreadsheet application, a slide presentation application, etc.) cannot be modified, removed, or repurposed in any way by a third party add-in developer. In this way, the file button 310 may be restricted from access by third party add-in developers. Similarly, once user interface components are added to a given user interface, those components may be designated as restricted to prevent additional third party add-in applications from modifying those user interface components provided by previous add-ins.

According to one embodiment of the invention, if a third party add-in application is uninstalled from a host application, a custom XML file applied to the host application user interface and used to add or modify user interface components of the uninstalled add-in application is parsed and changes to the host application user interface associated with the add-in application are disabled so that when the associated add-in application is uninstalled, changes to the host application user interface are no longer presented in the user interface. Similarly, when a document-based solution that modifies an original graphical user interface is closed, the graphical user interface is presented such that changes to the graphical user interface associated with modification of the XML representation are not presented in the graphical user interface.

By accessing the XML file 340 representing the host application user interface 305, existing user interface components can be repurposed such that the repurposed components exhibit substantially different behaviors. For example, existing user interface components may be enabled, disabled, or specified as being associated with a different application action. For example, a third party application developer may desire that a given button (e.g., a "print" button) in the host application user interface be used only to print a document according to a predetermined print setting associated with the third party application add-in. As another example, in the event that a given function conflicts with the operation of a function added to the host application by the add-in application, the given function button or control will be disabled and rendered unusable.

FIG. 4 is a computer screen display of an example contextual user interface that may be modified according to embodiments of the present invention. Those skilled in the art will appreciate that the contextual user interface illustrated in FIG. 4 is for purposes of example only and is not intended to limit the variety of different layouts or content types that may be applied to and included in the contextual user interface provided in association with the selected document object. As such, the contextual user interface illustrated in FIG. 4 is not intended to limit or restrict the claimed invention in any way. For example, as described below, the contextual user interface shown in FIG. 4 is in the form of a user interface menu deployed in relation to the selected object. However, this embodiment of the invention may be implemented in other forms, such as adding buttons or controls (e.g., tabs) to the original user interface 305 that are deployed in the user interface when an object is selected in an electronic document.

Referring to FIG. 4, a document including an embedded picture object 410 is shown in a word processing application workspace. According to embodiments of the present invention, context menu 420 may be loaded adjacent or near a selected object by various methods, including, but not limited to, selecting a given document object, such as exemplary picture object 410. The context menu 420 includes selectable functionality controls related to editing a selected object in a selected document. That is, the context menu 420 is placed within one or more selectable functionality controls that may be used to edit a particular selected object in a selected document. For example, referring to the context menu 420 shown in FIG. 4, the context menu is loaded in the context of the selected picture object 410, and thus, selectable functionality controls, such as a paste control, a copy control, a position control, a reset picture control, and the like, provide the user with the functionality to edit properties of the selected picture object 410. It will be appreciated by those skilled in the art that if the context menu 420 is loaded in the context of another object type, the selectable functionality controls placed in the context menu 420 will be associated with the other object type.

By exposing the XML file 340 and associated XML schema for the contextual user interface 420, third party developers can add and modify user interface content contained in the contextual user interface 420 to create customized contextual user interface content and components associated with plug-in functionality to be applied to selected document objects, according to embodiments of the present invention. For example, referring to FIG. 4, a third party add-in application provides additional functionality for modifying or formatting a picture object 410 over that provided in the host application context user interface 420. As described above with reference to FIG. 3, a third party add-in developer can add, disable, repurpose, or modify user interface components in the contextual user interface 420 associated with the third party application add-in. For example, if the add-in application provides additional formatting functionality that may be used for the picture object 410, additional formatting buttons or groups of formatting buttons and/or controls can be added to the contextual user interface 420 for presentation in the contextual user interface 420 when the picture object 410 is selected for editing. As described above, instead of modifying the contextual user interface 420, new functionality controls (e.g., new tabs) may be added to the user interface 305 (as shown in FIG. 3) that will be deployed in the user interface 305 when associated objects are selected.

Alternatively, a new contextual user interface may be added to the host application in addition to modifying the existing contextual user interface. For example, if a given add-in application inserts one or more functions for a selected object, the XML file 340 and associated XML schema are exposed to create a new contextual user interface for the associated add-in function that will be loaded in association with the selected object. According to this embodiment, a start-from-scratch XML file 340 will be provided to allow for the creation and deployment of a new contextual user interface 420, as described above.

FIG. 5 is a computer screen display showing an example gallery or set of selectable controls 430 that may be modified according to embodiments of the present invention. It should be understood that the gallery of user interface controls 430 shown in FIG. 5 is for purposes of example only and is not intended to limit the variety of different user interface layouts or user interface content types that may be applied to and included in the gallery of user interface controls as described herein. Referring to FIG. 5, a pop-up image library is shown adjacent to context menu 420. It should be understood that the gallery of controls may be deployed in other configurations. For example, a gallery of controls according to embodiments of the present invention may be deployed in a straight line in the user interface 200, or the gallery may be deployed as a drop-down user interface under a selected button or control in the user interface 200. According to an embodiment, each control 435, 440, 445 contained in the gallery of controls represents one or more functions, such as formatting functions that would be applied to the selected object if a given control in the gallery were selected.

A given add-in application may add functionality to the host application to provide additional functionality (e.g., formatting properties) for a selected object and for which one or more additional controls are to be desired in the host application's gallery of controls. For example, referring to the example gallery of controls as shown in FIG. 5, a third party add-in application may provide formatting settings that enable automatic placement of an example picture object into the upper left hand corner of an example document and simultaneous application of different formats to text contained in the document containing the picture object. If desired, the add-in application developer may modify the host application control gallery user interface 430 to provide additional controls to the selected document and picture objects showing the application format properties of the add-in. As described above with reference to FIGS. 3 and 4, the author of the add-in application can specify the location in the control gallery where the new control should be placed. For example, the add-in developer may specify that a new control should be inserted after the "top left" control 440.

In addition to modifying the gallery of existing user interface controls, a new gallery of user interface controls 430 may also be added to the host application. For example, if a given add-in application adds one or more functionalities for a selected object, the XML file and associated XML schema are exposed for creating a gallery of new user interface controls for the associated add-in functionality to be loaded in association with the selected object. According to this embodiment, a start-from-scratch XML file 340 may be provided as described above to allow for the creation and deployment of a new contextual user interface 420.

After adding user interface components to the host application user interface as described above with reference to fig. 3, 4, and 5, the host application may automatically scale the size of the added user interface components as the display space for the modified user interface is reduced or increased, according to one embodiment of the present invention. For example, if a logical group of buttons or controls is added to the host application user interface and the logical group of buttons or controls includes three large buttons associated with add-in functionality, the host application automatically zooms the displayed buttons if the size of the window containing the user interface is reduced. For example, if the window containing the user interface is reduced in size such that the three large example keys cannot be displayed without pinching in the proximity of other user interface components, the three large keys would be replaced with three smaller versions of the three large keys. If the size of the window is further reduced, the smaller versions of the three keys are also removed altogether, and the small text identifications of the three functions associated with the three keys may be used in place of the three large keys.

The customized user interface component may be automatically refreshed by the host application when conditions affecting the customized user interface component change. For example, if a customized user interface component is added to the host application user interface 305 that provides a flag or picture associated with a given type of information (first data), the flag or picture will be disabled and automatically refreshed when the associated information or data changes. For example, a plug-in weather application may provide a button or control to the host application user interface that, when selected, provides a weather forecast for a specified area. The button or control in the user interface may be decorated with a picture associated with the current weather, e.g., a sunny picture for sunny weather, a cloudy picture for cloudy weather, etc. According to embodiments of the present invention, if the current weather conditions change, the picture or logo provided for the added button or control is no longer available, the current picture or logo is automatically disabled, and the button or control is refreshed with a different picture or logo than is applicable to the current information (e.g., the current weather conditions).

According to an embodiment, the host application keeps track of the identity of each added or modified user interface component associated with the software application plug-in responsible for the added or modified user interface component. According to one embodiment, when hovering or focusing over an added user interface element, such as hovering a mouse pointer over an added user interface element, a tooltip or other dialog is presented to identify the software application plug-in responsible for the added or modified user interface element, and to guide the user with other information describing or providing the software application plug-in associated with the added or modified user interface element as help content.

As described herein, software application user interface programming is exposed to allow modification of existing application user interfaces, including adding, removing, disabling, enabling, and repurposing new or existing user interface components associated with plug-in applications. It will be apparent to those skilled in the art that various modifications or variations can be made in the present invention without departing from the spirit and scope of the invention. Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein.

Claims (13)

1. A method for modifying a graphical user interface, comprising:

receiving a plug-in application providing one or more additional functionalities for a host software application on a computer, wherein an initial XML file represents an initial ribbon graphical user interface in a graphical user interface of the host software application, the initial ribbon graphical user interface including a first tab including a gallery containing a first set of selectable controls, each selectable control in the gallery including an image representing a functionality to be applied if that selectable control is selected, and an XML schema file providing modified XML syntax and syntax rules for the initial XML file;

receiving a modified XML file that includes modifications to the initial XML file according to the XML schema file and that represents a modified ribbon graphical user interface, wherein a gallery of the modified ribbon graphical user interface includes, in addition to the controls in the gallery of the initial ribbon graphical user interface, one or more new selectable controls that provide access to one or more additional functionalities provided by the add-in application, each of the new selectable controls including an image that represents a functionality that would be applied if the new selectable control were selected;

receiving, from a user, a selection of one object in the document displayed in the graphical user interface after receiving the modified XML file; and

in response to receiving the selection of the object, a contextual control is presented in a gallery of the ribbon-shaped graphical user interface, the contextual control being one of the one or more new selectable controls.

2. The method of claim 1, further comprising: upon receiving focus on one of the new selectable controls, displaying identification information in the modified graphical user interface regarding modification of the initial ribbon-shaped graphical user interface.

3. The method of claim 1,

wherein the initial ribbon graphical user interface comprises a contextual user interface for displaying a second set of one or more selectable controls applicable to the selected object; and

wherein the modified ribbon graphical user interface comprises one or more additional selectable controls in the contextual user interface.

4. The method of claim 1,

wherein, prior to receiving the modified XML file, the gallery in the initial ribbon graphical user interface provides a second set of one or more selectable controls that provide one or more functionalities for the selected object.

5. A method for modifying a graphical user interface, comprising:

displaying, by a computing device, a user interface for a host software application, the user interface comprising an initial ribbon graphical user interface;

receiving, at the computing device, a plug-in application that provides one or more additional functions to the host software application;

wherein an initial XML file represents the initial ribbon graphical user interface, the initial ribbon graphical user interface including a gallery providing a first set of one or more selectable controls, each selectable control in the gallery including an image representing a function to be applied if the selectable control is selected, and an XML schema file providing modified XML grammar and syntax rules for the initial XML file;

receiving, at the computing device, a modified XML file that includes modifications to the initial XML file according to the XML schema file and that represents a modified ribbon graphical user interface, wherein the modified XML file includes a first additional XML element that corresponds to a context control;

receiving, from a user, a selection of one of the objects in the document displayed in the user interface after receiving the modified XML file; and

in response to receiving the selection of the object, exposing, by the computing device, a contextual control in the gallery, the contextual control including an image representing a given functionality to be applied to the selected object if the contextual control is selected, the given functionality being one of the additional functionalities provided by the add-in application.

6. The method of claim 5, further comprising: receiving, at the computing device, a selection of the gallery.

7. The method of claim 5, further comprising:

presenting the initial ribbon graphical user interface in place of the modified ribbon graphical user interface in response to closing the document-based solution that modified the initial ribbon graphical user interface.

8. The method of claim 5, further comprising:

disabling a second set of one or more selectable controls in the modified ribbon graphical user interface.

9. A method for modifying a graphical user interface, comprising:

displaying, by a computing device, a user interface for a host software application, the user interface comprising a ribbon graphical user interface comprising a first tab comprising a first logical grouping of controls comprising a first set of one or more selectable controls for selecting one or more functions of the host software application;

providing an extensible markup language (XML) representation of the ribbon graphical user interface, the XML representation including a first XML element corresponding to a first tab of the ribbon graphical user interface and a second XML element corresponding to the first logical grouping of controls;

providing an XML schema file for providing XML syntax and syntax rules for the modification of the XML representation;

receiving a plug-in application that provides one or more additional functions to the host software application;

receiving a modification to the XML representation, the modification to the XML representation including a modification to the XML representation according to the XML schema file, wherein a first plug-in XML element is inserted into the XML representation, the first plug-in XML element corresponding to a plug-in control;

receiving a selection of one object in a document from a user after receiving a modification to the XML representation, the document being displayed in the user interface; and

in response to receiving the selection of the object, presenting, by the computing device, the plug-in control in the ribbon graphical user interface;

applying a given functionality to the selected object in response to a user selection of the add-in control, the given functionality being one of the additional functionalities provided by the add-in application;

when the plug-in application is uninstalled from the host software application, the ribbon graphical user interface is presented as it was presented prior to receiving the modification to the XML representation.

10. The method of claim 9, wherein:

wherein, after receiving the modification, the ribbon graphical user interface includes a second tab that includes one or more additional selectable controls associated with the one or more additional functionalities provided by the add-in application.

11. The method of claim 9, further comprising: after receiving the modification to the XML representation and upon receiving focus on a plug-in control contained in the ribbon graphical user interface, displaying an identification of the plug-in application.

12. The method of claim 9, wherein:

wherein the ribbon-shaped graphical user interface comprises a gallery that provides a second set of one or more selectable controls that provide one or more functionalities for the selected object;

wherein, after receiving the modification, the gallery includes one or more additional selectable controls in addition to the second set of selectable controls.

13. The method of claim 9, further comprising: when a document-based solution that modifies the ribbon graphical user interface is closed, the ribbon graphical user interface is presented as it appeared prior to opening the document-based solution.