HK1175558B - System and method for creating and manipulating data structures using an interactive graphical interface - Google Patents

Abstract

The present invention relates to a system, method and computer program for creating, visualizing and manipulating a data structure using an intuitive and interactive graphical interface. It is operable to display a data structure and enable users to interact with the data structure by means of a user interface. The data structure includes data entities and relationships between the data entities. One or more user context properties are associable with the data entities. The user interface is populated with data entities from the data structure by enabling the users to associate visual properties of the user interface with the user context associable properties of the data structure. This enables the users to interact with the data structure by means of the user interface.

Description

System and method for creating and manipulating data structures using an interactive graphical interface

Cross-referencing

This patent application claims priority from U.S. patent application No. 12/615,703 filed on 11/10/2009, which is incorporated herein by reference.

Technical Field

The present invention relates generally to an interface for creating and manipulating data structures. The invention more particularly relates to a system, method and computer program for creating, visualizing and manipulating data structures using an intuitive interactive graphical interface.

Background

Knowledge can be represented using a variety of types of data structures, including graphs. One such graph is called a semantic network. A semantic network is a directed graph composed of vertices representing concepts and edges representing semantic relationships between concepts.

Semantic networks can become unwieldy as they grow.

Tags can be used to represent concepts in a semantic network. In an appearance referred to as a "tag cloud," a set of tags can be used to represent relationships between concepts. The tag cloud is a familiar data presentation device on the internet. Tag clouds are often used to represent tags in a meaningful way, for example, to describe the popularity of tags in blogs and other internet resources to users. The properties of the word, such as size, thickness or color, may represent the properties of the underlying data. The tag cloud may be generated manually or by computerized means.

Fig. 1A to 1C show examples of tag clouds. For example, as shown in FIG. 1A, a cloud may include tags of several different sizes, where larger tags in the cloud represent more use of the tags in a blog. As shown in fig. 1B and 1C, for example, the size of the tag may indicate the importance of the concept relative to the context.

Information can be encoded in tag attributes (such as size, thickness, or color), but absolute and relative positions have virtually little meaning in the tag cloud. The tags are typically individual words, which may be sorted alphabetically or otherwise. The words may be aligned on a baseline or arranged in some other way, but this is usually done to save space or to obtain different visual effects.

These tag clouds are not interactive, however, because they merely represent information graphically without any measures for feedback from the user. There is no way for the tag cloud to manipulate the relationships between the various tags.

The prior art does not discuss ways of presenting a simple visual representation of a hierarchical or multi-hierarchical data structure, such as term classification or semantic web, in order to allow an ordinary computer user to create, visualize or manipulate the data structure. While manual entry of new concepts and relationships is contemplated in the prior art to a limited extent, the use of tag clouds to create concepts and automatically infer relationships to existing concepts represented by tags is not disclosed in the prior art. There is no disclosure of a convenient way to visualize and manipulate the relationships between the concepts represented by the tags. In other words, the prior art does not teach the use of tag clouds as input means for the represented semantic network or other data structure.

U.S. patent application 11/548,894 to Lewis et al discloses a cloud of tags presented to a user, where each tag may result in n layers of related information.

U.S. patent application 11/540,628 to Hoskinson discloses a tag cloud generated by a computer in response to a search query. The tags containing the topical representations or indicia are linked to the associated website from which the information used to generate the cloud was initially collected.

U.S. patent application 11/533,058 to Blanchard et al discloses customizing the display for a presented tag cloud. These clouds are customizable in their characteristics, such as font color, font size, boundaries, 3D depth, shading, and so forth. While all of these characteristic changes contribute to the visual display of the tags in the tag cloud, there is no corresponding substantial effect on the information represented by the tag cloud.

None of the previous applications discuss the way in which a data structure is presented to a user to visually represent relationships that may exist between concepts represented by tags and allow the user to manipulate the data structure with the tags.

PCT/US2007/025873 to Lindermann et al discloses allowing a user to input, store, and output concepts expressed as words or word combinations and relationships between these concepts in a graphical user interface. The user provides concepts to a thought engine (thoughtengine) and specifies the type or nature of relationships between the concepts. The users establish and share the generated semantic network.

Lindermann et al is directed to allowing users who do not understand structured data to insert data into a structure. The user explicitly inserts data and relationships and therefore must know how the relationships are formed. No automation is provided for establishing relationships based on the way a user views data. While Lindermann et al discusses users classifying the types of new relationships, they do not discuss a simple way to establish relationships with minimal user input. Nor does it discuss at all the way in which existing relationships between concepts in a semantic network can be easily manipulated.

Accordingly, there is a need for a means to allow an average computer user to create, visualize or otherwise manipulate data structures using a tag cloud.

Disclosure of Invention

The present invention provides a computer network implemented method for displaying a data structure and allowing one or more users to interact with the data structure through a user interface, the data structure comprising data entities and relationships between the data entities, wherein one or more user context attributes may be associated with a data entity, the method being characterized by: populating (post) the user interface with data entities from the data structure by allowing one or more users to associate one or more visual attributes of the user interface with attributes of the data structure that may be associated with a user context, thereby allowing one or more users to interact with the data structure through the user interface using one or more computer processors.

The present invention also provides a system for performing a computer-implemented method for displaying a data structure comprising data entities and relationships between the data entities, wherein one or more user context attributes may be associated with the data entities and allow one or more users to interact with the data structure, including creating the data structure, the system being characterized by one or more computer devices comprising or linked to: (a) at least one display; and (b) a user interface utility adapted to: (i) presenting a user interface to one or more users through the at least one display; and (ii) populating the user interface with data entities from the data structure by allowing one or more users to associate one or more visual properties of the user interface with properties of the data structure that may be associated with a user context, thereby allowing one or more users to interact with the data structure through the user interface.

The present invention also provides a computer program product containing executable computer program instructions which, when executed by one or more computers having a display, present a user interface corresponding to a data structure and allow one or more users to interact with the data structure, including creating the data structure, the data structure including data entities and relationships between the data entities, characterized in that one or more user context attributes may be associated with the data entities, the interaction with the data structure on the user interface including: populating the user interface with data entities from the data structure by allowing one or more users to associate one or more visual properties of the user interface with properties of the data structure that may be associated with a user context, thereby allowing one or more users to interact with the data structure through the user interface using one or more computer processors.

In this respect, before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and to the arrangements of the components set forth in the following description or illustrated in the drawings. The invention is capable of other embodiments and of being practiced and carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting.

Drawings

FIG. 1A illustrates a prior art tag cloud.

FIG. 1B illustrates another tag cloud of the prior art.

FIG. 1C illustrates another tag cloud of the prior art.

FIG. 2 illustrates the thought cloud of the present invention.

FIG. 3A illustrates inserting a new concept into a thought cloud.

FIG. 3B illustrates changing a label corresponding to a new concept of the thought cloud.

FIG. 3C illustrates a changed label corresponding to a new concept of the thought cloud.

FIG. 4A illustrates a label corresponding to an existing concept of the thought cloud.

FIG. 4B illustrates a measure for resizing a markup font in a thought cloud.

FIG. 4C illustrates a resized font of a markup.

FIG. 5A shows a label for a thought cloud that is about to change location.

Fig. 5B shows changing the position of the mark.

FIG. 5C shows the marker undergoing a change in position.

Fig. 6A illustrates measures for changing content related to a concept.

FIG. 6B illustrates content related to a concept.

FIG. 7A illustrates a thought cloud.

Fig. 7B shows the proposed concept shown in the thought cloud.

FIG. 8 illustrates one example of a hierarchical semantic network suitable for the present invention.

Detailed Description

Definition of

The data structure includes measures to organize and access information composed of knowledge representations based on relational links between data entities in the data structure.

Data entities mean knowledge representations given by data structures.

Relationships include hierarchical, multi-hierarchical, associative, equivalent, or other relationships between data entities in a data structure.

User context attributes mean any quantitative or qualitative aspect of the concept that a user would like to portray in a data structure utilizing the interface provided by the present invention, including, for example, relative topic importance, time, chronology (chronology), and/or physical characteristics such as temperature, pressure, and the like.

In the context of a semantic network, concepts include embodiments of data entities, where each concept may represent a concept of a user.

Active concepts mean concepts being considered.

In the generation of a semantic network, depth means the number of specific relationships at a distance from the active concepts within which the semantic network is generated or displayed.

Overview

The present invention provides a user interface to create, visualize and manipulate data structures. The user interface provides input means for a data structure. The data structure may include data entities and relationships between data entities. One or more user context attributes may be associated with the data entity such that, for example, the data structure may reflect a user context. In this regard, the user context may (but need not) result in the relationship between data entities being hierarchical, multi-hierarchical, associative, or equivalent.

The data entities are displayed through a user interface, allowing one or more users to interact with the data structure. The user interface allows a user to define a relationship between a new data entity and an existing data entity without having to understand the nature of the relationship in the underlying data structure and without having to explicitly specify the desired changes to the relationship.

The data structure may be a semantic network, wherein the data entities may be concepts. The semantic network may be provided by a semantic engine such as that provided in PCT/CA2009/000567 to Sweeney et al. The semantic network may be structured in a variety of ways, including hierarchical, multi-hierarchical, cyclic, or acyclic data structures.

The user interface is designed to allow a user to easily create, visualize and manipulate data structures. In particular, the data structures are represented in an easily understood and intuitive format and allow users to seamlessly manipulate relationships in a semantic network. The user does not need to understand the nature of the relationship in the underlying data structure or even have knowledge about the existence of relationships between any of its concepts. Instead, relationships are defined based on the way users think about each concept with respect to each other.

For example, where the data structure is a semantic network, then the user interface allows for the creation and management of a context or graphical representation of the semantic network. The user interface is populated with interface components that match or correspond to the concepts and relationships. The interface component is, for example, a label corresponding to a conceptual representation.

The user interface allows a non-professional computer user to interact with the semantic engine in order to generate and refine the semantic network (where, for example, relationships between concepts may be managed and concepts added, edited, or removed at one or more relationship levels in the semantic network).

Relationships are implicitly managed through user interaction with a user interface without requiring the user to explicitly define changes to the relationships in the semantic network. Tags or other interface components populated on the user interface are mapped to the relationships. Relationships are inferred from the mapping of concepts on the user interface managed by the user, which may reflect the manner in which the users think about each concept with respect to each other. The user can also create, edit, or delete concepts from the semantic network by manipulating the interface component. By managing such a context or graphical representation of the semantic network, the user in effect causes the present invention to automatically create or manipulate relationships of the semantic network.

The representation may include a plurality of dimensions within a visual space to map one or more of various attributes or characteristics about the data entity. The one or more attributes may be, for example, visual attributes including location, size, color, distance, typeface, underlining, contour, thickness, gradient, time, and any other visually significant characteristic.

The user interface may be understood as a thought cloud, i.e. a tag cloud in which the user's thought is represented. The thought cloud is adapted to display to the user a plurality of labels reflecting respective concepts that are related in the underlying semantic network, the concepts being arranged into labels in the thought cloud based on their relationships. In this manner, the relationships between the user context attributes of the underlying data structure and the entities in the data structure may be represented during the rendering process.

The user may interact with the user interface to further create, visualize, and manipulate the semantic network to manage the semantic network, for example, in a manner that reflects the way the users think about each other. Tools may be provided for the user by the user interface, including adding concepts to the semantic network and manipulating relationships in the semantic network. In this way, the user is not burdened with managing or even understanding the complexity of the corresponding data transformation form being processed according to the user's interaction with the semantic network. Regardless, there is no need for the user interface to explicitly display to the user the relationships between the concepts that make up the semantic network.

The present invention also includes a user interface for a semantic network in which a user is provided with a plurality of suggested tags representing concepts in the semantic network that are potentially related to one or more tags input to the user interface by the user. The semantic network may include concepts and relationships between concepts. The user may optionally approve suggested tokens, the mapping of which implicitly results in inferred relationships to other concepts in the semantic network, thereby allowing the creation of new relationships from existing concepts to the concepts represented by the suggested tokens.

In the following description, a semantic network is used as an exemplary data structure, and a thought cloud is used as an exemplary user interface for the semantic network. It should be understood that the data structure may be any data structure that includes data entities and relationships between data entities, where user context attributes may be associated with the data entities. It should also be understood that the term "thought cloud" is used merely for convenience.

Thought cloud

The present invention includes a user interface to create, visualize and manipulate data structures, such as semantic networks, that include concepts and hierarchical or multi-hierarchical relationships between concepts. The semantic network may be provided by a semantic engine, such as that provided in PCT/CA2009/000567 to Sweeney et al, for example, and may originate from active concepts. The user interface provides input measures for the semantic network that allow a user to manipulate relationships between new and existing concepts without having to understand the nature of the relationships in the underlying semantic network and without having to explicitly specify desired changes to the relationships.

The user interface allows for the creation of a contextual or graphical representation of the semantic network. The user interface may be a thought cloud. The idea cloud displays to the user one or more interface components, which may be, for example, labels, corresponding to concepts related in a hierarchical or multi-hierarchical manner in the underlying semantic network. The tag may have one or more attributes, where each attribute may be represented by context or graphical means. These properties allow each concept to be represented in an organized manner by the user interface. This allows the user interface to represent the relationships without explicitly displaying the relationships to the user, thereby avoiding the frustration of users who do not understand the existence nature of the relationships.

While the tag cloud is not interactive, the thought cloud is interactive. While a tag cloud has been used as an output device, a thought cloud may also be used as an input device to allow a user to create, visualize, and manipulate a semantic network. Tools may be provided for the user by the user interface, including adding concepts or modifying concepts in the semantic network, and measures through which hierarchical or multi-hierarchical relationships in the semantic network are manipulated without the user needing to know that they are doing so. In this way, the user is not burdened with managing or even knowing the complexity of the respective data transformation forms being processed according to the user's interaction with the semantic network.

Users of the thought cloud, for example, can move the tags relative to each other and change the attributes of the tags, resulting in changes to the underlying semantic network without exposing the users to the associated transformations. Thus, the thought cloud allows a user to capture his ideas on a computer by creating, setting, and manipulating tokens representing concepts in any way meaningful to the user. For example, there is no need for any particular alignment or setting of the markers in the thought cloud, as the markers can be set and reset as desired by the user.

Workflow example

A user may use the present invention to create, visualize and manipulate data structures, which may be, for example, semantic networks. For example, the following workflow is one specific use case of the present invention. It will be appreciated that other uses are possible, for example where the data structure is one other than a semantic network, or where the semantic network is generated entirely by the user rather than the speech engine.

The user may initially access the user interface and measures for entering active concepts and optionally also depth are provided by the user interface. The user may enter active concepts, which are obtained by the semantic engine. According to its implementation, the semantic engine generates a semantic network. The user interface displays to the user a thought cloud representing the semantic network. The thought cloud is optionally constrained to those concepts within the depth of the active concept (if the depth is specified by the user). It should be understood that the semantic engine may use depth to limit the generation of semantic networks from active concepts.

The user may then visualize the semantic network through the thought cloud. The user may manipulate (navigator) the thought cloud, for example, by panning through the thought cloud or zooming in and out of the thought cloud. The user can manipulate labels representing concepts displayed in the thought cloud. For example, a user may move one or more concepts on a user interface or may manipulate one or more properties of one or more concepts. Each manipulation is processed by the present invention, which automatically alters the semantic network according to transformation rules to reflect the user's manipulation of the concept.

The user may also add or delete concepts in the thought cloud. When adding a concept, a user may define attributes for the concept. Concepts and their attributes can be processed by the present invention, which automatically modifies the semantic network based on its specific attributes to reflect the addition of concepts according to transformation rules.

The user interface may provide the user with means by which to save the semantic network they create and manipulate. The user may maintain a plurality of semantic networks that he or she has created and manipulated, such as representing different historical points or dates. The user interface may also provide the user with means by which to restore any saved semantic networks.

Visualizing thought clouds

The user interface may be a screen or a portion of a screen used as a boundary for visualizing the thought cloud.

User interaction may be used to determine a starting point for initially establishing a thought cloud in order to prevent an awkward number of concepts from appearing in the thought cloud. For example, a semantic engine for generating a semantic network may be provided to receive user interaction information (context) from which active concepts may be derived. The semantic engine may be, for example, as described in PCT/CA 2009/000567. The active concepts may be considered as a starting point for visualizing a semantic network. A depth may be provided such that only those concepts in the semantic network that are relevant to the active concept within the depth are displayed by the user interface.

Alternatively, to cope with spatial constraints, the user interface may include the ability to change resolution, thereby mimicking motion in the z-direction by zooming in and out. The user interface may also provide the ability to pan vertically and horizontally to navigate through different areas of the thought cloud. When performing the swipe, concepts that happen to fall near (e.g., at the center of) a particular area of the user interface may be used as active concepts to further display the semantic network.

According to a set of transformation rules, concepts and relationships between concepts may be represented by one or more attributes or characteristics (e.g., including visual attributes). These properties may include x, y, and z coordinates, size, color, distance, typeface, underlining, contour, thickness, gradient, and any other visually significant property. Each of the attributes may be associated with an item of user context attributes of the underlying data structure (such as meaning of a concept or a relationship type defined by a transformation rule, including, for example, order, confidence, time, or popularity). These extensible properties allow the present invention to work with an arbitrary range of complex concepts and complex relationship types (including hierarchical, associative, or equivalent relationships). To associate the attributes of the underlying data structure with the attributes of the presentation and interaction, a number of transformation rules may be provided.

FIG. 2 illustrates a thought cloud in one particular example. The importance or depth (from active nodes or concepts) in a semantic network may be represented in the thought cloud as the size of the tokens, the parenthood may be represented as the relative position between concepts, with the closest larger concept being the parent, the order of sibling concepts may be represented by the distance from common origin or the distance from the parent concept, and the source of the concept (e.g., user-provided concept) may be represented by the color. It should be understood that the specific visual attributes used to represent the relationships between the various concepts in the thought cloud may vary.

For example, consider a semantic network that includes the hierarchical structure shown in FIG. 8. In this example, the concept with the largest word size is at the root level of the semantic network, while the concept with the smallest word size is at the leaf level. For any other concept, the more recent larger concept is its parent (e.g., a concept with a font size of 8 has its parent as the most recent concept with a size greater than 8).

As shown in fig. 2, for example, the active concept "miles davis" is a parent concept to "improvision", "trumpt", and "Jazz", since it is the most recent larger concept to each of these child concepts. Similarly, "Fusion" and "bebo" both have "Jazz" as parents, "Columbia" has "trumpt" as parents, and "Modal" has "improvement" as parents. It can also be seen that "Bebop" is more closely related to "Jazz" than "Modal".

The depth of the relationship to be displayed by the user interface may be constrained by the depth, as mentioned previously.

Thus, the provided user interface may be used to visualize hierarchical or multi-hierarchical semantic networks. In a strict hierarchy, each concept may have and only one parent (except for the root concept, which has no parents), while in a multi-hierarchy, each concept may have multiple parents. For example, a multi-hierarchy may be visualized by placing child concepts equidistant from a plurality of identically sized parent concepts.

Alternatively, these or other attributes may be divided into configurable units or ranges, as it may be difficult to set the concepts to be exactly equidistant or of the same size. For example, the actual attribute values may differ within a certain unit or range, but the semantic network may treat them the same. Alternatively, the distance and text size can be "snapped" to the nearest value on a configurable grid that will allow simpler placement of concepts.

Adding concepts

The invention includes the use of thought clouds as a means for users to create concepts in a semantic network and automatically generate correspondences in the semantic network. Once the thought cloud is surfaced by the user, the user may add concepts to the thought cloud, which are then reflected as new concepts in the semantic network. The new concepts will have relationships in the semantic network based on the user's configuration of the tagged attributes that represent the concepts. Likewise, the user need not understand that the attributes reflect the relationships of the concepts to other concepts in the semantic network.

Fig. 3 shows the addition of a new concept. The user may select a command to add a new concept that causes a new concept field to be displayed on the user interface. FIG. 3A illustrates a label for a new concept added to a user interface. The tag may be clearly marked as new to allow the user to easily determine that it is a new and configurable tag.

The user can create meaningful marks for the new concept, as shown in FIG. 3B. Once the user enters the token, the token may become part of the thought cloud, as shown in FIG. 3C. Once added, the attributes of one or more new concepts in the underlying semantic network are automatically generated based on the tagged attributes in the thought cloud, as described above. The user may further manipulate the generated attributes or create attributes for the mark.

Manipulation or deletion concepts

The present invention also includes a user interface that allows a user to manipulate concepts in a thought cloud, resulting in automatic management of concepts and their relationships in a semantic network.

For example, the user may change the attributes or location of new or existing tags. Fig. 4A shows a piece of thought cloud. The user may select a concept to be manipulated, for example, by hovering a mouse pointer over or clicking on a marker corresponding to the concept. Fig. 4B shows one selected marker. Once selected, a measure for configuring the attribute may be provided adjacent to the indicia to be configured, which may be, for example, a drop down list or text box listing all possible configurations of the attribute. The user may be given options including measures for configuring each of the attributes. The user may be presented with an option to delete the token altogether, which will remove the concept from the semantic network.

For example, the user may configure the font size of the markup. The user interface may provide up and down arrows to increase and decrease font size, respectively. The user may, for example, select a down arrow to reduce the font size of the mark and, once the user is satisfied with the change in the property, deselect the mark to apply the configuration, as shown in fig. 4C.

The location of the tags can be changed to alter the relationships between concepts of the semantic network. Fig. 5A shows a marker corresponding to a concept that a user wishes to move. The user may select and move the marker, for example, by clicking on the marker and dragging it to the desired location, as shown in fig. 5B. The user may deselect the marker in order to apply the configuration of the concept, as shown in fig. 5C.

User manipulation of tag attributes may be handled by automatically modifying relationships between concepts in the underlying semantic network according to transformation rules.

Similarly, concepts may also be removed from the semantic network. For example, the user may select a delete command to remove the concept. Concepts related to active concepts of the semantic network only via the removed concepts may also optionally be removed, but concepts having multi-level links to active concepts of the semantic network may only be severed with links passing through the removed concepts and other links not passing through the removed concepts may remain intact.

Similar processing may be provided in order to configure any property of the mark, e.g. geometrical properties including color, thickness, gradient, etc.

Once the configuration is applied, user manipulation of attributes can be handled by automatically manipulating concepts of the semantic network and relationships between concepts according to transformation rules.

In some data structures, particularly those that are not multi-tiered data structures, it may be desirable to resolve binding (tie) due to ambiguous manipulation of new tokens or existing tokens. The solution stitching may also be used with multi-hierarchy data structures, for example, to limit the number of multi-hierarchy relationships.

For example, in a semantic network, a concept placed equidistantly between two potential parent concepts may result in a binding. With the development of the underlying semantic network, the probability of stitching increases. Breaking the set of binding rules may thus be provided in order to resolve the binding. For example, when using an earlier set of sample transformation rules, the system may need to break the binding when determining parent-child relationships. If a concept has more probable concepts with the same shortest distance to it, the following criteria can determine which of them is the parent. Examples of breaking binding rule sets for this purpose may include in order: (1) the larger concept is the parent; (2) ordering some other established attribute about each concept, such as time of most recent change, confidence of relationship, distance from common origin, or alphabetical order (the attribute selected for this ordering can be any attribute that makes sense to a particular thought cloud); (3) a number of attributes generated or user-defined for the token corresponding to the concept; (4) if multi-hierarchy is not allowed, a random selection is made; (5) prompting the user to select a parent; (6) if multiple hierarchies are allowed and the tie is not broken by the previous first or second criterion, all candidate concepts may be considered parents of the child concept.

Continuing with the same example, the system may also need to break the binding when determining the sibling order. If two or more concepts are the same distance from a common origin (or the same distance from their parent concept if the parent concept is used instead of an origin), the following criteria may determine which sibling is in front of: (1) the larger concept is an older sibling; (2) ordering some other established attribute about each concept, such as time of most recent change, confidence of relationship, distance from common origin, or alphabetical order (the attribute selected for this ordering can be any attribute that makes sense to a particular thought cloud); (3) a random selection is made or the user is prompted to select a sibling order.

Creation and manipulation

Further, the concepts of the semantic network can be manipulated by adding detail to the concepts. For example, while the primary representation of a concept is its markup, the concept can also include content that is normally hidden from view, such as tags, annotations, graphics, links to audio or video, or other content.

For example, a user may select a marker for a concept of the semantic network that may result in measures for viewing content related to the concept, which may be presented, for example, by a drop down menu or a text box displaying the content or a portion thereof, as shown in FIG. 6A. The user may select to view the content, causing the content to be displayed, as shown in fig. 6B. The content may be displayed in an editable format, allowing a user to manipulate the content. Once the marker is deselected, the content may be hidden again and the content may be applied back to the semantic network. The user may also access measures for viewing the content for the purpose of creating content corresponding to the indicia.

Amplification of

The user interface may also provide the user with a plurality of suggested tags that represent concepts that are potentially related to one or more tags input to the user interface by the user and/or tags displayed to the user on the user interface. The user may select one or more of these entered and/or displayed tokens (which are directly or indirectly related in the underlying data structure) in order to display suggested tokens from the semantic engine.

The suggested tags may change based on one or more attributes of one or more tags input by the user, such as based on the location or size of the tag. The suggested tags may change based on changes to the one or more attributes and may change as other tags are added, deleted, or modified.

The user may optionally approve the suggested tags, allowing new relationships to be automatically generated from existing concepts to the concepts represented by the suggested tags. Likewise, a semantic network will be used as an exemplary data structure, but the following may be applied to any other data structure as well.

Concepts related to those represented in the thought cloud may be suggested by an associated semantic engine, such as the semantic engine described in PCT/CA 2009/000567. These concepts may be from another semantic network (created by the user or someone/something else), from one information domain, or from a combination thereof. The user may set a minimum and/or maximum value for some criteria (e.g., density or confidence) for including the concept in the suggested set. If the minimum and maximum values are set, the user can create a window for inclusion instead of a simple bar.

The relevant concepts may optionally be distinguished from the user's own concepts by using distinguishing measures, such as color, shading, opacity, gradient, or geometric properties. Consider, for example, the thought cloud of FIG. 7A. The markup for the related concepts can be presented in a panel of the user interface that is separate from the panel containing the user's original thought cloud. Alternatively, as shown in FIG. 7B, the labels for the user concepts may be displayed with different font properties than the labels for the suggested concepts.

Additional implementation

Instead of the proximity of tokens representing the closeness of relationship between concepts and the size representing the level of a concept in the semantic network, the method may be enhanced by representing different confidence levels in each concept with a gradient of text from deep to shallow for its tokens. If a concept suggested by the semantic networking system has a confidence of 0.1, its label may be shown in lighter text. If the user selects the concept, its confidence level may change to 1.0 and its label will become darker.

In addition to the thought cloud for semantic networking, other extensions to the user interface of the present invention for creating, visualizing, and manipulating data structures may be provided.

A different example may have the goal planning application translate x-coordinate to date, y to expected cost, and z (size) to importance. The user interface may include explicit axes corresponding to these physical dimensions to help the user interpret their meaning. Here, the date dimension may employ a text gradient parallel to the x-coordinate, with earlier targets having lighter labels.

A text gradient may also be applied to represent time for these examples. Thus, the gradient will represent time rather than confidence.

Claims (42)

1. A computer network implemented method for displaying a data structure and allowing one or more users to interact with the data structure through a user interface, the data structure comprising data entities and relationships between the data entities, wherein one or more user context attributes are associated with a data entity, the method characterized by:

populating the user interface with data entities from the data structure by allowing one or more users to associate one or more visual attributes of the user interface with attributes of the data structure associated with the user context, thereby allowing the one or more users to interact with the data structure through the user interface using one or more computer processors;

wherein the relationship is hierarchical or multi-hierarchical;

wherein a larger data entity having a greater number of visual attributes determines a higher level of hierarchy in the user interface and whereby operation by the user interface utility disassembles the multi-hierarchy relationship.

2. The method of claim 1, wherein one or more users modify visual attributes to interact with the data structure.

3. The method of claim 2, wherein the data structure is modified to provide a modified data structure that reflects visual attributes of the results of the user interaction, the visual attributes being displayed on the user interface.

4. A method according to claim 3, wherein the modified data structure is stored, and wherein the method comprises the further step of restoring the modified data structure to a particular historical point in time.

5. The method of claim 1, associating visual attributes of a user interface with interface components corresponding to data entities, and further comprising allowing one or more users to add or delete one or more interface components using the user interface and defining visual attributes for the one or more interface components, resulting in modifying a data structure to reflect the interface components.

6. The method of claim 5, wherein the interface component is a contextual or graphical indicia.

7. The method of claim 1, wherein the visual attribute comprises a size, and wherein a most recent larger sized data entity for another particular data entity represents a hierarchical relationship between the data entities.

8. The method of claim 2, wherein the visual attribute is selected from a group consisting of: location, size, color, distance, typeface, underline, contour, thickness, gradient, time, and any combination thereof.

9. The method of claim 1, wherein the data structure is a semantic network such that the data entities include concepts and the relationships include semantic relationships between concepts.

10. The method of claim 1, wherein the user interface is adapted to suggest a set of one or more additional data entities that are related to the one or more data entities displayed by the user interface.

11. The method of claim 10, wherein one or more users select one or more data entities displayed by the user interface to initiate the user interface to suggest one or more additional data entities.

12. The method of claim 10, further characterized by allowing one or more users to modify the visual attributes to interact with the data structure, wherein the user interface is adapted to suggest a second set of one or more additional data entities related to the data entities displayed by the modified user interface.

13. The method of claim 2, wherein the visual attributes can be accessed and manipulated by one or more users through a user interface.

14. The method of claim 1, wherein one or more users can create one or more visual attributes that have not been generated.

15. The method of claim 1, wherein for one or more data entities, one or more users can add or manipulate content selected from the group consisting of: a tag, an annotation, a graphic, a link to audio, a link to video, or any combination thereof.

16. The method of claim 7, wherein a multi-tiered relationship is represented on a user interface by data entities having at least two data entities of equal and larger size.

17. The method of claim 16, further comprising enforcing one or more rules to resolve a multi-tiered relationship based on: one or more visual attributes of the data entities involved in the relationship; a number of generated or user-defined visual attributes corresponding to respective data entities in the relationship; a confidence of the relationship; randomly selecting; selecting by a user; allowing binding; or any combination thereof.

18. A method as claimed in claim 1, wherein one or more users can browse data entities and relationships by panning through and/or zooming in and out of a user interface.

19. The method of claim 7, wherein a hierarchical relationship can be defined by a user or by a data structure.

20. The method of claim 9, wherein the user interface is adapted to suggest a set of one or more additional concepts related to the one or more concepts displayed by the user interface.

21. The method of claim 20, further characterized by allowing one or more users to modify the visual attributes to interact with the semantic network, wherein the user interface is adapted to suggest a second set of one or more additional concepts related to the concepts displayed by the modified user interface.

22. The method of claim 20, wherein one or more users can select any portion of the displayed concepts that are used to suggest one or more additional concepts.

23. The method of claim 22, wherein the selected portion of the displayed concepts are indirectly related.

24. A system for performing a computer-implemented method for displaying a data structure comprising data entities and relationships between the data entities, wherein one or more user context attributes are associated with the data entities and allow one or more users to interact with the data structure, including creating the data structure, the system characterized by one or more computer devices comprising or linked to:

(a) at least one display; and

(b) a user interface utility adapted to perform the following operations:

(i) presenting a user interface to one or more users through the at least one display; and

(ii) populating the user interface with data entities from the data structure by allowing one or more users to associate one or more visual attributes of the user interface with attributes of the data structure associated with the user context, thereby allowing the one or more users to interact with the data structure through the user interface;

wherein the relationship is hierarchical or multi-hierarchical;

wherein a larger data entity having a greater number of visual attributes determines a higher level of the hierarchy in the user interface, and whereby operation by the user interface utility disassembles the multi-hierarchy relationship.

25. The system of claim 24, wherein the data entity is represented by an interface component that includes a contextual or graphical indicia.

26. The system of claim 25, wherein the visual attributes include a size, wherein a most recent data entity of a larger size for another particular data entity represents a hierarchical relationship between the data entities.

27. The system of claim 26, wherein the hierarchical relationship between data entities is manipulated by changing a size of at least one of the data entities.

28. The system of claim 26, wherein the hierarchical relationship between data entities is manipulated by changing a distance between at least two data entities.

29. The system of claim 24, wherein the user interface utility is adapted to create a network of data entities on a user interface and initiate the network to be stored in a storage device linked to one or more computer devices.

30. The system of claim 29, wherein the data entities stored in the memory are retrieved as related networks.

31. The system of claim 26, wherein the system is adapted to map hierarchical relationships between data entities to a database of semantically synthesized conceptual relationships.

32. The system of claim 31, wherein visual attributes of the user interface are associated with data entities, and wherein the plurality of visual attributes appear in a drop down menu of a database of conceptual relationships derived from the semantic synthesis by operation of a user interface utility, such that modification of one or more visual attributes modifies a data structure.

33. The system of claim 31, wherein the user interface is adapted to present a drop-down menu of visual attributes to the user such that the visual attributes change if the hierarchical relationship between data entities changes.

34. The system of claim 31, wherein the system allows one or more users to add visual attributes to data entities to be recorded for the database.

35. The system of claim 24, adapted to allow one or more users to add and edit notes and graphics for data entities after clicking on an icon presented by the user interface.

36. The system of claim 26, wherein a multi-tiered relationship is represented on the user interface by data entities having at least two equally-spaced and equally-larger data entities.

37. The system of claim 24, wherein the user interface utility allows a user to scroll vertically or horizontally through the user interface to automatically adapt to an expanding network of data entities.

38. The system of claim 24, wherein the user interface utility allows one or more users to zoom in and out on a portion of a data entity.

39. The system of claim 26, wherein the hierarchical relationship is user-defined or defined by a data structure.

40. A system for performing a computer-implemented method for displaying a data structure comprising data entities and relationships between the data entities, wherein one or more user context attributes are associated with the data entities and allow one or more users to interact with the data structure, including creating the data structure, the system characterized by one or more computer devices comprising or linked to:

(a) at least one display; and

(b) a user interface utility adapted to perform the following operations:

(i) presenting a user interface to one or more users through the at least one display; and

(ii) populating the user interface with data entities from the data structure by allowing one or more users to associate one or more visual attributes of the user interface with attributes of the data structure associated with the user context, thereby allowing the one or more users to interact with the data structure through the user interface;

wherein the data entity is represented by an interface component comprising a context or graphical indicia;

wherein the visual attribute in which the relationship is hierarchical or multi-hierarchical comprises a size, wherein a most recent larger sized data entity for another particular data entity represents a hierarchical relationship between the data entities;

wherein a multi-hierarchical relationship is represented on the user interface by data entities having at least two data entities of equal and larger size;

where a larger data entity with a greater number of visual attributes determines a higher level of the hierarchy and thus the multi-hierarchy relationship is broken down by operation of the user interface utility.

41. A system for performing a computer-implemented method for displaying a data structure comprising data entities and relationships between the data entities, wherein one or more user context attributes are associated with the data entities and allow one or more users to interact with the data structure, including creating the data structure, the system characterized by one or more computer devices comprising or linked to:

(a) at least one display; and

(b) a user interface utility adapted to perform the following operations:

(i) presenting a user interface to one or more users through the at least one display; and

(ii) populating the user interface with data entities from the data structure by allowing one or more users to associate one or more visual attributes of the user interface with attributes of the data structure associated with the user context, thereby allowing the one or more users to interact with the data structure through the user interface;

wherein the data entity is represented by an interface component comprising a context or graphical indicia;

wherein the visual attribute in which the relationship is hierarchical or multi-hierarchical comprises a size, wherein a most recent larger sized data entity for another particular data entity represents a hierarchical relationship between the data entities;

wherein a multi-hierarchical relationship is represented on the user interface by data entities having at least two data entities of equal and larger size;

where a larger data entity with a higher confidence value determines a higher level of the hierarchy and thus the multi-hierarchy relationship is broken down by operation of the user interface utility.

42. A system for performing a computer-implemented method for displaying a data structure comprising data entities and relationships between the data entities, wherein one or more user context attributes are associated with the data entities and allow one or more users to interact with the data structure, including creating the data structure, the system characterized by one or more computer devices comprising or linked to:

(a) at least one display; and

(b) a user interface utility adapted to perform the following operations:

(i) presenting a user interface to one or more users through the at least one display; and

(ii) populating the user interface with data entities from the data structure by allowing one or more users to associate one or more visual attributes of the user interface with attributes of the data structure associated with the user context, thereby allowing the one or more users to interact with the data structure through the user interface;

wherein the data entity is represented by an interface component comprising a context or graphical indicia;

wherein the visual attribute in which the relationship is hierarchical or multi-hierarchical comprises a size, wherein a most recent larger sized data entity for another particular data entity represents a hierarchical relationship between the data entities;

wherein a multi-hierarchical relationship is represented on the user interface by data entities having at least two data entities of equal and larger size;

wherein the user interface utility allows one or more users to disarm the multi-tiered relationship by selecting one of the larger data entities to determine a higher level of the hierarchy.