CN100570549C - System and method for data modeling in a project-based storage platform - Google Patents

Abstract

Embodiments of the invention are directed to a data store (302) comprising items, elements, and relationships. An item is a data store unit in a data store (302) and further includes the element and the relationship. An element is an instance of a type that includes one or more fields. A relationship is a link between at least two items. The data store (302) also includes a core schema (340) that defines a set of core items that the hardware/software interface system understands and can directly process using a predetermined and predictable manner. The core items are derived from a common single base item, which is a base item in the base schema.

Description

Systems and methods for data modeling in an item-based storage platform

cross reference

The subject matter of this application is related to the invention disclosed in the following commonly-assigned application: SYSTEMS AND METHODS FOR REPRESENTING UNITS OFINFORMATION MANAGEABLE BY A HARDWARE/SOFTWARE INTERFACE SYSTEM BUT INDEPENDENT OF PHYSICAL REPRESENTATION, filed concurrently with this application Hardware/Software Interface System and Method for Information Units Managed Yet Independent of Physical Representation)" (Application No. Unassigned) (Attorney Docket MSFT-1748); filed concurrently with this application, entitled "SYSTEMS AND METHODS FOR SEPARATING UNITS OF INFORMATIONMANAGEABLE BY A HARDWARE/SOFTWARE INTERFACE SYSTEM FROMTHEIR PHYSICAL ORGANIZATION (system and method for separating information units manageable by hardware/software interface systems from their physical organization)" (application number unassigned ) (Attorney Docket No. MSFT-1749); filed concurrently with this application, entitled "SYSTEMS AND METHODS FORTHE IMPLEMENTATION OF A BASE SCHEMA FOR ORGANIZING UNITS OFINFORMATION MANAGEABLE BY A HARDWARE/SOFTWARE INTERFACE SYSTEM System and Method for Basic Modes of Information Units Managed by a Software Interface System)" (Application No. Unassigned) (Attorney Docket MSFT-1750); filed concurrently with this application, entitled "SYSTEMS AND METHODS FOR THE IMPLEMENTATION OF A CORE SCHEMA FOR PROVIDING A TOP-LEVELSTRUCTURE FOR ORGANIZING UNITS OF INFORMATION MANAGEABLE BYA HARDWARE/SOFTWARE INTERFACE SYSTEM )" (Application No. Unassigned) (Attorney Docket MSFT-1751); filed concurrently with this application and entitled "SYSTEMS A ND METHOD FOR REPRESENTINGRELATIONSHIPSBETWEEN UNITS OF INFORMATION MANAGEABLE BY AHARDWARE/SOFTWARE INTERFACE SYSTEM (System and Method for Representing Relationships Between Information Units Manageable by a Hardware/Software Interface System)" (Application No. Unassigned) (Attorney Docket No. MSFT-1752); filed concurrently with this application, U.S. Patent Application (Application Number Unassigned) (Attorney Docket No. MSFT-2733); and, filed concurrently with this application, entitled "STORAGE PLATFORM FOR ORGANIZING, SEARCHING AND SHARING DATA (Storage Platform for Organizing, Searching, and Sharing Data) "U.S. Patent Application (Application No. Unassigned) (Attorney Docket No. MSFT-2734).

technical field

The present invention relates generally to the field of information storage and retrieval, and more particularly to an active storage platform for organizing, searching and sharing different types of data in computerized systems.

Background of the invention

During the last ten years, the capacity of a single disk has increased by approximately seventy percent (70%) per year. Moore's Law has accurately predicted the incredible growth in central processing unit (CPU) power over the past few years. Wired and wireless technologies have provided massive amounts of connectivity and bandwidth. Assuming current trends continue, within a few years the typical laptop computer will have about a terabyte (TB) of storage and contain millions of files, with 500 gigabyte (500GB) drives becoming common.

Consumers use their computers primarily to communicate and organize personal information, whether they be traditional Personal Information Manager (PIM) style data or media such as digital music or photos. The volume of digital content and the ability to store raw bytes has grown enormously; however, the methods available to consumers to organize and unify this data have not kept pace. Knowledge workers spend a significant amount of time managing and sharing information, with some studies estimating that knowledge workers spend 15-25% of their time on activities related to invalid information. Another study estimates that the typical knowledge worker spends approximately 2.5 hours per day searching for information.

Developers and information technology (IT) departments invest significant time and money building their own data stores for common storage abstractions to represent things like people, places, times, and events. Not only does this result in duplication of effort, but it also creates silos of common data, with no mechanism for collectively searching or sharing that data. Just consider how many address books there are on computers running the Microsoft Windows operating system today. Many applications, such as email clients and personal finance programs, maintain their own address books, and there is little sharing between the applications of the address book data that each such program maintains separately. Thus, financial programs such as Microsoft Money do not share payer addresses with addresses maintained in email contact folders such as those in Microsoft Outlook. Indeed, many users have multiple devices between which their personal data should be logically synchronized and between various additional sources including mobile phone numbers to commercial services such as MSN and AOL; however collaboration on shared documents Much of this is done by attaching documents to email messages - which is manual and inefficient.

One reason for the lack of collaboration is that traditional methods of organizing information in computer systems have focused on using a file-folder-directory-based system ("file system") to divide more files organized into a directory hierarchy of folders. The Multics operating system, developed in the 1960s, is considered a pioneer in the use of files, folders, and directories to manage storable data units at the operating system level. Specifically, Multics uses symbolic addresses in the hierarchical structure of files (thus introducing the concept of file paths), where the physical address of a file is opaque to users (applications and end users). This file system takes no account of the file format of any individual file at all, and the relationships within and between files are considered independent at the operating system level (ie, distinct from the location of the files in the hierarchy). Thanks to Multics, storable data at the operating system level is organized into files, folders, and directories. These files typically include the file hierarchy itself ("directory") placed within a specific file maintained by the file system. This directory in turn maintains a list of entries corresponding to all other files in the directory and the node positions of those files in the hierarchy (here folders). This has been the state of the art for nearly 40 years.

However, while providing a reasonable representation of information residing in a computer's physical storage system, the file system is an abstraction of the physical storage system, and thus utilization of a file requires the user to process what (with context, characteristics, and relationships to other units) A layer of indirection (interpretation) between what the operating system provides (files, folders, and directories). As a result, users (applications and/or end users) have to force information units into file system structures, even if doing so is inefficient, inconsistent, or undesirable. Furthermore, existing file systems know very little about the structure of the data stored in individual files, and thus, most information remains closed within the files, only accessible (and understandable) by the applications that wrote that data. Therefore, the lack of a schema description of information and a mechanism to manage the information results in the formation of FIFO storage buffers (silos) of data, with very little data being shared between FIFO storage buffers. For example, many personal computer (PC) users have more than 5 disparate stores that contain information about the people they interact with at some level - such as Outlook contacts, online account addresses, Windows address book, Quicken payees and Instant Messaging (IM) buddy lists - because organizing files presents important challenges to these PC users. Since most existing file systems utilize a nested folder metaphor to organize files and folders, the effort necessary to maintain a flexible and efficient organizational model becomes overwhelming as the number of files increases. In these cases, having multiple catalogs for a single file is very useful; however using hard and soft links in existing file systems is cumbersome and difficult to maintain.

Several unsuccessful attempts have been made in the past to overcome the shortcomings of file systems. Some of these previous attempts have involved the use of content-addressable memory to provide mechanisms by which data can be accessed by content rather than by physical address. However, these efforts proved unsuccessful, and while content-addressable memory proved useful for small-scale use by devices such as cache memory and memory management units, it was not effective for large-scale Using is not yet possible for various reasons, so a solution like that simply doesn't exist. Other attempts have been made using object-oriented database (OODB) systems, but these attempts, while strongly database-like, and good non-file representations, are not efficient at handling file representations and cannot be reproduced on hardware/ Speed, efficiency, and simplicity of software interfaces to hierarchically structured files and folders at the system level. Other attempts, such as the attempt to use SmallTalk (and other derivatives), proved to be quite effective in dealing with file and non-file representations, but lacked the database features necessary to effectively organize and exploit the relationships that existed between the various data files, so The overall effectiveness of that system is unacceptable. Yet another attempt to use BeOS (and others such as Operating System Research), while capable of adequately representing files while providing some of the necessary database features, proved inadequate in dealing with non-file representations, which are traditional file The same core shortcomings of the system.

Database technology is another area of expertise that presents similar challenges. For example, although the relational database model has achieved great commercial success, in practice Independent Software Vendors (ISVs) typically utilize a small fraction of the functionality available in relational database software products such as Microsoft SQL Server. Instead, most of an application's interaction with such a product is in the form of simple "gets" and "puts". While there are several obvious reasons for this (such as being platform or database agnostic), a key reason that is often overlooked is that databases don't necessarily provide the precise abstraction that major business application distributors really need. For example, while the real world has concepts of "items" like "customers" or "orders" (and orders' embedded "row items" as items within them and the items themselves), relational databases only aspect to discuss. As a result, while applications may wish to have aspects of consistency, locking, security, and/or triggers at the item level (to name a few), typically databases only provide these features at the table/row level. Although it would work fine if each item mapped to a single row of a table in the database, in the case of orders with multiple row items, there is a reason for an item to actually map to multiple tables, and In this case, a single relational database system cannot exactly provide the correct abstraction. Therefore, applications must build logic on top of the database to provide these basic abstractions. In other words, the basic relational model does not provide an adequate platform for storing data on which advanced applications are easily developed, because the basic relational model requires a layer of indirection between the application and the storage system, which is only possible in certain cases of the application See the semantic structure of the data. Although some database vendors are building advanced functionality into their products (such as providing object-relational capabilities, new organizational models, etc.), none have yet provided the comprehensive solution needed, and where a truly comprehensive solution is useful for The domain abstractions (such as "person", "location", "event", etc.) provide solutions for useful data model abstractions (such as "item", "extension", "relationship", etc.).

In view of the aforementioned shortcomings in existing data storage and database technologies, there is a need for a new storage platform that provides an improved ability to organize, search and share all types of data in computer systems - a storage platform that It extends and augments the data platform beyond existing file systems and database systems, and is designed to store all types of data. The present invention fulfills this need.

Summary of the invention

The following summary provides an overview of various aspects of the invention. This overview is not intended to provide an exhaustive description of all important aspects of the invention, nor is it intended to define the scope of the invention. Rather, this summary is intended to serve as an introduction to the following detailed description and accompanying drawings.

The present invention is directed to a storage platform for organizing, searching and sharing data. The storage platform of the present invention expands and expands the concept of data storage beyond existing file systems and database systems, and is designed to store all types of data, including structured, unstructured or semi-structured data.

According to an aspect of the present invention, the storage platform of the present invention includes data storage implemented on a database engine. In various embodiments of the invention, the database engine includes a relational database engine with object-relational extensions. The data store implements a data model that supports organization, search, sharing, synchronization, and security of data. Concrete data types are described in schemas, and the platform provides a mechanism for extending the set of schemas to define new data types (essentially subtypes of the base types provided by the schemas). A synchronization capability facilitates the sharing of data between users or systems. Provides file system-like capabilities that allow interoperability of this data store with existing file systems without the limitations of such traditional file systems. A change tracking mechanism provides the ability to track changes to the data store. The storage platform also includes a set of application program interfaces, which enable applications to access all the above-mentioned capabilities of the storage platform, and to access the data described in the schema.

According to another aspect of the invention, the data model implemented by the data store defines the units of the data store in terms of items, elements and relationships. Items are units of data that can be stored in a data store and can include one or more elements and relationships. An element is an instance of a type that includes one or more fields (also referred to here as attributes). A relationship is a link between two items. (As used herein, these and other specific terms may be capitalized to separate them from other terms similarly used, however, no distinction is made between capitalized terms such as "Item" and non-capitalized terms same term, such as "item", and no such distinction should be assumed or implied.)

According to another aspect of the present invention, a computer system includes a plurality of items, each of which constitutes a discrete unit of storable information manipulable by a hardware/software interface system; a plurality of project folders, which constitute the an organizational structure; and a hardware/software interface system for manipulating a plurality of projects, and wherein each project belongs to at least one project folder, and may belong to more than one project folder.

According to another aspect of the invention, a computer system includes a plurality of items, wherein each item constitutes a discrete unit of information manipulable by a hardware/software interface system, and the item or certain item attributes are, as opposed to being derived from persistent storage, Values can be calculated dynamically. In other words, the hardware/software interface system does not require items to be stored, and supports certain operations, such as the ability to enumerate the current set of items, or to give an identifier for an item on the storage platform (described in Application Programming Interface or API described more fully in the section on ), the ability to retrieve items—for example, an item could be the current location of a cell phone, or the temperature read from a temperature sensor.

According to another aspect of the present invention, a hardware/software interface system for a computer system further includes a plurality of relationally interconnected items managed by the hardware/software interface system, wherein the hardware/software interface system manipulates a plurality of project. According to another aspect of the present invention, a hardware/software interface system for a computer system, wherein said hardware/software interface system manipulates a plurality of discrete information units having attributes understandable by said hardware/software interface system. According to another aspect of the present invention, a hardware/software interface system for a computer system includes a kernel schema to define that said hardware/software interface system can understand and directly perform in a predetermined and predictable manner A core set of items to work on. In accordance with another aspect of the present invention, a method for manipulating a plurality of discrete units of information ("items") in a hardware/software interface system for a computer system is disclosed, the method comprising combining the items with multiple Relationships are interconnected and the relationships are managed at the hardware/software interface system level.

According to another feature of the invention, the storage platform's API provides data classes for each item, item extension, and relationship defined in the storage platform's schema set. In addition, the application programming interface provides a set of framework classes that define a set of common behaviors for the data classes and, together with the data classes, provide the basic programming model for the storage platform API. In accordance with another feature of the invention, the storage platform API provides a simplified query model that enables application programmers to form queries based on data storage in a manner that isolates the application programmer from the details of the query language of the underlying database engine. Queries for various properties of an item. According to yet another aspect of the storage platform API of the present invention, the API collects changes made to items by applications and then organizes them into the correct updates required by the database engine (or any kind of storage engine) implementing the data store middle. This enables application programmers to make changes to items in memory, leaving the complexity of data store updates to the API.

Through its common storage base and schematized data, the storage platform of the present invention enables more efficient application development for consumers, knowledge workers and enterprises. It provides a rich and extensible application programming interface that not only makes available the capabilities inherent in its data model, but also includes and extends existing file system and database access methods.

Other features and advantages of the present invention will become apparent by reading the following detailed description of the invention and accompanying drawings.

Brief description of the drawings

The foregoing general description, as well as the following detailed description of the invention, are better understood when read in conjunction with the accompanying drawings. Exemplary embodiments of various aspects of the invention are shown in the drawings for purposes of explaining the invention; however, the invention is not limited to the precise methods and instrumentalities disclosed. In the attached picture:

Figure 1 is a block diagram representing a computer system in which aspects of the present invention may be incorporated;

2 is a block diagram showing a computer system divided into three component groups: hardware components, hardware/software system interface components, and application components;

Figure 2A shows a conventional tree-based hierarchical structure for files grouped into folders within directories in a file-based operating system;

Figure 3 is a block diagram illustrating a storage platform in accordance with the present invention;

Fig. 4 shows the structural relationship between projects, project folders and categories in various embodiments of the present invention;

Figure 5A shows a block diagram of the structure of the project;

Figure 5B is a block diagram illustrating complex attribute types for the item of Figure 5A;

Figure 5C is a block diagram showing the "Location (position)" item, wherein its complex type is further described (explicitly listed);

Figure 6A shows items that are subtypes of items found in the base schema;

Fig. 6B is a block diagram showing the subtype item of Fig. 6A, which explicitly lists the types it inherits (in addition to its immediate attributes);

Figure 7 is a block diagram showing the base schema, including its two top-level class types, Item (item) and PropertyBase (property base), and additional base schema types derived therefrom;

FIG. 8A is a block diagram illustrating items in a core schema;

Figure 8B is a block diagram illustrating attribute types in a core schema;

9 is a block diagram illustrating a project folder, its member projects, and the interconnection relationship between the project folder and its member projects;

Figure 10 is a block diagram showing a category (which is itself an item), its member items, and the interconnection between the category and its member items;

FIG. 11 is a diagram illustrating a reference type hierarchy of a data model of a storage platform according to the present invention;

Figure 12 is a diagram illustrating how relationships are classified according to one embodiment of the invention;

Figure 13 is a diagram illustrating a notification mechanism according to one embodiment of the present invention;

Figure 14 is a diagram showing an example where both transactions insert new records into the same B-tree;

Figure 15 shows a data change detection process according to one embodiment of the present invention;

Figure 16 shows an exemplary directory tree;

17 shows an example in which folders of an existing directory-based file system are moved into the storage platform data store according to an aspect of the present invention;

Figure 18 illustrates the concept of containing folders in accordance with an aspect of the invention;

Figure 19 shows the basic architecture of the storage platform API;

Figure 20 schematically represents the various components of the storage platform API stack;

21A and 21B are graphical representations of exemplary contact schemas (items and elements);

Figure 22 shows the runtime framework of the Storage Platform API in accordance with an aspect of the invention;

Figure 23 illustrates execution of a "FindAll" operation in accordance with one embodiment of the invention;

Figure 24 illustrates a process for generating storage platform API classes from a storage platform schema in accordance with an aspect of the invention;

Figure 25 shows the schema on which the File (file) API is based according to another aspect of the present invention;

Figure 26 is a schematic diagram illustrating an access mask format for data security purposes according to one embodiment of the present invention;

Fig. 27 (a), (b), (c) have provided a new security zone that protects equally from existing security zone according to an embodiment of an aspect of the present invention;

28 is a schematic diagram illustrating the concept of an item search view according to one embodiment of an aspect of the present invention;

Figure 29 is a diagram illustrating an exemplary project hierarchy according to one embodiment of the present invention;

Detailed description of the invention

Table of contents

A. Exemplary Computing Environment ................................................ .16

B. Traditional File-Based Storage .................................19

II. New Storage Platforms for Organizing, Searching, and Sharing Data................................20

A. Glossary................................................ ......20

B. Storage Platform Overview ................................................ ..twenty one

C. Data Model ................................................ .....twenty two

1. Items................................................ ........twenty three

2. Item Identification ................................................ ......26

a) Project references................................................... ......26

(1)ItemIDReference................................................26

(2)ItemPathReference................................................26

b) Reference Type Hierarchy ................................................... 27

3. Project folders and categories ................................................ 27

4. Mode................................................ ..........28

a)Basic mode................................................ ......28

b) Kernel mode................................................... ......29

5. Relationships ................................................ ..........30

a) Statement of Relationship .............................................. ......31

b) hold a relationship ................................................ ......32

c)Embedding relationship................................................ ......33

d) Citation relationship................................... ......33

e) Rules and Constraints ................................................ ....34

f) Ordering of relations ................................................ ....34

6. Scalability................................................ ......38

a)Project Expansion................................................ ......39

b) Extended NestedElement type....................................42

D. Database Engine ................................................ ...43

1. Implementation of data storage using UDT...................................................44

2. Project Mapping................................................ ..45

3. Extended Mapping................................................ ..47

4. Nested Element Mapping................................................... ...48

5. Subject identity ................................................ ..48

6. SQL Object Naming................................................ ...49

7. Column naming ................................................ ......50

8. Search View ................................................ ...50

a) Items................................................... ......50

(1) Main item search view................................................... 51

(2) Typed item search view...................................51

b)Project Expansion................................... ....52

(1) Main Extended Search View ................................................ 52

(2) Typed Extended Search View...................................52

c) Nested elements................................................ ....53

d) Relationship ................................................ ......53

(1) Main relational search view................................................... 53

(2) Relation instance search view....................................53

9. Update................................................ ......54

10. Changing Tracking and Tombstones....................................54

a) Change Tracking ................................................ ....55

(1) Change Tracking in the "Main" Search View .................................. 55

(2) Change Tracking in "Typed" Search Views ................................................................ 56

b) Tombstone................................................ ......56

(1) Project Tombstone................................... ....57

(2) Extended tombstone................................................ ....57

(3) Relationship tombstone................................... ....57

(4)Tombstone removal................................... ....58

11. Helper API and functions................................................... 58

a) Function [System.Storage].GetItem................................58

b) Function [System.Storage].GetExtension.............58

c) Function [System.Storage].GetRelationship..........59

12. Metadata ................................................ ......59

a) Schema Metadata ................................................ ...59

b) Instance Metadata ................................................ ...59

E.Security................................................ ......59

1. Overview................................................ ......59

2. Detailed description of the security model...................................63

a) Security Descriptor Structure ................................................ 64

(1) Access mask format................................................... ..65

(2) Generic access rights................................................... ..65

(3) Standard Access Rights................................................... .65

b) Project-specific permissions....................................66

(1) Special permissions for file and directory objects....................66

(2)WinFSItemRead................................................67

(3)WinFSItemReadAttributes................................68

(4)WinFSItemWriteAttributes................................68

(5)WinFSItemWrite...................................................68

(6)WinFSItemAddLink...................................69

(7)WinFSItemDeleteLink...................................69

(8) Permission to delete items...................................69

(9) Permission to copy items...................................69

(10) Permission to move items....................................70

(11) Permission to view the security policy on the project..................... 70

(12) Permission to change the security policy on the project................... 70

(13) Privilege with No Direct Equivalent ................................. 70

3. Realization................................................ ......71

a) Create a new project in the container....................................71

b) Adding an explicit ACL to the project...................................71

c) Adding Holder Relationships to Projects .................................. 72

d) Deleting a Holding Relationship from a Project .................................................. 72

e) Removing explicit ACLs from the project....................................72

f) Modifying the ACL associated with the project....................................72

F. Notification and Change Tracking .................................73

1. Storage change event.....................................73

a) Events................................................ ......73

b) Monitoring program ................................................ ..74

2. Change Tracking and Notification Generation Mechanisms .................................75

a) Change Tracking ................................................ ..76

b) Time Stamp Management .................................. 77

c) Data Change Detection - Event Detection .................................77

G.Synchronization................................................ ......78

1. Synchronization from storage platform to storage platform....................78

a) Synchronous (Sync) Control Application ........................ 79

b) Schema Comments ................................................ ..79

c) Synchronization Configuration ................................................ ..80

(1) Community Folder - Mapping..................................................81

(2) Overview ................................................. 81

(3) Schedule....................................82

d) Conflict handling....................................82

(1) Conflict detection....................................83

(a) Knowledge-Based Conflict .................................. 83

(b) Constraint-Based Conflicts ................................. 83

(2) Conflict handling....................................83

(a) Automatic Conflict Resolution ................................. 84

(b) Conflict Logging ................................. 84

(c) Conflict Checking and Resolution ................................. 85

(d) Convergence of Replicas and Propagation of Conflict Resolution... 85

2. Synchronization of data storage on non-storage platforms.....................85

a) Synchronization Service....................................86

(1) Changing the enumeration.................................................86

(2) Changing the application....................................87

(3) Conflict resolution....................................87

b) Adapter implementation ................................. 87

3. Security.................................................88

4. Manageability.................................................88

G. Legacy Documentation Interoperability .................................88

1. Interoperability Model .................................89

2. Data Storage Characteristics .................................90

a) Non-volume................................................90

b)Storage structure...................................90

c) Not Migrating All Files ................................. 90

d) NTFS Namespace Access to Storage Platform Files 91

e) Expected Namespace/Drive Letter ................................................91

I. Storage Platform API...................................91

1. Overview ................................................. 91

2. Nomenclature and Scope...................................92

3. Storage Platform API Components ................................. 94

4. Data classes .................................................94

5. Runtime Architecture .................................. 101

a)Runtime architecture class....................................101

(1)ItemContext....................................101

(2)ItemSearcher....................................102

(a) Target Type.................................................102

(b) Filter ................................................. 102

(c) Preparing to search....................................103

(d) Search Options....................................103

(3) Item Result Stream ("FindResult")...................................104

b) Runtime Architecture in Operation ................................................. 105

c) Common programming mode....................................106

(1) Opening and closing the ItemContext object....................106

(2) Search object....................................107

(a) Search Options....................................108

(b) FindOne and FindOnly....................................108

(c) Search for shortcuts on ItemContext ................................. 109

(d) Search by ID or path....................................109

(e) GetSearcher mode....................................110

(3) Updating storage....................................110

6. Security ................................................. 112

7. Support for Relationships .................................112

a) Basic relationship type....................................113

(1)Relationship class....................................113

(2) ItemReference class....................................114

(3) ItemIdReference class....................................114

(4) ItemPathReference class....................................115

(5) RelationshipId structure....................................115

(6)VirtualRelationshipCollection class...................116

b) Generated relationship types....................................117

(1) Generated relation types....................................118

(2) RelationshipPrototype class....................................118

(3)RelationshipPrototpyeCollection class... 119

c)Relationship support in the Item class.................................................119

(1) Item class....................................................119

(2)RelationshipCollection class....................................119

d) Relational Support in Search Expressions .................................. 120

(1) From item traversal to relationship....................................120

(2) From relationship traversal to items....................................120

(3) Combination relationship traversal....................................121

e) Example use of relational support ..................................121

(1) Search relationship.................................................121

(2) Navigating from the relationship to the source and target items...................................122

(3) Navigating from the source project to the relationship ................................. 123

(4) Create relationships (and items)...................................124

(5) Deleting relationships (and items) ................................................. 125

8. "Extending" the storage platform API....................................125

a) Domain Behavior..................................................126

b) Value-Adding Behavior ................................. 126

c) Value-Added Behavior as a Service Provider ..................................127

9. Design-Time Architecture .................................128

10. Inquiry form....................................128

a) Filter Basics .................................129

b) Type coercion....................................130

c) Filter Syntax .................................. 130

11. Remote Control ............................................. 131

a) Local/Remote Transparency in API ................................................. 131

b) Realization of remote control storage platform....................132

c)Accessing non-storage platform storage...................................132

d) Relationship to DFS..................................................132

e) Relationship with GXA/Indigo...................................132

12. Constraints ................................................. 133

13. Sharing ................................................. 134

a) Indicates sharing ................................................. 135

b) Manage Shares ................................. 135

c) Accessing Shares.................................................. 135

d) Discoverability ................................................. 136

14.Semantics of Find....................................136

15. Storage Platform Contacts API....................................136

a) Overview of System.Storage.Contact ................................................. 137

b) Domain Behavior..................................................137

16. Storage Platform File API....................................138

a) Introduction ................................................. 138

(1) Reflecting NTFS volumes in the storage platform....................139

(2) Create files and directories in the storage platform name space... 139

b) File mode ................................................ .140

c) Overview of System.Storage.Files...................................140

d) Code example................................................ .140

(1) Open the file and write to it...................................140

(2) Use query................................................ .141

e) Domain Behavior ................................................ ...141

J.Summary................................................ ....141

I. Introduction

The subject matter of the invention is described in detail to satisfy statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors contemplate that the claimed subject matter can be implemented in other ways as well, to include different steps or combinations of steps similar to those described in this document, in conjunction with other present and future technologies. Furthermore, while the term "step" may be used herein to refer to various elements of the method employed, the term should not be construed to imply a specific order between the steps disclosed herein unless an explicit description of the various steps is explicitly described. order.

A. Exemplary Computing Environment

Many embodiments of the invention can be implemented on computers. Figure 1 and the following discussion are intended to provide a brief description of a suitable computing environment in which the invention may be implemented. Although not required, aspects of the invention can be described in the general context of computer-executable instructions, such as program modules, being executed on a computer, such as a client workstation or server. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In addition, the invention may be practiced with other computer system configurations, including handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframes, and the like. The invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in local or remote memory storage devices.

As shown in FIG. 1 , an exemplary general purpose computing system includes a conventional personal computer 20 or the like including a processing unit 21 , a system memory 22 and a system bus 23 coupling various system components including the system memory to the processing unit 21 . System bus 23 may be any of several bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. System memory includes read only memory (ROM) 24 and random access memory (RAM) 25 . The basic input/output system 26 (BIOS), which contains the basic routines that help transfer information between the elements of the personal computer 20, such as at start-up, is stored in the ROM 24. The personal computer 20 may also include a hard disk drive 27 for reading and writing a hard disk (not shown), a magnetic disk drive 28 for reading and writing a removable magnetic disk 29, and an optical disk drive 30 for reading and writing a removable optical disk 31 such as a CDROM or other optical media. Hard disk drive 27, magnetic disk drive 28, and optical disk drive 30 are connected to system bus 23 through hard disk drive interface 32, magnetic disk drive interface 33, and optical drive interface 34, respectively. The drives and their associated computer-readable media provide nonvolatile storage of computer-readable instructions, data structures, program modules and other data for the personal computer 20 . Although the exemplary environment described here employs a hard disk, removable magnetic disk 29, and removable optical disk 31, those skilled in the art will understand that other types of computers that can store data that can be accessed by the computer can also be used in the exemplary operating environment. Readable media such as cassette tapes, flash memory cards, digital video disks, Bernoulli cassettes, random access memory (RAM), read only memory (ROM), etc. Similarly, the example environment may also include many types of monitoring equipment, such as thermal and security or fire alarm systems, and other sources of information.

Several program modules can be stored on a hard disk, magnetic disk 29, optical disk 31, ROM 24 or RAM 25, including an operating system 35, one or more application programs 36, other program modules 37, and program data 38. A user can enter commands and information into personal computer 20 through input devices such as keyboard 40 and pointing device 42 . Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, and the like. These and other input devices are often connected to processing unit 21 through a serial interface 46 coupled to the system bus, but may also be connected through other interfaces, such as a parallel port, a game port or a universal serial bus (USB). A monitor 47 or other type of display device is also connected to system bus 23 through an interface such as video adapter 48 . In addition to the monitor 47, personal computers typically include other peripheral output devices (not shown), such as speakers and a printer. The example system of FIG. 1 also includes a host adapter 55 , a small computer system interface (SCSI) bus 56 , and an external storage device 62 connected to the SCSI bus 56 .

Personal computer 20 may operate in a network environment using logical connections to one or more remote computers, such as remote computer 49 . Remote computer 49 may be another personal computer, server, router, network PC, peer-to-peer device, or other common network node, and typically includes many or all of the elements described above for personal computer 20, although only shown in FIG. Out of the memory storage device 50. The logical connections depicted in FIG. 1 include a local area network (LAN) 51 and a wide area network (WAN) 52 . Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

When used in a LAN network environment, the personal computer 20 is connected to the LAN 51 through a network interface or adapter 53. When used in a WAN networking environment, the personal computer 20 typically includes a modem 54 or other means for establishing communications over a wide area network 52, such as the Internet. A built-in or external modem 54 is connected to the system bus 23 through the serial port interface 46 . In a network environment, program modules depicted relative to the personal computer 20, or portions thereof, may be stored in the remote memory storage device. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

As shown in the block diagram of FIG. 2 , computer system 200 can be roughly divided into three component groups: hardware components 202, hardware/software interface system components 204, and application program components 206 (also referred to in some contexts herein as "user components"). Component" or "Software Component").

1, in various embodiments of computer system 200, hardware components 202 may include central processing unit (CPU) 21, memory (ROM 24 and RAM 25), basic input/output system (BIOS) 26, and various Input/output (I/O) devices, such as keyboard 40, mouse 42, monitor 47, and/or printer (not shown), etc. Hardware components 202 comprise the basic physical infrastructure of computer system 200 .

Application program components 206 include various software programs including, but not limited to, compilers, database systems, word processing programs, business programs, video games, and the like. Applications provide the means by which computer resources are used to solve problems, provide solutions, and manipulate data for various users (machines, other computer systems, and/or end users).

The hardware/software interface system component 204 includes (and in some embodiments may include only) the operating system, which in most cases itself includes a shell and a kernel. An "operating system" (OS) is a special program that acts as an intermediary between application programs and computer hardware. The hardware/software interface system component 204 may also include a virtual machine manager (VMM), a common language runtime (CLR) or its functional equivalent, a Java virtual machine (JVM) or its functional equivalent, or in a computer system In place of the operating system or other software components in addition to the operating system. The purpose of a hardware/software interface system is to provide an environment in which a user can execute an application. The goal of any hardware/software interface system is to facilitate the use of a computer system and to utilize computer hardware in an efficient manner.

A hardware/software interface system is typically loaded into a computer system at startup and then manages all application programs in the computer system. Applications interact with the hardware/software interface system by requesting services through an application programming interface (API). Certain applications enable an end user to interact with a hardware/software interface system through a user interface such as a command language or a graphical user interface (GUI).

Hardware/software interface systems traditionally perform various services for applications. In a multitasking hardware/software interface system where multiple programs run simultaneously, the hardware/software interface system determines which applications should run in what order and how much each application is allowed to run before switching to another application for its turn time. The hardware/software interface system also manages the sharing of internal memory among multiple applications and handles input and output to and from attached hardware devices such as hard disks, printers, and dial-up ports. The hardware/software interface system also sends messages to each application (and in some cases to the end user) about the status of the operation and any errors that may have occurred. The hardware/software interface system can also offload the management of batch jobs (such as printing), so that the launched application can get rid of the job and resume other processing and/or operations. On computers capable of parallel processing, the hardware/software interface system also manages partitioning the program so that it runs on multiple processors simultaneously.

The hardware/software interface system shell (referred to herein as "the shell") is the interactive end-user interface to the hardware/software interface system. (The shell is also known as the "command interpreter", or in operating systems, the "operating system shell"). A shell is the outer layer of a hardware/software interface system that is directly accessible by applications and/or end users. In contrast to the shell, the kernel is the innermost layer of the hardware/software interface system that directly interacts with hardware components.

While many embodiments of the invention are envisioned to be particularly applicable to computerized systems, nothing in this document is meant to limit the invention to those embodiments. Rather, the term "computer system" as used herein is intended to include any and all devices capable of storing and processing information and/or using stored information to control the behavior or performance of the device itself, whether or not those devices are electronic in nature, mechanical, logical, or virtual.

B. Traditional file-based storage

In most computer systems today, a "file" is a unit of storable information, which may include hardware/software interface systems and applications, data sets, etc. In all modern hardware/software interface systems (Windows, Unix, Linux, MacOS, virtual machine systems, etc.), a file is the fundamental discrete (storable and retrievable) unit of information that can be processed by the hardware/software interface system. Groups of files are usually organized into "folders". In Microsoft Windows, Macintosh OS, and other hardware/software interface systems, a folder is a collection of files that can be retrieved, moved, and manipulated as a single unit of information. These folders are in turn organized into tree-based hierarchical arrangements called "directories" (discussed in more detail later). In other hardware/software interface systems such as Dos, z/OS, and most Unix-based operating systems, the terms "directory" and/or "folder" are used interchangeably, early Apple computer systems such as the Apple IIe) The term "category" is used in place of a directory; however, as used herein, all these terms are considered synonymous and used interchangeably, and are intended to also include reference to hierarchical information storage structures and their folders and files References to all other equivalent terms for components.

Traditionally, directories (aka directories of folders) are tree-based hierarchies, where files are grouped into folders, which in turn are arranged by relative node positions that make up the directory tree. For example, as shown in FIG. 2A, a base folder (or "root directory") 212 of a DOS-based file system may include a plurality of folders 214, each of which may also include additional folders (such as the "root directory" for a particular folder). subfolders") 216, and each of these includes additional folders 218, up to infinity. Each of these folders may have one or more files 220, although at the hardware/software interface system level, the individual files in the folder have little in common other than their location in the tree hierarchy. Not surprisingly, the method of organizing files into a file hierarchy indirectly mirrors the physical organization of typical storage media (eg, hard disks, floppy disks, CD-ROMs, etc.) used to store these files.

In addition to the above, each folder is a container for its subfolders and its files - that is, each folder owns its subfolders and files. For example, when a folder is deleted by the hardware/software interface system, the folder's subfolders and files are also deleted (and in the case of each subfolder recursively including its own subfolders and files). Also, each file is typically owned by only one folder, and while a file can be copied and the copies are in different folders, the copies of the file are themselves distinct and independent units that have no direct connection to the original file (e.g. Changes at the hardware/software interface system level are not reflected in the copy file). Files and folders are therefore "physical" in nature in this respect, in that folders are treated like physical containers, whereas files are treated as distinct and separate physical elements within those containers.

II. A new storage platform for organizing, searching and sharing data

The present invention is directed to a storage platform for organizing, searching and sharing data. The storage platform of the present invention extends and broadens the data platform beyond the categories of file systems and database systems discussed above, and is designed to store all types of data, including new forms of data called items.

A. Glossary

The terms used here and in the claims have the following meanings:

An "item" is a unit that can store information accessible to the hardware/software interface system, unlike a simple file, it is an object with a basic set of attributes commonly supported by all objects that are presented to the end user by the hardware/software interface system shell. Items also have attributes and relationships supported common to all item types, including features that allow the introduction of new attributes and relationships (discussed in detail below).

An "operating system" (OS) is a special program that acts as an intermediary between application programs and computer hardware. In most cases, an operating system consists of a shell and a kernel.

A "hardware/software interface system" is software, or a combination of hardware and software, that functions as an interface between the underlying hardware components of a computer system and application programs executing on the computer system. A hardware/software interface system typically includes (and in some embodiments only) an operating system. The hardware/software interface system can also include a virtual machine manager (VMM), a common language runtime (CLR) or its functional equivalent, a Java virtual machine (JVM) or its functional equivalent, or replace the operating A system or other software component other than an operating system. The purpose of a hardware/software interface system is to provide an environment in which users can execute applications. The goal of any hardware/software interface system is to make the computer system easy to use and utilize the computer hardware in an efficient manner.

B. Overview of storage platforms

Referring to FIG. 3 , a storage platform 300 includes a data store 302 implemented on a database engine 314 in accordance with the present invention. In one embodiment, the database engine includes a relational database engine with object-relational extensions. In one embodiment, relational database engine 314 includes a Microsoft SQL Server relational database engine.

Data store 302 implements data model 304 that supports organization, search, sharing, synchronization, and security of data. Certain data types are described in schemas, such as schema 340, and storage platform 300 provides tools 346 for adopting these schemas and for extending these schemas, as detailed below.

A change tracking mechanism 306 implemented in the data store 302 provides the ability to track changes to the data store. Data store 302 also provides security capabilities 308 and upgrade/downgrade capabilities 310, both of which are detailed below. Data store 302 also provides a set of application programming interfaces 312 to expose the capabilities of data store 302 to other storage platform components and applications utilizing the storage platform, such as applications 350a, 350b, and 350c.

The storage platform of the present invention also includes an application programming interface (API) 322 that enables applications such as applications 350a, 350b, and 350c to access all of the above-described functions of the storage platform and to access data described in the schema. Applications can use the storage platform API 322 in conjunction with other APIs such as OLE DB API 324 and Microsoft Windows Win 32 API 326.

The storage platform 300 of the present invention can provide various services to applications, including a synchronization service 330 that facilitates sharing data between users or systems. For example, synchronization service 330 allows interoperability with other data stores 340 having the same format as data store 302 and accesses data stores 342 having other formats. Storage platform 300 also provides file system capabilities that allow data store 302 to interoperate with existing file systems such as Windows NTFS file system 318.

In at least some embodiments, storage platform 320 can also provide applications with additional capabilities to allow acting on data and to allow interaction with other systems. These capabilities may be embodied in the form of additional services 328 such as Info Agent service 334 and notification service 332, as well as in the form of other utilities 336.

In at least some embodiments, the storage platform is implemented in, or forms an integral part of, a hardware/software interface system of a computer system. For example and without limitation, the storage platform of the present invention can use an operating system, a virtual machine manager (VMM), a common language runtime (CLR) or its functional equivalent, or a Java virtual machine (JVM) or its functional equivalent to carry out, or form an integral part of.

Through its common storage base and schematized data, the storage platform of the present invention enables more efficient application development for consumers, knowledge workers and business operators. It provides a rich and extensible programming surface area not only to access the capabilities inherent in its data model, but also to include and extend existing file system and database access methods.

In the following description and in the various drawings, the storage platform 300 of the present invention may be referred to as "WinFS". However, the use of this name to refer to the storage platform is for descriptive convenience only and no limitation is intended.

C. Data Model

The data store 302 of the storage platform 300 of the present invention implements a data model that supports organization, search, sharing, synchronization and security of data residing in the store. In the data model of the present invention, "item" is the basic unit of storing information. The data model provides a mechanism for declaring projects and extensions of projects, for establishing relationships between projects, and for organizing projects into project folders and categories, as described more fully below.

This data model relies on two primitive mechanisms: types and relations. Types are structures that provide a format that governs the form of instances of a type. Formats are expressed as ordered groups of attributes. A property is the name of a value or set of values of a given type. For example, the USPostalAddress (United States postal address) type has attributes Street (street), City (city), Zip (zip code), State (state), where Street, City, and State are of type String, and Zip is of type Int32. Street can be multivalued (ie, a set of values), allowing addresses to have more than one value for the Street property. The system defines certain primitive types that can be used in other types of construction, including String (string), Binary (binary), Boolean (Boolean), Int16 (16-bit integer), Int32 (32-bit integer), Int64 (64-bit Integer), Single (single precision), Double (double precision), Byte (byte) DateTime (date time), Decimal (decimal) and GUID. Properties of a type may be defined using any primitive type (with some restrictions noted below) or any constructed type. For example, a Location type may be defined with attributes Coordinate and Address, where the Address attribute is the above-mentioned type USPostalAddress. Properties can also be required or optional.

A relationship can be declared and represents a mapping between two sets of instances of a type. For example, there could be a relationship declared between a Person type and a Location type, called LivesAt, which determines who lives at what location. A relationship has a name and two endpoints, a source and a target. Relationships can also have ordered sets of attributes. Both source and destination endpoints have names and types. For example, a LivesAt relationship has a source called Occupant of type Person and a target called Dwelling of type Location, and furthermore has attributes StartDate and EndDate representing the resident's life time period in the housing. Note that individuals can live in multiple homes over time, and that homes can have multiple residents, so the most likely place to put the StartDate and EndDate information is at the relationship itself.

A relationship defines a mapping between instances constrained by a type given as an endpoint type. For example a LivesAt relationship cannot be one in which Automobile (car) is Occupant (resident) because Automobile is not Person.

Data models allow defining subtype-supertype relationships between types. A subtype-supertype relationship, also known as a BaseType relationship, is defined in such a way that if a type A is a base type of a type B, then it must be the case that every instance of B is also an instance of A. Another way to express it is that every instance that conforms to B must also conform to A. For example, if A has an attribute Name (name) of type String, and B has an attribute Age (age) of type Int16, it follows that any instance of B must have both Name and Age. A hierarchy of types can be thought of as a tree with a single supertype at the root. Branches at the root provide first-level subtypes, which provide second-level subtypes, and so on, up to leaf-most subtypes that no longer have any subtypes themselves. Trees are not limited to uniform depth, but cannot contain any cycles. A given type can have zero or more subtypes and zero or one supertype. A given instance may conform to at most one type and that type's supertypes. In other words, for a given instance at any level in the tree, the instance can conform to at most one subtype at that level.

A type may be considered abstract if instances of the type must also be instances of subtypes of that type.

1. Project

An item is a unit of storable information, unlike a simple file, it is an object with a basic set of properties common to all objects presented by the storage platform to end users or applications. Items also have attributes and relationships supported common to all item types, including features described below that allow the introduction of new attributes and relationships.

Items are objects for common operations such as copy, delete, move, open, print, backup, restore, duplicate, and so on. Items are units that can be stored and retrieved, and all forms of storable information handled by the storage platform exist as items, attributes of items, or relationships between items, each of which is discussed in more detail below.

Projects are designed to represent realistic and easy-to-understand data units, such as Contacts (contacts), People (people), Services (services), Locations (locations), (various types of) Documents (documents), etc. FIG. 5A is a block diagram showing the structure of an item. The unqualified name for this item is "Location". The qualified name of the item is "Core.Location", which indicates that the item structure is defined as a specific type of item in the Core schema (the Core schema is discussed in detail below).

The Location item has several attributes, including EAddress (email address), MetropolitanRegion (metropolitan region), Neighborhood (neighborhood), and PostalAddress (postal address). Each item-specific property is indicated immediately following the property name, separated from the property name by a colon (":"). To the right of the type name, the number of values allowed for that attribute type is indicated between square brackets ("[]"), where an asterisk ("*") to the right of a colon (":") indicates unspecified and/ or an unlimited number ("many"). A "1" to the right of the colon indicates at most one value. A zero ("0") to the left of the colon indicates that the attribute is optional (may have no value at all). A "1" to the left of the colon indicates that there must be at least one value (the attribute is required). Both Neighborhood and MetropolitanRegin are of type "nvarchar" (or equivalent), which is a predefined data type or "simple type" (represented here by the lack of capitalization). However EAddress and PostalAddress are properties of defined types or "complex types" (marked here in capitals) of types EAddress and PostalAddress respectively. A complex type is a type that is derived from one or more simple data types and/or from other complex types. Complex types of properties of items also constitute "nested elements" because details of complex types are nested into immediate items to define their properties, while information concerning these complex types is maintained with items having these properties (in the item's within the boundaries, as discussed later). These concepts of type are well known and readily understood by those skilled in the art.

Figure 5B is a block diagram illustrating complex attribute types PostalAddress and EAddress. The PostalAddress attribute type defines that an item of attribute type PostalAddress can expect zero or one City (city) value, zero or one CountryCode (country code) value, zero or one MailStop (mailbox code) value, and any number (zero to many) PostalAddress type and so on. In this way, the shape of the data for a particular attribute in the item is defined. The EAddress property type is similarly defined as shown. Although optionally used here in this application, another way to represent complex types in a Location item is to derive the item with individual properties for each complex type listed therein. Figure 5C is a block diagram showing the Location item, in which its complex type is further described. It should be understood, however, that an alternative representation of the Location item in FIG. 5C is just the same item shown in FIG. 5A. The storage platform of the present invention also allows subtyping so that one property type is a subtype of another (where one property class inherits properties from another parent property type).

Similar to, but distinct from, properties and their property types, Items inherit their own Item type, which is also the subject of subcategories. In other words, the storage platform in several embodiments of the invention allows an item to be a subtype of another item (so that an item inherits the properties of another parent item). Furthermore, for various embodiments of the present invention, each item is a subtype of the "Item" item type, which is the first and fundamental item type found in the base schema (the base schema is also discussed in detail below). Figure 6A shows that an item (in this example the Location item) is a subtype of the Item item type found in the base schema. In this figure, the arrows indicate that the Location item (like all other items) is a subtype of the Item item type. The Item item type, which is the base item from which all other items are derived, has several important properties such as ItemId (item ID) and various time stamps, thereby defining standard properties for all items in the operating system. In this figure, these properties of the Item item type are inherited by Location and thus become properties of Location.

Another way to represent properties in a Location item that inherits from the Item item type is to derive a Location with individual properties from each property type of the parent item listed therein. Fig. 6B is a block diagram showing the Location item, in which its inherited types are described in addition to its immediate attributes. It should be noted and understood that this item is the same item shown in Figure 5A, although in this figure, Location is shown with all its attributes, including direct attributes (shown in this figure and in Figure 5A) and inherited attributes (in Shown in this figure but not in Figure 5A) (whereas in Figure 5A these attributes are referenced by arrows showing that the Location item is a subtype of the Item item type).

Items are self-contained objects, so deleting an item also deletes all of the item's direct and inherited properties. Similarly, when an item is retrieved, what is received is the item and all of its direct and inherited attributes (including information concerning its complex attribute types). Certain embodiments of the present invention may enable one to request a subset of attributes when retrieving a particular item; however the default for many such embodiments is to provide an item with all its direct and inherited attributes when retrieving. In addition, the properties of an item can also be extended by adding new properties to the existing properties of the item's type. These "extensions" are followed by the actual properties of the item, and subtypes of the item type may automatically include extended properties.

The "boundary" of an item is represented by its attributes (including complex attribute types, extensions, etc.). The boundaries of an item also represent the limits of operations performed on the item, including copying, deleting, moving, creating, and so on. For example, in several embodiments of the invention, when an item is copied, all content within the boundaries of the item is also copied. For each project, the boundary includes the following:

The item type of an item, and if the item is a subtype of another item (as in the case of several embodiments of the invention where all items are derived from a single item and item type of the base schema), any applicable subtype Information (that is, information referring to the parent item type). The original item being copied is a subtype of another item, and the copy can also be a subtype of that same item.

· Item's complex type properties and extensions (if any). If the original item has properties of a complex type (original or extended), the copy can also have the same complex type.

• A record of items on an "ownership relationship", ie, an item-owned list of what other items ("catalogue items") owns what this item ("owning item") owns. This specifically relates to the project folders discussed fully below and the rule that all projects must belong to at least one project folder. Furthermore, with respect to embedded items (discussed more fully below), an embedded item is considered to be the part of the item in which operations such as copying, deleting, etc. are embedded.

2. Project identification

ItemID is used to uniquely identify items in the global item space. The Base.Item type defines a field ItemID of type GUID that stores the identity of the item. An item must have only one identity in the data store 302 .

a) Project References

Item references are data structures that contain information to locate and identify items. In this data model, define an abstract type called ItemReference (item reference), from which all item reference types are derived. The ItemReference type defines a virtual method named Resolve. The Resolve method resolves the ItemReference and returns an item. This method is overridden by a concrete subtype of ItemReference that implements functionality for retrieving an item given a reference. Call the Resolve method as part of the Storage Platform API 322.

(1)ItemIDReference

ItemIDReference (item ID reference) is a subtype of ItemReference. It defines the Locator (locator) and ItemID fields. The Locator field names (ie identifies) the project field. It is handled by the locator resolve method that resolves the value of the Locator to the project field. The ItemID field is of type ItemID.

(2)ItemPathReference

ItemPathReference (item path reference) is a specialization of ItemReference that defines the Locator and Path fields. The Locator field identifies the project domain. It is handled by the locator resolve method that resolves the value of the Locator to the project field. The Path field contains a (relative) path in the storage platform namespace rooted at the project domain provided by the Locator.

Such references cannot be used in set operations. References generally must be resolved through the path resolution process. The Resolve method of the Storage Platform API 322 provides this functionality.

b) Reference type hierarchy

The citation forms discussed above are represented by the citation type hierarchy shown in FIG. 11 . Other reference types that inherit from these types can be defined in the schema. They can be used as the type of the target field in the relation declaration.

3. Project folders and categories

As will be discussed more fully below, groups of projects can be organized into special projects called project folders (not to be confused with folders of files). Unlike most file systems, however, a project can belong to multiple project folders, so that when a project is accessed and revised in one project folder, the revised project can then be accessed directly from another project folder. Essentially, although access to a project can occur from different project folders, what is actually being accessed is the same project. However, a project folder does not have to own all of its member projects, or simply own projects in conjunction with other folders, so that the deletion of one project folder does not necessarily result in the deletion of projects. However, in several embodiments of the invention, a project must belong to at least one project folder, so that if the unique project folder for a particular project is deleted, then for some embodiments, the project is automatically deleted, or in other implementations In this case, the project automatically becomes a member of the default project folder (for example, the "TrashCan" project folder is conceptually similar to similarly named folders used in various file and folder-based systems. )

As discussed more fully below, items may also belong to categories based on commonly described characteristics such as: (a) item type (or types), (b) specific direct or inherited attributes (or attributes), or (c ) corresponds to a specific value (or values) of an item property. For example, items that include certain attributes of personal contact information can automatically belong to the Contact category, as do any items that have the attribute of contact information. Likewise, any item that has a location attribute with a value of "New York City" can automatically belong to the NewYorkCity (New York City) category.

Categories are conceptually different from project folders in that project folders may contain items that are unrelated to each other (i.e., have no common descriptive characteristics), whereas each item in a category has a common type, attribute, or value ("commonality"), it is this commonality that forms the basis for its relationship to other items or items in the class. Furthermore, while membership of an item in a particular folder is not mandatory based on any particular aspect of the item, for some embodiments, all items having a taxonomically common Membership of this class is automatic at the system level. Conceptually, categories can also be seen as virtual project folders, whose membership is based on the results of a specific query (such as in the context of a database), and projects that satisfy the conditions of this query (determined by the commonality of the category) should form the category membership.

Figure 4 shows the structural relationship among projects, project folders and categories in various embodiments of the present invention. A plurality of projects 402 , 404 , 406 , 408 , 410 , 412 , 414 , 416 , 418 , and 420 are members of respective project folders 422 , 424 , 426 , 428 , and 430 . Certain projects belong to more than one project folder, such as project 402 belonging to project folders 422 and 424 . Certain items, such as items 402, 404, 406, 408, 410, and 412 are also members of one or more categories 432, 434, and 436, while other items, such as items 44, 416, 418, and 420, may not belong to any category ( This is mostly not like in some embodiments though, where having any attribute automatically implies membership in a class, so in that embodiment an item should be completely featureless in order to not be a member of any class). In contrast to the hierarchical structure of folders, both category and project folders have a structure more like a directed graph as shown. In any case, projects, project folders, and categories are projects (albeit different project types).

Contrary to files, folders and directories, the project, project folder and category features of the present invention are not "physical" in nature, because they have no conceptual equivalent of physical containers, so projects can exist in a the position above. The ability for projects to exist in more than one project file location and be organized into categories provides an enhanced and rich degree of data processing and storage structure capabilities at the hardware/software interface level beyond what is currently available in the art.

4. Mode

a) Basic pattern

To provide a common basis for creating and using items, embodiments of the storage platform of the present invention include an underlying schema that establishes a conceptual framework for creating and organizing items and attributes. The base schema defines the items and properties of some specific types, and the characteristics of these specific base types from which subtypes are further derived. Using this basic pattern enables programmers to conceptually distinguish items (and their respective types) from attributes (and their respective types). Additionally, the base schema lists the base set of properties that all items can have, since all items (and their corresponding item types) are derived from this base item (and their corresponding item types) of the base schema.

As shown in FIG. 7 , for several embodiments of the present invention, the basic schema defines three top-level types: Item (item), Extension (extension) and PropertyBase (property base). As shown, the item type is defined through the properties of this base "Item" item type. In contrast, the top-level property type "PropertyBase" has no predefined properties, and is merely an anchor point from which all other property types are derived, and through which all derived property types relate to each other (collectively derived from a single property type). The Extension type attribute defines which project the extension extends, and defines the identity that distinguishes one extension from another project, since a project can have multiple extensions.

ItemFolder (item folder) is a subtype of the Item item type, which, in addition to inheriting properties from Item, characterizes the relationship used to establish links to its members (if any), although Identitykey (identity key) and Property ( Attributes) are subtypes of PropertyBase. CategoryRef (category reference) is in turn a subtype of IdentityKey.

b) Kernel mode

Various embodiments of the storage platform of the present invention also include a core schema that provides a conceptual framework for the top-level project type structure. FIG. 8A is a block diagram showing items in the core schema, and FIG. 8B is a block diagram showing attribute types in the core schema. The distinction made between files with different extensions (*.com, *.exe, *.bat, *.sys, etc.) and other criteria in the file- and folder-based system is a kernel-mode-like function. In an item-based hardware/software interface system, the core schema defines a core set of item types that directly (by item type) or indirectly (by item subtype) characterize all items into an item-based hardware/software interface system One or more core schema item types that are understood and can be directly processed in a predetermined or predictable manner. The predefined item types reflect the most commonly used items in the item-based hardware/software interface system, and thus the level of validity is derived by the item-based hardware/software interface system that understands these predefined item types that make up the core schema.

In some embodiments, the core schema is not extensible, ie, no additional types can be subclassed directly from item types in the base schema, except for specific predefined derived item types that are part of the core schema. By prohibiting extensions to the core schema (ie, by prohibiting the addition of new items to the core schema), the storage platform hosts the use of the core schema item type, since each subsequent item type must be a subtype of the core schema item type. This structure allows a reasonable degree of flexibility in defining additional item types while maintaining the benefits of having a predetermined set of core item types.

Referring to FIG. 8A, for various embodiments of the present invention, specific item types supported by the core schema may include one or more of the following:

• Category: Items of this item type (and subtypes derived from it) represent valid categories in the item-based hardware/software interface system.

• Commodity: An item that is an identifiable thing as a value.

• Device: An item having a logical structure supporting information processing capabilities.

• Document: An item with content that cannot be interpreted by an item-based hardware/software interface system but instead by an application corresponding to the document type.

· Event (event): records some events in the environment of the project.

• Location: An item representing a physical location (eg, geographic location).

• Message: An item for communication between two or more principals (defined below).

• Principal: An item that has at least one positively verifiable identity (eg, identification of a person, organization, group, family, author, service, etc.) other than the ItemId.

• Statement: An item with specific information about an environment, including but not limited to: Policies, Subscriptions, Credentials, etc.

Referring similarly to FIG. 8B , specific attribute types supported by the core schema may include one or more of the following:

Certificate (certificate) (derived from the base PropertyBase type in the base schema)

· PrincipalIdentityKey (subject identity key) (derived from the IdentityKey type in the underlying schema)

· PostalAddress (postal address) (derived from the Property type in the basic mode)

RichText (rich text) (derived from the Property type in the basic mode)

EAddress (email geology) (derived from the Property type in the base schema)

· IdentitySecnrityPackage (identity security package) (exported from the Relationship type in the basic model)

·RoleOccupancy (resident role) (derived from the Relationship type in the basic model)

• BasicPresence (derived from the Relationship type in the Basic Schema) These items and attributes are further described by the respective attributes listed in Figures 8A and 8B.

5. Relationship

Relationships are binary, where one item is designated as the source and the other as the target. Source and target items are related by a relationship. The source project generally controls the life cycle of the relationship. That is, when the source item is deleted, the relationship between the items is also deleted.

Relationships are classified into: Containment and Reference relationships. Contains relationships control the lifecycle of the target item, while references relationships do not provide any lifecycle management semantics. Fig. 12 shows how relationships are classified.

Containment relationship types are further classified into Holding and Embedding relationships. When all holdings to an item are removed, the item is deleted. The holding relationship controls the life cycle of the target through the reference counting mechanism. Embedding relationships can model composite items and can be viewed as exclusive holding relationships. An item can be the target of one or more holding relationships; but an item can only be the target of one embedded relationship. An item that is the target of an embedded relationship cannot be the target of any other holding or embedded relationship.

Reference relationships do not control the lifecycle of the target project. They can be dangling - the target item can be absent. Reference relationships can be used to model references to projects anywhere in the global project namespace (ie, including remote data stores).

Obtaining an item does not automatically acquire its relationship, the application must explicitly request the item's relationship. Also, modifying a relationship does not modify the source or target items; similarly, adding a relationship does not affect the source/target items.

a) Relationship statement

Explicit relation types are defined with the following elements;

Specify the relationship name in the Name attribute

• A relationship type of one of the following: holds, embeds, references. This is specified in the Type attribute.

• Source and destination endpoints. Each endpoint specifies the name and type of the item being referenced.

• The source endpoint field is typically of type ItemID (undeclared), but must refer to an item in the same data store as the relation instance.

• For holding and embedding relationships, the target endpoint field must be of type ItemIDReference, and it must refer to an item in the same store as the relationship instance. For reference relationships, the target endpoint can be of any ItemReference type and can reference items in other storage platform data stores.

• One or more fields of type scalar or PropertyBase can optionally be declared. These fields can contain data associated with the relationship.

• Relationship instances are stored in the global relationship table.

• Each relation instance is uniquely identified by the combination (source ItemID, relation ID). Relation IDs are unique within a given source ItemID for all relations originating from a given Item, regardless of their type.

The source project is the owner of the relationship. While the project designated as the owner controls the lifecycle of the relationship, the relationships themselves are separate from the projects they are related to. The storage platform API 322 provides mechanisms for exposing the relationships associated with items.

Here is an example of a relationship declaration.

<Relationship Name="Employment"BaseType="Reference">

<Source Name="Employee" ItemType="Contact.Person"/>

<Target Name="Employer" ItemType="Contact.Organization"

     ReferenceType = "ItemIDReference" />

<Property Name="StartDate"Type="the storage

platformTypes.DateTime″/>

<Property Name="EndDate"Type="the storage

platformTypes.DateTime″/>

<Property Name="Office"Type="the storage

platformTypes.DateTime″/>

</Relationship>

This is an example of a reference relationship. This relationship cannot be created if the personal item referenced by the source reference does not exist. And if the individual item is deleted, the relationship instance between the individual and the organization is also deleted. However, if the organization item is deleted, the relationship is not deleted, but it is suspended.

b) holds a relationship

Holding relationships are used to model reference counting based on the lifecycle management of the target project.

An item can be the source endpoint of zero or more relationships to an item. Items that are not embedded items can be targets in one or more holding relationships.

The target endpoint reference type must be ItemIDReference, and it must refer to an item in the same store as the relation instance.

Holding relationships enforces lifecycle management of target endpoints. The creation of the holding relation instance and the item as the target is an atomic operation. Additional holding relationship instances can be created that target the same project. When the last holding relationship instance with the given item as target endpoint is deleted, the target item is also deleted.

The type of an endpoint item specified in a relationship declaration is generally enforced when an instance of that relationship is created. The type of an endpoint item cannot be changed after the relationship is established.

Holding relationships play a key role in forming a project's namespace. They contain the "Name" attribute, which defines the name of the target project relative to the source project. Relative names are unique across all holding relationships originating from a given project. An ordered list of relative names from the root project to a given project forms the project's full name.

The holding relationships form a directed acyclic graph (DAG). When creating holding relationships, the system ensures that no loops are created, thereby ensuring that the project's namespace forms a DAG.

Although the hold relationship controls the lifecycle of the target item, it does not control the consistency of operation of the target endpoint item. The target item is operationally independent of the item that owns it through the holding relationship. Copying, moving, backing up, and other operations on items that are the source of a holding relationship do not affect items that are the target of the same relationship—for example, backing up a folder item does not automatically back up all items in that folder (FolderMember(folder member) target in a relationship).

Here is an example of a hold relationship:

<Relationship Name="FolderMembers"BaseType="Holding">

<Source Name="Folder" ItemType="Base.Folder"/>

<Target Name="Item"ItemType="Base.Item"

     ReferenceType = "ItemIDReference" />

</Relationship>

The FolderMember relationship makes the concept of a folder a generic collection of items.

c) Embedding relationships

The embedding relationship models the concept of exclusive control of the life cycle of the target item. They allow the notion of compositing items.

The creation of an embedded relation instance and an item as a target is an atomic operation. Items can be the source of zero or more embedded relationships. However an item can be the object of one and only one embedded relation. An item that is the target of an embedded relationship cannot be the target of a holding relationship.

The target endpoint reference type must be ItemIDReference, and it must refer to an item in the same data store as the relation instance.

The type of an endpoint item specified in a relationship declaration is generally enforced when an instance of that relationship is created. The type of an endpoint item cannot be changed after the relationship is established.

The embedding relationship controls the operational consistency of that target endpoint. For example, the operation of serializing an item may include serializing all embedded relationships originating from the item and all of its targets; copying an item also copies all of its embedded items.

Here is an example declaration:

<Relationship Name="ArchiveMembers"BaseType="Embedding">

<Source Name="Archive" ItemType="Zip.Archive"/>

<Target Name="Member" ItemType="Base.Item"

       ReferenceType="ItemIDReference"/>

<Property Name="ZipSize"Type="the storage

platformTypes.bigint″/>

<Property Name="SizeReduction"Type="the storage

platformTypes.float″/>

</Relationship>

d) Reference relationship

A reference relationship does not control the lifecycle of the item it references. In particular, a referencing relationship does not guarantee the existence of the target, nor does it guarantee that the target is of the type as specified in the relationship declaration. This means that reference relationships can be dangling. And reference relationships can reference items in other data stores. A citation relationship can be viewed as a concept similar to a link on a web page.

The following is an example of a reference relationship declaration:

<Relationship Name="DocumentAuthor"BaseType="Reference">

<Sourc ItemType="Document"

  ItemType = "Base.Document"/>

<Target ItemType="Author"ItemType="Base.Author"

     ReferenceType = "ItemIDReference" />

<Property Type="Role"Type="Core.CategoryRef"/>

<Property Type="DisplayName"Type="the storage

platformTypes.nvarchar(256)″/>

</Relationship>

Any reference type is allowed on the target's endpoint. Items participating in a reference relationship can be of any item type.

Reference relationships are used to model most non-lifecycle management relationships between projects. Reference relationships are convenient for modeling loosely coupled relationships because the existence of the target is not enforced. Reference relationships can be used for target items in other stores, including stores on other computers.

e) Rules and constraints

The following additional rules and constraints apply to relationships:

1. The item must be the target of (only one embedded relationship) or (one or more holding relationships). One exception is the root project. An item can be the target of zero or more reference relationships.

2. An item that is the target of an embedded relationship cannot be the source of the holding relationship. It can be the source of a reference relationship.

3. If the project is upgraded from a file, it cannot be the source of the holding relationship. It can be the source of embedding and referencing relationships.

4. Items promoted from files cannot be targets of embedded relationships.

f) Ordering of relations

In at least one embodiment, the storage platform of the present invention supports ordering of relationships. Sorting is done through an attribute called "Order" in the base relation definition. There is no uniqueness constraint in the Order field. The order of relations with the same "Order" property value is not guaranteed, however they are guaranteed to be sorted after relations with lower "Order" values and before relations with higher "Order" field values.

The application gets the default order of relationships by sorting on the combination (SourceItem ID, RelationshipID, Order). All relationship instances originating from a given item are sorted into a single collection, regardless of the type of relationship in the collection. However this guarantees that all relations of a given type (eg, FolderMembers) are an ordered subset of a given item's relation collection.

The data store API 312 for manipulating relations implements a set of operations that support ordering of relations. The following terms are introduced to help explain those operations:

RelFirst is the first relation in the ordered set with the order value OrdFirst;

RelLast is the last relation in the ordered set with the order value OrdLast;

RelX is a given relation in a collection with an order value OrdX;

RelPrev is the closest relation in the set to RelX with an order value OrdPrev less than OrdX; and

RelNext is the closest relation in the set to RelX with an order value OrdNext greater than OrdX.

InsertBeforeFirst(SourceItemID, Relationship)

Inserts a relation as the first relation in the collection. The value of the "Order" attribute of the new relationship can be less than OrdFirst.

InsertAfterLast(SourceItemID, Relationship)

Inserts a relation as the last relation in the collection. The "Order" property of the new relationship can have a value greater than OrdLast.

InsertAt(SourceItemID, ord, Relationship)

Inserts a relationship with the value specified for the "Order" property.

InsertBefore(SourceItemID, ord, Relationship)

Inserts this relation before the relation with the given order value. New relations may be assigned the "Order" value, which is between, but not including, OrdPrev and ord.

InsertAfter(SourceItemID, ord, Relationship)

Inserts this relation after the relation with the given order value. New relations may be assigned the "Order" value, which is between but not including ord and OrdNext.

MoveBefore(SourceItemID, ord, Relationship)

Moves the relation with the given relation ID before the relation with the specified "Order" value. Relationships can be assigned a new "Order" value, which is between, but not including, OrdPrev and ord.

MoveAfter(SourceItemID, ord, Relationship)

Moves the relationship with the given relationship ID after the relationship with the specified "Order" value. The relationship can be assigned a new order value, which is between but not including ord and OrdNext.

As mentioned earlier, each project must be a member of the project folder. By relationship, each project must have a relationship with a project folder. In several embodiments of the invention, certain relationships are represented by relationships that exist between items.

As implemented for various embodiments of the invention, relationships provide directed binary relationships extending from one item (source) to another item (target). A relationship is owned by the source item (the item that extends it), so the relationship is removed if the source is removed (eg, the relationship is deleted when the source item is deleted). Also in some cases, relationships can share (co-owned) ownership of the target item, and that ownership can only be reflected in the relationship's IsOwned property (or its equivalent) (see Figure 7 shown for relational attribute types). In these embodiments, creating a new IsOwned relationship automatically increments the reference count on the target project, while deleting such a relationship decrements the reference count on the target project. For these particular embodiments, an item persists if it has a reference count greater than zero, and is automatically deleted if the item count reaches zero. Again, a project folder is a project that has (or can have) a set of relationships with other projects, including membership of the project folder. Other practical implementations of relationships are possible and contemplated by the present invention to achieve the functionality described herein.

Regardless of the actual implementation, a relationship is an optional connection from one object to another. The ability for a project to belong to more than one project folder and one or more categories, whether those projects, folders and categories are public or private, is determined by the meaning given to the presence (or lack thereof) in project-based structures . These logical relationships are the meanings assigned to a set of relationships, regardless of its physical implementation designed to perform the functions described herein. Logical relationships are established between projects and their folders or categories (or vice versa), because essentially project folders and categories are each a specific type of project. Thus, project folders and categories can be acted upon like other projects (copied, added to e-mail messages, embedded in documents, etc. without limitation), and project folders and categories can be used like other projects using the same Mechanisms for serialization and deserialization (import and export). (eg in XML all items can have a serialized format and this grid applies equally to item folders, categories and items).

The relationships described above, which represent relationships between projects and their project folders, can logically extend from projects to project folders, from project folders to projects, or both. A relationship that logically extends from a project to a project folder indicates that the project folder is public to the project and shares its membership information with the project; conversely, a lack of a logical relationship from the project to the project folder indicates that the project folder is private to the project and does not share membership information with the project. Similarly, a relationship that logically extends from a project folder to a project indicates that the project is public and can be shared with the project folder, whereas the lack of a relationship that extends logically from the project folder to the project indicates that the project is private and cannot shared. Therefore, when exporting a project folder to another system, it is "public" to the project, it is shared in the new environment, and it is "public" when the project searches its project folder for other shareable projects A project folder that provides the project with information about shareable projects that belong to it.

FIG. 9 is a block diagram showing a project folder (which is itself a project), its member projects, and the interconnection relationship between the project folder and its member projects. Project folder 900 has a number of projects 902, 904, and 906 as its members. Project folder 900 has a relationship 912 from itself to project 902, which indicates that project 902 is public and can be shared with project folder 900, its members 904 and 906, and any other project files that can access project folder 900 Folders, categories, or items (not shown) are shared. However, there is no relationship from project 902 to project folder project 900 , indicating that project folder 900 is private to project 902 and does not share its membership information with project 902 . On the other hand, project 904 does have a relationship 924 from itself to project folder 900 , which indicates that project folder 900 is public and shares its membership information with project 904 . However there is no relationship from project folder 900 to project 904, which indicates that project 904 is private and not related to project folder 900, its other members 902, 906, and any other project folder that can access project folder 900 , category, or project (not shown) sharing. In contrast to its relationships (or lack thereof) to projects 902 and 904, project folder 900 has a relationship 916 from itself to project 906, and project 906 has a relationship 926 back to project folder 900, together indicating that project 906 is public, and can be shared with project folder 900, its members 902 and 904, and any other project folder, category, or project (not shown) that can access project folder 900, and project folder 900 is public , and share its membership information with project 906.

As discussed previously, projects within project folders need not share commonality because project folders are not "described". A category, on the other hand, is described by a commonality common to all its member projects. Thus, membership of a class is inherently limited to items that have a described commonality, and in some embodiments, all items that satisfy a class's description automatically become members of that class. Thus, while project folders allow unimportant type structures to be represented by their membership, categories allow membership based on defined commonality.

Of course, class descriptions are logical in nature, so classes can be described by any logical representation of types, attributes and/or values. For example, a logical representation of a category could be its membership to include items with either or both attributes. If the attributes described for a class are "A" and "B," membership in that class can include items with attribute A but not B, items with attribute B but not A, and items with both attributes A and B project. The logical representation of the attribute is described by the logical operator "OR", where the member group described by the category is an item with attribute A OR B. As will be appreciated by those skilled in the art, similar logical operators (including but not limited to "AND (and)", "XOR (exclusive or)" and "NOT (not)" alone or in combination thereof can also be used to Describe the category.

Although there is a distinction between project folders (not described) and categories (described), in many embodiments of the invention, in principle, category relationships to projects and project relationships to categories above face project folders and Items are revealed in the same way.

Fig. 10 is a block diagram showing a category (which is itself an item), its member items, and the interconnection relationship between the category and its member items. Class 1000 has a number of items 1002, 1004, and 1006 as members, all of which share some combination of common attributes, values, and types 1008 (commonality description 1008') described by class 1000. Class 1000 has a relationship from itself to project 1002, which indicates that project 1002 is public and can be shared with class 1000, its members 1004 and 1006, and any other class, project folder, or project that has access to class 1000 (not shown) shared. However, there is no relationship from project 1002 to category 1000 , indicating that category 1000 is private to project 1002 and does not share membership information with project 1002 . On the other hand, item 1004 does have a relationship 1024 from itself to category 1000 , which indicates that category 1000 is public and shares its membership information with item 1004 . However, there is no relationship extending from class 1000 to item 1004, indicating that item 1004 is private and cannot be shared with class 1000, its other members 1002 and 1006, and any other class, item folder, Or project (not shown) sharing. In contrast to its relationship (or lack thereof) to items 1002 and 1004, class 1000 has a relationship 1016 from itself to item 1006, and item 1006 has a relationship 1026 back to class 1000, which together indicate that item 1006 is public , and can be shared with class 1000, its project members 1002 and 1004, and any other class, project folder, or project (not shown) that can access class 1000, and class 1000 is public and shares its Membership information.

Finally, since categories and project folders are themselves projects, and projects can relate to each other, categories can relate to project folders and vice versa, and in some further embodiments, categories, project folders and projects can each relate to Other categories, project folders and projects. In various embodiments, however, the project folder structure and/or class structure prohibits containment loops at the hardware/software interface system level. While the project folder and category structure resembles a directed graph, an embodiment that prohibits loops resembles a directed acyclic graph (DAG), which, according to the mathematical definition in the field of graph theory, is one in which no path begins and ends at the same vertex directed graph.

6. Scalability

As mentioned above, the present storage platform aims to provide an initial schema set 340 . However, at least in some embodiments, the storage platform also allows customers, including independent software vendors (ISVs), to create new schemas 344 (ie, new items and nested element types). This section discusses the mechanism for creating this schema by extending the item types and nested element types (or simply "element" types) defined in the initial schema set 340 .

Preferably, the expansion of the initial set of item and nested element types is constrained as follows:

Allow ISVs to introduce a new item type, the subtype Base.Item;

Allow ISVs to introduce a new nested element type, the subtype Base.NestedElement;

ISVs are allowed to introduce new extensions, the subtype Base.NestedElement; but

ISVs cannot subtype any type (item, embedded element, or extension type) defined by the storage platform's initial schema set 340 .

Since an item type or embedded element type defined by a storage platform's initial schema set may exactly match the needs of an ISV application, the ISV must be allowed to customize the type. This takes into account the concept of extension. Extensions are strongly typed instances, but (a) they cannot stand alone, and (b) they must be attached to an item or nested element.

In addition to addressing the need for schema extensibility, the extension also aims to address "multityping". In some embodiments, because storage platforms may not support multiple inheritance or overlapping subtypes, applications may use extensions as a way to model instances of overlapping types (eg, a document that is both a legal document and a secure document).

a) Project extension

To provide extensibility of the project, the data model also defines an abstract type called Base.Extension. This is the root type of the hierarchy of extension types. Applications can subtype Base.Extension to create specific extension types.

Define the Base.Extension type in the base schema as follows:

<Type Name="Base.Extension" IsAbstract="True">

<Propety Name="ItemID"

Type = "the storage platformTypes.uniqueidentified"

  Nullable = "false"

    MultiValued="false"/>

<Property Name="ExtensionID"

Type = "the storage platformTypes.uniqueidentified"

Nullable = "false"

      MultiValued="false"/>

</Type>

The ItemID field contains the ItemID of the item associated with this extension. An item with this ItemID must exist. If an item with the given ItemID does not exist, the extension cannot be created. When an item is deleted, all extensions with the same ItemID are deleted. The tuple (ItemID, ExtensionID) uniquely identifies the extension instance.

The structure of an extension type is similar to that of an item type:

Extension types have fields;

fields can be primitive or nested element types; and

Extension types can be subcategorized.

The following restrictions apply to extension types

An extension cannot be the source and target of a relationship;

An instance of an extension type cannot exist independently of the item; and

Extension types cannot be used as field types in storage platform type definitions

There are no constraints on the types of extensions that can be associated with a given item type. Any extension type allows extension of any item type. When multiple extension instances are attached to a project, they are independent of each other in structure and behavior.

Extension instances are stored separately and accessed from projects. All extension type instances are accessible from the global extension view. An efficient query can be composed that returns all instances of an extension of a given type, regardless of what type of item they are associated with. The Storage Platform API provides a programming model for storing, retrieving and modifying project extensions.

An extension type can be a type that is subtyped using the storage platform's single inheritance model. Creates a new extension type derived from an extension type. An extended structure or behavior cannot override or replace the structure or behavior of the item type hierarchy.

Similar to item types, extension type instances can be accessed directly through the views associated with the extension type. Extended ItemIDs indicate which item they belong to and can be used to retrieve the corresponding item object from the global item view.

For operational consistency purposes, extensions are considered part of the project. Copy/move, backup/restore, and other common operations defined by the storage platform can operate on extensions that are part of the project.

Consider the following example. Define the Contact type in the Windows type set.

<Type Name="Contact"BaseType="Base.Item">

<Property Name="Name"

     Type = "String"

Nullable = "false"

       MultiValued="false"/>

<Property Name="Address"

     Type = "Address"

                                            

       MultiValued="false"/>

</Type>

CRM application developers like to attach CRM application extensions to contacts stored in storage platforms. Application developers define CRM extensions that contain additional data structures that the application can handle.

<Type Name="CRMExtension"BaseType="Base.Extension">

<Property Name="CustomerID"

      Type = "String"

                               

    MultiValued="false"/>

...

</Type>

HR application developers wish to attach additional data to contacts as well. This data is separate from CRM application data. Application developers can also create extensions

<Type Name="HRExtension"EBaseType="Base.Extension">

<Property Name="EmployeeID"

     Type = "String"

Nullable = "false"

       MultiValued="false"/>

...

</Type>

CRMExtension and HRExtension are two separate extensions that can be attached to contact items. They can be created and accessed independently of each other.

In the above example, the fields and methods of the CRMExtension type cannot override the fields and methods of the contact hierarchy. It should be noted that instances of the CRMExtension type can be attached to item types other than Contacts.

When retrieving a contact item, its item extension is not automatically retrieved. Given a contact item, its related item extensions can be accessed by querying the global extensions view for extensions with the same ItemID.

All CRMExtension extensions in the system, regardless of the project they belong to, can be accessed through the CRMExtension type view. All project extensions of a project share the same project ID. In the above example, the Contact item instance and the attached CRMExtension and HRExtension instances share the same ItemID.

The following table summarizes the similarities and differences between the Item, Extension, and NestedElement types:

Item, ItemExtension and NestedElement

b) Extend the NestedElement type

Nested element types are not extended by the same mechanism as item types. Nested element extensions are stored and accessed using the same mechanisms as nested element type fields.

The data model defines the root of a nested element type named Element:

<Type Name="Element"

IsAbstract="True">

<Property Name="ElementID"

Type = "the storage platformTypes.uniqueidentifier"

Nullable = "false"

       MultiValued="false"/>

</Type>

The NestedElement type inherits from this type. The NestedElement element type additionally defines one field, which is the multiset element.

<Type Name="NestedElement"BaseType="Base.Element"

IsAbstract="True">

<Property Name="Extensions"

      Type = "Base.Element"

                                      

MultiValued="true"/>

</Type>

NestedElement extensions differ from item extensions in the following ways:

Nested element expansion is not an expansion type. They do not belong to the extension type hierarchy rooted at the Base.Extension type.

Nested element extensions are stored with the item's other fields and are not globally accessible - queries that retrieve all instances of a given extension type cannot be composed.

These extensions are stored like other nested elements (or items). Like other nested sets, NestedElement extensions are stored in UDTs. They are accessible through the Extension field of the nested element type.

The collection interfaces used to access multivalued properties are also used for access and iteration over collections of type extensions.

The following table summarizes and compares Item extensions and NestedElement extensions.

Item extension and NestedElement extension

D. Database engine

As mentioned above, data storage is implemented on the database engine. In this embodiment, the database engine includes a relational database engine that implements SQL query language and has object-relational extensions, such as Microsoft SQL Server engine. This section describes the mapping from the data model implemented by the data storage to the relational storage according to this embodiment, and provides the information used by the client of the storage platform on the logic API. It will be appreciated, however, that different mappings may be used when different database engines are used. Indeed, in addition to implementing the storage platform conceptual data model on the relational database engine, it can also be implemented on other types of databases, such as object-oriented and XML databases.

Object-oriented (OO) database systems provide persistence and transactions for programming language objects (eg, C++, Java). The storage platform concept of "items" maps well to objects in object-oriented systems, although embedded collections must be added to objects. Other storage platform type concepts like inheritance and nested element types also map to the object-oriented type system. Object-oriented systems typically already support object identities; thus, item identities can be mapped to object identities. The behavior (operations) of items map nicely to object methods. However, object-oriented systems often lack organizational capabilities and are poor at searching. Also, object-oriented systems do not provide support for unstructured and semi-structured data. To support the full storage platform data model described here, concepts such as relationships, folders, and extensions need to be added to the object data model. Additionally, mechanisms such as upgrades, synchronization, notifications, and security need to be implemented.

Similar to object-oriented systems, XML databases based on XSD (XML Schema Definition) support systems based on single-inheritance types. The item type system of the present invention is mappable to an XSD type model. XSD does not provide support for behaviors either. The project's XSD must add the project's behavior. XML databases deal with a single XSD document and lack organization and broad search capabilities. As with object-oriented databases, to support the data model described here, other concepts such as relationships and folders need to be incorporated into the XML database; and mechanisms such as synchronization, notification and security need to be implemented.

1. Data storage implementation using UDT

In this embodiment, the relational database engine 314, which in one embodiment includes the Microsoft SQL Server engine, supports built-in scalar types. The built-in scalar types are "old" and "simple". They are primitive in the sense that users cannot define their own types; they are simple in the sense that users cannot encapsulate complex structures. User-defined types (hereinafter referred to as UDTs) provide a mechanism for type extensibility beyond or beyond the original scalar type system by enabling users to extend the type system by defining complex structured types. Once defined by the user, UDTs can be used anywhere in the type system where built-in scalar types can be used.

According to one aspect of the invention, storage platform schemas are mapped to UDT classes in database engine storage. Data store items are mapped to UDT classes derived from the Base.Item type. Similar to projects, extensions can also be mapped to UDT classes and use inheritance. The root extension type is Base.Extension, from which all extension types are derived.

A UDT is a CLR class that has state (ie data fields) and behavior (ie routines). Define UDTs using any managed language (c#, VB.NET, etc.). UDT methods and operators can be invoked on instances of this type in Transact-SQL. A UDT can be: the type of a column in a row, the type of a parameter of a routine in T-SQL, or the type of a variable in T-SQL.

The following example shows the basics of a UDT. Assume MapLib.dll has an assembly called MapLib. In that assembly, under the namespace BaseTypes, there is a class called Point:

namespace BaseTypes

{

public class Point

{

// Returns the distance from the specified point.

public double Distance(Point p)

{

//Returns the distance between point p and this point

}

//Other content in this class

}

}

The following T-SQL code binds the class Point to a SQL Server UDT called Point. The first step calls "Create Assembly", which loads the MapLib assembly into the database. The second step calls "Create Type" to create the user-defined type "Point" and bind it to the managed type BastTypes.Point:

CREATE ASSEMBLY Maplib

FROM'\\myserv\share\MapLib.dll'

go

CREATE TYPE Point

EXTERNAL NAME'BaseTypes.Point'

go

Once created, the "Point" UDT can be used as a column in a table, and the method can be called in T-SQL as follows:

Create table Cities(

Name varchar(20),

State varchar(20),

Location Point)

-- Retrieve the distance from the city to coordinates (32, 23)

Declare @p point(32, 23), @distance float

Select Location::Distance(@p)

From Cities

The mapping of storage platform schemas to UDT classes is completely straightforward at a high level. In general, storage platform schemas are mapped to CLR namespaces. Storage platform types are mapped to CLR classes. CLR class inheritance mirrors storage platform type inheritance, and storage platform attributes are mapped to CLR class attributes.

The project hierarchy shown in Figure 29 is used as an example in this document. It shows the Base.Item type from which all item types are derived, and a set of exported item types (eg, Contact.Person and Contact.Employee), where inheritance is indicated by arrows.

2. Project Mapping

In order to hope that items can be searched globally, and support inheritance and type substitutability in the relational data of this embodiment, a possible implementation of item storage in database storage is in a class with type Base.Item Store all items in a single table of columns. Using type substitutability, items of all types can be stored and searches can be filtered by item type and subtype using Yukon's "is of(type)" operator.

However, due to the overhead involved in this embodiment that is associated with this approach, items are divided by top-level type such that items for each "family" of types are stored into separate tables. In this partitioning schema, a table is created for each item type that directly inherits from Base.Item. As noted above, types inheriting from the following use type substitutability stored in the appropriate type family table. Only the first level of inheritance from Base.Item is handled exclusively. For the example project hierarchy shown in Figure 29, this results in the following type family table:

create tabele Contact. [Table! Person](

_Item Contact.Person not null

{Change tracking information}

)

create table Doc. [Table! Document](

_Item Doc. Document not null,

{Change tracking information}

)

Use a "shadow" table to store a copy of all items' globally searchable attributes. This table can be maintained by the Update() method of the storage platform API, through which all data changes are made. Unlike the type family table, the global item table only contains the item's top-level scalar attributes, rather than the full UDT item object. The structure of the global items table is as follows:

create table Base. [Table! Item](

ItemID uniqueidentifier not null constraint[PK_Clu_Item! ItemID] primary key clustered,

TypeID uniqueidentifier nul null,

{Additional Properties of Base.Item},

{Change tracking information}

)

The Global Items table allows navigation to Item objects stored in the Type Family table by exposing ItemID and TypeID. ItemID typically uniquely identifies an item in the data store. Metadata not described here can be used to map a TypeID to a type name and a view containing the item.

Since finding an item by its ItemID is a common operation in the context of the global item table as well as in other cases, a GetItem() function is provided to retrieve an item object given an item's ItemId. The function has the following declaration:

Base.Item Base.GetItem(uniqueidentifier ItemID)

For ease of access and to hide implementation details as much as possible, all queries of a project can be made against views built on the tables of said project. Specifically, for each item type, create a view on the family table of the appropriate type. These type views select the associated type, including all items of subtypes. For convenience, in addition to UDT objects, views can display columns for all top-level fields of that type including inherited fields. A view of the sample project hierarchy shown in Figure 29 is as follows:

create view Contact.Person as

select_Item.ItemID, {Property of Base.Item}, {Property of Contact.Person}, {Change Tracking Information}, _Item

from Contact. [Table! Person]

-- Note that the Contact.Employee view uses the "where" judgment

-- to limit the set of items found to instances of Contact.Employee

create view Contact.Employee as

select_Item.ItemID, {Attribute of Base.Item}, {Attribute of Contact.Person}, {Attribute of Contact.Employee},

{Change tracking information}, cast(_Item as Contact.Employee)

from Contact. [Table! Person]

where_Item is of(Contact.Employee)

create view Doc.Document as

select_Item.ItemID, {Property of Base.Item}, {Property of Doc.Document}, {Change Tracking Information}, _Item

from Doc[Table! Document]

-- Note that the Doc.WordDocument view uses the "where" judgment

-- to limit the set of found items to instances of Doc.WordDocument

create view Doc.WordDocument as

select_Item.ItemID, {Property of Base.Item}, {Property of Doc.Document}, {Property of Doc.WordDocument},

{Change Tracking Information}, cast(_Item as Doc.WordDocument)

from Doc. [Table! Document]

where_Item is of(Doc.WordDocument)

For completeness, views can also be created on global project tables. The view initially displays the same columns as the table:

create view Base.Item as

select ItemID, TypeID, {Base.Item properties}, {change tracking information}

from Base. [Table! Item]

3. Extended mapping

Extensions are very similar to projects and have some of the same requirements. As another root type that supports inheritance, extensions are subject to many of the same considerations and trade-offs compared to storage. For this, a similar type family mapping is applied to extensions, rather than a single table approach. Of course, in other embodiments, a single table approach could be used.

In this embodiment, an extension is associated with only one item through an ItemID, and contains an ExtensionID that is unique within the context of the item. An extension table has the following definitions:

create table Base. [Table! Extension](

ItemID uniqueidentifier not null,

ExtensionID uniqueidentifier not null,

TypeID uniqueidentifier not null,

{Properties of Base.Extension},

{CHANGE TRACKING INFO},

constraint[PK_Clu_Extension! ItemID! ExtensionED]

primary key clustered(ItemID asc, ExtensionID asc)

)

As with Items, given an Identity comprising ItemID and ExtensionID pairs, a function may be provided for retrieving Extensions. The function has the following declaration:

Base.Extension Base.GetExtension(uniqueidentifier ItemID, uniqueidentifier ExtensionID)

Similar to item type views, views can be created for each extension type. Assume an extension hierarchy parallel to the example project hierarchy, with the following types: Base.Extension, Contact.PersonExtension, Contact.EmployeeExtension. The following views can be created:

create view Base.Extension as

select ItemID, ExtensionID, TypeID, {Properties of Base.Extension}, {Change Tracking Information}

from Base. [Table! Extension]

create view Contact. [Extension! PersonExtension]as

select_Extension.ItemID, _Extension.ExtensionID, {Properties of Base Extension},

{Properties of Contact.PersonExtension},

{Change Tracking Information}, _Extension

from Base. [Table! PersonExtension]

create view Contact. [Extension! EmployeeExtension]as

select_Extension.ItemID, _Extension.ExtensionID, {Properties of Base.Extension},

{Properties of Contact.PersonExtension},

{Properties of Contact.EmployeeExtension}, {Change Tracking Information},

  cast(_Extension as Contact. EmployeeExtension)

from Base. [Table! PersonExtension]

where_Extension is of(Contact.EmployeeExtension)

4. Nested element mapping

Nested elements are types that can be embedded into items, extensions, relationships, or other nested elements to form deeply nested structures. Similar to items and extensions, nested elements are implemented as UDTs, but they are stored in items and extensions. As such, nested elements have no storage mapping beyond their items and expanding containers. In other words, there are no tables in the system that directly store instances of the NestedElement type, and no views dedicated to nested elements.

5. Object identity

Every entity in the data model, ie every item, extension and relationship has a unique key. An item is uniquely identified by its ItemId. Extensions are uniquely identified by composite keys (ItemId, ExtensionId). Relations are identified by composite keys (ItemId, RelationId). ItemId, ExtensionId and RelationshipId are all GUID values.

6.SQL object naming

All objects created in the data store can be stored in a SQL schema name derived from the storage platform schema name. For example, the storage platform base schema (often referred to as "Base") can generate types in the "[System.Storage]" SQL schema, such as "[System.Storage].Item". The generated names can be prefixed with a qualifier to eliminate naming conflicts. Where appropriate, exclamation points (!) can be used as separators for each logical part of the name. The following table summarizes the naming conventions used for objects in the data store. Each schema element (item, extension, relationship, and view) is listed along with the decorated naming convention used to access instances in the data store.

object name modification describe example main project search view Master! Item Provides an overview of projects in the current project domain [System.Storage].[Master! Item] Typed item search view project type Provides all property data from the item and any parent types [AcmeCorp.Doc].[OfficeDoc] main extended search view Master! Extension Provides an overview of all extensions in the current project domain [System.Storage].[Master! Extension] Typed Extended Search Views Extension! extension type Provide all attribute data to the extension [AcmeCorp.Doc].[Extension! SlickyNote] main relationship view Master! Relationship Provides an overview of all relationships in the current project domain [System.Storage].[Master! Relationship] relationship view Relationship! relationship name Provides all data associated with a given relation [AcmeCorp.Doc].[Relationship! AuthorsFromDocument] view View! view name Based on pattern view [AcmeCorp.Doc].

Defined to provide column/type [View! DocumentTitles]

7. Column naming

When mapping either object model to storage, naming conflicts may occur due to the additional information stored with application objects. To avoid naming conflicts, all non-type-specific columns (columns that do not map directly to named properties in the type declaration) are prefixed with an underscore character (_). In this embodiment, the underscore character (_) is not allowed as the beginning character of any identifier attribute. Furthermore, to unify naming between the CLR and the datastore, all attributes (relations, etc.) of a storage platform type or schema element shall have an uppercase first character.

8. Search View

Views are provided by the storage platform for searching stored content. SQL views are provided for each project and extension type. Additionally, views are provided to support relationships and views (defined by the data model). All SQL views and underlying tables in the storage platform are read-only. As will be described more fully below, data may be stored or changed using the Update( ) method of the storage platform API.

Every view explicitly defined in the storage platform schema (defined by the schema designer, not automatically generated by the storage platform) can be named by the SQL view [<schema name>].[View! <view name>] to access. For example, a view named "BookSales" in the schema "AcmePublisher.Books" can be accessed using the name "[AcmePublisher.Books].[View!BookSales]. Because the view's output format is customized on a per-view basis (by any query definition provided by the party defining the view), the columns are mapped directly based on the schema view definition.

All SQL search views in the storage platform data store use the following collation convention for columns:

1. Logical "key" columns for view results such as ItemId, ElementId, RelationshipId, etc.

2. Metadata information about the result type such as TypeId.

3. Such as Create Version (create version), UpdateVersion (update version) and other change tracking columns

4. Type-specific columns (properties of the declared type)

5. Type-specific views (family views) also contain object columns for returned objects

Members of each type family are searchable using a series of item views, one view for each item type in the data store.

a) project

A per-item search view contains one row for each instance of an item of a particular type or a subtype thereof. For example, a document view can return Document (document), LegalDocument (legal document), and ReviewDocument (review document) instances. Given this example, the project view can be conceptualized as shown in FIG. 28 .

(1) Main Item Search View

Each instance of a storage platform data store defines a special item view called the Master Item View. This view provides overview information about each item in the data store. The view provides one column for each item type attribute, one describing the type of item, and several columns providing change tracking and synchronization information. Use the name "[System.Storage].[Master!Item]" in the data store to identify the master item view.

List type describe ItemId ItemId The project's storage platform identity _TypeId TypeId TypeId of the item - identifies the exact type of the item and can be used to retrieve information about the type using the metadata category _RootItemId ItemId The ItemId of the first non-embedded ancestor that controls the lifetime of this item <global change tracking> ... Global change tracking information <project properties> none There is one column for each item type attribute

(2) Typed item search view

Each item type also has a search view. Although similar to the root item view, this view also provides access to item objects through the "_Item" column. Each typed item search view is identified in the data store by the name [schema name].[item type name]. For example [AcmeCorp. Dod]. [OfficeDoc].

List type describe ItemId ItemId The project's storage platform identity <type change tracking> ... type change tracking information <parent attribute> <property only> One column for each parent attribute <project properties> <property only> There is a column for each exclusive attribute of this type Item The CLR type of the project CLR Object - the type of declared item

b) Project extension

All project extensions in WinFs storage can also be accessed using the search view.

(1) Main Extended Search View

Each instance of the data store defines a special extension view called the master extension view (Master Extension View). This view provides overview information about each extent in the datastore. The view has one column for each extension attribute, one describing the type of extension, and several columns providing change tracking and synchronization information. Use the name "[System.Storage].[Master!Extension]" to identify the master extension view in the data store.

List type describe ItemId ItemId The storage platform identity of the project associated with this extension ExtensionId ExtensionId(GUID) The id of this extension instance _TypeId TypeId TypeId of the extension - identifies the exact type of the extension and can be used to retrieve information about the extension using metadata categories <global change tracking> ... Global change tracking information <extended attribute> <property only> There is a column for each extension type attribute

(2) Typed extended search view

Each extension type also has a search view. Although similar to the main extensions view, this view also provides access to project objects through the _Extension column. Use the name [schema-name].[Extension! extension_type_name] identifies each typed extension search view. For example [AcmeCorp.Doc].[Extension! OfficeDocExt].

List type describe ItemId ItemId The storage platform identity of the project associated with this extension ExtensionId ExtensionId(GUID) The Id of this extension instance <type change tracking> ... type change tracking information <parent attribute> <property only> One column for each parent attribute <extended attribute> <property only> There is a column for each exclusive attribute of this type _Extension The CLR type of the extension instance CLR object - the type of the declared extension

c) nested elements

All nested elements are stored in Item, Extension or Relationship instances. Therefore, they can be accessed by querying the appropriate project, extension or relational search view.

d) relationship

As discussed above, relationships form the basic unit of linking between items in a storage platform data store.

(1) Main relation search view

Each data store provides a master relational view. This view provides information about all relational instances in the datastore. Use the name "[System.Storage].[Master!Relationship]" in the data store to identify the master relational view.

List type describe ItemId ItemId The identity of the source endpoint (ItemId) RelationshipId RelationshipId(GUID) The id of the relation instance _RelTypeId RelationshipsPTypeId RelTypeId for this relation - uses metadata category to identify the type of this relation instance <global change tracking> ... Global change tracking information TargetItemReference ItemReference The identity of the target endpoint _Relationship Relationship An instance of the Relationship object for this instance

(2) Relation instance search view

Each declared relationship also has a search view that returns all instances of that particular relationship. Although similar to the main relational view, this view provides named columns for each attribute of the relational data. Use the name [schema-name].[Relationship! relation name] to identify each relation instance in the search view. For example [AcmeCorp.Doc].[Relationship! DocumentAuthor].

List type describe ItemId ItemId The identity of the source endpoint (ItemId) RelationshipId RelationshipId(GUID) The id of the relation instance

<type change tracking> ... type change tracking information TargetItemReference ItemReference The identity of the target endpoint <source name> ItemId Named property of source endpoint identity (alias for ItemId) <target name> ItemReference or exported class Named properties of the target endpoint identity (alias and type cast to TargetItemReference) <relationship attribute> <property only> There is a column for each attribute defined by the relation _Relationship The CLR type of the relation instance Types of CLR object-declaration relationships

9. Update

All views in the storage platform data store are read-only. To create new instances of data model elements (items, extensions, or relationships), or to update existing instances, the ProcessOperation or ProcessUpdategram methods of the storage platform API must be used. A ProcessOperation method is a single stored procedure defined by a datastore that consumes an "operation" that refines the action to be performed. The ProcessUpdategram method is a stored procedure that takes an ordered set of operations called "updategrams" that together refine the set of actions to be performed.

The operation format is extensible and provides various operations on schema elements. Some common operations include:

1. Project operation:

a.CreateItem (create a new item in the context of embedding or holding a relationship)

b.UpdateItem (update an existing item)

2. Relational operations:

a.CreateRelationship (create a reference or hold an instance of the relationship)

b.UpdateRelationship (update a relationship instance)

c.DeleteRelationship (remove a relationship instance)

3. Extended operation

a.CreateExtension (add an extension to an existing project)

b.UpdateExtension (update an existing extension)

c.DeleteExtension (delete an extension)

10. Change tracking and tombstones

As discussed more fully below, change tracking and tombstone services are provided by the data store. This section provides an overview of the change tracking information presented in the data store

a) change tracking

Each search view provided by the data store contains columns for providing change tracking information; those columns are common to all project, extension and relational views. A storage platform schema view explicitly defined by the schema designer does not automatically provide change tracking information - this information is provided indirectly through the search view upon which the view itself is built.

For each element in the data store, change tracking information is available from two places: the "main" element view and the "typed" element view. For example, change tracking information on the AcmeCorp.Document.Document item type is available from the master item view '[System.Storage].[Master!Item]' and the typed item view [AcmeCorp.Document].[Document].

(1) Change tracking in the "main" search view

The change tracking information in the main search view provides information about the created and updated versions of an element, information about which sync partner created the element, information about which sync partner last updated the element, and information from each partner for creating and the updated version number. A partner key is used to identify a partner in a synchronization relationship (described below). A single UDT object named _ChangeTrackingInfo of type [System.Storge.Store].ChangeTrackingInfo contains all this information. Define types in the System.Storage schema. _ChangeTrackingInfo is available in all global search views for projects, extensions, and relationships. The type definition of _ChangeTrackingInfo is:

<Type Name="ChangeTrackingInfo"BaseType="Base.NestedElement">

<FieldProperty Name="CreationLocalTS"Type="SqlTypes.SqlInt64"

Nullable="False"/>

<FieldProperty Name="CreatingpartnerKey"Type="SqlTypes.SqlInt32"

Nullable="False"/>

<FieldProperty Name="CreatingPartnerTS"Type="SqlTypes.SqlInt64"

Nullable="False"/>

<FieldProperty Name="LastUpdateLocalTS"Type="SqlTypes.SqlInt64"

Nullable="False"/>

<FieldProperty Name="LastUpdatingPartnerKey"Type="SqlTypes.SqlInt32"

Nullable="False"/>

<FieldProperty Name="LastUpdatingPartnerTS"Type="SqlTypes.SqlInt64"

Nullable="False"/>

</Type>

These properties contain the following information:

List describe _CreationLocalTS Creation time stamp of the local machine _CreatingPartnerKey The PartnerKey of the partner who created this entity. If the entity was created locally, this is the PartnerKey of the local machine _CreatingPartnerTS A timestamp of the time this entity was created at the partner corresponding to _CreatingPartnerKey _LastUpdateLocalTS A local timestamp corresponding to the update time of the local machine _LastUpdatePartnerKey The PartnerKey of the partner who last updated this entity. This is the PartnerKey of the local machine if the last update to this entity was done locally. _LastUpdatePartnerTS A timestamp of when this entity was updated at the partner corresponding to _LastUpdatingPartnerKey.

(2) Change Tracking in "Typed" Search Views

In addition to providing the same information as the global search view, each typed search view provides additional information about the synchronization status of each element recorded in the synchronization topology.

List type describe <global change tracking> ... Information from global change tracking _ChangeUnitVersions MultiSet<change unit version> A description of the version number of the unit of change in a particular element _ElementSyncMetadata ElementSyncMetadata Additional version-independent metadata about items that are only interested in synchronizing the runtime _VersionSyncMetadata VersionSyncMetadata Additional version-specific metadata about versions that are only interested in synchronizing the runtime

b) tombstone

Datastores provide tombstone information for projects, extensions, and relationships. The tombstone view provides information about both live and tombstone entities (items, extensions and relationships) in one place. Project and extension tombstone views provide no access to corresponding objects, while relational tombstone views provide access to relational objects (relational objects are null in case of tombstoned relations).

(1) Project Tombstone

Via view[System.Storage].[Tombstone! item] Retrieves the item tombstone from the system.

List type describe ItemId ItemId The identity of the project _TypeID TypeId type of item <project properties> ... Attributes defined for all items _RootItemId ItemId ItemId of the first non-embedded item that contains this item _ChangeTrackingInfo A CLR instance of the ChangeTrackingInfo type Change tracking information for this project _IsDeleted BIT This is the flag, 0 is a live project, 1 is a tombstone project _DeletionWallclock UTCDATETIME By the UTC wall clock datetime of the partner of the deleted item, it is empty if the item is alive

(2) Extended Tombstone

Use the view [System.Storage].[Tombstone! Extension] Retrieves the extension tombstone from the system. Extension change tracking information is similar to that provided for projects with the addition of the ExtensionId property.

List type describe ItemId ItemId The identity of the project that owns the extension ExtensionId ExtensionId The ExtensionId of the extension _TypeID TypeId The type of extension _ChangeTrackingInfo A CLR instance of the ChangeTrackingInfo type Change tracking information for this extension _IsDeleted BIT This is the flag, 0 is the live project, 1 is the tombstone extension _DeletionWallclock UTCDATETIME Press the UTC wall clock date period of the partner that removed the extension. It is empty if the extension is active

(3) Relationship Tombstone

Via view[System.Storage].[Tombstone! Relationship] Retrieves relationship tombstones from the system. Relationship tombstone information is similar to the information provided for extensions. However, additional information is provided on the target ItemRef of the relation instance. Also, select the relationship object.

List type describe ItemId ItemId The identity of the project that owns the relationship (the identity of the source endpoint of the relationship) RelationshipId RelationshipId The RelationshipId of the relationship _TypeID TypeId type of relationship _ChangeTrackingInfo A CLR instance of the ChangeTrackingInfo type Change tracking information for this relationship _IsDeleted BIT This is the flag, 0 is the live project, 1 is the tombstone extension _DeletionWallclock UTCDATETIME By the UTC wall clock datetime of the partner who deleted the relationship. It is null if the relation is alive _Relationship The CLR instance of the relationship This is the relationship object for live relationships, it is empty for tombstone relationships TargetItemReference ItemReference The identity of the target endpoint

(4) Tombstone removal

To prevent unlimited growth of tombstone information, the data store provides a tombstone removal task. This task determines when tombstone information can be discarded. This task computes the locally created/updated version bounds and then truncates the tombstone information by discarding all earlier tombstone versions.

11. Helper API and functions

The base map also provides several helper functions. These functions are provided to facilitate common operations on this data model.

a) Function [System.Storage].GetItem

//Given ItemId returns an item object

Item GetItem(ItemId ItemId)

b) Function [System.Storage].GetExtension

//Given ItemId and ExtensionId return an extension object

Extension GetExtension(ItemId ItemId, ExtensionId ExtensionId)

c) Function [System.Storage].GetRelationship

//Given ItemId and RelationshipId returns -Relationship object

Relationship GetRelationship(ItemId ItemId, RelationshipId RelationshipId)

12. Metadata

There are two types of metadata represented in the store: instance metadata (type of item, etc.), and type metadata.

a) schema metadata

Schema metadata is stored in the data store as instances of item types from the meta-schema.

b) instance metadata

Applications use instance metadata to query an item's type and to find extensions associated with the item. Given an item's ItemId, an application can query the global item view to return the item's type, and use this value to query the Meta.Type view to return information about the item's declared type. For example,

// Returns the metadata item object for the given item instance

SELECT m._Item AS metadataInfoObj

FROM[System.Storage].[Item]i INNER JOIN[Meta].[Type]m ON i._TypeId＝m.ItemId

WHERE i.ItemId = @ItemId

E. Security

This section describes the security model for the storage platform of the present invention according to one embodiment.

1. Overview

According to the present embodiment, the granularity of specifying and enforcing the storage platform's security policy is at the level of various operations on items in a given data store; there is no ability to secure parts of an item separately from the whole. This security model specifies, through access control lists (ACLs), the subjects that can be granted or denied access to perform these operations on items. Each ACL is an ordered collection of Access Control Entries (ACEs).

The security policy for a project can be fully described by freely selected access control policies and system access control policies. Each of these is a set of ACLs. The first set (DACL) describes the discretionary access rights granted to various principals by the project's owner, while the second set of ACLs is known as the SACL (System Access Control List), which specifies when objects are manipulated in certain ways How to complete system monitoring. In addition to these, each item in the data store is associated with a SID corresponding to the owner of the item (owner SID).

The primary mechanism used to organize items in a storage platform data store is the one that includes a hierarchical structure. Containment hierarchies are implemented using holding relationships between items. A holding relationship expressed as "A contains B" between two projects A and B enables project A to influence the lifecycle of project B. In general, an item in a data store cannot exist until it has a holding relationship from another item to it. In addition to controlling the lifecycle of an item, the holding relationship provides the mechanism necessary to propagate the item's security policy.

The security policy specified for each project consists of two parts - the part explicitly specified for that project and the part inherited from the project's parent project in the data store. A well-defined security policy for any item consists of two parts - the part that governs access to the item under consideration, and the part that affects the security policy inherited by all its descendants in the containing hierarchy. Security policies inherited by descendants are determined by well-defined policies and inherited policies.

Since security policies are propagated through holding relationships and can also be overridden at any project, it is necessary to specify how to determine the project's effective security policy. In this embodiment, the data store contains items in the hierarchy that inherit the ACL along each path from the root of the store to the item.

In an ACL inherited for any given path, the ordering of the ACEs in the ACL determines the final security policy that is enforced in real time. The following notation is used to describe the ordering of ACEs in an ACL. The ordering of ACEs in an ACL inherited by items is determined by the following two rules −

The first rule satisfies the ACEs inherited from each item in the path from the root containing the hierarchy to item I. ACEs inherited from closer containers take precedence over entries inherited from farther containers. Intuitively, this allows administrators to override inherited ACEs from further down the containment hierarchy. The rule is as follows:

For all inherited ACL L on project I

For all items I1, I2

For all ACEs A1 and A2 in L

I1 is an ancestor of I2, and

I2 is an ancestor of I3, and

A1 is an ACE inherited from I1, and

A2 is an ACE inherited from I2

mean

A2 takes precedence over A1 in L

The second rule orders ACEs that deny access to an item before ACEs that grant access to an item.

For all inherited ACL L on project I

For all items I1

For all ACEs A1 and A2 in L,

I1 is an ancestor of I2, and

A1 is ACCESS_DENIED_ACE inherited from I1, and

A2 is ACCESS_GRANTED_ACE inherited from I1

mean

A1 takes precedence over A2 in L

In the case where the containment hierarchy is a tree, there is only one path from the root of the tree to the item, and the item has only one inherited ACL. In these cases, the ACLs inherited by the project match the ACLs inherited by the file (project) in the existing Windows security model, by the relative ordering of the ACEs therein.

However, the containment hierarchy in the data store is a directed acyclic graph (DAG), since multiple holding relationships are allowed for items. In these cases, there are multiple paths from the root node containing the hierarchy to the item. Since items bear an ACL along each path, each item is associated with a set of ACLs rather than a single ACL. Note that this differs from the traditional filesystem model, where only one ACL is associated with a file or folder.

When the containing hierarchy is a DAG rather than a tree, there are usually two aspects to elaborate. There needs to be a description of how the effective security policy is computed for an item when it inherits more than one ACL from its parent, and how the organization and representation of items has direct implications for the management of the security model of a storage platform data store.

The following algorithm evaluates the access rights of a given principal to a given item. Throughout this document, the following notation is used to describe ACLs associated with items.

Inherited_ACLs (ItemId) - A set of ACLs inherited by the item whose item identity is ItemId from its parent item in the store.

Explicit_ACL(ItemId) - ACL explicitly defined for the item whose identity is ItemID.

NTSTATUS

ACLAccessCheck(

PSID pOwnerSid,

PDACL pDacl,

DWORD DesiredAccess,

HANDLE ClientToken,

PPRIVILEGE_SET pPrivilegeSet,

DWORD *pGrantedAccess)

If the desired access rights are not explicitly denied, the above routine returns STATUS_SUCCESS, and pGrantedAccess determines which of the user's desired rights are granted by the specified ACL. If the desired access is explicitly denied, this routine returns STATUS_ACCESS_DENIED.

NTSTATUS

ItemAccessCheck(

OS_ITEMID ItemId,

DWORD DesiredAccess,

HANDLE ClientToken,

PPRIVILEGE_SET pPrivilegeSet)

{

STATUS Status;

PDACL pExplicitACL=NULL;

PDACL pInheritedACLs=NULL;

DWORD NumberOfInheritedACLs=0;

pExplicitACL = GetExplicitACLForItem(ItemId);

GetInheritedACLsForItem(ItemId, &pInheritedACLs, &NumberOfInheritedACLs)

Status＝ACLAccessCheck(

pOwnerSid,

pExplicitACL,

DesiredAccess,

ClientToken,

pPrivilegeSet,

&GrantedAccess);

if(Status!=STATUS_SUCCESS)

return Status;

if(DesiredAccess==GrantedAccess)

return STATUS_SUCCESS;

for(

i=0;

(i<NumberOfInheritedACLs&&Status==STATUS_SUCCESS);

i++){

GrantedAccessForACL=0;

Status＝ACLAccessCheck(

pOwnerSid,

pExplicitACL,

DesiredAccess,

ClientToken,

pPrivilegeSet,

&GrantedAccessForACL);

if(Status==STATUS_SUCCESS){

GrantedAccess|=GrantedAccessForACL;

}

}

If((Status==STATUS_SUCCESS)&&

(GrantedAccess!=DesiredAccess)){

Status = STATUS_ACCESS_DENIED;

}

return Status;

}

A security policy defined at any project extends to descendants of that project in the containment hierarchy defined on the datastore. For all items that define an explicit policy, the effect is as if defining a policy that is inherited by all of its descendants in the containing hierarchy. An effective ACL inherited by all descendants can be obtained by taking the ACL inherited by the project and adding the inheritable ACEs from the explicit ACL to the beginning of the ACL. This is known as the set of inheritable ACLs associated with the project.

When the root node lacks an explicit specification of security in the containing hierarchy at a folder item, the folder's security specification applies to all descendants of that item in the containing hierarchy. Thus, each item for which an explicit security policy designation is provided defines a similarly protected area of items, and the effective ACL for all items within that area is the set of inheritable ACLs for that item. In cases where the containing hierarchy is a tree, this will completely define the region. If each area is to be associated with a number, it is sufficient to include with the item only the area to which the item belongs.

However, for a containment hierarchy that is a DAG, the point in the containment hierarchy at which the effective security policy changes is generally determined by two types of items. The first category is projects that have an explicit ACL assigned to them. Typically, these are the points in the containment hierarchy at which ACLs are explicitly specified by the administrator. The second category is items that have more than one parent node, each parent node having a different security policy associated with it. Typically, these are entries at the convergence point of the security policy specified for the volume, and indicate the start of a new security policy.

Using this definition, items in the data store fall into one of two categories - those items that are the root node of a similarly protected security area, and those items that are not the root node. Projects that do not define a security zone generally belong to only one security zone. As in the case of trees, the effective security of an item can be specified by specifying the zone to which the item belongs and the item. This leads to a straightforward model for managing the security of storage platform data storage based on each similarly protected area in storage.

2. Detailed description of the security model

This section provides details on how items are secured by describing how individual permissions within security descriptors and the ACLs they contain affect various operations.

a) Security Descriptor Structure

Before describing the details of the security model, a basic discussion of security descriptors is helpful. A security descriptor contains security information associated with a securable object. A security descriptor consists of a SECURITY_DESCRIPTOR (security descriptor) structure and its associated security information. A security descriptor can include the following security information:

1. The owner of the object and the SID of the primary group.

2. A DACL that specifies the access rights that are allowed or denied to specific users or groups of users.

3. A SACL that specifies the type of access attempt that generates an inspection record for the object.

4. A set of control bits that define the meaning of a security descriptor or its individual members.

Preferably, applications cannot directly manipulate the contents of the security descriptor. Functions exist for setting and retrieving security information in an object's security descriptor. In addition, functions exist for creating and initializing security descriptors for new objects.

A discretionary access control list (DACL) identifies trustees who are allowed or denied access to a protectable object. When a process attempts to access a protectable object, the system checks the ACE in the object's DACL to determine whether to grant access to it. If the object has no DACL, the system grants full access to everyone. If the object's DACL does not have an ACE, the system denies the attempt to access the object because the DACL does not allow any access rights. The system checks the ACEs in turn until it finds one or more ACEs that allow all of the requested access rights, or until any of the requested access rights are denied.

System Access Control Lists (SACLs) enable administrators to log attempts to access protected objects. Each ACE specifies the type of access attempt for a given trustee that causes the system to generate a record in the security event log. ACEs in a SACL can generate audit records when an access attempt fails, when it succeeds, or both. SACL alerts when an unauthorized user attempts to gain access to an object.

All types of ACEs contain the following access control information:

1. A security identifier (SID) identifying the trustee to which the ACE is applied.

2. Specify the access mask for the access rights controlled by this ACE.

3. A flag indicating the type of the ACE.

4. A set of bit flags that determine whether a child container or object can inherit an ACE from the main object to which the ACL is attached.

The table below lists the three ACE types supported by all protectable objects.

type describe Access Denied ACE Used in DACL to deny access to trustees. Access Allow ACE Used in DACLs to grant access to trustees. System Monitoring ACE Used in SACL to generate audit records when the trustee attempts to exercise the specified access rights.

(1) Access mask format

All protectable objects can have their access rights assigned using the access mask format shown in FIG. 26 . In this format, the lower 16 bits are used for object-specific access rights, the next 7 bits are used for standard access rights that apply to most types of objects, and the highest 4 bits are used to specify that each object type can be mapped to a Generic access for group-standard and object-specific permissions. The ACCESS_SYSTEM_SECURITY (Access System Security) bit corresponds to the permission to access the object's SACL.

(2) Generic Access Rights

Generic permissions are specified in the 4 most significant bits within the mask. Each class of manageable objects maps these bits to its set of standard and object-specific access rights. For example, a file object may map GENERIC_READ bits to READ_CONTROL and SYNCHRONIZE standard access rights, and to FILE_READ_DATA, FILE_READ_EA, and FILE_READ_ATTRIBUTES object-specific access rights. Objects of other types map the GENERIC_READ bits to the access rights groups applicable to objects of that type.

Generic access can be used to specify the type of access required when opening a handle to an object. This is usually simpler than specifying all corresponding standard and private permissions. The table below shows the constants defined for generic access rights.

constant generic meaning GENERIC_ALL Read, write and execute access GENERIC_EXECUTE execute access GENERIC_READ read access GENERIC_WRITE write access

(3) Standard Access Rights

Each class of maintainable objects has a set of access rights that correspond to operations specific to that class of objects. In addition to these object-specific access rights, there is a set of standard access rights corresponding to operations common to most types of securable objects. The table below shows the constants defined for standard access rights.

constant meaning DELETE The permission to delete an object. READ_CONTROL The right to read the information in the object's security descriptor, excluding the information in the SACL. SYNCHRONIZE Permissions to use this object for synchronization. This enables the thread to wait until the object is in a notified state. Certain object types do not support this access right. WRITE_DAC Permission to modify the DACL in the object's security descriptor. WRITE_OWNER Change the owner's permissions in the object's security descriptor.

b) Project-specific permissions

In the access mask structure of FIG. 26, item-specific rights are placed in the object-specific rights field (lower 16 bits). Since, in this embodiment, the storage platform exposes two sets of APIs to manage security—Win32 and the storage platform API, file system object-specific permissions must be considered to motivate the design of storage platform object-specific permissions.

(1) File and directory object-specific permissions

Consider the following table:

Table of contents directory description document File Description value FILE_LIST_DIRECTORY Permission to list the contents of a directory FILE_READ_DATA Permission to read corresponding file data 0x0001 FILE_ADD_FILE Permission to create files in the directory FILE_WRITE_DATA Permission to write data to the file 0x0002 FILE_ADDSUBDIRECTORY Permission to create subdirectories FILE_APPEND_DATA Permission to append data to the file 0x0004 FILE_READ_EA Permission to read extended file attributes FILE_READ_EA Permission to read extended file attributes 0x0008 FILE_WRITE_EA Permission to write extended file attributes FILE_WRITE_EA Permission to write extended file attributes 0x0010 FILE_TRAVERSE Permission to traverse directories FILE_EXECUTE For native code 0x0020

limit file, the permission to execute the file FILE_DELETE_CHILD Permission to delete a directory and all files it contains none none 0x0040 FILE_READ_ATTRIBUTES Permission to read directory attributes FILE_READ_ATTRIBUTES Permission to read file attributes 0x0080 FILE_WRITE_ATTRIBUTES Permission to write directory attributes FILE_WRITE_ATTRIBUTES Permission to write file attributes 0x0100

Referring to the above table, note that the file system makes a fundamental difference between files and directories, which is why file and directory permissions can overlap at the same bit. The file system defines very granular permissions, allowing applications to control behavior on these objects. For example, they allow applications to distinguish between attributes associated with files (FILE_READ/WRITE_ATTRIBUTES), extended attributes, and data streams.

The goal of the security model of the storage platform of the present invention is to simplify the rights assignment model, so that for example applications operating on data storage items (contacts, emails, etc.) generally do not need to distinguish between attributes, extended attributes and data streams. However, for files and folders, granular Win32 permissions are preserved and access semantics through the storage platform are defined so that compatibility with Win32 applications can be provided. This mapping is discussed for each project authority specified below.

The following project permissions are specified with their associated allowable actions. The equivalent Win32 permissions behind each of these project permissions are also provided.

(2)WinFSItemRead

This permission allows read access to all elements of an item, including items linked to the item via an embedding relationship. It also allows enumeration of items linked to the item by holding relationships (also known as directory listings). This includes the names of items linked through reference relationships. This permission maps to:

document:

(FILE_READ_DATA|SYNCHRONIZA)

folder:

(FILE_LIST_DIRECTORY|SYNCHRONIZE)

The semantics are that the security application can set WinFSItemReadData (WinFS item read data), and specify the permission mask as a combination of the file permissions specified above.

(3)WinFSItemReadAttributes

This permission allows read access to basic attributes of the item, much like the file system distinction between basic file attributes and data streams. Preferably, these base properties are those properties residing in the base item from which all items are derived. This permission maps to:

document:

(FILE_READ_ATTRIBUTES)

folder:

(FILE_READ_ATTRIBUTES)

(4)WinFSItemWriteAttributes

This permission allows write access to basic attributes of the item, much like the file system distinction between basic file attributes and data streams. Preferably, these base properties reside in those properties in the base item from which all items are derived. This permission maps to:

document:

(FILE_WRITE_ATTRIBUTES)

folder:

(FILE_WRITE_ATTRIBUTES)

(5)WinFSItemWrite

This permission allows the ability to write on all elements of an item, including items linked by embedding relationships. This permission also allows the ability to add or remove embedded relationships to other items. This permission maps to:

document:

(FILE_WRITE_DATA)

folder:

(FILE_ADD_FILE)

In a storage platform data store, there is no distinction between items and folders, since items can also have holding relationships to other items in the data store. Thus, if you have the FILE_ADD_SUBDIRECTORY (or FILE_APPEND_DATA) permission, you can make a project the source of relationships to other projects.

(6)WinFSItemAddLink

This permission allows the ability to add hold relationships to items in the store. It should be noted that WRITE_DAC is required on the destination item due to the security model for multiple holding relationships changing security on the item, and if coming from a higher point in the hierarchy, the change can bypass WRITE_DAC Ability to create relationships to it. This permission maps to:

document:

(FILE_APPEND_DATA)

folder:

(FILE_ADD_SUBSIRECTORY)

(7)WinFSItemDeleteLink

This permission allows the ability to delete a holding relationship to an item even when the principal is not granted permission to delete the item. This is consistent with the file system model and facilitates cleanup. This permission maps to:

document:

(FILE_DELETE_CHILD) - Note that the filesystem has no equivalent of this permission for files, but has the notion of having a holding relationship to other items, and thus implements this permission for non-folders as well.

folder:

(FILE_DELETE_CHILD)

(8) Permission to delete items

If the last holding relationship to an item disappears, the item is deleted. There is no explicit notation for removing an item. There is a cleanup operation that removes all holding relationships to items, but it is a higher level tool and not a system primitive.

Any item specified with a path can be unlinked if either (1) a parent item along that path grants write permission to the item, or (2) standard permissions on the item itself authorize DELETE . When the last relationship is removed, the item disappears from the system. Any item specified with an ItemID can be unlinked if the standard permissions on the item itself authorize DELETE.

(9) Permission to copy items

Items can be copied from the source to the destination folder if the principal is authorized for WinFSItemRead on the item and WinFSItemWrite on the destination folder.

(10) Permission to move items

Moving a file in the file system requires only DELETE permission on the source file and FILE_ADD_FILE on the destination directory, since it preserves the ACL on the destination. However, a flag can be specified in the MoveFileEx call (MOVEFILE_COPY_ALLOWED) that lets the application specify that CopyFile semantics are tolerated in the case of moves across volumes. There are four possible options for what happens to the security descriptor after the move:

1. Carry the entire ACL along with the file - the default intra-volume move semantics.

2. Carry the entire ACL along with the file and mark the ACL as protected.

3. Only carry explicit ACE spans and re-inherit on the destination.

4. Carry nothing and re-inherit on the destination - the default inter-volume move semantics - as if copying the file.

In this security model, if the application specifies the MOVEFILE_COPY_ALLOWED flag, the fourth option is enforced for both intra-volume and inter-volume cases. If this flag is not specified, the second option is executed unless the destination is also in the same security zone (ie, same inheritance semantics). Storage platform-level moves also implement a fourth option, and require READ_DATA on the source, as does copying.

(11) View the permissions of the security policy on the project

View security for a project if the project grants the standard permission READ_CONTROL to the principal.

(12) Change the permissions of the security policy on the project

If the project grants the standard permission WRITE_DAC to the principal, the security of the project can be changed. However, since the data store provides implicit inheritance, this has implications for how security can vary hierarchically. The rule is that if the root of the hierarchy authorizes WRTIE_DAC, then the security policy is changed across the entire hierarchy, regardless of whether a particular item in the hierarchy (or DAG) does not authorize WRITE_DAC to the principal.

(13) Authority without direct equivalent

In this embodiment, FILE_EXECUTE (FILE_TRAVERSE for directories) has no direct equivalent in storage platforms. The model preserves these permissions for Win32 compatibility, but does not make any access decisions for projects based on these permissions. As with FILE_READ/WRITE_EA, the semantics of this bit are not provided since data store items do not have a notation for extended attributes. However, this bit is reserved for Win32 compatibility.

3. Realize

All items that define areas that are also protected have an entry associated with them in the security table. The security table is defined as follows:

project identity Project orderly path Explicit project ACL Path ACL Zone ACLs

The project identity entry is the project identity of the root of the security area that is also protected. An item ordered path is an ordered path (ordpath) associated with the root of a security area that is also protected. An explicit project ACL entry is an explicit ACL defined for the root of a security zone that is also protected. This can be empty in some cases, for example when defining a new security area because the project has multiple parent projects belonging to different areas. Path ACLs are the set of ACLs inherited by the project, and zone ACL entries are the set of ACLs defined for similarly protected security zones associated with the project.

Computation of the effective security for any item in a given store makes full use of this table. To determine the security policy associated with a project, get the security zone associated with the project and retrieve the ACL associated with that zone.

This security table is kept up-to-date when the security policy associated with a project is changed, either directly by adding explicit ACLs or by indirectly adding holding relationships that result in the formation of new security zones, to ensure the Algorithms work.

The various changes to storage and accompanying algorithms for maintaining the safety table are as follows:

a) Create a new project in the container

When an item is newly created in a container, it inherits all ACLs associated with that container. Since the newly created project has only one parent project, it belongs to the same region as its parent project. As a result, no new entries need to be created in the security table.

b) Add an explicit ACL to the project

When an ACL is added to a project, it defines new security zones for all of its descendants in the containing hierarchy that belong to the same security zone as the given project itself. For all items in the containment hierarchy that belong to other security areas but are descendants of the given item, the security area remains unchanged, but the effective ACL associated with that area is changed to reflect the addition of the new ACL.

The introduction of this new security zone may trigger further definition of zones for all items that have multiple holding relationships with ancestors spanning the old security zone and the newly defined security zone. For all such projects, new security zones need to be defined and the process repeated.

Figures 27(a), (b) and (c) depict new security zones that are also protected from existing security zones by introducing new explicit ACLs. This is indicated by the node labeled 2. However, the introduction of this new area results in the creation of an additional area 3 because the project has multiple holding relationships.

The following sequence of updates to the security table reflects a breakdown of the security regions that are also protected.

c) Add a holding relationship to the project

When adding a holding relationship to a project, it raises one of three possibilities. If the target of the holding relationship, ie the item under consideration, is the root of a security zone, the effective ACL associated with that zone is changed and no further modification of the security table is required. If the security zone of the source of the new holding relationship is the same as the security zone of the project's existing parent project, no changes are required. However, if the project now has a parent project that belongs to a different security area, a new security area is formed with the given project as the root of that security area. This change is propagated to all items in the containing hierarchy by modifying the security zone associated with that item. All items and their descendants in the containment hierarchy that belong to the same security area as the item under consideration need to be changed. Once a change is made, all projects with multiple holding relationships need to be checked to determine if further changes are required. If any of these projects have parent projects with different security zones, further changes are required.

d) Remove the holding relationship from the project

When removing a holding relationship from a project, it is possible to collapse a security area with its parent area if certain conditions are met. More precisely, this can be done under the following conditions: (1) if the removal of the holding relationship results in an item that has a parent item and no explicit ACL is specified for that item; (2) if the removal of the holding relationship results in A project whose parents are both in the same security zone and for which no explicit ACL is defined. In these cases, these security areas can be marked the same as the parent project. This flag needs to be applied to all items whose safe area corresponds to the collapsed area.

e) Remove the explicit ACL from the project

When explicit ACLs are removed from a project, it is possible to collapse the security zone rooted at that project with the security zone of its parent project. More precisely, this process can be achieved if the removal of the explicit ACL results in an item whose parent item in the containment hierarchy belongs to the same security area. In these cases, the safe area can be marked the same as the parent item, and the change is applied to all items whose safe area corresponds to the collapsed area.

f) Modify the ACL associated with the project

In this case, no new additions to the security table are required. The effective ACL associated with the zone is updated, and the new ACL change is propagated to the security zones affected by it.

F. Notifications and Change Tracking

According to another aspect of the invention, the storage platform provides notification capabilities that allow applications to track data changes. This trait is primarily intended for use by applications that maintain volatile state or perform business logic on data change events. Applications register for notifications on items, item extensions, and item relationships. Notifications are delivered asynchronously after data changes are committed. Applications can filter notifications by project, extension and relationship type, and action type.

According to one embodiment, the storage platform API 322 provides two types of interfaces for notifications. First, the application registers for simple data change events triggered by changes to items, item extensions, and item relationships. Second, the application creates a "watcher" object to monitor groups of items, item extensions, and relationships between items. The state of the watchdog object can be saved and recreated after a system failure or system offline for an extended period of time. A single notification can reflect multiple updates.

1. Storage change event

This section provides several examples of how to use the notification interfaces provided by Storage Platform API 322.

a) event

Items, Item Extensions, and Item Relationships represent data change events used by applications to register for data change notifications. The following code example shows the definition of the ItemModified (item modified) and ItemRemoved (item removed) event handlers on the Item base class.

//event

public event ItemModifiedEventHandler Item ItemModified ;

public event ItemRemovedEventHandler Item ItemRemoved ;

All notifications carry enough data to retrieve changed items from the data store. The following code samples show how to register for events on items, item extensions, or item relationships:

myItem.ItemModified+=new ItemModifiedEventHandler(this.onItemUpdate);

myItem.ItemRemoved+=new ItemRemovedEventHandler(this.onItemDelete);

In this embodiment, the storage platform ensures that the application will be notified if the corresponding item has been modified or deleted since the last notification was delivered, or in the case of a new registration since the last fetch from the data store.

b) Monitoring program

In this embodiment, the storage platform defines watcher classes for monitoring objects associated with (1) folders or folder hierarchies, (2) project contexts, or (3) specific projects. For each of these three categories, the storage platform provides specific watcher classes that monitor associated items, item extensions, or item relationships, for example, the storage platform provides corresponding FolderItemWatcher (folder item watcher), FolderRelationshipWatcher (file Folder relationship monitor) and FolderExtensionWatcher (folder extension monitor) class.

When creating a watcher, an application can request notifications on pre-existing items, ie, items, extensions, or relationships. This option is mostly used by applications that maintain a private item cache. If not requested, the application receives notifications of all updates that have occurred since the watcher object was created.

Along with delivery notifications, the storage platform provides a "WatcherState" object. WatcherState can be serialized and saved on disk. The watchdog state can then be used to recreate the corresponding watchdog on failure or when reconnecting after going offline. A new recreated watcher will regenerate unacknowledged notifications. The application indicates delivery of a notification by calling an "Exclude" method on the corresponding watchdog state providing a reference to the notification.

The storage platform delivers a separate copy of the watchdog state to each event handler. Received watchdog state on subsequent invocations of the same event handler assumes delivery of all previously received notifications.

As an example, the following code sample shows the definition of FolderItemWatcher.

public class FolderItemWatcher：Watcher

{

//Constructor

public FolderItemWatcher_Constructor(Folder folder);

 public FolderItemWatcher_Constructor1(Folder folder, Type itemType);

 public FolderItemWatcher_Constructor2(ItemContext context, ItemId folderId);

 public FolderItemWatcher_Constructor3(Folder folder, Type itemType,

FolderItemWatcherOptions options);

 public FolderItemWatcher_Constructor4(ItemContext context, ItemId folderId, Type itemType);

  public FolderItemWatcher_Constructor5(ItemContext context, ItemId, folderId, Type itemType,

FolderItemWatcherOptions options);

//Attributes

 public ItemId FolderItemWatcher_Folderld{get;}

 public Type FolderItemWatcher_ItemType{get;}

 public FolderItemWatcherOptions FolderItemWatcher_Options{get;}

//event

public event ItemChangedEventHandler FolderItemWatcher_ItemChanged;

}

The following code example shows how to create a folder watcher object for monitoring the contents of a folder. The watchdog generates notifications, ie events, when new music items are added or existing music items are updated or deleted. Folder Watcher monitors a specific folder or all folders in a folder hierarchy.

myFolderItemWatcher = new FolderItemWatcher(myFolder, typeof(Music));

myFolderItemWatcher.ItemChanged+=new ItemChangedEventHandler(this.onItemChanged);

2. Change tracking and notification generation mechanism

The storage platform provides a simple and efficient mechanism to track data changes and generate notifications. Clients retrieve notifications on the same connection used to retrieve data. This greatly simplifies security checks, removing delays and constraints on possible network configurations. Notifications are retrieved by issuing a select statement. To prevent polling, clients can use a "wait for" feature provided by database engine 314 . Figure 13 illustrates the basic storage platform notification concept. Awaiting queries can be executed synchronously, in which case the calling thread is blocked until the result is available, or asynchronously, in which case the thread is not blocked and returns on a separate thread when the result is available result.

The combination of "wait" and "select" is attractive for monitoring data changes appropriate for a particular data range, since changes can be monitored by setting a notification lock on the corresponding data range. This applies to many common storage platform scenarios. Changes to individual items can be monitored by setting notification locks on the corresponding data ranges. Changes to folders and folder trees can be monitored by setting notification locks on path scopes. Changes to a type and its subtypes can be monitored by setting a notification lock on the type scope.

In general, there are three distinct phases associated with processing notifications: (1) data change or even detection, (2) subscription matching, and (3) notification delivery. Excluding synchronous notification delivery, that is, notification delivery as part of a transaction that performs data changes, storage platforms can implement the following two forms of notification delivery:

1) Direct event detection: Perform event detection and subscription matching as part of an update transaction. Notifications are inserted into tables monitored by subscribers; and

2) Deferred event detection: Event detection and subscription matching are performed after the update transaction is committed. The actual subscriber or intermediary then detects the event and generates a notification.

Direct event detection requires additional code to be executed as part of the update operation. This allows capturing all events of interest, including events indicating relative state changes.

Deferred event detection removes the requirement to add extra code to update operations. Event detection is done by the end subscriber. Deferred event detection typically batches event detection and event delivery, and is well suited to the query execution infrastructure of a database engine 314 (eg, SQL Server).

Deferred event detection relies on logs or traces left by update operations. The storage platform maintains a set of logical timestamps and tombstones for deleted data items. When scanning for changes in a data store, the client provides a set of timestamps that define a low watermark for detecting changes as well as a set of timestamps that prevent duplicate notifications. The application may receive notifications of all changes that occurred after the time indicated by the low watermark.

Complex applications that have access to core views can also optimize and reduce the number of SQL statements required to monitor a potentially large set of items by creating dedicated parameter and repeat filter tables. Applications with special needs, such as those that must support rich views, can use the available change tracking framework to monitor data changes and refresh their private snapshots.

Therefore, preferably, in one embodiment, the storage platform implements a deferred time detection method, as described more fully below.

a) change tracking

All project, extension and project relationship definitions carry a unique identifier. Change Tracking maintains a set of logical timestamps for all data items to record creation, update and deletion times. Tombstone entries are used to represent deleted data items.

Applications use this information to efficiently monitor whether specific items, item extensions, or item relationships have been newly added, updated, or deleted since the application last accessed the data store. The following example shows the mechanism.

create table[item-extension-relationship-table-template](

identifier uniqueidentifier not null default newid()

created bigint, not null, --@@stored when created

updated bignit, not null, --@@stored when last updated

...

)

All deleted items, item extensions and relationships are recorded in the corresponding tombstone tables. A template is shown below.

create table[item-extension-relationship-tombstone table-template](

identifier uniqueidentifier not null,

Deleted bigint, not null, --@@stored when deleted,

created bigint, not null, --@@ stored when created,

updated bigint, not null, --@@stored when last updated

...

)

For efficiency reasons, the storage platform maintains a set of global tables for items, item extensions, relationships, and pathnames. These global lookup tables can be used by applications to efficiently monitor data ranges and retrieve associated timestamp and type information.

b) Time stamp management

Logical time stamps are "local" to database storage, ie storage platform volumes. Timestamps are monotonically increasing 64-bit values. Retaining a single timestamp is usually sufficient to detect whether data has changed since the last connection to the storage platform volume. However, in the most realistic case, slightly more timestamps need to be saved to check for duplicates. The reason is explained below.

A relational database table is a logical abstraction built on a set of physical data structures, such as B-trees, heaps, etc. Assigning a timestamp to a newly created or updated record is not an atomic action. Inserting the record into the underlying data structure may occur at a different time, whereby the application may see the records as being out of order.

Figure 14 shows two transactions inserting new records into the same B-tree. Since transaction T3 inserts its records before transaction T2's insert is scheduled, an application scanning the B-tree may see that records inserted by transaction T3 precede those inserted by T2. Thus, the reader may incorrectly assume that he saw all records created before time "10." To solve this problem, the database engine 314 provides a function to return a low watermark before which all updates are committed and all updates are inserted into the respective underlying data structures. In the above example, the returned low watermark is "5", assuming the reader started before committing transaction T2. The low watermark provided by the database engine 314 allows an application to efficiently determine which items to ignore when scanning a database or range of data for data changes. In general, ACID transactions are assumed to last for a very short time, and thus a low watermark is expected to be very close to the most recently distributed timestamp. In the presence of long-duration transactions, applications may have to keep individual timestamps to detect and discard duplicates.

c) Data Change Detection - Event Detection

Applications get a low watermark when querying the data store. The application then uses this watermark to scan the data store for entries whose creation, update, or deletion timestamps are greater than the returned low watermark. Figure 15 illustrates this process.

To prevent duplicate notifications, the app remembers timestamps greater than the returned low watermark and uses these to filter out duplicates. Applications create session-local temporary tables to efficiently handle large sets of repeating timestamps. Before issuing a select statement, the application inserts all previously returned duplicate timestamps and removes timestamps older than the last low watermark returned, as shown below.

delete from $dupilcates where ts<@oldLowWaterMarkl

insert into $duplicates(ts)values(...),...,(...);

waitfor(select*, getLowWaterMark() as newLowWaterMark

from[global! items]

where updated>=@oldLowWaterMark

and updated not in(select＊from$duplicates))

G. Synchronization

According to another aspect of the invention, the storage platform provides a synchronization service 330 which (1) allows multiple instances of the storage platform (each with its own data store 302) to synchronize individual instances of their content according to a flexible set of rules. part, and (ii) provide an infrastructure for third parties to synchronize data storage of the storage platform of the present invention with other data sources implementing proprietary protocols.

Storage platform-to-storage platform synchronization occurs across a set of participating replicas. For example, referring to FIG. 3 , it is desirable to provide synchronization between data store 302 of storage platform 300 and another remote data store 338 under the control of another instance of the storage platform, presumably running on a different computer system. The total membership of the group need not be known by any given replica at any given time.

Different replicas can make changes independently (ie, concurrently). A synchronization process is defined to make each replica aware of changes made by other replicas. This synchronization capability is multi-master in nature.

The synchronization capabilities of the present invention allow each replica to:

Determine what changes the other replica knows about;

Request information about changes not known to this replica;

· convey information about changes unknown to other replicas;

Determine when two changes conflict with each other;

· Local application changes;

· Communicate conflict resolution to other replicas to ensure convergence; and

• Decompose conflicts based on policies specified for conflict resolution.

1. Synchronization from storage platform to storage platform

The main application of the synchronization service 300 of the storage platform of the present invention is to synchronize multiple instances of a storage platform (each with its own data store). The synchronization service operates at the storage platform schema level (rather than in the underlying tables of the database engine 314). Thus, for example "Scope" is used to define the synchronization groups discussed below.

Synchronization services operate on the principle of "net change". Instead of recording and sending individual operations (as transactional replication does), the synchronization service sends the final results of those operations, thus often combining the results of multiple operations into a single final result.

Synchronization services generally do not respect transaction boundaries. In other words, if two changes are made to a storage platform data store in a single transaction, there is no guarantee that these changes are applied atomically to all other replicas - one change can be shown without the other. The exception to this principle is that if two changes are made to the same item in the same transaction, those changes are guaranteed to be sent and applied to the other replicas atomically. Thus, a project is a unit of consistency for the synchronization service.

a) Sync control application

Any application can connect to the sync service and initiate a sync operation. That application provides all the parameters needed to perform a synchronization (see Synchronization Overview below). Such applications are referred to herein as Synchronous Control Applications (SCA).

When synchronizing two storage platform instances, the synchronization is initiated by SCA on one side. The SCA notifies the local synchronization service to synchronize with the remote partner. On the other side, the sync service is woken up by a message sent by the sync service from the originating machine. It responds based on persistent configuration information (see mapping below) that exists on the target machine. Synchronization services can run on a schedule or in response to events. In these cases, the synchronization service that implements the schedule becomes the SCA.

To enable synchronization, two steps need to be taken. First, schema designers must annotate storage platform schemas with appropriate synchronization semantics (specifying change units as described below). Second, synchronization must be properly configured on all machines with instances of the storage platform participating in the synchronization (described below).

b) schema annotation

The basic concept of synchronization service is the concept of change unit (Change Unit). A unit of change is the smallest schema fragment tracked individually by the storage platform. For each unit of change, the synchronization service can determine whether it has changed or remained unchanged since the last synchronization.

Changing elements in the specified schema serves several purposes. First, it establishes how wordy the online sync service is. When a change is made within a change unit, the entire change unit is sent to the other replicas because the synchronization service does not know which part of the change unit was changed. Second, it determines the granularity of collision detection. When two concurrent changes are made to the same change unit (these terms are defined in detail in subsequent chapters), the synchronization service causes a conflict; on the other hand, if concurrent changes are made to different change units, no conflict occurs and the changes are automatically merge. Third, it severely affects the amount of metadata maintained by the system. A lot of sync service metadata is maintained for each unit of change; therefore, making the unit of change smaller increases the overhead of synchronization.

Defining the changing unit requires finding the right tradeoff. To this end, Synchronization Services allows schema designers to participate in the process.

In one embodiment, the synchronization service does not support change units larger than one element. However, it supports the ability for schema designers to specify units of change smaller than an element - that is, to combine multiple attributes of an element into a single unit of change. In this embodiment, this is accomplished using the following syntax:

<Type Name="Appointment" MajorVersion="1"MinorVersion="0"ExtendsType="Base.Item"

ExtendsVersion="1">

<Field Name="MeetingStatus"Type="the storage platformTypes uniqueidentifier Nullable="False"/>

<Fileld Name="OrganizerName"Type="the storage platformTypes.nvarchar(512)"Nullable="False"/>

<Filed Name="OrganizerEmail"Type="the storage platformTypes.nvarchar(512)"

TypeMajorVersion="1" MultiValued="True"/>

...

<ChangeUnit Name="CU_Status">

<Field Name="MeetingStatus"/>

</ChangeUnit>

<ChangeUnit Name="CU_Organizer"/>

<Field Name="OrganizerName"/>

<Field Name="OrganizerEmail"/>

</ChangeUnit>

...

</Type>

c) Synchronization configuration

A group of storage platform partners who wish to keep some parts of their data in sync is called a sync community. Although members of the community wish to remain in sync, they do not need to represent the data in exactly the same way; in other words, sync partners can transform the data they are synchronizing.

In a peer-to-peer situation, it is impractical for peers to maintain translation maps for all their partners. Instead, the sync service takes the approach of defining "community folders". A community folder is an abstraction that represents a hypothetical "shared folder" that all community members are syncing with.

This concept is best illustrated with an example. If Joe wants to keep the My Documents (My Documents) folders of several of his computers synchronized, Joe defines a community folder, such as called JoeDocuments. Then on each computer, Joe configures a mapping between the hypothetical JoeDocuments folder and the local My Documents folder. From this point on, when Joe's computers synchronize with each other, they talk by means of documents in JoeDocuments rather than their local projects. In this way, all Joe's computers understand each other without having to know who the others are—the community folder becomes the lingua franca of the sync community.

Configuring the synchronization service consists of three steps: (1) defining the mapping between local folders and community folders; (2) defining the synchronization profiles that determine which gets synchronized (such as who to synchronize with, and which subset should be sent, which is received); and (3) define a schedule on which the different synchronization profiles should run, or run them manually.

(1) Community Folder - Mapping

Community folder maps are stored on individual machines as XML configuration files. Each map has the following schema:

/mappings/communityFolder

This element names the mapped community folder. The names follow the syntax of folders.

/mappings/localFolder

This element names the local folder to which the mapping is translated. This name follows the syntax rules for folders. For the mapping to work, the folder must already exist. Items in this folder are considered for each synchronization of this mapping.

/mappings/transformations

This element defines how to convert projects from community folders to local folders and vice versa. If missing or empty, no conversion is performed. Specifically, this means that no IDs are mapped. This configuration is mainly used to create a cache of folders.

/mappings/transformations/mapIDs

This element requests that a newly generated local ID be given to all projects mapped from the community folder, rather than reusing the community ID. The sync runtime maintains ID mappings to convert items back and forth.

/mappings/transformations/localRoot

This element requests all root items in the community folder to be children of the specified root.

/mappings/runAs

This element controls under whose authority requests for this map are processed. If not present, the sender is assumed.

/mappings/runAs/sender

The presence of this element indicates that the message sender for this mapping must be impersonal and process requests under his credentials.

(2) Overview

A synchronization profile is the overall set of parameters required for a separation synchronization. It is provided by SCA to the sync runtime to start the sync. The synchronization profile for storage platform-to-storage platform synchronization contains the following information:

· Local folders, used as source and target for changes;

The name of the remote folder to synchronize with - this folder must be published from the remote partner via the mapping defined above;

· Direction-sync service supports send-only, receive-only, and send-receive synchronization;

• Local Filter - Select what local information to send to the remote partner. Expressed as a storage platform query on a local folder;

Remote Filters - select what remote information to retrieve from remote partners - expressed as storage platform queries on community folders;

Conversion - define how to convert between project and native formats;

· Local Security - specifies whether changes retrieved from the remote endpoint are applied with the permission of the remote endpoint (personalization), or whether the user initiates the sync locally; and

• Conflict Resolution Policy - specifies whether conflicts should be rejected, logged, or automatically resolved - in the latter case, which conflict resolver to use and its configuration parameters.

The synchronization service provides runtime CLR classes that allow easy construction of synchronization profiles. Profiles can be serialized to and from XML files for easy storage (often along with timetables). However, there is no standard place to store all profiles in a storage platform; SCA is welcome to build profiles at points that do not have to be persistent. Note that it is not necessary to have a local mapping to initiate a sync. All synchronization information can be specified in the profile. However, to respond to a synchronization request initiated by a remote party, a mapping is required.

(3) Schedule

In one embodiment, the synchronization service does not provide its own scheduling infrastructure. Instead, it relies on another component to accomplish this task - the WindowsScheduler available in the Microsoft Windows operating system. The sync service includes a command line utility that acts as an SCA and triggers syncs based on sync profiles saved in XML files. This utility makes it easy to configure Windows Scheduler to run synchronizations on a schedule or in response to events such as user logon or logout.

d) Conflict handling

Conflict handling in the synchronization service is divided into three phases: (1) conflict detection that occurs when changes are applied - this step determines whether the changes can be safely applied; (2) automatic conflict resolution and logging - this step ( Occurs immediately after conflict detection) consults the automatic conflict resolver to see if the conflict can be resolved - if not, optionally logs the conflict; and (3) conflict checking and resolution - if some conflicts have been logged is logged, and occurs outside the context of a sync session, this step is taken - at this point, the logged conflict can be resolved and removed from the log.

(1) Conflict detection

In this embodiment, the synchronization service detects two types of conflicts: knowledge-based conflicts and constraint-based conflicts.

(a) Knowledge-Based Conflict

Knowledge-based conflicts arise when two replicas make independent changes to the same change unit. Two changes are said to be independent if they were made without knowledge of each other—in other words, the version of the first was not overwritten by knowledge of the second, and vice versa. Based on knowledge of replicas as described above, the synchronization service automatically detects all such conflicts and handles them as described herein below.

Sometimes it's helpful to think of conflicts as forks in the version history that change a unit. If no conflicts occur during the life of a changing unit, its version history is a simple chain - each change happens after the previous one. In the case of a knowledge-based conflict, two changes happen in parallel, causing the chain to split and become a version tree.

(b) Constraint-based conflicts

There are cases where independent changes violate integrity constraints when applied together. For example, creating two copies of a file with the same name in the same directory would cause such a conflict to occur.

Constraint-based conflicts involve two separate changes (as for knowledge-based conflicts); however, they do not affect the same change unit. Instead, they affect different change units, but there are constraints between them.

The synchronization service detects constraint violations when changing the application and automatically raises constraint-based conflicts. Resolving constraint-based conflicts typically requires modifying the changed custom code in a way that does not violate the constraints; the synchronization service does not provide a general mechanism for doing this.

(2) Conflict handling

When a conflict is detected, the sync service can take one of 3 actions (chosen by the sync initiator in the sync profile): (1) reject the change, returning it to the sender; (2) log the conflict to the conflict log or (3) Automatically resolve conflicts.

If the change is rejected, then the synchronization service works if the change does not reach the replica. A negative acknowledgment is sent back to the initiator. This decomposition strategy is mainly useful on headless replicas (such as file servers) where conflict logging is not feasible. Instead, such replicas force other replicas to handle conflicts by rejecting changes.

A sync initiator configures conflict resolution in its sync profile. The synchronization service supports combining multiple conflict resolvers in a single profile by - first, specifying a list of conflict handlers to try one after the other until one succeeds; second, associating a conflict handler with a conflict type, For example, point update-update knowledge-based conflicts to one conflict handler, and all other conflicts to the log.

(a) Automatic Conflict Resolution

The synchronization service provides various default conflict resolvers. The list includes:

Local wins: discards incoming changes if they conflict with locally stored data;

Remote wins: local data is discarded if it conflicts with an incoming change;

Last Writer Wins: picks either a local winner or a remote winner based on the changed timestamp (note that in general sync services do not depend on clock values; this conflict resolver is the only exception to that rule);

• Deterministic: Winners are picked in a way that is guaranteed to be the same across all replicas, but not otherwise meaningful - one embodiment of the synchronization service might implement this feature using a lexicographic comparison of partner IDs.

Additionally, ISVs can implement and install their own conflict handlers. Custom conflict handlers can accept configuration parameters; such parameters must be specified by the SCA in the conflict resolution section of the synchronization profile.

When the conflict resolver resolves a conflict, it returns a list of operations that need to be performed (instead of conflicting changes) to the runtime. The synchronization service then applies these operations, adjusting the remote knowledge appropriately to include information already considered by the conflict handler.

Another conflict may have been detected while the decomposition was being applied. In this case, the new conflict must be resolved before the original processing can be rerun.

When viewing conflicts as branches in a project's version history, conflict resolution can be viewed as a junction - combining two branches to form a single point. Thus, conflict resolution turns the version history into a directed acyclic graph (DAG).

(b) Conflict logging

A very specific type of conflict handler is a conflict logger. The synchronization service logs conflicts as items of type ConflictRecord. These records are in turn related to the conflicting project (unless the project itself has been deleted). Each conflict record contains: the incoming change that caused the conflict; the type of conflict: update-update, update-delete, delete-update, insert-insert, or constraint; and knowledge of the version of the incoming change and the replica that sent it . Logged conflicts are available for inspection and resolution as described below.

(c) Conflict checking and resolution

The synchronization service provides an API for applications to examine the conflict log and suggest resolution methods for conflicts therein. The API allows an application to enumerate all conflicts or conflicts related to a given item. It also allows these applications to resolve logged conflicts in one of 3 ways: (1) remote wins - accepts the logged changes and overrides conflicting local changes; (2) local wins - ignoring conflicting parts of the logged changes; and (3) proposing new changes - where the application proposes a merge that resolves the conflicts in its view. Once conflicts are resolved by the application, the synchronization service removes them from the log.

(d) Convergence of replicas and propagation of conflict resolution

In complex synchronization situations, the same conflict can be detected on multiple replicas. If this happens, a number of things can happen: (1) conflicts are resolved on one replica and the resolution is sent to the other; (2) conflicts are automatically resolved on both replicas; or (3) Manually resolve conflicts on both replicas (via the conflict checking API).

To ensure convergence, the synchronization service passes conflict resolution to other replicas. When changes that resolve conflicts arrive at replicas, the sync service automatically finds any conflicting records in the log that were resolved by the update, and deletes them. In this case, conflict resolution on one replica will be bound on all other replicas.

If different replicas choose different winners for the same conflict, the synchronization service applies the principles of bound conflict resolution and automatically picks one of the two resolutions over the other. The winner is picked in a deterministic manner that is guaranteed to produce the same result all the time (one embodiment uses replica ID dictionary comparison).

If different replicas propose different "new changes" for the same conflict, the sync service treats this new conflict as a special conflict and uses the conflict logger to prevent it from propagating to other replicas. This situation often leads to manual conflict resolution.

2. Synchronization of non-storage platform data storage

According to another aspect of the storage platform of the present invention, the storage platform provides an architecture for the ISV to implement a sync adapter that enables the storage platform to synchronize with legacy systems such as Microsoft Exchange, AD, Hotmail, etc. A sync adapter benefits from a number of synchronization services provided by the synchronization services described below.

Despite its name, a sync adapter does not need to be implemented as a plug-in to a storage platform architecture. A "sync adapter" can simply be any application that utilizes the sync service runtime interface to obtain services such as change enumeration and application when needed.

To make it easier for others to configure and run synchronization to a given backend, writers of sync adapters are encouraged to expose a standard sync adapter interface that performs synchronization given the synchronization profile above. The profile provides configuration information to the adapter, and the adapter passes certain information to the synchronization runtime to control runtime services (eg, folders to synchronize).

a) Synchronization service

Synchronization Services Provides several synchronization services to adapter writers. In the remainder of this section, it is convenient to refer to the machine on which the storage platform is synchronizing as the "client", and the non-storage platform backend that the adapter is talking to as the "server".

(1) Change the enumeration

Based on the change tracking data maintained by the sync service, change enumeration allows the sync adapter to easily enumerate the changes that have occurred to the data store folder since the last sync attempt with the partner was made.

Changes are enumerated based on the concept of an "anchor" - an opaque structure representing information about the last synchronization. As mentioned in previous chapters, anchors take the form of storing platform knowledge. Synchronization adapters that utilize change enumeration services fall into two broad categories: adapters that use "stored anchors" and adapters that use "provided anchors".

The distinction is based on where the information about the last sync is stored - on the client or on the server. Adapters often easily store this information on the client - backends often cannot. On the other hand, if multiple clients are syncing with the same backend, storing this information on the client is inefficient and in some cases incorrect - it keeps one client from knowing that other clients have pushed to Server changes. If the adapter wishes to use the server stored anchor, the adapter needs to send it back to the storage platform when changing the enumeration.

In order for the storage platform to maintain anchors (for local or remote storage), the storage platform needs to know about changes that were successfully applied on the server. These and only these changes can be included in the anchor point. During change enumeration, the sync adapter uses the Acknowledgment interface to report which changes have been successfully applied. At the end of the synchronization, adapters using the provided anchor must read the new anchor (which incorporates all successfully applied changes) and send it to their backends.

Each adapter often needs to store adapter specific data as well as items that are inserted into the storage platform data store. Common examples of this data store are remote IDs and remote versions (time stamps). The sync service provides the mechanism for storing this data, while the change enumeration provides the mechanism for receiving this additional data and the changes to return. In most cases, this eliminates the need for the adapter to re-query the database.

(2) Change application

Change Apply allows sync adapters to apply changes received from their backends to the local storage platform. The adapter is expected to transition the change to storage platform mode.

The main function of the change application is to automatically detect conflicts. As in the case of storage platform-to-storage platform synchronization, a conflict is defined as two overlapping changes made without mutual knowledge. When adapters use changes to apply, they must specify an anchor on which to perform conflict detection. If overlapping local changes not covered by the adapter's knowledge are detected, the change application causes a conflict. Similar to changing enumerations, adapters can use stored or provided anchors. Changes the application to support efficient storage of adapter-specific metadata. Such data can be appended by the adapter to the changes to be applied, and can be stored by the synchronization service. Data can be returned the next time the enumeration is changed.

(3) Conflict resolution

The conflict resolution mechanisms described above, including logging and automatic resolution options, are also available for sync adapters. A sync adapter can specify a strategy for conflict resolution when applying changes. If specified, conflicts can be passed to the specified conflict handler and resolved (if possible). Conflicts can also be logged. The adapter can detect conflicts when attempting to apply local changes to the backend. In that case, the adapter can still pass conflicts to the synchronization runtime for resolution by policy. Additionally, sync adapters can request that any conflicts detected by sync services be sent back to them for processing. This is especially handy in cases where the backend can store or resolve conflicts.

b) Adapter implementation

While some "adapters" are simply applications that utilize the runtime interface, adapters are encouraged to implement the standard adapter interface. These interfaces allow a synchronization control application to: request that the adapter perform a synchronization for a given synchronization profile; cancel an ongoing synchronization; and receive a progress report (percent complete) on the ongoing synchronization.

3. Security

The synchronization service strives to introduce as little synchronization as possible into the security model implemented by the storage platform. Instead of defining new permissions for synchronization, existing permissions are used. specifically,

Anyone who can read a datastore item can enumerate changes to that item;

Anyone who can write to a datastore item can apply changes to that item; and

• Anyone who can extend a data storage item can associate synchronization metadata with that item.

The synchronization service does not maintain security authorization information. When a change is made by user U at replica A, and forwarded to replica B, the fact that the change was originally made at A (by U) is lost. If B forwards this change to replica C, this is done under B's authority, not A's. This leads to the restriction that if a replica is not trusted to make its own changes to an item, it cannot forward changes made by other replicas.

This is done by the sync control application when the sync service is started. The synchronization service personifies the identity of the SCA and completes all operations (local and remote) under that identity. As an illustration, observe that user U cannot cause the local synchronization service to retrieve changes to items for which user U does not have read access from the remote storage platform.

4. Manageability

Monitoring a distributed community of replicas is a complex problem. The synchronization service may use a "scan" algorithm to collect and distribute information about the state of the replica. The properties of the scanning algorithm ensure that information about all configured replicas is eventually collected and the failed (non-responsive) replica is detected.

Community-wide surveillance information is available on each replica. Monitoring tools can be run on any chosen replica to examine this monitoring information and make management decisions. Configuration changes must be made directly on the affected replica.

G. Legacy Document Interoperability

As mentioned above, at least in some embodiments, the storage platform of the present invention is intended to be implemented as an integral part of the hardware/software interface system of a computer system. For example, the storage platform of the present invention may be implemented as an integral part of an operating system such as the Microsoft Windows family of operating systems. In this respect, the storage platform API becomes part of the operating system API through which applications interact with the operating system. Thus, the storage platform becomes the device by which applications store information on the operating system, and thus the item-based data model of the storage platform replaces the traditional file system of this operating system. For example, when implemented in the Microsoft Windows family of operating systems, the storage platform can replace the NTFS file system implemented in that operating system. Currently, applications access the services of the NTFS file system through the Win32 API exposed by the Windows family of operating systems.

However, it should be recognized that completely replacing the NTFS file system with the storage platform of the present invention requires recoding existing Win32-based applications, and that such recoding may be undesirable, so the storage platform of the present invention provides Some kind of interoperability with existing filesystems would be beneficial. Thus, in one embodiment of the present invention, the storage platform enables applications relying on the Win32 programming model to simultaneously access the data storage of the storage platform as well as the content of the traditional NTFS file system. To this end, the storage platform uses a naming convention that is a superset of the Win32 naming convention to facilitate easy interoperability. In addition, the storage platform supports accessing files and directories stored in storage platform volumes through Win32API.

1. Interoperability Model

In accordance with this aspect of the invention, and in accordance with the exemplary embodiments described above, a storage platform implements a namespace in which non-file and file items can be organized. Using this model, the following advantages can be obtained:

1. Folders in a data store can contain both file and non-file items, thereby presenting a single namespace for both file and schematized data. In addition, it provides the same security, sharing and management model for all user data.

2. Since both file and non-file items can be accessed using the storage platform API, and no special rules are imposed for files in this approach, it presents a clearer programming model for application developers to work with.

3. All namespace operations go through the storage platform and are therefore processed synchronously. It's important to note that deep attribute upgrades (expansion of file contents) still happen asynchronously, but synchronous operations provide a more predictable environment for users and applications.

As a result of this model, in this embodiment, search capabilities may not be provided on data sources that are incorporated into the storage platform data store. This includes files on removable media, remote servers, and local disks. Provides a sync adapter that displays proxy server items (shortcuts + promoted metadata) in storage platforms for items residing in external filesystems. The Proxy Server project does not attempt to mimic files according to the namespace hierarchy or security of the data source.

The symmetry achieved in the namespace and programming model between file and non-file content provides a more structured project for applications to migrate content from the file system to storage platform data stores over time. good path. By providing a native file item type in the storage platform's data store, an application can transfer file data to the storage platform while still being able to manipulate that data through Win32. Eventually, applications may be fully ported to the storage platform API and structure their data in terms of storage platform items rather than files.

2. Data storage characteristics

To provide the desired level of interoperability, in one embodiment, the following features of storage platform datastores are implemented.

a) Non-volume

Storage platform datastores are not exposed as separate file system volumes. The storage platform takes full advantage of FILESTREAM hosted directly on NTFS. Thus, there are no changes to the on-disk format, thereby eliminating the need to expose the storage platform as a new file system at the volume level.

Instead, data stores (namespaces) are structured corresponding to NTFS volumes. The database and FILESTREAM for this portion of the fallback namespace reside on NTFS volumes associated with the storage platform datastore. Data storage corresponding to system volumes is also provided.

b) storage structure

The structure of storage is best illustrated with an example. As an example, consider the directory tree on the system volume of a machine named HomeMachine, shown in Figure 16. According to the file system interoperability feature of the present invention, corresponding to the c:\ drive, there is a storage platform data store exposed to the Win32 API through a UNC share, for example called "WinFSOnC". This makes the associated datastore accessible by the following UNC names:

\\HomeMachine\WinFSOnC.

In this embodiment, files and/or folders need to be explicitly migrated from NTFS to the storage platform. Therefore, if the user wishes to move the My Documents folder to the storage platform data store to take advantage of all the additional search/categorization features provided by the storage platform, the hierarchy would be as shown in Figure 17. It is important to note that these folders are actually moved in this example. Another point to note is that the namespace was moved into the storage platform, the actual stream was renamed to FILESTREAM, and the appropriate pointers were hooked into the storage platform.

c) do not migrate all files

Files that correspond to user data or require search/categorization provided by the storage platform are candidates for porting to storage platform data storage. Preferably, in order to limit the application compatibility problem with the storage platform, in the context of the Microsoft Windows operating system, the group of files transplanted to the storage platform of the present invention is limited to files in the MyDocuments folder, Internet Explorer (IE) Favorites , IE History, and the Desktop.ini file in the Documents and Settings directory. Preferably, transplantation of Windows system files is not allowed.

d) NTFS namespace access to storage platform files

In the embodiments described herein, files intended to be migrated to the storage platform cannot be accessed through the NTFS namespace, even though the actual file stream is stored in NTFS. In this way, complex locking and security issues caused by multi-threaded implementations are avoided.

e) expected namespace/drive letter

Access to files and folders in the storage platform is provided through UNC names of the form \\<machine name>\<WinFS share name>. For application classes that require a drive letter for operation, a drive letter can be mapped to this UNC name.

I. Storage Platform API

As noted above, the storage platform includes APIs that enable applications to access the features and capabilities of the storage platform discussed above, and to access items stored in the data store. This section describes one embodiment of the storage platform API of the storage platform of the present invention.

FIG. 19 shows the basic architecture of the storage platform API according to this embodiment. The storage platform API uses SQL client 1900 to talk to local data store 302 and also uses SQL client 1900 to talk to remote data stores such as data store 340 . Local storage can also talk to remote data storage 340 using DQP (Distributed Query Processor) or through the storage platform synchronization service ("Sync") described above. Storage platform API 322 also acts as a bridge API for data storage notifications, passing the application's subscript to notification engine 332 and routing the notifications to the application (eg, application 350a, 350b, or 350c) as described above. In one embodiment, the storage platform API 322 also defines a restricted "provider" architecture that enables it to access data in Microsoft Exchange and AD.

1. Overview

The data access mechanism of this embodiment of the storage platform API of the present invention addresses four areas: query, navigation, action, and event.

Inquire

In one embodiment, the storage platform data store is implemented on the relational database engine 314; as a result, the full expressive capabilities of the SQL language are inherent in the storage platform. Higher-level query objects provide a simplified model for querying the store, but may not encapsulate the full expressive capabilities of the store.

navigation

The storage platform data model builds a rich, extensible type system on top of the underlying database abstraction. For developers, storing platform data is the network of the project. The storage platform API allows navigation between projects via filtering, relationships, folders, etc. This is a higher-level abstraction than basic SQL queries; at the same time, it allows the use of rich filtering and navigation capabilities over familiar CLR coding patterns.

action

The storage platform API exposes actions common to all items - Create, Delete, Update; these are exposed as methods on objects. In addition, domain-specific actions such as SendMail (send mail), CheckFreeBusy (check free and busy), etc. are also available as methods. The API framework uses well-defined patterns that ISVs can use to add value by defining additional actions.

event

Data in the storage platform is dynamic. To enable applications to react when data in storage changes, the API exposes developers to rich event, subscription, and notification capabilities.

2. Naming and scope

It is useful to distinguish between namespaces and naming. As is often used, the term namespace refers to the set of all names available in a system. A system can be an XML schema, a program, a web, a collection of all ftp sites (and their content), etc. Naming is the process or algorithm used to assign unique names to all entries of interest within a namespace. Thus, the naming is of interest because it is expected to unambiguously refer to a given unit within the namespace. Thus, as used herein, the term "namespace" refers to the set of all names available across all storage platform instances in the universe. Projects are named entities in the namespace of the storage platform. The UNC naming convention is used to ensure unique project names. Each item in each storage platform's storage in the universe is addressable by a UNC name.

The highest organizational level in the storage platform namespace is a service - which is simply an instance of a storage platform. The next level of organization is volumes. A volume is the largest anonymous container for a project. Each storage platform instance contains one or more volumes. Inside the volume are the projects. Items are data atoms in the storage platform.

Data in the real world is almost always organized according to some system that makes sense in a given domain. Underlying all such data organization patterns is the concept of dividing the universe of data into named groups. As mentioned above, this concept is modeled in the storage platform by the concept of folders. Folders are a special type of item; there are two types of folders: containing folders and virtual folders.

Referring to FIG. 18, a containing folder is an item that contains a holding relationship with other items, and is equivalent to a file system folder of a general concept. Every project is "contained" in at least one containing folder.

A virtual folder is a more dynamic way of organizing a collection of items; it is simply the name of a given set of items - either explicitly enumerated or specified via a query. A virtual folder is itself an item, and can be thought of as representing a set of (non-holding) relationships with a set of items.

Sometimes, a tighter concept of containment needs to be modeled; for example, a Word document embedded in an email message is in some sense more tightly bound to its container than, for example, a file contained in a folder. This concept is expressed by the concept of embedded items. An embedded item has a special relationship that refers to another item; the referenced item can only be bound to the containing item or manipulated only within the context of the containing item.

Finally, the storage platform provides the concept of categories as a way of categorizing items and elements. Each item or element in the storage platform may have one or more categories associated with it. Categories are essentially just names that are tagged to items/elements. This name can be used in searches. The storage platform data model allows the definition of a hierarchy of categories, thus allowing a tree-like classification of data.

An unambiguous name for a project is the triplet: (<service name>, <volume ID>, <project ID>). Some items (notably folders and virtual folders) are collections of other items. This leads to an alternate way of identifying projects: (<service name>, <volume ID>, <project path>).

Storage platform names include the notion of a service context: a service context is a name that maps to (<volume name>, <path>) pairs. It identifies an item or group of items - for example, a folder, virtual folder, etc. For service contexts, the UNC name used to store any item in the platform namespace becomes:

\\<service name>\<service context>\<project path>

Users can create and delete service contexts. Likewise, the root directory within each volume has a predefined context: volumename$.

The item context scopes queries (eg, find operations) by limiting the results returned to items that live in the specified path.

3. Storage platform API components

Figure 20 schematically represents various components of the storage platform API in accordance with this embodiment of the invention. The storage platform API includes the following components: (1) data classes 2002, which represent storage platform elements and item types; (2) runtime library framework 2004, which manages object persistence and provides support classes 2006; and (3) tools 2008, It is used to generate CLR classes from stored platform schemas.

According to one aspect of the invention, at design time, a schema author submits schema documents 2010 and code 2012 for domain methods to a set of storage platform API design time tools 2008 . These tools generate client-side data classes 2002 and a storage schema 2014 and a storage class definition 2016 for the schema. "Domain" refers to a particular schema; for example, talking about domain methods of classes in the Contact schema, and so on. These data classes 2002 are used at runtime by application developers in cooperation with the storage platform API runtime architecture classes 2006 to manipulate storage platform data.

To illustrate aspects of the storage platform API of the present invention, several examples are presented based on an exemplary contact schema. Pictorial representations of this exemplary mode are shown in Figures 21A and 21B.

4. Data class

According to an aspect of the invention, each item, item extension and element type, and each relationship in the storage platform data store has a corresponding class in the storage platform API. Roughly, the fields of a type map to the fields of a class. Every item, item extension, and element in the storage platform is available as an object of a corresponding class in the storage platform API. Developers can query, create, modify or delete these objects.

The storage platform includes an initial set of schemas. Each schema defines a set of item and element types, and a set of relationships. The following is one embodiment of an algorithm for generating data classes from these schema entities:

For each mode S:

For each item I in S, a class named System.Storage.S.I is generated. This class has the following members:

Overloaded constructors, including ones that allow specifying the initial folder and name of the new project.

• Attributes for each field in I. If the field is multivalued, the attribute will be a collection of the corresponding element type.

• An overloaded static method (eg, a method named "FindAll") that finds multiple items matching the filter.

• An overloaded static method (eg, a method named "FindOne") that finds a single item matching the filter.

• A static method to find an item given its ID (eg a method named "FindByID").

• Find a static method of an item given its name relative to the item context (eg, a method named "FindByName").

• A method to save changes to the item (eg, a method named "Update").

· An overloaded static Create method that creates a new instance of an item. These methods allow specifying the project's initial folder in various ways.

For each element E in S, generate a class named System.Storage.S.E. This class has the following members:

• Attributes for each field in E. If the field is multivalued, the attribute will be a collection of the corresponding element type.

For each element E in S, generate a class named System.Storage.S.ECollection. This class follows the general .NET Framework guidelines for strongly typed collection classes. For relationship-based element types, the class will also include the following members:

- Find an overloaded method for multiple objects that match items that implicitly include a filter where the collection appears in the source role. Overloads include certain methods that allow filtering based on item subtypes (eg, a method named "FindAllTargetItems").

• Find an overloaded method for a single item object that matches a filter that implicitly includes items where the set appears in the source role. Overloads include certain methods that allow filtering based on item subtypes (eg, a method named "FindOneTargetItem").

• Find an overloaded method (eg, a method named "FindAllRelationships") that matches objects of nested element types that implicitly include a filter for items where the collection occurs in the source role.

• Find an overloaded method (eg, a method named "FindAllRelationshipsForTarget") that matches objects of nested element types that implicitly include a filter for items where the collection occurs in the source role.

• Find an overloaded method (eg, a method named "FindOneRelationship") that matches a single object of a nested element type that implicitly includes a filter of items where the collection occurs in the source role.

• Find an overloaded method (eg, a method named "FindOneRelationshipForTarget") that matches a single object of a nested element type that implicitly includes a filter of items where the collection occurs in the source role.

For a relation R in S, a class named System.Storage.S.R is generated. This class has one or two subclasses, depending on whether one or two relationship roles specify endpoint fields.

You can also extend the Create class for each project you create in this way.

Data classes exist in the System.Storage.<schema name> namespace, where <schema name> is the name of the corresponding schema - such as Contacts, Files, etc. For example, all classes corresponding to the Contacts schema are in the System.Storage.Contacts namespace.

As an example, referring to Figures 21A and 21B, the Contact schema results in the following classes contained in the System.Storage.Contact namespace:

Items: Item, Folder, WellKnownFolder, LocalMachineDataFolder, UserDataFolder, Principal, Service, GroupService, PersonService, PresenceService, ContactService, ADService, Person, User, Group, Organization, HouseHold

·元素：NestedElementBase、NestedElement、IdentityKey、SecurityID、EAddress、ContactEAddress、TelephoneNumber、SMTPEAddress、InstantMessagingAddress、Template、TemplateRelationship、LocationRelationship、FamilyEventLocationRelationship、HouseHoldLocationRelationship、RoleOccupancy、EmployeeData、GroupMemberShip、OrganizationLocationRelationship、HouseHoldMemberData、FamilyData、SpouseData、ChildData

As another example, the detailed structure of the Person type as defined in the Contacts schema is shown in XML as follows:

<Type Name="Person" MajorVersion="1" MinorVersion="0"

ExtendsType="Core.Principal"ExtendsVersion="1">

<Field Name="Birthdate"Type="the storage platformTypes.datetime"

Nullable="true"TypeMajorVersion="1"/>

<Field Name="Gender"Type="Base.CategoryRef"

Nullable="true"MultiValued="false"

TypeMajorVersion="1"/>

<Field Name="PersonalNames"Type="Contact.FullName"

Nullable="true"MultiValued="true"

 TypeMajorVersion="1"/>

<Field Name="PersonalEAddresses"Type="Core.EAddress"

Nullable="true"MultiValued="true"

 TypeMajorVersion="1"/>

<Field Name="PersonalPostalAddresses"

Type = "Core. PostalAddress" Nullable = "true"

MultiValued="true"TypeMajorVersion="1"/>

<Field Name="PersonalPicture"Type="the storage platformTypes.image"

Nullable="true"TypeMajorVersion="1"/>

<Field Name="Notes"Type="Core.RichText"Nullable="true"

MultiValued="true"TypeMajorVersion="1"/>

<Field Name="Profession"Type="Base.CategoryRef"

Nullable="true"MultiValued="true"

 TypeMajorVersion="1"/>

<Field Name="DataSource"Type="Base.IdentityKey"

Nullable="true"MultiValued="true"

 TypeMajorVersion="1"/>

<Field Name="ExpirationDate"Type="the storage platformTypes.datetime"

Nullable="true"TypeMajorVersion="1"/>

<Field Name="HasAllAddressBookData"Type="the storage platformTypes.bit"

Nullable="true"TypeMajorVersion="1"/>

<Field Name="EmployeeOf"Type="Contact.EmployeeData"

    Nullable = "true" MultiValued = "true"

   TypeMajorVersion="1"/>

</Type>

This type results in the following class (only public members shown):

partial public class Person:

System.Storage.Core.Principal,

System.Windows.Data.IDataUnit

{

 public System.Data.SqlTypes.SqlDateTime

Birthdate { get; set; }

 public System.Storage.Base.CategoryRef

Gender{get; set:}

public System.Storage.Contact.FullNameCollection

 Person alNames{get;}

 public System.Storage.Core.EAddressCollection

  Personal EAddresses{get;}

 public System.Storage.Core.PostalAddressCollection

  PersonalPostalAddresses { get; }

public System.Data.SqlTypes.SqlBinary

  PersonalPicture { get; set; }

 public System.Storage.Core.RichTextCollection

Notes { get; }

 public System.Storage.Base.CategoryRefCollection

Profession{get;}

 public System.Storage.Base.IdentityKeyCollection

  DataSource { get; }

 public System.Data.SqlTypes.SqlDateTime

  Expiration Date { get; set; }

public System.Data.SqlTypes.SqlBoolean

  HasAllAddressBookData { get; set; }

public System.Storage.Contact.EmployeeDataCollection

EmployeeOf{get;}

public Person();

public Person(System.Storage.Base.Folder folder, string name);

public static new System.Storage.FindResult

FindAll(System.Storage.ItemStore store);

public static new System.Storage.FindResult

FindAll(

System.Storage.ItemStore store,

string filter);

public static new Person

FindOne(

System.Storage.ItemStore store,

string filter);

public new event

System.Windows.Data.PropertyChangedEventHandler

PropertyChangedHandler;

public static new Person

FindByID(

System.Storage.ItemStore store,

long long item_key);

}

As yet another example, the detailed structure of the TelephoneNumber type defined in the contact schema is shown in XML as follows:

<Type Name="TelephoneNumber" ExtendsType="Core.EAddress"

MajorVersion="1"MinorVersion="0"ExtendsVersion="1">

<Field Name="CountryCode"Type="the storage platformTypes.nvarchar(50)"

    Nullable = "true" MultiValued = "false"

   TypeMajorVersion="1"/>

<Field Name="AreaCode"Type="the storage platformTypes.nvarchar(256)"

Nullable="true"TypeMajorVersion="1"/>

<Field Name="Number"Type="the storage platformTypes.nvarchar(256)"

Nullable="true"TypeMajorVersion="1"/>

<Field Name="Extension"Type="the storage platformTypes.nvarchar(256)"

Nullable="true"TypeMajorVersion="1"/>

<Field Name="PIN"Type="the storage platformTypes.nvarchar(50)"

Nullable="true"TypeMajorVersion="1"/>

</Type>

This type results in the following class (only public members shown):

partial public class TelephoneNumber:

System.Storage.Core.EAddress,

System.Windows.Data.IDataUnit

{

 public System.Data.SqlTypes.SqlString CountryCode

{ get; set; }

 public System.Data.SqlTypes.SqlString AreaCode

{ get; set; }

 public System.Data.SqlTypes.SqlString Number

{ get; set; }

 public System.Data.SqlTypes.SqlString Extension

{ get; set; }

 public System.Data.SqlTypes.SqlString PIN

{ get; set; }

The resulting class hierarchy for a given schema directly reflects the hierarchy within that schema. As an example, consider the Item type defined in the Contacts schema (see Figures 21A and 21B). The class hierarchy corresponding to this type in the storage platform API is as follows:

Object

DataClass

ElementBase

RootItemBase

Item

Principal

                     

Household

Organization

Person

User

Service

PresenceService

ContactService

ADService

RootNestedBase

...(element class)

Yet another schema, one that allows representation of all audio/video media in the system (broken audio files, audio CDs, DVDs, home videos, etc.) enables users/applications to store, organize, search, and manipulate audio of different kinds /video media. The basic media document schema is general enough to represent any media, and extensions to this basic schema are designed to handle domain-specific properties separately for audio and video media. This mode, as well as many others, are conceived to operate directly or indirectly under the core mode.

5. Runtime architecture

The basic storage platform API programming model is object persistence. Applications perform searches on storage and retrieve objects representing data from storage. Applications modify retrieved objects or create new ones, then have their changes propagated to storage. The process is managed by the ItemContext (item context) object. Searches are performed using an ItemSearcher object, and search results are accessed through a FindResult object.

a) Runtime architecture class

According to another inventive aspect of the present invention, the runtime framework implements a plurality of classes to support operations on data classes. These architectural classes define a common set of behaviors for the data classes, and together with the data classes provide the basic programming model for the storage platform API. The classes in the runtime framework belong to the System.Storage namespace. In this embodiment, the framework classes include the following main classes: ItemContext, ItemSearcher and FindResult. Other secondary classes, enumeration values, and delegates may also be provided.

(1) ItemContext

The ItemContext object (i) represents the set of item domains that the application wishes to search, (ii) maintains for each object state information representing the state of the data retrieved from the storage platform, and (iii) manages the transactions and any file system that interoperates with the storage platform.

As an object persistence engine, ItemContext provides the following services:

1. Deserialize the data read from storage into an object.

2. Maintain object identity (the same object is used to represent a given item, no matter how many times the item is included in the query results).

3. Track object state.

ItemContext also implements several services unique to the storage platform:

1. Generate and execute the storage platform update element operations required to persist changes.

2. Create the connections to multiple data stores required to allow seamless navigation of reference relationships and to allow retrieval of objects to be modified and saved from multi-domain searches.

3. Ensuring that items in backing files are correctly updated when object changes indicate that the item is to be saved.

4. Manage transactions across multiple storage platform connections and manage transactionally processed file systems when updating data stored in file-backed items and file stream attributes.

5. Perform create, copy, move, and delete operations that take into account storage platform relational semantics, items that support files, and stream-typed attributes.

Appendix A provides the source code listing the ItemContext class according to one embodiment thereof.

(2) Item Searcher

The ItemSearcher class supports simple searches that return entire Item objects, streams of Item objects, or streams of values projected from Item objects. ItemSearcher encapsulates all common core functionality to: the concept of a target type and the parameterized filters applied to that target type. ItemSearcher also allows precompiling or preparing searchers as an optimization when performing the same operation with multiple types. Appendix B provides the source code listing the ItemSearcher class and several closely related classes according to one embodiment thereof.

(a) Target type

The search target type is set when the ItemSearcher is constructed. Target types are CLR types that map to ranges that can be queried by the data store. Specifically, it is the CLR type that maps to Item, Relationship, and Relationship Extension types, as well as schematized views.

When retrieving a searcher using the ItemContext.GetSearcher method, the searcher's target type is specified as a parameter. When calling a static GetSeacher method (for example, Person.GetSearcher) on an Item, Relationship, or ItemExtension type, the target type is the Item, Relationship, or ItemExtension type.

Search expressions provided in an ItemSearcher (for example, search filters and pass query operations, or projection definitions) are always relative to the search target type. These expressions may specify properties of the target type (including properties of nested elements), and may specify joins to relationship and item extensions as described elsewhere.

The search target type is available through a read-only property (for example, the Item.Searcher.Type property).

(b) filter

ItemSearcher contains properties that specify filters (for example, a property named "Filters" as a collection of SearchExpression objects) that defines the filters used in the search. All filters in a collection are combined using the logical AND operator when performing a search. Filters can contain parameter references. Parameter values are specified through the Parameters property.

(c) Prepare to search

In cases where it is possible to perform the same search repeatedly with only one parameter change, some performance can be improved by precompiling or preparing the search. This is done with a set of prepare methods on the ItemSearcher (e.g. a method to prepare a Find that returns one or more items, possibly named "PrepareFind"; and a method to prepare a Find that returns a projection, possibly named "PrepareProject" ) to achieve. For example:

ItemSearcher searcher = ...;

PreparedFind pf = searcher. PrepareFind();

...

result = pf. FindAll();

...

result = pf. FindAll();

(d) Find options

There are several options available for simple searches. These can be specified, for example, in a FindOptions (find options) object and passed to the Find method. For example:

ItemSearcher searcher = Person. GetSearcher(context);

FindOptions options = new FindOptions();

options.MaxResults = 10;

options.SortOptions.Add("PersonalNames.Surname", SortOrder.Ascending);

FindResult result = searcher. FindAll(options);

As a convenience, sorting options can also be passed directly to the Find method:

ItemSearcher searcher = Person. GetSearcher(context);

FindResult result = searcher. FindAll(

new SortOption("PersonalNames. Surname", SortOrder. Ascending));

The DelayLoad (delay load) option determines whether to load the values of large binary attributes when retrieving search results, or whether to delay loading until after they are referenced. The MaxResults option determines the maximum number of results returned. This is equivalent to specifying TOP in the SQL query. It is usually used in conjunction with sorting.

A sequence of SortOption (sort options) objects can be specified (eg, using the FindOptions.SortOptions property). The search results will be sorted as specified by the first SortOption object, then as specified by the second SortOption object, and so on. SortOption specifies a search expression indicating the properties to be used for sorting. The expression specifies one of the following:

1. Search for scalar properties in the target type;

2. A scalar attribute in a nested element reachable from the search target type by traversing the single-valued attribute; or

3. The result of an aggregate function with valid arguments (eg, Max applied to a scalar attribute in a nested element reachable from a search target type by traversing a multivalued attribute or relationship).

For example, assuming the search target type is System.Storage.Contact.Person:

1. "Birthdate" - Valid, Birthdate is a scalar property of type Person;

2. "PersonalNames.Surname" - Invalid, PersonalNames is a multi-valued attribute and no aggregate functions are used;

3. "Count(PersonalNames)" - valid, the count of PersonalNames;

4. "Case(Contact.MemberOfHousehold).Household.HouseholdEAddresses.StartDate" - Invalid, using relationships and multivalued properties without aggregate functions.

5. "Max(Cast(Contact.MemberOfHousehold).Household.HouseholdEAddresses.StartDate)" - valid, most recent start date of home electronic addresses.

(3) Item result stream ("FindResult")

The ItemSearcher (eg, via the FindAll method) returns an object (eg, a "FindResult" object) that can be used to access the objects returned by the search. Appendix C provides the source code listing the FindResult class and several closely related classes according to one embodiment thereof.

There are two different methods for obtaining results from the FindResult object: using the reader pattern defined by IObjectReader (object reader interface) (and IAsyncObjectReader (asynchronous object reader interface)), and using IEnumerable (enumerable Enumeration interface) and the enumerator pattern defined by IEnumerator (enumerator interface). The enumerator pattern is standard in the CLR and supports language constructs such as C#'s foreach. For example:

ItemSearcher searcher = Person. GetSearcher(context);

searcher.Filters.Add("PersonalNames.Surname='Smith'");

FindResult result = searcher. FindAll();

foreach(Person person in result)...;

The Reader pattern is supported as it allows for more efficient processing of results by eliminating data copies in some cases. For example:

ItemSearcher searcher = Person. GetSearcher(context);

searcher.Filters.Add("PersonalNames.SurName='Smith'");

FindResult result = searcher. FindAll();

while(result. Read())

{

Person person = (Person) result.Current;

...

}

Additionally, the Reader pattern supports asynchronous operations:

ItemSearcher searcher = Person. GetSearcher(context);

searcher.Filters.Add("PersonalNames.SurName='Smith'");

FindResult result = searcher. FindAll();

IAysncResult asyncResult = result.BeginRead(new AsyncCallback(MyCallback));

void MyCallback(IAsyncResult asyncResult)

{

if(result. EndRead(asyncResult))

{

Person person = (Person) result.Current;

...

}

}

In this embodiment, FindResult should be closed when it is no longer needed. This can be done by calling the Close method or using a language construct such as the statement used by C#. For example:

ItemSearcher searcher = Person. GetSearcher(context);

searcher.Filters.Add("PersonalNames.SurName='Smith'");

using(FindResult result=searcher. FindAll())

{

while(result. Read())

{

Person person = (Person) result.Current;

...

}

}

b) Runtime architecture in operation

Figure 22 shows the runtime architecture in operation. The runtime architecture operates as follows:

1. The application 350a, 350b or 350c is bound to an item in the storage platform.

2. The framework 2004 creates an ItemContext (item context) object 2202 corresponding to the bound item and returns it to the application.

3. The application submits a Find on this ItemContext to obtain a collection of items; the returned collection is conceptually an object graph 2204 (due to relationships).

4. The application changes, deletes and inserts data.

5. The application saves the changes by calling the Update() method.

c) Common programming mode

This section provides various examples of how the storage platform API schema classes can be used to manipulate items in a data store.

(1) Open and close the ItemContext object

An application obtains an ItemContext object that it will use to interact with the data store, for example, by calling the static ItemContext.Open method and providing one or more paths identifying the item domains with which the ItemContext will be associated. The item scope scopes searches performed using the ItemContext such that only the domain item and items contained within that item will be subject to the search. Examples are as follows:

Open an ItemContext that has a DefaultStore storage platform share on the local machine

ItemContext ic = ItemContext. Open()

Open an ItemContext with the given storage platform share

ItemContext ic = ItemContext.Open(@″\\myserver1\DefaultStore″);

Open the ItemContext with the item under the storage platform share

ItemContext ic = ItemContext.Open(@″\\myserver1\WinFSSpecs\api\m6″);

Open ItemContext with multiple item fields

ItemContext ic = ItemContext.Open(@″\\myserver1\My Documents″,

@″\\jane1\My Documents″,

@"\\jane2\My Documents");

When the ItemContext is no longer needed, it must be closed.

Explicitly close the ItemContext

ItemContext ic = ItemContext. Open();

...

ic.Close();

Use statement with ItemContext to close

using(ItemContext ic=ItemContext. Open())

{

...;

}

(2) Search object

In accordance with another aspect of the present invention, the storage platform API provides features that enable an application programmer to formulate queries based on various attributes of items in the data store in a manner that isolates the application programmer from the details of the query language of the underlying database engine. Simplified query model.

An application can perform a search across the fields specified when opening the ItemContext using the ItemSearcher object returned by the ItemContext.GetSearcher method. Search results are accessed using the FindResult object. Assume the following declaration for the following example:

ItemContext ic = ...;

ItemSearcher searcher = null;

FindResult result = null;

Item item = null;

Relationship relationship=null;

ItemExtension itemExtension = null;

The basic search mode involves using the ItemSearcher object retrieved from the ItemContext by calling the GetSearcher (get searcher) method.

Search for all items of a given type

searcher = ic. GetSearcher(typeof(Person));

result = searcher. FindAll();

foreach(Person p in result)...;

Search for items of a given type that satisfy a filter

searcher = ic. GetSearcher(typeof(Person));

searcher.Filters.Add("PersonalNames.Surname='Smith'");

result = searcher. FindAll();

foreach(Person p in result)...;

Use the parameters in the filter string

searcher = ic. GetSearcher(typeof(Person));

searcher.Filters.Add("Birthdate<@Date");

searcher.Parameters["Date"] = someDate;

result = searcher. FindAll();

foreach(Person p in result)...;

Searches for relations of a given type that satisfy a filter

searcher=ic.GetSearcher(typeof(EmployeeEmployer));

searcher.Filters.Add("StartDate<=@Date AND(EndDate>=@Date OR isnull(EndDate))");

searcher.Parameters["Date"] = someDate;

result = searcher. FindAll();

Foreach(EmployeeEmployer ee in result)...;

Searches for items with a relationship of the given type that satisfies the filter

searcher = ic. GetSearcher(typeof(Folder));

searcher.Filters.Add("MemberRelationships.Name like'A%'");//See [ApiRel]

result = searcher. FindAll();

Foreach(Folder f in result)...;

Searches for project extensions of a given type that satisfy the filter

searcher=ic.GetSearcher(typeof(ShellExtension));

searcher.Filters.Add("Keywords.Value='Foo'");

result = searcher. FindAll();

foreach(ShellExtension se in result)...;

Searches for an item with an item extension of the given type that satisfies the filter

searcher = ic. GetSearcher(typeof(Person));

searcher.Filters.Add("Extensions.Cast(@Type).Keywords.Value='Foo'");//See [ApiExt]

searcher.Parameters["Type"] = typeof(ShellExtension);

result = searcher. FindAll();

foreach(Person p in result)...;

(a) Search options

Various options can be specified when performing a search, including sorting, lazy loading, and limiting the number of results.

sort search results

searcher = ic. GetSearcher(typeof(Person));

searcher.Filters.Add("PersonalNames.Surname='Smith'");

SearchOptions options = new SearchOptions();

options.SortOptions.Add(new SortOption("Birthdate", SortOrder.Ascending));

result = searcher. FindAll(options);

foreach(Person p in result)...;

// There are shortcuts available

searcher = ic. GetSearcher(typeof(Person));

searcher.Filters.Add("PersonalNames.Surname='Smith'");

result = searcher.FindAll(new SortOption("Birthdate", SortOrder.Ascending));

foreach(Person p in result)...;

limit result count

searcher = ic. GetSearcher(typeof(Person));

searcher.Filters.Add("PersonalNames.Surname='Smith'");

SearchOptions options = new SearchOptions();

options.MaxResults = 10;

result = searcher. FindAll(options);

foreach(Person p in result)...;

(b) FindOne and FindOnly

Sometimes it is useful to retrieve only the first result, especially when specifying a sorting criterion. Also, some searches are expected to return only one object, and are not expected to return no objects.

search for an object

searcher = ic. GetSearcher(typeof(Person));

searcher.Filters.Add("PersonalNames.Surname='Smith'");

Person p = searcher.FindOne(new SortOption("Birthdate"SortOrder.Ascending)) as Person;

if(p!=null)...;

Search for a single object that is expected to always exist

searcher = ic. GetSearcher(typeof(Person));

searcher.Filters.Add("PersonalNames[Surname='Smith'AND Givenname 'John']");

try

{

Person p = searcher. FindOnly();

...;

}

catch(Exception e)

{

...;

}

(c) Search for shortcuts on the ItemContext

There are also several shortcuts on the ItemContext to make simple searches as easy as possible.

Search using the ItemContext.FindAll shortcut

result=ic.FindAll(typeof(Person), "PersonalNames.Surname='Smith'");

foreach(Person p in result)...;

Search using the Item.Context.FindOne shortcut

Person p=ic.FindOne(typeof(Person), "PersonalNames.Surname='Smith'") as Person;

(d) Find by ID or path

Additionally, items, relationships, and item extensions can be retrieved by providing their ids. Items can also be retrieved by path.

Get an item, relationship, and item extension given an id

item = ic.FindltemByld(iid);

relationship=ic.FindRelationshipByld(iid,rid);

itemExtension = ic.FindltemExtensionByld(iid, eid);

Get an item given a path

// single field only

item = ic.FindltemByPath(@″temp\foo.txt″);

// single or multiple fields

result = ic.FindAllItemsByPath(@″temp\foo.txt″);

foreach(Item I in result)...;

(e) GetSearcher mode

There are many places in the storage platform API where it is desirable to provide helper methods that perform searches within the context of another object or with specific parameters. The GetSearcher pattern allows for these situations. There are many GerSearcher methods in the API. Each of these returns an ItemSearcher pre-configured to perform a given search. For example:

searcher = itemContext. GetSearcher();

searcher = Person. GetSearcher();

searcher = EmployeeEmployer. GetSearcherGivenEmployer(organization);

searcher = person. GetSearcherForReports();

Additional filters can be added before performing the search:

searcher = person. GetSearcherForReports();

searcher.Filters.Add("PersonalNames.Surname='Smith'");

Optionally choose how you want the result:

FindResult findResult = searcher. FindAll();

Person person = searcher. FindOne();

(3) Update storage

Once an object has been retrieved through a search, it can be modified as required by the application. New objects can also be created and associated with existing objects. Once the application has made all the changes that form a logical group, the application calls ItemContext.Update to persist these changes to storage. According to yet another aspect of the storage platform API of the present invention, the API collects changes made to items by applications and then organizes them into the correct updates required by the database engine (or any kind of storage engine) on which the data storage is implemented. . This enables application programmers to make changes to items in memory, leaving the complexity of data store updates to the API.

Save changes to a single project

Person p=ic.FindItemByld(pid)as Person;

p.DisplayName = "foo";

p.TelephoneNumbers.Add(new TelephoneNumber(″425-555-1234″));

ic.Update();

Save changes to multiple projects

Household h1＝ic.FindItemByld(hid1)as Household;

Household h2＝ic.FindItemByld(hid2)as Household;

Person p=ic.FindItemByld(pid)as Person;

h1.MemberRelationships.Remove(p);

h2.MemberRelationships.Add(p);

ic.Update();

create new project

Folder f＝ic.FindItemByld(fid)as Folder;

Person p = new Person();

p.DisplayName = "foo";

f.Relationships.Add(new FolderMember(p, "foo"));

ic.Update();

// or use a shortcut...

Folder f＝ic.FindItemByld(fid)as Folder;

Person p = new Person();

p.DisplayName = "foo";

f.MemberRelationships.Add(p, "foo");

ic.Update();

Delete the relationship (and possibly the target item)

searcher = ic. GetSearcher(typeof(FolderMember));

searcher.Filters.Add("SourceItemId=@fid");

searcher.Filters.Add("TargetItemId=@pid");

searcher.Parameters.Add("fid", fid);

searcher.Parameters.Add("pid", pid);

foreach(FolderMember fm in searcher. FindAll()) fm.MarkForDelete();

ic.Update();

// or use a shortcut...

Folder f＝ic.FindItemByld(fid)as Folder;

f. MemberRelationships. Remove(pid);

ic.Update();

Add project extension

Item item = ic.FindItemByld(iid);

MyExtension me = new MyExtension();

me.Foo="bar";

item.Extensions.Add(me);

ic.Update();

remove project extension

searcher = ic. GetSearcher(typeof(MyExtension));

searcher.FiIters.Add("ItemId=@iid");

searcher.Parameters.Add("iid", iid);

foreach(MyExtension me in searcher.FindAll())me.MarkForDelete();

ic.Update();

// or use a shortcut...

Item i = ic.FindItemByld(iid);

i.Extensions.Remove(typeof(MyExtension));

ic.Update();

6. Security

Referring to Section II.E (Security) above, in this embodiment of the storage platform API, there are five methods available on the item context for retrieving and modifying the security policy associated with an item in storage. These methods are:

1. GetItemSecurity;

2. SetItemSecurity;

3. GetPathSecurity;

4. SetPathSecurity and

5. GetEffectiveItemSecurity.

GetItemSecurity and SetItemSecurity provide mechanisms to retrieve and modify explicit ACLs associated with items. This ACL is independent of the path that exists to the item, and is in progress independent of the holding relationship that has the item as the target. This enables an administrator to infer the security of an item as desired independently of the paths that exist to the item.

GetPathSecurity and SetPathSecurity provide mechanisms for retrieving and modifying ACLs that exist on an item due to a holding relationship from another folder. This ACL consists of the ACLs from the respective ancestors for the path under consideration to the item and the explicit ACL for that path, if provided. The difference between this ACL and the ACL in the previous case is that this ACL is kept in progress only if the corresponding holding relationship exists, whereas an explicit item ACL is independent of any holding relationship to the item.

The ACLs that can be set on items with SetItemSecurity and SetPathSecurity are limited to inheritable and object-specific ACEs. They cannot contain any ACEs that are marked as inherited.

GetEffectiveItemSecurity retrieves various paths based on ACLs as well as explicit ACLs on items. This reflects the authorization policy in effect on a given project.

7. Support for relationships

As mentioned above, the storage platform's data model defines "relationships" that allow items to be related to each other. When generating the schema's data classes, the following classes are generated for each relation type:

1. The class that represents the relationship itself. This class is derived from the Relationship class and contains members specific to the relationship type.

2. Strongly typed "virtual" collection classes. This class is derived from VirtualRelationshipCollection and allows creation and deletion of relationship instances.

This section describes the support for relationships in the Storage Platform API.

a) Basic relationship type

The Storage Platform API provides several types in the System.Storage namespace that form the basis of the relational API. These types are:

1. Relationship (relationship) - the base type of all relationship classes

2. VirtualRelationshipCollection (virtual relationship collection) - the basic type of all relationship collections

3. ItemReference (item reference), ItemIdReference (item ID reference), ItemPathReference (item path reference) - represent item reference types; the relationship between these types is shown in FIG. 11 .

(1) Relationship class

public abstract class Relationship:StoreObject

{

//Create with default values

protected Relationship(ItemIDReference targetItemReference);

//Notify the relation that it was added to the relation collection. The object will interrogate the collection to determine the source item, item context, etc.

internalAddedToCollection(virtualRelationshipCollection collection);

//Relationship ID

 public RelationshipId RelationshipId{get;}

//The ID of the source project

 public ItemId SourceItemId{get;}

// Get the source item

public Item SourceItem{get;}

//Reference to the target project

 public ItemIdReference TargetItemReference{get;}

//Get the target item (call TargetItemReference.GetItem())

public Item TargetItem{get;}

//Determine if the ItemContext already has a connection to the domain of the target item

//(call TargetItemReference, IsDomainConnected)

public bool is TargetDomainConnected{get;}

//The name of the target project in the namespace. The name must be unique across all source project holding relationships

 public OptionalValue<string>Name{get;set;}

//Determine whether this is a holding or reference relationship

 public OptionalValue<bool>IsOwned{get;set;}

}

(2) ItemReference class

The following are the base classes for item reference types.

public abstract class ItemReference:NestedElement

{

//Create with default values

protected Item Reference();

// Returns the referenced item

public virtual Item GetItem();

// Determine if a connection to the domain of the referenced project is established

public virtual bool IsDomainConnected();

}

An ItemReference object may identify an item that exists in a storage other than the storage in which the item reference itself resides. Each exported type specifies how references to remote storage are constructed and used. The implementations of GetItem and IsDomainConnected in the exported class use the ItemContext's multi-domain support to load the item from the desired domain and determine whether a connection to that domain has already been established.

(3) ItemIdReference class

Below is the ItemIdReference class - an item reference that uses the item ID to identify the target item.

public class ItemIdReference:ItemReference

{

// Construct a new ItemIdReference with default values

public ItemIdReference();

//Construct a new ItemIdReference to the specified item. The domain associated with the item is used as the locator

public ItemIdReference(Item item);

// Construct a new ItemIdReference with an empty locator and the given target item ID

public ItemIdReference(ItemId itemId);

//Construct a new ItemIdReference with the given locator and item ID value

public ItemIdReference(string locator, ItemId itemId);

//The ID of the target project

 public ItemId ItemId{get;set;}

// Identifies the path to the WinFS project that contains the target project in its domain. If empty, the domain containing the item is unknown.

 public OptionalValue<string>Locator{get;set;}

// Determine if a connection to the referenced project's domain is established

public override bool IsDomainConnected();

//Retrieve the referenced item

public override Item GetItem();

}

GetItem and IsDomainConnected use the ItemContext's multi-domain support to load the item from the desired domain and determine if a connection to that domain has already been established. This feature is not yet implemented.

(4) ItemPathReference class

The ItemPathReference class is an item reference that uses a path to identify the target item. The code for this class is as follows:

ou public class ItemPathReference:ItemReference

{

//Construct project path reference with default value

public ItemPathReference();

//Construct project path reference with given path without any locator

public ItemPathReference(string path);

//Construct project path reference with given locator and path

 public ItemPathReference(string locator, string path);

//Identifies the path to the WinFS project that contains the target project in its domain

 public OptionalValue<string>Locator{get;set;}

//The target project path relative to the project field specified by the locator

 public string Path{get;set;}

// Determine if a connection to the referenced project's domain has been established

public override bool IsDomainConnected();

//Retrieve the referenced item

public override Item Getltem();

}

GetItem and IsDomainConnected use the ItemContext's multi-domain support to load the item from the desired domain and determine if a connection to that domain has already been established.

(5) RelationshipId structure

The RelationshipId structure encapsulates the relationship ID GUID.

publicc lass RelationshipId

{

//Generate a new relationship ID GUID

public static RelationshipId NewRelationshipId();

//Initialize with the new relationship ID GUID

public RelationshipId();

//Initialize with the specified GUID

public RelationshipId(Guid Id);

//Initialize with GUID string identifier

public RelationshipId(string id);

//Returns the string ID of the relationship ID GUID

public override string ToString();

//Convert the System.Guid instance into a RelationshipId instance

public static implicit operator RelationshipId(Guid guid);

//Convert the RelationshipId instance into a System.Guid instance

public static implicit operator Guid(RelationshipId relationshipId);

}

This value type wraps a GUID so that parameters and properties can be strongly typed as relation IDs. OptionalValue<RelationshipId> should be used when the relationship ID is nullable. Empty values such as those provided by System.Guid.Empty are not represented. RelationshipId cannot be constructed with a null value. When using the default constructor to create a RelationshipId, a new GUID is created.

(6) VirtualRelationshipCollection class

The VirtualRelationshipCollection class implements a collection of relationship objects that includes objects from the data store plus new objects that are added to the collection, but not objects that are removed from the store. Objects of the specified relationship type for the given source item ID are included in this collection.

This is the base class for the relation collection classes generated for each relation type. This class can be used as the type of an attribute in the source item type to provide access and easy manipulation of the relationship for a given item.

Enumerating the contents of a VirtualRelationshipCollection may require loading a large number of relationship objects from storage. Applications should use the Count property to determine how many relationships can be loaded before enumerating the contents of the collection. Adding/removing objects to/from a collection does not require loading relationships from storage.

For efficiency, it is preferable for the application to search for relationships that meet certain criteria, rather than enumerating all item relationships using a VirtualRelationshipCollection object. Adding a relationship object to the collection causes the represented relationship to be created in the store when ItemContext.Update is called. Removing a relationship object from a collection causes the represented relationship to be deleted in storage when ItemContext.Update is called. The virtual collection contains the correct set of objects regardless of whether relationship objects were added/removed via the Item.Relationships collection or any other relationship collection on the item.

The following code defines the VirtualRelationshipCollection class:

public abstract class VirtualRelationshipCollection:ICollection

{

//The collection contains relationships of the type owned by the item identified by the item itemId

protected VirtualRelatiohshipCollection(ItemContext itemContext,

ItemId itemId,

Type relationshipType);

// The enumerator will return all objects retrieved from storage minus objects with status Inserted (already inserted)

//any object with status Deleted (already deleted)

public IEnumerator GetEnumerator();

// Returns a count of the number of relation objects returned by the enumerator. This count can be calculated without retrieving all objects from storage.

public int Count{get;}

//Always return false

public bool Icollection.IsSynchronized(){get;}

//Always return this object

 public object ICollection.SyncRoot{get;}

//Search for the desired object in storage

public void Refresh();

//Add the specified relationship to the collection. Objects must have status Constructed (constructed) or Removed (removed).

// If the status is Constructed, it is changed to Added (added). If the status is Removed, it appropriately

//Change to Retrieved (retrieved) or Modified (modified). The source item ID of the relationship must be the same as when constructing the collection

// The provided source project ID is the same.

protected void Add(Relationship relationship);

//Remove the specified relationship from the collection. The status of the object must be Added, Retrieved, or Modified.

// If the object status is Added, it will be set to Constructed. If the object status is Retrieved or Modified,

//It will be set to Removed. The source item ID of the relation must be the same as the source item ID provided when constructing the collection.

protected void Remove(Relationship relationship);

// Objects removed from the collection

public ICollection RemovedRelationships{get;}

// Objects added to the collection

public ICollection AddedRelationships{get;}

// The object retrieved from storage. The collection will be empty until the VirtualRelationshipCollection is enumerated

// or after calling Refresh (obtaining the value of this property does not cause the collection to be filled).

public ICollection StoredRelationships{get;}

//Anonymous method

public IAsyncResult BeginGetCount(IAsyncCallback callback, object state);

public int EndGetCount(IAsyncResult asyncResult);

public IAsyncResult BeginRefresh(IAsyncCallback callback, object state);

public void EndRefresh(IAsyncResult asyncResult);

}

b) Generated relationship types

When generating classes for storage platform schemas, a class is generated for each relationship declaration. In addition to the class representing the relationship itself, a relationship collection class is also generated for each relationship. These classes are used as attribute types in the source or target item classes of the relationship.

This section describes the classes generated using several "prototype" classes. That is, given the specified relationship declarations, describe the generated classes. It is important to note that the class, type and endpoint names used in the prototype class are placeholders for the names specified in the relational schema and should not be interpreted literally.

(1) Generated relationship type

This section describes the classes generated for each relation type. For example:

<Relationship Name="RelationshipPrototype" BaseType="Holding">

<Source Name="Head" ItemType="Foo"/>

<Target Name="Tail"ItemType="Bar"ReferenceType="ItemIDReference"/>

<Property Name="SomeProperty"Type="WinFSTypes.String"/>

</Relationship>

Given the relationship definition RelationshipPrototype, a RelationshipPrototypeCollection class will be generated. The RelationshipPrototype class represents the relationship itself. The RelationshipPrototypeCollection class provides access to RelationshipPrototype instances with the specified project as the source endpoint.

(2) RelationshipPrototype class

This is the prototype relationship class for a holding relationship named "HoldingRelationshipPrototype" where the source endpoint is named "Head" and the item type "Foo" is specified, and the target endpoint is named "Tail" , and the 'Bar' item type is specified. It is defined as follows:

public class RelationshipPrototype: Relationship

{

public RelationshipPrototype(Bar tailItem);

public Re/ationshipPrototype(Bar tailItem, string name);

 public RelationshipPrototype(Bar tailItem, string name, bool IsOwned);

public RelationshipPrototype(Bar tailItem, bool IsOwned);

public RelationshipPrototype(ItemIdReference tailItemReference);

// Get the Head item (call base.SourceItem)

public Foo HeadItem{get;}

//Get Tail item (call base.TargetItem)

public Bar TailItem{get;}

//Represents other attributes declared in the relational schema. These are generated just like attributes on item or nested element types.

public string SomeProperty{get;set;}

public static ItemSearcher GetSearcher(ItemContext itemContext);

public static ItemSearcher GetSearcher(Foo headItem);

public static FindResult FindAll(string filter);

public static RelationshipPrototype FindOne(string filter);

public static RelationshipPrototype FindOnly(string filter);

}

(3) RelationshipPrototpyeCollection class

This is the prototype class generated with the RelationshipPrototype class, which maintains a collection of RelationshipPrototype relationship instances owned by the specified project. It is defined as follows:

public class RelationshipPrototypeCollection:VirtualRelationshipCollection

{

public RelationshipPrototypeCollection(ItemContext itemContext,

ItemId headItemId);

public void Add(RelationshipPrototype relationship);

public RelationshipPrototype Add(Bar bar);

public RelationshipPrototype Add(Bar bar, string name);

public RelationshipPrototype Add(Bar bar, string name, bool IsOwned);

public RelationshipPrototype Add(Bar bar, bool IsOwned);

public void Remove(RelationshipPrototype relationship);

public void Remove(Bar bar);

public void Remove(ItemId barItemId);

public void Remove(RelationshipId relationshipId);

public void Remove(string name);

}

c) Relationship support in the Item class

The Item class contains a Relationships property that provides access to the relationship of which the item is the source of the relationship. The Relationships property has type RelationshipCollection.

(1) Item class

The following code shows the relational context property of the Item class:

public abstract class Item:StoreObject

{

...

//A collection of relationships where the item is the source

public RelationshipCollection Relationships{get;}

...

}

(2) RelationshipCollection class

This class provides access to the relation instance where the given item is the source of the relation. It is defined as follows:

public class RelationshipCollection:VirtualRelationshipCollection

{

public RelationshipCollection(ItemContext itemContext,

ItemId headItemId);

public void Add(Relationship relationship);

public Relationship Add(Bar bar);

public Relationship Add(Bar bar, string name);

public Relationship Add(Bar bar, string name, bool IsOwned);

public Relationship Add(Bar bar, bool IsOwned);

public void Remove(Relationship relationship);

public void Remove(Bar bar);

public void Remove(ItemId barItemId);

public void Remove(RelationshipId relationshipId);

public void Remove(string name);

}

d) Relation support in search expressions

A traversal of the joins between relations and related items may be specified in the search expression.

(1) From item traversal to relationship

When the current context of the search expression is a set of items, the join between the item and the instance of the relationship of which the item is the source can be done using the Item.Relationships property. Relationships that join to specific types can be specified using the Cast (type cast) operator of search expressions.

Strongly typed relationship collections (for example, Folder.MemberRelationships ) can also be used in search expressions. Type casts to relational types are implicit.

Once the set of relationships is established, the attributes of the relationships can be used in decisions or as targets for projections. When used to specify the target of a projection, the set of relations is returned. For example, the following statement will find all owners involved in organizations where the StartDate attribute of the relationship has a value greater than or equal to "1/1/2000".

FindResult result = Person. FindAll(context,

″Relationships.Cast(Contact.EmployeeOfOrganization).StartDate＞’1/1/2000’″);

If the Person type had a property EmployerContext of type EmployeeSideEmployerEmployeeRelationships (generated for the EmployeeEmployer relationship type), then this would be written as:

FindResult result = Person. FindAll(context,

"EmployerRelationships.StartDate > '1/1/2000'");

(2) From relationship traversal to items

When the current context of the search expression is a set of relations, the joins from the relation to any endpoint of the relation can be traversed by specifying the endpoint name. Once the set of related items is established, the attributes of those items can be used for determination or as targets for projections. When used to specify the target of a projection, the set of items is returned. For example, the following statement will find all EmployeeOfOrganization relationships (regardless of the organization) where the employee's last name is the first name "Smith":

FindResult result = EmployeeOfOrganization. FindAll(context,

"Employee. PersonalNames[SurName='Smith']");

The search expression Cast operator can be used to filter the type of endpoint items. For example, to find all MemberOfFoler (folder member) relationship instances where the member is a Person item with the last name "Smith":

FindResult result = MemberOfFolder. FindAll(context,

"Member.Cast(Contact.Person).PersonalNames[Surname='Smith']");

(3) Combination relationship traversal

The first two patterns, item traversal to relations and relational traversal to items, can be combined to achieve arbitrarily complex traversals. For example, to find all organizations that have employees with the last name "Smith":

FindResult result = Organization. FindAll(context,

"EmployeeRelationships."+

"Employee." +

"PersonalNames[SurName='Smith']");

The following example will find all Person items that represent people living in households in the "New York" area (planned items: this is no longer supported...it's a replacement).

FindResult result = Person. FindAll(context,

"Relationships. Cast(Contact. MemberOfHousehold)." +

"Household." +

"Relationships. Cast(Contact. LocationOfHousehold)." +

"MetropolitonRegion = 'New York'");

e) Example usage of relationship support

Below is an example of how to manipulate relationships using the relationship support in the storage platform API. For the following examples, assume the following declarations:

ItemContext ic = ...;

ItemId fid=...;//ID of the folder item

Folder folder = Folder.FindByld(ic, fid);

ItemId sid=...;//The ID of the source item

Item source = Item.FindByld(ic, sid);

ItemId tid=...;//ID of the target item

Item target = Item. FindByld(ic, tid);

ItemSearcher searcher = null;

(1) Search relationship

May search for source or target relationships. Filters can be used to select relationships of a specified type with a given property value. Filters can also be used to select related item types or attribute values based on relationships. For example, the following searches can be performed:

all relationships where the given item is a source

searcher = Relationship. GetSearcher(folder);

foreach(Relationship relationship in searcher. FindAll())...;

where the given item is all relations with a source that has a name that matches "A%"

searcher = Relationship. GetSearcher(folder);

searcher.Filters.Add("Name like'A%'");

foreach(Relationship relationship in searcher. FindAll())...;

All FolderMember relationships where the given item is a source

searcher = FolderMember. GetSearcher(folder);

foreach(FolderMember folderMember in searcher. FindAll())...;

All FolderMember relationships where the given item is a source and have a name like "A%"

searcher = FolderMember. GetSearcher(folder);

searcher.Filters.Add("Name like'A%'");

foreach(FolderMember folderMember in searcher. FindAll())...;

where the target items are all FolderMember relationships of Person

searcher = FolderMember. GetSearcher(folder);

searcher.Filters.Add("MemberItem.Cast(Person)");

foreach(FolderMember folderMember in searcher. FindAll())...;

where the target item is all FolderMember relationships for a Person with last name "Smith"

searcher = FolderMember. GetSearcher(folder);

searcher.Filters.Add("MemberItem.Cast(Person).PersonalNames.Surname='Smith'");

foreach(FolderMember folderMember in searcher. FindAll())...;

In addition to the GerSearcher API shown above, each relationship class supports static FindAll, FineOne and FindOnly APIs. Additionally, the relationship type can be specified when calling ItemContext.GetSearcher, ItemContext.FindAll, ItemContext.FindOne, or ItemContext.FindOnly.

(2) Navigate from the relationship to the source and target items

Once the relational object has been retrieved by searching, it is possible to "navigate" to the target or source item. The base relationship class provides SourceItem (source item) and TargetItem (target item) properties that return an Item object. The generated relationship classes provide equivalent strongly typed and named properties (eg, FolderMember.FolderItem and FolderMember.MemberItem). For example:

Navigate to the source and target projects of the relationship named "Foo"

searcher = Relationship. GetSearcher();

searcher.Filters.Add("Name='Foo'");

foreach(Relationship relationship in searcher. FindAll())

{

  Item source = relationship. SourceItem;

  Item target = relationship.TargetItem;

}

Navigate to the target project

searcher = FolderMember. GetSearcher(folder);

searcher.Filters.Add("Name like'A%'");

foreach(FolderMember folderMember in searcher. FindAll())

{

  Item member = folderMember.TargetItem;

...

}

Navigating to the target item works even in domains where the target item does not find the relationship. In this case, the storage platform API opens a connection to the desired target domain. Applications can determine whether a connection is required before retrieving the target item.

Check target project in unconnected domain

searcher = Relationship. GetSearcner(source);

foreach(Relationship relationship in searcher. FindAll())

{

If(relationship. IsTargetDomaiinConnected)

{

Item member = relationship.TargetItem;

...

}

}

(3) Navigate from the source item to the relationship

Given an item object, it is possible to navigate to relationships for which the item is a source, without performing an explicit search. This can be done using the Item.Relationships collection property or a strongly typed collection property such as Folder.MemberRelationships. From a relationship, it is possible to navigate to the target item. This navigation works even when the target item is not in the item scope associated with the source item's ItemContext, including when the target item is not in the same store as the source item. For example:

Navigate from source item to relationship to target item

Console.WriteLine("Item{0} is the source of the following relationships:", source.ItemId);

foreach(Relationship relationship in source.Relationships)

{

Item target = relationship.TargetItem;

Console.WriteLine({0}==>{1}″, relationship.Relationshipld, target.ItemId);

}

Navigate from folder item to folder membership to target item

console.WriteLine("Item{0} is the source of the following relationships:", folder.ItemId);

foreach(FolderMember folderMember in folder.MemberRelationships)

{

Item target = folderMember. GetMemberItem();

Console.WriteLine("{0}==>{1}", folderMember.RelationshipId, target.ItemId);

Items can have many relationships, so applications should use caution when enumerating collections of relationships. In general, searches should be used to identify specific relationships of interest rather than to enumerate the entire collection. However, projects that have a set-based programming model for relations and are valuable enough to have many relations are sufficiently rare to justify the risk of developer abuse. An application can check the number of relationships in the collection and use a different programming model if needed. For example:

Check size of relationship collection

if(folder.MemberRelationships.Count＞1000)

{

Console.WriteLine("Too many relationships!");

}

else

{

...

}

The relationship collections described above are "virtual" in the sense that they are not actually populated with objects representing each relationship, unless the application attempts to enumerate the collection. If the collection is enumerated, the result reflects what is in storage, plus what the application added but did not save, but does not reflect any relationships that were removed but not saved by the application.

(4) Create relationships (and items)

New relationships are created by creating a relationship object, adding it to the relationship collection in the source project, and updating the project context. In order to create a new item, a hold or embed relationship must be created. For example:

Add a new item to an existing folder

Bar bar = new Bar();

folder.Relationships.Add(new FolderMember(bar, "name"));

ic.Update();

//or

Bar bar = new Bar();

folder.MemberRelationships.Add(new FolderMember(bar, "name"));

ic.Update();

//or

Bar bar = new Bar();

folder.MemberRelationships.Add(bar, name);

ic.Update();

Add an existing item to an existing folder

folder.MemberRelationships.Add(target, "name");

ic.Update();

Add existing items to new folder

Folder existingFolder＝ic.FindItemByld(fid)as Folder;

Folder newFolder = new Folder();

existingFolder.MemberRelationships.Add(newFolder, "a name");

newFolder.MemberRelationships.Add(target, "a name");

ic.Update();

Add a new item to a new folder

Folder existingFolder＝ic.FindItemByld(fid)as Folder;

Folder newFolder = new Folder();

existingFolder.MemberRelationships.Add(newFolder, "a name");

Bar bar = new Bar();

newFolder.MemberRelationships.Add(bar, "a name");

ic.Update();

(5) Delete relationships (and items)

delete holding relationship

// If the source item and relationship IDs are known...

RelationshipId rid =...;

Relationship r = ic. FindRelationshipByld(fid, rid);

r.MarkForDelete;

ic.Update();

//otherwise...

folder.MemberRelationships.Remove(target);

ic.Update();

8. "Extended" storage platform API

As mentioned above, each storage platform pattern results in a set of classes. These classes have standard methods such as Find ^* , and also have properties for setting and getting field values. These classes and associated methods form the basis of the Storage Platform API.

a) Domain Behavior

In addition to these standard methods, each pattern has a set of domain-specific methods for that pattern. Make these methods domain behaviors. For example, some field behaviors in the contact schema are:

· Is the email address valid?

• Given a folder, get a collection of all members of that folder.

• Given an item ID, get an object representing that item.

• Given a person, get their online status.

·Helper function to create a new contact or a temporary contact.

·etc.

It's important to note that although a distinction is made between "standard" behavior (Find ^*, etc.) and domain behavior, they appear to the programmer as mere methods. The difference between these methods is that the standard behavior is automatically generated from the schema file by the storage platform API design-time tools, while the domain behavior is hard-coded.

By its very nature, domain behavior should be handcrafted. This leads to a practical problem: the initial version of C# required the entire implementation of a class to be in a single file. Thus, this forces the automatically generated class files to have to be edited to add domain behavior. This is a problem in itself.

A feature called local classes was introduced in C# for problems like this. At its most basic, local classes allow class implementations to span multiple files. A local class is the same as a regular class, except that its declaration is preceded by the keyword

partial:

partial public class Person:DerivedItemBase

{

//accomplish

}

Now, the domain behavior of Person can be placed in different files, such as:

partial public class Person

{

public EmailAddress PrimaryEmailAddress

{

      get{/*implementation*/}

}

}

b) Value-added behavior

Data classes with domain behavior form the basis upon which application developers build. However, it is neither possible nor desirable that a data class exhibit every conceivable behavior related to that data. The storage platform allows developers to build on the basic functionality provided by the storage platform API. The underlying pattern here is to write a class whose methods take one or more storage platform data classes as parameters. For example, a value-added class for sending email using Microsoft Outlook or using Microsoft Windows Messenger could be as follows:

MailMessage m = MailMessage. FindOne(...);

OutlookEMailServices.SendMessage(m);

Person p = Person. FindOne(...);

WindowsMessagerServices m = new WindowsMessagerServices(p);

m.MessageReceived+=new MessageReceivedHandler(f);

m.SendMessage("Hello");

These value-added classes can be registered with the storage platform. The registration data is associated with schema metadata that the storage platform maintains for each installed storage platform type. This metadata is stored as a storage platform item and can be queried.

Registration of value-added classes is a powerful feature; for example, it allows the following scenario: Right-click on a Person object in the Shell Explorer, and the allowed set of actions can be derived from the value-added class registered with Person.

c) Value-added behavior as a service provider

In this embodiment, the storage platform API provides a mechanism by which value-added classes can be registered as "services" of a given type. This enables applications to set and get service providers (=value-added classes) of a given type. Value-added classes wishing to take advantage of this mechanism should implement a well-known interface; for example:

interface IChatServices

{

void SendMessage(string msg);

event MessageReceivedHandler MessageReceived;

}

class WindowsMessengerServices:IChatServices

{

...

}

class YahooMessengerServices:IChatServices

{

...

}

All storage platform API data classes implement the ICachedServiceProvider (cached service provider) interface. This interface extends the System.IServiceProvider interface, as follows:

interface ICachedServiceProvider: System.IServiceProvider

{

void SetService(System.Type type, Object provider);

void RemoteService(System.Type type);

}

Using this interface, applications can set service provider instances and request specific types of service providers.

To support this interface, the storage platform data class maintains a hash table of service providers keyed by type. When requesting a service provider, the implementation first looks in the hash table to see if a service provider of the specified type is set. If not, use the registered service provider infrastructure to identify a service provider of the specified type. An instance of that provider is then created, added to the hash table, and returned. Note that it is also possible for a shared method on a data class to request a service provider and forward operations to that provider. For example, this could be used to provide a Send method on a mail message class that uses an email system specified by the user.

9. Design-time architecture

This section describes how the storage platform pattern translates to the storage platform API on the client and UDT classes on the server according to this embodiment of the invention. The diagram of Figure 24 shows the components involved.

Referring to Figure 24, the types in the schema are contained in the XML file (box 1). This file also contains field-level and item-level constraints associated with the schema. The storage platform class generator (xfs2cs.exe - box 2) takes this file and generates a partial class for the storage UDT (box 5) and a partial class for the client class (box 3). For each schema domain, there are other methods - called domain behaviors. There is meaningful domain behavior in storage (box 7), client (box 6), and both (box 4). The code in boxes 4, 6 and 7 is handwritten (not automatically generated). The partial classes in boxes 3, 4 and 6 together form the complete class implementation of the storage platform API domain classes. Boxes 3, 4 and 6 are compiled (box 8) to form the storage platform API class - box 11 (actually the storage platform API is the result of compiling boxes 3, 4 and 6 from all initial schema domains). In addition to domain classes, there are other classes that implement value-added behavior. These classes utilize one or more classes from one or more schema domains. This is represented by box 10. The partial classes in boxes 4, 5 and 7 together form the complete class implementation of the server UDT class. Boxes 4, 5, and 7 are compiled (box 9) to form the server-side UDT assembly - box 12 (actually, the server-side UDT program level is the compiler plus compiles boxes 4, 5, and 7 results). The DDL command generator module (box 13) takes the UDT assembly (box 12) and the schema file (box 1) and installs them on the data store. The process involves table generation and viewing of types in each schema, among other things.

10. Inquiry form

When stripped down to the basics, the pattern for an application when using the storage platform API is to: open an ItemContext; use Find with filter criteria to retrieve the desired object; operate on the object; and send changes back to the storage. This section deals with the syntax for what goes into the filter string.

The filter string provided when looking up a storage platform data object describes the constraints that the object's properties must satisfy in order to be returned. The syntax used by the storage platform API supports type casting and relation traversal.

a) Filter base

The filter string is either empty, indicating that all objects of the specified type are to be returned, or a Boolean expression that must be satisfied by each returned object. An expression refers to a property of an object. The storage platform API runtime knows how these property names map to storage platform type field names and ultimately to the SQL views maintained by the storage platform storage.

Consider the following example:

//find everyone

FindResult res1 = Person. FindAll(ctx)

//Find out all the people whose Gender (gender) attribute value is equal to "Male" (male)

FindResult res2 = Person.FindAll(ctx, "Gender = 'Male'")

// Find people whose Gender attribute value is equal to "Male" and who were born in the last century

FindResult res3 = Person.FindAll(

ctx,

"Gender='Male'And Birthdate<'1/1/2001"")

Properties of nested objects can also be used in filters. For example:

//Find out everyone modified in the last 24 hours

FindResult res1 = Person.FindAll(

ctx,

String.Format("Item.Modified＞'{0}'", DateTime.NowSubstract(nes TimeSpan(24, 0, 0))));

For collections, it is possible to filter members using conditions enclosed in square brackets. For example:

//Find all people whose name is "John" and whose last name is "Smith"

FindResult res1 = Person.FindAll(

ctx,

"PersonalNames[GivenName='John'And Surname='Smith']")

//Find out all persons whose live address is from provider "x" and whose presence class is "y"

FindResult res2 = Person.FindAll(

ctx,

″PersonalRealtimeAddress[ProviderURI=′x′].BasicPresence″+

     "OnlineStatus.Category='y'")

The following example lists all people born since 12/31/1999:

ItemContext ctx = ItemContext. Open("Work Contacts");

FindResult results=

  Person.FindAll(ctx, "Birthdate＞'12/31/1999'");

foreach(Person person in results)

Console.WriteLine(person.DisplayName);

ctx. Close();

Line 1 creates a new ItemContext object to access the "Work Contacts" on the storage platform share on the local computer. Lines 3 and 4 obtain a collection of Person objects in which the Birthday attribute satisfies a date later than 12/31/1999, as specified by the expression "Birthdate>'12/31/1999'". The execution of this FindAll operation is shown in FIG. 23 .

b) Type coercion

It is often the case that the type of the value stored in the property is derived from the type declared by the property. For example, the PersonalEAddress (personal electronic address) attribute in Person contains a collection of types derived from EAdress (electronic address) such as EMailAddress (email address) and TelephoneNumber (telephone number). To filter based on telephone area codes, you must cast from the EAddress type to the TelephoneNumber type:

//Find all people in the phone number 425 area code

FindResult res1 = Person.FindAll(

ctx,

"PersonalEA Addresses."+

″Cast(System.Storage.Contact.TelephoneNumber)).″+

"AreaCode='425'");

// Alternatively, the type name can be passed as follows:

FindResult res1 = Person.FindAll(

ctx,

String.Format("PersonalEAddresses.Cast({0})).AreaCode='425'",

typeof(TelephoneNumber).FullName))

c) filter syntax

The following is a description of the filter syntax supported by the storage platform API according to one embodiment.

filter ::= empty filter | condition

empty filter ::=

condition::=simple condition | compound condition | parenthesized condition

simple condition ::= existence check|comparison

Existence Check ::= Property Reference

comparison ::= property reference comparison operator constant

Compound Condition ::= Simple Condition Boolean Operator Condition

Bracketed condition::='('condition')'

Comparison operator ::='! ='|'=='|'='|'<'|'>'|'>='|'<='

Boolean Operators::='And'|'&&'|'Or'|'||'

constant ::= string constant | number constant

String constant::=′″(any Unicode character) ^* ′″

Note: Embedded ' characters are escaped by copying

Numeric constant::=0-9 ^*

Attribute Reference ::= Simple Attribute Name|Compound Attribute Name

Simple property name ::= (all Unicode characters except '.' and space) ^* filter?

filter::='('condition')'

Composite property name::=(type coercion|relation traversal|simple property name)'.'property reference

Type cast ::= 'Cast('type name')'

Relation Traversal ::= Traverse to Source | Traverse to Target

Traverse to source ::= 'Source('full relation name')'

Traverse to target::='Target('full relation name')'

typename::= fully qualified CLR typename

Full relation name ::= schema name '.' relation name

schema name::=storage platform name

Relationship name::=storage platform name

storage platform name ::= as defined in [schema definition]

11. Remote control

a) Local/remote transparency in the API

The target of data access in the storage platform is the local storage platform instance. If the query (or parts thereof) involve remote data, the local instance acts as a router. Thus, the API layer provides local/remote transparency: there is no structural API difference between local and remote data access. It is purely determined by the scope requested.

The storage platform data store also enables distributed queries; thus, it is possible to connect to a local storage platform instance and execute queries involving items from different volumes, some of which are on local storage and others on remote storage. The store performs operations on the results and presents them to the application. From the point of view of the storage platform API (and thus the point of view of the application developer), any remote access is completely seamless and transparent.

The storage platform API allows applications to determine whether a given ItemContext object (as returned by the ItemContext.Open method) represents a local or remote connection using the IsRemote property, which is a property on the ItemContext object. In particular, an app might want to provide video feedback to help set user expectations for performance, reliability, and more.

b) Realization of remote storage platform

Storage Platform Datastores talk to each other using a special OLEDB provider, which runs over HTTP (the default OLEDG provider uses TDS). In one embodiment, the distributed query is through the default OPENROWSET function of the relational database engine. Provide a special user-defined function (UDF): DoRemoteQuery (server, queryText), to complete the actual remote control.

c) Access to non-storage platform storage

In one embodiment of the storage platform of the present invention, there is no non-generic provider architecture that allows any storage to participate in storage platform data access. However, a limited provider architecture is provided for the specific case of Microsoft Exchange and Microsoft Active Directory (AD). This means that developers can use the storage platform API and access data in AD and Exchange just as they can in the storage platform, but the data they can access is limited to the type of storage platform schema. Thus, address books (=collections of type Person of storage platform) are supported in AD, and mail, calendar and contacts are supported for Exchange.

d) Relationship with DFS

The storage platform attribute upgrader does not upgrade past fixed points. Even if namespaces are rich enough to be accessed through fixed points, queries cannot pass through them. Storage platform volumes can appear as leaf nodes in the DFS tree.

e) Relationship with GXA/Indigo

Developers can use the storage platform API to expose a "GXA header" on top of the data storage. Conceptually, there is no difference in creating any other web service. The Storage Platform API does not use GXA to talk to Storage Platform datastores. As mentioned above, the API uses TDS to talk to local storage; any remote control is handled by local storage using the sync service.

12. Constraints

The storage platform data model allows value constraints on types. These constraints are evaluated automatically on storage, and this process is transparent to the user. Note that constraints are checked at the server. With this in mind, it is sometimes desirable to give developers the flexibility to verify that input data satisfies constraints without incurring the overhead of a round trip to the server. This is especially useful in interactive applications where end users enter data to populate objects. The Storage Platform API provides this facility.

Recall that the storage platform schema is specified in an XML file, which is used by the storage platform to generate the appropriate database objects representing the schema. It is also used by the design-time schema of the Storage Platform API to automatically generate classes.

Here is a partial listing of the XML file used to generate the Contacts schema:

<Schema Name="Contacts" MajorVersion="1"MinorVersion="8">

<ReferencedSchema Name="Base" MajorVersion="1"/>

<Type Name="Person" MajorVersion="1" MinorVersion="0"

ExtendsType="Principal"ExtendsVersion="1">

<Field Name="Birthdate"Type="the storage platformTypes.datetime"

Nullable="true"MultiValued="false"/>

<Field Name="Gender"Type="the storage platformTypes.nvarchar(16)"

Nullable="true"MultiValued="false"/>

<Field Name="PersonalNames"Type="FullName"TypeMajorVersion="1"

Nullable="true"MultiValued="true"/>

<Field Name="PersonalEAddresses"Type="EAddress"

TypeMajorVersion="1"Nullable="true"MultiValued="true"/>

<Field Name="PersonalPostalAddresses"Type="PostalAddress"

TypeMajorVersion="1"Nullable="true"MultiValued="true"/>

<Check>expression</Check>

</Type>

...

...

</Schema>

The Check tag in the XML above specifies constraints on the Person type. There can be more than one Check tag. The above constraints are generally checked in storage. To specify constraints that can also be checked explicitly by the application, the above XML is modified to:

<Schema Name="Contacts" MajorVersion="1"MinorVersion="8">

<ReferencedSchema Name="Base" MajorVersion="1"/>

<Type Name="Person"...>

<Field Name="Birthdate"Type="the storage platformTypes.datetime"

Nullable="true"MultiValued="false"/>

...

<Check InApplication="true">expression</Check>

</Type>

...

...

</Schema>

Note that the new "InApplication" attribute on the <Check> element is set to true. This causes the storage platform API to surface constraints in the API through an instance method called Validate() on the Person class. Applications can call this method on the object to ensure that the data is valid and to prevent possible useless round trips to the server. This returns a boolean indicating the result of the confirmation. Note that value constraints are still applied at the server, regardless of whether the client calls the <object>.Validate() method. Here is an example of how to use Validate:

ItemContext ctx = ItemContext. Open();

// Create a contact in the user's My Contacts folder.

Folder f = UserDataFolder.FindMyPersonalContactsFolder(ctx);

Person p = new Person(f);

//set the person's birthday

p.Birthdate = new DateTime(1959, 6, 9);

//Add names classified as person names

FullName name = new FullName(FullName.Category.PrimaryName);

name.GivenName="Joe";

name.Surname="Smith";

p.PersonalNames.Add(name);

//Confirm the Person object

if(p.Validate()==false)

{

//The data does not represent a valid person

}

//save changes

p.Update();

ctx. Close();

There are multiple access paths to Storage Platform storage - Storage Platform API, ADO.NET, ODBC, OLEDB and ADO. This raises the problem of authoritative constraint checking - ie, how to ensure that data written from eg ODBC passes the same data integrity constraints as data written from the storage platform API. Since all constraints are checked at the store, the constraints are now authoritative. Regardless of the API path used to reach the storage, all writes to the storage are filtered by constraint checks at the storage.

13. Sharing

Shares in the storage platform are of the form:

\\<DNS name>\<context service>

where <DNS name> is the DNS name of the machine and <context service> is a containing folder, virtual folder, or item in a volume on that machine. For example, assume that the machine "Johns Desktop" has a volume named Johns Information, and within that volume there is a folder named Contacts_Categories; this folder contains a folder named Work, which has John's work contacts:

\\Johns_Desktop\Johns_Information$\Contacts_Categories\Work This can be shared as "WorkContacts". Using the shared definition, \\Johns_Desktop\WorkContacts\JaneSmith is a valid storage platform name and identifies the Person project JaneSmith.

a) means sharing

A shared item type has the following attributes: share name and share target (this can be a non-holding link). For example, the name of the above share is WorkContacts and the target is Contacts_Categories\Work on the volume Johns_Information. Here is the schema snippet for the Share type:

<Schema

xmlns="http://schemas.microsoft.com/winfs/2002/11/18/schema"

Name="Share"MajorVersion="1"MinorVersion="0">

<ReferencedSchema Name="Base" MajorVersion="1"/>

<ReferencedSchema Name="the storage platformTypes" MajorVersion="1"/>

<Type Name="Share" MajorVersion="1"MinorVersion="0"

ExtendsType="Base.Item"ExtendsVersion="1">

<Field Name="Name"Type="the storage platformTypes.nvarchar(512)"

   TypeMajorVersion="1"/>

<Field Name="Target"Type="Base.RelationshipData"TypeMajorVersion="1"/>

</Type>

</Schema>

b) Manage shares

Since a share is a project, a share can be managed like any other project. Shares can be created, deleted and modified. Shares are also protected in the same way as other storage platform projects.

c) access share

The application accesses the remote storage platform share by passing the share name (eg, \\Johns_Desktop\WorkContacts) to the storage platform API in the ItemContext.Open() method call. ItemContext.Open returns the ItemContext object instance. The storage platform API then talks to the local storage platform service (recall that accessing remote storage platforms is done through the local storage platform). The local storage platform service in turn talks to a remote storage platform service (eg, on machine Johns_Desktop) with a given share name (eg, WorkContacts). The remote storage platform service then translates the WorkContacts into Contacts_Categories\Work and opens it. After that, queries and other operations are performed as with other scopes.

d) Discoverability

In one embodiment, an application can discover the shares available on a given <DNS name> in the following manner. According to the first approach, the Storage Platform API accepts a DNS name (eg, Johns_Desktop) as a scope parameter in the ItemContext.Open() method. The storage platform API then uses the DNS name as part of the connection string to connect to the storage platform storage. With this connection, the only thing the application can do is call ItemContext.FindAll(typeof(Share)). The storage platform service then performs a union operation on all shares on all attached volumes and returns the set of shares. According to the second method, on the local machine, the administrator can find shares on a specific volume through FindAll(typeof(Share)), or through FindAll(typeof(Share), "Target(ShareDestination).Id=folderId") Discover specific folders.

14.Semantics of Find

The Find ^* methods (whether they are called on the ItemContext object or on individual items) generally apply to items (including embedded items) in a given context. Nested elements do not have Find - they cannot be searched independently of their containing item. This is consistent with the expected semantics of the storage platform data model, where nested elements derive their "identity" from the containing item. To make this concept clearer, here are examples of valid and invalid lookup operations:

a) Show all phone numbers in the system that have area code 206?

doesn't work because that lookup is done on the phone number (element) without referencing the item.

b) Show all phone numbers with area code 206 within all Persons?

Invalid, even though Person(=item) is referenced, the search criteria does not refer to the item.

c) Show all phone numbers of Murali (=single person) with area code 206?

Valid because there are search criteria on the item (Person named "Murali"). The exception to this rule is for nested elements derived directly or indirectly from the Base.Relationship type. These types can be queried individually through relationship classes. This query can be supported because the storage platform implementation uses a "master linked list" to store the Relationship elements instead of embedding them in the item UDT.

15. Storage platform Contacts API

This section gives an overview of the storage platform Contacts API. The schema behind the Contacts API is shown in Figures 21A and 21B.

a) Overview of System.Storage.Contact

The storage platform API includes namespaces for working with items and elements in the contact schema. The namespace is called System.Stroage.Contact.

The schema has for example the following classes:

Projects: UserDataFolder, User, Person, ADService, Service, Group, Organization, Principal, Location

Elements: Profile, PostalAddress, EmailAddress, TelephoneNumber, RealTimeAddress, Eaddress, FullName, BasicPresence, GroupMembership, RoleOccupancy

b) Domain Behavior

The following is a list of domain behaviors for contact mode. When viewed at a high enough level, domain behavior falls into well-organized categories:

· Static helpers, such as Person.CreatePersonalContact() to create a new personal contact;

· Instance helpers, such as user.AutoLoginToAllProfiles() that log a user (an instance of the User class) into all profiles that are marked for automatic login;

· Category GUID, such as Category.Home, Category.Work, etc.;

Export properties, eg emailAddress.Address() - returns a string combining the username and domain fields of a given emailAddress (an instance of the EmailAddress class); and

• Export collections eg person.PersonalEmailAddress - Given an instance of the Person class, get its personal email address.

The following table gives a list of these methods and the category they belong to for each class in Contacts that has domain behavior.

16. Storage platform File API

This section provides an overview of the File API of the storage platform according to an embodiment of the present invention.

a) introduction

(1) Reflect NTFS volumes in the storage platform

The storage platform provides methods for indexing on content in existing NTFS volumes. This is accomplished by extracting ("updating") attributes from each file stream or directory in NTFS and storing these attributes as items in the storage platform.

The file mode of the storage platform defines two item types - File (file) and Directory (directory), to store the updated file system entities. The Directory type is a subtype of the Folder type; it is a containing folder that contains other Directory items or File items.

A Directory item can contain Directory and File items; it cannot contain any other type of item. As for the storage platforms considered, the Directory and File items are read-only from any data access API. The File System Upgrade Manager (FSPM) service asynchronously upgrades changed attributes into the storage platform. The properties of File and Directory items can be changed by Win32 API. The storage platform API can be used to read any attribute of these items, including streams associated with File items.

(2) Create files and directories in the storage platform namespace

When an NTFS volume is upgraded to a storage platform volume, all files and directories within it are in a specific section of the volume. This area is read-only from the storage platform's point of view; FSPM can create new directories and files, and/or change attributes of existing items.

The rest of the volume's namespace may contain the usual full range of storage platform item types - Principal, Organization, Document, Folder, etc. The storage platform also allows files and directories to be created in any part of the storage platform namespace. These "native" files and directories have no counterpart in the NTFS file system; they are stored entirely within the storage platform. Furthermore, changes to properties are immediately visible.

However, the programming model remains the same; as far as storage platform data access APIs are concerned, they are still read-only. "Native" files and directories must be updated using the Win32 API. This simplifies the developer's mental model, which is:

1. Any storage platform project type can be created anywhere in the namespace (unless, of course, prohibited by the license);

2. Any storage platform item type can be read using the storage platform API;

3. All storage platform item types can be written using the storage platform API with the exception of File and Directory

4. Use the Win32 API for writing to File and Directory items regardless of where they are in the namespace; and

5. Changes to File/Directory items in an "upgraded" namespace may not appear directly in the storage platform; in a "non-upgraded" namespace, changes are reflected directly in the storage platform.

b) file mode

Figure 25 shows the schema on which the File API is based.

c) Overview of System.Storage.Files

The storage platform API includes namespaces for working with file objects. The namespace is called System.Storage.Files. The data members of the classes in System.Storage.Files directly reflect the information stored in the storage platform's storage; this information is "upgraded" from the file system object, or can be created natively using the Win32 API. The System.Storage.Files namespace has two classes: FileItem (file item) and DirectoryItem (directory item). The members of these classes and their methods can be easily guessed by looking at the schema diagram in Figure 25. FileItem and DirectoryItem are read-only from the storage platform API. To modify them, you must use the Win32 API or the classes in System.IO.

d) Code example

In this section, three code samples are provided showing the use of the classes in System.Storage.Files.

(1) Open the file and write to it

This example shows how "traditional" file manipulation is done.

ItemContext ctx = ItemContext. Open();

FileItem f = FileItem.FindByPath(ctx, @″\My Documents\billg.ppt″);

// Example of handling file attributes - making sure the file is not read-only

if (! f. IsReadOnly)

{

     FileStream fs = f. OpenWrite();

     //Read, write, and close the file stream fs

}

ctx. Close();

Line 3 uses the FindByPath (find by path) method to open the file. Line 7 shows the use of the upgraded property IsReadOnly to check if the file is writable. If so, use the OpenWrite() method on the FileItem object on line 9 to get the file stream.

(2) Use query

Rich queries may be easily done on files due to the storage platform storage holding properties upgraded from the file system. In this example, all files modified in the last three days are listed:

//List all files modified in the last three days

FindResult result = FileItem.FindAll(

ctx,

"Modified>='{0}'",

DateTime. Now. AddDays(-3));

foreach(FileItem file in result)

{

...

}

Here is another example of using a query - this one finds all writable files of a certain type (=extension):

//Find all writable .cs files in a specific directory

//Equivalent to: dir c:\win\src\api\*.cs/a-r-d

DirectoryItem dir=

DirectoryItem.FindByPath(ctx, @″c:\win\src\api″);

Find Result = dir. GetFiles(

"Extension='cs' and IsReadOnly=false");

foreach(File file in result)

{

...

}

e) Domain Behavior

In one embodiment, file classes also have domain behavior (hand-coded properties and methods) in addition to standard properties and methods. These behaviors are generally based on methods in the corresponding System.IO classes.

J. Summary

As noted above, the present invention is directed to a storage platform for organizing, searching and sharing data. The storage platform of the present invention expands and broadens the concept of data storage beyond existing file systems and database systems, and is designed to store all types of data, including structured, unstructured or semi-structured data, Such as relational (table) data, XML, and new forms of data called items. Through its common storage base and schematized data, the storage platform of the present invention allows for more efficient application development for consumers, knowledge workers and enterprises. It provides a rich and extensible application programming interface that not only makes available the capabilities inherent in its data model, but also includes and extends existing file system and database access methods. It will be appreciated that changes may be made to the above-described embodiments without departing from their broad inventive concepts. Therefore, this invention is not limited to the particular embodiments disclosed, but it is intended to cover modifications which come within the spirit and scope of the present invention as defined by the appended claims.

As is apparent from the above, all or part of the various systems, methods and aspects of the invention can be implemented in the form of program code (ie, instructions). The program code can be stored on a computer readable medium, such as magnetic, electronic, optical storage medium, including but not limited to, floppy disk, CD-ROM, CD-RW, DVD-ROM, DVD-RAM, magnetic tape, flash memory, hard disk drive, or any other machine-readable storage medium in which, when the program code is loaded into and executed by a machine such as a computer or a server, the machine becomes a means for implementing the present invention. The present invention can also be implemented in the form of program code transmitted through some transmission medium, such as by wire or cable, by optical fiber, by a network including the Internet or an intranet, or by any other form of transmission, wherein when the program code When received by a machine such as a computer and loaded into it for execution, the machine becomes a means for carrying out the invention. When implemented on a general-purpose processor, the program code combines with the processor to provide a unique means that operates analogously to specific logic circuits.

Appendix A

namespace System. Storage

{

abstract class ItemContext: IDisposable, IServiceProvider

{

ItemContext creates and manages members

//Applications cannot directly create ItemContext objects, nor can they derive classes from ItemContext.

internal itemContext();

//Create can be used to search for the specified path, or if no path is specified,

//Create an ItemContext that can be used to search the default storage on the local computer.

public static ItemContext Open();

public static ItemContext Open(string path);

public static ItemContext Open(params string[]paths);

// Returns the path specified when creating the ItemContext.

public string[]GetOpenPaths();

//Create a copy of the ItemContext. This replica will have independent transaction, cache, and update state. cache

//Initially empty. It is expected that using a cloned ItemContext will be more efficient than opening a new ItemContext with the same item scope.

public ItemContext Clone();

//Close the ItemContext. Any attempt to use the ItemContext after it has been closed will result in an ObjectDisposedException.

public void Close();

void IDisposable. Dispose();

//True if any domain specified by the ItemContext resolves to the remote computer when opened.

public bool IsRemote{get;}

// Returns an object that provides the requested service type. Returns NULL if the requested service cannot be provided.

//The use of the IServiceProvider mode allows APIs that are not normally used and may cause developer confusion to be excluded

// Outside the ItemContext class. ItemContext can provide the following types of services: ItemSerialization, IStoreObjectCache

public object GerService(Type serviceType);

update related members

// Save the changes represented by all modified objects and all objects passed to MarkForCreate or MarkForDelete.

// An UpdateCollisionException may be thrown if an update conflict is detected.

public void Update();

// Save the changes represented by the specified object. Object must have been modified or passed to MarkForCreate or

//MarkForDelete, otherwise ArgumentException is thrown. If an update conflict is detected,

//UpdateCollisionException can be thrown.

public void Update(object objectToUpdate);

public void Update(IEnumerable objectsToUpdate);

// Refresh the contents of the specified object from storage. If the object has been modified, the changes are overwritten and the object is no longer considered

//changed. An ArgumentException is thrown if anything other than an item, itemextension, or relationship object is specified.

public void Refresh(object objectToRefresh);

public void Refresh(IEnumerable objectsToRefresh);

// Raised when update detects that data has changed in storage between retrieving the modified object and attempting to save it. If not registered

// Any event handler, update throws an exception. If an event handler is registered, it can throw an exception to abort the update,

//Wrap the modified object to overwrite the data in the store, or merge changes made in the store and in the object.

public event ChangeCollisionEventHandler UpdateCollision;

//Raised at various points in update processing to provide update progress information.

public event UpdateProgressEventhandler UpdateProgress;

//Asynchronous version of Update

public IAsyncResult BeginUpdate(IAsyncCallback callback, object state);

public IAsyncResult BeginUpdate(object objectToUpdate,

IAsyncCallback callback,

object state);

 public IAsyncResult BeginUpdate(IEnumerable objectsToUpdate,

IAsyncCallback callback,

object state);

public void EndUpdate(IAsyncResult result);

//Asynchronous version of Refresh

public IAsyncResult BeginRefresh(object objectToRefresh,

IAsyncCallback callback,

object state);

 public IAsyncResult BeginRefresh(IEnumerable objectsToRefresh,

IAsyncCallback callback,

object state);

public void EndRefresh(IAsyncResult result);

business related members

//Start a transaction with the specified isolation level. The default isolation level is ReadCommited. in all situations,

//Start a distributed transaction as it may have to include changing stream typed item properties.

public Transaction BeginTransaction();

public Transaction BeginTransaction(System.Data.IsolationLevel isolationLevel);

Search related members

//Create an ItemSearcher that searches for objects of the specified type in the context of this item. If an except item, relationship, or

//ArgumentException is thrown for types other than project extension.

public ItemSearcher GetSearcher(Type type);

//Find an item given its ID.

public Item FindltemByld(ItemId ItemId);

//Find an item given its path. Paths can be absolute or relative. If it is relative, then if when opening the ItemContext

//If multiple project domains are specified, NotSupportedException will be thrown. Returns NULL if no such item exists.

//Create a connection to the \\machine\share part of the domain to retrieve items. The project will be associated with this domain.

public Item FinditemByPath(string path);

//Find an item given its path. Paths are relative to the specified project domain. Creates a connection to the specified domain to retrieve items. The project will

// Associated with this domain. Returns NULL if no such item exists.

public Item FindltemByPath(string domain, string path);

//Find a set of items given their path. The path is relative to the item scope specified when opening the ItemContext. if not present

// Such an item returns an empty result.

public FindResult FindAllItemsByPath(string path);

//Find a relationship given its ID.

 public Relationship FindRelationshipByld(ItemId itemID,

Relationshipld relationshipld);

//Find out the project extension given its ID.

  public ItemExtension FindItemExtensionByld(ItemId itemId,

ItemExtensionId itemExtensionId);

//Find all items, relationships, or item extensions of the specified type that arbitrarily satisfy the given filter. if specified

//A type other than one of these types throws an ArgumentException.

public FindResult FindAll(Tyoe type);

public FindResult FindAll(Type type, string filter);

// Find any items, relationships, or item extensions of the specified type that satisfy the given filter. If one of these types is specified

//Types other than ArgumentException will be thrown. Returns NULL if no such object is found.

public object FindOne(Type type, string filter);

// Find items, relationships, or item extensions of the specified type that satisfy the given filter. If one of these types other than

//Type, ArgumentException is thrown. If no such object is found, throws

//ObjectNotFoundException. If more than one object is found, MultipleObjectsFoundException is thrown.

public object FindOnly(Type type, string filter);

// Returns true if there is an item, relationship, or item extension of the specified type that satisfies the given filter. If these are specified

//An ArgumentException is thrown for a type other than one of the types.

public bool Exists(Type type, string filter);

// Specifies how objects returned by searches involving objects identify the map maintained by the ItemContext.

public SearchCollisionMode SearchCollisionMode{get;set;}

//Raised when PreserveModifiedObject is specified for ResultMapping. This event allows an application to optionally use the

//Update the modified object with the Russian data retrieved by this search.

public event ChangeCollisionEventHandler SearchCollision;

// Combine objects from another ItemContext into this item context. If denoting the same item, relationship, or item extension

// If the object does not exist in the ItemContext's identity map, create a clone of the object and add it to the map. if

//If the object does exist, update the object with the state and content of the specified object in a manner consistent with SearchCollisionMode.

public Item IncorporateItem(Item item);

public Relationship IncorporateRelationship(Relationship relationship);

public ItemExtension IncorporateItemExtension(ItemExtension itemExtension);

}

//Handlers for the ItemContext.UpdateCollision and ItemSearcher.SearchCollision events.

public delegate void ChangeCollisionEventHandler(object source,

ChangeCollisionEventArgs args);

//Arguments for the ChangeCollisionEventHandler delegate

public class ChangeCollisionEventArgs:EventArgs

{

//Modified item, item extension or relationship object.

public object ModifiedObject{get;}

// Attributes from storage

public IDictionary StoredProperties{get;}

}

//Handler for ItemContext.UpdateProgress

public delegate void UpdateProgressEventHandler(ItemContext itemContext,

UpdateProgressEventArgs args);

//Arguments for the UpdateProgressEventHandler delegate

public class ChangeCollisionEventArgs:EventArgs

{

//Current update operation

public UpdateOperation CurrentOperation{get;}

//Currently updated object

public object CurrentObject{get;}

}

// Specifies how objects returned by searches involving objects identify the map maintained by the ItemContext.

public enum SearchCollisionMode

{

// Indicates that a new object should be created and returned. Objects in the identity map that represent the same item, item extension, or relationship are ignored. if

//If this option is specified, the SearchCollision event will not be triggered.

DoNotMapSearchResults,

// Indicates that an object from the identity map should be returned. If the application modifies the contents of the object, save the modified contents of the object.

// If the object has not been modified, update its contents with the data returned by the search. Apps available for SearchCollision

//Handler for the event and optionally updates the object if needed.

PreserveModifiedObjects,

// Indicates that an object from the identity map should be returned. Update the object's content with the data returned by the search, even if the object has been applied

//Program modification. If this option is specified, the SearchCollision event is not raised.

  OverwriteModifiedObjects

}

//Current update operation

public enum UpdateOperation

{

// Provided when Update is called for the first time. CurrentObject will be null.

OverallUpdateStarting,

// Provided only before Update returns after a successful update. CurrentObject will be null.

OverallUpdateCompletedSuccessfully,

// Provided only before Update throws an exception. CurrentObject will be the exception object.

OverallUpdateCompletedUnsuccessfully,

// Provided when the update of the object is started. CurrentObject will refer to the object used for updates.

ObjectUpdateStarting,

// Provided when a new connection is required. CurrentObject will be the one containing the id passed to ItemContext.Open or from a relation

//The string of the path of the project field retrieved by the Location field.

OpeningConnection

}

}

Appendix B

namespace System.Storage

{

// Perform a search across specific types in the project context.

public class ItemSearcher

{

Constructor

public ItemSearcher();

public ItemSearcher(Type targetType, ItemContext context);

public ItemSearcher(Type targetType, ItemContext context,

params SearchExpression[] filters);

Attributes

//Filter used to identify matching objects.

public SearchExpressionCollection Filters { get; }

//Specifies the ItemContext of the field that will be searched.

public ItemContext ItemContext { get; set; }

//A collection of search parameters.

public ParameterCollection Parameters{get;}

// The type of searcher that will be used for operations. For simple searches, this is the type of object that will be returned.

public Type TargetType { get; set; }

search method

// Find objects of TargetType that meet the conditions specified by Filters. Returns empty if no such object exists

//FindResult.

public FindResult FindAll();

public FindResult FindAll(FindOptions findOptions);

public FindResult FindAll(params SortOption[]sortOptions);

//Find out any object of TargetType that satisfies the condition specified by Filters. Returns null if no such object exists.

public object FindOne();

public object FindOne(FindOptions findOptions);

public object FindOne(params SortOption[]sortOptions);

//Find the TargetType that satisfies the conditions specified by Filters. thrown if no such object is found

//ObjectNotFoundException. If more than one object is found, MultipleObjectsFoundException is thrown.

public object FindOnly();

public object FindOnly(FindOptions findOptions);

//Determine whether an object of TargetType that satisfies the conditions specified by Filters exists.

public bool Exists();

//Create objects that can be used to more efficiently perform the same search repeatedly.

public PreparedFind PrepareFind();

public PreparedFind PrepareFind(FindOptions findOptions);

public PreparedFind PrepareFind(params SortOption[]sortOptions);

// Retrieve the number of records that will be returned by FindAll().

public int GetCount();

//Asynchronous versions of various methods

public IAsyncResult BeginFindAll(AsyncCallback callback,

object state);

public IAsyncResult BeginFindAll(FindOptions findOptions,

AsyncCallback callback,

object state);

public IAsyncResult BeginFindAll(SortOption[]sortOptions,

AsyncCallback callback,

object state);

public FindResult EndFindAll(IAsyncResult ar);

public IAsyncResult BeginFindOne(AsyncCallback callback,

object state);

public IAsyncResult BeginFindOne(FindOptions findOptions,

AsyncCallback callback,

object state);

public IAsyncResult BeginFindOne(SortOption[]sortOptions,

AsyncCallback callback,

object state);

public object EndFindOne(IAsyncResult asyncResult);

public IAsyncResult BeginFindOnly(AsyncCallback callback,

object state);

public IAsyncResult BeginFindOnly(FindOptions findOptions,

AsyncCallback callback,

object state);

public IAsyncResult BeginFindOnly(SortOption[]sortOptions,

AsyncCallback callback,

object state);

public object EndFindOnly(IAsyncResult asyncResult);

public IAsyncResult BeginGetCount(AsyncCallback callback,

object state);

public int EndGetCount(IAsyncResult asyncResult);

public IAsyncResult BeginExists(AsyncCallback callback,

object state);

public bool EndExists(IAsyncResult asyncResult);

}

// options to use when performing the search

public class FindOptions

{

public FindOptions();

public FindOptions(params SortOption[]sortOptions);

//Specify whether lazy-loadable fields should be lazy-loaded.

public bool DelayLoad{get;set;}

//The number of matches returned.

public int MaxResults { get; set; }

//A collection of sorting options.

public SortOptionCollextion SortOptions{get;}

}

//Represent the parameter name and value

public class Parameter

{

// Initialize the Parameter object with name and value

public Parameter(string name, object value);

//parameter name

public string Name{get;}

//parameter value

public object Value{get;set;}

}

//Collection of parameter name/value pairs

public class ParameterCollection:ICollection

{

public ParameterCollection();

public int Count{get;}

public object this[string name]{get;set;}

public object SyncRoot{get;}

public void Add(Parameter parameter);

public Parameter Add(string name, object value);

public bool Contains(Parameter parameter);

public bool Contains(string name);

public void CopyTo(Parameter[]array, int index);

void ICollection.CopyTo(Array array, int index);

IEnumerator IEnumerable. GetEnumerator();

public void Remove(Parameter parameter);

public void Remove(string name);

}

// Indicates a search that has been optimized for repeated execution.

public class PreparedFind

{

public ItemContext ItemContext{get;}

public ParameterCollection Parameters{get:}

public FindResult FindAll();

public object FindOne();

public object FindOnly();

public bool Exists();

}

// Specifies the sorting options used in the search.

public class SortOption

{

// Initialize the object with default values.

public SortOption();

//Use SearchExpression and order to initialize the SortOptions object.

public sortOption(SearchExpression searchExpression, SortOrder order);

// The search SearchExpression that identifies the attribute that will be sorted.

public SearchExpression Expression{get;set;}

//Specify ascending or descending sort order

public SortOrder Order{get;set;}

}

//Collection of sort option objects

public class SortOptionCollection: IList

{

public SortOptionCollection();

public SortOption this[int index]{get;set;}

public int Add(SortOption value);

public int Add(SearchExpression expression, SortOrder order);

int IList. Add(object value);

public void Clear();

public bool Contains(SortOption value);

bool IList. Contains(object value);

public void CopyTo(SortOption[]array, int index);

void ICollection.CopyTo(Array array, int index);

public int Count{get;}

IEnumerator IEnumerable. GetEnumerator();

public void Insert(int index, SortOption value);

void IList. Insert(int index, object value);

public int indexOf(SortOption value);

int IList. IndexOf(object value);

public void Remove(SortOption value);

void IList. Remove(object value);

public void RemoveAt(int index);

public object SyncRoot{get;}

}

//Use the SortOption object to specify the sort order

public enum SortOrder

{

Ascending,

Descending

}

}

Appendix C

namespace System.Storage

{

public abstract class FindResult:IAsyncObjectReader

{

public FindResult();

//Move the FindResult to the next position in the results.

public bool Read();

public IAsyncResult BeginRead(AsyncCallback callback, object state);

public bool EndRead(IAsyncResult asyncResult);

//Current object.

public object Current{get;}

//Returns whether the FindResult contains any objects.

public bool HasResults{get;}

//Returns whether FindResult is closed.

public bool IsClosed{get;}

// Return the type of the item in this FindResult.

public Type ObjectType{get;}

//Close FindResult

public void Close();

void IDisposable. Dispose();

// Returns the enumerator on FindResult, starting from the current position. Advancing any enumerator on FindResult will advance all

// Enumerator and FindResult itself.

IEnumerator IEnumerable. GetEnumerator();

public FindResultEnumerator GetEnumerator();

}

public abstract class FindResultEnumerator:IEnumerator, IDisposable

{

public abstract object Current{get;}

public abstract bool MoveNext();

public abstract void Reset();

public abstract void Close();

void IDisposable. Dispose();

}

}

namespace System

{

//Public interface for iterating over objects

public interface iObjectReader: IEnumerable, IDisposable

{

objectCurrent{get;}

bool IsClosed{get;}

bool HasResults{get;}

Type objectType { get; }

bool Read();

void Close();

}

//Add an asynchronous method to IObjectReader

public interface IAsyncObjectReader:IObjectReader

{

  IAsyncResult BeginRead(AsyncCallback callback, object state);

bool EndRead(IAsyncResult result);

}

}

Claims (61)

1. A method for organizing items in a data store, said items comprising (a) discrete units of storable information manipulable by an operating system, (b) at least one element comprising one or more An instance of the type of the field, and (c) a relationship to at least one other item; the method includes steps for having the item be a member of at least one item folder but not owned by any of the item folders such that any Deletion of the project folder does not automatically result in the deletion of the project means. 2. The method of claim 1, wherein the project is automatically deleted when it no longer belongs to any project folder. 3. The method of claim 1, wherein the project automatically becomes a member of a default project folder when the project no longer belongs to any project folder. 4. The method of claim 1, wherein the project is automatically deleted when it is a member of only one project folder and the project folder is deleted. 5. The method of claim 1, wherein when the project is a member of only one project folder and the project folder is deleted, the project automatically becomes a member of a default project folder. 6. A method for organizing items in a data store, comprising: manipulating a plurality of items including at least one item, wherein each of said plurality of items constitutes a discrete storable unit of information manipulable by a hardware/software interface system; manipulating a plurality of project folders including at least one project folder, wherein the plurality of project folders constitute an organizational structure for the plurality of projects; and wherein the item includes at least one element that is an instance of a type that includes one or more fields and a relationship to at least one other item, the item is a member of at least one item folder but is not used by the item Any one of the folders owns such that deletion of any of the project folders does not automatically result in deletion of the project. 7. The method of claim 6, wherein a project is automatically deleted when it no longer belongs to any project folder. 8. The method of claim 6, wherein the project is automatically deleted when it is a member of only one project folder and the project folder is deleted. 9. The method of claim 6, further comprising manipulating a plurality of categories comprising at least one category, wherein the plurality of categories also constitutes another organizational structure for the plurality of projects. 10. The method of claim 9, wherein categories are defined by item attributes. 11. The method of claim 9, wherein a class of the plurality of classes is defined by an item attribute, and only items that include an item attribute for the class are members of the class. 12. The method of claim 10, wherein each of the plurality of categories is defined by item attributes, and only items that include a particular item attribute for a category are members of that category. 13. The method of claim 6, the discrete unit of storable information comprising a set of basic properties commonly supported across objects exposed by the operating system shell. 14. The method of claim 13, wherein the item is a basic unit of information manipulated by an operating system. 15. The method of claim 6, the method further comprising interconnecting the item with a plurality of relationships, and managing the relationships at a hardware/software interface system level. 16. The method of claim 15, wherein each relation from the plurality of relations constitutes a mapping between a pair of items interconnected by the relation at a hardware/software interface system level . 17. The method of claim 16, wherein each relationship has attributes. 18. The method of claim 17, wherein each relationship includes an attribute "Target" for identifying a target item of the relationship. 19. The method of claim 18, wherein each relationship further includes an attribute "IsOwned" for defining ownership of the target item of the relationship. 20. A computer system for organizing items in a data store, comprising: means for manipulating a plurality of items comprising at least one item, wherein each of said plurality of items constitutes a discrete storable unit of information manipulable by a hardware/software interface system; means for manipulating a plurality of project folders comprising at least one project folder, wherein the plurality of project folders constitute an organizational structure for the plurality of projects; and wherein the item includes at least one element that is an instance of a type that includes one or more fields and a relationship to at least one other item, the item is a member of at least one item folder but is not used by the item Any one of the folders owns such that deletion of any of the project folders does not automatically result in deletion of the project. 21. The computer system of claim 20, said items being interconnected by a plurality of relationships managed by said hardware/software interface system. 22. The computer system of claim 21, wherein the first item has a relationship from itself to the second item. 23. The computer system of claim 22, wherein the first project is a project folder. 24. The computer system of claim 23, wherein the second project is a project folder. 25. The computer system of claim 23, wherein the second item is a category. 26. The computer system of claim 23, wherein the second item is an item that is neither a project folder nor a category. 27. The computer system of claim 23, wherein the subset of projects comprises project folders. 28. The computer system of claim 23, wherein the subset of items comprises categories. 29. The computer system of claim 23, wherein the subset of projects includes projects that are neither project folders nor categories. 30. The computer system of claim 20, said discrete units of storable information having attributes understandable by said hardware/software interface system. 31. The computer system of claim 30, wherein the hardware/software interface system includes a base schema defining at least one item and at least one attribute. 32. The computer system of claim 31 , wherein at least one item in the base schema is a base item that constitutes a base item type from which the hardware/ All other items manipulated in the software interface system. 33. The computer system of claim 32, wherein the base item type includes an attribute for unique identification of the base item in a hardware/software interface system. 34. The computer system of claim 31 , wherein at least one attribute in the base schema is a base attribute, which constitutes a base attribute type from which the hardware / All other attributes used in the software interface system. 35. The computer system of claim 31, wherein: at least one item in said base schema is a base item, which constitutes a base item type from which all other items manipulated in said hardware/software interface system are derived; and At least one attribute in the base schema is a base attribute, which constitutes a base attribute type from which all other attributes used in the hardware/software interface system can be derived. 36. The computer system of claim 35, wherein the base schema further includes a second project derived from the basic project type "first project", the second project constituting the , and the second item expresses a relationship with the first item. 37. The computer system of claim 35, wherein said base schema further includes a second attribute derived from said base attribute type "first attribute", said second attribute constituting a Base type for cell key properties. 38. The computer system of claim 37, wherein the base schema further includes a third attribute derived from the second attribute, the third attribute constituting a base type for class references. 39. The computer system of claim 35, wherein the base schema further includes a second attribute derived from the base attribute type "first attribute", the second attribute constituting the basic type. 40. The computer system of claim 30, wherein the hardware/software interface system includes a core schema defining a set of core items that the hardware/software interface system understands and can use in a predetermined and predictable manner Deal directly with the core project. 41. The computer system of claim 40, wherein each item from the set of core items is directly or indirectly derived from a common single base item. 42. The computer system of claim 41, wherein the common single base item is a base item in a base schema. 43. The computer system of claim 42, wherein the kernel schema further defines a set of kernel attributes that the hardware/software interface system understands and can directly process in a predetermined and predictable manner . 44. The computer system of claim 43, wherein each attribute from the set of core attributes is derived directly or indirectly from at least one base attribute. 45. The computer system of claim 44, wherein the kernel schema includes entries for devices. 46. The computer system of claim 44, wherein the kernel schema includes an entry for events. 47. The computer system of claim 44, wherein the core schema includes items for items. 48. The computer system of claim 44, wherein the kernel schema includes an item for messages. 49. The computer system of claim 44, wherein the core schema includes an item for a principal. 50. The computer system of claim 44, wherein the kernel schema includes an entry for location. 51. The computer system of claim 44, wherein the core schema includes an item for documentation. 52. The computer system of claim 44, wherein the kernel schema includes an entry for statements. 53. The computer system of claim 44, wherein the core schema includes an entry for contacts. 54. The computer system of claim 44, wherein the core schema includes attributes for certificates. 55. The computer system of claim 44, wherein the core schema includes an attribute for a subject id unit key. 56. The computer system of claim 44, wherein the core schema includes an attribute for a postal address. 57. The computer system of claim 44, wherein the core schema includes attributes for rich text elements. 58. The computer system of claim 44, wherein the kernel schema includes attributes for electronic addresses. 59. The computer system of claim 44, wherein the kernel schema includes an attribute for an id unit security package. 60. The computer system of claim 44, wherein the core schema includes a relationship for occupying a role between two contacts. 61. The computer system of claim 44, wherein the kernel schema includes attributes for fundamental existence.

Images