WO1998057265B1

WO1998057265B1 - Network object cache engine

Info

Publication number: WO1998057265B1
Application number: PCT/US1998/011834
Authority: WO
Filing date: 1998-06-09
Publication date: 1999-03-11
Anticipated expiration: 1999-12-09

Abstract

The invention provides a method and system for caching information objects transmitted using a computer network. A cache engine determines directly when and where to store those objects in a memory (such as RAM) and mass storage (such as one or more disk drives), so as to optimally write those objects to mass storage and later read them from mass storage, without having to maintain them persistently. Thec cache engine actively allocates those objects to memory or to disk, determines where on disk to store those objects, retrieves those objects in response to their network identifiers (such as their URLs), and determines which objects to remove from the cache so as to maintain sufficient operating space. The cache engine collects information to be written to disk in write episodes, so as to maximize efficiency when writing information to disk and so as to maximize efficiency when later reading that information from disk. The cache engine performs write episodes so as to atomically commit changes to disk during each write episode, so the cache engine does not fail in response to loss of power or storage, or other intermediate failure of portions of the cache. The cache engine also stores key system objects on each one of a plurality of disks, so as to maintain the cache holographic in the sense that loss of any subset of the disks merely decreases the amount of available cache. The cache engine also collects information to be deleted from disk in delete episodes, so as to maximize efficiency when deleting information from disk and so as to maximize efficiency when later writing to those areas having former deleted information. The cache engine responds to the addition or deletion of disks as the expansion or contraction of the amount of available cache.

Claims

AMENDED CLAIMS[received by the International Bureau on 15 January 1999 (15.01.99); new claims 2-14 added; remaining claim unchanged (3 pages)]

1. A system for objects on a network, said system including: a receiver coupled to said network; and a cache engine operative to record an object from said network on a mass storage; wherein said cache engine is capable of selecting times to record said object, selecting locations to record said object, storing said object holographically so as to continue operation after loss of a portion of said mass storage, or minimizing time needed to write to said mass storage.

2. An apparatus for maintaining a set of network objects including: a processor for controlling a cache mechanism configured to cache a plurality of objects including said set of network objects, said processor being capable of communication with a network .and being configured to send at least one of said set of network objects over s.aid network; a m.ass storage, associated with said cache mechanism, in communication with said processor; a memory, associated with said cache mechanism, in communication with said processor and said mass storage; a hash mechanism, configured to locate any of said plurality of objects in said cache mechanism, responsive to an object identifier; and an object storage mechanism, responsive to said hash mechanism, configured to transfer one or more of said plurality of objects between s d memory .and said mass storage.

3. The apparatus of claim 2, wherein said cache mechanism is organized into a plur ity of blocks .and the h.ash mech.anism returns a block pointer to access any of said plurality of objects stored in said cache mechanism.

4. The apparatus of claim 3, wherein said plurality of blocks of the mass storage are directly accessed independent of any filesystem imposed on the mass storage.

5. The apparatus of claim 2, wherein the object storage mech-anism further includes a delayed write means for performing an atomic write episode to write a plurality of blocks containing one or more of said plurality of objects to said mass storage.

6. The apparatus of claims 3, 4 or 5, wherein the object storage mechanism further includes an optimization means for minimizing the time required to transfer said plurality of blocks.

7. The apparatus of claims 2, 3, 4, 5, or 6, wherein the mass storage includes a plurality of disk drives and each of said plurality of disk drives is associated with a corresponding disk set descriptor object, each of said corresponding disk set descriptor objects referenced by a disk set object.

8. The apparatus of claim 7, further including a dynamic mass storage configuration means for updating said disk set object and creating or modifying said corresponding disk set descriptor object responsive to the addition, removal, failure or replacement of one or more of said plurality of disk drives while said apparatus continues to operate.

9. A computer controlled method for maintaining a set of network objects including steps of: controlling a cache mechanism configured to cache a plurality of objects including said set of network objects; locating any of said plurality of objects in said cache mechanism, responsive to an object identifier; atomically transferring, responsive the step of locating, one or more of said plurality of objects between a memory and a mass storage; and sending at least one of said set of network objects over said network.

10. The computer controlled method of claim 9, wherein said cache mechanism is organized into a plurality of blocks and a hash mechanism performs the step of returning a block pointer to access any of said plurality of objects stored in said cache mechanism.

11. The computer controlled method of claim 9, wherein the method further includes the step of performing an atomic write episode to write a plurality of blocks containing one or more of said plurality of objects to said mass storage.

12. The apparatus of claims 9, 10 or 11, further including the step of minimizing the time required to transfer said plurality of blocks.

32

13. The computer controlled method of claims 9, 10, 11, or 12, wherein said mass storage includes a plurality of disk drives and each of said plurality of disk drives is associated with a corresponding disk set descriptor object, each of said corresponding disk set descriptor objects referenced by a disk set object, the method further includes the step of maintaining said disk set object and said corresponding disk set descriptor object to holographically store said set of network objects on said mass storage.

14. The computer controlled method of claim 13, further including steps of: updating said disk set object; and modifying said corresponding disk set descriptor object responsive to the addition, removal, failure or replacement of one or more of said plurality of disk drives.

33 Statement under Article 19

Applicant believes the amendments to the claims do not make any impact on the description or the drawings.

The invention includes a highly reliable and efficient network cache (and associated methods).

Yammine (US 5,564,011) discloses a system and method for regeneration of on-disk dynamic control blocks for the file system. Yammine assumes a mass storage device upon which is imposed a hierarchical file system that contains directories and files. Yammine uses information saved during file system initialization to regenerate the file system for a disk should the existing file system becomes inconsistent.

Jones (EP 0 359 384) discloses a posted- write queue method and apparatus for queuing multiple single sector writes to a disk device. If successive write requests were queued to the disk for successive sectors on the disk, the disk driver performs a multi-sector write instead of multiple single sector writes.

The invention is a stand-alone network cache device used to efficiently cache network objects. The invention need not use a file oriented mass storage system. Instead, objects (including networked objects) are stored directly to data blocks of a mass storage device during write episodes used to optimize storage and retrieval of objects. In addition, objects are atomically transferred between memory and the mass storage device during these write episodes so that the state of the saved information is always consistent on the mass storage. Furthermore, where the mass storage consists of multiple drives, the invention automatically detects the addition, removal, and failure of any of the drives and continues to perform its caching operation with the additional or reduced mass storage.

34