A thread-safe concurrent in-memory cache.

Cache supports full concurrency of retrievals and a high expected concurrency for updates.

Cache utilizes a lock-free concurrent hash table cht::SegmentedHashMap from the cht crate for the central key-value storage. Cache performs a best-effort bounding of the map using an entry replacement algorithm to determine which entries to evict when the capacity is exceeded.

Examples

Cache entries are manually added using insert method, and are stored in the cache until either evicted or manually invalidated.

Here’s an example of reading and updating a cache by using multiple threads:

use moka::sync::Cache;

use std::thread;

fn value(n: usize) -> String {
    format!("value {}", n)
}

const NUM_THREADS: usize = 16;
const NUM_KEYS_PER_THREAD: usize = 64;

// Create a cache that can store up to 10,000 entries.
let cache = Cache::new(10_000);

// Spawn threads and read and update the cache simultaneously.
let threads: Vec<_> = (0..NUM_THREADS)
    .map(|i| {
        // To share the same cache across the threads, clone it.
        // This is a cheap operation.
        let my_cache = cache.clone();
        let start = i * NUM_KEYS_PER_THREAD;
        let end = (i + 1) * NUM_KEYS_PER_THREAD;

        thread::spawn(move || {
            // Insert 64 entries. (NUM_KEYS_PER_THREAD = 64)
            for key in start..end {
                my_cache.insert(key, value(key));
                // get() returns Option<String>, a clone of the stored value.
                assert_eq!(my_cache.get(&key), Some(value(key)));
            }

            // Invalidate every 4 element of the inserted entries.
            for key in (start..end).step_by(4) {
                my_cache.invalidate(&key);
            }
        })
    })
    .collect();

// Wait for all threads to complete.
threads.into_iter().for_each(|t| t.join().expect("Failed"));

// Verify the result.
for key in 0..(NUM_THREADS * NUM_KEYS_PER_THREAD) {
    if key % 4 == 0 {
        assert_eq!(cache.get(&key), None);
    } else {
        assert_eq!(cache.get(&key), Some(value(key)));
    }
}

Thread Safety

All methods provided by the Cache are considered thread-safe, and can be safely accessed by multiple concurrent threads.

Cache<K, V, S> will implement Send when all of the following conditions meet:

• K (key) and V (value) implement Send and Sync.
• S (the hash-map state) implements Send.

and will implement Sync when all of the following conditions meet:

• K (key) and V (value) implement Send and Sync.
• S (the hash-map state) implements Sync.

Sharing a cache across threads

To share a cache across threads, do one of the followings:

• Create a clone of the cache by calling its clone method and pass it to other thread.
• Wrap the cache by a sync::OnceCell or sync::Lazy from once_cell create, and set it to a static variable.

Cloning is a cheap operation for Cache as it only creates thread-safe reference-counted pointers to the internal data structures.

Avoiding to clone the value at get

The return type of get method is Option<V> instead of Option<&V>. Every time get is called for an existing key, it creates a clone of the stored value V and returns it. This is because the Cache allows concurrent updates from threads so a value stored in the cache can be dropped or replaced at any time by any other thread. get cannot return a reference &V as it is impossible to guarantee the value outlives the reference.

If you want to store values that will be expensive to clone, wrap them by std::sync::Arc before storing in a cache. Arc is a thread-safe reference-counted pointer and its clone() method is cheap.

Expiration Policies

Cache supports the following expiration policies:

• Time to live: A cached entry will be expired after the specified duration past from insert.
• Time to idle: A cached entry will be expired after the specified duration past from get or insert.

See the CacheBuilder’s doc for how to configure a cache with them.

Hashing Algorithm

By default, Cache uses a hashing algorithm selected to provide resistance against HashDoS attacks.

The default hashing algorithm is the one used by std::collections::HashMap, which is currently SipHash 1-3.

While its performance is very competitive for medium sized keys, other hashing algorithms will outperform it for small keys such as integers as well as large keys such as long strings. However those algorithms will typically not protect against attacks such as HashDoS.

The hashing algorithm can be replaced on a per-Cache basis using the build_with_hasher method of the CacheBuilder. Many alternative algorithms are available on crates.io, such as the aHash crate.