IET Framework

Cache an expensive read

The question this guide answers: "This read is slow and repeated. Can I remember its result for a few seconds?"

For a derived query result, caching is a performance layer, not a correctness layer: add it when a read is expensive, repeated, and a slightly stale answer is acceptable. If freshness requirements, low reuse, or invalidation complexity outweigh the saving, skip it. Choosing capabilities explains that decision, and this page shows the current API: a typed MemoryCache<T>, a CacheKey::scoped key, read-through with get_or_load, and invalidation after a successful commit.

This scope matters. An authoritative store is a different thing. The session cache integration (iet-auth-session-cache::CacheSessions) is a correctness- sensitive SessionStore, and authentication or revocation decisions must not be treated as derived results; see authentication for its authoritative-store semantics. This guide is about caching derived reads, not about moving an authoritative store into memory.

What you add

The trait and key live in iet-cache. The available backend is the in-process iet-cache-memory. A service crate names the trait (iet-cache); the composition root that builds the concrete cache depends on iet-cache-memory. No serialization crate is needed: a cached value must be Clone + Send + Sync, not Serialize.

Minimal usage

Construct one cache per value type, then wrap the one eligible read. This example is complete and compiled against this checkout (see the commands at the end):

rust
use iet_cache::{Cache, CacheKey};
use iet_cache_memory::MemoryCache;
use std::sync::{
    Arc,
    atomic::{AtomicUsize, Ordering},
};
use std::time::Duration;

#[derive(Clone, Debug, PartialEq)]
struct Task {
    id: u128,
    title: String,
}

// Stand-in for the expensive query you are avoiding.
async fn load_tasks(owner: &str) -> iet_core::AppResult<Vec<Task>> {
    Ok(vec![Task {
        id: 1,
        title: owner.to_owned(),
    }])
}

#[tokio::test]
async fn read_through_serves_the_second_call_then_invalidates() -> iet_core::AppResult<()> {
    let cache = MemoryCache::<Vec<Task>>::new(256)?;
    let key = CacheKey::scoped("tasks.v1", "list", &["owner-1"]);
    let loads = Arc::new(AtomicUsize::new(0));

    let first_loads = loads.clone();
    let first = cache
        .get_or_load(&key, Duration::from_secs(15), false, || async move {
            first_loads.fetch_add(1, Ordering::SeqCst);
            load_tasks("owner-1").await
        })
        .await?;

    let second_loads = loads.clone();
    let second = cache
        .get_or_load(&key, Duration::from_secs(15), false, || async move {
            second_loads.fetch_add(1, Ordering::SeqCst);
            load_tasks("owner-1").await
        })
        .await?;

    // The second call is served from the cache, so the loader ran once.
    assert_eq!(first, second);
    assert_eq!(loads.load(Ordering::SeqCst), 1);

    // Invalidation mechanics only: this test has no database transaction, so it
    // models a write that has already committed rather than proving commit order.
    cache.invalidate(&key).await?;
    assert!(cache.get(&key).await?.is_none());
    Ok(())
}

MemoryCache::<T>::new(capacity)? returns an error if capacity is zero. Capacity bounds the number of entries, not the heap owned by T, so keep the values small and the capacity honest. Clones share the same storage, which is how you inject one cache into several services.

The key is part of the design

CacheKey::scoped(namespace, operation, &[parts...]) length-prefixes every component, so adjacent components cannot collide (["tenant:a", "b"] differs from ["tenant", "a:b"]). Include:

  • the visibility scope — owner, tenant, actor subject, profile;
  • the operation and schema version — the namespace/operation arguments;
  • every argument that changes the result — filters, page, cursor, locale, representation, feature flags, and a revision if that is how you invalidate.

The framework does not inject a tenant, owner, or any other scope into the key. You put it there. Authorization still runs on every request: the cache and its key are not an authorization boundary. A key that omits the visible-owner scope will serve one caller's data to another.

Never put credentials, cookies, CSRF tokens, or raw secrets in a key. A stable, non-secret scope identifier such as an owner or user ID is acceptable; secret material is not. Prefer a stable opaque scope identifier when the raw identifier itself is sensitive.

Reads inside a transaction bypass the shared cache

get_or_load(&key, ttl, transaction_active, loader) returns the cached value on a hit, otherwise awaits the loader and stores its value. Pass transaction_active = true whenever the read participates in a transaction. The backend then calls the loader directly and performs no get or put, so the shared cache cannot leak a value read outside the transaction (imports from iet_core elided):

rust,ignore
let mut session = database.session();
let transaction_active = session.state() != TransactionState::Idle;
let value = cache
    .get_or_load(&key, ttl, transaction_active, || async {
        repository.list(&actor.subject, query, page, &mut scope).await
    })
    .await?;

For a read outside a transaction pass false. The starter's service computes transaction_active exactly this way from the session state.

Invalidate only after the write commits

Choose the invalidation shape before you ship, and perform it only after the mutation is durable:

  • Exact result key: cache.invalidate(&key).await? for a key you can reconstruct.
  • A deliberate family: cache.invalidate_prefix(&prefix).await?, where the prefix is built with the same scoped encoding and empty trailing components, e.g. CacheKey::scoped("tasks.v1", "list", &[]). Prefix invalidation matches the encoded key string prefix; keep components aligned so it cannot catch an unrelated key.

If invalidation fails after the commit, the write is already durable. Do not roll back, and do not retry the mutation because of a cache failure. Surface or record the cache error and accept the bounded stale window; TTL bounds it but does not remove it immediately.

The starter instead puts a monotonic revision counter in the key and bumps it after a successful commit. Old keys are simply no longer requested, and they age out by TTL. Either pattern is fine; pick one and test it.

Cache errors are errors, not misses

Propagate cache failures with ? unless the application defines and tests an explicit fallback policy. A failed loader also propagates and does not populate the cache. The following is complete and compiled:

rust
use iet_cache::{Cache, CacheKey};
use iet_cache_memory::MemoryCache;
use std::time::Duration;

#[tokio::test]
async fn loader_error_propagates_and_is_not_cached() -> iet_core::AppResult<()> {
    let cache = MemoryCache::<u64>::new(4)?;
    let key = CacheKey::scoped("tasks.v1", "count", &["owner-1"]);
    let outcome = cache
        .get_or_load(&key, Duration::from_secs(15), false, || async {
            Err::<u64, iet_core::AppError>(iet_core::AppError::new(
                iet_core::ErrorKind::Unavailable,
                "db.down",
                "database unavailable",
            ))
        })
        .await;
    assert!(outcome.is_err());
    assert!(cache.get(&key).await?.is_none());
    Ok(())
}

A zero TTL bypasses read-through storage; put with a zero TTL removes the key.

Limits this backend does not promise

State these limits rather than assuming stronger behavior:

  • Process-local and volatile. A restart loses entries; two application processes do not share a cache.
  • No eviction of live entries. At capacity, a new key fails with Unavailable; existing live entries are never evicted. Updating an existing key still succeeds. Expired entries may be reclaimed lazily on access or write.
  • No single-flight loading. Concurrent misses each run the loader; the cache does not deduplicate them.
  • No cross-process consistency and no race-safe commit invalidation. TTL is a bound on staleness, not an immediate-consistency mechanism.

Use separate MemoryCache<T> instances for different value types. For counters that back admission control, see the increment_window method in rate limiting; that is a different problem from caching a read result. The full contract, including optional analytics, is described in the technical notes in docs/caching.md.

Test what you actually claim

Unit-test the policy, not just the backend: hit and miss, key separation across owners and arguments, transaction bypass, loader failure, zero TTL, and post-commit invalidation. The complete tests on this page were executed in an isolated crate against this checkout. To reproduce them, put the test blocks in a test module with path dependencies on iet-core, iet-cache, and iet-cache-memory, plus Tokio with its macros and runtime features. Then run that crate's cargo test.

IET Framework documentation — built with the IET Framework.