liveBook · Manning

Memory usage

As explained in Vibe Engineering

Memory usage is the amount of computer memory consumed by a running application or its data.

Memory usage becomes an engineering constraint when resident model weights, caches, or application data compete with available memory. More memory can buy lower latency or stronger local models, but eager retention can create startup delays and future memory problems. section 9.3.4 section 7.4.2

Evaluate memory usage through these practical thresholds and trade-offs:

  • Under 16GB, local inference is limited to tab completion and simple pattern continuation; around 24–32GB supports single-file work and begins multi-file reasoning; 48GB or more enables larger mixture-of-experts models and multi-file agent work. section 9.3.4
  • Eager caching improves runtime performance by keeping data resident, but increases startup time and can waste memory when many entries are rarely used. section 7.4.1 section 7.4.2
  • Set explicit bounds before choosing a cache strategy; a dictionary below 10MB makes an eager cache reasonable, while larger or complex objects require further sizing. section 7.4.2 section 7.4.1

Fast caching with a memory cost

A cached word lookup reduces the benchmark from 63,631.677 ms/op to 0.008 ms/op. Gatling results also favor eager caching, but retaining every word indefinitely increases memory usage and delays application startup. The correct choice depends on the dictionary's size and access distribution. section 7.4.1 section 7.4.2

Cursor
public class CachedWordsService implements WordsService {
    private static final int MULTIPLY_FACTOR = 100;
    private static final IntSupplier DEFAULT_INDEX_PROVIDER =
CachedWordsService::getIndexForToday;
    private final Path filePath;
    private final IntSupplier indexProvider;
    private final Set<String> wordCache;
    private final List<String> wordList; #A
    private final long cacheLoadTime;
    private final int totalWords;
    public CachedWordsService(Path filePath) { #B
        this(filePath, DEFAULT_INDEX_PROVIDER);
    }
 
    public CachedWordsService(Path filePath, IntSupplier indexProvider) {
        this.filePath = filePath;
        this.indexProvider = indexProvider;
        long startTime = System.nanoTime();
        this.wordCache = loadWordCache(); #C
        this.wordList = loadWordList();
        this.cacheLoadTime = System.nanoTime() - startTime;
        this.totalWords = wordList.size();
        System.out.println(String.format(“Dictionary loaded: %,d words in %.2f ms", 
            totalWords, cacheLoadTime / 1_000_000.0));
    }
    @Override
    public String getWordOfTheDay() {
        int index = indexProvider.getAsInt();
        if (index >= 0 && index < wordList.size()) { #E
            return wordList.get(index); #D
        }
        return "No word today.";
    }
    @Override
    public boolean wordExists(String word) {
        return wordCache.contains(word); #F
    }
    private Set<String> loadWordCache() { 
            Set<String> cache = new HashSet<>(); #G
            List<String> lines = Files.readAllLines(filePath);
            cache.addAll(lines);
            return cache;
}
    private List<String> loadWordList() { #H
            return Files.readAllLines(filePath);
    }
}

Inspect the cached service to see resident word collections, cache loading, and startup-time measurement in the implementation. section 7.4.1

How Vibe Engineering covers it

How deep it goes
Compares memory thresholds, resident model parameters, eager caches, startup cost, and dictionary-size constraints.
Examples
Uses local-model RAM tiers, a cached word dictionary, JMH timings, Gatling latency results, and a 10MB sizing constraint.
What it assumes
Assumes awareness of application memory and latency while teaching how data size and access distribution constrain caching decisions.
What you can do after
Evaluate cache and local-inference choices by weighing memory capacity against latency, startup time, model capability, and data size.
How it connects
Connects memory usage with caching, eager loading, lazy loading, startup time, latency, and mixture-of-experts models.

Also explained in

Compare how 13 books explain memory usage →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.