Memory usage becomes an engineering constraint when resident model weights, caches, or application data compete with available memory. More memory can buy lower latency or stronger local models, but eager retention can create startup delays and future memory problems. section 9.3.4 section 7.4.2
Evaluate memory usage through these practical thresholds and trade-offs:
- Under 16GB, local inference is limited to tab completion and simple pattern continuation; around 24–32GB supports single-file work and begins multi-file reasoning; 48GB or more enables larger mixture-of-experts models and multi-file agent work. section 9.3.4
- Eager caching improves runtime performance by keeping data resident, but increases startup time and can waste memory when many entries are rarely used. section 7.4.1 section 7.4.2
- Set explicit bounds before choosing a cache strategy; a dictionary below 10MB makes an eager cache reasonable, while larger or complex objects require further sizing. section 7.4.2 section 7.4.1
Fast caching with a memory cost
A cached word lookup reduces the benchmark from 63,631.677 ms/op to 0.008 ms/op. Gatling results also favor eager caching, but retaining every word indefinitely increases memory usage and delays application startup. The correct choice depends on the dictionary's size and access distribution. section 7.4.1 section 7.4.2
public class CachedWordsService implements WordsService {
private static final int MULTIPLY_FACTOR = 100;
private static final IntSupplier DEFAULT_INDEX_PROVIDER =
CachedWordsService::getIndexForToday;
private final Path filePath;
private final IntSupplier indexProvider;
private final Set<String> wordCache;
private final List<String> wordList; #A
private final long cacheLoadTime;
private final int totalWords;
public CachedWordsService(Path filePath) { #B
this(filePath, DEFAULT_INDEX_PROVIDER);
}
public CachedWordsService(Path filePath, IntSupplier indexProvider) {
this.filePath = filePath;
this.indexProvider = indexProvider;
long startTime = System.nanoTime();
this.wordCache = loadWordCache(); #C
this.wordList = loadWordList();
this.cacheLoadTime = System.nanoTime() - startTime;
this.totalWords = wordList.size();
System.out.println(String.format(“Dictionary loaded: %,d words in %.2f ms",
totalWords, cacheLoadTime / 1_000_000.0));
}
@Override
public String getWordOfTheDay() {
int index = indexProvider.getAsInt();
if (index >= 0 && index < wordList.size()) { #E
return wordList.get(index); #D
}
return "No word today.";
}
@Override
public boolean wordExists(String word) {
return wordCache.contains(word); #F
}
private Set<String> loadWordCache() {
Set<String> cache = new HashSet<>(); #G
List<String> lines = Files.readAllLines(filePath);
cache.addAll(lines);
return cache;
}
private List<String> loadWordList() { #H
return Files.readAllLines(filePath);
}
}Inspect the cached service to see resident word collections, cache loading, and startup-time measurement in the implementation. section 7.4.1












