Streaming processes data incrementally instead of constructing and retaining a complete result in memory. It is especially valuable when a feature must handle large or concurrent workloads without weakening its output: obtain a bounded batch, transform or write it, then release it before continuing. The central engineering question is not whether a declarative pipeline looks concise, but whether its traversals, intermediate results, and retained data fit the workload. section 10.3.4 section 10.2.2
Export rows without retaining the workbook
An Excel export that built its whole document in memory appeared harmless with limited local data and one user, but concurrent production exports exhausted the heap. A streaming redesign fetched database rows in batches, wrote each batch immediately to a temporary Excel file, and released those rows from memory as generation continued. The feature kept its full exported data and synchronous user experience while reducing memory pressure. section 10.3.4
Java streams are useful for expressing transformations as a readable sequence such as filter, map, sort, and collect. That clarity does not automatically make a solution a memory-efficient streaming design: separate stream operations can mean separate traversals, and collecting or sorting necessarily retains output data. For large datasets, first identify the actual constraint, then choose between a direct loop, a single-pass collector, or parallel execution. section 6.1.2 section 10.2.2
Listing 10.5 Single-stream constraint using a custom collector
record Stats(Person youngest, Person oldest, List<String> names) {}
record OldestAndNames(Optional<Person> oldest, List<String> names) {}
Stats stats = persons.stream().collect(
Collectors.teeing(
Collectors.minBy( #1
Comparator.comparingInt(Person::age)), #1
Collectors.teeing( #2
Collectors.maxBy( #3
Comparator.comparingInt(Person::age)), #3
Collectors.mapping( #4
Person::name, #4
Collectors.collectingAndThen( #4
Collectors.toList(), #4
list -> { Collections.sort(list); #4
return list; } #4
) #4
), OldestAndNames::new #5
),
(minOpt, resultOldestAndNames) -> new Stats( #6
minOpt.orElse(null), #6
resultOldestAndNames.oldest().orElse(null), #6
resultOldestAndNames.names() #6
) #6
) #6
);
log.info("Youngest: {}, oldest: {}, sorted names: {}",
stats.youngest(),
stats.oldest(),
stats.names());
A custom collector illustrates consolidating minimum, maximum, and sorted names into one stream operation when a single traversal is a real constraint. section 10.2.2
Use streaming deliberately:
- Stream batches to a durable destination when retaining the full generated artifact causes memory pressure. section 10.3.4
- Treat repeated traversals and sorting as costs that matter only once data volume, object size, or transformation work creates noticeable delay. section 10.2.2
- Avoid parallel streams for small or blocking workloads; splitting, coordination, merging, and shared-pool contention can outweigh concurrency gains. section 10.2.2
