liveBook · Manning

Major compaction

As explained in ScyllaDB in Action

Major compaction is a user-triggered compaction that processes every SSTable in a selected scope.

Major compaction is a manual compaction run with `nodetool compact`. Instead of waiting for automatic minor compactions, it processes every SSTable in the chosen scope. That scope can be limited to a keyspace, table, or even a token range, rather than the whole cluster. section 6.2.3 section 8.4.1

Compaction merges SSTables that contain overlapping data, keeping the newest version and discarding older versions when they are included together. A major compaction can therefore help when a partition has accumulated tombstones or obsolete data that routine compaction has not yet cleaned up. section 6.2.3 section 8.4.1

Figure 6.6 Compaction combines multiple SSTables into a newer SSTable, skipping obsolete rows and eligible-for-deletion tombstones.
figure

Compaction can remove an obsolete row only when the newer version that shadows it is part of the same compaction. Tombstones also remain until they are eligible for deletion. section 6.2.3

Use major compaction cautiously. It pushes the selected data to the highest tier or level of the table’s compaction strategy. Those highest tiers or levels are compacted infrequently, so old versions can become difficult for later minor compactions to merge away. Repeated major compactions may then be needed to regain the same effect. section 8.4.1 section 9.3.3

Before triggering a major compaction, check the operational trade-offs:

  • It consumes cluster resources while it compacts all selected table data. section 9.3.3
  • A large or persistent compaction backlog can affect cluster performance; `nodetool compactionstats` shows ongoing work by keyspace and table on a node. section 9.3.3
  • Removing a tombstone also depends on `gc_grace_seconds`; compacting does not make a tombstone immediately safe to delete. section 6.2.3

Targeted cleanup decision

If reads from one table are slow because its partitions contain many tombstones or obsolete values, a major compaction of that table may improve the immediate data layout. Monitor compaction work first, and recognize that the cleanup can make future automatic compactions less effective. section 8.4.1 section 9.3.3

How ScyllaDB in Action covers it

How deep it goes
Explains SSTable merging, strategy-tier consequences, tombstone eligibility, resource costs, and monitoring rather than command syntax details.
Examples
Uses a tombstone-heavy partition with slow reads and targeted table, keyspace, or token-range compaction as operational scenarios.
What you can do after
Operate a manual compaction with `nodetool compact` while weighing immediate cleanup against resource use and future compaction difficulty.
How it connects
Connects major compaction to minor compaction, SSTables, tombstones, gc_grace_seconds, compaction strategies, and compactionstats.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of ScyllaDB in Action. The section links point to the parts of the book it draws on.