Refresh

A refresh makes recently indexed Elasticsearch documents searchable. A shard first holds incoming documents in an in-memory buffer. When refresh runs, Lucene collects buffered documents into a new segment containing document data and inverted indexes; searches can then see them. The usual refresh interval is one second, so indexing is near-real-time rather than instant. section 5.1.2 section 15.2.3

Refresh batches work because disk I/O is expensive. Creating segments too often adds I/O and can become a bottleneck under heavy indexing. Lucene also treats segments as immutable and merges them cumulatively in groups of three. Refresh therefore affects both how quickly new content appears in search and the resources consumed while indexing. section 5.1.2 section 5.1.3

Tune refresh at the index level according to when new documents must appear in search:

• Use a longer interval, such as 60 seconds or one minute, when delayed search visibility is acceptable and reducing refresh-related I/O matters. section 5.1.3 section 15.2.3
• Set refresh_interval to -1 during a large migration or bulk load when documents should remain unsearchable until indexing completes. section 5.1.3 section 15.2.3
• Re-enable or force refresh after the load; otherwise searches will not return documents indexed while refresh was disabled. section 5.1.3 section 15.2.3
• Use the refresh query parameter on index, delete, update, or bulk requests when client-side control is needed. section 5.1.3

Figure 5.4 Mechanics of indexing documents

Follow the path from a shard’s in-memory buffer to a searchable Lucene segment during indexing. section 5.1.2

NOTE: Increasing the interval reduces refresh activity but increases the period in which indexed documents are uncommitted and not durable; a server failure can lose them. Choose an interval that balances indexing throughput, search freshness, and acceptable loss risk. section 5.1.3