liveBook · Manning

Index rollover

As explained in Elasticsearch in Action, Second Edition

Index rollover is the replacement of a current Elasticsearch index with a new index while directing subsequent writes to the new index without copying existing documents.

Index rollover replaces the active write index with a new empty index as data accumulates. Existing documents stay in the old index rather than being copied; the old index becomes read-only, while later writes go to the new index. This keeps ever-growing indexes from becoming unwieldy and supports continuous ingestion. section 6.8.3 section 6.0

Rollover is especially useful for time-series data such as application logs. A sequence such as logs-18-sept-21 and logs-19-sept-21, or cars_2021-000001 and cars_2021-000002, separates incoming data into successive indexes. Elasticsearch increments a positive numeric suffix and pads the next suffix with zeros. section 6.8.3 section 6.9

A manual rollover uses an alias as the stable write name:

  • Create the first index with a positive numeric suffix, such as cars_2021-000001. section 6.8.3
  • Add an alias and mark that backing index with is_write_index: true, so the alias has a writable target. section 6.8.3
  • Call POST <index_alias>/_rollover. Elasticsearch creates the next index and moves subsequent writes through the alias to it. section 6.8.3
  • Keep searching through an alias that targets the relevant indexes, rather than naming each rolled index in every query. section 3.2.3
Listing 6.45 Creating an alias for an existing index
POST _aliases                           #1
{
  "actions": [                          #2
    {
      "add": {                          #3
        "index": "cars_2021-000001",    #4
        "alias": "latest_cars_a",       #5
        "is_write_index": true          #6
      }
    }
  ]
}

This alias setup makes cars_2021-000001 the writable backing index for latest_cars_a. Afterward, a rollover request to that alias can create the next numbered index. section 6.8.3

Automating rollover with ILM

Index lifecycle management (ILM) can roll over an index when any configured condition is met, such as max_age: 1d, max_docs: 10000, or max_size: 10gb. Attach the policy and its rollover alias through an index template, then create the initial indexed name with that alias marked writable. A multi-phase policy can subsequently shrink an index in a warm phase and delete it later; deletion is irreversible, so back up data first. Policies are scanned every 10 minutes by default, which can make short development tests appear inactive. section 6.9.3 section 6.9.3

Rollover is different from index splitting: splitting redistributes existing data into more shards, whereas rollover starts a blank successor without copying documents. For recurring data, aliases and data streams provide a single search-facing resource across multiple time-based indexes. section 6.8.3 section 3.2.3

How Elasticsearch in Action, Second Edition covers it

How deep it goes
Explains alias setup and _rollover calls, then configures ILM conditions, phases, templates, and polling behavior.
Examples
Uses sequential car indexes, daily log indexes, and a MySQL logs ILM policy with age, document, and size limits.
What it assumes
Introduces rollover from basic index and alias operations; treats ILM as an advanced option for later use.
What you can do after
Operate/configure it through a write alias or ILM policy to rotate incoming data into new indexes.
How it connects
Connects rollover with aliases, index lifecycle management, index templates, data streams, index splitting, and time-series data.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Elasticsearch in Action, Second Edition. The section links point to the parts of the book it draws on.