liveBook · Manning

Index splitting

As explained in Elasticsearch in Action, Second Edition

Index splitting is the Elasticsearch operation of creating a new index with more primary shards and copying data from an existing index into it.

Index splitting expands an overloaded Elasticsearch index by creating a new index with more primary shards and copying the source data into it. More shards redistribute documents and can provide additional space while helping mitigate slower search responses. section 6.8.1

Expanding an overloaded cars index

Suppose all_cars was created with two primary shards and its data is growing rapidly. Make the source index read-only, then split it into all_cars_new with a larger shard count. The split request waits synchronously until the new index is complete and contains the source data. section 6.8.1

Listing 6.41 Making sure the index is read-only
PUT all_cars/_settings            #1
{
  "settings":{
    "index.blocks.write":"true"   #2
  }
}

Set index.blocks.write to true before splitting so the source index cannot accept indexing operations during the operation. section 6.8.1

Listing 6.42 Splitting the all_cars index
POST all_cars/_split/all_cars_new   #1
{
  "settings": {
    "index.number_of_shards": 12    #2
  }
}
#1 _split expects a target index.
#2 Sets the number of shards on the new index

Call the _split API with the source and target index names, then set the target's index.number_of_shards value. section 6.8.1

Account for these operational conditions:

  • Ensure the target index does not already exist. section 6.8.1
  • Choose a target shard count that satisfies Elasticsearch's splitting rules; the source shard count must be a factor of the target count. section 6.8.1
  • Use splitting to add primary shards; shrinking performs the opposite resizing direction. section 6.10

How Elasticsearch in Action, Second Edition covers it

How deep it goes
Explains the read-only prerequisite, synchronous API operation, target-index requirement, and shard-count factor rule.
Examples
Uses the overloaded all_cars index and demonstrates creating all_cars_new with a larger primary-shard count.
What it assumes
Requires an existing index and teaches making that index read-only before invoking the split operation.
What you can do after
Operate/configure index resizing through the settings API and Elasticsearch's _split API.
How it connects
Connects index splitting with primary shards, index settings, read-only indexes, the _split API, and index shrinking.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Elasticsearch in Action, Second Edition. The section links point to the parts of the book it draws on.