Snapshot is the mechanism to take the backup of an index or group of indices or the entire cluster from a running Elasticsearch cluster. This is a very effective and stable approach. I have used this approach in production clusters.

Restore is the mechanism to restore the snapshot. This restore operation can be performed on the same Elasticsearch cluster or on a different Elasticsearch cluster. I usually get requests from development teams to take snapshot of a set of indices from the production Elasticsearch cluster and restore it on the UAT cluster. I have a shared NFS mounted to the production cluster and UAT cluster. I register repositories on the production cluster and UAT cluster pointing to the same mount drive. The production repository will be with read-write permission and the UAT repository will be read-only.

Sometime I get requirement to rename the index while doing the replacement. This is possibly due to the existence of an index with the same name.

Here I am explaining the technique to rename an index or indices while doing the restore from a snapshot. A Snapshot may contain one or more indices. We can restore a specific index or a set of indices from a snapshot by specifying their names while doing the restore.

The restore operation must be performed on a functioning cluster. However, an existing index can be only restored if it’s closed and has the same number of shards as the index in the snapshot. The restore operation automatically opens restored indices if they were closed and creates new indices if they didn’t exist in the cluster.

A Sample restore request is given below. The payload is optional and by default it restores all the indices available in the snapshot.

POST /_snapshot/my_bkup_repos/snapshot_01/_restore
{
  "indices": "insights-index,data-index,logs-index",
  "ignore_unavailable": true,
  "include_global_state": false,              
  "rename_pattern": "(.+)",
  "rename_replacement": "restored_$1",
  "include_aliases": false
}

In the above request, I am doing a restore of three indices insights-index , data-index, logs-index from a snapshot with the name snapshot_01 registered in the snapshot repository my_bkup_repos.

The request body has a set of attributes.

  • indices – The name of the index or indices to restore from the snapshot.
  • ignore_unavailable – This is to ignore or consider the unavailable indices present in the snapshot.
  • include_global_state – This refers to the snapshot’s cluster state and feature states. False means, this will be ignored.
  • rename_pattern – This is an option used to rename an index while doing the restore using regular expression.
  • rename_replacement – This is an option used to rename an index while doing the restore by referencing the matched pattern text.
  • include_aliases – This controls whether to restore the aliases associated with the indices in the snaphot. False means, ignore the aliases.

In the above sample request, the actual name of indices and its name after the restore from snapshot are given below

  • insights-index –> restored_insights-index
  • data-index –> restored_data-index
  • logs-index –> restored_logs-index

This is achieved by using the rename_pattern and rename_replacement. The rename_pattern picks the text using the regex from the index name and it gets applied on the rename_replacement logic. In this example, the regex pattern picks the complete name of the index and the replacement logic is to append the picked up text to a prefix restored_. So the restored index will get a name restored_<indexname>.

You can define the regex patterns and replacement logic based on your requirement. The rename_pattern supports java regex. So you can define your own regex in the request.

I hope this explanation is clear. This is a very simple tip. If you are facing any issues, feel free to comment on this blog post. I will respond back.

Advertisement