
Upgrading Elasticsearch can be a daunting task, but with the right approach, you can ensure a smooth transition to the latest version.
Elasticsearch 7.10 was a major release that introduced significant changes to the query DSL, including the deprecation of the QueryBuilder.
Before upgrading, it's essential to assess your current Elasticsearch setup and identify potential issues that may arise during the upgrade process.
Elasticsearch 7.10 also introduced a new feature called "index aliases", which allows you to create multiple aliases for a single index, making it easier to manage your indices and improve query performance.
Related reading: Elasticsearch 7 中文文档
Prepare
Before you start the Elasticsearch upgrade process, there's some important prep work to do. Elasticsearch nodes cannot be downgraded after upgrading, so make sure you're ready to move forward with a higher version.
You should review the requirements and considerations for the upgrade, including the fact that Elasticsearch nodes cannot be downgraded after upgrading. This will help you avoid any potential issues down the line.
Make a note of the current state of your Elasticsearch nodes, as this will be crucial for the upgrade process.
A different take: Upgrading My Home Internet to Wifi 6
Node Upgrade
Before upgrading your Elasticsearch nodes, it's essential to categorize them into master-eligible and master-ineligible nodes. Master-eligible nodes can act as master nodes, while master-ineligible nodes cannot.
To upgrade your nodes, follow the correct order: start by upgrading non-master-eligible nodes, then proceed tier-by-tier, beginning with the frozen tier. This ensures that the data can still flow through the tiers during the upgrade.
Here's a step-by-step guide to upgrading nodes in a rolling upgrade: make sure your cluster is stable, disable unnecessary indexing, and stop tasks associated with ML jobs and datafeeds. Then, disable shard allocation, stop the Elasticsearch node, upgrade Elasticsearch, and upgrade plugins. Finally, re-enable shard allocation and check that the upgraded node has rejoined the cluster.
To upgrade nodes in a full cluster restart, shut down all Elasticsearch nodes at once, upgrade them, and then restart each node. Note that in a full cluster restart, the master nodes must be started before the non-master nodes.
A rolling restart upgrade will upgrade a cluster without any downtime, but it's not possible to carry out rolling restart upgrades involving a change in major versions, with the exception of a few specific upgrades.
Recommended read: Wix Website Upgrades
Compatibility Matrix
Compatibility Matrix is a crucial tool to ensure a smooth upgrade process. It helps you understand the compatibility of your current Elasticsearch version with the one you want to upgrade to.
To check the compatibility, you'll want to look at the upgrade matrix, which is a table that outlines the compatibility of different Elasticsearch versions.
Here's a breakdown of the upgrade matrix:
Note that Elasticsearch version 7.17 is the last version that works with JDK 1.8.
Correct Node Upgrade Order
To upgrade your Elasticsearch nodes correctly, it's essential to follow the right order. Before starting the upgrade, you need to divide your cluster into master-eligible nodes and master-ineligible nodes.
Master-ineligible nodes can always join a cluster with a lower version master node, but lower version master-ineligible nodes cannot join a higher version cluster. Therefore, you should first upgrade all master-ineligible nodes.
The correct order for upgrading nodes is to begin by upgrading nodes that are not master-eligible, which can be found using the GET /_nodes/_all,master:false/_none API call or by locating nodes configured with node.master: false.
Here's a step-by-step guide to upgrading nodes:
1. Upgrade master-ineligible nodes first.
2. Proceed with the upgrade tier-by-tier, starting with the frozen tier, followed by the cold, warm, and finally the hot tier.
3. Upgrade the master-eligible nodes last.
By following this order, you can ensure that all nodes can join the cluster during the upgrade process.
Backup and Testing
Before you start the Elasticsearch upgrade process, it's essential to test it in a non-production environment to catch and fix any issues that may arise. This is crucial to avoid any potential problems in your production cluster.
You should set up a test environment to run the upgrade process. This will allow you to identify and resolve any potential issues before upgrading your production cluster. Testing beforehand will save you a lot of headaches in the long run.
Take snapshots of all your Elasticsearch indices before starting the upgrade process. This is a critical step, as it's not possible to downgrade an Elasticsearch node, and the only way to recover your data in case of a failed upgrade is to restore it from snapshots.
For your interest: Elasticsearch Cluster Health
Test Environment Setup
Setting up a test environment is a crucial step before upgrading your Elasticsearch cluster. This allows you to test the upgrade process in a controlled environment, identify potential issues, and resolve them before upgrading your production cluster.
You should test the upgrade process in a test or staging environment before upgrading your production cluster. This is recommended in Example 10: "Set up a test environment".
To set up a test environment, you can create a clone of your production cluster with a different name. This will allow you to test the upgrade process without affecting your production cluster.
A full cluster restart upgrade (offline upgrade) is usually easier than an online upgrade since you don't have to manage a cluster with different versions of nodes at the same time. This type of upgrade will inevitably require your Elasticsearch cluster to be down for the duration of the upgrading process.
Take a Backup and Snapshots
Taking a backup and snapshots is crucial before upgrading your Elasticsearch cluster. You can't downgrade an Elasticsearch node, so it's essential to have a backup plan in place.
It's not possible to reverse a failed upgrade, and the only way to recover your data is to create a new cluster with the old version and recover your data from snapshots. Therefore, it's essential to take snapshots of all Elasticsearch indices before starting the upgrade process.
To take a backup, you can use a tool like Google Cloud Storage and Kibana or Amazon S3 and Kibana, as mentioned in the article. Don't forget to backup Kibana by exporting all of its Saved Objects as well.
You should take a snapshot of your current Elasticsearch cluster, which ensures that you can restore your data if anything goes wrong during the upgrade process. This is a critical step in the upgrade process, and it's essential to do it before proceeding.
Additional reading: Data Lake Gen2 Upgrade
Here are the essential steps to take a backup and snapshots:
- Take a snapshot of your current Elasticsearch cluster
- Backup Kibana by exporting all of its Saved Objects
- Use a tool like Google Cloud Storage and Kibana or Amazon S3 and Kibana to take a backup of your data
By following these steps, you can ensure that you have a backup plan in place and can recover your data if anything goes wrong during the upgrade process.
Plugins and Settings
Before upgrading Elasticsearch, it's essential to consider the plugins you're using. If you're relying on Elasticsearch plugins, check their availability and compatibility with the new version.
Incompatible plugins can lead to a malfunctioning upgraded cluster, so it's crucial to upgrade all plugins to their compatible versions before upgrading Elasticsearch itself.
Elastic Product End of Life Dates
Elastic product EOL (End Of Life) dates are crucial to consider when planning your Elasticsearch setup. Elasticsearch explicitly mentions that each major version is supported for 18 months after the release.
Each major version of Elasticsearch has its own end of life date. Here's a list of some notable EOL dates:
Make sure to check the EOL date for your version of Elasticsearch to avoid any potential issues.
Updating Instance Settings

Updating instance settings is a crucial step after upgrading Elasticsearch. You'll need to change the ElasticsearchMajorVersion instance setting from 6 to 7.
After completing the upgrade, change the ElasticsearchMajorVersion instance setting from 6 to 7. This is a straightforward step that ensures your instance is properly configured for the new version.
Here's a quick rundown of the steps:
- Change the ElasticsearchMajorVersion instance setting from 6 to 7.
This setting update is a one-time process that helps you take full advantage of the new features and improvements in Elasticsearch 7.
Log Monitoring and Allocation
Monitoring your plugin's logs is crucial to ensure everything is running smoothly. This is especially true after making changes to your plugin's settings.
Carefully review your logs for any errors that may have occurred. You can do this by checking the node's logs after restarting it.
Ensure that shards are being allocated properly, as this can impact your plugin's performance.
Kibana and Data
Kibana is a powerful data visualization tool that helps you make sense of your Elasticsearch data.
By default, Kibana connects to Elasticsearch, allowing you to easily view and interact with your data in real-time.
The Elasticsearch upgrade process doesn't change how Kibana works, but it does improve the underlying data storage and retrieval capabilities that Kibana relies on.
Assistant in Kibana
The Upgrade Assistant in Kibana is a game-changer for ensuring a smooth transition to the next major version.
You can find the Upgrade Assistant in the last minor version of each major version, such as 6.8.23 or 7.17.10.
It's essential to clear all issues using the assistant before attempting to upgrade to the next major version, as this will prevent any potential problems down the line.
The Upgrade Assistant in Kibana is accessible through the menu, where you can navigate to it and start the process of ensuring compatibility with the next version.
Expand your knowledge: Elasticsearch Kibana
Updating Kibana
Updating Kibana is a crucial step in keeping your stack up to date.
To update Kibana, you'll need to upgrade the version field in your manifest to 7.8.0. This is the same approach we used when updating Elasticsearch.
After making the change, reapply the deployment by running the snippet below, allowing ECK to take over and upgrade Kibana.
The version field in the manifest should be updated to 7.8.0:
Post-Upgrade
After upgrading your Elasticsearch cluster, it's essential to monitor logs for any errors. This is crucial to ensure that your cluster is running smoothly.
Carefully review the logs for any signs of issues, such as errors or warnings. This will help you identify and fix any problems before they escalate.
Once all nodes are upgraded, it's time to reenable shard allocation. This allows Elasticsearch to balance shards across the cluster, ensuring optimal performance and data distribution.
Reenabling shard allocation is a straightforward process, but it's critical to do it after all nodes are upgraded to avoid any conflicts or data inconsistencies.
Explore further: Facebook Messenger Upgraded the Security of This Chat
Single Node and Remaining
Upgrading a single node in your Elasticsearch cluster requires a bit of finesse, but it's a crucial step in ensuring a smooth transition. You need to make sure your cluster is stable and green before proceeding.
To achieve this, follow the process outlined in the previous steps, but keep in mind that you should upgrade all non-master-eligible nodes first. This means you'll need to repeat the process for each node, one by one.
Here's a quick rundown of the steps involved in upgrading a single node:
- Stop the Elasticsearch node and upgrade it.
- Upgrade plugins to ensure they're compatible with the new version of Elasticsearch.
- Start Elasticsearch and re-enable shard allocation.
- Check that the upgraded node has rejoined the cluster and wait for the cluster status to turn green.
Once you've completed the upgrade for a single node, you can move on to upgrading the remaining nodes in your cluster. The process is similar, but you can follow the same procedure as in Step 6, which involves shutting down, upgrading, and restarting each node one by one.
Single Node
Upgrading Elasticsearch on a single node is a good idea before rolling it out to other nodes. This will help verify that there are no issues with the new version in your environment.
Shutting down a single node is the first step in the process. This ensures that the node is not processing any data while the upgrade is taking place.
You'll need to upgrade the software package on the node. This can be done using the usual package manager tools.
Restarting the node after the upgrade is complete. This will allow you to verify that the new version is working correctly.
Shutting down a single node before upgrading it is a good practice to avoid any potential issues. This helps you troubleshoot any problems that may arise during the upgrade process.
Remaining Nodes
Now that you've upgraded your first node, it's time to tackle the remaining nodes. Make sure to follow the same procedure as before, which involves shutting down, upgrading, and restarting each node one by one.
To ensure a smooth process, it's essential to wait for the cluster to fully recover before moving on to the next node. This means waiting for the cluster status to turn green, which can take some time.
Before upgrading each node, disable shard allocation to prevent the cluster from reallocating shards to another node. You can do this by running a specific command, which is mentioned in the previous section.
Once you've completed the upgrade process for each node, re-enable shard allocation and check that the upgraded node has rejoined the cluster. You can do this by running a command that shows you how many nodes are in the cluster.
Here's a quick summary of the steps to follow for each remaining node:
- Shut down the node
- Upgrade Elasticsearch and plugins
- Start Elasticsearch
- Re-enable shard allocation
- Check that the node has rejoined the cluster
- Wait for the cluster to turn green
By following these steps, you'll be able to upgrade your remaining nodes without any issues. Remember to take your time and wait for the cluster to fully recover before moving on to the next node.
Frequently Asked Questions
How to upgrade Elasticsearch 7.17 to 8?
To upgrade from Elasticsearch 7.17 to 8, follow these two steps: upgrade to 7.17 first, then migrate to 8.1. Start by upgrading to the latest 7. X version, then advance to 8.1 for a seamless transition.
How to upgrade Elasticsearch 6 to 7?
To upgrade Elasticsearch 6 to 7, install and configure Elasticsearch 7, then back up your application-specific indexes and blacklist the bundled Liferay Connector to Elasticsearch 6. This ensures a smooth transition to the latest Elasticsearch version.
Featured Images: pexels.com


