OpenSearch Upgrades
OpenSearch has been upgraded a couple of times here at ElectricBrain (.... -> 2.6 -> 2.7 -> 2.8 -> 2.9 -> 2.13 -> 2.14Cluster ), however this is the first time anything about the process has been documented from an ElectricBrain perspective.
OpenSearch 2.9.0+2.13.0 -> 2.14.0 cluster
This software required a corresponding hardware upgrade to remain AWS compatible. There are now 4 docker swarm nodes capable of running OpenSearch official containers.
The OpenSearch cluster now consists of 5 containers - 2 x Data, 1 x Coordination, 1 x Manager (as per official OpenSearch documentation) and 1 x OpenSearch-Dashboards.
Moving the indices over was completed by using the snapshot technique where a shared snapshot repository is used. A snapshot is taken of the indices of interest and that is then restored to the new cluster.
Once all the indices where restored they finish up with names like restore_indexName. This requires a renaming operation to make this compatible with the existing software. This operation also make the indices take on the default index properties (shards, replicas etc.)
POST _reindex
{
"source": {
"index": "restored_*"
},
"dest": {
"index": "restored"
},
"script": {
"lang": "painless",
"source": "ctx._index = (ctx._index.substring('restored_'.length(), ctx._index.length()))"
}
}
OpenSearch 2.9.0 -> 2.13.0
The 2.10.0 upgrade was abandonned. It was decided to go directly to 2.13.0. Not all indices were copied over. Hardware was upgraded to Raspberry Pi 5 since the official containers nolonger run on Pi 4 or lower. At this point only a single Pi 5 node is installed.
OpenSearch 2.9.0 -> 2.10.0
Immediate drama!
"Fatal glibc error: This version of Amazon Linux requires a newer ARM64 processor compliant with at least ARM architecture 8.2-a with Cryptographic extensions."
No more Raspberry Pi4 or CM4 or BPi-CM4.
Testing on the Raspberry Pi 5 8GB reveals the CPU complies and the docker 2.10.0 image runs. It also runs on ODroid C4 with its Amlogic S905X3 12nm Processor (ARMv8-A architecture with Neon and Crypto extensions).
A quick test on a dockerized host is to execute "docker run -it opensearchproject/opensearch:2.10.0 /bin/bash" which either ends up at a command prompt or at the dreaded "Fatal glibc error".
Official response from OpenSearch's Docker container team:
Hi @electricbrain-code we switched from Amazon Linux 2 to Amazon Linux 2023 on the base image to increase glibc version as well as in preparation of AL2 deprecation.
This is expected behavior and we currently have not supported RPi officially on our compatibility chart.
https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/
If needed I think you can contact AmazonLinux team regarding their container images on https://github.com/amazonlinux/container-images or https://github.com/amazonlinux/amazon-linux-2023/, as opensearch is just using the existing image provided from them. Thanks.
It is interesting to note that the official response links to a page in the OpenSearch documentation that currently does not list AmazonLinux as a supported operating system (2024-03-29). ** This page has now been update.
The Amazon Linux 2023 requirements page now outlines CPU requirements. It is specifically notes that any Raspberry Pi earlier than the Pi 5 can no longer run AL2023.
Note
"Because of the AL2023 aarch64 base CPU requirements, all Raspberry Pi systems prior to the Raspberry Pi 5 do not meet the minimum CPU requirements."