xoxys/thegeeklab
1
0
Fork 0
Code Issues 1 Pull Requests Releases Wiki Activity

post: How to (not) migrate Graylog to Opensearch #126

Merged
xoxys merged 2 commits from new-post into main 2022-07-14 14:19:01 +00:00
Conversation 0 Commits 2 Files Changed 3 +51
3 changed files with 51 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit c2abb819fa - Show all commits

5
.dictionary
View File

@ -46,3 +46,8 @@ OSCP
JSON
DockerHub
starlark
Graylog
Opensearch
Elasticsearch
EOL
lifecycle

BIN
content/posts/2022/how-to-not-migrate-graylog-to-opensearch/images/feature.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 470 KiB

46
content/posts/2022/how-to-not-migrate-graylog-to-opensearch/index.md Normal file
View File

@ -0,0 +1,46 @@
---
title: "How to (not) migrate Graylog to Opensearch"
date: 2022-07-14T14:45:51+02:00
authors:
- robert-kaussow
tags:
- sysadmin
resources:
- name: feature
src: "images/feature.jpg"
params:
anchor: Center
credits: >
[CHUTTERSNAP](https://unsplash.com/@chuttersnap) on
[Unsplash](https://unsplash.com/s/photos/fail)
---
Graylog is a centralized log management solution to capture, store and analyze log files in real-time. Starting with the latest minor release 4.3 Graylog announced to no longer support Elasticsearch (ES) due to licensing and structural changes Elastic introduced in v7.11. For this reason, the last supported ES version is 7.10, which has already reached EOL on May 11, 2022.
<!--more-->
Fortunately, Graylog also knows this and recommends users to switch even if it is currently not enforced and ES 7.10 continues to work for now <!-- spellchecker-disable -->[^elastic]<!-- spellchecker-enable -->
. As you usually don't wants to operate software that no longer receives security updates, I have started to look into a migration and prepared the Container and Ansible setup. My fist mistake on this journey was to believe I can just use the latest Opensearch (OS) release. Had I read the documentation <!-- spellchecker-disable -->[^graylog]<!-- spellchecker-enable -->
more carefully I would have saved myself a lot of trouble...
Anyway, the actual migration of the cluster from ES v7.10 to OS v2.1 succeeded surprisingly smoothly. Well, almost, after all I had to rewrite the complete Ansible role because OS 2.x has changed almost all configuration parameters and API calls :tada: But as you can imagine, everything explodes while trying to start Graylog again. Dang. Just downgrading Opensearch was also not possible as the cluster and all indices were migrated successfully already. To get it back in a working state I decided to reset the entire cluster and restore the snapshots from the S3 backup repository before I start to start a next try, this time with a supported OS 1.x version :fingers_crossed: At least I have already completed the ES disaster recovery test for this year.
Lessons Learned:
- Read documentations/upgrade instructions more carefully
- Ensure to have a working backup
- Test your recovery process frequently to stay calm and comfortable in case of an emergency
- Test upgrades in a staging environment whenever possible
What annoys me a bit about the whole situation is the back and forth and the rather bad communication in the past from Graylog <!-- spellchecker-disable -->[^gl-github]<!-- spellchecker-enable -->
. Furthermore, the situation with Opensearch is not really better, as it is unclear if e.g. version 1.3 is still supported or not and a general lifecycle information is still missing <!-- spellchecker-disable -->[^os-github]<!-- spellchecker-enable -->
.
<!-- spellchecker-disable -->
[^elastic]: https://www.graylog.org/post/graylog-to-add-support-for-opensearch
[^graylog]: https://docs.graylog.org/docs/migrating-to-opensearch
[^gl-github]: https://github.com/Graylog2/graylog2-server/issues/11804
[^os-github]: https://github.com/opensearch-project/project-website/issues/661
<!-- spellchecker-enable -->