Compare commits

..

124 Commits

Author SHA1 Message Date
35bd685cfe not needed 2026-08-07 23:00:58 +02:00
ce5d7f51c6 streamline table api 2026-08-07 22:59:33 +02:00
59331fc721 satisfy linter 2026-07-17 10:13:32 +02:00
aba5985a2b fix too wide tables when in !tty mode 2026-07-17 10:09:05 +02:00
281499166b fix header count on task ls 2026-07-17 10:08:39 +02:00
744cdcb733 fix word wrapping 2026-07-17 09:52:55 +02:00
6f3ca3243a fix crash 2026-07-17 08:55:48 +02:00
T. von Dein
522898f187 add index copy (aka reindex), fix typoe indicies to indices (#100) 2026-07-16 23:21:45 +02:00
T. von Dein
59cdc03ea2 hide ds backend indices as any other hidden indices (#99) 2026-07-16 21:46:50 +02:00
T. von Dein
d69984b01f satisfy linter (#98) 2026-07-15 23:57:02 +02:00
T. von Dein
00c5d79794 use mapmap.Slicer to filter index templates ls and index ls (#96) 2026-07-15 23:47:19 +02:00
T. von Dein
55a1005857 add index template completion, index pattern filter and name filter (#95) 2026-07-15 14:52:30 +02:00
T. von Dein
2f13e08536 fix #93 crash index template, check nil ptr (#94) 2026-07-15 13:54:39 +02:00
T. von Dein
97a509da9e adopt latest golang enhancements (#91) 2026-07-13 14:33:41 +02:00
c5a21bd677 add esql sample 2026-07-10 15:14:23 +02:00
T. von Dein
f61fda0697 add feature esql search (#90) 2026-07-10 15:07:57 +02:00
T. von Dein
15dbf7ada0 fix api header spec (#89) 2026-07-10 13:58:56 +02:00
T. von Dein
311a8474d6 feature/node-usage (#88) 2026-07-10 13:37:59 +02:00
5644046c78 api repl: print output directly if not json 2026-07-10 11:28:25 +02:00
T. von Dein
9a42b86d62 add support for human readable /_cat endpoints: 'api repl -H' (#87) 2026-07-10 11:21:45 +02:00
T. von Dein
53abe666e8 fix 'node show' and 'shard explain' crashes (#86) 2026-07-09 13:41:07 +02:00
1e9e419e9b fix crash in ByteSize conversion 2026-07-08 15:07:01 +02:00
T. von Dein
c128e32ca1 add more important node stats (#82) 2026-07-08 15:03:31 +02:00
2969522913 rename var 2026-07-08 15:00:21 +02:00
51480e6b35 add 'index du <index>' 2026-07-08 14:58:07 +02:00
99bf703997 add table.AddRowLate() to add rows after sorting 2026-07-08 14:57:46 +02:00
T. von Dein
76ad6c7afa remove workaround to load config in root.Before() (#83)
see: https://github.com/urfave/cli/issues/2348
2026-07-08 14:13:47 +02:00
T. von Dein
8614e09004 show green and failed indices in status, rename red to failed in index ls (#81) 2026-07-08 10:23:01 +02:00
03cfeb4336 fix go tool 2026-07-08 09:34:58 +02:00
T. von Dein
3d3dfb7a42 fix #79: remove compatibility header (#80) 2026-07-08 09:33:22 +02:00
1d45a3cd4c set go version for releaser 2026-07-08 00:00:37 +02:00
0104e1bf58 fix go releaser go version 2026-07-07 23:53:47 +02:00
2dc716d1c9 fix version newline 2026-07-07 23:49:57 +02:00
T. von Dein
0d63e7c4d2 satisfy full linter (#77) 2026-07-07 23:46:43 +02:00
T. von Dein
b8e0ee074c internal/gofix and fix error wrapping (#76) 2026-07-07 10:41:34 +02:00
T. von Dein
f10366dbde mv index allocate to shard allocate, add explanations, fixes #71 (#75) 2026-07-07 08:39:32 +02:00
T. von Dein
c35ffa5e4b internal/rework-table (#74) 2026-07-07 07:29:03 +02:00
92865c0fe2 cosmetics 2026-07-06 14:29:36 +02:00
5475b9986d typo 2026-07-06 14:28:31 +02:00
T. von Dein
82c017144a various fixes (#70)
- dont show ccr status if license insufficient
- add automated tests
- fix config loading
- fix search output
- fix `license show` panics
- fix `ilm show` completion
- print ask passwd prompt to stderr
2026-07-06 14:22:59 +02:00
T. von Dein
a07f38e945 add 'license show', add printer.NewTableEmpty() along .WithHeaders() (#69) 2026-07-03 13:52:36 +02:00
T. von Dein
c197e138ef fix repl output: didn't print json smaller than term height, add debug-http to api repl as well (#68) 2026-07-03 13:30:37 +02:00
T. von Dein
fb6f2389fe add 'cluster settings set' settings completion (#67) 2026-07-03 12:52:13 +02:00
T. von Dein
c9fb572935 get rid of getJsonPath, use flat_settings query flag instead (#66) 2026-07-03 10:25:57 +02:00
T. von Dein
1fc2004474 add external pager support via ES_JSON_PAGER (#65) 2026-07-03 10:07:11 +02:00
T. von Dein
7b576c976c add option -f to usage and filter support (#64) 2026-07-03 08:54:19 +02:00
T. von Dein
b51ee109f2 tune cluster ls: parallized and use tcp connect instead of https (#63) 2026-07-03 08:30:35 +02:00
T. von Dein
b573d63c54 enhance cluster status, add word wrap to wide table rows (#62) 2026-07-02 12:16:59 +02:00
T. von Dein
e3b96b3e3b add node show + node clients (#61) 2026-07-01 13:13:16 +02:00
T. von Dein
6436bcfb22 ask for pass if missing or use env pass, add token support (#60) 2026-07-01 10:58:39 +02:00
T. von Dein
f5c8a23589 add cluster reroute commands, fix usage width bug (#59)
fixes #36
2026-06-29 13:08:05 +02:00
T. von Dein
f9eded5f92 enhance ilm show, add ilm update, rename modify to update everywhere (#58) 2026-06-29 11:48:29 +02:00
T. von Dein
b7a87051c0 add 'ilm show -t' to show a better visualization of a ilm policy (#57) 2026-06-29 10:11:50 +02:00
T. von Dein
81987b75f2 turn --show-command-tree into sub command, hide hidden commands (#56) 2026-06-29 08:38:03 +02:00
34bf4fbed9 add ilm forecast example 2026-06-26 11:41:55 +02:00
T. von Dein
a87f6d0c4f add --hidden flag (default off) to 'index ls' and 'ilm forecast ls' (#55) 2026-06-26 11:32:53 +02:00
T. von Dein
bbe72a474b fix #53: shardsize to string, fix nil pointer, add cold phase (#54) 2026-06-26 11:25:30 +02:00
e3aa50af37 remove debug print 2026-06-26 10:07:42 +02:00
T. von Dein
d2a0812579 feature add ilm-explain (#52) 2026-06-26 10:02:41 +02:00
T. von Dein
6200b5e7b0 enhance/ilm-forecast (#51) 2026-06-26 09:08:14 +02:00
b1b701303c optimzie virtualAge() a little 2026-06-25 20:30:42 +02:00
8f2c68b5b6 fix index response, fix linter issues 2026-06-25 14:19:43 +02:00
d60983411e refactored large func 2026-06-25 14:16:35 +02:00
e1291d1d92 add ilm forecast show 2026-06-25 13:29:24 +02:00
5f60696b01 add ilm forecast data collection, generalized parallel api calls 2026-06-25 11:54:06 +02:00
T. von Dein
825b3ce5d1 fix #49: add ilm policy to index+index-template create+modify (#50) 2026-06-25 07:52:54 +02:00
75a94fade3 fix doc 2026-06-24 10:30:53 +02:00
3e36f97e4b upd documentation 2026-06-24 10:29:17 +02:00
1b9309bca0 upd readme 2026-06-24 09:44:21 +02:00
20b79ce168 left align ints by default again, add -I to right align on request 2026-06-24 09:35:40 +02:00
T. von Dein
ffcb8adb70 enhanced default cluster selection, add 'cluster switch', overhaul json header setting (#47) 2026-06-24 09:25:50 +02:00
3c6819151f generate command tree from within (esctl --show-command-tree) 2026-06-23 14:02:58 +02:00
faa8c0fe26 fix typo 2026-06-23 14:02:40 +02:00
92ddf4f1f0 spelling fix 2026-06-23 13:04:25 +02:00
76e039d124 fix badges, add release badge 2026-06-23 13:02:43 +02:00
a5ae8f61a9 add anti ai paragraph and badge 2026-06-23 12:57:11 +02:00
9a19473ff1 upd features 2026-06-23 12:35:35 +02:00
T. von Dein
e16f2bfd0b add pager tui for 'api show' (#46) 2026-06-23 11:58:52 +02:00
T. von Dein
6d31d18e0c fix panic if using 'cluser ls -d' (#45) 2026-06-23 10:54:00 +02:00
355e3051b4 updtree 2026-06-22 14:13:15 +02:00
T. von Dein
a81d6b94e9 enhance/clusterstatus (#44)
fixes #35
2026-06-22 14:11:32 +02:00
T. von Dein
9b5ba0775b add ilm management support (#43) 2026-06-22 13:52:02 +02:00
T. von Dein
802217158c feature/add-profiler (#42) 2026-06-18 08:54:00 +02:00
5d345aaa9a fixed another panic 2026-06-17 14:26:28 +02:00
2f029304f3 fix panic and width determination 2026-06-17 14:16:26 +02:00
49bc2c7acb use real terminal width to determine markdown width in 'api show' 2026-06-17 13:33:03 +02:00
T. von Dein
f1ff361c12 add API viewer, mv repl subcommand to api (#41) 2026-06-17 13:11:09 +02:00
072e949147 upd command tree 2026-06-16 13:51:46 +02:00
T. von Dein
ce4a60f8e2 add alias+datastream rollover, and auto rollover on index tpl change (#40)
fixes #37
2026-06-16 13:48:41 +02:00
T. von Dein
97004e0957 add task support (#39)
fixes #34
2026-06-16 08:25:25 +02:00
T. von Dein
d1f3082e5a refactored IndexTemplateModify() (#38) 2026-06-16 08:22:02 +02:00
T. von Dein
d0d1d69791 add index template {ls,sh} (#33) 2026-06-15 15:18:21 +02:00
T. von Dein
ec1ce56b2d add datastream support, refactor custom completion (#32) 2026-06-10 15:43:14 +02:00
925b21823a marshall explain output only in debug mode 2026-06-08 14:11:26 +02:00
9b9b394539 upd tree 2026-06-08 14:05:25 +02:00
T. von Dein
b9eb3e3e2f add search validate+explain (#31) 2026-06-08 14:00:22 +02:00
T. von Dein
f30c837d53 fix search output order (last on bottom) and index ls output partials (#30) 2026-06-08 12:09:01 +02:00
c4143093fd typo 2026-06-04 17:09:23 +02:00
7e9a9f82a7 add intro and feature list 2026-06-04 17:07:34 +02:00
0f48a17536 +todos 2026-06-03 13:13:44 +02:00
T. von Dein
01e0fd024b fix-index-show-completion, add more completions (#29) 2026-06-03 13:07:11 +02:00
T. von Dein
5ecba5abe6 add index fields filter flags (#27) 2026-06-03 10:14:41 +02:00
T. von Dein
18af1b6073 add role diff (#26) 2026-06-03 08:47:54 +02:00
0696095bb0 add roles ls+sh 2026-06-01 13:20:06 +02:00
T. von Dein
504db9835d add sort support to search, add index fields command, fix error messages (#25) 2026-06-01 12:30:41 +02:00
f1877c6903 fix doc add id creation, use rand.Int64() 2026-05-29 14:42:36 +02:00
T. von Dein
2b2094b155 allow multiline json data in repl (#24) 2026-05-29 12:42:02 +02:00
1c72b5ec9a Merge branch 'main' of ssh://codeberg.org/scip/esctl 2026-05-29 11:28:17 +02:00
aeded16b89 catch out-of-bounds error 2026-05-29 11:27:48 +02:00
57bbe680a7 fix http debugger 2026-05-29 11:27:40 +02:00
T. von Dein
130d0b879d add doc delete (#23) 2026-05-29 11:05:57 +02:00
T. von Dein
f7302ea506 more search enhancements (#22) 2026-05-29 10:19:18 +02:00
b2da6f0f29 also fix color status in ccr status 2026-05-27 09:59:56 +02:00
c92f10fb5d show cluster color status directly to make it clearer 2026-05-27 09:25:55 +02:00
501deec538 fix tab completion 2026-05-26 08:01:28 +02:00
T. von Dein
98a577f68a add doc show, enhance search, add jsonpath to search and doc show 2026-05-22 13:50:17 +02:00
T. von Dein
89e2498aad add index alias delete (#20) 2026-05-21 14:00:35 +02:00
T. von Dein
0870ceb3df Refactor output code, add JSON, TSV and YAML output modes, add -o (#19), add version subcommand 2026-05-20 13:06:39 +02:00
32de4863af upd command tree 2026-05-19 13:59:19 +02:00
T. von Dein
58d94a4f56 shards, aliases and filtering (#18) 2026-05-19 13:58:08 +02:00
565d92b491 add command tree 2026-05-18 14:29:46 +02:00
T. von Dein
40e0612bef add explain index allocation 2026-05-18 13:58:08 +02:00
T. von Dein
db50072a7b add support for json color output in repl using jq, if installed (#16) 2026-05-18 13:56:07 +02:00
T. von Dein
b71819f9e8 update deps: cli v3.9.0 and some minor updates (#15) 2026-05-18 12:19:45 +02:00
96 changed files with 174202 additions and 1145 deletions

9
.gitignore vendored
View File

@@ -26,3 +26,12 @@ go.work.sum
.env
esctl
*.sh
*.json
*.log
cpu.profile
t
single

110
.golangci.yml Normal file
View File

@@ -0,0 +1,110 @@
version: "2"
linters:
enable:
- errcheck
- govet
- ineffassign
- staticcheck
- unused
- arangolint
- asasalint
- asciicheck
- bidichk
- bodyclose
- canonicalheader
- clickhouselint
- containedctx
#- contextcheck
- copyloopvar
#- cyclop
- decorder
- dogsled
- dupword
- durationcheck
- embeddedstructfieldcheck
- errchkjson
- errname
#- errorlint
- exhaustive
- exptostd
- fatcontext
- funcorder
#- funlen
- ginkgolinter
- gocheckcompilerdirectives
- gochecknoinits
- gochecksumtype
#- gocognit
- gocritic
#- gocyclo
#- godoclint
- goheader
- gomoddirectives
- gomodguard_v2
- goprintffuncname
#- gosec
- gosmopolitan
- grouper
- iface
- importas
- inamedparam
- interfacebloat
- intrange
- iotamixing
- lll
- loggercheck
- makezero
- misspell
- modernize
- nakedret
#- nestif
- nilerr
- nilnesserr
- nlreturn
- nonamedreturns
- nosprintfhostport
- paralleltest
- perfsprint
- prealloc
#- predeclared
- promlinter
- protogetter
- reassign
- recvcheck
#- revive
- rowserrcheck
- sloglint
- spancheck
- sqlclosecheck
- tagalign
- testableexamples
- testifylint
- testpackage
- thelper
- tparallel
- unconvert
- unparam
- unqueryvet
- usestdlibvars
- usetesting
- varnamelen
- wastedassign
- whitespace
- wsl_v5
- zerologlint
settings:
varnamelen:
ignore-names:
- err
- wg
- mu
- ts
- to
- es
- op
- fd
- id
- fn

View File

@@ -2,7 +2,7 @@ matrix:
platform:
- linux/amd64
goversion:
- 1.25
- 1.26.4
labels:
platform: ${platform}
@@ -21,6 +21,6 @@ steps:
event: [push,manual]
image: golang:${goversion}
commands:
- curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/HEAD/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.5.0
- curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/HEAD/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.12.2
- golangci-lint --version
- golangci-lint run ./...

View File

@@ -12,6 +12,13 @@ The programming language used for this project will always be
# Contributing
Everyone can contribute to this project except you are using "AI"
(LLM) generated content (code, issue text, etc), even if only a part
of it is. This is a human made project. It is being made to have fun
at work and learning coding and APIs. If you want fast results without
the hassle to invest time then please refrain from
contributing.
You can contribute to this project in various ways:
## Open an issue

View File

@@ -23,11 +23,34 @@ UID = root
GID = 0
HAVE_POD := $(shell pod2text -h 2>/dev/null)
BRANCH = $(shell git branch --show-current)
COMMIT = $(shell git rev-parse --short=8 HEAD)
BUILD = $(shell date +%Y.%m.%d.%H%M%S)
GOVERSION = $(shell grep -E "^go" go.mod | cut -d' ' -f2)
APIVERSION = $(shell grep github.com/elastic/go-elasticsearch go.mod | cut -d' ' -f2)
APISPEC = "https://www.elastic.co/docs/api/doc/elasticsearch.json"
all: buildlocal
buildlocal:
CGO_LDFLAGS='-static' go build -tags osusergo,netgo -ldflags "-extldflags=-static" -o $(tool)
# pkg/es/openspec.go:
# echo "package es" > pkg/es/openspec.go
# echo >> pkg/es/openspec.go
# echo "const Openspec = \`" >> pkg/es/openspec.go
# curl -o - $(APISPEC) | sed "s/\`/'/g" >> pkg/es/openspec.go
# echo "\`" >> pkg/es/openspec.go
assets/openapi.json:
curl -o assets/openapi.json $(APISPEC)
buildlocal: assets/openapi.json
CGO_LDFLAGS='-static' go build -tags osusergo,netgo -ldflags "-extldflags=-static" \
-ldflags "-X codeberg.org/scip/esctl/pkg/cfg.BRANCH=$(BRANCH) \
-X codeberg.org/scip/esctl/pkg/cfg.COMMIT=$(COMMIT) \
-X codeberg.org/scip/esctl/pkg/cfg.GOVERSION=$(GOVERSION) \
-X codeberg.org/scip/esctl/pkg/cfg.APIVERSION=$(APIVERSION) \
-X codeberg.org/scip/esctl/pkg/cfg.BUILD=$(BUILD)" \
-o $(tool)
strip $(tool)
install: buildlocal
@@ -37,19 +60,18 @@ install: buildlocal
install -o $(UID) -g $(GID) -m 444 $(tool).1 $(PREFIX)/man/man1/
clean:
rm -rf $(tool) coverage.out testdata t/out
rm -rf $(tool) coverage.out testdata t/out pkg/es/openspec.go
test: clean
mkdir -p t/out
go test ./... $(ARGS)
test: clean buildlocal
make -C t test
testlint: test lint
lint:
golangci-lint run
lint-basic:
golangci-lint run --enable-only errcheck,govet,ineffassign,staticcheck,unused
lint-full:
golangci-lint run --enable-all --exclude-use-default --disable exhaustivestruct,exhaustruct,depguard,interfacer,deadcode,golint,structcheck,scopelint,varcheck,ifshort,maligned,nosnakecase,godot,funlen,gofumpt,cyclop,noctx,gochecknoglobals,paralleltest
lint:
golangci-lint run --show-stats=false
testfuzzy: clean
go test -fuzz ./... $(ARGS)
@@ -83,5 +105,17 @@ show-versions: buildlocal
@echo "### go version used for building:"
@grep -m 1 go go.mod
# lint:
# golangci-lint run -p bugs -p unused
.PHONY: profile
profile: buildlocal
./esctl api ls --profile-file cpu.profile
go tool pprof -text esctl cpu.profile
go tool pprof --http localhost:8888 ./esctl cpu.profile
docker-up:
make -C t up
docker-waitup:
make -C t up
docker-down:
make -C t down

629
README.md
View File

@@ -1,67 +1,65 @@
[![status-badge](https://ci.codeberg.org/api/badges/16999/status.svg)](https://ci.codeberg.org/repos/16999)
[![humanmande](https://img.shields.io/badge/human-made-green)](CONTRIBUTING.md)
[![License](https://img.shields.io/badge/license-GPL-blue.svg)](https://codeberg.org/scip/esctl/blob/master/LICENSE)
[![Go Report Card](https://goreportcard.com/badge/codeberg.org/scip/esctl)](https://goreportcard.com/report/codeberg.org/scip/esctl)
[![Latest Release](https://flat.badgen.net/codeberg/release/scip/esctl)](https://codeberg.org/scip/esctl/releases)
# esctl
Elasticsearch CLI
## Usage
```console
NAME:
esctl - manage elasticsearch from cli
USAGE:
esctl [global options] [command [command options]]
VERSION:
v0.0.4
COMMANDS:
search, / search within an index
index, i manage indicies
snapshot, snap manage snapshots
cluster, c manage cluster[s]
node, snap manage nodes
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--debug, -d enable debugging [$ES_DEBUG]
--config string, -c string config file [$ES_CONFIG]
--cluster string, -C string cluster alias to work with
--help, -h show help
--version, -v print the version
```
Configure `esctl` with environment variables:
- `ES_URI`: elasticsearch uri
- `ES_USER`: username
- `ES_PASS`: password
Or create a config file such as this:
```yaml
clusters:
default:
uri: https://es.foo.bar:9200/
user: elastic
pass: 123456
other:
uri: https://myes.foo:9200/
user: elastic
pass: asdasdasd
```
and specify it with `-c configfile`. You may also put clusters into a
default config file in `~/.config/esctl/config.yaml`. In this case you
can omit `-c ...`.
If you want to work on a specific cluster, specify its name with the
global `-C` option.
## Introduction
FIXME
This is a handy cli tool which interfaces to an elasticsearch cluster
(or two of them if you're using cross cluster replication). It is a
work-in-progress project yet, things might change occasionally. Expect
a stable release once we reach major version 1.0.0.
Features:
- Configuration of cluster credentials using environment vars or
config file. Multiple clusters can be configured. `esctl cluster ls`
shows which one is reachable.
- Shell completion support (bash, zsh and fish). Put this into your
rc: `source <(esctl completion bash)`.
- Cluster settings can be viewed and modified.
- Comprehensive cluster status.
- Cluster reroute support.
- Search: you can search indices using full text or by fields, select
logical condition (OR, AND), use PIT, limit datetime (ES date math
can be used), etc. It is however not yet possible to create
recursive searches like: `(cond1 AND cond2) OR (cond3 OR cond4)`.
- Search using ES|QL language: `esctl searchql`.
- Cross cluster replication (ccr): view, pause, resume, delete
replication. You can also manage follower configuration.
- Index management: manage aliases, create, modify, delete indices,
display field mappings etc. Automatic rollover of aliases supported.
- Index template management: create, modify, delete etc
- Index alias management: create, modify, delete etc
- Node management: only list nodes yet.
- Shard management: only list shards yet.
- Snapshot management: only list snapshots yet.
- ILM management: list, create, delete etc.
- Task management: list and cancel tasks
- Role management: only list roles yet. There's also a `role diff`
subcommand, which is for internal use. It can be used to verify if
role defs in a CSV match the deployed roles.
- API documentation (`api list` and `api show <path>`) with
interactive markdown pager for endpoint documentation.
- Repl: this is an interactive REPL (read eval print loop) towards the
elasticsearch API. You can run API calls on the current selected
cluster w/o the hassle to specify the whole url, credentials etc. It
has line editing and history support. If `jq` is installed output
JSON will be syntax highlighted. A simple internal pager will be
used if output exceeds the terminal height. You can tweak this using
the `$ES_JSON_PAGER` environment variable (I'd recommend [fx](https://fx.wtf/)).
- Doc support. You can put, delete and show docs for an index. Very
handy if you want to play with it. Just create a new index:
`esctl index create foo` and then insert docs into it for search
experiments:
```console
esctl doc add -i foo '{"title":"curry in a hurry", "message":"australian thai"}'
```
## Installation
@@ -113,6 +111,525 @@ make
sudo make install
```
## Configure
Configure `esctl` with environment variables:
- `ES_URI`: elasticsearch uri
- `ES_USER`: username
- `ES_PASS`: password
- `ES_TOKEN`: API token, instead of user+password
Or create a config file such as this:
```yaml
clusters:
foobar:
uri: https://es.foo.bar:9200/
user: elastic
pass: ******
other:
uri: https://myes.foo:9200/
token: ******
```
and specify it with `-c configfile`. You may also put clusters into a
default config file in `~/.config/esctl/config.yaml`. In this case you
can omit `-c ...`.
If you want to work on a specific cluster, you need to make it the
current default one. You can either manually configure it in the
config:
```yaml
clusters:
foobar:
uri: https://es.foo.bar:9200/
user: elastic
pass: 123456
default: true
```
or - if the cluster already exists in the config - issue this command:
```console
esctl cluster switch foobar
```
You may also temporary set a cluster as the current default with the
global `-C` option.
The following rules apply for default cluster selection:
1. If there's just one cluster configured (either via environment
variables or config), this one will be selected.
2. If a cluster have been specified with `-C <name>`, use this.
3. If there is one cluster configured with the `default` flag `true`,
use this one.
Use `esctl cluster ls` to check status and see which one would be
used.
Please be aware, that prior to use a cluster and send API requests to
it, `esctl` checks the connectivity to it first.
## Usage
Once you have configured one or more (and marked one as the default,
see above) elasticsearch clusters, you can start using `esctl`. Since
there are just too many things you can do with it, we'll just outline
a couple of examples here.
To see the status of a cluster:
```console
$ esctl cluster status
DEFAULT STATUS
Cluster Name eck-cluster-dev-s1
ES Status green
ES Version 9.4.2
Is Leader true
Active Shards 447
Active Primary Shards 221
Unassigned Shards 0
Unassigned Primary Shards 0
Pending Tasks 0
Nodes 3
Red Indices 0
Long Running Tasks 2
```
you may also add the `-v` flag to see more:
```console
$ esctl cluster status -v
DEFAULT STATUS
Cluster Name eck-cluster-dev-s1
ES Status green
ES Version 9.4.2
Is Leader true
Active Shards 447
Active Primary Shards 221
Unassigned Shards 0
Unassigned Primary Shards 0
Pending Tasks 0
Nodes 3
Red Indices 0
Long Running Tasks 2
indices 221
Docs 9854777
Total Size 3.4 GB
Total Queries 4210411
Shards Primaries 824642680872
Shards Total 824642680808
Storage 6.9 GB/6.9 GB
JVM Heap Memory 16 GB/39 GB
JVM Threads 458
JVM Version OpenJDK 64-Bit Server VM 26.0.1+8-34
CPUs 18
CPU Usage 0%
Open FDs 1423
```
View cluster settings:
```console
$ esctl cluster settings list
SETTING VALUE
cluster.remote.svct13.mode proxy
cluster.remote.svct13.proxy_address 100.127.161.24:9300
cluster.remote.svct13.proxy_socket_connections 18
cluster.remote.svct13.skip_unavailable false
cluster.routing.allocation.enable all
indices.recovery.max_bytes_per_sec 320mb
```
Settings can be changed, use json path to specify a setting:
```console
$ esctl cluster settings set indices.recovery.max_bytes_per_sec:480mb
```
Searching is pretty easy. First check which indices exist:
```console
$ esctl index ls
NAME SIZE DOCSCOUNT
[..]
foo 249b 0
foo2 31.4kb 14
mock 9.6kb 20
mr-strange 249b 0
silo18 5.2kb 1
```
Like a dataview in kibana you can specify an index pattern for search:
```console
$ esctl search -i foo*
{"id":"1779873299", "score":1.0000, "index":"foo2", "source":{"message":"foo bar"}}{"id":"1779870997", "score":1.0000, "index":"foo2", "source":{"message":"start","time":"27.05.2026T10:55:33+0200","title":"zeitbuchung"}}{"id":"1779870750", "score":1.0000, "index":"foo2", "source":{"message":"pause","time":"27.05.2026T10:33:45+0200","title":"zeitbuchung"}}{"id":"1779870103", "score":1.0000, "index":"foo2", "source":{"message":"punktlich gebucht","time":"27.05.2026T10:20:01+0200","title":"zeitbuchuung"}}{"id":"1779432806", "score":1.0000, "index":"foo2", "source":{"message":"strange compound 2"}}{"id":"1779432750", "score":1.0000, "index":"foo2", "source":{"message":"strange compound 1"}}{"id":"1778600947", "score":1.0000, "index":"foo2", "source":{"id":"d8d8d","user":"scip"}}{"id":"1778600945", "score":1.0000, "index":"foo2", "source":{"id":"d8d8d","user":"scip"}}{"id":"1778600857", "score":1.0000, "index":"foo2", "source":{"id":"d8d8d","user":"scip"}}{"id":"1778601788", "score":1.0000, "index":"foo2", "source":{"id":"new1","message":"live long and prosper"}}{"id":"1778601422", "score":1.0000, "index":"foo2", "source":{"id":"new1","message":"love long and prosper"}}{"id":"1779447242", "score":1.0000, "index":"foo2", "source":{"message":"australian thai","title":"curry in a hurry"}}{"id":"1779447284", "score":1.0000, "index":"foo2", "source":{"message":"german killer food","title":"wuerger burger"}}{"id":"1779364394", "score":1.0000, "index":"foo2", "source":{"id":"1","message":"non-dead human looks curious at TargetOne"}}
```
Oops, that looks gibberish. It's because `esctl` just prints the raw
JSON search results. There are multiple ways to post-process them:
You can use `jq` (here, we're using `-l 1` to limit the output to 1 document):
```console
$ esctl search -i foo* -l 1 | jq
{
"id": "1779364394",
"score": 1.0000,
"index": "foo2",
"source": {
"id": "1",
"message": "non-dead human looks curious at TargetOne"
}
}
```
However, you might as well use internal filtering using jsonpaths:
```console
$ esctl search -i foo* -l 1 -p source.message
non-dead human looks curious at TargetOne
```
To get more help about internal jsonpath filtering, run `esctl
help-jsonpath`.
A full text search looks like this:
```console
$ esctl search -i foo* thai | jq
{
"id": "1779447242",
"score": 2.3842,
"index": "foo2",
"source": {
"message": "australian thai",
"title": "curry in a hurry"
}
}
```
Instead of a costly full text you might as well specify the field to
search within:
```console
$ esctl search -i foo* message=thai | jq
{
"id": "1779447242",
"score": 2.3842,
"index": "foo2",
"source": {
"message": "australian thai",
"title": "curry in a hurry"
}
}
```
Of course you can search for multiple fields (or multiple terms in
full text search). By default search terms are applied using logical
AND, which you can change with the option `--or`.
You can also change the sorting (`--sort-by`) and the time range
(`--timerange`).
To further narrow down search results you can add filters, which must
match literally:
```console
$ esctl search -i foo* -F title=zeitbuchung message=pause | jq
{
"id": "1779870750",
"score": 2.8426,
"index": "foo2",
"source": {
"message": "pause",
"time": "27.05.2026T10:33:45+0200",
"title": "zeitbuchung"
}
}
```
You can also search using [ES|QL](https://www.elastic.co/docs/reference/query-languages/esql/esql-getting-started):
```console
$ esctl searchql "from hyperdrive | sort @timestamp | limit 5"
@TIMESTAMP MESSAGE TAG
2026-06-24T08:24:56.000Z arosu loop
2026-06-24T08:24:58.000Z hami loop
2026-06-24T08:24:59.000Z ishininu loop
2026-06-24T08:25:00.000Z uyomoruron loop
2026-06-24T08:25:02.000Z ishimime loop
```
There are several output modes (json, yaml, csv), to get esql output as CSV:
```console
$ esctl searchql "from hyperdrive | sort @timestamp | limit 5" -o csv
@timestamp,message,tag
2026-06-24T08:24:56.000Z,arosu,loop
2026-06-24T08:24:58.000Z,hami,loop
2026-06-24T08:24:59.000Z,ishininu,loop
2026-06-24T08:25:00.000Z,uyomoruron,loop
2026-06-24T08:25:02.000Z,ishimime,loop
```
To check which field mappings are available for an index:
```console
$ esctl index show foo2
INDEX-PROPERTY VALUE
name foo2
replicas 1
shards 1
created 2026-05-12 17:47:37
uuid XmRtlh_JSreIzTB7jJNNJA
fields id,message,time,title,user
```
Now, let's create a new index:
```console
$ esctl index create hyperdrive @timestamp:date message:keyword tag:text
esctl index show hyperdrive
INDEX-PROPERTY VALUE
name hyperdrive
replicas 1
shards 1
created 2026-06-24 10:16:23
uuid gYfUj8TiR6aMB_BHmMYr3Q
fields @timestamp,message,tag
```
Once we have an index, we can insert documents into it (you'll need
[gfn](https://codeberg.org/scip/gfn) for this):
```console
$ for name in $(gfn JapaneseNamesDiverse -n 20); do \
esctl doc add -i hyperdrive \
"{\"message\":\"$name\",\"tag\":\"loop\",\"@timestamp\":\"$(date
--iso-8601=second)\"}"; \
sleep 1; \
done
3644770736267480972
1249004675657072369
5499556813583921546
947619563748335783
6804953496467561176
1716813071323879201
463762522269463473
4674152788118344945
7741776772191747414
8531321287239151007
3335734000533108391
7020487792409206103
4926995290680601261
374509677635815298
49895974689085266
7603943644788310
2102931816420108217
696238814931581311
2463257155346426021
596556586001412147
```
Now, let's search them:
```console
$ esctl search -i hyperdrive -l 2 | jq
{
"id": "2463257155346426021",
"score": 0.0000,
"index": "hyperdrive",
"source": {
"@timestamp": "2026-06-24T10:25:19+02:00",
"message": "inenen",
"tag": "loop"
}
}
{
"id": "596556586001412147",
"score": 0.0000,
"index": "hyperdrive",
"source": {
"@timestamp": "2026-06-24T10:25:20+02:00",
"message": "ominuhen",
"tag": "loop"
}
}
```
There are also some uniq features which are not directly available via
the ES API or GUI. Here's one example: say you have a number of
indices with ILM policies each. At some day there were a surge in
incoming data in some of them and you want to know, when the excess
data will be rolled over to cold storage:
```console
$ esctl ilm forecast sh -f warm -w 3d
1.1 TB bytes of data in warm phase will be rolled within 72h0m0s to the next phase
```
You may also look at a detailed ilm forecast list:
```console
$ esctl ilm forecast ls -f warm -w 3d
INDEX CURRENT-SIZE CURRENT-AGE VIRTUAL-AGE MIN-AGE MIN-SIZE CURRENT-PHASE NEXT-PHASE
foobar-n1-p01-elastic-000126 12 GB 2d:8h:12m 3d:5h:16m 7d 25 GB hot warm
foobar-n1-p01-misc-000070 16 GB 5d:16h:31m 4d:10h:31m 7d 25 GB hot warm
foobar-n1-q01-elastic-000122 23 GB 5d:0h:11m 6d:10h:33m 7d 25 GB hot warm
foobar-n1-q01-misc-000072 18 GB 6d:15h:21m 4d:21h:36m 7d 25 GB hot warm
delaware-f1-p01-kafka-000023 1.3 MB 15h:12m 0s 7d 25 GB hot warm
delaware-f1-p01-misc-000024 17 GB 6d:16h:1m 4d:16h:33m 7d 25 GB hot warm
delaware-f1-q01-kafka-000023 1.5 MB 15h:12m 0s 7d 25 GB hot warm
delaware-f1-q01-misc-000024 16 GB 6d:15h:31m 4d:13h:52m 7d 25 GB hot warm
delaware-n1-p01-elastic-000417 16 GB 12h:42m 4d:10h:31m 7d 25 GB hot warm
```
So you can see, which index will be rolled over when. Note the
**VIRTUAL-AGE** field however: it is calculated from the current
storage usage of the index in relation to rollover max shard size. So
you can see, when an index will be rolled over either because it aged
out or because its storage exceeded the limit.
---
Please note, that `esctl` is still in its early stages and things are
changing heavily every now and then. New commands are being added
constantly as well.
### Command tree:
```console
api - api access and documentation
list - list index of API calls
show - show an API doc
repl - interactive API repl
ccr - manage cross cluster replication
status - cross cluster replication status (yaml config with 2 clusters required)
pause - pause shard allocation
resume - resume shard allocation
follower - manage ccr follower indices
show - show ccr follower index details
add - add ccr follower index
delete - delete ccr follower index
unfollow - unfollow ccr follower index
pause - pause ccr index to follow
resume - resume ccr index to follow
renew - renew ccr follower index
info - show ccr remote info
cluster - manage cluster[s]
status - show cluster status
switch - set current elasticsearch cluster
list - list configured clusters
settings - cluster settings management
list - show cluster settings
set - set|update cluster settings
reroute - manually change the allocation of individual shards in the cluster.
move - move shard to another node
allocate-replica - allocate-replica replica to another node
cancel - cancel a reroute operation
allocate-empty-primary - allocate an empty primary shard to a node
allocate-stale-primary - allocate a stale primary shard to a node
datastream - manage data streams
list - list indices
show - show details about an data stream
create - create a new data stream
delete - delete a data stream
rollover - roll over a data stream
doc - manage documents
add - add JSON document index
show - show a JSON document
delete - delete JSON document[s] from index[es]
ilm - manage index lifecycle
retry - retry applying an ILM profile to an index
status - get the current index lifecycle management status
list - list index lifecycle policies
show - show details about an index lifecycle policy
create - create a new lifecycle policy
update - update an lifecycle policy
forecast - calculate index phase movements
list - list index rollover config
show - show rollover forecast over all indices
explain - explain ilm condition of an index
index - manage indices
list - list indices
show - show details about an index
create - create a new index
update - update an index
delete - delete an index
close - close an index
fields - show info about field capabilities
ilm - show ilm status
du - show index disk usage
copy - copy (reindex) documents from one index to another
alias - manage index aliases
create - create an index alias
list - list index aliases
delete - delete an index alias
rollover - roll over an index alias
template - manage index templates
list - list index templates
show - show details about an index template
create - create a new index template
update - update a new index template
delete - delete an index template
license - manage cluster license
show - show details about the cluster license
node - manage nodes
list - list nodes
show - show details about a node
clients - show node http clients
usage - show node usage stats
role - manage roles
list - list roles
show - show details about a role
diff - show differences between roles and CSV baseline
search - search within an index
searchql - search using ES/QL language
shard - manage shards
list - list shards
show - show details about a shard
allocation - explain shard allocation
snapshot - manage snapshots
list - list snapshots
show - show details about a snapshot
task - manage tasks
list - list tasks
cancel - cancel running task
version - show esctl version information
debug - developer only
help-jsonpath - show jsonpath help
help-usage - show overview of all available commands
help-esql - show esql help
```
# Development
## To test completion
Add the flag `--generate-shell-completion` to any command, e.g.:
```console
./esctl role show --generate-shell-completion
machine_learning_admin
rollup_admin
editor
reporting_user
snapshot_user
fcn_admin
machine_learning_user
kibana_system
beats_admin
kibana_user
fcns_space
transport_client
transform_user
[..]
```
# Report bugs
[Please open an issue](https://codeberg.org/scip/esctl/issues). Thanks!

View File

@@ -1,7 +1,7 @@
- [Go client docs](https://www.elastic.co/docs/reference/elasticsearch/clients/go/typed-api)
- [ES API docs](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get)
- Fix index names custom completion
- index show: add more details, see screenshots
- add shard explain, aka:
get /_cluster/allocation/explain {"index":"yourindex", "primary": true, "shard":0}
- add datastream support:
https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-data-stream

53
assets/assets.go Normal file
View File

@@ -0,0 +1,53 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package assets
import (
"embed"
"github.com/go-openapi/loads"
"github.com/go-openapi/swag/loading"
)
//go:embed *.json *.md
var AssetFS embed.FS
var OpenAPI *loads.Document
var EsQlCheatSheet string
func LoadAssetOpenApi() {
doc, err := loads.Spec(
"openapi.json",
loads.WithLoadingOptions(
loading.WithFS(AssetFS),
))
if err != nil {
panic(err)
}
OpenAPI = doc
}
func LoadEsql() {
md, err := AssetFS.ReadFile("esql.md")
if err != nil {
panic(err)
}
EsQlCheatSheet = string(md)
}

951
assets/esql.md Normal file
View File

@@ -0,0 +1,951 @@
<!-- courtesy https://github.com/linkan-per/ES-QL-Cheat-Sheet by |linkan-per -->
# ES|QL (Elasticsearch Query Language) Cheat Sheet
## Table of Contents
1. [Introduction](#introduction)
2. [Basic Query Structure](#basic-query-structure)
3. [Source Commands](#source-commands)
4. [Processing Commands](#processing-commands)
5. [Data Selection & Filtering](#data-selection--filtering)
6. [Aggregations & Statistics](#aggregations--statistics)
7. [String Functions](#string-functions)
8. [Mathematical Functions](#mathematical-functions)
9. [Date/Time Functions](#datetime-functions)
10. [Type Conversion Functions](#type-conversion-functions)
11. [Conditional Functions](#conditional-functions)
12. [Array Functions](#array-functions)
13. [Join Operations (LOOKUP)](#join-operations-lookup)
14. [Sorting & Limiting](#sorting--limiting)
15. [Grouping & Aggregating](#grouping--aggregating)
16. [Advanced Patterns](#advanced-patterns)
---
## Introduction
ES|QL is Elasticsearch's new query language designed for data exploration, analysis, and transformation. It uses a pipe (`|`) syntax to chain commands together.
**Basic Syntax:**
```
FROM <data-source>
| <processing-command>
| <processing-command>
| ...
```
---
## Basic Query Structure
### Simple Query
```esql
FROM logs-*
| LIMIT 10
```
### Query with Multiple Commands
```esql
FROM employees
| WHERE department == "Engineering"
| KEEP name, salary, hire_date
| SORT salary DESC
| LIMIT 5
```
---
## Source Commands
### FROM - Specify Data Source
```esql
// From a single index
FROM logs-2024
// From multiple indices with wildcard
FROM logs-*, metrics-*
// From specific indices
FROM index1, index2, index3
// With metadata
FROM logs-* METADATA _id, _index
```
### ROW - Generate Inline Data
```esql
// Create a single row
ROW name = "John", age = 30, city = "NYC"
// Multiple rows
ROW a = 1, b = "x"
| EVAL c = a * 10
```
---
## Processing Commands
### KEEP - Select Specific Fields
```esql
FROM employees
| KEEP name, department, salary
// Keep with pattern
FROM logs-*
| KEEP @timestamp, message, host.*
```
### DROP - Remove Specific Fields
```esql
FROM employees
| DROP password, ssn, internal_notes
// Drop with pattern
FROM logs-*
| DROP *.keyword
```
### RENAME - Rename Fields
```esql
FROM employees
| RENAME emp_name AS name, emp_dept AS department
// Multiple renames
FROM logs-*
| RENAME source.ip AS src_ip, destination.ip AS dst_ip
```
---
## Data Selection & Filtering
### WHERE - Filter Rows
```esql
// Equality
FROM employees
| WHERE department == "Sales"
// Comparison operators
FROM products
| WHERE price > 100 AND stock < 50
// IS NULL / IS NOT NULL
FROM logs-*
| WHERE error_code IS NOT NULL
// IN operator
FROM employees
| WHERE department IN ("Sales", "Marketing", "HR")
// LIKE operator (wildcards)
FROM logs-*
| WHERE message LIKE "*error*"
// RLIKE operator (regex)
FROM logs-*
| WHERE message RLIKE "error|exception|failure"
// NOT operator
FROM employees
| WHERE NOT department == "IT"
// Multiple conditions
FROM orders
| WHERE status == "completed"
AND total_amount > 1000
AND order_date >= "2024-01-01"
```
---
## Aggregations & Statistics
### STATS - Aggregate Functions
#### COUNT
```esql
// Count all rows
FROM logs-*
| STATS count = COUNT()
// Count distinct
FROM employees
| STATS unique_departments = COUNT_DISTINCT(department)
// Count by group
FROM logs-*
| STATS event_count = COUNT() BY log_level
```
#### SUM, AVG, MIN, MAX
```esql
FROM sales
| STATS
total_revenue = SUM(amount),
avg_sale = AVG(amount),
min_sale = MIN(amount),
max_sale = MAX(amount)
// With grouping
FROM sales
| STATS
total = SUM(amount),
average = AVG(amount)
BY product_category
```
#### MEDIAN, PERCENTILE
```esql
FROM response_times
| STATS
median_time = MEDIAN(duration),
p95 = PERCENTILE(duration, 95),
p99 = PERCENTILE(duration, 99)
```
#### Multiple Aggregations
```esql
FROM orders
| STATS
order_count = COUNT(),
total_revenue = SUM(amount),
avg_order_value = AVG(amount),
unique_customers = COUNT_DISTINCT(customer_id)
BY region, product_category
```
---
## String Functions
### CONCAT - Concatenate Strings
```esql
FROM employees
| EVAL full_name = CONCAT(first_name, " ", last_name)
// With separator
FROM logs-*
| EVAL log_info = CONCAT(level, ": ", message)
```
### SUBSTRING - Extract Substring
```esql
FROM employees
| EVAL first_initial = SUBSTRING(first_name, 0, 1)
// Extract with length
FROM products
| EVAL short_code = SUBSTRING(product_id, 0, 5)
```
### LENGTH - String Length
```esql
FROM messages
| EVAL message_length = LENGTH(message)
| WHERE message_length > 100
```
### TRIM, LTRIM, RTRIM - Remove Whitespace
```esql
FROM user_input
| EVAL cleaned = TRIM(input_field)
| EVAL left_trimmed = LTRIM(input_field)
| EVAL right_trimmed = RTRIM(input_field)
```
### UPPER, LOWER - Case Conversion
```esql
FROM employees
| EVAL name_upper = UPPER(name)
| EVAL email_lower = LOWER(email)
```
### REPLACE - Replace String
```esql
FROM logs-*
| EVAL cleaned_message = REPLACE(message, "ERROR", "Warning")
```
### SPLIT - Split String into Array
```esql
FROM logs-*
| EVAL tags_array = SPLIT(tags, ",")
```
### STARTS_WITH, ENDS_WITH
```esql
FROM files
| WHERE STARTS_WITH(filename, "log_")
| WHERE ENDS_WITH(filename, ".txt")
```
---
## Mathematical Functions
### Basic Operations
```esql
FROM sales
| EVAL
total = price * quantity,
discount_price = price * 0.9,
tax = price * 0.08
// Multiple operations
FROM metrics
| EVAL
sum_val = field1 + field2,
diff_val = field1 - field2,
product = field1 * field2,
ratio = field1 / field2,
remainder = field1 % field2
```
### ABS - Absolute Value
```esql
FROM transactions
| EVAL abs_amount = ABS(transaction_amount)
```
### ROUND, FLOOR, CEIL
```esql
FROM measurements
| EVAL
rounded = ROUND(value, 2),
floored = FLOOR(value),
ceiled = CEIL(value)
```
### POW - Power
```esql
FROM data
| EVAL squared = POW(value, 2)
| EVAL cubed = POW(value, 3)
```
### SQRT - Square Root
```esql
FROM measurements
| EVAL sqrt_value = SQRT(value)
```
### LOG, LOG10
```esql
FROM data
| EVAL
natural_log = LOG(value),
log_base_10 = LOG10(value)
```
### GREATEST, LEAST
```esql
FROM comparisons
| EVAL
max_val = GREATEST(val1, val2, val3),
min_val = LEAST(val1, val2, val3)
```
---
## Date/Time Functions
### NOW - Current Timestamp
```esql
FROM logs-*
| EVAL current_time = NOW()
```
### DATE_EXTRACT - Extract Date Parts
```esql
FROM events
| EVAL
year = DATE_EXTRACT("year", @timestamp),
month = DATE_EXTRACT("month", @timestamp),
day = DATE_EXTRACT("day", @timestamp),
hour = DATE_EXTRACT("hour", @timestamp),
minute = DATE_EXTRACT("minute", @timestamp),
day_of_week = DATE_EXTRACT("day_of_week", @timestamp)
```
### DATE_FORMAT - Format Date
```esql
FROM events
| EVAL formatted_date = DATE_FORMAT("yyyy-MM-dd", @timestamp)
| EVAL custom_format = DATE_FORMAT("MMM dd, yyyy HH:mm", @timestamp)
```
### DATE_TRUNC - Truncate Date
```esql
FROM logs-*
| EVAL
hour_bucket = DATE_TRUNC("hour", @timestamp),
day_bucket = DATE_TRUNC("day", @timestamp),
month_bucket = DATE_TRUNC("month", @timestamp)
```
### DATE_DIFF - Date Difference
```esql
FROM orders
| EVAL days_since_order = DATE_DIFF("days", order_date, NOW())
| EVAL hours_to_delivery = DATE_DIFF("hours", order_date, delivery_date)
```
### DATE_PARSE - Parse String to Date
```esql
FROM data
| EVAL parsed_date = DATE_PARSE("yyyy-MM-dd", date_string)
```
---
## Type Conversion Functions
### TO_STRING - Convert to String
```esql
FROM data
| EVAL id_string = TO_STRING(id)
| EVAL amount_string = TO_STRING(amount)
```
### TO_INTEGER, TO_LONG - Convert to Integer
```esql
FROM data
| EVAL age_int = TO_INTEGER(age_string)
| EVAL id_long = TO_LONG(id_string)
```
### TO_DOUBLE - Convert to Double
```esql
FROM data
| EVAL price_double = TO_DOUBLE(price_string)
```
### TO_BOOLEAN - Convert to Boolean
```esql
FROM data
| EVAL is_active = TO_BOOLEAN(active_string)
```
### TO_DATETIME - Convert to DateTime
```esql
FROM data
| EVAL timestamp = TO_DATETIME(date_string)
```
### TO_IP - Convert to IP Address
```esql
FROM logs-*
| EVAL ip_address = TO_IP(ip_string)
```
---
## Conditional Functions
### CASE - Conditional Logic
```esql
FROM employees
| EVAL salary_grade = CASE(
salary < 50000, "Entry",
salary < 80000, "Mid",
salary < 120000, "Senior",
"Executive"
)
// With multiple conditions
FROM orders
| EVAL order_status = CASE(
status == "pending" AND days_old > 7, "Overdue",
status == "pending", "Processing",
status == "shipped", "In Transit",
status == "delivered", "Completed",
"Unknown"
)
```
### COALESCE - Return First Non-Null Value
```esql
FROM data
| EVAL display_name = COALESCE(nickname, first_name, username, "Unknown")
```
### IF - Simple Conditional
```esql
FROM products
| EVAL stock_status =
CASE(stock > 0, "Available", "Out of Stock")
// Nested conditions
FROM employees
| EVAL bonus = CASE(
performance_rating >= 4.5, salary * 0.15,
performance_rating >= 3.5, salary * 0.10,
performance_rating >= 2.5, salary * 0.05,
0
)
```
---
## Array Functions
### MV_COUNT - Count Array Elements
```esql
FROM logs-*
| EVAL tag_count = MV_COUNT(tags)
| WHERE tag_count > 3
```
### MV_AVG, MV_SUM, MV_MIN, MV_MAX - Array Aggregations
```esql
FROM metrics
| EVAL
avg_value = MV_AVG(values),
total = MV_SUM(values),
min_value = MV_MIN(values),
max_value = MV_MAX(values)
```
### MV_CONCAT - Concatenate Array Elements
```esql
FROM logs-*
| EVAL all_tags = MV_CONCAT(tags, ", ")
```
### MV_DEDUPE - Remove Duplicates from Array
```esql
FROM data
| EVAL unique_values = MV_DEDUPE(values)
```
### MV_FIRST, MV_LAST - Get First/Last Element
```esql
FROM logs-*
| EVAL first_tag = MV_FIRST(tags)
| EVAL last_tag = MV_LAST(tags)
```
### MV_SLICE - Extract Array Slice
```esql
FROM data
| EVAL first_three = MV_SLICE(values, 0, 3)
```
---
## Join Operations (LOOKUP)
ES|QL uses ENRICH (similar to LOOKUP/JOIN) to join data from enrich policies.
### Prerequisites: Create Enrich Policy
First, create an enrich policy in Kibana Dev Tools:
```json
PUT /_enrich/policy/user_lookup
{
"match": {
"indices": "users",
"match_field": "user_id",
"enrich_fields": ["username", "email", "department"]
}
}
POST /_enrich/policy/user_lookup/_execute
```
### ENRICH - Join/Lookup Data
```esql
FROM logs-*
| ENRICH user_lookup ON user_id
| KEEP @timestamp, user_id, username, email, message
// With field renaming
FROM transactions
| ENRICH product_lookup ON product_id WITH product_name, category, price
| KEEP transaction_id, product_name, category, quantity, price
// Multiple enrichments
FROM orders
| ENRICH customer_lookup ON customer_id WITH customer_name, customer_tier
| ENRICH product_lookup ON product_id WITH product_name, product_category
| KEEP order_id, customer_name, product_name, order_amount
```
### Complex Join Example
```esql
FROM orders
| ENRICH customer_lookup ON customer_id
WITH customer_name, customer_email, customer_segment
| ENRICH product_lookup ON product_id
WITH product_name, product_category, product_price
| EVAL total_price = quantity * product_price
| WHERE customer_segment == "Premium"
| STATS
total_orders = COUNT(),
total_revenue = SUM(total_price)
BY customer_name, product_category
| SORT total_revenue DESC
```
---
## Sorting & Limiting
### SORT - Order Results
```esql
// Ascending order (default)
FROM employees
| SORT salary
// Descending order
FROM employees
| SORT salary DESC
// Multiple fields
FROM employees
| SORT department ASC, salary DESC
// With nulls first/last
FROM data
| SORT value DESC NULLS FIRST
```
### LIMIT - Limit Results
```esql
// Get first 10 rows
FROM logs-*
| LIMIT 10
// Top 5 highest salaries
FROM employees
| SORT salary DESC
| LIMIT 5
// Pagination (skip and limit)
FROM products
| SORT price
| LIMIT 20 // Results 0-19
```
### HEAD - Get First N Rows (Alias for LIMIT)
```esql
FROM logs-*
| HEAD 100
```
---
## Grouping & Aggregating
### GROUP BY with STATS
```esql
// Single field grouping
FROM sales
| STATS total_sales = SUM(amount) BY region
// Multiple field grouping
FROM orders
| STATS
order_count = COUNT(),
total_revenue = SUM(amount)
BY region, product_category, sales_rep
// Time-based grouping
FROM logs-*
| EVAL hour = DATE_TRUNC("hour", @timestamp)
| STATS event_count = COUNT() BY hour, log_level
| SORT hour DESC
```
### Complex Aggregation Example
```esql
FROM sales_data
| WHERE order_date >= "2024-01-01"
| EVAL month = DATE_TRUNC("month", order_date)
| STATS
total_orders = COUNT(),
total_revenue = SUM(amount),
avg_order_value = AVG(amount),
unique_customers = COUNT_DISTINCT(customer_id),
max_order = MAX(amount),
min_order = MIN(amount)
BY month, region, product_category
| EVAL revenue_per_customer = total_revenue / unique_customers
| WHERE total_orders > 100
| SORT month DESC, total_revenue DESC
| LIMIT 50
```
---
## Advanced Patterns
### Window Functions Pattern
```esql
// Running total by group
FROM sales
| SORT date
| STATS
daily_sales = SUM(amount),
order_count = COUNT()
BY date, region
| SORT region, date
```
### Pivoting Data
```esql
// Count by status and priority
FROM tickets
| STATS ticket_count = COUNT() BY status, priority
| SORT status, priority
```
### Finding Duplicates
```esql
FROM users
| STATS count = COUNT() BY email
| WHERE count > 1
| SORT count DESC
```
### Time Series Analysis
```esql
FROM metrics-*
| EVAL
hour = DATE_TRUNC("hour", @timestamp),
day = DATE_EXTRACT("day", @timestamp)
| STATS
avg_cpu = AVG(cpu_percent),
max_cpu = MAX(cpu_percent),
avg_memory = AVG(memory_percent)
BY hour, host
| WHERE avg_cpu > 80
| SORT hour DESC
```
### Percentage Calculations
```esql
FROM sales
| STATS
total_sales = SUM(amount),
count = COUNT()
BY product_category
| EVAL percentage = ROUND(total_sales / SUM(total_sales) * 100, 2)
| SORT percentage DESC
```
### Top N per Group
```esql
// Top 3 products per category by sales
FROM sales
| STATS total_sales = SUM(amount) BY product_category, product_name
| SORT product_category, total_sales DESC
// Note: ES|QL doesn't have native PARTITION BY,
// so you may need to process this in multiple queries or use aggregations
```
### Data Cleaning
```esql
FROM raw_data
| EVAL
// Clean whitespace
cleaned_name = TRIM(name),
// Standardize case
email_lower = LOWER(email),
// Replace values
status = REPLACE(status, "N/A", "Unknown"),
// Handle nulls
age = COALESCE(age, 0),
// Validate ranges
valid_age = CASE(age < 0 OR age > 150, NULL, age)
| WHERE cleaned_name IS NOT NULL
| DROP name, email
| RENAME cleaned_name AS name, email_lower AS email
```
### Cohort Analysis
```esql
FROM user_events
| EVAL
signup_month = DATE_TRUNC("month", signup_date),
event_month = DATE_TRUNC("month", event_date)
| STATS
active_users = COUNT_DISTINCT(user_id)
BY signup_month, event_month
| SORT signup_month, event_month
```
### Anomaly Detection Pattern
```esql
FROM metrics-*
| EVAL hour = DATE_TRUNC("hour", @timestamp)
| STATS
avg_value = AVG(value),
stddev = SQRT(AVG(POW(value - AVG(value), 2)))
BY hour
| EVAL
upper_bound = avg_value + (2 * stddev),
lower_bound = avg_value - (2 * stddev)
```
---
## Complete Real-World Examples
### Example 1: User Activity Dashboard
```esql
FROM user_logs-*
| WHERE @timestamp >= NOW() - 7 days
| ENRICH user_lookup ON user_id WITH username, user_tier
| EVAL day = DATE_TRUNC("day", @timestamp)
| STATS
daily_active_users = COUNT_DISTINCT(user_id),
total_sessions = COUNT(),
avg_session_duration = AVG(session_duration)
BY day, user_tier
| EVAL avg_duration_minutes = ROUND(avg_session_duration / 60, 2)
| SORT day DESC, user_tier
```
### Example 2: E-commerce Sales Report
```esql
FROM orders
| WHERE order_date >= "2024-01-01"
| ENRICH customer_lookup ON customer_id
WITH customer_name, customer_segment
| ENRICH product_lookup ON product_id
WITH product_name, product_category, cost_price
| EVAL
profit = (price - cost_price) * quantity,
month = DATE_TRUNC("month", order_date)
| STATS
total_orders = COUNT(),
total_revenue = SUM(price * quantity),
total_profit = SUM(profit),
avg_order_value = AVG(price * quantity),
unique_customers = COUNT_DISTINCT(customer_id)
BY month, product_category, customer_segment
| EVAL profit_margin = ROUND(total_profit / total_revenue * 100, 2)
| WHERE total_revenue > 10000
| SORT month DESC, total_revenue DESC
| LIMIT 100
```
### Example 3: Security Log Analysis
```esql
FROM security-logs-*
| WHERE @timestamp >= NOW() - 24 hours
| WHERE event_type IN ("login_failed", "suspicious_activity")
| EVAL hour = DATE_TRUNC("hour", @timestamp)
| STATS
event_count = COUNT(),
unique_ips = COUNT_DISTINCT(source_ip),
unique_users = COUNT_DISTINCT(username)
BY hour, event_type, country
| WHERE event_count > 100
| SORT hour DESC, event_count DESC
```
### Example 4: Application Performance Monitoring
```esql
FROM apm-*
| WHERE @timestamp >= NOW() - 1 hour
| EVAL
response_category = CASE(
response_time < 100, "Fast",
response_time < 500, "Medium",
response_time < 1000, "Slow",
"Very Slow"
),
minute = DATE_TRUNC("minute", @timestamp)
| STATS
request_count = COUNT(),
avg_response = AVG(response_time),
p95_response = PERCENTILE(response_time, 95),
p99_response = PERCENTILE(response_time, 99),
error_count = COUNT() WHERE status_code >= 400
BY minute, endpoint, response_category
| EVAL error_rate = ROUND(error_count / request_count * 100, 2)
| WHERE request_count > 10
| SORT minute DESC, avg_response DESC
```
---
## Tips & Best Practices
1. **Use KEEP instead of SELECT** - More explicit about which fields to retain
2. **Filter early with WHERE** - Reduce data processing by filtering before aggregations
3. **Use DATE_TRUNC for time bucketing** - Essential for time series analysis
4. **Leverage ENRICH for joins** - Pre-create enrich policies for frequently joined data
5. **Use EVAL for calculated fields** - Create derived fields before aggregation
6. **Combine multiple conditions in WHERE** - More efficient than multiple WHERE clauses
7. **Use STATS with BY for grouping** - Replaces traditional GROUP BY
8. **Sort after aggregation** - More efficient than sorting before
9. **Use LIMIT to control output size** - Especially important for large datasets
10. **Use metadata fields when needed** - Access _id, _index with METADATA keyword
---
## Common Patterns Cheat Sheet
```esql
// Count by field
FROM index | STATS count = COUNT() BY field
// Top N
FROM index | STATS value = SUM(amount) BY category | SORT value DESC | LIMIT 10
// Time series
FROM index | EVAL bucket = DATE_TRUNC("hour", @timestamp) | STATS count = COUNT() BY bucket
// Percentage of total
FROM index | STATS total = SUM(amount) BY category | EVAL pct = total / SUM(total) * 100
// Filter nulls
FROM index | WHERE field IS NOT NULL
// String matching
FROM index | WHERE field LIKE "*pattern*"
// Date range
FROM index | WHERE @timestamp >= NOW() - 7 days
// Multiple aggregations
FROM index | STATS count = COUNT(), sum = SUM(val), avg = AVG(val) BY group
// Conditional aggregation
FROM index | STATS error_count = COUNT() WHERE status == "error" BY service
```
---
## Comparison with Traditional SQL
| SQL | ES|QL |
|-----|-------|
| SELECT * | FROM index |
| SELECT field1, field2 | FROM index \| KEEP field1, field2 |
| WHERE condition | WHERE condition (same) |
| GROUP BY field | STATS ... BY field |
| ORDER BY field | SORT field |
| LIMIT 10 | LIMIT 10 (same) |
| COUNT(*) | STATS count = COUNT() |
| SUM(field) | STATS total = SUM(field) |
| AVG(field) | STATS avg = AVG(field) |
| JOIN | ENRICH (using enrich policies) |
| CASE WHEN | CASE(...) |
| CONCAT(a, b) | CONCAT(a, b) (same) |
---
## Resources
- Official ES|QL Documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/esql.html
- ES|QL Functions Reference: https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-functions.html
- Enrich Processor: https://www.elastic.co/guide/en/elasticsearch/reference/current/enrich-processor.html
---
*Last Updated: 2024*
*ES|QL is actively evolving - check official documentation for latest features*

160985
assets/openapi.json Normal file

File diff suppressed because one or more lines are too long

106
cmd/api.go Normal file
View File

@@ -0,0 +1,106 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func Api(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "api",
Usage: "api access and documentation",
Commands: []*cli.Command{
ApiList(conf),
ApiShow(conf),
ApiRepl(conf),
},
}
}
func ApiList(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "list",
Aliases: []string{"ls"},
Usage: "list index of API calls",
UsageText: "list [<pattern>]",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "tag",
Usage: "list only operations matching <tag>",
Destination: &conf.Tag,
Aliases: []string{"t"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.ApiList(conf, cmd.Args().Get(0))
},
}
}
func ApiShow(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "show",
Aliases: []string{"sh"},
Usage: "show an API doc",
UsageText: "show <pattern> [<verb>]",
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.ApiShow(conf, cmd.Args().Get(0), cmd.Args().Get(1))
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Capi)
},
}
}
func ApiRepl(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "repl",
Aliases: []string{"shell"},
Usage: "interactive API repl",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "pager",
Usage: "external viewer program (default:internal)",
Destination: &conf.Pager,
Aliases: []string{"p"},
Sources: cli.EnvVars("PAGER", "ES_JSON_PAGER"),
},
&cli.BoolFlag{
Name: "human-readable-cat",
Usage: "enable human readable /_cat output",
Destination: &conf.HumanCat,
Aliases: []string{"H"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.ApiRepl(conf)
},
}
}

View File

@@ -44,19 +44,24 @@ func Ccr(conf *cfg.Config) *cli.Command {
func CcrStatus(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "status",
Aliases: []string{"st"},
Usage: "cross cluster replication status (yaml config with 2 clusters required)",
Name: "status",
Aliases: []string{"st"},
Usage: "cross cluster replication status (yaml config with 2 clusters required)",
UsageText: "status <leader> <follower>",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "exclude",
Usage: "regexp of indicies to exclude",
Usage: "regexp of indices to exclude",
Destination: &conf.Exclude,
Aliases: []string{"e"},
},
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Ccluster)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
leader := cmd.Args().Get(0)
follower := cmd.Args().Get(1)
@@ -109,6 +114,10 @@ func CcrRemoteInfo(conf *cfg.Config) *cli.Command {
Usage: "show ccr remote info",
UsageText: "info [options] [<index>]",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.CcrRemoteInfo(conf, cmd.Args().Get(0))
},

View File

@@ -59,6 +59,10 @@ func CcrFollowerRenew(conf *cfg.Config) *cli.Command {
},
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
@@ -77,6 +81,10 @@ func CcrFollowerResume(conf *cfg.Config) *cli.Command {
Usage: "resume ccr index to follow",
UsageText: "resume [options] <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
@@ -95,6 +103,10 @@ func CcrFollowerPause(conf *cfg.Config) *cli.Command {
Usage: "pause ccr index to follow",
UsageText: "pause [options] <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
@@ -113,6 +125,10 @@ func CcrFollowerUnfollow(conf *cfg.Config) *cli.Command {
Usage: "unfollow ccr follower index",
UsageText: "unfollow [options] <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
@@ -141,6 +157,10 @@ func CcrFollowerAdd(conf *cfg.Config) *cli.Command {
},
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
@@ -160,6 +180,10 @@ func CcrFollowerDelete(conf *cfg.Config) *cli.Command {
Usage: "delete ccr follower index",
UsageText: "delete <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
@@ -180,6 +204,10 @@ func CcrFollowerShow(conf *cfg.Config) *cli.Command {
Usage: "show ccr follower index details",
UsageText: "show <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()

View File

@@ -18,6 +18,7 @@ package cmd
import (
"context"
"errors"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
@@ -33,8 +34,10 @@ func Cluster(conf *cfg.Config) *cli.Command {
Commands: []*cli.Command{
ClusterStatus(conf),
ClusterSwitch(conf),
ClusterList(conf),
ClusterSettings(conf),
ClusterReroute(conf),
},
}
}
@@ -58,12 +61,6 @@ func ClusterStatus(conf *cfg.Config) *cli.Command {
Aliases: []string{"s"},
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "all",
Usage: "show status of all clusters",
Destination: &conf.All,
Aliases: []string{"a"},
},
&cli.BoolFlag{
Name: "verbose",
Usage: "include verbose statistics",
@@ -77,3 +74,25 @@ func ClusterStatus(conf *cfg.Config) *cli.Command {
},
}
}
func ClusterSwitch(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "switch",
Usage: "set current elasticsearch cluster",
UsageText: "switch <cluster-name>",
Aliases: []string{"ctx"},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Ccluster)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
name := cmd.Args().Get(0)
if name == "" {
return errors.New("no cluster name specified")
}
return conf.SwitchCluster(name)
},
}
}

224
cmd/cluster_reroute.go Normal file
View File

@@ -0,0 +1,224 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"errors"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func ClusterReroute(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "reroute",
Usage: "manually change the allocation of individual shards in the cluster.",
UsageText: "reroute <cluster-name>",
Aliases: []string{"ctx"},
Commands: []*cli.Command{
ClusterRerouteMove(conf),
ClusterRerouteAllocateReplica(conf),
ClusterRerouteCancel(conf),
ClusterRerouteAllocatePrimary(conf, false),
ClusterRerouteAllocatePrimary(conf, true),
},
}
}
func ClusterRerouteMove(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "move",
Usage: "move shard to another node",
UsageText: "move [options] <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Flags: []cli.Flag{
&cli.IntFlag{
Name: "shard",
Usage: "shard to move",
Destination: &conf.Shards,
Aliases: []string{"s"},
Required: true,
},
&cli.StringFlag{
Name: "from-node",
Usage: "current node",
Destination: &conf.FromNode,
Aliases: []string{"f"},
Required: true,
},
&cli.StringFlag{
Name: "to-node",
Usage: "node to move to",
Destination: &conf.ToNode,
Aliases: []string{"t"},
Required: true,
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.ClusterRerouteMove(conf, index)
},
}
}
func ClusterRerouteAllocateReplica(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "allocate-replica",
Usage: "allocate-replica replica to another node",
UsageText: "allocate-replica [options] <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Flags: []cli.Flag{
&cli.IntFlag{
Name: "shard",
Usage: "shard to move",
Destination: &conf.Shards,
Aliases: []string{"s"},
Required: true,
},
&cli.StringFlag{
Name: "to-node",
Usage: "node to move to",
Destination: &conf.ToNode,
Aliases: []string{"t"},
Required: true,
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.ClusterRerouteAllocateReplica(conf, index)
},
}
}
func ClusterRerouteCancel(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "cancel",
Usage: "cancel a reroute operation",
UsageText: "cancel [options] <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Flags: []cli.Flag{
&cli.IntFlag{
Name: "shard",
Usage: "shard to move",
Destination: &conf.Shards,
Aliases: []string{"s"},
Required: true,
},
&cli.StringFlag{
Name: "to-node",
Usage: "node to move to",
Destination: &conf.ToNode,
Aliases: []string{"t"},
Required: true,
},
&cli.BoolFlag{
Name: "allow-primary",
Usage: "allow primary shard to cancel",
Destination: &conf.AllowPrimary,
Aliases: []string{"a"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.ClusterRerouteCancel(conf, index)
},
}
}
func ClusterRerouteAllocatePrimary(conf *cfg.Config, stale bool) *cli.Command {
name := "allocate-empty-primary"
usage := "allocate an empty primary shard to a node"
if stale {
name = "allocate-stale-primary"
usage = "allocate a stale primary shard to a node"
}
return &cli.Command{
Name: name,
Usage: usage,
UsageText: name + " [options] <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Flags: []cli.Flag{
&cli.IntFlag{
Name: "shard",
Usage: "shard to move",
Destination: &conf.Shards,
Aliases: []string{"s"},
Required: true,
},
&cli.StringFlag{
Name: "to-node",
Usage: "node to move to",
Destination: &conf.ToNode,
Aliases: []string{"t"},
Required: true,
},
&cli.BoolFlag{
Name: "accept-data-loss",
Usage: "",
Destination: &conf.AcceptDataLoss,
Aliases: []string{"a"},
Required: true,
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.ClusterRerouteAllocatePrimary(conf, index, stale)
},
}
}

View File

@@ -27,7 +27,7 @@ import (
)
const (
SETTINGS = `https://www.elastic.co/docs/reference/elasticsearch/configuration-reference`
SETTINGS = ``
)
func ClusterSettings(conf *cfg.Config) *cli.Command {
@@ -85,7 +85,9 @@ func ClusterSettingsSet(conf *cfg.Config) *cli.Command {
Name: "set",
Usage: "set|update cluster settings",
Aliases: []string{"set", "update"},
UsageText: "set [options] setting:value [setting:value ...]\n\nReference: " + SETTINGS,
UsageText: "set [options] setting:value [setting:value ...]",
CustomHelpTemplate: addReference(
"https://www.elastic.co/docs/reference/elasticsearch/configuration-reference"),
Flags: []cli.Flag{
&cli.BoolFlag{
@@ -102,6 +104,10 @@ func ClusterSettingsSet(conf *cfg.Config) *cli.Command {
},
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cclustersettings)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()

80
cmd/completion.go Normal file
View File

@@ -0,0 +1,80 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"fmt"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
const (
Cindex = iota
Cdatastream
Crole
Ccluster
Capi
Cilm
Cnode
Cclustersettings
Cindextemplate
)
func complete(conf *cfg.Config, cmd *cli.Command, what int) {
if cmd.NArg() > 0 {
return
}
var (
list []string
err error
)
switch what {
case Cindex:
list, err = es.IndexNames(conf)
case Crole:
list, err = es.RoleNames(conf)
case Ccluster:
for cluster := range conf.Clusters {
list = append(list, cluster)
}
case Cdatastream:
list, err = es.DatastreamNames(conf)
case Capi:
list = es.ApiPathNames()
case Cilm:
list, err = es.IlmNames(conf)
case Cnode:
list, err = es.NodeNames(conf)
case Cclustersettings:
list, err = es.ClusterSettingsNames(conf)
case Cindextemplate:
list, err = es.IndexTemplateNames(conf)
}
if err != nil {
return
}
for _, item := range list {
fmt.Println(item)
}
}

220
cmd/datastream.go Normal file
View File

@@ -0,0 +1,220 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"errors"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func Datastream(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "datastream",
Aliases: []string{"ds"},
Usage: "manage data streams",
Commands: []*cli.Command{
DatastreamList(conf),
DatastreamShow(conf),
DatastreamCreate(conf),
DatastreamDelete(conf),
DatastreamRollover(conf),
},
}
}
func DatastreamList(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "list",
Aliases: []string{"ls"},
Usage: "list indices",
Flags: []cli.Flag{
&cli.IntFlag{
Name: "max",
Usage: "max number of items to process",
Destination: &conf.MaxItems,
Aliases: []string{"m"},
},
&cli.BoolFlag{
Name: "hidden",
Usage: "include hidden data streams",
Destination: &conf.Hidden,
Aliases: []string{"H"},
},
&cli.StringSliceFlag{
Name: "filter",
Usage: "show only data streams matching the filter",
Destination: &conf.Filter,
Aliases: []string{"F"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.DatastreamList(conf)
},
}
}
func DatastreamShow(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "show",
Aliases: []string{"sh"},
Usage: "show details about an data stream",
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no data stream specified")
}
return es.DatastreamShow(conf, cmd.Args().Get(0))
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cdatastream)
},
}
}
func DatastreamCreate(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "create",
Aliases: []string{"+"},
Usage: "create a new data stream",
UsageText: "create [options] <name>",
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no data stream specified")
}
return es.DatastreamCreate(conf, cmd.Args().Get(0))
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cdatastream)
},
}
}
func DatastreamDelete(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "delete",
Aliases: []string{"rm"},
Usage: "delete a data stream",
UsageText: "delete [options] <name>",
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no data stream specified")
}
return es.DatastreamDelete(conf, cmd.Args().Get(0))
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cdatastream)
},
}
}
func DatastreamRollover(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "rollover",
Aliases: []string{"roll"},
Usage: "roll over a data stream",
UsageText: "rollover <data stream>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "wait",
Usage: "wait for active shards",
Destination: &conf.Wait,
Aliases: []string{"w"},
},
&cli.BoolFlag{
Name: "dry-run",
Usage: "checks conditions but does not perform a rollover",
Destination: &conf.Wait,
Aliases: []string{"n"},
},
&cli.StringFlag{
Name: "max-age",
Usage: "roll over after max age (eg: 7d, 2m, 8h)",
Destination: &conf.MaxAge,
},
&cli.Int64Flag{
Name: "max-docs",
Usage: "roll over after max docs",
Destination: &conf.MaxDocs,
},
&cli.IntFlag{
Name: "max-primary-shard-size",
Usage: "roll over when primary shard size reaches size",
Destination: &conf.MaxShardSize,
},
&cli.IntFlag{
Name: "max-primary-shard-docs",
Usage: "roll over after max docs in primary shard reached",
Destination: &conf.MaxShardDocs,
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
alias := cmd.Args().Get(0)
if alias == "" {
return errors.New("no data stream specified")
}
return es.DatastreamRollover(conf, alias)
},
}
}
func DatastreamIlm(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "ilm",
Usage: "show ilm status",
UsageText: "ds ilm <name>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cdatastream)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
ds := cmd.Args().Get(0)
if ds == "" {
return errors.New("no ds name specified")
}
return es.IlmExplain(conf, ds)
},
}
}

View File

@@ -33,7 +33,8 @@ func Doc(conf *cfg.Config) *cli.Command {
Commands: []*cli.Command{
DocAdd(conf),
//Delete(conf),
DocShow(conf),
DocDelete(conf),
},
}
}
@@ -43,16 +44,120 @@ func DocAdd(conf *cfg.Config) *cli.Command {
Name: "add",
Aliases: []string{"+"},
Usage: "add JSON document index",
UsageText: "add [options] <index> '<json-doc>'",
UsageText: "add [options] -i <index> '<json-doc>'",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "index",
Usage: "index to work on",
Sources: cli.EnvVars("ES_INDEX"),
Destination: &conf.Index,
Aliases: []string{"i"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
if args.Len() != 2 {
return errors.New("missing arguments: <index> <json-doc>")
if args.Len() != 1 {
return errors.New("missing arguments: <json-doc>")
}
return es.DocAdd(conf, cmd.Args().Get(0), cmd.Args().Get(1))
return es.DocAdd(conf, cmd.Args().Get(0))
},
}
}
func DocShow(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "show",
Aliases: []string{"sh"},
Usage: "show a JSON document",
UsageText: "show [options] -i <index> <id>",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "index",
Usage: "index to work on",
Sources: cli.EnvVars("ES_INDEX"),
Destination: &conf.Index,
Aliases: []string{"i"},
},
&cli.StringFlag{
Name: "path",
Usage: "jsonPath filter (e.g. source.message)",
Destination: &conf.Path,
Aliases: []string{"p"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
if args.Len() != 1 {
return errors.New("missing arguments: <id>")
}
return es.DocShow(conf, cmd.Args().Get(0))
},
}
}
func DocDelete(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "delete",
Aliases: []string{"rm"},
Usage: "delete JSON document[s] from index[es]",
UsageText: "delete [options] -i <index> [<[field<sep>]pattern> ...]\n" + SearchUsage,
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "all",
Usage: "delete all docs (dangerous!)",
Destination: &conf.All,
Aliases: []string{"a"},
},
&cli.StringFlag{
Name: "index",
Usage: "index to work on",
Sources: cli.EnvVars("ES_INDEX"),
Destination: &conf.Index,
Aliases: []string{"i"},
},
&cli.StringSliceFlag{
Name: "filter",
Usage: "additional boolean filters. format: key=value",
Destination: &conf.Filter,
Aliases: []string{"F"},
},
&cli.StringFlag{
Name: "timerange",
Usage: "field:<date> to <date> (e.g. @timestamp:2026-05-05 to 2026-05-15)",
Destination: &conf.Range,
Aliases: []string{"r"},
},
&cli.StringFlag{
Name: "timestamp-format",
Usage: "a valid ES builtin timestamp or custom format",
Destination: &conf.TimestampFormat,
Value: "strict_date_hour_minute",
},
&cli.BoolFlag{
Name: "or",
Usage: "logical operator (default: and)",
Destination: &conf.Or,
Aliases: []string{"O"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
if args.Len() == 0 && !conf.All {
return errors.New("missing arguments: <query>")
}
return es.DocDelete(conf, cmd.Args().Slice())
},
}
}

261
cmd/ilm.go Normal file
View File

@@ -0,0 +1,261 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"errors"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func Ilm(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "ilm",
Usage: "manage index lifecycle",
Commands: []*cli.Command{
IlmRetry(conf),
IlmStatus(conf),
IlmList(conf),
IlmShow(conf),
IlmCreate(conf, false),
IlmCreate(conf, true),
IlmForecast(conf),
IlmExplain(conf),
},
}
}
func IlmRetry(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "retry",
Usage: "retry applying an ILM profile to an index",
UsageText: "retry <index>",
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.IlmRetry(conf, cmd.Args().Get(0))
},
}
}
func IlmExplain(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "explain",
Usage: "explain ilm condition of an index",
UsageText: "explain <index>",
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.IlmExplain(conf, index)
},
}
}
func IlmStatus(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "status",
Usage: "get the current index lifecycle management status",
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.IlmStatus(conf)
},
}
}
func IlmList(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "list",
Aliases: []string{"ls"},
Usage: "list index lifecycle policies",
UsageText: "list [<policy-pattern>]",
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.IlmList(conf, cmd.Args().Get(0))
},
}
}
func IlmShow(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "show",
Aliases: []string{"sh"},
Usage: "show details about an index lifecycle policy",
UsageText: "show <policy>",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "tree",
Usage: "display policy as a tree",
Destination: &conf.Ilm.Tree,
Aliases: []string{"t"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
policy := cmd.Args().Get(0)
if policy == "" {
return errors.New("no policy specified")
}
return es.IlmShow(conf, cmd.Args().Get(0))
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cilm)
},
}
}
func IlmCreate(conf *cfg.Config, modify bool) *cli.Command {
name := "create"
alias := "+"
usage := "create a new lifecycle policy"
if modify {
name = "update"
usage = "update an lifecycle policy"
alias = "upd"
}
return &cli.Command{
Name: name,
Aliases: []string{alias},
Usage: usage,
UsageText: name + " [options] <policy>",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "description",
Usage: "policy description",
Destination: &conf.Ilm.Description,
Aliases: []string{"D"},
},
// hot
&cli.StringFlag{
Name: "hot-min-age",
Usage: "hot phase: min age",
Destination: &conf.Ilm.HotMinAge,
},
&cli.StringFlag{
Name: "hot-rollover-max-age",
Usage: "hot phase: rollover after",
Destination: &conf.Ilm.HotRolloverMaxAge,
},
&cli.StringFlag{
Name: "hot-rollover-max-primary-shard-size",
Usage: "hot phase: max primary shard size",
Destination: &conf.Ilm.HotRolloverMaxPrimaryShardSize,
},
&cli.Int64Flag{
Name: "hot-rollover-max-docs",
Usage: "hot phase: max docs",
Destination: &conf.Ilm.HotRolloverMaxDocs,
},
&cli.BoolFlag{
Name: "hot-readonly",
Usage: "hot phase: make index ro before rollover",
Destination: &conf.Ilm.HotReadonly,
},
// warm
&cli.StringFlag{
Name: "warm-min-age",
Usage: "warm phase: min age",
Destination: &conf.Ilm.WarmMinAge,
},
&cli.IntFlag{
Name: "warm-force-merge-segments",
Usage: "warm phase: force merge segments per shard",
Destination: &conf.Ilm.WarmForceMerge,
},
&cli.IntFlag{
Name: "warm-shrink-shards",
Usage: "warm phase: shrink on number of shards",
Destination: &conf.Ilm.WarmShrinkShards,
},
&cli.IntFlag{
Name: "warm-priority",
Usage: "warm phase: priority",
Destination: &conf.Ilm.WarmPriority,
},
// cold
&cli.StringFlag{
Name: "cold-min-age",
Usage: "cold phase: min age",
Destination: &conf.Ilm.ColdMinAge,
},
&cli.StringFlag{
Name: "cold-searchable-snapshot-repo",
Usage: "cold phase: searchable snapshot repo",
Destination: &conf.Ilm.ColdSearchableSnapshotRepo,
},
&cli.IntFlag{
Name: "cold-force-merge-segments",
Usage: "cold phase: force merge segments per shard",
Destination: &conf.Ilm.ColdForceMerge,
},
&cli.IntFlag{
Name: "cold-priority",
Usage: "cold phase: priority",
Destination: &conf.Ilm.ColdPriority,
},
// frozen
&cli.StringFlag{
Name: "frozen-min-age",
Usage: "frozen phase: min age",
Destination: &conf.Ilm.FrozenMinAge,
},
&cli.StringFlag{
Name: "frozen-searchable-snapshot-repo",
Usage: "frozen phase: searchable snapshot repo",
Destination: &conf.Ilm.FrozenSearchableSnapshotRepo,
},
// delete
&cli.StringFlag{
Name: "delete-min-age",
Usage: "delete phase: min age",
Destination: &conf.Ilm.DeleteMinAge,
},
&cli.BoolFlag{
Name: "delete-searchable-snapshots",
Usage: "delete phase: delete searchable snapshots",
Destination: &conf.Ilm.DeleteSearchableSnapshots,
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
policy := cmd.Args().Get(0)
if policy == "" {
return errors.New("no policy name specified")
}
return es.IlmCreate(conf, cmd.Args().Get(0))
},
}
}

139
cmd/ilm_forecast.go Normal file
View File

@@ -0,0 +1,139 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"errors"
"slices"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func IlmForecast(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "forecast",
Usage: "calculate index phase movements",
Commands: []*cli.Command{
IlmForecastList(conf),
IlmForecastShow(conf),
},
}
}
const IlmForeCastListReference = `<duration> must be a string which consists one or more of these elements:
d - days
h - hours
m - minutes
s - seconds
Examples:
58d12h - 58 days and 12 hours
122h30m - 122 hours and 30 minutes
Specs may be mixed:
8h12d - 12 days and 8 hours`
func IlmForecastList(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "list",
Aliases: []string{"ls"},
Usage: "list index rollover config",
UsageText: "list [options] [<filter>]",
CustomHelpTemplate: addReference(IlmForeCastListReference),
Flags: []cli.Flag{
&cli.StringFlag{
Name: "from-phase",
Usage: "which ilm phase to forecast from",
Destination: &conf.Ilm.FromPhase,
Aliases: []string{"f"},
},
&cli.StringFlag{
Name: "within",
Usage: "<duration> within which to forecast",
Destination: &conf.Ilm.Within,
Aliases: []string{"w"},
},
&cli.StringFlag{
Name: "min-age",
Usage: "show only indices older than <duration>",
Destination: &conf.Ilm.MinAge,
Aliases: []string{"m"},
},
&cli.BoolFlag{
Name: "verbose",
Usage: "show ilm policy as well",
Destination: &conf.Verbose,
Aliases: []string{"v"},
},
&cli.BoolFlag{
Name: "hidden",
Usage: "include hidden indices",
Destination: &conf.Hidden,
Aliases: []string{"H"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
valid := []string{"hot", "warm", "cold", "frozen"}
if !slices.Contains(valid, conf.Ilm.FromPhase) {
return errors.New("invalid from phase, allowed: hot, warm, cold, frozen")
}
return es.IlmForecastList(conf, cmd.Args().Get(0))
},
}
}
func IlmForecastShow(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "show",
Aliases: []string{"sh"},
Usage: "show rollover forecast over all indices",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "from-phase",
Usage: "which ilm phase to forecast from",
Destination: &conf.Ilm.FromPhase,
Aliases: []string{"f"},
},
&cli.StringFlag{
Name: "within",
Usage: "duration within which to forecast",
Destination: &conf.Ilm.Within,
Aliases: []string{"w"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
valid := []string{"hot", "warm", "cold", "frozen"}
if !slices.Contains(valid, conf.Ilm.FromPhase) {
return errors.New("invalid from phase, allowed: hot, warm, cold, frozen")
}
return es.IlmForecastShow(conf)
},
}
}

View File

@@ -18,7 +18,7 @@ package cmd
import (
"context"
"fmt"
"errors"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
@@ -30,14 +30,23 @@ func Index(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "index",
Aliases: []string{"i"},
Usage: "manage indicies",
Usage: "manage indices",
Commands: []*cli.Command{
IndexList(conf),
IndexShow(conf),
IndexCreate(conf),
IndexCreate(conf, false),
IndexCreate(conf, true),
IndexDelete(conf),
IndexClose(conf),
IndexFields(conf),
IndexIlm(conf),
IndexDu(conf),
IndexCopy(conf),
// sub commands
IndexAlias(conf),
IndexTemplate(conf),
},
}
}
@@ -46,7 +55,7 @@ func IndexList(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "list",
Aliases: []string{"ls"},
Usage: "list indicies",
Usage: "list indices",
Flags: []cli.Flag{
&cli.IntFlag{
@@ -57,15 +66,27 @@ func IndexList(conf *cfg.Config) *cli.Command {
},
&cli.BoolFlag{
Name: "partials",
Usage: "include partial indicies",
Usage: "include partial indices",
Destination: &conf.Partials,
Aliases: []string{"p"},
},
&cli.BoolFlag{
Name: "reds",
Usage: "include only red failed indicies",
Name: "hidden",
Usage: "include hidden indices",
Destination: &conf.Hidden,
Aliases: []string{"H"},
},
&cli.BoolFlag{
Name: "failed",
Usage: "include only red failed indices",
Destination: &conf.Failed,
Aliases: []string{"r"},
Aliases: []string{"f"},
},
&cli.StringSliceFlag{
Name: "filter",
Usage: "show only indices matching the filter",
Destination: &conf.Filter,
Aliases: []string{"F"},
},
},
@@ -82,35 +103,35 @@ func IndexShow(conf *cfg.Config) *cli.Command {
Usage: "show details about an index",
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.IndexShow(conf, cmd.Args().Get(0))
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.IndexShow(conf, cmd.Args().Get(0))
},
// FIXME: doesn't work at all
// FIXME: also it would ONLY work if the user uses env vars, -C would not be
// there when the completion output is being generated
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
if cmd.NArg() > 0 {
return
}
indices, err := es.IndexNames(conf)
if err != nil {
return
}
for _, index := range indices {
fmt.Println(index)
}
complete(conf, cmd, Cindex)
},
}
}
func IndexCreate(conf *cfg.Config) *cli.Command {
func IndexCreate(conf *cfg.Config, modify bool) *cli.Command {
name := "create"
usage := "create a new index"
if modify {
name = "update"
usage = "update an index"
}
return &cli.Command{
Name: "create",
Name: name,
Aliases: []string{"+"},
Usage: "create a new index",
Usage: usage,
UsageText: name + ` <name> <fieldmapping:type>...
Valid field mapping types: integer, text, date, keyword`,
Flags: []cli.Flag{
&cli.BoolFlag{
@@ -130,13 +151,20 @@ func IndexCreate(conf *cfg.Config) *cli.Command {
Usage: "number of replicas to create",
Destination: &conf.Replicas,
Aliases: []string{"r"},
Value: 1,
},
&cli.StringFlag{
Name: "ilm-policy",
Usage: "ilm policy to apply",
Destination: &conf.Policy,
Aliases: []string{"p"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
if args.Len() == 0 {
return fmt.Errorf("no index specified")
return errors.New("no index specified")
}
mappings := args.Slice()[1:]
@@ -148,11 +176,21 @@ func IndexCreate(conf *cfg.Config) *cli.Command {
func IndexDelete(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "delete",
Aliases: []string{"rm"},
Usage: "delete an index",
Name: "delete",
Aliases: []string{"rm"},
Usage: "delete an index",
UsageText: "delete <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.IndexDelete(conf, cmd.Args().Get(0))
},
}
@@ -160,11 +198,184 @@ func IndexDelete(conf *cfg.Config) *cli.Command {
func IndexClose(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "close",
Usage: "close an index",
Name: "close",
Usage: "close an index",
UsageText: "close <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.IndexClose(conf, cmd.Args().Get(0))
},
}
}
func IndexFields(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "fields",
Usage: "show info about field capabilities",
UsageText: "index fields <index>",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "aggretable",
Usage: "include only aggretable fields",
Destination: &conf.Aggretable,
Aliases: []string{"a"},
},
&cli.BoolFlag{
Name: "searchable",
Usage: "include only searchable fields",
Destination: &conf.Searchable,
Aliases: []string{"s"},
},
&cli.StringSliceFlag{
Name: "type",
Usage: "show only fields of this type",
Destination: &conf.Filter,
Aliases: []string{"t"},
},
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.IndexFields(conf, index)
},
}
}
func IndexIlm(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "ilm",
Usage: "show ilm status",
UsageText: "index ilm <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.IlmExplain(conf, index)
},
}
}
func IndexDu(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "du",
Usage: "show index disk usage",
UsageText: "index du <index>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.IndexDiskusage(conf, index)
},
}
}
func IndexCopy(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "copy",
Aliases: []string{"alias", "cp"},
Usage: "copy (reindex) documents from one index to another",
UsageText: "index copy [options] -s <source-index> -t <dest-index>",
Flags: []cli.Flag{
// FIXME: not implemented by typed API
// &cli.BoolFlag{
// Name: "missing", // op_type
// Usage: "copy only missing docs",
// Destination: &conf.Missing,
// Aliases: []string{"m"},
// },
// &cli.BoolFlag{
// Name: "sync", // version_type to external
// Usage: "create missing docs and update outdated docs",
// Destination: &conf.Sync,
// Aliases: []string{"S"},
// },
&cli.BoolFlag{
Name: "force", // conflicts to proceed
Usage: "continue reindexing even when conflicts happen",
Destination: &conf.Force,
Aliases: []string{"f"},
},
&cli.Float64Flag{
Name: "requests-per-second",
Usage: "the maximum number of documents to index per second (-1 turns off throttling)",
Destination: &conf.RequestsPerSecond,
Aliases: []string{"R"},
},
&cli.Int64Flag{
Name: "max-docs",
Usage: "the maximum number of documents to reindex",
Destination: &conf.MaxDocs,
Aliases: []string{"m"},
},
&cli.DurationFlag{
Name: "timeout",
Usage: "timeout for write operations (e.g. 300m or 120s)",
Destination: &conf.Timeout,
Aliases: []string{"T"},
},
&cli.BoolFlag{
Name: "refresh",
Usage: "refresh affected shards to make this operation visible to search",
Destination: &conf.Refresh,
Aliases: []string{"r"},
},
&cli.BoolFlag{
Name: "wait",
Usage: "wait for active shards",
Destination: &conf.Wait,
Aliases: []string{"w"},
},
&cli.StringSliceFlag{
Name: "source",
Usage: "source index (multiple supported)",
Destination: &conf.SourceIndices,
Aliases: []string{"s"},
Required: true,
},
&cli.StringFlag{
Name: "target",
Usage: "target index",
Destination: &conf.Index,
Aliases: []string{"t"},
Required: true,
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.IndexCopy(conf)
},
}
}

172
cmd/index_alias.go Normal file
View File

@@ -0,0 +1,172 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"errors"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func IndexAlias(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "alias",
Aliases: []string{"a"},
Usage: "manage index aliases",
Commands: []*cli.Command{
IndexAliasCreate(conf),
IndexAliasList(conf),
IndexAliasDelete(conf),
IndexAliasRollover(conf),
// FIXME: implement IndexAliasAdd
// see https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-update-aliases
// IndexAliasAdd(conf),
},
}
}
func IndexAliasCreate(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "create",
Aliases: []string{"+"},
Usage: "create an index alias",
UsageText: "create <index> <alias>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
alias := cmd.Args().Get(1)
if index == "" || alias == "" {
return errors.New("no index or alias specified")
}
return es.IndexAliasCreate(conf, index, alias)
},
}
}
func IndexAliasDelete(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "delete",
Aliases: []string{"rm"},
Usage: "delete an index alias",
UsageText: "delete <index> <alias>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
alias := cmd.Args().Get(1)
if index == "" || alias == "" {
return errors.New("no index or alias specified")
}
return es.IndexAliasDelete(conf, index, alias)
},
}
}
func IndexAliasRollover(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "rollover",
Aliases: []string{"roll"},
Usage: "roll over an index alias",
UsageText: "rollover <alias>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindex)
},
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "wait",
Usage: "wait for active shards",
Destination: &conf.Wait,
Aliases: []string{"w"},
},
&cli.BoolFlag{
Name: "dry-run",
Usage: "checks conditions but does not perform a rollover",
Destination: &conf.Wait,
Aliases: []string{"n"},
},
&cli.StringFlag{
Name: "max-age",
Usage: "roll over after max age (eg: 7d, 2m, 8h)",
Destination: &conf.MaxAge,
},
&cli.Int64Flag{
Name: "max-docs",
Usage: "roll over after max docs",
Destination: &conf.MaxDocs,
},
&cli.IntFlag{
Name: "max-primary-shard-size",
Usage: "roll over when primary shard size reaches size",
Destination: &conf.MaxShardSize,
},
&cli.IntFlag{
Name: "max-primary-shard-docs",
Usage: "roll over after max docs in primary shard reached",
Destination: &conf.MaxShardDocs,
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
alias := cmd.Args().Get(0)
if alias == "" {
return errors.New("no alias specified")
}
return es.IndexAliasRollover(conf, alias)
},
}
}
func IndexAliasList(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "list",
Aliases: []string{"ls"},
Usage: "list index aliases",
Flags: []cli.Flag{
&cli.StringSliceFlag{
Name: "filter",
Usage: "show only aliases for indices matching the filter",
Destination: &conf.Filter,
Aliases: []string{"F"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.IndexAliasList(conf)
},
}
}

236
cmd/index_template.go Normal file
View File

@@ -0,0 +1,236 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"errors"
"slices"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func IndexTemplate(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "template",
Aliases: []string{"tpl"},
Usage: "manage index templates",
Commands: []*cli.Command{
IndexTemplateList(conf),
IndexTemplateShow(conf),
IndexTemplateCreate(conf, false),
IndexTemplateCreate(conf, true),
IndexTemplateDelete(conf),
},
}
}
func IndexTemplateList(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "list",
Aliases: []string{"ls"},
Usage: "list index templates",
Flags: []cli.Flag{
&cli.StringSliceFlag{
Name: "filter-index-pattern",
Usage: "show only index templates which use patterns, which match the filter",
Destination: &conf.Filter,
Aliases: []string{"F"},
},
&cli.BoolFlag{
Name: "hidden",
Usage: "show hidden index templates as well",
Destination: &conf.Hidden,
Aliases: []string{"H"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.IndexTemplateList(conf, cmd.Args().Get(0))
},
}
}
func IndexTemplateShow(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "show",
Aliases: []string{"sh"},
Usage: "show details about an index template",
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index template specified")
}
return es.IndexTemplateShow(conf, cmd.Args().Get(0))
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindextemplate)
},
}
}
func IndexTemplateCreate(conf *cfg.Config, modify bool) *cli.Command {
name := "create"
alias := "+"
required := true
if modify {
name = "update"
alias = "upd"
required = false
}
return &cli.Command{
Name: name,
Aliases: []string{alias},
Usage: name + " a new index template",
UsageText: name + ` <name> <fieldmapping:type>...`,
CustomHelpTemplate: addReference(
`Valid field mapping types: integer, text, date, keyword
Doc for index settings:
https://www.elastic.co/docs/reference/elasticsearch/index-settings
`),
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "data-stream",
Usage: "template creates a data stream",
Destination: &conf.Stream,
Aliases: []string{"S"},
},
&cli.BoolFlag{
Name: "allow-auto-create",
Usage: "automatically create indices",
Destination: &conf.AutoCreate,
Aliases: []string{"a"},
},
&cli.BoolFlag{
Name: "rollover",
Usage: "automatically rollover associated aliases",
Destination: &conf.Rollover,
Aliases: []string{"R"},
},
&cli.StringFlag{
Name: "mode",
Usage: "index mode, one of: standard, timeseries, logsdb or lookup",
Destination: &conf.Mode,
Aliases: []string{"m"},
},
&cli.StringFlag{
Name: "retention",
Usage: "data retention, e.g. -1 (infinitely), 30d, 3m or 1y (data stream only)",
Destination: &conf.Retention,
Aliases: []string{"r"},
Required: required,
},
&cli.IntFlag{
Name: "priority",
Usage: "index template priority",
Destination: &conf.Priority,
Aliases: []string{"p"},
},
&cli.StringSliceFlag{
Name: "index-pattern",
Usage: "index matching pattern",
Destination: &conf.Patterns,
Aliases: []string{"i"},
Required: required,
},
&cli.StringSliceFlag{
Name: "settings",
Usage: "individual template settings, format: name:value, multiple possible",
Destination: &conf.Settings,
Aliases: []string{"s"},
},
&cli.StringSliceFlag{
Name: "component",
Usage: "include component template, multiple possible",
Destination: &conf.Components,
Aliases: []string{"c"},
},
&cli.StringSliceFlag{
Name: "meta",
Usage: "custom meta field, format: name:value, multiple possible",
Destination: &conf.Meta,
Aliases: []string{"M"},
},
&cli.StringSliceFlag{
Name: "aliases",
Usage: "set up aliases to associate with indices",
Destination: &conf.Aliases,
Aliases: []string{"A"},
},
&cli.StringFlag{
Name: "ilm-policy",
Usage: "ilm policy to apply",
Destination: &conf.Policy,
Aliases: []string{"p"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
if args.Len() == 0 {
return errors.New("no name specified")
}
if conf.Mode != "" {
if !slices.Contains([]string{"standard", "timeseries", "logsdb", "lookup"}, conf.Mode) {
return errors.New("mode must be one of: standard, timeseries, logsdb or lookup")
}
}
mappings := args.Slice()[1:]
if modify {
return es.IndexTemplateModify(conf, args.Get(0), mappings)
}
return es.IndexTemplateCreate(conf, args.Get(0), mappings)
},
}
}
func IndexTemplateDelete(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "delete",
Aliases: []string{"rm"},
Usage: "delete an index template",
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index template specified")
}
return es.IndexTemplateDelete(conf, cmd.Args().Get(0))
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cindextemplate)
},
}
}

49
cmd/license.go Normal file
View File

@@ -0,0 +1,49 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func License(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "license",
Usage: "manage cluster license",
Commands: []*cli.Command{
LicenseShow(conf),
},
}
}
func LicenseShow(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "show",
Aliases: []string{"sh"},
Usage: "show details about the cluster license",
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.LicenseShow(conf)
},
}
}

View File

@@ -18,6 +18,7 @@ package cmd
import (
"context"
"errors"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
@@ -34,6 +35,8 @@ func Node(conf *cfg.Config) *cli.Command {
Commands: []*cli.Command{
NodeList(conf),
NodeShow(conf),
NodeClients(conf),
NodeUsage(conf),
},
}
}
@@ -57,9 +60,63 @@ func NodeShow(conf *cfg.Config) *cli.Command {
Usage: "show details about a node",
UsageText: "show [options] <node>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cnode)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
// return es.NodeShow(conf, cmd.Args().Get(0))
return nil
node := cmd.Args().Get(0)
if node == "" {
return errors.New("no node specified")
}
return es.NodeShow(conf, node)
},
}
}
func NodeClients(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "clients",
Usage: "show node http clients",
UsageText: "clients [options] <node>",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "query",
Usage: "include query parameters",
Destination: &conf.All,
Aliases: []string{"q"},
},
},
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cnode)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
node := cmd.Args().Get(0)
if node == "" {
return errors.New("no node specified")
}
return es.NodeClients(conf, node)
},
}
}
func NodeUsage(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "usage",
Usage: "show node usage stats",
UsageText: "usage [options] [<node>]",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Cnode)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.NodeUsage(conf, cmd.Args().Get(0))
},
}
}

View File

@@ -15,24 +15,3 @@ You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func Repl(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "repl",
Aliases: []string{"shell"},
Usage: "interactive API repl",
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.Repl(conf)
},
}
}

141
cmd/roles.go Normal file
View File

@@ -0,0 +1,141 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"errors"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func Roles(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "role",
Usage: "manage roles",
Commands: []*cli.Command{
RoleList(conf),
RoleShow(conf),
RoleDiff(conf),
},
}
}
func RoleList(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "list",
Aliases: []string{"ls"},
Usage: "list roles",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "orphaned",
Usage: "include only orphaned roles",
Destination: &conf.Failed,
Aliases: []string{"o"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.RoleList(conf)
},
}
}
func RoleShow(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "show",
Aliases: []string{"sh"},
Usage: "show details about a role",
UsageText: "show [options] <role>",
ShellComplete: func(ctx context.Context, cmd *cli.Command) {
complete(conf, cmd, Crole)
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no role specified")
}
return es.RoleShow(conf, cmd.Args().Get(0))
},
}
}
func RoleDiff(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "diff",
Usage: "show differences between roles and CSV baseline",
UsageText: `diff [options] <file.csv> [<role>]
CSV format:
index_name;role;index_privilege;cluster_privilege;ad_group;space;retention;kibana_privilege;field_privilege`,
MutuallyExclusiveFlags: []cli.MutuallyExclusiveFlags{{
Flags: [][]cli.Flag{
{
&cli.BoolFlag{
Name: "not-deployed",
Usage: "include only not deployed but defined roles",
Destination: &conf.NotDeployed,
Aliases: []string{"n"},
},
},
{
&cli.BoolFlag{
Name: "undefined",
Usage: "include only deployed but undefined roles",
Destination: &conf.Undefined,
Aliases: []string{"u"},
},
},
{
&cli.BoolFlag{
Name: "diff",
Usage: "include only differing roles",
Destination: &conf.Diff,
Aliases: []string{"D"},
},
},
}},
},
Flags: []cli.Flag{
&cli.StringFlag{
Name: "separator",
Usage: "CSV field separator",
Destination: &conf.Separator,
Aliases: []string{"s"},
Value: ",",
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
csvfile := cmd.Args().Get(0)
if csvfile == "" {
return errors.New("no CSV file specified")
}
return es.RoleDiff(conf, cmd.Args().Get(0), cmd.Args().Get(1))
},
}
}

View File

@@ -19,17 +19,77 @@ package cmd
import (
"context"
"fmt"
golog "log"
"os"
"runtime/debug"
"runtime/pprof"
"strings"
"codeberg.org/scip/esctl/assets"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"codeberg.org/scip/esctl/pkg/log"
"codeberg.org/scip/esctl/pkg/printer"
markdown "github.com/MichaelMure/go-term-markdown"
"github.com/urfave/cli/v3"
)
const (
versionFmt = `esctl version: %s
build: %s
branch: %s
commit: %s
go version: %s
API Version: %s
`
jsonPathUsage = `jsonPath usage:
name.last >> "Anderson"
age >> 37
children >> ["Sara","Alex","Jack"]
children.# >> 3
children.1 >> "Alex"
child*.2 >> "Jack"
c?ildren.0 >> "Sara"
fav\.movie >> "Deer Hunter"
friends.#.first >> ["Dale","Roger","Jane"]
friends.1.last >> "Craig"
You can also query an array for the first match by using #(...), or
find all matches with #(...)#. Queries support the ==, !=, <, <=, >,
>= comparison operators and the simple pattern matching % (like) and
!% (not like) operators. Eg:
friends.#(last=="Murphy").first >> "Dale"
friends.#(last=="Murphy")#.first >> ["Dale","Jane"]
friends.#(age>45)#.last >> ["Craig","Murphy"]
friends.#(first%"D*").last >> "Murphy"
friends.#(first!%"D*").last >> "Craig"
friends.#(nets.#(=="fb"))#.first >> ["Dale","Roger"]
To extract more than one field, use:
{"ns":source.namespace_name,"time":source.@timestamp} >>
{
"ns": "3f80316965c64405-275a75782e984bcc8d82",
"time": "2026-06-10T04:23:38.914080361+00:00"
}
{
"ns": "3f80316965c64405-f0e1dcef668e45bb9afd",
"time": "2026-06-10T09:39:00.554892542+00:00"
}
Documentation: https://github.com/tidwall/gjson/blob/master/SYNTAX.md`
)
func Finish(err error) int {
if err != nil {
fmt.Fprintln(os.Stderr, "Error: ", err.Error())
return 1
}
@@ -42,7 +102,6 @@ func Main() int {
cmd := &cli.Command{
Name: "esctl",
Usage: "manage elasticsearch from cli",
Version: cfg.Version,
EnableShellCompletion: true,
Flags: []cli.Flag{
@@ -54,6 +113,25 @@ func Main() int {
Sources: cli.EnvVars("ES_DEBUG"),
Destination: &conf.Debug,
},
&cli.BoolFlag{
Name: "debug-http",
Value: false,
Usage: "enable HTTP debugging",
Destination: &conf.DebugHTTP,
},
&cli.BoolFlag{
Name: "debug-goroutines",
Value: false,
Usage: "enable goroutine debugging",
Destination: &conf.DebugGoRoutines,
},
&cli.BoolFlag{
Name: "align-ints",
Aliases: []string{"I"},
Value: false,
Usage: "right align integers in tabular output",
Destination: &conf.AlignInts,
},
&cli.StringFlag{
Name: "config",
Aliases: []string{"c"},
@@ -69,17 +147,42 @@ func Main() int {
Usage: "cluster alias to work with",
Destination: &conf.CurrentCluster,
},
&cli.StringFlag{
Name: "output",
Aliases: []string{"o"},
Value: "",
Usage: "output mode (tsv, csv, json, yaml) default: tsv",
Destination: &conf.Output,
},
&cli.StringFlag{
Name: "profile-file",
Usage: "golang profiler output file",
Destination: &conf.ProfileFile,
Hidden: true,
},
},
Commands: []*cli.Command{
Search(conf),
Index(conf),
Snapshot(conf),
Cluster(conf),
Api(conf),
Ccr(conf),
Node(conf),
Cluster(conf),
Datastream(conf),
Doc(conf),
Repl(conf),
Ilm(conf),
Index(conf),
License(conf),
Node(conf),
Roles(conf),
Search(conf),
SearchQL(conf),
Shard(conf),
Snapshot(conf),
Task(conf),
Version(conf),
Debug(conf),
HelpJsonPath(conf),
HelpUsage(conf),
HelpEsQl(conf),
},
Before: func(ctx context.Context, cmd *cli.Command) (context.Context, error) {
@@ -88,15 +191,212 @@ func Main() int {
return nil, err
} else {
fmt.Println(cmd.UsageText)
return nil, nil
}
}
log.Init(conf)
if conf.ProfileFile != "" {
// enable cpu profiling. Do NOT use q to stop the game but
// close the window to get a profile
fd, err := os.Create(conf.ProfileFile)
if err != nil {
return nil, err
}
defer func() {
if err := fd.Close(); err != nil {
golog.Fatal(err)
}
}()
if err := pprof.StartCPUProfile(fd); err != nil {
golog.Fatal(err)
}
defer pprof.StopCPUProfile()
}
return nil, nil
},
}
return Finish(cmd.Run(context.Background(), os.Args))
}
func HelpJsonPath(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "help-jsonpath",
Usage: "show jsonpath help",
Action: func(ctx context.Context, cmd *cli.Command) error {
_, err := fmt.Println(jsonPathUsage)
return err
},
}
}
func HelpEsQl(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "help-esql",
Usage: "show esql help",
Action: func(ctx context.Context, cmd *cli.Command) error {
assets.LoadEsql()
width := cfg.GetTermWidth()
printer.Pager("esql cheat sheet", string(markdown.Render(assets.EsQlCheatSheet, width, cfg.DefaultMargin)))
return nil
},
}
}
func Version(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "version",
Usage: "show esctl version information",
Action: func(ctx context.Context, cmd *cli.Command) error {
info, _ := debug.ReadBuildInfo()
if strings.Contains(info.Main.Version, "+dirty") {
cfg.COMMIT += "+dirty"
}
_, err := fmt.Printf(versionFmt,
cfg.Version, cfg.BUILD, cfg.BRANCH, cfg.COMMIT, cfg.GOVERSION, cfg.APIVERSION)
return err
},
}
}
func Debug(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "debug",
Usage: "developer only",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "index",
Usage: "index to search within",
Sources: cli.EnvVars("ES_INDEX"),
Destination: &conf.Index,
Aliases: []string{"i"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.Debug(conf)
},
}
}
func HelpUsage(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "help-usage",
Usage: "show overview of all available commands",
UsageText: "help-usage [<filter>]",
Aliases: []string{"usage"},
CustomHelpTemplate: addReference(`<filter> implies -f`),
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "hidden",
Usage: "include hidden commands",
Destination: &conf.Hidden,
Aliases: []string{"H"},
},
&cli.BoolFlag{
Name: "full-commands",
Usage: "show full commands",
Destination: &conf.Force,
Aliases: []string{"f"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
maxCommandWidth := 0
filter := cmd.Args().Get(0)
if filter != "" {
conf.Force = true
}
// first pass, determine max command width
if err := walkVisible(conf, cmd.Root(), func(cmd *cli.Command) error {
path := cmd.Path()
size := len(path[len(path)-1])
if conf.Force {
path := strings.Join(cmd.Path(), " ")
size = len(path)
}
if size > maxCommandWidth {
maxCommandWidth = size
}
return nil
}); err != nil {
return err
}
maxCommandWidth += 4 // account for indent width
// second pass, build tree
return walkVisible(conf, cmd.Root(), func(cmd *cli.Command) error {
path := cmd.Path()
if filter != "" {
if !strings.Contains(strings.Join(path, " "), filter) {
return nil
}
}
command := path[len(path)-1]
if conf.Force {
command = strings.Join(cmd.Path(), " ")
}
if len(path) == 1 || strings.HasSuffix(command, "help") {
return nil
}
indent := strings.Repeat(" ", len(path[1:])-1)
space := strings.Repeat(" ", maxCommandWidth-(len(command)+len(indent)))
fmt.Printf("%s%s %s - %s\n", indent, command, space, cmd.Usage)
return nil
})
},
}
}
// copy of cmd.Walk() with the exception to skip hidden commands and its siblings
// see: https://github.com/urfave/cli/issues/2372
func walkVisible(conf *cfg.Config, cmd *cli.Command, fn func(*cli.Command) error) error {
if fn == nil {
return nil
}
if !conf.Hidden && cmd.Hidden {
return nil
}
if err := fn(cmd); err != nil {
return err
}
for _, sub := range cmd.Commands {
if err := walkVisible(conf, sub, fn); err != nil {
return err
}
}
return nil
}

View File

@@ -18,7 +18,7 @@ package cmd
import (
"context"
"errors"
"strings"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
@@ -26,12 +26,38 @@ import (
"github.com/urfave/cli/v3"
)
const SearchUsage = `field <sep> might be one of:
=: Must match
!=: Must not match
You can omit a field spec and thereby search across all fields.
By default all queries contribute to matches (logical AND), use
-O to apply a logical OR operator.
You can also search multiple fields by separating them with comma, eg:
user,group=root
Use filters to further restrict results, they must match literally.
For datetime range format refer to:
https://www.elastic.co/docs/reference/elasticsearch/rest-apis/common-options#date-math
For timestamp formats refer to:
https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/mapping-date-format`
const QlUsage = `ES|QL documentation:
https://www.elastic.co/docs/reference/query-languages/esql/esql-getting-started
See also: esctl help-esql`
func Search(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "search",
Aliases: []string{"/"},
Usage: "search within an index",
UsageText: "search [options] <field=pattern> ...",
Name: "search",
Aliases: []string{"/"},
Usage: "search within an index",
UsageText: "search [options] [<[field<sep>]pattern> ...]\n",
CustomHelpTemplate: addReference(SearchUsage),
Flags: []cli.Flag{
&cli.StringFlag{
@@ -43,33 +69,105 @@ func Search(conf *cfg.Config) *cli.Command {
},
&cli.IntFlag{
Name: "from",
Usage: "show results FROM (default 0)",
Usage: "show results starting at <from>",
Destination: &conf.From,
Value: 0,
Aliases: []string{"f"},
},
&cli.IntFlag{
Name: "to",
Usage: "show results to (default 10)",
Name: "len",
Usage: "number of results to show (-1: all[max:10k], caution: might be slow)",
Destination: &conf.To,
Value: 10,
Aliases: []string{"t"},
Value: 20,
Aliases: []string{"l"},
},
&cli.StringSliceFlag{
Name: "filter",
Usage: "additional filters. format: key=value",
Usage: "additional boolean filters. format: key=value",
Destination: &conf.Filter,
Aliases: []string{"F"},
},
&cli.StringFlag{
Name: "jsonpath",
Usage: "jsonPath filter (e.g. source.message)",
Destination: &conf.Path,
Aliases: []string{"p"},
},
&cli.StringFlag{
Name: "timerange",
Usage: "field:<date> to <date> (e.g. @timestamp:2026-05-05 to 2026-05-15)",
Destination: &conf.Range,
Aliases: []string{"r"},
},
&cli.StringFlag{
Name: "timestamp-format",
Usage: "a valid ES builtin timestamp or custom format",
Destination: &conf.TimestampFormat,
Value: "strict_date_hour_minute",
},
&cli.StringFlag{
Name: "sort-by",
Usage: "sort by a field",
Destination: &conf.SortBy,
Value: "@timestamp",
Aliases: []string{"k"},
},
&cli.BoolFlag{
Name: "ascending",
Usage: "sort in ascending order (default: descending)",
Destination: &conf.Ascending,
Aliases: []string{"a"},
},
&cli.BoolFlag{
Name: "tail",
Usage: "follow search live, like tail -f",
Destination: &conf.Tail,
Aliases: []string{"T"},
},
&cli.BoolFlag{
Name: "or",
Usage: "logical operator (default: and)",
Destination: &conf.Or,
Aliases: []string{"O"},
},
&cli.BoolFlag{
Name: "validate",
Usage: "validate search query",
Destination: &conf.Validate,
Aliases: []string{"v"},
},
&cli.BoolFlag{
Name: "explain",
Usage: "explain search query",
Destination: &conf.Explain,
Aliases: []string{"e"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
if args.Len() == 0 {
return errors.New("at least one query must be specified (format: field=pattern)")
if conf.To == -1 {
conf.To = 10000
}
return es.Search(conf, args.Slice())
},
}
}
func SearchQL(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "searchql",
Aliases: []string{"/"},
Usage: "search using ES/QL language",
UsageText: "search <ES/QL term>",
CustomHelpTemplate: addReference(QlUsage),
Action: func(ctx context.Context, cmd *cli.Command) error {
args := cmd.Args()
return es.SearchQL(conf, strings.Join(args.Slice(), " "))
},
}
}

135
cmd/shards.go Normal file
View File

@@ -0,0 +1,135 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"errors"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func Shard(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "shard",
Aliases: []string{"s"},
Usage: "manage shards",
Commands: []*cli.Command{
ShardList(conf),
ShardShow(conf),
ShardAllocation(conf),
},
}
}
func ShardList(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "list",
Aliases: []string{"ls"},
Usage: "list shards",
Flags: []cli.Flag{
&cli.IntFlag{
Name: "max",
Usage: "max number of items to process",
Destination: &conf.MaxItems,
Aliases: []string{"m"},
},
&cli.BoolFlag{
Name: "reds",
Usage: "include only failed shards",
Destination: &conf.Failed,
Aliases: []string{"r"},
},
&cli.BoolFlag{
Name: "primaries",
Usage: "show only primary shards",
Destination: &conf.Primary,
Aliases: []string{"p"},
},
&cli.BoolFlag{
Name: "verbose",
Usage: "show node name and ip as well",
Destination: &conf.Verbose,
Aliases: []string{"v"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.ShardList(conf)
},
}
}
func ShardShow(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "show",
Aliases: []string{"sh"},
Usage: "show details about a shard",
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.ShardShow(conf, index)
},
}
}
func ShardAllocation(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "allocation",
Aliases: []string{"a"},
Usage: "explain shard allocation",
Flags: []cli.Flag{
&cli.IntFlag{
Name: "shard",
Usage: "shard number to explain for",
Destination: &conf.Shards,
Aliases: []string{"s"},
},
&cli.BoolFlag{
Name: "primary",
Usage: "explain primary allocation (default true)",
Destination: &conf.Primary,
Aliases: []string{"p"},
},
&cli.StringFlag{
Name: "node",
Usage: "explain a shard only if it is currently located on the specified node name or node ID",
Destination: &conf.FromNode,
Aliases: []string{"n"},
},
},
Action: func(ctx context.Context, cmd *cli.Command) error {
index := cmd.Args().Get(0)
if index == "" {
return errors.New("no index specified")
}
return es.ShardAllocation(conf, cmd.Args().Get(0))
},
}
}

69
cmd/tasks.go Normal file
View File

@@ -0,0 +1,69 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"context"
"errors"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/es"
"github.com/urfave/cli/v3"
)
func Task(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "task",
Usage: "manage tasks",
Commands: []*cli.Command{
TaskList(conf),
TaskCancel(conf),
},
}
}
func TaskList(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "list",
Aliases: []string{"ls"},
Usage: "list tasks",
Action: func(ctx context.Context, cmd *cli.Command) error {
return es.TaskList(conf)
},
}
}
func TaskCancel(conf *cfg.Config) *cli.Command {
return &cli.Command{
Name: "cancel",
Aliases: []string{"rm"},
Usage: "cancel running task",
UsageText: "cancel <taskId>",
Action: func(ctx context.Context, cmd *cli.Command) error {
taskId := cmd.Args().Get(0)
if taskId == "" {
return errors.New("no taskId specified")
}
return es.TaskCancel(conf, cmd.Args().Get(0))
},
}
}

36
cmd/utilities.go Normal file
View File

@@ -0,0 +1,36 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"fmt"
"strings"
"github.com/urfave/cli/v3"
)
func addReference(ref string) string {
indentedRef := []string{}
for line := range strings.SplitSeq(ref, "\n") {
indentedRef = append(indentedRef, " "+line)
}
return fmt.Sprintf("%s\nREFERENCE:\n%s\n",
cli.SubcommandHelpTemplate,
strings.Join(indentedRef, "\n"))
}

92
go.mod
View File

@@ -14,35 +14,83 @@
// along with this program. If not, see <http://www.gnu.org/licenses/>.
module codeberg.org/scip/esctl
go 1.25.0
go 1.26
require (
github.com/alecthomas/repr v0.5.2 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/chzyer/readline v1.5.1 // indirect
github.com/clipperhouse/displaywidth v0.10.0 // indirect
github.com/clipperhouse/uax29/v2 v2.6.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/elastic/elastic-transport-go/v8 v8.11.0 // indirect
github.com/elastic/go-elasticsearch/v9 v9.3.2 // indirect
github.com/fatih/color v1.19.0 // indirect
codeberg.org/scip/mapmap v0.0.2
github.com/MichaelMure/go-term-markdown v0.1.4
github.com/alecthomas/repr v0.5.3
github.com/charmbracelet/bubbles v1.0.0
github.com/charmbracelet/bubbletea v1.3.10
github.com/charmbracelet/lipgloss v1.1.0
github.com/chzyer/readline v1.5.1
github.com/dustin/go-humanize v1.0.1
github.com/elastic/elastic-transport-go/v8 v8.11.0
github.com/elastic/go-elasticsearch/v9 v9.3.4
github.com/fatih/color v1.19.0
github.com/go-openapi/loads v0.24.0
github.com/go-openapi/spec v0.22.5
github.com/go-openapi/swag/loading v0.26.1
github.com/mattn/go-isatty v0.0.22
github.com/tidwall/gjson v1.19.0
github.com/tlinden/yadu v0.1.3
github.com/urfave/cli/v3 v3.10.1-0.20260623012112-f980ca84bf65
golang.org/x/term v0.44.0
gopkg.in/yaml.v3 v3.0.1
)
require (
github.com/MichaelMure/go-term-text v0.3.1 // indirect
github.com/alecthomas/chroma v0.7.1 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/charmbracelet/colorprofile v0.4.1 // indirect
github.com/charmbracelet/x/ansi v0.11.6 // indirect
github.com/charmbracelet/x/cellbuf v0.0.15 // indirect
github.com/charmbracelet/x/term v0.2.2 // indirect
github.com/clipperhouse/displaywidth v0.11.0 // indirect
github.com/clipperhouse/uax29/v2 v2.7.0 // indirect
github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964 // indirect
github.com/disintegration/imaging v1.6.2 // indirect
github.com/dlclark/regexp2 v1.11.0 // indirect
github.com/eliukblau/pixterm/pkg/ansimage v0.0.0-20191210081756-9fb6cf8c2f75 // indirect
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/lmittmann/tint v1.1.3 // indirect
github.com/go-openapi/analysis v0.25.2 // indirect
github.com/go-openapi/errors v0.22.8 // indirect
github.com/go-openapi/jsonpointer v0.23.1 // indirect
github.com/go-openapi/jsonreference v0.21.6 // indirect
github.com/go-openapi/strfmt v0.26.3 // indirect
github.com/go-openapi/swag/conv v0.26.1 // indirect
github.com/go-openapi/swag/jsonname v0.26.1 // indirect
github.com/go-openapi/swag/jsonutils v0.26.1 // indirect
github.com/go-openapi/swag/mangling v0.26.1 // indirect
github.com/go-openapi/swag/stringutils v0.26.1 // indirect
github.com/go-openapi/swag/typeutils v0.26.1 // indirect
github.com/go-openapi/swag/yamlutils v0.26.1 // indirect
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
github.com/gomarkdown/markdown v0.0.0-20191123064959-2c17d62f5098 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/kyokomi/emoji/v2 v2.2.8 // indirect
github.com/lucasb-eyer/go-colorful v1.4.0 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.21 // indirect
github.com/mattn/go-runewidth v0.0.19 // indirect
github.com/olekukonko/cat v0.0.0-20250911104152-50322a0618f6 // indirect
github.com/olekukonko/errors v1.2.0 // indirect
github.com/olekukonko/ll v0.1.6 // indirect
github.com/olekukonko/tablewriter v1.1.4 // indirect
github.com/tlinden/yadu v0.1.3 // indirect
github.com/urfave/cli/v3 v3.8.0 // indirect
github.com/mattn/go-localereader v0.0.1 // indirect
github.com/mattn/go-runewidth v0.0.23 // indirect
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.16.0 // indirect
github.com/oklog/ulid/v2 v2.1.1 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.0 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/otel v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f // indirect
golang.org/x/sys v0.42.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/image v0.0.0-20191206065243-da761ea9ff43 // indirect
golang.org/x/net v0.55.0 // indirect
golang.org/x/sys v0.46.0 // indirect
golang.org/x/text v0.37.0 // indirect
)

229
go.sum
View File

@@ -1,27 +1,66 @@
github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs=
github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
codeberg.org/scip/mapmap v0.0.2 h1:0i61jOUwFmGVwPukrMzrx0Fi4T9Qru2nlmibuaJimBo=
codeberg.org/scip/mapmap v0.0.2/go.mod h1:/ojYo2P7dMA2FWEu+jHKmsKPeq5yDcCvXeHevqZd5OI=
github.com/MichaelMure/go-term-markdown v0.1.4 h1:Ir3kBXDUtOX7dEv0EaQV8CNPpH+T7AfTh0eniMOtNcs=
github.com/MichaelMure/go-term-markdown v0.1.4/go.mod h1:EhcA3+pKYnlUsxYKBJ5Sn1cTQmmBMjeNlpV8nRb+JxA=
github.com/MichaelMure/go-term-text v0.3.1 h1:Kw9kZanyZWiCHOYu9v/8pWEgDQ6UVN9/ix2Vd2zzWf0=
github.com/MichaelMure/go-term-text v0.3.1/go.mod h1:QgVjAEDUnRMlzpS6ky5CGblux7ebeiLnuy9dAaFZu8o=
github.com/alecthomas/assert v0.0.0-20170929043011-405dbfeb8e38 h1:smF2tmSOzy2Mm+0dGI2AIUHY+w0BUc+4tn40djz7+6U=
github.com/alecthomas/assert v0.0.0-20170929043011-405dbfeb8e38/go.mod h1:r7bzyVFMNntcxPZXK3/+KdruV1H5KSlyVY0gc+NgInI=
github.com/alecthomas/chroma v0.7.1 h1:G1i02OhUbRi2nJxcNkwJaY/J1gHXj9tt72qN6ZouLFQ=
github.com/alecthomas/chroma v0.7.1/go.mod h1:gHw09mkX1Qp80JlYbmN9L3+4R5o6DJJ3GRShh+AICNc=
github.com/alecthomas/colour v0.0.0-20160524082231-60882d9e2721 h1:JHZL0hZKJ1VENNfmXvHbgYlbUOvpzYzvy2aZU5gXVeo=
github.com/alecthomas/colour v0.0.0-20160524082231-60882d9e2721/go.mod h1:QO9JBoKquHd+jz9nshCh40fOfO+JzsoXy8qTHF68zU0=
github.com/alecthomas/kong v0.2.1-0.20190708041108-0548c6b1afae/go.mod h1:+inYUSluD+p4L8KdviBSgzcqEjUQOfC5fQDRFuc36lI=
github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897/go.mod h1:xTS7Pm1pD1mvyM075QCDSRqH6qRLXylzS24ZTpRiSzQ=
github.com/alecthomas/repr v0.5.3 h1:Ebk3yZ0kvrHC7TkTHLDJGDq1LxKeu9sQBQREFMcesS8=
github.com/alecthomas/repr v0.5.3/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8=
github.com/charmbracelet/bubbles v1.0.0 h1:12J8/ak/uCZEMQ6KU7pcfwceyjLlWsDLAxB5fXonfvc=
github.com/charmbracelet/bubbles v1.0.0/go.mod h1:9d/Zd5GdnauMI5ivUIVisuEm3ave1XwXtD1ckyV6r3E=
github.com/charmbracelet/bubbletea v1.3.10 h1:otUDHWMMzQSB0Pkc87rm691KZ3SWa4KUlvF9nRvCICw=
github.com/charmbracelet/bubbletea v1.3.10/go.mod h1:ORQfo0fk8U+po9VaNvnV95UPWA1BitP1E0N6xJPlHr4=
github.com/charmbracelet/colorprofile v0.4.1 h1:a1lO03qTrSIRaK8c3JRxJDZOvhvIeSco3ej+ngLk1kk=
github.com/charmbracelet/colorprofile v0.4.1/go.mod h1:U1d9Dljmdf9DLegaJ0nGZNJvoXAhayhmidOdcBwAvKk=
github.com/charmbracelet/lipgloss v1.1.0 h1:vYXsiLHVkK7fp74RkV7b2kq9+zDLoEU4MZoFqR/noCY=
github.com/charmbracelet/lipgloss v1.1.0/go.mod h1:/6Q8FR2o+kj8rz4Dq0zQc3vYf7X+B0binUUBwA0aL30=
github.com/charmbracelet/x/ansi v0.11.6 h1:GhV21SiDz/45W9AnV2R61xZMRri5NlLnl6CVF7ihZW8=
github.com/charmbracelet/x/ansi v0.11.6/go.mod h1:2JNYLgQUsyqaiLovhU2Rv/pb8r6ydXKS3NIttu3VGZQ=
github.com/charmbracelet/x/cellbuf v0.0.15 h1:ur3pZy0o6z/R7EylET877CBxaiE1Sp1GMxoFPAIztPI=
github.com/charmbracelet/x/cellbuf v0.0.15/go.mod h1:J1YVbR7MUuEGIFPCaaZ96KDl5NoS0DAWkskup+mOY+Q=
github.com/charmbracelet/x/term v0.2.2 h1:xVRT/S2ZcKdhhOuSP4t5cLi5o+JxklsoEObBSgfgZRk=
github.com/charmbracelet/x/term v0.2.2/go.mod h1:kF8CY5RddLWrsgVwpw4kAa6TESp6EB5y3uxGLeCqzAI=
github.com/chzyer/logex v1.2.1 h1:XHDu3E6q+gdHgsdTPH6ImJMIp436vR6MPtH8gP05QzM=
github.com/chzyer/logex v1.2.1/go.mod h1:JLbx6lG2kDbNRFnfkgvh4eRJRPX1QCoOIWomwysCBrQ=
github.com/chzyer/readline v1.5.1 h1:upd/6fQk4src78LMRzh5vItIt361/o4uq553V8B5sGI=
github.com/chzyer/readline v1.5.1/go.mod h1:Eh+b79XXUwfKfcPLepksvw2tcLE/Ct21YObkaSkeBlk=
github.com/chzyer/test v1.0.0 h1:p3BQDXSxOhOG0P9z6/hGnII4LGiEPOYBhs8asl/fC04=
github.com/chzyer/test v1.0.0/go.mod h1:2JlltgoNkt4TW/z9V/IzDdFaMTM2JPIi26O1pF38GC8=
github.com/clipperhouse/displaywidth v0.10.0 h1:GhBG8WuerxjFQQYeuZAeVTuyxuX+UraiZGD4HJQ3Y8g=
github.com/clipperhouse/displaywidth v0.10.0/go.mod h1:XqJajYsaiEwkxOj4bowCTMcT1SgvHo9flfF3jQasdbs=
github.com/clipperhouse/uax29/v2 v2.6.0 h1:z0cDbUV+aPASdFb2/ndFnS9ts/WNXgTNNGFoKXuhpos=
github.com/clipperhouse/uax29/v2 v2.6.0/go.mod h1:Wn1g7MK6OoeDT0vL+Q0SQLDz/KpfsVRgg6W7ihQeh4g=
github.com/clipperhouse/displaywidth v0.11.0 h1:lBc6kY44VFw+TDx4I8opi/EtL9m20WSEFgwIwO+UVM8=
github.com/clipperhouse/displaywidth v0.11.0/go.mod h1:bkrFNkf81G8HyVqmKGxsPufD3JhNl3dSqnGhOoSD/o0=
github.com/clipperhouse/uax29/v2 v2.7.0 h1:+gs4oBZ2gPfVrKPthwbMzWZDaAFPGYK72F0NJv2v7Vk=
github.com/clipperhouse/uax29/v2 v2.7.0/go.mod h1:EFJ2TJMRUaplDxHKj1qAEhCtQPW2tJSwu5BF98AuoVM=
github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964 h1:y5HC9v93H5EPKqaS1UYVg1uYah5Xf51mBfIoWehClUQ=
github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964/go.mod h1:Xd9hchkHSWYkEqJwUGisez3G1QY8Ryz0sdWrLPMGjLk=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/disintegration/imaging v1.6.2 h1:w1LecBlG2Lnp8B3jk5zSuNqd7b4DXhcjwek1ei82L+c=
github.com/disintegration/imaging v1.6.2/go.mod h1:44/5580QXChDfwIclfc/PCwrr44amcmDAg8hxG0Ewe4=
github.com/dlclark/regexp2 v1.1.6/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/elastic/elastic-transport-go/v8 v8.9.0 h1:KeT/2P54F0xS0S8Y3Pf+tFDg4HmBgReQMB+BMz8dDAs=
github.com/elastic/elastic-transport-go/v8 v8.9.0/go.mod h1:ssMTvNS2hwf7CaiGsRRsx4gQHFZ/jS/DkLcISxekWzc=
github.com/elastic/elastic-transport-go/v8 v8.11.0 h1:taYmqC2M6+fZt/+W+ENYh/W5L9+KrlJGOSbEJs8egWc=
github.com/elastic/elastic-transport-go/v8 v8.11.0/go.mod h1:DZQ0szCNywc9F+C9l/Kkd4n69SvJVj0I3yK1Of7s3l8=
github.com/elastic/go-elasticsearch/v9 v9.3.2 h1:nvtvfN/Gsp/rzUPz/9yILwDAsYJ3s5L0VmhR16zPKCA=
github.com/elastic/go-elasticsearch/v9 v9.3.2/go.mod h1:ubKUMJCJbX5V/gW5MIn2NQZyaEZ61ubXwJmD5UMNrM8=
github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM=
github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE=
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
github.com/elastic/go-elasticsearch/v9 v9.3.4 h1:vnqXl6jnlA+ZwlfRaKF9BR9woZC1KSB17gtyvgf8zVU=
github.com/elastic/go-elasticsearch/v9 v9.3.4/go.mod h1:ubKUMJCJbX5V/gW5MIn2NQZyaEZ61ubXwJmD5UMNrM8=
github.com/eliukblau/pixterm/pkg/ansimage v0.0.0-20191210081756-9fb6cf8c2f75 h1:vbix8DDQ/rfatfFr/8cf/sJfIL69i4BcZfjrVOxsMqk=
github.com/eliukblau/pixterm/pkg/ansimage v0.0.0-20191210081756-9fb6cf8c2f75/go.mod h1:0gZuvTO1ikSA5LtTI6E13LEOdWQNjIo5MTQOvrV0eFg=
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f h1:Y/CXytFA4m6baUTXGLOoWe4PQhGxaX0KpnayAqC48p4=
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM=
github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL+zU=
github.com/fatih/color v1.19.0 h1:Zp3PiM21/9Ld6FzSKyL5c/BULoe/ONr9KlbYVOfG8+w=
github.com/fatih/color v1.19.0/go.mod h1:zNk67I0ZUT1bEGsSGyCZYZNrHuTkJJB+r6Q9VuMi0LE=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
@@ -29,47 +68,149 @@ github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/lmittmann/tint v1.1.3 h1:Hv4EaHWXQr+GTFnOU4VKf8UvAtZgn0VuKT+G0wFlO3I=
github.com/lmittmann/tint v1.1.3/go.mod h1:HIS3gSy7qNwGCj+5oRjAutErFBl4BzdQP6cJZ0NfMwE=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/go-openapi/analysis v0.25.2 h1:I0vy4n3alz+DHTiN1PRhCb7QZxkK6g5YmswZKv2TKuw=
github.com/go-openapi/analysis v0.25.2/go.mod h1:Uhs1t/2XR10EnwONYILGEzw8gcfGIG5Xk5K2AxnhqDo=
github.com/go-openapi/errors v0.22.8 h1:oP7sW7TWc3wFFjrzzj0nI83H2qMBkNjNfSd+XRejk/I=
github.com/go-openapi/errors v0.22.8/go.mod h1:BuUoHcYrU6E7V9gfj1I5wLQqgtIHnup/alXZ8KdgQ0w=
github.com/go-openapi/jsonpointer v0.23.1 h1:1HBACs7XIwR2RcmItfdSFlALhGbe6S92p0ry4d1GWg4=
github.com/go-openapi/jsonpointer v0.23.1/go.mod h1:iWRmZTrGn7XwYhtPt/fvdSFj1OfNBngqRT2UG3BxSqY=
github.com/go-openapi/jsonreference v0.21.6 h1:NZ5nGfnaM1n4I43Xjm1e5/M2GjOwQwndQz22uhxwD+Y=
github.com/go-openapi/jsonreference v0.21.6/go.mod h1:xzbgtQ3ZbWxvET3AxdzCJlJt6vkovbf+IfSPJjD0tUY=
github.com/go-openapi/loads v0.24.0 h1:4LLorXRPTzIN9V6ngMUZbAscsBOUBk3Oa8cClu/bFrQ=
github.com/go-openapi/loads v0.24.0/go.mod h1:xQMgX+hw5xRAhGrcDXxeMw78IFqUpIzhleu3HqPhyF4=
github.com/go-openapi/spec v0.22.5 h1:KhO7RBlKQfonUWX2WzQCoLIXVA6AcNqDGZ3a1Dutdlo=
github.com/go-openapi/spec v0.22.5/go.mod h1:vxpOtMya5TXtENXKE5bKqv5NjocVhyhxHrlZfvKnZ74=
github.com/go-openapi/strfmt v0.26.3 h1:rzmslHarJgBbf2qfGge+X3htclQfmXqBZMm0Too0HhU=
github.com/go-openapi/strfmt v0.26.3/go.mod h1:a5nsUw0oRpQzZeOwx8bi6cKbzFZslpbCKt1LEot+KnQ=
github.com/go-openapi/swag/conv v0.26.1 h1:slr5FVkg9Wc3Y5zcwenD8Sd/PQ94b2I/QJI7N7KTBpg=
github.com/go-openapi/swag/conv v0.26.1/go.mod h1:mvQXgPptZk9GTrFgGwWvT4q+dN+zQej9JfmGwnipz1A=
github.com/go-openapi/swag/jsonname v0.26.1 h1:VReupaV6WxlAsCn0e4DUfgV6bPmINnPpyJDLqSfNPcE=
github.com/go-openapi/swag/jsonname v0.26.1/go.mod h1:OvdW6BoWoj33pTfi7x9vFrgmT+fk7aw0BRwvCE0YOuc=
github.com/go-openapi/swag/jsonutils v0.26.1 h1:2hdBfFkHg+7Wrz2VsCbeyR6hzkRDs7AztnMR2u84yOY=
github.com/go-openapi/swag/jsonutils v0.26.1/go.mod h1:U+RMJH3wa+6BRiphuRtIyI8fW9HPFqFQ4sHk2oRx0UQ=
github.com/go-openapi/swag/jsonutils/fixtures_test v0.26.1 h1:1CD7NiLLb/TXl3tOnFYU4b+mNfb5rtgHkaA+q7RMYYQ=
github.com/go-openapi/swag/jsonutils/fixtures_test v0.26.1/go.mod h1:ZWafc8nMdYzTE3uYY6W86f0n46+IF0g4uUyRhJw/kXc=
github.com/go-openapi/swag/loading v0.26.1 h1:E9K4wqXeROlhjFQ13K9zMz6ojFGXIggGe+ad1odrK9w=
github.com/go-openapi/swag/loading v0.26.1/go.mod h1:3qvRIlWzWdq1HvmldwmuJ2ohpcAryN6xVt2OTKd0/7E=
github.com/go-openapi/swag/mangling v0.26.1 h1:gpYI4WuPKFJJVjV5cDLGlDVJhFIxYjQc7yN5eEb4CqM=
github.com/go-openapi/swag/mangling v0.26.1/go.mod h1:POETDH01hqAdASXfw7ISEd9bCOE6xBHOt8NHmGZRmYM=
github.com/go-openapi/swag/stringutils v0.26.1 h1:f88uYyTso7TnHrKM/bUBsQ5e2wKf37cpgo6pvbzd9yU=
github.com/go-openapi/swag/stringutils v0.26.1/go.mod h1:Sc6d3bU8fgk5AyZR8/8jEQ+Is/Ald+TD/IIggPN8UJk=
github.com/go-openapi/swag/typeutils v0.26.1 h1:yg42FgMzRR6PVQ3M3qHz1s+Y6/P4HoJ3cBarXa3OVnU=
github.com/go-openapi/swag/typeutils v0.26.1/go.mod h1:VfnV+oUtSP2vCSCn2aJgnr8OevUYemyIzzS1VOzS10o=
github.com/go-openapi/swag/yamlutils v0.26.1 h1:0TSLK+lXs9vfIhAWzBeI/lOzEnIoot6WTCO1aAeWFTk=
github.com/go-openapi/swag/yamlutils v0.26.1/go.mod h1:7W5b7PRX9MxwL7TjeG7H8HkyBGRsIDRObhyMWFgBI2M=
github.com/go-openapi/testify/enable/yaml/v2 v2.5.1 h1:q9NtHwK4qHF7yZziBPvZyv7zWAIk8ok88Gh2mR6Jpc8=
github.com/go-openapi/testify/enable/yaml/v2 v2.5.1/go.mod h1:JW0MXIotCYps/XsgJnG3a8Q7rE5xAiBwoOD5OfaIQBk=
github.com/go-openapi/testify/v2 v2.5.1 h1:TMdhCaw8fUNraVSf3Omoob1dO/AzBfhtFAPW0an6sBo=
github.com/go-openapi/testify/v2 v2.5.1/go.mod h1:SgsVHtfooshd0tublTtJ50FPKhujf47YRqauXXOUxfw=
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/gomarkdown/markdown v0.0.0-20191123064959-2c17d62f5098 h1:Qxs3bNRWe8GTcKMxYOSXm0jx6j0de8XUtb/fsP3GZ0I=
github.com/gomarkdown/markdown v0.0.0-20191123064959-2c17d62f5098/go.mod h1:aii0r/K0ZnHv7G0KF7xy1v0A7s2Ljrb5byB7MO5p6TU=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kyokomi/emoji/v2 v2.2.8 h1:jcofPxjHWEkJtkIbcLHvZhxKgCPl6C7MyjTrD4KDqUE=
github.com/kyokomi/emoji/v2 v2.2.8/go.mod h1:JUcn42DTdsXJo1SWanHh4HKDEyPaR5CqkmoirZZP9qE=
github.com/lucasb-eyer/go-colorful v1.0.3/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/lucasb-eyer/go-colorful v1.4.0 h1:UtrWVfLdarDgc44HcS7pYloGHJUjHV/4FwW4TvVgFr4=
github.com/lucasb-eyer/go-colorful v1.4.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.21 h1:xYae+lCNBP7QuW4PUnNG61ffM4hVIfm+zUzDuSzYLGs=
github.com/mattn/go-isatty v0.0.21/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4=
github.com/mattn/go-runewidth v0.0.19 h1:v++JhqYnZuu5jSKrk9RbgF5v4CGUjqRfBm05byFGLdw=
github.com/mattn/go-runewidth v0.0.19/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/olekukonko/cat v0.0.0-20250911104152-50322a0618f6 h1:zrbMGy9YXpIeTnGj4EljqMiZsIcE09mmF8XsD5AYOJc=
github.com/olekukonko/cat v0.0.0-20250911104152-50322a0618f6/go.mod h1:rEKTHC9roVVicUIfZK7DYrdIoM0EOr8mK1Hj5s3JjH0=
github.com/olekukonko/errors v1.2.0 h1:10Zcn4GeV59t/EGqJc8fUjtFT/FuUh5bTMzZ1XwmCRo=
github.com/olekukonko/errors v1.2.0/go.mod h1:ppzxA5jBKcO1vIpCXQ9ZqgDh8iwODz6OXIGKU8r5m4Y=
github.com/olekukonko/ll v0.1.6 h1:lGVTHO+Qc4Qm+fce/2h2m5y9LvqaW+DCN7xW9hsU3uA=
github.com/olekukonko/ll v0.1.6/go.mod h1:NVUmjBb/aCtUpjKk75BhWrOlARz3dqsM+OtszpY4o88=
github.com/olekukonko/tablewriter v1.1.4 h1:ORUMI3dXbMnRlRggJX3+q7OzQFDdvgbN9nVWj1drm6I=
github.com/olekukonko/tablewriter v1.1.4/go.mod h1:+kedxuyTtgoZLwif3P1Em4hARJs+mVnzKxmsCL/C5RY=
github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
github.com/mattn/go-isatty v0.0.22 h1:j8l17JJ9i6VGPUFUYoTUKPSgKe/83EYU2zBC7YNKMw4=
github.com/mattn/go-isatty v0.0.22/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4=
github.com/mattn/go-localereader v0.0.1 h1:ygSAOl7ZXTx4RdPYinUpg6W99U8jWvWi9Ye2JC/oIi4=
github.com/mattn/go-localereader v0.0.1/go.mod h1:8fBrzywKY7BI3czFoHkuzRoWE9C+EiG4R1k4Cjx5p88=
github.com/mattn/go-runewidth v0.0.12/go.mod h1:RAqKPSqVFrSLVXbA8x7dzmKdmGzieGRCM46jaSJTDAk=
github.com/mattn/go-runewidth v0.0.23 h1:7ykA0T0jkPpzSvMS5i9uoNn2Xy3R383f9HDx3RybWcw=
github.com/mattn/go-runewidth v0.0.23/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 h1:ZK8zHtRHOkbHy6Mmr5D264iyp3TiX5OmNcI5cIARiQI=
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6/go.mod h1:CJlz5H+gyd6CUWT45Oy4q24RdLyn7Md9Vj2/ldJBSIo=
github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELUXHmA=
github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIfMUZ4EgX0QYo=
github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc=
github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk=
github.com/oklog/ulid/v2 v2.1.1 h1:suPZ4ARWLOJLegGFiZZ1dFAkqzhMjL3J1TzI+5wHz8s=
github.com/oklog/ulid/v2 v2.1.1/go.mod h1:rcEKHmBBKfef9DhnvX7y1HZBYxjXb0cP5ExxNsTT1QQ=
github.com/pborman/getopt v0.0.0-20170112200414-7148bc3a4c30/go.mod h1:85jBQOZwpVEaDAr341tbn15RS4fCAsIst0qp7i8ex1o=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o=
github.com/sergi/go-diff v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/tidwall/gjson v1.19.0 h1:xwxm7n691Uf3u5OFjzngavjGTh55KX5q/9w9xHW88JU=
github.com/tidwall/gjson v1.19.0/go.mod h1:V37/opeE/JbLUOfH0QTXiNez2l0RUjYUhpT4szFQAfc=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tlinden/yadu v0.1.3 h1:5cRCUmj+l5yvlM2irtpFBIJwVV2DPEgYSaWvF19FtcY=
github.com/tlinden/yadu v0.1.3/go.mod h1:l3bRmHKL9zGAR6pnBHY2HRPxBecf7L74BoBgOOpTcUA=
github.com/urfave/cli/v3 v3.8.0 h1:XqKPrm0q4P0q5JpoclYoCAv0/MIvH/jZ2umzuf8pNTI=
github.com/urfave/cli/v3 v3.8.0/go.mod h1:ysVLtOEmg2tOy6PknnYVhDoouyC/6N42TMeoMzskhso=
github.com/urfave/cli/v3 v3.10.1-0.20260623012112-f980ca84bf65 h1:NXitXXO9DupDLEFj8hIyYtsubRxSFTp5JXqohElT0nU=
github.com/urfave/cli/v3 v3.10.1-0.20260623012112-f980ca84bf65/go.mod h1:ysVLtOEmg2tOy6PknnYVhDoouyC/6N42TMeoMzskhso=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/otel v1.35.0 h1:xKWKPxrxB6OtMCbmMY021CqC45J+3Onta9MqjhnusiQ=
go.opentelemetry.io/otel v1.35.0/go.mod h1:UEqy8Zp11hpkUrL73gSlELM0DupHoiq72dR+Zqel/+Y=
go.opentelemetry.io/otel/metric v1.35.0 h1:0znxYu2SNyuMSQT4Y9WDWej0VpcsxkuklLa4/siN90M=
go.opentelemetry.io/otel/metric v1.35.0/go.mod h1:nKVFgxBZ2fReX6IlyW28MgZojkoAkJGaE8CpgeAU3oE=
go.opentelemetry.io/otel/sdk v1.29.0 h1:vkqKjk7gwhS8VaWb0POZKmIEDimRCMsopNYnriHyryo=
go.opentelemetry.io/otel/sdk v1.29.0/go.mod h1:pM8Dx5WKnvxLCb+8lG1PRNIDxu9g9b9g59Qr7hfAAok=
go.opentelemetry.io/otel/trace v1.35.0 h1:dPpEfJu1sDIqruz7BHFG3c7528f6ddfSWfFDVt/xgMs=
go.opentelemetry.io/otel/trace v1.35.0/go.mod h1:WUk7DtFp1Aw2MkvqGdwiXYDZZNvA/1J8o6xRXLrIkyc=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/dl v0.0.0-20190829154251-82a15e2f2ead/go.mod h1:IUMfjQLJQd4UTqG1Z90tenwKoCX93Gn3MAQJMOSBsDQ=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/exp v0.0.0-20231006140011-7918f672742d h1:jtJma62tbqLibJ5sFQz8bKtEM8rJBtfilJ2qTU199MI=
golang.org/x/exp v0.0.0-20231006140011-7918f672742d/go.mod h1:ldy0pHrwJyGW56pPQzzkH36rKxoZW1tw7ZJpeKx+hdo=
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/image v0.0.0-20191206065243-da761ea9ff43 h1:gQ6GUSD102fPgli+Yb4cR/cGaHF7tNBt+GYoRCpGC7s=
golang.org/x/image v0.0.0-20191206065243-da761ea9ff43/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
golang.org/x/sys v0.0.0-20181128092732-4ed8d59d0b35/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.44.0 h1:0rLvDRCtNj0gZkyIXhCyOb2OAzEhLVqc4B+hrsBhrmc=
golang.org/x/term v0.44.0/go.mod h1:7ze4MdzUzLXpSAoFP1H0bOI9aXDqveSvatT5vKcFh2Y=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

235
pkg/cfg/cluster.go Normal file
View File

@@ -0,0 +1,235 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cfg
import (
"crypto/tls"
"errors"
"fmt"
"net"
"net/http"
"os"
"strings"
"syscall"
"time"
"github.com/elastic/elastic-transport-go/v8/elastictransport"
"github.com/elastic/go-elasticsearch/v9"
"golang.org/x/term"
"gopkg.in/yaml.v3"
)
// used in general config struct
type Cluster struct {
Name, Uri, User, Pass, Token string
client *elasticsearch.TypedClient
Default, DebugHTTP bool
}
// used just for writing back to the config file
type ClusterConfig struct {
Uri, User, Pass, Token string
Default bool
}
// to write the config, we avoid all other config settings
type WriteConfig struct {
Clusters map[string]*ClusterConfig
}
func (cluster *Cluster) SetClient(client *elasticsearch.TypedClient) {
cluster.client = client
}
// return the go-elasticsearch client object but before doing that,
// check if we need to tune auth
func (cluster *Cluster) ES() *elasticsearch.TypedClient {
if cluster.client == nil {
fmt.Println("no current cluster, use 'esctl cluster switch <name>' to set one")
os.Exit(1)
}
if err := cluster.CheckAuth(); err != nil {
fmt.Printf("Error: %s", err)
os.Exit(1)
}
return cluster.client
}
// add authentication to es client, if not yet done
func (cluster *Cluster) CheckAuth() error {
if cluster.Pass == "" && cluster.User != "" && cluster.Token == "" && cluster.Default {
// no token - user is set, but no password.
// check if the env var is set
pass := os.Getenv("ES_PASS")
if pass != "" {
cluster.Pass = pass
} else {
// k, try interactively
fmt.Fprintf(os.Stderr, "Enter password for elasticsearch user %s@%s: ", cluster.User, cluster.Name)
pass, err := term.ReadPassword(syscall.Stdin)
if err != nil {
return err
}
passwd := strings.TrimSpace(string(pass))
if passwd == "" {
return errors.New("password empty")
}
cluster.Pass = string(pass)
fmt.Println()
}
opts := cluster.getDefaultOptions()
opts = append(opts, elasticsearch.WithBasicAuth(cluster.User, cluster.Pass))
es, err := elasticsearch.NewTyped(opts...)
if err != nil {
return fmt.Errorf("failed to setup elasticsearch connection: %w", err)
}
cluster.SetClient(es)
}
return nil
}
// set Default=true for the given cluster in the config (if exists)
func (conf *Config) SwitchCluster(name string) error {
_, exists := conf.Clusters[name]
if !exists {
return errors.New("no cluster with that name configured")
}
cfg := WriteConfig{Clusters: map[string]*ClusterConfig{}}
for clustername, cluster := range conf.Clusters {
cfg.Clusters[clustername] = &ClusterConfig{
Uri: cluster.Uri,
User: cluster.User,
Pass: cluster.Pass,
Token: cluster.Token,
Default: false,
}
if clustername == name {
cfg.Clusters[clustername].Default = true
}
}
raw, err := yaml.Marshal(cfg)
if err != nil {
return fmt.Errorf("failed to marshal cluster config: %w", err)
}
outfile := getDefaultPath()
if conf.ConfigFile != "" {
outfile = conf.ConfigFile
}
if err := os.WriteFile(outfile, raw, 0600); err != nil {
return err
}
return nil
}
// We do NOT use go-elasticsearch to check for cluster reachability,
// because at this stage, auth may not have been configured. So
// instead we just connect to the cluster using plan net/tcp
func (cluster *Cluster) IsReachable() (bool, error) {
timeout := 500 * time.Millisecond
url := strings.TrimPrefix(strings.TrimPrefix(cluster.Uri, "https://"), "http://")
host := strings.Split(url, "/")
if !strings.Contains(host[0], ":") {
host[0] += ":443"
}
conn, err := net.DialTimeout("tcp", host[0], timeout)
if err != nil {
return false, err
}
return true, conn.Close()
}
func (cluster *Cluster) getDefaultOptions() []elasticsearch.Option {
// These headers are not needed with ES 9, but with ES 8, we set
// them here so every API call uses it. The only exception being
// the api repl, which does it on its own.
headers := http.Header{}
headers.Add("Content-Type", "application/json")
headers.Add("Accept", "application/json")
return []elasticsearch.Option{
elasticsearch.WithAddresses(cluster.Uri),
elasticsearch.WithTransportOptions(
cluster.getTransport(),
elastictransport.WithHeader(headers),
),
elasticsearch.WithCompatibilityMode(),
}
}
func (cluster *Cluster) getTransport() elastictransport.Option {
transport := new(http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
})
if cluster.DebugHTTP {
return elastictransport.WithTransport(
new(DebugTransport{Transport: transport}),
)
}
return elastictransport.WithTransport(
new(CompatibilityTransport{Transport: transport}),
)
}
func (conf *Config) SetupES() error {
for name, cluster := range conf.Clusters {
cluster.Name = name
cluster.DebugHTTP = conf.DebugHTTP
opts := cluster.getDefaultOptions()
switch {
case cluster.Pass != "" && cluster.User != "":
opts = append(opts, elasticsearch.WithBasicAuth(cluster.User, cluster.Pass))
case cluster.Token != "":
opts = append(opts, elasticsearch.WithAPIKey(cluster.Token))
}
es, err := elasticsearch.NewTyped(opts...)
if err != nil {
return fmt.Errorf("failed to setup elasticsearch connection: %w", err)
}
cluster.SetClient(es)
}
return nil
}

View File

@@ -17,57 +17,127 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
package cfg
import (
"context"
"crypto/tls"
"errors"
"fmt"
"net/http"
"os"
"path/filepath"
"reflect"
"time"
"github.com/alecthomas/repr"
"github.com/elastic/elastic-transport-go/v8/elastictransport"
"github.com/elastic/go-elasticsearch/v9"
"gopkg.in/yaml.v3"
)
const (
Version string = `v0.0.9`
Version string = `v0.0.27`
)
type Cluster struct {
Uri, User, Pass string
ES *elasticsearch.TypedClient
}
var (
// initialized during build, see Makefile:buildlocal
APIVERSION, GOVERSION, BUILD, COMMIT, BRANCH string
)
type Config struct {
ConfigFile string // -c
CurrentCluster string // -C
Debug bool // -d
Clusters map[string]*Cluster
DefaultCluster *Cluster
Index string // index: -i
Failed, Partials bool // index: flags
Shards, Replicas int // index create: -s -r
Wait bool // index create: -w
From, To, MaxItems int // search: flags
Filter []string // search: -F
Exclude string // cluster compare: -e (regexp)
All, Verbose bool // cluster status: -a -v
Persistent, Transient, Default bool // -p -t -D cluster settings set
Force bool // ccr follower renew: -f
ConfigFile string // -c
CurrentCluster string // -C
Debug bool // -d
Output string // -o <mode>
Clusters map[string]*Cluster
DefaultCluster *Cluster
HaveJQ bool // determined at runtime by ourselfes
ProfileFile string // for internal use (golang profiling)
AlignInts bool // -I
Index string // index: -i
Failed, Partials bool // index: flags
Shards, Replicas int // index create+allocation: -s -r
Wait bool // index create: -w
Policy string // index create: -p
Primary bool // index allocation: -p
Searchable bool // index fields: -s
Aggretable bool // index fields: -a
Priority int // index template create: -p
Settings []string // index template create: -s
Patterns []string // index template create: -i
Components []string // index template create: -c
Meta []string // index template create: -M
Aliases []string // index template create: -A
Stream bool // index template create: -S
AutoCreate bool // index template create: -a
Mode string // index template create: -a
Retention string // index template create: -r
Rollover bool // index template create: -R
Missing bool // index copy: -m
Sync bool // index copy: -S
RequestsPerSecond float64 // index copy: -R
Timeout time.Duration // index copy: -T
Refresh bool // index copy: -r
SourceIndices []string // index copy: -s
From, To, MaxItems int // search: flags
Filter []string // search: -F
Path string // search+doc sh: -p
Subhelp bool // search+doc sh: -H
Tail bool // search: -f [tail]
Or bool // search: -O
Range string // search: -r
TimestampFormat string // search: --timestamp-format
Explain bool // search: -e
Validate bool // search: --validate
SortBy string // sort: -k
Ascending bool // sort: -a
All bool // doc + node ls: -a
Exclude string // cluster compare: -e (regexp)
Verbose bool // cluster status: -v
Persistent, Transient, Default bool // cluster settings set: -p -t -D
Force bool // ccr follower renew: -f
DebugHTTP bool // root: --debug-http
DebugGoRoutines bool // root: --debug-goroutines
Separator string // role diff: -s
NotDeployed bool // role diff: -n
Undefined bool // role diff: -u
Diff bool // role diff: -D
Hidden bool // ds ls: -H
// rollover
MaxAge string
MaxDocs int64 // roll over, plus others
MaxShardSize, MaxShardDocs int // roll over
DryRun bool // rollover: -n
Tag string // api ls: -t
HumanCat bool // api repl: -H
Ilm Ilm // ilm create
FromNode, ToNode string // cluster reroute move: -f + -t
AllowPrimary, AcceptDataLoss bool // cluster reroute cancel: -p,-a
Pager string // api repl: -p || PAGER
}
func NewConfig() *Config {
return &Config{Clusters: map[string]*Cluster{}}
return new(Config{Clusters: map[string]*Cluster{}})
}
func getDefaultPath() string {
return filepath.Join([]string{os.Getenv("HOME"), ".config", "esctl", "config.yaml"}...)
}
func (conf *Config) Init() error {
DefaultConfig := os.Getenv("HOME") + "/.config/esctl/config.yaml"
DefaultConfig := getDefaultPath()
if conf.ConfigFile == "" && fileExists(DefaultConfig) {
conf.ConfigFile = DefaultConfig
}
switch {
case fileExists(DefaultConfig):
conf.ConfigFile = DefaultConfig
fallthrough
case conf.ConfigFile != "":
if err := conf.LoadConfig(); err != nil {
return err
@@ -82,50 +152,44 @@ func (conf *Config) Init() error {
return err
}
if conf.CurrentCluster != "" {
current, exists := conf.Clusters[conf.CurrentCluster]
if !exists {
return fmt.Errorf("no cluster with alias %s configured", conf.CurrentCluster)
} else {
conf.DefaultCluster = current
}
} else {
if len(conf.Clusters) == 1 {
for name, cluster := range conf.Clusters {
conf.DefaultCluster = cluster
conf.CurrentCluster = name
}
} else {
for name, cluster := range conf.Clusters {
_, err := cluster.ES.Cluster.Health().
Header("content-type", "application/json").
Header("accept", "application/json").
Do(context.Background())
if err == nil {
conf.DefaultCluster = cluster
conf.CurrentCluster = name
}
}
}
if err := conf.determineDefaultCluster(); err != nil {
return err
}
conf.HaveJQ = isJQinstalled()
conf.PrintDebug()
return nil
}
// we are using reflect to clone a config obj w/o the ES stuff for
// shorter repr.Println() output (the ES structure is just too large)
func (conf *Config) Clone() Config {
clone := Config{}
ref := reflect.ValueOf(*conf)
typeOfS := ref.Type()
for idx := range ref.NumField() {
field := typeOfS.Field(idx).Name
if field == "Clusters" || field == "DefaultCluster" || !ref.Field(idx).CanInterface() {
continue
}
reflect.ValueOf(&clone).Elem().FieldByName(field).Set(reflect.ValueOf(ref.Field(idx).Interface()))
}
return clone
}
func (conf *Config) PrintDebug() {
if !conf.Debug {
return
}
clone := *conf
for name := range clone.Clusters {
clone.Clusters[name] = nil
}
clone.DefaultCluster = nil
clone := conf.Clone()
fmt.Println("config:")
repr.Println(clone)
@@ -133,18 +197,17 @@ func (conf *Config) PrintDebug() {
func (conf *Config) LoadEnv() error {
cluster := Cluster{
Uri: os.Getenv("ES_URI"),
User: os.Getenv("ES_USER"),
Pass: os.Getenv("ES_PASS"),
Uri: os.Getenv("ES_URI"),
User: os.Getenv("ES_USER"),
Pass: os.Getenv("ES_PASS"),
Token: os.Getenv("ES_TOKEN"),
}
switch {
case cluster.Uri == "":
return errors.New("ES_URI unset")
case cluster.User == "":
return errors.New("ES_USER unset")
case cluster.Pass == "":
return errors.New("ES_PASS unset")
case cluster.User == "" || cluster.Token == "":
return errors.New("ES_USER and ES_TOKEN unset")
}
conf.Clusters["default"] = &cluster
@@ -163,7 +226,7 @@ func (conf *Config) LoadConfig() error {
return fmt.Errorf("failed to read config file: %w", err)
}
newconf := &Config{}
newconf := new(Config{})
err = yaml.Unmarshal(data, newconf)
if err != nil {
@@ -173,40 +236,56 @@ func (conf *Config) LoadConfig() error {
if len(newconf.Clusters) > 0 {
conf.Clusters = newconf.Clusters
_, exists := conf.Clusters["default"]
if !exists {
// no "default", just use the first we stumble upon
for _, cluster := range conf.Clusters {
for _, cluster := range conf.Clusters {
if cluster.Default {
conf.DefaultCluster = cluster
break
}
} else {
conf.DefaultCluster = newconf.Clusters["default"]
}
if conf.DefaultCluster == nil {
conf.DefaultCluster = &Cluster{}
}
}
return nil
}
func (conf *Config) SetupES() error {
for _, cluster := range conf.Clusters {
es, err := elasticsearch.NewTyped(
elasticsearch.WithAddresses(cluster.Uri),
elasticsearch.WithBasicAuth(cluster.User, cluster.Pass),
elasticsearch.WithTransportOptions(
elastictransport.WithTransport(
&http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
),
),
)
func (conf *Config) determineDefaultCluster() error {
if conf.CurrentCluster != "" {
// -C specified, set current cluster explicitly, no matter what the config says
current, exists := conf.Clusters[conf.CurrentCluster]
if !exists {
return fmt.Errorf("no cluster with alias %s configured", conf.CurrentCluster)
} else {
conf.DefaultCluster = current
if err != nil {
return fmt.Errorf("failed to setup elasticsearch connection: %w", err)
// disable all others
for _, cluster := range conf.Clusters {
cluster.Default = false
}
conf.DefaultCluster.Default = true
}
} else {
// we need to determine ourselfes
if len(conf.Clusters) == 1 {
// ok, just one cluster configured, use this, of course
for name, cluster := range conf.Clusters {
conf.DefaultCluster = cluster
conf.CurrentCluster = name
conf.DefaultCluster.Default = true
}
} else {
// multiple ones exists, look if one is set as default
for name, cluster := range conf.Clusters {
if cluster.Default {
conf.DefaultCluster = cluster
conf.CurrentCluster = name
}
}
}
cluster.ES = es
}
return nil
@@ -214,7 +293,6 @@ func (conf *Config) SetupES() error {
func fileExists(filename string) bool {
info, err := os.Stat(filename)
if err != nil {
// return false on any error
return false

76
pkg/cfg/ilm.go Normal file
View File

@@ -0,0 +1,76 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cfg
type Ilm struct {
Description string
HotMinAge string
HotRolloverMaxAge string
HotRolloverMaxPrimaryShardSize string
HotRolloverMaxDocs int64
HotReadonly bool
WarmMinAge string
WarmForceMerge int
WarmShrinkShards int
WarmPriority int
ColdMinAge string
ColdSearchableSnapshotRepo string
ColdForceMerge int
ColdPriority int
FrozenMinAge string
FrozenSearchableSnapshotRepo string
DeleteMinAge string
DeleteSearchableSnapshots bool
MinAge, FromPhase, Within string // forecast
Tree bool
}
func (cfg *Ilm) HaveHot() bool {
return cfg.HotMinAge != "" ||
cfg.HotRolloverMaxAge != "" ||
cfg.HotRolloverMaxDocs > 0 ||
cfg.HotRolloverMaxPrimaryShardSize != "" ||
cfg.HotReadonly
}
func (cfg *Ilm) HaveWarm() bool {
return cfg.WarmMinAge != "" ||
cfg.WarmForceMerge > 0 ||
cfg.WarmShrinkShards > 0 ||
cfg.WarmPriority > 0
}
func (cfg *Ilm) HaveCold() bool {
return cfg.ColdMinAge != "" ||
cfg.ColdSearchableSnapshotRepo != "" ||
cfg.ColdForceMerge > 0 ||
cfg.ColdPriority > 0
}
func (cfg *Ilm) HaveFrozen() bool {
return cfg.FrozenMinAge != ""
}
func (cfg *Ilm) HaveDelete() bool {
return cfg.DeleteSearchableSnapshots || cfg.DeleteMinAge != ""
}

39
pkg/cfg/jq.go Normal file
View File

@@ -0,0 +1,39 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cfg
import (
"context"
"os/exec"
"strings"
)
func isJQinstalled() bool {
cmd := exec.CommandContext(context.Background(), "jq", "-h")
out, err := cmd.Output()
if err != nil {
return false
}
if strings.Contains(string(out), "Usage") {
return true
}
return false
}

49
pkg/cfg/term.go Normal file
View File

@@ -0,0 +1,49 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cfg
import (
"os"
"golang.org/x/term"
)
const (
DefaultMargin = 4
)
func GetTermWidth() int {
if term.IsTerminal(int(os.Stdout.Fd())) {
width, _, err := term.GetSize(int(os.Stdout.Fd()))
if err == nil {
return width - DefaultMargin
}
}
return 80
}
func GetTermHeight() int {
if term.IsTerminal(int(os.Stdout.Fd())) {
_, height, err := term.GetSize(int(os.Stdout.Fd()))
if err == nil {
return height
}
}
return 25
}

93
pkg/cfg/transport.go Normal file
View File

@@ -0,0 +1,93 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cfg
import (
"bytes"
"encoding/json"
"fmt"
"log/slog"
"net/http"
)
// used to print uri, path and body of a request made by the go-client
type DebugTransport struct {
Transport http.RoundTripper
}
func (t *DebugTransport) RoundTrip(req *http.Request) (*http.Response, error) {
content := ""
contentline := ""
if req.ContentLength > 0 {
buf := new(bytes.Buffer)
body, _ := req.GetBody()
_, err := buf.ReadFrom(body)
if err != nil {
return nil, err
}
var pretty bytes.Buffer
err = json.Indent(&pretty, buf.Bytes(), "", "\t")
if err != nil {
return nil, fmt.Errorf("json parse error: %w", err)
}
content = pretty.String()
contentline = buf.String()
}
req = fixRequestHeaders(req)
slog.Info("req", "host", req.URL.Host, "uri", req.URL.Path,
"body", content, "bodyline", contentline,
"headers", req.Header,
)
return t.Transport.RoundTrip(req)
}
type CompatibilityTransport struct {
Transport http.RoundTripper
}
func (t *CompatibilityTransport) RoundTrip(req *http.Request) (*http.Response, error) {
req = fixRequestHeaders(req)
return t.Transport.RoundTrip(req)
}
// Fixes https://codeberg.org/scip/esctl/issues/79:
// the API client has this Accept header hardcoded everywhere:
// req.Header.Set("Accept", "application/vnd.elasticsearch+json;compatible-with=9")
// While this works pretty well with ES9, it doesn't with ES8. So, we replace
// this header with a new one without the compatibility part.
func fixRequestHeaders(req *http.Request) *http.Request {
if req.Header.Get("Accept") != "" {
req.Header.Del("Accept")
req.Header.Add("Accept", "application/json")
}
if req.Header.Get("Content-Type") != "" {
req.Header.Del("Content-Type")
req.Header.Add("Content-Type", "application/json")
}
return req
}

591
pkg/es/api.go Normal file
View File

@@ -0,0 +1,591 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"bufio"
"bytes"
"context"
"crypto/tls"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"log/slog"
"maps"
"net/http"
"os"
"os/exec"
"regexp"
"slices"
"strings"
"codeberg.org/scip/esctl/assets"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
markdown "github.com/MichaelMure/go-term-markdown"
"github.com/charmbracelet/lipgloss"
"github.com/chzyer/readline"
"github.com/go-openapi/spec"
)
const (
intro = `# Input format: verb path [data]"
#
# Example:
#
# post /yourindex/_ccr/pause_follow
# put /yourindex/_settings {"number_of_replicas": 1}
#
# You can also put multiline JSON after the path like:
#
# put /yourindex/_settings
# {
# "number_of_replicas": 1
# }
#
# If you do NOT supply a JSON in the first line, you need to hit ENTER
# twice to complete.
#
# Supply the flag --human-readable-cat, -H to view /_cat API calls in
# human readable form.`
)
// holds an API operation via go-openapi/spec
type Op struct {
Verb string
Op *spec.Operation
}
type Param struct {
Description, Param string
}
// API operation parameter type
type Params struct {
Path []Param
Query []Param
}
func ApiRepl(conf *cfg.Config) error {
verbs := []string{"post", "get", "put", "delete"}
fmt.Println(intro)
fmt.Println()
reader, err := readline.NewEx(&readline.Config{
Prompt: "> ",
HistoryFile: os.Getenv("HOME") + "/.config/esctl/history",
HistoryLimit: 500,
InterruptPrompt: "^C",
EOFPrompt: "exit",
HistorySearchFold: true,
})
if err != nil {
return fmt.Errorf("failed to initialize readline lib: %w", err)
}
for {
text, err := reader.Readline()
if err != nil {
break
}
text = strings.TrimSpace(text)
if text == "" {
continue
}
parts := strings.SplitN(strings.TrimSpace(text), " ", 3)
if len(parts) < 2 {
fmt.Println("error: you need to input a verb, uri [and post data]")
continue
}
if !slices.Contains(verbs, strings.ToLower(parts[0])) {
fmt.Println("error: verb must be one of " + strings.Join(verbs, ","))
continue
}
if !strings.HasPrefix(parts[1], "/") {
parts[1] = "/" + parts[1]
}
json := ""
if len(parts) == 3 {
// put /uri {json}
json = parts[2]
}
// put /uri/<Ret> [json]
data, err := readJSON(json)
if err != nil {
fmt.Println(err)
}
raw, err := CallAPI(conf, parts[0], parts[1], data)
if err != nil {
fmt.Printf("failed to call API: %s\n", esErrorString(err))
}
if conf.HumanCat && strings.HasPrefix(parts[1], "/_cat") {
fmt.Println(string(raw))
} else {
pageJsonOutput(conf, raw)
}
}
//nolint:nilerr
return nil
}
func pageJsonOutput(conf *cfg.Config, raw []byte) {
tmpconf := new(cfg.Config{HaveJQ: conf.HaveJQ})
if conf.Pager != "" {
tmpconf.HaveJQ = false
}
output, err := prettyfiJson(tmpconf, raw)
if err != nil {
fmt.Println(err)
}
lines := len(strings.Split(output, "\n"))
height := cfg.GetTermHeight()
if lines > height {
if conf.Pager != "" {
cmd := strings.Split(conf.Pager, " ")
pager := exec.Command(cmd[0], cmd[1:]...)
var buf bytes.Buffer
buf.WriteString(output)
pager.Stdout = os.Stdout
pager.Stdin = &buf
pager.Stderr = os.Stderr
err := pager.Run()
if err != nil {
fmt.Printf("failed to execute pager '%s': %s", conf.Pager, err)
}
} else {
printer.Pager("json output", output)
}
return
}
fmt.Println(output)
}
func encodeAuth(username, password string) string {
return base64.StdEncoding.EncodeToString([]byte(username + ":" + password))
}
func CallAPI(conf *cfg.Config, verb, path, data string) ([]byte, error) {
verb = strings.ToUpper(verb)
// we're using port-forwards anyway
noVerifyTransport := new(http.Transport{
TLSClientConfig: new(tls.Config{InsecureSkipVerify: true}),
})
client := new(http.Client{Transport: noVerifyTransport})
if conf.DebugHTTP {
client = new(http.Client{
Transport: new(cfg.DebugTransport{
Transport: noVerifyTransport})})
}
req, err := http.NewRequest(verb, conf.DefaultCluster.Uri+path, bytes.NewBuffer([]byte(data)))
if err != nil {
return nil, err
}
if !conf.HumanCat || (conf.HumanCat && !strings.HasPrefix(path, "/_cat")) {
req.Header.Add("Content-Type", "application/json")
req.Header.Add("Accept", "application/json")
}
// make sure we have got all we need
if err := conf.DefaultCluster.CheckAuth(); err != nil {
return nil, err
}
if conf.DefaultCluster.Token != "" {
req.Header.Add("Authorization", "APIKey "+conf.DefaultCluster.Token)
} else {
req.Header.Add("Authorization", "Basic "+encodeAuth(conf.DefaultCluster.User, conf.DefaultCluster.Pass))
}
// actually execute the request
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer func() {
if err := resp.Body.Close(); err != nil {
log.Fatal(err)
}
}()
// Read and print response
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("failed to read response body: %w", err)
}
return body, nil
}
func prettyfiJson(conf *cfg.Config, raw []byte) (string, error) {
if conf.HaveJQ {
cmd := exec.CommandContext(context.Background(), "jq", "-C")
cmd.Stdin = bytes.NewReader(raw)
var out bytes.Buffer
cmd.Stdout = &out
err := cmd.Run()
if err != nil {
return "", err
}
return out.String(), nil
}
var pretty bytes.Buffer
err := json.Indent(&pretty, raw, "", "\t")
if err != nil {
//nolint:nilerr
return string(raw), nil
}
return pretty.String(), nil
}
// interactively read arbitrary JSON data from STDIN, which is
// virtually a repl inside the primary repl
func readJSON(input string) (string, error) {
data := strings.Builder{}
if input != "" {
data.WriteString(input)
} else {
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
break
}
data.WriteString(line)
}
}
if data.Len() == 0 {
return "", nil
}
// validate
check := map[string]any{}
err := json.Unmarshal([]byte(data.String()), &check)
if err != nil {
return "", fmt.Errorf("error: input data is not proper JSON: %w", err)
}
return data.String(), nil
}
func ApiList(conf *cfg.Config, pattern string) error {
assets.LoadAssetOpenApi()
filter := regexp.MustCompile(pattern)
table := printer.NewTable(conf).WithSize(4, 0)
table.Addheaders("path", "http verb", "tag", "description")
for path, item := range assets.OpenAPI.Spec().Paths.Paths {
if pattern != "" {
if !filter.MatchString(path) {
continue
}
}
ops := findOperation(item.PathItemProps)
for _, op := range ops {
tags := findTags(op)
if conf.Tag != "" {
if !slices.Contains(tags, conf.Tag) {
continue
}
}
table.AddRow(
path,
op.Verb,
strings.Join(tags, ","),
strings.TrimSpace(op.Op.Summary),
)
}
}
table.Sort()
return table.Print()
}
// for completion
func ApiPathNames() []string {
assets.LoadAssetOpenApi()
return slices.Collect(maps.Keys(assets.OpenAPI.Spec().Paths.Paths))
}
func ApiShow(conf *cfg.Config, showpath, verb string) error {
assets.LoadAssetOpenApi()
out := printer.Builder{}
op, err := matchOperation(showpath, verb)
if err != nil {
return err
}
slog.Debug("api operation", "op", op)
cleanMarkup := regexp.MustCompile(`<[^<>]+>`)
width := cfg.GetTermWidth()
params := getApiParameters(op, showpath, width)
sample := getApiExample(op)
description := markdown.Render(cleanMarkup.ReplaceAllString(op.Op.Description, ""), width, cfg.DefaultMargin)
var (
bold = lipgloss.NewStyle().
Bold(true)
paragraph = lipgloss.NewStyle().
MarginBottom(1).
MarginLeft(cfg.DefaultMargin)
boldparagraph = lipgloss.NewStyle().
MarginBottom(1).
MarginLeft(cfg.DefaultMargin).
Bold(true)
indentparagraph = lipgloss.NewStyle().
MarginBottom(1).
MarginLeft(2)
)
out.WriteStringLine(bold.Render("ID: " + op.Op.ID))
out.WriteStringLine(paragraph.Render(fmt.Sprintf("%s %s", op.Verb, showpath)))
out.WriteStringLine(bold.Render("Tags"))
out.WriteStringLine(paragraph.Render(strings.Join(op.Op.Tags, ", ")))
out.WriteStringLine(bold.Render("Summary"))
out.WriteStringLine(paragraph.Render(strings.TrimSpace(op.Op.Summary)))
out.WriteStringLine(bold.Render("Description"))
out.WriteStringLine(string(description)) // already indented
if len(params.Path) > 0 {
out.WriteStringLine(bold.Render("Path Parameters"))
for _, param := range params.Path {
out.WriteStringLine(boldparagraph.Render(param.Param))
if param.Description != "" {
out.WriteStringLine(param.Description)
}
}
}
if len(params.Query) > 0 {
out.WriteStringLine(bold.Render("Query Parameters"))
for _, param := range params.Query {
out.WriteStringLine(boldparagraph.Render(param.Param))
if param.Description != "" {
out.WriteStringLine(indentparagraph.Render(param.Description))
}
}
}
if sample != "" {
out.WriteStringLine(bold.Render("Example"))
out.WriteStringLine(paragraph.Render(sample))
}
printer.Pager(showpath, out.String())
return nil
}
// find all tags associated with op
func findTags(op *Op) []string {
return op.Op.Tags
}
// Get API parameters
func getApiParameters(op *Op, path string, width int) Params {
params := Params{}
pathParams := []string{}
pathParamsReg := regexp.MustCompile(`{([a-z_]+)}`)
// find params in url path
for _, match := range pathParamsReg.FindAllStringSubmatch(path, -1) {
if len(match) == 2 {
pathParams = append(pathParams, match[1])
}
}
empty := spec.ParamProps{}
for _, param := range op.Op.Parameters {
if param.ParamProps == empty {
for _, token := range param.Refable.Ref.Ref.GetPointer().DecodedTokens() {
parts := strings.Split(token, "-")
if len(parts) == 2 {
param := parts[1]
if slices.Contains(pathParams, param) {
params.Path = append(params.Path, Param{Param: param})
} else {
params.Query = append(params.Query, Param{Param: param})
}
}
}
} else {
par := Param{
Description: string(markdown.Render(param.Description, width, cfg.DefaultMargin)),
Param: param.Name,
}
if param.In == "query" {
params.Query = append(params.Query, par)
} else {
params.Path = append(params.Path, par)
}
}
}
return params
}
// Get console example
func getApiExample(op *Op) string {
sample := ""
samples, sampleexist := op.Op.Extensions["x-codeSamples"]
if sampleexist {
for _, itemany := range samples.([]any) {
item := itemany.(map[string]any)
lang, haslang := item["lang"]
if haslang {
if lang == "Console" {
source, hassource := item["source"]
if hassource {
sample = strings.TrimSpace(source.(string))
}
}
}
}
}
return sample
}
// Find an API operation which matches given path and verb.
// If no verb is given and only 1 op exists, return this one,
// otherwise showpath+verb have to match precisely.
func matchOperation(showpath, verb string) (*Op, error) {
ops := []*Op{}
op := new(Op{})
var found bool
for path, item := range assets.OpenAPI.Spec().Paths.Paths {
if path == showpath {
ops = findOperation(item.PathItemProps)
break
}
}
if len(ops) == 0 {
return nil, errors.New("no matching API call found")
}
if len(ops) == 1 && verb == "" {
return ops[0], nil
}
for _, item := range ops {
if strings.ToLower(item.Verb) == verb {
op = item
found = true
break
}
}
if !found {
return nil, errors.New("no matching API call found for given path+verb")
}
return op, nil
}
func findOperation(item spec.PathItemProps) []*Op {
ops := []*Op{}
if item.Post != nil {
ops = append(ops, &Op{"POST", item.Post})
}
if item.Get != nil {
ops = append(ops, &Op{"GET", item.Get})
}
if item.Delete != nil {
ops = append(ops, &Op{"DELETE", item.Delete})
}
if item.Put != nil {
ops = append(ops, &Op{"PUT", item.Put})
}
if item.Patch != nil {
ops = append(ops, &Op{"PATCH", item.Patch})
}
return ops
}

View File

@@ -23,6 +23,7 @@ import (
"log/slog"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
@@ -45,23 +46,18 @@ func CcrStatus(conf *cfg.Config, leader, follower string) error {
}
// reverse attribution
f := follower
follower = leader
leader = f
follower, leader = leader, follower
slog.Debug("leader/follower attribution is invalid, reversing", "leader", leader, "follower", follower)
}
indices := ClusterIndices{}
for _, alias := range []string{leader, follower} {
cat := conf.Clusters[alias].ES.Cat.Indices().
// we need to add custom request headers, required for older ES instances
Header("content-type", "application/json").
Header("accept", "application/json")
res, err := cat.Do(context.Background())
res, err := conf.Clusters[alias].ES().Cat.Indices().
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get indicies on %s: %s", alias, err)
return fmt.Errorf("failed to get indices on %s: %w", alias, esErrorString(err))
}
indices[alias] = make(map[string]*types.IndicesRecord, len(res))
@@ -87,27 +83,27 @@ func CcrStatus(conf *cfg.Config, leader, follower string) error {
}
func CcrRemoteInfo(conf *cfg.Config, index string) error {
res, err := conf.DefaultCluster.ES.Cluster.RemoteInfo().
Header("content-type", "application/json").
Header("accept", "application/json").
res, err := conf.DefaultCluster.ES().Cluster.RemoteInfo().
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to retrieve follower info: %s", err)
return fmt.Errorf("failed to retrieve follower info: %w", esErrorString(err))
}
slog.Debug("ccr remote info", "info", res)
remote := ""
var info *types.ClusterRemoteProxyInfo
for name, data := range res {
remote = name
info = data.(*types.ClusterRemoteProxyInfo)
break
}
if remote == "" {
return fmt.Errorf("cluster doesn't follow any other: %s", err)
return errors.New("cluster doesn't follow any other")
}
mode := "follower"
@@ -115,18 +111,18 @@ func CcrRemoteInfo(conf *cfg.Config, index string) error {
mode = "leader"
}
table := NewTable(2, 5)
table.Addheaders("field", "value")
table := printer.NewTable(conf).WithSize(2, 5)
table.Addheaders("ccr remote property", "value")
table.entries = [][]string{
table.Entries = [][]any{
{"Remote Cluster", remote},
{"CCR Mode", mode},
{"Connected", fmt.Sprintf("%t", info.Connected)},
{"Num Proxy Sockets Connected", fmt.Sprintf("%d", info.NumProxySocketsConnected)},
{"Connected", info.Connected},
{"Num Proxy Sockets Connected", info.NumProxySocketsConnected},
{"Proxy Address", info.ProxyAddress},
}
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
return err
}

View File

@@ -18,29 +18,30 @@ package es
import (
"context"
"errors"
"fmt"
"log/slog"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
)
func getRemoteName(conf *cfg.Config) (string, error) {
res, err := conf.DefaultCluster.ES.Cluster.RemoteInfo().
Header("content-type", "application/json").
Header("accept", "application/json").
res, err := conf.DefaultCluster.ES().Cluster.RemoteInfo().
Do(context.Background())
if err != nil {
return "", fmt.Errorf("failed to retrieve follower info: %s", err)
return "", fmt.Errorf("failed to retrieve follower info: %w", esErrorString(err))
}
remote := ""
for name := range res {
remote = name
break
}
if remote == "" {
return "", fmt.Errorf("cluster doesn't have a follower: %s", err)
return "", errors.New("cluster doesn't have a follower")
}
return remote, nil
@@ -62,68 +63,61 @@ func CcrFollowerRenew(conf *cfg.Config, index string) error {
if err := wrapError(IndexClose, conf, index); err != nil {
return err
}
fmt.Printf("closed %s", index)
if err := wrapError(CcrFollowerPause, conf, index); err != nil {
return err
}
fmt.Printf("paused %s", index)
if err := wrapError(CcrFollowerUnfollow, conf, index); err != nil {
return err
}
fmt.Printf("unfollowed %s", index)
if err := wrapError(IndexDelete, conf, index); err != nil {
return err
}
fmt.Printf("deleted %s", index)
if err := CcrFollowerAdd(conf, index); err != nil {
return err
}
fmt.Printf("added follower %s", index)
return nil
}
func CcrFollowerResume(conf *cfg.Config, index string) error {
create := conf.DefaultCluster.ES.Ccr.ResumeFollow(index).
Header("content-type", "application/json").
Header("accept", "application/json")
_, err := create.Do(context.Background())
_, err := conf.DefaultCluster.ES().Ccr.ResumeFollow(index).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to resume ccr following: %s", err)
return fmt.Errorf("failed to resume ccr following: %w", esErrorString(err))
}
return nil
}
func CcrFollowerPause(conf *cfg.Config, index string) error {
create := conf.DefaultCluster.ES.Ccr.PauseFollow(index).
Header("content-type", "application/json").
Header("accept", "application/json")
_, err := create.Do(context.Background())
_, err := conf.DefaultCluster.ES().Ccr.PauseFollow(index).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to pause ccr following: %s", err)
return fmt.Errorf("failed to pause ccr following: %w", esErrorString(err))
}
return nil
}
func CcrFollowerUnfollow(conf *cfg.Config, index string) error {
create := conf.DefaultCluster.ES.Ccr.ForgetFollower(index).
Header("content-type", "application/json").
Header("accept", "application/json")
_, err := create.Do(context.Background())
_, err := conf.DefaultCluster.ES().Ccr.ForgetFollower(index).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to unfollow index: %s", err)
return fmt.Errorf("failed to unfollow index: %w", esErrorString(err))
}
return nil
@@ -135,32 +129,27 @@ func CcrFollowerAdd(conf *cfg.Config, index string) error {
return err
}
create := conf.DefaultCluster.ES.Ccr.Follow(index).
create := conf.DefaultCluster.ES().Ccr.Follow(index).
LeaderIndex(index).
RemoteCluster(remote).
Header("content-type", "application/json").
Header("accept", "application/json")
RemoteCluster(remote)
if conf.Wait {
create.WaitForActiveShards("all")
}
_, err = create.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to create follower index: %s", err)
return fmt.Errorf("failed to create follower index: %w", esErrorString(err))
}
return nil
}
func CcrFollowerShow(conf *cfg.Config, index string) error {
res, err := conf.DefaultCluster.ES.Ccr.FollowStats(index).
Header("content-type", "application/json").
Header("accept", "application/json").
res, err := conf.DefaultCluster.ES().Ccr.FollowStats(index).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to retrieve follower index info: %s", err)
return fmt.Errorf("failed to retrieve follower index info: %w", esErrorString(err))
}
slog.Debug("ES result", "follower stats", res.Indices)
@@ -175,22 +164,22 @@ func CcrFollowerShow(conf *cfg.Config, index string) error {
follower := res.Indices[0].Shards[0]
table := NewTable(2, 9)
table.Addheaders("field", "value")
table := printer.NewTable(conf).WithSize(2, 9)
table.Addheaders("ccr follower property", "value")
table.entries = [][]string{
table.Entries = [][]any{
{"name", index},
{"remote_cluster", follower.RemoteCluster},
{"leader_checkpoint", fmt.Sprintf("%d", follower.LeaderGlobalCheckpoint)},
{"follower_checkpoint", fmt.Sprintf("%d", follower.FollowerGlobalCheckpoint)},
{"bytes_read", fmt.Sprintf("%d", follower.BytesRead)},
{"failed_read_requests", fmt.Sprintf("%d", follower.FailedReadRequests)},
{"failed_write_requests", fmt.Sprintf("%d", follower.FailedWriteRequests)},
{"successful_read_requests", fmt.Sprintf("%d", follower.SuccessfulReadRequests)},
{"successful_write_requests", fmt.Sprintf("%d", follower.SuccessfulWriteRequests)},
{"leader_checkpoint", follower.LeaderGlobalCheckpoint},
{"follower_checkpoint", follower.FollowerGlobalCheckpoint},
{"bytes_read", follower.BytesRead},
{"failed_read_requests", follower.FailedReadRequests},
{"failed_write_requests", follower.FailedWriteRequests},
{"successful_read_requests", follower.SuccessfulReadRequests},
{"successful_write_requests", follower.SuccessfulWriteRequests},
}
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
return err
}

View File

@@ -0,0 +1,74 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
// "encoding/json/jsontext"
// "encoding/json/v2"
"encoding/json"
"fmt"
"codeberg.org/scip/esctl/pkg/cfg"
)
type HealthReport struct {
Indicators map[string]HealthReportIndicator
}
type HealthReportIndicator struct {
Status, Symptom string
Diagnosis []HealthReportDiagnosis
}
type HealthReportDiagnosis struct {
Id, Cause, Action string
AffectedResources map[string][]string `json:"affected_resources"`
}
// we do not use the go-elasticsearch client API here but call the ES
// API directly, because the returned structure (a
// healthreport.Response) is not iterable, you'd have to explicitly
// call every indicator type and every cause etc which also have
// different types each. To check which is !green would result in a
// gigantic function.
func getHealthReport(conf *cfg.Config) (*HealthReport, error) {
raw, err := CallAPI(conf, "GET", "/_health_report", "")
if err != nil {
return nil, err
}
report := new(HealthReport{})
// FIXME: use this once jsonv2 is no more experimental it already
// builds and works like intended, but golangci-lint doesn't
// recognize it with: go: unknown GOEXPERIMENT jsonv2
//
// if err := json.UnmarshalDecode(
// jsontext.NewDecoder(
// bytes.NewBuffer(raw)),
// &report); err != nil {
// return nil, fmt.Errorf("failed to unmarshal healthreport response: %w", err)
// }
if err := json.Unmarshal(raw, &report); err != nil {
return nil, fmt.Errorf("failed to unmarshal healthreport response: %w", err)
}
return report, nil
}

View File

@@ -17,178 +17,265 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
package es
import (
"context"
"errors"
"fmt"
"log/slog"
"slices"
"strings"
"sync"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/dustin/go-humanize"
"github.com/elastic/go-elasticsearch/v9/typedapi/ccr/stats"
"github.com/elastic/go-elasticsearch/v9/typedapi/cluster/health"
clusterstats "github.com/elastic/go-elasticsearch/v9/typedapi/cluster/stats"
"github.com/elastic/go-elasticsearch/v9/typedapi/core/info"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
const (
ResponseHealth = iota
ResponseInfo
ResponseCcr
ResponseStats
)
type ClusterIndices map[string]map[string]*types.IndicesRecord
type apiResponse struct {
error error
info *info.Response
health *health.Response
ccr *stats.Response
stats *clusterstats.Response
which int
type clusterReachable struct {
reachable bool
err error
}
func ClusterList(conf *cfg.Config) error {
table := NewTable(3, len(conf.Clusters))
var (
mu sync.Mutex
wg sync.WaitGroup
)
table.Addheaders("cluster", "uri", "default")
reachable := make(map[string]clusterReachable, len(conf.Clusters))
// check endpoints in parallel to speed things up
for name, cluster := range conf.Clusters {
wg.Go(func() {
online, err := cluster.IsReachable()
mu.Lock()
reachable[name] = clusterReachable{reachable: online, err: err}
mu.Unlock()
})
}
wg.Wait()
table := printer.NewTable(conf).WithSize(5, len(conf.Clusters))
table.Addheaders("cluster", "uri", "reachable", "current", "error")
idx := 0
names := make([]string, len(conf.Clusters))
for name := range conf.Clusters {
names[idx] = name
for name, cluster := range conf.Clusters {
reachableStr := "no"
current := "no"
errmsg := ""
if reachable[name].reachable {
reachableStr = printer.Colorize(conf, "green", "reachable")
}
if cluster.Default {
current = printer.Colorize(conf, "green", "yes")
if !reachable[name].reachable {
reachableStr = printer.Colorize(conf, "red", "no")
errmsg = reachable[name].err.Error()
}
}
table.Entries[idx] = []any{name, cluster.Uri, reachableStr, current, errmsg}
idx++
}
slices.Sort(names)
table.Sort()
for idx, name := range names {
current := name == "default" || name == conf.CurrentCluster
cluster := conf.Clusters[name]
return table.Print()
}
_, err := cluster.ES.Cluster.Health().
Header("content-type", "application/json").
Header("accept", "application/json").
Do(context.Background())
if err == nil {
name = Colorize("green", name)
}
table.entries[idx] = []string{name, cluster.Uri, fmt.Sprintf("%t", current)}
// We're using goroutines here to parallelize API requests, since we
// have to do 3 of'em for each cluster. This speeds things up.
func getClusterStatus(conf *cfg.Config) (*apiResponse, error) {
gocount := 6
if conf.Verbose {
gocount++
}
if err := table.PrintMarkdown(); err != nil {
es := conf.DefaultCluster.ES()
responses := make(chan apiResponse, gocount)
wg := new(sync.WaitGroup{})
wg.Go(func() {
getApiData(conf, es, responses, "health")
})
wg.Go(func() {
getApiData(conf, es, responses, "healthreport")
})
wg.Go(func() {
getApiData(conf, es, responses, "info")
})
wg.Go(func() {
getApiData(conf, es, responses, "ccr")
})
wg.Go(func() {
getApiData(conf, es, responses, "indices")
})
wg.Go(func() {
getApiData(conf, es, responses, "tasks")
})
if conf.Verbose {
getApiData(conf, es, responses, "stats")
}
wg.Wait()
all := new(apiResponse{})
var err error
for range gocount {
res := <-responses
err = errors.Join(err, res.error)
switch res.which {
case ResponseHealth:
all.health = res.health
case ResponseCcr:
all.ccr = res.ccr
case ResponseInfo:
all.info = res.info
case ResponseStats:
all.stats = res.stats
case ResponseIndices:
all.indices = res.indices
case ResponseTasks:
all.tasks = res.tasks
case ResponseHealthReport:
all.healthreport = res.healthreport
}
}
return all, err
}
func ClusterStatus(conf *cfg.Config) error {
res, err := getClusterStatus(conf)
if err != nil && !strings.Contains(err.Error(), "current license is non-compliant") {
return err
}
slog.Debug("ES result", "cluster health", res.health)
var (
isleader bool
ccrfollowing string
)
if res.ccr != nil {
isleader = len(res.ccr.AutoFollowStats.AutoFollowedClusters) == 0
if len(res.ccr.AutoFollowStats.AutoFollowedClusters) > 0 {
// is following another cluster
ccrfollowing = fmt.Sprintf("%s (%d/%d)",
res.ccr.AutoFollowStats.AutoFollowedClusters[0].ClusterName,
res.ccr.AutoFollowStats.NumberOfSuccessfulFollowIndices,
res.ccr.AutoFollowStats.NumberOfFailedFollowIndices,
)
}
}
// look for red indices, if any
failedIndices := 0
greenIndices := 0
for _, index := range *res.indices {
switch {
case *index.Health != "green":
failedIndices++
case !strings.HasPrefix(*index.Index, "."):
// don't count internal indices
greenIndices++
}
}
// look for long running tasks
longtasks := 0
for _, task := range *res.tasks {
if strings.Contains(*task.RunningTime, "d") {
longtasks++
}
}
table := printer.NewTable(conf).WithSize(2, 7)
table.Addheaders(conf.DefaultCluster.Name, "status")
table.Entries = [][]any{
{"Cluster Name", res.health.ClusterName},
{"ES Status", printer.Colorize(conf, res.health.Status.Name, res.health.Status.Name)},
{"ES Version", res.info.Version.Int},
{"Is Leader", isleader},
{"Active Shards", res.health.ActiveShards},
{"Active Primary Shards", res.health.ActivePrimaryShards},
{"Unassigned Shards", res.health.UnassignedShards},
{"Unassigned Primary Shards", res.health.UnassignedPrimaryShards},
{"Pending Tasks", res.health.NumberOfPendingTasks},
{"Nodes", res.health.NumberOfNodes},
{"Green Indices", greenIndices},
{"Failed Indices", failedIndices},
{"Long Running Tasks", longtasks},
}
if !isleader && res.ccr != nil {
table.Entries = append(table.Entries, [][]any{
{"AutoFollow (success/failed indices)", ccrfollowing},
{"Followed Indices", len(res.ccr.FollowStats.Indices)},
}...)
}
if conf.Verbose {
table = gatherClusterStats(res.stats, table)
}
if res.health.Status.Name != "green" {
for name, indicator := range res.healthreport.Indicators {
if indicator.Status != "green" {
table.Entries = append(table.Entries, []any{
printer.Colorize(conf, indicator.Status, "Bad health "+name), indicator.Symptom,
})
for _, diag := range indicator.Diagnosis {
table.Entries = append(table.Entries, []any{" -> cause", diag.Cause})
for resource, items := range diag.AffectedResources {
table.Entries = append(table.Entries, []any{" -> affected " + resource, strings.Join(items, ",")})
}
if diag.Action != "" {
table.AddRow(" -> suggested action to fix", diag.Action)
}
}
}
}
}
if err := table.Print(); err != nil {
return err
}
return nil
}
// We're using goroutines here to parallelize API requests, since we
// have to do 3 of'em for each cluster. This speeds things up.
func ClusterStatus(conf *cfg.Config) error {
clusters := []string{}
gocount := 3
if conf.Verbose {
gocount++
}
if conf.All {
for key := range conf.Clusters {
clusters = append(clusters, key)
}
} else {
clusters = []string{"default"}
}
for _, cluster := range clusters {
es := conf.DefaultCluster.ES
if cluster != "default" {
es = conf.Clusters[cluster].ES
}
responses := make(chan apiResponse, gocount)
wg := &sync.WaitGroup{}
wg.Add(gocount)
go getClusterData(es, wg, responses, "health")
go getClusterData(es, wg, responses, "info")
go getClusterData(es, wg, responses, "ccrstats")
if conf.Verbose {
go getClusterData(es, wg, responses, "stats")
}
wg.Wait()
var clusterhealth *health.Response
var info *info.Response
var ccrstats *stats.Response
var clusterstats *clusterstats.Response
for i := 0; i < gocount; i++ {
r := <-responses
if r.error != nil {
return r.error
}
switch r.which {
case ResponseHealth:
clusterhealth = r.health
case ResponseCcr:
ccrstats = r.ccr
case ResponseInfo:
info = r.info
case ResponseStats:
clusterstats = r.stats
}
}
slog.Debug("ES result", "cluster health", clusterhealth)
ccrfollowing := ""
if len(ccrstats.AutoFollowStats.AutoFollowedClusters) > 0 {
// is following another cluster
ccrfollowing = fmt.Sprintf("%s (%d/%d)",
ccrstats.AutoFollowStats.AutoFollowedClusters[0].ClusterName,
ccrstats.AutoFollowStats.NumberOfSuccessfulFollowIndices,
ccrstats.AutoFollowStats.NumberOfFailedFollowIndices,
)
}
table := NewTable(2, 5)
table.Addheaders(cluster, "status")
table.entries = [][]string{
{"Cluster Name", Colorize(clusterhealth.Status.Name, clusterhealth.ClusterName)},
{"ES Version", info.Version.Int},
{"Active Shards", fmt.Sprintf("%d", clusterhealth.ActiveShards)},
{"Active Primary Shards", fmt.Sprintf("%d", clusterhealth.ActivePrimaryShards)},
{"Nodes", fmt.Sprintf("%d", clusterhealth.NumberOfNodes)},
{"AutoFollow (success/failed indices)", ccrfollowing},
}
if conf.Verbose {
table = gatherClusterStats(conf, clusterstats, table)
}
if err := table.PrintMarkdown(); err != nil {
return err
}
}
return nil
}
func gatherClusterStats(conf *cfg.Config, clusterstats *clusterstats.Response, table *Table) *Table {
var querycount int64
var vmversion string
func gatherClusterStats(clusterstats *clusterstats.Response, table *printer.Table) *printer.Table {
var (
querycount int64
vmversion string
)
for _, count := range clusterstats.Indices.Search.Queries {
querycount += count
@@ -200,31 +287,28 @@ func gatherClusterStats(conf *cfg.Config, clusterstats *clusterstats.Response, t
clusterstats.Nodes.Jvm.Versions[0].VmVersion}, " ")
}
isleader := checkClusterIsLeader(conf, conf.CurrentCluster)
table.entries = append(table.entries, [][]string{
{"Indicies", fmt.Sprintf("%d", clusterstats.Indices.Count)},
{"Is Leader", fmt.Sprintf("%t", isleader)},
{"Docs", fmt.Sprintf("%d", clusterstats.Indices.Docs.Count)},
{"Total Size", humanize.Bytes(uint64(clusterstats.Indices.Docs.TotalSizeInBytes))},
{"Total Queries", fmt.Sprintf("%d", querycount)},
{"Shards Primaries", fmt.Sprintf("%d", clusterstats.Indices.Shards.Primaries)},
{"Shards Total", fmt.Sprintf("%d", clusterstats.Indices.Shards.Total)},
table.Entries = append(table.Entries, [][]any{
{"indices", clusterstats.Indices.Count},
{"Docs", clusterstats.Indices.Docs.Count},
{"Total Size", printer.Bytes(clusterstats.Indices.Docs.TotalSizeInBytes)},
{"Total Queries", "%d", querycount},
{"Shards Primaries", clusterstats.Indices.Shards.Primaries},
{"Shards Total", clusterstats.Indices.Shards.Total},
{"Storage", fmt.Sprintf(
"%s/%s",
humanize.Bytes(uint64(clusterstats.Indices.Store.SizeInBytes)),
humanize.Bytes(uint64(*clusterstats.Indices.Store.TotalDataSetSizeInBytes)),
)},
{"JVM Heap", fmt.Sprintf(
{"JVM Heap Memory", fmt.Sprintf(
"%s/%s",
humanize.Bytes(uint64(clusterstats.Nodes.Jvm.Mem.HeapUsedInBytes)),
humanize.Bytes(uint64(clusterstats.Nodes.Jvm.Mem.HeapMaxInBytes)),
)},
{"JVM Threads", fmt.Sprintf("%d", clusterstats.Nodes.Jvm.Threads)},
{"JVM Threads", clusterstats.Nodes.Jvm.Threads},
{"JVM Version", vmversion},
{"CPUs", fmt.Sprintf("%d", clusterstats.Nodes.Os.AllocatedProcessors)},
{"CPUs", clusterstats.Nodes.Os.AllocatedProcessors},
{"CPU Usage", fmt.Sprintf("%d%%", clusterstats.Nodes.Process.Cpu.Percent)},
{"Open FDs", fmt.Sprintf("%d", clusterstats.Nodes.Process.OpenFileDescriptors.Avg)},
{"Open FDs", clusterstats.Nodes.Process.OpenFileDescriptors.Avg},
}...)
return table

126
pkg/es/cluster_reroute.go Normal file
View File

@@ -0,0 +1,126 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"fmt"
"codeberg.org/scip/esctl/pkg/cfg"
"github.com/elastic/go-elasticsearch/v9/typedapi/esdsl"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
func ClusterRerouteMove(conf *cfg.Config, index string) error {
move := conf.DefaultCluster.ES().Cluster.Reroute()
commands := esdsl.NewCommand()
moveCommand := new(types.CommandMoveAction{
Shard: conf.Shards,
FromNode: conf.FromNode,
ToNode: conf.ToNode,
Index: index,
})
commands.CommandCaster().Move = moveCommand
move.Commands(commands)
_, err := move.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to reroute move: %w", err)
}
return nil
}
func ClusterRerouteAllocateReplica(conf *cfg.Config, index string) error {
move := conf.DefaultCluster.ES().Cluster.Reroute()
commands := esdsl.NewCommand()
allocCommand := new(types.CommandAllocateReplicaAction{
Shard: conf.Shards,
Node: conf.ToNode,
Index: index,
})
commands.CommandCaster().AllocateReplica = allocCommand
move.Commands(commands)
_, err := move.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to allocate a replica shard: %w", err)
}
return nil
}
func ClusterRerouteCancel(conf *cfg.Config, index string) error {
move := conf.DefaultCluster.ES().Cluster.Reroute()
commands := esdsl.NewCommand()
cancelCommand := new(types.CommandCancelAction{
Shard: conf.Shards,
Node: conf.ToNode,
Index: index,
AllowPrimary: &conf.AllowPrimary,
})
commands.CommandCaster().Cancel = cancelCommand
move.Commands(commands)
_, err := move.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to cancel a reroute process: %w", err)
}
return nil
}
func ClusterRerouteAllocatePrimary(conf *cfg.Config, index string, stale bool) error {
move := conf.DefaultCluster.ES().Cluster.Reroute()
commands := esdsl.NewCommand()
allocCommand := new(types.CommandAllocatePrimaryAction{
Shard: conf.Shards,
Node: conf.ToNode,
Index: index,
AcceptDataLoss: conf.AcceptDataLoss,
})
if stale {
commands.CommandCaster().AllocateStalePrimary = allocCommand
} else {
commands.CommandCaster().AllocateEmptyPrimary = allocCommand
}
move.Commands(commands)
_, err := move.Do(context.Background())
if err != nil {
which := "empty"
if stale {
which = "stale"
}
return fmt.Errorf("failed to allocate %s primary shard: %w", which, err)
}
return nil
}

View File

@@ -20,54 +20,28 @@ import (
"context"
"encoding/json"
"fmt"
"log/slog"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/urfave/cli/v3"
)
// recursively traverse the raw settings hash and build a flat map
// consisting of the translated path and its value.
//
// e.g.
// logger:
//
// org:
// elasticsearch:
// transport:
// OutboundHandler: "ERROR"
//
// gets:
//
// logger.org.elasticsearch.transport.OutboundHandler: "ERROR"
func getJsonPath(raw map[string]any, topic string) map[string]string {
paths := map[string]string{}
for name, data := range raw {
path := topic + "." + name
switch value := data.(type) {
case string:
paths[path] = value
case map[string]any:
paths = getJsonPath(value, path)
}
}
return paths
func ClusterSettingsNames(conf *cfg.Config) ([]string, error) {
return validClusterSettings, nil
}
func ClusterSettingsList(conf *cfg.Config) error {
res, err := conf.DefaultCluster.ES.Cluster.GetSettings().
Header("content-type", "application/json").
Header("accept", "application/json").
res, err := conf.DefaultCluster.ES().Cluster.GetSettings().
FlatSettings(true).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get cluster settings: %s", err)
return fmt.Errorf("failed to get cluster settings: %w", esErrorString(err))
}
table := NewTable(2, 0)
table := printer.NewTable(conf).WithSize(2, 0)
table.Addheaders("setting", "value")
entries := [][]string{}
entries := [][]any{}
settingshash := res.Persistent // == map[string]json.RawMessage
@@ -79,25 +53,13 @@ func ClusterSettingsList(conf *cfg.Config) error {
}
for topic, val := range settingshash {
data := map[string]any{}
err := json.Unmarshal(val, &data)
if err != nil {
return fmt.Errorf("failed to unmarshall setting for topic %s: %s", topic, err)
}
paths := getJsonPath(data, topic)
slog.Debug("settings", topic, paths)
for setting, value := range paths {
entries = append(entries, []string{setting, fmt.Sprintf("%v", value)})
}
entries = append(entries, []any{topic, string(val)})
}
table.entries = entries
table.Entries = entries
table.Sort()
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
return err
}
@@ -105,7 +67,7 @@ func ClusterSettingsList(conf *cfg.Config) error {
}
func ClusterSettingsSet(conf *cfg.Config, args cli.Args) error {
put := conf.Clusters[conf.CurrentCluster].ES.Cluster.PutSettings()
put := conf.DefaultCluster.ES().Cluster.PutSettings()
for _, arg := range args.Slice() {
setting, value := splitArg(arg)
@@ -114,27 +76,25 @@ func ClusterSettingsSet(conf *cfg.Config, args cli.Args) error {
case conf.Transient:
message, err := json.Marshal(value)
if err != nil {
return fmt.Errorf("failed to marshall transient value <%v> to valid JSON: %s", value, err)
return fmt.Errorf("failed to marshall transient value <%v> to valid JSON: %w", value, err)
}
put.AddTransient(setting, message)
case conf.Persistent:
fallthrough
default:
message, err := json.Marshal(value)
if err != nil {
return fmt.Errorf("failed to marshall persistent value <%v> to valid JSON: %s", value, err)
return fmt.Errorf("failed to marshall persistent value <%v> to valid JSON: %w", value, err)
}
put.AddPersistent(setting, message)
}
}
_, err := put.
Header("content-type", "application/json").
Header("accept", "application/json").
Do(context.Background())
_, err := put.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to set settings: %s", err)
return fmt.Errorf("failed to set settings: %w", esErrorString(err))
}
return nil
@@ -143,17 +103,14 @@ func ClusterSettingsSet(conf *cfg.Config, args cli.Args) error {
func ClusterSettingsSetSingle(conf *cfg.Config, setting, value string) error {
message, err := json.Marshal(value)
if err != nil {
return fmt.Errorf("failed to marshall persistent value <%v> to valid JSON: %s", value, err)
return fmt.Errorf("failed to marshall persistent value <%v> to valid JSON: %w", value, err)
}
_, err = conf.Clusters[conf.CurrentCluster].ES.Cluster.PutSettings().
_, err = conf.DefaultCluster.ES().Cluster.PutSettings().
AddPersistent(setting, message).
Header("content-type", "application/json").
Header("accept", "application/json").
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to set %s: %s", setting, err)
return fmt.Errorf("failed to set %s: %w", setting, esErrorString(err))
}
return nil

View File

@@ -0,0 +1,730 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
// manually extracted from https://www.elastic.co/docs/reference/elasticsearch/configuration-reference
var validClusterSettings []string = []string{
"action.auto_create_index",
"action.destructive_requires_name",
"attribute_delimiters.groups",
"attribute_patterns.dn",
"attribute_patterns.groups",
"attribute_patterns.mail",
"attribute_patterns.name",
"attribute_patterns.principal",
"attributes.dn",
"attributes.groups",
"attributes.mail",
"attributes.name",
"attributes.principal",
"authentication.enabled",
"auth.secure_password",
"auth.username",
"breaker.eql_sequence.limit",
"breaker.eql_sequence.overhead",
"breaker.eql_sequence.type",
"breaker.model_inference.limit",
"breaker.model_inference.overhead",
"breaker.model_inference.type",
"cache.hash_algo",
"cache.max_users",
"cache.ttl",
"ccr.indices.recovery.chunk_size",
"ccr.indices.recovery.internal_action_timeout",
"ccr.indices.recovery.max_bytes_per_sec",
"ccr.indices.recovery.max_concurrent_file_chunks",
"ccr.indices.recovery.recovery_activity_timeout",
"claim_patterns.dn",
"claim_patterns.group",
"claim_patterns.groups",
"claim_patterns.mail",
"claim_patterns.name",
"claim_patterns.principal",
"claims.dn",
"claims.groups",
"claims.mail",
"claims.name",
"claims.principal",
"client_authentication.rotation_grace_period",
"client_authentication.shared_secret",
"client_authentication.type",
"cluster_alerts.management.blacklist",
"cluster_alerts.management.enabled",
"cluster.auto_shrink_voting_configuration",
"cluster.discovery_configuration_check.interval",
"cluster.election.duration",
"cluster.election.initial_timeout",
"cluster.election.max_timeout",
"cluster.fault_detection.follower_check.retry_count",
"cluster.fault_detection.follower_check.timeout",
"cluster.fault_detection.leader_check.interval",
"cluster.fault_detection.leader_check.retry_count",
"cluster.fault_detection.leader_check.timeout",
"cluster.follower_lag.timeout",
"cluster.indices.close.enable",
"cluster.info.update.interval",
"cluster.initial_master_nodes",
"cluster.join_validation.cache_timeout",
"cluster.lifecycle.default.rollover",
"cluster.max_voting_config_exclusions",
"cluster.no_master_block",
"cluster.publish.info_timeout",
"cluster.publish.timeout",
"cluster.reindex.pit.keep_alive",
"cluster.remote.initial_connect_timeout",
"cluster.remote.node.attr",
"cluster.remote.signing.certificate_authorities",
"cluster.remote.signing.diagnose.trust",
"cluster.remote.signing.truststore.algorithm",
"cluster.remote.signing.truststore.path",
"cluster.remote.signing.truststore.secure_password",
"cluster.remote.signing.truststore.type",
"cluster.routing.allocation.allow_rebalance",
"cluster.routing.allocation.awareness.attributes",
"cluster.routing.allocation.awareness.force.*",
"cluster.routing.allocation.balance.disk_usage",
"cluster.routing.allocation.balance.index",
"cluster.routing.allocation.balance.shard",
"cluster.routing.allocation.balance.threshold",
"cluster.routing.allocation.balance.write_load",
"cluster.routing.allocation.cluster_concurrent_rebalance",
"cluster.routing.allocation.disk.threshold_enabled",
"cluster.routing.allocation.disk.watermark.enable_for_single_data_node",
"cluster.routing.allocation.disk.watermark.flood_stage",
"cluster.routing.allocation.disk.watermark.flood_stage.frozen",
"cluster.routing.allocation.disk.watermark.flood_stage.frozen.max_headroom",
"cluster.routing.allocation.disk.watermark.flood_stage.max_headroom",
"cluster.routing.allocation.disk.watermark.high",
"cluster.routing.allocation.disk.watermark.high.max_headroom",
"cluster.routing.allocation.disk.watermark.low",
"cluster.routing.allocation.disk.watermark.low.max_headroom",
"cluster.routing.allocation.enable",
"cluster.routing.allocation.exclude.{{attribute}}",
"cluster.routing.allocation.include.{{attribute}}",
"cluster.routing.allocation.node_concurrent_incoming_recoveries",
"cluster.routing.allocation.node_concurrent_outgoing_recoveries",
"cluster.routing.allocation.node_concurrent_recoveries",
"cluster.routing.allocation.node_initial_primaries_recoveries",
"cluster.routing.allocation.require.{{attribute}}",
"cluster.routing.allocation.same_shard.host",
"cluster.routing.allocation.stats.cache.ttl",
"cluster.routing.allocation.total_shards_per_node",
"cluster.routing.allocation.type",
"cluster.routing.rebalance.enable",
"connection.read_timeout",
"connection.timeout",
"data_streams.lifecycle.downsampling.max_indices_in_progress",
"data_streams.lifecycle.poll_interval",
"data_streams.lifecycle.retention.default",
"data_streams.lifecycle.retention.max",
"data_streams.lifecycle.signalling.error_retry_interval",
"data_streams.lifecycle.target.merge.policy.floor_segment",
"data_streams.lifecycle.target.merge.policy.merge_factor",
"delegation.enabled",
"discovery.cluster_formation_warning_timeout",
"discovery.find_peers_interval",
"discovery.find_peers_warning_timeout",
"discovery.probe.connect_timeout",
"discovery.probe.handshake_timeout",
"discovery.request_peers_timeout",
"discovery.seed_hosts",
"discovery.seed_providers",
"discovery.seed_resolver.max_concurrent_resolvers",
"discovery.seed_resolver.timeout",
"discovery.type",
"dlm.frozen.cleanup.poll_interval",
"dlm.frozen.transition.poll_interval",
"dlm.frozen.transition.thread_pool.queue_size",
"dlm.frozen.transition.thread_pool.size",
"email_defaults.*",
"encryption.certificate",
"encryption.key",
"encryption.keystore.alias",
"encryption.keystore.path",
"encryption.keystore.secure_key_password",
"encryption.keystore.secure_password",
"encryption.keystore.type",
"encryption.secure_key_passphrase",
"enrich.cache_size",
"enrich.cleanup_period",
"enrich.coordinator_proxy.max_concurrent_requests",
"enrich.coordinator_proxy.max_lookups_per_request",
"enrich.coordinator_proxy.queue_capacity",
"enrich.fetch_size",
"enrich.max_concurrent_policy_executions",
"enrich.max_force_merge_attempts",
"fallback_claims.aud",
"fallback_claims.sub",
"files.role_mapping",
"gateway.expected_data_nodes",
"gateway.recover_after_data_nodes",
"gateway.recover_after_time",
"group_search.base_dn",
"group_search.filter",
"group_search.scope",
"group_search.user_attribute",
"health.ilm.max_retries_per_step",
"health.ilm.max_time_on_action",
"health.ilm.max_time_on_step",
"health.master_history.has_master_lookup_timeframe",
"health.master_history.identity_changes_threshold",
"health.master_history.no_master_transitions_threshold",
"health.node.enabled",
"health.periodic_logger.enabled",
"health.periodic_logger.poll_interval",
"health.reporting.local.monitor.interval",
"health.shard_capacity.unhealthy_threshold.red",
"health.shard_capacity.unhealthy_threshold.yellow",
"http.bind_host",
"http.client_stats.closed_channels.max_age",
"http.client_stats.closed_channels.max_count",
"http.client_stats.enabled",
"http.compression",
"http.compression_level",
"http.connection_pool_ttl",
"http.connection_read_timeout",
"http.connect_timeout",
"http.cors.enabled",
"http.detailed_errors.enabled",
"http.host",
"http.max_connections",
"http.max_content_length",
"http.max_endpoint_connections",
"http.max_header_size",
"http.max_initial_line_length",
"http.max_warning_header_count",
"http.max_warning_header_size",
"http.pipelining.max_events",
"http.port",
"http.proxy.host",
"http.proxy.port",
"http.proxy.scheme",
"http.publish_host",
"http.publish_port",
"http.socket_timeout",
"http.tcp.keep_alive",
"http.tcp.keep_count",
"http.tcp.keep_idle",
"http.tcp.keep_interval",
"http.tcp.no_delay",
"http.tcp.receive_buffer_size",
"http.tcp.reuse_address",
"http.tcp.send_buffer_size",
"idp.entity_id",
"idp.metadata.http.connect_timeout",
"idp.metadata.http.fail_on_error",
"idp.metadata.http.minimum_refresh",
"idp.metadata.http.read_timeout",
"idp.metadata.http.refresh",
"idp.metadata.path",
"idp.use_single_logout",
"index.dlm.frozen.created",
"indexing_pressure.memory.limit",
"index.lifecycle.indexing_complete",
"index.lifecycle.name",
"index.lifecycle.origination_date",
"index.lifecycle.parse_origination_date",
"index.lifecycle.prefer_ilm",
"index.lifecycle.rollover_alias",
"index.lifecycle.step.wait_time_threshold",
"index.name.time_format",
"index.queries.cache.enabled",
"indices.breaker.fielddata.limit",
"indices.breaker.fielddata.overhead",
"indices.breaker.request.limit",
"indices.breaker.request.overhead",
"indices.breaker.total.limit",
"indices.breaker.total.use_real_memory",
"indices.fielddata.cache.size",
"indices.lifecycle.history_index_enabled",
"indices.lifecycle.poll_interval",
"indices.lifecycle.rollover.only_if_has_documents",
"indices.memory.index_buffer_size",
"indices.memory.max_index_buffer_size",
"indices.memory.min_index_buffer_size",
"indices.queries.cache.size",
"indices.query.bool.max_clause_count",
"indices.query.bool.max_nested_depth",
"indices.recovery.max_bytes_per_sec",
"indices.recovery.max_concurrent_file_chunks",
"indices.recovery.max_concurrent_operations",
"indices.recovery.max_concurrent_snapshot_file_downloads",
"indices.recovery.max_concurrent_snapshot_file_downloads_per_node",
"indices.recovery.use_snapshots",
"indices.requests.cache.expire",
"indices.requests.cache.size",
"jwt.cache.size",
"jwt.cache.ttl",
"keytab.path",
"krb.debug",
"load_balance.cache_ttl",
"load_balance.type",
"master_history.max_age",
"monitor.fs.health.enabled",
"monitor.fs.health.refresh_interval",
"monitor.fs.health.slow_path_logging_threshold",
"nameid.allow_create",
"nameid.sp_qualifier",
"network.bind_host",
"network.breaker.inflight_requests.limit",
"network.breaker.inflight_requests.overhead",
"network.host",
"network.publish_host",
"network.tcp.keep_alive",
"network.tcp.keep_count",
"network.tcp.keep_idle",
"network.tcp.keep_interval",
"network.tcp.no_delay",
"network.tcp.receive_buffer_size",
"network.tcp.reuse_address",
"network.tcp.send_buffer_size",
"network.thread.watchdog.interval",
"network.thread.watchdog.quiet_time",
"node.bandwidth.recovery.disk.read",
"node.bandwidth.recovery.disk.write",
"node.bandwidth.recovery.factor.read",
"node.bandwidth.recovery.factor.write",
"node.bandwidth.recovery.network",
"node.bandwidth.recovery.operator.factor",
"node.bandwidth.recovery.operator.factor.max_overcommit",
"node.bandwidth.recovery.operator.factor.read",
"node.bandwidth.recovery.operator.factor.write",
"node.roles: [ ml ]",
"node.roles: [ transform ]",
"op.authorization_endpoint",
"op.endsession_endpoint",
"op.issuer",
"op.jwkset_path",
"op.token_endpoint",
"op.userinfo_endpoint",
"org.elasticsearch.common.network.ThreadWatchdog",
"org.elasticsearch.transport.InboundHandler",
"org.elasticsearch.transport.OutboundHandler",
"pkc_jwkset_reload.enabled",
"pkc_jwkset_reload.file_interval",
"pkc_jwkset_reload.url_interval_max",
"pkc_jwkset_reload.url_interval_min",
"proxy.base_path",
"reindex.remote.blocklist",
"reindex.remote.whitelist",
"reindex.ssl.certificate",
"reindex.ssl.certificate_authorities",
"reindex.ssl.key",
"reindex.ssl.key_passphrase",
"reindex.ssl.keystore.key_password",
"reindex.ssl.keystore.password",
"reindex.ssl.keystore.path",
"reindex.ssl.keystore.secure_key_password",
"reindex.ssl.keystore.secure_password",
"reindex.ssl.keystore.type",
"reindex.ssl.secure_key_passphrase",
"reindex.ssl.truststore.password",
"reindex.ssl.truststore.path",
"reindex.ssl.truststore.secure_password",
"reindex.ssl.truststore.type",
"reindex.ssl.verification_mode",
"remote_cluster.bind_host",
"remote_cluster.host",
"remote_cluster.port",
"remote_cluster.publish_host",
"remote_cluster.publish_port",
"remote_cluster_server.enabled",
"remote_cluster.tcp.keep_alive",
"remote_cluster.tcp.keep_count",
"remote_cluster.tcp.keep_idle",
"remote_cluster.tcp.keep_interval",
"remote_cluster.tcp.no_delay",
"remote_cluster.tcp.receive_buffer_size",
"remote_cluster.tcp.reuse_address",
"remote_cluster.tcp.send_buffer_size",
"repositories.default_repository",
"rp.client_auth_jwt_signature_algorithm",
"rp.client_auth_method",
"rp.client_id",
"rp.client_secret",
"rp.post_logout_redirect_uri",
"rp.redirect_uri",
"rp.requested_scopes",
"rp.response_type",
"rp.signature_algorithm",
"script.max_compilations_rate",
"script.painless.regex.enabled",
"search.aggs.allowed_inline_metric_scripts",
"search.aggs.allowed_stored_metric_scripts",
"search.aggs.only_allowed_metric_scripts",
"search.max_buckets",
"search.task_watchdog.cooldown_period",
"search.task_watchdog.coordinator_threshold",
"search.task_watchdog.data_node_threshold",
"search.task_watchdog.enabled",
"search.task_watchdog.interval",
"signing.certificate",
"signing.key",
"signing.keystore.alias",
"signing.keystore.path",
"signing.keystore.secure_key_password",
"signing.keystore.secure_password",
"signing.keystore.type",
"signing.saml_messages",
"signing.secure_key_passphrase",
"slm.health.failed_snapshot_warn_threshold",
"slm.history_index_enabled",
"slm.retention_duration",
"slm.retention_schedule",
"smtp.auth",
"smtp.connection_timeout",
"smtp.host",
"smtp.local_address",
"smtp.local_port",
"smtp.port",
"smtp.secure_password",
"smtp.send_partial",
"smtp.ssl.trust",
"smtp.starttls.enable",
"smtp.starttls.required",
"smtp.timeout",
"smtp.user",
"smtp.wait_on_quit",
"smtp.write_timeout",
"snapshot.max_concurrent_operations",
"sp.acs",
"sp.entity_id",
"sp.logout",
"ssl.certificate",
"ssl.certificate_authorities",
"ssl.cipher_suites",
"ssl.key",
"ssl.key_passphrase",
"ssl.keystore.key_password",
"ssl.keystore.password",
"ssl.keystore.path",
"ssl.keystore.secure_key_password",
"ssl.keystore.secure_password",
"ssl.keystore.type",
"ssl.secure_key_passphrase",
"ssl.secure_keystore.password",
"ssl.supported_protocols",
"ssl.truststore.password",
"ssl.truststore.path",
"ssl.truststore.secure_password",
"ssl.truststore.type",
"ssl.verification_mode",
"stack.templates.enabled",
"timeout.ldap_search",
"timeout.response",
"timeout.tcp_connect",
"timeout.tcp_read",
"transport.bind_host",
"transport.compress",
"transport.compression_scheme",
"transport.connect_timeout",
"transport.host",
"transport.ping_schedule",
"transport.port",
"transport.profiles.$PROFILE.xpack.security.filter.allow",
"transport.profiles.$PROFILE.xpack.security.filter.deny",
"transport.publish_host",
"transport.publish_port",
"transport.tcp.keep_alive",
"transport.tcp.keep_count",
"transport.tcp.keep_idle",
"transport.tcp.keep_interval",
"transport.tcp.no_delay",
"transport.tcp.receive_buffer_size",
"transport.tcp.reuse_address",
"transport.tcp.send_buffer_size",
"truststore.algorithm",
"truststore.password",
"truststore.path",
"truststore.secure_password",
"user_search.attribute",
"user_search.base_dn",
"user_search.down_level_filter",
"user_search.filter",
"user_search.pool.enabled",
"user_search.pool.health_check.dn",
"user_search.pool.health_check.enabled",
"user_search.pool.health_check.interval",
"user_search.pool.initial_size",
"user_search.pool.size",
"user_search.scope",
"user_search.upn_filter",
"vectors.indexing.use_gpu",
"wait_master.timeout",
"xpack.http.connection_pool_ttl",
"xpack.http.default_connection_timeout",
"xpack.http.default_read_timeout",
"xpack.http.max_response_size",
"xpack.http.proxy.host",
"xpack.http.proxy.port",
"xpack.http.proxy.scheme",
"xpack.http.ssl.certificate",
"xpack.http.ssl.certificate_authorities",
"xpack.http.ssl.cipher_suites",
"xpack.http.ssl.key",
"xpack.http.ssl.keystore.path",
"xpack.http.ssl.keystore.secure_key_password",
"xpack.http.ssl.keystore.secure_password",
"xpack.http.ssl.keystore.type",
"xpack.http.ssl.secure_key_passphrase",
"xpack.http.ssl.supported_protocols",
"xpack.http.ssl.truststore.path",
"xpack.http.ssl.truststore.secure_password",
"xpack.http.ssl.truststore.type",
"xpack.http.ssl.verification_mode",
"xpack.http.tcp.keep_alive",
"xpack.http.whitelist",
"xpack.ilm.enabled",
"xpack.inference.ccm.cache.expiry_time",
"xpack.inference.ccm.cache.weight",
"xpack.inference.endpoint.cache.enabled",
"xpack.inference.endpoint.cache.expiry_time",
"xpack.inference.endpoint.cache.weight",
"xpack.inference.http.connection_eviction_interval",
"xpack.inference.http.connection_eviction_max_idle_time",
"xpack.inference.http.max_response_size",
"xpack.inference.http.max_route_connections",
"xpack.inference.http.max_total_connections",
"xpack.inference.http.request_executor.queue_capacity",
"xpack.inference.http.retry.initial_delay",
"xpack.inference.http.retry.max_delay_bound",
"xpack.inference.http.retry.timeout",
"xpack.inference.logging.reset_interval",
"xpack.inference.logging.wait_duration",
"xpack.inference.oauth2.token_cache.enabled",
"xpack.inference.oauth2.token_cache.expiry_time",
"xpack.inference.oauth2.token_cache.weight",
"xpack.inference.query_timeout",
"xpack.inference.truncator.reduction_percentage",
"xpack.license.self_generated.type",
"xpack.ml.anomalies.heal_reindexed_v7.enabled",
"xpack.ml.enable_config_migration",
"xpack.ml.enabled",
"xpack.ml.idle_job_auto_close_timeout",
"xpack.ml.inference_model.cache_size",
"xpack.ml.inference_model.time_to_live",
"xpack.ml.max_anomaly_records",
"xpack.ml.max_inference_processors",
"xpack.ml.max_lazy_ml_nodes",
"xpack.ml.max_machine_memory_percent",
"xpack.ml.max_ml_node_size",
"xpack.ml.max_model_memory_limit",
"xpack.ml.max_open_jobs",
"xpack.ml.model_repository",
"xpack.ml.nightly_maintenance_requests_per_second",
"xpack.ml.node_concurrent_job_allocations",
"xpack.ml.persist_results_max_retries",
"xpack.ml.process_connect_timeout",
"xpack.ml.results_index_rollover_max_size",
"xpack.ml.trained_models.graph_validation_enabled",
"xpack.ml.use_auto_machine_memory_percent",
"xpack.monitoring.collection.cluster.stats.timeout",
"xpack.monitoring.collection.enabled",
"xpack.monitoring.collection.index.recovery.active_only",
"xpack.monitoring.collection.index.recovery.timeout",
"xpack.monitoring.collection.index.stats.timeout",
"xpack.monitoring.collection.indices",
"xpack.monitoring.collection.interval",
"xpack.monitoring.collection.node.stats.timeout",
"xpack.monitoring.elasticsearch.collection.enabled",
"xpack.monitoring.enabled",
"xpack.monitoring.exporters",
"xpack.monitoring.exporters.$NAME.ssl.certificate",
"xpack.monitoring.exporters.$NAME.ssl.certificate_authorities",
"xpack.monitoring.exporters.$NAME.ssl.cipher_suites",
"xpack.monitoring.exporters.$NAME.ssl.key",
"xpack.monitoring.exporters.$NAME.ssl.key_passphrase",
"xpack.monitoring.exporters.$NAME.ssl.keystore.key_password",
"xpack.monitoring.exporters.$NAME.ssl.keystore.password",
"xpack.monitoring.exporters.$NAME.ssl.keystore.path",
"xpack.monitoring.exporters.$NAME.ssl.keystore.secure_key_password",
"xpack.monitoring.exporters.$NAME.ssl.keystore.secure_password",
"xpack.monitoring.exporters.$NAME.ssl.keystore.type",
"xpack.monitoring.exporters.$NAME.ssl.secure_key_passphrase",
"xpack.monitoring.exporters.$NAME.ssl.supported_protocols",
"xpack.monitoring.exporters.$NAME.ssl.truststore.password",
"xpack.monitoring.exporters.$NAME.ssl.truststore.path",
"xpack.monitoring.exporters.$NAME.ssl.truststore.secure_password",
"xpack.monitoring.exporters.$NAME.ssl.truststore.type",
"xpack.monitoring.exporters.$NAME.ssl.verification_mode",
"xpack.monitoring.history.duration",
"xpack.notification.email.account",
"xpack.notification.email.account.domain_allowlist",
"xpack.notification.email.default_account",
"xpack.notification.email.html.sanitization.allow",
"xpack.notification.email.html.sanitization.disallow",
"xpack.notification.email.html.sanitization.enabled",
"xpack.notification.email.recipient_allowlist",
"xpack.notification.email.ssl.certificate",
"xpack.notification.email.ssl.certificate_authorities",
"xpack.notification.email.ssl.cipher_suites",
"xpack.notification.email.ssl.key",
"xpack.notification.email.ssl.keystore.path",
"xpack.notification.email.ssl.keystore.secure_key_password",
"xpack.notification.email.ssl.keystore.secure_password",
"xpack.notification.email.ssl.keystore.type",
"xpack.notification.email.ssl.secure_key_passphrase",
"xpack.notification.email.ssl.supported_protocols",
"xpack.notification.email.ssl.truststore.path",
"xpack.notification.email.ssl.truststore.secure_password",
"xpack.notification.email.ssl.truststore.type",
"xpack.notification.email.ssl.verification_mode",
"xpack.notification.jira.account",
"xpack.notification.jira.default_account",
"xpack.notification.pagerduty",
"xpack.notification.pagerduty.account",
"xpack.notification.pagerduty.default_account",
"xpack.notification.slack",
"xpack.notification.slack.account",
"xpack.notification.slack.default_account",
"xpack.notification.webhook.additional_token_enabled",
"xpack.otel_data.histogram_field_type",
"xpack.otel_data.registry.enabled",
"xpack.profiling.enabled",
"xpack.profiling.templates.enabled",
"xpack.security.audit.enabled",
"xpack.security.audit.logfile.emit_node_host_address",
"xpack.security.audit.logfile.emit_node_host_name",
"xpack.security.audit.logfile.emit_node_id",
"xpack.security.audit.logfile.emit_node_name",
"xpack.security.audit.logfile.events.emit_request_body",
"xpack.security.audit.logfile.events.exclude",
"xpack.security.audit.logfile.events.include",
"xpack.security.authc.anonymous.authz_exception",
"xpack.security.authc.anonymous.roles",
"xpack.security.authc.anonymous.username",
"xpack.security.authc.api_key.cache.hash_algo",
"xpack.security.authc.api_key.cache.max_keys",
"xpack.security.authc.api_key.cache.ttl",
"xpack.security.authc.api_key.delete.interval",
"xpack.security.authc.api_key.delete.retention_period",
"xpack.security.authc.api_key.delete.timeout",
"xpack.security.authc.api_key.enabled",
"xpack.security.authc.api_key.hashing.algorithm",
"xpack.security.authc.password_hashing.algorithm",
"xpack.security.authc.realms.jwt.*",
"xpack.security.authc.realms.kerberos.*",
"xpack.security.authc.realms.oidc.*",
"xpack.security.authc.realms.saml.*",
"xpack.security.authc.token.enabled",
"xpack.security.authc.token.timeout",
"xpack.security.autoconfiguration.enabled",
"xpack.security.automata.cache.enabled",
"xpack.security.automata.cache.size",
"xpack.security.automata.cache.ttl",
"xpack.security.automata.max_determinized_states",
"xpack.security.dls.bitset.cache.size",
"xpack.security.dls.bitset.cache.ttl",
"xpack.security.dls_fls.enabled",
"xpack.security.enabled",
"xpack.security.enrollment.enabled",
"xpack.security.fips_mode.enabled",
"xpack.security.fips_mode.required_providers",
"xpack.security.hide_settings",
"xpack.security.http.filter.allow",
"xpack.security.http.filter.deny",
"xpack.security.http.ssl.certificate",
"xpack.security.http.ssl.certificate_authorities",
"xpack.security.http.ssl.cipher_suites",
"xpack.security.http.ssl.client_authentication",
"xpack.security.http.ssl.enabled",
"xpack.security.http.ssl.key",
"xpack.security.http.ssl.key_passphrase",
"xpack.security.http.ssl.keystore.key_password",
"xpack.security.http.ssl.keystore.password",
"xpack.security.http.ssl.keystore.path",
"xpack.security.http.ssl.keystore.secure_key_password",
"xpack.security.http.ssl.keystore.secure_password",
"xpack.security.http.ssl.keystore.type",
"xpack.security.http.ssl.secure_key_passphrase",
"xpack.security.http.ssl.supported_protocols",
"xpack.security.http.ssl.truststore.password",
"xpack.security.http.ssl.truststore.path",
"xpack.security.http.ssl.truststore.secure_password",
"xpack.security.http.ssl.truststore.type",
"xpack.security.http.ssl.verification_mode",
"xpack.security.loginAssistanceMessage",
"xpack.security.remote_cluster_client.ssl.certificate",
"xpack.security.remote_cluster_client.ssl.certificate_authorities",
"xpack.security.remote_cluster_client.ssl.cipher_suites",
"xpack.security.remote_cluster_client.ssl.enabled",
"xpack.security.remote_cluster_client.ssl.handshake_timeout",
"xpack.security.remote_cluster_client.ssl.key",
"xpack.security.remote_cluster_client.ssl.keystore.path",
"xpack.security.remote_cluster_client.ssl.keystore.secure_key_password",
"xpack.security.remote_cluster_client.ssl.keystore.secure_password",
"xpack.security.remote_cluster_client.ssl.keystore.type",
"xpack.security.remote_cluster_client.ssl.secure_key_passphrase",
"xpack.security.remote_cluster_client.ssl.supported_protocols",
"xpack.security.remote_cluster_client.ssl.truststore.path",
"xpack.security.remote_cluster_client.ssl.truststore.secure_password",
"xpack.security.remote_cluster_client.ssl.truststore.type",
"xpack.security.remote_cluster_client.ssl.verification_mode",
"xpack.security.remote_cluster.filter.allow",
"xpack.security.remote_cluster.filter.deny",
"xpack.security.remote_cluster_server.ssl.certificate",
"xpack.security.remote_cluster_server.ssl.certificate_authorities",
"xpack.security.remote_cluster_server.ssl.cipher_suites",
"xpack.security.remote_cluster_server.ssl.client_authentication",
"xpack.security.remote_cluster_server.ssl.enabled",
"xpack.security.remote_cluster_server.ssl.handshake_timeout",
"xpack.security.remote_cluster_server.ssl.key",
"xpack.security.remote_cluster_server.ssl.keystore.path",
"xpack.security.remote_cluster_server.ssl.keystore.secure_key_password",
"xpack.security.remote_cluster_server.ssl.keystore.secure_password",
"xpack.security.remote_cluster_server.ssl.keystore.type",
"xpack.security.remote_cluster_server.ssl.secure_key_passphrase",
"xpack.security.remote_cluster_server.ssl.supported_protocols",
"xpack.security.remote_cluster_server.ssl.truststore.path",
"xpack.security.remote_cluster_server.ssl.truststore.secure_password",
"xpack.security.remote_cluster_server.ssl.truststore.type",
"xpack.security.remote_cluster_server.ssl.verification_mode",
"xpack.security.ssl.diagnose.trust",
"xpack.security.transport.filter.allow",
"xpack.security.transport.filter.deny",
"xpack.security.transport.ssl.certificate",
"xpack.security.transport.ssl.certificate_authorities",
"xpack.security.transport.ssl.cipher_suites",
"xpack.security.transport.ssl.client_authentication",
"xpack.security.transport.ssl.enabled",
"xpack.security.transport.ssl.handshake_timeout",
"xpack.security.transport.ssl.key",
"xpack.security.transport.ssl.key_passphrase",
"xpack.security.transport.ssl.keystore.key_password",
"xpack.security.transport.ssl.keystore.password",
"xpack.security.transport.ssl.keystore.path",
"xpack.security.transport.ssl.keystore.secure_key_password",
"xpack.security.transport.ssl.keystore.secure_password",
"xpack.security.transport.ssl.keystore.type",
"xpack.security.transport.ssl.secure_key_passphrase",
"xpack.security.transport.ssl.supported_protocols",
"xpack.security.transport.ssl.trust_restrictions.x509_fields",
"xpack.security.transport.ssl.truststore.password",
"xpack.security.transport.ssl.truststore.path",
"xpack.security.transport.ssl.truststore.secure_password",
"xpack.security.transport.ssl.truststore.type",
"xpack.security.transport.ssl.verification_mode",
"xpack.transform.enabled",
"xpack.transform.num_transform_failure_retries",
"xpack.watcher.enabled",
"xpack.watcher.encryption_key",
"xpack.watcher.encrypt_sensitive_data",
"xpack.watcher.history.cleaner_service.enabled",
"xpack.watcher.max.history.record.size",
"xpack.watcher.trigger.schedule.engine",
}

View File

@@ -18,14 +18,12 @@ package es
import (
"context"
"errors"
"fmt"
"regexp"
"strings"
"sync"
"codeberg.org/scip/esctl/pkg/cfg"
"github.com/elastic/go-elasticsearch/v9"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/elastic/go-elasticsearch/v9/typedapi/cluster/health"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
@@ -35,12 +33,11 @@ const (
)
func checkClusterIsLeader(conf *cfg.Config, leader string) bool {
stats, err := conf.Clusters[leader].ES.Ccr.Stats().
Header("content-type", "application/json").
Header("accept", "application/json").
stats, err := conf.Clusters[leader].ES().Ccr.Stats().
Do(context.Background())
if err != nil {
fmt.Printf("failed to get ccr stats from %s: %s", leader, err)
fmt.Printf("failed to get ccr stats from %s: %s", leader, esErrorString(err))
return false
}
@@ -51,6 +48,7 @@ func checkClusterIsLeader(conf *cfg.Config, leader string) bool {
if stats.AutoFollowStats.AutoFollowedClusters[0].ClusterName != "" {
fmt.Println("leader/follower attribution is invalid, reverse cluster attribution and retry")
return false
}
@@ -62,50 +60,53 @@ func checkClusterStatus(conf *cfg.Config, leader, follower string) bool {
status := map[string]*health.Response{}
for _, cluster := range []string{leader, follower} {
st, err := conf.Clusters[leader].ES.Cluster.Health().
Header("content-type", "application/json").
Header("accept", "application/json").
clusterHealth, err := conf.Clusters[leader].ES().Cluster.Health().
Do(context.Background())
if err != nil {
fmt.Printf("failed to get health from %s: %s", cluster, err)
fmt.Printf("failed to get health from %s: %s", cluster, esErrorString(err))
return false
}
status[cluster] = st
status[cluster] = clusterHealth
}
table := NewTable(3, 5)
table := printer.NewTable(conf).WithSize(3, 6)
table.Addheaders("setting", "leader:"+leader, "follower:"+follower)
table.entries = [][]string{
{"Cluster Name",
Colorize(status[leader].Status.Name, status[leader].ClusterName),
Colorize(status[follower].Status.Name, status[follower].ClusterName),
table.Entries = [][]any{
{"Cluster Name", status[leader].ClusterName, status[follower].ClusterName},
{"Cluster Status",
printer.Colorize(conf, status[leader].Status.Name, status[leader].Status.Name),
printer.Colorize(conf, status[follower].Status.Name, status[follower].Status.Name),
},
{"Active Shards",
fmt.Sprintf("%d", status[leader].ActiveShards),
fmt.Sprintf("%d", status[follower].ActiveShards),
status[leader].ActiveShards,
status[follower].ActiveShards,
},
{"Active Primary Shards",
fmt.Sprintf("%d", status[leader].ActivePrimaryShards),
fmt.Sprintf("%d", status[follower].ActivePrimaryShards),
status[leader].ActivePrimaryShards,
status[follower].ActivePrimaryShards,
},
{"Indicies",
fmt.Sprintf("%d", len(status[leader].Indices)),
fmt.Sprintf("%d", len(status[follower].Indices)),
{"indices",
len(status[leader].Indices),
len(status[follower].Indices),
},
{"Nodes",
fmt.Sprintf("%d", status[leader].NumberOfNodes),
fmt.Sprintf("%d", status[follower].NumberOfNodes),
status[leader].NumberOfNodes,
status[follower].NumberOfNodes,
},
}
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
fmt.Println(err)
return false
}
fmt.Println()
if status[leader].Status.Name == "green" && status[follower].Status.Name == "green" {
return true
}
@@ -115,26 +116,25 @@ func checkClusterStatus(conf *cfg.Config, leader, follower string) bool {
// finds indices on both clusters which have ilm errors
func findIlmErrors(conf *cfg.Config, leader, follower string) bool {
failed := map[string]map[string]string{}
failed := map[string]map[string]any{}
for _, cluster := range []string{leader, follower} {
ilm, err := conf.Clusters[cluster].ES.Ilm.ExplainLifecycle("_all").
ilm, err := conf.Clusters[cluster].ES().Ilm.ExplainLifecycle("_all").
OnlyManaged(true).
Header("content-type", "application/json").
Header("accept", "application/json").
Do(context.Background())
if err != nil {
fmt.Printf("failed to get ilm status from %s: %s", cluster, err)
fmt.Printf("failed to get ilm status from %s: %s", cluster, esErrorString(err))
return false
}
failed[cluster] = map[string]string{}
failed[cluster] = map[string]any{}
for name, ilmstate := range ilm.Indices {
count := ilmstate.(*types.LifecycleExplainManaged).FailedStepRetryCount
if count != nil && *count > 0 {
failed[cluster][name] = fmt.Sprintf("%d", *count)
failed[cluster][name] = *count
}
}
}
@@ -151,18 +151,19 @@ func findIlmErrors(conf *cfg.Config, leader, follower string) bool {
if len(failed[cluster]) > 0 {
idx := 0
table := NewTable(2, len(failed[cluster]))
table := printer.NewTable(conf).WithSize(2, len(failed[cluster]))
table.Addheaders("ilm errors on "+which, "errors")
for name, count := range failed[cluster] {
table.entries[idx] = []string{name, count}
table.Entries[idx] = []any{name, count}
idx++
}
table.Sort()
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
fmt.Println(err)
return false
}
}
@@ -171,7 +172,7 @@ func findIlmErrors(conf *cfg.Config, leader, follower string) bool {
return false
}
// find unsynchronized indicies only present on leader
// find unsynchronized indices only present on leader
func findIndicesOnlyOnLeader(conf *cfg.Config, indices ClusterIndices, leader, follower string) bool {
exclude := regexp.MustCompile(DefaultExclude)
if conf.Exclude != "" {
@@ -179,6 +180,7 @@ func findIndicesOnlyOnLeader(conf *cfg.Config, indices ClusterIndices, leader, f
}
indexOnlyOnLeader := map[string]*types.IndicesRecord{}
for name, index := range indices[leader] {
if exclude.MatchString(name) {
continue
@@ -187,9 +189,7 @@ func findIndicesOnlyOnLeader(conf *cfg.Config, indices ClusterIndices, leader, f
_, followerHasIt := indices[follower][name]
if !followerHasIt {
// fetch index details
res, err := conf.Clusters[leader].ES.Indices.Get(name).
Header("content-type", "application/json").
Header("accept", "application/json").
res, err := conf.Clusters[leader].ES().Indices.Get(name).
Do(context.Background())
if err != nil {
continue // ignore it then
@@ -202,9 +202,11 @@ func findIndicesOnlyOnLeader(conf *cfg.Config, indices ClusterIndices, leader, f
}
isWritable := false
for _, alias := range res[name].Aliases {
if *alias.IsWriteIndex {
isWritable = true
break
}
}
@@ -223,19 +225,21 @@ func findIndicesOnlyOnLeader(conf *cfg.Config, indices ClusterIndices, leader, f
}
idx := 0
table := NewTable(3, len(indexOnlyOnLeader))
table := printer.NewTable(conf).WithSize(3, len(indexOnlyOnLeader))
table.Addheaders("index only on leader", "size", "docscount")
for name, index := range indexOnlyOnLeader {
name := Colorize("red", name)
name := printer.Colorize(conf, "red", name)
table.entries[idx] = []string{name, *index.DatasetSize, *index.DocsCount}
table.Entries[idx] = []any{name, *index.DatasetSize, *index.DocsCount}
idx++
}
table.Sort()
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
fmt.Println(err)
return false
}
@@ -250,9 +254,7 @@ func findOrphanedIndices(conf *cfg.Config, indices ClusterIndices, leader, follo
_, leaderHasIt := indices[leader][name]
if !leaderHasIt {
// fetch index details
res, err := conf.Clusters[follower].ES.Indices.Get(name).
Header("content-type", "application/json").
Header("accept", "application/json").
res, err := conf.Clusters[follower].ES().Indices.Get(name).
Do(context.Background())
if err != nil {
continue // ignore it then
@@ -273,19 +275,21 @@ func findOrphanedIndices(conf *cfg.Config, indices ClusterIndices, leader, follo
}
idx := 0
table := NewTable(3, len(orphaned))
table := printer.NewTable(conf).WithSize(3, len(orphaned))
table.Addheaders("orphaned index on follower", "size", "docscount")
for name, index := range orphaned {
name := Colorize("red", name)
name := printer.Colorize(conf, "red", name)
table.entries[idx] = []string{name, *index.DatasetSize, *index.DocsCount}
table.Entries[idx] = []any{name, *index.DatasetSize, *index.DocsCount}
idx++
}
table.Sort()
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
fmt.Println(err)
return false
}
@@ -307,19 +311,21 @@ func findFailedFollowerIndices(conf *cfg.Config, indices ClusterIndices, followe
}
idx := 0
table := NewTable(3, len(red))
table := printer.NewTable(conf).WithSize(3, len(red))
table.Addheaders("red index on follower", "size", "docscount")
for name, index := range red {
name := Colorize("red", name)
name := printer.Colorize(conf, "red", name)
table.entries[idx] = []string{name, *index.DatasetSize, *index.DocsCount}
table.Entries[idx] = []any{name, *index.DatasetSize, *index.DocsCount}
idx++
}
table.Sort()
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
fmt.Println(err)
return false
}
@@ -338,58 +344,3 @@ func splitArg(arg string) (string, string) {
return parts[0], parts[1]
}
}
func getClusterData(es *elasticsearch.TypedClient, wg *sync.WaitGroup, reschan chan apiResponse, which string) {
defer wg.Done()
ar := apiResponse{}
arerr := errors.New("")
switch which {
case "health":
res, err := es.Cluster.Health().
Header("content-type", "application/json").
Header("accept", "application/json").
Do(context.Background())
ar.health = res
ar.which = ResponseHealth
arerr = err
case "info":
res, err := es.Info().
Header("content-type", "application/json").
Header("accept", "application/json").
Do(context.Background())
ar.info = res
ar.which = ResponseInfo
arerr = err
case "ccrstats":
res, err := es.Ccr.Stats().
Header("content-type", "application/json").
Header("accept", "application/json").
Do(context.Background())
ar.ccr = res
ar.which = ResponseCcr
arerr = err
case "stats":
res, err := es.Cluster.Stats().
Header("content-type", "application/json").
Header("accept", "application/json").
Do(context.Background())
ar.stats = res
ar.which = ResponseStats
arerr = err
}
if arerr != nil {
ar.error = fmt.Errorf("failed to get cluster health: %s", arerr)
}
reschan <- ar
}

247
pkg/es/datastream.go Normal file
View File

@@ -0,0 +1,247 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"errors"
"fmt"
"log/slog"
"regexp"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
func DatastreamNames(conf *cfg.Config) ([]string, error) {
res, err := conf.DefaultCluster.ES().Indices.GetDataStream().
Do(context.Background())
if err != nil {
return nil, fmt.Errorf("failed to get data streams: %w", esErrorString(err))
}
dss := make([]string, len(res.DataStreams))
for idx, ds := range res.DataStreams {
dss[idx] = ds.Name
}
return dss, nil
}
func DatastreamList(conf *cfg.Config) error {
res, err := conf.DefaultCluster.ES().Indices.GetDataStream().
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get data streams: %w", esErrorString(err))
}
slog.Debug("ES result", "data streams", res)
list := filterDatastreams(conf, res.DataStreams)
size := len(list)
if conf.MaxItems > 0 {
if size > conf.MaxItems {
size = conf.MaxItems
}
}
table := printer.NewTable(conf).WithSize(7, size)
table.Addheaders("name", "ilm policy", "hidden", "system", "replicated", "generation", "timestamp field")
for idx, datastream := range list {
name := printer.Colorize(conf, datastream.Status.String(), datastream.Name)
policy := ""
if datastream.IlmPolicy != nil {
policy = *datastream.IlmPolicy
}
table.Entries[idx] = []any{
name,
policy,
datastream.Hidden,
*datastream.System,
*datastream.Replicated,
datastream.Generation,
datastream.TimestampField.Name,
}
if idx == size-1 {
break
}
}
table.Sort()
if err := table.Print(); err != nil {
return err
}
return nil
}
func filterDatastreams(conf *cfg.Config, list []types.DataStream) []types.DataStream {
selected := []types.DataStream{}
for _, ds := range list {
if !conf.Hidden && ds.Hidden {
continue
}
selected = append(selected, ds)
}
if len(conf.Filter) == 0 {
return selected
}
// we support just one filter here, for now
filter := *regexp.MustCompile(conf.Filter[0])
newlist := []types.DataStream{}
for _, ds := range selected {
if filter.MatchString(ds.Name) {
newlist = append(newlist, ds)
}
}
return newlist
}
func DatastreamShow(conf *cfg.Config, dsname string) error {
res, err := conf.DefaultCluster.ES().Indices.GetDataStream().
Name(dsname).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get data stream: %w", esErrorString(err))
}
slog.Debug("ES result", "data stream", res)
if len(res.DataStreams) == 0 {
return errors.New("no data stream found with that name")
}
stats, err := conf.DefaultCluster.ES().Indices.DataStreamsStats().
Name(dsname).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get data stream stats: %w", esErrorString(err))
}
table := printer.NewTable(conf).WithSize(11, 2)
table.Addheaders("data stream property", "value")
datastream := res.DataStreams[0]
name := printer.Colorize(conf, datastream.Status.String(), datastream.Name)
policy := ""
if datastream.IlmPolicy != nil {
policy = *datastream.IlmPolicy
}
table.Entries = [][]any{
{"name", name},
{"ilm policy", policy},
{"hidden", datastream.Hidden},
{"system", *datastream.System},
{"replicated", *datastream.Replicated},
{"generation", datastream.Generation},
{"timestamp field", datastream.TimestampField.Name},
{"backing indices", stats.BackingIndices},
{"size", printer.Bytes(stats.TotalStoreSizeBytes)},
{"shards-failed", stats.Shards_.Failed},
{"shards-successful", stats.Shards_.Successful},
}
if err := table.Print(); err != nil {
return err
}
table = printer.NewTable(conf).WithSize(5, len(datastream.Indices))
table.Addheaders("backing index name", "uuid", "prefer ilm", "ilm policy", "managed by")
for idx, index := range datastream.Indices {
policy := ""
if datastream.IlmPolicy != nil {
policy = *datastream.IlmPolicy
}
table.Entries[idx] = []any{
index.IndexName,
index.IndexUuid,
*index.PreferIlm,
policy,
index.ManagedBy.Name,
}
if idx < len(datastream.Indices) {
fmt.Println()
}
}
table.Sort()
if err := table.Print(); err != nil {
return err
}
return nil
}
func DatastreamCreate(conf *cfg.Config, dsname string) error {
_, err := conf.DefaultCluster.ES().Indices.CreateDataStream(dsname).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to create datastream: %w", esErrorString(err))
}
return nil
}
func DatastreamDelete(conf *cfg.Config, dsname string) error {
_, err := conf.DefaultCluster.ES().Indices.DeleteDataStream(dsname).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to delete datastream: %w", esErrorString(err))
}
return nil
}
func DatastreamRollover(conf *cfg.Config, ds string) error {
res, err := RolloverAlias(conf, ds)
if err != nil {
return fmt.Errorf("failed to rollover data stream: %w", esErrorString(err))
}
table := printer.NewTable(conf).WithSize(2, 5)
table.Addheaders("rollover response", "value")
table.Entries = [][]any{
{"acknowledged", res.Acknowledged},
{"rolled over", res.RolledOver},
{"shards acknowledged", res.ShardsAcknowledged},
{"old index", res.OldIndex},
{"new index", res.NewIndex},
}
return table.Print()
}

48
pkg/es/debug.go Normal file
View File

@@ -0,0 +1,48 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"codeberg.org/scip/esctl/pkg/cfg"
"github.com/alecthomas/repr"
)
func Debug(conf *cfg.Config) error {
report, err := getHealthReport(conf)
if err != nil {
return err
}
repr.Println(report)
/*
res, err := conf.DefaultCluster.ES().Search().
Index(conf.Index).
Size(0).
Aggregations(map[string]types.Aggregations{
"min_ts": *esdsl.NewMinAggregation().Field("@timestamp").AggregationsCaster(),
"max_ts": *esdsl.NewMaxAggregation().Field("@timestamp").AggregationsCaster(),
}).
Do(context.Background())
if err != nil {
return err
}
repr.Println(res)
*/
return nil
}

View File

@@ -19,10 +19,16 @@ package es
import (
"context"
"encoding/json"
"errors"
"fmt"
"time"
"math/rand/v2"
"strconv"
"strings"
"codeberg.org/scip/esctl/pkg/cfg"
"github.com/elastic/go-elasticsearch/v9/typedapi/core/deletebyquery"
"github.com/elastic/go-elasticsearch/v9/typedapi/esdsl"
"github.com/tidwall/gjson"
)
// create index with:
@@ -32,23 +38,87 @@ import (
// then add a doc:
//
// esctl doc add foo2 '{"id":"d8d8d","user":"scip"}'
func DocAdd(conf *cfg.Config, index, jsondoc string) error {
func DocAdd(conf *cfg.Config, jsondoc string) error {
data := map[string]any{}
err := json.Unmarshal([]byte(jsondoc), &data)
if err != nil {
return fmt.Errorf("supplied document was not valid JSON: %s", err)
return fmt.Errorf("supplied document was not valid JSON: %w", err)
}
now := fmt.Sprintf("%d", time.Now().Unix())
now := strconv.FormatInt(rand.Int64(), 10)
_, err = conf.DefaultCluster.ES.Create(index, now).
res, err := conf.DefaultCluster.ES().Create(conf.Index, now).
Document(data).
Header("content-type", "application/json").
Header("accept", "application/json").
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to create new doc in index %s: %s", index, err)
return fmt.Errorf("failed to create new doc in index %s: %w", conf.Index, esErrorString(err))
}
fmt.Println(res.Id_)
return nil
}
func DocShow(conf *cfg.Config, id string) error {
res, err := conf.DefaultCluster.ES().Get(conf.Index, id).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to retrieve doc in index %s: %w", conf.Index, esErrorString(err))
}
if !res.Found {
return errors.New("no document with that id found")
}
docjson := fmt.Sprintf(`{"id":%s, "index":"%s", "source":%s}`,
res.Id_,
res.Index_,
res.Source_)
if conf.Path != "" {
value := gjson.Get(docjson, conf.Path)
fmt.Println(value.String())
} else {
fmt.Println(docjson)
}
fmt.Println()
return nil
}
func DocDelete(conf *cfg.Config, queries []string) error {
if len(queries) == 1 && strings.Contains(queries[0], "id=") {
id, _ := strings.CutPrefix(queries[0], "id=")
_, err := conf.DefaultCluster.ES().Delete(conf.Index, id).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to delete doc in index %s: %w", conf.Index, esErrorString(err))
}
return nil
}
req := new(deletebyquery.Request{})
if len(queries) == 0 && conf.All {
req.Query = esdsl.NewMatchAllQuery().QueryCaster()
} else {
queryCaster, err := prepareQuery(conf, queries)
if err != nil {
return err
}
req.Query = queryCaster
}
_, err := conf.DefaultCluster.ES().DeleteByQuery(conf.Index).
Request(req).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to delete docs in index %s: %w", conf.Index, esErrorString(err))
}
return nil

127
pkg/es/duration.go Normal file
View File

@@ -0,0 +1,127 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"fmt"
"math"
"regexp"
"strconv"
"time"
)
var (
matchDuration = regexp.MustCompile(`(\d+)([dhms])`)
)
type esDuration struct {
days, hours, minutes, seconds int
}
/*
We could use time.ParseDuration(), but this doesn't support days.
We could also use github.com/xhit/go-str2duration/v2, which does
the job, but it's just another dependency, just for this little
gem. And we don't need a time.Time value. And int is good enough
for duration comparison.
Convert a duration into an integer. Valid time units are "s", "m", "h" and "d".
via https://codeberg.org/scip/tablizer/src/branch/main/lib/sort.go#L113
*/
func parseDuration(duration string) time.Duration {
seconds := 0
for _, match := range matchDuration.FindAllStringSubmatch(duration, -1) {
if len(match) == 3 {
durationvalue, _ := strconv.Atoi(match[1])
switch match[2][0] {
case 'd':
seconds += durationvalue * 86400
case 'h':
seconds += durationvalue * 3600
case 'm':
seconds += durationvalue * 60
case 's':
seconds += durationvalue
}
}
}
return time.Duration(seconds) * time.Second
}
// pretty much the same as above, but return a struct filled with int duration values
func extractDuration(duration string) esDuration {
esd := esDuration{}
for _, match := range matchDuration.FindAllStringSubmatch(duration, -1) {
if len(match) == 3 {
durationvalue, _ := strconv.Atoi(match[1])
switch match[2][0] {
case 'd':
esd.days = durationvalue
case 'h':
esd.hours = durationvalue
case 'm':
esd.minutes = durationvalue
case 's':
esd.seconds = durationvalue
}
}
}
return esd
}
// convert a duration like 156h6m15.095s-6d/12h/6m => 6d:12h:6m
func formatDuration(val time.Duration) string {
esd := extractDuration(val.String())
daysF := float64(esd.hours) / 24.0
days := math.Floor(daysF)
hoursLeft := (daysF - days) * 24
switch {
case val.Minutes() < 1:
return fmt.Sprintf("%ds", int64(esd.seconds))
case val.Hours() < 1:
return fmt.Sprintf("%dm", int64(esd.minutes))
case val.Hours() < 24:
if esd.minutes == 0 {
return fmt.Sprintf("%dh", esd.hours)
}
return fmt.Sprintf("%dh:%dm", esd.hours, int64(esd.minutes))
default:
if esd.minutes == 0 && hoursLeft == 0 {
return fmt.Sprintf("%dd", int64(days))
}
return fmt.Sprintf("%dd:%dh:%dm",
int64(days),
int64(hoursLeft),
esd.minutes,
)
}
}

47
pkg/es/errors.go Normal file
View File

@@ -0,0 +1,47 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"errors"
"fmt"
"strings"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
func esErrorString(err error) error {
msg := err.Error()
//nolint:gocritic
switch errVal := err.(type) {
case *types.ElasticsearchError:
var causes strings.Builder
for _, cause := range errVal.ErrorCause.RootCause {
fmt.Fprintf(&causes, "%s\n", *cause.Reason)
}
if errVal.ErrorCause.Reason != nil {
msg = *errVal.ErrorCause.Reason + ": " + causes.String()
} else {
msg = fmt.Sprintf("http status %d: ", errVal.Status)
}
}
return errors.New(msg)
}

530
pkg/es/ilm.go Normal file
View File

@@ -0,0 +1,530 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"maps"
"slices"
"strings"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/alecthomas/repr"
"github.com/charmbracelet/lipgloss"
"github.com/elastic/go-elasticsearch/v9/typedapi/esdsl"
"github.com/elastic/go-elasticsearch/v9/typedapi/ilm/putlifecycle"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
func IlmRetry(conf *cfg.Config, index string) error {
_, err := conf.DefaultCluster.ES().Ilm.Retry(index).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to retry ilm: %w", esErrorString(err))
}
return nil
}
func IlmStatus(conf *cfg.Config) error {
res, err := conf.DefaultCluster.ES().Ilm.GetStatus().
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get ilm status: %w", esErrorString(err))
}
fmt.Println(res.OperationMode.Name)
return nil
}
func IlmNames(conf *cfg.Config) ([]string, error) {
res, err := conf.DefaultCluster.ES().Ilm.GetLifecycle().
Do(context.Background())
if err != nil {
return nil, fmt.Errorf("failed to get ilm policies: %w", esErrorString(err))
}
names := slices.Collect(maps.Keys(res))
return names, nil
}
func IlmList(conf *cfg.Config, pattern string) error {
ilm := conf.DefaultCluster.ES().Ilm.GetLifecycle()
if pattern != "" {
ilm.FilterPath(pattern)
}
res, err := ilm.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get ilm policies: %w", esErrorString(err))
}
if conf.Debug {
repr.Println(res)
}
table := printer.NewTable(conf).WithSize(5, 0)
table.Addheaders("ilm policy", "hot", "warm", "frozen", "delete")
for name, ilm := range res {
table.AddRow(name,
ilmPhaseString(ilm.Policy.Phases.Hot, true),
ilmPhaseString(ilm.Policy.Phases.Warm, true),
ilmPhaseString(ilm.Policy.Phases.Frozen, true),
ilmPhaseString(ilm.Policy.Phases.Delete, true),
)
}
table.Sort()
return table.Print()
}
func IlmShow(conf *cfg.Config, policy string) error {
res, err := conf.DefaultCluster.ES().Ilm.GetLifecycle().
Policy(policy).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get ilm status: %w", esErrorString(err))
}
if conf.Debug {
repr.Println(res)
}
ilm, exists := res[policy]
if !exists {
return errors.New("no ilm policy retrieved")
}
if conf.Ilm.Tree {
return IlmShowTree(conf, ilm.Policy)
}
table := printer.NewTable(conf).WithSize(2, 5)
table.Addheaders("ilm policy setting", "value")
table.Entries = [][]any{
{"policy", policy},
{"hot phase", ilmPhaseString(ilm.Policy.Phases.Hot, false)},
{"warm phase", ilmPhaseString(ilm.Policy.Phases.Warm, false)},
{"frozen phase", ilmPhaseString(ilm.Policy.Phases.Frozen, false)},
{"delete phase", ilmPhaseString(ilm.Policy.Phases.Delete, false)},
}
return table.Print()
}
// Visualize phases or a ILM policy, taking into account that the
// minAge for the current phases is set in the next phase
func IlmShowTree(conf *cfg.Config, ilm types.IlmPolicy) error {
indent := ""
table := printer.NewTable(conf).WithSize(4, 0)
table.Addheaders("phase", "min age", "min size", "snapshot repo")
for _, phase := range IlmPhaseOrder {
if phase == "hot" {
fmt.Printf("%s%s phase:\n%s rollover after %s\n",
indent, phase,
indent, formatDuration(parseDuration(ilm.Phases.Hot.Actions.Rollover.MaxAge.(string))),
)
if ilm.Phases.Hot.Actions.Rollover.MaxPrimaryShardSize != "" {
fmt.Printf("%s rollover when storage > %s\n",
indent, ilm.Phases.Hot.Actions.Rollover.MaxPrimaryShardSize)
}
if ilm.Phases.Hot.Actions.Rollover.MaxDocs != nil {
fmt.Printf("%s rollover docs > %d\n",
indent, *ilm.Phases.Hot.Actions.Rollover.MaxDocs)
}
indent += " "
continue
}
current := getIlmCurrentPhase(ilm, phase)
if current == nil {
continue
}
next := findNextPhase(ilm, phase)
fmt.Printf("%s%s phase:\n", indent, phase)
if next != nil {
fmt.Printf("%s rollover after %s\n",
indent, formatDuration(next.minage),
)
if current.Actions.SearchableSnapshot != nil {
fmt.Printf("%s roll to snapshot repo: %s\n",
indent, current.Actions.SearchableSnapshot.SnapshotRepository)
}
if current.Actions.Forcemerge != nil {
fmt.Printf("%s force merge segments: %d\n",
indent, current.Actions.Forcemerge.MaxNumSegments)
}
if current.Actions.Shrink != nil {
fmt.Printf("%s shrink shards: %d\n",
indent, *current.Actions.Shrink.NumberOfShards)
}
if current.Actions.SetPriority != nil {
fmt.Printf("%s priority: %d\n",
indent, *current.Actions.SetPriority.Priority)
}
if current.Actions.Delete != nil && current.Actions.Delete.DeleteSearchableSnapshot != nil {
fmt.Printf("%s delete searchable snapshots: %t\n",
indent, *current.Actions.Delete.DeleteSearchableSnapshot)
}
}
if phase == "delete" {
fmt.Printf("%s delete immediately\n", indent)
}
indent += " "
}
return nil
}
func getIlmCurrentPhase(policy types.IlmPolicy, currentPhase string) *types.Phase {
switch currentPhase {
case "hot":
return policy.Phases.Hot
case "warm":
return policy.Phases.Warm
case "cold":
return policy.Phases.Cold
case "frozen":
return policy.Phases.Frozen
default:
return policy.Phases.Delete
}
}
func ilmPhaseString(phase *types.Phase, short bool) string {
if phase == nil {
return ""
}
out := []string{fmt.Sprintf("move-after:%s", phase.MinAge)}
if short {
return strings.Join(out, ",")
}
if phase.Actions.Rollover != nil {
if phase.Actions.Rollover.MaxAge != "" {
out = append(out, fmt.Sprintf("rollover:%s", phase.Actions.Rollover.MaxAge))
if phase.Actions.Rollover.MaxPrimaryShardSize != nil {
out = append(out, fmt.Sprintf("shardsize:%s", phase.Actions.Rollover.MaxPrimaryShardSize))
}
}
}
if phase.Actions.Delete != nil {
out = append(out, fmt.Sprintf("delete-searchable-snapshot:%t", *phase.Actions.Delete.DeleteSearchableSnapshot))
}
if phase.Actions.SearchableSnapshot != nil {
out = append(out, "searchable-snapshot:"+phase.Actions.SearchableSnapshot.SnapshotRepository)
}
return strings.Join(out, ",")
}
func IlmExplain(conf *cfg.Config, index string) error {
res, err := conf.DefaultCluster.ES().Ilm.ExplainLifecycle(index).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get ilm state: %w", esErrorString(err))
}
slog.Debug("ilm status", "ilm", res)
explain, exists := res.Indices[index]
if !exists {
return errors.New("no ilm status retrieved")
}
ilm := explain.(*types.LifecycleExplainManaged)
table := printer.NewTable(conf).WithSize(2, 0)
table.Addheaders("ilm status field", "value")
info := ""
if len(ilm.StepInfo["reason"]) > 0 {
err = json.Unmarshal(ilm.StepInfo["reason"], &info)
if err != nil {
return fmt.Errorf("failed to unmarshal step info: %w", err)
}
}
table.Entries = [][]any{
{"index", ilm.Index},
{"ilm policy", *ilm.Policy},
{"action", *ilm.Action},
{"age", ilm.Age},
{"managed", ilm.Managed},
{"phase", *ilm.Phase},
{"phase execution", ilmPhaseString(ilm.PhaseExecution.PhaseDefinition, false)},
{"step", *ilm.Step},
}
if ilm.FailedStep != nil {
table.AddRow("failed step", *ilm.FailedStep)
table.AddRow("failed step retry count", ilm.FailedStepRetryCount)
}
if err := table.Print(); err != nil {
return err
}
if info != "" {
var bold = lipgloss.NewStyle().Bold(true)
fmt.Printf("\n%s:\n%s\n",
bold.Render("Step Reason"),
info)
}
return nil
}
func IlmCreate(conf *cfg.Config, policyname string) error {
var policy *types.IlmPolicy = nil
res, err := conf.DefaultCluster.ES().Ilm.GetLifecycle().
Policy(policyname).
Do(context.Background())
if err == nil {
if conf.Debug {
repr.Println(res)
}
ilm, exists := res[policyname]
if !exists {
return errors.New("no ilm policy retrieved")
}
policy = &ilm.Policy
}
ilm := conf.DefaultCluster.ES().Ilm.PutLifecycle(policyname)
cfg := conf.Ilm
var phases types.PhasesVariant = esdsl.NewPhases()
if cfg.HaveHot() {
hot := types.Phase{}
var actions types.IlmActionsVariant = esdsl.NewIlmActions()
rollover := new(types.RolloverAction{})
haveroll := false
if policy != nil {
// update
actions = policy.Phases.Hot.Actions
if policy.Phases.Hot.Actions.Rollover != nil {
rollover = policy.Phases.Hot.Actions.Rollover
haveroll = true
}
}
if cfg.HotMinAge != "" {
hot.MinAge = cfg.HotMinAge
}
if cfg.HotRolloverMaxAge != "" {
rollover.MaxAge = cfg.HotRolloverMaxAge
haveroll = true
}
if cfg.HotRolloverMaxDocs != 0 {
rollover.MaxDocs = &cfg.HotRolloverMaxDocs
haveroll = true
}
if cfg.HotRolloverMaxPrimaryShardSize != "" {
rollover.MaxPrimaryShardSize = &cfg.HotRolloverMaxPrimaryShardSize
}
if haveroll {
actions.IlmActionsCaster().Rollover = rollover
}
hot.Actions = actions.IlmActionsCaster()
phases.PhasesCaster().Hot = &hot
} else if policy != nil {
// update
phases.PhasesCaster().Hot = policy.Phases.Hot
}
if cfg.HaveWarm() {
warm := types.Phase{}
var actions types.IlmActionsVariant = esdsl.NewIlmActions()
if policy != nil {
// update
actions = policy.Phases.Warm.Actions
}
if cfg.WarmForceMerge != 0 {
actions.IlmActionsCaster().Forcemerge = &types.ForceMergeAction{MaxNumSegments: cfg.WarmForceMerge}
}
if cfg.WarmMinAge != "" {
warm.MinAge = cfg.WarmMinAge
}
if cfg.WarmPriority != 0 {
actions.IlmActionsCaster().SetPriority = &types.SetPriorityAction{Priority: &cfg.WarmPriority}
}
if cfg.WarmShrinkShards != 0 {
actions.IlmActionsCaster().Shrink = &types.ShrinkAction{NumberOfShards: &cfg.WarmShrinkShards}
}
warm.Actions = actions.IlmActionsCaster()
phases.PhasesCaster().Warm = &warm
} else if policy != nil && policy.Phases.Warm != nil {
// update
phases.PhasesCaster().Warm = policy.Phases.Warm
}
if cfg.HaveCold() {
cold := types.Phase{}
var actions types.IlmActionsVariant = esdsl.NewIlmActions()
if policy != nil {
// update
actions = policy.Phases.Cold.Actions
}
if cfg.ColdForceMerge != 0 {
actions.IlmActionsCaster().Forcemerge = &types.ForceMergeAction{MaxNumSegments: cfg.ColdForceMerge}
}
if cfg.ColdMinAge != "" {
cold.MinAge = cfg.ColdMinAge
}
if cfg.ColdPriority != 0 {
actions.IlmActionsCaster().SetPriority = &types.SetPriorityAction{Priority: &cfg.ColdPriority}
}
if cfg.ColdSearchableSnapshotRepo != "" {
actions.IlmActionsCaster().SearchableSnapshot =
&types.SearchableSnapshotAction{SnapshotRepository: cfg.ColdSearchableSnapshotRepo}
}
cold.Actions = actions.IlmActionsCaster()
phases.PhasesCaster().Cold = &cold
} else if policy != nil && policy.Phases.Cold != nil {
// update
phases.PhasesCaster().Cold = policy.Phases.Cold
}
if cfg.HaveFrozen() {
froze := types.Phase{}
var actions types.IlmActionsVariant = esdsl.NewIlmActions()
if policy != nil {
// update
actions = policy.Phases.Frozen.Actions
}
if cfg.FrozenMinAge != "" {
froze.MinAge = cfg.FrozenMinAge
}
if cfg.FrozenSearchableSnapshotRepo != "" {
actions.IlmActionsCaster().SearchableSnapshot =
&types.SearchableSnapshotAction{SnapshotRepository: cfg.FrozenSearchableSnapshotRepo}
}
froze.Actions = actions.IlmActionsCaster()
phases.PhasesCaster().Frozen = &froze
} else if policy != nil && policy.Phases.Frozen != nil {
// update
phases.PhasesCaster().Frozen = policy.Phases.Frozen
}
if cfg.HaveDelete() {
del := types.Phase{}
delete := types.DeleteAction{}
var actions types.IlmActionsVariant = esdsl.NewIlmActions()
if policy != nil {
// update
actions = policy.Phases.Delete.Actions
}
if cfg.DeleteMinAge != "" {
del.MinAge = cfg.DeleteMinAge
}
if cfg.DeleteSearchableSnapshots {
delete.DeleteSearchableSnapshot = &cfg.DeleteSearchableSnapshots
}
actions.IlmActionsCaster().Delete = &delete
del.Actions = actions.IlmActionsCaster()
phases.PhasesCaster().Delete = &del
} else if policy != nil && policy.Phases.Delete != nil {
// update
phases.PhasesCaster().Delete = policy.Phases.Delete
}
put := new(putlifecycle.Request{})
newpolicy := new(types.IlmPolicy{})
newpolicy.IlmPolicyCaster().Phases = *phases.PhasesCaster()
put.Policy = newpolicy
ilm.Request(put)
_, err = ilm.Do(context.Background())
if err != nil {
return fmt.Errorf("failed create ilm policy: %w", esErrorString(err))
}
return nil
}

398
pkg/es/ilm_forecast.go Normal file
View File

@@ -0,0 +1,398 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"fmt"
"regexp"
"strconv"
"strings"
"sync"
"time"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/dustin/go-humanize"
"github.com/elastic/go-elasticsearch/v9/typedapi/cat/indices"
"github.com/elastic/go-elasticsearch/v9/typedapi/ilm/explainlifecycle"
"github.com/elastic/go-elasticsearch/v9/typedapi/ilm/getlifecycle"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
var (
IlmPhaseOrder = []string{"hot", "warm", "cold", "frozen", "delete"}
)
type NextPhase struct {
minage time.Duration
minsize int64
phase string
previousHot bool
}
type PhaseData struct {
index string
policy string
size int64
age time.Duration
minage time.Duration
minsize int64
currentPhase, nextPhase string
}
/*
ilm stages:
types.Lifecycle{
Policy: types.IlmPolicy{
Phases: types.Phases{
Hot: &types.Phase{
Actions: &types.IlmActions{
Rollover: &types.RolloverAction{
MaxAge: "7d",
MaxPrimaryShardSize: "25gb",
},
},
MinAge: "0ms",
},
Warm: &types.Phase{
MinAge: "0d",
},
Frozen: &types.Phase{
MinAge: "14d",
},
Delete: &types.Phase{
MinAge: "60d",
},
},
},
}
hot for 7 days or 25Gig, then Rollover => warm 14 days => frozen 60 days => delete
*/
func IlmForecastList(conf *cfg.Config, filter string) error {
phaseData, err := getIlmPhaseData(conf)
if err != nil {
return err
}
flt := regexp.MustCompile(filter)
minage := time.Duration(0)
if conf.Ilm.MinAge != "" {
minage = parseDuration(conf.Ilm.MinAge)
}
headers := []string{"index", "current size", "current age",
"virtual age", "min age", "min size", "current phase", "next phase"}
if conf.Verbose {
headers = append(headers, "ilm policy")
}
table := printer.NewTable(conf).WithSize(len(headers), 0)
table.Addheaders(headers...)
for _, phase := range phaseData {
if filter != "" && !flt.MatchString(phase.index) {
continue
}
if conf.Ilm.MinAge != "" && phase.age < minage {
continue
}
virtualAge := virtualAge(&phase)
row := []any{
phase.index,
printer.Bytes(phase.size),
formatDuration(phase.age),
formatDuration(virtualAge),
formatDuration(phase.minage),
printer.Bytes(phase.minsize),
phase.currentPhase,
phase.nextPhase,
}
if conf.Verbose {
row = append(row, phase.policy)
}
table.AddRow(row...)
}
table.Sort()
return table.Print()
}
// Loop over all current index phases of the current phase
// (conf.Ilm.FromPhase). Calculate virtualAge if phase.minsize >0. If
// the actual age is smaller than the virtual age, use this as a base,
// otherwise use virtual age. Then look if the rollover would happen
// within the phase.minage window (conf.Ilm.Within + current age) and
// add the size.
func IlmForecastShow(conf *cfg.Config) error {
phaseData, err := getIlmPhaseData(conf)
if err != nil {
return err
}
within := parseDuration(conf.Ilm.Within)
var toBeFreed int64 = 0
for _, phase := range phaseData {
age := max(virtualAge(&phase), phase.age)
if age+within >= phase.minage {
toBeFreed += phase.size
}
}
fmt.Printf("%s bytes of data in %s phase will be rolled within %s to the next phase\n",
humanize.Bytes(uint64(toBeFreed)),
conf.Ilm.FromPhase,
within)
return nil
}
func virtualAge(phase *PhaseData) time.Duration {
if phase.minsize == 0 {
return time.Duration(0)
}
return phase.minage * time.Duration(100*phase.size/phase.minsize) / 100
}
// Retrieve all index, ilm-explain and ilm-policies in parallel
func getIlmPhaseData(conf *cfg.Config) ([]PhaseData, error) {
responses := make(chan apiResponse, 3)
wg := new(sync.WaitGroup{})
wg.Go(func() {
getApiData(conf, conf.DefaultCluster.ES(), responses, "indicesbytes")
})
wg.Go(func() {
getApiData(conf, conf.DefaultCluster.ES(), responses, "explain")
})
wg.Go(func() {
getApiData(conf, conf.DefaultCluster.ES(), responses, "policies")
})
wg.Wait()
var (
ilmdetails *explainlifecycle.Response
indicesres *indices.Response
ilmpolicies getlifecycle.Response
)
for range 3 {
res := <-responses
if res.error != nil {
return nil, res.error
}
switch res.which {
case ResponseExplain:
ilmdetails = res.explainlifecycle
case ResponseIndices:
indicesres = res.indicesbytes
case ResponseLifecycle:
ilmpolicies = *res.lifecycle
}
}
list := []PhaseData{}
indices := make(map[string]types.IndicesRecord)
for _, index := range *indicesres {
indices[*index.Index] = index
}
for indexName, explain := range ilmdetails.Indices {
if !conf.Hidden && strings.HasPrefix(indexName, ".") {
continue
}
// avoid types.LifecycleExplainUnmanaged casting error
switch explain.(type) {
case *types.LifecycleExplainManaged:
default:
continue
}
explain := explain.(*types.LifecycleExplainManaged)
if explain.Phase == nil {
continue
}
phase := *explain.Phase
if phase != conf.Ilm.FromPhase {
continue
}
size, err := strconv.ParseInt(*indices[indexName].DatasetSize, 10, 64)
if err != nil {
return nil, fmt.Errorf("failed to convert index.DatasetSize to int: %w", err)
}
age := time.Duration(*explain.AgeInMillis * int64(time.Millisecond))
policy := ilmpolicies[*explain.Policy].Policy
nextPhase := findNextPhase(policy, phase)
if nextPhase == nil {
continue
}
list = append(list, PhaseData{
index: indexName,
policy: *explain.Policy,
size: size,
age: age,
minage: nextPhase.minage,
minsize: nextPhase.minsize,
currentPhase: conf.Ilm.FromPhase,
nextPhase: nextPhase.phase,
})
}
return list, nil
}
func registerPhases(policy types.IlmPolicy, currentPhase string) (map[string]*types.Phase, int) {
// register phases configured in the policy
phases := map[string]*types.Phase{
"hot": nil,
"warm": nil,
"cold": nil,
"frozen": nil,
"delete": nil,
}
// looking for a defined previous phase
start := 0
// register phases
for idx, phase := range IlmPhaseOrder {
switch phase {
case "hot":
if policy.Phases.Hot != nil {
phases[phase] = policy.Phases.Hot
}
case "warm":
if policy.Phases.Warm != nil {
phases[phase] = policy.Phases.Warm
}
case "cold":
if policy.Phases.Cold != nil {
phases[phase] = policy.Phases.Cold
}
case "frozen":
if policy.Phases.Frozen != nil {
phases[phase] = policy.Phases.Frozen
}
case "delete":
if policy.Phases.Delete != nil {
phases[phase] = policy.Phases.Delete
}
}
if phase == currentPhase {
start = idx
}
}
return phases, start
}
func findNextPhase(policy types.IlmPolicy, currentPhase string) *NextPhase {
// phase list to determine which comes next
phases, start := registerPhases(policy, currentPhase)
nextPhase := new(NextPhase{})
// finally determine which phase comes next
// exception: hot, where we look for rollover rules
for idx, phase := range IlmPhaseOrder {
if idx < start {
continue
}
if phase == "hot" {
// if we're starting here, do not look for the next phase
maxage := parseDuration(phases[phase].Actions.Rollover.MaxAge.(string))
var maxsize int64 = 0
switch val := phases[phase].Actions.Rollover.MaxPrimaryShardSize.(type) {
case int64:
maxsize = val
case string:
size, err := humanize.ParseBytes(val)
if err != nil {
panic(err)
}
maxsize = int64(size)
}
nextPhase.minage = maxage
nextPhase.minsize = maxsize
nextPhase.previousHot = true
continue
}
if phase == currentPhase {
continue // use the next one
}
if phases[phase] == nil {
continue // skip it, since empty
}
nextPhase.phase = phase
if nextPhase.previousHot {
return nextPhase
}
minage := parseDuration(phases[phase].MinAge.(string))
nextPhase.minage = minage
nextPhase.minsize = 0
return nextPhase
}
return nil
}

View File

@@ -18,26 +18,32 @@ package es
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"regexp"
"slices"
"strconv"
"strings"
"time"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"codeberg.org/scip/mapmap"
"github.com/charmbracelet/lipgloss"
"github.com/elastic/go-elasticsearch/v9/typedapi/cat/indices"
"github.com/elastic/go-elasticsearch/v9/typedapi/esdsl"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
"github.com/elastic/go-elasticsearch/v9/typedapi/types/enums/healthstatus"
)
// used for completion
func IndexNames(conf *cfg.Config) ([]string, error) {
res, err := conf.DefaultCluster.ES.Cat.Indices().
Header("content-type", "application/json").
Header("accept", "application/json").
res, err := conf.DefaultCluster.ES().Cat.Indices().
Do(context.Background())
if err != nil {
return nil, fmt.Errorf("failed to get indicies: %s", err)
return nil, fmt.Errorf("failed to get indices: %w", esErrorString(err))
}
indices := make([]string, len(res))
@@ -48,11 +54,34 @@ func IndexNames(conf *cfg.Config) ([]string, error) {
return indices, nil
}
func filterIndices(conf *cfg.Config, list indices.Response) indices.Response {
var filter *regexp.Regexp
if len(conf.Filter) > 0 {
filter = regexp.MustCompile(conf.Filter[0])
}
return mapmap.NewSlicer(list).MapSliceValuesImmutable(func(index types.IndicesRecord) bool {
if !conf.Partials && strings.HasPrefix(*index.Index, "partial-") {
return false
}
if !conf.Hidden && strings.HasPrefix(*index.Index, ".") {
return false
}
if len(conf.Filter) > 0 {
if !filter.MatchString(*index.Index) {
return false
}
}
return true
})
}
func IndexList(conf *cfg.Config) error {
cat := conf.DefaultCluster.ES.Cat.Indices().
// we need to add custom request headers, required for older ES instances
Header("content-type", "application/json").
Header("accept", "application/json")
cat := conf.DefaultCluster.ES().Cat.Indices()
if conf.Failed {
cat = cat.Health(healthstatus.Red)
@@ -60,12 +89,14 @@ func IndexList(conf *cfg.Config) error {
res, err := cat.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get indicies: %s", err)
return fmt.Errorf("failed to get indices: %w", esErrorString(err))
}
slog.Debug("ES result", "indicies", res)
slog.Debug("ES result", "indices", res)
size := len(res)
list := filterIndices(conf, res)
size := len(list)
if conf.MaxItems > 0 {
if size > conf.MaxItems {
@@ -73,13 +104,13 @@ func IndexList(conf *cfg.Config) error {
}
}
table := NewTable(3, size)
table := printer.NewTable(conf).WithSize(3, size)
table.Addheaders("name", "size", "docscount")
for idx, index := range res {
name := Colorize(*index.Health, *index.Index)
for idx, index := range list {
name := printer.Colorize(conf, *index.Health, *index.Index)
table.entries[idx] = []string{name, *index.DatasetSize, *index.DocsCount}
table.Entries[idx] = []any{name, *index.DatasetSize, *index.DocsCount}
if idx == size-1 {
break
@@ -87,49 +118,60 @@ func IndexList(conf *cfg.Config) error {
}
table.Sort()
if err := table.PrintMarkdown(); err != nil {
return err
}
return nil
return table.Print()
}
func IndexShow(conf *cfg.Config, index string) error {
if index == "" {
return fmt.Errorf("no index specified")
}
res, err := conf.DefaultCluster.ES.Indices.Get(index).
// we need to add custom request headers, required for older ES instances
Header("content-type", "application/json").
Header("accept", "application/json").
func IndexShow(conf *cfg.Config, indexpattern string) error {
res, err := conf.DefaultCluster.ES().Indices.Get(indexpattern).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get index: %s", err)
return fmt.Errorf("failed to get index: %w", esErrorString(err))
}
slog.Debug("ES result", "index", res)
slog.Debug("index show", "index", res)
table := NewTable(2, 5)
table.Addheaders("field", "value")
for name, index := range res {
fields := make([]string, len(index.Mappings.Properties))
ts, err := strconv.ParseInt(res[index].Settings.Index.CreationDate.(string), 10, 64)
if err != nil {
ts = 0
}
idx := 0
for field := range index.Mappings.Properties {
fields[idx] = field
idx++
}
created := time.Unix(ts/1000, 0)
table := printer.NewTable(conf).WithSize(2, 7)
table.Addheaders("index property", "value")
table.entries = [][]string{
{"name", index},
{"replicas", *res[index].Settings.Index.NumberOfReplicas},
{"shards", *res[index].Settings.Index.NumberOfShards},
{"created", created.Format("2006-01-02 15:04:05")},
{"uuid", *res[index].Settings.Index.Uuid},
}
ts, err := strconv.ParseInt(index.Settings.Index.CreationDate.(string), 10, 64)
if err != nil {
ts = 0
}
if err := table.PrintMarkdown(); err != nil {
return err
created := time.Unix(ts/1000, 0)
table.Entries = [][]any{
{"name", name},
{"replicas", *index.Settings.Index.NumberOfReplicas},
{"shards", *index.Settings.Index.NumberOfShards},
{"created", created},
{"uuid", *index.Settings.Index.Uuid},
{"version", *index.Settings.Index.Version.Created},
{"fields", fields},
}
if index.Settings.Index.Lifecycle != nil {
table.Entries = append(table.Entries, [][]any{
{"ilm policy", *index.Settings.Index.Lifecycle.Name},
{"ilm rollover alias", *index.Settings.Index.Lifecycle.RolloverAlias},
}...)
}
if err := table.Print(); err != nil {
return err
}
fmt.Println()
}
return nil
@@ -138,19 +180,18 @@ func IndexShow(conf *cfg.Config, index string) error {
func IndexCreate(conf *cfg.Config, index string, mappings []string) error {
settings := esdsl.NewIndexSettings()
create := conf.DefaultCluster.ES.Indices.Create(index).
Header("content-type", "application/json").
Header("accept", "application/json")
create := conf.DefaultCluster.ES().Indices.Create(index)
if conf.Wait {
create.WaitForActiveShards("all")
}
if conf.Shards > 0 {
settings = settings.NumberOfShards(strconv.Itoa(conf.Shards))
settings.NumberOfShards(strconv.Itoa(conf.Shards))
}
if conf.Replicas > 0 {
settings = settings.NumberOfReplicas(strconv.Itoa(conf.Replicas))
settings.NumberOfReplicas(strconv.Itoa(conf.Replicas))
}
if len(mappings) > 0 {
@@ -177,41 +218,159 @@ func IndexCreate(conf *cfg.Config, index string, mappings []string) error {
create.Mappings(maps)
}
if conf.Policy != "" {
settings.Lifecycle(
esdsl.NewIndexSettingsLifecycle().
Name(conf.Policy).
RolloverAlias(index))
}
_, err := create.Settings(settings).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to create index: %s", err)
return fmt.Errorf("failed to create index: %w", esErrorString(err))
}
return nil
}
func IndexDelete(conf *cfg.Config, index string) error {
if index == "" {
return fmt.Errorf("no index specified")
}
_, err := conf.DefaultCluster.ES.Indices.Delete(index).
Header("content-type", "application/json").
Header("accept", "application/json").
_, err := conf.DefaultCluster.ES().Indices.Delete(index).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to delete index: %s", err)
return fmt.Errorf("failed to delete index: %w", esErrorString(err))
}
return nil
}
func IndexClose(conf *cfg.Config, index string) error {
create := conf.DefaultCluster.ES.Indices.Close(index).
Header("content-type", "application/json").
Header("accept", "application/json")
_, err := create.Do(context.Background())
_, err := conf.DefaultCluster.ES().Indices.Close(index).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to close index: %s", err)
return fmt.Errorf("failed to close index: %w", esErrorString(err))
}
return nil
}
func IndexFields(conf *cfg.Config, index string) error {
res, err := conf.DefaultCluster.ES().FieldCaps().
Index(index).
Fields("*").
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to retrieve field capabilties: %w", esErrorString(err))
}
table := printer.NewTable(conf).WithSize(5, 0)
table.Addheaders("field", "type", "searchable", "aggretable", "metadata")
idx := 0
for name, field := range res.Fields {
for fieldtype, caps := range field {
// fields only have 1 type, so this one is it
switch {
case conf.Searchable && !caps.Searchable:
continue
case conf.Aggretable && !caps.Aggregatable:
continue
case len(conf.Filter) > 0 && !slices.Contains(conf.Filter, fieldtype):
continue
}
table.AddRow(name, fieldtype,
caps.Searchable,
caps.Aggregatable,
*caps.MetadataField)
break
}
idx++
}
table.Sort()
if err := table.Print(); err != nil {
return err
}
return nil
}
type Diskusage struct {
Total int64 `json:"total_in_bytes"`
Points int64 `json:"points_in_bytes"`
Norms int64 `json:"norms_in_bytes"`
TermVectors int64 `json:"term_vectors_in_bytes"`
KnnVectors int64 `json:"knn_vectors_in_bytes"`
BloomFilter int64 `json:"bloom_filter_in_bytes"`
}
type IndexDiskUsage struct {
AllFields Diskusage `json:"all_fields"`
Fields map[string]Diskusage `json:"fields"`
}
type ResIndexDiskUsage map[string]IndexDiskUsage
func IndexDiskusage(conf *cfg.Config, index string) error {
var bold = lipgloss.NewStyle().Bold(true)
res, err := conf.DefaultCluster.ES().Indices.DiskUsage(index).
RunExpensiveTasks(true).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to retrieve index disk usage: %w", esErrorString(err))
}
duRes := ResIndexDiskUsage{}
if err := json.Unmarshal(res, &duRes); err != nil {
return fmt.Errorf("failed to unmarshal disk usage response: %w", err)
}
diskusage, exists := duRes[index]
if !exists {
return errors.New("no disk usage reported for index")
}
table := printer.NewTable(conf).
WithHeaders("field", "bloom filter", "norms", "points", "term vectors", "knn vectors", "total")
for name, field := range diskusage.Fields {
if strings.HasPrefix(name, "_") || strings.HasSuffix(name, ".keyword") {
continue
}
table.AddRow(
name,
printer.Bytes(field.BloomFilter),
printer.Bytes(field.Norms),
printer.Bytes(field.Points),
printer.Bytes(field.TermVectors),
printer.Bytes(field.KnnVectors),
printer.Bytes(field.Total),
)
}
all := diskusage.AllFields
table.Sort()
table.AddRowLate(
bold.Render("Summary"),
printer.Bytes(all.BloomFilter),
printer.Bytes(all.Norms),
printer.Bytes(all.Points),
printer.Bytes(all.TermVectors),
printer.Bytes(all.KnnVectors),
printer.Bytes(all.Total),
)
if err := table.Print(); err != nil {
return err
}
return nil

127
pkg/es/index_alias.go Normal file
View File

@@ -0,0 +1,127 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"fmt"
"log/slog"
"regexp"
"strings"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
)
// FIXME: add filter support, see IndexCreate mapping
func IndexAliasCreate(conf *cfg.Config, index, alias string) error {
res, err := conf.DefaultCluster.ES().Indices.PutAlias(index, alias).
Do(context.Background())
slog.Debug("create alias", "result", res)
if err != nil {
return fmt.Errorf("failed to create index alias: %w", esErrorString(err))
}
return nil
}
func IndexAliasList(conf *cfg.Config) error {
filter := regexp.Regexp{}
if len(conf.Filter) > 0 {
// we support just one filter here, for now
filter = *regexp.MustCompile(conf.Filter[0])
}
res, err := conf.DefaultCluster.ES().Indices.GetAlias().
Index("_all").
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to list index aliases: %w", esErrorString(err))
}
slog.Debug("aliases list", "result", res)
aliaslist := map[string][]string{} // index => []aliases
for index, aliases := range res {
if len(conf.Filter) > 0 {
if !filter.MatchString(index) {
continue
}
}
for alias := range aliases.Aliases {
aliaslist[index] = append(aliaslist[index], alias)
}
}
table := printer.NewTable(conf).WithSize(2, len(aliaslist))
table.Addheaders("index", "alias")
idx := 0
for index, aliases := range aliaslist {
table.Entries[idx] = []any{
index,
strings.Join(aliases, ","),
}
idx++
}
table.Sort()
if err := table.Print(); err != nil {
return err
}
return nil
}
func IndexAliasDelete(conf *cfg.Config, index, alias string) error {
res, err := conf.DefaultCluster.ES().Indices.DeleteAlias(index, alias).
Do(context.Background())
slog.Debug("delete alias", "result", res)
if err != nil {
return fmt.Errorf("failed to delete index alias: %w", esErrorString(err))
}
return nil
}
func IndexAliasRollover(conf *cfg.Config, alias string) error {
res, err := RolloverAlias(conf, alias)
if err != nil {
return fmt.Errorf("failed to rollover index alias: %w", esErrorString(err))
}
table := printer.NewTable(conf).WithSize(2, 5)
table.Addheaders("rollover response", "value")
table.Entries = [][]any{
{"acknowledged", res.Acknowledged},
{"rolled over", res.RolledOver},
{"shards acknowledged", res.ShardsAcknowledged},
{"old index", res.OldIndex},
{"new index", res.NewIndex},
}
return table.Print()
}

83
pkg/es/index_copy.go Normal file
View File

@@ -0,0 +1,83 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"fmt"
"time"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/elastic/go-elasticsearch/v9/typedapi/esdsl"
"github.com/elastic/go-elasticsearch/v9/typedapi/types/enums/conflicts"
)
func IndexCopy(conf *cfg.Config) error {
copy := conf.DefaultCluster.ES().Reindex()
if conf.Force {
copy.Conflicts(conflicts.Proceed)
}
if conf.RequestsPerSecond != 0 {
copy.RequestsPerSecond(fmt.Sprintf("%.2f", conf.RequestsPerSecond))
}
if conf.MaxDocs > 0 {
copy.MaxDocs(conf.MaxDocs)
}
if conf.Timeout > 0 {
copy.Timeout(formatDuration(conf.Timeout))
}
if conf.Wait {
copy.WaitForActiveShards("all")
}
if conf.Refresh {
copy.Refresh(true)
}
copy.Source(esdsl.NewReindexSource().Index(conf.SourceIndices...))
copy.Dest(esdsl.NewReindexDestination().Index(conf.Index))
res, err := copy.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to copy indices: %w", esErrorString(err))
}
table := printer.NewTable(conf).WithSize(2, 0).
WithHeaders("Reindex metrtic", "value")
table.Entries = [][]any{
{"Source indices", conf.SourceIndices},
{"Target index", conf.Index},
{"Batches", *res.Batches},
{"Documents total", *res.Total},
{"Documents created", *res.Created},
{"Documents deleted", *res.Deleted},
{"Documents updated", *res.Updated},
{"Requests/s", *res.RequestsPerSecond},
{"Timed out", *res.TimedOut},
{"Time elapsed", time.Duration(*res.Took) * time.Millisecond},
{"Version conflicts", *res.VersionConflicts},
}
return table.Print()
}

530
pkg/es/index_template.go Normal file
View File

@@ -0,0 +1,530 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"maps"
"strconv"
"strings"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"codeberg.org/scip/mapmap"
"github.com/elastic/go-elasticsearch/v9/typedapi/esdsl"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
// used for completion
func IndexTemplateNames(conf *cfg.Config) ([]string, error) {
res, err := conf.DefaultCluster.ES().Indices.GetIndexTemplate().
Do(context.Background())
if err != nil {
return nil, fmt.Errorf("failed to get index templates: %w", esErrorString(err))
}
names := make([]string, len(res.IndexTemplates))
for idx, tpl := range res.IndexTemplates {
names[idx] = tpl.Name
}
return names, nil
}
func IndexTemplateList(conf *cfg.Config, filter string) error {
res, err := conf.DefaultCluster.ES().Indices.GetIndexTemplate().
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get index templates: %w", esErrorString(err))
}
slog.Debug("res", "index templates", res)
table := printer.NewTable(conf).WithSize(5, 0)
table.Addheaders("name", "description", "index patterns", "priority")
tplList := filterIndexTemplates(conf, filter, res.IndexTemplates)
for _, tpl := range tplList {
desc := strings.TrimPrefix(strings.TrimSuffix(string(tpl.IndexTemplate.Meta_["description"]), `"`), `"`)
var prio int64
if tpl.IndexTemplate.Priority != nil {
prio = *tpl.IndexTemplate.Priority
}
table.AddRow(tpl.Name, desc, tpl.IndexTemplate.IndexPatterns, prio)
}
table.Sort()
return table.Print()
}
// Filter index templates by name, index pattern or hidden flag, using
// mapmap.Slicer
func filterIndexTemplates(conf *cfg.Config, nameFilter string,
templates []types.IndexTemplateItem) []types.IndexTemplateItem {
return mapmap.NewSlicer(templates).MapSliceValuesImmutable(func(tpl types.IndexTemplateItem) bool {
if !conf.Hidden && strings.HasPrefix(tpl.Name, ".") {
return false
}
if nameFilter != "" && !strings.Contains(tpl.Name, nameFilter) {
return false
}
if len(conf.Filter) > 0 {
if !mapmap.NewSlicer(tpl.IndexTemplate.IndexPatterns).
FindSliceInSlice(conf.Filter, strings.Contains) {
return false
}
}
return true
})
}
func IndexTemplateShow(conf *cfg.Config, tplname string) error {
res, err := conf.DefaultCluster.ES().Indices.GetIndexTemplate().
Name(tplname).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get index template: %w", esErrorString(err))
}
slog.Debug("res", "index template", res)
if len(res.IndexTemplates) != 1 {
return errors.New("multiple or no index template matched the pattern")
}
tpl := res.IndexTemplates[0]
table := printer.NewTable(conf).WithSize(2, 6)
table.Addheaders("index template property", "value")
desc, err := json.Marshal(tpl.IndexTemplate.Meta_["description"])
if err != nil {
return fmt.Errorf("failed to unmarshal meta json data: %w", err)
}
hasds := tpl.IndexTemplate.DataStream != nil
aliases := []string{}
for alias := range tpl.IndexTemplate.Template.Aliases {
aliases = append(aliases, alias)
}
table.Entries = [][]any{
{"name", tpl.Name},
{"description", desc},
{"index patterns", tpl.IndexTemplate.IndexPatterns},
{"composed of", tpl.IndexTemplate.ComposedOf},
{"data stream enabled", hasds},
{"aliases", aliases},
}
if err := table.Print(); err != nil {
return err
}
table = printer.NewTable(conf).WithSize(2, 0)
table.Addheaders("index setting property", "value")
err = getIndexTemplateSettings(conf, tplname, table)
if err != nil {
return err
}
fmt.Println()
if err := table.Print(); err != nil {
return err
}
table = printer.NewTable(conf).WithSize(2, 0)
table.Addheaders("index field mapping", "type")
if tpl.IndexTemplate.Template.Mappings != nil {
for name, field := range tpl.IndexTemplate.Template.Mappings.Properties {
typeval := ""
switch val := field.(type) {
case *types.IntegerNumberProperty:
typeval = val.Type
case *types.KeywordProperty:
typeval = val.Type
case *types.DateProperty:
typeval = val.Type
}
table.AddRow(name, typeval)
}
fmt.Println()
if err := table.Print(); err != nil {
return err
}
}
return nil
}
func IndexTemplateCreate(conf *cfg.Config, name string, mappings []string) error {
settings := esdsl.NewIndexSettings()
maps := esdsl.NewIndexTemplateMapping()
create := conf.DefaultCluster.ES().Indices.PutIndexTemplate(name)
if conf.Shards > 0 {
settings.NumberOfShards(strconv.Itoa(conf.Shards))
}
if conf.Replicas > 0 {
settings.NumberOfReplicas(strconv.Itoa(conf.Replicas))
}
if conf.Policy != "" {
settings.Lifecycle(
esdsl.NewIndexSettingsLifecycle().
Name(conf.Policy))
}
if conf.Stream {
create.DataStream(esdsl.NewDataStreamVisibility())
if conf.Retention != "" {
maps.Lifecycle(
esdsl.NewDataStreamLifecycle().
DataRetention(
esdsl.NewDuration().String(conf.Retention)))
}
}
if conf.AutoCreate {
create.AllowAutoCreate(true)
}
if conf.Mode != "" {
settings = settings.Mode(conf.Mode)
}
if len(mappings) > 0 {
typemaps, err := modMappings(mappings)
if err != nil {
return err
}
maps.Mappings(typemaps)
}
if len(conf.Components) > 0 {
create.ComposedOf(conf.Components...)
}
for _, alias := range conf.Aliases {
maps.AddAlias(alias, esdsl.NewAlias())
}
if len(conf.Meta) > 0 {
metadata, err := modMeta(conf, nil)
if err != nil {
return err
}
create.Meta_(esdsl.NewMetadata(metadata))
}
if len(conf.Settings) > 0 {
usersettings, err := modSettings(conf)
if err != nil {
return err
}
settings = settings.IndexSettings(usersettings)
}
maps.Settings(settings)
create.Template(maps)
create.IndexPatterns(conf.Patterns...)
_, err := create.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to create index template: %w", esErrorString(err))
}
return nil
}
// FIXME: func's too long, refactor
// FIXME: it's not yet possible to remove items in lists (aliases, mappings, settings), just overwrite them
func IndexTemplateModify(conf *cfg.Config, name string, mappings []string) error {
maps := esdsl.NewIndexTemplateMapping()
// load existing index mapping
res, err := conf.DefaultCluster.ES().Indices.GetIndexTemplate().
Name(name).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get index template: %w", esErrorString(err))
}
slog.Debug("res", "index template", res)
if len(res.IndexTemplates) != 1 {
return errors.New("multiple or no index template matched the pattern")
}
tpl := res.IndexTemplates[0]
// our modify PUT request
modify := conf.DefaultCluster.ES().Indices.PutIndexTemplate(name)
// load existing settings, if any
settings := tpl.IndexTemplate.Template.Settings
// pre fill mappings and aliases
maps.Mappings(tpl.IndexTemplate.Template.Mappings)
maps.Aliases(tpl.IndexTemplate.Template.Aliases)
// pre fill components
modify.ComposedOf(tpl.IndexTemplate.ComposedOf...)
// pre fill data stream config
if conf.Stream {
modify.DataStream(esdsl.NewDataStreamVisibility())
if conf.Retention != "" {
maps.Lifecycle(
esdsl.NewDataStreamLifecycle().
DataRetention(
esdsl.NewDuration().String(conf.Retention)))
}
}
modify.AllowAutoCreate(conf.AutoCreate)
if conf.Mode != "" {
*settings.Mode = conf.Mode
}
if conf.Policy != "" {
settings.Lifecycle.Name = &conf.Policy
}
if len(mappings) > 0 {
typemaps, err := modMappings(mappings)
if err != nil {
return err
}
maps.Mappings(typemaps)
}
if len(conf.Components) > 0 {
modify.ComposedOf(conf.Components...)
}
for _, alias := range conf.Aliases {
maps.AddAlias(alias, esdsl.NewAlias())
}
if len(conf.Meta) > 0 {
// pre fill meta, if any
metadata, err := modMeta(conf, tpl.IndexTemplate.Meta_)
if err != nil {
return err
}
modify.Meta_(esdsl.NewMetadata(metadata))
}
if len(conf.Settings) > 0 {
usersettings, err := modSettings(conf)
if err != nil {
return err
}
settings.IndexSettings = usersettings
}
patterns := tpl.IndexTemplate.IndexPatterns
if len(conf.Patterns) > 0 {
patterns = conf.Patterns
}
maps.Settings(settings)
modify.Template(maps)
modify.IndexPatterns(patterns...)
_, err = modify.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to modify index template: %w", esErrorString(err))
}
if conf.Rollover {
if err := rolloverAliasIndexTemplate(conf, name); err != nil {
return err
}
}
return nil
}
func IndexTemplateDelete(conf *cfg.Config, name string) error {
_, err := conf.DefaultCluster.ES().Indices.DeleteIndexTemplate(name).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to delete index template: %w", esErrorString(err))
}
return nil
}
// Based on given index template find the associated index patterns,
// find indices matching those, find their associated aliases and
// rollover all we find. Only run when conf.Rollover==true
func rolloverAliasIndexTemplate(conf *cfg.Config, name string) error {
res, err := conf.DefaultCluster.ES().Indices.GetIndexTemplate().
Name(name).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get index template: %w", esErrorString(err))
}
slog.Debug("res", "index template", res)
if len(res.IndexTemplates) != 1 {
return errors.New("multiple or no index template matched the pattern")
}
patterns := res.IndexTemplates[0].IndexTemplate.IndexPatterns
aliases := map[string]int{}
// find all aliases matching the patterns
for _, pattern := range patterns {
res, err := conf.DefaultCluster.ES().Indices.ResolveIndex(pattern).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to resolve index pattern: %w", esErrorString(err))
}
for _, index := range res.Indices {
for _, alias := range index.Aliases {
aliases[alias] = 1
}
}
}
if len(aliases) == 0 {
return nil
}
table := printer.NewTable(conf).WithSize(4, 0)
table.Addheaders("rollover alias", "status", "ack", "new index")
// apply rollover to all matching aliases, if any
for alias := range aliases {
res, err := RolloverAlias(conf, alias)
if err != nil {
return err
}
table.AddRow(alias, res.RolledOver, res.Acknowledged, res.NewIndex)
if err := table.Print(); err != nil {
return err
}
}
return nil
}
func modMappings(mappings []string) (types.TypeMappingVariant, error) {
typemaps := esdsl.NewTypeMapping()
for _, mapping := range mappings {
parts := strings.Split(mapping, ":")
if len(parts) != 2 {
return nil, fmt.Errorf(
"invalid mapping %s, expect <name:type> (type: integer, text, date, keyword)",
mapping)
}
switch parts[1] {
case "text":
typemaps.AddProperty(parts[0], esdsl.NewTextProperty())
case "integer":
typemaps.AddProperty(parts[0], esdsl.NewIntegerNumberProperty())
case "date":
typemaps.AddProperty(parts[0], esdsl.NewDateProperty())
case "keyword":
typemaps.AddProperty(parts[0], esdsl.NewKeywordProperty())
}
}
return typemaps, nil
}
func modMeta(conf *cfg.Config, meta types.Metadata) (map[string]json.RawMessage, error) {
metadata := map[string]json.RawMessage{}
maps.Copy(metadata, meta)
for _, meta := range conf.Meta {
parts := strings.Split(meta, ":")
if len(parts) != 2 {
return nil, errors.New("meta data must be in the form key:value")
}
msg, err := json.Marshal(parts[1])
if err != nil {
return nil, fmt.Errorf("failed to json marshal metadata %s: %w", meta, err)
}
metadata[parts[0]] = msg
}
return metadata, nil
}
func modSettings(conf *cfg.Config) (map[string]json.RawMessage, error) {
usersettings := map[string]json.RawMessage{}
for _, meta := range conf.Settings {
parts := strings.Split(meta, ":")
if len(parts) != 2 {
return nil, errors.New("settings data must be in the form key:value")
}
msg, err := json.Marshal(parts[1])
if err != nil {
return nil, fmt.Errorf("failed to json marshal metadata %s: %w", meta, err)
}
usersettings[parts[0]] = msg
}
return usersettings, nil
}

View File

@@ -0,0 +1,60 @@
package es
import (
"encoding/json"
"fmt"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
)
type TplTemplateData struct {
Settings map[string]any `json:"settings"`
Aliases map[string]any `json:"aliases"`
Lifecycle map[string]any `json:"lifecycle"`
Mappings map[string]any `json:"mappings"`
}
type TplTemplate struct {
Template TplTemplateData `json:"template"`
}
type TplTpl struct {
IndexTemplate TplTemplate `json:"index_template"`
}
type Tpl struct {
IndexTemplates []TplTpl `json:"index_templates"`
}
func getIndexTemplateSettings(conf *cfg.Config, tplname string, table *printer.Table) error {
raw, err := CallAPI(conf, "GET", "/_index_template/"+tplname+"?flat_settings", "")
if err != nil {
return err
}
data := Tpl{}
if err = json.Unmarshal(raw, &data); err != nil {
return err
}
if conf.Debug {
output, err := prettyfiJson(conf, raw)
if err != nil {
return err
}
fmt.Println(output)
}
if len(data.IndexTemplates) == 0 {
return fmt.Errorf("index_template %s not found", tplname)
}
tpl := data.IndexTemplates[0].IndexTemplate.Template.Settings
for topic, val := range tpl {
table.AddRow(topic, val)
}
return nil
}

72
pkg/es/license.go Normal file
View File

@@ -0,0 +1,72 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"fmt"
"log/slog"
"strconv"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
)
func LicenseShow(conf *cfg.Config) error {
res, err := conf.DefaultCluster.ES().License.Get().
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get license: %w", esErrorString(err))
}
slog.Debug("license show", "license", res)
table := printer.NewTable(conf).WithHeaders("license setting", "value")
lic := res.License
var (
maxnodes = "infinite"
maxunits = "infinite"
expire = "never"
)
if lic.MaxNodes != nil {
maxnodes = strconv.FormatInt(*lic.MaxNodes, 10)
}
if lic.ExpiryDate != nil {
expire = lic.ExpiryDate.(string)
}
if lic.MaxResourceUnits != nil {
maxunits = strconv.Itoa(*lic.MaxResourceUnits)
}
table.Entries = [][]any{
{"UID", lic.Uid},
{"Issued to", lic.IssuedTo},
{"Expires", expire},
{"Issued", lic.IssueDate},
{"Max nodes", maxnodes},
{"Max resource units", maxunits},
{"Type", lic.Type.Name},
{"Status", lic.Status.Name},
}
return table.Print()
}

View File

@@ -20,38 +20,287 @@ import (
"context"
"fmt"
"log/slog"
"strconv"
"strings"
"time"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
)
func NodeList(conf *cfg.Config) error {
// get nodes
nodes, err := conf.DefaultCluster.ES.Cat.Nodes().Do(context.Background())
nodes, err := conf.DefaultCluster.ES().Cat.Nodes().Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get nodes: %s", err)
return fmt.Errorf("failed to get nodes: %w", esErrorString(err))
}
slog.Debug("ES result", "nodes", nodes)
table := NewTable(7, len(nodes))
table.Addheaders("name", "ip", "load1m", "load5m", "load15m", "ram %", "heap %")
table := printer.NewTable(conf).WithHeaders(
"name", "ip", "load1m", "load5m", "load15m", "ram %", "heap %")
for idx, node := range nodes {
table.entries[idx] = []string{
for _, node := range nodes {
table.AddRow(
*node.Name,
*node.Ip,
*node.Load1M,
*node.Load5M,
*node.Load15M,
node.RamPercent.(string),
node.HeapPercent.(string),
}
node.RamPercent,
node.HeapPercent,
)
}
table.Sort()
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
return err
}
return nil
}
func NodeNames(conf *cfg.Config) ([]string, error) {
nodes, err := conf.DefaultCluster.ES().Cat.Nodes().
Do(context.Background())
if err != nil {
return nil, fmt.Errorf("failed to get nodes: %w", esErrorString(err))
}
slog.Debug("ES result", "nodes", nodes)
nodelist := make([]string, len(nodes))
for idx, node := range nodes {
nodelist[idx] = *node.Name
}
return nodelist, err
}
// FIXME: adding the settings metric leads to json unmarshall error:
// https://github.com/elastic/go-elasticsearch/issues/1524
func NodeShow(conf *cfg.Config, nodename string) error {
res, err := conf.DefaultCluster.ES().Nodes.Info().
NodeId(nodename).
Metric("os, jvm, thread_pool, remote_cluster_server").
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get node info: %w", esErrorString(err))
}
slog.Debug("ES result", "node", res)
stats, err := conf.DefaultCluster.ES().Nodes.Stats().
NodeId(nodename).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get node stats: %w", esErrorString(err))
}
slog.Debug("ES result", "stat", stats)
for id, info := range res.Nodes {
stat := stats.Nodes[id]
table := printer.NewTable(conf).WithSize(2, 0)
table.Addheaders(nodename+" property", "value")
roles := make([]string, len(info.Roles))
for idx, role := range info.Roles {
roles[idx] = role.Name
}
k8snode := info.Attributes["k8s_node_name"]
rank := "none"
adsel, exists := stat.AdaptiveSelection[id]
if exists {
rank = *adsel.Rank
}
table.Entries = [][]any{
{"Id", id},
{"Name", nodename},
{"Kubernetes node", k8snode},
{"Ip address", info.Ip},
{"Node rank", rank},
{"JVM", info.Jvm.VmName + " " + info.Jvm.Version},
{"JVM Started", time.UnixMilli(info.Jvm.StartTimeInMillis)},
{"OS", info.Os.PrettyName + " " + info.Os.Version},
{"Node roles", roles},
{"Node version", info.Version},
// FIXME: not implemented upstream
// see: https://github.com/elastic/go-elasticsearch/issues/1526
// {"Allocated shards", stat.Allocations.XXX},
{"HTTP clients", *stat.Http.CurrentOpen},
{"CPUs", *info.Os.AllocatedProcessors},
{"Load 15m/5m/1m", fmt.Sprintf("%.2f/%.2f/%.2f",
stat.Os.Cpu.LoadAverage["15m"],
stat.Os.Cpu.LoadAverage["5m"],
stat.Os.Cpu.LoadAverage["1m"],
)},
{"Open FD's", *stat.Process.OpenFileDescriptors},
{"HTTP sesssions current/total", fmt.Sprintf("%d/%d",
*stat.Http.CurrentOpen,
*stat.Http.TotalOpened,
)},
{"Traffic rx/tx",
printer.ByteString(*stat.Transport.RxSizeInBytes) + " / " + printer.ByteString(*stat.Transport.TxSizeInBytes)},
{"Response time avg", time.Duration(*stat.AdaptiveSelection[id].AvgResponseTimeNs)},
{"Memory usage (used/avail)",
printer.ByteString(*stat.Os.Mem.UsedInBytes) + " / " + printer.ByteString(*stat.Os.Mem.TotalInBytes)},
{"Search queries current/total", fmt.Sprintf("%d/%d",
stat.Indices.Search.QueryCurrent,
stat.Indices.Search.QueryTotal,
)},
{"Search efficiency", stat.Indices.Search.QueryTimeInMillis / stat.Indices.Search.QueryTotal},
{"Docs count", stat.Indices.Docs.Count},
{"Merges current/total", fmt.Sprintf("%d/%d",
stat.Indices.Merges.Current,
stat.Indices.Merges.Total,
)},
{"Merge docs count current/total", fmt.Sprintf("%d/%d",
stat.Indices.Merges.CurrentDocs,
stat.Indices.Merges.TotalDocs,
)},
{"Merge size current/total", fmt.Sprintf("%s/%s",
printer.ByteString(stat.Indices.Merges.CurrentSizeInBytes),
printer.ByteString(stat.Indices.Merges.TotalSizeInBytes),
)},
{"CircuitBreaker trip count", *stat.Breakers["fielddata"].Tripped},
}
if len(stat.Fs.Data) > 0 {
fs := stat.Fs.Data[0]
table.AddRow("Storage usage (used/avail)",
printer.ByteString(*fs.AvailableInBytes)+" / "+printer.ByteString(*fs.TotalInBytes))
table.AddRow("Storage mount", *fs.Mount)
}
if err := table.Print(); err != nil {
return err
}
}
return nil
}
func NodeClients(conf *cfg.Config, nodename string) error {
stats, err := conf.DefaultCluster.ES().Nodes.Stats().
NodeId(nodename).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get node stats: %w", esErrorString(err))
}
slog.Debug("ES result", "stat", stats)
table := printer.NewTable(conf).WithSize(5, 0)
table.Addheaders("agent", "id", "when", "from host", "url")
for _, stat := range stats.Nodes {
for _, client := range stat.Http.Clients {
if client.ClosedTimeMillis == nil {
agent := ""
if client.Agent != nil {
agent = *client.Agent
}
uri := ""
if client.LastUri != nil {
uri = *client.LastUri
if !conf.All {
parts := strings.Split(uri, "?")
uri = parts[0]
}
}
table.AddRow(
agent,
strconv.FormatInt(*client.Id, 10),
time.UnixMilli(*client.LastRequestTimeMillis).String(),
*client.RemoteAddress,
uri,
)
}
}
break
}
table.Sort()
return table.Print()
}
func NodeUsage(conf *cfg.Config, nodeid string) error {
usage := conf.DefaultCluster.ES().Nodes.Usage()
if nodeid != "" {
usage.NodeId(nodeid)
}
stats, err := usage.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get node usage: %w", esErrorString(err))
}
slog.Debug("ES result", "usage", stats)
table := printer.NewTable(conf).WithHeaders(
"node",
"bulk",
"doc get",
"doc mget",
"doc update",
"index doc",
"index stats",
"search",
"msearch",
"open pit",
)
for id, actions := range stats.Nodes {
node, err := getNodeName(conf, id)
if err != nil {
return err
}
stat := actions.RestActions
table.AddRow(
node,
stat["bulk_action"],
stat["document_get_action"],
stat["document_mget_action"],
stat["document_update_action"],
stat["document_index_action"],
stat["indices_stats_action"],
stat["search_action"],
stat["msearch_action"],
stat["open_point_in_time"],
)
}
return table.Print()
}
func getNodeName(conf *cfg.Config, id string) (string, error) {
res, err := conf.DefaultCluster.ES().Nodes.Info().
Metric("os").
Do(context.Background())
if err != nil {
return "", fmt.Errorf("failed to get node info: %w", esErrorString(err))
}
for nodeid, node := range res.Nodes {
if id == nodeid {
return node.Name, nil
}
}
return "", nil
}

158
pkg/es/parallel.go Normal file
View File

@@ -0,0 +1,158 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"fmt"
"codeberg.org/scip/esctl/pkg/cfg"
"github.com/elastic/go-elasticsearch/v9"
"github.com/elastic/go-elasticsearch/v9/typedapi/cat/indices"
"github.com/elastic/go-elasticsearch/v9/typedapi/cat/tasks"
"github.com/elastic/go-elasticsearch/v9/typedapi/ccr/stats"
"github.com/elastic/go-elasticsearch/v9/typedapi/cluster/health"
clusterstats "github.com/elastic/go-elasticsearch/v9/typedapi/cluster/stats"
"github.com/elastic/go-elasticsearch/v9/typedapi/core/info"
"github.com/elastic/go-elasticsearch/v9/typedapi/ilm/explainlifecycle"
"github.com/elastic/go-elasticsearch/v9/typedapi/ilm/getlifecycle"
"github.com/elastic/go-elasticsearch/v9/typedapi/types/enums/bytes"
)
const (
ResponseHealth = iota
ResponseInfo
ResponseCcr
ResponseStats
ResponseIndices
ResponseTasks
ResponseExplain
ResponseLifecycle
ResponseHealthReport
)
type apiResponse struct {
error error
info *info.Response
health *health.Response
healthreport *HealthReport
ccr *stats.Response
stats *clusterstats.Response
indices *indices.Response
indicesbytes *indices.Response
tasks *tasks.Response
lifecycle *getlifecycle.Response
explainlifecycle *explainlifecycle.Response
which int
}
func getApiData(conf *cfg.Config, es *elasticsearch.TypedClient, reschan chan apiResponse, which string) {
apiRes := apiResponse{}
var arerr error
switch which {
case "health":
res, err := es.Cluster.Health().
Do(context.Background())
apiRes.health = res
apiRes.which = ResponseHealth
arerr = err
case "healthreport":
report, err := getHealthReport(conf)
apiRes.healthreport = report
apiRes.which = ResponseHealthReport
arerr = err
case "info":
res, err := es.Info().
Do(context.Background())
apiRes.info = res
apiRes.which = ResponseInfo
arerr = err
case "ccr":
res, err := es.Ccr.Stats().
Do(context.Background())
apiRes.ccr = res
apiRes.which = ResponseCcr
arerr = err
case "stats":
res, err := es.Cluster.Stats().
Do(context.Background())
apiRes.stats = res
apiRes.which = ResponseStats
arerr = err
case "indices":
res, err := es.Cat.Indices().
Do(context.Background())
apiRes.indices = &res
apiRes.which = ResponseIndices
arerr = err
case "indicesbytes":
res, err := es.Cat.
Indices().
Bytes(bytes.Bytes{Name: "b"}).
Do(context.Background())
apiRes.indicesbytes = &res
apiRes.which = ResponseIndices
arerr = err
case "tasks":
res, err := es.Cat.Tasks().
Do(context.Background())
apiRes.tasks = &res
apiRes.which = ResponseTasks
arerr = err
case "explain":
res, err := es.Ilm.
ExplainLifecycle("_all").
Do(context.Background())
apiRes.explainlifecycle = res
apiRes.which = ResponseExplain
arerr = err
case "policies":
res, err := es.Ilm.
GetLifecycle().
Do(context.Background())
apiRes.lifecycle = &res
apiRes.which = ResponseLifecycle
arerr = err
}
if arerr != nil {
apiRes.error = fmt.Errorf("failed to get data from API: %w", arerr)
}
reschan <- apiRes
}

View File

@@ -1,153 +0,0 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"bytes"
"crypto/tls"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"slices"
"strings"
"codeberg.org/scip/esctl/pkg/cfg"
"github.com/chzyer/readline"
)
func encodeAuth(username, password string) string {
return base64.StdEncoding.EncodeToString([]byte(username + ":" + password))
}
func CallAPI(conf *cfg.Config, input []string) error {
var data string
verb := strings.ToUpper(input[0])
path := input[1]
if len(input) == 3 {
data = input[2]
}
// we're using port-forwards anyway
tr := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
req, err := http.NewRequest(verb, conf.DefaultCluster.Uri+path, bytes.NewBuffer([]byte(data)))
if err != nil {
return err
}
req.Header.Add("Content-Type", "application/json")
req.Header.Add("accept", "application/json")
req.Header.Add("Authorization", "Basic "+encodeAuth(conf.DefaultCluster.User, conf.DefaultCluster.Pass))
// actually execute the request
resp, err := client.Do(req)
if err != nil {
return err
}
// Read and print response
body, err := io.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("failed to read response body: %s", err)
}
var pretty bytes.Buffer
error := json.Indent(&pretty, body, "", "\t")
if error != nil {
return fmt.Errorf("json parse error: %s", err)
}
fmt.Println(pretty.String())
return nil
}
func Repl(conf *cfg.Config) error {
verbs := []string{"post", "get", "put", "delete"}
fmt.Println("Input format: verb path [data]")
fmt.Println("example: post /yourindex/_ccr/pause_follow")
reader, err := readline.NewEx(&readline.Config{
Prompt: "> ",
HistoryFile: os.Getenv("HOME") + "/.config/esctl/history",
HistoryLimit: 500,
InterruptPrompt: "^C",
EOFPrompt: "exit",
HistorySearchFold: true,
})
if err != nil {
return fmt.Errorf("failed to initialize readline lib: %s", err)
}
for {
text, err := reader.Readline()
if err != nil {
break
}
text = strings.TrimSpace(text)
if text == "" {
continue
}
parts := strings.SplitN(strings.TrimSpace(text), " ", 3)
if len(parts) < 2 {
fmt.Println("error: you need to input a verb, uri [and post data]")
continue
}
if !slices.Contains(verbs, strings.ToLower(parts[0])) {
fmt.Println("error: verb must be one of " + strings.Join(verbs, ","))
continue
}
if !strings.HasPrefix(parts[1], "/") {
fmt.Println("error: url path must start with /")
continue
}
if len(parts) == 3 {
data := map[string]any{}
err := json.Unmarshal([]byte(parts[2]), &data)
if err != nil {
fmt.Printf("error: input data is not proper JSON: %s", err)
continue
}
}
err = CallAPI(conf, parts)
if err != nil {
fmt.Printf("failed to call API: %s\n", err)
}
reader.SetPrompt("> ")
}
return nil
}

243
pkg/es/role.go Normal file
View File

@@ -0,0 +1,243 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"fmt"
"log/slog"
"maps"
"slices"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
func RoleNames(conf *cfg.Config) ([]string, error) {
res, err := conf.DefaultCluster.ES().Security.GetRole().
Do(context.Background())
if err != nil {
return nil, fmt.Errorf("failed to get roles: %w", esErrorString(err))
}
return slices.Collect(maps.Keys(res)), nil
}
func RoleList(conf *cfg.Config) error {
res, err := conf.DefaultCluster.ES().Security.GetRole().
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get roles: %w", esErrorString(err))
}
slog.Debug("ES result", "roles", res)
table := printer.NewTable(conf).WithSize(3, len(res))
table.Addheaders("role", "index roles", "cluster roles")
idx := 0
for name, role := range res {
table.Entries[idx] = []any{
name,
role.Cluster,
role.Indices,
}
idx++
}
table.Sort()
return table.Print()
}
func RoleShow(conf *cfg.Config, rolename string) error {
res, err := conf.DefaultCluster.ES().Security.GetRole().
Name(rolename).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get role: %w", esErrorString(err))
}
slog.Debug("ES result", "role", res)
role, exists := res[rolename]
if !exists {
return fmt.Errorf("role %s does not exist", rolename)
}
if len(role.Indices) > 0 {
if err := roleIndices(conf, role); err != nil {
return err
}
fmt.Println()
}
if len(role.RemoteIndices) > 0 {
if err := roleRemoteIndices(conf, role); err != nil {
return err
}
fmt.Println()
}
if len(role.Cluster) > 0 {
if err := roleClusters(conf, role); err != nil {
return err
}
fmt.Println()
}
if len(role.RemoteCluster) > 0 {
if err := roleRemoteClusters(conf, role); err != nil {
return err
}
fmt.Println()
}
if len(role.Applications) > 0 {
if err := roleApplications(conf, role); err != nil {
return err
}
}
return nil
}
func roleRemoteClusters(conf *cfg.Config, role types.Role) error {
if len(role.RemoteCluster) == 0 {
return nil
}
table := printer.NewTable(conf).WithSize(2, len(role.RemoteCluster))
table.Addheaders("remote cluster", "privilege")
idx := 0
for _, priv := range role.RemoteCluster {
perms := []string{}
for _, perm := range priv.Privileges {
perms = append(perms, perm.Name)
}
table.Entries[idx] = []any{priv.Clusters, perms}
idx++
}
table.Sort()
return table.Print()
}
func roleClusters(conf *cfg.Config, role types.Role) error {
if len(role.Cluster) == 0 {
return nil
}
table := printer.NewTable(conf).WithSize(1, len(role.Cluster))
table.Addheaders("cluster rights")
idx := 0
for _, cluster := range role.Cluster {
table.Entries[idx] = []any{cluster.Name}
idx++
}
table.Sort()
return table.Print()
}
func roleRemoteIndices(conf *cfg.Config, role types.Role) error {
if len(role.RemoteIndices) == 0 {
return nil
}
table := printer.NewTable(conf).WithSize(3, len(role.RemoteIndices))
table.Addheaders("remote index names", "index permissions", "allow restricted")
idx := 0
for _, priv := range role.RemoteIndices {
perms := []string{}
for _, perm := range priv.Privileges {
perms = append(perms, perm.Name)
}
table.Entries[idx] = []any{priv.Names, perms, *priv.AllowRestrictedIndices}
idx++
}
table.Sort()
return table.Print()
}
func roleIndices(conf *cfg.Config, role types.Role) error {
if len(role.Indices) == 0 {
return nil
}
table := printer.NewTable(conf).WithSize(3, len(role.Indices))
table.Addheaders("index names", "index permissions", "allow restricted")
idx := 0
for _, priv := range role.Indices {
perms := []string{}
for _, perm := range priv.Privileges {
perms = append(perms, perm.Name)
}
table.Entries[idx] = []any{priv.Names, perms, *priv.AllowRestrictedIndices}
idx++
}
table.Sort()
return table.Print()
}
func roleApplications(conf *cfg.Config, role types.Role) error {
if len(role.Applications) == 0 {
return nil
}
table := printer.NewTable(conf).WithSize(3, len(role.Applications))
table.Addheaders("application", "privileges", "resources")
idx := 0
for _, priv := range role.Applications {
table.Entries[idx] = []any{
priv.Application,
priv.Privileges,
priv.Resources,
}
idx++
}
table.Sort()
return table.Print()
}

362
pkg/es/role_diff.go Normal file
View File

@@ -0,0 +1,362 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"bufio"
"bytes"
"context"
"encoding/csv"
"fmt"
"log"
"log/slog"
"os"
"slices"
"strings"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/alecthomas/repr"
"github.com/elastic/go-elasticsearch/v9/typedapi/security/getrole"
)
// use static csv record positions as const vars so we can modify it
// if the csv format ever changes
const (
Rindexname = iota
Rrole
Rindexprivilege
Rclusterprivilege
Radgroup
Rspace
Rretention
Rkibanaprivilege
Rfieldprivilege
)
type Record struct {
// filled from CSV input
index_name string
role string
index_privilege string
cluster_privilege []string
ad_group []string
space string
retention string
kibana_privilege string
field_privilege string
// set by ourselfes
defined bool
}
type Register struct {
name string
deployed, defined bool
}
// generic variant, we do not account for multiple rows of the same
// role, in such cases an entry will simply overwritten. Use
// getCsvRecord() for a single role.
func getCsvRecords(conf *cfg.Config, csvfile string) (map[string]Record, error) {
data, err := os.ReadFile(csvfile)
if err != nil {
return nil, fmt.Errorf("failed to read CSV file: %w", err)
}
csvreader := csv.NewReader(bytes.NewReader(data))
csvreader.Comma = rune(conf.Separator[0])
csvreader.Comment = '#'
csvreader.TrimLeadingSpace = true
rows, err := csvreader.ReadAll()
if err != nil {
return nil, fmt.Errorf("failed to parse CSV: %w", err)
}
records := make(map[string]Record, len(rows)-1)
for idx, row := range rows {
if idx == 0 {
continue // header
}
records[row[1]] = Record{
index_name: row[Rindexname],
role: row[Rrole],
index_privilege: row[Rindexprivilege],
cluster_privilege: []string{row[Rindexprivilege]},
ad_group: []string{row[Rclusterprivilege]},
space: row[Rspace],
retention: row[Rretention],
kibana_privilege: row[Rkibanaprivilege],
field_privilege: row[Rfieldprivilege],
defined: true,
}
}
return records, nil
}
// same thing as above but for one specific role. supports multiple
// rows of the same record with different values which will be
// combined.
func getCsvRecord(conf *cfg.Config, csvfile, rolename string) (*Record, error) {
fd, err := os.Open(csvfile)
if err != nil {
return nil, fmt.Errorf("failed to open CSV file: %w", err)
}
defer func() {
if err := fd.Close(); err != nil {
log.Fatalf("failed to close file: %s", err)
}
}()
scanner := bufio.NewScanner(fd)
record := new(Record{role: rolename})
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if strings.HasPrefix(line, "#") || line == "" {
continue
}
row := strings.Split(line, conf.Separator)
if row[Rrole] == rolename {
record.index_name = row[Rindexname]
record.index_privilege = row[Rindexprivilege]
record.cluster_privilege = strings.Split(row[Rclusterprivilege], ",")
record.ad_group = append(record.ad_group, row[Radgroup])
record.space = row[Rspace]
record.retention = row[Rretention]
record.kibana_privilege = row[Rkibanaprivilege]
record.field_privilege = row[Rfieldprivilege]
record.defined = true
}
}
return record, nil
}
func diffRoles(conf *cfg.Config, records map[string]Record, res getrole.Response) []Register {
rows := []Register{}
filtered := []Register{}
deployed := map[string]int{}
// iterate over deployed roles
for name := range res {
reg := Register{name: name}
_, defined := records[name]
if defined {
reg.deployed = true
reg.defined = true
} else {
reg.deployed = true
reg.defined = false
}
deployed[name] = 1
rows = append(rows, reg)
}
// iterate over records from CSV and register only those which are not deployed
for name := range records {
reg := Register{name: name, defined: true}
_, deployed := deployed[name]
if !deployed {
rows = append(rows, reg)
}
}
for _, reg := range rows {
if (conf.NotDeployed && !reg.deployed) ||
(conf.Undefined && !reg.defined) ||
(conf.Diff && reg.deployed != reg.defined) ||
(!conf.Undefined && !conf.NotDeployed && !conf.Diff) {
filtered = append(filtered, reg)
}
}
return filtered
}
func RoleDiff(conf *cfg.Config, csvfile, role string) error {
records, err := getCsvRecords(conf, csvfile)
if err != nil {
return err
}
if role != "" {
return RoleDiffSingle(conf, csvfile, role)
}
res, err := conf.DefaultCluster.ES().Security.GetRole().
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get roles: %w", esErrorString(err))
}
rows := diffRoles(conf, records, res)
table := printer.NewTable(conf).WithSize(3, len(rows))
table.Addheaders("role", "is deployed", "is defined")
for idx, row := range rows {
deployed := printer.Colorize(conf, "green", "deployed")
if !row.deployed {
deployed = printer.Colorize(conf, "red", "not deployed")
}
defined := printer.Colorize(conf, "green", "defined")
if !row.defined {
defined = printer.Colorize(conf, "red", "undefined")
}
table.Entries[idx] = []any{
row.name,
deployed,
defined,
}
}
table.Sort()
return table.Print()
}
func getRoleMappingGroups(conf *cfg.Config, rolename string) ([]string, error) {
mappings, err := conf.DefaultCluster.ES().Security.GetRoleMapping().
Do(context.Background())
if err != nil {
return nil, fmt.Errorf("failed to get role mappings: %w", esErrorString(err))
}
groups := []string{}
for _, mapping := range mappings {
if slices.Contains(mapping.Roles, rolename) {
for _, rule := range mapping.Rules.Any {
for _, group := range rule.Field["groups"] {
groups = append(groups, group.(string))
}
}
}
}
return groups, nil
}
func compareSlices(name string, left, right []string) {
slices.Sort(left)
slices.Sort(right)
if slices.Compare(left, right) != 0 {
fmt.Printf("%s differs:\ndeployed: %s\n csv: %s\n",
name, strings.Join(left, ","), strings.Join(right, ","))
} else {
fmt.Printf("deployed %s matches csv definition\n", name)
}
}
func RoleDiffSingle(conf *cfg.Config, csvfile, rolename string) error {
res, err := conf.DefaultCluster.ES().Security.GetRole().
Name(rolename).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get role: %w", esErrorString(err))
}
record, err := getCsvRecord(conf, csvfile, rolename)
if err != nil {
return err
}
if !record.defined {
fmt.Printf("role %s is not defined\n", rolename)
return nil
}
role, exists := res[rolename]
if !exists {
fmt.Printf("role %s is not deployed\n", rolename)
return nil
} else {
fmt.Printf("role %s is deployed\n", rolename)
}
slog.Debug("found role", "role", role)
if conf.Debug {
// slog.Debug doesn't print it, for whatever reason
repr.Println(record)
}
groups, err := getRoleMappingGroups(conf, rolename)
if err != nil {
return err
}
slog.Debug("group mappings", "groups", groups)
// check cluster setting
clusters := []string{}
for _, cluster := range role.Cluster {
clusters = append(clusters, cluster.Name)
}
// check index names+privs
indices := []string{}
privs := []string{}
for _, index := range role.Indices {
for _, name := range index.Names {
indices = append(indices, strings.ReplaceAll(name, "**", "*"))
}
for _, priv := range index.Privileges {
privs = append(privs, priv.Name)
}
}
// check kibana application space
spaces := []string{}
for _, app := range role.Applications {
for _, resource := range app.Resources {
if strings.Contains(resource, "space:") {
parts := strings.Split(resource, ":")
if len(parts) == 2 {
spaces = append(spaces, parts[1])
}
}
}
}
compareSlices("cluster_privilege", clusters, record.cluster_privilege)
compareSlices("ad_group", groups, record.ad_group)
compareSlices("index_name", indices, []string{record.index_name})
compareSlices("index_privilege", privs, []string{record.index_privilege})
compareSlices("space", spaces, []string{record.space})
return nil
}

68
pkg/es/rollover.go Normal file
View File

@@ -0,0 +1,68 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"codeberg.org/scip/esctl/pkg/cfg"
"github.com/elastic/go-elasticsearch/v9/typedapi/esdsl"
"github.com/elastic/go-elasticsearch/v9/typedapi/indices/rollover"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
func RolloverConditions(conf *cfg.Config) types.RolloverConditionsVariant {
cond := esdsl.NewRolloverConditions()
if conf.MaxAge != "" {
cond.MaxAge(esdsl.NewDuration().String(conf.MaxAge))
}
if conf.MaxDocs > 0 {
cond.MaxDocs(conf.MaxDocs)
}
if conf.MaxShardSize > 0 {
bs := esdsl.NewByteSize()
bs.Int64(int64(conf.MaxShardSize))
cond.MaxPrimaryShardSize(bs)
}
if conf.MaxShardDocs > 0 {
cond.MaxPrimaryShardDocs(int64(conf.MaxShardDocs))
}
return cond
}
func RolloverAlias(conf *cfg.Config, alias string) (*rollover.Response, error) {
roll := conf.DefaultCluster.ES().Indices.Rollover(alias)
if conf.Wait {
roll.WaitForActiveShards("all")
}
if conf.DryRun {
roll.DryRun(true)
}
cond := RolloverConditions(conf)
roll.Conditions(cond)
return roll.Do(context.Background())
}

View File

@@ -18,14 +18,24 @@ package es
import (
"context"
"encoding/json"
"fmt"
"log"
"log/slog"
"strings"
"time"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/elastic/go-elasticsearch/v9/typedapi/core/search"
"github.com/elastic/go-elasticsearch/v9/typedapi/esdsl"
"github.com/elastic/go-elasticsearch/v9/typedapi/indices/validatequery"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
"github.com/elastic/go-elasticsearch/v9/typedapi/types/enums/sortorder"
"github.com/tidwall/gjson"
)
const (
MAXPAGE = 5000
)
/*
@@ -35,49 +45,202 @@ Execute an ES search.
additional filters can be given as -F key=value
*/
func Search(conf *cfg.Config, queries []string) error {
query := esdsl.NewBoolQuery()
if conf.Validate {
return validateSearch(conf, queries)
}
for _, q := range queries {
parts := strings.Split(q, "=")
if len(parts) != 2 {
return fmt.Errorf("search queries must be in the form field=pattern")
searchEs := conf.DefaultCluster.ES().Search().Index(conf.Index)
queryCaster, err := prepareQuery(conf, queries)
if err != nil {
return err
}
req := new(search.Request{Query: queryCaster})
searchEs.Request(req)
searchEs = addSort(conf, searchEs)
switch {
case conf.Tail:
return searchTail(conf, searchEs)
case conf.Explain:
return explainSearch(conf, searchEs)
default:
if conf.To > MAXPAGE {
return searchPit(conf, req)
} else {
return searchOnce(conf, searchEs)
}
query.Must(esdsl.NewMatchQuery(parts[0], parts[1]))
}
}
filters := make([]types.QueryVariant, len(conf.Filter))
for idx, filter := range conf.Filter {
parts := strings.Split(filter, "=")
if len(parts) != 2 {
return fmt.Errorf("invalid filter spec: %s, expecting key=value", filter)
}
filters[idx] = esdsl.NewTermQuery(parts[0], esdsl.NewFieldValue().String(parts[1]))
}
if len(filters) > 0 {
query.Filter(filters...)
}
res, err := conf.DefaultCluster.ES.Search().
Index(conf.Index).
Request(&search.Request{
Query: query.QueryCaster(),
From: &conf.From,
Size: &conf.To,
}).
func explainSearch(conf *cfg.Config, search *search.Search) error {
res, err := search.
Explain(true).
Size(1). // one's enough for explain
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to run search (esdsl): %s", err)
return fmt.Errorf("failed to call explain search (esdsl): %w", esErrorString(err))
}
slog.Debug("ES result", "search", res)
if conf.Debug {
raw, err := json.Marshal(res)
if err != nil {
return fmt.Errorf("failed to marshal explain result: %w", err)
}
for _, hit := range res.Hits.Hits {
fmt.Printf("%s\n", hit.Source_)
value := gjson.Get(string(raw), "hits.hits.0._explanation")
fmt.Println(value.String())
}
if len(res.Hits.Hits) > 0 {
ex := res.Hits.Hits[0].Explanation_
fmt.Println(ex.Description)
fmt.Println(ex.Value)
// recurse into explanation details (it's a tree)
for _, ex := range ex.Details {
explain(&ex, " ")
}
}
return nil
}
func explain(res *types.ExplanationDetail, indent string) {
fmt.Println(indent + "- " + res.Description)
for _, ex := range res.Details {
fmt.Println(indent + " - " + ex.Description)
fmt.Println(indent + fmt.Sprintf(" score: %f", res.Value))
explain(&ex, indent+" ")
}
}
func validateSearch(conf *cfg.Config, queries []string) error {
validate := conf.DefaultCluster.ES().Indices.ValidateQuery()
queryCaster, err := prepareQuery(conf, queries)
if err != nil {
return err
}
req := new(validatequery.Request{Query: queryCaster})
validate.Request(req)
res, err := validate.
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to validate search (esdsl): %w", esErrorString(err))
}
slog.Debug("ES result", "search", res)
if res.Valid {
fmt.Println(printer.Colorize(conf, "green", "valid"))
} else {
fmt.Println(printer.Colorize(conf, "red", "invalid"))
}
return nil
}
func searchOnce(conf *cfg.Config, search *search.Search) error {
res, err := search.
From(conf.From).
Size(conf.To).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to run search (esdsl): %w", esErrorString(err))
}
slog.Debug("ES result", "search", res)
printer.PrintDocs(conf, res.Hits.Hits)
return nil
}
// https://www.elastic.co/docs/reference/elasticsearch/clients/go/using-the-api/searching#_pit_search_after
func searchPit(conf *cfg.Config, req *search.Request) error {
ctx := context.Background()
pit, err := conf.DefaultCluster.ES().OpenPointInTime(conf.Index).KeepAlive("1m").Do(ctx)
if err != nil {
return fmt.Errorf("failed to open point-in-time request for search: %w", err)
}
defer func() {
_, err := conf.DefaultCluster.ES().ClosePointInTime().Id(pit.Id).Do(ctx)
if err != nil {
log.Fatalf("failed to close PIT: %s", err)
}
}()
search := conf.DefaultCluster.ES().Search().
Request(req).
Pit(esdsl.NewPointInTimeReference().
Id(pit.Id).
KeepAlive(esdsl.NewDuration().String("1m"))).
Sort(esdsl.NewSortOptions().
AddSortOption("_shard_doc", esdsl.NewFieldSort(sortorder.Asc))).
Size(conf.To)
search = addSort(conf, search)
for {
res, err := search.Do(ctx)
if err != nil {
return fmt.Errorf("failed to run search (esdsl pit): %w", esErrorString(err))
}
if len(res.Hits.Hits) == 0 {
break
}
printer.PrintDocs(conf, res.Hits.Hits)
last := res.Hits.Hits[len(res.Hits.Hits)-1]
search = search.SearchAfterValues(last.Sort)
if res.PitId != nil {
search = search.Pit(esdsl.NewPointInTimeReference().
Id(*res.PitId).
KeepAlive(esdsl.NewDuration().String("1m")))
}
}
return nil
}
func searchTail(conf *cfg.Config, search *search.Search) error {
docs := map[string]int{}
fmt.Println("enter ctrl-c to abort...")
for {
res, err := search.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to run search (esdsl): %w", esErrorString(err))
}
slog.Debug("ES result", "search", res)
for _, hit := range res.Hits.Hits {
_, exists := docs[*hit.Id_]
if exists {
continue
}
printer.PrintDoc(conf, hit)
fmt.Println()
docs[*hit.Id_] = 1
}
time.Sleep(100 * time.Millisecond)
}
}

279
pkg/es/search_filter.go Normal file
View File

@@ -0,0 +1,279 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"errors"
"fmt"
"log/slog"
"strings"
"codeberg.org/scip/esctl/pkg/cfg"
"github.com/elastic/go-elasticsearch/v9/typedapi/core/search"
"github.com/elastic/go-elasticsearch/v9/typedapi/esdsl"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
"github.com/elastic/go-elasticsearch/v9/typedapi/types/enums/operator"
"github.com/elastic/go-elasticsearch/v9/typedapi/types/enums/sortorder"
)
const (
Fmust = iota
Fmustnot
Fshould
)
// hold filter configuration, can be used multiple times through a search
type filter struct {
term, filter string // message=foo
mterm []string // multi_match
criteria int // must(=), mustnot(!=), should(?)
multi bool // message,title=foo => filter:foo, multi: []string{"message","title"}
}
// Build a new filter object. We use this to build our elastic query
// out of it. We support different types of queries:
//
// - nop filter: no query at all, just return the first N documents.
//
// - simple ones like: "authenticated", which match across all fields
//
// - simple ones combined like "authenticated monitoring", if -O was set,
// apply an OR logic
//
// - specific field queries: "message=authenticated" (can be combined too like above)
//
// - and specific field queries with negation: "message!=authenticated"
//
// All queries support additional filters using -F field=value, which
// must match literally, and range filters using -r "@timestamp:2026-05-28T10:00:00 to now"
func NewFilter(query string) (*filter, error) {
var separator string
var criteria int // we use the constants on top for this
switch {
case strings.Contains(query, "!="):
criteria = Fmustnot
separator = "!="
default:
criteria = Fmust
separator = "="
}
part := strings.Split(query, separator)
if len(part) != 2 {
return nil, errors.New("search queries must be in the form field<sep>pattern where <sep> must be one of: = or !=")
}
flt := new(filter{term: part[0], filter: part[1], criteria: criteria})
if strings.Contains(part[0], ",") {
// a MultiMatchQuery, match across multiple fields at once
multi := strings.Split(part[0], ",")
flt.multi = true
flt.mterm = multi
}
return flt, nil
}
// Build a complex query set for queries containing field[!]=pattern
func mkMatchQueries(queries []string, op operator.Operator) ([]types.QueryVariant, []types.QueryVariant, error) {
matchqueries := []types.QueryVariant{}
matchNotqueries := []types.QueryVariant{}
for _, q := range queries {
filter, err := NewFilter(q)
if err != nil {
return nil, nil, err
}
// by default we match on a single field
var match types.QueryVariant = esdsl.NewMatchQuery(filter.term, filter.filter)
if filter.multi {
// ok, match across multiple given fields
match = esdsl.NewMultiMatchQuery(filter.filter).Fields(filter.mterm...).Operator(op)
}
if filter.criteria == Fmustnot {
matchNotqueries = append(matchNotqueries, match)
} else {
matchqueries = append(matchqueries, match)
}
}
return matchqueries, matchNotqueries, nil
}
// Prepare user search query and turn it into a proper search request
func prepareQuery(conf *cfg.Config, queries []string) (*types.Query, error) {
// logical operator for simple and multimatch queries
op := operator.Operator{Name: "AND"}
if conf.Or {
op = operator.Operator{Name: "OR"}
}
query := esdsl.NewBoolQuery()
wholeQuery := strings.Join(queries, " ")
switch {
case len(queries) == 0:
// nothing provided via ARGs, so match any docs
query.Must(esdsl.NewMatchAllQuery())
case !strings.ContainsAny(wholeQuery, "!="):
// simple query w/o any field[!]=pattern style
query.Must(esdsl.NewSimpleQueryStringQuery(wholeQuery).DefaultOperator(op))
default:
// a complex query
matchqueries, matchNotqueries, err := mkMatchQueries(queries, op)
if err != nil {
return nil, err
}
// apply boolean logic
if conf.Or {
query.Should(matchqueries...)
} else {
// by default we use AND
query.Must(matchqueries...)
}
if matchNotqueries != nil {
query.MustNot(matchNotqueries...)
}
}
slog.Debug("complex query", "query", query)
// there might be boolean or range filters like -Ffield=value as well
filters, err := addFilters(conf)
if err != nil {
return nil, err
}
if filters != nil {
query.Filter(filters...)
}
return query.QueryCaster(), nil
}
// Build a slice of filters, to be fed into query.Filter() including static ones
// like -Ffield=value and ranges like -r "@timestamp:2026-05-28T10:00:00 to now"
func addFilters(conf *cfg.Config) ([]types.QueryVariant, error) {
count := len(conf.Filter)
if conf.Range != "" {
count++
}
if count == 0 {
return nil, nil
}
filters := make([]types.QueryVariant, count)
// static filters
for idx, filter := range conf.Filter {
parts := strings.Split(filter, "=")
if len(parts) != 2 {
return nil, fmt.Errorf("invalid filter spec: %s, expecting key=value", filter)
}
filters[idx] = esdsl.NewTermQuery(parts[0], esdsl.NewFieldValue().String(parts[1]))
}
// range filters. format: @timestamp:2026-05-05 to 2026-05-15
// see: https://www.elastic.co/docs/reference/elasticsearch/rest-apis/common-options#date-math
if conf.Range != "" {
parts := strings.SplitN(conf.Range, ":", 2)
if len(parts) != 2 {
return nil, errors.New("invalid range format, expected field:range")
}
field := parts[0]
parts = strings.Split(parts[1], " to ")
if len(parts) != 2 {
return nil, errors.New("invalid date range format, expected '<start> to <end>'")
}
from := parts[0]
to := parts[1]
slog.Debug("time range filter",
"from", from,
"to", to,
"format", conf.TimestampFormat,
"count", count,
)
rng := esdsl.NewDateRangeQuery(field).
Gte(from).
Lte(to)
if strings.Contains(parts[1], ":") {
// specific format not needed as long as there are no times specified
rng.Format(conf.TimestampFormat)
}
filters[count-1] = rng
}
return filters, nil
}
// check if the conf.SortBy field (default: @timestamp) is searchable
// by using the field capabilities API.
func addSort(conf *cfg.Config, search *search.Search) *search.Search {
res, err := conf.DefaultCluster.ES().FieldCaps().
Index(conf.Index).
Fields(conf.SortBy).
Do(context.Background())
if err != nil {
// whatever it was, do not add Sort()
slog.Debug("get field capabilities", "field", conf.SortBy, "error", esErrorString(err))
return search
}
field, exists := res.Fields[conf.SortBy]
if exists {
// good, the field exists
for _, cap := range field {
if cap.Searchable {
// ok, ES would be willing to sort by this field
order := esdsl.NewFieldSort(sortorder.Desc)
if conf.Ascending {
order = esdsl.NewFieldSort(sortorder.Asc)
}
search = search.Sort(
esdsl.NewSortOptions().AddSortOption(conf.SortBy, order),
)
break
}
}
}
return search
}

79
pkg/es/searchql.go Normal file
View File

@@ -0,0 +1,79 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
)
type searchQlColumn struct {
Name string `json:"name"`
Type string `json:"type"`
}
type searchQlResult struct {
Partial bool `json:"is_partial"`
Docs int `json:"documents_found"`
Columns []searchQlColumn `json:"columns"`
Values [][]any `json:"values"`
}
func SearchQL(conf *cfg.Config, querystring string) error {
query := conf.DefaultCluster.ES().Esql.Query().
Query(querystring).
DropNullColumns(true)
res, err := query.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to run esql query: %w", esErrorString(err))
}
if conf.Debug {
fmt.Println(string(res))
}
qlResult := searchQlResult{}
if err = json.Unmarshal(res, &qlResult); err != nil {
return fmt.Errorf("failed to unmarshal esql result: %w", err)
}
slog.Debug("searchql result", "qlres", qlResult)
if qlResult.Docs == 0 {
return nil
}
table := printer.NewTable(conf).WithSize(len(qlResult.Columns), len(qlResult.Values))
headers := make([]string, len(qlResult.Columns))
for idx, col := range qlResult.Columns {
headers[idx] = col.Name
}
table.Addheaders(headers...)
table.Entries = qlResult.Values
return table.Print()
}

214
pkg/es/shard.go Normal file
View File

@@ -0,0 +1,214 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"fmt"
"log/slog"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
"github.com/elastic/go-elasticsearch/v9/typedapi/cat/shards"
)
func colorzizeShard(conf *cfg.Config, state, name string) string {
color := "red" // in case of RELOCATING and UNASSIGNED
switch state {
case "STARTED":
color = "green"
case "INITIALIZING":
color = "yellow"
}
return printer.Colorize(conf, color, name)
}
func resolvePrirep(state string) string {
if state == "p" {
return "primary"
}
return "replica"
}
func filterShards(conf *cfg.Config, shardlist shards.Response) shards.Response {
filtered := shards.Response{}
size := len(shardlist)
if conf.MaxItems > 0 {
if size > conf.MaxItems {
size = conf.MaxItems
}
}
for idx, shard := range shardlist {
if conf.Failed {
if *shard.State == "STARTED" {
continue
}
}
if conf.Primary {
if *shard.Prirep != "p" {
continue
}
}
if idx == size-1 {
break
}
filtered = append(filtered, shard)
}
return filtered
}
func ShardList(conf *cfg.Config) error {
res, err := conf.DefaultCluster.ES().Cat.Shards().
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get shards: %w", esErrorString(err))
}
shardlist := filterShards(conf, res)
slog.Debug("ES result", "shards", res)
return printShards(conf, shardlist)
}
func printShards(conf *cfg.Config, shardlist shards.Response) error {
headers := []string{"index", "shard", "is primary", "store", "dataset", "docs"}
if conf.Verbose {
headers = append(headers, "node", "ip")
}
table := printer.NewTable(conf).WithSize(len(headers), len(shardlist))
table.Addheaders(headers...)
for idx, shard := range shardlist {
name := colorzizeShard(conf, *shard.State, *shard.Index)
table.Entries[idx] = []any{
name,
*shard.Shard,
resolvePrirep(*shard.Prirep),
*shard.Store,
*shard.Dataset,
*shard.Docs,
}
if conf.Verbose {
table.Entries[idx] = append(table.Entries[idx],
*shard.Node,
*shard.Ip,
)
}
}
table.Sort()
if err := table.Print(); err != nil {
return err
}
return nil
}
func ShardShow(conf *cfg.Config, index string) error {
res, err := conf.DefaultCluster.ES().Cat.Shards().Index(index).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get shards: %w", esErrorString(err))
}
slog.Debug("ES result", "shards", res)
conf.Verbose = true
return printShards(conf, res)
}
func ShardAllocation(conf *cfg.Config, index string) error {
explain := conf.DefaultCluster.ES().Cluster.AllocationExplain().
Index(index).
Primary(conf.Primary).
Shard(conf.Shards)
if conf.FromNode != "" {
explain.CurrentNode(conf.FromNode)
}
res, err := explain.Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get shard allocation explain: %w", esErrorString(err))
}
slog.Debug("ES result", "explain", res)
table := printer.NewTable(conf).WithSize(2, 10)
table.Addheaders("shard allocation setting", "value")
table.Entries = [][]any{
{"Index", index},
{"Current state", res.CurrentState},
{"Can rebalance cluster", res.CanRebalanceCluster.Name},
{"Can rebalance to another node", res.CanRebalanceToOtherNode.Name},
{"Can remain on current node", res.CanRemainOnCurrentNode.Name},
}
if res.CurrentNode != nil {
currentNode := res.CurrentNode
roles := make([]string, len(currentNode.Roles))
for idx, role := range currentNode.Roles {
roles[idx] = role.Name
}
table.Entries = append(table.Entries, [][]any{
{"Current node", currentNode.Name},
{"Current k8s node", currentNode.Attributes["k8s_node_name"]},
{"Current node address", currentNode.TransportAddress},
{"Current node id", currentNode.Id},
{"Current node weight", currentNode.WeightRanking},
{"Current node roles", roles},
}...)
} else {
table.AddRow("Current node", "not currently assigned to any node")
}
if res.CurrentState == "unassigned" {
table.AddRow("Unassignment reason", res.UnassignedInfo.Reason.String()+" at "+res.UnassignedInfo.At.(string))
}
for _, nodeDecision := range res.NodeAllocationDecisions {
for _, decider := range nodeDecision.Deciders {
table.AddRow("Allocation decider", decider.Decider)
table.AddRow(" -> decision", decider.Decision.String())
table.AddRow(" -> explanation", decider.Explanation)
}
}
if err := table.Print(); err != nil {
return err
}
return nil
}

View File

@@ -25,6 +25,7 @@ import (
"strings"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
)
var (
@@ -41,39 +42,39 @@ type Snapshot struct {
}
func SnapshotList(conf *cfg.Config) error {
// get partial indicies
ires, err := conf.DefaultCluster.ES.Cat.Indices().Do(context.Background())
// get partial indices
ires, err := conf.DefaultCluster.ES().Cat.Indices().Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get indicies: %s", err)
return fmt.Errorf("failed to get indices: %w", esErrorString(err))
}
indicies := map[string]int{}
indices := map[string]int{}
for _, index := range ires {
name := strings.ReplaceAll(*index.Index, "partial-", "")
indicies[name] = 1
indices[name] = 1
}
// get snapshots
sres, err := conf.DefaultCluster.ES.Cat.Snapshots().Do(context.Background())
sres, err := conf.DefaultCluster.ES().Cat.Snapshots().Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get snapshots: %s", err)
return fmt.Errorf("failed to get snapshots: %w", esErrorString(err))
}
slog.Debug("ES result", "indicies", sres)
slog.Debug("ES result", "indices", sres)
snapshots := []*Snapshot{} // original snapshot names
for _, snapshot := range sres {
snap := &Snapshot{
snap := new(Snapshot{
Name: *snapshot.Id,
Status: *snapshot.Status,
Start: fmt.Sprintf("%s", snapshot.StartTime),
Forindex: indexFromSnapshot(*snapshot.Id),
Orphaned: "no",
}
})
_, exists := indicies[snap.Forindex]
_, exists := indices[snap.Forindex]
if !exists {
snap.Orphaned = "orphaned"
}
@@ -83,11 +84,11 @@ func SnapshotList(conf *cfg.Config) error {
}
}
table := NewTable(5, len(snapshots))
table := printer.NewTable(conf).WithSize(5, len(snapshots))
table.Addheaders("name", "index", "start", "orphaned", "status")
for idx, snap := range snapshots {
table.entries[idx] = []string{
table.Entries[idx] = []any{
snap.Name,
snap.Forindex,
snap.Start,
@@ -97,7 +98,8 @@ func SnapshotList(conf *cfg.Config) error {
}
table.Sort()
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
return err
}
@@ -105,9 +107,9 @@ func SnapshotList(conf *cfg.Config) error {
}
func SnapshotShow(conf *cfg.Config, snapshot string) error {
res, err := conf.DefaultCluster.ES.Snapshot.Get("*", snapshot).Do(context.Background())
res, err := conf.DefaultCluster.ES().Snapshot.Get("*", snapshot).Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get snapshot: %s", err)
return fmt.Errorf("failed to get snapshot: %w", esErrorString(err))
}
slog.Debug("ES result", "snapshot", res)
@@ -116,28 +118,28 @@ func SnapshotShow(conf *cfg.Config, snapshot string) error {
return errors.New("no snapshot retrieved")
}
table := NewTable(2, 17)
table.Addheaders("field", "value")
table := printer.NewTable(conf).WithSize(2, 17)
table.Addheaders("snapshot property", "value")
snap := res.Snapshots[0]
table.entries = [][]string{
table.Entries = [][]any{
{"snapshot", snapshot},
{"uuid", snap.Uuid},
{"repository", *snap.Repository},
{"version_id", fmt.Sprintf("%d", snap.VersionId)},
{"version_id", snap.VersionId},
{"version", *snap.Version},
{"include_global_state", fmt.Sprintf("%t", *snap.IncludeGlobalState)},
{"include_global_state", *snap.IncludeGlobalState},
{"state", *snap.State},
{"start_time", fmt.Sprintf("%s", snap.StartTime)},
{"end_time", fmt.Sprintf("%s", snap.EndTime)},
{"duration_in_millis", fmt.Sprintf("%d", *snap.DurationInMillis)},
{"shards-total", fmt.Sprintf("%d", snap.Shards.Total)},
{"shards-failed", fmt.Sprintf("%d", snap.Shards.Failed)},
{"shards-successful", fmt.Sprintf("%d", snap.Shards.Successful)},
{"start_time", snap.StartTime},
{"end_time", snap.EndTime},
{"duration_in_millis", *snap.DurationInMillis},
{"shards-total", snap.Shards.Total},
{"shards-failed", snap.Shards.Failed},
{"shards-successful", snap.Shards.Successful},
}
if err := table.PrintMarkdown(); err != nil {
if err := table.Print(); err != nil {
return err
}

View File

@@ -1,114 +0,0 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"fmt"
"sort"
"strings"
"github.com/olekukonko/tablewriter"
"github.com/olekukonko/tablewriter/renderer"
"github.com/olekukonko/tablewriter/tw"
)
type Table struct {
headers []string
entries [][]string
}
func NewTable(columns, rows int) *Table {
table := Table{}
table.headers = make([]string, columns)
table.entries = make([][]string, rows)
return &table
}
func (data *Table) PrintMarkdown() error {
tableString := &strings.Builder{}
table := tablewriter.NewTable(tableString,
tablewriter.WithRenderer(
renderer.NewBlueprint(
tw.Rendition{
Borders: tw.Border{
Left: tw.On,
Right: tw.On,
Top: tw.Off,
Bottom: tw.Off,
},
Settings: tw.Settings{
Separators: tw.Separators{
ShowHeader: tw.On,
ShowFooter: tw.Off,
BetweenRows: tw.Off,
BetweenColumns: 0,
},
},
Symbols: tw.NewSymbols(tw.StyleMarkdown),
})),
tablewriter.WithConfig(
tablewriter.Config{
Header: tw.CellConfig{
Formatting: tw.CellFormatting{
Alignment: tw.AlignLeft,
AutoFormat: tw.Off,
},
},
Row: tw.CellConfig{
Formatting: tw.CellFormatting{
Alignment: tw.AlignLeft,
},
},
},
),
)
table.Header(data.headers)
if err := table.Bulk(data.entries); err != nil {
return fmt.Errorf("failed to add data to table renderer: %s", err)
}
if err := table.Render(); err != nil {
return fmt.Errorf("failed to render table: %s", err)
}
fmt.Println(tableString.String())
return nil
}
func (data *Table) Sort() {
// sanity checks
if len(data.entries) == 0 {
return
}
sort.Slice(data.entries, func(i, j int) bool {
return data.entries[i][0] < data.entries[j][0]
})
}
func (data *Table) Addheaders(headers ...string) {
for idx, header := range headers {
data.headers[idx] = bold(strings.ToUpper(header))
}
}

74
pkg/es/task.go Normal file
View File

@@ -0,0 +1,74 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
import (
"context"
"fmt"
"log/slog"
"strconv"
"time"
"codeberg.org/scip/esctl/pkg/cfg"
"codeberg.org/scip/esctl/pkg/printer"
)
func TaskList(conf *cfg.Config) error {
res, err := conf.DefaultCluster.ES().Cat.Tasks().
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to get index templates: %w", esErrorString(err))
}
slog.Debug("res", "tasks", res)
table := printer.NewTable(conf).WithSize(6, len(res))
table.Addheaders("task id", "action", "start time", "run time", "node", "type")
for idx, task := range res {
var ts time.Time
epc, err := strconv.Atoi(*task.StartTime)
if err != nil {
ts = time.Now()
} else {
ts = time.Unix(int64(epc/1000), 0)
}
table.Entries[idx] = []any{
*task.TaskId,
*task.Action,
ts,
*task.RunningTime,
*task.Node,
*task.Type,
}
}
return table.Print()
}
func TaskCancel(conf *cfg.Config, taskid string) error {
_, err := conf.DefaultCluster.ES().Tasks.Cancel().
TaskId(taskid).
Do(context.Background())
if err != nil {
return fmt.Errorf("failed to cancel task: %w", esErrorString(err))
}
return nil
}

View File

@@ -29,13 +29,13 @@ import (
const LevelNotice = slog.Level(2)
func Init(conf *cfg.Config) {
logLevel := &slog.LevelVar{}
logLevel := new(slog.LevelVar{})
opts := &yadu.Options{
opts := new(yadu.Options{
Level: logLevel,
AddSource: true,
NoColor: !isatty.IsTerminal(os.Stdout.Fd()),
}
})
buildInfo, _ := debug.ReadBuildInfo()

28
pkg/printer/builder.go Normal file
View File

@@ -0,0 +1,28 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package printer
import "strings"
// a wrapper around strings.Builder for convenience
type Builder struct {
strings.Builder
}
func (builder *Builder) WriteStringLine(in string) {
builder.WriteString(in + "\n")
}

37
pkg/printer/bytes.go Normal file
View File

@@ -0,0 +1,37 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package printer
import "github.com/dustin/go-humanize"
type ByteSize struct {
size uint64
}
func (b *ByteSize) String() string {
return humanize.Bytes(b.size)
}
func Bytes(size int64) *ByteSize {
return new(ByteSize{size: uint64(size)})
}
func ByteString(size int64) string {
b := ByteSize{size: uint64(size)}
return b.String()
}

105
pkg/printer/cast.go Normal file
View File

@@ -0,0 +1,105 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package printer
import (
"fmt"
"strconv"
"strings"
"time"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
)
func any2string(in any) string {
//nolint:gocritic
switch val := in.(type) {
case string:
return val
case bool:
return strconv.FormatBool(val)
case int64:
return strconv.FormatInt(val, 10)
case int:
return strconv.Itoa(val)
case float64:
return fmt.Sprintf("%.2f", val)
case float32:
return fmt.Sprintf("%.2f", val)
case []string:
return strings.Join(val, ",")
case ByteSize:
return val.String()
case *ByteSize:
return val.String()
case time.Time:
return val.Format("2006-01-02 15:04:05")
case time.Duration:
return val.String()
case []byte:
return string(val)
case nil:
return "null"
case types.Percentage, types.DateTime:
return val.(string)
}
return ""
}
func (data *Table) preprocessRows() {
if data.processed {
// only do it once
return
}
// convert entries to strings
data.rows = make([][]string, len(data.Entries))
for rowidx, entries := range data.Entries {
data.rows[rowidx] = make([]string, len(entries))
for colidx, entry := range data.Entries[rowidx] {
data.rows[rowidx][colidx] = any2string(entry)
}
}
// determine header lenght's
for idx, head := range data.Headers {
data.lenHeaders[idx] = visibleLen(head)
}
// determine max width per column
for _, entries := range data.rows {
currentWidth := 0
for idx, entry := range entries {
length := visibleLen(entry)
if data.lenHeaders[idx] < length {
if length > currentWidth+data.maxwidth {
data.lenHeaders[idx] = data.maxwidth - currentWidth
} else {
data.lenHeaders[idx] = length
}
}
currentWidth += data.lenHeaders[idx]
}
}
data.processed = true
}

View File

@@ -15,18 +15,34 @@ You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package es
package printer
import "github.com/fatih/color"
import (
"os"
"codeberg.org/scip/esctl/pkg/cfg"
"github.com/fatih/color"
"github.com/mattn/go-isatty"
)
var (
green = color.New(color.BgGreen, color.FgHiWhite).SprintFunc()
yellow = color.New(color.BgYellow, color.FgHiWhite).SprintFunc()
orange = color.BgRGB(255, 128, 0).AddRGB(0, 0, 0).SprintFunc()
red = color.New(color.BgRed, color.FgHiWhite).SprintFunc()
bold = color.New(color.Bold).SprintFunc()
)
func Colorize(col, what string) string {
func Colorize(conf *cfg.Config, col, what string) string {
if !isatty.IsTerminal(os.Stdout.Fd()) {
return what
}
switch conf.Output {
case "json", "yaml":
return what
}
switch col {
case "green":
what = green(what)
@@ -34,7 +50,17 @@ func Colorize(col, what string) string {
what = orange(what)
case "red":
what = red(what)
case "yellow":
what = yellow(what)
}
return what
}
func Bold(what string) string {
if !isatty.IsTerminal(os.Stdout.Fd()) {
return what
}
return bold(what)
}

56
pkg/printer/doc.go Normal file
View File

@@ -0,0 +1,56 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package printer
import (
"fmt"
"slices"
"codeberg.org/scip/esctl/pkg/cfg"
"github.com/elastic/go-elasticsearch/v9/typedapi/types"
"github.com/tidwall/gjson"
)
func PrintDocs(conf *cfg.Config, hits []types.Hit) {
if !conf.Ascending {
slices.Reverse(hits)
}
for _, hit := range hits {
PrintDoc(conf, hit)
}
}
func PrintDoc(conf *cfg.Config, hit types.Hit) {
var score types.Float64
if hit.Score_ != nil {
score = *hit.Score_
}
docjson := fmt.Sprintf(`{"id":"%s", "score":%0.4f, "index":"%s", "source":%s}`,
*hit.Id_,
score,
hit.Index_,
hit.Source_)
if conf.Path != "" {
value := gjson.Get(docjson, conf.Path)
fmt.Println(value.String())
} else {
fmt.Println(docjson)
}
}

55
pkg/printer/metrics.go Normal file
View File

@@ -0,0 +1,55 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package printer
import (
"fmt"
"runtime/metrics"
"strconv"
)
func printGoRoutineMetrics() {
fmt.Println("\nGoroutine metrics:")
printMetric("/sched/goroutines-created:goroutines", "Created")
printMetric("/sched/goroutines:goroutines", "Live")
printMetric("/sched/goroutines/not-in-go:goroutines", "Syscall/CGO")
printMetric("/sched/goroutines/runnable:goroutines", "Runnable")
printMetric("/sched/goroutines/running:goroutines", "Running")
printMetric("/sched/goroutines/waiting:goroutines", "Waiting")
fmt.Println("Thread metrics:")
printMetric("/sched/gomaxprocs:threads", "Max")
printMetric("/sched/threads/total:threads", "Live")
}
func printMetric(name string, descr string) {
sample := []metrics.Sample{{Name: name}}
metrics.Read(sample)
var val string
switch sample[0].Value.Kind() {
case metrics.KindFloat64, metrics.KindFloat64Histogram:
val = fmt.Sprintf("%.2f", sample[0].Value.Float64())
case metrics.KindUint64:
val = strconv.FormatUint(sample[0].Value.Uint64(), 10)
case metrics.KindBad:
val = "n/a"
}
fmt.Printf(" %s: %v\n", descr, val)
}

133
pkg/printer/pager.go Normal file
View File

@@ -0,0 +1,133 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package printer
// pager setup using bubbletea
// file shamlelessly copied from:
// https://github.com/charmbracelet/bubbletea/tree/main/examples/pager
import (
"fmt"
"os"
"strings"
"github.com/charmbracelet/bubbles/viewport"
tea "github.com/charmbracelet/bubbletea"
"github.com/charmbracelet/lipgloss"
)
var (
titleStyle = func() lipgloss.Style {
b := lipgloss.RoundedBorder()
b.Right = "├"
return lipgloss.NewStyle().BorderStyle(b).Padding(0, 1)
}()
infoStyle = func() lipgloss.Style {
b := lipgloss.RoundedBorder()
b.Left = "┤"
return titleStyle.BorderStyle(b)
}()
)
type model struct {
content string
title string
ready bool
viewport viewport.Model
}
func (m model) Init() tea.Cmd {
return nil
}
func (m model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
var cmd tea.Cmd
cmds := make([]tea.Cmd, 1)
switch msg := msg.(type) {
case tea.KeyMsg:
if k := msg.String(); k == "ctrl+c" || k == "q" || k == "esc" {
return m, tea.Quit
}
case tea.WindowSizeMsg:
headerHeight := lipgloss.Height(m.headerView())
footerHeight := lipgloss.Height(m.footerView())
verticalMarginHeight := headerHeight + footerHeight
if !m.ready {
// Since this program is using the full size of the viewport we
// need to wait until we've received the window dimensions before
// we can initialize the viewport. The initial dimensions come in
// quickly, though asynchronously, which is why we wait for them
// here.
m.viewport = viewport.New(msg.Width, msg.Height-verticalMarginHeight)
m.viewport.YPosition = headerHeight
m.viewport.SetContent(m.content)
m.ready = true
} else {
m.viewport.Width = msg.Width
m.viewport.Height = msg.Height - verticalMarginHeight
}
}
// Handle keyboard and mouse events in the viewport
m.viewport, cmd = m.viewport.Update(msg)
cmds[0] = cmd
return m, tea.Batch(cmds...)
}
func (m model) View() string {
if !m.ready {
return "\n Initializing..."
}
return fmt.Sprintf("%s\n%s\n%s", m.headerView(), m.viewport.View(), m.footerView())
}
func (m model) headerView() string {
// title := titleStyle.Render("RPN Help Overview")
title := titleStyle.Render(m.title)
line := strings.Repeat("─", max(0, m.viewport.Width-lipgloss.Width(title)))
return lipgloss.JoinHorizontal(lipgloss.Center, title, line)
}
func (m model) footerView() string {
info := infoStyle.Render(fmt.Sprintf("%3.f%%", m.viewport.ScrollPercent()*100))
line := strings.Repeat("─", max(0, m.viewport.Width-lipgloss.Width(info)))
return lipgloss.JoinHorizontal(lipgloss.Center, line, info)
}
func Pager(title, message string) {
pager := tea.NewProgram(
model{content: message, title: title},
tea.WithAltScreen(), // use the full size of the terminal in its "alternate screen buffer"
tea.WithMouseCellMotion(), // turn on mouse support so we can track the mouse wheel
)
if _, err := pager.Run(); err != nil {
fmt.Println("could not run pager:", err)
os.Exit(1)
}
}

325
pkg/printer/table.go Normal file
View File

@@ -0,0 +1,325 @@
/*
Copyright © 2026 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package printer
import (
"encoding/json"
"fmt"
"regexp"
"sort"
"strconv"
"strings"
"codeberg.org/scip/esctl/pkg/cfg"
"gopkg.in/yaml.v3"
)
type Table struct {
Mode string // tsv, json, yaml
Headers []string
RawHeaders []string
Entries [][]any
rows [][]string // representation used for printing
processed bool
lenHeaders []int
alignInts bool
maxwidth int
debugGoRoutines bool
}
func NewTable(conf *cfg.Config) *Table {
return new(Table{
Mode: conf.Output,
maxwidth: cfg.GetTermWidth(),
debugGoRoutines: conf.DebugGoRoutines,
Headers: []string{},
RawHeaders: []string{},
lenHeaders: []int{},
alignInts: conf.AlignInts,
})
}
func (table *Table) WithSize(columns, rows int) *Table {
if columns > 0 {
table.Headers = make([]string, columns)
table.RawHeaders = make([]string, columns)
table.lenHeaders = make([]int, columns)
}
if rows > 0 {
table.Entries = make([][]any, rows)
}
return table
}
func (table *Table) WithHeaders(headers ...string) *Table {
count := len(headers)
table.WithSize(count, 0).Addheaders(headers...)
return table
}
func (table *Table) Print() error {
var err error
switch table.Mode {
case "json":
err = table.PrintJSON()
case "yaml":
err = table.PrintYAML()
case "csv":
err = table.PrintCSV()
default:
err = table.PrintTSV()
}
if table.debugGoRoutines {
printGoRoutineMetrics()
}
return err
}
var (
ansiCtrlSeq = regexp.MustCompile(`\033.[0-9;]+m`)
)
func (table *Table) PrintYAML() error {
raw := table.toMap()
body, err := yaml.Marshal(raw)
if err != nil {
return fmt.Errorf("failed to produce YAML output: %w", err)
}
fmt.Println(string(body))
return nil
}
func (table *Table) PrintJSON() error {
raw := table.toMap()
body, err := json.MarshalIndent(raw, "", " ")
if err != nil {
return fmt.Errorf("failed to produce JSON output: %w", err)
}
fmt.Println(string(body))
return nil
}
func (table *Table) PrintTSV() error {
// length's, convert cell types
table.preprocessRows()
// output headers
for idx, header := range table.Headers {
if idx+1 != len(table.Headers) {
fmt.Print(header, strings.Repeat(" ", table.lenHeaders[idx]-visibleLen(header)))
} else {
// no padding for last header
fmt.Print(header)
}
if idx < len(table.Headers)-1 {
fmt.Print(" ")
}
}
fmt.Println()
for _, entries := range table.rows {
currentWidth := 0
columns := len(entries)
for idx, entry := range entries {
length := visibleLen(entry)
if length+currentWidth > table.maxwidth &&
table.maxwidth-currentWidth > 1 &&
idx == columns-1 {
// text is too wide to be put into one line, and
// it's the last cell, so wrap it
entry = wrap(table.maxwidth-currentWidth, currentWidth+2, entry)
}
currentWidth += table.lenHeaders[idx]
switch {
case isInt(entry) && table.alignInts:
// align right
fmt.Print(strings.Repeat(" ", table.lenHeaders[idx]-length), entry)
case length < table.lenHeaders[idx] && idx+1 != len(entries):
// pad right, if required
fmt.Print(entry, strings.Repeat(" ", table.lenHeaders[idx]-length))
default:
// no padding for last entry
fmt.Print(entry)
}
if idx < len(table.Headers)-1 {
fmt.Print(" ")
}
}
fmt.Println()
}
return nil
}
// Wrap a text into multiple lines, first line is not indented, all
// further lines will be indented. Used within Print() to print large
// cell text.
func wrap(width, indent int, text string) string {
if len(text) <= width {
return text
}
wrapped := ""
line := ""
for word := range strings.FieldsSeq(text) {
if len(line)+len(word)+1 <= width {
// appending word to current line doesn't exceed width
if line != "" {
line += " "
}
line += word
} else {
// it exceeds it, so we need to wrap
if wrapped == "" {
// beginning of output, no indenting here
wrapped = line + "\n"
} else {
// we're in the middle of the text, so add the indent
wrapped += strings.Repeat(" ", indent) + line + "\n"
}
// remember the current word for the next round
line = word
}
}
if line != "" {
// last line, no newline needed here
wrapped += strings.Repeat(" ", indent) + line
}
return wrapped
}
func (table *Table) PrintCSV() error {
table.preprocessRows()
fmt.Println(strings.Join(table.RawHeaders, ","))
for _, entries := range table.rows {
row := make([]string, len(entries))
for idx, entry := range entries {
if strings.Contains(entry, " ") || strings.Contains(entry, ",") {
row[idx] = `"` + entry + `"`
} else {
row[idx] = entry
}
}
fmt.Println(strings.Join(row, ","))
}
return nil
}
func (table *Table) Sort() {
// sanity checks
if len(table.Entries) == 0 {
return
}
table.preprocessRows()
sort.Slice(table.rows, func(i, j int) bool {
return table.rows[i][0] < table.rows[j][0]
})
}
func (table *Table) Addheaders(headers ...string) {
for idx, header := range headers {
switch table.Mode {
case "json", "yaml":
table.Headers[idx] = strings.ReplaceAll(strings.ToLower(header), " ", "_")
default:
table.Headers[idx] = Bold(strings.ReplaceAll(strings.ToUpper(header), " ", "-"))
}
table.RawHeaders[idx] = header
}
}
func (table *Table) AddRow(fields ...any) {
table.Entries = append(table.Entries, fields)
}
func (table *Table) AddRowLate(fields ...any) {
table.AddRow(fields)
if !table.processed {
return
}
row := make([]string, len(fields))
for idx, field := range fields {
row[idx] = any2string(field)
}
table.rows = append(table.rows, row)
}
// needed for json and yaml output
func (table *Table) toMap() []map[string]any {
raw := make([]map[string]any, len(table.Entries))
for idx, entries := range table.Entries {
raw[idx] = make(map[string]any, len(table.Headers))
for eidx, entry := range entries {
raw[idx][table.Headers[eidx]] = entry
}
}
return raw
}
// return the length of a string but only visible chars, w/o ansi color escapes
func visibleLen(word string) int {
return len(ansiCtrlSeq.ReplaceAllLiteralString(word, ""))
}
func isInt(num string) bool {
if _, err := strconv.Atoi(num); err == nil {
return true
}
return false
}

6
t/.env Normal file
View File

@@ -0,0 +1,6 @@
ES_PASS=tuscador1
STACK_VERSION=9.4.2
CLUSTER_NAME=domdoc
LICENSE=basic
ES_PORT=9200
MEM_LIMIT=1073741824

48
t/Makefile Normal file
View File

@@ -0,0 +1,48 @@
.PHONY: test clean cluster docs search up down delete runenv
# docker stuff
up:
@echo "booting up docker containers"
docker compose up -d --remove-orphans
waitup:
@echo "wait til es cluster is up and running"
mosscap msc-actions.yaml msc-docker.yaml -n -t 300
down:
@echo "shutting down docker containers"
docker compose down
delete:
@echo "delete docker volumes"
docker volume rm t_certs t_esdata01 t_esdata02 t_esdata03
clean-docker: down delete
runenv: up waitup cluster docs wait
# mosscap esctl stuff
test: up waitup cluster docs wait search
cluster:
@echo "setting up elastisearch cluster"
mosscap msc-actions.yaml msc-cluster.yaml -n
docs:
@echo "put some docs"
mosscap msc-actions.yaml msc-docs.yaml -n -c 1000 -p 100
wait:
sleep 10
search:
@echo "make some searches"
mosscap msc-actions.yaml msc-search.yaml -n -t 60
clean:
@echo "cleaning up cluster"
mosscap msc-actions.yaml msc-clean.yaml -n
rm -f *.state debug.log

77
t/README.md Normal file
View File

@@ -0,0 +1,77 @@
# Testing
Building regular unit tests would require to create a mock
elasticsearch cluster, which is too much a chore for a one man
show. Running a regular elasticsearch cluster on Codeberg CI is not
economically reasonable.
Therefore I run tests manually. Here's how.
# Dependencies
The following is required:
- linux, any distro will do
- docker in a decent version
- [mosscap](https://codeberg.org/scip/mosscap) version `0.0.17` or higher.
# Docker
There's a ready to use docker compose file, which fires up an
elasticsearch cluster with 3 nodes. To start it manually just execute
`make up`.
When it's up and running it should look like this:
```console
$ docker ps
NAMES STATUS PORTS
t-es03-1 Up 16 minutes (healthy) 9200/tcp, 9300/tcp
t-es02-1 Up 16 minutes (healthy) 9200/tcp, 9300/tcp
t-es01-1 Up 16 minutes (healthy) 0.0.0.0:9200->9200/tcp, [::]:9200->9200/tcp, 9300/tcp
```
There's an esctl config file `cluster.yaml`, which you can use to
access that elasticsearch cluster, e.g.:
```console
$ esctl -c cluster.yaml cluster status
DOCKER/DOMDOC STATUS
Cluster Name domdoc
ES Status green
ES Version 9.4.2
Is Leader false
Active Shards 8
Active Primary Shards 4
Unassigned Shards 0
Unassigned Primary Shards 0
Pending Tasks 0
Nodes 3
Red Indices 0
Long Running Tasks 0
```
# Automated Tests
To execute the automated tests, just execute `make test`. This boots
up the elasticsearch containers, waits until they are up and running,
creates data on it and queries it.
# Cleanup
To just clean up the elasticsearch cluster run `make clean`.
To remove everything, run `make clean-docker`.
# Reference
## Mosscap configs
| CONFIG | DESCRIPTION |
|------------------|----------------------------------------------------------------------|
| msc-actions.yaml | contains all actions, used by all item configs |
| msc-docker.yaml | single item: used to wait until elasticsearch containers are healthy |
| msc-cluster.yaml | single item: create indices etc |
| msc-docs.yaml | generator items: put docs into the index |
| msc-search.yaml | single item: do some searches |
| msc-clean.yaml | single item: clean up the elasticsearch cluster |

7
t/cluster.yaml Normal file
View File

@@ -0,0 +1,7 @@
# esctl config
clusters:
docker/domdoc:
uri: https://elastic:9200
user: elastic
token: ""
default: false

207
t/docker-compose.yaml Normal file
View File

@@ -0,0 +1,207 @@
services:
setup:
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
volumes:
- certs:/usr/share/elasticsearch/config/certs
user: "0"
command: >
bash -c '
if [ x${ES_PASS} == x ]; then
echo "Set the ES_PASS environment variable in the .env file";
exit 1;
fi;
if [ ! -f config/certs/ca.zip ]; then
echo "Creating CA";
bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
unzip config/certs/ca.zip -d config/certs;
fi;
if [ ! -f config/certs/certs.zip ]; then
echo "Creating certs";
echo -ne \
"instances:\n"\
" - name: es01\n"\
" dns:\n"\
" - es01\n"\
" - localhost\n"\
" ip:\n"\
" - 127.0.0.1\n"\
" - name: es02\n"\
" dns:\n"\
" - es02\n"\
" - localhost\n"\
" ip:\n"\
" - 127.0.0.1\n"\
" - name: es03\n"\
" dns:\n"\
" - es03\n"\
" - localhost\n"\
" ip:\n"\
" - 127.0.0.1\n"\
> config/certs/instances.yml;
bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
unzip config/certs/certs.zip -d config/certs;
fi;
echo "Setting file permissions"
chown -R root:root config/certs;
find . -type d -exec chmod 750 \{\} \;;
find . -type f -exec chmod 640 \{\} \;;
echo "Waiting for Elasticsearch availability";
until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
echo "All done!";
'
healthcheck:
test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
interval: 1s
timeout: 5s
retries: 120
es01:
depends_on:
setup:
condition: service_healthy
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
volumes:
- certs:/usr/share/elasticsearch/config/certs
- esdata01:/usr/share/elasticsearch/data
ports:
- ${ES_PORT}:9200
environment:
- node.name=es01
- cluster.name=${CLUSTER_NAME}
- cluster.initial_master_nodes=es01
- discovery.seed_hosts=es02
- ELASTIC_PASSWORD=${ES_PASS}
- bootstrap.memory_lock=true
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/es01/es01.key
- xpack.security.http.ssl.certificate=certs/es01/es01.crt
- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/es01/es01.key
- xpack.security.transport.ssl.certificate=certs/es01/es01.crt
- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.license.self_generated.type=${LICENSE}
- xpack.ml.use_auto_machine_memory_percent=true
- node_roles=data_content
- xpack.searchable.snapshot.shared_cache.size="1gb"
- cluster.routing.allocation.disk.threshold_enabled=false
- cluster.routing.allocation.disk.watermark.low="2gb"
- cluster.routing.allocation.disk.watermark.high="1gb"
- cluster.routing.allocation.disk.watermark.flood_stage="500mb"
mem_limit: ${MEM_LIMIT}
ulimits:
memlock:
soft: -1
hard: -1
healthcheck:
test:
[
"CMD-SHELL",
"curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
]
interval: 10s
timeout: 10s
retries: 120
es02:
depends_on:
- es01
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
volumes:
- certs:/usr/share/elasticsearch/config/certs
- esdata02:/usr/share/elasticsearch/data
environment:
- node.name=es02
- cluster.name=${CLUSTER_NAME}
- cluster.initial_master_nodes=es01,es02
- discovery.seed_hosts=es01
- ELASTIC_PASSWORD=${ES_PASS}
- bootstrap.memory_lock=true
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/es02/es02.key
- xpack.security.http.ssl.certificate=certs/es02/es02.crt
- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/es02/es02.key
- xpack.security.transport.ssl.certificate=certs/es02/es02.crt
- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.license.self_generated.type=${LICENSE}
- xpack.ml.use_auto_machine_memory_percent=true
- cluster.routing.allocation.disk.threshold_enabled=false
- cluster.routing.allocation.disk.watermark.low="2gb"
- cluster.routing.allocation.disk.watermark.high="1gb"
- cluster.routing.allocation.disk.watermark.flood_stage="500mb"
mem_limit: ${MEM_LIMIT}
ulimits:
memlock:
soft: -1
hard: -1
healthcheck:
test:
[
"CMD-SHELL",
"curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
]
interval: 10s
timeout: 10s
retries: 120
es03:
depends_on:
- es01
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
volumes:
- certs:/usr/share/elasticsearch/config/certs
- esdata03:/usr/share/elasticsearch/data
environment:
- node.name=es03
- cluster.name=${CLUSTER_NAME}
- cluster.initial_master_nodes=es01,es03
- discovery.seed_hosts=es01
- ELASTIC_PASSWORD=${ES_PASS}
- bootstrap.memory_lock=true
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/es03/es03.key
- xpack.security.http.ssl.certificate=certs/es03/es03.crt
- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/es03/es03.key
- xpack.security.transport.ssl.certificate=certs/es03/es03.crt
- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.license.self_generated.type=${LICENSE}
- xpack.ml.use_auto_machine_memory_percent=true
- cluster.routing.allocation.disk.threshold_enabled=false
- cluster.routing.allocation.disk.watermark.low="2gb"
- cluster.routing.allocation.disk.watermark.high="1gb"
- cluster.routing.allocation.disk.watermark.flood_stage="500mb"
mem_limit: ${MEM_LIMIT}
ulimits:
memlock:
soft: -1
hard: -1
healthcheck:
test:
[
"CMD-SHELL",
"curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
]
interval: 10s
timeout: 10s
retries: 120
volumes:
certs:
driver: local
esdata01:
driver: local
esdata02:
driver: local
esdata03:
driver: local

96
t/msc-actions.yaml Normal file
View File

@@ -0,0 +1,96 @@
#
# this is a mosscap action file, see: https://codeberg.org/scip/mosscap
vars:
esctl: ../esctl -c cluster.yaml
actions:
wait: |
sleep [[.time]]
docker_wait: |
for i in {1..20}; do
count=$(docker ps | grep healthy | wc -l)
if test $count -eq 3; then
echo "all containers are healthy"
exit
fi
sleep 5s
done
echo "not all containers are healthy"
docker ps
cluster_ls: |
[[.esctl]] cluster ls | grep -E domdoc.*reachable.*yes
cluster_status: |
[[.esctl]] cluster status | grep green
create_ilm: |
[[.esctl]] ilm create --hot-rollover-max-age 7d \
--hot-rollover-max-primary-shard-size 25g \
--warm-min-age 0 \
--delete-min-age 14d [[.policy]]
describe_ilm: |
[[.esctl]] ilm show [[.policy]] -t | grep -E "rollover when storage > 25g"
create_index_template: |
[[.esctl]] index template create [[.index]] -i [[.index]] \
-s number_of_shards:3 \
-s index.sort.field:@timestamp \
-r 1m --ilm-policy [[.policy]] \
user:keyword message:text @timestamp:date
describe_index_template: |
[[.esctl]] index template show [[.index]] | grep -E "index.sort.field.*@timestamp"
create_index: |
[[.esctl]] index create -s 2 -r 2 [[.index]]
describe_index: |
[[.esctl]] index show [[.index]] | grep -E "fields.*@timestamp"
index_ls: |
[[.esctl]] index ls | grep [[.index]]
index_template_rm: |
[[.esctl]] index template rm [[.index]]
index_rm: |
[[.esctl]] index rm [[.index]]
timestamp: |
date --iso-8601=second
date: |
date +%Y-%m-%d
doc_add: |
[[.esctl]] doc add -i [[.index]] '{"user": "[[.user]]", "message":"[[.message]]", "@timestamp":"[[.ts]]"}'
search_any: |
[[.esctl]] search -i [[.index]] -l 1 | jq .source.message
search_date: |
[[.esctl]] search -i [[.index]] -l 1 | jq '.source."@timestamp"' | grep [[.today]]
search_count: |
[[.esctl]] search -i [[.index]] -l 5000 | wc -l | grep 1000
search_content: |
[[.esctl]] search -i [[.index]] [[.content_pattern]] -l 1 | jq .source.message
search_content_and: |
[[.esctl]] search -i [[.index]] [[.content_and_pattern]] -l 1 | jq .source.message
nodes: |
[[.esctl]] node ls | grep es01
describe_node: |
[[.esctl]] node show [[.node]] | grep "Node version"
license: |
[[.esctl]] license show | grep -E "Expires.*never"

14
t/msc-clean.yaml Normal file
View File

@@ -0,0 +1,14 @@
#
# this is a mosscap item file, see: https://codeberg.org/scip/mosscap
items:
- name: docker/domdoc
vars:
index: test
tasks:
- name: delete_index
action: index_rm
- name: delete_template
action: index_template_rm

48
t/msc-cluster.yaml Normal file
View File

@@ -0,0 +1,48 @@
#
# this is a mosscap item file, see: https://codeberg.org/scip/mosscap
statefile: cluster.state
items:
- name: docker/domdoc
vars:
index: test
node: es01
policy: testpolicy
tasks:
- name: ls
action: cluster_ls
- name: license
action: license
- name: status
action: cluster_status
- name: ilm
action: create_ilm
- name: check_ilm
action: describe_ilm
- name: template
action: create_index_template
- name: check_template
action: describe_index_template
- name: index
action: create_index
- name: indexls
action: index_ls
- name: describe
action: describe_index
- name: nodes
action: nodes
- name: check_node
action: describe_node

12
t/msc-docker.yaml Normal file
View File

@@ -0,0 +1,12 @@
#
# this is a mosscap item file, see: https://codeberg.org/scip/mosscap
items:
- name: docker/domdoc
vars:
index: test
time: 5s
tasks:
- name: wait
action: docker_wait

21
t/msc-docs.yaml Normal file
View File

@@ -0,0 +1,21 @@
#
# this is a mosscap generator item file, see: https://codeberg.org/scip/mosscap
vars:
index: test
tasks:
- name: ts
action: timestamp
- name: add
action: doc_add
args:
ts: ts.result
generate:
user: $name
# will become something like:
# "content34 iBAXQ2Va Ko5gMwwu FEH2O99B"
# so we can search for "content3*" and get multiple matches
message: "content$int8 $rand make$int8 $rand"

31
t/msc-search.yaml Normal file
View File

@@ -0,0 +1,31 @@
#
# this is a mosscap item file, see: https://codeberg.org/scip/mosscap
items:
- name: docker/domdoc
vars:
index: test
time: 10s
content_pattern: "content3*"
content_and_pattern: "content3* make"
tasks:
- name: today
action: date
- name: search_any
action: search_any
- name: date
action: search_date
args:
today: today.result
- name: count
action: search_count
- name: content
action: search_content
- name: content_and
action: search_content_and