[![status-badge](https://ci.codeberg.org/api/badges/16999/status.svg)](https://ci.codeberg.org/repos/16999) [![humanmande](https://img.shields.io/badge/human-made-green)](CONTRIBUTING.md) [![License](https://img.shields.io/badge/license-GPL-blue.svg)](https://codeberg.org/scip/esctl/blob/master/LICENSE) [![Go Report Card](https://goreportcard.com/badge/codeberg.org/scip/esctl)](https://goreportcard.com/report/codeberg.org/scip/esctl) [![Latest Release](https://flat.badgen.net/codeberg/release/scip/esctl)](https://codeberg.org/scip/esctl/releases) # esctl Elasticsearch CLI ## Introduction This is a handy cli tool which interfaces to an elasticsearch cluster (or two of them if you're using cross cluster replication). It is a work-in-progress project yet, things might change occasionally. Expect a stable release once we reach major version 1.0.0. Features: - Configuration of cluster credentials using environment vars or config file. Multiple clusters can be configured. `esctl cluster ls` shows which one is reachable. - Shell completion support (bash, zsh and fish). Put this into your rc: `source <(esctl completion bash)`. - Cluster settings can be viewed and modified. - Comprehensive cluster status. - Cluster reroute support. - Search: you can search indices using full text or by fields, select logical condition (OR, AND), use PIT, limit datetime (ES date math can be used), etc. It is however not yet possible to create recursive searches like: `(cond1 AND cond2) OR (cond3 OR cond4)`. - Cross cluster replication (ccr): view, pause, resume, delete replication. You can also manage follower configuration. - Index management: manage aliases, create, modify, delete indices, display field mappings etc. Automatic rollover of aliases supported. - Index template management: create, modify, delete etc - Index alias management: create, modify, delete etc - Node management: only list nodes yet. - Shard management: only list shards yet. - Snapshot management: only list snapshots yet. - ILM management: list, create, delete etc. - Task management: list and cancel tasks - Role management: only list roles yet. There's also a `role diff` subcommand, which is for internal use. It can be used to verify if role defs in a CSV match the deployed roles. - API documentation (`api list` and `api show `) with interactive markdown pager for endpoint documentation. - Repl: this is an interactive REPL (read eval print loop) towards the elasticsearch API. You can run API calls on the current selected cluster w/o the hassle to specify the whole url, credentials etc. It has line editing and history support. If `jq` is installed output JSON will be syntax highlighted. A simple internal pager will be used if output exceeds the terminal height. You can tweak this using the `$ES_JSON_PAGER` environment variable (I'd recommend [fx](https://fx.wtf/)). - Doc support. You can put, delete and show docs for an index. Very handy if you want to play with it. Just create a new index: `esctl index create foo` and then insert docs into it for search experiments: ```console esctl doc add -i foo '{"title":"curry in a hurry", "message":"australian thai"}' ``` ## Installation The tool does not have any dependencies. Just download the binary for your platform from the releases page and you're good to go. ### Installation using a pre-compiled binary You can use [stew](https://github.com/marwanhawari/stew) to install esctl: ```default stew install https://codeberg.org/scip/esctl ``` Or go to the [latest release page](https://codeberg.org/scip/esctl/releases/) and look for your OS and platform. There are two options to install the binary: Directly download the binary for your platform, e.g. `esctl-linux-amd64-0.0.2`, rename it to `esctl` (or whatever you like more!) and put it into your bin dir (e.g. `$HOME/bin` or as root to `/usr/local/bin`). Be sure to verify the signature of the binary file. For this also download the matching `esctl-linux-amd64-0.0.2.sha256` file and: ```shell cat esctl-linux-amd64-0.0.2.sha25 && sha256sum esctl-linux-amd64-0.0.2 ``` You should see the same SHA256 hash. You may also download a binary tarball for your platform, e.g. `esctl-linux-amd64-0.0.2.tar.gz`, unpack and install it. GNU Make is required for this: ```shell tar xvfz esctl-linux-amd64-0.0.2.tar.gz cd esctl-linux-amd64-0.0.2 sudo make install ``` ### Installation from source Check out the repository and execute `go build`, then copy the compiled binary to your `$PATH`. Or, if you have GNU Make installed, just execute: ```default make sudo make install ``` ## Configure Configure `esctl` with environment variables: - `ES_URI`: elasticsearch uri - `ES_USER`: username - `ES_PASS`: password - `ES_TOKEN`: API token, instead of user+password Or create a config file such as this: ```yaml clusters: foobar: uri: https://es.foo.bar:9200/ user: elastic pass: ****** other: uri: https://myes.foo:9200/ token: ****** ``` and specify it with `-c configfile`. You may also put clusters into a default config file in `~/.config/esctl/config.yaml`. In this case you can omit `-c ...`. If you want to work on a specific cluster, you need to make it the current default one. You can either manually configure it in the config: ```yaml clusters: foobar: uri: https://es.foo.bar:9200/ user: elastic pass: 123456 default: true ``` or - if the cluster already exists in the config - issue this command: ```console esctl cluster switch foobar ``` You may also temporary set a cluster as the current default with the global `-C` option. The following rules apply for default cluster selection: 1. If there's just one cluster configured (either via environment variables or config), this one will be selected. 2. If a cluster have been specified with `-C `, use this. 3. If there is one cluster configured with the `default` flag `true`, use this one. Use `esctl cluster ls` to check status and see which one would be used. Please be aware, that prior to use a cluster and send API requests to it, `esctl` checks the connectivity to it first. ## Usage Once you have configured one or more (and marked one as the default, see above) elasticsearch clusters, you can start using `esctl`. Since there are just too many things you can do with it, we'll just outline a couple of examples here. To see the status of a cluster: ```console $ esctl cluster status DEFAULT STATUS Cluster Name eck-cluster-dev-s1 ES Status green ES Version 9.4.2 Is Leader true Active Shards 447 Active Primary Shards 221 Unassigned Shards 0 Unassigned Primary Shards 0 Pending Tasks 0 Nodes 3 Red Indices 0 Long Running Tasks 2 ``` you may also add the `-v` flag to see more: ```console $ esctl cluster status -v DEFAULT STATUS Cluster Name eck-cluster-dev-s1 ES Status green ES Version 9.4.2 Is Leader true Active Shards 447 Active Primary Shards 221 Unassigned Shards 0 Unassigned Primary Shards 0 Pending Tasks 0 Nodes 3 Red Indices 0 Long Running Tasks 2 Indicies 221 Docs 9854777 Total Size 3.4 GB Total Queries 4210411 Shards Primaries 824642680872 Shards Total 824642680808 Storage 6.9 GB/6.9 GB JVM Heap Memory 16 GB/39 GB JVM Threads 458 JVM Version OpenJDK 64-Bit Server VM 26.0.1+8-34 CPUs 18 CPU Usage 0% Open FDs 1423 ``` View cluster settings: ```console $ esctl cluster settings list SETTING VALUE cluster.remote.svct13.mode proxy cluster.remote.svct13.proxy_address 100.127.161.24:9300 cluster.remote.svct13.proxy_socket_connections 18 cluster.remote.svct13.skip_unavailable false cluster.routing.allocation.enable all indices.recovery.max_bytes_per_sec 320mb ``` Settings can be changed, use json path to specify a setting: ```console $ esctl cluster settings set indices.recovery.max_bytes_per_sec:480mb ``` Searching is pretty easy. First check which indices exist: ```console $ esctl index ls NAME SIZE DOCSCOUNT [..] foo 249b 0 foo2 31.4kb 14 mock 9.6kb 20 mr-strange 249b 0 silo18 5.2kb 1 ``` Like a dataview in kibana you can specify an index pattern for search: ```console $ esctl search -i foo* {"id":"1779873299", "score":1.0000, "index":"foo2", "source":{"message":"foo bar"}}{"id":"1779870997", "score":1.0000, "index":"foo2", "source":{"message":"start","time":"27.05.2026T10:55:33+0200","title":"zeitbuchung"}}{"id":"1779870750", "score":1.0000, "index":"foo2", "source":{"message":"pause","time":"27.05.2026T10:33:45+0200","title":"zeitbuchung"}}{"id":"1779870103", "score":1.0000, "index":"foo2", "source":{"message":"punktlich gebucht","time":"27.05.2026T10:20:01+0200","title":"zeitbuchuung"}}{"id":"1779432806", "score":1.0000, "index":"foo2", "source":{"message":"strange compound 2"}}{"id":"1779432750", "score":1.0000, "index":"foo2", "source":{"message":"strange compound 1"}}{"id":"1778600947", "score":1.0000, "index":"foo2", "source":{"id":"d8d8d","user":"scip"}}{"id":"1778600945", "score":1.0000, "index":"foo2", "source":{"id":"d8d8d","user":"scip"}}{"id":"1778600857", "score":1.0000, "index":"foo2", "source":{"id":"d8d8d","user":"scip"}}{"id":"1778601788", "score":1.0000, "index":"foo2", "source":{"id":"new1","message":"live long and prosper"}}{"id":"1778601422", "score":1.0000, "index":"foo2", "source":{"id":"new1","message":"love long and prosper"}}{"id":"1779447242", "score":1.0000, "index":"foo2", "source":{"message":"australian thai","title":"curry in a hurry"}}{"id":"1779447284", "score":1.0000, "index":"foo2", "source":{"message":"german killer food","title":"wuerger burger"}}{"id":"1779364394", "score":1.0000, "index":"foo2", "source":{"id":"1","message":"non-dead human looks curious at TargetOne"}} ``` Oops, that looks gibberish. It's because `esctl` just prints the raw JSON search results. There are multiple ways to post-process them: You can use `jq` (here, we're using `-l 1` to limit the output to 1 document): ```console $ esctl search -i foo* -l 1 | jq { "id": "1779364394", "score": 1.0000, "index": "foo2", "source": { "id": "1", "message": "non-dead human looks curious at TargetOne" } } ``` However, you might as well use internal filtering using jsonpaths: ```console $ esctl search -i foo* -l 1 -p source.message non-dead human looks curious at TargetOne ``` To get more help about internal jsonpath filtering, run `esctl help-jsonpath`. A full text search looks like this: ```console $ esctl search -i foo* thai | jq { "id": "1779447242", "score": 2.3842, "index": "foo2", "source": { "message": "australian thai", "title": "curry in a hurry" } } ``` Instead of a costly full text you might as well specify the field to search within: ```console $ esctl search -i foo* message=thai | jq { "id": "1779447242", "score": 2.3842, "index": "foo2", "source": { "message": "australian thai", "title": "curry in a hurry" } } ``` Of course you can search for multiple fields (or multiple terms in full text search). By default search terms are applied using logical AND, which you can change with the option `--or`. You can also change the sorting (`--sort-by`) and the time range (`--timerange`). To further narrow down search results you can add filters, which must match literally: ```console $ esctl search -i foo* -F title=zeitbuchung message=pause | jq { "id": "1779870750", "score": 2.8426, "index": "foo2", "source": { "message": "pause", "time": "27.05.2026T10:33:45+0200", "title": "zeitbuchung" } } ``` To check which field mappings are available for an index: ```console $ esctl index show foo2 INDEX-PROPERTY VALUE name foo2 replicas 1 shards 1 created 2026-05-12 17:47:37 uuid XmRtlh_JSreIzTB7jJNNJA fields id,message,time,title,user ``` Now, let's create a new index: ```console $ esctl index create hyperdrive @timestamp:date message:keyword tag:text esctl index show hyperdrive INDEX-PROPERTY VALUE name hyperdrive replicas 1 shards 1 created 2026-06-24 10:16:23 uuid gYfUj8TiR6aMB_BHmMYr3Q fields @timestamp,message,tag ``` Once we have an index, we can insert documents into it (you'll need [gfn](https://codeberg.org/scip/gfn) for this): ```console $ for name in $(gfn JapaneseNamesDiverse -n 20); do \ esctl doc add -i hyperdrive \ "{\"message\":\"$name\",\"tag\":\"loop\",\"@timestamp\":\"$(date --iso-8601=second)\"}"; \ sleep 1; \ done 3644770736267480972 1249004675657072369 5499556813583921546 947619563748335783 6804953496467561176 1716813071323879201 463762522269463473 4674152788118344945 7741776772191747414 8531321287239151007 3335734000533108391 7020487792409206103 4926995290680601261 374509677635815298 49895974689085266 7603943644788310 2102931816420108217 696238814931581311 2463257155346426021 596556586001412147 ``` Now, let's search them: ```console $ esctl search -i hyperdrive -l 2 | jq { "id": "2463257155346426021", "score": 0.0000, "index": "hyperdrive", "source": { "@timestamp": "2026-06-24T10:25:19+02:00", "message": "inenen", "tag": "loop" } } { "id": "596556586001412147", "score": 0.0000, "index": "hyperdrive", "source": { "@timestamp": "2026-06-24T10:25:20+02:00", "message": "ominuhen", "tag": "loop" } } ``` There are also some uniq features which are not directly available via the ES API or GUI. Here's one example: say you have a number of indices with ILM policies each. At some day there were a surge in incoming data in some of them and you want to know, when the excess data will be rolled over to cold storage: ```console $ esctl ilm forecast sh -f warm -w 3d 1.1 TB bytes of data in warm phase will be rolled within 72h0m0s to the next phase ``` You may also look at a detailed ilm forecast list: ```console $ esctl ilm forecast ls -f warm -w 3d INDEX CURRENT-SIZE CURRENT-AGE VIRTUAL-AGE MIN-AGE MIN-SIZE CURRENT-PHASE NEXT-PHASE foobar-n1-p01-elastic-000126 12 GB 2d:8h:12m 3d:5h:16m 7d 25 GB hot warm foobar-n1-p01-misc-000070 16 GB 5d:16h:31m 4d:10h:31m 7d 25 GB hot warm foobar-n1-q01-elastic-000122 23 GB 5d:0h:11m 6d:10h:33m 7d 25 GB hot warm foobar-n1-q01-misc-000072 18 GB 6d:15h:21m 4d:21h:36m 7d 25 GB hot warm delaware-f1-p01-kafka-000023 1.3 MB 15h:12m 0s 7d 25 GB hot warm delaware-f1-p01-misc-000024 17 GB 6d:16h:1m 4d:16h:33m 7d 25 GB hot warm delaware-f1-q01-kafka-000023 1.5 MB 15h:12m 0s 7d 25 GB hot warm delaware-f1-q01-misc-000024 16 GB 6d:15h:31m 4d:13h:52m 7d 25 GB hot warm delaware-n1-p01-elastic-000417 16 GB 12h:42m 4d:10h:31m 7d 25 GB hot warm ``` So you can see, which index will be rolled over when. Note the **VIRTUAL-AGE** field however: it is calculated from the current storage usage of the index in relation to rollover max shard size. So you can see, when an index will be rolled over either because it aged out or because its storage exceeded the limit. --- Please note, that `esctl` is still in its early stages and things are changing heavily every now and then. New commands are being added constantly as well. ### Command tree: ```console api - api access and documentation list - list index of API calls show - show an API doc repl - interactive API repl ccr - manage cross cluster replication status - cross cluster replication status (yaml config with 2 clusters required) pause - pause shard allocation resume - resume shard allocation follower - manage ccr follower indices show - show ccr follower index details add - add ccr follower index delete - delete ccr follower index unfollow - unfollow ccr follower index pause - pause ccr index to follow resume - resume ccr index to follow renew - renew ccr follower index info - show ccr remote info cluster - manage cluster[s] status - show cluster status switch - set current elasticsearch cluster list - list configured clusters settings - cluster settings management list - show cluster settings set - set|update cluster settings reroute - manually change the allocation of individual shards in the cluster. move - move shard to another node allocate-replica - allocate-replica replica to another node cancel - cancel a reroute operation allocate-empty-primary - allocate an empty primary shard to a node allocate-stale-primary - allocate a stale primary shard to a node datastream - manage data streams list - list indicies show - show details about an data stream create - create a new data stream delete - delete a data stream rollover - roll over a data stream doc - manage documents add - add JSON document index show - show a JSON document delete - delete JSON document[s] from index[es] ilm - manage index lifecycle retry - retry applying an ILM profile to an index status - get the current index lifecycle management status list - list index lifecycle policies show - show details about an index lifecycle policy create - create a new lifecycle policy update - update an lifecycle policy forecast - calculate index phase movements list - list index rollover config show - show rollover forecast over all indices explain - explain ilm condition of an index index - manage indicies list - list indicies show - show details about an index create - create a new index update - update an index delete - delete an index close - close an index fields - show info about field capabilities ilm - show ilm status alias - manage index aliases create - create an index alias list - list index aliases delete - delete an index alias rollover - roll over an index alias template - manage index templates list - list index templates show - show details about an index template create - create a new index template update - update a new index template delete - delete an index template license - manage cluster license show - show details about the cluster license node - manage nodes list - list nodes show - show details about a node clients - show node http clients role - manage roles list - list roles show - show details about a role diff - show differences between roles and CSV baseline search - search within an index shard - manage shards list - list shards show - show details about a shard allocation - explain shard allocation snapshot - manage snapshots list - list snapshots show - show details about a snapshot task - manage tasks list - list tasks cancel - cancel running task version - show esctl version information debug - developer only help-jsonpath - show jsonpath help help-usage - show overview of all available commands ``` # Development ## To test completion Add the flag `--generate-shell-completion` to any command, e.g.: ```console ./esctl role show --generate-shell-completion machine_learning_admin rollup_admin editor reporting_user snapshot_user fcn_admin machine_learning_user kibana_system beats_admin kibana_user fcns_space transport_client transform_user [..] ``` # Report bugs [Please open an issue](https://codeberg.org/scip/esctl/issues). Thanks! # License This work is licensed under the terms of the General Public Licens version 3. # Author Copyleft (c) 2026 Thomas von Dein