scip/esctl
1
0
Fork 0
mirror of https://codeberg.org/scip/esctl.git synced 2026-08-24 03:34:18 +02:00
Code Issues Packages Projects Releases Wiki Activity

upd documentation

Browse Source
This commit is contained in:
Thomas von Dein
2026-06-24 10:29:17 +02:00
parent 1b9309bca0
commit 3e36f97e4b
1 changed files with 380 additions and 111 deletions
Download Patch File Download Diff File Expand all files Collapse all files

491
README.md
View File

@@ -56,9 +56,388 @@ Features:
esctl doc add -i foo '{"title":"curry in a hurry", "message":"australian thai"}'
```
## Installation
The tool does not have any dependencies. Just download the binary for
your platform from the releases page and you're good to go.
### Installation using a pre-compiled binary
You can use [stew](https://github.com/marwanhawari/stew) to install esctl:
```default
stew install https://codeberg.org/scip/esctl
```
Or go to the [latest release page](https://codeberg.org/scip/esctl/releases/)
and look for your OS and platform. There are two options to install the binary:
Directly download the binary for your platform,
e.g. `esctl-linux-amd64-0.0.2`, rename it to `esctl` (or whatever
you like more!) and put it into your bin dir (e.g. `$HOME/bin` or as
root to `/usr/local/bin`).
Be sure to verify the signature of the binary file. For this also
download the matching `esctl-linux-amd64-0.0.2.sha256` file and:
```shell
cat esctl-linux-amd64-0.0.2.sha25 && sha256sum esctl-linux-amd64-0.0.2
```
You should see the same SHA256 hash.
You may also download a binary tarball for your platform, e.g.
`esctl-linux-amd64-0.0.2.tar.gz`, unpack and install it. GNU Make is
required for this:
```shell
tar xvfz esctl-linux-amd64-0.0.2.tar.gz
cd esctl-linux-amd64-0.0.2
sudo make install
```
### Installation from source
Check out the repository and execute `go build`, then copy the
compiled binary to your `$PATH`.
Or, if you have GNU Make installed, just execute:
```default
make
sudo make install
```
## Configure
Configure `esctl` with environment variables:
- `ES_URI`: elasticsearch uri
- `ES_USER`: username
- `ES_PASS`: password
Or create a config file such as this:
```yaml
clusters:
foobar:
uri: https://es.foo.bar:9200/
user: elastic
pass: 123456
other:
uri: https://myes.foo:9200/
user: elastic
pass: asdasdasd
```
and specify it with `-c configfile`. You may also put clusters into a
default config file in `~/.config/esctl/config.yaml`. In this case you
can omit `-c ...`.
If you want to work on a specific cluster, you need to make it the
current default one. You can either manually configure it in the
config:
```yaml
clusters:
foobar:
uri: https://es.foo.bar:9200/
user: elastic
pass: 123456
default: true
```
or - if the cluster already exists in the config - issue this command:
```console
esctl cluster switch foobar
```
You may also temporary set a cluster as the current default with the
global `-C` option.
The following rules apply for default cluster selection:
1. If there's just one cluster configured (either via environment
variables or config), this one will be selected.
2. If a cluster have been specified with `-C <name>`, use this.
3. If there is one cluster configured with the `default` flag `true`,
use this one.
Use `esctl cluster ls` to check status and see which one would be
used.
Please be aware, that prior to use a cluster and send API requests to
it, `esctl` checks the connectivity to it first.
## Usage
Command tree:
Once you have configured one or more (and marked one as the default,
see above) elasticsearch clusters, you can start using `esctl`. Since
there are just too many things you can do with it, we'll just outline
a couple of examples here.
To see the status of a cluster:
```console
$ esctl cluster status
DEFAULT STATUS
Cluster Name eck-cluster-dev-s1
ES Status green
ES Version 9.4.2
Is Leader true
Active Shards 447
Active Primary Shards 221
Unassigned Shards 0
Unassigned Primary Shards 0
Pending Tasks 0
Nodes 3
Red Indices 0
Long Running Tasks 2
```
you may also add the `-v` flag to see more:
```console
esctl cluster status -v
DEFAULT STATUS
Cluster Name eck-cluster-dev-s1
ES Status green
ES Version 9.4.2
Is Leader true
Active Shards 447
Active Primary Shards 221
Unassigned Shards 0
Unassigned Primary Shards 0
Pending Tasks 0
Nodes 3
Red Indices 0
Long Running Tasks 2
Indicies 221
Docs 9854777
Total Size 3.4 GB
Total Queries 4210411
Shards Primaries 824642680872
Shards Total 824642680808
Storage 6.9 GB/6.9 GB
JVM Heap Memory 16 GB/39 GB
JVM Threads 458
JVM Version OpenJDK 64-Bit Server VM 26.0.1+8-34
CPUs 18
CPU Usage 0%
Open FDs 1423
```
View cluster settings:
```console
$ esctl cluster settings list
SETTING VALUE
cluster.remote.svct13.mode proxy
cluster.remote.svct13.proxy_address 100.127.161.24:9300
cluster.remote.svct13.proxy_socket_connections 18
cluster.remote.svct13.skip_unavailable false
cluster.routing.allocation.enable all
indices.recovery.max_bytes_per_sec 320mb
```
Settings can be changed, use json path to specify a setting:
```console
$ esctl cluster settings set indices.recovery.max_bytes_per_sec:480mb
```
Searching is pretty easy. First check which indices exist:
```console
$ esctl index ls
NAME SIZE DOCSCOUNT
[..]
foo 249b 0
foo2 31.4kb 14
mock 9.6kb 20
mr-strange 249b 0
silo18 5.2kb 1
```
Like a dataview in kibana you can specify an index pattern for search:
```console
$ esctl search -i foo*
{"id":"1779873299", "score":1.0000, "index":"foo2", "source":{"message":"foo bar"}}{"id":"1779870997", "score":1.0000, "index":"foo2", "source":{"message":"start","time":"27.05.2026T10:55:33+0200","title":"zeitbuchung"}}{"id":"1779870750", "score":1.0000, "index":"foo2", "source":{"message":"pause","time":"27.05.2026T10:33:45+0200","title":"zeitbuchung"}}{"id":"1779870103", "score":1.0000, "index":"foo2", "source":{"message":"punktlich gebucht","time":"27.05.2026T10:20:01+0200","title":"zeitbuchuung"}}{"id":"1779432806", "score":1.0000, "index":"foo2", "source":{"message":"strange compound 2"}}{"id":"1779432750", "score":1.0000, "index":"foo2", "source":{"message":"strange compound 1"}}{"id":"1778600947", "score":1.0000, "index":"foo2", "source":{"id":"d8d8d","user":"scip"}}{"id":"1778600945", "score":1.0000, "index":"foo2", "source":{"id":"d8d8d","user":"scip"}}{"id":"1778600857", "score":1.0000, "index":"foo2", "source":{"id":"d8d8d","user":"scip"}}{"id":"1778601788", "score":1.0000, "index":"foo2", "source":{"id":"new1","message":"live long and prosper"}}{"id":"1778601422", "score":1.0000, "index":"foo2", "source":{"id":"new1","message":"love long and prosper"}}{"id":"1779447242", "score":1.0000, "index":"foo2", "source":{"message":"australian thai","title":"curry in a hurry"}}{"id":"1779447284", "score":1.0000, "index":"foo2", "source":{"message":"german killer food","title":"wuerger burger"}}{"id":"1779364394", "score":1.0000, "index":"foo2", "source":{"id":"1","message":"non-dead human looks curious at TargetOne"}}
```
Oops, that looks gibberish. It's because `esctl` just prints the raw
JSON search results. There are multiple ways to post-process them:
You can use `jq` (here, we're using `-l 1` to limit the output to 1 document):
```console
$ esctl search -i foo* -l 1 | jq
{
"id": "1779364394",
"score": 1.0000,
"index": "foo2",
"source": {
"id": "1",
"message": "non-dead human looks curious at TargetOne"
}
}
```
However, you might as well use internal filtering using jsonpaths:
```console
$ esctl search -i foo* -l 1 -p source.message
non-dead human looks curious at TargetOne
```
To get more help about internal jsonpath filtering, run `esctl
help-jsonpath`.
A full text search looks like this:
```console
$ esctl search -i foo* thai | jq
{
"id": "1779447242",
"score": 2.3842,
"index": "foo2",
"source": {
"message": "australian thai",
"title": "curry in a hurry"
}
}
```
Instead of a costly full text you might as well specify the field to
search within:
```console
$ esctl search -i foo* message=thai | jq
{
"id": "1779447242",
"score": 2.3842,
"index": "foo2",
"source": {
"message": "australian thai",
"title": "curry in a hurry"
}
}
```
Of course you can search for multiple fields (or multiple terms in
full text search). By default search terms are applied using logical
AND, which you can change with the option `--or`.
You can also change the sorting (`--sort-by`) and the time range
(`--timerange`).
To further narrow down search results you can add filters, which must
match literally:
```console
$ esctl search -i foo* -F title=zeitbuchung message=pause | jq
{
"id": "1779870750",
"score": 2.8426,
"index": "foo2",
"source": {
"message": "pause",
"time": "27.05.2026T10:33:45+0200",
"title": "zeitbuchung"
}
}
```
To check which field mappings are available for an index:
```console
$ esctl index show foo2
INDEX-PROPERTY VALUE
name foo2
replicas 1
shards 1
created 2026-05-12 17:47:37
uuid XmRtlh_JSreIzTB7jJNNJA
fields id,message,time,title,user
```
Now, let's create a new index:
```console
$ esctl index create hyperdrive @timestamp:date message:keyword tag:text
esctl index show hyperdrive
INDEX-PROPERTY VALUE
name hyperdrive
replicas 1
shards 1
created 2026-06-24 10:16:23
uuid gYfUj8TiR6aMB_BHmMYr3Q
fields @timestamp,message,tag
```
Once we have an index, we can insert documents into it (you'll need
[gfn](https://codeberg.org/scip/gfn) for this):
```console
$ for name in $(gfn JapaneseNamesDiverse -n 20); do \
esctl doc add -i hyperdrive \
"{\"message\":\"$name\",\"tag\":\"loop\",\"@timestamp\":\"$(date
--iso-8601=second)\"}"; \
sleep 1; \
done
3644770736267480972
1249004675657072369
5499556813583921546
947619563748335783
6804953496467561176
1716813071323879201
463762522269463473
4674152788118344945
7741776772191747414
8531321287239151007
3335734000533108391
7020487792409206103
4926995290680601261
374509677635815298
49895974689085266
7603943644788310
2102931816420108217
696238814931581311
2463257155346426021
596556586001412147
```
Now, let's search them:
```console
$ esctl search -i hyperdrive -l 2 | jq
{
"id": "2463257155346426021",
"score": 0.0000,
"index": "hyperdrive",
"source": {
"@timestamp": "2026-06-24T10:25:19+02:00",
"message": "inenen",
"tag": "loop"
}
}
{
"id": "596556586001412147",
"score": 0.0000,
"index": "hyperdrive",
"source": {
"@timestamp": "2026-06-24T10:25:20+02:00",
"message": "ominuhen",
"tag": "loop"
}
}
```
Please note, that `esctl` is still in its early stages and things are
changing heavily every now and then. New commands are being added
constantly as well.
### Command tree:
```console
api - api access and documentation
@@ -149,116 +528,6 @@ completion - Output shell completion script for bash, zsh, fish, or P
pwsh - Output pwsh completion script
```
Configure `esctl` with environment variables:
- `ES_URI`: elasticsearch uri
- `ES_USER`: username
- `ES_PASS`: password
Or create a config file such as this:
```yaml
clusters:
foobar:
uri: https://es.foo.bar:9200/
user: elastic
pass: 123456
other:
uri: https://myes.foo:9200/
user: elastic
pass: asdasdasd
```
and specify it with `-c configfile`. You may also put clusters into a
default config file in `~/.config/esctl/config.yaml`. In this case you
can omit `-c ...`.
If you want to work on a specific cluster, you need to make it the
current default one. You can either manually configure it in the
config:
```yaml
clusters:
foobar:
uri: https://es.foo.bar:9200/
user: elastic
pass: 123456
default: true
```
or - if the cluster already exists in the config - issue this command:
```console
esctl cluster switch foobar
```
You may also temporary set a cluster as the current default with the
global `-C` option.
The following rules apply for default cluster selection:
1. If there's just one cluster configured (either via environment
variables or config), this one will be selected.
2. If a cluster have been specified with `-C <name>`, use this.
3. If there is one cluster configured with the `default` flag `true`,
use this one.
Use `esctl cluster ls` to check status and see which one would be
used.
Please be aware, that prior to use a cluster and send API requests to
it, `esctl` checks the connectivity to it first.
## Installation
The tool does not have any dependencies. Just download the binary for
your platform from the releases page and you're good to go.
### Installation using a pre-compiled binary
You can use [stew](https://github.com/marwanhawari/stew) to install esctl:
```default
stew install https://codeberg.org/scip/esctl
```
Or go to the [latest release page](https://codeberg.org/scip/esctl/releases/)
and look for your OS and platform. There are two options to install the binary:
Directly download the binary for your platform,
e.g. `esctl-linux-amd64-0.0.2`, rename it to `esctl` (or whatever
you like more!) and put it into your bin dir (e.g. `$HOME/bin` or as
root to `/usr/local/bin`).
Be sure to verify the signature of the binary file. For this also
download the matching `esctl-linux-amd64-0.0.2.sha256` file and:
```shell
cat esctl-linux-amd64-0.0.2.sha25 && sha256sum esctl-linux-amd64-0.0.2
```
You should see the same SHA256 hash.
You may also download a binary tarball for your platform, e.g.
`esctl-linux-amd64-0.0.2.tar.gz`, unpack and install it. GNU Make is
required for this:
```shell
tar xvfz esctl-linux-amd64-0.0.2.tar.gz
cd esctl-linux-amd64-0.0.2
sudo make install
```
### Installation from source
Check out the repository and execute `go build`, then copy the
compiled binary to your `$PATH`.
Or, if you have GNU Make installed, just execute:
```default
make
sudo make install
```
# Development
## To test completion