scip/anydb
1
0
Fork 0
mirror of https://codeberg.org/scip/anydb.git synced 2025-12-17 12:31:02 +01:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: scip:v0.0.5
Branches Tags
scip:main scip:github scip:codeberg scip:ui
scip:v0.2.6-codeberg1 scip:v0.2.0 scip:v0.1.4 scip:v0.1.3 scip:v0.1.1 scip:v0.1.0 scip:v0.0.7 scip:v0.0.6 scip:v0.0.5 scip:v0.0.4 scip:v0.0.3 scip:v0.0.2 scip:v0.0.1
...
compare: scip:v0.0.7
Branches Tags
scip:main scip:github scip:codeberg scip:ui
scip:v0.2.6-codeberg1 scip:v0.2.0 scip:v0.1.4 scip:v0.1.3 scip:v0.1.1 scip:v0.1.0 scip:v0.0.7 scip:v0.0.6 scip:v0.0.5 scip:v0.0.4 scip:v0.0.3 scip:v0.0.2 scip:v0.0.1

8 Commits
v0.0.5 ... v0.0.7

Author SHA1 Message Date
Thomas von Dein
a777c9cab4 added -i option to list to search case insensitive 2024-12-23 19:49:42 +01:00
Thomas von Dein
2f652dc57d todo 2024-12-23 19:31:50 +01:00
T.v.Dein
f5f3760e67 Merge pull request #9 from TLINDEN/edit 
Updated manual
 2024-12-23 16:23:12 +01:00
T.v.Dein
2253acf19d Merge branch 'main' into edit 2024-12-23 16:21:26 +01:00
Thomas von Dein
b7e3267695 updated manual 2024-12-23 16:19:27 +01:00
Thomas von Dein
6adeb618b7 add dbtool note 2024-12-22 21:12:16 +01:00
Thomas von Dein
b262c73746 added edit command, fixed set -o - 2024-12-22 17:50:40 +01:00
Thomas von Dein
66d6bd2a41 added edit command, fixed set -o - 2024-12-22 17:46:44 +01:00
12 changed files with 1985 additions and 194 deletions
Expand all files Collapse all files

10
README.md
View File

@@ -27,6 +27,8 @@ reasons:
- templates for custom output for maximum flexibility
- includes a tiny web server, which serves a restful API
And I wrote a very similar [tool](https://www.daemon.de/projects/dbtool/) 24 years ago and wanted to do it again wit go.
**anydb** can do all the things you can do with skate:
```shell
@@ -145,6 +147,14 @@ curl localhost:8787/anydb/v1/foo
# list keys
curl localhost:8787/anydb/v1/
# as you might correctly suspect you can store multi-line values or
# the content of text files. but what to do if you want to change it?
# here's one way:
anydb get contract24 > file.txt && vi file.txt && anydb set contract24 -r file.txt
# annoying. better do this
anydb edit contract24
# sometimes you need to know some details about the current database
# add -d for more details
anydb info

3
TODO.md
View File

@@ -1,3 +1,4 @@
- repl
- mime-type => exec app + value
- [edit command](https://github.com/TLINDEN/rpnc/blob/master/command.go#L249)
- add waitgroup to db.go funcs
- RestList does not support any params?

564
anydb.1
View File

@@ -133,7 +133,7 @@
.\" ========================================================================
.\"
.IX Title "ANYDB 1"
.TH ANYDB 1 "2024-12-22" "1" "User Commands"
.TH ANYDB 1 "2024-12-23" "1" "User Commands"
.\" For nroff, turn off justification. Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
@@ -151,14 +151,20 @@ anydb \- a personal key value store
\& Available Commands:
\& completion Generate the autocompletion script for the specified shell
\& del Delete key
\& edit Edit a key
\& export Export database to json
\& get Retrieve value for a key
\& help Help about any command
\& import Import database dump
\& info info
\& list List database contents
\& man show manual page
\& serve run REST API listener
\& set Insert key/value pair
\&
\& Flags:
\& \-b, \-\-bucket string use other bucket (default: data) (default "data")
\& \-c, \-\-config string toml config file
\& \-f, \-\-dbfile string DB file to use (default "/home/scip/.config/anydb/default.db")
\& \-d, \-\-debug Enable debugging
\& \-h, \-\-help help for anydb
@@ -168,9 +174,559 @@ anydb \- a personal key value store
.Ve
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
Anydb is a simple to use commandline tool to store anything you'd
like, even binary files etc. It uses a key/value store (bbolt) in your
home directory.
Anydb is a commandline personal key value store, it is simple to use
and can be used to store anything you'd like, even binary files
etc. It uses a key/value store (bbolt) in your home directory.
.PP
The tool provides a number of subcommands to use it, there are global
options and each subcommand has its own set of options.
.SH "GLOBAL OPTIONS"
.IX Header "GLOBAL OPTIONS"
.ie n .IP """\-f, \-\-dbfile filename""" 4
.el .IP "\f(CW\-f, \-\-dbfile filename\fR" 4
.IX Item "-f, --dbfile filename"
The default location of your databas is
\&\f(CW\*(C`$HOME/.config/anydb/default.db\*(C'\fR. You can change this with the \f(CW\*(C`\-f\*(C'\fR
option.
.ie n .IP """\-b, \-\-bucket name""" 4
.el .IP "\f(CW\-b, \-\-bucket name\fR" 4
.IX Item "-b, --bucket name"
Data in a bbolt key-value-store are managed in so called
buckets. These are kind of namespaces, where each key must be
unique. However, a database may contain more than one bucket.
.Sp
By default anydb uses a bucket named \*(L"data\*(R", but you can change this
using the option \f(CW\*(C`\-b\*(C'\fR.
.Sp
Buckets can be configured to always encrypt values, see \s-1ENCRYTPTION\s0.
.ie n .IP """\-c, \-\-config filename""" 4
.el .IP "\f(CW\-c, \-\-config filename\fR" 4
.IX Item "-c, --config filename"
Under normal circumstances you don't need a configuration file. But if
you want, you can provide one using the option \f(CW\*(C`\-c\*(C'\fR.
.Sp
Anydb looks for a couple of default locations for a config file. You
only need this option if you want to supply a configuration on a
non-standard location. See \s-1CONFIGURATION\s0 for more details.
.ie n .IP """\-d, \-\-debug""" 4
.el .IP "\f(CW\-d, \-\-debug\fR" 4
.IX Item "-d, --debug"
Enable debug output.
.ie n .IP """\-h, \-\-help""" 4
.el .IP "\f(CW\-h, \-\-help\fR" 4
.IX Item "-h, --help"
Show the usage of anydb.
.ie n .IP """\-v, \-\-version""" 4
.el .IP "\f(CW\-v, \-\-version\fR" 4
.IX Item "-v, --version"
Show the program version.
.PP
All of these options can be used with subcommands as well.
.SH "SUBCOMMANDS"
.IX Header "SUBCOMMANDS"
.SS "completion"
.IX Subsection "completion"
The \fBcompletion\fR command can be used to setup completion for
anydb. Just put something like this into your shell's configuration
file:
.PP
.Vb 1
\& source <(anydb completion bash)
.Ve
.PP
If you use another shell, specify it instead of bash, of course.
.SS "set"
.IX Subsection "set"
The \fBset\fR command is being used to insert or update a key-value pair.
.PP
Usage:
.PP
.Vb 2
\& Usage:
\& anydb set <key> [<value> | \-r <file>] [\-t <tag>] [flags]
\&
\& Aliases:
\& set, add, s, +
\&
\& Flags:
\& \-e, \-\-encrypt encrypt value
\& \-r, \-\-file string Filename or \- for STDIN
\& \-h, \-\-help help for set
\& \-t, \-\-tags tag,tag,... tags, multiple allowed
.Ve
.PP
The standard way to insert a new entry is really simple:
.PP
.Vb 1
\& anydb set key value
.Ve
.PP
If you don't specify a value, anydb expects you to feed it some data
via \s-1STDIN.\s0 For example:
.PP
.Vb 1
\& anydb set key < file
.Ve
.PP
You might as well specify a file directly using the \f(CW\*(C`\-f\*(C'\fR option:
.PP
.Vb 1
\& anydb set key \-f file
.Ve
.PP
Values can be encrypted using \fBChaCha20Poly1305\fR when you specify the
\&\f(CW\*(C`\-e\*(C'\fR option. Anydb will ask you interactively for a passphrase. You
might as well provide the passphrase using the environment variable
\&\f(CW\*(C`ANYDB_PASSWORD\*(C'\fR. To encrypt the value, a cryptographically secure
key will be derived from the passphrase using the ArgonID2
algorithm. Each value can be encrypted with another passphrase. So,
the database itself is not encrypted, just the values.
.PP
You can supply tags by using the option \f(CW\*(C`\-t\*(C'\fR. Multiple tags can be
provided either by separating them with a comma or by using multiple
\&\f(CW\*(C`\-t\*(C'\fR parameters:
.PP
.Vb 2
\& anydb set key value \-t tag1,tag2
\& anydb set key value \-t tag1 \-t tag2
.Ve
.PP
You can later filter entries by tag or by a combination of tags.
.PP
To edit or modify an entry, just use the \fBset\fR command with the same
key, the value in the database will be overwritten with the new
value. An alternative option is the \fBedit\fR command, see below.
.SS "get"
.IX Subsection "get"
To retrieve the value of a key, use the \fBget\fR subcommand.
.PP
Usage:
.PP
.Vb 2
\& Usage:
\& anydb get <key> [\-o <file>] [\-m <mode>] [\-n \-N] [\-T <tpl>] [flags]
\&
\& Aliases:
\& get, show, g, .
\&
\& Flags:
\& \-h, \-\-help help for get
\& \-m, \-\-mode string output format (simple|wide|json|template) (default \*(Aqsimple\*(Aq)
\& \-n, \-\-no\-headers omit headers in tables
\& \-N, \-\-no\-human do not translate to human readable values
\& \-o, \-\-output string output value to file (ignores \-m)
\& \-T, \-\-template string go template for \*(Aq\-m template\*(Aq
.Ve
.PP
In its simplest form you just call the \fBget\fR subcommand with the key
you want to have the value for. The value is being printed to \s-1STDOUT\s0
by default:
.PP
.Vb 1
\& anydb get key
.Ve
.PP
If the value is binary content, it will not just being printed. In
those cases you need to either redirect output into a file or use the
option \f(CW\*(C`\-o\*(C'\fR to write to a file:
.PP
.Vb 2
\& anydb get key > file
\& anydb get key \-o file
.Ve
.PP
If the value is encrypted, you will be asked for the passphrase to
decrypt it. If the environment variable \f(CW\*(C`ANYDB_PASSWORD\*(C'\fR is set, its
value will be used instead.
.PP
There are different output modes you can choos from: simple, wide and
json. The \*(L"simple\*(R" mode is the default one, it just prints the value
as is. The \*(L"wide\*(R" mode prints a tabular output similar to the \fBlist\fR
subcommand, see there for more details. The options \f(CW\*(C`\-n\*(C'\fR and \f(CW\*(C`\-N\*(C'\fR
have the same meaning as in the list command. The \*(L"json\*(R" mode prints
the raw \s-1JSON\s0 representation of the whole database entry. Decryption
will only take place in \*(L"simple\*(R" and \*(L"json\*(R" mode. The \*(L"template\*(R" mode
provides the most flexibily, it is detailed in the section
\&\s-1TEMPLATES\s0.
.SS "list"
.IX Subsection "list"
The \fBlist\fR subcommand displays a list of all database entries.
.PP
Usage:
.PP
.Vb 2
\& Usage:
\& anydb list [<filter\-regex>] [\-t <tag>] [\-m <mode>] [\-n \-N] [\-T <tpl>] [flags]
\&
\& Aliases:
\& list, /, ls
\&
\& Flags:
\& \-h, \-\-help help for list
\& \-m, \-\-mode string output format (table|wide|json|template), wide is a verbose table. (default \*(Aqtable\*(Aq)
\& \-n, \-\-no\-headers omit headers in tables
\& \-N, \-\-no\-human do not translate to human readable values
\& \-t, \-\-tags stringArray tags, multiple allowed
\& \-T, \-\-template string go template for \*(Aq\-m template\*(Aq
\& \-l, \-\-wide\-output output mode: wide
.Ve
.PP
In its simplest form \- without any options \- , the \fBlist\fR command
just prints all keys with their values to \s-1STDOUT.\s0 Values are being
truncated to maximum of 60 characters, that is, multiline values are
not completely shown in order to keep the tabular view readable.
.PP
To get more informations about each entry, use the \f(CW\*(C`\-o wide\*(C'\fR or \f(CW\*(C`\-l\*(C'\fR
option. In addition to the key and value also the size, update
timestamp and tags will be printed. Time and size values are converted
into a human readable form, you can suppress this behavior with the
\&\f(CW\*(C`\-N\*(C'\fR option. You may omit the headers using the option \f(CW\*(C`\-n\*(C'\fR
.PP
Sometimes you might want to filter the list of entries. Either because
your database grew too large or because you're searching for
something. In that case you have two options: You may supply one or
more tags or provide a filter regexp. To filter by tag, do:
.PP
.Vb 3
\& anydb list \-t tag1
\& anydb list \-t tag1,tag2
\& anydb list \-t tag1 \-t tag2
.Ve
.PP
To filter using a regular expression, do:
.PP
.Vb 1
\& anydb list "foo.*bar"
.Ve
.PP
Regular expressions follow the golang \fBre2\fR syntax. For more details
about the syntax, refer to
<https://github.com/google/re2/wiki/Syntax>. Please note, that this
regexp dialect is not \s-1PCRE\s0 compatible, but supports most of its
features.
.PP
You can \- as with the \fBget\fR command \- use other output modes. The
default mode is \*(L"table\*(R". The \*(L"wide\*(R" mode is, as already mentioned, a
more detailed table. Also supported is \*(L"json\*(R" mode and \*(L"template\*(R"
mode. For details about using templates see \s-1TEMPLATES\s0.
.SS "del"
.IX Subsection "del"
Use the \fBdel\fR command to delete database entries.
.PP
Usage:
.PP
.Vb 2
\& Usage:
\& anydb del <key> [flags]
\&
\& Aliases:
\& del, d, rm
\&
\& Flags:
\& \-h, \-\-help help for del
.Ve
.PP
The subcommand \fBdel\fR does not provide any further options, it just
deletes the entry referred to by the given key. No questions are being
asked.
.SS "edit"
.IX Subsection "edit"
The \fBedit\fR command makes it easier to modify larger entries.
.PP
Usage:
.PP
.Vb 2
\& Usage:
\& anydb edit <key> [flags]
\&
\& Aliases:
\& edit, modify, mod, ed, vi
\&
\& Flags:
\& \-h, \-\-help help for edit
.Ve
.PP
The subcommand \fBedit\fR does not provide any further options. It
works like this:
.IP "1. Write the value info a temporary file." 4
.IX Item "1. Write the value info a temporary file."
.PD 0
.IP "2. Execute the editor (which one, see below!) with that file." 4
.IX Item "2. Execute the editor (which one, see below!) with that file."
.IP "3. Now you can edit the file and save+close it when done." 4
.IX Item "3. Now you can edit the file and save+close it when done."
.IP "4. Anydb picks up the file and if the content has changed, puts its value into the \s-1DB.\s0" 4
.IX Item "4. Anydb picks up the file and if the content has changed, puts its value into the DB."
.PD
.PP
By default anydb executes the \f(CW\*(C`vi\*(C'\fR command. You can modify this
behavior by setting the environment variable \f(CW\*(C`EDITOR\*(C'\fR appropriately.
.PP
Please note, that this does not work with binary content!
.SS "export"
.IX Subsection "export"
Since the bbold database file is not portable across platforms (it is
bound to the endianess of the \s-1CPU\s0 it was being created on), you might
want to create a backup file of your database. You can do this with
the \fBexport\fR subcommand.
.PP
Usage:
.PP
.Vb 2
\& Usage:
\& anydb export [\-o <json filename>] [flags]
\&
\& Aliases:
\& export, dump, backup
\&
\& Flags:
\& \-h, \-\-help help for export
\& \-o, \-\-output string output to file
.Ve
.PP
The database dump is a \s-1JSON\s0 representation of the whole database and
will be printed to \s-1STDOUT\s0 by default. Redirect it to a file or use the
\&\f(CW\*(C`\-o\*(C'\fR option:
.PP
.Vb 2
\& anydb export > dump.json
\& anydb export \-o dump.json
.Ve
.PP
Please note, that encrypted values will not be decrypted. This might
change in a future version of anydb.
.SS "import"
.IX Subsection "import"
The \fBimport\fR subcommand can be used to restore a database from a \s-1JSON\s0
dump.
.PP
Usage:
.PP
.Vb 2
\& Usage:
\& anydb import [<json file>] [flags]
\&
\& Aliases:
\& import, restore
\&
\& Flags:
\& \-r, \-\-file string Filename or \- for STDIN
\& \-h, \-\-help help for import
\& \-t, \-\-tags stringArray tags, multiple allowed
.Ve
.PP
By default the \f(CW\*(C`import\*(C'\fR subcommand reads the \s-1JSON\s0 contents from
\&\s-1STDIN.\s0 You might pipe the dump into it or use the option \f(CW\*(C`\-r\*(C'\fR:
.PP
.Vb 3
\& anydb import < dump.json
\& anydb import \-r dump.json
\& cat dump.json | anydb import
.Ve
.PP
If there is already a database, it will be saved by appending a
timestamp and a new database with the contents of the dump will be
created.
.SS "serve"
.IX Subsection "serve"
Anydb provides a RESTful \s-1API,\s0 which you can use to manage the database
from somewhere else. The \s-1API\s0 does not provide any authentication or
any other security measures, so better only use it on localhost.
.PP
Usage:
.PP
.Vb 2
\& Usage:
\& anydb serve [\-l host:port] [flags]
\&
\& Flags:
\& \-h, \-\-help help for serve
\& \-l, \-\-listen string host:port (default "localhost:8787")
.Ve
.PP
To start the listener, just execute the \fBserve\fR subcommand. You can
tweak the ip address and tcp port using the \f(CW\*(C`\-l\*(C'\fR option. The listener
will not fork and run in the foreground. Logs are being printed to
\&\s-1STDOUT\s0 as long as the listener runs.
.PP
For more details about the \s-1API,\s0 please see the \*(L"\s-1REST API\*(R"\s0 section.
.SS "info"
.IX Subsection "info"
The \fBinfo\fR subcommand shows you some information about your current
database.
.PP
Usage:
.PP
.Vb 2
\& Usage:
\& anydb info [flags]
\&
\& Flags:
\& \-h, \-\-help help for info
\& \-N, \-\-no\-human do not translate to human readable values
.Ve
.PP
Data being shown are: filename and size, number of keys per bucket. If
you supply the \f(CW\*(C`\-d\*(C'\fR option (debug), some bbolt internals are being
displayed as well.
.SS "man"
.IX Subsection "man"
The \fBman\fR subcommand shows an unformatted text variant of the manual
page (which are currently reading).
.PP
Usage:
.PP
.Vb 2
\& Usage:
\& anydb man [flags]
\&
\& Flags:
\& \-h, \-\-help help for man
.Ve
.PP
The manual is being piped into the \f(CW\*(C`more\*(C'\fR command, which is being
expected to exist according to the \s-1POSIX\s0 standard on all supported
unix platforms. It might not work on Windows.
.SH "TEMPLATES"
.IX Header "TEMPLATES"
The \fBget\fR and \fBlist\fR commands support a template feature, which is
very handy to create you own kind of formatting. The template syntax
being used is the \s-1GO\s0 template language, refer to
<https://pkg.go.dev/text/template> for details.
.PP
Each template operates on one or more entries, no loop construct is
required, the template provided applies to every matching entry
separatley.
.PP
The following template variables can be used:
.IP "\fBKey\fR \- string =item \fBValue\fR \- string =item \fBBin\fR \- []byte =item \fBCreated\fR \- time.Time =item \fBTags\fR \- []string =item \fBEncrypted\fR bool" 4
.IX Item "Key - string =item Value - string =item Bin - []byte =item Created - time.Time =item Tags - []string =item Encrypted bool"
.PP
Prepend a single dot (\*(L".\*(R") before each variable name.
.PP
Here are some examples how to use the feature:
.PP
Only show the keys of all entries:
.PP
.Vb 1
\& anydb list \-m template \-T "{{ .Key }}"
.Ve
.PP
Format the list in a way so that is possible to evaluate it in a
shell:
.PP
.Vb 2
\& eval $(anydb get foo \-m template \-T "key=\*(Aq{{ .Key }}\*(Aq value=\*(Aq{{ .Value }}\*(Aq ts=\*(Aq{{ .Created}}\*(Aq")
\& echo "Key: $key, Value: $value"
.Ve
.PP
Print the values in \s-1CSV\s0 format \s-1ONLY\s0 if they have some tag:
.PP
.Vb 1
\& anydb list \-m template \-T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created}}{{ end }}"
.Ve
.SH "CONFIGURATION"
.IX Header "CONFIGURATION"
Anydb looks at the following location for a configuration file, in
that order:
.ie n .IP """$HOME/.config/anydb/anydb.toml""" 4
.el .IP "\f(CW$HOME/.config/anydb/anydb.toml\fR" 4
.IX Item "$HOME/.config/anydb/anydb.toml"
.PD 0
.ie n .IP """$HOME/.anydb.toml""" 4
.el .IP "\f(CW$HOME/.anydb.toml\fR" 4
.IX Item "$HOME/.anydb.toml"
.ie n .IP """anydb.toml"" in the current directory" 4
.el .IP "\f(CWanydb.toml\fR in the current directory" 4
.IX Item "anydb.toml in the current directory"
.ie n .IP "or specify one using ""\-c""" 4
.el .IP "or specify one using \f(CW\-c\fR" 4
.IX Item "or specify one using -c"
.PD
The configuration format uses the \s-1TOML\s0 language, refer to
<https://toml.io/en/> for more details. The key names correspond to
the commandline options in most cases.
.Sp
Configuration follows a certain precedence: the files are tried to be
read in the given order, followed by commandline options. That is, the
last configuration file wins, unless the user provides a commandline
option, then this setting will be taken.
.Sp
A complete configuration file might look like this:
.Sp
.Vb 7
\& # defaults
\& dbfile = "~/.config/anydb/default.db"
\& dbbucket = "data"
\& noheaders = false
\& nohumanize = false
\& encrypt = false
\& listen = "localhost:8787"
\&
\& # different setups for different buckets
\& [buckets.data]
\& encrypt = true
\&
\& [buckets.test]
\& encrypt = false
.Ve
.Sp
Under normal circumstances you don't need a configuration
file. However, if you want to use different buckets, then this might
be a handy option. Buckets are being configured in ini-style with the
term \*(L"bucket.\*(R" followed by the bucket name. In the example above we
enable encryption for the default bucket \*(L"data\*(R" and disable it for a
bucket \*(L"test\*(R". To use different buckets, use the \f(CW\*(C`\-b\*(C'\fR option.
.SH "REST API"
.IX Header "REST API"
The subcommand \fBserve\fR starts a simple \s-1HTTP\s0 service, which responds
to RESTful \s-1HTTP\s0 requests. The listener responds to all requests with a
\&\s-1JSON\s0 encoded response. The response contains the status and the
content \- if any \- of the requested resource.
.PP
The following requests are supported:
.IP "\fB\s-1GET\s0 /anydb/v1/\fR" 4
.IX Item "GET /anydb/v1/"
Returns a \s-1JSON\s0 encoded list of all entries.
.IP "\fB\s-1GET\s0 /anydb/v1/key\fR" 4
.IX Item "GET /anydb/v1/key"
Returns the \s-1JSON\s0 encoded entry, if found.
.IP "\fB\s-1PUT\s0 /anydb/v1/\fR" 4
.IX Item "PUT /anydb/v1/"
Create an entry. Expects a \s-1JSON\s0 encoded request object in \s-1POST\s0 data.
.IP "\fB\s-1DELETE\s0 /anydb/v1/key\fR" 4
.IX Item "DELETE /anydb/v1/key"
Delete an entry.
.PP
Some curl example calls to the \s-1API:\s0
.PP
Post a new key:
curl \-X \s-1PUT\s0 localhost:8787/anydb/v1/ \e
\-H 'Content\-Type: application/json' \e
\-d '{\*(L"key\*(R":\*(L"foo\*(R",\*(L"val\*(R":\*(L"bar\*(R"}'
.PP
Retrieve the value:
.PP
.Vb 1
\& curl localhost:8787/anydb/v1/foo
.Ve
.PP
List all keys:
.PP
.Vb 1
\& curl localhost:8787/anydb/v1/
.Ve
.SH "BUGS"
.IX Header "BUGS"
In order to report a bug, unexpected behavior, feature requests
or to submit a patch, please open an issue on github:
<https://github.com/TLINDEN/anydb/issues>.
.PP
Please repeat the failing command with debugging enabled \f(CW\*(C`\-d\*(C'\fR and
include the output in the issue.
.SH "LIMITATIONS"
.IX Header "LIMITATIONS"
The \s-1REST API\s0 list request doesn't provide any filtering capabilities yet.
.SH "LICENSE"
.IX Header "LICENSE"
This software is licensed under the \s-1GNU GENERAL PUBLIC LICENSE\s0 version 3.

552
anydb.pod
View File

@@ -11,14 +11,20 @@ anydb - a personal key value store
Available Commands:
completion Generate the autocompletion script for the specified shell
del Delete key
edit Edit a key
export Export database to json
get Retrieve value for a key
help Help about any command
import Import database dump
info info
list List database contents
man show manual page
serve run REST API listener
set Insert key/value pair
Flags:
-b, --bucket string use other bucket (default: data) (default "data")
-c, --config string toml config file
-f, --dbfile string DB file to use (default "/home/scip/.config/anydb/default.db")
-d, --debug Enable debugging
-h, --help help for anydb
@@ -26,11 +32,551 @@ anydb - a personal key value store
Use "anydb [command] --help" for more information about a command.
=head1 DESCRIPTION
Anydb is a simple to use commandline tool to store anything you'd
like, even binary files etc. It uses a key/value store (bbolt) in your
home directory.
Anydb is a commandline personal key value store, it is simple to use
and can be used to store anything you'd like, even binary files
etc. It uses a key/value store (bbolt) in your home directory.
The tool provides a number of subcommands to use it, there are global
options and each subcommand has its own set of options.
=head1 GLOBAL OPTIONS
=over
=item C<-f, --dbfile filename>
The default location of your databas is
C<$HOME/.config/anydb/default.db>. You can change this with the C<-f>
option.
=item C<-b, --bucket name>
Data in a bbolt key-value-store are managed in so called
buckets. These are kind of namespaces, where each key must be
unique. However, a database may contain more than one bucket.
By default anydb uses a bucket named "data", but you can change this
using the option C<-b>.
Buckets can be configured to always encrypt values, see L<ENCRYTPTION>.
=item C<-c, --config filename>
Under normal circumstances you don't need a configuration file. But if
you want, you can provide one using the option C<-c>.
Anydb looks for a couple of default locations for a config file. You
only need this option if you want to supply a configuration on a
non-standard location. See L<CONFIGURATION> for more details.
=item C<-d, --debug>
Enable debug output.
=item C<-h, --help>
Show the usage of anydb.
=item C<-v, --version>
Show the program version.
=back
All of these options can be used with subcommands as well.
=head1 SUBCOMMANDS
=head2 completion
The B<completion> command can be used to setup completion for
anydb. Just put something like this into your shell's configuration
file:
source <(anydb completion bash)
If you use another shell, specify it instead of bash, of course.
=head2 set
The B<set> command is being used to insert or update a key-value pair.
Usage:
Usage:
anydb set <key> [<value> | -r <file>] [-t <tag>] [flags]
Aliases:
set, add, s, +
Flags:
-e, --encrypt encrypt value
-r, --file string Filename or - for STDIN
-h, --help help for set
-t, --tags tag,tag,... tags, multiple allowed
The standard way to insert a new entry is really simple:
anydb set key value
If you don't specify a value, anydb expects you to feed it some data
via STDIN. For example:
anydb set key < file
You might as well specify a file directly using the C<-f> option:
anydb set key -f file
Values can be encrypted using B<ChaCha20Poly1305> when you specify the
C<-e> option. Anydb will ask you interactively for a passphrase. You
might as well provide the passphrase using the environment variable
C<ANYDB_PASSWORD>. To encrypt the value, a cryptographically secure
key will be derived from the passphrase using the ArgonID2
algorithm. Each value can be encrypted with another passphrase. So,
the database itself is not encrypted, just the values.
You can supply tags by using the option C<-t>. Multiple tags can be
provided either by separating them with a comma or by using multiple
C<-t> parameters:
anydb set key value -t tag1,tag2
anydb set key value -t tag1 -t tag2
You can later filter entries by tag or by a combination of tags.
To edit or modify an entry, just use the B<set> command with the same
key, the value in the database will be overwritten with the new
value. An alternative option is the B<edit> command, see below.
=head2 get
To retrieve the value of a key, use the B<get> subcommand.
Usage:
Usage:
anydb get <key> [-o <file>] [-m <mode>] [-n -N] [-T <tpl>] [flags]
Aliases:
get, show, g, .
Flags:
-h, --help help for get
-m, --mode string output format (simple|wide|json|template) (default 'simple')
-n, --no-headers omit headers in tables
-N, --no-human do not translate to human readable values
-o, --output string output value to file (ignores -m)
-T, --template string go template for '-m template'
In its simplest form you just call the B<get> subcommand with the key
you want to have the value for. The value is being printed to STDOUT
by default:
anydb get key
If the value is binary content, it will not just being printed. In
those cases you need to either redirect output into a file or use the
option C<-o> to write to a file:
anydb get key > file
anydb get key -o file
If the value is encrypted, you will be asked for the passphrase to
decrypt it. If the environment variable C<ANYDB_PASSWORD> is set, its
value will be used instead.
There are different output modes you can choos from: simple, wide and
json. The "simple" mode is the default one, it just prints the value
as is. The "wide" mode prints a tabular output similar to the B<list>
subcommand, see there for more details. The options C<-n> and C<-N>
have the same meaning as in the list command. The "json" mode prints
the raw JSON representation of the whole database entry. Decryption
will only take place in "simple" and "json" mode. The "template" mode
provides the most flexibily, it is detailed in the section
L<TEMPLATES>.
=head2 list
The B<list> subcommand displays a list of all database entries.
Usage:
Usage:
anydb list [<filter-regex>] [-t <tag>] [-m <mode>] [-n -N] [-T <tpl>] [-i] [flags]
Aliases:
list, /, ls
Flags:
-i, --case-insensitive filter case insensitive
-h, --help help for list
-m, --mode string output format (table|wide|json|template),
wide is a verbose table. (default 'table')
-n, --no-headers omit headers in tables
-N, --no-human do not translate to human readable values
-t, --tags stringArray tags, multiple allowed
-T, --template string go template for '-m template'
-l, --wide-output output mode: wide
In its simplest form - without any options - , the B<list> command
just prints all keys with their values to STDOUT. Values are being
truncated to maximum of 60 characters, that is, multiline values are
not completely shown in order to keep the tabular view readable.
To get more informations about each entry, use the C<-o wide> or C<-l>
option. In addition to the key and value also the size, update
timestamp and tags will be printed. Time and size values are converted
into a human readable form, you can suppress this behavior with the
C<-N> option. You may omit the headers using the option C<-n>
Sometimes you might want to filter the list of entries. Either because
your database grew too large or because you're searching for
something. In that case you have two options: You may supply one or
more tags or provide a filter regexp. To filter by tag, do:
anydb list -t tag1
anydb list -t tag1,tag2
anydb list -t tag1 -t tag2
To filter using a regular expression, do:
anydb list "foo.*bar"
Regular expressions follow the golang B<re2> syntax. For more details
about the syntax, refer to
L<https://github.com/google/re2/wiki/Syntax>. Please note, that this
regexp dialect is not PCRE compatible, but supports most of its
features.
If you want to search case insensitive, add the option C<-i>.
You can - as with the B<get> command - use other output modes. The
default mode is "table". The "wide" mode is, as already mentioned, a
more detailed table. Also supported is "json" mode and "template"
mode. For details about using templates see L<TEMPLATES>.
=head2 del
Use the B<del> command to delete database entries.
Usage:
Usage:
anydb del <key> [flags]
Aliases:
del, d, rm
Flags:
-h, --help help for del
The subcommand B<del> does not provide any further options, it just
deletes the entry referred to by the given key. No questions are being
asked.
=head2 edit
The B<edit> command makes it easier to modify larger entries.
Usage:
Usage:
anydb edit <key> [flags]
Aliases:
edit, modify, mod, ed, vi
Flags:
-h, --help help for edit
The subcommand B<edit> does not provide any further options. It
works like this:
=over
=item 1. Write the value info a temporary file.
=item 2. Execute the editor (which one, see below!) with that file.
=item 3. Now you can edit the file and save+close it when done.
=item 4. Anydb picks up the file and if the content has changed, puts its value into the DB.
=back
By default anydb executes the C<vi> command. You can modify this
behavior by setting the environment variable C<EDITOR> appropriately.
Please note, that this does not work with binary content!
=head2 export
Since the bbold database file is not portable across platforms (it is
bound to the endianess of the CPU it was being created on), you might
want to create a backup file of your database. You can do this with
the B<export> subcommand.
Usage:
Usage:
anydb export [-o <json filename>] [flags]
Aliases:
export, dump, backup
Flags:
-h, --help help for export
-o, --output string output to file
The database dump is a JSON representation of the whole database and
will be printed to STDOUT by default. Redirect it to a file or use the
C<-o> option:
anydb export > dump.json
anydb export -o dump.json
Please note, that encrypted values will not be decrypted. This might
change in a future version of anydb.
=head2 import
The B<import> subcommand can be used to restore a database from a JSON
dump.
Usage:
Usage:
anydb import [<json file>] [flags]
Aliases:
import, restore
Flags:
-r, --file string Filename or - for STDIN
-h, --help help for import
-t, --tags stringArray tags, multiple allowed
By default the C<import> subcommand reads the JSON contents from
STDIN. You might pipe the dump into it or use the option C<-r>:
anydb import < dump.json
anydb import -r dump.json
cat dump.json | anydb import
If there is already a database, it will be saved by appending a
timestamp and a new database with the contents of the dump will be
created.
=head2 serve
Anydb provides a RESTful API, which you can use to manage the database
from somewhere else. The API does not provide any authentication or
any other security measures, so better only use it on localhost.
Usage:
Usage:
anydb serve [-l host:port] [flags]
Flags:
-h, --help help for serve
-l, --listen string host:port (default "localhost:8787")
To start the listener, just execute the B<serve> subcommand. You can
tweak the ip address and tcp port using the C<-l> option. The listener
will not fork and run in the foreground. Logs are being printed to
STDOUT as long as the listener runs.
For more details about the API, please see the L<REST API> section.
=head2 info
The B<info> subcommand shows you some information about your current
database.
Usage:
Usage:
anydb info [flags]
Flags:
-h, --help help for info
-N, --no-human do not translate to human readable values
Data being shown are: filename and size, number of keys per bucket. If
you supply the C<-d> option (debug), some bbolt internals are being
displayed as well.
=head2 man
The B<man> subcommand shows an unformatted text variant of the manual
page (which are currently reading).
Usage:
Usage:
anydb man [flags]
Flags:
-h, --help help for man
The manual is being piped into the C<more> command, which is being
expected to exist according to the POSIX standard on all supported
unix platforms. It might not work on Windows.
=head1 TEMPLATES
The B<get> and B<list> commands support a template feature, which is
very handy to create you own kind of formatting. The template syntax
being used is the GO template language, refer to
L<https://pkg.go.dev/text/template> for details.
Each template operates on one or more entries, no loop construct is
required, the template provided applies to every matching entry
separatley.
The following template variables can be used:
=over
=item B<Key> - string
=item B<Value> - string
=item B<Bin> - []byte
=item B<Created> - time.Time
=item B<Tags> - []string
=item B<Encrypted> bool
=back
Prepend a single dot (".") before each variable name.
Here are some examples how to use the feature:
Only show the keys of all entries:
anydb list -m template -T "{{ .Key }}"
Format the list in a way so that is possible to evaluate it in a
shell:
eval $(anydb get foo -m template -T "key='{{ .Key }}' value='{{ .Value }}' ts='{{ .Created}}'")
echo "Key: $key, Value: $value"
Print the values in CSV format ONLY if they have some tag:
anydb list -m template -T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created}}{{ end }}"
=head1 CONFIGURATION
Anydb looks at the following location for a configuration file, in
that order:
=over
=item C<$HOME/.config/anydb/anydb.toml>
=item C<$HOME/.anydb.toml>
=item C<anydb.toml> in the current directory
=item or specify one using C<-c>
The configuration format uses the TOML language, refer to
L<https://toml.io/en/> for more details. The key names correspond to
the commandline options in most cases.
Configuration follows a certain precedence: the files are tried to be
read in the given order, followed by commandline options. That is, the
last configuration file wins, unless the user provides a commandline
option, then this setting will be taken.
A complete configuration file might look like this:
# defaults
dbfile = "~/.config/anydb/default.db"
dbbucket = "data"
noheaders = false
nohumanize = false
encrypt = false
listen = "localhost:8787"
# different setups for different buckets
[buckets.data]
encrypt = true
[buckets.test]
encrypt = false
Under normal circumstances you don't need a configuration
file. However, if you want to use different buckets, then this might
be a handy option. Buckets are being configured in ini-style with the
term "bucket." followed by the bucket name. In the example above we
enable encryption for the default bucket "data" and disable it for a
bucket "test". To use different buckets, use the C<-b> option.
=back
=head1 REST API
The subcommand B<serve> starts a simple HTTP service, which responds
to RESTful HTTP requests. The listener responds to all requests with a
JSON encoded response. The response contains the status and the
content - if any - of the requested resource.
The following requests are supported:
=over
=item B<GET /anydb/v1/>
Returns a JSON encoded list of all entries.
=item B<GET /anydb/v1/key>
Returns the JSON encoded entry, if found.
=item B<PUT /anydb/v1/>
Create an entry. Expects a JSON encoded request object in POST data.
=item B<DELETE /anydb/v1/key>
Delete an entry.
=back
Some curl example calls to the API:
Post a new key:
curl -X PUT localhost:8787/anydb/v1/ \
-H 'Content-Type: application/json' \
-d '{"key":"foo","val":"bar"}'
Retrieve the value:
curl localhost:8787/anydb/v1/foo
List all keys:
curl localhost:8787/anydb/v1/
=head1 BUGS
In order to report a bug, unexpected behavior, feature requests
or to submit a patch, please open an issue on github:
L<https://github.com/TLINDEN/anydb/issues>.
Please repeat the failing command with debugging enabled C<-d> and
include the output in the issue.
=head1 LIMITATIONS
The REST API list request doesn't provide any filtering capabilities yet.
=head1 LICENSE

7
app/db.go
View File

@@ -76,6 +76,13 @@ func (entry *DbEntry) Normalize() {
entry.Size = len(entry.Bin)
}
if strings.Contains(entry.Value, "\n") {
parts := strings.Split(entry.Value, "\n")
if len(parts) > 0 {
entry.Value = parts[0]
}
}
if len(entry.Value) > MaxValueWidth {
entry.Value = entry.Value[0:MaxValueWidth] + "..."
}

23
cfg/config.go
View File

@@ -26,23 +26,24 @@ import (
"github.com/tlinden/anydb/common"
)
var Version string = "v0.0.5"
var Version string = "v0.0.7"
type BucketConfig struct {
Encrypt bool
}
type Config struct {
Debug bool
Dbfile string
Dbbucket string
Template string
Mode string // wide, table, yaml, json
NoHeaders bool
NoHumanize bool
Encrypt bool // one entry
Listen string
Buckets map[string]BucketConfig // config file only
Debug bool
Dbfile string
Dbbucket string
Template string
Mode string // wide, table, yaml, json
NoHeaders bool
NoHumanize bool
Encrypt bool // one entry
CaseInsensitive bool
Listen string
Buckets map[string]BucketConfig // config file only
Tags []string // internal
DB *app.DB // internal

498
cmd/anydb.go
View File

@@ -1,19 +1,3 @@
/*
Copyright © 2024 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
var manpage = `
@@ -28,14 +12,20 @@ SYNOPSIS
Available Commands:
completion Generate the autocompletion script for the specified shell
del Delete key
edit Edit a key
export Export database to json
get Retrieve value for a key
help Help about any command
import Import database dump
info info
list List database contents
man show manual page
serve run REST API listener
set Insert key/value pair
Flags:
-b, --bucket string use other bucket (default: data) (default "data")
-c, --config string toml config file
-f, --dbfile string DB file to use (default "/home/scip/.config/anydb/default.db")
-d, --debug Enable debugging
-h, --help help for anydb
@@ -44,9 +34,479 @@ SYNOPSIS
Use "anydb [command] --help" for more information about a command.
DESCRIPTION
Anydb is a simple to use commandline tool to store anything you'd like,
even binary files etc. It uses a key/value store (bbolt) in your home
directory.
Anydb is a commandline personal key value store, it is simple to use and
can be used to store anything you'd like, even binary files etc. It uses
a key/value store (bbolt) in your home directory.
The tool provides a number of subcommands to use it, there are global
options and each subcommand has its own set of options.
GLOBAL OPTIONS
"-f, --dbfile filename"
The default location of your databas is
"$HOME/.config/anydb/default.db". You can change this with the "-f"
option.
"-b, --bucket name"
Data in a bbolt key-value-store are managed in so called buckets.
These are kind of namespaces, where each key must be unique.
However, a database may contain more than one bucket.
By default anydb uses a bucket named "data", but you can change this
using the option "-b".
Buckets can be configured to always encrypt values, see ENCRYTPTION.
"-c, --config filename"
Under normal circumstances you don't need a configuration file. But
if you want, you can provide one using the option "-c".
Anydb looks for a couple of default locations for a config file. You
only need this option if you want to supply a configuration on a
non-standard location. See CONFIGURATION for more details.
"-d, --debug"
Enable debug output.
"-h, --help"
Show the usage of anydb.
"-v, --version"
Show the program version.
All of these options can be used with subcommands as well.
SUBCOMMANDS
completion
The completion command can be used to setup completion for anydb. Just
put something like this into your shell's configuration file:
source <(anydb completion bash)
If you use another shell, specify it instead of bash, of course.
set
The set command is being used to insert or update a key-value pair.
Usage:
Usage:
anydb set <key> [<value> | -r <file>] [-t <tag>] [flags]
Aliases:
set, add, s, +
Flags:
-e, --encrypt encrypt value
-r, --file string Filename or - for STDIN
-h, --help help for set
-t, --tags tag,tag,... tags, multiple allowed
The standard way to insert a new entry is really simple:
anydb set key value
If you don't specify a value, anydb expects you to feed it some data via
STDIN. For example:
anydb set key < file
You might as well specify a file directly using the "-f" option:
anydb set key -f file
Values can be encrypted using ChaCha20Poly1305 when you specify the "-e"
option. Anydb will ask you interactively for a passphrase. You might as
well provide the passphrase using the environment variable
"ANYDB_PASSWORD". To encrypt the value, a cryptographically secure key
will be derived from the passphrase using the ArgonID2 algorithm. Each
value can be encrypted with another passphrase. So, the database itself
is not encrypted, just the values.
You can supply tags by using the option "-t". Multiple tags can be
provided either by separating them with a comma or by using multiple
"-t" parameters:
anydb set key value -t tag1,tag2
anydb set key value -t tag1 -t tag2
You can later filter entries by tag or by a combination of tags.
To edit or modify an entry, just use the set command with the same key,
the value in the database will be overwritten with the new value. An
alternative option is the edit command, see below.
get
To retrieve the value of a key, use the get subcommand.
Usage:
Usage:
anydb get <key> [-o <file>] [-m <mode>] [-n -N] [-T <tpl>] [flags]
Aliases:
get, show, g, .
Flags:
-h, --help help for get
-m, --mode string output format (simple|wide|json|template) (default 'simple')
-n, --no-headers omit headers in tables
-N, --no-human do not translate to human readable values
-o, --output string output value to file (ignores -m)
-T, --template string go template for '-m template'
In its simplest form you just call the get subcommand with the key you
want to have the value for. The value is being printed to STDOUT by
default:
anydb get key
If the value is binary content, it will not just being printed. In those
cases you need to either redirect output into a file or use the option
"-o" to write to a file:
anydb get key > file
anydb get key -o file
If the value is encrypted, you will be asked for the passphrase to
decrypt it. If the environment variable "ANYDB_PASSWORD" is set, its
value will be used instead.
There are different output modes you can choos from: simple, wide and
json. The "simple" mode is the default one, it just prints the value as
is. The "wide" mode prints a tabular output similar to the list
subcommand, see there for more details. The options "-n" and "-N" have
the same meaning as in the list command. The "json" mode prints the raw
JSON representation of the whole database entry. Decryption will only
take place in "simple" and "json" mode. The "template" mode provides the
most flexibily, it is detailed in the section TEMPLATES.
list
The list subcommand displays a list of all database entries.
Usage:
Usage:
anydb list [<filter-regex>] [-t <tag>] [-m <mode>] [-n -N] [-T <tpl>] [flags]
Aliases:
list, /, ls
Flags:
-h, --help help for list
-m, --mode string output format (table|wide|json|template), wide is a verbose table. (default 'table')
-n, --no-headers omit headers in tables
-N, --no-human do not translate to human readable values
-t, --tags stringArray tags, multiple allowed
-T, --template string go template for '-m template'
-l, --wide-output output mode: wide
In its simplest form - without any options - , the list command just
prints all keys with their values to STDOUT. Values are being truncated
to maximum of 60 characters, that is, multiline values are not
completely shown in order to keep the tabular view readable.
To get more informations about each entry, use the "-o wide" or "-l"
option. In addition to the key and value also the size, update timestamp
and tags will be printed. Time and size values are converted into a
human readable form, you can suppress this behavior with the "-N"
option. You may omit the headers using the option "-n"
Sometimes you might want to filter the list of entries. Either because
your database grew too large or because you're searching for something.
In that case you have two options: You may supply one or more tags or
provide a filter regexp. To filter by tag, do:
anydb list -t tag1
anydb list -t tag1,tag2
anydb list -t tag1 -t tag2
To filter using a regular expression, do:
anydb list "foo.*bar"
Regular expressions follow the golang re2 syntax. For more details about
the syntax, refer to <https://github.com/google/re2/wiki/Syntax>. Please
note, that this regexp dialect is not PCRE compatible, but supports most
of its features.
You can - as with the get command - use other output modes. The default
mode is "table". The "wide" mode is, as already mentioned, a more
detailed table. Also supported is "json" mode and "template" mode. For
details about using templates see TEMPLATES.
del
Use the del command to delete database entries.
Usage:
Usage:
anydb del <key> [flags]
Aliases:
del, d, rm
Flags:
-h, --help help for del
The subcommand del does not provide any further options, it just deletes
the entry referred to by the given key. No questions are being asked.
edit
The edit command makes it easier to modify larger entries.
Usage:
Usage:
anydb edit <key> [flags]
Aliases:
edit, modify, mod, ed, vi
Flags:
-h, --help help for edit
The subcommand edit does not provide any further options. It works like
this:
1. Write the value info a temporary file.
2. Execute the editor (which one, see below!) with that file.
3. Now you can edit the file and save+close it when done.
4. Anydb picks up the file and if the content has changed, puts its
value into the DB.
By default anydb executes the "vi" command. You can modify this behavior
by setting the environment variable "EDITOR" appropriately.
Please note, that this does not work with binary content!
export
Since the bbold database file is not portable across platforms (it is
bound to the endianess of the CPU it was being created on), you might
want to create a backup file of your database. You can do this with the
export subcommand.
Usage:
Usage:
anydb export [-o <json filename>] [flags]
Aliases:
export, dump, backup
Flags:
-h, --help help for export
-o, --output string output to file
The database dump is a JSON representation of the whole database and
will be printed to STDOUT by default. Redirect it to a file or use the
"-o" option:
anydb export > dump.json
anydb export -o dump.json
Please note, that encrypted values will not be decrypted. This might
change in a future version of anydb.
import
The import subcommand can be used to restore a database from a JSON
dump.
Usage:
Usage:
anydb import [<json file>] [flags]
Aliases:
import, restore
Flags:
-r, --file string Filename or - for STDIN
-h, --help help for import
-t, --tags stringArray tags, multiple allowed
By default the "import" subcommand reads the JSON contents from STDIN.
You might pipe the dump into it or use the option "-r":
anydb import < dump.json
anydb import -r dump.json
cat dump.json | anydb import
If there is already a database, it will be saved by appending a
timestamp and a new database with the contents of the dump will be
created.
serve
Anydb provides a RESTful API, which you can use to manage the database
from somewhere else. The API does not provide any authentication or any
other security measures, so better only use it on localhost.
Usage:
Usage:
anydb serve [-l host:port] [flags]
Flags:
-h, --help help for serve
-l, --listen string host:port (default "localhost:8787")
To start the listener, just execute the serve subcommand. You can tweak
the ip address and tcp port using the "-l" option. The listener will not
fork and run in the foreground. Logs are being printed to STDOUT as long
as the listener runs.
For more details about the API, please see the "REST API" section.
info
The info subcommand shows you some information about your current
database.
Usage:
Usage:
anydb info [flags]
Flags:
-h, --help help for info
-N, --no-human do not translate to human readable values
Data being shown are: filename and size, number of keys per bucket. If
you supply the "-d" option (debug), some bbolt internals are being
displayed as well.
man
The man subcommand shows an unformatted text variant of the manual page
(which are currently reading).
Usage:
Usage:
anydb man [flags]
Flags:
-h, --help help for man
The manual is being piped into the "more" command, which is being
expected to exist according to the POSIX standard on all supported unix
platforms. It might not work on Windows.
TEMPLATES
The get and list commands support a template feature, which is very
handy to create you own kind of formatting. The template syntax being
used is the GO template language, refer to
<https://pkg.go.dev/text/template> for details.
Each template operates on one or more entries, no loop construct is
required, the template provided applies to every matching entry
separatley.
The following template variables can be used:
Key - string =item Value - string =item Bin - []byte =item Created -
time.Time =item Tags - []string =item Encrypted bool
Prepend a single dot (".") before each variable name.
Here are some examples how to use the feature:
Only show the keys of all entries:
anydb list -m template -T "{{ .Key }}"
Format the list in a way so that is possible to evaluate it in a shell:
eval $(anydb get foo -m template -T "key='{{ .Key }}' value='{{ .Value }}' ts='{{ .Created}}'")
echo "Key: $key, Value: $value"
Print the values in CSV format ONLY if they have some tag:
anydb list -m template -T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created}}{{ end }}"
CONFIGURATION
Anydb looks at the following location for a configuration file, in that
order:
"$HOME/.config/anydb/anydb.toml"
"$HOME/.anydb.toml"
"anydb.toml" in the current directory
or specify one using "-c"
The configuration format uses the TOML language, refer to
<https://toml.io/en/> for more details. The key names correspond to
the commandline options in most cases.
Configuration follows a certain precedence: the files are tried to
be read in the given order, followed by commandline options. That
is, the last configuration file wins, unless the user provides a
commandline option, then this setting will be taken.
A complete configuration file might look like this:
# defaults
dbfile = "~/.config/anydb/default.db"
dbbucket = "data"
noheaders = false
nohumanize = false
encrypt = false
listen = "localhost:8787"
# different setups for different buckets
[buckets.data]
encrypt = true
[buckets.test]
encrypt = false
Under normal circumstances you don't need a configuration file.
However, if you want to use different buckets, then this might be a
handy option. Buckets are being configured in ini-style with the
term "bucket." followed by the bucket name. In the example above we
enable encryption for the default bucket "data" and disable it for a
bucket "test". To use different buckets, use the "-b" option.
REST API
The subcommand serve starts a simple HTTP service, which responds to
RESTful HTTP requests. The listener responds to all requests with a JSON
encoded response. The response contains the status and the content - if
any - of the requested resource.
The following requests are supported:
GET /anydb/v1/
Returns a JSON encoded list of all entries.
GET /anydb/v1/key
Returns the JSON encoded entry, if found.
PUT /anydb/v1/
Create an entry. Expects a JSON encoded request object in POST data.
DELETE /anydb/v1/key
Delete an entry.
Some curl example calls to the API:
Post a new key: curl -X PUT localhost:8787/anydb/v1/ \ -H 'Content-Type:
application/json' \ -d '{"key":"foo","val":"bar"}'
Retrieve the value:
curl localhost:8787/anydb/v1/foo
List all keys:
curl localhost:8787/anydb/v1/
BUGS
In order to report a bug, unexpected behavior, feature requests or to
submit a patch, please open an issue on github:
<https://github.com/TLINDEN/anydb/issues>.
Please repeat the failing command with debugging enabled "-d" and
include the output in the issue.
LIMITATIONS
The REST API list request doesn't provide any filtering capabilities
yet.
LICENSE
This software is licensed under the GNU GENERAL PUBLIC LICENSE version

159
cmd/crud.go
View File

@@ -17,11 +17,8 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"bytes"
"errors"
"fmt"
"os"
"os/exec"
"strings"
"unicode/utf8"
@@ -29,7 +26,6 @@ import (
"github.com/tlinden/anydb/app"
"github.com/tlinden/anydb/cfg"
"github.com/tlinden/anydb/output"
"github.com/tlinden/anydb/rest"
)
func Set(conf *cfg.Config) *cobra.Command {
@@ -142,7 +138,7 @@ func Get(conf *cfg.Config) *cobra.Command {
}
cmd.PersistentFlags().StringVarP(&attr.File, "output", "o", "", "output value to file (ignores -m)")
cmd.PersistentFlags().StringVarP(&conf.Mode, "mode", "m", "", "output format (simple|wide|json) (default 'simple')")
cmd.PersistentFlags().StringVarP(&conf.Mode, "mode", "m", "", "output format (simple|wide|json|template) (default 'simple')")
cmd.PersistentFlags().BoolVarP(&conf.NoHeaders, "no-headers", "n", false, "omit headers in tables")
cmd.PersistentFlags().BoolVarP(&conf.NoHumanize, "no-human", "N", false, "do not translate to human readable values")
cmd.PersistentFlags().StringVarP(&conf.Template, "template", "T", "", "go template for '-m template'")
@@ -185,38 +181,6 @@ func Del(conf *cfg.Config) *cobra.Command {
return cmd
}
func Export(conf *cfg.Config) *cobra.Command {
var (
attr app.DbAttr
)
var cmd = &cobra.Command{
Use: "export [-o <json filename>]",
Short: "Export database to json",
Long: `Export database to json`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
conf.Mode = "json"
entries, err := conf.DB.List(&attr)
if err != nil {
return err
}
return output.WriteJSON(&attr, conf, entries)
},
}
cmd.PersistentFlags().StringVarP(&attr.File, "output", "o", "", "output to file")
cmd.Aliases = append(cmd.Aliases, "dump")
cmd.Aliases = append(cmd.Aliases, "backup")
return cmd
}
func List(conf *cfg.Config) *cobra.Command {
var (
attr app.DbAttr
@@ -224,7 +188,7 @@ func List(conf *cfg.Config) *cobra.Command {
)
var cmd = &cobra.Command{
Use: "list [<filter-regex>] [-t <tag>] [-m <mode>] [-n -N] [-T <tpl>]",
Use: "list [<filter-regex>] [-t <tag>] [-m <mode>] [-n -N] [-T <tpl>] [-i]",
Short: "List database contents",
Long: `List database contents`,
RunE: func(cmd *cobra.Command, args []string) error {
@@ -232,7 +196,11 @@ func List(conf *cfg.Config) *cobra.Command {
cmd.SilenceUsage = true
if len(args) > 0 {
attr.Args = args
if conf.CaseInsensitive {
attr.Args = []string{"(?i)" + args[0]}
} else {
attr.Args = args
}
}
// turn comma list into slice, if needed
@@ -253,11 +221,12 @@ func List(conf *cfg.Config) *cobra.Command {
},
}
cmd.PersistentFlags().StringVarP(&conf.Mode, "mode", "m", "", "output format (table|wide|json), wide is a verbose table. (default 'table')")
cmd.PersistentFlags().StringVarP(&conf.Mode, "mode", "m", "", "output format (table|wide|json|template), wide is a verbose table. (default 'table')")
cmd.PersistentFlags().StringVarP(&conf.Template, "template", "T", "", "go template for '-m template'")
cmd.PersistentFlags().BoolVarP(&wide, "wide-output", "l", false, "output mode: wide")
cmd.PersistentFlags().BoolVarP(&conf.NoHeaders, "no-headers", "n", false, "omit headers in tables")
cmd.PersistentFlags().BoolVarP(&conf.NoHumanize, "no-human", "N", false, "do not translate to human readable values")
cmd.PersistentFlags().BoolVarP(&conf.CaseInsensitive, "case-insensitive", "i", false, "filter case insensitive")
cmd.PersistentFlags().StringArrayVarP(&attr.Tags, "tags", "t", nil, "tags, multiple allowed")
cmd.Aliases = append(cmd.Aliases, "/")
@@ -266,116 +235,6 @@ func List(conf *cfg.Config) *cobra.Command {
return cmd
}
func Import(conf *cfg.Config) *cobra.Command {
var (
attr app.DbAttr
)
var cmd = &cobra.Command{
Use: "import [<json file>]",
Short: "Import database dump",
Long: `Import database dump`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
out, err := conf.DB.Import(&attr)
if err != nil {
return err
}
fmt.Print(out)
return nil
},
}
cmd.PersistentFlags().StringVarP(&attr.File, "file", "r", "", "Filename or - for STDIN")
cmd.PersistentFlags().StringArrayVarP(&attr.Tags, "tags", "t", nil, "tags, multiple allowed")
cmd.Aliases = append(cmd.Aliases, "add")
cmd.Aliases = append(cmd.Aliases, "s")
cmd.Aliases = append(cmd.Aliases, "+")
return cmd
}
func Help(conf *cfg.Config) *cobra.Command {
return nil
}
func Man(conf *cfg.Config) *cobra.Command {
var cmd = &cobra.Command{
Use: "man",
Short: "show manual page",
Long: `show manual page`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
man := exec.Command("less", "-")
var b bytes.Buffer
b.WriteString(manpage)
man.Stdout = os.Stdout
man.Stdin = &b
man.Stderr = os.Stderr
err := man.Run()
if err != nil {
return fmt.Errorf("failed to execute 'less': %w", err)
}
return nil
},
}
return cmd
}
func Serve(conf *cfg.Config) *cobra.Command {
var cmd = &cobra.Command{
Use: "serve [-l host:port]",
Short: "run REST API listener",
Long: `run REST API listener`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
return rest.Runserver(conf, nil)
},
}
cmd.PersistentFlags().StringVarP(&conf.Listen, "listen", "l", "localhost:8787", "host:port")
return cmd
}
func Info(conf *cfg.Config) *cobra.Command {
var cmd = &cobra.Command{
Use: "info",
Short: "info",
Long: `show info about database`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
info, err := conf.DB.Info()
if err != nil {
return err
}
return output.Info(os.Stdout, conf, info)
},
}
cmd.PersistentFlags().BoolVarP(&conf.NoHumanize, "no-human", "N", false, "do not translate to human readable values")
return cmd
}
func getPassword() ([]byte, error) {
var pass []byte

326
cmd/extra.go Normal file
View File

@@ -0,0 +1,326 @@
/*
Copyright © 2024 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package cmd
import (
"bytes"
"errors"
"fmt"
"io"
"os"
"os/exec"
"unicode/utf8"
"github.com/spf13/cobra"
"github.com/tlinden/anydb/app"
"github.com/tlinden/anydb/cfg"
"github.com/tlinden/anydb/output"
"github.com/tlinden/anydb/rest"
)
func Export(conf *cfg.Config) *cobra.Command {
var (
attr app.DbAttr
)
var cmd = &cobra.Command{
Use: "export [-o <json filename>]",
Short: "Export database to json",
Long: `Export database to json`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
conf.Mode = "json"
entries, err := conf.DB.List(&attr)
if err != nil {
return err
}
return output.WriteJSON(&attr, conf, entries)
},
}
cmd.PersistentFlags().StringVarP(&attr.File, "output", "o", "", "output to file")
cmd.Aliases = append(cmd.Aliases, "dump")
cmd.Aliases = append(cmd.Aliases, "backup")
return cmd
}
func Import(conf *cfg.Config) *cobra.Command {
var (
attr app.DbAttr
)
var cmd = &cobra.Command{
Use: "import [<json file>]",
Short: "Import database dump",
Long: `Import database dump`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
out, err := conf.DB.Import(&attr)
if err != nil {
return err
}
fmt.Print(out)
return nil
},
}
cmd.PersistentFlags().StringVarP(&attr.File, "file", "r", "", "Filename or - for STDIN")
cmd.PersistentFlags().StringArrayVarP(&attr.Tags, "tags", "t", nil, "tags, multiple allowed")
cmd.Aliases = append(cmd.Aliases, "restore")
return cmd
}
func Help(conf *cfg.Config) *cobra.Command {
return nil
}
func Man(conf *cfg.Config) *cobra.Command {
var cmd = &cobra.Command{
Use: "man",
Short: "show manual page",
Long: `show manual page`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
man := exec.Command("less", "-")
var b bytes.Buffer
b.WriteString(manpage)
man.Stdout = os.Stdout
man.Stdin = &b
man.Stderr = os.Stderr
err := man.Run()
if err != nil {
return fmt.Errorf("failed to execute 'less': %w", err)
}
return nil
},
}
return cmd
}
func Serve(conf *cfg.Config) *cobra.Command {
var cmd = &cobra.Command{
Use: "serve [-l host:port]",
Short: "run REST API listener",
Long: `run REST API listener`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
return rest.Runserver(conf, nil)
},
}
cmd.PersistentFlags().StringVarP(&conf.Listen, "listen", "l", "localhost:8787", "host:port")
return cmd
}
func Info(conf *cfg.Config) *cobra.Command {
var cmd = &cobra.Command{
Use: "info",
Short: "info",
Long: `show info about database`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
info, err := conf.DB.Info()
if err != nil {
return err
}
return output.Info(os.Stdout, conf, info)
},
}
cmd.PersistentFlags().BoolVarP(&conf.NoHumanize, "no-human", "N", false, "do not translate to human readable values")
return cmd
}
func Edit(conf *cfg.Config) *cobra.Command {
var (
attr app.DbAttr
)
var cmd = &cobra.Command{
Use: "edit <key>",
Short: "Edit a key",
Long: `Edit a key`,
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
return errors.New("no key specified")
}
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
password := []byte{}
if len(args) > 0 {
attr.Key = args[0]
}
// fetch entry
entry, err := conf.DB.Get(&attr)
if err != nil {
return err
}
if len(entry.Value) == 0 && len(entry.Bin) > 0 {
return errors.New("key contains binary uneditable content")
}
// decrypt if needed
if entry.Encrypted {
pass, err := getPassword()
if err != nil {
return err
}
password = pass
clear, err := app.Decrypt(pass, entry.Value)
if err != nil {
return err
}
if utf8.ValidString(string(clear)) {
entry.Value = string(clear)
} else {
entry.Bin = clear
}
entry.Encrypted = false
}
// determine editor, vi is default
editor := getEditor()
// save file to a temp file, call the editor with it, read
// it back in and compare the content with the original
// one
newcontent, err := editContent(editor, entry.Value)
if err != nil {
return err
}
// all is valid, fill our DB feeder
newattr := app.DbAttr{
Key: attr.Key,
Tags: attr.Tags,
Encrypted: attr.Encrypted,
Val: newcontent,
}
// encrypt if needed
if conf.Encrypt {
err = app.Encrypt(password, &attr)
if err != nil {
return err
}
}
// done
return conf.DB.Set(&newattr)
},
}
cmd.Aliases = append(cmd.Aliases, "modify")
cmd.Aliases = append(cmd.Aliases, "mod")
cmd.Aliases = append(cmd.Aliases, "ed")
cmd.Aliases = append(cmd.Aliases, "vi")
return cmd
}
func getEditor() string {
editor := "vi"
enveditor, present := os.LookupEnv("EDITOR")
if present {
if editor != "" {
editor = enveditor
}
}
return editor
}
// taken from github.com/tlinden/rpn/ (my own program)
func editContent(editor string, content string) (string, error) {
// create a temp file
tmp, err := os.CreateTemp("", "stack")
if err != nil {
return "", fmt.Errorf("failed to create templ file: %w", err)
}
defer os.Remove(tmp.Name())
// put the content into a tmp file
_, err = tmp.WriteString(content)
if err != nil {
return "", fmt.Errorf("failed to write value to temp file: %w", err)
}
// execute editor with our tmp file containing current stack
cmd := exec.Command(editor, tmp.Name())
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
err = cmd.Run()
if err != nil {
return "", fmt.Errorf("failed to run editor command %s: %w", editor, err)
}
// read the file back in
modified, err := os.Open(tmp.Name())
if err != nil {
return "", fmt.Errorf("failed to open temp file: %w", err)
}
defer modified.Close()
newcontent, err := io.ReadAll(modified)
if err != nil {
return "", fmt.Errorf("failed to read from temp file: %w", err)
}
newcontentstr := string(newcontent)
if content == newcontentstr {
return "", fmt.Errorf("content not modified, aborting")
}
return newcontentstr, nil
}

8
cmd/root.go
View File

@@ -119,15 +119,23 @@ func Execute() {
app.BucketData, "use other bucket (default: "+app.BucketData+")")
rootCmd.PersistentFlags().StringVarP(&configfile, "config", "c", "", "toml config file")
// CRUD
rootCmd.AddCommand(Set(&conf))
rootCmd.AddCommand(List(&conf))
rootCmd.AddCommand(Get(&conf))
rootCmd.AddCommand(Del(&conf))
// backup
rootCmd.AddCommand(Export(&conf))
rootCmd.AddCommand(Import(&conf))
// REST API
rootCmd.AddCommand(Serve(&conf))
// auxiliary
rootCmd.AddCommand(Man(&conf))
rootCmd.AddCommand(Info(&conf))
rootCmd.AddCommand(Edit(&conf))
err = rootCmd.Execute()
if err != nil {

22
output/single.go
View File

@@ -71,15 +71,25 @@ func Print(writer io.Writer, conf *cfg.Config, attr *app.DbAttr, entry *app.DbEn
}
func WriteFile(writer io.Writer, conf *cfg.Config, attr *app.DbAttr, entry *app.DbEntry) error {
fd, err := os.OpenFile(attr.File, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0755)
if err != nil {
return fmt.Errorf("failed to open file %s for writing: %w", attr.File, err)
var fileHandle *os.File
var err error
if attr.File == "-" {
fileHandle = os.Stdout
} else {
fd, err := os.OpenFile(attr.File, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0755)
if err != nil {
return fmt.Errorf("failed to open file %s for writing: %w", attr.File, err)
}
defer fd.Close()
fileHandle = fd
}
defer fd.Close()
if len(entry.Bin) > 0 {
// binary file content
_, err = fd.Write(entry.Bin)
_, err = fileHandle.Write(entry.Bin)
} else {
val := entry.Value
if !strings.HasSuffix(val, "\n") {
@@ -87,7 +97,7 @@ func WriteFile(writer io.Writer, conf *cfg.Config, attr *app.DbAttr, entry *app.
val += "\n"
}
_, err = fd.Write([]byte(val))
_, err = fileHandle.Write([]byte(val))
}
if err != nil {

7
t/workflow.txtar
View File

@@ -18,6 +18,9 @@
# simple entry
exec anydb -f test.db set foo bar
# single entry uc()
exec anydb -f test.db set MUCHAS gracias
# entry with tags
exec anydb -f test.db set color grey -t flower,plant
@@ -37,6 +40,10 @@ exec anydb -f test.db list -t flower
exec anydb -f test.db list b.r
stdout bar
# list with -i filter
exec anydb -f test.db list -i mucha
stdout MUCHA
# get single entry
exec anydb -f test.db get color
stdout grey