30 Commits

Author SHA1 Message Date
83de01b349 fix release builder 2025-02-10 15:49:22 +01:00
b4cf1e4d1c link to the bug 2025-02-10 15:41:30 +01:00
42dc598deb Merge branch 'main' of github.com:TLINDEN/anydb 2025-02-10 15:40:05 +01:00
ff169a7198 add warning about #19 2025-02-10 15:39:40 +01:00
T.v.Dein
6c4f119bfa Fixes and additions: (#20)
- fix encryption bug #19, which was a regression
- added encryption unit test
- added debug logging here and there

Co-authored-by: Thomas von Dein <tom@vondein.org>
2025-02-10 15:34:04 +01:00
16b6075329 ci tuning 2025-02-10 15:33:21 +01:00
07808187d5 enhance changelog generation, update release builder 2025-02-05 17:49:26 +01:00
T.v.Dein
bc1b08d6f9 fix #16: add documentation about how to use .Created.AsTime (#18)
Co-authored-by: Thomas von Dein <tom@vondein.org>
2025-02-03 17:50:28 +01:00
dependabot[bot]
bd5c1e3abb Bump docker/build-push-action from 6.10.0 to 6.13.0 (#13)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.10.0 to 6.13.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](48aba3b46d...ca877d9245)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-02-03 17:37:55 +01:00
dependabot[bot]
57026e11aa Bump actions/setup-go from 1 to 5 (#12)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 1 to 5.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v1...v5)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-02-03 17:37:41 +01:00
dependabot[bot]
d0fb560bb2 Bump actions/checkout from 2 to 4 (#14)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-02-03 17:37:24 +01:00
T.v.Dein
f2bc21e03a Feat/releasenotes (#17)
* add release notes, start with PR mode
2025-02-03 17:36:49 +01:00
717510a2f9 fix gifs 2025-02-02 19:45:06 +01:00
T.v.Dein
d794cc8608 Feature/vhs demo (#15)
* add vhs made demo gif

* add support for ANYDB_DB env var

* left one section

* fixed data type bug, added demo gifs, upgraded dependencies

---------

Co-authored-by: Thomas von Dein <tom@vondein.org>
2025-02-02 19:42:12 +01:00
359d134d6e do not push binaries locally anymore 2025-01-18 10:50:33 +01:00
a674c9c602 build release binaries with ci workflow 2025-01-18 10:48:37 +01:00
e26c61e26f use correct tag 2025-01-18 10:36:31 +01:00
T.v.Dein
e054d1e530 Add release ci (#11)
* add release binary builder
2025-01-18 10:32:01 +01:00
9afca91159 fixed crash in api list, added filter support to api list 2025-01-01 18:08:43 +01:00
T.v.Dein
675d10d8fa Merge pull request #10 from TLINDEN/proto
Use protobuf for internal data storage instead of json
2024-12-30 12:33:03 +01:00
659e3472bb fixed copylocks linter warning 2024-12-30 12:23:13 +01:00
1eb5efae0c finalized conversion to protobuf:
- fixed import+export
- generalized file options
- always store keys as lowercase
- fixed+enhanced docs
- fixed tests
2024-12-30 12:12:02 +01:00
Thomas von Dein
bb5c268ca8 todo 2024-12-29 23:29:23 +01:00
a4b6a3cfdf restructured data storage, values now have their own sub bucket 2024-12-29 18:29:43 +01:00
c144e99b41 switch to use protobuf for internal data structure in DB 2024-12-29 13:01:16 +01:00
Thomas von Dein
2a6a651b91 fix pod bug 2024-12-25 23:53:34 +01:00
Thomas von Dein
33d638aff0 fix typos and formatting 2024-12-23 23:51:17 +01:00
8a8888a3cf +badges 2024-12-23 20:02:38 +01:00
a590eaf903 +todo 2024-12-23 20:00:48 +01:00
cb6abecaf9 +man page badge 2024-12-23 19:58:51 +01:00
35 changed files with 1339 additions and 453 deletions

View File

@@ -1,10 +1,19 @@
name: build-and-test
on: [push, pull_request]
#on: [push, pull_request]
on:
pull_request:
branches:
- main
push:
branches:
- main
jobs:
build:
strategy:
matrix:
version: [1.21]
version: ['1.23']
os: [ubuntu-latest, macos-latest]
name: Build
runs-on: ${{ matrix.os }}
@@ -41,7 +50,7 @@ jobs:
steps:
- uses: actions/setup-go@v5
with:
go-version: 1.21
go-version: 1.23
- uses: actions/checkout@v4
- name: golangci-lint
uses: golangci/golangci-lint-action@v6

View File

@@ -22,13 +22,13 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push Docker image
uses: docker/build-push-action@48aba3b46d1b1fec4febb7c5d0c644b249a11355
uses: docker/build-push-action@ca877d9245402d1537745e0e356eab47c3520991
with:
push: true
tags: ghcr.io/tlinden/anydb:${{ github.ref_name}}
- name: Build and push latest Docker image
uses: docker/build-push-action@48aba3b46d1b1fec4febb7c5d0c644b249a11355
uses: docker/build-push-action@ca877d9245402d1537745e0e356eab47c3520991
with:
push: true
tags: ghcr.io/tlinden/anydb:latest

87
.github/workflows/release.yaml vendored Normal file
View File

@@ -0,0 +1,87 @@
name: build-release
on:
push:
tags:
- "v*.*.*"
jobs:
release:
name: Build Release Assets
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: 1.23.5
- name: Build the executables
run: ./mkrel.sh anydb ${{ github.ref_name}}
- name: List the executables
run: ls -l ./releases
- name: Upload the binaries
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
tag: ${{ github.ref_name }}
file: ./releases/*
file_glob: true
- name: Build Changelog
id: github_release
uses: mikepenz/release-changelog-builder-action@v5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
mode: "PR"
configurationJson: |
{
"template": "#{{CHANGELOG}}\n\n**Full Changelog**: #{{RELEASE_DIFF}}",
"pr_template": "- #{{TITLE}} (##{{NUMBER}}) by #{{AUTHOR}}\n#{{BODY}}",
"empty_template": "- no changes",
"categories": [
{
"title": "## New Features",
"labels": ["add", "feature"]
},
{
"title": "## Bug Fixes",
"labels": ["fix", "bug", "revert"]
},
{
"title": "## Documentation Enhancements",
"labels": ["doc"]
},
{
"title": "## Refactoring Efforts",
"labels": ["refactor"]
},
{
"title": "## Miscellaneus Changes",
"labels": []
}
],
"ignore_labels": [
"duplicate", "good first issue", "help wanted", "invalid", "question", "wontfix"
],
"label_extractor": [
{
"pattern": "(.) (.+)",
"target": "$1"
},
{
"pattern": "(.) (.+)",
"target": "$1",
"on_property": "title"
}
]
}
- name: Create Release
uses: softprops/action-gh-release@v2
with:
body: ${{steps.github_release.outputs.changelog}}

View File

@@ -14,7 +14,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
FROM golang:1.22-alpine as builder
FROM golang:1.23-alpine as builder
RUN apk update
RUN apk upgrade

View File

@@ -29,7 +29,7 @@ BUILD = $(shell date +%Y.%m.%d.%H%M%S)
VERSION := $(if $(filter $(BRANCH), development),$(version)-$(BRANCH)-$(COMMIT)-$(BUILD),$(version))
HAVE_POD := $(shell pod2text -h 2>/dev/null)
all: $(tool).1 cmd/$(tool).go buildlocal
all: $(tool).1 cmd/$(tool).go app/dbentry.pb.go buildlocal
%.1: %.pod
ifdef HAVE_POD
@@ -49,12 +49,17 @@ endif
# awk '/SYNOPS/{f=1;next} /DESCR/{f=0} f' $*.pod | sed 's/^ //' >> cmd/$*.go
# echo "\`" >> cmd/$*.go
app/dbentry.pb.go: app/dbentry.proto
protoc -I=. --go_out=app app/dbentry.proto
mv app/github.com/tlinden/anydb/app/dbentry.pb.go app/dbentry.pb.go
rm -rf app/github.com
buildlocal:
go build -ldflags "-X 'github.com/tlinden/anydb/cfg.VERSION=$(VERSION)'"
# binaries are being built by ci workflow on tag creation
release:
./mkrel.sh $(tool) $(version)
gh release create $(version) --generate-notes releases/*
gh release create $(version) --generate-notes
install: buildlocal
install -d -o $(UID) -g $(GID) $(PREFIX)/bin
@@ -98,3 +103,9 @@ lint:
lint-full:
golangci-lint run --enable-all --exclude-use-default --disable exhaustivestruct,exhaustruct,depguard,interfacer,deadcode,golint,structcheck,scopelint,varcheck,ifshort,maligned,nosnakecase,godot,funlen,gofumpt,cyclop,noctx,gochecknoglobals,paralleltest,forbidigo,gci,godox,goimports,ireturn,stylecheck,testpackage,mirror,nestif,revive,goerr113,gomnd
gocritic check -enableAll *.go
demo:
make -C demo demo
.PHONY: demo

139
README.md
View File

@@ -2,7 +2,16 @@
[![Actions](https://github.com/tlinden/anydb/actions/workflows/ci.yaml/badge.svg)](https://github.com/tlinden/anydb/actions)
[![License](https://img.shields.io/badge/license-GPL-blue.svg)](https://github.com/tlinden/anydb/blob/master/LICENSE)
[![Go Coverage](https://github.com/tlinden/anydb/wiki/coverage.svg)](https://raw.githack.com/wiki/tlinden/anydb/coverage.html)
[![Go Report Card](https://goreportcard.com/badge/github.com/tlinden/anydb)](https://goreportcard.com/report/github.com/tlinden/anydb)
[![GitHub release](https://img.shields.io/github/v/release/tlinden/anydb?color=%2300a719)](https://github.com/TLINDEN/anydb/releases/latest)
[![Documentation](https://img.shields.io/badge/manpage-documentation-blue)](https://github.com/TLINDEN/anydb/blob/master/anydb.pod)
> [!CAUTION]
> Version 0.1.3 introduced a [regression bug](https://github.com/TLINDEN/anydb/issues/19),
> which caused the encryption feature not to work correctly anymore.
> If you are using anydb 0.1.3, you are urgently advised to
> upgrade to 0.2.0
Anydb is a simple to use commandline tool to store anything you'd
like, even binary files etc. It is a re-implementation of
@@ -31,137 +40,11 @@ And I wrote a very similar [tool](https://www.daemon.de/projects/dbtool/) 24 yea
**anydb** can do all the things you can do with skate:
```shell
# Store something (and sync it to the network)
anydb set kitty meow
# Fetch something (from the local cache)
anydb get kitty
# Whats in the store?
anydb list
# Spaces are fine
anydb set "kitty litter" "smells great"
# You can store binary data, too
anydb set profile-pic < my-cute-pic.jpg
anydb get profile-pic > here-it-is.jpg
# Unicode also works, of course
anydb set 猫咪 喵
anydb get 猫咪
# For more info
anydb --help
# Do creative things with anydb list
anydb set penelope marmalade
anydb set christian tacos
anydb set muesli muesli
anydb list | xargs -n 2 printf '%s loves %s.\n'
```
![simple demo](https://github.com/TLINDEN/anydb/blob/main/demo/intro.gif)
However, there are more features than just that!
```shell
# you can assign tags
anydb set foo bar -t note,important
# and filter for them
anydb list -t important
# beside tags filtering you can also use regexps for searching
anydb list '[a-z]+\d'
# anydb also supports a wide output
anydb list -o wide
KEY TAGS SIZE AGE VALUE
blah important 4 B 7 seconds ago haha
foo 3 B 15 seconds ago bar
猫咪 3 B 3 seconds ago 喵
# there are shortcuts as well
anydb ls -l
anydb /
# other outputs are possible as well
anydb list -o json
# you can backup your database
anydb export -o backup.json
# and import it somewhere else
anydb import -r backup.json
# you can encrypt entries. anydb asks for a passphrase
# and will do the same when you retrieve the key using the
# get command. anydb will ask you interactively for a password
anydb set mypassword -e
# but you can provide it via an environment variable too
ANYDB_PASSWORD=foo anydb set -e secretkey blahblah
# too tiresome to add -e every time you add an entry?
# use a per bucket config
cat ~/.config/anydb/anydb.toml
[buckets.data]
encrypt = true
anydb set foo bar # will be encrypted
# speaking of buckets, you can use different buckets
anydb -b test set foo bar
# and speaking of configs, you can place a config file at these places:
# ~/.config/anydb/anydb.toml
# ~/.anydb.toml
# anydb.toml (current directory)
# or specify one using -c <filename>
# look at example.toml
# using template output mode you can freely design how to print stuff
# here, we print the values in CSV format ONLY if they have some tag
anydb ls -m template -T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created}}{{ end }}"
# or, to simulate skate's -k or -v
anydb ls -m template -T "{{ .Key }}"
anydb ls -m template -T "{{ .Value }}"
# maybe you want to digest the item in a shell script? also
# note, that both the list and get commands support templates
eval $(anydb get foo -m template -T "key='{{ .Key }}' value='{{ .Value }}' ts='{{ .Created}}'")
echo "$key: $value"
# run the restful api server
anydb serve
# post a new key
curl -X PUT localhost:8787/anydb/v1/ \
-H 'Content-Type: application/json' \
-d '{"key":"foo","val":"bar"}'
# retrieve it
curl localhost:8787/anydb/v1/foo
# list keys
curl localhost:8787/anydb/v1/
# as you might correctly suspect you can store multi-line values or
# the content of text files. but what to do if you want to change it?
# here's one way:
anydb get contract24 > file.txt && vi file.txt && anydb set contract24 -r file.txt
# annoying. better do this
anydb edit contract24
# sometimes you need to know some details about the current database
# add -d for more details
anydb info
# it comes with a manpage builtin
anydb man
```
![advanced demo](https://github.com/TLINDEN/anydb/blob/main/demo/advanced.gif)
## Installation

15
TODO.md
View File

@@ -1,4 +1,19 @@
## Features
- repl
- mime-type => exec app + value
- add waitgroup to db.go funcs
- RestList does not support any params?
## DB Structure
- put tags into sub bucket see #1
tags bucket
key/tag => tag/key
tag/key => tag
A delete would just delete all keys from all values and then:
lookup in tags bucket for all key/*, then iterate over the values and
remove all tag/key's. Then deleting a key would not leave any residue
behind.

78
anydb.1
View File

@@ -133,7 +133,7 @@
.\" ========================================================================
.\"
.IX Title "ANYDB 1"
.TH ANYDB 1 "2024-12-23" "1" "User Commands"
.TH ANYDB 1 "2025-02-10" "1" "User Commands"
.\" For nroff, turn off justification. Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
@@ -276,7 +276,7 @@ You might as well specify a file directly using the \f(CW\*(C`\-f\*(C'\fR option
.PP
Values can be encrypted using \fBChaCha20Poly1305\fR when you specify the
\&\f(CW\*(C`\-e\*(C'\fR option. Anydb will ask you interactively for a passphrase. You
might as well provide the passphrase using the environment variable
can also provide the passphrase using the environment variable
\&\f(CW\*(C`ANYDB_PASSWORD\*(C'\fR. To encrypt the value, a cryptographically secure
key will be derived from the passphrase using the ArgonID2
algorithm. Each value can be encrypted with another passphrase. So,
@@ -339,7 +339,7 @@ If the value is encrypted, you will be asked for the passphrase to
decrypt it. If the environment variable \f(CW\*(C`ANYDB_PASSWORD\*(C'\fR is set, its
value will be used instead.
.PP
There are different output modes you can choos from: simple, wide and
There are different output modes you can choose from: simple, wide and
json. The \*(L"simple\*(R" mode is the default one, it just prints the value
as is. The \*(L"wide\*(R" mode prints a tabular output similar to the \fBlist\fR
subcommand, see there for more details. The options \f(CW\*(C`\-n\*(C'\fR and \f(CW\*(C`\-N\*(C'\fR
@@ -356,16 +356,18 @@ Usage:
.PP
.Vb 2
\& Usage:
\& anydb list [<filter\-regex>] [\-t <tag>] [\-m <mode>] [\-n \-N] [\-T <tpl>] [flags]
\& anydb list [<filter\-regex> | \-t <tag> ] [\-m <mode>] [\-nNif] [\-T <tpl>] [flags]
\&
\& Aliases:
\& list, /, ls
\& list, ls, /, find, search
\&
\& Flags:
\& \-i, \-\-case\-insensitive filter case insensitive
\& \-h, \-\-help help for list
\& \-m, \-\-mode string output format (table|wide|json|template), wide is a verbose table. (default \*(Aqtable\*(Aq)
\& \-n, \-\-no\-headers omit headers in tables
\& \-N, \-\-no\-human do not translate to human readable values
\& \-s, \-\-search\-fulltext perform a full text search
\& \-t, \-\-tags stringArray tags, multiple allowed
\& \-T, \-\-template string go template for \*(Aq\-m template\*(Aq
\& \-l, \-\-wide\-output output mode: wide
@@ -405,6 +407,12 @@ about the syntax, refer to
regexp dialect is not \s-1PCRE\s0 compatible, but supports most of its
features.
.PP
If you want to search case insensitive, add the option \f(CW\*(C`\-i\*(C'\fR.
.PP
By default anydb only searches through the keys. If you want to search
through the values as well, then use the \f(CW\*(C`\-s\*(C'\fR option, which enables
full-text search.
.PP
You can \- as with the \fBget\fR command \- use other output modes. The
default mode is \*(L"table\*(R". The \*(L"wide\*(R" mode is, as already mentioned, a
more detailed table. Also supported is \*(L"json\*(R" mode and \*(L"template\*(R"
@@ -465,7 +473,7 @@ behavior by setting the environment variable \f(CW\*(C`EDITOR\*(C'\fR appropriat
Please note, that this does not work with binary content!
.SS "export"
.IX Subsection "export"
Since the bbold database file is not portable across platforms (it is
Since the bbolt database file is not portable across platforms (it is
bound to the endianess of the \s-1CPU\s0 it was being created on), you might
want to create a backup file of your database. You can do this with
the \fBexport\fR subcommand.
@@ -474,7 +482,7 @@ Usage:
.PP
.Vb 2
\& Usage:
\& anydb export [\-o <json filename>] [flags]
\& anydb export \-o <json filename> [flags]
\&
\& Aliases:
\& export, dump, backup
@@ -485,12 +493,12 @@ Usage:
.Ve
.PP
The database dump is a \s-1JSON\s0 representation of the whole database and
will be printed to \s-1STDOUT\s0 by default. Redirect it to a file or use the
\&\f(CW\*(C`\-o\*(C'\fR option:
will be printed to the file specified with the \f(CW\*(C`\-o\*(C'\fR option. If you
specify \*(L"\-\*(R" as the filename, it will be written to \s-1STDIN.\s0
.PP
.Vb 2
\& anydb export > dump.json
\& anydb export \-o dump.json
\& anydb export \-o \- > dump.json
.Ve
.PP
Please note, that encrypted values will not be decrypted. This might
@@ -504,7 +512,7 @@ Usage:
.PP
.Vb 2
\& Usage:
\& anydb import [<json file>] [flags]
\& anydb import \-i <json file> [flags]
\&
\& Aliases:
\& import, restore
@@ -515,13 +523,14 @@ Usage:
\& \-t, \-\-tags stringArray tags, multiple allowed
.Ve
.PP
By default the \f(CW\*(C`import\*(C'\fR subcommand reads the \s-1JSON\s0 contents from
\&\s-1STDIN.\s0 You might pipe the dump into it or use the option \f(CW\*(C`\-r\*(C'\fR:
The \f(CW\*(C`import\*(C'\fR subcommand reads the \s-1JSON\s0 contents from
the file specified with the \f(CW\*(C`\-i\*(C'\fR option. If you specify \*(L"\-\*(R" as the
filename, it will be read from \s-1STDIN.\s0
.PP
.Vb 3
\& anydb import < dump.json
\& anydb import \-r dump.json
\& cat dump.json | anydb import
\& anydb import \-i \- < dump.json
\& anydb import \-i dump.json
\& cat dump.json | anydb import \-i \-
.Ve
.PP
If there is already a database, it will be saved by appending a
@@ -599,8 +608,24 @@ required, the template provided applies to every matching entry
separatley.
.PP
The following template variables can be used:
.IP "\fBKey\fR \- string =item \fBValue\fR \- string =item \fBBin\fR \- []byte =item \fBCreated\fR \- time.Time =item \fBTags\fR \- []string =item \fBEncrypted\fR bool" 4
.IX Item "Key - string =item Value - string =item Bin - []byte =item Created - time.Time =item Tags - []string =item Encrypted bool"
.IP "\fB.Key\fR \- string" 4
.IX Item ".Key - string"
.PD 0
.IP "\fB.Value\fR \- string" 4
.IX Item ".Value - string"
.IP "\fB.Bin\fR \- []byte" 4
.IX Item ".Bin - []byte"
.IP "\fB.Created\fR \- timestamp.Time" 4
.IX Item ".Created - timestamp.Time"
.PD
To retrieve a string representation of the timestamp, use \f(CW\*(C`.Created.AsTime\*(C'\fR.
If you need a unix timestamp since epoch, use \f(CW\*(C`.Created.Unix\*(C'\fR.
.IP "\fB.Tags\fR \- []string" 4
.IX Item ".Tags - []string"
.PD 0
.IP "\fB.Encrypted\fR bool" 4
.IX Item ".Encrypted bool"
.PD
.PP
Prepend a single dot (\*(L".\*(R") before each variable name.
.PP
@@ -616,18 +641,18 @@ Format the list in a way so that is possible to evaluate it in a
shell:
.PP
.Vb 2
\& eval $(anydb get foo \-m template \-T "key=\*(Aq{{ .Key }}\*(Aq value=\*(Aq{{ .Value }}\*(Aq ts=\*(Aq{{ .Created}}\*(Aq")
\& echo "Key: $key, Value: $value"
\& eval $(anydb get foo \-m template \-T "key=\*(Aq{{ .Key }}\*(Aq value=\*(Aq{{ .Value }}\*(Aq ts=\*(Aq{{ .Created.AsTime}}\*(Aq")
\& echo "Key: $key, Value: $value, When: $ts"
.Ve
.PP
Print the values in \s-1CSV\s0 format \s-1ONLY\s0 if they have some tag:
.PP
.Vb 1
\& anydb list \-m template \-T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created}}{{ end }}"
\& anydb list \-m template \-T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created.AsTime}}{{ end }}"
.Ve
.SH "CONFIGURATION"
.IX Header "CONFIGURATION"
Anydb looks at the following location for a configuration file, in
Anydb looks at the following locations for a configuration file, in
that order:
.ie n .IP """$HOME/.config/anydb/anydb.toml""" 4
.el .IP "\f(CW$HOME/.config/anydb/anydb.toml\fR" 4
@@ -643,17 +668,18 @@ that order:
.el .IP "or specify one using \f(CW\-c\fR" 4
.IX Item "or specify one using -c"
.PD
.PP
The configuration format uses the \s-1TOML\s0 language, refer to
<https://toml.io/en/> for more details. The key names correspond to
the commandline options in most cases.
.Sp
.PP
Configuration follows a certain precedence: the files are tried to be
read in the given order, followed by commandline options. That is, the
last configuration file wins, unless the user provides a commandline
option, then this setting will be taken.
.Sp
.PP
A complete configuration file might look like this:
.Sp
.PP
.Vb 7
\& # defaults
\& dbfile = "~/.config/anydb/default.db"
@@ -670,7 +696,7 @@ A complete configuration file might look like this:
\& [buckets.test]
\& encrypt = false
.Ve
.Sp
.PP
Under normal circumstances you don't need a configuration
file. However, if you want to use different buckets, then this might
be a handy option. Buckets are being configured in ini-style with the

View File

@@ -133,7 +133,7 @@ You might as well specify a file directly using the C<-f> option:
Values can be encrypted using B<ChaCha20Poly1305> when you specify the
C<-e> option. Anydb will ask you interactively for a passphrase. You
might as well provide the passphrase using the environment variable
can also provide the passphrase using the environment variable
C<ANYDB_PASSWORD>. To encrypt the value, a cryptographically secure
key will be derived from the passphrase using the ArgonID2
algorithm. Each value can be encrypted with another passphrase. So,
@@ -189,7 +189,7 @@ If the value is encrypted, you will be asked for the passphrase to
decrypt it. If the environment variable C<ANYDB_PASSWORD> is set, its
value will be used instead.
There are different output modes you can choos from: simple, wide and
There are different output modes you can choose from: simple, wide and
json. The "simple" mode is the default one, it just prints the value
as is. The "wide" mode prints a tabular output similar to the B<list>
subcommand, see there for more details. The options C<-n> and C<-N>
@@ -206,18 +206,18 @@ The B<list> subcommand displays a list of all database entries.
Usage:
Usage:
anydb list [<filter-regex>] [-t <tag>] [-m <mode>] [-n -N] [-T <tpl>] [-i] [flags]
anydb list [<filter-regex> | -t <tag> ] [-m <mode>] [-nNif] [-T <tpl>] [flags]
Aliases:
list, /, ls
list, ls, /, find, search
Flags:
-i, --case-insensitive filter case insensitive
-h, --help help for list
-m, --mode string output format (table|wide|json|template),
wide is a verbose table. (default 'table')
-m, --mode string output format (table|wide|json|template), wide is a verbose table. (default 'table')
-n, --no-headers omit headers in tables
-N, --no-human do not translate to human readable values
-s, --search-fulltext perform a full text search
-t, --tags stringArray tags, multiple allowed
-T, --template string go template for '-m template'
-l, --wide-output output mode: wide
@@ -254,6 +254,10 @@ features.
If you want to search case insensitive, add the option C<-i>.
By default anydb only searches through the keys. If you want to search
through the values as well, then use the C<-s> option, which enables
full-text search.
You can - as with the B<get> command - use other output modes. The
default mode is "table". The "wide" mode is, as already mentioned, a
more detailed table. Also supported is "json" mode and "template"
@@ -315,7 +319,7 @@ Please note, that this does not work with binary content!
=head2 export
Since the bbold database file is not portable across platforms (it is
Since the bbolt database file is not portable across platforms (it is
bound to the endianess of the CPU it was being created on), you might
want to create a backup file of your database. You can do this with
the B<export> subcommand.
@@ -323,7 +327,7 @@ the B<export> subcommand.
Usage:
Usage:
anydb export [-o <json filename>] [flags]
anydb export -o <json filename> [flags]
Aliases:
export, dump, backup
@@ -333,11 +337,11 @@ Usage:
-o, --output string output to file
The database dump is a JSON representation of the whole database and
will be printed to STDOUT by default. Redirect it to a file or use the
C<-o> option:
will be printed to the file specified with the C<-o> option. If you
specify "-" as the filename, it will be written to STDIN.
anydb export > dump.json
anydb export -o dump.json
anydb export -o - > dump.json
Please note, that encrypted values will not be decrypted. This might
change in a future version of anydb.
@@ -350,7 +354,7 @@ dump.
Usage:
Usage:
anydb import [<json file>] [flags]
anydb import -i <json file> [flags]
Aliases:
import, restore
@@ -360,12 +364,13 @@ Usage:
-h, --help help for import
-t, --tags stringArray tags, multiple allowed
By default the C<import> subcommand reads the JSON contents from
STDIN. You might pipe the dump into it or use the option C<-r>:
The C<import> subcommand reads the JSON contents from
the file specified with the C<-i> option. If you specify "-" as the
filename, it will be read from STDIN.
anydb import < dump.json
anydb import -r dump.json
cat dump.json | anydb import
anydb import -i - < dump.json
anydb import -i dump.json
cat dump.json | anydb import -i -
If there is already a database, it will be saved by appending a
timestamp and a new database with the contents of the dump will be
@@ -443,12 +448,20 @@ The following template variables can be used:
=over
=item B<Key> - string
=item B<Value> - string
=item B<Bin> - []byte
=item B<Created> - time.Time
=item B<Tags> - []string
=item B<Encrypted> bool
=item B<.Key> - string
=item B<.Value> - string
=item B<.Bin> - []byte
=item B<.Created> - timestamp.Time
To retrieve a string representation of the timestamp, use C<.Created.AsTime>.
If you need a unix timestamp since epoch, use C<.Created.Unix>.
=item B<.Tags> - []string
=item B<.Encrypted> bool
=back
@@ -463,17 +476,17 @@ Only show the keys of all entries:
Format the list in a way so that is possible to evaluate it in a
shell:
eval $(anydb get foo -m template -T "key='{{ .Key }}' value='{{ .Value }}' ts='{{ .Created}}'")
echo "Key: $key, Value: $value"
eval $(anydb get foo -m template -T "key='{{ .Key }}' value='{{ .Value }}' ts='{{ .Created.AsTime}}'")
echo "Key: $key, Value: $value, When: $ts"
Print the values in CSV format ONLY if they have some tag:
anydb list -m template -T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created}}{{ end }}"
anydb list -m template -T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created.AsTime}}{{ end }}"
=head1 CONFIGURATION
Anydb looks at the following location for a configuration file, in
Anydb looks at the following locations for a configuration file, in
that order:
=over
@@ -486,6 +499,8 @@ that order:
=item or specify one using C<-c>
=back
The configuration format uses the TOML language, refer to
L<https://toml.io/en/> for more details. The key names correspond to
the commandline options in most cases.
@@ -519,7 +534,6 @@ term "bucket." followed by the bucket name. In the example above we
enable encryption for the default bucket "data" and disable it for a
bucket "test". To use different buckets, use the C<-b> option.
=back
=head1 REST API

View File

@@ -20,30 +20,37 @@ import (
"fmt"
"io"
"os"
"strings"
"unicode/utf8"
)
type DbAttr struct {
Key string
Val string
Bin []byte
Preview string
Val []byte
Args []string
Tags []string
File string
Encrypted bool
Binary bool
// conf flags, needed for incoming rest requests
Fulltext bool
}
// check if value is to be read from a file or stdin, setup preview
// text according to flags, lowercase key
func (attr *DbAttr) ParseKV() error {
attr.Key = strings.ToLower(attr.Args[0])
switch len(attr.Args) {
case 1:
// 1 arg = key + read from file or stdin
attr.Key = attr.Args[0]
if attr.File == "" {
attr.File = "-"
}
case 2:
attr.Key = attr.Args[0]
attr.Val = attr.Args[1]
attr.Val = []byte(attr.Args[1])
if attr.Args[1] == "-" {
attr.File = "-"
@@ -51,7 +58,29 @@ func (attr *DbAttr) ParseKV() error {
}
if attr.File != "" {
return attr.GetFileValue()
if err := attr.GetFileValue(); err != nil {
return err
}
}
switch {
case attr.Binary:
attr.Preview = "<binary-content>"
case attr.Encrypted:
attr.Preview = "<encrypted-content>"
default:
if len(attr.Val) > MaxValueWidth {
attr.Preview = string(attr.Val)[0:MaxValueWidth] + "..."
if strings.Contains(attr.Preview, "\n") {
parts := strings.Split(attr.Preview, "\n")
if len(parts) > 0 {
attr.Preview = parts[0]
}
}
} else {
attr.Preview = string(attr.Val)
}
}
return nil
@@ -82,11 +111,12 @@ func (attr *DbAttr) GetFileValue() error {
}
// poor man's text file test
sdata := string(data)
if utf8.ValidString(sdata) {
attr.Val = sdata
attr.Val = data
if utf8.ValidString(string(data)) {
attr.Binary = false
} else {
attr.Bin = data
attr.Binary = true
}
} else {
// read from console stdin
@@ -101,7 +131,7 @@ func (attr *DbAttr) GetFileValue() error {
data += input + "\n"
}
attr.Val = data
attr.Val = []byte(data)
}
return nil

View File

@@ -18,9 +18,9 @@ package app
import (
"crypto/rand"
"encoding/base64"
"errors"
"fmt"
"log/slog"
"os"
"syscall"
@@ -35,7 +35,7 @@ const (
ArgonParallel uint8 = 2
ArgonSaltLen int = 16
ArgonKeyLen uint32 = 32
B64SaltLen int = 22
B64SaltLen int = 16 //22
)
type Key struct {
@@ -85,7 +85,11 @@ func DeriveKey(password []byte, salt []byte) (*Key, error) {
ArgonKeyLen,
)
return &Key{Key: hash, Salt: salt}, nil
key := &Key{Key: hash, Salt: salt}
slog.Debug("derived key", "key", string(key.Key), "salt", string(key.Salt))
return key, nil
}
// Retrieve a random chunk of given size
@@ -104,7 +108,7 @@ func GetRandom(size int, capacity int) ([]byte, error) {
// modifying it.
//
// The cipher text consists of:
// base64(password-salt) + base64(12 byte nonce + ciphertext + 16 byte mac)
// password-salt) + (12 byte nonce + ciphertext + 16 byte mac)
func Encrypt(pass []byte, attr *DbAttr) error {
key, err := DeriveKey(pass, nil)
if err != nil {
@@ -116,47 +120,38 @@ func Encrypt(pass []byte, attr *DbAttr) error {
return fmt.Errorf("failed to create AEAD cipher: %w", err)
}
var plain []byte
if attr.Val != "" {
plain = []byte(attr.Val)
} else {
plain = attr.Bin
}
total := aead.NonceSize() + len(plain) + aead.Overhead()
total := aead.NonceSize() + len(attr.Val) + aead.Overhead()
nonce, err := GetRandom(aead.NonceSize(), total)
if err != nil {
return err
}
cipher := aead.Seal(nonce, nonce, plain, nil)
cipher := aead.Seal(nonce, nonce, attr.Val, nil)
attr.Bin = nil
attr.Val = base64.RawStdEncoding.EncodeToString(key.Salt) +
base64.RawStdEncoding.EncodeToString(cipher)
attr.Val = key.Salt
attr.Val = append(attr.Val, cipher...)
attr.Encrypted = true
attr.Preview = "<encrypted-content>"
slog.Debug("encrypted attr", "salt", string(key.Salt), "cipher", string(attr.Val))
return nil
}
// Do the reverse
func Decrypt(pass []byte, cipherb64 string) ([]byte, error) {
salt, err := base64.RawStdEncoding.Strict().DecodeString(cipherb64[0:B64SaltLen])
if err != nil {
return nil, fmt.Errorf("failed to encode to base64: %w", err)
func Decrypt(pass []byte, cipherb []byte) ([]byte, error) {
if len(cipherb) < B64SaltLen {
return nil, fmt.Errorf("encrypted cipher block too small")
}
key, err := DeriveKey(pass, salt)
key, err := DeriveKey(pass, cipherb[0:B64SaltLen])
if err != nil {
return nil, err
}
cipher, err := base64.RawStdEncoding.Strict().DecodeString(cipherb64[B64SaltLen:])
if err != nil {
return nil, fmt.Errorf("failed to encode to base64: %w", err)
}
cipher := cipherb[B64SaltLen:]
aead, err := chacha20poly1305.New(key.Key)
if err != nil {
@@ -169,5 +164,12 @@ func Decrypt(pass []byte, cipherb64 string) ([]byte, error) {
nonce, ciphertext := cipher[:aead.NonceSize()], cipher[aead.NonceSize():]
return aead.Open(nil, nonce, ciphertext, nil)
clear, err := aead.Open(nil, nonce, ciphertext, nil)
if err != nil {
return nil, err
}
slog.Debug("decrypted attr", "salt", string(key.Salt), "clear", string(clear))
return clear, err
}

306
app/db.go
View File

@@ -20,6 +20,7 @@ import (
"encoding/json"
"errors"
"fmt"
"log/slog"
"os"
"path/filepath"
"regexp"
@@ -27,6 +28,8 @@ import (
"time"
bolt "go.etcd.io/bbolt"
"google.golang.org/protobuf/proto"
timestamppb "google.golang.org/protobuf/types/known/timestamppb"
)
const MaxValueWidth int = 60
@@ -38,17 +41,6 @@ type DB struct {
DB *bolt.DB
}
type DbEntry struct {
Id string `json:"id"`
Key string `json:"key"`
Value string `json:"value"`
Encrypted bool `json:"encrypted"`
Bin []byte `json:"bin"`
Tags []string `json:"tags"`
Created time.Time `json:"created"`
Size int
}
type BucketInfo struct {
Name string
Keys int
@@ -62,33 +54,7 @@ type DbInfo struct {
Path string
}
// Post process an entry for list output.
// Do NOT call it during write processing!
func (entry *DbEntry) Normalize() {
entry.Size = len(entry.Value)
if entry.Encrypted {
entry.Value = "<encrypted-content>"
}
if len(entry.Bin) > 0 {
entry.Value = "<binary-content>"
entry.Size = len(entry.Bin)
}
if strings.Contains(entry.Value, "\n") {
parts := strings.Split(entry.Value, "\n")
if len(parts) > 0 {
entry.Value = parts[0]
}
}
if len(entry.Value) > MaxValueWidth {
entry.Value = entry.Value[0:MaxValueWidth] + "..."
}
}
type DbEntries []DbEntry
type DbEntries []*DbEntry
type DbTag struct {
Keys []string `json:"key"`
@@ -96,11 +62,31 @@ type DbTag struct {
const BucketData string = "data"
func GetDbFile(file string) string {
if file != "" {
return file
}
file = os.Getenv("ANYDB_DB")
if file != "" {
return file
}
home, err := os.UserHomeDir()
if err != nil {
panic(err)
}
return filepath.Join(home, ".config", "anydb", "default.db")
}
func New(file string, bucket string, debug bool) (*DB, error) {
return &DB{Debug: debug, Dbfile: file, Bucket: bucket}, nil
}
func (db *DB) Open() error {
slog.Debug("opening DB", "dbfile", db.Dbfile)
if _, err := os.Stat(filepath.Dir(db.Dbfile)); os.IsNotExist(err) {
if err := os.MkdirAll(filepath.Dir(db.Dbfile), 0700); err != nil {
return err
@@ -120,7 +106,7 @@ func (db *DB) Close() error {
return db.DB.Close()
}
func (db *DB) List(attr *DbAttr) (DbEntries, error) {
func (db *DB) List(attr *DbAttr, fulltext bool) (DbEntries, error) {
if err := db.Open(); err != nil {
return nil, err
}
@@ -130,31 +116,63 @@ func (db *DB) List(attr *DbAttr) (DbEntries, error) {
var filter *regexp.Regexp
if len(attr.Args) > 0 {
// via cli
filter = regexp.MustCompile(attr.Args[0])
}
err := db.DB.View(func(tx *bolt.Tx) error {
if len(attr.Key) > 0 {
// via api
filter = regexp.MustCompile(attr.Key)
}
bucket := tx.Bucket([]byte(db.Bucket))
err := db.DB.View(func(tx *bolt.Tx) error {
root := tx.Bucket([]byte(db.Bucket))
if root == nil {
return nil
}
slog.Debug("opened root bucket", "root", root)
bucket := root.Bucket([]byte("meta"))
if bucket == nil {
return nil
}
err := bucket.ForEach(func(key, jsonentry []byte) error {
slog.Debug("opened buckets", "root", root, "data", bucket)
databucket := root.Bucket([]byte("data"))
if databucket == nil {
return fmt.Errorf("failed to retrieve data sub bucket")
}
err := bucket.ForEach(func(key, pbentry []byte) error {
var entry DbEntry
if err := json.Unmarshal(jsonentry, &entry); err != nil {
return fmt.Errorf("failed to unmarshal from json: %w", err)
if err := proto.Unmarshal(pbentry, &entry); err != nil {
return fmt.Errorf("failed to unmarshal from protobuf: %w", err)
}
if fulltext {
// avoid crash due to access fault
value := databucket.Get([]byte(entry.Key)) // empty is ok
vc := make([]byte, len(value))
copy(vc, value)
entry.Value = string(vc)
}
var include bool
switch {
case filter != nil:
if filter.MatchString(entry.Value) ||
filter.MatchString(entry.Key) ||
if filter.MatchString(entry.Key) ||
filter.MatchString(strings.Join(entry.Tags, " ")) {
include = true
}
if !entry.Binary && !include && fulltext {
if filter.MatchString(string(entry.Value)) {
include = true
}
}
case len(attr.Tags) > 0:
for _, search := range attr.Tags {
for _, tag := range entry.Tags {
@@ -173,7 +191,7 @@ func (db *DB) List(attr *DbAttr) (DbEntries, error) {
}
if include {
entries = append(entries, entry)
entries = append(entries, &entry)
}
return nil
@@ -181,6 +199,7 @@ func (db *DB) List(attr *DbAttr) (DbEntries, error) {
return err
})
return entries, err
}
@@ -192,30 +211,39 @@ func (db *DB) Set(attr *DbAttr) error {
entry := DbEntry{
Key: attr.Key,
Value: attr.Val,
Bin: attr.Bin,
Binary: attr.Binary,
Tags: attr.Tags,
Encrypted: attr.Encrypted,
Created: time.Now(),
Created: timestamppb.Now(),
Size: uint64(len(attr.Val)),
Preview: attr.Preview,
}
// check if the entry already exists and if yes, check if it has
// any tags. if so, we initialize our update struct with these
// tags unless it has new tags configured.
// FIXME: use Get()
err := db.DB.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(db.Bucket))
root := tx.Bucket([]byte(db.Bucket))
if root == nil {
return nil
}
bucket := root.Bucket([]byte("meta"))
if bucket == nil {
return nil
}
jsonentry := bucket.Get([]byte(entry.Key))
if jsonentry == nil {
slog.Debug("opened buckets", "root", root, "data", bucket)
pbentry := bucket.Get([]byte(entry.Key))
if pbentry == nil {
return nil
}
var oldentry DbEntry
if err := json.Unmarshal(jsonentry, &oldentry); err != nil {
return fmt.Errorf("failed to unmarshal from json: %w", err)
if err := proto.Unmarshal(pbentry, &oldentry); err != nil {
return fmt.Errorf("failed to unmarshal from protobuf: %w", err)
}
if len(oldentry.Tags) > 0 && len(entry.Tags) == 0 {
@@ -230,19 +258,41 @@ func (db *DB) Set(attr *DbAttr) error {
return err
}
// marshall our data
pbentry, err := proto.Marshal(&entry)
if err != nil {
return fmt.Errorf("failed to marshall protobuf: %w", err)
}
err = db.DB.Update(func(tx *bolt.Tx) error {
// insert data
bucket, err := tx.CreateBucketIfNotExists([]byte(db.Bucket))
// create root bucket
root, err := tx.CreateBucketIfNotExists([]byte(db.Bucket))
if err != nil {
return fmt.Errorf("failed to create DB bucket: %w", err)
}
jsonentry, err := json.Marshal(entry)
// create meta bucket
bucket, err := root.CreateBucketIfNotExists([]byte("meta"))
if err != nil {
return fmt.Errorf("failed to marshall json: %w", err)
return fmt.Errorf("failed to create DB meta sub bucket: %w", err)
}
err = bucket.Put([]byte(entry.Key), []byte(jsonentry))
slog.Debug("opened/created buckets", "root", root, "data", bucket)
// write meta data
err = bucket.Put([]byte(entry.Key), []byte(pbentry))
if err != nil {
return fmt.Errorf("failed to insert data: %w", err)
}
// create data bucket
databucket, err := root.CreateBucketIfNotExists([]byte("data"))
if err != nil {
return fmt.Errorf("failed to create DB data sub bucket: %w", err)
}
// write value
err = databucket.Put([]byte(entry.Key), attr.Val)
if err != nil {
return fmt.Errorf("failed to insert data: %w", err)
}
@@ -266,21 +316,50 @@ func (db *DB) Get(attr *DbAttr) (*DbEntry, error) {
entry := DbEntry{}
err := db.DB.View(func(tx *bolt.Tx) error {
bucket := tx.Bucket([]byte(db.Bucket))
// root bucket
root := tx.Bucket([]byte(db.Bucket))
if root == nil {
return nil
}
// get meta sub bucket
bucket := root.Bucket([]byte("meta"))
if bucket == nil {
return nil
}
jsonentry := bucket.Get([]byte(attr.Key))
if jsonentry == nil {
// FIXME: shall we return a key not found error?
return nil
slog.Debug("opened buckets", "root", root, "data", bucket)
// retrieve meta data
pbentry := bucket.Get([]byte(attr.Key))
if pbentry == nil {
return fmt.Errorf("no such key: %s", attr.Key)
}
if err := json.Unmarshal(jsonentry, &entry); err != nil {
return fmt.Errorf("failed to unmarshal from json: %w", err)
// put into struct
if err := proto.Unmarshal(pbentry, &entry); err != nil {
return fmt.Errorf("failed to unmarshal from protobuf: %w", err)
}
// get data sub bucket
databucket := root.Bucket([]byte("data"))
if databucket == nil {
return fmt.Errorf("failed to retrieve data sub bucket")
}
// retrieve actual data value
value := databucket.Get([]byte(attr.Key))
if len(value) == 0 {
return fmt.Errorf("no such key: %s", attr.Key)
}
// we need to make a copy of it, otherwise we'll get an
// "unexpected fault address" error
vc := make([]byte, len(value))
copy(vc, value)
entry.Value = string(vc)
return nil
})
@@ -292,7 +371,6 @@ func (db *DB) Get(attr *DbAttr) (*DbEntry, error) {
}
func (db *DB) Del(attr *DbAttr) error {
// FIXME: check if it exists prior to just call bucket.Delete()?
if err := db.Open(); err != nil {
return err
}
@@ -305,6 +383,8 @@ func (db *DB) Del(attr *DbAttr) error {
return nil
}
slog.Debug("opened buckets", "data", bucket)
return bucket.Delete([]byte(attr.Key))
})
@@ -317,7 +397,7 @@ func (db *DB) Import(attr *DbAttr) (string, error) {
return "", err
}
if attr.Val == "" {
if len(attr.Val) == 0 {
return "", errors.New("empty json file")
}
@@ -345,22 +425,43 @@ func (db *DB) Import(attr *DbAttr) (string, error) {
defer db.Close()
err := db.DB.Update(func(tx *bolt.Tx) error {
// insert data
bucket, err := tx.CreateBucketIfNotExists([]byte(db.Bucket))
// create root bucket
root, err := tx.CreateBucketIfNotExists([]byte(db.Bucket))
if err != nil {
return fmt.Errorf("failed to create bucket: %w", err)
return fmt.Errorf("failed to create DB bucket: %w", err)
}
// create meta bucket
bucket, err := root.CreateBucketIfNotExists([]byte("meta"))
if err != nil {
return fmt.Errorf("failed to create DB meta sub bucket: %w", err)
}
slog.Debug("opened buckets", "root", root, "data", bucket)
for _, entry := range entries {
jsonentry, err := json.Marshal(entry)
pbentry, err := proto.Marshal(entry)
if err != nil {
return fmt.Errorf("failed to marshall json: %w", err)
return fmt.Errorf("failed to marshall protobuf: %w", err)
}
err = bucket.Put([]byte(entry.Key), []byte(jsonentry))
// write meta data
err = bucket.Put([]byte(entry.Key), []byte(pbentry))
if err != nil {
return fmt.Errorf("failed to insert data into DB: %w", err)
}
// create data bucket
databucket, err := root.CreateBucketIfNotExists([]byte("data"))
if err != nil {
return fmt.Errorf("failed to create DB data sub bucket: %w", err)
}
// write value
err = databucket.Put([]byte(entry.Key), []byte(entry.Value))
if err != nil {
return fmt.Errorf("failed to insert data: %w", err)
}
}
return nil
@@ -417,3 +518,58 @@ func (db *DB) Info() (*DbInfo, error) {
return info, err
}
func (db *DB) Getall(attr *DbAttr) (DbEntries, error) {
if err := db.Open(); err != nil {
return nil, err
}
defer db.Close()
var entries DbEntries
err := db.DB.View(func(tx *bolt.Tx) error {
// root bucket
root := tx.Bucket([]byte(db.Bucket))
if root == nil {
return nil
}
// get meta sub bucket
bucket := root.Bucket([]byte("meta"))
if bucket == nil {
return nil
}
// get data sub bucket
databucket := root.Bucket([]byte("data"))
if databucket == nil {
return fmt.Errorf("failed to retrieve data sub bucket")
}
slog.Debug("opened buckets", "root", root, "data", bucket)
// iterate over all db entries in meta sub bucket
err := bucket.ForEach(func(key, pbentry []byte) error {
var entry DbEntry
if err := proto.Unmarshal(pbentry, &entry); err != nil {
return fmt.Errorf("failed to unmarshal from protobuf: %w", err)
}
// retrieve the value from the data sub bucket
value := databucket.Get([]byte(entry.Key))
// we need to make a copy of it, otherwise we'll get an
// "unexpected fault address" error
vc := make([]byte, len(value))
copy(vc, value)
entry.Value = string(vc)
entries = append(entries, &entry)
return nil
})
return err
})
return entries, err
}

210
app/dbentry.pb.go Normal file
View File

@@ -0,0 +1,210 @@
// -*-c++-*-
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.1
// protoc v3.21.12
// source: app/dbentry.proto
package app
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
timestamppb "google.golang.org/protobuf/types/known/timestamppb"
reflect "reflect"
sync "sync"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type DbEntry struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id string `protobuf:"bytes,1,opt,name=Id,proto3" json:"Id,omitempty"`
Key string `protobuf:"bytes,2,opt,name=Key,proto3" json:"Key,omitempty"`
Preview string `protobuf:"bytes,3,opt,name=Preview,proto3" json:"Preview,omitempty"`
Tags []string `protobuf:"bytes,4,rep,name=Tags,proto3" json:"Tags,omitempty"`
Created *timestamppb.Timestamp `protobuf:"bytes,5,opt,name=Created,proto3" json:"Created,omitempty"`
Size uint64 `protobuf:"varint,6,opt,name=Size,proto3" json:"Size,omitempty"`
Encrypted bool `protobuf:"varint,7,opt,name=Encrypted,proto3" json:"Encrypted,omitempty"`
Binary bool `protobuf:"varint,8,opt,name=Binary,proto3" json:"Binary,omitempty"`
Value string `protobuf:"bytes,9,opt,name=Value,proto3" json:"Value,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *DbEntry) Reset() {
*x = DbEntry{}
mi := &file_app_dbentry_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *DbEntry) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*DbEntry) ProtoMessage() {}
func (x *DbEntry) ProtoReflect() protoreflect.Message {
mi := &file_app_dbentry_proto_msgTypes[0]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use DbEntry.ProtoReflect.Descriptor instead.
func (*DbEntry) Descriptor() ([]byte, []int) {
return file_app_dbentry_proto_rawDescGZIP(), []int{0}
}
func (x *DbEntry) GetId() string {
if x != nil {
return x.Id
}
return ""
}
func (x *DbEntry) GetKey() string {
if x != nil {
return x.Key
}
return ""
}
func (x *DbEntry) GetPreview() string {
if x != nil {
return x.Preview
}
return ""
}
func (x *DbEntry) GetTags() []string {
if x != nil {
return x.Tags
}
return nil
}
func (x *DbEntry) GetCreated() *timestamppb.Timestamp {
if x != nil {
return x.Created
}
return nil
}
func (x *DbEntry) GetSize() uint64 {
if x != nil {
return x.Size
}
return 0
}
func (x *DbEntry) GetEncrypted() bool {
if x != nil {
return x.Encrypted
}
return false
}
func (x *DbEntry) GetBinary() bool {
if x != nil {
return x.Binary
}
return false
}
func (x *DbEntry) GetValue() string {
if x != nil {
return x.Value
}
return ""
}
var File_app_dbentry_proto protoreflect.FileDescriptor
var file_app_dbentry_proto_rawDesc = []byte{
0x0a, 0x11, 0x61, 0x70, 0x70, 0x2f, 0x64, 0x62, 0x65, 0x6e, 0x74, 0x72, 0x79, 0x2e, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x12, 0x03, 0x61, 0x70, 0x70, 0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65,
0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74,
0x61, 0x6d, 0x70, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0xef, 0x01, 0x0a, 0x07, 0x44, 0x62,
0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x0e, 0x0a, 0x02, 0x49, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28,
0x09, 0x52, 0x02, 0x49, 0x64, 0x12, 0x10, 0x0a, 0x03, 0x4b, 0x65, 0x79, 0x18, 0x02, 0x20, 0x01,
0x28, 0x09, 0x52, 0x03, 0x4b, 0x65, 0x79, 0x12, 0x18, 0x0a, 0x07, 0x50, 0x72, 0x65, 0x76, 0x69,
0x65, 0x77, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x50, 0x72, 0x65, 0x76, 0x69, 0x65,
0x77, 0x12, 0x12, 0x0a, 0x04, 0x54, 0x61, 0x67, 0x73, 0x18, 0x04, 0x20, 0x03, 0x28, 0x09, 0x52,
0x04, 0x54, 0x61, 0x67, 0x73, 0x12, 0x34, 0x0a, 0x07, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x64,
0x18, 0x05, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61,
0x6d, 0x70, 0x52, 0x07, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x53,
0x69, 0x7a, 0x65, 0x18, 0x06, 0x20, 0x01, 0x28, 0x04, 0x52, 0x04, 0x53, 0x69, 0x7a, 0x65, 0x12,
0x1c, 0x0a, 0x09, 0x45, 0x6e, 0x63, 0x72, 0x79, 0x70, 0x74, 0x65, 0x64, 0x18, 0x07, 0x20, 0x01,
0x28, 0x08, 0x52, 0x09, 0x45, 0x6e, 0x63, 0x72, 0x79, 0x70, 0x74, 0x65, 0x64, 0x12, 0x16, 0x0a,
0x06, 0x42, 0x69, 0x6e, 0x61, 0x72, 0x79, 0x18, 0x08, 0x20, 0x01, 0x28, 0x08, 0x52, 0x06, 0x42,
0x69, 0x6e, 0x61, 0x72, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x09,
0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x56, 0x61, 0x6c, 0x75, 0x65, 0x42, 0x1e, 0x5a, 0x1c, 0x67,
0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x74, 0x6c, 0x69, 0x6e, 0x64, 0x65,
0x6e, 0x2f, 0x61, 0x6e, 0x79, 0x64, 0x62, 0x2f, 0x61, 0x70, 0x70, 0x62, 0x06, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x33,
}
var (
file_app_dbentry_proto_rawDescOnce sync.Once
file_app_dbentry_proto_rawDescData = file_app_dbentry_proto_rawDesc
)
func file_app_dbentry_proto_rawDescGZIP() []byte {
file_app_dbentry_proto_rawDescOnce.Do(func() {
file_app_dbentry_proto_rawDescData = protoimpl.X.CompressGZIP(file_app_dbentry_proto_rawDescData)
})
return file_app_dbentry_proto_rawDescData
}
var file_app_dbentry_proto_msgTypes = make([]protoimpl.MessageInfo, 1)
var file_app_dbentry_proto_goTypes = []any{
(*DbEntry)(nil), // 0: app.DbEntry
(*timestamppb.Timestamp)(nil), // 1: google.protobuf.Timestamp
}
var file_app_dbentry_proto_depIdxs = []int32{
1, // 0: app.DbEntry.Created:type_name -> google.protobuf.Timestamp
1, // [1:1] is the sub-list for method output_type
1, // [1:1] is the sub-list for method input_type
1, // [1:1] is the sub-list for extension type_name
1, // [1:1] is the sub-list for extension extendee
0, // [0:1] is the sub-list for field type_name
}
func init() { file_app_dbentry_proto_init() }
func file_app_dbentry_proto_init() {
if File_app_dbentry_proto != nil {
return
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_app_dbentry_proto_rawDesc,
NumEnums: 0,
NumMessages: 1,
NumExtensions: 0,
NumServices: 0,
},
GoTypes: file_app_dbentry_proto_goTypes,
DependencyIndexes: file_app_dbentry_proto_depIdxs,
MessageInfos: file_app_dbentry_proto_msgTypes,
}.Build()
File_app_dbentry_proto = out.File
file_app_dbentry_proto_rawDesc = nil
file_app_dbentry_proto_goTypes = nil
file_app_dbentry_proto_depIdxs = nil
}

20
app/dbentry.proto Normal file
View File

@@ -0,0 +1,20 @@
// -*-c++-*-
syntax = "proto3";
package app;
import "google/protobuf/timestamp.proto";
option go_package = "github.com/tlinden/anydb/app";
message DbEntry {
string Id = 1;
string Key = 2;
string Preview = 3;
repeated string Tags = 4;
google.protobuf.Timestamp Created = 5;
uint64 Size = 6;
bool Encrypted = 7;
bool Binary = 8;
string Value = 9;
}

View File

@@ -26,7 +26,7 @@ import (
"github.com/tlinden/anydb/common"
)
var Version string = "v0.0.7"
var Version string = "v0.2.0"
type BucketConfig struct {
Encrypt bool
@@ -42,6 +42,7 @@ type Config struct {
NoHumanize bool
Encrypt bool // one entry
CaseInsensitive bool
Fulltext bool
Listen string
Buckets map[string]BucketConfig // config file only

View File

@@ -116,12 +116,12 @@ SUBCOMMANDS
anydb set key -f file
Values can be encrypted using ChaCha20Poly1305 when you specify the "-e"
option. Anydb will ask you interactively for a passphrase. You might as
well provide the passphrase using the environment variable
"ANYDB_PASSWORD". To encrypt the value, a cryptographically secure key
will be derived from the passphrase using the ArgonID2 algorithm. Each
value can be encrypted with another passphrase. So, the database itself
is not encrypted, just the values.
option. Anydb will ask you interactively for a passphrase. You can also
provide the passphrase using the environment variable "ANYDB_PASSWORD".
To encrypt the value, a cryptographically secure key will be derived
from the passphrase using the ArgonID2 algorithm. Each value can be
encrypted with another passphrase. So, the database itself is not
encrypted, just the values.
You can supply tags by using the option "-t". Multiple tags can be
provided either by separating them with a comma or by using multiple
@@ -172,7 +172,7 @@ SUBCOMMANDS
decrypt it. If the environment variable "ANYDB_PASSWORD" is set, its
value will be used instead.
There are different output modes you can choos from: simple, wide and
There are different output modes you can choose from: simple, wide and
json. The "simple" mode is the default one, it just prints the value as
is. The "wide" mode prints a tabular output similar to the list
subcommand, see there for more details. The options "-n" and "-N" have
@@ -187,16 +187,18 @@ SUBCOMMANDS
Usage:
Usage:
anydb list [<filter-regex>] [-t <tag>] [-m <mode>] [-n -N] [-T <tpl>] [flags]
anydb list [<filter-regex> | -t <tag> ] [-m <mode>] [-nNif] [-T <tpl>] [flags]
Aliases:
list, /, ls
list, ls, /, find, search
Flags:
-i, --case-insensitive filter case insensitive
-h, --help help for list
-m, --mode string output format (table|wide|json|template), wide is a verbose table. (default 'table')
-n, --no-headers omit headers in tables
-N, --no-human do not translate to human readable values
-s, --search-fulltext perform a full text search
-t, --tags stringArray tags, multiple allowed
-T, --template string go template for '-m template'
-l, --wide-output output mode: wide
@@ -230,6 +232,12 @@ SUBCOMMANDS
note, that this regexp dialect is not PCRE compatible, but supports most
of its features.
If you want to search case insensitive, add the option "-i".
By default anydb only searches through the keys. If you want to search
through the values as well, then use the "-s" option, which enables
full-text search.
You can - as with the get command - use other output modes. The default
mode is "table". The "wide" mode is, as already mentioned, a more
detailed table. Also supported is "json" mode and "template" mode. For
@@ -281,7 +289,7 @@ SUBCOMMANDS
Please note, that this does not work with binary content!
export
Since the bbold database file is not portable across platforms (it is
Since the bbolt database file is not portable across platforms (it is
bound to the endianess of the CPU it was being created on), you might
want to create a backup file of your database. You can do this with the
export subcommand.
@@ -289,7 +297,7 @@ SUBCOMMANDS
Usage:
Usage:
anydb export [-o <json filename>] [flags]
anydb export -o <json filename> [flags]
Aliases:
export, dump, backup
@@ -299,11 +307,11 @@ SUBCOMMANDS
-o, --output string output to file
The database dump is a JSON representation of the whole database and
will be printed to STDOUT by default. Redirect it to a file or use the
"-o" option:
will be printed to the file specified with the "-o" option. If you
specify "-" as the filename, it will be written to STDIN.
anydb export > dump.json
anydb export -o dump.json
anydb export -o - > dump.json
Please note, that encrypted values will not be decrypted. This might
change in a future version of anydb.
@@ -315,7 +323,7 @@ SUBCOMMANDS
Usage:
Usage:
anydb import [<json file>] [flags]
anydb import -i <json file> [flags]
Aliases:
import, restore
@@ -325,12 +333,13 @@ SUBCOMMANDS
-h, --help help for import
-t, --tags stringArray tags, multiple allowed
By default the "import" subcommand reads the JSON contents from STDIN.
You might pipe the dump into it or use the option "-r":
The "import" subcommand reads the JSON contents from the file specified
with the "-i" option. If you specify "-" as the filename, it will be
read from STDIN.
anydb import < dump.json
anydb import -r dump.json
cat dump.json | anydb import
anydb import -i - < dump.json
anydb import -i dump.json
cat dump.json | anydb import -i -
If there is already a database, it will be saved by appending a
timestamp and a new database with the contents of the dump will be
@@ -402,8 +411,16 @@ TEMPLATES
The following template variables can be used:
Key - string =item Value - string =item Bin - []byte =item Created -
time.Time =item Tags - []string =item Encrypted bool
.Key - string
.Value - string
.Bin - []byte
.Created - timestamp.Time
To retrieve a string representation of the timestamp, use
".Created.AsTime". If you need a unix timestamp since epoch, use
".Created.Unix".
.Tags - []string
.Encrypted bool
Prepend a single dot (".") before each variable name.
@@ -415,29 +432,30 @@ TEMPLATES
Format the list in a way so that is possible to evaluate it in a shell:
eval $(anydb get foo -m template -T "key='{{ .Key }}' value='{{ .Value }}' ts='{{ .Created}}'")
echo "Key: $key, Value: $value"
eval $(anydb get foo -m template -T "key='{{ .Key }}' value='{{ .Value }}' ts='{{ .Created.AsTime}}'")
echo "Key: $key, Value: $value, When: $ts"
Print the values in CSV format ONLY if they have some tag:
anydb list -m template -T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created}}{{ end }}"
anydb list -m template -T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created.AsTime}}{{ end }}"
CONFIGURATION
Anydb looks at the following location for a configuration file, in that
Anydb looks at the following locations for a configuration file, in that
order:
"$HOME/.config/anydb/anydb.toml"
"$HOME/.anydb.toml"
"anydb.toml" in the current directory
or specify one using "-c"
The configuration format uses the TOML language, refer to
<https://toml.io/en/> for more details. The key names correspond to
the commandline options in most cases.
Configuration follows a certain precedence: the files are tried to
be read in the given order, followed by commandline options. That
is, the last configuration file wins, unless the user provides a
commandline option, then this setting will be taken.
The configuration format uses the TOML language, refer to
<https://toml.io/en/> for more details. The key names correspond to the
commandline options in most cases.
Configuration follows a certain precedence: the files are tried to be
read in the given order, followed by commandline options. That is, the
last configuration file wins, unless the user provides a commandline
option, then this setting will be taken.
A complete configuration file might look like this:
@@ -456,12 +474,12 @@ CONFIGURATION
[buckets.test]
encrypt = false
Under normal circumstances you don't need a configuration file.
However, if you want to use different buckets, then this might be a
handy option. Buckets are being configured in ini-style with the
term "bucket." followed by the bucket name. In the example above we
enable encryption for the default bucket "data" and disable it for a
bucket "test". To use different buckets, use the "-b" option.
Under normal circumstances you don't need a configuration file. However,
if you want to use different buckets, then this might be a handy option.
Buckets are being configured in ini-style with the term "bucket."
followed by the bucket name. In the example above we enable encryption
for the default bucket "data" and disable it for a bucket "test". To use
different buckets, use the "-b" option.
REST API
The subcommand serve starts a simple HTTP service, which responds to

View File

@@ -20,7 +20,6 @@ import (
"errors"
"os"
"strings"
"unicode/utf8"
"github.com/spf13/cobra"
"github.com/tlinden/anydb/app"
@@ -119,17 +118,13 @@ func Get(conf *cfg.Config) *cobra.Command {
return err
}
clear, err := app.Decrypt(pass, entry.Value)
clear, err := app.Decrypt(pass, []byte(entry.Value))
if err != nil {
return err
}
if utf8.ValidString(string(clear)) {
entry.Value = string(clear)
} else {
entry.Bin = clear
}
entry.Size = uint64(len(entry.Value))
entry.Encrypted = false
}
@@ -188,7 +183,7 @@ func List(conf *cfg.Config) *cobra.Command {
)
var cmd = &cobra.Command{
Use: "list [<filter-regex>] [-t <tag>] [-m <mode>] [-n -N] [-T <tpl>] [-i]",
Use: "list [<filter-regex> | -t <tag> ] [-m <mode>] [-nNis] [-T <tpl>]",
Short: "List database contents",
Long: `List database contents`,
RunE: func(cmd *cobra.Command, args []string) error {
@@ -212,7 +207,7 @@ func List(conf *cfg.Config) *cobra.Command {
conf.Mode = "wide"
}
entries, err := conf.DB.List(&attr)
entries, err := conf.DB.List(&attr, conf.Fulltext)
if err != nil {
return err
}
@@ -227,10 +222,13 @@ func List(conf *cfg.Config) *cobra.Command {
cmd.PersistentFlags().BoolVarP(&conf.NoHeaders, "no-headers", "n", false, "omit headers in tables")
cmd.PersistentFlags().BoolVarP(&conf.NoHumanize, "no-human", "N", false, "do not translate to human readable values")
cmd.PersistentFlags().BoolVarP(&conf.CaseInsensitive, "case-insensitive", "i", false, "filter case insensitive")
cmd.PersistentFlags().BoolVarP(&conf.Fulltext, "search-fulltext", "s", false, "perform a full text search")
cmd.PersistentFlags().StringArrayVarP(&attr.Tags, "tags", "t", nil, "tags, multiple allowed")
cmd.Aliases = append(cmd.Aliases, "/")
cmd.Aliases = append(cmd.Aliases, "ls")
cmd.Aliases = append(cmd.Aliases, "/")
cmd.Aliases = append(cmd.Aliases, "find")
cmd.Aliases = append(cmd.Aliases, "search")
return cmd
}

View File

@@ -23,7 +23,6 @@ import (
"io"
"os"
"os/exec"
"unicode/utf8"
"github.com/spf13/cobra"
"github.com/tlinden/anydb/app"
@@ -38,16 +37,16 @@ func Export(conf *cfg.Config) *cobra.Command {
)
var cmd = &cobra.Command{
Use: "export [-o <json filename>]",
Short: "Export database to json",
Long: `Export database to json`,
Use: "export -o <json filename>",
Short: "Export database to json file",
Long: `Export database to json file`,
RunE: func(cmd *cobra.Command, args []string) error {
// errors at this stage do not cause the usage to be shown
cmd.SilenceUsage = true
conf.Mode = "json"
entries, err := conf.DB.List(&attr)
entries, err := conf.DB.Getall(&attr)
if err != nil {
return err
}
@@ -56,7 +55,10 @@ func Export(conf *cfg.Config) *cobra.Command {
},
}
cmd.PersistentFlags().StringVarP(&attr.File, "output", "o", "", "output to file")
cmd.PersistentFlags().StringVarP(&attr.File, "output-file", "o", "", "filename or - for STDIN")
if err := cmd.MarkPersistentFlagRequired("output-file"); err != nil {
panic(err)
}
cmd.Aliases = append(cmd.Aliases, "dump")
cmd.Aliases = append(cmd.Aliases, "backup")
@@ -70,7 +72,7 @@ func Import(conf *cfg.Config) *cobra.Command {
)
var cmd = &cobra.Command{
Use: "import [<json file>]",
Use: "import -i <json file>",
Short: "Import database dump",
Long: `Import database dump`,
RunE: func(cmd *cobra.Command, args []string) error {
@@ -87,8 +89,11 @@ func Import(conf *cfg.Config) *cobra.Command {
},
}
cmd.PersistentFlags().StringVarP(&attr.File, "file", "r", "", "Filename or - for STDIN")
cmd.PersistentFlags().StringVarP(&attr.File, "import-file", "i", "", "filename or - for STDIN")
cmd.PersistentFlags().StringArrayVarP(&attr.Tags, "tags", "t", nil, "tags, multiple allowed")
if err := cmd.MarkPersistentFlagRequired("import-file"); err != nil {
panic(err)
}
cmd.Aliases = append(cmd.Aliases, "restore")
@@ -199,7 +204,7 @@ func Edit(conf *cfg.Config) *cobra.Command {
return err
}
if len(entry.Value) == 0 && len(entry.Bin) > 0 {
if len(entry.Value) == 0 && entry.Binary {
return errors.New("key contains binary uneditable content")
}
@@ -211,17 +216,12 @@ func Edit(conf *cfg.Config) *cobra.Command {
}
password = pass
clear, err := app.Decrypt(pass, entry.Value)
clear, err := app.Decrypt(pass, []byte(entry.Value))
if err != nil {
return err
}
if utf8.ValidString(string(clear)) {
entry.Value = string(clear)
} else {
entry.Bin = clear
}
entry.Encrypted = false
}
@@ -231,7 +231,7 @@ func Edit(conf *cfg.Config) *cobra.Command {
// save file to a temp file, call the editor with it, read
// it back in and compare the content with the original
// one
newcontent, err := editContent(editor, entry.Value)
newcontent, err := editContent(editor, string(entry.Value))
if err != nil {
return err
}
@@ -241,7 +241,7 @@ func Edit(conf *cfg.Config) *cobra.Command {
Key: attr.Key,
Tags: attr.Tags,
Encrypted: attr.Encrypted,
Val: newcontent,
Val: []byte(newcontent),
}
// encrypt if needed

View File

@@ -19,13 +19,15 @@ package cmd
import (
"errors"
"fmt"
"log/slog"
"os"
"path/filepath"
"runtime/debug"
"github.com/alecthomas/repr"
"github.com/spf13/cobra"
"github.com/tlinden/anydb/app"
"github.com/tlinden/anydb/cfg"
"github.com/tlinden/yadu"
)
func completion(cmd *cobra.Command, mode string) error {
@@ -67,12 +69,6 @@ func Execute() {
Short: "anydb",
Long: `A personal key value store`,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
db, err := app.New(conf.Dbfile, conf.Dbbucket, conf.Debug)
if err != nil {
return err
}
conf.DB = db
var configs []string
if configfile != "" {
@@ -86,9 +82,34 @@ func Execute() {
}
if conf.Debug {
repr.Println(conf)
buildInfo, _ := debug.ReadBuildInfo()
opts := &yadu.Options{
Level: slog.LevelDebug,
AddSource: true,
}
slog.SetLogLoggerLevel(slog.LevelDebug)
handler := yadu.NewHandler(os.Stdout, opts)
debuglogger := slog.New(handler).With(
slog.Group("program_info",
slog.Int("pid", os.Getpid()),
slog.String("go_version", buildInfo.GoVersion),
),
)
slog.SetDefault(debuglogger)
slog.Debug("parsed config", "conf", conf)
}
dbfile := app.GetDbFile(conf.Dbfile)
db, err := app.New(dbfile, conf.Dbbucket, conf.Debug)
if err != nil {
return err
}
conf.DB = db
return nil
},
@@ -114,7 +135,7 @@ func Execute() {
rootCmd.PersistentFlags().BoolVarP(&ShowVersion, "version", "v", false, "Print program version")
rootCmd.PersistentFlags().BoolVarP(&conf.Debug, "debug", "d", false, "Enable debugging")
rootCmd.PersistentFlags().StringVarP(&conf.Dbfile, "dbfile", "f",
filepath.Join(home, ".config", "anydb", "default.db"), "DB file to use")
"", "DB file to use (default: ~/.config/anydb/default.db)")
rootCmd.PersistentFlags().StringVarP(&conf.Dbbucket, "bucket", "b",
app.BucketData, "use other bucket (default: "+app.BucketData+")")
rootCmd.PersistentFlags().StringVarP(&configfile, "config", "c", "", "toml config file")

21
demo/Makefile Normal file
View File

@@ -0,0 +1,21 @@
.PHONY: demo clean check clean-demo
VHS = vhs
clean-demo:
rm -f local.db*
%.gif: %.tape
@echo "vhs $<"
env PATH=..:$(PATH) ANYDB_DB=local.db vhs $<
clean:
rm -vf *.db* *.json
check:
ls -l ../anydb
demo: check clean-demo intro.gif advanced.gif

BIN
demo/advanced.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.9 MiB

182
demo/advanced.tape Normal file
View File

@@ -0,0 +1,182 @@
# -*-sh-*-
Output advanced.gif
Set FontSize 20
Set Width 1000
Set Height 800
Set Theme { "name": "Whimsy", "black": "#535178", "red": "#ef6487", "green": "#5eca89", "yellow": "#fdd877", "blue": "#65aef7", "magenta": "#aa7ff0", "cyan": "#43c1be", "white": "#ffffff", "brightBlack": "#535178", "brightRed": "#ef6487", "brightGreen": "#5eca89", "brightYellow": "#fdd877", "brightBlue": "#65aef7", "brightMagenta": "#aa7ff0", "brightCyan": "#43c1be", "brightWhite": "#ffffff", "background": "#29283b", "foreground": "#b3b0d6", "selection": "#3d3c58", "cursor": "#b3b0d6" }
Set WindowBar Colorful
Set BorderRadius 10
Set Shell zsh
Set FontFamily "IBM Plex Mono"
Set CursorBlink false
Set PlaybackSpeed 1
Set TypingSpeed .05
Hide
Type `PROMPT=''`
Enter
Type "setopt interactivecomments"
Enter
Type "autoload -U colors && colors"
Enter
Type `PS1="%{$fg[magenta]%}demo> %{$reset_color%}"`
Enter
Type "clear"
Enter
Show
Type "# you can assign tags"
Enter
Sleep 1s
Type "anydb set foo bar -t note,important"
Enter
Sleep 3s
Enter
Type "# and filter for them"
Enter
Sleep 1s
Type "anydb list -t important"
Enter
Sleep 3s
Enter
Type "# beside tags filtering you can also use regexps for searching"
Enter
Type "# note, by default the list command only searches through keys"
Enter
Sleep 1s
Type "anydb list '[a-z]+'"
Enter
Sleep 3s
Enter
Type "# do a full text search"
Enter
Sleep 1s
Type "anydb list '[a-z]+' -s"
Enter
Sleep 3s
Enter
Type "# anydb also supports a wide output"
Enter
Sleep 1s
Type "anydb list -m wide"
Enter
Sleep 3s
Enter
Type "# there are shortcuts as well"
Enter
Sleep 1s
Type "anydb ls -l"
Enter
Sleep 2s
Type "anydb /"
Enter
Sleep 3s
Enter
Type "# other outputs are possible as well"
Enter
Sleep 1s
Type "anydb list -m json"
Enter
Sleep 3s
Enter
Type "# you can backup your database"
Enter
Sleep 1s
Type "anydb export -o backup.json"
Enter
Sleep 3s
Enter
Type "# and import it somewhere else"
Enter
Sleep 1s
Type "rm local.db"
Enter
Sleep 1s
Type "anydb ls -l"
Enter
Sleep 1s
Type "anydb import -i backup.json"
Enter
Sleep 1s
Type "anydb ls -l"
Enter
Sleep 3s
Enter
Type "# you can encrypt entries. anydb asks for a passphrase"
Enter
Type "# and will do the same when you retrieve the key using the"
Enter
Type "# get command. anydb will ask you interactively for a password"
Enter
Sleep 1s
Type "anydb set address 'Beatstreet 42' -e"
Enter
Type "pass"
Enter
Sleep 3s
Enter
Type "# but you can provide it via an environment variable too"
Enter
Sleep 1s
Type "ANYDB_PASSWORD=foo anydb set -e secretkey blahblah"
Enter
Sleep 3s
Enter
Type "# using template output mode you can freely design how to print stuff"
Enter
Type "# here, we print the values in CSV format ONLY if they have some tag"
Enter
Type "# also note, that we're printing the creation timestamp as epoch"
Sleep 1s
Type `anydb ls -m template -T "{{ if .Tags }}{{ .Key }},{{ .Value }},{{ .Created.AsTime.Unix}}{{ end }}"`
Enter
Sleep 3s
Enter
Type "# or, to simulate skate's -k or -v"
Enter
Sleep 1s
Type `anydb ls -m template -T "{{ .Key }}"`
Enter
Sleep 1s
Type `anydb ls -m template -T "{{ .Value }}"`
Enter
Sleep 3s
Enter
Type "# maybe you want to digest the item in a shell script? also"
Enter
Type "# note, that both the list and get commands support templates"
Enter
Sleep 1s
Type `eval $(anydb get kitty -m template -T "value='{{ .Value }}'"); echo "value: $value"`
Enter
Sleep 3s
Enter
Type "# sometimes you need to know some details about the current database"
Enter
Type "# add -d for more details"
Enter
Sleep 1
Type "anydb info"
Enter
Sleep 3s
Enter
Type "# Try it out yourself: github.com/tlinden/anydb!"
Enter
Sleep 4s

BIN
demo/intro.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 260 KiB

76
demo/intro.tape Normal file
View File

@@ -0,0 +1,76 @@
# -*-sh-*-
Output intro.gif
Set FontSize 20
Set Width 1000
Set Height 800
Set Theme { "name": "Whimsy", "black": "#535178", "red": "#ef6487", "green": "#5eca89", "yellow": "#fdd877", "blue": "#65aef7", "magenta": "#aa7ff0", "cyan": "#43c1be", "white": "#ffffff", "brightBlack": "#535178", "brightRed": "#ef6487", "brightGreen": "#5eca89", "brightYellow": "#fdd877", "brightBlue": "#65aef7", "brightMagenta": "#aa7ff0", "brightCyan": "#43c1be", "brightWhite": "#ffffff", "background": "#29283b", "foreground": "#b3b0d6", "selection": "#3d3c58", "cursor": "#b3b0d6" }
Set WindowBar Colorful
Set BorderRadius 10
Set Shell zsh
Set FontFamily "IBM Plex Mono"
Set CursorBlink false
Set PlaybackSpeed 1
Set TypingSpeed .05
Hide
Type `PROMPT=''`
Enter
Type "setopt interactivecomments"
Enter
Type "autoload -U colors && colors"
Enter
Type `PS1="%{$fg[magenta]%}demo> %{$reset_color%}"`
Enter
Type "clear"
Enter
Show
Type "# Store something"
Enter
Sleep 1s
Type "anydb set kitty meow"
Enter
Sleep 3s
Enter
Type `# What's in the store?`
Enter
Sleep 1s
Type "anydb ls"
Enter
Sleep 3s
Enter
Type "# Fetch something"
Enter
Sleep 1s
Type "anydb get kitty"
Enter
Sleep 3s
Enter
Type "# Unicode also works, of course"
Enter
Sleep 1s
Type "anydb set 猫咪 喵"
Enter
Sleep 2s
Type "anydb get 猫咪"
Enter
Sleep 3s
Enter
Type "# Do creative things with anydb list"
Enter
Sleep 1s
Type "anydb set penelope marmalade"
Enter
Type "anydb set christian tacos"
Enter
Type "anydb set muesli muesli"
Enter
Type "anydb list | xargs -n 2 printf '%s loves %s.\n'"
Enter
Sleep 3s

45
go.mod
View File

@@ -1,32 +1,41 @@
module github.com/tlinden/anydb
go 1.22.1
go 1.23
toolchain go1.23.5
require (
github.com/alecthomas/repr v0.4.0 // indirect
github.com/andybalholm/brotli v1.1.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/gofiber/fiber/v2 v2.52.5 // indirect
github.com/alecthomas/repr v0.4.0
github.com/dustin/go-humanize v1.0.1
github.com/gofiber/fiber/v2 v2.52.6
github.com/inconshreveable/mousetrap v1.1.0
github.com/olekukonko/tablewriter v0.0.5
github.com/pelletier/go-toml v1.9.5
github.com/rogpeppe/go-internal v1.13.1
github.com/spf13/cobra v1.8.1
go.etcd.io/bbolt v1.4.0
golang.org/x/crypto v0.31.0
golang.org/x/term v0.27.0
google.golang.org/protobuf v1.36.5
)
require (
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/fatih/color v1.16.0 // indirect
github.com/gofiber/fiber/v3 v3.0.0-beta.3 // indirect
github.com/gofiber/utils/v2 v2.0.0-beta.4 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/klauspost/compress v1.17.9 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/klauspost/compress v1.17.11 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.15 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/pelletier/go-toml v1.9.5 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/rivo/uniseg v0.2.0 // indirect
github.com/rogpeppe/go-internal v1.13.1 // indirect
github.com/spf13/cobra v1.8.1 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/spf13/pflag v1.0.6 // indirect
github.com/tlinden/yadu v0.1.3 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasthttp v1.55.0 // indirect
github.com/valyala/tcplisten v1.0.0 // indirect
go.etcd.io/bbolt v1.3.11 // indirect
golang.org/x/crypto v0.31.0 // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/term v0.27.0 // indirect
golang.org/x/sys v0.29.0 // indirect
golang.org/x/tools v0.22.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

28
go.sum
View File

@@ -6,14 +6,20 @@ github.com/andybalholm/brotli v1.0.5 h1:8uQZIdzKmjc/iuPu7O2ioW48L81FgatrcpfFmiq/
github.com/andybalholm/brotli v1.0.5/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
github.com/andybalholm/brotli v1.1.0 h1:eLKJA0d02Lf0mVpIDgYnqXcUn0GqVmEFny3VuID1U3M=
github.com/andybalholm/brotli v1.1.0/go.mod h1:sms7XGricyQI9K10gOSf56VKKWS4oLer58Q+mhRPtnY=
github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA=
github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/asdine/storm/v3 v3.2.1 h1:I5AqhkPK6nBZ/qJXySdI7ot5BlXSZ7qvDY1zAn5ZJac=
github.com/asdine/storm/v3 v3.2.1/go.mod h1:LEpXwGt4pIqrE/XcTvCnZHT5MgZCV6Ub9q7yQzOFWr0=
github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM=
github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE=
github.com/gofiber/fiber/v2 v2.52.5 h1:tWoP1MJQjGEe4GB5TUGOi7P2E0ZMMRx5ZTG4rT+yGMo=
github.com/gofiber/fiber/v2 v2.52.5/go.mod h1:KEOE+cXMhXG0zHc9d8+E38hoX+ZN7bhOtgeF2oT6jrQ=
github.com/gofiber/fiber/v2 v2.52.6 h1:Rfp+ILPiYSvvVuIPvxrBns+HJp8qGLDnLJawAu27XVI=
github.com/gofiber/fiber/v2 v2.52.6/go.mod h1:YEcBbO/FB+5M1IZNBP9FO3J9281zgPAreiI1oqg8nDw=
github.com/gofiber/fiber/v3 v3.0.0-beta.3 h1:7Q2I+HsIqnIEEDB+9oe7Gadpakh6ZLhXpTYz/L20vrg=
github.com/gofiber/fiber/v3 v3.0.0-beta.3/go.mod h1:kcMur0Dxqk91R7p4vxEpJfDWZ9u5IfvrtQc8Bvv/JmY=
github.com/gofiber/utils/v2 v2.0.0-beta.4 h1:1gjbVFFwVwUb9arPcqiB6iEjHBwo7cHsyS41NeIW3co=
@@ -31,11 +37,15 @@ github.com/klauspost/compress v1.17.0 h1:Rnbp4K9EjcDuVuHtd0dgA4qNuv9yKDYKK1ulpJw
github.com/klauspost/compress v1.17.0/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA=
github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw=
github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc=
github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
@@ -43,6 +53,8 @@ github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/Qd
github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZgg3U=
github.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec=
github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY=
github.com/pelletier/go-toml v1.9.5 h1:4yBQzkHv+7BHq2PQUZF3Mx0IYxG7LsP222s7Agd3ve8=
@@ -57,7 +69,11 @@ github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=
github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/tlinden/yadu v0.1.3 h1:5cRCUmj+l5yvlM2irtpFBIJwVV2DPEgYSaWvF19FtcY=
github.com/tlinden/yadu v0.1.3/go.mod h1:l3bRmHKL9zGAR6pnBHY2HRPxBecf7L74BoBgOOpTcUA=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasthttp v1.51.0 h1:8b30A5JlZ6C7AS81RsWjYMQmrZG6feChmgAolCl1SqA=
@@ -67,9 +83,12 @@ github.com/valyala/fasthttp v1.55.0/go.mod h1:NkY9JtkrpPKmgwV3HTaS2HWaJss9RSIsRV
github.com/valyala/tcplisten v1.0.0 h1:rBHj/Xf+E1tRGZyWIWwJDiRY0zc1Js+CV5DqwacVSA8=
github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
github.com/vmihailenco/msgpack v4.0.4+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
go.etcd.io/bbolt v1.3.4/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ=
go.etcd.io/bbolt v1.3.11 h1:yGEzV1wPz2yVCLsD8ZAiGHhHVlczyC9d1rP43/VCRJ0=
go.etcd.io/bbolt v1.3.11/go.mod h1:dksAq7YMXoljX0xu6VF5DMZGbhYYoLUalEiSySYAS4I=
go.etcd.io/bbolt v1.4.0 h1:TU77id3TnN/zKr7CO/uk+fBCwF2jGcMuw2B/FMAzYIk=
go.etcd.io/bbolt v1.4.0/go.mod h1:AsD+OCi/qPN1giOX1aiLAha3o1U8rAz65bvN4j0sRuk=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
@@ -81,6 +100,8 @@ golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU=
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.27.0 h1:WP60Sv1nlK1T6SupCHbXzSaN0b9wUmsPoRS9b61A23Q=
golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -89,6 +110,13 @@ golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGm
golang.org/x/tools v0.22.0 h1:gqSGLZqv+AI9lIQzniJ0nZDRG5GBPsSi+DRNHWNz6yA=
golang.org/x/tools v0.22.0/go.mod h1:aCwcsjqvq7Yqt6TNyX7QMU2enbQ/Gt0bo6krSeEri+c=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/protobuf v1.36.1 h1:yBPeRvTftaleIgM3PZ/WBIZ7XM/eEYAaEyCwvyjq/gk=
google.golang.org/protobuf v1.36.1/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
google.golang.org/protobuf v1.36.4 h1:6A3ZDJHn/eNqc1i+IdefRzy/9PokBTPvcqMySR7NNIM=
google.golang.org/protobuf v1.36.4/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
google.golang.org/protobuf v1.36.5 h1:tPhr+woSbjfYvY6/GPufUoYizxw1cF/yFoxJ2fmpwlM=
google.golang.org/protobuf v1.36.5/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

23
main.go
View File

@@ -19,6 +19,7 @@ package main
import (
"bufio"
"fmt"
"log/slog"
"os"
"runtime"
@@ -27,9 +28,31 @@ import (
)
func main() {
const NoLogsLevel = 100
slog.SetLogLoggerLevel(NoLogsLevel)
Main()
}
func init() {
// if we're running on Windows AND if the user double clicked the
// exe file from explorer, we tell them and then wait until any
// key has been hit, which will make the cmd window disappear and
// thus give the user time to read it.
if runtime.GOOS == "windows" {
if mousetrap.StartedByExplorer() {
fmt.Println("Do no double click kleingebaeck.exe!")
fmt.Println("Please open a command shell and run it from there.")
fmt.Println()
fmt.Print("Press any key to quit: ")
_, err := bufio.NewReader(os.Stdin).ReadString('\n')
if err != nil {
panic(err)
}
}
}
}
func Main() int {
cmd.Execute()
return 0

View File

@@ -33,7 +33,6 @@ import (
)
func List(writer io.Writer, conf *cfg.Config, entries app.DbEntries) error {
// FIXME: call sort here
switch conf.Mode {
case "wide", "", "table":
return ListTable(writer, conf, entries)
@@ -65,8 +64,6 @@ func ListTemplate(writer io.Writer, conf *cfg.Config, entries app.DbEntries) err
buf := bytes.Buffer{}
for _, row := range entries {
row.Normalize()
buf.Reset()
err = tmpl.Execute(&buf, row)
if err != nil {
@@ -94,31 +91,28 @@ func ListTable(writer io.Writer, conf *cfg.Config, entries app.DbEntries) error
}
for _, row := range entries {
row.Normalize()
if conf.Mode == "wide" {
switch conf.NoHumanize {
case true:
table.Append([]string{
row.Key,
strings.Join(row.Tags, ","),
strconv.Itoa(row.Size),
row.Created.Format("02.01.2006T03:04.05"),
row.Value,
strconv.FormatUint(row.Size, 10),
row.Created.AsTime().Format("02.01.2006T03:04.05"),
row.Preview,
})
default:
table.Append([]string{
row.Key,
strings.Join(row.Tags, ","),
humanize.Bytes(uint64(row.Size)),
//row.Created.Format("02.01.2006T03:04.05"),
humanize.Time(row.Created),
row.Value,
humanize.Time(row.Created.AsTime()),
row.Preview,
})
}
} else {
table.Append([]string{row.Key, row.Value})
table.Append([]string{row.Key, row.Preview})
}
}

View File

@@ -22,7 +22,6 @@ import (
"io"
"os"
"reflect"
"strings"
"github.com/dustin/go-humanize"
"github.com/tlinden/anydb/app"
@@ -40,16 +39,15 @@ func Print(writer io.Writer, conf *cfg.Config, attr *app.DbAttr, entry *app.DbEn
switch conf.Mode {
case "simple", "":
if len(entry.Bin) > 0 {
if entry.Binary {
if isatty {
fmt.Println("binary data omitted")
} else {
os.Stdout.Write(entry.Bin)
os.Stdout.WriteString(entry.Value)
}
} else {
fmt.Print(entry.Value)
if !strings.HasSuffix(entry.Value, "\n") {
fmt.Print(string(entry.Value))
if entry.Value[entry.Size-1] != '\n' {
// always add a terminal newline
fmt.Println()
}
@@ -62,9 +60,9 @@ func Print(writer io.Writer, conf *cfg.Config, attr *app.DbAttr, entry *app.DbEn
fmt.Println(string(jsonentry))
case "wide":
return ListTable(writer, conf, app.DbEntries{*entry})
return ListTable(writer, conf, app.DbEntries{entry})
case "template":
return ListTemplate(writer, conf, app.DbEntries{*entry})
return ListTemplate(writer, conf, app.DbEntries{entry})
}
return nil
@@ -77,7 +75,6 @@ func WriteFile(writer io.Writer, conf *cfg.Config, attr *app.DbAttr, entry *app.
if attr.File == "-" {
fileHandle = os.Stdout
} else {
fd, err := os.OpenFile(attr.File, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0755)
if err != nil {
return fmt.Errorf("failed to open file %s for writing: %w", attr.File, err)
@@ -87,17 +84,14 @@ func WriteFile(writer io.Writer, conf *cfg.Config, attr *app.DbAttr, entry *app.
fileHandle = fd
}
if len(entry.Bin) > 0 {
// binary file content
_, err = fileHandle.Write(entry.Bin)
} else {
val := entry.Value
if !strings.HasSuffix(val, "\n") {
// always add a terminal newline
val += "\n"
}
// actually write file content
_, err = fileHandle.WriteString(entry.Value)
_, err = fileHandle.Write([]byte(val))
if !entry.Binary {
if entry.Value[entry.Size-1] != '\n' {
// always add a terminal newline
_, err = fileHandle.Write([]byte{'\n'})
}
}
if err != nil {

View File

@@ -44,7 +44,6 @@ func RestList(c *fiber.Ctx, conf *cfg.Config) error {
attr := new(app.DbAttr)
if len(c.Body()) > 0 {
if err := c.BodyParser(attr); err != nil {
return c.Status(fiber.StatusUnprocessableEntity).JSON(fiber.Map{
"errors": err.Error(),
@@ -54,7 +53,7 @@ func RestList(c *fiber.Ctx, conf *cfg.Config) error {
}
// get list
entries, err := conf.DB.List(attr)
entries, err := conf.DB.List(attr, attr.Fulltext)
if err != nil {
return JsonStatus(c, fiber.StatusForbidden,
"Unable to list keys: "+err.Error())

View File

@@ -42,6 +42,11 @@ func Runserver(conf *cfg.Config, args []string) error {
return RestList(c, conf)
})
api.Post("/", func(c *fiber.Ctx) error {
// same thing as above but allows to supply parameters, see app.Dbattr{}
return RestList(c, conf)
})
api.Get("/:key", func(c *fiber.Ctx) error {
return RestGet(c, conf)
})

37
t/crypt.txtar Normal file
View File

@@ -0,0 +1,37 @@
#
# Copyright © 2025 Thomas von Dein
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# encrypt something
exec env ANYDB_PASSWORD=12345 anydb -f test.db set -e secret eshishinusan
# retrieve it
exec env ANYDB_PASSWORD=12345 anydb -f test.db get secret
stdout eshishinusan
# but has it really been encrypted?
! exec env ANYDB_PASSWORD=8d8d8 anydb -f test.db get secret
! stdout eshishinusan
stderr 'message authentication failed'
# what about the listing
exec anydb -f test.db ls -l
stdout 'encrypted-content'
! stdout eshishinusan
# and the export?
exec anydb -f test.db export -o -
! stdout eshishinusan

View File

@@ -32,3 +32,20 @@ stdout 50
# look if it's inside the db
exec anydb -f test.db ls
stdout datum.*binary-content
# do the same thing with text content, start with a new text entry
exec anydb -f test.db set feed alpha
# which we write to a file
exec anydb -f test.db get feed -o out2.txt
exists out2.txt
# check if its filled (5 bytes + newline)
exec ls -l out2.txt
stdout 6
# compare content
exec cat out2.txt
stdout alpha

View File

@@ -23,10 +23,10 @@ exec anydb -f test.db export -o backup.json
stdout 'database contents exported to backup.json'
# import into new db
exec anydb -f new.db import -r backup.json
exec anydb -f new.db import -i backup.json
stdout 'imported.*entries'
# check contents
exec anydb -f new.db list
exec anydb -f new.db list bar -s
stdout foo.*bar

View File

@@ -37,12 +37,12 @@ exec anydb -f test.db list -t flower
! stdout bar
# list with filter
exec anydb -f test.db list b.r
exec anydb -f test.db list b.r -s
stdout bar
# list with -i filter
exec anydb -f test.db list -i mucha
stdout MUCHA
exec anydb -f test.db list -is mucha
stdout mucha
# get single entry
exec anydb -f test.db get color