added JSON export format:

Starting with version 1.4 JSON export format will be supported but
not be enabled by default. However a deprecation notice will appear if
a user still uses YAML format. YAML support will be removed in version
1.5.

Fixes https://github.com/TLINDEN/note/issues/10.
This commit is contained in:
2023-08-13 19:05:29 +02:00
parent 1b842625ed
commit 37e73950da
6 changed files with 431 additions and 344 deletions

223
Changelog
View File

@@ -1,94 +1,153 @@
================================================================================
1.3.26: fixed installer, on newer perls NOTEDB/* is being ignored
================================================================================
1.3.25: fixed indents
================================================================================
1.3.24: removed bashism in mysql installer.
================================================================================
1.3.23: defect distritbution. do not use.
================================================================================
1.3.22: fixed bug in mysql backend.
-*-text-*-
1.4.0:
Added support for JSON encoded export files instead of YAML.
Currently this is just an option, which can be set in the config
with UseJSON or commandline -j. The old export format is now
being considered deprecated (a warning will be shown) and will
be removed in 1.5. Fixes https://github.com/TLINDEN/note/issues/10.
A helper script has been added to help to combine two notedb's
into one: contrib/sync-combine-two-yaml-exports-into-one.pl. It is
still using YAML mode, JSON support will be added in the future.
1.3.26:
fixed installer, on newer perls NOTEDB/* is being ignored
1.3.25:
fixed indents
1.3.24:
removed bashism in mysql installer.
1.3.23:
defect distritbution. do not use.
1.3.22:
fixed bug in mysql backend.
added retry feature for NOTEDB::pwsafe3 backend save() password
entering. bails out after 5 retries.
================================================================================
1.3.21:
Changed note id generation in NOTEDB::pwsafe3::_uuid(), again.
Instead of crc checksums and the loop to avoid duplicates, I just
use a counter and sort the entries by ctime, so that older entries
keep their note id. Also this should fix a rare bug, where the
code hangs because of said loop.
================================================================================
1.3.20:
fixed bug in NOTEDB::pwsafe3::_retrieve(), it iterated over the
records unsorted which resulted in different note ids each time
the program runs.
================================================================================
1.3.19:
revert fix in NOTEDB::pwsafe3::filechanged(), use > again.
================================================================================
1.3.18:
fixed unit tests and added more (lots of them)
fixed several bugs in backend modules, which I found during
writing of the unit tests.
================================================================================
1.3.17:
applied patch by Bill Carlson, which fixes string length issues
in non-interactive search mode.
fixed usage text (-h) so that it now contains the available
options.
================================================================================
1.3.16:
fixed checking of encrypted notes when encryption is turned off.
instead of checking note id 1, we now check the first entry,
whatever id it may have.
================================================================================
1.3.15:
fixed bug in NOTEDB::pwsafe3 backend, it converted the date
of a note entry into the wrong formatted timestamp.
================================================================================
1.3.14:
fixed bug in NOTEDB::pwsafe3 backend, it used lockging on the
database file for reading, which is wrong.
================================================================================
1.3.13:
ADDED: new config parameter 'motd', a note entry which will be shown
on startup (if exists).
CHANGED: the prompt will now show if the current instance is running
in readonly mode.
================================================================================
1.3.12:
FIXED: NOTEDB::general data backend module did overwrite records if
there were some deleted ones in the database.
================================================================================
1.3.11:
FIXED: NOTEDB::pwsafe3 contained the old python stuff in delete code.
================================================================================
1.3.10:
FIXED: the YAML export/import change were missing too. grml...
FIXED: NOTEDB::general backend fixed loading Config::General
================================================================================
1.3.9:
FIXED: bin/note were missing. for whatever reason, I don't know.
================================================================================
1.3.8:
ADDED: New backend added: NOTEDB::pwsafe3, which adds support to store
notes in a Password Safe v3 database.
FIXED: -d didn't work, because of a typo in mode assignment.
================================================================================
1.3.7:
ADDED: added ticket feature, which adds a unique id to each
new note, which persists during imports/exports. the
id is a randomly generated string.
================================================================================
1.3.6:
ADDED: Added test cases for "make test"
ADDED: Added test for optional and required perl modules in
Makefile.PL
FIXED: NOTEDB::dumper version string were wrong, therefore
cpan didn't index is properly.
================================================================================
1.3.5:
FIXED: Applied patch by Elmar Loos which fixes misbehavior for
-t and -T (identical output)
@@ -108,24 +167,31 @@ CHANGED: default config and default settings without config have
ADDED: New interactive mode command: "c". It is now possible to
change note's behavior at runtime. No database related
parameters can be modified.
================================================================================
1.3.4:
ADDED: Each note now contains a "ticket number" which identifies
it against other notes. Those tickets are not changing when
notes will be reorganized.
ADDED: added support for less to view note entries instead of
just printing it to STDOUT. (interactive mode).
================================================================================
1.3.3:
ADDED: new configfile parameter PrintLines (default: YES), which
controls wether listings are separated by horizontal lines.
================================================================================
1.3.2:
FIXED: NOTEDB::mysql backend parameter fixed (dbtype)
FIXED: NOTEDB::mysql didn't fetch topics correctly
FIXED: NOTEDB::text didn't correctly return last highest note id
================================================================================
1.3.1:
FIXED: most config variables not related to drivers had
invalid sentence in note, so the new ones of the new config
@@ -136,7 +202,9 @@ FIXED: fixed handling of NOTEDB::crypt_supported, now encryption
ADDED: NOTEDB::text now supports internal caching too.
CHANGED: lock() sets the umask internally to 022, so that other
users are able to read the lockfile.
================================================================================
1.3.0:
ADDED: new config option: ReadOnly
ADDED: new database backend: NOTEDB::text, which uses the Storable
@@ -156,35 +224,35 @@ CHANGED: the configuration variables are now stored in a hash, this
CHANGED: the libpath variable has been removed, it didnt't work either.
use now .. instead, so that a local installation for a non
root user is still possible.
================================================================================
1.2.6:
FIXED: the binary driver (NOTEDB::binary) encounters now if a note
entry is bigger then MaxNoteByte. It prints the overlapping
part to STDERR, and a warning message and finally saves
the complete, unchanged note entry to an external text file.
================================================================================
1.2.5:
FIXED: removed any file/path actions using '/'. replaced by portable
functions by using File::Spec. This makes it possible to run
note unchanged on win32 (and possibly any other) environments.
FIXED: added a whitespace to the prompt in interactive mode to
circumvent a bug in the win32 Term::ReadLine module which causes
the cursor to be displayed on the left side (column 0) of
the screen.
FIXED: added "or die" code to some commands which are running inside
an eval{} block to fetch errors. Without the "or die"s no
error could ever catched.
CHANGED: removed HOME variable support of the noterc. in fact, if it
exists, no error will occur, but it will no longer be used.
It didn't work in older versions anyway.
ADDED: It is now possible to quit note using CTRL-D (or: EOF)
================================================================================
1.2.4:
CHANGED: in the function find_editor() the alternatives vim and pico
has been removed because they would never had a match.
@@ -197,7 +265,8 @@ ADDED: if the config variable FormatText is set to 'simple' then
be displayed formatted, instead of two.
ADDED: added Term::ReadLine support (auto-completion and history).
================================================================================
1.2.3:
ADDED: if FormatText is enabled one can now use a new special format
tag: //. If a text is surrounded by two slashes, i.e.: //blah//
@@ -206,7 +275,8 @@ ADDED: if FormatText is enabled one can now use a new special format
since no nobody can grab you password by looking at your
monitor, but you can copy&paste it.
================================================================================
1.2.2:
FIXED: oneliner note entries caused breaked displaying in interactive
mode.
@@ -221,7 +291,8 @@ ADDED: new config variable AutoClear, which is turned on by default,
which controls wether the screen shall be cleared after each
item (display, list and so on).
================================================================================
1.2.1:
CHANGED: added the correct installation instructions to the README file.
REMOVED: removed the usage guidelines from te README file. This information
@@ -234,7 +305,8 @@ ADDED: topics can now be abbreviated in interactive mode, which
match could be found, otherwise the available topics that
matches will be suggested.
================================================================================
1.2.0:
CHANGED: using Makemaker instead of self-written code in Makefile.PL
for installation. No more dependency checks built-in because
@@ -264,14 +336,14 @@ CHANGED: cosmetics.
NOTE: increased minor version number from 1 to 2 to indicate that
development begun after 2 1/2 years pause again :-)
================================================================================
1.1.2:
FIXED: Empty notes will no longer stored.
ADDED: A new config option which allows you to specify a time format
other than the default one which is used by note.
================================================================================
1.1.1:
FIXED: Some odd typos in README and note.pod.
@@ -285,7 +357,7 @@ CHANGED: the default colors are now visible both on black and white
ADDED: two more color values: <white_black> and <bold>.
CHANGED: the color hash is now in ::main instead of ::C.
================================================================================
1.1.0:
CHANGED: does no more use the external touch command to create a new
@@ -320,11 +392,11 @@ CHANGED: oop-ized and re-indented the modules dbm.pm, mysql.pm and
binary.pm.
ADDED: You can now specify a port for the mysql backend ("DbPort").
================================================================================
1.0.9: jumped directly to 1.1.0, too many additions.
================================================================================
1.0.8:
FIXED: typo in noterc shipped with package may caused confusion
@@ -337,14 +409,14 @@ CHANGED: changed the way note manages temporary filez. It uses now a
ADDED: A new config option allows the user to specify her own
temp-directory. The default is still /tmp.
================================================================================
1.0.7:
FIXED: there was a bug in the search expression, use now \Q and \E.
ADDED: --config <file> allows one to use another config than the
default.
================================================================================
1.0.6:
FIXED: there were some odd bugs in commandline parsing, some options
@@ -357,14 +429,14 @@ CHANGED: Optimized a little bit the output routine, now it is better to
ADDED: sub format and appropriate config-option for text formatting
capabilities.
CHANGED: changed getconfig regexp, which allows now also to use
Option = Param.
Option Param.
FIXED: was not possible to override config-options, which are set by
default to something.
ADDED: note checks now, if a database os actually really encrypted and
exits with an error if it s and the user turned off encryption.
This protects her from destroying it's own database ..
================================================================================
1.0.5:
FIXED: the T (and t respectively) printed nothing out since 1.0.3!
@@ -385,7 +457,6 @@ FIXED: Removed Getopt::Long option "bundling", causes errors with perl
FIXED: Added the Getopt::Long option "no_ignore_case". In 1.0.4 options
were case insensitive causing -i to be interpreted as --import
instead of --interactive ;-(((
Thanks to Peter Palmreuter for the following fixed/additions:
ADDED: a new config option $DEFAULT_LIST, which causes note,
if turned to "LONG", to use long-listing as default.
@@ -397,14 +468,14 @@ FIXED: sub search prints now an appropriate error-message in
CHANGED: Changed the text in the interactive help to reflect
changes of verion 1.0.3 (t and T).
================================================================================
1.0.4:
CHANGED: Moved from @ARGV-parsing to Getopt::Long, adding options is now
much easier and I do now understand my own code ;-)
ADDED: --raw, the "Raw Mode", which turns off any formatting of output.
================================================================================
1.0.3:
ADDED: "-" works also for --dump, but in the other direction. It causes
@@ -424,7 +495,7 @@ ADDED: -o commandline switch, which causes note to overwrite an
ADDED: the long-tree-view (-T) displays now also the note-number of
each note.
================================================================================
1.0.2:
ADDED: Topic-Tree overview command (-t or -T).
@@ -433,20 +504,20 @@ ADDED: Enhanced list command in interactive mode, you can now specify
CHANGED: updated the help and usage sections to reflect the additions
above.
================================================================================
1.0.1:
FIXED: fixed bug in NOTEDB::mysql, which caused note to store NULL
values in db, if encryption was off. A really dump failure :-(
================================================================================
1.0.0:
CHANGED: removed install.sh. use now a Makefile for installation.
ADDED: Encryption support. Note can now encrypt notes using IDEA
or DES as encryption-protocols(symetric).
================================================================================
0.9:
FIXED: There were many new bugs after my last changes *grrrrr*. fixed.
@@ -467,7 +538,6 @@ FIXED: there was a bug in NOTEDB::binary which caused not to recount
note numbers after deleting one :-(
================================================================================
0.8:
ADDED: NOTEDB::binary. so now 0.8 is ready for shipping !
@@ -492,7 +562,6 @@ FIXED: width of listings is now always the same independent of the
stringlength of a certain note.
================================================================================
0.7:
ADDED: one can now use the unix-like "cd" command to change to another
@@ -506,9 +575,6 @@ ADDED: a new config option "$KEEP_TIMESTAMP" allows a user to disable
================================================================================
0.6:
FIXED: oops - the new suptopic feature confused the commandline-mode of
note! quickly corrected! so subtopics also available from
@@ -539,10 +605,6 @@ FIXED: A bug at line 769 causing single note where smaller than note-
================================================================================
0.5:
ADDED: Topic support(requested). You can sort the various notes under
different topics now.
@@ -551,10 +613,6 @@ FIXED: There was another bug, which caused the list command to display
================================================================================
0.4.2:
ADDED: If run in interactive mode, note will at first do a list
command.
@@ -566,10 +624,6 @@ FIXED: Some more vars will be resetted during each loop in interactive
================================================================================
0.4.1:
ADDED: The install.sh script for the mysql version is no able to
install the required Mysql module directly from CPAN, thanks
@@ -589,10 +643,6 @@ FIXED: the function num_bereich() had a bug, which caused ot to ignore under
================================================================================
0.4:
CHANGED: ok, mysql support is back again (upon requests). therefore there
are two different version of the script in the same time with
@@ -609,10 +659,6 @@ ADDED: sub num_bereich(), which allows one to specify more then one
================================================================================
0.3:
CHANGED: it uses no more a mysql database, but a binary file instead.
This is much faster!
@@ -620,8 +666,6 @@ ADDED: note can display the notes with colors, it is turned off by
default
================================================================================
0.2:
FIXED: now any occurence of ' will be masked with \' before storage
@@ -631,7 +675,6 @@ FIXED: now numbers of notes will be recounted, if one delete one note,
CHANGED: the look of the list output has been changed, similar to a table
================================================================================
0.1:
INITIAL RELEASE.

9
README
View File

@@ -1,4 +1,4 @@
note 1.3.26 by T.v.Dein (09/02/2015)
note 1.4.0 by T.v.Dein (13/08/2023)
=======================================
Introduction
@@ -104,6 +104,11 @@ You need the following things:
you want to use the auto-completion and history functionality.
o Config::General if you want to use the NOTEDB::general
backend.
o YAML is needed to create backups using -D. Please note,
that this format is deprecated starting with 1.4.0. The
Support will be removed in 1.5.0. Please switch to JSON
format as soon as possible, either by using the -j
commandline option or the UseJSON configuration value.
Installation
@@ -207,4 +212,4 @@ and I'll add you.
Last changed
============
09/02/2015
13/08/2023

View File

@@ -1 +1 @@
1.3.26
1.4.0

View File

@@ -1,7 +1,7 @@
#!/usr/bin/perl
#
# note - console notes management with database and encryption support.
# Copyright (C) 1999-2017 T.v.Dein (see README for details!)
# Copyright (C) 1999-2023 T.v.Dein (see README for details!)
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
@@ -38,6 +38,7 @@ use Getopt::Long;
use FileHandle;
use File::Spec;
use YAML;
use JSON::PP;
#
@@ -71,6 +72,7 @@ my (
$opt_, $opt_i, $opt_r, $opt_e, $opt_d, $opt_enc,
$opt_s, $opt_t, $opt_T, $opt_l, $opt_L, $opt_c,
$opt_D, $opt_I, $opt_o, $opt_h, $opt_n, $opt_v,
$opt_j,
#
# set from commandline (or interactive)
@@ -133,7 +135,8 @@ my (
'printlines' => 0,
'cache' => 0,
'preferrededitor' => '',
'motd' => ''
'motd' => '',
'usejson' => 0, # will be the default in the future
);
# these are not customizable at runtime!
@@ -141,7 +144,7 @@ $hardparams = "(readonly|maxlen|dbdriver|useencryption|cryptmethod)";
$CONF = File::Spec->catfile($ENV{HOME}, ".noterc");
$USER = getlogin || getpwuid($<); chomp $USER;
$TOPIC = 1;
$VERSION = "1.3.26";
$VERSION = "1.4.0";
$CurDepth = 1; # the current depth inside the topic "directory" structure...
$maxlen = "auto";
$timelen = 22;
@@ -214,6 +217,7 @@ else {
"list|l:s" => \$opt_l, # string, optional
"longlist|L:s" => \$opt_L, # string, optional
"dump|Dump|D:s" => \$opt_D, # string, optional
"json|j" => \$opt_j, # bool, optional
"import|Import|I:s" => \$opt_I, # string, optional
"overwrite|o!" => \$opt_o, # no arg
"help|h|?!" => \$opt_h, # no arg
@@ -276,6 +280,10 @@ else {
else {
$dump_file = "-"; # use STDIN
}
if (defined $opt_j) {
$conf{usejson} = 1; # force JSON
}
}
elsif (defined $opt_I) {
$mode = "import";
@@ -923,6 +931,7 @@ sub edit {
sub dump {
my(%res, $num, $DUMP);
# $dump_file
if ($dump_file eq "-") {
$DUMP = *STDOUT;
@@ -931,8 +940,11 @@ sub dump {
open (DUMPFILE, ">$dump_file") or die "could not open $dump_file\n";
$DUMP = *DUMPFILE;
}
select $DUMP;
%res = $db->get_all();
# FIXME: prepare hashing in NOTEDB class
foreach $num (sort { $a <=> $b } keys %res) {
print STDOUT "dumping note number $num to $dump_file\n" if($dump_file ne "-");
@@ -947,7 +959,17 @@ sub dump {
my $date = $res{$num}->{date};
$res{$num} = { body => $body, title => $title, path => $path, date => $date};
}
if($conf{usejson}) {
my $json = JSON::PP->new->utf8->pretty;
print $json->encode(\%res);
}
else {
warn "Deprecation notice: YAML export format will not be supported in the future!
Enable JSON using the UseJSON config parameter or the -j commandline parameter!";
print Dump(\%res);
}
close(DUMPFILE);
select STDOUT;
}
@@ -964,9 +986,16 @@ sub import {
$DUMP = *DUMPFILE;
}
my $yaml = join '', <$DUMP>;
my $serialized = join '', <$DUMP>;
my $res = Load($yaml);
my $res;
if($serialized =~ /^\{/) {
$res = decode_json($serialized);
}
else {
$res = Load($serialized);
}
foreach my $number (keys %{$res}) {
my $note;

View File

@@ -283,6 +283,12 @@ AddTicket = NO
motd =
#
# Enable JSON formatted backups. The option will be removed
# in version 1.5 and become the only available option.
# In earlier versions: if unset or set to NO, YAML will be used.
UseJSON = YES
#
#
# That's all about it for now.

View File

@@ -77,6 +77,10 @@ the specified ones.
Dumps all notes to the textfile <file>. If <file> is a "-" it will
be printed out to standard output (STDOUT).
=item I<-j --json>
Use JSON format for exports created using -D. The importer determines
the format to be used automatically.
=item I<-I, --Import file | ->
@@ -534,6 +538,6 @@ T.v.Dein <tlinden@cpan.org>
=head1 VERSION
1.3.26
1.4.0
=cut