version bump

now use a const struct via func.

.github/workflows/ci.yaml

@@ -23,3 +23,36 @@ jobs:
 
     - name: test
       run: make test
+
+  golangci:
+    name: lint
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/setup-go@v3
+        with:
+          go-version: 1.17
+      - uses: actions/checkout@v3
+      - name: golangci-lint
+        uses: golangci/golangci-lint-action@v3
+        #with:
+          # Optional: version of golangci-lint to use in form of v1.2 or v1.2.3 or `latest` to use the latest version
+          # version: v1.29
+
+          # Optional: working directory, useful for monorepos
+          # working-directory: somedir
+
+          # Optional: golangci-lint command line arguments.
+          # args: --issues-exit-code=0
+
+          # Optional: show only new issues if it's a pull request. The default value is `false`.
+          # only-new-issues: true
+
+          # Optional: if set to true then the all caching functionality will be complete disabled,
+          #           takes precedence over all other caching options.
+          # skip-cache: true
+
+          # Optional: if set to true then the action don't cache or restore ~/go/pkg.
+          # skip-pkg-cache: true
+
+          # Optional: if set to true then the action don't cache or restore ~/.cache/go-build.
+          # skip-build-cache: true

CHANGELOG.md

@@ -0,0 +1,252 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org).
+
+## [v1.0.12](https://github.com/TLINDEN/tablizer/tree/v1.0.13) - 2022-11-03
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.12...v1.0.13)
+
+### Added
+
+- Added command line flag to generate shell completion code
+
+- Added an animated demo gif to the README to demonstrate the tool
+
+### Fixed
+
+- The `-A` flag wasn't implemented (default output mode).
+
+
+
+
+## [v1.0.12](https://github.com/TLINDEN/tablizer/tree/v1.0.12) - 2022-10-25
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.11...v1.0.12)
+
+### Added
+
+- Added support to parse CSV input
+
+- Added CSV output support
+
+- Added support for environment variables
+
+### Changed
+
+- We do not use the generated help message anymore, instead we use the
+  usage from the  manpage, which we have to maintain  anyway. It looks
+  better and has flag groups, which  cobra is still lacking as of this
+  writing.
+  
+-  More refactoring  and  re-organization,  runtime configuration  now
+  lives in the cfg module.
+  
+  
+### Fixed
+
+- Fixed  [Bug #5](https://github.com/TLINDEN/tablizer/issues/5), where
+  matches have not been highlighted correctly in some rare cases.
+
+
+
+## [v1.0.11](https://github.com/TLINDEN/tablizer/tree/v1.0.11) - 2022-10-19
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.10...v1.0.11)
+
+### Added
+
+- Added CI job golinter to regularly check for common mistakes.
+
+- Added YAML output mode.
+
+- Added more unit tests, we're over 95% in the lib module.
+
+### Changed
+
+- do  not use any global  variables anymore, makes the  code easier to
+  maintain, understand and test
+
+- using io.Writer  in print* functions, which is easier  to test, also
+  re-implemented the print tests.
+
+- replaced go-str2duration with my own implementation `duration2int()`.
+
+
+
+## [v1.0.10](https://github.com/TLINDEN/tablizer/tree/v1.0.10) - 2022-10-15
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.9...v1.0.10)
+
+### Added
+
+- Added various sort modes: sort by time, by duration, numerical (-a -t -i)
+
+- Added possibility to modify sort order to descending (-D)
+
+- Added support  to specify a regexp in column  selector -c, which can
+  also be mixed with numerical column spec
+  
+- More unit tests
+
+### Fixed
+
+- Column specification  allowed to specify duplicate  columns like `-c
+  1,2,1,2` unchecked. Now this list will be deduplicated before use.
+
+
+
+## [v1.0.9](https://github.com/TLINDEN/tablizer/tree/v1.0.9) - 2022-10-14
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.8...v1.0.9)
+
+### Added
+
+- Added Changelog, Contribution guidelines and no COC.
+
+### Changed
+
+- some minor changes to satisfy linter.
+
+
+
+## [v1.0.8](https://github.com/TLINDEN/tablizer/tree/v1.0.8) - 2022-10-13
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.7...v1.0.8)
+
+### Added
+
+- Added sort support with the new parameter -k (like sort(1)).
+
+
+
+## [v1.0.7](https://github.com/TLINDEN/tablizer/tree/v1.0.7) - 2022-10-11
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.6...v1.0.7)
+
+### Added
+
+- Added pattern highlighting support.
+
+- Added more unit tests.
+
+### Fixed
+
+- Fixed extended more output in combination with -c.
+
+- Fixed issue #4, the version string was missing.
+
+
+
+## [v1.0.6](https://github.com/TLINDEN/tablizer/tree/v1.0.6) - 2022-10-05
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.5...v1.0.6)
+
+### Added
+
+- Added documentation about regexp syntax in the manpage.
+
+- Added more unit tests.
+
+### Changed
+
+- Rewrote the input parser.
+
+- Some more refactoring work has been done.
+
+
+
+## [v1.0.5](https://github.com/TLINDEN/tablizer/tree/v1.0.5) - 2022-10-05
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.4...v1.0.5)
+
+### Added
+
+- A  new option has been  added: --invert-match -v which  behaves like
+  the same option in grep(1): it inverts the pattern match.
+  
+- A few more unit tests have been added.
+
+### Fixed
+
+- Pattern  matching did  not work, because  the (new)  help subcommand
+  lead  to  cobra  taking  care  of  the  first  arg  to  the  program
+  (argv[1]).  So now  there's a  new parameter  -m which  displays the
+  manpage and no more subcommands.
+  
+
+
+## [v1.0.4](https://github.com/TLINDEN/tablizer/tree/v1.0.4) - 2022-10-04
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.3...v1.0.4)
+
+### Added
+
+- Development version of the compiled binary now uses git vars
+  in addition to program version.
+  
+- Added  an option to display  the manual page (compiled  in) as text:
+  --help, for cases where a user just installed the binary.
+  
+### Changed
+
+- Fixed go module namespace.
+
+
+
+## [v1.0.3](https://github.com/TLINDEN/tablizer/tree/v1.0.3) - 2022-10-03
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.2...v1.0.3)
+
+### Added
+
+- Added a new output mode: shell mode, which allows the user
+  to use the output in a shell eval loop to further process
+  the data.
+  
+### Changed
+
+- More refactoring work has been done.
+
+
+
+## [v1.0.2](https://github.com/TLINDEN/tablizer/tree/v1.0.2) - 2022-10-02
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.1...v1.0.2)
+
+### Added
+
+- Added some basic unit tests.
+
+### Changed
+
+- Code has been refactored to be more efficient.
+
+- Replaced table generation code with Tablewriter.
+
+
+
+
+
+## [v1.0.1](https://github.com/TLINDEN/tablizer/tree/v1.0.1) - 2022-09-30
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.0...v1.0.1)
+
+### Added
+
+- Added a unix manual page.
+
+- Added release builder to Makefile
+
+### Changed
+
+- Various minor fixes.
+
+
+
+## [v1.0.0](https://github.com/TLINDEN/tablizer/tree/v1.0.0) - 2022-09-28
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/02a64a5c3fe4220df2c791ff1421d16ebd428c19...v1.0.0)
+
+Initial release.

CODE_OF_CONDUCT.md

@@ -0,0 +1,113 @@
+# No Code of Conduct
+
+*TL;DR:* This project  does **NOT** have a so called  Code of Conduct,
+nor will it ever have one.
+
+## The Rant
+
+The reasons are somewhat complicated and  I'll try my best to document
+them here.
+
+Ethical codes  or rules come  along like laws.  But how is  ethical or
+moral behavior defined? And who  defines which behavior is ethical and
+which is not? Certainly not me.
+
+Unless  you  live  in  a  dictatorship (and  more  than  half  of  the
+population of  planet earth  do as  of this  writing), laws  come into
+existence by democratic procedures. Laws  cover almost every aspect of
+live in  a society. Laws allow  and forbid behavior and  laws sanction
+infringements.
+
+A software project like  this one on the other hand  is not a society.
+There   are   not   enough   people  involved   to   form   democratic
+structures. And there will always be  a minority of users who have the
+right to commit or reject code. How could any maintainer of a software
+project dare to decree rules upon  others? Actually, am I, the current
+maintainer of this very project authorized to do so?
+
+I think the anser to this question clearly is NO.
+
+The  issue  is  being  complicated  by  the  fact,  that  open  source
+development these  days happens  on planetary  scale. And  this planet
+houses hundreds if not  thousands of different cultures, philosophies,
+ideologies and worldviews.  The answer  to many ethical questions will
+in most cases vague and nebulous.
+
+Ones joke will always be another ones insult.
+
+Then there  is the  problem of  language. I myself  am not  an english
+native, but I publish everyting using  the english language. I am able
+to communicate with  most people in the open  source community because
+of that.   But I am  certainly not  able to understand  everything and
+everyone. There  might be nuances to  a sentence I don't  sense, there
+might be  sarcastic connotations I  don't understand or  references to
+historical figures, events  or traditions I don't know  and never have
+heard of.
+
+Judging over other  peoples online behavior looks like  a titanic task
+to me. It is just not my job  to judge others, I am not legitimized or
+authorized to do so and I am not interested in this kind of business.
+
+Another huge  problem with ethical rules  is that you need  to outline
+and enforce sanctions  on those who violate the rules.  But since I am
+not an  elected authority  how would I  be able to  do this?   I don't
+know.  And  what happens if  someone complains about myself?   Shall I
+remove myself from my own project? Come on!
+
+Last but  not least there's  the law. So,  let's say someone  in india
+says  something insulting  to some  other developer  in an  issue.  Of
+course german  law does not apply  to indian people. More,  the insult
+might actually  not be an insult  in india. In the  end, nothing would
+happen.  Under  normal  circumstances, maintainers  would  delete  the
+posting, ban the user or remove push privileges etc.
+
+But then, is there a way for  the offending user to defend himself? Of
+course not, since neither indian or german law alone applies. I cannot
+go to  a german court  and sue the  guy and he  cannot do the  same in
+india. Or -  we possibly could but the judges  on both countries would
+just laugh and close the case.
+
+That  being said,  I  don't have  the  power nor  the  tools, nor  the
+authority to enforce serious sanctions  of any meaningful kind against
+others. Therefore I cannot outline any rules whatsoever.
+
+And let's not  even start talking about  there undemocratic "comitees"
+many projects  are forming to  circumvent this problem.  Some projects
+even  include  external  entities  like a  lawer  or  some  bureaucrat
+somewhere  just to  have the  ability  to complain  against a  comitee
+member. What a mess!
+
+
+
+## So, which are the ethical rules within this project then?
+
+Well, there are none.
+
+This project is  about code, not society. It doesn't  matter where you
+come from,  how you look,  how you think,  what you believe,  who your
+friends are, whay you  said or did sometime in the  past. I don't even
+care if you are a human being. You are an alien so bored that you need
+to submit code on github? Fine with me. You're a convicted criminal? I
+don't give a shit!
+
+**The only thing I am interested here is Code and only Code.**
+
+So if anyhing happens here I don't like  or I am obliged by law to act
+on,  I  will  decide  on  a  case to  case  basis  what  to  do.   And
+unfortunately,  since this  is the  nature  of a  github project,  you
+cannot complain, object or protest. I am very sorry!
+
+If you will, let's at least  outline these:
+
+- Please - just please -  behave towards others as you'd expect others
+  to behave towards yourself.
+
+- Don't judge others for any reason.
+
+- Only judge the code.
+
+But these are not rules, only a  friendly appeal to you as a developer
+and user.
+
+
+Thanks a lot!

CONTRIBUTING.md

@@ -0,0 +1,94 @@
+## Project Goals
+
+The goal of this  project is to build a small tool  which helps in day
+to day  work with tabular  output of various commandline  programs. It
+should be small,  fast and easy to understand. The  idea is to replace
+multiline  shell  pipes  using  awk,   sed  and  grep  with  just  one
+binary.
+
+There will be no GUI, no web interface, no public API of some sort, no
+builtin interpreter.
+
+The  programming  language  used  for  this  project  will  always  be
+[GOLANG](https://go.dev/)  with  the  exception of  the  documentation
+([Perl POD](https://perldoc.perl.org/perlpod)) and the Makefile.
+
+# Contributing
+
+You can contribute to this project in various ways:
+
+## Open an issue
+
+If you encounter  a problem or don't understand how  the program works
+or if you think the documentation is unclear, please don't hesitate to
+open an issue.
+
+Please add as much information about the case as possible, such as:
+
+- Your environment (operating system etc)
+- tablizer version (`tablizer --version`)
+- Input data. Please replace sensitive information with mock data!
+- Actual program output.
+- Expected program output.
+- Error message - if any.
+
+Be aware  that I  am working on  this (and some  other) project  in my
+spare  time which  is scarce.   Therefore  please don't  expect me  to
+respond to  your query within  hours or even  days. Be patient,  but I
+WILL respond.
+
+## Pull Requests
+
+Code and documentation help is  always much appreciated! Please follow
+thes guidelines to successfully contribute:
+
+-  Every  pull   request  shall  be  based   on  latest  `development`
+  branch. `main` is only used for releases.
+  
+- Execute the  unit tests before committing: `make  test`. There shall
+  be no errors.
+  
+- Strive  to be  backwards compatible  so that  users who  are already
+  using the program  don't have to change their habits  - unless it is
+  really neccessary.
+
+- Try to add a unit test for your addition.
+
+- Don't ever change existing unit tests!
+
+- Add a meaningful and comprehensive rationale about your contribution:
+  - Why do you think it might be useful for others?
+  - What did you actually change or add?
+  - Is there an open issue which  this PR fixes and if so, please link
+    to that issue.
+
+- [Re-]format your code with `gofmt -s`.
+
+- Avoid unneccesary dependencies, especially for very small functions.
+
+- **If** a  new dependency is being added, it  must be compatible with
+  our [license agreement](LICENSE).
+  
+- You  need to accept  that the  code or documentation  you contribute
+  will be redistributed under the  terms of said license agreement. If
+  your  contribution  is  considerably  large  or  if  you  contribute
+  regularly, then  feel free  to add  your name and  if you  want your
+  email    address     to    the    *AUTHORS*    section     of    the
+  [manpage](tablizer.pod).
+
+- Adhere to the above mentioned project goals.
+
+-  If  you are  unsure if  your addition or  change will  be accepted,
+  better ask before starting coding. Open an issue about your proposal
+  and let's  discuss it! That way  we avoid doing unnessesary  work on
+  both sides.
+    
+Each pull  request will be  carefully reviewed and  if it is  a useful
+addition  it  will  be  accepted. However,  please  be  prepared  that
+sometimes a  PR will be  rejected.  The reasons  may vary and  will be
+documented.   Perhaps the  above guidelines  are not  matched, or  the
+addition seems  to be not so  useful from my perspective,  maybe there
+are  too  much  changes  or  there  might  be  changes  I  don't  even
+understand.
+
+But whatever happens: your contribution is always welcome!

Makefile

@@ -18,15 +18,15 @@
 #
 # no need to modify anything below
 tool    = tablizer
-version = $(shell egrep "= .v" lib/common.go | cut -d'=' -f2 | cut -d'"' -f 2)
+version = $(shell egrep "= .v" cfg/config.go | cut -d'=' -f2 | cut -d'"' -f 2)
 archs   = android darwin freebsd linux netbsd openbsd windows
 PREFIX  = /usr/local
 UID     = root
 GID     = 0
-BRANCH  = $(shell git describe --all | cut -d/ -f2)
+BRANCH  = $(shell git branch --show-current)
 COMMIT  = $(shell git rev-parse --short=8 HEAD)
 BUILD   = $(shell date +%Y.%m.%d.%H%M%S) 
-VERSION:= $(if $(filter $(BRANCH), development),$(version)-$(BRANCH)-$(COMMIT)-$(BUILD))
+VERSION:= $(if $(filter $(BRANCH), development),$(version)-$(BRANCH)-$(COMMIT)-$(BUILD),$(version))
 
 
 all: $(tool).1 cmd/$(tool).go buildlocal
@@ -36,15 +36,21 @@ all: $(tool).1 cmd/$(tool).go buildlocal
 
 cmd/%.go: %.pod
 	echo "package cmd" > cmd/$*.go
+	echo >> cmd/$*.go
 	echo "var manpage = \`" >> cmd/$*.go
 	pod2text $*.pod >> cmd/$*.go
 	echo "\`" >> cmd/$*.go
 
+	echo "var usage = \`" >> cmd/$*.go
+	awk '/SYNOPS/{f=1;next} /DESCR/{f=0} f' $*.pod  | sed 's/^    //' >> cmd/$*.go
+	echo "\`" >> cmd/$*.go
+
 buildlocal:
-	go build -ldflags "-X 'github.com/tlinden/tablizer/lib.VERSION=$(VERSION)'"
+	go build -ldflags "-X 'github.com/tlinden/tablizer/cfg.VERSION=$(VERSION)'"
 
 release:
 	./mkrel.sh $(tool) $(version)
+	gh release create $(version) --generate-notes releases/*
 
 install: buildlocal
 	install -d -o $(UID) -g $(GID) $(PREFIX)/bin
@@ -53,7 +59,15 @@ install: buildlocal
 	install -o $(UID) -g $(GID) -m 444 $(tool).1 $(PREFIX)/man/man1/
 
 clean:
-	rm -rf $(tool) releases
+	rm -rf $(tool) releases coverage.out
 
 test:
 	go test -v ./...
+
+singletest:
+	@echo "Call like this: ''make singletest TEST=TestPrepareColumns MOD=lib"
+	go test -run $(TEST) github.com/tlinden/tablizer/$(MOD)
+
+cover-report:
+	go test ./... -cover -coverprofile=coverage.out
+	go tool cover -html=coverage.out

README.md

@@ -1,5 +1,6 @@
 [![Actions](https://github.com/tlinden/tablizer/actions/workflows/ci.yaml/badge.svg)](https://github.com/tlinden/tablizer/actions)
 [![License](https://img.shields.io/badge/license-GPL-blue.svg)](https://github.com/tlinden/tablizer/blob/master/LICENSE)
+[![Go Report Card](https://goreportcard.com/badge/github.com/tlinden/tablizer)](https://goreportcard.com/report/github.com/tlinden/tablizer)
 
 ## tablizer - Manipulate tabular output of other programs
 
@@ -20,13 +21,13 @@ But you're only interested in the  NAME and STATUS columns. Here's how
 to do this with tablizer:
 
 ```
-% kubectl get pods | ./tablizer 
+% kubectl get pods | tablizer 
 NAME(1)                      READY(2) STATUS(3)  RESTARTS(4)    AGE(5)
 repldepl-7bcd8d5b64-7zq4l    1/1      Running    1 (69m ago)    5h26m
 repldepl-7bcd8d5b64-m48n8    1/1      Running    1 (69m ago)    5h26m
 repldepl-7bcd8d5b64-q2bf4    1/1      Running    1 (69m ago)    5h26m
 
-% kubectl get pods | ./tablizer -c 1,3
+% kubectl get pods | tablizer -c 1,3
 NAME(1)                      STATUS(3)
 repldepl-7bcd8d5b64-7zq4l    Running
 repldepl-7bcd8d5b64-m48n8    Running
@@ -35,10 +36,11 @@ repldepl-7bcd8d5b64-q2bf4    Running
 
 Another use case is when the tabular  output is so wide that lines are
 being broken and the whole output  is completely distorted.  In such a
-case you can use the `-x` flag to get an output similar to `\x` in `psql`:
+case you can use the `-o extended  | -X` flag to get an output similar
+to `\x` in `psql`:
 
 ```
-% kubectl get pods | ./tablizer -x    
+% kubectl get pods | tablizer -X
     NAME: repldepl-7bcd8d5b64-7zq4l
    READY: 1/1
   STATUS: Running
@@ -63,17 +65,22 @@ Tablize can read one or more files or - if none specified - from STDIN.
 You can also specify a regex pattern to reduce the output:
 
 ```
-% kubectl get pods | ./tablizer q2bf4
+% kubectl get pods | tablizer q2bf4
 NAME(1)                      READY(2) STATUS(3)  RESTARTS(4)    AGE(5)
 repldepl-7bcd8d5b64-q2bf4    1/1      Running    1 (69m ago)    5h26m
 ```
 
+There are more output modes like org-mode (orgtbl) and markdown.
+
+## Demo
+
+[![asciicast](demo/tablizer-demo.gif)](https://asciinema.org/a/9FKc3HPnlg8D2X8otheleEa9t)
 
 ## Installation
 
 There are multiple ways to install **tablizer**:
 
-- Go to the [latest release page](https://github.com/muesli/mango/releases/latest),
+- Go to the [latest release page](https://github.com/tlinden/tablizer/releases/latest),
   locate the binary for your operating system and platform.
   
   Download it and put it into some directory within your `$PATH` variable.
@@ -106,6 +113,10 @@ https://github.com/TLINDEN/tablizer/blob/main/tablizer.pod
 Or if you cloned  the repository you can read it  this way (perl needs
 to be installed though): `perldoc tablizer.pod`.
 
+If you have the binary installed, you  can also read the man page with
+this command:
+
+    tablizer --man
 
 ## Getting help
 

TODO

@@ -1 +0,0 @@
-

TODO.md

@@ -0,0 +1,9 @@
+## Fixes to be implemented
+
+## Features to be implemented
+
+- add comment support (csf.NewReader().Comment = '#')
+
+- add --no-headers option
+
+

cfg/config.go

@@ -0,0 +1,200 @@
+/*
+Copyright © 2022 Thomas von Dein
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+package cfg
+
+import (
+	"errors"
+	"fmt"
+	"github.com/gookit/color"
+	"os"
+	"regexp"
+)
+
+const DefaultSeparator string = `(\s\s+|\t)`
+const Version string = "v1.0.13"
+
+var VERSION string // maintained by -x
+
+type Config struct {
+	Debug       bool
+	NoNumbering bool
+	Columns     string
+	UseColumns  []int
+	Separator   string
+	OutputMode  int
+	InvertMatch bool
+	Pattern     string
+	PatternR    *regexp.Regexp
+
+	SortMode       string
+	SortDescending bool
+	SortByColumn   int
+
+	/*
+	 FIXME: make configurable somehow, config file or ENV
+	 see https://github.com/gookit/color.
+	*/
+	ColorStyle color.Style
+
+	NoColor bool
+}
+
+// maps outputmode short flags to output mode, ie. -O => -o orgtbl
+type Modeflag struct {
+	X bool
+	O bool
+	M bool
+	S bool
+	Y bool
+	A bool
+	C bool
+}
+
+// used for switching printers
+const (
+	Extended = iota + 1
+	Orgtbl
+	Markdown
+	Shell
+	Yaml
+	CSV
+	Ascii
+)
+
+// various sort types
+type Sortmode struct {
+	Numeric bool
+	Time    bool
+	Age     bool
+}
+
+// default color schemes
+func Colors() map[color.Level]map[string]color.Color {
+	return map[color.Level]map[string]color.Color{
+		color.Level16: {
+			"bg": color.BgGreen, "fg": color.FgBlack,
+		},
+		color.Level256: {
+			"bg": color.BgLightGreen, "fg": color.FgBlack,
+		},
+		color.LevelRgb: {
+			// FIXME: maybe use something nicer
+			"bg": color.BgLightGreen, "fg": color.FgBlack,
+		},
+	}
+}
+
+// find supported color mode, modifies config based on constants
+func (c *Config) DetermineColormode() {
+	if !isTerminal(os.Stdout) {
+		color.Disable()
+	} else {
+		level := color.TermColorLevel()
+		colors := Colors()
+		c.ColorStyle = color.New(colors[level]["bg"], colors[level]["fg"])
+	}
+}
+
+// Return true if current terminal is interactive
+func isTerminal(f *os.File) bool {
+	o, _ := f.Stat()
+	if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice {
+		return true
+	} else {
+		return false
+	}
+}
+
+func Getversion() string {
+	// main program version
+
+	// generated  version string, used  by -v contains  lib.Version on
+	//  main  branch,   and  lib.Version-$branch-$lastcommit-$date  on
+	// development branch
+
+	return fmt.Sprintf("This is tablizer version %s", VERSION)
+}
+
+func (conf *Config) PrepareSortFlags(flag Sortmode) {
+	switch {
+	case flag.Numeric:
+		conf.SortMode = "numeric"
+	case flag.Age:
+		conf.SortMode = "duration"
+	case flag.Time:
+		conf.SortMode = "time"
+	default:
+		conf.SortMode = "string"
+	}
+}
+
+func (conf *Config) PrepareModeFlags(flag Modeflag) {
+	switch {
+	case flag.X:
+		conf.OutputMode = Extended
+	case flag.O:
+		conf.OutputMode = Orgtbl
+	case flag.M:
+		conf.OutputMode = Markdown
+	case flag.S:
+		conf.OutputMode = Shell
+	case flag.Y:
+		conf.OutputMode = Yaml
+	case flag.C:
+		conf.OutputMode = CSV
+	default:
+		conf.OutputMode = Ascii
+	}
+}
+
+func (c *Config) CheckEnv() {
+	// check for environment vars, command line flags have precedence,
+	// NO_COLOR is being checked by the color module itself.
+	if !c.NoNumbering {
+		_, set := os.LookupEnv("T_NO_HEADER_NUMBERING")
+		if set {
+			c.NoNumbering = true
+		}
+	}
+
+	if len(c.Columns) == 0 {
+		cols := os.Getenv("T_COLUMNS")
+		if len(cols) > 1 {
+			c.Columns = cols
+		}
+	}
+}
+
+func (c *Config) ApplyDefaults() {
+	// mode specific defaults
+	if c.OutputMode == Yaml || c.OutputMode == CSV {
+		c.NoNumbering = true
+	}
+}
+
+func (c *Config) PreparePattern(pattern string) error {
+	PatternR, err := regexp.Compile(pattern)
+
+	if err != nil {
+		return errors.Unwrap(fmt.Errorf("Regexp pattern %s is invalid: %w", c.Pattern, err))
+	}
+
+	c.PatternR = PatternR
+	c.Pattern = pattern
+
+	return nil
+}

cfg/config_test.go

@@ -0,0 +1,103 @@
+/*
+Copyright © 2022 Thomas von Dein
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package cfg
+
+import (
+	"fmt"
+	//	"reflect"
+	"testing"
+)
+
+func TestPrepareModeFlags(t *testing.T) {
+	var tests = []struct {
+		flag   Modeflag
+		expect int // output (constant enum)
+	}{
+		// short commandline flags like -M
+		{Modeflag{X: true}, Extended},
+		{Modeflag{S: true}, Shell},
+		{Modeflag{O: true}, Orgtbl},
+		{Modeflag{Y: true}, Yaml},
+		{Modeflag{M: true}, Markdown},
+		{Modeflag{}, Ascii},
+	}
+
+	// FIXME: use a map for easier printing
+	for _, tt := range tests {
+		testname := fmt.Sprintf("PrepareModeFlags-expect-%d", tt.expect)
+		t.Run(testname, func(t *testing.T) {
+			c := Config{}
+
+			c.PrepareModeFlags(tt.flag)
+			if c.OutputMode != tt.expect {
+				t.Errorf("got: %d, expect: %d", c.OutputMode, tt.expect)
+			}
+		})
+	}
+}
+
+func TestPrepareSortFlags(t *testing.T) {
+	var tests = []struct {
+		flag   Sortmode
+		expect string // output
+	}{
+		// short commandline flags like -M
+		{Sortmode{Numeric: true}, "numeric"},
+		{Sortmode{Age: true}, "duration"},
+		{Sortmode{Time: true}, "time"},
+		{Sortmode{}, "string"},
+	}
+
+	for _, tt := range tests {
+		testname := fmt.Sprintf("PrepareSortFlags-expect-%s", tt.expect)
+		t.Run(testname, func(t *testing.T) {
+			c := Config{}
+
+			c.PrepareSortFlags(tt.flag)
+
+			if c.SortMode != tt.expect {
+				t.Errorf("got: %s, expect: %s", c.SortMode, tt.expect)
+			}
+		})
+	}
+}
+
+func TestPreparePattern(t *testing.T) {
+	var tests = []struct {
+		pattern string
+		wanterr bool
+	}{
+		{"[A-Z]+", false},
+		{"[a-z", true},
+	}
+
+	for _, tt := range tests {
+		testname := fmt.Sprintf("PreparePattern-pattern-%s-wanterr-%t", tt.pattern, tt.wanterr)
+		t.Run(testname, func(t *testing.T) {
+			c := Config{}
+
+			err := c.PreparePattern(tt.pattern)
+
+			if err != nil {
+				if !tt.wanterr {
+					t.Errorf("PreparePattern returned error: %s", err)
+				}
+			}
+		})
+	}
+}

cmd/root.go

@@ -18,16 +18,17 @@ package cmd
 
 import (
 	"bytes"
+	"errors"
 	"fmt"
 	"github.com/spf13/cobra"
+	"github.com/tlinden/tablizer/cfg"
 	"github.com/tlinden/tablizer/lib"
 	"log"
 	"os"
 	"os/exec"
+	"strings"
 )
 
-var ShowManual = false
-
 func man() {
 	man := exec.Command("less", "-")
 
@@ -45,13 +46,38 @@ func man() {
 	}
 }
 
+func completion(cmd *cobra.Command, mode string) error {
+	switch mode {
+	case "bash":
+		return cmd.Root().GenBashCompletion(os.Stdout)
+	case "zsh":
+		return cmd.Root().GenZshCompletion(os.Stdout)
+	case "fish":
+		return cmd.Root().GenFishCompletion(os.Stdout, true)
+	case "powershell":
+		return cmd.Root().GenPowerShellCompletionWithDesc(os.Stdout)
+	default:
+		return errors.New("Invalid shell parameter! Valid ones: bash|zsh|fish|powershell")
+	}
+}
+
+func Execute() {
+	var (
+		conf           cfg.Config
+		ShowManual     bool
+		ShowVersion    bool
+		ShowCompletion string
+		modeflag       cfg.Modeflag
+		sortmode       cfg.Sortmode
+	)
+
 	var rootCmd = &cobra.Command{
 		Use:   "tablizer [regex] [file, ...]",
 		Short: "[Re-]tabularize tabular data",
 		Long:  `Manipulate tabular output of other programs`,
 		RunE: func(cmd *cobra.Command, args []string) error {
-		if lib.ShowVersion {
+			if ShowVersion {
-			fmt.Printf("This is tablizer version %s\n", lib.VERSION)
+				fmt.Println(cfg.Getversion())
 				return nil
 			}
 
@@ -60,47 +86,57 @@ var rootCmd = &cobra.Command{
 				return nil
 			}
 
-		err := lib.PrepareColumns()
+			if len(ShowCompletion) > 0 {
-		if err != nil {
+				return completion(cmd, ShowCompletion)
-			return err
 			}
 
-		err = lib.PrepareModeFlags()
+			// Setup
-		if err != nil {
+			conf.CheckEnv()
-			return err
+			conf.PrepareModeFlags(modeflag)
-		}
+			conf.PrepareSortFlags(sortmode)
+			conf.DetermineColormode()
+			conf.ApplyDefaults()
 
-		return lib.ProcessFiles(args)
+			// actual execution starts here
+			return lib.ProcessFiles(&conf, args)
 		},
 	}
 
-func Execute() {
+	// options
+	rootCmd.PersistentFlags().BoolVarP(&conf.Debug, "debug", "d", false, "Enable debugging")
+	rootCmd.PersistentFlags().BoolVarP(&conf.NoNumbering, "no-numbering", "n", false, "Disable header numbering")
+	rootCmd.PersistentFlags().BoolVarP(&conf.NoColor, "no-color", "N", false, "Disable pattern highlighting")
+	rootCmd.PersistentFlags().BoolVarP(&ShowVersion, "version", "V", false, "Print program version")
+	rootCmd.PersistentFlags().BoolVarP(&conf.InvertMatch, "invert-match", "v", false, "select non-matching rows")
+	rootCmd.PersistentFlags().BoolVarP(&ShowManual, "man", "m", false, "Display manual page")
+	rootCmd.PersistentFlags().StringVarP(&ShowCompletion, "completion", "", "", "Display completion code")
+	rootCmd.PersistentFlags().StringVarP(&conf.Separator, "separator", "s", cfg.DefaultSeparator, "Custom field separator")
+	rootCmd.PersistentFlags().StringVarP(&conf.Columns, "columns", "c", "", "Only show the speficied columns (separated by ,)")
+
+	// sort options
+	rootCmd.PersistentFlags().IntVarP(&conf.SortByColumn, "sort-by", "k", 0, "Sort by column (default: 1)")
+
+	// sort mode, only 1 allowed
+	rootCmd.PersistentFlags().BoolVarP(&conf.SortDescending, "sort-desc", "D", false, "Sort in descending order (default: ascending)")
+	rootCmd.PersistentFlags().BoolVarP(&sortmode.Numeric, "sort-numeric", "i", false, "sort according to string numerical value")
+	rootCmd.PersistentFlags().BoolVarP(&sortmode.Time, "sort-time", "t", false, "sort according to time string")
+	rootCmd.PersistentFlags().BoolVarP(&sortmode.Age, "sort-age", "a", false, "sort according to age (duration) string")
+	rootCmd.MarkFlagsMutuallyExclusive("sort-desc", "sort-numeric", "sort-time", "sort-age")
+
+	// output flags, only 1 allowed
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.X, "extended", "X", false, "Enable extended output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.M, "markdown", "M", false, "Enable markdown table output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.O, "orgtbl", "O", false, "Enable org-mode table output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.S, "shell", "S", false, "Enable shell mode output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.Y, "yaml", "Y", false, "Enable yaml output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.C, "csv", "C", false, "Enable CSV output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.A, "ascii", "A", false, "Enable ASCII output (default)")
+	rootCmd.MarkFlagsMutuallyExclusive("extended", "markdown", "orgtbl", "shell", "yaml", "csv")
+
+	rootCmd.SetUsageTemplate(strings.TrimSpace(usage) + "\n")
+
 	err := rootCmd.Execute()
 	if err != nil {
 		os.Exit(1)
 	}
 }
-
-func init() {
-	rootCmd.PersistentFlags().BoolVarP(&lib.Debug, "debug", "d", false, "Enable debugging")
-	rootCmd.PersistentFlags().BoolVarP(&lib.NoNumbering, "no-numbering", "n", false, "Disable header numbering")
-	rootCmd.PersistentFlags().BoolVarP(&lib.ShowVersion, "version", "V", false, "Print program version")
-	rootCmd.PersistentFlags().BoolVarP(&lib.InvertMatch, "invert-match", "v", false, "select non-matching rows")
-	rootCmd.PersistentFlags().BoolVarP(&ShowManual, "man", "m", false, "Display manual page")
-	rootCmd.PersistentFlags().StringVarP(&lib.Separator, "separator", "s", lib.DefaultSeparator, "Custom field separator")
-	rootCmd.PersistentFlags().StringVarP(&lib.Columns, "columns", "c", "", "Only show the speficied columns (separated by ,)")
-
-	// output flags, only 1 allowed, hidden, since just short cuts
-	rootCmd.PersistentFlags().BoolVarP(&lib.OutflagExtended, "extended", "X", false, "Enable extended output")
-	rootCmd.PersistentFlags().BoolVarP(&lib.OutflagMarkdown, "markdown", "M", false, "Enable markdown table output")
-	rootCmd.PersistentFlags().BoolVarP(&lib.OutflagOrgtable, "orgtbl", "O", false, "Enable org-mode table output")
-	rootCmd.PersistentFlags().BoolVarP(&lib.OutflagShell, "shell", "S", false, "Enable shell mode output")
-	rootCmd.MarkFlagsMutuallyExclusive("extended", "markdown", "orgtbl", "shell")
-	rootCmd.Flags().MarkHidden("extended")
-	rootCmd.Flags().MarkHidden("orgtbl")
-	rootCmd.Flags().MarkHidden("markdown")
-	rootCmd.Flags().MarkHidden("shell")
-
-	// same thing but more common, takes precedence over above group
-	rootCmd.PersistentFlags().StringVarP(&lib.OutputMode, "output", "o", "", "Output mode - one of: orgtbl, markdown, extended, shell, ascii(default)")
-}

cmd/tablizer.go

@@ -1,4 +1,5 @@
 package cmd
+
 var manpage = `
 NAME
     tablizer - Manipulate tabular output of other programs
@@ -7,27 +8,43 @@ SYNOPSIS
         Usage:
           tablizer [regex] [file, ...] [flags]
     
-        Flags:
+        Operational Flags:
           -c, --columns string     Only show the speficied columns (separated by ,)
-          -d, --debug              Enable debugging
-          -h, --help               help for tablizer
           -v, --invert-match       select non-matching rows
-          -m, --man                Display manual page
           -n, --no-numbering       Disable header numbering
-          -o, --output string      Output mode - one of: orgtbl, markdown, extended, ascii(default)
+          -N, --no-color           Disable pattern highlighting
+          -s, --separator string   Custom field separator
+          -k, --sort-by int        Sort by column (default: 1)
+
+        Output Flags (mutually exclusive):
           -X, --extended           Enable extended output
           -M, --markdown           Enable markdown table output
           -O, --orgtbl             Enable org-mode table output
-          -s, --separator string   Custom field separator
+          -S, --shell              Enable shell evaluable ouput
+          -Y, --yaml               Enable yaml output
+          -C, --csv                Enable CSV output
+          -A, --ascii              Default output mode, ascii tabular
+
+        Sort Mode Flags (mutually exclusive):
+          -a, --sort-age           sort according to age (duration) string
+          -D, --sort-desc          Sort in descending order (default: ascending)
+          -i, --sort-numeric       sort according to string numerical value
+          -t, --sort-time          sort according to time string
+
+        Other Flags:
+              --completion <shell> Generate the autocompletion script for <shell>
+          -d, --debug              Enable debugging
+          -h, --help               help for tablizer
+          -m, --man                Display manual page
           -v, --version            Print program version
 
 DESCRIPTION
     Many programs generate tabular output. But sometimes you need to
     post-process these tables, you may need to remove one or more columns or
-    you may want to filter for some pattern or you may need the output in
+    you may want to filter for some pattern (See PATTERNS) or you may need
-    another program and need to parse it somehow. Standard unix tools such
+    the output in another program and need to parse it somehow. Standard
-    as awk(1), grep(1) or column(1) may help, but sometimes it's a tedious
+    unix tools such as awk(1), grep(1) or column(1) may help, but sometimes
-    business.
+    it's a tedious business.
 
     Let's take the output of the tool kubectl. It contains cells with
     withespace and they do not separate columns by TAB characters. This is
@@ -35,7 +52,7 @@ DESCRIPTION
 
     You can use tablizer to do these and more things.
 
-    tablizer analyses the header fiels of a table, registers the column
+    tablizer analyses the header fields of a table, registers the column
     positions of each header field and separates columns by those positions.
 
     Without any options it reads its input from "STDIN", but you can also
@@ -61,7 +78,7 @@ DESCRIPTION
        NAME(1) READY(2) STATUS(3) RESTARTS(4) AGE(5)
 
     These numbers denote the column and you can use them to specify which
-    columns you want to have in your output:
+    columns you want to have in your output (see COLUMNS:
 
        kubectl get pods | tablizer -c1,3
 
@@ -70,13 +87,81 @@ DESCRIPTION
 
     The numbering can be suppressed by using the -n option.
 
-    Finally the -d option enables debugging output which is mostly usefull
+    By default, if a pattern has been speficied, matches will be
+    highlighted. You can disable this behavior with the -N option.
+
+    Use the -k option to specify by which column to sort the tabular data
+    (as in GNU sort(1)). The default sort column is the first one. To
+    disable sorting at all, supply 0 (Zero) to -k. The default sort order is
+    ascending. You can change this to descending order using the option -D.
+    The default sort order is by string, but there are other sort modes:
+
+    -a --sort-age
+        Sorts duration strings like "1d4h32m51s".
+
+    -i --sort-numeric
+        Sorts numeric fields.
+
+    -t --sort-time
+        Sorts timestamps.
+
+    Finally the -d option enables debugging output which is mostly useful
     for the developer.
 
+  PATTERNS
+    You can reduce the rows being displayed by using a regular expression
+    pattern. The regexp is PCRE compatible, refer to the syntax cheat sheet
+    here: <https://github.com/google/re2/wiki/Syntax>. If you want to read a
+    more comprehensive documentation about the topic and have perl installed
+    you can read it with:
+
+        perldoc perlre
+
+    Or read it online: <https://perldoc.perl.org/perlre>.
+
+    A note on modifiers: the regexp engine used in tablizer uses another
+    modifier syntax:
+
+        (?MODIFIER)
+
+    The most important modifiers are:
+
+    "i" ignore case "m" multiline mode "s" single line mode
+
+    Example for a case insensitive search:
+
+        kubectl get pods -A | tablizer "(?i)account"
+
+  COLUMNS
+    The parameter -c can be used to specify, which columns to display. By
+    default tablizer numerizes the header names and these numbers can be
+    used to specify which header to display, see example above.
+
+    However, beside numbers, you can also use regular expressions with -c,
+    also separated by comma. And you can mix column numbers with regexps.
+
+    Lets take this table:
+
+            PID TTY          TIME CMD
+          14001 pts/0    00:00:00 bash
+          42871 pts/0    00:00:00 ps
+          42872 pts/0    00:00:00 sed
+
+    We want to see only the CMD column and use a regex for this:
+
+        ps | tablizer -s '\s+' -c C
+        CMD(4)
+        bash
+        ps
+        tablizer
+        sed
+
+    where "C" is our regexp which matches CMD.
+
   OUTPUT MODES
     There might be cases when the tabular output of a program is way too
     large for your current terminal but you still need to see every column.
-    In such cases the -o extended or -X option can be usefull which enables
+    In such cases the -o extended or -X option can be useful which enables
     *extended mode*. In this mode, each row will be printed vertically,
     header left, value right, aligned by the field widths. Here's an
     example:
@@ -103,7 +188,63 @@ DESCRIPTION
 
     Beside normal ascii mode (the default) and extended mode there are more
     output modes available: orgtbl which prints an Emacs org-mode table and
-    markdown which prints a Markdown table.
+    markdown which prints a Markdown table, yaml, which prints yaml encoding
+    and CSV mode, which prints a comma separated value file.
+
+  ENVIRONMENT VARIABLES
+    tablizer supports certain environment variables which use can use to
+    influence program behavior. Commandline flags have always precedence
+    over environment variables.
+
+    <T_NO_HEADER_NUMBERING> - disable numbering of header fields, like -n.
+    <T_COLUMNS> - comma separated list of columns to output, like -c
+    <NO_COLORS> - disable colorization of matches, like -N
+
+  COMPLETION
+    Shell completion for command line options can be enabled by using the
+    --completion flag. The required parameter is the name of your shell.
+    Currently supported are: bash, zsh, fish and powershell.
+
+    Detailed instructions:
+
+    Bash:
+           source <(tablizer --completion bash)
+
+        To load completions for each session, execute once:
+
+          # Linux:
+          $ tablizer --completion bash > /etc/bash_completion.d/tablizer
+
+          # macOS:
+          $ tablizer --completion bash > $(brew --prefix)/etc/bash_completion.d/tablizer
+
+    Zsh:
+        If shell completion is not already enabled in your environment, you
+        will need to enable it. You can execute the following once:
+
+          echo "autoload -U compinit; compinit" >> ~/.zshrc
+
+        To load completions for each session, execute once:
+
+          $ tablizer --completion zsh > "${fpath[1]}/_tablizer"
+
+        You will need to start a new shell for this setup to take effect.
+
+    fish:
+           tablizer --completion fish | source
+
+        To load completions for each session, execute once:
+
+           tablizer --completion fish > ~/.config/fish/completions/tablizer.fish
+
+    PowerShell:
+           tablizer --completion powershell | Out-String | Invoke-Expression
+
+        To load completions for every new session, run:
+
+           tablizer --completion powershell > tablizer.ps1
+
+        and source this file from your PowerShell profile.
 
 BUGS
     In order to report a bug, unexpected behavior, feature requests or to
@@ -129,3 +270,40 @@ AUTHORS
     Thomas von Dein tom AT vondein DOT org
 
 `
+var usage = `
+
+Usage:
+  tablizer [regex] [file, ...] [flags]
+
+Operational Flags:
+  -c, --columns string     Only show the speficied columns (separated by ,)
+  -v, --invert-match       select non-matching rows
+  -n, --no-numbering       Disable header numbering
+  -N, --no-color           Disable pattern highlighting
+  -s, --separator string   Custom field separator
+  -k, --sort-by int        Sort by column (default: 1)
+
+Output Flags (mutually exclusive):
+  -X, --extended           Enable extended output
+  -M, --markdown           Enable markdown table output
+  -O, --orgtbl             Enable org-mode table output
+  -S, --shell              Enable shell evaluable ouput
+  -Y, --yaml               Enable yaml output
+  -C, --csv                Enable CSV output
+  -A, --ascii              Default output mode, ascii tabular
+
+Sort Mode Flags (mutually exclusive):
+  -a, --sort-age           sort according to age (duration) string
+  -D, --sort-desc          Sort in descending order (default: ascending)
+  -i, --sort-numeric       sort according to string numerical value
+  -t, --sort-time          sort according to time string
+
+Other Flags:
+      --completion <shell> Generate the autocompletion script for <shell>
+  -d, --debug              Enable debugging
+  -h, --help               help for tablizer
+  -m, --man                Display manual page
+  -v, --version            Print program version
+
+
+`

demo/Makefile

@@ -0,0 +1,3 @@
+all:
+	LC_ALL=en_US.UTF-8 asciinema rec --cols 50 --row 30 -c ./demo.sh --overwrite tmp.cast
+	agg tmp.cast tmp.gif

demo/demo.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+
+prompt() {
+    if test -n "$1"; then
+        echo
+        echo -n "% $*"
+        sleep 1
+        echo
+        $*
+        echo
+        echo -n "% "
+    else
+        echo -n "% "
+    fi
+}
+
+PATH=..:$PATH
+clear
+while IFS=$'\t' read -r flags table msg source _; do
+    echo "#"
+    echo "#   source tabular data:"
+    cat $table
+    echo
+    echo "#"
+    echo "#   $msg:"
+    prompt "tablizer $flags $table"
+  
+    sleep 4
+    clear
+done < <(yq -r tables.yaml \
+         | yq -r '.tables[] | [.flags, .table, .msg, .source] | @tsv')

demo/table.demo1

@@ -0,0 +1,4 @@
+NAME   DURATION   COUNT  WHEN
+beta   1d10h5m1s  33     3/1/2014
+alpha  4h35m      170    2013-Feb-03
+ceta   33d12h     9      06/Jan/2008 15:04:05 -0700

demo/table.demo2

@@ -0,0 +1,3 @@
+    PID TTY          TIME CMD
+  30912 pts/0    00:00:00 bash
+  49526 pts/0    00:00:00 ps

demo/tables.yaml

@@ -0,0 +1,54 @@
+tables:
+# OUTPUTS
+  - flags: -A
+    table: table.demo1
+    msg: default output mode
+  - flags: -O
+    table: table.demo1
+    msg: orgmode output mode
+  - flags: -M
+    table: table.demo1
+    msg: markdown output mode
+  - flags: -S
+    table: table.demo1
+    msg: shell output mode
+  - flags: -X
+    table: table.demo1
+    msg: extended output mode
+  - flags: -Y
+    table: table.demo1
+    msg: yaml output mode
+  - flags: -C
+    table: table.demo1
+    msg: CSV output mode
+
+  # SORTS
+  - flags: -A -k 3
+    table: table.demo1
+    msg: sort by column 3
+  - flags: -A -k 4 -t
+    table: table.demo1
+    msg: sort by column 4 and sort type time
+  - flags: -A -k 2 -a
+    table: table.demo1
+    msg: sort by column 2 and sort type duration
+
+  # REDUCE
+  - flags: -A -c 1,3
+    table: table.demo1
+    msg: only display column 1 and 3
+  - flags: -A -c AM,RA
+    table: table.demo1
+    msg: only display columns matching /(RA|AM)/
+  - flags: -X -c 1,3
+    table: table.demo1
+    msg: only display column 1 and 3 in extended mode
+
+  # SEARCH
+  - flags: /20 -A
+    table: table.demo1
+    msg: only show rows matching /20
+  - flags: /20 -A -v
+    table: table.demo1
+    msg: only show rows NOT matching /20
+

demo/tablizer-demo.cast

@@ -0,0 +1,119 @@
+{"version": 2, "width": 80, "height": 25, "timestamp": 1666890777, "env": {"SHELL": "/bin/bash", "TERM": "xterm-256color"}}
+[0.004618, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[0.010297, "o", "#\r\n#   source tabular data:\r\n"]
+[0.010898, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[0.011125, "o", "\r\n#\r\n"]
+[0.011177, "o", "#   default output mode:\r\n"]
+[0.011219, "o", "\r\n% tablizer -A table.demo1"]
+[1.011851, "o", "\r\n"]
+[1.013635, "o", "NAME(1)\tDURATION(2)\tCOUNT(3)\tWHEN(4)                    \r\nbeta   \t1d10h5m1s  \t33      \t3/1/2014                  \t\r\nalpha  \t4h35m      \t170     \t2013-Feb-03               \t\r\nceta   \t33d12h     \t9       \t06/Jan/2008 15:04:05 -0700\t\r\n"]
+[1.014021, "o", "\r\n% "]
+[5.015241, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[5.015339, "o", "#\r\n#   source tabular data:\r\n"]
+[5.015688, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[5.015776, "o", "\r\n#\r\n#   orgmode output mode:\r\n\r\n% tablizer -O table.demo1"]
+[6.016322, "o", "\r\n"]
+[6.01823, "o", "+---------+-------------+----------+----------------------------+\r\n| NAME(1) | DURATION(2) | COUNT(3) |          WHEN(4)           |\r\n+---------+-------------+----------+----------------------------+\r\n| beta    | 1d10h5m1s   |       33 | 3/1/2014                   |\r\n| alpha   | 4h35m       |      170 | 2013-Feb-03                |\r\n| ceta    | 33d12h      |        9 | 06/Jan/2008 15:04:05 -0700 |\r\n+---------+-------------+----------+----------------------------+\r\n"]
+[6.018497, "o", "\r\n% "]
+[10.020014, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[10.020112, "o", "#\r\n#   source tabular data:\r\n"]
+[10.020573, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[10.020643, "o", "\r\n#\r\n"]
+[10.02068, "o", "#   markdown output mode:\r\n\r\n% tablizer -M table.demo1"]
+[11.021559, "o", "\r\n"]
+[11.023551, "o", "| NAME(1) | DURATION(2) | COUNT(3) |          WHEN(4)           |\r\n|---------|-------------|----------|----------------------------|\r\n| beta    | 1d10h5m1s   |       33 | 3/1/2014                   |\r\n| alpha   | 4h35m       |      170 | 2013-Feb-03                |\r\n| ceta    | 33d12h      |        9 | 06/Jan/2008 15:04:05 -0700 |\r\n"]
+[11.023838, "o", "\r\n% "]
+[15.025244, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[15.025345, "o", "#\r\n#   source tabular data:\r\n"]
+[15.025829, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[15.025915, "o", "\r\n#\r\n#   shell output mode:\r\n"]
+[15.025931, "o", "\r\n"]
+[15.025948, "o", "% tablizer -S table.demo1"]
+[16.026714, "o", "\r\n"]
+[16.028606, "o", "NAME(1)=\"beta\" DURATION(2)=\"1d10h5m1s\" COUNT(3)=\"33\" WHEN(4)=\"3/1/2014\"\r\nNAME(1)=\"alpha\" DURATION(2)=\"4h35m\" COUNT(3)=\"170\" WHEN(4)=\"2013-Feb-03\"\r\nNAME(1)=\"ceta\" DURATION(2)=\"33d12h\" COUNT(3)=\"9\" WHEN(4)=\"06/Jan/2008 15:04:05 -0700\"\r\n"]
+[16.029144, "o", "\r\n% "]
+[20.030593, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[20.030706, "o", "#\r\n#   source tabular data:\r\n"]
+[20.03121, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[20.031277, "o", "\r\n#\r\n#   extended output mode:\r\n"]
+[20.031327, "o", "\r\n% tablizer -X table.demo1"]
+[21.032053, "o", "\r\n"]
+[21.033787, "o", "    NAME(1): beta\r\nDURATION(2): 1d10h5m1s\r\n   COUNT(3): 33\r\n    WHEN(4): 3/1/2014\r\n\r\n    NAME(1): alpha\r\nDURATION(2): 4h35m\r\n   COUNT(3): 170\r\n    WHEN(4): 2013-Feb-03\r\n\r\n    NAME(1): ceta\r\nDURATION(2): 33d12h\r\n   COUNT(3): 9\r\n    WHEN(4): 06/Jan/2008 15:04:05 -0700\r\n\r\n"]
+[21.034132, "o", "\r\n% "]
+[25.035531, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[25.035585, "o", "#\r\n"]
+[25.035681, "o", "#   source tabular data:\r\n"]
+[25.036179, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[25.036232, "o", "\r\n#\r\n"]
+[25.036274, "o", "#   yaml output mode:\r\n\r\n% tablizer -Y table.demo1"]
+[26.036928, "o", "\r\n"]
+[26.038674, "o", "entries:\r\n    - count: 33\r\n      duration: \"1d10h5m1s\"\r\n      name: \"beta\"\r\n      when: \"3/1/2014\"\r\n    - count: 170\r\n      duration: \"4h35m\"\r\n      name: \"alpha\"\r\n      when: \"2013-Feb-03\"\r\n    - count: 9\r\n      duration: \"33d12h\"\r\n      name: \"ceta\"\r\n      when: \"06/Jan/2008 15:04:05 -0700\"\r\n"]
+[26.038975, "o", "\r\n% "]
+[30.040539, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[30.040659, "o", "#\r\n#   source tabular data:\r\n"]
+[30.041167, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[30.041246, "o", "\r\n#\r\n#   CSV output mode:\r\n\r\n% tablizer -C table.demo1"]
+[31.042088, "o", "\r\n"]
+[31.043721, "o", "NAME,DURATION,COUNT,WHEN\r\nbeta,1d10h5m1s,33,3/1/2014\r\nalpha,4h35m,170,2013-Feb-03\r\nceta,33d12h,9,06/Jan/2008 15:04:05 -0700\r\n"]
+[31.043997, "o", "\r\n% "]
+[35.045523, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[35.04563, "o", "#\r\n#   source tabular data:\r\n"]
+[35.046209, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[35.046275, "o", "\r\n#\r\n#   sort by column 3:\r\n\r\n% tablizer -A -k 3 table.demo1"]
+[36.047083, "o", "\r\n"]
+[36.048793, "o", "NAME(1)\tDURATION(2)\tCOUNT(3)\tWHEN(4)                    \r\nalpha  \t4h35m      \t170     \t2013-Feb-03               \t\r\nbeta   \t1d10h5m1s  \t33      \t3/1/2014                  \t\r\nceta   \t33d12h     \t9       \t06/Jan/2008 15:04:05 -0700\t\r\n"]
+[36.049077, "o", "\r\n% "]
+[40.050739, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[40.050925, "o", "#\r\n#   source tabular data:\r\n"]
+[40.051481, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[40.051671, "o", "\r\n#\r\n#   sort by column 4 and sort type time:\r\n\r\n% tablizer -A -k 4 -t table.demo1"]
+[41.052486, "o", "\r\n"]
+[41.05454, "o", "NAME(1)\tDURATION(2)\tCOUNT(3)\tWHEN(4)                    \r\nceta   \t33d12h     \t9       \t06/Jan/2008 15:04:05 -0700\t\r\nalpha  \t4h35m      \t170     \t2013-Feb-03               \t\r\nbeta   \t1d10h5m1s  \t33      \t3/1/2014                  \t\r\n"]
+[41.054864, "o", "\r\n% "]
+[45.056297, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[45.056405, "o", "#\r\n#   source tabular data:\r\n"]
+[45.056895, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[45.056978, "o", "\r\n#\r\n"]
+[45.057023, "o", "#   sort by column 2 and sort type duration:\r\n"]
+[45.057073, "o", "\r\n% tablizer -A -k 2 -a table.demo1"]
+[46.057895, "o", "\r\n"]
+[46.059684, "o", "NAME(1)\tDURATION(2)\tCOUNT(3)\tWHEN(4)                    \r\nalpha  \t4h35m      \t170     \t2013-Feb-03               \t\r\nbeta   \t1d10h5m1s  \t33      \t3/1/2014                  \t\r\nceta   \t33d12h     \t9       \t06/Jan/2008 15:04:05 -0700\t\r\n"]
+[46.059988, "o", "\r\n% "]
+[50.061514, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[50.061622, "o", "#\r\n#   source tabular data:\r\n"]
+[50.062091, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[50.062188, "o", "\r\n#\r\n#   only display column 1 and 3:\r\n\r\n% tablizer -A -c 1,3 table.demo1"]
+[51.062985, "o", "\r\n"]
+[51.066293, "o", "NAME(1)\tCOUNT(3) \r\nbeta   \t33      \t\r\nalpha  \t170     \t\r\nceta   \t9       \t\r\n"]
+[51.066843, "o", "\r\n% "]
+[55.070781, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[55.071327, "o", "#\r\n#   source tabular data:\r\n"]
+[55.073499, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[55.073822, "o", "\r\n#\r\n#   only display columns matching /(RA|AM)/:\r\n"]
+[55.074188, "o", "\r\n% tablizer -A -c AM,RA table.demo1"]
+[56.07636, "o", "\r\n"]
+[56.078603, "o", "NAME(1)\tDURATION(2) \r\nbeta   \t1d10h5m1s  \t\r\nalpha  \t4h35m      \t\r\nceta   \t33d12h     \t\r\n"]
+[56.078957, "o", "\r\n% "]
+[60.080574, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[60.080734, "o", "#\r\n#   source tabular data:\r\n"]
+[60.081286, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[60.081418, "o", "\r\n#\r\n#   only display column 1 and 3 in extended mode:\r\n\r\n% tablizer -X -c 1,3 table.demo1"]
+[61.082844, "o", "\r\n"]
+[61.089822, "o", " NAME(1): beta\r\nCOUNT(3): 33\r\n\r\n NAME(1): alpha\r\nCOUNT(3): 170\r\n\r\n NAME(1): ceta\r\nCOUNT(3): 9\r\n\r\n"]
+[61.090969, "o", "\r\n% "]
+[65.096092, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[65.096571, "o", "#\r\n#   source tabular data:\r\n"]
+[65.098736, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[65.099085, "o", "\r\n#\r\n#   only show rows matching /20:\r\n"]
+[65.099283, "o", "\r\n% tablizer /20 -A table.demo1"]
+[66.101537, "o", "\r\n"]
+[66.109112, "o", "NAME(1)\tDURATION(2)\tCOUNT(3)\tWHEN(4)                    \r\nbeta   \t1d10h5m1s  \t33      \t3/1\u001b[102;30m/20\u001b[0m14                  \t\r\nceta   \t33d12h     \t9       \t06/Jan\u001b[102;30m/20\u001b[0m08 15:04:05 -0700\t\r\n"]
+[66.109405, "o", "\r\n% "]
+[70.11076, "o", "\u001b[H\u001b[2J\u001b[3J"]
+[70.110873, "o", "#\r\n#   source tabular data:\r\n"]
+[70.111365, "o", "NAME   DURATION   COUNT  WHEN\r\nbeta   1d10h5m1s  33     3/1/2014\r\nalpha  4h35m      170    2013-Feb-03\r\nceta   33d12h     9      06/Jan/2008 15:04:05 -0700\r\n"]
+[70.111469, "o", "\r\n#\r\n#   only show rows NOT matching /20:\r\n\r\n% tablizer /20 -A -v table.demo1"]
+[71.112738, "o", "\r\n"]
+[71.120032, "o", "NAME(1)\tDURATION(2)\tCOUNT(3)\tWHEN(4)     \r\nalpha  \t4h35m      \t170     \t2013-Feb-03\t\r\n"]
+[71.121127, "o", "\r\n% "]
+[75.126199, "o", "\u001b[H\u001b[2J\u001b[3J"]

go.mod

@@ -4,13 +4,18 @@ go 1.18
 
 require (
 	github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897
+	github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de
+	github.com/gookit/color v1.5.2
 	github.com/olekukonko/tablewriter v0.0.5
 	github.com/spf13/cobra v1.5.0
+	gopkg.in/yaml.v3 v3.0.1
 )
 
 require (
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
-	github.com/mattn/go-runewidth v0.0.9 // indirect
+	github.com/mattn/go-runewidth v0.0.10 // indirect
+	github.com/rivo/uniseg v0.1.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
-	github.com/stretchr/testify v1.8.0 // indirect
+	github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
+	golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 // indirect
 )

go.sum

@@ -1,27 +1,41 @@
 github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897 h1:p9Sln00KOTlrYkxI1zYWl1QLnEqAqEARBEYa8FQnQcY=
 github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897/go.mod h1:xTS7Pm1pD1mvyM075QCDSRqH6qRLXylzS24ZTpRiSzQ=
+github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de h1:FxWPpzIjnTlhPwqqXc4/vE0f7GvRjuAsbW+HOIe8KnA=
+github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de/go.mod h1:DCaWoUhZrYW9p1lxo/cm8EmUOOzAPSEZNGF2DK1dJgw=
 github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/gookit/color v1.5.2 h1:uLnfXcaFjlrDnQDT+NCBcfhrXqYTx/rcCa6xn01Y8yI=
+github.com/gookit/color v1.5.2/go.mod h1:w8h4bGiHeeBpvQVePTutdbERIUf3oJE5lZ8HM0UgXyg=
 github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
 github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
-github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0=
 github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
+github.com/mattn/go-runewidth v0.0.10 h1:CoZ3S2P7pvtP45xOtBw+/mDL2z0RKI576gSkzRRpdGg=
+github.com/mattn/go-runewidth v0.0.10/go.mod h1:RAqKPSqVFrSLVXbA8x7dzmKdmGzieGRCM46jaSJTDAk=
 github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec=
 github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/rivo/uniseg v0.1.0 h1:+2KBaVoUmb9XzDsrx/Ct0W/EYOSFf/nWTauy++DprtY=
+github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
+github.com/scylladb/termtables v0.0.0-20191203121021-c4c0b6d42ff4/go.mod h1:C1a7PQSMz9NShzorzCiG2fk9+xuCgLkPeCvMHYR2OWg=
 github.com/spf13/cobra v1.5.0 h1:X+jTBEBqF0bHN+9cSMgmfuvv2VHJ9ezmFNf9Y/XstYU=
 github.com/spf13/cobra v1.5.0/go.mod h1:dWXEIy2H428czQCjInthrTRUg7yKbok+2Qi/yBIJoUM=
 github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
 github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
+github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
+github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 h1:QldyIu/L63oPpyvQmHgvgickp1Yw510KJOqX7H24mg8=
+github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
+golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 h1:Bli41pIlzTzf3KEY06n+xnzK/BESIg2ze4Pgfh/aI8c=
+golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

lib/common.go

@@ -17,31 +17,10 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
 
 package lib
 
-var (
+// contains a whole parsed table
-	// command line flags
+type Tabdata struct {
-	Debug            bool
+	maxwidthHeader int      // longest header
-	XtendedOut       bool
+	columns        int      // count
-	NoNumbering      bool
+	headers        []string // [ "ID", "NAME", ...]
-	ShowVersion      bool
+	entries        [][]string
-	Columns          string
+}
-	UseColumns       []int
-	DefaultSeparator string = `(\s\s+|\t)`
-	Separator        string = `(\s\s+|\t)`
-	OutflagExtended  bool
-	OutflagMarkdown  bool
-	OutflagOrgtable  bool
-	OutflagShell     bool
-	OutputMode       string
-	InvertMatch      bool
-
-	// used for validation
-	validOutputmodes = "(orgtbl|markdown|extended|ascii)"
-
-	// main program version
-	Version = "v1.0.6"
-
-	// generated  version string, used  by -v contains  lib.Version on
-	//  main  branch,   and  lib.Version-$branch-$lastcommit-$date  on
-	// development branch
-	VERSION string
-)

lib/helpers.go

@@ -20,7 +20,11 @@ package lib
 import (
 	"errors"
 	"fmt"
+	"github.com/gookit/color"
+	"github.com/tlinden/tablizer/cfg"
+	"os"
 	"regexp"
+	"sort"
 	"strconv"
 	"strings"
 )
@@ -34,49 +38,99 @@ func contains(s []int, e int) bool {
 	return false
 }
 
-func PrepareColumns() error {
+// parse columns list given  with -c, modifies config.UseColumns based
-	if len(Columns) > 0 {
+// on eventually given regex
-		for _, use := range strings.Split(Columns, ",") {
+func PrepareColumns(c *cfg.Config, data *Tabdata) error {
-			usenum, err := strconv.Atoi(use)
+	if len(c.Columns) > 0 {
-			if err != nil {
+		for _, use := range strings.Split(c.Columns, ",") {
-				msg := fmt.Sprintf("Could not parse columns list %s: %v", Columns, err)
+			if len(use) == 0 {
+				msg := fmt.Sprintf("Could not parse columns list %s: empty column", c.Columns)
 				return errors.New(msg)
 			}
-			UseColumns = append(UseColumns, usenum)
+
+			usenum, err := strconv.Atoi(use)
+			if err != nil {
+				// might be a regexp
+				colPattern, err := regexp.Compile(use)
+				if err != nil {
+					msg := fmt.Sprintf("Could not parse columns list %s: %v", c.Columns, err)
+					return errors.New(msg)
 				}
+
+				// find matching header fields
+				for i, head := range data.headers {
+					if colPattern.MatchString(head) {
+						c.UseColumns = append(c.UseColumns, i+1)
+					}
+
+				}
+			} else {
+				// we digress from go  best practises here, because if
+				// a colum spec is not a number, we process them above
+				// inside the err handler  for atoi(). so only add the
+				// number, if it's really just a number.
+				c.UseColumns = append(c.UseColumns, usenum)
+			}
+		}
+
+		// deduplicate: put all values into a map (value gets map key)
+		// thereby  removing duplicates,  extract keys into  new slice
+		// and sort it
+		imap := make(map[int]int, len(c.UseColumns))
+		for _, i := range c.UseColumns {
+			imap[i] = 0
+		}
+		c.UseColumns = nil
+		for k := range imap {
+			c.UseColumns = append(c.UseColumns, k)
+		}
+		sort.Ints(c.UseColumns)
 	}
 	return nil
 }
 
-func numberizeHeaders(data *Tabdata) {
 // prepare headers: add numbers to headers
+func numberizeAndReduceHeaders(c cfg.Config, data *Tabdata) {
 	numberedHeaders := []string{}
+	maxwidth := 0 // start from scratch, so we only look at displayed column widths
+
 	for i, head := range data.headers {
-		if len(Columns) > 0 {
+		headlen := 0
+		if len(c.Columns) > 0 {
 			// -c specified
-			if !contains(UseColumns, i+1) {
+			if !contains(c.UseColumns, i+1) {
 				// ignore this one
 				continue
 			}
 		}
-		if NoNumbering {
+		if c.NoNumbering {
 			numberedHeaders = append(numberedHeaders, head)
+			headlen = len(head)
 		} else {
-			numberedHeaders = append(numberedHeaders, fmt.Sprintf("%s(%d)", head, i+1))
+			numhead := fmt.Sprintf("%s(%d)", head, i+1)
+			headlen = len(numhead)
+			numberedHeaders = append(numberedHeaders, numhead)
+		}
+
+		if headlen > maxwidth {
+			maxwidth = headlen
 		}
 	}
 	data.headers = numberedHeaders
+	if data.maxwidthHeader != maxwidth && maxwidth > 0 {
+		data.maxwidthHeader = maxwidth
+	}
 }
 
-func reduceColumns(data *Tabdata) {
 // exclude columns, if any
-	if len(Columns) > 0 {
+func reduceColumns(c cfg.Config, data *Tabdata) {
+	if len(c.Columns) > 0 {
 		reducedEntries := [][]string{}
-		reducedEntry := []string{}
+		var reducedEntry []string
 		for _, entry := range data.entries {
 			reducedEntry = nil
 			for i, value := range entry {
-				if !contains(UseColumns, i+1) {
+				if !contains(c.UseColumns, i+1) {
 					continue
 				}
 
@@ -88,39 +142,6 @@ func reduceColumns(data *Tabdata) {
 	}
 }
 
-func PrepareModeFlags() error {
-	if len(OutputMode) == 0 {
-		// associate short flags like -X with mode selector
-		switch {
-		case OutflagExtended:
-			OutputMode = "extended"
-		case OutflagMarkdown:
-			OutputMode = "markdown"
-		case OutflagOrgtable:
-			OutputMode = "orgtbl"
-		case OutflagShell:
-			OutputMode = "shell"
-			NoNumbering = true
-		default:
-			OutputMode = "ascii"
-		}
-	} else {
-		r, err := regexp.Compile(validOutputmodes)
-
-		if err != nil {
-			return errors.New("Failed to validate output mode spec!")
-		}
-
-		match := r.MatchString(OutputMode)
-
-		if !match {
-			return errors.New("Invalid output mode!")
-		}
-	}
-
-	return nil
-}
-
 func trimRow(row []string) []string {
 	// FIXME: remove this when we only use Tablewriter and strip in ParseFile()!
 	var fixedrow []string
@@ -130,3 +151,14 @@ func trimRow(row []string) []string {
 
 	return fixedrow
 }
+
+func colorizeData(c cfg.Config, output string) string {
+	if len(c.Pattern) > 0 && !c.NoColor && color.IsConsole(os.Stdout) {
+		r := regexp.MustCompile("(" + c.Pattern + ")")
+		return r.ReplaceAllStringFunc(output, func(in string) string {
+			return c.ColorStyle.Sprint(in)
+		})
+	} else {
+		return output
+	}
+}

lib/helpers_test.go

@@ -19,11 +19,12 @@ package lib
 
 import (
 	"fmt"
+	"github.com/tlinden/tablizer/cfg"
 	"reflect"
 	"testing"
 )
 
-func Testcontains(t *testing.T) {
+func TestContains(t *testing.T) {
 	var tests = []struct {
 		list   []int
 		search int
@@ -45,6 +46,19 @@ func Testcontains(t *testing.T) {
 }
 
 func TestPrepareColumns(t *testing.T) {
+	data := Tabdata{
+		maxwidthHeader: 5,
+		columns:        3,
+		headers: []string{
+			"ONE", "TWO", "THREE",
+		},
+		entries: [][]string{
+			{
+				"2", "3", "4",
+			},
+		},
+	}
+
 	var tests = []struct {
 		input     string
 		exp       []int
@@ -52,21 +66,23 @@ func TestPrepareColumns(t *testing.T) {
 	}{
 		{"1,2,3", []int{1, 2, 3}, false},
 		{"1,2,", []int{}, true},
-		{"a,b", []int{}, true},
+		{"T", []int{2, 3}, false},
+		{"T,2,3", []int{2, 3}, false},
+		{"[a-z,4,5", []int{4, 5}, true}, // invalid regexp
 	}
 
 	for _, tt := range tests {
 		testname := fmt.Sprintf("PrepareColumns-%s-%t", tt.input, tt.wanterror)
 		t.Run(testname, func(t *testing.T) {
-			Columns = tt.input
+			c := cfg.Config{Columns: tt.input}
-			err := PrepareColumns()
+			err := PrepareColumns(&c, &data)
 			if err != nil {
 				if !tt.wanterror {
 					t.Errorf("got error: %v", err)
 				}
 			} else {
-				if !reflect.DeepEqual(UseColumns, tt.exp) {
+				if !reflect.DeepEqual(c.UseColumns, tt.exp) {
-					t.Errorf("got: %v, expected: %v", UseColumns, tt.exp)
+					t.Errorf("got: %v, expected: %v", c.UseColumns, tt.exp)
 				}
 			}
 		})
@@ -79,15 +95,15 @@ func TestReduceColumns(t *testing.T) {
 		columns []int
 	}{
 		{
-			expect:  [][]string{[]string{"a", "b"}},
+			expect:  [][]string{{"a", "b"}},
 			columns: []int{1, 2},
 		},
 		{
-			expect:  [][]string{[]string{"a", "c"}},
+			expect:  [][]string{{"a", "c"}},
 			columns: []int{1, 3},
 		},
 		{
-			expect:  [][]string{[]string{"a"}},
+			expect:  [][]string{{"a"}},
 			columns: []int{1},
 		},
 		{
@@ -96,22 +112,46 @@ func TestReduceColumns(t *testing.T) {
 		},
 	}
 
-	input := [][]string{[]string{"a", "b", "c"}}
+	input := [][]string{{"a", "b", "c"}}
-
-	Columns = "y" // used as a flag with len(Columns)...
 
 	for _, tt := range tests {
 		testname := fmt.Sprintf("reduce-columns-by-%+v", tt.columns)
 		t.Run(testname, func(t *testing.T) {
-			UseColumns = tt.columns
+			c := cfg.Config{Columns: "x", UseColumns: tt.columns}
 			data := Tabdata{entries: input}
-			reduceColumns(&data)
+			reduceColumns(c, &data)
 			if !reflect.DeepEqual(data.entries, tt.expect) {
 				t.Errorf("reduceColumns returned invalid data:\ngot: %+v\nexp: %+v", data.entries, tt.expect)
 			}
 		})
 	}
-
+}
-	Columns = "" // reset for other tests
+
-	UseColumns = nil
+func TestNumberizeHeaders(t *testing.T) {
+	data := Tabdata{
+		headers: []string{"ONE", "TWO", "THREE"},
+	}
+
+	var tests = []struct {
+		expect  []string
+		columns []int
+		nonum   bool
+	}{
+		{[]string{"ONE(1)", "TWO(2)", "THREE(3)"}, []int{1, 2, 3}, false},
+		{[]string{"ONE(1)", "TWO(2)"}, []int{1, 2}, false},
+		{[]string{"ONE", "TWO"}, []int{1, 2}, true},
+	}
+
+	for _, tt := range tests {
+		testname := fmt.Sprintf("numberize-headers-columns-%+v-nonum-%t", tt.columns, tt.nonum)
+		t.Run(testname, func(t *testing.T) {
+			c := cfg.Config{Columns: "x", UseColumns: tt.columns, NoNumbering: tt.nonum}
+			usedata := data
+			numberizeAndReduceHeaders(c, &usedata)
+			if !reflect.DeepEqual(usedata.headers, tt.expect) {
+				t.Errorf("numberizeAndReduceHeaders returned invalid data:\ngot: %+v\nexp: %+v",
+					usedata.headers, tt.expect)
+			}
+		})
+	}
 }

lib/io.go

@@ -19,29 +19,40 @@ package lib
 
 import (
 	"errors"
+	"github.com/tlinden/tablizer/cfg"
 	"io"
 	"os"
 )
 
-func ProcessFiles(args []string) error {
+func ProcessFiles(c *cfg.Config, args []string) error {
-	fds, pattern, err := determineIO(args)
+	fds, pattern, err := determineIO(c, args)
 
 	if err != nil {
 		return err
 	}
 
+	if err := c.PreparePattern(pattern); err != nil {
+		return err
+	}
+
 	for _, fd := range fds {
-		data, err := parseFile(fd, pattern)
+		data, err := Parse(*c, fd)
 		if err != nil {
 			return err
 		}
-		printData(&data)
+
+		err = PrepareColumns(c, &data)
+		if err != nil {
+			return err
+		}
+
+		printData(os.Stdout, *c, &data)
 	}
 
 	return nil
 }
 
-func determineIO(args []string) ([]io.Reader, string, error) {
+func determineIO(c *cfg.Config, args []string) ([]io.Reader, string, error) {
 	var pattern string
 	var fds []io.Reader
 	var havefiles bool
@@ -52,6 +63,7 @@ func determineIO(args []string) ([]io.Reader, string, error) {
 			// first  one is  not a  file, consider  it as  regexp and
 			// shift arg list
 			pattern = args[0]
+			c.Pattern = args[0] // used for colorization by printData()
 			args = args[1:]
 		}
 

lib/parser.go

@@ -19,37 +19,94 @@ package lib
 
 import (
 	"bufio"
+	"encoding/csv"
 	"errors"
 	"fmt"
 	"github.com/alecthomas/repr"
+	"github.com/tlinden/tablizer/cfg"
 	"io"
 	"regexp"
 	"strings"
 )
 
-// contains a whole parsed table
+/*
-type Tabdata struct {
+   Parser switch
-	maxwidthHeader int      // longest header
+*/
-	maxwidthPerCol []int    // max width per column
+func Parse(c cfg.Config, input io.Reader) (Tabdata, error) {
-	columns        int      // count
+	if len(c.Separator) == 1 {
-	headers        []string // [ "ID", "NAME", ...]
+		return parseCSV(c, input)
-	entries        [][]string
+	}
+
+	return parseTabular(c, input)
+}
+
+/*
+   Parse CSV input.
+*/
+func parseCSV(c cfg.Config, input io.Reader) (Tabdata, error) {
+	var content io.Reader = input
+	data := Tabdata{}
+
+	if len(c.Pattern) > 0 {
+		scanner := bufio.NewScanner(input)
+		lines := []string{}
+		hadFirst := false
+		for scanner.Scan() {
+			line := strings.TrimSpace(scanner.Text())
+			if hadFirst {
+				// don't match 1st line, it's the header
+				if c.PatternR.MatchString(line) == c.InvertMatch {
+					// by default  -v is false, so if a  line does NOT
+					// match the pattern, we will ignore it. However,
+					// if the user specified -v, the matching is inverted,
+					// so we ignore all lines, which DO match.
+					continue
+				}
+			}
+			lines = append(lines, line)
+			hadFirst = true
+		}
+		content = strings.NewReader(strings.Join(lines, "\n"))
+	}
+
+	csvreader := csv.NewReader(content)
+	csvreader.Comma = rune(c.Separator[0])
+
+	records, err := csvreader.ReadAll()
+	if err != nil {
+		return data, errors.Unwrap(fmt.Errorf("Could not parse CSV input: %w", err))
+	}
+
+	if len(records) >= 1 {
+		data.headers = records[0]
+		data.columns = len(records)
+
+		for _, head := range data.headers {
+			// register widest header field
+			headerlen := len(head)
+			if headerlen > data.maxwidthHeader {
+				data.maxwidthHeader = headerlen
+			}
+		}
+
+		if len(records) > 1 {
+			data.entries = records[1:]
+		}
+	}
+
+	return data, nil
 }
 
 /*
    Parse tabular input.
 */
-func parseFile(input io.Reader, pattern string) (Tabdata, error) {
+func parseTabular(c cfg.Config, input io.Reader) (Tabdata, error) {
 	data := Tabdata{}
 
 	var scanner *bufio.Scanner
 
 	hadFirst := false
-	separate := regexp.MustCompile(Separator)
+	separate := regexp.MustCompile(c.Separator)
-	patternR, err := regexp.Compile(pattern)
-	if err != nil {
-		return data, errors.Unwrap(fmt.Errorf("Regexp pattern %s is invalid: %w", pattern, err))
-	}
 
 	scanner = bufio.NewScanner(input)
 
@@ -84,8 +141,8 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
 			}
 		} else {
 			// data processing
-			if len(pattern) > 0 {
+			if len(c.Pattern) > 0 {
-				if patternR.MatchString(line) == InvertMatch {
+				if c.PatternR.MatchString(line) == c.InvertMatch {
 					// by default  -v is false, so if a  line does NOT
 					// match the pattern, we will ignore it. However,
 					// if the user specified -v, the matching is inverted,
@@ -97,16 +154,6 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
 			idx := 0 // we cannot use the header index, because we could exclude columns
 			values := []string{}
 			for _, part := range parts {
-				width := len(strings.TrimSpace(part))
-
-				if len(data.maxwidthPerCol)-1 < idx {
-					data.maxwidthPerCol = append(data.maxwidthPerCol, width)
-				} else {
-					if width > data.maxwidthPerCol[idx] {
-						data.maxwidthPerCol[idx] = width
-					}
-				}
-
 				// if Debug {
 				// 	fmt.Printf("<%s> ", value)
 				// }
@@ -114,6 +161,12 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
 
 				idx++
 			}
+
+			// fill up missing fields, if any
+			for i := len(values); i < len(data.headers); i++ {
+				values = append(values, "")
+			}
+
 			data.entries = append(data.entries, values)
 		}
 	}
@@ -122,7 +175,7 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
 		return data, errors.Unwrap(fmt.Errorf("Failed to read from io.Reader: %w", scanner.Err()))
 	}
 
-	if Debug {
+	if c.Debug {
 		repr.Print(data)
 	}
 

lib/parser_test.go

@@ -19,45 +19,64 @@ package lib
 
 import (
 	"fmt"
+	"github.com/tlinden/tablizer/cfg"
 	"reflect"
 	"strings"
 	"testing"
 )
 
+var input = []struct {
+	name      string
+	text      string
+	separator string
+}{
+	{
+		name:      "tabular-data",
+		separator: cfg.DefaultSeparator,
+		text: `
+ONE    TWO    THREE  
+asd    igig   cxxxncnc  
+19191  EDD 1  X`,
+	},
+	{
+		name:      "csv-data",
+		separator: ",",
+		text: `
+ONE,TWO,THREE
+asd,igig,cxxxncnc
+19191,"EDD 1",X`,
+	},
+}
+
 func TestParser(t *testing.T) {
 	data := Tabdata{
 		maxwidthHeader: 5,
-		maxwidthPerCol: []int{
-			5, 5, 8,
-		},
 		columns:        3,
 		headers: []string{
 			"ONE", "TWO", "THREE",
 		},
 		entries: [][]string{
-			[]string{
+			{"asd", "igig", "cxxxncnc"},
-				"asd", "igig", "cxxxncnc",
+			{"19191", "EDD 1", "X"},
-			},
-			[]string{
-				"19191", "EDD 1", "X",
-			},
 		},
 	}
 
-	table := `ONE    TWO    THREE  
+	for _, in := range input {
-asd    igig   cxxxncnc  
+		testname := fmt.Sprintf("parse-%s", in.name)
-19191  EDD 1  X`
+		t.Run(testname, func(t *testing.T) {
-
+			readFd := strings.NewReader(strings.TrimSpace(in.text))
-	readFd := strings.NewReader(table)
+			c := cfg.Config{Separator: in.separator}
-	gotdata, err := parseFile(readFd, "")
+			gotdata, err := Parse(c, readFd)
-	Separator = DefaultSeparator
 
 			if err != nil {
 				t.Errorf("Parser returned error: %s\nData processed so far: %+v", err, gotdata)
 			}
 
 			if !reflect.DeepEqual(data, gotdata) {
-		t.Errorf("Parser returned invalid data, Regex: %s\nExp: %+v\nGot: %+v\n", Separator, data, gotdata)
+				t.Errorf("Parser returned invalid data\nExp: %+v\nGot: %+v\n",
+					data, gotdata)
+			}
+		})
 	}
 }
 
@@ -66,47 +85,81 @@ func TestParserPatternmatching(t *testing.T) {
 		entries [][]string
 		pattern string
 		invert  bool
+		want    bool
 	}{
 		{
 			entries: [][]string{
-				[]string{
+				{"asd", "igig", "cxxxncnc"},
-					"asd", "igig", "cxxxncnc",
-				},
 			},
 			pattern: "ig",
 			invert:  false,
 		},
 		{
 			entries: [][]string{
-				[]string{
+				{"19191", "EDD 1", "X"},
-					"19191", "EDD 1", "X",
-				},
 			},
 			pattern: "ig",
 			invert:  true,
 		},
 	}
 
-	table := `ONE    TWO    THREE  
+	for _, in := range input {
-asd    igig   cxxxncnc  
+		for _, tt := range tests {
+			testname := fmt.Sprintf("parse-%s-with-pattern-%s-inverted-%t",
+				in.name, tt.pattern, tt.invert)
+			t.Run(testname, func(t *testing.T) {
+				c := cfg.Config{InvertMatch: tt.invert, Pattern: tt.pattern,
+					Separator: in.separator}
+
+				_ = c.PreparePattern(tt.pattern)
+
+				readFd := strings.NewReader(strings.TrimSpace(in.text))
+				gotdata, err := Parse(c, readFd)
+
+				if err != nil {
+					if !tt.want {
+						t.Errorf("Parser returned error: %s\nData processed so far: %+v",
+							err, gotdata)
+					}
+				} else {
+					if !reflect.DeepEqual(tt.entries, gotdata.entries) {
+						t.Errorf("Parser returned invalid data (pattern: %s, invert: %t)\nExp: %+v\nGot: %+v\n",
+							tt.pattern, tt.invert, tt.entries, gotdata.entries)
+					}
+				}
+			})
+		}
+	}
+}
+
+func TestParserIncompleteRows(t *testing.T) {
+	data := Tabdata{
+		maxwidthHeader: 5,
+		columns:        3,
+		headers: []string{
+			"ONE", "TWO", "THREE",
+		},
+		entries: [][]string{
+			{"asd", "igig", ""},
+			{"19191", "EDD 1", "X"},
+		},
+	}
+
+	table := `
+ONE    TWO    THREE  
+asd    igig
 19191  EDD 1  X`
 
-	for _, tt := range tests {
+	readFd := strings.NewReader(strings.TrimSpace(table))
-		testname := fmt.Sprintf("parse-with-inverted-pattern-%t", tt.invert)
+	c := cfg.Config{Separator: cfg.DefaultSeparator}
-		t.Run(testname, func(t *testing.T) {
+	gotdata, err := Parse(c, readFd)
-			InvertMatch = tt.invert
-
-			readFd := strings.NewReader(table)
-			gotdata, err := parseFile(readFd, tt.pattern)
 
 	if err != nil {
 		t.Errorf("Parser returned error: %s\nData processed so far: %+v", err, gotdata)
 	}
 
-			if !reflect.DeepEqual(tt.entries, gotdata.entries) {
+	if !reflect.DeepEqual(data, gotdata) {
-				t.Errorf("Parser returned invalid data (pattern: %s, invert: %t)\nExp: %+v\nGot: %+v\n",
+		t.Errorf("Parser returned invalid data, Regex: %s\nExp: %+v\nGot: %+v\n",
-					tt.pattern, tt.invert, tt.entries, gotdata.entries)
+			c.Separator, data, gotdata)
-			}
-		})
 	}
 }

lib/printer.go

@@ -18,39 +18,60 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
 package lib
 
 import (
+	"encoding/csv"
 	"fmt"
+	"github.com/gookit/color"
 	"github.com/olekukonko/tablewriter"
-	"os"
+	"github.com/tlinden/tablizer/cfg"
+	"gopkg.in/yaml.v3"
+	"io"
+	"log"
 	"regexp"
+	"strconv"
 	"strings"
 )
 
-func printData(data *Tabdata) {
+func printData(w io.Writer, c cfg.Config, data *Tabdata) {
-	if OutputMode != "shell" {
+	// some output preparations:
-		numberizeHeaders(data)
-	}
-	reduceColumns(data)
 
-	switch OutputMode {
+	// add numbers to headers and remove this we're not interested in
-	case "extended":
+	numberizeAndReduceHeaders(c, data)
-		printExtendedData(data)
+
-	case "ascii":
+	// remove unwanted columns, if any
-		printAsciiData(data)
+	reduceColumns(c, data)
-	case "orgtbl":
+
-		printOrgmodeData(data)
+	// sort the data
-	case "markdown":
+	sortTable(c, data)
-		printMarkdownData(data)
+
-	case "shell":
+	switch c.OutputMode {
-		printShellData(data)
+	case cfg.Extended:
+		printExtendedData(w, c, data)
+	case cfg.Ascii:
+		printAsciiData(w, c, data)
+	case cfg.Orgtbl:
+		printOrgmodeData(w, c, data)
+	case cfg.Markdown:
+		printMarkdownData(w, c, data)
+	case cfg.Shell:
+		printShellData(w, c, data)
+	case cfg.Yaml:
+		printYamlData(w, c, data)
+	case cfg.CSV:
+		printCSVData(w, c, data)
 	default:
-		printAsciiData(data)
+		printAsciiData(w, c, data)
 	}
+
+}
+
+func output(w io.Writer, str string) {
+	fmt.Fprint(w, str)
 }
 
 /*
    Emacs org-mode compatible table (also orgtbl-mode)
 */
-func printOrgmodeData(data *Tabdata) {
+func printOrgmodeData(w io.Writer, c cfg.Config, data *Tabdata) {
 	tableString := &strings.Builder{}
 	table := tablewriter.NewWriter(tableString)
 
@@ -73,17 +94,21 @@ func printOrgmodeData(data *Tabdata) {
 	   | cell | cell |
 	   |------+------|
 	*/
-	leftR := regexp.MustCompile("(?m)^\\+")
+	leftR := regexp.MustCompile(`(?m)^\\+`)
-	rightR := regexp.MustCompile("\\+(?m)$")
+	rightR := regexp.MustCompile(`\\+(?m)$`)
 
-	fmt.Print(rightR.ReplaceAllString(leftR.ReplaceAllString(tableString.String(), "|"), "|"))
+	output(w, color.Sprint(
+		colorizeData(c,
+			rightR.ReplaceAllString(
+				leftR.ReplaceAllString(tableString.String(), "|"), "|"))))
 }
 
 /*
    Markdown table
 */
-func printMarkdownData(data *Tabdata) {
+func printMarkdownData(w io.Writer, c cfg.Config, data *Tabdata) {
-	table := tablewriter.NewWriter(os.Stdout)
+	tableString := &strings.Builder{}
+	table := tablewriter.NewWriter(tableString)
 
 	table.SetHeader(data.headers)
 
@@ -95,13 +120,15 @@ func printMarkdownData(data *Tabdata) {
 	table.SetCenterSeparator("|")
 
 	table.Render()
+	output(w, color.Sprint(colorizeData(c, tableString.String())))
 }
 
 /*
    Simple ASCII table without any borders etc, just like the input we expect
 */
-func printAsciiData(data *Tabdata) {
+func printAsciiData(w io.Writer, c cfg.Config, data *Tabdata) {
-	table := tablewriter.NewWriter(os.Stdout)
+	tableString := &strings.Builder{}
+	table := tablewriter.NewWriter(tableString)
 
 	table.SetHeader(data.headers)
 	table.AppendBulk(data.entries)
@@ -123,54 +150,101 @@ func printAsciiData(data *Tabdata) {
 	table.SetNoWhiteSpace(true)
 
 	table.Render()
+	output(w, color.Sprint(colorizeData(c, tableString.String())))
 }
 
 /*
    We simulate the \x command of psql (the PostgreSQL client)
 */
-func printExtendedData(data *Tabdata) {
+func printExtendedData(w io.Writer, c cfg.Config, data *Tabdata) {
 	// needed for data output
-	format := fmt.Sprintf("%%%ds: %%s\n", data.maxwidthHeader) // FIXME: re-calculate if -c has been set
+	format := fmt.Sprintf("%%%ds: %%s\n", data.maxwidthHeader)
-
+	out := ""
 	if len(data.entries) > 0 {
-		var idx int
 		for _, entry := range data.entries {
-			idx = 0
 			for i, value := range entry {
-				if len(Columns) > 0 {
+				out += color.Sprintf(format, data.headers[i], value)
-					if !contains(UseColumns, i+1) {
+			}
-						continue
+
+			out += "\n"
 		}
 	}
 
-				fmt.Printf(format, data.headers[idx], value)
+	output(w, colorizeData(c, out))
-				idx++
-			}
-			fmt.Println()
-		}
-	}
 }
 
 /*
    Shell output, ready to be eval'd. Just like FreeBSD stat(1)
 */
-func printShellData(data *Tabdata) {
+func printShellData(w io.Writer, c cfg.Config, data *Tabdata) {
+	out := ""
 	if len(data.entries) > 0 {
-		var idx int
 		for _, entry := range data.entries {
-			idx = 0
 			shentries := []string{}
 			for i, value := range entry {
-				if len(Columns) > 0 {
+				shentries = append(shentries, fmt.Sprintf("%s=\"%s\"",
-					if !contains(UseColumns, i+1) {
+					data.headers[i], value))
-						continue
+			}
+			out += fmt.Sprint(strings.Join(shentries, " ")) + "\n"
 		}
 	}
 
-				shentries = append(shentries, fmt.Sprintf("%s=\"%s\"", data.headers[idx], value))
+	// no colorization here
-				idx++
+	output(w, out)
 }
-			fmt.Println(strings.Join(shentries, " "))
+
+func printYamlData(w io.Writer, c cfg.Config, data *Tabdata) {
+	type D struct {
+		Entries []map[string]interface{} `yaml:"entries"`
+	}
+
+	d := D{}
+
+	for _, entry := range data.entries {
+		ml := map[string]interface{}{}
+
+		for i, entry := range entry {
+			style := yaml.TaggedStyle
+			_, err := strconv.Atoi(entry)
+			if err != nil {
+				style = yaml.DoubleQuotedStyle
+			}
+
+			ml[strings.ToLower(data.headers[i])] =
+				&yaml.Node{
+					Kind:  yaml.ScalarNode,
+					Style: style,
+					Value: entry}
+		}
+
+		d.Entries = append(d.Entries, ml)
+	}
+
+	yamlstr, err := yaml.Marshal(&d)
+
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	output(w, string(yamlstr))
+}
+
+func printCSVData(w io.Writer, c cfg.Config, data *Tabdata) {
+	csvout := csv.NewWriter(w)
+
+	if err := csvout.Write(data.headers); err != nil {
+		log.Fatalln("error writing record to csv:", err)
+	}
+
+	for _, entry := range data.entries {
+		if err := csvout.Write(entry); err != nil {
+			log.Fatalln("error writing record to csv:", err)
 		}
 	}
+
+	csvout.Flush()
+
+	if err := csvout.Error(); err != nil {
+		log.Fatal(err)
+	}
 }

lib/printer_test.go

@@ -18,82 +18,276 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
 package lib
 
 import (
+	"bytes"
 	"fmt"
-	"os"
+	//"github.com/alecthomas/repr"
+	"github.com/tlinden/tablizer/cfg"
 	"strings"
 	"testing"
 )
 
-func TestPrinter(t *testing.T) {
+func newData() Tabdata {
-	startdata := Tabdata{
+	return Tabdata{
-		maxwidthHeader: 5,
+		maxwidthHeader: 8,
-		maxwidthPerCol: []int{
+		columns:        4,
-			5,
-			5,
-			8,
-		},
-		columns: 3,
 		headers: []string{
-			"ONE", "TWO", "THREE",
+			"NAME",
+			"DURATION",
+			"COUNT",
+			"WHEN",
 		},
 		entries: [][]string{
-			[]string{
+			{
-				"asd", "igig", "cxxxncnc",
+				"beta",
+				"1d10h5m1s",
+				"33",
+				"3/1/2014",
 			},
-			[]string{
+			{
-				"19191", "EDD 1", "X",
+				"alpha",
+				"4h35m",
+				"170",
+				"2013-Feb-03",
+			},
+			{
+				"ceta",
+				"33d12h",
+				"9",
+				"06/Jan/2008 15:04:05 -0700",
 			},
 		},
 	}
-
-	expects := map[string]string{
-		"ascii": `ONE(1)	TWO(2)	THREE(3) 
-asd   	igig  	cxxxncnc	
-19191 	EDD 1 	X`,
-		"orgtbl": `|--------+--------+----------|
-| ONE(1) | TWO(2) | THREE(3) |
-|--------+--------+----------|
-| asd    | igig   | cxxxncnc |
-|  19191 | EDD 1  | X        |
-|--------+--------+----------|`,
-		"markdown": `| ONE(1) | TWO(2) | THREE(3) |
-|--------|--------|----------|
-| asd    | igig   | cxxxncnc |
-|  19191 | EDD 1  | X        |`,
-		"shell": `ONE="asd" TWO="igig" THREE="cxxxncnc"
-ONE="19191" TWO="EDD 1" THREE="X"`,
 }
 
-	r, w, err := os.Pipe()
+var tests = []struct {
-	if err != nil {
+	name string // so we can identify which one fails, can be the same
-		t.Fatal(err)
+	// for multiple tests, because flags will be appended to the name
-	}
+	sortby    string // empty == default
-	origStdout := os.Stdout
+	column    int    // sort by this column, 0 == default first or NO Sort
-	os.Stdout = w
+	desc      bool   // sort in descending order, default == ascending
+	nonum     bool   // hide numbering
+	mode      int    // shell, orgtbl, etc. empty == default: ascii
+	usecol    []int  // columns to display, empty == display all
+	usecolstr string // for testname, must match usecol
+	expect    string // rendered output we expect
+}{
+	// --------------------- Default settings mode tests ``
+	{
+		mode: cfg.Ascii,
+		name: "default",
+		expect: `
+NAME(1)	DURATION(2)	COUNT(3)	WHEN(4)                    
+beta   	1d10h5m1s  	33      	3/1/2014                  	
+alpha  	4h35m      	170     	2013-Feb-03               	
+ceta   	33d12h     	9       	06/Jan/2008 15:04:05 -0700`,
+	},
+	{
+		mode: cfg.CSV,
+		name: "csv",
+		expect: `
+NAME,DURATION,COUNT,WHEN
+beta,1d10h5m1s,33,3/1/2014
+alpha,4h35m,170,2013-Feb-03
+ceta,33d12h,9,06/Jan/2008 15:04:05 -0700`,
+	},
+	{
+		name: "default",
+		mode: cfg.Orgtbl,
+		expect: `
++---------+-------------+----------+----------------------------+
+| NAME(1) | DURATION(2) | COUNT(3) |          WHEN(4)           |
++---------+-------------+----------+----------------------------+
+| beta    | 1d10h5m1s   |       33 | 3/1/2014                   |
+| alpha   | 4h35m       |      170 | 2013-Feb-03                |
+| ceta    | 33d12h      |        9 | 06/Jan/2008 15:04:05 -0700 |
++---------+-------------+----------+----------------------------+`,
+	},
+	{
+		name: "default",
+		mode: cfg.Markdown,
+		expect: `
+| NAME(1) | DURATION(2) | COUNT(3) |          WHEN(4)           |
+|---------|-------------|----------|----------------------------|
+| beta    | 1d10h5m1s   |       33 | 3/1/2014                   |
+| alpha   | 4h35m       |      170 | 2013-Feb-03                |
+| ceta    | 33d12h      |        9 | 06/Jan/2008 15:04:05 -0700 |`,
+	},
+	{
+		name:  "default",
+		mode:  cfg.Shell,
+		nonum: true,
+		expect: `
+NAME="beta" DURATION="1d10h5m1s" COUNT="33" WHEN="3/1/2014"
+NAME="alpha" DURATION="4h35m" COUNT="170" WHEN="2013-Feb-03"
+NAME="ceta" DURATION="33d12h" COUNT="9" WHEN="06/Jan/2008 15:04:05 -0700"`,
+	},
+	{
+		name:  "default",
+		mode:  cfg.Yaml,
+		nonum: true,
+		expect: `
+entries:
+    - count: 33
+      duration: "1d10h5m1s"
+      name: "beta"
+      when: "3/1/2014"
+    - count: 170
+      duration: "4h35m"
+      name: "alpha"
+      when: "2013-Feb-03"
+    - count: 9
+      duration: "33d12h"
+      name: "ceta"
+      when: "06/Jan/2008 15:04:05 -0700"`,
+	},
+	{
+		name: "default",
+		mode: cfg.Extended,
+		expect: `
+    NAME(1): beta
+DURATION(2): 1d10h5m1s
+   COUNT(3): 33
+    WHEN(4): 3/1/2014
 
-	for mode, expect := range expects {
+    NAME(1): alpha
-		testname := fmt.Sprintf("print-%s", mode)
+DURATION(2): 4h35m
+   COUNT(3): 170
+    WHEN(4): 2013-Feb-03
+
+    NAME(1): ceta
+DURATION(2): 33d12h
+   COUNT(3): 9
+    WHEN(4): 06/Jan/2008 15:04:05 -0700`,
+	},
+
+	//------------------------ SORT TESTS
+	{
+		name:   "sortbycolumn",
+		column: 3,
+		sortby: "numeric",
+		desc:   false,
+		expect: `
+NAME(1)	DURATION(2)	COUNT(3)	WHEN(4)                    
+ceta   	33d12h     	9       	06/Jan/2008 15:04:05 -0700	
+beta   	1d10h5m1s  	33      	3/1/2014                  	
+alpha  	4h35m      	170     	2013-Feb-03`,
+	},
+	{
+		name:   "sortbycolumn",
+		column: 4,
+		sortby: "time",
+		desc:   false,
+		expect: `
+NAME(1)	DURATION(2)	COUNT(3)	WHEN(4)                    
+ceta   	33d12h     	9       	06/Jan/2008 15:04:05 -0700	
+alpha  	4h35m      	170     	2013-Feb-03               	
+beta   	1d10h5m1s  	33      	3/1/2014`,
+	},
+	{
+		name:   "sortbycolumn",
+		column: 2,
+		sortby: "duration",
+		desc:   false,
+		expect: `
+NAME(1)	DURATION(2)	COUNT(3)	WHEN(4)                    
+alpha  	4h35m      	170     	2013-Feb-03               	
+beta   	1d10h5m1s  	33      	3/1/2014                  	
+ceta   	33d12h     	9       	06/Jan/2008 15:04:05 -0700`,
+	},
+
+	//  -----------------------  UseColumns Tests
+	{
+		name:      "usecolumns",
+		usecol:    []int{1, 4},
+		usecolstr: "1,4",
+		expect: `
+NAME(1)	WHEN(4)                    
+beta   	3/1/2014                  	
+alpha  	2013-Feb-03               	
+ceta   	06/Jan/2008 15:04:05 -0700`,
+	},
+	{
+		name:      "usecolumns",
+		usecol:    []int{2},
+		usecolstr: "2",
+		expect: `
+DURATION(2) 
+1d10h5m1s  	
+4h35m      	
+33d12h`,
+	},
+	{
+		name:      "usecolumns",
+		usecol:    []int{3},
+		usecolstr: "3",
+		expect: `
+COUNT(3) 
+33      	
+170     	
+9`,
+	},
+	{
+		name:      "usecolumns",
+		column:    0,
+		usecol:    []int{1, 3},
+		usecolstr: "1,3",
+		expect: `
+NAME(1)	COUNT(3) 
+beta   	33      	
+alpha  	170     	
+ceta   	9`,
+	},
+	{
+		name:      "usecolumns",
+		usecol:    []int{2, 4},
+		usecolstr: "2,4",
+		expect: `
+DURATION(2)	WHEN(4)                    
+1d10h5m1s  	3/1/2014                  	
+4h35m      	2013-Feb-03               	
+33d12h     	06/Jan/2008 15:04:05 -0700`,
+	},
+}
+
+func TestPrinter(t *testing.T) {
+	for _, tt := range tests {
+		testname := fmt.Sprintf("print-sortcol-%d-desc-%t-sortby-%s-mode-%d-usecolumns-%s",
+			tt.column, tt.desc, tt.sortby, tt.mode, tt.usecolstr)
 		t.Run(testname, func(t *testing.T) {
+			// replaces os.Stdout, but we ignore it
+			var w bytes.Buffer
 
-			OutputMode = mode
+			// cmd flags
-			data := startdata // we need to reset our mock data, since it's being modified in printData()
+			c := cfg.Config{
-			printData(&data)
+				SortByColumn:   tt.column,
-
+				SortDescending: tt.desc,
-			buf := make([]byte, 1024)
+				SortMode:       tt.sortby,
-			n, err := r.Read(buf)
+				OutputMode:     tt.mode,
-			if err != nil {
+				NoNumbering:    tt.nonum,
-				t.Fatal(err)
+				UseColumns:     tt.usecol,
+				NoColor:        true,
 			}
-			buf = buf[:n]
-			output := strings.TrimSpace(string(buf))
 
-			if output != expect {
+			c.ApplyDefaults()
-				t.Errorf("output mode: %s, got:\n%s\nwant:\n%s\n (%d <=> %d)", mode, output, expect, len(output), len(expect))
+
+			// the test checks the len!
+			if len(tt.usecol) > 0 {
+				c.Columns = "yes"
+			} else {
+				c.Columns = ""
+			}
+
+			testdata := newData()
+			exp := strings.TrimSpace(tt.expect)
+
+			printData(&w, c, &testdata)
+
+			got := strings.TrimSpace(w.String())
+
+			if got != exp {
+				t.Errorf("not rendered correctly:\n+++ got:\n%s\n+++ want:\n%s",
+					got, exp)
 			}
 		})
 	}
-
-	// Restore
-	os.Stdout = origStdout
-
 }

lib/sort.go

@@ -0,0 +1,121 @@
+/*
+Copyright © 2022 Thomas von Dein
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package lib
+
+import (
+	"github.com/araddon/dateparse"
+	"github.com/tlinden/tablizer/cfg"
+	"regexp"
+	"sort"
+	"strconv"
+)
+
+func sortTable(c cfg.Config, data *Tabdata) {
+	if c.SortByColumn <= 0 {
+		// no sorting wanted
+		return
+	}
+
+	// slightly modified here to match internal array indicies
+	col := c.SortByColumn
+
+	col-- // ui starts counting by 1, but use 0 internally
+
+	// sanity checks
+	if len(data.entries) == 0 {
+		return
+	}
+
+	if col >= len(data.headers) {
+		// fall back to default column
+		col = 0
+	}
+
+	// actual sorting
+	sort.SliceStable(data.entries, func(i, j int) bool {
+		return compare(&c, data.entries[i][col], data.entries[j][col])
+	})
+}
+
+// config is not modified here, but it would be inefficient to copy it every loop
+func compare(c *cfg.Config, a string, b string) bool {
+	var comp bool
+
+	switch c.SortMode {
+	case "numeric":
+		left, err := strconv.Atoi(a)
+		if err != nil {
+			left = 0
+		}
+		right, err := strconv.Atoi(b)
+		if err != nil {
+			right = 0
+		}
+		comp = left < right
+	case "duration":
+		left := duration2int(a)
+		right := duration2int(b)
+		comp = left < right
+	case "time":
+		left, _ := dateparse.ParseAny(a)
+		right, _ := dateparse.ParseAny(b)
+		comp = left.Unix() < right.Unix()
+	default:
+		comp = a < b
+	}
+
+	if c.SortDescending {
+		comp = !comp
+	}
+
+	return comp
+}
+
+/*
+   We could use time.ParseDuration(), but this doesn't support days.
+
+   We  could also  use github.com/xhit/go-str2duration/v2,  which does
+   the job,  but it's  just another dependency,  just for  this little
+   gem. And  we don't need a  time.Time value. And int  is good enough
+   for duration comparision.
+
+   Convert a  durartion into  an integer.  Valid  time units  are "s",
+   "m", "h" and "d".
+*/
+func duration2int(duration string) int {
+	re := regexp.MustCompile(`(\d+)([dhms])`)
+	seconds := 0
+
+	for _, match := range re.FindAllStringSubmatch(duration, -1) {
+		if len(match) == 3 {
+			v, _ := strconv.Atoi(match[1])
+			switch match[2][0] {
+			case 'd':
+				seconds += v * 86400
+			case 'h':
+				seconds += v * 3600
+			case 'm':
+				seconds += v * 60
+			case 's':
+				seconds += v
+			}
+		}
+	}
+
+	return seconds
+}

lib/sort_test.go

@@ -0,0 +1,79 @@
+/*
+Copyright © 2022 Thomas von Dein
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package lib
+
+import (
+	"fmt"
+	"github.com/tlinden/tablizer/cfg"
+	"testing"
+)
+
+func TestDuration2Seconds(t *testing.T) {
+	var tests = []struct {
+		dur    string
+		expect int
+	}{
+		{"1d", 60 * 60 * 24},
+		{"1h", 60 * 60},
+		{"10m", 60 * 10},
+		{"2h4m10s", (60 * 120) + (4 * 60) + 10},
+		{"88u", 0},
+		{"19t77X what?4s", 4},
+	}
+
+	for _, tt := range tests {
+		testname := fmt.Sprintf("duration-%s", tt.dur)
+		t.Run(testname, func(t *testing.T) {
+			seconds := duration2int(tt.dur)
+			if seconds != tt.expect {
+				t.Errorf("got %d, want %d", seconds, tt.expect)
+			}
+		})
+	}
+}
+
+func TestCompare(t *testing.T) {
+	var tests = []struct {
+		mode string
+		a    string
+		b    string
+		want bool
+		desc bool
+	}{
+		// ascending
+		{"numeric", "10", "20", true, false},
+		{"duration", "2d4h5m", "45m", false, false},
+		{"time", "12/24/2022", "1/1/1970", false, false},
+
+		// descending
+		{"numeric", "10", "20", false, true},
+		{"duration", "2d4h5m", "45m", true, true},
+		{"time", "12/24/2022", "1/1/1970", true, true},
+	}
+
+	for _, tt := range tests {
+		testname := fmt.Sprintf("compare-mode-%s-a-%s-b-%s-desc-%t", tt.mode, tt.a, tt.b, tt.desc)
+		t.Run(testname, func(t *testing.T) {
+			c := cfg.Config{SortMode: tt.mode, SortDescending: tt.desc}
+			got := compare(&c, tt.a, tt.b)
+			if got != tt.want {
+				t.Errorf("got %t, want %t", got, tt.want)
+			}
+		})
+	}
+}

mkrel.sh

@@ -43,7 +43,7 @@ for D in $DIST; do
     tardir="${tool}-${os}-${arch}-${version}"
     tarfile="releases/${tool}-${os}-${arch}-${version}.tar.gz"
     set -x
-    GOOS=${os} GOARCH=${arch} go build -o ${binfile}
+    GOOS=${os} GOARCH=${arch} go build -o ${binfile} -ldflags "-X 'github.com/tlinden/tablizer/lib.VERSION=${version}'"
     mkdir -p ${tardir}
     cp ${binfile} README.md LICENSE ${tardir}/
     echo 'tool = tablizer

tablizer.1

@@ -133,7 +133,7 @@
 .\" ========================================================================
 .\"
 .IX Title "TABLIZER 1"
-.TH TABLIZER 1 "2022-10-05" "1" "User Commands"
+.TH TABLIZER 1 "2022-11-01" "1" "User Commands"
 .\" For nroff, turn off justification.  Always turn off hyphenation; it makes
 .\" way too many mistakes in technical documents.
 .if n .ad l
@@ -146,28 +146,44 @@ tablizer \- Manipulate tabular output of other programs
 \&    Usage:
 \&      tablizer [regex] [file, ...] [flags]
 \&    
-\&    Flags:
+\&    Operational Flags:
 \&      \-c, \-\-columns string     Only show the speficied columns (separated by ,)
-\&      \-d, \-\-debug              Enable debugging
-\&      \-h, \-\-help               help for tablizer
 \&      \-v, \-\-invert\-match       select non\-matching rows
-\&      \-m, \-\-man                Display manual page
 \&      \-n, \-\-no\-numbering       Disable header numbering
-\&      \-o, \-\-output string      Output mode \- one of: orgtbl, markdown, extended, ascii(default)
+\&      \-N, \-\-no\-color           Disable pattern highlighting
+\&      \-s, \-\-separator string   Custom field separator
+\&      \-k, \-\-sort\-by int        Sort by column (default: 1)
+\&
+\&    Output Flags (mutually exclusive):
 \&      \-X, \-\-extended           Enable extended output
 \&      \-M, \-\-markdown           Enable markdown table output
 \&      \-O, \-\-orgtbl             Enable org\-mode table output
-\&      \-s, \-\-separator string   Custom field separator
+\&      \-S, \-\-shell              Enable shell evaluable ouput
+\&      \-Y, \-\-yaml               Enable yaml output
+\&      \-C, \-\-csv                Enable CSV output
+\&      \-A, \-\-ascii              Default output mode, ascii tabular
+\&
+\&    Sort Mode Flags (mutually exclusive):
+\&      \-a, \-\-sort\-age           sort according to age (duration) string
+\&      \-D, \-\-sort\-desc          Sort in descending order (default: ascending)
+\&      \-i, \-\-sort\-numeric       sort according to string numerical value
+\&      \-t, \-\-sort\-time          sort according to time string
+\&
+\&    Other Flags:
+\&          \-\-completion <shell> Generate the autocompletion script for <shell>
+\&      \-d, \-\-debug              Enable debugging
+\&      \-h, \-\-help               help for tablizer
+\&      \-m, \-\-man                Display manual page
 \&      \-v, \-\-version            Print program version
 .Ve
 .SH "DESCRIPTION"
 .IX Header "DESCRIPTION"
 Many  programs generate  tabular output.   But sometimes  you need  to
 post-process these tables, you may need  to remove one or more columns
-or you may want to filter for  some pattern or you may need the output
+or you  may want to filter  for some pattern (See  \s-1PATTERNS\s0) or you
-in another program and need to  parse it somehow.  Standard unix tools
+may need the  output in another program and need  to parse it somehow.
-such as  \fBawk\fR\|(1), \fBgrep\fR\|(1) or \fBcolumn\fR\|(1)  may help, but sometimes  it's a
+Standard unix tools such as \fBawk\fR\|(1), \fBgrep\fR\|(1) or \fBcolumn\fR\|(1) may help, but
-tedious business.
+sometimes it's a tedious business.
 .PP
 Let's take  the output of  the tool  kubectl.  It contains  cells with
 withespace and they do not separate columns by \s-1TAB\s0 characters. This is
@@ -175,8 +191,8 @@ not easy to process.
 .PP
 You can use \fBtablizer\fR to do these and more things.
 .PP
-\&\fBtablizer\fR analyses the header fiels of a table, registers the column
+\&\fBtablizer\fR  analyses the  header  fields of  a  table, registers  the
-positions  of  each  header  field  and  separates  columns  by  those
+column positions of  each header field and separates  columns by those
 positions.
 .PP
 Without any options it reads its input from \f(CW\*(C`STDIN\*(C'\fR, but you can also
@@ -207,7 +223,7 @@ have a numer associated with it, e.g.:
 .Ve
 .PP
 These numbers denote the column and  you can use them to specify which
-columns you want to have in your output:
+columns you want to have in your output (see \s-1COLUMNS\s0:
 .PP
 .Vb 1
 \&   kubectl get pods | tablizer \-c1,3
@@ -218,14 +234,97 @@ the original order.
 .PP
 The numbering can be suppressed by using the \fB\-n\fR option.
 .PP
+By  default, if  a  \fBpattern\fR  has been  speficied,  matches will  be
+highlighted. You can disable this behavior with the \fB\-N\fR option.
+.PP
+Use the  \fB\-k\fR option to specify  by which column to  sort the tabular
+data (as in \s-1GNU\s0 \fBsort\fR\|(1)). The default sort column is the first one. To
+disable sorting at all, supply 0  (Zero) to \-k. The default sort order
+is ascending. You can change this to descending order using the option
+\&\fB\-D\fR. The default  sort order is by string, but  there are other sort
+modes:
+.IP "\fB\-a \-\-sort\-age\fR" 4
+.IX Item "-a --sort-age"
+Sorts duration strings like \*(L"1d4h32m51s\*(R".
+.IP "\fB\-i \-\-sort\-numeric\fR" 4
+.IX Item "-i --sort-numeric"
+Sorts numeric fields.
+.IP "\fB\-t \-\-sort\-time\fR" 4
+.IX Item "-t --sort-time"
+Sorts timestamps.
+.PP
 Finally the  \fB\-d\fR option  enables debugging  output which  is mostly
-usefull for the developer.
+useful for the developer.
+.SS "\s-1PATTERNS\s0"
+.IX Subsection "PATTERNS"
+You can reduce the rows being  displayed by using a regular expression
+pattern.  The  regexp is  \s-1PCRE\s0 compatible, refer  to the  syntax cheat
+sheet here: <https://github.com/google/re2/wiki/Syntax>.  If you want
+to read  a more comprehensive  documentation about the topic  and have
+perl installed you can read it with:
+.PP
+.Vb 1
+\&    perldoc perlre
+.Ve
+.PP
+Or read it online: <https://perldoc.perl.org/perlre>.
+.PP
+A note on  modifiers: the regexp engine used in  tablizer uses another
+modifier syntax:
+.PP
+.Vb 1
+\&    (?MODIFIER)
+.Ve
+.PP
+The most important modifiers are:
+.PP
+\&\f(CW\*(C`i\*(C'\fR ignore case
+\&\f(CW\*(C`m\*(C'\fR multiline mode
+\&\f(CW\*(C`s\*(C'\fR single line mode
+.PP
+Example for a case insensitive search:
+.PP
+.Vb 1
+\&    kubectl get pods \-A | tablizer "(?i)account"
+.Ve
+.SS "\s-1COLUMNS\s0"
+.IX Subsection "COLUMNS"
+The  parameter  \fB\-c\fR  can  be  used  to  specify,  which  columns  to
+display.  By default  tablizer numerizes  the header  names and  these
+numbers can  be used to specify  which header to display,  see example
+above.
+.PP
+However, beside  numbers, you  can also  use regular  expressions with
+\&\fB\-c\fR, also  separated by comma. And  you can mix column  numbers with
+regexps.
+.PP
+Lets take this table:
+.PP
+.Vb 4
+\&        PID TTY          TIME CMD
+\&      14001 pts/0    00:00:00 bash
+\&      42871 pts/0    00:00:00 ps
+\&      42872 pts/0    00:00:00 sed
+.Ve
+.PP
+We want to see only the \s-1CMD\s0 column and use a regex for this:
+.PP
+.Vb 6
+\&    ps | tablizer \-s \*(Aq\es+\*(Aq \-c C
+\&    CMD(4)
+\&    bash
+\&    ps
+\&    tablizer
+\&    sed
+.Ve
+.PP
+where \*(L"C\*(R" is our regexp which matches \s-1CMD.\s0
 .SS "\s-1OUTPUT MODES\s0"
 .IX Subsection "OUTPUT MODES"
 There might be cases  when the tabular output of a  program is way too
 large  for your  current  terminal but  you still  need  to see  every
 column.   In such  cases the  \fB\-o extended\fR  or \fB\-X\fR  option can  be
-usefull which enables \fIextended mode\fR. In this mode, each row will be
+useful which enables \fIextended mode\fR. In  this mode, each row will be
 printed vertically,  header left,  value right,  aligned by  the field
 widths. Here's an example:
 .PP
@@ -256,7 +355,84 @@ You can use this in an eval loop.
 .PP
 Beside normal  ascii mode  (the default) and  extended mode  there are
 more output modes available: \fBorgtbl\fR  which prints an Emacs org-mode
-table and \fBmarkdown\fR which prints a Markdown table.
+table and  \fBmarkdown\fR which prints  a Markdown table,  \fByaml\fR, which
+prints  yaml encoding  and \s-1CSV\s0  mode, which  prints a  comma separated
+value file.
+.SS "\s-1ENVIRONMENT VARIABLES\s0"
+.IX Subsection "ENVIRONMENT VARIABLES"
+\&\fBtablizer\fR supports  certain environment variables which  use can use
+to  influence   program  behavior.   Commandline  flags   have  always
+precedence over environment variables.
+.IP "<T_NO_HEADER_NUMBERING> \- disable numbering of header fields, like \fB\-n\fR." 4
+.IX Item "<T_NO_HEADER_NUMBERING> - disable numbering of header fields, like -n."
+.PD 0
+.IP "<T_COLUMNS> \- comma separated list of columns to output, like \fB\-c\fR" 4
+.IX Item "<T_COLUMNS> - comma separated list of columns to output, like -c"
+.IP "<\s-1NO_COLORS\s0> \- disable colorization of matches, like \fB\-N\fR" 4
+.IX Item "<NO_COLORS> - disable colorization of matches, like -N"
+.PD
+.SS "\s-1COMPLETION\s0"
+.IX Subsection "COMPLETION"
+Shell completion for command line options  can be enabled by using the
+\&\fB\-\-completion\fR  flag. The  required  parameter is  the  name of  your
+shell. Currently supported are: bash, zsh, fish and powershell.
+.PP
+Detailed instructions:
+.IP "Bash:" 4
+.IX Item "Bash:"
+.Vb 1
+\&   source <(tablizer \-\-completion bash)
+.Ve
+.Sp
+To load completions for each session, execute once:
+.Sp
+.Vb 2
+\&  # Linux:
+\&  $ tablizer \-\-completion bash > /etc/bash_completion.d/tablizer
+\&
+\&  # macOS:
+\&  $ tablizer \-\-completion bash > $(brew \-\-prefix)/etc/bash_completion.d/tablizer
+.Ve
+.IP "Zsh:" 4
+.IX Item "Zsh:"
+If shell completion is not already enabled in your environment,
+you will need to enable it.  You can execute the following once:
+.Sp
+.Vb 1
+\&  echo "autoload \-U compinit; compinit" >> ~/.zshrc
+.Ve
+.Sp
+To load completions for each session, execute once:
+.Sp
+.Vb 1
+\&  $ tablizer \-\-completion zsh > "${fpath[1]}/_tablizer"
+.Ve
+.Sp
+You will need to start a new shell for this setup to take effect.
+.IP "fish:" 4
+.IX Item "fish:"
+.Vb 1
+\&   tablizer \-\-completion fish | source
+.Ve
+.Sp
+To load completions for each session, execute once:
+.Sp
+.Vb 1
+\&   tablizer \-\-completion fish > ~/.config/fish/completions/tablizer.fish
+.Ve
+.IP "PowerShell:" 4
+.IX Item "PowerShell:"
+.Vb 1
+\&   tablizer \-\-completion powershell | Out\-String | Invoke\-Expression
+.Ve
+.Sp
+To load completions for every new session, run:
+.Sp
+.Vb 1
+\&   tablizer \-\-completion powershell > tablizer.ps1
+.Ve
+.Sp
+and source this file from your PowerShell profile.
 .SH "BUGS"
 .IX Header "BUGS"
 In order to report a bug, unexpected behavior, feature requests

tablizer.pod

@@ -7,18 +7,34 @@ tablizer - Manipulate tabular output of other programs
     Usage:
       tablizer [regex] [file, ...] [flags]
     
-    Flags:
+    Operational Flags:
       -c, --columns string     Only show the speficied columns (separated by ,)
-      -d, --debug              Enable debugging
-      -h, --help               help for tablizer
       -v, --invert-match       select non-matching rows
-      -m, --man                Display manual page
       -n, --no-numbering       Disable header numbering
-      -o, --output string      Output mode - one of: orgtbl, markdown, extended, ascii(default)
+      -N, --no-color           Disable pattern highlighting
+      -s, --separator string   Custom field separator
+      -k, --sort-by int        Sort by column (default: 1)
+
+    Output Flags (mutually exclusive):
       -X, --extended           Enable extended output
       -M, --markdown           Enable markdown table output
       -O, --orgtbl             Enable org-mode table output
-      -s, --separator string   Custom field separator
+      -S, --shell              Enable shell evaluable ouput
+      -Y, --yaml               Enable yaml output
+      -C, --csv                Enable CSV output
+      -A, --ascii              Default output mode, ascii tabular
+
+    Sort Mode Flags (mutually exclusive):
+      -a, --sort-age           sort according to age (duration) string
+      -D, --sort-desc          Sort in descending order (default: ascending)
+      -i, --sort-numeric       sort according to string numerical value
+      -t, --sort-time          sort according to time string
+
+    Other Flags:
+          --completion <shell> Generate the autocompletion script for <shell>
+      -d, --debug              Enable debugging
+      -h, --help               help for tablizer
+      -m, --man                Display manual page
       -v, --version            Print program version
 
 
@@ -37,8 +53,8 @@ not easy to process.
 
 You can use B<tablizer> to do these and more things.
 
-B<tablizer> analyses the header fiels of a table, registers the column
+B<tablizer>  analyses the  header  fields of  a  table, registers  the
-positions  of  each  header  field  and  separates  columns  by  those
+column positions of  each header field and separates  columns by those
 positions.
 
 Without any options it reads its input from C<STDIN>, but you can also
@@ -65,7 +81,7 @@ have a numer associated with it, e.g.:
    NAME(1) READY(2) STATUS(3) RESTARTS(4) AGE(5)
 
 These numbers denote the column and  you can use them to specify which
-columns you want to have in your output:
+columns you want to have in your output (see L<COLUMNS>:
 
    kubectl get pods | tablizer -c1,3
 
@@ -74,8 +90,34 @@ the original order.
 
 The numbering can be suppressed by using the B<-n> option.
 
+By  default, if  a  B<pattern>  has been  speficied,  matches will  be
+highlighted. You can disable this behavior with the B<-N> option.
+
+Use the  B<-k> option to specify  by which column to  sort the tabular
+data (as in GNU sort(1)). The default sort column is the first one. To
+disable sorting at all, supply 0  (Zero) to -k. The default sort order
+is ascending. You can change this to descending order using the option
+B<-D>. The default  sort order is by string, but  there are other sort
+modes:
+
+=over
+
+=item B<-a --sort-age>
+
+Sorts duration strings like "1d4h32m51s".
+
+=item B<-i --sort-numeric>
+
+Sorts numeric fields.
+
+=item B<-t --sort-time>
+
+Sorts timestamps.
+
+=back
+
 Finally the  B<-d> option  enables debugging  output which  is mostly
-usefull for the developer.
+useful for the developer.
 
 =head2 PATTERNS
 
@@ -100,17 +142,46 @@ C<i> ignore case
 C<m> multiline mode
 C<s> single line mode
 
-Example for a case insensitve search:
+Example for a case insensitive search:
 
     kubectl get pods -A | tablizer "(?i)account"
 
 
+=head2 COLUMNS
+
+The  parameter  B<-c>  can  be  used  to  specify,  which  columns  to
+display.  By default  tablizer numerizes  the header  names and  these
+numbers can  be used to specify  which header to display,  see example
+above.
+
+However, beside  numbers, you  can also  use regular  expressions with
+B<-c>, also  separated by comma. And  you can mix column  numbers with
+regexps.
+
+Lets take this table:
+
+        PID TTY          TIME CMD
+      14001 pts/0    00:00:00 bash
+      42871 pts/0    00:00:00 ps
+      42872 pts/0    00:00:00 sed
+
+We want to see only the CMD column and use a regex for this:
+
+    ps | tablizer -s '\s+' -c C
+    CMD(4)
+    bash
+    ps
+    tablizer
+    sed
+
+where "C" is our regexp which matches CMD.
+
 =head2 OUTPUT MODES
 
 There might be cases  when the tabular output of a  program is way too
 large  for your  current  terminal but  you still  need  to see  every
 column.   In such  cases the  B<-o extended>  or B<-X>  option can  be
-usefull which enables I<extended mode>. In this mode, each row will be
+useful which enables I<extended mode>. In  this mode, each row will be
 printed vertically,  header left,  value right,  aligned by  the field
 widths. Here's an example:
 
@@ -137,7 +208,80 @@ You can use this in an eval loop.
 
 Beside normal  ascii mode  (the default) and  extended mode  there are
 more output modes available: B<orgtbl>  which prints an Emacs org-mode
-table and B<markdown> which prints a Markdown table.
+table and  B<markdown> which prints  a Markdown table,  B<yaml>, which
+prints  yaml encoding  and CSV  mode, which  prints a  comma separated
+value file.
+
+=head2 ENVIRONMENT VARIABLES
+
+B<tablizer> supports  certain environment variables which  use can use
+to  influence   program  behavior.   Commandline  flags   have  always
+precedence over environment variables.
+
+=over
+
+=item <T_NO_HEADER_NUMBERING> - disable numbering of header fields, like B<-n>.
+
+=item <T_COLUMNS> - comma separated list of columns to output, like B<-c>
+
+=item <NO_COLORS> - disable colorization of matches, like B<-N>
+
+=back
+
+=head2 COMPLETION
+
+Shell completion for command line options  can be enabled by using the
+B<--completion>  flag. The  required  parameter is  the  name of  your
+shell. Currently supported are: bash, zsh, fish and powershell.
+
+Detailed instructions:
+
+=over
+
+=item Bash:
+
+   source <(tablizer --completion bash)
+
+To load completions for each session, execute once:
+
+  # Linux:
+  $ tablizer --completion bash > /etc/bash_completion.d/tablizer
+
+  # macOS:
+  $ tablizer --completion bash > $(brew --prefix)/etc/bash_completion.d/tablizer
+
+=item Zsh:
+
+If shell completion is not already enabled in your environment,
+you will need to enable it.  You can execute the following once:
+
+  echo "autoload -U compinit; compinit" >> ~/.zshrc
+
+To load completions for each session, execute once:
+
+  $ tablizer --completion zsh > "${fpath[1]}/_tablizer"
+
+You will need to start a new shell for this setup to take effect.
+
+=item fish:
+
+   tablizer --completion fish | source
+
+To load completions for each session, execute once:
+
+   tablizer --completion fish > ~/.config/fish/completions/tablizer.fish
+
+=item PowerShell:
+
+   tablizer --completion powershell | Out-String | Invoke-Expression
+
+To load completions for every new session, run:
+
+   tablizer --completion powershell > tablizer.ps1
+
+and source this file from your PowerShell profile.
+
+=back
 
 =head1 BUGS