re-organized pattern matching code

now use a const struct via func.

.github/workflows/ci.yaml

@@ -23,3 +23,36 @@ jobs:
 
     - name: test
       run: make test
+
+  golangci:
+    name: lint
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/setup-go@v3
+        with:
+          go-version: 1.17
+      - uses: actions/checkout@v3
+      - name: golangci-lint
+        uses: golangci/golangci-lint-action@v3
+        #with:
+          # Optional: version of golangci-lint to use in form of v1.2 or v1.2.3 or `latest` to use the latest version
+          # version: v1.29
+
+          # Optional: working directory, useful for monorepos
+          # working-directory: somedir
+
+          # Optional: golangci-lint command line arguments.
+          # args: --issues-exit-code=0
+
+          # Optional: show only new issues if it's a pull request. The default value is `false`.
+          # only-new-issues: true
+
+          # Optional: if set to true then the all caching functionality will be complete disabled,
+          #           takes precedence over all other caching options.
+          # skip-cache: true
+
+          # Optional: if set to true then the action don't cache or restore ~/go/pkg.
+          # skip-pkg-cache: true
+
+          # Optional: if set to true then the action don't cache or restore ~/.cache/go-build.
+          # skip-build-cache: true

CHANGELOG.md

@@ -4,6 +4,82 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org).
 
+## [v1.0.12](https://github.com/TLINDEN/tablizer/tree/v1.0.12) - 2022-10-25
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.11...v1.0.12)
+
+### Added
+
+- Added support to parse CSV input
+
+- Added CSV output support
+
+- Added support for environment variables
+
+### Changed
+
+- We do not use the generated help message anymore, instead we use the
+  usage from the  manpage, which we have to maintain  anyway. It looks
+  better and has flag groups, which  cobra is still lacking as of this
+  writing.
+  
+-  More refactoring  and  re-organization,  runtime configuration  now
+  lives in the cfg module.
+  
+  
+### Fixed
+
+- Fixed  [Bug #5](https://github.com/TLINDEN/tablizer/issues/5), where
+  matches have not been highlighted correctly in some rare cases.
+
+
+
+## [v1.0.11](https://github.com/TLINDEN/tablizer/tree/v1.0.11) - 2022-10-19
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.10...v1.0.11)
+
+### Added
+
+- Added CI job golinter to regularly check for common mistakes.
+
+- Added YAML output mode.
+
+- Added more unit tests, we're over 95% in the lib module.
+
+### Changed
+
+- do  not use any global  variables anymore, makes the  code easier to
+  maintain, understand and test
+
+- using io.Writer  in print* functions, which is easier  to test, also
+  re-implemented the print tests.
+
+- replaced go-str2duration with my own implementation `duration2int()`.
+
+
+
+## [v1.0.10](https://github.com/TLINDEN/tablizer/tree/v1.0.10) - 2022-10-15
+
+[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.9...v1.0.10)
+
+### Added
+
+- Added various sort modes: sort by time, by duration, numerical (-a -t -i)
+
+- Added possibility to modify sort order to descending (-D)
+
+- Added support  to specify a regexp in column  selector -c, which can
+  also be mixed with numerical column spec
+  
+- More unit tests
+
+### Fixed
+
+- Column specification  allowed to specify duplicate  columns like `-c
+  1,2,1,2` unchecked. Now this list will be deduplicated before use.
+
+
+
 ## [v1.0.9](https://github.com/TLINDEN/tablizer/tree/v1.0.9) - 2022-10-14
 
 [Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.8...v1.0.9)
@@ -24,7 +100,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ### Added
 
-- Added sort support with the new parameter -k (like sort(1).
+- Added sort support with the new parameter -k (like sort(1)).
 
 
 

CODE_OF_CONDUCT.md

@@ -1,7 +1,9 @@
 # No Code of Conduct
 
-This project does  **NOT** have a so called Code  of Conduct, nor will
-it ever get one.
+*TL;DR:* This project  does **NOT** have a so called  Code of Conduct,
+nor will it ever have one.
+
+## The Rant
 
 The reasons are somewhat complicated and  I'll try my best to document
 them here.
@@ -42,28 +44,22 @@ might be  sarcastic connotations I  don't understand or  references to
 historical figures, events  or traditions I don't know  and never have
 heard of.
 
-Juding over other peoples online behavior looks like a titanic task to
-me. It  is just not my  job to judge  others. I am not  legitimized or
-authorized to do so.
+Judging over other  peoples online behavior looks like  a titanic task
+to me. It is just not my job  to judge others, I am not legitimized or
+authorized to do so and I am not interested in this kind of business.
 
 Another huge  problem with ethical rules  is that you need  to outline
-and enforce  sanctions on thos who  violate the rules. But  since I am
-not an  elected authority  how would  I be  able to  do this?  I don't
-know.  And what  happens if  someone complains  about myself?  Shall I
+and enforce sanctions  on those who violate the rules.  But since I am
+not an  elected authority  how would I  be able to  do this?   I don't
+know.  And  what happens if  someone complains about myself?   Shall I
 remove myself from my own project? Come on!
 
-Last but  not least  there's the  law. I  am a  german citizen  and am
-living  in relatively  freedom.  Unlike  many other  people living  in
-democracies these days,  I myself fought for this very  freedom on the
-streets of  Leipzig in 1989.  I  saw the tanks, the  Stasi officers, I
-felt the  fear. But the laws  under I live  today and which I  have to
-adhere to, are only limited to the small speck on earth I am living on.
-
-So, let's say someone in india  says something insulting to some other
-developer in an issue.  Of course  german law does not apply to indian
-people. More, the insult might actually  not be an insult in india. In
-the end, nothing would happen. Under normal circumstances, maintainers
-would delete the posting, ban the user or remove push privileges etc.
+Last but  not least there's  the law. So,  let's say someone  in india
+says  something insulting  to some  other developer  in an  issue.  Of
+course german  law does not apply  to indian people. More,  the insult
+might actually  not be an insult  in india. In the  end, nothing would
+happen.  Under  normal  circumstances, maintainers  would  delete  the
+posting, ban the user or remove push privileges etc.
 
 But then, is there a way for  the offending user to defend himself? Of
 course not, since neither indian or german law alone applies. I cannot
@@ -71,13 +67,18 @@ go to  a german court  and sue the  guy and he  cannot do the  same in
 india. Or -  we possibly could but the judges  on both countries would
 just laugh and close the case.
 
-And let's not  even start talking about  there undemocratic "comitees"
-many projects are forming to  circumvent this problem.
-
 That  being said,  I  don't have  the  power nor  the  tools, nor  the
 authority to enforce serious sanctions  of any meaningful kind against
 others. Therefore I cannot outline any rules whatsoever.
 
+And let's not  even start talking about  there undemocratic "comitees"
+many projects  are forming to  circumvent this problem.  Some projects
+even  include  external  entities  like a  lawer  or  some  bureaucrat
+somewhere  just to  have the  ability  to complain  against a  comitee
+member. What a mess!
+
+
+
 ## So, which are the ethical rules within this project then?
 
 Well, there are none.
@@ -86,7 +87,8 @@ This project is  about code, not society. It doesn't  matter where you
 come from,  how you look,  how you think,  what you believe,  who your
 friends are, whay you  said or did sometime in the  past. I don't even
 care if you are a human being. You are an alien so bored that you need
-to submit code on github? Fine with me.
+to submit code on github? Fine with me. You're a convicted criminal? I
+don't give a shit!
 
 **The only thing I am interested here is Code and only Code.**
 

Makefile

@@ -18,12 +18,12 @@
 #
 # no need to modify anything below
 tool    = tablizer
-version = $(shell egrep "= .v" lib/common.go | cut -d'=' -f2 | cut -d'"' -f 2)
+version = $(shell egrep "= .v" cfg/config.go | cut -d'=' -f2 | cut -d'"' -f 2)
 archs   = android darwin freebsd linux netbsd openbsd windows
 PREFIX  = /usr/local
 UID     = root
 GID     = 0
-BRANCH  = $(shell git describe --all | cut -d/ -f2)
+BRANCH  = $(shell git branch --show-current)
 COMMIT  = $(shell git rev-parse --short=8 HEAD)
 BUILD   = $(shell date +%Y.%m.%d.%H%M%S) 
 VERSION:= $(if $(filter $(BRANCH), development),$(version)-$(BRANCH)-$(COMMIT)-$(BUILD),$(version))
@@ -41,8 +41,12 @@ cmd/%.go: %.pod
 	pod2text $*.pod >> cmd/$*.go
 	echo "\`" >> cmd/$*.go
 
+	echo "var usage = \`" >> cmd/$*.go
+	awk '/SYNOPS/{f=1;next} /DESCR/{f=0} f' $*.pod  | sed 's/^    //' >> cmd/$*.go
+	echo "\`" >> cmd/$*.go
+
 buildlocal:
-	go build -ldflags "-X 'github.com/tlinden/tablizer/lib.VERSION=$(VERSION)'"
+	go build -ldflags "-X 'github.com/tlinden/tablizer/cfg.VERSION=$(VERSION)'"
 
 release:
 	./mkrel.sh $(tool) $(version)
@@ -55,7 +59,15 @@ install: buildlocal
 	install -o $(UID) -g $(GID) -m 444 $(tool).1 $(PREFIX)/man/man1/
 
 clean:
-	rm -rf $(tool) releases
+	rm -rf $(tool) releases coverage.out
 
 test:
 	go test -v ./...
+
+singletest:
+	@echo "Call like this: ''make singletest TEST=TestPrepareColumns MOD=lib"
+	go test -run $(TEST) github.com/tlinden/tablizer/$(MOD)
+
+cover-report:
+	go test ./... -cover -coverprofile=coverage.out
+	go tool cover -html=coverage.out

TODO

@@ -1 +0,0 @@
-

TODO.md

@@ -0,0 +1,9 @@
+## Fixes to be implemented
+
+## Features to be implemented
+
+- add comment support (csf.NewReader().Comment = '#')
+
+- add --no-headers option
+
+

cfg/config.go

@@ -0,0 +1,200 @@
+/*
+Copyright © 2022 Thomas von Dein
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+package cfg
+
+import (
+	"errors"
+	"fmt"
+	"github.com/gookit/color"
+	"os"
+	"regexp"
+)
+
+const DefaultSeparator string = `(\s\s+|\t)`
+const Version string = "v1.0.12"
+
+var VERSION string // maintained by -x
+
+type Config struct {
+	Debug       bool
+	NoNumbering bool
+	Columns     string
+	UseColumns  []int
+	Separator   string
+	OutputMode  int
+	InvertMatch bool
+	Pattern     string
+	PatternR    *regexp.Regexp
+
+	SortMode       string
+	SortDescending bool
+	SortByColumn   int
+
+	/*
+	 FIXME: make configurable somehow, config file or ENV
+	 see https://github.com/gookit/color.
+	*/
+	ColorStyle color.Style
+
+	NoColor bool
+}
+
+// maps outputmode short flags to output mode, ie. -O => -o orgtbl
+type Modeflag struct {
+	X bool
+	O bool
+	M bool
+	S bool
+	Y bool
+	A bool
+	C bool
+}
+
+// used for switching printers
+const (
+	Extended = iota + 1
+	Orgtbl
+	Markdown
+	Shell
+	Yaml
+	CSV
+	Ascii
+)
+
+// various sort types
+type Sortmode struct {
+	Numeric bool
+	Time    bool
+	Age     bool
+}
+
+// default color schemes
+func Colors() map[color.Level]map[string]color.Color {
+	return map[color.Level]map[string]color.Color{
+		color.Level16: {
+			"bg": color.BgGreen, "fg": color.FgBlack,
+		},
+		color.Level256: {
+			"bg": color.BgLightGreen, "fg": color.FgBlack,
+		},
+		color.LevelRgb: {
+			// FIXME: maybe use something nicer
+			"bg": color.BgLightGreen, "fg": color.FgBlack,
+		},
+	}
+}
+
+// find supported color mode, modifies config based on constants
+func (c *Config) DetermineColormode() {
+	if !isTerminal(os.Stdout) {
+		color.Disable()
+	} else {
+		level := color.TermColorLevel()
+		colors := Colors()
+		c.ColorStyle = color.New(colors[level]["bg"], colors[level]["fg"])
+	}
+}
+
+// Return true if current terminal is interactive
+func isTerminal(f *os.File) bool {
+	o, _ := f.Stat()
+	if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice {
+		return true
+	} else {
+		return false
+	}
+}
+
+func Getversion() string {
+	// main program version
+
+	// generated  version string, used  by -v contains  lib.Version on
+	//  main  branch,   and  lib.Version-$branch-$lastcommit-$date  on
+	// development branch
+
+	return fmt.Sprintf("This is tablizer version %s", VERSION)
+}
+
+func (conf *Config) PrepareSortFlags(flag Sortmode) {
+	switch {
+	case flag.Numeric:
+		conf.SortMode = "numeric"
+	case flag.Age:
+		conf.SortMode = "duration"
+	case flag.Time:
+		conf.SortMode = "time"
+	default:
+		conf.SortMode = "string"
+	}
+}
+
+func (conf *Config) PrepareModeFlags(flag Modeflag) {
+	switch {
+	case flag.X:
+		conf.OutputMode = Extended
+	case flag.O:
+		conf.OutputMode = Orgtbl
+	case flag.M:
+		conf.OutputMode = Markdown
+	case flag.S:
+		conf.OutputMode = Shell
+	case flag.Y:
+		conf.OutputMode = Yaml
+	case flag.C:
+		conf.OutputMode = CSV
+	default:
+		conf.OutputMode = Ascii
+	}
+}
+
+func (c *Config) CheckEnv() {
+	// check for environment vars, command line flags have precedence,
+	// NO_COLOR is being checked by the color module itself.
+	if !c.NoNumbering {
+		_, set := os.LookupEnv("T_NO_HEADER_NUMBERING")
+		if set {
+			c.NoNumbering = true
+		}
+	}
+
+	if len(c.Columns) == 0 {
+		cols := os.Getenv("T_COLUMNS")
+		if len(cols) > 1 {
+			c.Columns = cols
+		}
+	}
+}
+
+func (c *Config) ApplyDefaults() {
+	// mode specific defaults
+	if c.OutputMode == Yaml || c.OutputMode == CSV {
+		c.NoNumbering = true
+	}
+}
+
+func (c *Config) PreparePattern(pattern string) error {
+	PatternR, err := regexp.Compile(pattern)
+
+	if err != nil {
+		return errors.Unwrap(fmt.Errorf("Regexp pattern %s is invalid: %w", c.Pattern, err))
+	}
+
+	c.PatternR = PatternR
+	c.Pattern = pattern
+
+	return nil
+}

cfg/config_test.go

@@ -0,0 +1,103 @@
+/*
+Copyright © 2022 Thomas von Dein
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package cfg
+
+import (
+	"fmt"
+	//	"reflect"
+	"testing"
+)
+
+func TestPrepareModeFlags(t *testing.T) {
+	var tests = []struct {
+		flag   Modeflag
+		expect int // output (constant enum)
+	}{
+		// short commandline flags like -M
+		{Modeflag{X: true}, Extended},
+		{Modeflag{S: true}, Shell},
+		{Modeflag{O: true}, Orgtbl},
+		{Modeflag{Y: true}, Yaml},
+		{Modeflag{M: true}, Markdown},
+		{Modeflag{}, Ascii},
+	}
+
+	// FIXME: use a map for easier printing
+	for _, tt := range tests {
+		testname := fmt.Sprintf("PrepareModeFlags-expect-%d", tt.expect)
+		t.Run(testname, func(t *testing.T) {
+			c := Config{}
+
+			c.PrepareModeFlags(tt.flag)
+			if c.OutputMode != tt.expect {
+				t.Errorf("got: %d, expect: %d", c.OutputMode, tt.expect)
+			}
+		})
+	}
+}
+
+func TestPrepareSortFlags(t *testing.T) {
+	var tests = []struct {
+		flag   Sortmode
+		expect string // output
+	}{
+		// short commandline flags like -M
+		{Sortmode{Numeric: true}, "numeric"},
+		{Sortmode{Age: true}, "duration"},
+		{Sortmode{Time: true}, "time"},
+		{Sortmode{}, "string"},
+	}
+
+	for _, tt := range tests {
+		testname := fmt.Sprintf("PrepareSortFlags-expect-%s", tt.expect)
+		t.Run(testname, func(t *testing.T) {
+			c := Config{}
+
+			c.PrepareSortFlags(tt.flag)
+
+			if c.SortMode != tt.expect {
+				t.Errorf("got: %s, expect: %s", c.SortMode, tt.expect)
+			}
+		})
+	}
+}
+
+func TestPreparePattern(t *testing.T) {
+	var tests = []struct {
+		pattern string
+		wanterr bool
+	}{
+		{"[A-Z]+", false},
+		{"[a-z", true},
+	}
+
+	for _, tt := range tests {
+		testname := fmt.Sprintf("PreparePattern-pattern-%s-wanterr-%t", tt.pattern, tt.wanterr)
+		t.Run(testname, func(t *testing.T) {
+			c := Config{}
+
+			err := c.PreparePattern(tt.pattern)
+
+			if err != nil {
+				if !tt.wanterr {
+					t.Errorf("PreparePattern returned error: %s", err)
+				}
+			}
+		})
+	}
+}

cmd/root.go

@@ -20,14 +20,14 @@ import (
 	"bytes"
 	"fmt"
 	"github.com/spf13/cobra"
+	"github.com/tlinden/tablizer/cfg"
 	"github.com/tlinden/tablizer/lib"
 	"log"
 	"os"
 	"os/exec"
+	"strings"
 )
 
-var ShowManual = false
-
 func man() {
 	man := exec.Command("less", "-")
 
@@ -45,64 +45,75 @@ func man() {
 	}
 }
 
-var rootCmd = &cobra.Command{
-	Use:   "tablizer [regex] [file, ...]",
-	Short: "[Re-]tabularize tabular data",
-	Long:  `Manipulate tabular output of other programs`,
-	RunE: func(cmd *cobra.Command, args []string) error {
-		if lib.ShowVersion {
-			fmt.Printf("This is tablizer version %s\n", lib.VERSION)
-			return nil
-		}
-
-		if ShowManual {
-			man()
-			return nil
-		}
-
-		err := lib.PrepareColumns()
-		if err != nil {
-			return err
-		}
-
-		err = lib.PrepareModeFlags()
-		if err != nil {
-			return err
-		}
-
-		return lib.ProcessFiles(args)
-	},
-}
-
 func Execute() {
+	var (
+		conf        cfg.Config
+		ShowManual  bool
+		ShowVersion bool
+		modeflag    cfg.Modeflag
+		sortmode    cfg.Sortmode
+	)
+
+	var rootCmd = &cobra.Command{
+		Use:   "tablizer [regex] [file, ...]",
+		Short: "[Re-]tabularize tabular data",
+		Long:  `Manipulate tabular output of other programs`,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			if ShowVersion {
+				fmt.Println(cfg.Getversion())
+				return nil
+			}
+
+			if ShowManual {
+				man()
+				return nil
+			}
+
+			// Setup
+			conf.CheckEnv()
+			conf.PrepareModeFlags(modeflag)
+			conf.PrepareSortFlags(sortmode)
+			conf.DetermineColormode()
+			conf.ApplyDefaults()
+
+			// actual execution starts here
+			return lib.ProcessFiles(&conf, args)
+		},
+	}
+
+	// options
+	rootCmd.PersistentFlags().BoolVarP(&conf.Debug, "debug", "d", false, "Enable debugging")
+	rootCmd.PersistentFlags().BoolVarP(&conf.NoNumbering, "no-numbering", "n", false, "Disable header numbering")
+	rootCmd.PersistentFlags().BoolVarP(&conf.NoColor, "no-color", "N", false, "Disable pattern highlighting")
+	rootCmd.PersistentFlags().BoolVarP(&ShowVersion, "version", "V", false, "Print program version")
+	rootCmd.PersistentFlags().BoolVarP(&conf.InvertMatch, "invert-match", "v", false, "select non-matching rows")
+	rootCmd.PersistentFlags().BoolVarP(&ShowManual, "man", "m", false, "Display manual page")
+	rootCmd.PersistentFlags().StringVarP(&conf.Separator, "separator", "s", cfg.DefaultSeparator, "Custom field separator")
+	rootCmd.PersistentFlags().StringVarP(&conf.Columns, "columns", "c", "", "Only show the speficied columns (separated by ,)")
+
+	// sort options
+	rootCmd.PersistentFlags().IntVarP(&conf.SortByColumn, "sort-by", "k", 0, "Sort by column (default: 1)")
+
+	// sort mode, only 1 allowed
+	rootCmd.PersistentFlags().BoolVarP(&conf.SortDescending, "sort-desc", "D", false, "Sort in descending order (default: ascending)")
+	rootCmd.PersistentFlags().BoolVarP(&sortmode.Numeric, "sort-numeric", "i", false, "sort according to string numerical value")
+	rootCmd.PersistentFlags().BoolVarP(&sortmode.Time, "sort-time", "t", false, "sort according to time string")
+	rootCmd.PersistentFlags().BoolVarP(&sortmode.Age, "sort-age", "a", false, "sort according to age (duration) string")
+	rootCmd.MarkFlagsMutuallyExclusive("sort-desc", "sort-numeric", "sort-time", "sort-age")
+
+	// output flags, only 1 allowed
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.X, "extended", "X", false, "Enable extended output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.M, "markdown", "M", false, "Enable markdown table output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.O, "orgtbl", "O", false, "Enable org-mode table output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.S, "shell", "S", false, "Enable shell mode output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.Y, "yaml", "Y", false, "Enable yaml output")
+	rootCmd.PersistentFlags().BoolVarP(&modeflag.C, "csv", "C", false, "Enable CSV output")
+	rootCmd.MarkFlagsMutuallyExclusive("extended", "markdown", "orgtbl", "shell", "yaml", "csv")
+
+	rootCmd.SetUsageTemplate(strings.TrimSpace(usage) + "\n")
+
 	err := rootCmd.Execute()
 	if err != nil {
 		os.Exit(1)
 	}
 }
-
-func init() {
-	rootCmd.PersistentFlags().BoolVarP(&lib.Debug, "debug", "d", false, "Enable debugging")
-	rootCmd.PersistentFlags().BoolVarP(&lib.NoNumbering, "no-numbering", "n", false, "Disable header numbering")
-	rootCmd.PersistentFlags().BoolVarP(&lib.NoColor, "no-color", "N", false, "Disable pattern highlighting")
-	rootCmd.PersistentFlags().BoolVarP(&lib.ShowVersion, "version", "V", false, "Print program version")
-	rootCmd.PersistentFlags().BoolVarP(&lib.InvertMatch, "invert-match", "v", false, "select non-matching rows")
-	rootCmd.PersistentFlags().BoolVarP(&ShowManual, "man", "m", false, "Display manual page")
-	rootCmd.PersistentFlags().StringVarP(&lib.Separator, "separator", "s", lib.DefaultSeparator, "Custom field separator")
-	rootCmd.PersistentFlags().StringVarP(&lib.Columns, "columns", "c", "", "Only show the speficied columns (separated by ,)")
-	rootCmd.PersistentFlags().IntVarP(&lib.SortByColumn, "sort-by", "k", 0, "Sort by column (default: 1)")
-
-	// output flags, only 1 allowed, hidden, since just short cuts
-	rootCmd.PersistentFlags().BoolVarP(&lib.OutflagExtended, "extended", "X", false, "Enable extended output")
-	rootCmd.PersistentFlags().BoolVarP(&lib.OutflagMarkdown, "markdown", "M", false, "Enable markdown table output")
-	rootCmd.PersistentFlags().BoolVarP(&lib.OutflagOrgtable, "orgtbl", "O", false, "Enable org-mode table output")
-	rootCmd.PersistentFlags().BoolVarP(&lib.OutflagShell, "shell", "S", false, "Enable shell mode output")
-	rootCmd.MarkFlagsMutuallyExclusive("extended", "markdown", "orgtbl", "shell")
-	rootCmd.Flags().MarkHidden("extended")
-	rootCmd.Flags().MarkHidden("orgtbl")
-	rootCmd.Flags().MarkHidden("markdown")
-	rootCmd.Flags().MarkHidden("shell")
-
-	// same thing but more common, takes precedence over above group
-	rootCmd.PersistentFlags().StringVarP(&lib.OutputMode, "output", "o", "", "Output mode - one of: orgtbl, markdown, extended, shell, ascii(default)")
-}

cmd/tablizer.go

@@ -8,20 +8,33 @@ SYNOPSIS
         Usage:
           tablizer [regex] [file, ...] [flags]
     
-        Flags:
+        Operational Flags:
           -c, --columns string     Only show the speficied columns (separated by ,)
-          -d, --debug              Enable debugging
-          -h, --help               help for tablizer
           -v, --invert-match       select non-matching rows
-          -m, --man                Display manual page
           -n, --no-numbering       Disable header numbering
           -N, --no-color           Disable pattern highlighting
-          -o, --output string      Output mode - one of: orgtbl, markdown, extended, ascii(default)
+          -s, --separator string   Custom field separator
+          -k, --sort-by int        Sort by column (default: 1)
+
+        Output Flags (mutually exclusive):
           -X, --extended           Enable extended output
           -M, --markdown           Enable markdown table output
           -O, --orgtbl             Enable org-mode table output
-          -s, --separator string   Custom field separator
-          -k, --sort-by int        Sort by column (default: 1)
+          -S, --shell              Enable shell evaluable ouput
+          -Y, --yaml               Enable yaml output
+          -C, --csv                Enable CSV output
+          -A, --ascii              Default output mode, ascii tabular
+
+        Sort Mode Flags (mutually exclusive):
+          -a, --sort-age           sort according to age (duration) string
+          -D, --sort-desc          Sort in descending order (default: ascending)
+          -i, --sort-numeric       sort according to string numerical value
+          -t, --sort-time          sort according to time string
+
+        Other Flags:
+          -d, --debug              Enable debugging
+          -h, --help               help for tablizer
+          -m, --man                Display manual page
           -v, --version            Print program version
 
 DESCRIPTION
@@ -64,7 +77,7 @@ DESCRIPTION
        NAME(1) READY(2) STATUS(3) RESTARTS(4) AGE(5)
 
     These numbers denote the column and you can use them to specify which
-    columns you want to have in your output:
+    columns you want to have in your output (see COLUMNS:
 
        kubectl get pods | tablizer -c1,3
 
@@ -78,7 +91,18 @@ DESCRIPTION
 
     Use the -k option to specify by which column to sort the tabular data
     (as in GNU sort(1)). The default sort column is the first one. To
-    disable sorting at all, supply 0 (Zero) to -k.
+    disable sorting at all, supply 0 (Zero) to -k. The default sort order is
+    ascending. You can change this to descending order using the option -D.
+    The default sort order is by string, but there are other sort modes:
+
+    -a --sort-age
+        Sorts duration strings like "1d4h32m51s".
+
+    -i --sort-numeric
+        Sorts numeric fields.
+
+    -t --sort-time
+        Sorts timestamps.
 
     Finally the -d option enables debugging output which is mostly useful
     for the developer.
@@ -107,6 +131,32 @@ DESCRIPTION
 
         kubectl get pods -A | tablizer "(?i)account"
 
+  COLUMNS
+    The parameter -c can be used to specify, which columns to display. By
+    default tablizer numerizes the header names and these numbers can be
+    used to specify which header to display, see example above.
+
+    However, beside numbers, you can also use regular expressions with -c,
+    also separated by comma. And you can mix column numbers with regexps.
+
+    Lets take this table:
+
+            PID TTY          TIME CMD
+          14001 pts/0    00:00:00 bash
+          42871 pts/0    00:00:00 ps
+          42872 pts/0    00:00:00 sed
+
+    We want to see only the CMD column and use a regex for this:
+
+        ps | tablizer -s '\s+' -c C
+        CMD(4)
+        bash
+        ps
+        tablizer
+        sed
+
+    where "C" is our regexp which matches CMD.
+
   OUTPUT MODES
     There might be cases when the tabular output of a program is way too
     large for your current terminal but you still need to see every column.
@@ -137,7 +187,17 @@ DESCRIPTION
 
     Beside normal ascii mode (the default) and extended mode there are more
     output modes available: orgtbl which prints an Emacs org-mode table and
-    markdown which prints a Markdown table.
+    markdown which prints a Markdown table, yaml, which prints yaml encoding
+    and CSV mode, which prints a comma separated value file.
+
+  ENVIRONMENT VARIABLES
+    tablizer supports certain environment variables which use can use to
+    influence program behavior. Commandline flags have always precedence
+    over environment variables.
+
+    <T_NO_HEADER_NUMBERING> - disable numbering of header fields, like -n.
+    <T_COLUMNS> - comma separated list of columns to output, like -c
+    <NO_COLORS> - disable colorization of matches, like -N
 
 BUGS
     In order to report a bug, unexpected behavior, feature requests or to
@@ -163,3 +223,39 @@ AUTHORS
     Thomas von Dein tom AT vondein DOT org
 
 `
+var usage = `
+
+Usage:
+  tablizer [regex] [file, ...] [flags]
+
+Operational Flags:
+  -c, --columns string     Only show the speficied columns (separated by ,)
+  -v, --invert-match       select non-matching rows
+  -n, --no-numbering       Disable header numbering
+  -N, --no-color           Disable pattern highlighting
+  -s, --separator string   Custom field separator
+  -k, --sort-by int        Sort by column (default: 1)
+
+Output Flags (mutually exclusive):
+  -X, --extended           Enable extended output
+  -M, --markdown           Enable markdown table output
+  -O, --orgtbl             Enable org-mode table output
+  -S, --shell              Enable shell evaluable ouput
+  -Y, --yaml               Enable yaml output
+  -C, --csv                Enable CSV output
+  -A, --ascii              Default output mode, ascii tabular
+
+Sort Mode Flags (mutually exclusive):
+  -a, --sort-age           sort according to age (duration) string
+  -D, --sort-desc          Sort in descending order (default: ascending)
+  -i, --sort-numeric       sort according to string numerical value
+  -t, --sort-time          sort according to time string
+
+Other Flags:
+  -d, --debug              Enable debugging
+  -h, --help               help for tablizer
+  -m, --man                Display manual page
+  -v, --version            Print program version
+
+
+`

go.mod

@@ -4,15 +4,18 @@ go 1.18
 
 require (
 	github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897
+	github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de
 	github.com/gookit/color v1.5.2
 	github.com/olekukonko/tablewriter v0.0.5
 	github.com/spf13/cobra v1.5.0
-	github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778
+	gopkg.in/yaml.v3 v3.0.1
 )
 
 require (
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
-	github.com/mattn/go-runewidth v0.0.9 // indirect
+	github.com/mattn/go-runewidth v0.0.10 // indirect
+	github.com/rivo/uniseg v0.1.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
+	github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 // indirect
 	golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 // indirect
 )

go.sum

@@ -1,5 +1,7 @@
 github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897 h1:p9Sln00KOTlrYkxI1zYWl1QLnEqAqEARBEYa8FQnQcY=
 github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897/go.mod h1:xTS7Pm1pD1mvyM075QCDSRqH6qRLXylzS24ZTpRiSzQ=
+github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de h1:FxWPpzIjnTlhPwqqXc4/vE0f7GvRjuAsbW+HOIe8KnA=
+github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de/go.mod h1:DCaWoUhZrYW9p1lxo/cm8EmUOOzAPSEZNGF2DK1dJgw=
 github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
@@ -8,19 +10,24 @@ github.com/gookit/color v1.5.2 h1:uLnfXcaFjlrDnQDT+NCBcfhrXqYTx/rcCa6xn01Y8yI=
 github.com/gookit/color v1.5.2/go.mod h1:w8h4bGiHeeBpvQVePTutdbERIUf3oJE5lZ8HM0UgXyg=
 github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
 github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
-github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0=
 github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
+github.com/mattn/go-runewidth v0.0.10 h1:CoZ3S2P7pvtP45xOtBw+/mDL2z0RKI576gSkzRRpdGg=
+github.com/mattn/go-runewidth v0.0.10/go.mod h1:RAqKPSqVFrSLVXbA8x7dzmKdmGzieGRCM46jaSJTDAk=
 github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec=
 github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/rivo/uniseg v0.1.0 h1:+2KBaVoUmb9XzDsrx/Ct0W/EYOSFf/nWTauy++DprtY=
+github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
+github.com/scylladb/termtables v0.0.0-20191203121021-c4c0b6d42ff4/go.mod h1:C1a7PQSMz9NShzorzCiG2fk9+xuCgLkPeCvMHYR2OWg=
 github.com/spf13/cobra v1.5.0 h1:X+jTBEBqF0bHN+9cSMgmfuvv2VHJ9ezmFNf9Y/XstYU=
 github.com/spf13/cobra v1.5.0/go.mod h1:dWXEIy2H428czQCjInthrTRUg7yKbok+2Qi/yBIJoUM=
 github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
 github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
+github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
@@ -28,6 +35,7 @@ github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 h1:QldyIu/L63oPpyvQmHg
 github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
 golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 h1:Bli41pIlzTzf3KEY06n+xnzK/BESIg2ze4Pgfh/aI8c=
 golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

lib/common.go

@@ -17,72 +17,9 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
 
 package lib
 
-import (
-	"github.com/gookit/color"
-	//"github.com/xo/terminfo"
-)
-
-var (
-	// command line flags
-	Debug            bool
-	XtendedOut       bool
-	NoNumbering      bool
-	ShowVersion      bool
-	Columns          string
-	UseColumns       []int
-	DefaultSeparator string = `(\s\s+|\t)`
-	Separator        string = `(\s\s+|\t)`
-	OutflagExtended  bool
-	OutflagMarkdown  bool
-	OutflagOrgtable  bool
-	OutflagShell     bool
-	OutputMode       string
-	InvertMatch      bool
-	Pattern          string
-
-	/*
-	 FIXME: make configurable somehow, config file or ENV
-	 see https://github.com/gookit/color will be set by
-	 io.ProcessFiles() according to currently supported
-	 color mode.
-	*/
-	MatchFG string
-	MatchBG string
-	NoColor bool
-
-	// colors to be used per supported color mode
-	Colors = map[color.Level]map[string]string{
-		color.Level16: {
-			"bg": "green", "fg": "black",
-		},
-		color.Level256: {
-			"bg": "lightGreen", "fg": "black",
-		},
-		color.LevelRgb: {
-			// FIXME: maybe use something nicer
-			"bg": "lightGreen", "fg": "black",
-		},
-	}
-
-	// used for validation
-	validOutputmodes = "(orgtbl|markdown|extended|ascii)"
-
-	// main program version
-	Version = "v1.0.9"
-
-	// generated  version string, used  by -v contains  lib.Version on
-	//  main  branch,   and  lib.Version-$branch-$lastcommit-$date  on
-	// development branch
-	VERSION string
-
-	// sorting
-	SortByColumn int
-)
-
 // contains a whole parsed table
 type Tabdata struct {
 	maxwidthHeader int      // longest header
-	maxwidthPerCol []int    // max width per column
 	columns        int      // count
 	headers        []string // [ "ID", "NAME", ...]
 	entries        [][]string

lib/helpers.go

@@ -21,8 +21,10 @@ import (
 	"errors"
 	"fmt"
 	"github.com/gookit/color"
+	"github.com/tlinden/tablizer/cfg"
 	"os"
 	"regexp"
+	"sort"
 	"strconv"
 	"strings"
 )
@@ -36,35 +38,72 @@ func contains(s []int, e int) bool {
 	return false
 }
 
-func PrepareColumns() error {
-	if len(Columns) > 0 {
-		for _, use := range strings.Split(Columns, ",") {
-			usenum, err := strconv.Atoi(use)
-			if err != nil {
-				msg := fmt.Sprintf("Could not parse columns list %s: %v", Columns, err)
+// parse columns list given  with -c, modifies config.UseColumns based
+// on eventually given regex
+func PrepareColumns(c *cfg.Config, data *Tabdata) error {
+	if len(c.Columns) > 0 {
+		for _, use := range strings.Split(c.Columns, ",") {
+			if len(use) == 0 {
+				msg := fmt.Sprintf("Could not parse columns list %s: empty column", c.Columns)
 				return errors.New(msg)
 			}
-			UseColumns = append(UseColumns, usenum)
+
+			usenum, err := strconv.Atoi(use)
+			if err != nil {
+				// might be a regexp
+				colPattern, err := regexp.Compile(use)
+				if err != nil {
+					msg := fmt.Sprintf("Could not parse columns list %s: %v", c.Columns, err)
+					return errors.New(msg)
+				}
+
+				// find matching header fields
+				for i, head := range data.headers {
+					if colPattern.MatchString(head) {
+						c.UseColumns = append(c.UseColumns, i+1)
+					}
+
+				}
+			} else {
+				// we digress from go  best practises here, because if
+				// a colum spec is not a number, we process them above
+				// inside the err handler  for atoi(). so only add the
+				// number, if it's really just a number.
+				c.UseColumns = append(c.UseColumns, usenum)
+			}
 		}
+
+		// deduplicate: put all values into a map (value gets map key)
+		// thereby  removing duplicates,  extract keys into  new slice
+		// and sort it
+		imap := make(map[int]int, len(c.UseColumns))
+		for _, i := range c.UseColumns {
+			imap[i] = 0
+		}
+		c.UseColumns = nil
+		for k := range imap {
+			c.UseColumns = append(c.UseColumns, k)
+		}
+		sort.Ints(c.UseColumns)
 	}
 	return nil
 }
 
-func numberizeHeaders(data *Tabdata) {
-	// prepare headers: add numbers to headers
+// prepare headers: add numbers to headers
+func numberizeAndReduceHeaders(c cfg.Config, data *Tabdata) {
 	numberedHeaders := []string{}
 	maxwidth := 0 // start from scratch, so we only look at displayed column widths
 
 	for i, head := range data.headers {
 		headlen := 0
-		if len(Columns) > 0 {
+		if len(c.Columns) > 0 {
 			// -c specified
-			if !contains(UseColumns, i+1) {
+			if !contains(c.UseColumns, i+1) {
 				// ignore this one
 				continue
 			}
 		}
-		if NoNumbering {
+		if c.NoNumbering {
 			numberedHeaders = append(numberedHeaders, head)
 			headlen = len(head)
 		} else {
@@ -83,15 +122,15 @@ func numberizeHeaders(data *Tabdata) {
 	}
 }
 
-func reduceColumns(data *Tabdata) {
-	// exclude columns, if any
-	if len(Columns) > 0 {
+// exclude columns, if any
+func reduceColumns(c cfg.Config, data *Tabdata) {
+	if len(c.Columns) > 0 {
 		reducedEntries := [][]string{}
 		var reducedEntry []string
 		for _, entry := range data.entries {
 			reducedEntry = nil
 			for i, value := range entry {
-				if !contains(UseColumns, i+1) {
+				if !contains(c.UseColumns, i+1) {
 					continue
 				}
 
@@ -103,39 +142,6 @@ func reduceColumns(data *Tabdata) {
 	}
 }
 
-func PrepareModeFlags() error {
-	if len(OutputMode) == 0 {
-		// associate short flags like -X with mode selector
-		switch {
-		case OutflagExtended:
-			OutputMode = "extended"
-		case OutflagMarkdown:
-			OutputMode = "markdown"
-		case OutflagOrgtable:
-			OutputMode = "orgtbl"
-		case OutflagShell:
-			OutputMode = "shell"
-			NoNumbering = true
-		default:
-			OutputMode = "ascii"
-		}
-	} else {
-		r, err := regexp.Compile(validOutputmodes)
-
-		if err != nil {
-			return errors.New("Failed to validate output mode spec!")
-		}
-
-		match := r.MatchString(OutputMode)
-
-		if !match {
-			return errors.New("Invalid output mode!")
-		}
-	}
-
-	return nil
-}
-
 func trimRow(row []string) []string {
 	// FIXME: remove this when we only use Tablewriter and strip in ParseFile()!
 	var fixedrow []string
@@ -146,20 +152,13 @@ func trimRow(row []string) []string {
 	return fixedrow
 }
 
-func colorizeData(output string) string {
-	if len(Pattern) > 0 && !NoColor && color.IsConsole(os.Stdout) {
-		r := regexp.MustCompile("(" + Pattern + ")")
-		return r.ReplaceAllString(output, "<bg="+MatchBG+";fg="+MatchFG+">$1</>")
+func colorizeData(c cfg.Config, output string) string {
+	if len(c.Pattern) > 0 && !c.NoColor && color.IsConsole(os.Stdout) {
+		r := regexp.MustCompile("(" + c.Pattern + ")")
+		return r.ReplaceAllStringFunc(output, func(in string) string {
+			return c.ColorStyle.Sprint(in)
+		})
 	} else {
 		return output
 	}
 }
-
-func isTerminal(f *os.File) bool {
-	o, _ := f.Stat()
-	if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice {
-		return true
-	} else {
-		return false
-	}
-}

lib/helpers_test.go

@@ -19,11 +19,12 @@ package lib
 
 import (
 	"fmt"
+	"github.com/tlinden/tablizer/cfg"
 	"reflect"
 	"testing"
 )
 
-func Testcontains(t *testing.T) {
+func TestContains(t *testing.T) {
 	var tests = []struct {
 		list   []int
 		search int
@@ -45,6 +46,19 @@ func Testcontains(t *testing.T) {
 }
 
 func TestPrepareColumns(t *testing.T) {
+	data := Tabdata{
+		maxwidthHeader: 5,
+		columns:        3,
+		headers: []string{
+			"ONE", "TWO", "THREE",
+		},
+		entries: [][]string{
+			{
+				"2", "3", "4",
+			},
+		},
+	}
+
 	var tests = []struct {
 		input     string
 		exp       []int
@@ -52,21 +66,23 @@ func TestPrepareColumns(t *testing.T) {
 	}{
 		{"1,2,3", []int{1, 2, 3}, false},
 		{"1,2,", []int{}, true},
-		{"a,b", []int{}, true},
+		{"T", []int{2, 3}, false},
+		{"T,2,3", []int{2, 3}, false},
+		{"[a-z,4,5", []int{4, 5}, true}, // invalid regexp
 	}
 
 	for _, tt := range tests {
 		testname := fmt.Sprintf("PrepareColumns-%s-%t", tt.input, tt.wanterror)
 		t.Run(testname, func(t *testing.T) {
-			Columns = tt.input
-			err := PrepareColumns()
+			c := cfg.Config{Columns: tt.input}
+			err := PrepareColumns(&c, &data)
 			if err != nil {
 				if !tt.wanterror {
 					t.Errorf("got error: %v", err)
 				}
 			} else {
-				if !reflect.DeepEqual(UseColumns, tt.exp) {
-					t.Errorf("got: %v, expected: %v", UseColumns, tt.exp)
+				if !reflect.DeepEqual(c.UseColumns, tt.exp) {
+					t.Errorf("got: %v, expected: %v", c.UseColumns, tt.exp)
 				}
 			}
 		})
@@ -98,20 +114,44 @@ func TestReduceColumns(t *testing.T) {
 
 	input := [][]string{{"a", "b", "c"}}
 
-	Columns = "y" // used as a flag with len(Columns)...
-
 	for _, tt := range tests {
 		testname := fmt.Sprintf("reduce-columns-by-%+v", tt.columns)
 		t.Run(testname, func(t *testing.T) {
-			UseColumns = tt.columns
+			c := cfg.Config{Columns: "x", UseColumns: tt.columns}
 			data := Tabdata{entries: input}
-			reduceColumns(&data)
+			reduceColumns(c, &data)
 			if !reflect.DeepEqual(data.entries, tt.expect) {
 				t.Errorf("reduceColumns returned invalid data:\ngot: %+v\nexp: %+v", data.entries, tt.expect)
 			}
 		})
 	}
-
-	Columns = "" // reset for other tests
-	UseColumns = nil
+}
+
+func TestNumberizeHeaders(t *testing.T) {
+	data := Tabdata{
+		headers: []string{"ONE", "TWO", "THREE"},
+	}
+
+	var tests = []struct {
+		expect  []string
+		columns []int
+		nonum   bool
+	}{
+		{[]string{"ONE(1)", "TWO(2)", "THREE(3)"}, []int{1, 2, 3}, false},
+		{[]string{"ONE(1)", "TWO(2)"}, []int{1, 2}, false},
+		{[]string{"ONE", "TWO"}, []int{1, 2}, true},
+	}
+
+	for _, tt := range tests {
+		testname := fmt.Sprintf("numberize-headers-columns-%+v-nonum-%t", tt.columns, tt.nonum)
+		t.Run(testname, func(t *testing.T) {
+			c := cfg.Config{Columns: "x", UseColumns: tt.columns, NoNumbering: tt.nonum}
+			usedata := data
+			numberizeAndReduceHeaders(c, &usedata)
+			if !reflect.DeepEqual(usedata.headers, tt.expect) {
+				t.Errorf("numberizeAndReduceHeaders returned invalid data:\ngot: %+v\nexp: %+v",
+					usedata.headers, tt.expect)
+			}
+		})
+	}
 }

lib/io.go

@@ -19,38 +19,40 @@ package lib
 
 import (
 	"errors"
-	"github.com/gookit/color"
+	"github.com/tlinden/tablizer/cfg"
 	"io"
 	"os"
 )
 
-func ProcessFiles(args []string) error {
-	fds, pattern, err := determineIO(args)
-
-	if !isTerminal(os.Stdout) {
-		color.Disable()
-	} else {
-		level := color.TermColorLevel()
-		MatchFG = Colors[level]["fg"]
-		MatchBG = Colors[level]["bg"]
-	}
+func ProcessFiles(c *cfg.Config, args []string) error {
+	fds, pattern, err := determineIO(c, args)
 
 	if err != nil {
 		return err
 	}
 
+	if err := c.PreparePattern(pattern); err != nil {
+		return err
+	}
+
 	for _, fd := range fds {
-		data, err := parseFile(fd, pattern)
+		data, err := Parse(*c, fd)
 		if err != nil {
 			return err
 		}
-		printData(&data)
+
+		err = PrepareColumns(c, &data)
+		if err != nil {
+			return err
+		}
+
+		printData(os.Stdout, *c, &data)
 	}
 
 	return nil
 }
 
-func determineIO(args []string) ([]io.Reader, string, error) {
+func determineIO(c *cfg.Config, args []string) ([]io.Reader, string, error) {
 	var pattern string
 	var fds []io.Reader
 	var havefiles bool
@@ -61,7 +63,7 @@ func determineIO(args []string) ([]io.Reader, string, error) {
 			// first  one is  not a  file, consider  it as  regexp and
 			// shift arg list
 			pattern = args[0]
-			Pattern = args[0] // FIXME
+			c.Pattern = args[0] // used for colorization by printData()
 			args = args[1:]
 		}
 

lib/parser.go

@@ -19,28 +19,94 @@ package lib
 
 import (
 	"bufio"
+	"encoding/csv"
 	"errors"
 	"fmt"
 	"github.com/alecthomas/repr"
+	"github.com/tlinden/tablizer/cfg"
 	"io"
 	"regexp"
 	"strings"
 )
 
+/*
+   Parser switch
+*/
+func Parse(c cfg.Config, input io.Reader) (Tabdata, error) {
+	if len(c.Separator) == 1 {
+		return parseCSV(c, input)
+	}
+
+	return parseTabular(c, input)
+}
+
+/*
+   Parse CSV input.
+*/
+func parseCSV(c cfg.Config, input io.Reader) (Tabdata, error) {
+	var content io.Reader = input
+	data := Tabdata{}
+
+	if len(c.Pattern) > 0 {
+		scanner := bufio.NewScanner(input)
+		lines := []string{}
+		hadFirst := false
+		for scanner.Scan() {
+			line := strings.TrimSpace(scanner.Text())
+			if hadFirst {
+				// don't match 1st line, it's the header
+				if c.PatternR.MatchString(line) == c.InvertMatch {
+					// by default  -v is false, so if a  line does NOT
+					// match the pattern, we will ignore it. However,
+					// if the user specified -v, the matching is inverted,
+					// so we ignore all lines, which DO match.
+					continue
+				}
+			}
+			lines = append(lines, line)
+			hadFirst = true
+		}
+		content = strings.NewReader(strings.Join(lines, "\n"))
+	}
+
+	csvreader := csv.NewReader(content)
+	csvreader.Comma = rune(c.Separator[0])
+
+	records, err := csvreader.ReadAll()
+	if err != nil {
+		return data, errors.Unwrap(fmt.Errorf("Could not parse CSV input: %w", err))
+	}
+
+	if len(records) >= 1 {
+		data.headers = records[0]
+		data.columns = len(records)
+
+		for _, head := range data.headers {
+			// register widest header field
+			headerlen := len(head)
+			if headerlen > data.maxwidthHeader {
+				data.maxwidthHeader = headerlen
+			}
+		}
+
+		if len(records) > 1 {
+			data.entries = records[1:]
+		}
+	}
+
+	return data, nil
+}
+
 /*
    Parse tabular input.
 */
-func parseFile(input io.Reader, pattern string) (Tabdata, error) {
+func parseTabular(c cfg.Config, input io.Reader) (Tabdata, error) {
 	data := Tabdata{}
 
 	var scanner *bufio.Scanner
 
 	hadFirst := false
-	separate := regexp.MustCompile(Separator)
-	patternR, err := regexp.Compile(pattern)
-	if err != nil {
-		return data, errors.Unwrap(fmt.Errorf("Regexp pattern %s is invalid: %w", pattern, err))
-	}
+	separate := regexp.MustCompile(c.Separator)
 
 	scanner = bufio.NewScanner(input)
 
@@ -75,8 +141,8 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
 			}
 		} else {
 			// data processing
-			if len(pattern) > 0 {
-				if patternR.MatchString(line) == InvertMatch {
+			if len(c.Pattern) > 0 {
+				if c.PatternR.MatchString(line) == c.InvertMatch {
 					// by default  -v is false, so if a  line does NOT
 					// match the pattern, we will ignore it. However,
 					// if the user specified -v, the matching is inverted,
@@ -88,16 +154,6 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
 			idx := 0 // we cannot use the header index, because we could exclude columns
 			values := []string{}
 			for _, part := range parts {
-				width := len(strings.TrimSpace(part))
-
-				if len(data.maxwidthPerCol)-1 < idx {
-					data.maxwidthPerCol = append(data.maxwidthPerCol, width)
-				} else {
-					if width > data.maxwidthPerCol[idx] {
-						data.maxwidthPerCol[idx] = width
-					}
-				}
-
 				// if Debug {
 				// 	fmt.Printf("<%s> ", value)
 				// }
@@ -105,6 +161,12 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
 
 				idx++
 			}
+
+			// fill up missing fields, if any
+			for i := len(values); i < len(data.headers); i++ {
+				values = append(values, "")
+			}
+
 			data.entries = append(data.entries, values)
 		}
 	}
@@ -113,7 +175,7 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
 		return data, errors.Unwrap(fmt.Errorf("Failed to read from io.Reader: %w", scanner.Err()))
 	}
 
-	if Debug {
+	if c.Debug {
 		repr.Print(data)
 	}
 

lib/parser_test.go

@@ -19,45 +19,64 @@ package lib
 
 import (
 	"fmt"
+	"github.com/tlinden/tablizer/cfg"
 	"reflect"
 	"strings"
 	"testing"
 )
 
+var input = []struct {
+	name      string
+	text      string
+	separator string
+}{
+	{
+		name:      "tabular-data",
+		separator: cfg.DefaultSeparator,
+		text: `
+ONE    TWO    THREE  
+asd    igig   cxxxncnc  
+19191  EDD 1  X`,
+	},
+	{
+		name:      "csv-data",
+		separator: ",",
+		text: `
+ONE,TWO,THREE
+asd,igig,cxxxncnc
+19191,"EDD 1",X`,
+	},
+}
+
 func TestParser(t *testing.T) {
 	data := Tabdata{
 		maxwidthHeader: 5,
-		maxwidthPerCol: []int{
-			5, 5, 8,
-		},
-		columns: 3,
+		columns:        3,
 		headers: []string{
 			"ONE", "TWO", "THREE",
 		},
 		entries: [][]string{
-			{
-				"asd", "igig", "cxxxncnc",
-			},
-			{
-				"19191", "EDD 1", "X",
-			},
+			{"asd", "igig", "cxxxncnc"},
+			{"19191", "EDD 1", "X"},
 		},
 	}
 
-	table := `ONE    TWO    THREE  
-asd    igig   cxxxncnc  
-19191  EDD 1  X`
+	for _, in := range input {
+		testname := fmt.Sprintf("parse-%s", in.name)
+		t.Run(testname, func(t *testing.T) {
+			readFd := strings.NewReader(strings.TrimSpace(in.text))
+			c := cfg.Config{Separator: in.separator}
+			gotdata, err := Parse(c, readFd)
 
-	readFd := strings.NewReader(table)
-	gotdata, err := parseFile(readFd, "")
-	Separator = DefaultSeparator
+			if err != nil {
+				t.Errorf("Parser returned error: %s\nData processed so far: %+v", err, gotdata)
+			}
 
-	if err != nil {
-		t.Errorf("Parser returned error: %s\nData processed so far: %+v", err, gotdata)
-	}
-
-	if !reflect.DeepEqual(data, gotdata) {
-		t.Errorf("Parser returned invalid data, Regex: %s\nExp: %+v\nGot: %+v\n", Separator, data, gotdata)
+			if !reflect.DeepEqual(data, gotdata) {
+				t.Errorf("Parser returned invalid data\nExp: %+v\nGot: %+v\n",
+					data, gotdata)
+			}
+		})
 	}
 }
 
@@ -66,47 +85,81 @@ func TestParserPatternmatching(t *testing.T) {
 		entries [][]string
 		pattern string
 		invert  bool
+		want    bool
 	}{
 		{
 			entries: [][]string{
-				{
-					"asd", "igig", "cxxxncnc",
-				},
+				{"asd", "igig", "cxxxncnc"},
 			},
 			pattern: "ig",
 			invert:  false,
 		},
 		{
 			entries: [][]string{
-				{
-					"19191", "EDD 1", "X",
-				},
+				{"19191", "EDD 1", "X"},
 			},
 			pattern: "ig",
 			invert:  true,
 		},
 	}
 
-	table := `ONE    TWO    THREE  
-asd    igig   cxxxncnc  
-19191  EDD 1  X`
+	for _, in := range input {
+		for _, tt := range tests {
+			testname := fmt.Sprintf("parse-%s-with-pattern-%s-inverted-%t",
+				in.name, tt.pattern, tt.invert)
+			t.Run(testname, func(t *testing.T) {
+				c := cfg.Config{InvertMatch: tt.invert, Pattern: tt.pattern,
+					Separator: in.separator}
 
-	for _, tt := range tests {
-		testname := fmt.Sprintf("parse-with-inverted-pattern-%t", tt.invert)
-		t.Run(testname, func(t *testing.T) {
-			InvertMatch = tt.invert
+				_ = c.PreparePattern(tt.pattern)
 
-			readFd := strings.NewReader(table)
-			gotdata, err := parseFile(readFd, tt.pattern)
+				readFd := strings.NewReader(strings.TrimSpace(in.text))
+				gotdata, err := Parse(c, readFd)
 
-			if err != nil {
-				t.Errorf("Parser returned error: %s\nData processed so far: %+v", err, gotdata)
-			}
-
-			if !reflect.DeepEqual(tt.entries, gotdata.entries) {
-				t.Errorf("Parser returned invalid data (pattern: %s, invert: %t)\nExp: %+v\nGot: %+v\n",
-					tt.pattern, tt.invert, tt.entries, gotdata.entries)
-			}
-		})
+				if err != nil {
+					if !tt.want {
+						t.Errorf("Parser returned error: %s\nData processed so far: %+v",
+							err, gotdata)
+					}
+				} else {
+					if !reflect.DeepEqual(tt.entries, gotdata.entries) {
+						t.Errorf("Parser returned invalid data (pattern: %s, invert: %t)\nExp: %+v\nGot: %+v\n",
+							tt.pattern, tt.invert, tt.entries, gotdata.entries)
+					}
+				}
+			})
+		}
+	}
+}
+
+func TestParserIncompleteRows(t *testing.T) {
+	data := Tabdata{
+		maxwidthHeader: 5,
+		columns:        3,
+		headers: []string{
+			"ONE", "TWO", "THREE",
+		},
+		entries: [][]string{
+			{"asd", "igig", ""},
+			{"19191", "EDD 1", "X"},
+		},
+	}
+
+	table := `
+ONE    TWO    THREE  
+asd    igig
+19191  EDD 1  X`
+
+	readFd := strings.NewReader(strings.TrimSpace(table))
+	c := cfg.Config{Separator: cfg.DefaultSeparator}
+	gotdata, err := Parse(c, readFd)
+
+	if err != nil {
+		t.Errorf("Parser returned error: %s\nData processed so far: %+v", err, gotdata)
+	}
+
+	if !reflect.DeepEqual(data, gotdata) {
+		t.Errorf("Parser returned invalid data, Regex: %s\nExp: %+v\nGot: %+v\n",
+			c.Separator, data, gotdata)
 	}
 }

lib/printer.go

@@ -18,47 +18,60 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
 package lib
 
 import (
+	"encoding/csv"
 	"fmt"
 	"github.com/gookit/color"
 	"github.com/olekukonko/tablewriter"
+	"github.com/tlinden/tablizer/cfg"
+	"gopkg.in/yaml.v3"
+	"io"
+	"log"
 	"regexp"
+	"strconv"
 	"strings"
 )
 
-func printData(data *Tabdata) {
+func printData(w io.Writer, c cfg.Config, data *Tabdata) {
 	// some output preparations:
 
-	if OutputMode != "shell" {
-		// not needed in eval string
-		numberizeHeaders(data)
-	}
+	// add numbers to headers and remove this we're not interested in
+	numberizeAndReduceHeaders(c, data)
 
 	// remove unwanted columns, if any
-	reduceColumns(data)
+	reduceColumns(c, data)
 
 	// sort the data
-	sortTable(data, SortByColumn)
+	sortTable(c, data)
 
-	switch OutputMode {
-	case "extended":
-		printExtendedData(data)
-	case "ascii":
-		printAsciiData(data)
-	case "orgtbl":
-		printOrgmodeData(data)
-	case "markdown":
-		printMarkdownData(data)
-	case "shell":
-		printShellData(data)
+	switch c.OutputMode {
+	case cfg.Extended:
+		printExtendedData(w, c, data)
+	case cfg.Ascii:
+		printAsciiData(w, c, data)
+	case cfg.Orgtbl:
+		printOrgmodeData(w, c, data)
+	case cfg.Markdown:
+		printMarkdownData(w, c, data)
+	case cfg.Shell:
+		printShellData(w, c, data)
+	case cfg.Yaml:
+		printYamlData(w, c, data)
+	case cfg.CSV:
+		printCSVData(w, c, data)
 	default:
-		printAsciiData(data)
+		printAsciiData(w, c, data)
 	}
+
+}
+
+func output(w io.Writer, str string) {
+	fmt.Fprint(w, str)
 }
 
 /*
    Emacs org-mode compatible table (also orgtbl-mode)
 */
-func printOrgmodeData(data *Tabdata) {
+func printOrgmodeData(w io.Writer, c cfg.Config, data *Tabdata) {
 	tableString := &strings.Builder{}
 	table := tablewriter.NewWriter(tableString)
 
@@ -81,19 +94,19 @@ func printOrgmodeData(data *Tabdata) {
 	   | cell | cell |
 	   |------+------|
 	*/
-	leftR := regexp.MustCompile("(?m)^\\+")
-	rightR := regexp.MustCompile("\\+(?m)$")
+	leftR := regexp.MustCompile(`(?m)^\\+`)
+	rightR := regexp.MustCompile(`\\+(?m)$`)
 
-	color.Print(
-		colorizeData(
+	output(w, color.Sprint(
+		colorizeData(c,
 			rightR.ReplaceAllString(
-				leftR.ReplaceAllString(tableString.String(), "|"), "|")))
+				leftR.ReplaceAllString(tableString.String(), "|"), "|"))))
 }
 
 /*
    Markdown table
 */
-func printMarkdownData(data *Tabdata) {
+func printMarkdownData(w io.Writer, c cfg.Config, data *Tabdata) {
 	tableString := &strings.Builder{}
 	table := tablewriter.NewWriter(tableString)
 
@@ -107,13 +120,13 @@ func printMarkdownData(data *Tabdata) {
 	table.SetCenterSeparator("|")
 
 	table.Render()
-	color.Print(colorizeData(tableString.String()))
+	output(w, color.Sprint(colorizeData(c, tableString.String())))
 }
 
 /*
    Simple ASCII table without any borders etc, just like the input we expect
 */
-func printAsciiData(data *Tabdata) {
+func printAsciiData(w io.Writer, c cfg.Config, data *Tabdata) {
 	tableString := &strings.Builder{}
 	table := tablewriter.NewWriter(tableString)
 
@@ -137,47 +150,101 @@ func printAsciiData(data *Tabdata) {
 	table.SetNoWhiteSpace(true)
 
 	table.Render()
-	color.Print(colorizeData(tableString.String()))
+	output(w, color.Sprint(colorizeData(c, tableString.String())))
 }
 
 /*
    We simulate the \x command of psql (the PostgreSQL client)
 */
-func printExtendedData(data *Tabdata) {
+func printExtendedData(w io.Writer, c cfg.Config, data *Tabdata) {
 	// needed for data output
 	format := fmt.Sprintf("%%%ds: %%s\n", data.maxwidthHeader)
-
+	out := ""
 	if len(data.entries) > 0 {
 		for _, entry := range data.entries {
 			for i, value := range entry {
-				color.Printf(format, data.headers[i], value)
+				out += color.Sprintf(format, data.headers[i], value)
 			}
 
-			fmt.Println()
+			out += "\n"
 		}
 	}
+
+	output(w, colorizeData(c, out))
 }
 
 /*
    Shell output, ready to be eval'd. Just like FreeBSD stat(1)
 */
-func printShellData(data *Tabdata) {
+func printShellData(w io.Writer, c cfg.Config, data *Tabdata) {
+	out := ""
 	if len(data.entries) > 0 {
-		var idx int
 		for _, entry := range data.entries {
-			idx = 0
 			shentries := []string{}
 			for i, value := range entry {
-				if len(Columns) > 0 {
-					if !contains(UseColumns, i+1) {
-						continue
-					}
-				}
-
-				shentries = append(shentries, fmt.Sprintf("%s=\"%s\"", data.headers[idx], value))
-				idx++
+				shentries = append(shentries, fmt.Sprintf("%s=\"%s\"",
+					data.headers[i], value))
 			}
-			fmt.Println(strings.Join(shentries, " "))
+			out += fmt.Sprint(strings.Join(shentries, " ")) + "\n"
 		}
 	}
+
+	// no colorization here
+	output(w, out)
+}
+
+func printYamlData(w io.Writer, c cfg.Config, data *Tabdata) {
+	type D struct {
+		Entries []map[string]interface{} `yaml:"entries"`
+	}
+
+	d := D{}
+
+	for _, entry := range data.entries {
+		ml := map[string]interface{}{}
+
+		for i, entry := range entry {
+			style := yaml.TaggedStyle
+			_, err := strconv.Atoi(entry)
+			if err != nil {
+				style = yaml.DoubleQuotedStyle
+			}
+
+			ml[strings.ToLower(data.headers[i])] =
+				&yaml.Node{
+					Kind:  yaml.ScalarNode,
+					Style: style,
+					Value: entry}
+		}
+
+		d.Entries = append(d.Entries, ml)
+	}
+
+	yamlstr, err := yaml.Marshal(&d)
+
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	output(w, string(yamlstr))
+}
+
+func printCSVData(w io.Writer, c cfg.Config, data *Tabdata) {
+	csvout := csv.NewWriter(w)
+
+	if err := csvout.Write(data.headers); err != nil {
+		log.Fatalln("error writing record to csv:", err)
+	}
+
+	for _, entry := range data.entries {
+		if err := csvout.Write(entry); err != nil {
+			log.Fatalln("error writing record to csv:", err)
+		}
+	}
+
+	csvout.Flush()
+
+	if err := csvout.Error(); err != nil {
+		log.Fatal(err)
+	}
 }

lib/printer_test.go

@@ -18,197 +18,276 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
 package lib
 
 import (
+	"bytes"
 	"fmt"
-	"github.com/gookit/color"
-	"os"
+	//"github.com/alecthomas/repr"
+	"github.com/tlinden/tablizer/cfg"
 	"strings"
 	"testing"
 )
 
-func stdout2pipe(t *testing.T) (*os.File, *os.File) {
-	reader, writer, err := os.Pipe()
-	if err != nil {
-		t.Fatal(err)
+func newData() Tabdata {
+	return Tabdata{
+		maxwidthHeader: 8,
+		columns:        4,
+		headers: []string{
+			"NAME",
+			"DURATION",
+			"COUNT",
+			"WHEN",
+		},
+		entries: [][]string{
+			{
+				"beta",
+				"1d10h5m1s",
+				"33",
+				"3/1/2014",
+			},
+			{
+				"alpha",
+				"4h35m",
+				"170",
+				"2013-Feb-03",
+			},
+			{
+				"ceta",
+				"33d12h",
+				"9",
+				"06/Jan/2008 15:04:05 -0700",
+			},
+		},
 	}
-	origStdout := os.Stdout
-	os.Stdout = writer
+}
 
-	// we need to tell the color mode the io.Writer, even if we don't usw colorization
-	color.SetOutput(writer)
+var tests = []struct {
+	name string // so we can identify which one fails, can be the same
+	// for multiple tests, because flags will be appended to the name
+	sortby    string // empty == default
+	column    int    // sort by this column, 0 == default first or NO Sort
+	desc      bool   // sort in descending order, default == ascending
+	nonum     bool   // hide numbering
+	mode      int    // shell, orgtbl, etc. empty == default: ascii
+	usecol    []int  // columns to display, empty == display all
+	usecolstr string // for testname, must match usecol
+	expect    string // rendered output we expect
+}{
+	// --------------------- Default settings mode tests ``
+	{
+		mode: cfg.Ascii,
+		name: "default",
+		expect: `
+NAME(1)	DURATION(2)	COUNT(3)	WHEN(4)                    
+beta   	1d10h5m1s  	33      	3/1/2014                  	
+alpha  	4h35m      	170     	2013-Feb-03               	
+ceta   	33d12h     	9       	06/Jan/2008 15:04:05 -0700`,
+	},
+	{
+		mode: cfg.CSV,
+		name: "csv",
+		expect: `
+NAME,DURATION,COUNT,WHEN
+beta,1d10h5m1s,33,3/1/2014
+alpha,4h35m,170,2013-Feb-03
+ceta,33d12h,9,06/Jan/2008 15:04:05 -0700`,
+	},
+	{
+		name: "default",
+		mode: cfg.Orgtbl,
+		expect: `
++---------+-------------+----------+----------------------------+
+| NAME(1) | DURATION(2) | COUNT(3) |          WHEN(4)           |
++---------+-------------+----------+----------------------------+
+| beta    | 1d10h5m1s   |       33 | 3/1/2014                   |
+| alpha   | 4h35m       |      170 | 2013-Feb-03                |
+| ceta    | 33d12h      |        9 | 06/Jan/2008 15:04:05 -0700 |
++---------+-------------+----------+----------------------------+`,
+	},
+	{
+		name: "default",
+		mode: cfg.Markdown,
+		expect: `
+| NAME(1) | DURATION(2) | COUNT(3) |          WHEN(4)           |
+|---------|-------------|----------|----------------------------|
+| beta    | 1d10h5m1s   |       33 | 3/1/2014                   |
+| alpha   | 4h35m       |      170 | 2013-Feb-03                |
+| ceta    | 33d12h      |        9 | 06/Jan/2008 15:04:05 -0700 |`,
+	},
+	{
+		name:  "default",
+		mode:  cfg.Shell,
+		nonum: true,
+		expect: `
+NAME="beta" DURATION="1d10h5m1s" COUNT="33" WHEN="3/1/2014"
+NAME="alpha" DURATION="4h35m" COUNT="170" WHEN="2013-Feb-03"
+NAME="ceta" DURATION="33d12h" COUNT="9" WHEN="06/Jan/2008 15:04:05 -0700"`,
+	},
+	{
+		name:  "default",
+		mode:  cfg.Yaml,
+		nonum: true,
+		expect: `
+entries:
+    - count: 33
+      duration: "1d10h5m1s"
+      name: "beta"
+      when: "3/1/2014"
+    - count: 170
+      duration: "4h35m"
+      name: "alpha"
+      when: "2013-Feb-03"
+    - count: 9
+      duration: "33d12h"
+      name: "ceta"
+      when: "06/Jan/2008 15:04:05 -0700"`,
+	},
+	{
+		name: "default",
+		mode: cfg.Extended,
+		expect: `
+    NAME(1): beta
+DURATION(2): 1d10h5m1s
+   COUNT(3): 33
+    WHEN(4): 3/1/2014
 
-	return origStdout, reader
+    NAME(1): alpha
+DURATION(2): 4h35m
+   COUNT(3): 170
+    WHEN(4): 2013-Feb-03
+
+    NAME(1): ceta
+DURATION(2): 33d12h
+   COUNT(3): 9
+    WHEN(4): 06/Jan/2008 15:04:05 -0700`,
+	},
+
+	//------------------------ SORT TESTS
+	{
+		name:   "sortbycolumn",
+		column: 3,
+		sortby: "numeric",
+		desc:   false,
+		expect: `
+NAME(1)	DURATION(2)	COUNT(3)	WHEN(4)                    
+ceta   	33d12h     	9       	06/Jan/2008 15:04:05 -0700	
+beta   	1d10h5m1s  	33      	3/1/2014                  	
+alpha  	4h35m      	170     	2013-Feb-03`,
+	},
+	{
+		name:   "sortbycolumn",
+		column: 4,
+		sortby: "time",
+		desc:   false,
+		expect: `
+NAME(1)	DURATION(2)	COUNT(3)	WHEN(4)                    
+ceta   	33d12h     	9       	06/Jan/2008 15:04:05 -0700	
+alpha  	4h35m      	170     	2013-Feb-03               	
+beta   	1d10h5m1s  	33      	3/1/2014`,
+	},
+	{
+		name:   "sortbycolumn",
+		column: 2,
+		sortby: "duration",
+		desc:   false,
+		expect: `
+NAME(1)	DURATION(2)	COUNT(3)	WHEN(4)                    
+alpha  	4h35m      	170     	2013-Feb-03               	
+beta   	1d10h5m1s  	33      	3/1/2014                  	
+ceta   	33d12h     	9       	06/Jan/2008 15:04:05 -0700`,
+	},
+
+	//  -----------------------  UseColumns Tests
+	{
+		name:      "usecolumns",
+		usecol:    []int{1, 4},
+		usecolstr: "1,4",
+		expect: `
+NAME(1)	WHEN(4)                    
+beta   	3/1/2014                  	
+alpha  	2013-Feb-03               	
+ceta   	06/Jan/2008 15:04:05 -0700`,
+	},
+	{
+		name:      "usecolumns",
+		usecol:    []int{2},
+		usecolstr: "2",
+		expect: `
+DURATION(2) 
+1d10h5m1s  	
+4h35m      	
+33d12h`,
+	},
+	{
+		name:      "usecolumns",
+		usecol:    []int{3},
+		usecolstr: "3",
+		expect: `
+COUNT(3) 
+33      	
+170     	
+9`,
+	},
+	{
+		name:      "usecolumns",
+		column:    0,
+		usecol:    []int{1, 3},
+		usecolstr: "1,3",
+		expect: `
+NAME(1)	COUNT(3) 
+beta   	33      	
+alpha  	170     	
+ceta   	9`,
+	},
+	{
+		name:      "usecolumns",
+		usecol:    []int{2, 4},
+		usecolstr: "2,4",
+		expect: `
+DURATION(2)	WHEN(4)                    
+1d10h5m1s  	3/1/2014                  	
+4h35m      	2013-Feb-03               	
+33d12h     	06/Jan/2008 15:04:05 -0700`,
+	},
 }
 
 func TestPrinter(t *testing.T) {
-	startdata := Tabdata{
-		maxwidthHeader: 5,
-		maxwidthPerCol: []int{
-			5,
-			5,
-			8,
-		},
-		columns: 3,
-		headers: []string{
-			"ONE", "TWO", "THREE",
-		},
-		entries: [][]string{
-			{
-				"asd", "igig", "cxxxncnc",
-			},
-			{
-				"19191", "EDD 1", "X",
-			},
-		},
-	}
-
-	expects := map[string]string{
-		"ascii": `ONE(1)	TWO(2)	THREE(3) 
-asd   	igig  	cxxxncnc	
-19191 	EDD 1 	X`,
-
-		"orgtbl": `|--------+--------+----------|
-| ONE(1) | TWO(2) | THREE(3) |
-|--------+--------+----------|
-| asd    | igig   | cxxxncnc |
-|  19191 | EDD 1  | X        |
-|--------+--------+----------|`,
-
-		"markdown": `| ONE(1) | TWO(2) | THREE(3) |
-|--------|--------|----------|
-| asd    | igig   | cxxxncnc |
-|  19191 | EDD 1  | X        |`,
-
-		"shell": `ONE="asd" TWO="igig" THREE="cxxxncnc"
-ONE="19191" TWO="EDD 1" THREE="X"`,
-
-		"extended": `ONE(1): asd
-  TWO(2): igig
-THREE(3): cxxxncnc
-
-  ONE(1): 19191
-  TWO(2): EDD 1
-THREE(3): X`,
-	}
-
-	NoColor = true
-	SortByColumn = 0 // disable sorting
-
-	origStdout, reader := stdout2pipe(t)
-
-	for mode, expect := range expects {
-		testname := fmt.Sprintf("print-%s", mode)
-		t.Run(testname, func(t *testing.T) {
-
-			OutputMode = mode
-			//  we need  to reset  our  mock data,  since it's  being
-			// modified in printData()
-			data := startdata
-			printData(&data)
-
-			buf := make([]byte, 1024)
-			n, err := reader.Read(buf)
-			if err != nil {
-				t.Fatal(err)
-			}
-			buf = buf[:n]
-			output := strings.TrimSpace(string(buf))
-
-			if output != expect {
-				t.Errorf("output mode: %s, got:\n%s\nwant:\n%s\n (%d <=> %d)",
-					mode, output, expect, len(output), len(expect))
-			}
-		})
-	}
-
-	// Restore
-	os.Stdout = origStdout
-
-}
-
-func TestSortPrinter(t *testing.T) {
-	startdata := Tabdata{
-		maxwidthHeader: 5,
-		maxwidthPerCol: []int{
-			3,
-			3,
-			2,
-		},
-		columns: 3,
-		headers: []string{
-			"ONE", "TWO", "THREE",
-		},
-		entries: [][]string{
-			{
-				"abc", "345", "b1",
-			},
-			{
-				"bcd", "234", "a2",
-			},
-			{
-				"cde", "123", "c3",
-			},
-		},
-	}
-
-	var tests = []struct {
-		data   Tabdata
-		sortby int
-		expect string
-	}{
-		{
-			data:   startdata,
-			sortby: 1,
-			expect: `ONE(1)	TWO(2)	THREE(3) 
-abc   	345   	b1      	
-bcd   	234   	a2      	
-cde   	123   	c3`,
-		},
-
-		{
-			data:   startdata,
-			sortby: 2,
-			expect: `ONE(1)	TWO(2)	THREE(3) 
-cde   	123   	c3      	
-bcd   	234   	a2      	
-abc   	345   	b1`,
-		},
-
-		{
-			data:   startdata,
-			sortby: 3,
-			expect: `ONE(1)	TWO(2)	THREE(3) 
-bcd   	234   	a2      	
-abc   	345   	b1      	
-cde   	123   	c3`,
-		},
-	}
-
-	NoColor = true
-	OutputMode = "ascii"
-	origStdout, reader := stdout2pipe(t)
-
 	for _, tt := range tests {
-		testname := fmt.Sprintf("print-sorted-table-%d", tt.sortby)
+		testname := fmt.Sprintf("print-sortcol-%d-desc-%t-sortby-%s-mode-%d-usecolumns-%s",
+			tt.column, tt.desc, tt.sortby, tt.mode, tt.usecolstr)
 		t.Run(testname, func(t *testing.T) {
-			SortByColumn = tt.sortby
+			// replaces os.Stdout, but we ignore it
+			var w bytes.Buffer
 
-			printData(&tt.data)
-
-			buf := make([]byte, 1024)
-			n, err := reader.Read(buf)
-			if err != nil {
-				t.Fatal(err)
+			// cmd flags
+			c := cfg.Config{
+				SortByColumn:   tt.column,
+				SortDescending: tt.desc,
+				SortMode:       tt.sortby,
+				OutputMode:     tt.mode,
+				NoNumbering:    tt.nonum,
+				UseColumns:     tt.usecol,
+				NoColor:        true,
 			}
-			buf = buf[:n]
-			output := strings.TrimSpace(string(buf))
 
-			if output != tt.expect {
-				t.Errorf("sort column: %d, got:\n%s\nwant:\n%s",
-					tt.sortby, output, tt.expect)
+			c.ApplyDefaults()
+
+			// the test checks the len!
+			if len(tt.usecol) > 0 {
+				c.Columns = "yes"
+			} else {
+				c.Columns = ""
+			}
+
+			testdata := newData()
+			exp := strings.TrimSpace(tt.expect)
+
+			printData(&w, c, &testdata)
+
+			got := strings.TrimSpace(w.String())
+
+			if got != exp {
+				t.Errorf("not rendered correctly:\n+++ got:\n%s\n+++ want:\n%s",
+					got, exp)
 			}
 		})
 	}
-
-	// Restore
-	os.Stdout = origStdout
 }

lib/sort.go

@@ -18,15 +18,22 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
 package lib
 
 import (
+	"github.com/araddon/dateparse"
+	"github.com/tlinden/tablizer/cfg"
+	"regexp"
 	"sort"
+	"strconv"
 )
 
-func sortTable(data *Tabdata, col int) {
-	if col <= 0 {
+func sortTable(c cfg.Config, data *Tabdata) {
+	if c.SortByColumn <= 0 {
 		// no sorting wanted
 		return
 	}
 
+	// slightly modified here to match internal array indicies
+	col := c.SortByColumn
+
 	col-- // ui starts counting by 1, but use 0 internally
 
 	// sanity checks
@@ -41,6 +48,74 @@ func sortTable(data *Tabdata, col int) {
 
 	// actual sorting
 	sort.SliceStable(data.entries, func(i, j int) bool {
-		return data.entries[i][col] < data.entries[j][col]
+		return compare(&c, data.entries[i][col], data.entries[j][col])
 	})
 }
+
+// config is not modified here, but it would be inefficient to copy it every loop
+func compare(c *cfg.Config, a string, b string) bool {
+	var comp bool
+
+	switch c.SortMode {
+	case "numeric":
+		left, err := strconv.Atoi(a)
+		if err != nil {
+			left = 0
+		}
+		right, err := strconv.Atoi(b)
+		if err != nil {
+			right = 0
+		}
+		comp = left < right
+	case "duration":
+		left := duration2int(a)
+		right := duration2int(b)
+		comp = left < right
+	case "time":
+		left, _ := dateparse.ParseAny(a)
+		right, _ := dateparse.ParseAny(b)
+		comp = left.Unix() < right.Unix()
+	default:
+		comp = a < b
+	}
+
+	if c.SortDescending {
+		comp = !comp
+	}
+
+	return comp
+}
+
+/*
+   We could use time.ParseDuration(), but this doesn't support days.
+
+   We  could also  use github.com/xhit/go-str2duration/v2,  which does
+   the job,  but it's  just another dependency,  just for  this little
+   gem. And  we don't need a  time.Time value. And int  is good enough
+   for duration comparision.
+
+   Convert a  durartion into  an integer.  Valid  time units  are "s",
+   "m", "h" and "d".
+*/
+func duration2int(duration string) int {
+	re := regexp.MustCompile(`(\d+)([dhms])`)
+	seconds := 0
+
+	for _, match := range re.FindAllStringSubmatch(duration, -1) {
+		if len(match) == 3 {
+			v, _ := strconv.Atoi(match[1])
+			switch match[2][0] {
+			case 'd':
+				seconds += v * 86400
+			case 'h':
+				seconds += v * 3600
+			case 'm':
+				seconds += v * 60
+			case 's':
+				seconds += v
+			}
+		}
+	}
+
+	return seconds
+}

lib/sort_test.go

@@ -0,0 +1,79 @@
+/*
+Copyright © 2022 Thomas von Dein
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+package lib
+
+import (
+	"fmt"
+	"github.com/tlinden/tablizer/cfg"
+	"testing"
+)
+
+func TestDuration2Seconds(t *testing.T) {
+	var tests = []struct {
+		dur    string
+		expect int
+	}{
+		{"1d", 60 * 60 * 24},
+		{"1h", 60 * 60},
+		{"10m", 60 * 10},
+		{"2h4m10s", (60 * 120) + (4 * 60) + 10},
+		{"88u", 0},
+		{"19t77X what?4s", 4},
+	}
+
+	for _, tt := range tests {
+		testname := fmt.Sprintf("duration-%s", tt.dur)
+		t.Run(testname, func(t *testing.T) {
+			seconds := duration2int(tt.dur)
+			if seconds != tt.expect {
+				t.Errorf("got %d, want %d", seconds, tt.expect)
+			}
+		})
+	}
+}
+
+func TestCompare(t *testing.T) {
+	var tests = []struct {
+		mode string
+		a    string
+		b    string
+		want bool
+		desc bool
+	}{
+		// ascending
+		{"numeric", "10", "20", true, false},
+		{"duration", "2d4h5m", "45m", false, false},
+		{"time", "12/24/2022", "1/1/1970", false, false},
+
+		// descending
+		{"numeric", "10", "20", false, true},
+		{"duration", "2d4h5m", "45m", true, true},
+		{"time", "12/24/2022", "1/1/1970", true, true},
+	}
+
+	for _, tt := range tests {
+		testname := fmt.Sprintf("compare-mode-%s-a-%s-b-%s-desc-%t", tt.mode, tt.a, tt.b, tt.desc)
+		t.Run(testname, func(t *testing.T) {
+			c := cfg.Config{SortMode: tt.mode, SortDescending: tt.desc}
+			got := compare(&c, tt.a, tt.b)
+			if got != tt.want {
+				t.Errorf("got %t, want %t", got, tt.want)
+			}
+		})
+	}
+}

tablizer.1

@@ -133,7 +133,7 @@
 .\" ========================================================================
 .\"
 .IX Title "TABLIZER 1"
-.TH TABLIZER 1 "2022-10-14" "1" "User Commands"
+.TH TABLIZER 1 "2022-10-23" "1" "User Commands"
 .\" For nroff, turn off justification.  Always turn off hyphenation; it makes
 .\" way too many mistakes in technical documents.
 .if n .ad l
@@ -146,20 +146,33 @@ tablizer \- Manipulate tabular output of other programs
 \&    Usage:
 \&      tablizer [regex] [file, ...] [flags]
 \&    
-\&    Flags:
+\&    Operational Flags:
 \&      \-c, \-\-columns string     Only show the speficied columns (separated by ,)
-\&      \-d, \-\-debug              Enable debugging
-\&      \-h, \-\-help               help for tablizer
 \&      \-v, \-\-invert\-match       select non\-matching rows
-\&      \-m, \-\-man                Display manual page
 \&      \-n, \-\-no\-numbering       Disable header numbering
 \&      \-N, \-\-no\-color           Disable pattern highlighting
-\&      \-o, \-\-output string      Output mode \- one of: orgtbl, markdown, extended, ascii(default)
+\&      \-s, \-\-separator string   Custom field separator
+\&      \-k, \-\-sort\-by int        Sort by column (default: 1)
+\&
+\&    Output Flags (mutually exclusive):
 \&      \-X, \-\-extended           Enable extended output
 \&      \-M, \-\-markdown           Enable markdown table output
 \&      \-O, \-\-orgtbl             Enable org\-mode table output
-\&      \-s, \-\-separator string   Custom field separator
-\&      \-k, \-\-sort\-by int        Sort by column (default: 1)
+\&      \-S, \-\-shell              Enable shell evaluable ouput
+\&      \-Y, \-\-yaml               Enable yaml output
+\&      \-C, \-\-csv                Enable CSV output
+\&      \-A, \-\-ascii              Default output mode, ascii tabular
+\&
+\&    Sort Mode Flags (mutually exclusive):
+\&      \-a, \-\-sort\-age           sort according to age (duration) string
+\&      \-D, \-\-sort\-desc          Sort in descending order (default: ascending)
+\&      \-i, \-\-sort\-numeric       sort according to string numerical value
+\&      \-t, \-\-sort\-time          sort according to time string
+\&
+\&    Other Flags:
+\&      \-d, \-\-debug              Enable debugging
+\&      \-h, \-\-help               help for tablizer
+\&      \-m, \-\-man                Display manual page
 \&      \-v, \-\-version            Print program version
 .Ve
 .SH "DESCRIPTION"
@@ -209,7 +222,7 @@ have a numer associated with it, e.g.:
 .Ve
 .PP
 These numbers denote the column and  you can use them to specify which
-columns you want to have in your output:
+columns you want to have in your output (see \s-1COLUMNS\s0:
 .PP
 .Vb 1
 \&   kubectl get pods | tablizer \-c1,3
@@ -225,7 +238,19 @@ highlighted. You can disable this behavior with the \fB\-N\fR option.
 .PP
 Use the  \fB\-k\fR option to specify  by which column to  sort the tabular
 data (as in \s-1GNU\s0 \fBsort\fR\|(1)). The default sort column is the first one. To
-disable sorting at all, supply 0 (Zero) to \-k.
+disable sorting at all, supply 0  (Zero) to \-k. The default sort order
+is ascending. You can change this to descending order using the option
+\&\fB\-D\fR. The default  sort order is by string, but  there are other sort
+modes:
+.IP "\fB\-a \-\-sort\-age\fR" 4
+.IX Item "-a --sort-age"
+Sorts duration strings like \*(L"1d4h32m51s\*(R".
+.IP "\fB\-i \-\-sort\-numeric\fR" 4
+.IX Item "-i --sort-numeric"
+Sorts numeric fields.
+.IP "\fB\-t \-\-sort\-time\fR" 4
+.IX Item "-t --sort-time"
+Sorts timestamps.
 .PP
 Finally the  \fB\-d\fR option  enables debugging  output which  is mostly
 useful for the developer.
@@ -261,6 +286,38 @@ Example for a case insensitive search:
 .Vb 1
 \&    kubectl get pods \-A | tablizer "(?i)account"
 .Ve
+.SS "\s-1COLUMNS\s0"
+.IX Subsection "COLUMNS"
+The  parameter  \fB\-c\fR  can  be  used  to  specify,  which  columns  to
+display.  By default  tablizer numerizes  the header  names and  these
+numbers can  be used to specify  which header to display,  see example
+above.
+.PP
+However, beside  numbers, you  can also  use regular  expressions with
+\&\fB\-c\fR, also  separated by comma. And  you can mix column  numbers with
+regexps.
+.PP
+Lets take this table:
+.PP
+.Vb 4
+\&        PID TTY          TIME CMD
+\&      14001 pts/0    00:00:00 bash
+\&      42871 pts/0    00:00:00 ps
+\&      42872 pts/0    00:00:00 sed
+.Ve
+.PP
+We want to see only the \s-1CMD\s0 column and use a regex for this:
+.PP
+.Vb 6
+\&    ps | tablizer \-s \*(Aq\es+\*(Aq \-c C
+\&    CMD(4)
+\&    bash
+\&    ps
+\&    tablizer
+\&    sed
+.Ve
+.PP
+where \*(L"C\*(R" is our regexp which matches \s-1CMD.\s0
 .SS "\s-1OUTPUT MODES\s0"
 .IX Subsection "OUTPUT MODES"
 There might be cases  when the tabular output of a  program is way too
@@ -297,7 +354,22 @@ You can use this in an eval loop.
 .PP
 Beside normal  ascii mode  (the default) and  extended mode  there are
 more output modes available: \fBorgtbl\fR  which prints an Emacs org-mode
-table and \fBmarkdown\fR which prints a Markdown table.
+table and  \fBmarkdown\fR which prints  a Markdown table,  \fByaml\fR, which
+prints  yaml encoding  and \s-1CSV\s0  mode, which  prints a  comma separated
+value file.
+.SS "\s-1ENVIRONMENT VARIABLES\s0"
+.IX Subsection "ENVIRONMENT VARIABLES"
+\&\fBtablizer\fR supports  certain environment variables which  use can use
+to  influence   program  behavior.   Commandline  flags   have  always
+precedence over environment variables.
+.IP "<T_NO_HEADER_NUMBERING> \- disable numbering of header fields, like \fB\-n\fR." 4
+.IX Item "<T_NO_HEADER_NUMBERING> - disable numbering of header fields, like -n."
+.PD 0
+.IP "<T_COLUMNS> \- comma separated list of columns to output, like \fB\-c\fR" 4
+.IX Item "<T_COLUMNS> - comma separated list of columns to output, like -c"
+.IP "<\s-1NO_COLORS\s0> \- disable colorization of matches, like \fB\-N\fR" 4
+.IX Item "<NO_COLORS> - disable colorization of matches, like -N"
+.PD
 .SH "BUGS"
 .IX Header "BUGS"
 In order to report a bug, unexpected behavior, feature requests

tablizer.pod

@@ -7,20 +7,33 @@ tablizer - Manipulate tabular output of other programs
     Usage:
       tablizer [regex] [file, ...] [flags]
     
-    Flags:
+    Operational Flags:
       -c, --columns string     Only show the speficied columns (separated by ,)
-      -d, --debug              Enable debugging
-      -h, --help               help for tablizer
       -v, --invert-match       select non-matching rows
-      -m, --man                Display manual page
       -n, --no-numbering       Disable header numbering
       -N, --no-color           Disable pattern highlighting
-      -o, --output string      Output mode - one of: orgtbl, markdown, extended, ascii(default)
+      -s, --separator string   Custom field separator
+      -k, --sort-by int        Sort by column (default: 1)
+
+    Output Flags (mutually exclusive):
       -X, --extended           Enable extended output
       -M, --markdown           Enable markdown table output
       -O, --orgtbl             Enable org-mode table output
-      -s, --separator string   Custom field separator
-      -k, --sort-by int        Sort by column (default: 1)
+      -S, --shell              Enable shell evaluable ouput
+      -Y, --yaml               Enable yaml output
+      -C, --csv                Enable CSV output
+      -A, --ascii              Default output mode, ascii tabular
+
+    Sort Mode Flags (mutually exclusive):
+      -a, --sort-age           sort according to age (duration) string
+      -D, --sort-desc          Sort in descending order (default: ascending)
+      -i, --sort-numeric       sort according to string numerical value
+      -t, --sort-time          sort according to time string
+
+    Other Flags:
+      -d, --debug              Enable debugging
+      -h, --help               help for tablizer
+      -m, --man                Display manual page
       -v, --version            Print program version
 
 
@@ -67,7 +80,7 @@ have a numer associated with it, e.g.:
    NAME(1) READY(2) STATUS(3) RESTARTS(4) AGE(5)
 
 These numbers denote the column and  you can use them to specify which
-columns you want to have in your output:
+columns you want to have in your output (see L<COLUMNS>:
 
    kubectl get pods | tablizer -c1,3
 
@@ -81,7 +94,26 @@ highlighted. You can disable this behavior with the B<-N> option.
 
 Use the  B<-k> option to specify  by which column to  sort the tabular
 data (as in GNU sort(1)). The default sort column is the first one. To
-disable sorting at all, supply 0 (Zero) to -k.
+disable sorting at all, supply 0  (Zero) to -k. The default sort order
+is ascending. You can change this to descending order using the option
+B<-D>. The default  sort order is by string, but  there are other sort
+modes:
+
+=over
+
+=item B<-a --sort-age>
+
+Sorts duration strings like "1d4h32m51s".
+
+=item B<-i --sort-numeric>
+
+Sorts numeric fields.
+
+=item B<-t --sort-time>
+
+Sorts timestamps.
+
+=back
 
 Finally the  B<-d> option  enables debugging  output which  is mostly
 useful for the developer.
@@ -114,6 +146,35 @@ Example for a case insensitive search:
     kubectl get pods -A | tablizer "(?i)account"
 
 
+=head2 COLUMNS
+
+The  parameter  B<-c>  can  be  used  to  specify,  which  columns  to
+display.  By default  tablizer numerizes  the header  names and  these
+numbers can  be used to specify  which header to display,  see example
+above.
+
+However, beside  numbers, you  can also  use regular  expressions with
+B<-c>, also  separated by comma. And  you can mix column  numbers with
+regexps.
+
+Lets take this table:
+
+        PID TTY          TIME CMD
+      14001 pts/0    00:00:00 bash
+      42871 pts/0    00:00:00 ps
+      42872 pts/0    00:00:00 sed
+
+We want to see only the CMD column and use a regex for this:
+
+    ps | tablizer -s '\s+' -c C
+    CMD(4)
+    bash
+    ps
+    tablizer
+    sed
+
+where "C" is our regexp which matches CMD.
+
 =head2 OUTPUT MODES
 
 There might be cases  when the tabular output of a  program is way too
@@ -146,7 +207,26 @@ You can use this in an eval loop.
 
 Beside normal  ascii mode  (the default) and  extended mode  there are
 more output modes available: B<orgtbl>  which prints an Emacs org-mode
-table and B<markdown> which prints a Markdown table.
+table and  B<markdown> which prints  a Markdown table,  B<yaml>, which
+prints  yaml encoding  and CSV  mode, which  prints a  comma separated
+value file.
+
+=head2 ENVIRONMENT VARIABLES
+
+B<tablizer> supports  certain environment variables which  use can use
+to  influence   program  behavior.   Commandline  flags   have  always
+precedence over environment variables.
+
+=over
+
+=item <T_NO_HEADER_NUMBERING> - disable numbering of header fields, like B<-n>.
+
+=item <T_COLUMNS> - comma separated list of columns to output, like B<-c>
+
+=item <NO_COLORS> - disable colorization of matches, like B<-N>
+
+=back
+
 
 =head1 BUGS