Compare commits

...

14 Commits

Author SHA1 Message Date
8e2ba58ddb added -k parameter to sort by columns 2022-10-13 18:56:34 +02:00
6eedb60a6a minor update to current state 2022-10-11 18:50:58 +02:00
81fac864f1 using gh for release generation, fixed mkrel.sh to add version 2022-10-11 18:42:10 +02:00
e868b50c0f Merge branch 'development' 2022-10-11 18:34:11 +02:00
b9ed7d8cb7 fixed -X output in combination with -c 2022-10-11 13:47:34 +02:00
6ae4a1b6d9 added test for -X output 2022-10-11 09:11:33 +02:00
f890596b4c added pattern highlighting support 2022-10-10 20:14:51 +02:00
22ee24cfdf Merge branch 'development' 2022-10-06 20:05:20 +02:00
34e2b8d855 fixed issuer #4, version string missing, and added some docs about pattern syntax 2022-10-06 20:02:40 +02:00
196833ed3c Merge branch 'development' 2022-10-05 19:17:34 +02:00
85277bbf5e more refactoring, fixed bug in shell mode output, fixed default
Separator and fixed #3
2022-10-05 16:43:51 +02:00
26e50cf908 fix #1: use scanner.Split() instead of splitting by header position
boundaries, since this splitting cuts utf-8 chars which causes
distorted output.
2022-10-05 12:55:33 +02:00
5be18e27c9 Merge branch 'development' 2022-10-05 09:20:02 +02:00
2c410e1cb3 added -v flag, replace 'help' subcommand wich -m, more tests 2022-10-05 09:12:46 +02:00
18 changed files with 693 additions and 243 deletions

View File

@@ -26,7 +26,7 @@ GID = 0
BRANCH = $(shell git describe --all | cut -d/ -f2)
COMMIT = $(shell git rev-parse --short=8 HEAD)
BUILD = $(shell date +%Y.%m.%d.%H%M%S)
VERSION:= $(if $(filter $(BRANCH), development),$(version)-$(BRANCH)-$(COMMIT)-$(BUILD))
VERSION:= $(if $(filter $(BRANCH), development),$(version)-$(BRANCH)-$(COMMIT)-$(BUILD),$(version))
all: $(tool).1 cmd/$(tool).go buildlocal
@@ -45,6 +45,7 @@ buildlocal:
release:
./mkrel.sh $(tool) $(version)
gh release create $(version) --generate-notes releases/*
install: buildlocal
install -d -o $(UID) -g $(GID) $(PREFIX)/bin

View File

@@ -20,13 +20,13 @@ But you're only interested in the NAME and STATUS columns. Here's how
to do this with tablizer:
```
% kubectl get pods | ./tablizer
% kubectl get pods | tablizer
NAME(1) READY(2) STATUS(3) RESTARTS(4) AGE(5)
repldepl-7bcd8d5b64-7zq4l 1/1 Running 1 (69m ago) 5h26m
repldepl-7bcd8d5b64-m48n8 1/1 Running 1 (69m ago) 5h26m
repldepl-7bcd8d5b64-q2bf4 1/1 Running 1 (69m ago) 5h26m
% kubectl get pods | ./tablizer -c 1,3
% kubectl get pods | tablizer -c 1,3
NAME(1) STATUS(3)
repldepl-7bcd8d5b64-7zq4l Running
repldepl-7bcd8d5b64-m48n8 Running
@@ -34,27 +34,28 @@ repldepl-7bcd8d5b64-q2bf4 Running
```
Another use case is when the tabular output is so wide that lines are
being broken and the whole output is completely distorted. In such a
case you can use the `-x` flag to get an output similar to `\x` in `psql`:
being broken and the whole output is completely distorted. In such a
case you can use the `-o extended | -X` flag to get an output similar
to `\x` in `psql`:
```
% kubectl get pods | ./tablizer -x
NAME: repldepl-7bcd8d5b64-7zq4l
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
% kubectl get pods | tablizer -X
NAME: repldepl-7bcd8d5b64-7zq4l
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
AGE: 5h28m
NAME: repldepl-7bcd8d5b64-m48n8
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
NAME: repldepl-7bcd8d5b64-m48n8
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
AGE: 5h28m
NAME: repldepl-7bcd8d5b64-q2bf4
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
NAME: repldepl-7bcd8d5b64-q2bf4
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
AGE: 5h28m
```
@@ -63,11 +64,12 @@ Tablize can read one or more files or - if none specified - from STDIN.
You can also specify a regex pattern to reduce the output:
```
% kubectl get pods | ./tablizer q2bf4
% kubectl get pods | tablizer q2bf4
NAME(1) READY(2) STATUS(3) RESTARTS(4) AGE(5)
repldepl-7bcd8d5b64-q2bf4 1/1 Running 1 (69m ago) 5h26m
```
There are more output modes like org-mode (orgtbl) and markdown.
## Installation
@@ -106,6 +108,10 @@ https://github.com/TLINDEN/tablizer/blob/main/tablizer.pod
Or if you cloned the repository you can read it this way (perl needs
to be installed though): `perldoc tablizer.pod`.
If you have the binary installed, you can also read the man page with
this command:
tablizer --man
## Getting help

View File

@@ -18,33 +18,31 @@ package cmd
import (
"bytes"
"github.com/tlinden/tablizer/lib"
"fmt"
"github.com/spf13/cobra"
"github.com/tlinden/tablizer/lib"
"log"
"os"
"os/exec"
)
var helpCmd = &cobra.Command{
Use: "help",
Short: "Show documentation",
Run: func(cmd *cobra.Command, args []string) {
man := exec.Command("less", "-")
var ShowManual = false
var b bytes.Buffer
b.Write([]byte(manpage))
func man() {
man := exec.Command("less", "-")
man.Stdout = os.Stdout
man.Stdin = &b
man.Stderr = os.Stderr
var b bytes.Buffer
b.Write([]byte(manpage))
err := man.Run()
man.Stdout = os.Stdout
man.Stdin = &b
man.Stderr = os.Stderr
if err != nil {
log.Fatal(err)
}
},
err := man.Run()
if err != nil {
log.Fatal(err)
}
}
var rootCmd = &cobra.Command{
@@ -57,6 +55,11 @@ var rootCmd = &cobra.Command{
return nil
}
if ShowManual {
man()
return nil
}
err := lib.PrepareColumns()
if err != nil {
return err
@@ -81,9 +84,13 @@ func Execute() {
func init() {
rootCmd.PersistentFlags().BoolVarP(&lib.Debug, "debug", "d", false, "Enable debugging")
rootCmd.PersistentFlags().BoolVarP(&lib.NoNumbering, "no-numbering", "n", false, "Disable header numbering")
rootCmd.PersistentFlags().BoolVarP(&lib.ShowVersion, "version", "v", false, "Print program version")
rootCmd.PersistentFlags().StringVarP(&lib.Separator, "separator", "s", "", "Custom field separator")
rootCmd.PersistentFlags().BoolVarP(&lib.NoColor, "no-color", "N", false, "Disable pattern highlighting")
rootCmd.PersistentFlags().BoolVarP(&lib.ShowVersion, "version", "V", false, "Print program version")
rootCmd.PersistentFlags().BoolVarP(&lib.InvertMatch, "invert-match", "v", false, "select non-matching rows")
rootCmd.PersistentFlags().BoolVarP(&ShowManual, "man", "m", false, "Display manual page")
rootCmd.PersistentFlags().StringVarP(&lib.Separator, "separator", "s", lib.DefaultSeparator, "Custom field separator")
rootCmd.PersistentFlags().StringVarP(&lib.Columns, "columns", "c", "", "Only show the speficied columns (separated by ,)")
rootCmd.PersistentFlags().IntVarP(&lib.SortByColumn, "sort-by", "k", 0, "Sort by column (default: 1)")
// output flags, only 1 allowed, hidden, since just short cuts
rootCmd.PersistentFlags().BoolVarP(&lib.OutflagExtended, "extended", "X", false, "Enable extended output")
@@ -98,11 +105,4 @@ func init() {
// same thing but more common, takes precedence over above group
rootCmd.PersistentFlags().StringVarP(&lib.OutputMode, "output", "o", "", "Output mode - one of: orgtbl, markdown, extended, shell, ascii(default)")
rootCmd.AddCommand(helpCmd)
rootCmd.SetHelpCommand(&cobra.Command{
Use: "no-help",
Hidden: true,
})
}

View File

@@ -11,21 +11,25 @@ SYNOPSIS
-c, --columns string Only show the speficied columns (separated by ,)
-d, --debug Enable debugging
-h, --help help for tablizer
-v, --invert-match select non-matching rows
-m, --man Display manual page
-n, --no-numbering Disable header numbering
-N, --no-color Disable pattern highlighting
-o, --output string Output mode - one of: orgtbl, markdown, extended, ascii(default)
-X, --extended Enable extended output
-M, --markdown Enable markdown table output
-O, --orgtbl Enable org-mode table output
-s, --separator string Custom field separator
-k, --sort-by int Sort by column (default: 1)
-v, --version Print program version
DESCRIPTION
Many programs generate tabular output. But sometimes you need to
post-process these tables, you may need to remove one or more columns or
you may want to filter for some pattern or you may need the output in
another program and need to parse it somehow. Standard unix tools such
as awk(1), grep(1) or column(1) may help, but sometimes it's a tedious
business.
you may want to filter for some pattern (See PATTERNS) or you may need
the output in another program and need to parse it somehow. Standard
unix tools such as awk(1), grep(1) or column(1) may help, but sometimes
it's a tedious business.
Let's take the output of the tool kubectl. It contains cells with
withespace and they do not separate columns by TAB characters. This is
@@ -38,7 +42,8 @@ DESCRIPTION
Without any options it reads its input from "STDIN", but you can also
specify a file as a parameter. If you want to reduce the output by some
regular expression, just specify it as its first parameters. Hence:
regular expression, just specify it as its first parameter. You may also
use the -v option to exclude all rows which match the pattern. Hence:
# read from STDIN
kubectl get pods | tablizer
@@ -67,9 +72,40 @@ DESCRIPTION
The numbering can be suppressed by using the -n option.
By default, if a pattern has been speficied, matches will be
highlighted. You can disable this behavior with the -N option.
Use the -k option to specify by which column to sort the tabular data
(as in GNU sort(1)). The default sort column is the first one. To
disable sorting at all, supply 0 (Zero) to -k.
Finally the -d option enables debugging output which is mostly usefull
for the developer.
PATTERNS
You can reduce the rows being displayed by using a regular expression
pattern. The regexp is PCRE compatible, refer to the syntax cheat sheet
here: <https://github.com/google/re2/wiki/Syntax>. If you want to read a
more comprehensive documentation about the topic and have perl installed
you can read it with:
perldoc perlre
Or read it online: <https://perldoc.perl.org/perlre>.
A note on modifiers: the regexp engine used in tablizer uses another
modifier syntax:
(?MODIFIER)
The most important modifiers are:
"i" ignore case "m" multiline mode "s" single line mode
Example for a case insensitve search:
kubectl get pods -A | tablizer "(?i)account"
OUTPUT MODES
There might be cases when the tabular output of a program is way too
large for your current terminal but you still need to see every column.

4
go.mod
View File

@@ -4,13 +4,15 @@ go 1.18
require (
github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897
github.com/gookit/color v1.5.2
github.com/olekukonko/tablewriter v0.0.5
github.com/spf13/cobra v1.5.0
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778
)
require (
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/mattn/go-runewidth v0.0.9 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stretchr/testify v1.8.0 // indirect
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 // indirect
)

6
go.sum
View File

@@ -4,6 +4,8 @@ github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46t
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/gookit/color v1.5.2 h1:uLnfXcaFjlrDnQDT+NCBcfhrXqYTx/rcCa6xn01Y8yI=
github.com/gookit/color v1.5.2/go.mod h1:w8h4bGiHeeBpvQVePTutdbERIUf3oJE5lZ8HM0UgXyg=
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0=
@@ -22,6 +24,10 @@ github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSS
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 h1:QldyIu/L63oPpyvQmHgvgickp1Yw510KJOqX7H24mg8=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 h1:Bli41pIlzTzf3KEY06n+xnzK/BESIg2ze4Pgfh/aI8c=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -17,29 +17,73 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
package lib
import (
"github.com/gookit/color"
//"github.com/xo/terminfo"
)
var (
// command line flags
Debug bool
XtendedOut bool
NoNumbering bool
ShowVersion bool
Columns string
UseColumns []int
Separator string
OutflagExtended bool
OutflagMarkdown bool
OutflagOrgtable bool
OutflagShell bool
OutputMode string
Debug bool
XtendedOut bool
NoNumbering bool
ShowVersion bool
Columns string
UseColumns []int
DefaultSeparator string = `(\s\s+|\t)`
Separator string = `(\s\s+|\t)`
OutflagExtended bool
OutflagMarkdown bool
OutflagOrgtable bool
OutflagShell bool
OutputMode string
InvertMatch bool
Pattern string
/*
FIXME: make configurable somehow, config file or ENV
see https://github.com/gookit/color will be set by
io.ProcessFiles() according to currently supported
color mode.
*/
MatchFG string
MatchBG string
NoColor bool
// colors to be used per supported color mode
Colors = map[color.Level]map[string]string{
color.Level16: map[string]string{
"bg": "green", "fg": "black",
},
color.Level256: map[string]string{
"bg": "lightGreen", "fg": "black",
},
color.LevelRgb: map[string]string{
// FIXME: maybe use something nicer
"bg": "lightGreen", "fg": "black",
},
}
// used for validation
validOutputmodes = "(orgtbl|markdown|extended|ascii)"
// main program version
Version = "v1.0.4"
Version = "v1.0.8"
// generated version string, used by -v contains lib.Version on
// main branch, and lib.Version-$branch-$lastcommit-$date on
// development branch
VERSION string
// sorting
SortByColumn int
)
// contains a whole parsed table
type Tabdata struct {
maxwidthHeader int // longest header
maxwidthPerCol []int // max width per column
columns int // count
headers []string // [ "ID", "NAME", ...]
entries [][]string
}

View File

@@ -20,6 +20,8 @@ package lib
import (
"errors"
"fmt"
"github.com/gookit/color"
"os"
"regexp"
"strconv"
"strings"
@@ -48,8 +50,62 @@ func PrepareColumns() error {
return nil
}
func numberizeHeaders(data *Tabdata) {
// prepare headers: add numbers to headers
numberedHeaders := []string{}
maxwidth := 0 // start from scratch, so we only look at displayed column widths
for i, head := range data.headers {
headlen := 0
if len(Columns) > 0 {
// -c specified
if !contains(UseColumns, i+1) {
// ignore this one
continue
}
}
if NoNumbering {
numberedHeaders = append(numberedHeaders, head)
headlen = len(head)
} else {
numhead := fmt.Sprintf("%s(%d)", head, i+1)
headlen = len(numhead)
numberedHeaders = append(numberedHeaders, numhead)
}
if headlen > maxwidth {
maxwidth = headlen
}
}
data.headers = numberedHeaders
if data.maxwidthHeader != maxwidth && maxwidth > 0 {
data.maxwidthHeader = maxwidth
}
}
func reduceColumns(data *Tabdata) {
// exclude columns, if any
if len(Columns) > 0 {
reducedEntries := [][]string{}
reducedEntry := []string{}
for _, entry := range data.entries {
reducedEntry = nil
for i, value := range entry {
if !contains(UseColumns, i+1) {
continue
}
reducedEntry = append(reducedEntry, value)
}
reducedEntries = append(reducedEntries, reducedEntry)
}
data.entries = reducedEntries
}
}
func PrepareModeFlags() error {
if len(OutputMode) == 0 {
// associate short flags like -X with mode selector
switch {
case OutflagExtended:
OutputMode = "extended"
@@ -89,3 +145,21 @@ func trimRow(row []string) []string {
return fixedrow
}
func colorizeData(output string) string {
if len(Pattern) > 0 && !NoColor && color.IsConsole(os.Stdout) {
r := regexp.MustCompile("(" + Pattern + ")")
return r.ReplaceAllString(output, "<bg="+MatchBG+";fg="+MatchFG+">$1</>")
} else {
return output
}
}
func isTerminal(f *os.File) bool {
o, _ := f.Stat()
if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice {
return true
} else {
return false
}
}

View File

@@ -52,6 +52,7 @@ func TestPrepareColumns(t *testing.T) {
}{
{"1,2,3", []int{1, 2, 3}, false},
{"1,2,", []int{}, true},
{"a,b", []int{}, true},
}
for _, tt := range tests {
@@ -71,3 +72,46 @@ func TestPrepareColumns(t *testing.T) {
})
}
}
func TestReduceColumns(t *testing.T) {
var tests = []struct {
expect [][]string
columns []int
}{
{
expect: [][]string{[]string{"a", "b"}},
columns: []int{1, 2},
},
{
expect: [][]string{[]string{"a", "c"}},
columns: []int{1, 3},
},
{
expect: [][]string{[]string{"a"}},
columns: []int{1},
},
{
expect: [][]string{nil},
columns: []int{4},
},
}
input := [][]string{[]string{"a", "b", "c"}}
Columns = "y" // used as a flag with len(Columns)...
for _, tt := range tests {
testname := fmt.Sprintf("reduce-columns-by-%+v", tt.columns)
t.Run(testname, func(t *testing.T) {
UseColumns = tt.columns
data := Tabdata{entries: input}
reduceColumns(&data)
if !reflect.DeepEqual(data.entries, tt.expect) {
t.Errorf("reduceColumns returned invalid data:\ngot: %+v\nexp: %+v", data.entries, tt.expect)
}
})
}
Columns = "" // reset for other tests
UseColumns = nil
}

View File

@@ -19,6 +19,7 @@ package lib
import (
"errors"
"github.com/gookit/color"
"io"
"os"
)
@@ -26,6 +27,14 @@ import (
func ProcessFiles(args []string) error {
fds, pattern, err := determineIO(args)
if !isTerminal(os.Stdout) {
color.Disable()
} else {
level := color.TermColorLevel()
MatchFG = Colors[level]["fg"]
MatchBG = Colors[level]["bg"]
}
if err != nil {
return err
}
@@ -52,6 +61,7 @@ func determineIO(args []string) ([]io.Reader, string, error) {
// first one is not a file, consider it as regexp and
// shift arg list
pattern = args[0]
Pattern = args[0] // FIXME
args = args[1:]
}

View File

@@ -27,51 +27,29 @@ import (
"strings"
)
// contains a whole parsed table
type Tabdata struct {
maxwidthHeader int // longest header
maxwidthPerCol []int // max width per column
columns int
headerIndices []map[string]int // [ {beg=>0, end=>17}, ... ]
headers []string // [ "ID", "NAME", ...]
entries [][]string
}
/*
Parse tabular input. We split the header (first line) by 2 or more
spaces, remember the positions of the header fields. We then split
the data (everything after the first line) by those positions. That
way we can turn "tabular data" (with fields containing whitespaces)
into real tabular data. We re-tabulate our input if you will.
Parse tabular input.
*/
func parseFile(input io.Reader, pattern string) (Tabdata, error) {
data := Tabdata{}
var scanner *bufio.Scanner
var spaces = `\s\s+|$`
if len(Separator) > 0 {
spaces = Separator
}
hadFirst := false
spacefinder := regexp.MustCompile(spaces)
beg := 0
separate := regexp.MustCompile(Separator)
patternR, err := regexp.Compile(pattern)
if err != nil {
return data, errors.Unwrap(fmt.Errorf("Regexp pattern %s is invalid: %w", pattern, err))
}
scanner = bufio.NewScanner(input)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
values := []string{}
patternR, err := regexp.Compile(pattern)
if err != nil {
return data, errors.Unwrap(fmt.Errorf("Regexp pattern %s is invalid: %w", pattern, err))
}
parts := separate.Split(line, -1)
if !hadFirst {
// header processing
parts := spacefinder.FindAllStringIndex(line, -1)
data.columns = len(parts)
// if Debug {
// fmt.Println(parts)
@@ -83,30 +61,14 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
// fmt.Printf("Part: <%s>\n", string(line[beg:part[0]]))
//}
// current field
head := string(line[beg:part[0]])
// register begin and end of field within line
indices := make(map[string]int)
indices["beg"] = beg
if part[0] == part[1] {
indices["end"] = 0
} else {
indices["end"] = part[1] - 1
}
// register widest header field
headerlen := len(head)
headerlen := len(part)
if headerlen > data.maxwidthHeader {
data.maxwidthHeader = headerlen
}
// register fields data
data.headerIndices = append(data.headerIndices, indices)
data.headers = append(data.headers, head)
// end of current field == begin of next one
beg = part[1]
data.headers = append(data.headers, strings.TrimSpace(part))
// done
hadFirst = true
@@ -114,22 +76,19 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
} else {
// data processing
if len(pattern) > 0 {
if !patternR.MatchString(line) {
if patternR.MatchString(line) == InvertMatch {
// by default -v is false, so if a line does NOT
// match the pattern, we will ignore it. However,
// if the user specified -v, the matching is inverted,
// so we ignore all lines, which DO match.
continue
}
}
idx := 0 // we cannot use the header index, because we could exclude columns
for _, index := range data.headerIndices {
value := ""
if index["end"] == 0 {
value = string(line[index["beg"]:])
} else {
value = string(line[index["beg"]:index["end"]])
}
width := len(strings.TrimSpace(value))
values := []string{}
for _, part := range parts {
width := len(strings.TrimSpace(part))
if len(data.maxwidthPerCol)-1 < idx {
data.maxwidthPerCol = append(data.maxwidthPerCol, width)
@@ -142,7 +101,7 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
// if Debug {
// fmt.Printf("<%s> ", value)
// }
values = append(values, strings.TrimSpace(value))
values = append(values, strings.TrimSpace(part))
idx++
}
@@ -151,7 +110,7 @@ func parseFile(input io.Reader, pattern string) (Tabdata, error) {
}
if scanner.Err() != nil {
return data, errors.Unwrap(fmt.Errorf("Regexp pattern %s is invalid: %w", pattern, scanner.Err()))
return data, errors.Unwrap(fmt.Errorf("Failed to read from io.Reader: %w", scanner.Err()))
}
if Debug {

View File

@@ -18,6 +18,7 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
package lib
import (
"fmt"
"reflect"
"strings"
"testing"
@@ -27,40 +28,18 @@ func TestParser(t *testing.T) {
data := Tabdata{
maxwidthHeader: 5,
maxwidthPerCol: []int{
5,
5,
8,
5, 5, 8,
},
columns: 3,
headerIndices: []map[string]int{
map[string]int{
"beg": 0,
"end": 6,
},
map[string]int{
"end": 13,
"beg": 7,
},
map[string]int{
"beg": 14,
"end": 0,
},
},
headers: []string{
"ONE",
"TWO",
"THREE",
"ONE", "TWO", "THREE",
},
entries: [][]string{
[]string{
"asd",
"igig",
"cxxxncnc",
"asd", "igig", "cxxxncnc",
},
[]string{
"19191",
"EDD 1",
"X",
"19191", "EDD 1", "X",
},
},
}
@@ -71,12 +50,63 @@ asd igig cxxxncnc
readFd := strings.NewReader(table)
gotdata, err := parseFile(readFd, "")
Separator = DefaultSeparator
if err != nil {
t.Errorf("Parser returned error: %s\nData processed so far: %+v", err, gotdata)
}
if !reflect.DeepEqual(data, gotdata) {
t.Errorf("Parser returned invalid data\nExp: %+v\nGot: %+v\n", data, gotdata)
t.Errorf("Parser returned invalid data, Regex: %s\nExp: %+v\nGot: %+v\n", Separator, data, gotdata)
}
}
func TestParserPatternmatching(t *testing.T) {
var tests = []struct {
entries [][]string
pattern string
invert bool
}{
{
entries: [][]string{
[]string{
"asd", "igig", "cxxxncnc",
},
},
pattern: "ig",
invert: false,
},
{
entries: [][]string{
[]string{
"19191", "EDD 1", "X",
},
},
pattern: "ig",
invert: true,
},
}
table := `ONE TWO THREE
asd igig cxxxncnc
19191 EDD 1 X`
for _, tt := range tests {
testname := fmt.Sprintf("parse-with-inverted-pattern-%t", tt.invert)
t.Run(testname, func(t *testing.T) {
InvertMatch = tt.invert
readFd := strings.NewReader(table)
gotdata, err := parseFile(readFd, tt.pattern)
if err != nil {
t.Errorf("Parser returned error: %s\nData processed so far: %+v", err, gotdata)
}
if !reflect.DeepEqual(tt.entries, gotdata.entries) {
t.Errorf("Parser returned invalid data (pattern: %s, invert: %t)\nExp: %+v\nGot: %+v\n",
tt.pattern, tt.invert, tt.entries, gotdata.entries)
}
})
}
}

View File

@@ -19,49 +19,26 @@ package lib
import (
"fmt"
"github.com/gookit/color"
"github.com/olekukonko/tablewriter"
"os"
"regexp"
"strings"
)
func printData(data *Tabdata) {
// prepare headers: add numbers to headers
numberedHeaders := []string{}
for i, head := range data.headers {
if len(Columns) > 0 {
// -c specified
if !contains(UseColumns, i+1) {
// ignore this one
continue
}
}
if NoNumbering {
numberedHeaders = append(numberedHeaders, head)
} else {
numberedHeaders = append(numberedHeaders, fmt.Sprintf("%s(%d)", head, i+1))
}
}
data.headers = numberedHeaders
// some output preparations:
// prepare data
if len(Columns) > 0 {
reducedEntries := [][]string{}
reducedEntry := []string{}
for _, entry := range data.entries {
reducedEntry = nil
for i, value := range entry {
if !contains(UseColumns, i+1) {
continue
}
reducedEntry = append(reducedEntry, value)
}
reducedEntries = append(reducedEntries, reducedEntry)
}
data.entries = reducedEntries
if OutputMode != "shell" {
// not needed in eval string
numberizeHeaders(data)
}
// remove unwanted columns, if any
reduceColumns(data)
// sort the data
sortTable(data, SortByColumn)
switch OutputMode {
case "extended":
printExtendedData(data)
@@ -107,14 +84,18 @@ func printOrgmodeData(data *Tabdata) {
leftR := regexp.MustCompile("(?m)^\\+")
rightR := regexp.MustCompile("\\+(?m)$")
fmt.Print(rightR.ReplaceAllString(leftR.ReplaceAllString(tableString.String(), "|"), "|"))
color.Print(
colorizeData(
rightR.ReplaceAllString(
leftR.ReplaceAllString(tableString.String(), "|"), "|")))
}
/*
Markdown table
*/
func printMarkdownData(data *Tabdata) {
table := tablewriter.NewWriter(os.Stdout)
tableString := &strings.Builder{}
table := tablewriter.NewWriter(tableString)
table.SetHeader(data.headers)
@@ -126,13 +107,15 @@ func printMarkdownData(data *Tabdata) {
table.SetCenterSeparator("|")
table.Render()
color.Print(colorizeData(tableString.String()))
}
/*
Simple ASCII table without any borders etc, just like the input we expect
*/
func printAsciiData(data *Tabdata) {
table := tablewriter.NewWriter(os.Stdout)
tableString := &strings.Builder{}
table := tablewriter.NewWriter(tableString)
table.SetHeader(data.headers)
table.AppendBulk(data.entries)
@@ -154,6 +137,7 @@ func printAsciiData(data *Tabdata) {
table.SetNoWhiteSpace(true)
table.Render()
color.Print(colorizeData(tableString.String()))
}
/*
@@ -161,22 +145,14 @@ func printAsciiData(data *Tabdata) {
*/
func printExtendedData(data *Tabdata) {
// needed for data output
format := fmt.Sprintf("%%%ds: %%s\n", data.maxwidthHeader) // FIXME: re-calculate if -c has been set
format := fmt.Sprintf("%%%ds: %%s\n", data.maxwidthHeader)
if len(data.entries) > 0 {
var idx int
for _, entry := range data.entries {
idx = 0
for i, value := range entry {
if len(Columns) > 0 {
if !contains(UseColumns, i+1) {
continue
}
}
fmt.Printf(format, data.headers[idx], value)
idx++
color.Printf(format, data.headers[i], value)
}
fmt.Println()
}
}
@@ -190,6 +166,7 @@ func printShellData(data *Tabdata) {
var idx int
for _, entry := range data.entries {
idx = 0
shentries := []string{}
for i, value := range entry {
if len(Columns) > 0 {
if !contains(UseColumns, i+1) {
@@ -197,10 +174,10 @@ func printShellData(data *Tabdata) {
}
}
fmt.Printf("%s=\"%s\" ", data.headers[idx], value)
shentries = append(shentries, fmt.Sprintf("%s=\"%s\"", data.headers[idx], value))
idx++
}
fmt.Println()
fmt.Println(strings.Join(shentries, " "))
}
}
}

View File

@@ -18,64 +18,195 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
package lib
import (
"fmt"
"github.com/gookit/color"
"os"
"strings"
"testing"
)
func stdout2pipe(t *testing.T) (*os.File, *os.File) {
reader, writer, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
origStdout := os.Stdout
os.Stdout = writer
// we need to tell the color mode the io.Writer, even if we don't usw colorization
color.SetOutput(writer)
return origStdout, reader
}
func TestPrinter(t *testing.T) {
table := `ONE TWO THREE
asd igig cxxxncnc
19191 EDD 1 X`
startdata := Tabdata{
maxwidthHeader: 5,
maxwidthPerCol: []int{
5,
5,
8,
},
columns: 3,
headers: []string{
"ONE", "TWO", "THREE",
},
entries: [][]string{
[]string{
"asd", "igig", "cxxxncnc",
},
[]string{
"19191", "EDD 1", "X",
},
},
}
expects := map[string]string{
"ascii": `ONE(1) TWO(2) THREE(3)
asd igig cxxxncnc
19191 EDD 1 X`,
"orgtbl": `|--------+--------+----------|
| ONE(1) | TWO(2) | THREE(3) |
|--------+--------+----------|
| asd | igig | cxxxncnc |
| 19191 | EDD 1 | X |
|--------+--------+----------|`,
"markdown": `| ONE(1) | TWO(2) | THREE(3) |
|--------|--------|----------|
| asd | igig | cxxxncnc |
| 19191 | EDD 1 | X |`,
"shell": `ONE="asd" TWO="igig" THREE="cxxxncnc"
ONE="19191" TWO="EDD 1" THREE="X"`,
"extended": `ONE(1): asd
TWO(2): igig
THREE(3): cxxxncnc
ONE(1): 19191
TWO(2): EDD 1
THREE(3): X`,
}
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
origStdout := os.Stdout
os.Stdout = w
NoColor = true
SortByColumn = 0 // disable sorting
origStdout, reader := stdout2pipe(t)
for mode, expect := range expects {
OutputMode = mode
fd := strings.NewReader(table)
data, err := parseFile(fd, "")
testname := fmt.Sprintf("print-%s", mode)
t.Run(testname, func(t *testing.T) {
if err != nil {
t.Errorf("Parser returned error: %s\nData processed so far: %+v", err, data)
}
OutputMode = mode
data := startdata // we need to reset our mock data, since it's being modified in printData()
printData(&data)
printData(&data)
buf := make([]byte, 1024)
n, err := reader.Read(buf)
if err != nil {
t.Fatal(err)
}
buf = buf[:n]
output := strings.TrimSpace(string(buf))
buf := make([]byte, 1024)
n, err := r.Read(buf)
if err != nil {
t.Fatal(err)
}
buf = buf[:n]
output := strings.TrimSpace(string(buf))
if output != expect {
t.Errorf("output mode: %s, got:\n%s\nwant:\n%s\n (%d <=> %d)", mode, output, expect, len(output), len(expect))
}
if output != expect {
t.Errorf("output mode: %s, got:\n%s\nwant:\n%s\n (%d <=> %d)",
mode, output, expect, len(output), len(expect))
}
})
}
// Restore
os.Stdout = origStdout
}
func TestSortPrinter(t *testing.T) {
startdata := Tabdata{
maxwidthHeader: 5,
maxwidthPerCol: []int{
3,
3,
2,
},
columns: 3,
headers: []string{
"ONE", "TWO", "THREE",
},
entries: [][]string{
[]string{
"abc", "345", "b1",
},
[]string{
"bcd", "234", "a2",
},
[]string{
"cde", "123", "c3",
},
},
}
var tests = []struct {
data Tabdata
sortby int
expect string
}{
{
data: startdata,
sortby: 1,
expect: `ONE(1) TWO(2) THREE(3)
abc 345 b1
bcd 234 a2
cde 123 c3`,
},
{
data: startdata,
sortby: 2,
expect: `ONE(1) TWO(2) THREE(3)
cde 123 c3
bcd 234 a2
abc 345 b1`,
},
{
data: startdata,
sortby: 3,
expect: `ONE(1) TWO(2) THREE(3)
bcd 234 a2
abc 345 b1
cde 123 c3`,
},
}
NoColor = true
OutputMode = "ascii"
origStdout, reader := stdout2pipe(t)
for _, tt := range tests {
testname := fmt.Sprintf("print-sorted-table-%d", tt.sortby)
t.Run(testname, func(t *testing.T) {
SortByColumn = tt.sortby
printData(&tt.data)
buf := make([]byte, 1024)
n, err := reader.Read(buf)
if err != nil {
t.Fatal(err)
}
buf = buf[:n]
output := strings.TrimSpace(string(buf))
if output != tt.expect {
t.Errorf("sort column: %d, got:\n%s\nwant:\n%s",
tt.sortby, output, tt.expect)
}
})
}
// Restore
os.Stdout = origStdout
}

46
lib/sort.go Normal file
View File

@@ -0,0 +1,46 @@
/*
Copyright © 2022 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package lib
import (
"sort"
)
func sortTable(data *Tabdata, col int) {
if col <= 0 {
// no sorting wanted
return
}
col-- // ui starts counting by 1, but use 0 internally
// sanity checks
if len(data.entries) == 0 {
return
}
if col >= len(data.headers) {
// fall back to default column
col = 0
}
// actual sorting
sort.SliceStable(data.entries, func(i, j int) bool {
return data.entries[i][col] < data.entries[j][col]
})
}

View File

@@ -43,7 +43,7 @@ for D in $DIST; do
tardir="${tool}-${os}-${arch}-${version}"
tarfile="releases/${tool}-${os}-${arch}-${version}.tar.gz"
set -x
GOOS=${os} GOARCH=${arch} go build -o ${binfile}
GOOS=${os} GOARCH=${arch} go build -o ${binfile} -ldflags "-X 'github.com/tlinden/tablizer/lib.VERSION=${version}'"
mkdir -p ${tardir}
cp ${binfile} README.md LICENSE ${tardir}/
echo 'tool = tablizer

View File

@@ -133,7 +133,7 @@
.\" ========================================================================
.\"
.IX Title "TABLIZER 1"
.TH TABLIZER 1 "2022-10-04" "1" "User Commands"
.TH TABLIZER 1 "2022-10-13" "1" "User Commands"
.\" For nroff, turn off justification. Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
@@ -150,22 +150,26 @@ tablizer \- Manipulate tabular output of other programs
\& \-c, \-\-columns string Only show the speficied columns (separated by ,)
\& \-d, \-\-debug Enable debugging
\& \-h, \-\-help help for tablizer
\& \-v, \-\-invert\-match select non\-matching rows
\& \-m, \-\-man Display manual page
\& \-n, \-\-no\-numbering Disable header numbering
\& \-N, \-\-no\-color Disable pattern highlighting
\& \-o, \-\-output string Output mode \- one of: orgtbl, markdown, extended, ascii(default)
\& \-X, \-\-extended Enable extended output
\& \-M, \-\-markdown Enable markdown table output
\& \-O, \-\-orgtbl Enable org\-mode table output
\& \-s, \-\-separator string Custom field separator
\& \-k, \-\-sort\-by int Sort by column (default: 1)
\& \-v, \-\-version Print program version
.Ve
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
Many programs generate tabular output. But sometimes you need to
Many programs generate tabular output. But sometimes you need to
post-process these tables, you may need to remove one or more columns
or you may want to filter for some pattern or you may need the output
in another program and need to parse it somehow. Standard unix tools
such as \fBawk\fR\|(1), \fBgrep\fR\|(1) or \fBcolumn\fR\|(1) may help, but sometimes it's a
tedious business.
or you may want to filter for some pattern (See \s-1PATTERNS\s0) or you
may need the output in another program and need to parse it somehow.
Standard unix tools such as \fBawk\fR\|(1), \fBgrep\fR\|(1) or \fBcolumn\fR\|(1) may help, but
sometimes it's a tedious business.
.PP
Let's take the output of the tool kubectl. It contains cells with
withespace and they do not separate columns by \s-1TAB\s0 characters. This is
@@ -179,8 +183,9 @@ positions.
.PP
Without any options it reads its input from \f(CW\*(C`STDIN\*(C'\fR, but you can also
specify a file as a parameter. If you want to reduce the output by
some regular expression, just specify it as its first
parameters. Hence:
some regular expression, just specify it as its first parameter. You
may also use the \fB\-v\fR option to exclude all rows which match the
pattern. Hence:
.PP
.Vb 2
\& # read from STDIN
@@ -215,8 +220,47 @@ the original order.
.PP
The numbering can be suppressed by using the \fB\-n\fR option.
.PP
By default, if a \fBpattern\fR has been speficied, matches will be
highlighted. You can disable this behavior with the \fB\-N\fR option.
.PP
Use the \fB\-k\fR option to specify by which column to sort the tabular
data (as in \s-1GNU\s0 \fBsort\fR\|(1)). The default sort column is the first one. To
disable sorting at all, supply 0 (Zero) to \-k.
.PP
Finally the \fB\-d\fR option enables debugging output which is mostly
usefull for the developer.
.SS "\s-1PATTERNS\s0"
.IX Subsection "PATTERNS"
You can reduce the rows being displayed by using a regular expression
pattern. The regexp is \s-1PCRE\s0 compatible, refer to the syntax cheat
sheet here: <https://github.com/google/re2/wiki/Syntax>. If you want
to read a more comprehensive documentation about the topic and have
perl installed you can read it with:
.PP
.Vb 1
\& perldoc perlre
.Ve
.PP
Or read it online: <https://perldoc.perl.org/perlre>.
.PP
A note on modifiers: the regexp engine used in tablizer uses another
modifier syntax:
.PP
.Vb 1
\& (?MODIFIER)
.Ve
.PP
The most important modifiers are:
.PP
\&\f(CW\*(C`i\*(C'\fR ignore case
\&\f(CW\*(C`m\*(C'\fR multiline mode
\&\f(CW\*(C`s\*(C'\fR single line mode
.PP
Example for a case insensitve search:
.PP
.Vb 1
\& kubectl get pods \-A | tablizer "(?i)account"
.Ve
.SS "\s-1OUTPUT MODES\s0"
.IX Subsection "OUTPUT MODES"
There might be cases when the tabular output of a program is way too

View File

@@ -11,23 +11,27 @@ tablizer - Manipulate tabular output of other programs
-c, --columns string Only show the speficied columns (separated by ,)
-d, --debug Enable debugging
-h, --help help for tablizer
-v, --invert-match select non-matching rows
-m, --man Display manual page
-n, --no-numbering Disable header numbering
-N, --no-color Disable pattern highlighting
-o, --output string Output mode - one of: orgtbl, markdown, extended, ascii(default)
-X, --extended Enable extended output
-M, --markdown Enable markdown table output
-O, --orgtbl Enable org-mode table output
-s, --separator string Custom field separator
-k, --sort-by int Sort by column (default: 1)
-v, --version Print program version
=head1 DESCRIPTION
Many programs generate tabular output. But sometimes you need to
Many programs generate tabular output. But sometimes you need to
post-process these tables, you may need to remove one or more columns
or you may want to filter for some pattern or you may need the output
in another program and need to parse it somehow. Standard unix tools
such as awk(1), grep(1) or column(1) may help, but sometimes it's a
tedious business.
or you may want to filter for some pattern (See L<PATTERNS>) or you
may need the output in another program and need to parse it somehow.
Standard unix tools such as awk(1), grep(1) or column(1) may help, but
sometimes it's a tedious business.
Let's take the output of the tool kubectl. It contains cells with
withespace and they do not separate columns by TAB characters. This is
@@ -41,8 +45,9 @@ positions.
Without any options it reads its input from C<STDIN>, but you can also
specify a file as a parameter. If you want to reduce the output by
some regular expression, just specify it as its first
parameters. Hence:
some regular expression, just specify it as its first parameter. You
may also use the B<-v> option to exclude all rows which match the
pattern. Hence:
# read from STDIN
kubectl get pods | tablizer
@@ -71,9 +76,44 @@ the original order.
The numbering can be suppressed by using the B<-n> option.
By default, if a B<pattern> has been speficied, matches will be
highlighted. You can disable this behavior with the B<-N> option.
Use the B<-k> option to specify by which column to sort the tabular
data (as in GNU sort(1)). The default sort column is the first one. To
disable sorting at all, supply 0 (Zero) to -k.
Finally the B<-d> option enables debugging output which is mostly
usefull for the developer.
=head2 PATTERNS
You can reduce the rows being displayed by using a regular expression
pattern. The regexp is PCRE compatible, refer to the syntax cheat
sheet here: L<https://github.com/google/re2/wiki/Syntax>. If you want
to read a more comprehensive documentation about the topic and have
perl installed you can read it with:
perldoc perlre
Or read it online: L<https://perldoc.perl.org/perlre>.
A note on modifiers: the regexp engine used in tablizer uses another
modifier syntax:
(?MODIFIER)
The most important modifiers are:
C<i> ignore case
C<m> multiline mode
C<s> single line mode
Example for a case insensitve search:
kubectl get pods -A | tablizer "(?i)account"
=head2 OUTPUT MODES
There might be cases when the tabular output of a program is way too