Compare commits

...

13 Commits

21 changed files with 800 additions and 89 deletions

159
CHANGELOG.md Normal file
View File

@@ -0,0 +1,159 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org).
## [v1.0.9](https://github.com/TLINDEN/tablizer/tree/v1.0.9) - 2022-10-14
[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.8...v1.0.9)
### Added
- Added Changelog, Contribution guidelines and no COC.
### Changed
- some minor changes to satisfy linter.
## [v1.0.8](https://github.com/TLINDEN/tablizer/tree/v1.0.8) - 2022-10-13
[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.7...v1.0.8)
### Added
- Added sort support with the new parameter -k (like sort(1).
## [v1.0.7](https://github.com/TLINDEN/tablizer/tree/v1.0.7) - 2022-10-11
[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.6...v1.0.7)
### Added
- Added pattern highlighting support.
- Added more unit tests.
### Fixed
- Fixed extended more output in combination with -c.
- Fixed issue #4, the version string was missing.
## [v1.0.6](https://github.com/TLINDEN/tablizer/tree/v1.0.6) - 2022-10-05
[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.5...v1.0.6)
### Added
- Added documentation about regexp syntax in the manpage.
- Added more unit tests.
### Changed
- Rewrote the input parser.
- Some more refactoring work has been done.
## [v1.0.5](https://github.com/TLINDEN/tablizer/tree/v1.0.5) - 2022-10-05
[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.4...v1.0.5)
### Added
- A new option has been added: --invert-match -v which behaves like
the same option in grep(1): it inverts the pattern match.
- A few more unit tests have been added.
### Fixed
- Pattern matching did not work, because the (new) help subcommand
lead to cobra taking care of the first arg to the program
(argv[1]). So now there's a new parameter -m which displays the
manpage and no more subcommands.
## [v1.0.4](https://github.com/TLINDEN/tablizer/tree/v1.0.4) - 2022-10-04
[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.3...v1.0.4)
### Added
- Development version of the compiled binary now uses git vars
in addition to program version.
- Added an option to display the manual page (compiled in) as text:
--help, for cases where a user just installed the binary.
### Changed
- Fixed go module namespace.
## [v1.0.3](https://github.com/TLINDEN/tablizer/tree/v1.0.3) - 2022-10-03
[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.2...v1.0.3)
### Added
- Added a new output mode: shell mode, which allows the user
to use the output in a shell eval loop to further process
the data.
### Changed
- More refactoring work has been done.
## [v1.0.2](https://github.com/TLINDEN/tablizer/tree/v1.0.2) - 2022-10-02
[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.1...v1.0.2)
### Added
- Added some basic unit tests.
### Changed
- Code has been refactored to be more efficient.
- Replaced table generation code with Tablewriter.
## [v1.0.1](https://github.com/TLINDEN/tablizer/tree/v1.0.1) - 2022-09-30
[Full Changelog](https://github.com/TLINDEN/tablizer/compare/v1.0.0...v1.0.1)
### Added
- Added a unix manual page.
- Added release builder to Makefile
### Changed
- Various minor fixes.
## [v1.0.0](https://github.com/TLINDEN/tablizer/tree/v1.0.0) - 2022-09-28
[Full Changelog](https://github.com/TLINDEN/tablizer/compare/02a64a5c3fe4220df2c791ff1421d16ebd428c19...v1.0.0)
Initial release.

111
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,111 @@
# No Code of Conduct
This project does **NOT** have a so called Code of Conduct, nor will
it ever get one.
The reasons are somewhat complicated and I'll try my best to document
them here.
Ethical codes or rules come along like laws. But how is ethical or
moral behavior defined? And who defines which behavior is ethical and
which is not? Certainly not me.
Unless you live in a dictatorship (and more than half of the
population of planet earth do as of this writing), laws come into
existence by democratic procedures. Laws cover almost every aspect of
live in a society. Laws allow and forbid behavior and laws sanction
infringements.
A software project like this one on the other hand is not a society.
There are not enough people involved to form democratic
structures. And there will always be a minority of users who have the
right to commit or reject code. How could any maintainer of a software
project dare to decree rules upon others? Actually, am I, the current
maintainer of this very project authorized to do so?
I think the anser to this question clearly is NO.
The issue is being complicated by the fact, that open source
development these days happens on planetary scale. And this planet
houses hundreds if not thousands of different cultures, philosophies,
ideologies and worldviews. The answer to many ethical questions will
in most cases vague and nebulous.
Ones joke will always be another ones insult.
Then there is the problem of language. I myself am not an english
native, but I publish everyting using the english language. I am able
to communicate with most people in the open source community because
of that. But I am certainly not able to understand everything and
everyone. There might be nuances to a sentence I don't sense, there
might be sarcastic connotations I don't understand or references to
historical figures, events or traditions I don't know and never have
heard of.
Juding over other peoples online behavior looks like a titanic task to
me. It is just not my job to judge others. I am not legitimized or
authorized to do so.
Another huge problem with ethical rules is that you need to outline
and enforce sanctions on thos who violate the rules. But since I am
not an elected authority how would I be able to do this? I don't
know. And what happens if someone complains about myself? Shall I
remove myself from my own project? Come on!
Last but not least there's the law. I am a german citizen and am
living in relatively freedom. Unlike many other people living in
democracies these days, I myself fought for this very freedom on the
streets of Leipzig in 1989. I saw the tanks, the Stasi officers, I
felt the fear. But the laws under I live today and which I have to
adhere to, are only limited to the small speck on earth I am living on.
So, let's say someone in india says something insulting to some other
developer in an issue. Of course german law does not apply to indian
people. More, the insult might actually not be an insult in india. In
the end, nothing would happen. Under normal circumstances, maintainers
would delete the posting, ban the user or remove push privileges etc.
But then, is there a way for the offending user to defend himself? Of
course not, since neither indian or german law alone applies. I cannot
go to a german court and sue the guy and he cannot do the same in
india. Or - we possibly could but the judges on both countries would
just laugh and close the case.
And let's not even start talking about there undemocratic "comitees"
many projects are forming to circumvent this problem.
That being said, I don't have the power nor the tools, nor the
authority to enforce serious sanctions of any meaningful kind against
others. Therefore I cannot outline any rules whatsoever.
## So, which are the ethical rules within this project then?
Well, there are none.
This project is about code, not society. It doesn't matter where you
come from, how you look, how you think, what you believe, who your
friends are, whay you said or did sometime in the past. I don't even
care if you are a human being. You are an alien so bored that you need
to submit code on github? Fine with me.
**The only thing I am interested here is Code and only Code.**
So if anyhing happens here I don't like or I am obliged by law to act
on, I will decide on a case to case basis what to do. And
unfortunately, since this is the nature of a github project, you
cannot complain, object or protest. I am very sorry!
If you will, let's at least outline these:
- Please - just please - behave towards others as you'd expect others
to behave towards yourself.
- Don't judge others for any reason.
- Only judge the code.
But these are not rules, only a friendly appeal to you as a developer
and user.
Thanks a lot!

94
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,94 @@
## Project Goals
The goal of this project is to build a small tool which helps in day
to day work with tabular output of various commandline programs. It
should be small, fast and easy to understand. The idea is to replace
multiline shell pipes using awk, sed and grep with just one
binary.
There will be no GUI, no web interface, no public API of some sort, no
builtin interpreter.
The programming language used for this project will always be
[GOLANG](https://go.dev/) with the exception of the documentation
([Perl POD](https://perldoc.perl.org/perlpod)) and the Makefile.
# Contributing
You can contribute to this project in various ways:
## Open an issue
If you encounter a problem or don't understand how the program works
or if you think the documentation is unclear, please don't hesitate to
open an issue.
Please add as much information about the case as possible, such as:
- Your environment (operating system etc)
- tablizer version (`tablizer --version`)
- Input data. Please replace sensitive information with mock data!
- Actual program output.
- Expected program output.
- Error message - if any.
Be aware that I am working on this (and some other) project in my
spare time which is scarce. Therefore please don't expect me to
respond to your query within hours or even days. Be patient, but I
WILL respond.
## Pull Requests
Code and documentation help is always much appreciated! Please follow
thes guidelines to successfully contribute:
- Every pull request shall be based on latest `development`
branch. `main` is only used for releases.
- Execute the unit tests before committing: `make test`. There shall
be no errors.
- Strive to be backwards compatible so that users who are already
using the program don't have to change their habits - unless it is
really neccessary.
- Try to add a unit test for your addition.
- Don't ever change existing unit tests!
- Add a meaningful and comprehensive rationale about your contribution:
- Why do you think it might be useful for others?
- What did you actually change or add?
- Is there an open issue which this PR fixes and if so, please link
to that issue.
- [Re-]format your code with `gofmt -s`.
- Avoid unneccesary dependencies, especially for very small functions.
- **If** a new dependency is being added, it must be compatible with
our [license agreement](LICENSE).
- You need to accept that the code or documentation you contribute
will be redistributed under the terms of said license agreement. If
your contribution is considerably large or if you contribute
regularly, then feel free to add your name and if you want your
email address to the *AUTHORS* section of the
[manpage](tablizer.pod).
- Adhere to the above mentioned project goals.
- If you are unsure if your addition or change will be accepted,
better ask before starting coding. Open an issue about your proposal
and let's discuss it! That way we avoid doing unnessesary work on
both sides.
Each pull request will be carefully reviewed and if it is a useful
addition it will be accepted. However, please be prepared that
sometimes a PR will be rejected. The reasons may vary and will be
documented. Perhaps the above guidelines are not matched, or the
addition seems to be not so useful from my perspective, maybe there
are too much changes or there might be changes I don't even
understand.
But whatever happens: your contribution is always welcome!

View File

@@ -26,7 +26,7 @@ GID = 0
BRANCH = $(shell git describe --all | cut -d/ -f2)
COMMIT = $(shell git rev-parse --short=8 HEAD)
BUILD = $(shell date +%Y.%m.%d.%H%M%S)
VERSION:= $(if $(filter $(BRANCH), development),$(version)-$(BRANCH)-$(COMMIT)-$(BUILD))
VERSION:= $(if $(filter $(BRANCH), development),$(version)-$(BRANCH)-$(COMMIT)-$(BUILD),$(version))
all: $(tool).1 cmd/$(tool).go buildlocal
@@ -36,6 +36,7 @@ all: $(tool).1 cmd/$(tool).go buildlocal
cmd/%.go: %.pod
echo "package cmd" > cmd/$*.go
echo >> cmd/$*.go
echo "var manpage = \`" >> cmd/$*.go
pod2text $*.pod >> cmd/$*.go
echo "\`" >> cmd/$*.go
@@ -45,6 +46,7 @@ buildlocal:
release:
./mkrel.sh $(tool) $(version)
gh release create $(version) --generate-notes releases/*
install: buildlocal
install -d -o $(UID) -g $(GID) $(PREFIX)/bin

View File

@@ -1,5 +1,6 @@
[![Actions](https://github.com/tlinden/tablizer/actions/workflows/ci.yaml/badge.svg)](https://github.com/tlinden/tablizer/actions)
[![License](https://img.shields.io/badge/license-GPL-blue.svg)](https://github.com/tlinden/tablizer/blob/master/LICENSE)
[![Go Report Card](https://goreportcard.com/badge/github.com/tlinden/tablizer)](https://goreportcard.com/report/github.com/tlinden/tablizer)
## tablizer - Manipulate tabular output of other programs
@@ -20,13 +21,13 @@ But you're only interested in the NAME and STATUS columns. Here's how
to do this with tablizer:
```
% kubectl get pods | ./tablizer
% kubectl get pods | tablizer
NAME(1) READY(2) STATUS(3) RESTARTS(4) AGE(5)
repldepl-7bcd8d5b64-7zq4l 1/1 Running 1 (69m ago) 5h26m
repldepl-7bcd8d5b64-m48n8 1/1 Running 1 (69m ago) 5h26m
repldepl-7bcd8d5b64-q2bf4 1/1 Running 1 (69m ago) 5h26m
% kubectl get pods | ./tablizer -c 1,3
% kubectl get pods | tablizer -c 1,3
NAME(1) STATUS(3)
repldepl-7bcd8d5b64-7zq4l Running
repldepl-7bcd8d5b64-m48n8 Running
@@ -34,27 +35,28 @@ repldepl-7bcd8d5b64-q2bf4 Running
```
Another use case is when the tabular output is so wide that lines are
being broken and the whole output is completely distorted. In such a
case you can use the `-x` flag to get an output similar to `\x` in `psql`:
being broken and the whole output is completely distorted. In such a
case you can use the `-o extended | -X` flag to get an output similar
to `\x` in `psql`:
```
% kubectl get pods | ./tablizer -x
NAME: repldepl-7bcd8d5b64-7zq4l
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
% kubectl get pods | tablizer -X
NAME: repldepl-7bcd8d5b64-7zq4l
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
AGE: 5h28m
NAME: repldepl-7bcd8d5b64-m48n8
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
NAME: repldepl-7bcd8d5b64-m48n8
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
AGE: 5h28m
NAME: repldepl-7bcd8d5b64-q2bf4
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
NAME: repldepl-7bcd8d5b64-q2bf4
READY: 1/1
STATUS: Running
RESTARTS: 1 (71m ago)
AGE: 5h28m
```
@@ -63,11 +65,12 @@ Tablize can read one or more files or - if none specified - from STDIN.
You can also specify a regex pattern to reduce the output:
```
% kubectl get pods | ./tablizer q2bf4
% kubectl get pods | tablizer q2bf4
NAME(1) READY(2) STATUS(3) RESTARTS(4) AGE(5)
repldepl-7bcd8d5b64-q2bf4 1/1 Running 1 (69m ago) 5h26m
```
There are more output modes like org-mode (orgtbl) and markdown.
## Installation
@@ -106,6 +109,10 @@ https://github.com/TLINDEN/tablizer/blob/main/tablizer.pod
Or if you cloned the repository you can read it this way (perl needs
to be installed though): `perldoc tablizer.pod`.
If you have the binary installed, you can also read the man page with
this command:
tablizer --man
## Getting help

View File

@@ -84,11 +84,13 @@ func Execute() {
func init() {
rootCmd.PersistentFlags().BoolVarP(&lib.Debug, "debug", "d", false, "Enable debugging")
rootCmd.PersistentFlags().BoolVarP(&lib.NoNumbering, "no-numbering", "n", false, "Disable header numbering")
rootCmd.PersistentFlags().BoolVarP(&lib.NoColor, "no-color", "N", false, "Disable pattern highlighting")
rootCmd.PersistentFlags().BoolVarP(&lib.ShowVersion, "version", "V", false, "Print program version")
rootCmd.PersistentFlags().BoolVarP(&lib.InvertMatch, "invert-match", "v", false, "select non-matching rows")
rootCmd.PersistentFlags().BoolVarP(&ShowManual, "man", "m", false, "Display manual page")
rootCmd.PersistentFlags().StringVarP(&lib.Separator, "separator", "s", lib.DefaultSeparator, "Custom field separator")
rootCmd.PersistentFlags().StringVarP(&lib.Columns, "columns", "c", "", "Only show the speficied columns (separated by ,)")
rootCmd.PersistentFlags().IntVarP(&lib.SortByColumn, "sort-by", "k", 0, "Sort by column (default: 1)")
// output flags, only 1 allowed, hidden, since just short cuts
rootCmd.PersistentFlags().BoolVarP(&lib.OutflagExtended, "extended", "X", false, "Enable extended output")

View File

@@ -1,4 +1,5 @@
package cmd
var manpage = `
NAME
tablizer - Manipulate tabular output of other programs
@@ -14,20 +15,22 @@ SYNOPSIS
-v, --invert-match select non-matching rows
-m, --man Display manual page
-n, --no-numbering Disable header numbering
-N, --no-color Disable pattern highlighting
-o, --output string Output mode - one of: orgtbl, markdown, extended, ascii(default)
-X, --extended Enable extended output
-M, --markdown Enable markdown table output
-O, --orgtbl Enable org-mode table output
-s, --separator string Custom field separator
-k, --sort-by int Sort by column (default: 1)
-v, --version Print program version
DESCRIPTION
Many programs generate tabular output. But sometimes you need to
post-process these tables, you may need to remove one or more columns or
you may want to filter for some pattern or you may need the output in
another program and need to parse it somehow. Standard unix tools such
as awk(1), grep(1) or column(1) may help, but sometimes it's a tedious
business.
you may want to filter for some pattern (See PATTERNS) or you may need
the output in another program and need to parse it somehow. Standard
unix tools such as awk(1), grep(1) or column(1) may help, but sometimes
it's a tedious business.
Let's take the output of the tool kubectl. It contains cells with
withespace and they do not separate columns by TAB characters. This is
@@ -35,7 +38,7 @@ DESCRIPTION
You can use tablizer to do these and more things.
tablizer analyses the header fiels of a table, registers the column
tablizer analyses the header fields of a table, registers the column
positions of each header field and separates columns by those positions.
Without any options it reads its input from "STDIN", but you can also
@@ -70,13 +73,44 @@ DESCRIPTION
The numbering can be suppressed by using the -n option.
Finally the -d option enables debugging output which is mostly usefull
By default, if a pattern has been speficied, matches will be
highlighted. You can disable this behavior with the -N option.
Use the -k option to specify by which column to sort the tabular data
(as in GNU sort(1)). The default sort column is the first one. To
disable sorting at all, supply 0 (Zero) to -k.
Finally the -d option enables debugging output which is mostly useful
for the developer.
PATTERNS
You can reduce the rows being displayed by using a regular expression
pattern. The regexp is PCRE compatible, refer to the syntax cheat sheet
here: <https://github.com/google/re2/wiki/Syntax>. If you want to read a
more comprehensive documentation about the topic and have perl installed
you can read it with:
perldoc perlre
Or read it online: <https://perldoc.perl.org/perlre>.
A note on modifiers: the regexp engine used in tablizer uses another
modifier syntax:
(?MODIFIER)
The most important modifiers are:
"i" ignore case "m" multiline mode "s" single line mode
Example for a case insensitive search:
kubectl get pods -A | tablizer "(?i)account"
OUTPUT MODES
There might be cases when the tabular output of a program is way too
large for your current terminal but you still need to see every column.
In such cases the -o extended or -X option can be usefull which enables
In such cases the -o extended or -X option can be useful which enables
*extended mode*. In this mode, each row will be printed vertically,
header left, value right, aligned by the field widths. Here's an
example:

4
go.mod
View File

@@ -4,13 +4,15 @@ go 1.18
require (
github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897
github.com/gookit/color v1.5.2
github.com/olekukonko/tablewriter v0.0.5
github.com/spf13/cobra v1.5.0
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778
)
require (
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/mattn/go-runewidth v0.0.9 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stretchr/testify v1.8.0 // indirect
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 // indirect
)

6
go.sum
View File

@@ -4,6 +4,8 @@ github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46t
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/gookit/color v1.5.2 h1:uLnfXcaFjlrDnQDT+NCBcfhrXqYTx/rcCa6xn01Y8yI=
github.com/gookit/color v1.5.2/go.mod h1:w8h4bGiHeeBpvQVePTutdbERIUf3oJE5lZ8HM0UgXyg=
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/mattn/go-runewidth v0.0.9 h1:Lm995f3rfxdpd6TSmuVCHVb/QhupuXlYr8sCI/QdE+0=
@@ -22,6 +24,10 @@ github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSS
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778 h1:QldyIu/L63oPpyvQmHgvgickp1Yw510KJOqX7H24mg8=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 h1:Bli41pIlzTzf3KEY06n+xnzK/BESIg2ze4Pgfh/aI8c=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -17,6 +17,11 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
package lib
import (
"github.com/gookit/color"
//"github.com/xo/terminfo"
)
var (
// command line flags
Debug bool
@@ -33,15 +38,52 @@ var (
OutflagShell bool
OutputMode string
InvertMatch bool
Pattern string
/*
FIXME: make configurable somehow, config file or ENV
see https://github.com/gookit/color will be set by
io.ProcessFiles() according to currently supported
color mode.
*/
MatchFG string
MatchBG string
NoColor bool
// colors to be used per supported color mode
Colors = map[color.Level]map[string]string{
color.Level16: {
"bg": "green", "fg": "black",
},
color.Level256: {
"bg": "lightGreen", "fg": "black",
},
color.LevelRgb: {
// FIXME: maybe use something nicer
"bg": "lightGreen", "fg": "black",
},
}
// used for validation
validOutputmodes = "(orgtbl|markdown|extended|ascii)"
// main program version
Version = "v1.0.6"
Version = "v1.0.9"
// generated version string, used by -v contains lib.Version on
// main branch, and lib.Version-$branch-$lastcommit-$date on
// development branch
VERSION string
// sorting
SortByColumn int
)
// contains a whole parsed table
type Tabdata struct {
maxwidthHeader int // longest header
maxwidthPerCol []int // max width per column
columns int // count
headers []string // [ "ID", "NAME", ...]
entries [][]string
}

View File

@@ -20,6 +20,8 @@ package lib
import (
"errors"
"fmt"
"github.com/gookit/color"
"os"
"regexp"
"strconv"
"strings"
@@ -51,7 +53,10 @@ func PrepareColumns() error {
func numberizeHeaders(data *Tabdata) {
// prepare headers: add numbers to headers
numberedHeaders := []string{}
maxwidth := 0 // start from scratch, so we only look at displayed column widths
for i, head := range data.headers {
headlen := 0
if len(Columns) > 0 {
// -c specified
if !contains(UseColumns, i+1) {
@@ -61,18 +66,28 @@ func numberizeHeaders(data *Tabdata) {
}
if NoNumbering {
numberedHeaders = append(numberedHeaders, head)
headlen = len(head)
} else {
numberedHeaders = append(numberedHeaders, fmt.Sprintf("%s(%d)", head, i+1))
numhead := fmt.Sprintf("%s(%d)", head, i+1)
headlen = len(numhead)
numberedHeaders = append(numberedHeaders, numhead)
}
if headlen > maxwidth {
maxwidth = headlen
}
}
data.headers = numberedHeaders
if data.maxwidthHeader != maxwidth && maxwidth > 0 {
data.maxwidthHeader = maxwidth
}
}
func reduceColumns(data *Tabdata) {
// exclude columns, if any
if len(Columns) > 0 {
reducedEntries := [][]string{}
reducedEntry := []string{}
var reducedEntry []string
for _, entry := range data.entries {
reducedEntry = nil
for i, value := range entry {
@@ -130,3 +145,21 @@ func trimRow(row []string) []string {
return fixedrow
}
func colorizeData(output string) string {
if len(Pattern) > 0 && !NoColor && color.IsConsole(os.Stdout) {
r := regexp.MustCompile("(" + Pattern + ")")
return r.ReplaceAllString(output, "<bg="+MatchBG+";fg="+MatchFG+">$1</>")
} else {
return output
}
}
func isTerminal(f *os.File) bool {
o, _ := f.Stat()
if (o.Mode() & os.ModeCharDevice) == os.ModeCharDevice {
return true
} else {
return false
}
}

View File

@@ -79,15 +79,15 @@ func TestReduceColumns(t *testing.T) {
columns []int
}{
{
expect: [][]string{[]string{"a", "b"}},
expect: [][]string{{"a", "b"}},
columns: []int{1, 2},
},
{
expect: [][]string{[]string{"a", "c"}},
expect: [][]string{{"a", "c"}},
columns: []int{1, 3},
},
{
expect: [][]string{[]string{"a"}},
expect: [][]string{{"a"}},
columns: []int{1},
},
{
@@ -96,7 +96,7 @@ func TestReduceColumns(t *testing.T) {
},
}
input := [][]string{[]string{"a", "b", "c"}}
input := [][]string{{"a", "b", "c"}}
Columns = "y" // used as a flag with len(Columns)...

View File

@@ -19,6 +19,7 @@ package lib
import (
"errors"
"github.com/gookit/color"
"io"
"os"
)
@@ -26,6 +27,14 @@ import (
func ProcessFiles(args []string) error {
fds, pattern, err := determineIO(args)
if !isTerminal(os.Stdout) {
color.Disable()
} else {
level := color.TermColorLevel()
MatchFG = Colors[level]["fg"]
MatchBG = Colors[level]["bg"]
}
if err != nil {
return err
}
@@ -52,6 +61,7 @@ func determineIO(args []string) ([]io.Reader, string, error) {
// first one is not a file, consider it as regexp and
// shift arg list
pattern = args[0]
Pattern = args[0] // FIXME
args = args[1:]
}

View File

@@ -27,15 +27,6 @@ import (
"strings"
)
// contains a whole parsed table
type Tabdata struct {
maxwidthHeader int // longest header
maxwidthPerCol []int // max width per column
columns int // count
headers []string // [ "ID", "NAME", ...]
entries [][]string
}
/*
Parse tabular input.
*/

View File

@@ -35,10 +35,10 @@ func TestParser(t *testing.T) {
"ONE", "TWO", "THREE",
},
entries: [][]string{
[]string{
{
"asd", "igig", "cxxxncnc",
},
[]string{
{
"19191", "EDD 1", "X",
},
},
@@ -69,7 +69,7 @@ func TestParserPatternmatching(t *testing.T) {
}{
{
entries: [][]string{
[]string{
{
"asd", "igig", "cxxxncnc",
},
},
@@ -78,7 +78,7 @@ func TestParserPatternmatching(t *testing.T) {
},
{
entries: [][]string{
[]string{
{
"19191", "EDD 1", "X",
},
},

View File

@@ -19,18 +19,26 @@ package lib
import (
"fmt"
"github.com/gookit/color"
"github.com/olekukonko/tablewriter"
"os"
"regexp"
"strings"
)
func printData(data *Tabdata) {
// some output preparations:
if OutputMode != "shell" {
// not needed in eval string
numberizeHeaders(data)
}
// remove unwanted columns, if any
reduceColumns(data)
// sort the data
sortTable(data, SortByColumn)
switch OutputMode {
case "extended":
printExtendedData(data)
@@ -76,14 +84,18 @@ func printOrgmodeData(data *Tabdata) {
leftR := regexp.MustCompile("(?m)^\\+")
rightR := regexp.MustCompile("\\+(?m)$")
fmt.Print(rightR.ReplaceAllString(leftR.ReplaceAllString(tableString.String(), "|"), "|"))
color.Print(
colorizeData(
rightR.ReplaceAllString(
leftR.ReplaceAllString(tableString.String(), "|"), "|")))
}
/*
Markdown table
*/
func printMarkdownData(data *Tabdata) {
table := tablewriter.NewWriter(os.Stdout)
tableString := &strings.Builder{}
table := tablewriter.NewWriter(tableString)
table.SetHeader(data.headers)
@@ -95,13 +107,15 @@ func printMarkdownData(data *Tabdata) {
table.SetCenterSeparator("|")
table.Render()
color.Print(colorizeData(tableString.String()))
}
/*
Simple ASCII table without any borders etc, just like the input we expect
*/
func printAsciiData(data *Tabdata) {
table := tablewriter.NewWriter(os.Stdout)
tableString := &strings.Builder{}
table := tablewriter.NewWriter(tableString)
table.SetHeader(data.headers)
table.AppendBulk(data.entries)
@@ -123,6 +137,7 @@ func printAsciiData(data *Tabdata) {
table.SetNoWhiteSpace(true)
table.Render()
color.Print(colorizeData(tableString.String()))
}
/*
@@ -130,22 +145,14 @@ func printAsciiData(data *Tabdata) {
*/
func printExtendedData(data *Tabdata) {
// needed for data output
format := fmt.Sprintf("%%%ds: %%s\n", data.maxwidthHeader) // FIXME: re-calculate if -c has been set
format := fmt.Sprintf("%%%ds: %%s\n", data.maxwidthHeader)
if len(data.entries) > 0 {
var idx int
for _, entry := range data.entries {
idx = 0
for i, value := range entry {
if len(Columns) > 0 {
if !contains(UseColumns, i+1) {
continue
}
}
fmt.Printf(format, data.headers[idx], value)
idx++
color.Printf(format, data.headers[i], value)
}
fmt.Println()
}
}

View File

@@ -19,11 +19,26 @@ package lib
import (
"fmt"
"github.com/gookit/color"
"os"
"strings"
"testing"
)
func stdout2pipe(t *testing.T) (*os.File, *os.File) {
reader, writer, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
origStdout := os.Stdout
os.Stdout = writer
// we need to tell the color mode the io.Writer, even if we don't usw colorization
color.SetOutput(writer)
return origStdout, reader
}
func TestPrinter(t *testing.T) {
startdata := Tabdata{
maxwidthHeader: 5,
@@ -37,10 +52,10 @@ func TestPrinter(t *testing.T) {
"ONE", "TWO", "THREE",
},
entries: [][]string{
[]string{
{
"asd", "igig", "cxxxncnc",
},
[]string{
{
"19191", "EDD 1", "X",
},
},
@@ -50,37 +65,48 @@ func TestPrinter(t *testing.T) {
"ascii": `ONE(1) TWO(2) THREE(3)
asd igig cxxxncnc
19191 EDD 1 X`,
"orgtbl": `|--------+--------+----------|
| ONE(1) | TWO(2) | THREE(3) |
|--------+--------+----------|
| asd | igig | cxxxncnc |
| 19191 | EDD 1 | X |
|--------+--------+----------|`,
"markdown": `| ONE(1) | TWO(2) | THREE(3) |
|--------|--------|----------|
| asd | igig | cxxxncnc |
| 19191 | EDD 1 | X |`,
"shell": `ONE="asd" TWO="igig" THREE="cxxxncnc"
ONE="19191" TWO="EDD 1" THREE="X"`,
"extended": `ONE(1): asd
TWO(2): igig
THREE(3): cxxxncnc
ONE(1): 19191
TWO(2): EDD 1
THREE(3): X`,
}
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
origStdout := os.Stdout
os.Stdout = w
NoColor = true
SortByColumn = 0 // disable sorting
origStdout, reader := stdout2pipe(t)
for mode, expect := range expects {
testname := fmt.Sprintf("print-%s", mode)
t.Run(testname, func(t *testing.T) {
OutputMode = mode
data := startdata // we need to reset our mock data, since it's being modified in printData()
// we need to reset our mock data, since it's being
// modified in printData()
data := startdata
printData(&data)
buf := make([]byte, 1024)
n, err := r.Read(buf)
n, err := reader.Read(buf)
if err != nil {
t.Fatal(err)
}
@@ -88,7 +114,8 @@ ONE="19191" TWO="EDD 1" THREE="X"`,
output := strings.TrimSpace(string(buf))
if output != expect {
t.Errorf("output mode: %s, got:\n%s\nwant:\n%s\n (%d <=> %d)", mode, output, expect, len(output), len(expect))
t.Errorf("output mode: %s, got:\n%s\nwant:\n%s\n (%d <=> %d)",
mode, output, expect, len(output), len(expect))
}
})
}
@@ -97,3 +124,91 @@ ONE="19191" TWO="EDD 1" THREE="X"`,
os.Stdout = origStdout
}
func TestSortPrinter(t *testing.T) {
startdata := Tabdata{
maxwidthHeader: 5,
maxwidthPerCol: []int{
3,
3,
2,
},
columns: 3,
headers: []string{
"ONE", "TWO", "THREE",
},
entries: [][]string{
{
"abc", "345", "b1",
},
{
"bcd", "234", "a2",
},
{
"cde", "123", "c3",
},
},
}
var tests = []struct {
data Tabdata
sortby int
expect string
}{
{
data: startdata,
sortby: 1,
expect: `ONE(1) TWO(2) THREE(3)
abc 345 b1
bcd 234 a2
cde 123 c3`,
},
{
data: startdata,
sortby: 2,
expect: `ONE(1) TWO(2) THREE(3)
cde 123 c3
bcd 234 a2
abc 345 b1`,
},
{
data: startdata,
sortby: 3,
expect: `ONE(1) TWO(2) THREE(3)
bcd 234 a2
abc 345 b1
cde 123 c3`,
},
}
NoColor = true
OutputMode = "ascii"
origStdout, reader := stdout2pipe(t)
for _, tt := range tests {
testname := fmt.Sprintf("print-sorted-table-%d", tt.sortby)
t.Run(testname, func(t *testing.T) {
SortByColumn = tt.sortby
printData(&tt.data)
buf := make([]byte, 1024)
n, err := reader.Read(buf)
if err != nil {
t.Fatal(err)
}
buf = buf[:n]
output := strings.TrimSpace(string(buf))
if output != tt.expect {
t.Errorf("sort column: %d, got:\n%s\nwant:\n%s",
tt.sortby, output, tt.expect)
}
})
}
// Restore
os.Stdout = origStdout
}

46
lib/sort.go Normal file
View File

@@ -0,0 +1,46 @@
/*
Copyright © 2022 Thomas von Dein
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package lib
import (
"sort"
)
func sortTable(data *Tabdata, col int) {
if col <= 0 {
// no sorting wanted
return
}
col-- // ui starts counting by 1, but use 0 internally
// sanity checks
if len(data.entries) == 0 {
return
}
if col >= len(data.headers) {
// fall back to default column
col = 0
}
// actual sorting
sort.SliceStable(data.entries, func(i, j int) bool {
return data.entries[i][col] < data.entries[j][col]
})
}

View File

@@ -43,7 +43,7 @@ for D in $DIST; do
tardir="${tool}-${os}-${arch}-${version}"
tarfile="releases/${tool}-${os}-${arch}-${version}.tar.gz"
set -x
GOOS=${os} GOARCH=${arch} go build -o ${binfile}
GOOS=${os} GOARCH=${arch} go build -o ${binfile} -ldflags "-X 'github.com/tlinden/tablizer/lib.VERSION=${version}'"
mkdir -p ${tardir}
cp ${binfile} README.md LICENSE ${tardir}/
echo 'tool = tablizer

View File

@@ -133,7 +133,7 @@
.\" ========================================================================
.\"
.IX Title "TABLIZER 1"
.TH TABLIZER 1 "2022-10-05" "1" "User Commands"
.TH TABLIZER 1 "2022-10-14" "1" "User Commands"
.\" For nroff, turn off justification. Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
@@ -153,21 +153,23 @@ tablizer \- Manipulate tabular output of other programs
\& \-v, \-\-invert\-match select non\-matching rows
\& \-m, \-\-man Display manual page
\& \-n, \-\-no\-numbering Disable header numbering
\& \-N, \-\-no\-color Disable pattern highlighting
\& \-o, \-\-output string Output mode \- one of: orgtbl, markdown, extended, ascii(default)
\& \-X, \-\-extended Enable extended output
\& \-M, \-\-markdown Enable markdown table output
\& \-O, \-\-orgtbl Enable org\-mode table output
\& \-s, \-\-separator string Custom field separator
\& \-k, \-\-sort\-by int Sort by column (default: 1)
\& \-v, \-\-version Print program version
.Ve
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
Many programs generate tabular output. But sometimes you need to
Many programs generate tabular output. But sometimes you need to
post-process these tables, you may need to remove one or more columns
or you may want to filter for some pattern or you may need the output
in another program and need to parse it somehow. Standard unix tools
such as \fBawk\fR\|(1), \fBgrep\fR\|(1) or \fBcolumn\fR\|(1) may help, but sometimes it's a
tedious business.
or you may want to filter for some pattern (See \s-1PATTERNS\s0) or you
may need the output in another program and need to parse it somehow.
Standard unix tools such as \fBawk\fR\|(1), \fBgrep\fR\|(1) or \fBcolumn\fR\|(1) may help, but
sometimes it's a tedious business.
.PP
Let's take the output of the tool kubectl. It contains cells with
withespace and they do not separate columns by \s-1TAB\s0 characters. This is
@@ -175,8 +177,8 @@ not easy to process.
.PP
You can use \fBtablizer\fR to do these and more things.
.PP
\&\fBtablizer\fR analyses the header fiels of a table, registers the column
positions of each header field and separates columns by those
\&\fBtablizer\fR analyses the header fields of a table, registers the
column positions of each header field and separates columns by those
positions.
.PP
Without any options it reads its input from \f(CW\*(C`STDIN\*(C'\fR, but you can also
@@ -218,14 +220,53 @@ the original order.
.PP
The numbering can be suppressed by using the \fB\-n\fR option.
.PP
By default, if a \fBpattern\fR has been speficied, matches will be
highlighted. You can disable this behavior with the \fB\-N\fR option.
.PP
Use the \fB\-k\fR option to specify by which column to sort the tabular
data (as in \s-1GNU\s0 \fBsort\fR\|(1)). The default sort column is the first one. To
disable sorting at all, supply 0 (Zero) to \-k.
.PP
Finally the \fB\-d\fR option enables debugging output which is mostly
usefull for the developer.
useful for the developer.
.SS "\s-1PATTERNS\s0"
.IX Subsection "PATTERNS"
You can reduce the rows being displayed by using a regular expression
pattern. The regexp is \s-1PCRE\s0 compatible, refer to the syntax cheat
sheet here: <https://github.com/google/re2/wiki/Syntax>. If you want
to read a more comprehensive documentation about the topic and have
perl installed you can read it with:
.PP
.Vb 1
\& perldoc perlre
.Ve
.PP
Or read it online: <https://perldoc.perl.org/perlre>.
.PP
A note on modifiers: the regexp engine used in tablizer uses another
modifier syntax:
.PP
.Vb 1
\& (?MODIFIER)
.Ve
.PP
The most important modifiers are:
.PP
\&\f(CW\*(C`i\*(C'\fR ignore case
\&\f(CW\*(C`m\*(C'\fR multiline mode
\&\f(CW\*(C`s\*(C'\fR single line mode
.PP
Example for a case insensitive search:
.PP
.Vb 1
\& kubectl get pods \-A | tablizer "(?i)account"
.Ve
.SS "\s-1OUTPUT MODES\s0"
.IX Subsection "OUTPUT MODES"
There might be cases when the tabular output of a program is way too
large for your current terminal but you still need to see every
column. In such cases the \fB\-o extended\fR or \fB\-X\fR option can be
usefull which enables \fIextended mode\fR. In this mode, each row will be
useful which enables \fIextended mode\fR. In this mode, each row will be
printed vertically, header left, value right, aligned by the field
widths. Here's an example:
.PP

View File

@@ -14,11 +14,13 @@ tablizer - Manipulate tabular output of other programs
-v, --invert-match select non-matching rows
-m, --man Display manual page
-n, --no-numbering Disable header numbering
-N, --no-color Disable pattern highlighting
-o, --output string Output mode - one of: orgtbl, markdown, extended, ascii(default)
-X, --extended Enable extended output
-M, --markdown Enable markdown table output
-O, --orgtbl Enable org-mode table output
-s, --separator string Custom field separator
-k, --sort-by int Sort by column (default: 1)
-v, --version Print program version
@@ -37,8 +39,8 @@ not easy to process.
You can use B<tablizer> to do these and more things.
B<tablizer> analyses the header fiels of a table, registers the column
positions of each header field and separates columns by those
B<tablizer> analyses the header fields of a table, registers the
column positions of each header field and separates columns by those
positions.
Without any options it reads its input from C<STDIN>, but you can also
@@ -74,8 +76,15 @@ the original order.
The numbering can be suppressed by using the B<-n> option.
By default, if a B<pattern> has been speficied, matches will be
highlighted. You can disable this behavior with the B<-N> option.
Use the B<-k> option to specify by which column to sort the tabular
data (as in GNU sort(1)). The default sort column is the first one. To
disable sorting at all, supply 0 (Zero) to -k.
Finally the B<-d> option enables debugging output which is mostly
usefull for the developer.
useful for the developer.
=head2 PATTERNS
@@ -100,7 +109,7 @@ C<i> ignore case
C<m> multiline mode
C<s> single line mode
Example for a case insensitve search:
Example for a case insensitive search:
kubectl get pods -A | tablizer "(?i)account"
@@ -110,7 +119,7 @@ Example for a case insensitve search:
There might be cases when the tabular output of a program is way too
large for your current terminal but you still need to see every
column. In such cases the B<-o extended> or B<-X> option can be
usefull which enables I<extended mode>. In this mode, each row will be
useful which enables I<extended mode>. In this mode, each row will be
printed vertically, header left, value right, aligned by the field
widths. Here's an example: