Thursday, August 25, 2016

Sh-Utils

I wrote a set of simple shell commands a little while back.  I actually ended up using them pretty regularly (quick and dirty source control, manual merging of configuration files, editing large compressed and delimited files in place, etc...).  Since they save some non-trivial keystrokes, I cleaned up the code and pushed it to github.  Likewise, I built a python package and published it via PyPI.

The package is called sh-utils.  The commands are...

pm - move p to p'
cpm - copy p to p'
upm - undo p to p'
sw - swap two paths
pt - pivot file over a command
pts - pivot file over a command (stdin)

There are descriptions on the github and PyPI pages, but I'll try to go into a bit more detail.

The three pm commands are mostly meant to work with path (file, directory, pipe, socket, etc...) backups.  For example, it's fairly reasonable to move or copy a file to a .bak (.old, .orig, etc...).  Vice versa, it's fairly reasonable to want to undo it.  So, pm takes a list of paths and moves them safely to a backup by prefixing (-p) or suffixing (-s) a value to the path name.  Likewise, cpm does the same, but copies the path instead of moving it.  upm safely undoes both of them.

For example, making a backup before editing a file...

~ cpm foo

To recover it...

~ upm foo\'

They're additive and subtractive as well, so it's easy to keep a stack of path(s) backups.  Unfortunately, that can make things a bit messy and possibly painful.  So, instead of having to run upm a bunch of times to unwind an stack of backups, upm has a flag (-a) that conflates the latest and earliest paths in a single command.

The next command is sw, which swaps two paths.  Normally swapping two paths takes a few commands and a pivot file (move a path to a pivot, move second to first, move the pivot to the second).  Instead, sw does everything in one command and as close to atomically as possible.

For example, swapping a file with its backup...

~ sw foo foo\'

or...

~ sw foo* 

Last are two pivot commands.  Similar to sw, doing something in place on a file normally needs more than one step and a pivot file.  Instead, pt and pts do everything in one command and as close to atomically as possible.

Normally I would sort a file...

~ sort foo > foo\'
~ mv foo\' foo

With pt...

~ pt sort foo

Finally, pts is similar to pt except it passes the file on stdin, which can be useful for pipelines.

For example, if you wanted to uncompress a delimited file, sort it, parse it, square a column, and re-compress it, normally it would be...

~ xz -dc foo.xz | sort | awk '{ print \$1, \$2 * \$2 }' | xz > foo.xz\'
~ mv foo.xz\' foo.xz

Instead, with pts...

~ pts sh -c "xz -d | sort | awk '{ print \$1, \$2 * \$2 }' | xz" foo.xz

If anyone ends up using these and you have any questions, feel free to ask.