Adam Prescott

Completion files and man pages

Tab completion makes life at the command line that little bit easier. Your shell (bash, or whatever), will complete commands for you, based on what’s available in your PATH. You can also get tab completion of options and flags, so that git push --d<TAB> completes to git push --dry-run (providing you have the git completion file set up correctly). Great. But what about when that isn’t available? You can’t, for example, type rsync --vers<TAB> and have it complete to rsync --version unless you have a sourced completion file for rsync, so you need to check rsync --help or the man page.

So a few weeks ago, I decided to see if I could put something together which would rip out the options from a man page, and maybe try to use that for tab completion, automating the whole thing. I did eventually get a solution of sorts. I may have also reinvented a few wheels. The end result is skel-complete, available on GitHub.

The key part of all this is taking some man page and getting the relevant options and flags based on usual conventions. If you’re expecting some beautifully polished line-parsing code, look away now. Don’t say I didn’t warn you.

/
  ^\s+
  -(\w)(?:\ <?[\w-]+>?)?(?:,\ --([\w-]+)(?:[=\ ]<?[\w-]+>?)?)?
  (?=\s+)
|
  ^\s+
  -(\w)(?:\[<?[\w-]+>?\])?(?:,\ --([\w-]+)(?:\[[=\ ]<?[\w-]+>?\])?)?
  (?=\s+)$
|
  ^\s+
  --([\w\-]+)(?:[=\ ]<?[\w-]+?>?)?
  (?=\s+)$
/x

If you stare at this hard enough, you can see what it does: finds -f and --foo in some text. There’s extra stuff thrown in there for dealing with commands in the style of --foo[=<bar>]. I have by no means tested it on a lot of man pages, but it seems to work okay for a proof-of-concept. Here’s fully-working Ruby example:

%q{
       --allow-empty

       --allow-empty-message
           Like --allow-empty this command is primarily for use by foreign SCM

       --cleanup=<mode>
           whitespace lines and strip removes both whitespace and commentary.

       -e, --edit
           file with -C are usually used as the commit log message unmodified.
           This option lets you further edit the message taken from these
           sources.

       --amend
           includes the usual -i/-o and explicit paths), and the commit log
           editor is seeded with the commit message from the tip of the
           parents -- so the current top commit is discarded.

           It is a rough equivalent for:

                       $ git reset --soft HEAD^
                       $ ... do something else to come up with the right tree ...
                       $ git commit -c ORIG_HEAD

       -i, --include
           Before making a commit out of staged contents so far, stage the
           merge.

       -o, --only
           Make a commit only from the paths specified on the command line,
           option is specified together with --amend, then no paths need to be

       -u[<mode>], --untracked-files[=<mode>]
           Show untracked files.

           The mode parameter is optional (defaults to all), and is used to
           specify the handling of untracked files; when -u is not used, the
           default is normal, i.e. show untracked files and directories.

           The possible options are:

           o    no - Show no untracked files

           o    normal - Shows untracked files and directories

           o    all - Also shows individual files in untracked directories.

               The default can be changed using the status.showUntrackedFiles
               configuration variable documented in git-config(1).
}.scan(
/
  ^\s+
  -(\w)(?:\ <?[\w-]+>?)?(?:,\ --([\w-]+)(?:[=\ ]<?[\w-]+>?)?)?
  (?=\s+)
|
  ^\s+
  -(\w)(?:\[<?[\w-]+>?\])?(?:,\ --([\w-]+)(?:\[[=\ ]<?[\w-]+>?\])?)?
  (?=\s+)$
|
  ^\s+
  --([\w\-]+)(?:[=\ ]<?[\w-]+?>?)?
  (?=\s+)$
/x
).map! { |ary| ary.compact! }

#=> [["allow-empty"], ["allow-empty-message"], ["cleanup"], ["e", "edit"],
#    ["amend"], ["i", "include"], ["o", "only"], ["u", "untracked-files"]]

So, what to do with this thing?

I read up on bash completion files basics, and in doing so discovered that someone actually had the same idea to use the man page (as you might expect). The approach is similar, but requires explicitly listing each command you want to on-the-fly complete for. I thought about some programmatic way of dynamically producing a list of all such commands, by perhaps looking through everything in $PATH, but that doesn’t seem to be possible. Neither does it seem to be possible to tell the complete command that you want your completion function to fire on everything.

So as an alternative to on-the-fly completion based on man pages, you can use this horrifying regular expression, along with a few pipes, to produce a fixed bash completion file.

As a first step, we need some helper function,

__options_from_man() {
	ruby -ne '$_.scan(
	/
	  ^\s+
	  -(\w)(?:\ <?[\w-]+>?)?(?:,\ --([\w-]+)(?:[=\ ]<?[\w-]+>?)?)?
	  (?=\s+)
	|
	  ^\s+
	  -(\w)(?:\[<?[\w-]+>?\])?(?:,\ --([\w-]+)(?:\[[=\ ]<?[\w-]+>?\])?)?
	  (?=\s+)$
	|
	  ^\s+
	  --([\w\-]+)(?:[=\ ]<?[\w-]+?>?)?
	  (?=\s+)$
	/x
	).map! { |e| e.compact! }.each do |a|
	  s = a.map do |e|
	    if e.length > 1
	      "--#{e}"
	    else
	      "-#{e}"
	    end
	  end.join(", ")
	  puts s
	end
	'
}

which you can then pipe a man page into.

man rsync | col -b | __options_from_man

The col -b is to “reverse the line feeds filter”. To see what that means, compare the files produced by man rsync > /tmp/without-col-b and man rsync | col -b > /tmp/with-col-b. Anyway, the output of that looks like

-v, --verbose
-4, --ipv4
-h, --help
--copy-links
--links
--copy-unsafe-links

and at this point you could use, say,

man rsync | col -b | __options_from_man | sort | uniq

made suitably generic, to give you a minimal list of available options as a quick reminder (relying on the regex to work properly). That’s not really any gain over simply using --help, but maybe you don’t care.

At this point, all you need to do is turn the output into a single line of options, insert that into a bash completion file template, redirect the output into a file that you source, like ~/etc/<command>-completion.bash, and, hey presto, a very hacky way of producing a completion file for a given command.

skel-complete rsync > ~/etc/rsync-completion.bash
. ~/etc/rsync-completion.bash
rsync --vers<TAB>

It’s not pretty, but maybe it’s useful, in which case it’s on GitHub.