Tricks to improve console programs usability

Command line options should always have a long mnemotechnic name easy to remember that starts by two hyphens -- . For example, --help, --out-file, etc.

Sometimes is difficult to choose a name. For example, should I use pass, password or passwd? I personally like to choose the more descriptive, in this case password, and set the rest as alias.

For the most used command line options and flags a short option should be created. The sort option should be only a letter or number, preceded by one hyphen -. Usually lowercase letters are used. Uppercase letters are used in the case that the intuitive lowercase letter is used. For example, curl uses -H to indicate an HTTP header since -h is used for help.

The flags (boolean options without value) normally can be used together with just one hyphen -. For example, ls -lah is the same as ls -l -a -h. Some programs use short options of two or more letters, like -out, but this limits the capacity of joining short flags.

In Windows environments, the slash / used to be used instead of hyphen. However, nowadays there are many tools that use hyphens, so I prefer to maintain hyphens. Use hyphens also avoid problems with path arguments that follows the Unix format, that can begin with an slash.

There are certain parameters that have a common meaning known by the community and should not be used for other purposes (without a reason). The most important are the following:

• -h/--help -> Show the help of the program. Every program should implement this.

• -v/--verbose -> Use to increase the program verbosity. Usually it is accumulative, allowing to specify the detail level by using it many times. For example, -v for a low detail and -vvv for a high level of detail.

• -V/--version -> Show the version of the program.

Moreover, the tools with similar or related behaviours should use the same name for parameters. It is a good practice to use as parameter name those used by famous tools.

For example, if you are programming a tool that performs HTTP requests as it main functionality, you should use parameters similar to those used by curl or wget. In this case, you could use -H to add a custom header as curl or -A for setting the user agent. This way it is easier to pick parameter names and also helps to reduce the user learning curve.

A feature that I have found incredibly comfortable in programs is when those have parameters that are flexible and accept many different argument types or deduce the type of argument without being specified.

For instance, imagine a bruteforcing program that accepts list of users and passwords. It would be very comfortable that the same parameter -u/--user could accept both an username or a file with usernames (and same for passwords). This way you don't have to remember 2 different parameters such as -u/--user and -U/--user-list for different type of arguments, and you can do something like the following:

$ ./bruteleaks -u foo -p p4ss
Valid credentials: foo:p4ss

$ ./bruteleaks -u users.txt -p p4ss
Valid credentials: jack:p4ss
Valid credentials: john:p4ss
Valid credentials: samuel:p4ss

$ ./bruteleaks -u users.txt -p passwords.txt
Valid credentials: jack:p4ss
Valid credentials: Batman:Bruc3
Valid credentials: Flash:1mf4st

Do you see? Many different attack posibilities without having to remember different parameters since the program checks if the arguments are files or not.

I found this so useful, that I have created an sample python snippet that allows to get the input for a parameter from a file, argument or stdin.

Other example is the tar utility that decompress file without needing to specify the format of the target file. E.g tar -xf file.tgz or tar -xf file.tar.xz. Pretty comfortable.

The main idea is that the program should deduce the most from the minimum input, saving the user from specifying to much information that could be redundant.

Notwithstanding, this flexibility must be intuitive for the user. Be careful with parsing parameters in extrange ways that leads to unpredictable behaviours.

Every programming language has at least one known library to handle parameters. Use it. It is easier, faster and cleaner than parse the command line by yourself. I let you some examples of libraries that parse parameters:

• C -> argp

• Haskell -> GetOpt

• Python -> argparse

• Powershell -> Built-In

• Rust -> clap

Sometimes you have to parse some arguments manually, but try to handle as much as you can from command line from inside of the library. They usually include many options such as:

• Auto generated usage for -h/--help

• Use of flags (boolean parameters that doesn't accept a value)

• Parameters that only accept a group of choices

• Parameters that can be used many times (like verbose)

• Parameters that cannot be used together (exclusive)

• Define the value type accepted for a parameter, if it is a number, string, file, etc

• Hooks for adding custom routines to parse parameters (very useful)

In case you use a configuration file, instead of invent a custom format, you can use one of following: TOML, INI or YAML. They were designed to be human friendly and are perfect for configuration files. They are easy to read and to edit with any text editor, and many languages have libraries to parse them (be careful parsing YAML since it can lead to arbitrary code execution).

There are tools that use XML or JSON formats for configuration files, but I don't recommend to use them because of my experience, since to edit and read them can be a little cumbersome for a human. I think they are more appropiate to be used to exchange data between programs, like in the case of REST/SOAP APIs or databases.

When reading configuration files in Linux is a common practice to look in some default places.

The configuration files that contains system configurations are usually stored in the /etc directory, where they can be placed in the following places:

• File with the program name following by .conf <program>.conf, like /etc/krb5.conf.

• Directory with the program name, like /etc/nginx/

• Directory with the program name following by .d <program>.d, like /etc/cron.d.

On the other side, the files with specific user configurations are stored in the user directory. Usually these are hidden files (that start with .) that can be found in the following places:

• File with program name .<program>, like ~/.ssh/.

• Directory with program name following by .d .<program>.d, like ~/.emacs.d/.

• In the .config directory, with a directory/file with the program name .config/<program>, like ~/.config/i3/.

Additionally, many programs also contain a parameter that allows to specify a custom location for a configuration file/directory.

Each program is connected to two output streams: stdout and stderr.

Stdout should be used to write the useful output data of the program, whereas stderr should be used to show other additional information like the program errors, warnings, debug messages, etc.

In Unix environments, the use of pipes | to connect the program output (stdout) to the input of the following (stdin) is a common practice. Therefore, if all the output is sent to stdout, including the program banner or warning messages, parsing the result with utilities like grep or cut can be more difficult or even infeasible.

For example, check this program to test user credentials:

$ credbrute -u /tmp/users.txt -p superman -v
>>> Credbrute: The fastest credential checker <<<
[INFO] Testing admin:superman -> Fail
[INFO] Testing bar:superman -> Success
bar:superman
[INFO] Testing foo:superman -> Success
foo:superman

If everything is written to stdout and we only want to get the name of the successful accounts, things can get ugly:

$ credbrute -u /tmp/users.txt -p superman -v | cut -d ':' -f 1 > superman_users.txt

$ cat superman_users.txt
>>> Credbrute
[INFO] Testing admin
[INFO] Testing bar
bar
[INFO] Testing foo
foo

Notwithstanding, if only the result is written to stdout, and the rest to stderr, then the result is the expected one:

$ credbrute -u /tmp/users.txt -p superman -v | cut -d ':' -f 1 > superman_users.txt
>>> Credbrute: The fastest credential checker <<<
[INFO] Testing admin:superman -> Fail
[INFO] Testing bar:superman -> Success
[INFO] Testing foo:superman -> Success

$ cat superman_users.txt
bar
foo

As you can see, an output well redirected to stdout and stderr can improve the program usability, specially when is used with other programs.

As we have seen, usually the output of a program can be used as the input of others, and this is something that Unix hackers know. That's the reason there are so many tools to parse text line such as grep, cut, sed, etc. If you format the output of your program in a way that can interact with the rest of tools through pipes |, the usability raises.

For example, imagine that you want to get the users that use "superman" as password:

$ credbrute -u /tmp/users.txt -p superman
>>> Credbrute: The fastest credential checker <<<
Fail: Incorrect password for user admin -> superman
Success!! 
The password of user foo is superman

Success!! 
The password of user bar is superman

$ credbrute -u /tmp/users.txt -p superman | grep "The password of" | cut -d ' ' -f 6
foo
bar

You can increase the effectivity with a more concise output:

$ credbrute -u /tmp/users.txt -p superman
foo:superman
bar:superman

$ credbrute -u /tmp/users.txt -p superman | cut -d ':' -f 1
foo
bar

And, in case you want more detailed, you could print it to stderr:

$ credbrute -u /tmp/users.txt -p superman | cut -d ':' -f 1
[INFO] Testing admin:superman -> Fail
[INFO] Testing foo:superman -> Success
foo
[INFO] Testing bar:superman -> Success
bar

You could also count the number of users that has the password "superman":

$ credbrute -u /tmp/users.txt -p superman | wc -l
2

This way you can use other tools to process your output easily, which leads to user to manipulate your data without the need to add new functionalities to your program.

Other case is the Powershell console, that in contrast to sh consoles, is an expert managing objects instead of text. Then, in Powershell scripts/cmdlets is preferable to return objects instead of text.

Some programs returns complex (or not so complex) data that can be useful to structure in a format like XML or JSON, in order to be parsed by other programs.

For example, nmap allows to save the results of the scans in a XML file, which is very useful to parse the results of many different scans many times with a single program.

I personally prefer to use the JSON format, since it is simpler than XML. Besides, it can be filtered/tranformed with tools like jq or gron.

Anyway, if your program produces a structured format output, don't forget to indicate an schema of the output data and types. In XML can be done by using DTDs, and JSON Schema in case of JSON.

When the output is intended to be read by humans, the use of colors can improve the reading and identification of important data, specially in programs with hundreds of lines.

When used, even if the meaning of the colors depends on the program, it is important to be consistent and give only one meaning to each color, so the user mind can map the color to an event in the program execution.

For example, red for program errors, green for good news, yellow for informational messages and blue for debug.

In order to let the user to focus in the important details, the output of a program should be minimum. However, the program also should allow to increase its verbosity in case a user want to know what is happening in order to understand or debug the program.

For example, the cp utility doesn't display output if not is requested, it just copy files that user indicates. But the verbosity level can be increased if required:

$ cp /tmp/a /tmp/b
$ cp /tmp/a /tmp/b -v
'/tmp/a' -> '/tmp/b'

In depends on the program, but a rule I like to follow is, by default, to only show the program result and the error messages (if any), and use the verbose parameter -v to allow user to see informational or debug messages.

Besides, a practice that I found annoying is to shown a banner. A banner is an irrelevant piece of information that and displaces the previous command outputs, thus increasing the necessary scrolling to review the previous command results and sometimes making impossible to take decent screenshots of the terminal with all the relevant information.

$ awesomeprogram
   _____                                              
  /  _  \__  _  __ ____   __________   _____   ____   
 /  /_\  \ \/ \/ // __ \ /  ___/  _ \ /     \_/ __ \  
/    |    \     /\  ___/ \___ (  <_> )  Y Y  \  ___/  
\____|__  /\/\_/  \___  >____  >____/|__|_|  /\___  > 
        \/            \/     \/            \/     \/  
__________                                            
\______   \_______  ____   ________________    _____  
 |     ___/\_  __ \/  _ \ / ___\_  __ \__  \  /     \ 
 |    |     |  | \(  <_> ) /_/  >  | \// __ \|  Y Y  \
 |____|     |__|   \____/\___  /|__|  (____  /__|_|  /
                        /_____/            \/      \/ 


Hey, I'm the useful output, do you see me?

I know it is super cool to design and show a banner, I have did this in the past, but it occuppies too much terminal space. Please, do not show banners. Or at least create an specific functionality to show it.