In this post I goint to present you some tricks that, in my experience, allow to greatly improve the usability of the console programs. I write them down here to avoid to forget them and in the hope that they can be useful to other people.
The key is to make the programs simple, which normally is not as easy as it sounds, but I hope these tricks will help you to achieve that.
I usually program small tools/scripts in Linux, so these advices are focused on that platform, but also can be applied to others.
Besides, many of these practices were inspired by The Art of Unix Programming, so if you like programming and Unix/Linux, you should check it.
I will asume basic knowledge of Linux systems. If you don't know Linux, I strongly recommend you to learn how to use it.
Now, let's go to the topic.
Make programs easy to install
Maybe this can be tedious the first time you do cause many times doesn't imply to program, but to prepare your program to be easily deployed. However, it is incredibly comfortable when you or other people need to use the program.
Once your tool is available to download, it should be easy to install/compile. It is a common practice that programs are compiled and installed by using makefiles. For example:
This is a common practice in C programs, where the
make command compiles the
program and the
make install command installs it. Even if your program is not
a C program, I recommend you to create a makefile for it in order to standarize
the program installation and make it more comfortable to other users.
For example, if your program is a Python program, it doesn't need to be
compiled, but the
make command could install the dependencies by executing
pip install -r requirements.txt and
make install could execute
pip install . to install your program in the system.
Maybe your tool is not a python package but a simply python script, then
install could execute
cp ./myscript.py /usr/bin/myscript (remember to make
your script executable (
chmod +x) and adding a shebang to it).
Note: In case you want to install a program for the user (cause you don't have
root privileges or whatever), you can copy it to
~/.local/bin (remember to add
this folder to your PATH, and create it if it is necessary).
For Python programs, I recommend you to create a
setup.py file to allow
install it easily, even without a makefile. Or, at least, to create a
requirements.txt file to indicate your dependencies. Do not specify
your dependencies in your readme, it is incredibly annoying to have to install
Actually, the installation of a program should be as simple as possible. In case you think that your tool is useful, you can include it in a package registry to make it easy to install for anyone.
For example, with Python tools you can registry them in pypi to be able to
install then with
pip install mytool. In case of Rust, you can register it in
the crate registry, to install it with
cargo install mytool. Moreover, you
also can create a package for a Linux distro and install it with
To sum up, even little programs are created to being used, and first step is to install them. Make them easy to install, even if it is tedious, saves users a lot of headaches.
The command line options or parameters are a crucial part of every console program. They are one of the main input methods, so it is important to handle them correctly. For parameters I like to use the following rules:
Note: The argument is the value that user pass to the parameter.
Command line options should always have a long mnemotechnic name easy to
remember that starts by two hyphens
-- . For example,
Sometimes is difficult to choose a name. For example, should I use
passwd? I personally like to choose the more descriptive, in this
password, and set the rest as alias.
For the most used command line options and flags a short option should be
created. The sort option should be only a letter or number, preceded by one
-. Usually lowercase letters are used. Uppercase letters are used in
the case that the intuitive lowercase letter is used. For example, curl uses
-H to indicate an HTTP header since
-h is used for help.
The flags (boolean options without value) normally can be used together with
just one hyphen
-. For example,
ls -lah is the same as
ls -l -a -h. Some
programs use short options of two or more letters, like
-out, but this limits
the capacity of joining short flags.
In Windows environments, the slash
/ used to be used instead of
hyphen. However, nowadays there are many tools that use hyphens, so I prefer to
maintain hyphens. Use hyphens also avoid problems with path arguments that
follows the Unix format, that can begin with an slash.
Follow paremeters conventions
There are certain parameters that have a common meaning known by the community and should not be used for other purposes (without a reason). The most important are the following:
-h/--help-> Show the help of the program. Every program should implement this.
-v/--verbose-> Use to increase the program verbosity. Usually it is accumulative, allowing to specify the detail level by using it many times. For example,
-vfor a low detail and
-vvvfor a high level of detail.
-V/--version-> Show the version of the program.
Moreover, the tools with similar or related behaviours should use the same name for parameters. It is a good practice to use as parameter name those used by famous tools.
For example, if you are programming a tool that performs HTTP requests as it
main functionality, you should use parameters similar to those used by
wget. In this case, you could use
-H to add a custom header as curl or
for setting the user agent. This way it is easier to pick parameter names and also
helps to reduce the user learning curve.
Have flexible parameters
A feature that I have found incredibly comfortable in programs is when those have parameters that are flexible and accept many different argument types or deduce the type of argument without being specified.
For instance, imagine a bruteforcing program that accepts list of users and
passwords. It would be very comfortable that the same parameter
could accept both an username or a file with usernames (and same for
passwords). This way you don't have to remember 2 different parameters such as
-U/--user-list for different type of arguments, and you can do
something like the following:
Do you see? Many different attack posibilities without having to remember different parameters since the program checks if the arguments are files or not.
Other example is the
tar utility that decompress file without needing to
specify the format of the target file. E.g
tar -xf file.tgz or
tar -xf file.tar.xz. Pretty comfortable.
The main idea is that the program should deduce the most from the minimum input, saving the user from specifying to much information that could be redundant.
Notwithstanding, this flexibility must be intuitive for the user. Be careful with parsing parameters in extrange ways that leads to unpredictable behaviours.
Use a library for handling parameters
Every programming language has at least one known library to handle parameters. Use it. It is easier, faster and cleaner than parse the command line by yourself. I let you some examples of libraries that parse parameters:
Sometimes you have to parse some arguments manually, but try to handle as much as you can from command line from inside of the library. They usually include many options such as:
Auto generated usage for
Use of flags (boolean parameters that doesn't accept a value)
Parameters that only accept a group of choices
Parameters that can be used many times (like verbose)
Parameters that cannot be used together (exclusive)
Define the value type accepted for a parameter, if it is a number, string, file, etc
Hooks for adding custom routines to parse parameters (very useful)
Configuration files are widely used by programs, specially by servers (nginx, ssh) or daemons (cron), but also clients (git, proxychains) or interactive programs (i3, emacs).
In case of using configuration files, following certain rules can improve their usability.
Use easily editable configuration files
In case you use a configuration file, instead of invent a custom format, you can use one of following: TOML, INI or YAML. They were designed to be human friendly and are perfect for configuration files. They are easy to read and to edit with any text editor, and many languages have libraries to parse them (be careful parsing YAML since it can lead to arbitrary code execution).
There are tools that use XML or JSON formats for configuration files, but I don't recommend to use them because of my experience, since to edit and read them can be a little cumbersome for a human. I think they are more appropiate to be used to exchange data between programs, like in the case of REST/SOAP APIs or databases.
Configuration files default places
When reading configuration files in Linux is a common practice to look in some default places.
The configuration files that contains system configurations are usually stored
/etc directory, where they can be placed in the following places:
File with the program name following by .conf
Directory with the program name, like
Directory with the program name following by .d
On the other side, the files with specific user configurations are stored in the user directory. Usually these are hidden files (that start with .) that can be found in the following places:
File with program name
Directory with program name following by .d
In the .config directory, with a directory/file with the program name
Additionally, many programs also contain a parameter that allows to specify a custom location for a configuration file/directory.
The program output is one the main channels to interact with the user and other programs. Thus, having a clear output that gives useful information is essential when creating a program.
Here are some tips about output.
Use stdout and stderr
Each program is connected to two output streams: stdout and stderr.
Stdout should be used to write the useful output data of the program, whereas stderr should be used to show other additional information like the program errors, warnings, debug messages, etc.
In Unix environments, the use of pipes
| to connect the program output
(stdout) to the input of the following (stdin) is a common practice. Therefore,
if all the output is sent to stdout, including the program banner or warning
messages, parsing the result with utilities like
cut can be more
difficult or even infeasible.
For example, check this program to test user credentials:
$ credbrute -u /tmp/users.txt -p superman -v >>> Credbrute: The fastest credential checker <<< [INFO] Testing admin:superman -> Fail [INFO] Testing bar:superman -> Success bar:superman [INFO] Testing foo:superman -> Success foo:superman
If everything is written to stdout and we only want to get the name of the successful accounts, things can get ugly:
$ credbrute -u /tmp/users.txt -p superman -v | cut -d ':' -f 1 > superman_users.txt $ cat superman_users.txt >>> Credbrute [INFO] Testing admin [INFO] Testing bar bar [INFO] Testing foo foo
Notwithstanding, if only the result is written to stdout, and the rest to stderr, then the result is the expected one:
$ credbrute -u /tmp/users.txt -p superman -v | cut -d ':' -f 1 > superman_users.txt >>> Credbrute: The fastest credential checker <<< [INFO] Testing admin:superman -> Fail [INFO] Testing bar:superman -> Success [INFO] Testing foo:superman -> Success $ cat superman_users.txt bar foo
As you can see, an output well redirected to stdout and stderr can improve the program usability, specially when is used with other programs.
As we have seen, usually the output of a program can be used as the input of
others, and this is something that Unix hackers know. That's the reason there are
so many tools to parse text line such as
sed, etc. If you
format the output of your program in a way that can interact with the rest of
tools through pipes
|, the usability raises.
For example, imagine that you want to get the users that use "superman" as password:
$ credbrute -u /tmp/users.txt -p superman >>> Credbrute: The fastest credential checker <<< Fail: Incorrect password for user admin -> superman Success!! The password of user foo is superman Success!! The password of user bar is superman $ credbrute -u /tmp/users.txt -p superman | grep "The password of" | cut -d ' ' -f 6 foo bar
You can increase the effectivity with a more concise output:
$ credbrute -u /tmp/users.txt -p superman foo:superman bar:superman $ credbrute -u /tmp/users.txt -p superman | cut -d ':' -f 1 foo bar
And, in case you want more detailed, you could print it to stderr:
$ credbrute -u /tmp/users.txt -p superman | cut -d ':' -f 1 [INFO] Testing admin:superman -> Fail [INFO] Testing foo:superman -> Success foo [INFO] Testing bar:superman -> Success bar
You could also count the number of users that has the password "superman":
$ credbrute -u /tmp/users.txt -p superman | wc -l 2
This way you can use other tools to process your output easily, which leads to user to manipulate your data without the need to add new functionalities to your program.
Other case is the Powershell console, that in contrast to sh consoles, is an expert managing objects instead of text. Then, in Powershell scripts/cmdlets is preferable to return objects instead of text.
Some programs returns complex (or not so complex) data that can be useful to structure in a format like XML or JSON, in order to be parsed by other programs.
For example, nmap allows to save the results of the scans in a XML file, which is very useful to parse the results of many different scans many times with a single program.
Anyway, if your program produces a structured format output, don't forget to indicate an schema of the output data and types. In XML can be done by using DTDs, and JSON Schema in case of JSON.
When the output is intended to be read by humans, the use of colors can improve the reading and identification of important data, specially in programs with hundreds of lines.
When used, even if the meaning of the colors depends on the program, it is important to be consistent and give only one meaning to each color, so the user mind can map the color to an event in the program execution.
For example, red for program errors, green for good news, yellow for informational messages and blue for debug.
In order to let the user to focus in the important details, the output of a program should be minimum. However, the program also should allow to increase its verbosity in case a user want to know what is happening in order to understand or debug the program.
For example, the
cp utility doesn't display output if not is requested, it
just copy files that user indicates. But the verbosity level can be increased
$ cp /tmp/a /tmp/b $ cp /tmp/a /tmp/b -v '/tmp/a' -> '/tmp/b'
In depends on the program, but a rule I like to follow is, by default, to only
show the program result and the error messages (if any), and use the verbose
-v to allow user to see informational or debug messages.
Besides, a practice that I found annoying is to shown a banner. A banner is an irrelevant piece of information that and displaces the previous command outputs, thus increasing the necessary scrolling to review the previous command results and sometimes making impossible to take decent screenshots of the terminal with all the relevant information.
I know it is super cool to design and show a banner, I have did this in the past, but it occuppies too much terminal space. Please, do not show banners. Or at least create an specific functionality to show it.
Documentation: Give examples
Documentation is hard and bored. Everyone knows that. However, a trick to make the documentation useful is to provide examples of the program use. People is lazy, so if we have an example of use, we can just simply copy, paste and modify it for our purposes.
At least you should have examples for the common use of the tool, where you can show both the input and output of the tool, allowing the users to know what to expect from the result. For instance, the readme of Rubeus contains many examples of what can be done.
Moreover, if you add examples of uses of your in combination with other tools it can be give the users an idea of when to use it.
Likewise, give examples of the files used by your program. On one hand, you can show templates of the configuration files, like the configuration of Responder. On the other hand, you can give examples and schemas for the structured files (JSON, XML, etc) produced by the program, in order to make other people aware of the format and thus allowing them to create another programs to parse your results.
Other advice in order to make your readme is to not include the output of the
help command (
-h/--help), it can be seen by using the help command and it is
hard to maintain. This doesn't mean that you can't explain the options in Readme
(and provide examples of use).
Well, that's all. I hope this practices allow you to make your programs more useful for you and the others. In case you know other tricks, please share them!!