Useful utilities in Unix for developers

After some years working as a software developer in different companies I have used many programs and utilities in *nix environments that I’d like to share.

This is not an exhaustive tutorial about how to get the most of them, many options will be just ignored and I will list only how I use them on a daily basis.

Text processing

I use these utilities to get info from a log or to quickly change a config file on a server.

tail

It prints the last part of a file, it accepts some parameters like -f which will print the last part of a file and waits to print what it’s appended to the file (useful to see live logs).

$ tail -f logs/log.txt

Or if you want just the last n lines you can do:

$ tail -n 200 logs/log.txt

Further reading: Manual Page

grep

The grep command searches a file, the standard input or multiple files for a pattern and prints the lines that match that pattern. As a pattern you can use a string or a regular expression, here are some examples.

$ grep error logs/log.txt # searchs error in log

The same result using the standard input with a pipe:

$ cat logs/log.txt | grep error # searchs error in log

If you want to search only for the whole word, not including parts of it in the results you can with -w:

$ grep -w "error" logs/log.txt

In order to find a text in a tree of files you can use -r:

$ grep -r "error" logs

If you want to show the number of lines that match that pattern:

$ grep -c "error" logs/log.txt

Further reading: Manual Page

diff

Compares two files, I tend to use it without options.

$ diff file1.txt file2.txt

Further reading: Manual Page

vi

Well I could say many things about this editor. In my computer I use Vim on a daily basis but in many servers this is not installed by default where vi is so I use the later for small edits in config files.

$ vi nginx.conf


Network

curl

Allows you to transfer data with many protocols (HTTP, FTP…etc). I tend to use it to download something from the Internet.

$ curl http://example.com/file.zip

You can also use patterns like:

$ curl http://example.com/[2000-2014]/part{a,b,c}.zip

Further reading: Manual Page

scp

It’s like the combination of SSH and cp (altough it actually stands for secure copy), basically you can copy files through the SSH protocol, a simple example:

$ scp user@1.1.1.1:/files/1.txt .

It also works the other way around:

$ scp 1.txt user@1.1.1.1:/files/

Further reading: Manual Page

rsync

Rsync it’s a very complete tool but I tend to use like a simple “Dropbox” for certain directories.

Totally worth it to read the whole man page: http://ss64.com/bash/rsync.html.

Management

htop

An interactive process viewer, it’s like an improved top. You just need to execute htop and it will start. It isn’t installed by default in many distributions but it’s a common package. If you can’t install it you can use top, which is very similar.

Screenshot of htop

Further reading: Manual Page

iostat

Useful to monitor devices. It comes in handy when you don’t know what’s happening with a certain disk.

Screenshot of iostat on a Mac

Further reading: Manual Page

##Directories & Files

A lot in Unix systems has to do with files in some OS like Plan9 you can even treat a network interface as a file. These utilities are very basic but I could not leave them out.

mv / cd / cp / ls

Very basic utilities to move files or directories, change current directory, copy files or directories and list the content of a certain directory respectively.

pwd

Response to the question: where am I?

rm

Remove a file or tree of directories (with -r option).

ln

Link between two files or directories.

$ ln -f origin new_link

Further reading: Manual Page

find

I use it to find a file by name on a server:

$ find / -name foo

Further reading: Manual Page

Best utility: man

I remember that in my first class of OSs the teacher taught us the man utility. With man you can find everything you need, here you can see how to use it.