We aim to collect practical, well-explained bash one-liners, and promote best practices in shell scripting. To get the latest bash one-liners, follow @bashoneliners on Twitter, or subscribe to our rss feed. If you find any problems, report a bug on GitHub.

Tags

0

Sort du output in Human-readable format

 $ for i in G M K; do du -hsx * | grep "[0-9]$i\b" | sort -nr; done 2>/dev/null

— by openiduser3 on April 14, 2012, 11:06 a.m.

Explanation

  • The reason to use a for loop is to sort results with G or M or K values separately, otherwise sort -n would just sort everything by the numbers regardless of G M K suffix.
  • grep "[0-9]$i\b" matches lines containing a digit followed by G or M or K followed by a "word boundary"
0

Sort du output in Human-readable format

 $ for i in $(echo -e 'G\nM\nK'); do du -hsx /* 2>/dev/null | grep '[0-9]'$i | sort -rn; done

— by jasembo on April 14, 2012, 6:02 a.m.

Explanation

  • echo -e prints G for Gigabytes, M for Megabytes and K for Kilobytes in a line each.
  • 2>/dev/null send stderr to /dev/null
  • sort -rn sorts in reverse numerical order. Largest first
1

Convert a decimal number to octal, hexadecimal, binary, or anything

 $ bc <<<'obase=2;1234'

— by openiduser43 on April 12, 2012, 8 p.m.

Explanation

  • <<< A variant of here documents
0

Convert a decimal number to octal, hexadecimal, binary, or anything

 $ echo 'obase=2;1234' | bc

— by openiduser3 on April 11, 2012, 11:20 p.m.

Explanation

  • bc is an arbitrary precision calculator language.
  • obase defines the conversion base for output numbers, in this example 2 (binary)
  • ; is a statement separator in bc
  • 1234 is the decimal number to convert
  • By piping the command to bc we get 1234 in binary format
0

Convert from avi format to mp4 encoding

 $ ffmpeg -i file.avi file.mp4

— by openiduser3 on April 11, 2012, 11:10 p.m.

Explanation

FFmpeg is a complete, cross-platform solution to record, convert and stream audio and video. It includes libavcodec - the leading audio/video codec library.

Limitations

It is not a standard package in most systems and distros.

0

Format input into multiple columns, like a table, useful or pretty-printing

 $ mount | column -t

— by openiduser3 on April 8, 2012, 4:08 p.m.

Explanation

column is a utility for formatting text. With the -t flag it detects the number of columns in the input so it can format the text into a table-like format.

For more details see man column.

0

A simple function to conveniently extract columns from an input stream

 $ col() { awk '{print $'$(echo $* | sed -e 's/ /,$/g')'}'; }

— by openiduser3 on April 5, 2012, 11:36 p.m.

Explanation

Something I do a lot is extract columns from some input where cut is not suitable because the columns are separated by not a single character but multiple spaces or tabs. So I often do things like:

... | awk '{print $7, $8}'

... which is a lot of typing, additionally slowed down when typing symbols like '{}$ ... Using the simple one-line function above makes it easier and faster:

... | col 7 8

How it works:

  • The one-liner defines a new function with name col
  • The function will execute awk, and it expects standard input (coming from a pipe or input redirection)
  • The function arguments are processed with sed to use them with awk: replace all spaces with ,$ so that for example 1 2 3 becomes 1,$2,$3, which is inserted into the awk command to become the well formatted shell command: awk '{print $1,$2,$3}'
0

Resize an image proportionally to some specified width or height

 $ mogrify -geometry x31 path/to/image.gif

— by openiduser3 on April 3, 2012, 9:48 p.m.

Explanation

  • mogrify is part of ImageMagick, an image manipulation software suite
  • mogrify manipulates the specified images. If you prefer to keep the original image untouched and write the manipulated image to a different file, simply replace mogrify with convert, the syntax is the same, but the last command line argument will be the target image to write to.
  • The -geometry flag is to resize the image, it requires a dimension parameter in the format WIDTHxHEIGHT
  • The dimension in this example has no width, which means the image will be resized to height=31 pixels, and the width will be proportional.

Limitations

ImageMagick is not a standard package, though it is open source and available in many systems.

0

Do something in another directory without going there

 $ (cd /path/to/somewhere; tar c .) > somewhere.tar

— by openiduser3 on April 2, 2012, 10:24 p.m.

Explanation

As explained superbly in man bash:

   (list) list is executed in a subshell environment (see  COMMAND  EXECU-
          TION  ENVIRONMENT below).  Variable assignments and builtin com-
          mands that affect the  shell's  environment  do  not  remain  in
          effect  after  the  command completes.  The return status is the
          exit status of list.

In other words, this is a handy way to do something somewhere else without having to go there and coming back.

0

Remove carriage return '\r' character in many files, without looping and intermediary files

 $ recode pc..l1 file1 file2 file3

— by Anon8MaLEqEp on March 31, 2012, 5:23 p.m.

Explanation

The recode utility installed on many systems converts between character sets. This command is shorthand for recode IBM-PC..latin1 file1 file2 file3 which converts the given files from CRLF to LF line endings.

0

Find the target path a symlink is pointing to

 $ readlink a_symbolic_link_to_somewhere

— by openiduser3 on March 31, 2012, 3:23 p.m.

Explanation

Sure, you could figure out the link target from the output of ls -l a_symbolic_link_to_somewhere too, but the output of readlink is simply the target of the symbolic link itself, so it is cleaner and easier to read.

1

Remove carriage return '\r' character in many files, without looping and intermediary files

 $ vi +'bufdo set ff=unix' +'bufdo %s/^M$//' +q file1 file2 file3

— by openiduser3 on March 30, 2012, 3:50 p.m.

Explanation

  • The arguments starting with + are commands in vi that will be executed
  • set ff=unix is a shortcut for set fileformat=unix and means to use "unix" file format, i.e. without the carriage return \r character.
  • %s/^M$// is a pattern substitution for all lines in the entire buffer, the pattern is "carriage return at end of the line", where ^M is not two literal characters but actually one, to enter it on the command line press Ctrl followed by Enter/Return
  • bufdo means to run command in all buffers (each file is opened in a separate buffer)
  • q is to quit vi

Note: the set ff=unix is necessary, otherwise the pattern substitution will not do anything if all the lines end with \r = the file is in dos format, because in that case the line ending character will not be considered as part of the line.

Note: if a shell-script has "accidentally" some carriage returns in it, then when you try to execute you may get an error: bad interpreter: No such file or directory. This one-liner fixes that problem. If you know that all the lines in the file have the carriage return, and there is only one file to fix, then a simplified version of the one-liner is enough:

vi +'set ff=unix' +wq file1
0

Get the octal, hexadecimal and decimal codes of the ASCII character set

 $ man ascii

— by openiduser3 on March 29, 2012, 9:48 a.m.

Explanation

Knowing the octal, hexadecimal or decimal code of the ASCII character set can be handy at times. In the past, too often I did things like:

perl -e 'for my $n (1 .. 255) { print $n, chr($n), $n, "\n"; }'

... when a simple man ascii would have done the trick...

On a related note, these all print the letter "A":

echo -e '\0101'
printf '\101'
printf '\x41'
perl -e 'print "\x41"'
1

Sort and remove duplicate lines in a file in one step without intermediary files

 $ vi +'%!sort | uniq' +wq file.txt

— by openiduser3 on March 22, 2012, 1:09 p.m.

Explanation

We open a file with vi and run two vi commands (specified with +):

  1. %!sort | uniq
    • % = range definition, it means all the lines in the current buffer.
    • ! = run filter for the range specified. Filter is an external program, in this example sort | uniq
  2. wq = write buffer contents to file and exit.
0

Show files containing "foo" and "bar" and "baz"

 $ grep -l 'baz' $(grep -l 'bar' $(grep -lr 'foo' *) )

— by Anon5eqErEbE on March 16, 2012, 5:37 a.m.

Explanation

Most people familiar with extended regular expressions know you can use the pipe symbol | to represent "or", so to see files containing any of "foo", "bar", or "baz" you could run:

grep -Elr 'foo|bar|baz' *

There is no corresponding symbol representing "and", but you can achieve the same effect by nesting invocations to grep. grep -lr 'foo' * returns a list of filenames in or below the current directory containing "foo". Via the $( ... ) syntax, this list is then operated on by grep -l 'bar', returning a list of filenames containing both 'foo' and 'bar', which finally is operated on by grep -l "baz". The end result is a list of filenames containing all three terms.

Limitations

This one-liner results in scanning files multiple times. You will want to put the term you expect to match the fewest number of times farthest to the right (that is, in the same position as "foo") and the one you expect to match most frequently farthest to the left (the same position as "baz"). This way, you will weed out the largest number of files sooner, making the one-liner complete more quickly.

0

List or edit and re-execute commands from the history list

 $ fc -l

— by openiduser3 on March 15, 2012, 12:10 p.m.

Explanation

fc is a little known but very useful bash built-in.

  • fc -l will list the most recent 16 commands
  • fc will open the last command in a text editor defined in the environmental variable FCEDIT or EDITOR or else vi, and re-execute when you exit
  • fc 5 9 will open the history entries 5 to 9 in a text editor
  • fc -s pat=sub will run the last command after substituting pat with sub in it (does not open editor)
  • fc -s pat=sub cc is the same but on the last command starting with cc
  • fc -s cc will run the last command starting with cc

For more info see help fc.

0

Find the most recently modified files in a directory and all subdirectories

 $ find /path/to/dir -type f -mtime -7 -print0 | xargs -0 ls -lt | head

— by openiduser3 on March 8, 2012, 5:10 p.m.

Explanation

  • find /path/to/dir -type f -mtime -7 -print0 prints all the files in the directory tree that have been modified within the last 7 days, with null character as the delimiter
  • xargs -0 ls -lt expects a null delimited list of filenames and will sort the files by modification time, in descending order from most recent to oldest
  • Since we are looking for the most recent files, with head we get the first 10 lines only

Note that if there are too many files in the output of find, xargs will run multiple ls -lt commands and the output will be incorrect. This is because the maximum command line length is getconf ARG_MAX and if this is exceeded xargs has to split the execution to multiple commands. So depending on your use case you may need to tweak the -mtime parameter to make sure there are not too many lines in the output.

0

List open files

 $ lsof -n

— by openiduser3 on March 2, 2012, 10:01 a.m.

Explanation

With the -n flag it will not try to resolve network numbers to host names for network files, making it run a bit faster.

With the -c option you can select processes executing a matching command. And with the -t flag the output will be simply process ids without a header, suitable to use with kill. For example you can kill Google Chrome process gone crazy like this:

kill -HUP $(lsof -n -c /google/i -t)

Here /google/i is a regular expression pattern with case insensitive matching.

0

Set a colorful bash prompt per dev test prod environments

 $ PS1='\[\e[1;31m\][\u@\h \W]\$\[\e[0m\] '

— by openiduser3 on Feb. 25, 2012, 2:46 p.m.

Explanation

It is useful to set a different color for the shell prompt in different deployment environments like dev/test/production, so that you don't mix up your multiple windows and do something by accident in the wrong window.

  • PS1 contains the format of the primary prompt
  • \[\e[1;31m\] sets the foreground color to red
  • \u will be substituted with the current username
  • \h will be substituted with the hostname
  • \W will be substituted with the current directory name
  • \[\e[0m\] is the end marker of the color setting

To make the color stand out even more for root users, the inverse color can be interesting too:

PS1='\[\e[7;31m\][\u@\h \W]\$\[\e[0m\] '

Other color examples:

#PS1='\[\e[1;32m\][\u@\h \W]\$\[\e[0m\] ' # green
#PS1='\[\e[1;33m\][\u@\h \W]\$\[\e[0m\] ' # yellow
#PS1='\[\e[1;34m\][\u@\h \W]\$\[\e[0m\] ' # blue

You can learn more in man bash, search for "PROMPTING".

Limitations

Your terminal program must support colors, of course ;-)

0

Find in files, recursively

 $ grep -rn 'nameserver' /etc 2>/dev/null

— by atpessoa on Feb. 19, 2012, 8:24 a.m.

Explanation

  • -r make a search recursively;
  • -n print line numbers;
  • -H is not need, is default;

Limitations

  • -i use for case insensitive search;
1

How to send an http POST to a website with a file input field

 $ curl -L -v -F "value=@myfile" "http://domain.tld/whatever.php"

— by openiduser14 on Feb. 15, 2012, 11:26 p.m.

Explanation

  • curl read "man curl" if you need to info like using cookies,etc. you can also use wget
  • -L follow redirects
  • -v be verbose
  • -F an input field
  • value= the name of the input field
  • @myfile the file you want uploaded
  • "http://domain.tld/whatever.php" the url that will take the file
0

Calculate the total disk space used by a list of files or directories

 $ du -cshx ./a ./b

— by openiduser14 on Feb. 15, 2012, 10:43 p.m.

Explanation

  • -s, --summarize; display only a total for each argument
  • -c, --total; produce a grand total
  • -x, --one-file-system; skip directories on different file systems
  • -h, --human-readable; print sizes in human readable format (e.g., 1K 234M 2G)
0

Create a compressed tar file that rsync will transfer efficiently

 $ GZIP='--rsyncable' tar cvzf bobsbackup.tar.gz /home/bob

— by Anon6y5E4Use on Feb. 15, 2012, 12:24 p.m.

Explanation

rsync works by comparing files on the local and remote machine and only sending those parts of the file that have changed. The normal way compression works, it results in everything after the modification changing, meaning lots of data ends up going over the network when you try to rsync compressed files.

The --rsyncable option to gzip changes the compression scheme so that modifications to the input file only affect the part of the file where they're located. This allows rsync to work its magic.

In this one-liner, the z option to tar calls gzip, which recognizes and uses any options specified in the GZIP environment variable.

Limitations

Using the --rsyncable option results in a slightly larger compressed file.

Not all versions of gzip include this feature - use the --help option to see if it's available on your system.

0

Cut select pages from a pdf file and create a new file from those pages.

 $ ps2pdf -dFirstPage=3 -dLastPage=10 input.pdf output.pdf

— by Anon6y5E4Use on Feb. 15, 2012, 11:08 a.m.

Explanation

ps2pdf is a script that comes with Ghostscript - despite the name, it can accept PDF files as input, not just postscript files.

Limitations

Only a single contiguous range of pages can be specified.

0

Calculate the total disk space used by a list of files or directories

 $ du -c

— by openiduser30 on Feb. 14, 2012, 1:34 a.m.

Explanation

-c option of du prints the total size of the arguments