We collect practical, well-explained Bash one-liners, and promote best practices in Bash shell scripting. To get the latest Bash one-liners, follow @bashoneliners on Twitter. If you find any problems, report a bug on GitHub.

Tags

0

Sort du output in Human-readable format

 $ du -hsx * | sort -rh

— by Vaevictus on April 26, 2012, 9:08 p.m.

Explanation

sort supports -h for human readable number sorting.

Limitations

Probably a newer GNU only option for sort. :D

0

Replace symlinks with the actual files they are pointing at

 $ find /path/to/dir -type l -exec sh -c 'cp --remove-destination "$(readlink "{}")" "{}"' \; 

— by Janos on April 24, 2012, 3:29 p.m.

Explanation

  • All the double quoting is necessary to handle filenames with spaces.
  • Calling sh with -exec is necessary to evaluate readlink for each symlink

Limitations

The BSD implementation of cp does not have the --remove-destination flag.

0

Expire a user's password immediately

 $ chage -d 0 USERNAME

— by Janos on April 23, 2012, 11:05 p.m.

Explanation

This will effectively force the user to change his/her password at next login.

Limitations

Not in BSD. Yes in Linux. Don't know in UNIX.

0

Convert any 16:9 video to play on a QHD widescreen Android phone

 $ ffmpeg -i $1 -y -threads 0 -subq 6 -deinterlace -level 30 -f mp4 -acodec libfaac -ab 160k -ar 24000 -ac 2 -vcodec libx264 -b 1000k -maxrate 1000k -bufsize 2000k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -coder 0 -refs 2 -flags +loop -vol 256 -trellis 1 -me_method umh -async 1 $2

— by openiduser44 on April 18, 2012, 8:18 p.m.

Explanation

The Android video player is somewhat fussy about the formats it can play. This ffmpeg script will take any movie file in 16:9 widescreen format and convert it into a form that can be played on one of the current leading QHD phones such as HTC Sensation, Samsung Galaxy S2 or Motorola Atrix. Files that are merely marked as widescreen (e.g. DVD VOBs) will have to be processed into a true widescreen format such as .m2t first.

Parameters:

$1 the input file

$2 the output file

Output file suffix should be .mp4

Requires:

ffmpeg and codecs

0

Sort du output in Human-readable format

 $ for i in G M K; do du -hsx * | grep "[0-9]$i\b" | sort -nr; done 2>/dev/null

— by Janos on April 14, 2012, 11:06 a.m.

Explanation

  • The reason to use a for loop is to sort results with G or M or K values separately, otherwise sort -n would just sort everything by the numbers regardless of G M K suffix.
  • grep "[0-9]$i\b" matches lines containing a digit followed by G or M or K followed by a "word boundary"

0

Sort du output in Human-readable format

 $ for i in $(echo -e 'G\nM\nK'); do du -hsx /* 2>/dev/null | grep '[0-9]'$i | sort -rn; done

— by jasembo on April 14, 2012, 6:02 a.m.

Explanation

  • echo -e prints G for Gigabytes, M for Megabytes and K for Kilobytes in a line each.
  • 2>/dev/null send stderr to /dev/null
  • sort -rn sorts in reverse numerical order. Largest first

0

Convert a decimal number to octal, hexadecimal, binary, or anything

 $ echo 'obase=2;1234' | bc

— by Janos on April 11, 2012, 11:20 p.m.

Explanation

  • bc is an arbitrary precision calculator language.
  • obase defines the conversion base for output numbers, in this example 2 (binary)
  • ; is a statement separator in bc
  • 1234 is the decimal number to convert
  • By piping the command to bc we get 1234 in binary format

0

Convert from avi format to mp4 encoding

 $ ffmpeg -i file.avi file.mp4

— by Janos on April 11, 2012, 11:10 p.m.

Explanation

FFmpeg is a complete, cross-platform solution to record, convert and stream audio and video. It includes libavcodec - the leading audio/video codec library.

Limitations

It is not a standard package in most systems and distros.

0

Format input into multiple columns, like a table, useful or pretty-printing

 $ mount | column -t

— by Janos on April 8, 2012, 4:08 p.m.

Explanation

column is a utility for formatting text. With the -t flag it detects the number of columns in the input so it can format the text into a table-like format.

For more details see man column.

0

Function to extract columns from an input stream

 $ col() { awk '{print $'$(echo $* | sed -e 's/ /,$/g')'}'; }

— by Janos on April 5, 2012, 11:36 p.m.

Explanation

Something I do a lot is extract columns from some input where cut is not suitable because the columns are separated by not a single character but multiple spaces or tabs. So I often do things like:

... | awk '{print $7, $8}'

... which is a lot of typing, additionally slowed down when typing symbols like '{}$ ... Using the simple one-line function above makes it easier and faster:

... | col 7 8

How it works:

  • The one-liner defines a new function with name col
  • The function will execute awk, and it expects standard input (coming from a pipe or input redirection)
  • The function arguments are processed with sed to use them with awk: replace all spaces with ,$ so that for example 1 2 3 becomes 1,$2,$3, which is inserted into the awk command to become the well formatted shell command: awk '{print $1,$2,$3}'

0

Resize an image proportionally to some specified width or height

 $ mogrify -geometry x31 path/to/image.gif

— by Janos on April 3, 2012, 9:48 p.m.

Explanation

  • mogrify is part of ImageMagick, an image manipulation software suite
  • mogrify manipulates the specified images. If you prefer to keep the original image untouched and write the manipulated image to a different file, simply replace mogrify with convert, the syntax is the same, but the last command line argument will be the target image to write to.
  • The -geometry flag is to resize the image, it requires a dimension parameter in the format WIDTHxHEIGHT
  • The dimension in this example has no width, which means the image will be resized to height=31 pixels, and the width will be proportional.

Limitations

ImageMagick is not a standard package, though it is open source and available in many systems.

0

Do something in another directory without going there

 $ (cd /path/to/somewhere; tar c .) > somewhere.tar

— by Janos on April 2, 2012, 10:24 p.m.

Explanation

As explained superbly in man bash:

   (list) list is executed in a subshell environment (see  COMMAND  EXECU-
          TION  ENVIRONMENT below).  Variable assignments and builtin com-
          mands that affect the  shell's  environment  do  not  remain  in
          effect  after  the  command completes.  The return status is the
          exit status of list.

In other words, this is a handy way to do something somewhere else without having to go there and coming back.

0

Remove carriage return '\r' character in many files, without looping and intermediary files

 $ recode pc..l1 file1 file2 file3

— by Anon8MaLEqEp on March 31, 2012, 5:23 p.m.

Explanation

The recode utility installed on many systems converts between character sets. This command is shorthand for recode IBM-PC..latin1 file1 file2 file3 which converts the given files from CRLF to LF line endings.

0

Find the target path a symlink is pointing to

 $ readlink a_symbolic_link_to_somewhere

— by Janos on March 31, 2012, 3:23 p.m.

Explanation

Sure, you could figure out the link target from the output of ls -l a_symbolic_link_to_somewhere too, but the output of readlink is simply the target of the symbolic link itself, so it is cleaner and easier to read.

0

Get the octal, hexadecimal and decimal codes of the ASCII character set

 $ man ascii

— by Janos on March 29, 2012, 9:48 a.m.

Explanation

Knowing the octal, hexadecimal or decimal code of the ASCII character set can be handy at times. In the past, too often I did things like:

perl -e 'for my $n (1 .. 255) { print $n, chr($n), $n, "\n"; }'

... when a simple man ascii would have done the trick...

On a related note, these all print the letter "A":

echo -e '\0101'
printf '\101'
printf '\x41'
perl -e 'print "\x41"'

0

List or edit and re-execute commands from the history list

 $ fc -l

— by Janos on March 15, 2012, 12:10 p.m.

Explanation

fc is a little known but very useful bash built-in.

  • fc -l will list the most recent 16 commands
  • fc will open the last command in a text editor defined in the environmental variable FCEDIT or EDITOR or else vi, and re-execute when you exit
  • fc 5 9 will open the history entries 5 to 9 in a text editor
  • fc -s pat=sub will run the last command after substituting pat with sub in it (does not open editor)
  • fc -s pat=sub cc is the same but on the last command starting with cc
  • fc -s cc will run the last command starting with cc

For more info see help fc.

0

Find the most recently modified files in a directory and all subdirectories

 $ find /path/to/dir -type f -mtime -7 -print0 | xargs -0 ls -lt | head

— by Janos on March 8, 2012, 5:10 p.m.

Explanation

  • find /path/to/dir -type f -mtime -7 -print0 prints all the files in the directory tree that have been modified within the last 7 days, with null character as the delimiter
  • xargs -0 ls -lt expects a null delimited list of filenames and will sort the files by modification time, in descending order from most recent to oldest
  • Since we are looking for the most recent files, with head we get the first 10 lines only

Note that if there are too many files in the output of find, xargs will run multiple ls -lt commands and the output will be incorrect. This is because the maximum command line length is getconf ARG_MAX and if this is exceeded xargs has to split the execution to multiple commands. So depending on your use case you may need to tweak the -mtime parameter to make sure there are not too many lines in the output.

0

List open files

 $ lsof -n

— by Janos on March 2, 2012, 10:01 a.m.

Explanation

With the -n flag it will not try to resolve network numbers to host names for network files, making it run a bit faster.

With the -c option you can select processes executing a matching command. And with the -t flag the output will be simply process ids without a header, suitable to use with kill. For example you can kill Google Chrome process gone crazy like this:

kill -HUP $(lsof -n -c /google/i -t)

Here /google/i is a regular expression pattern with case insensitive matching.

0

Set a colorful bash prompt per dev test prod environments

 $ PS1='\[\e[1;31m\][\u@\h \W]\$\[\e[0m\] '

— by Janos on Feb. 25, 2012, 2:46 p.m.

Explanation

It is useful to set a different color for the shell prompt in different deployment environments like dev/test/production, so that you don't mix up your multiple windows and do something by accident in the wrong window.

  • PS1 contains the format of the primary prompt
  • \[\e[1;31m\] sets the foreground color to red
  • \u will be substituted with the current username
  • \h will be substituted with the hostname
  • \W will be substituted with the current directory name
  • \[\e[0m\] is the end marker of the color setting

To make the color stand out even more for root users, the inverse color can be interesting too:

PS1='\[\e[7;31m\][\u@\h \W]\$\[\e[0m\] '

Other color examples:

#PS1='\[\e[1;32m\][\u@\h \W]\$\[\e[0m\] ' # green
#PS1='\[\e[1;33m\][\u@\h \W]\$\[\e[0m\] ' # yellow
#PS1='\[\e[1;34m\][\u@\h \W]\$\[\e[0m\] ' # blue

You can learn more in man bash, search for "PROMPTING".

Limitations

Your terminal program must support colors, of course ;-)

0

Calculate the total disk space used by a list of files or directories

 $ du -c

— by openiduser30 on Feb. 14, 2012, 1:34 a.m.

Explanation

-c option of du prints the total size of the arguments

0

View a file with line numbers

 $ cat -n /path/to/file | less

— by openiduser28 on Feb. 13, 2012, 5:14 p.m.

Explanation

cat -n will number all lines of a file.

Limitations

It will add some white spaces as padding.

0

Print the lines of file2 that are missing in file1

 $ grep -vxFf file1 file2

— by Janos on Feb. 8, 2012, 2:42 p.m.

Explanation

  • -f is to specify a file with the list of patterns: file1
  • -F is to treat the patterns fixed strings, without using regular expressions
  • -x is to match exactly the whole line
  • -v is to select non-matching lines

The result is effectively the same as:

diff file1 file2 | grep '^>' | sed -e s/..//

Limitations

The flags of grep might work differently depending on the system. So yeah you might prefer the second way which should work everywhere. Nonetheless the various of flags of grep are interesting.

0

Find in files, recursively

 $ find /etc -type f -print0 2>/dev/null | xargs -0 grep --color=AUTO -Hn 'nameserver' 2>/dev/null

— by openiduser21 on Feb. 2, 2012, 7:32 p.m.

Explanation

In the example above, find and display every file in /etc containing the string nameserver with the corresponding line, including line number, sample output:

/etc/ppp/ip-up.d/0dns-up:9:# Rev. Dec 22 1999 to put dynamic nameservers last.

/etc/ppp/ip-up.d/0dns-up:23:# nameservers given by the administrator. Those for which 'Dynamic' was chosen

/etc/ppp/ip-up.d/0dns-up:24:# are empty. 0dns-up fills in the nameservers when pppd gets them from the

/etc/ppp/ip-up.d/0dns-up:26:# 'search' or 'domain' directives or additional nameservers. Read the

/etc/ppp/ip-up.d/0dns-up:77:# nameserver lines to the temp file.

0

Concatenate PDF files using GhostScript

 $ gs -dNOPAUSE -sDEVICE=pdfwrite -sOUTPUTFILE=output.pdf -dBATCH file1.pdf file2.pdf file3.pdf

— by Janos on Jan. 26, 2012, 8:51 a.m.

Explanation

Free PDF editing software might become more and more available, but this method has been working for a long time, and likely will continue to do so.

Limitations

It may not work with all PDFs, for example files that don't conform to Adobe's published PDF specification.

0

Format text with long lines to text with fixed width

 $ fmt -s -w80 file.txt

— by Janos on Jan. 22, 2012, 10:08 a.m.

Explanation

  • It will break lines longer than 80 characters at appropriate white spaces to make them less than 80 characters long.
  • The -s flag will collapse multiple consecutive white spaces into one, or at the end of a sentence a double space.