We aim to collect practical, well-explained bash one-liners, and promote best practices in shell scripting. To get the latest bash one-liners, follow @bashoneliners on Twitter, or subscribe to our rss feed. If you find any problems, report a bug on GitHub.

Tags

0

Put an ssh session in the background

 $ ~^z

— by openiduser3 on Dec. 9, 2011, 7:44 p.m.

Explanation

  • Normally, ^z (read: ctrl-z) pauses the execution of the current foreground task. That doesn't work in an ssh session, because it is intercepted by the remote shell. ~^z is a special escape character for this case, to pause the ssh session and drop you back to the local shell.
  • For all escape characters see ~?
  • The ~ escape character must always follow a newline to be interpreted as special.
  • See man ssh for more details, search for ESCAPE CHARACTERS
1

Rotate a movie file with mencoder

 $ mencoder video.avi -o rotated-right.avi -oac copy -ovc lavc -vf rotate=1

— by openiduser3 on Dec. 2, 2011, 10:30 p.m.

Explanation

mencoder is part of mplayer.

Other possible values of the rotate parameter:

  • 0: Rotate by 90 degrees clockwise and flip (default).
  • 1: Rotate by 90 degrees clockwise.
  • 2: Rotate by 90 degrees counterclockwise.
  • 3: Rotate by 90 degrees counterclockwise and flip.
0

Recursively remove all empty sub-directories from a directory tree

 $ find . -type d | tac | xargs rmdir 2>/dev/null

— by openiduser3 on Nov. 29, 2011, 8:01 p.m.

Explanation

  • find will output all the directories
  • tac reverses the ordering of the lines, so "leaf" directories come first
  • The reordering is important, because rmdir removes only empty directories
  • We redirect error messages (about the non-empty directories) to /dev/null

Limitations

In UNIX and BSD systems you might not have tac, you can try the less intuitive tail -r instead.

0

Remove all the versioned-but-empty directories from a Subversion checkout

 $ find . -name .svn -type d | while read ss; do dir=$(dirname "$ss"); test $(ls -a "$dir" | wc -l) == 3 && echo "svn rm \"$dir\""; done

— by openiduser3 on Nov. 27, 2011, 8:38 a.m.

Explanation

Empty directories in version control stink. Most probably they shouldn't be there. Such directories have a single subdirectory in them named ".svn", and no other files or subdirectories.

  • The "find" searches for files files named .svn that are directories
  • The "while" assigns each line in the input to the variable ss
  • The "dirname" gets the parent directory of a path, the quotes are necessary for paths with spaces
  • ls -a should output 3 lines if the directory is in fact empty: ".", "..", and ".svn"
  • If the test is true and there are precisely 3 files in the directory, echo what we want to do
  • If the output of the one-liner looks good, pipe it to | sh to really execute
0

Create a sequence of integer numbers

 $ echo {4..-9}

— by openiduser3 on Nov. 24, 2011, 10:07 p.m.

Explanation

  • Useful for counters. For example, to do something 10 times, you could do for i in {1..10}; do something; done
  • Be careful, there cannot be spaces between the braces
  • As the example shows, can count backwards too

Limitations

Does not work in /bin/sh, this is bash specific.

0

Redirect the output of the time builtin command

 $ { time command; } > out.out 2> time+err.out

— by openiduser3 on Nov. 20, 2011, 8:34 p.m.

Explanation

  • time is a bash builtin command, and redirecting its output does not work the same way as with proper executables
  • If you execute within braces like above, the output of time will go to stderr (standard error), so you can capture it with 2>time.out
  • An alternative is to use the /usr/bin/time executable, by referring to its full path. (The path may be different depending on your system.)
0

Copy a directory with a large number of files to another server

 $ tar cp -C /path/to/dir . | ssh server2 'tar x -C /path/to/target'

— by openiduser3 on Nov. 17, 2011, 12:19 p.m.

Explanation

With a large number of files, scp or rsync can take very very long. It's much faster to tar up on one side and extract on the other. Without the -f flag tar writes output to standard output and expects input from standard input, so piping to ssh can work this way, without creating any intermediary files.

You may (or may not) gain an extra speed boost by compression, either with the z flag for tar, or with the -C flag for ssh, or with gzip pipes in the middle, like this:

tar cp -C /path/to/dir . | gzip | ssh server2 'gzip -cd | tar x -C /path/to/target'

Limitations

Depending on your system and version of tar, you may need to hyphenate the flags, for example tar -cp, and tar -x. The -C flag might also not work, but that shouldn't be too difficult to work around.

0

Redirect the output of multiple commands

 $ { cmd1 ; cmd2 ; cmd3 ; } > out.out 2> err.out

— by openiduser3 on Nov. 14, 2011, 10:08 a.m.

Explanation

  • Curly braces are very helpful for grouping several commands together
  • Be careful with the syntax: 1. there must be whitespace after the opening brace 2. there must be a semicolon after the last command and before the closing brace
  • Another practical use case: test something || { echo message; exit 1; }
0

View a file with line numbers

 $ grep -n ^ /path/to/file | less

— by openiduser3 on Nov. 9, 2011, 11:05 p.m.

Explanation

  • grep ^ will match all lines in a file
  • grep -n will prefix each line of output with the line number within its input file

Limitations

In some systems you might have to use egrep instead of grep.

0

Print the n-th and m-th line of a file

 $ sed -ne '101 p' -e '106 p' /path/to/the/file

— by openiduser3 on Nov. 6, 2011, 11:20 p.m.

Explanation

  • The above command will print the 101th and 106th lines of the specified file.
  • The -n switch will make sed not print all lines by default.
  • The -e switch is to specify a sed command, you can use it multiple times at once.

Some sed command examples:

  • 45 p - print line #45
  • 34,55 p - print lines #34-#55
  • 99,$ p - print lines #99-end of the file
0

Repeat the previous command but with a string replacement

 $ ^geomtry^geometry

— by openiduser3 on Nov. 4, 2011, 7:10 a.m.

Explanation

This can be very useful for example after a mistyped command like this:

convert -crop 745x845+0+150 my_logo.png -geomtry 400x my_logo2.png
^geomtry^geometry

"-geomtry" should have been "-geometry", the one-liner above will re-run the command with a replacement that fixes the typo.

1

Rename all files in the current directory by capitalizing the first letter of every word in the filenames

 $ ls | perl -ne 'chomp; $f=$_; tr/A-Z/a-z/; s/(?<![.'"'"'])\b\w/\u$&/g; print qq{mv "$f" "$_"\n}'

— by openiduser3 on Nov. 1, 2011, 12:51 p.m.

Explanation

  • When you pipe something to perl -ne, each input line is substituted into the $_ variable. The chomp, tr///, s/// perl functions in the above command all operate on the $_ variable by default.
  • The tr/A-Z/a-z/ will convert all letters to lowercase.
  • The regular expression pattern (?<![.'])\b\w matches any word character that follows a non-word character except a dot or a single quote.
  • The messy-looking '"'"' in the middle of the regex pattern is not a typo, but necessary for inserting a single quote into the pattern. (The first single quote closes the single quote that started the perl command, followed by a single quote enclosed within double quotes, followed by another single quote to continue the perl command.) We could have used double quotes to enclose the perl command, but then we would have to escape all the dollar signs which would make everything less readable.
  • In the replacement string $& is the letter that was matched, and by putting \u in front it will be converted to uppercase.
  • qq{} in perl works like double quotes, and can make things easier to read, like in this case above when we want to include double quotes within the string to be quoted.
  • After the conversions we print a correctly escaped mv command. Pipe this to bash to really execute the rename with | sh.

Limitations

The above command will not work for files with double quotes in the name, and possibly other corner cases.

0

Do not save command history of current bash session

 $ HISTFILE=

— by openiduser3 on Oct. 29, 2011, 2:41 p.m.

Explanation

The command history of the current bash session is saved to $HISTFILE when you exit the shell. If this variable is blank, the command history will not be saved.

0

Use rsync instead of cp to get a progress indicator when copying large files

 $ rsync --progress largefile.gz somewhere/else/

— by openiduser3 on Oct. 19, 2011, 12:48 a.m.

Explanation

Although rsync is famous for synchronizing files across machines, it also works locally on the same machine. And although the cp command doesn't have progress indicator, which can be annoying when copying large files, but rsync does have it, so there you go.

Limitations

When copying directories be careful that the meaning of a trailing slash when specifying directories can be slightly different from cp.

0

Unpack all of the .tar.bz2 files in current directory

 $ for FILE in *; do tar -jxf $FILE; done

— by versorge on Oct. 17, 2011, 11:23 p.m.

Explanation

  • for: begins the loop syntax in Bash.
  • FILE: can be any variable name you create.
  • do: this is the action you're going to tell Bash to perform on each pass of the loop.
0

Create and restore backups using cpio

 $ find . -xdev -print0 | cpio -oa0V | gzip > path_to_save.cpio.gz

— by openiduser3 on Oct. 17, 2011, 11:06 p.m.

Explanation

To restore:

gzip -cd path_to_save.cpio.gz | cpio -imV

Why not use tar instead? cpio is slightly more accurate!

  • find . -xdev -print0 finds all files and directories without crossing over to other partitions and prints a null delimited list of filenames
  • cpio -oa0V takes the list of files to archive from stdin and creates an archive file preserving timestamps and permissions
  • cpio -imV extracts the files and directories from stdin while preserving timestamps and permissions
0

Alert me by email when a disconnected or unreachable server comes back online

 $ while ! ping -c1 the_host_down; do sleep 1; done && date | mail -s 'the host is back!' me@example.com

— by janos on Oct. 10, 2011, 8:26 p.m.

Explanation

  • ping -c1 host sends just one ping and exits with success or error.
  • while will keep running until the ping succeeds
  • When the ping succeeds the while loop will end, and an email will be sent with the date as the message

Limitations

  • Depending on your system the parameter for sending a single ping might be different from -c1
  • Depending on your system the mail command might be different or work differently
0

Add timestamp to the output of ping

 $ ping some_host | while read LINE; do echo $(date): $LINE; done

— by janos on Oct. 10, 2011, 8:13 p.m.

Explanation

The while loop reads the output of ping line by line, and echoes it back with $(date) prepended.

0

Test a one-liner with echo commands first, pipe to bash when ready

 $ paste <(ls) <(ls | tr A-Z a-z) | while read OLD NEW; do echo mv -v $OLD $NEW; done | sh

— by janos on Oct. 8, 2011, 1 p.m.

Explanation

You can do a lot of damage with a single one-liner. So while testing the sequence and syntax of commands it's good to just echo the commands that you will want to execute in the end, rather than really executing them. If the output looks good, don't bother removing the echo-s (which can be error prone), it's easier and safer to just pipe the echoed commands to bash to execute.

0

Find the most recently modified files in a directory and all subdirectories

 $ find /path/to/dir -type f | perl -ne 'chomp(@files = <>); my $p = 9; foreach my $f (sort { (stat($a))[$p] <=> (stat($b))[$p] } @files) { print scalar localtime((stat($f))[$p]), "\t", $f, "\n" }' | tail

— by janos on Oct. 4, 2011, 10:25 p.m.

Explanation

  • find path_to_dir -type f prints all the files in the directory tree
  • chomp(@files = <>); reads all the lines into an array
  • stat($a) is an array of interesting info about a file. Index 7 is size, 8 is access time, 9 is modification time, etc. (See man perlfunc for details and search for stat EXPR.)
  • sort { (stat($a))[9] <=> (stat($b))[9] } @files sorts the files by modification time
  • print scalar localtime((stat($f))[9]), "\t", $f, "\n" - prints the modification time formatted nicely, followed by a tab and the filename
0

View specific column of data from a large file with long lines

 $ cat /tmp/log.data |colrm 1 155|colrm 60 300

— by versorge on Oct. 4, 2011, 9:55 p.m.

Explanation

  • cat: prints the file to standard output
  • colrm: removes selected columns from a file
0

Delete unversioned files in a Subversion checkout directory and all subdirectories

 $ svn st | grep ^? | sed -e 's/^? *//' | xargs -i{} echo rm -fr "{}"

— by janos on Oct. 3, 2011, 8:22 p.m.

Explanation

If there are no spaces in the file names, a simpler command will be enough. This one-liner works even if there are spaces and certain special characters in the file names.

  • svn st shows the changes in the Subversion checkout
  • | grep ^? matches only lines starting with question mark (= unversioned files)
  • | sed -e 's/^? *//' removes the question mark at the beginning of the line and the space characters following it
  • | xargs -i{} echo rm -fr "{}" executes an echo command for each line in the input, where the command is formed by inserting the input line in the {} placeholder. Confirm the result looks good and remove the echo to perform the rm.

Limitations

The command will not work with files that have " (double quote) in the name...

0

Get the available space on a partition as a single numeric value

 $ df /path/to/dir | sed -ne 2p | awk '{print $4}'

— by openiduser3 on Oct. 2, 2011, 5:41 p.m.

Explanation

  • sed -ne 2p prints the 2nd line
  • awk '{print $4}' prints the 4th column

Limitations

The output of the df command might be different depending on the system, and the available space might not be the 4th column. Make the necessary adjustments depending on your system.

0

Schedule a one-time task using "at" command and intuitive date specifications

 $ at now + 30 minutes

— by openiduser3 on Sept. 25, 2011, 11:30 a.m.

Explanation

  • The example will run something 30 minutes from now.
  • Another example: at 0815 wed -- run something at 8:15am on the next Wednesday.
  • With atq you can see the list of scheduled jobs, this is good to confirm that you specified the date correctly and the job will run when you want.
  • With atrm you can cancel scheduled jobs, using the job id as parameter, which you can find in the atq output.
0

Aliases the ls command to display the way I like it

 $ alias ls='ls -lhGpt --color=always'

— by versorge on Sept. 22, 2011, 12:25 a.m.

Explanation

alias: allows you to define a shortcut to a longer command. In this case 'ls' with the flags set for long listing, human readable no group and in color, not displaying the group, appending a slash to directories and color coding the output.