We collect practical, well-explained Bash one-liners, and promote best practices in Bash shell scripting. To get the latest Bash one-liners, follow @bashoneliners on Twitter. If you find any problems, report a bug on GitHub.

Tags

0

`tail -f` a file until text is seen

 $ tail -f /path/to/file.log | sed '/^Finished: SUCCESS$/ q'

— by Janos on Aug. 22, 2012, 8:29 a.m.

Explanation

tail -f until this exact line is seen:

Finished: SUCCESS

The exit condition does not have to be an exact line, it could just well be a simple pattern:

... | sed '/Finished/ q'

0

Run command multiple times with a for loop and a sequence expression

 $ for i in {1..10}; do date; sleep 1; done

— by Janos on Aug. 19, 2012, 9:27 a.m.

Explanation

This is just a regular for loop with a sequence expression. A sequence expression has the form {x..y[..incr]}, where x and y are either integers or single characters, and incr an optional increment.

More examples:

  • {a..f} = a b c d e f
  • {a..f..2} = a c e
  • {0..1}{0..1} = 00 01 10 11

Limitations

Don't try a large range like {1..10000000000000000}, it may render your computer unusable until killed.

1

Recording SSH sessions

 $ ssh -l USER HOST | tee -a /path/to/file

— by LeandroToledo on Aug. 15, 2012, 5:04 p.m.

Explanation

tee is a command which displays or pipes the output of a command and copies it into a file or a variable.

The -a option appends the output to the end of file instead of writing over it.

You can also create an alias in ~/.bashrc to record your session when using ssh:

function sshlog () { \ssh $@ 2>&1 | tee -a $(date +%Y%m%d).log; }
alias ssh=sshlog

0

Clear the swap space forcing everything back to main memory in Linux

 $ sudo swapoff -a; sudo swapon -a

— by Janos on Aug. 14, 2012, 11:21 a.m.

Explanation

Note: if you don't have enough main memory the swapoff will fail.

Limitations

This works only in Linux.

0

Redirection operator to override the noclobber option

 $ some_command >| output.txt

— by Janos on Aug. 11, 2012, 9:21 a.m.

Explanation

Normally the > operator overwrites the target file.

If the noclobber option is set (using: set -o noclobber), the > operator will fail if the target file exists.

The >| overrides the noclobber setting and overwrites the target file.

If the noclobber option is not set, then >| is equivalent to >, naturally.

0

How to set the ip address in Solaris 11

 $ ipadm create-addr -T static -a 192.168.1.10/24 eth0/staticaddr

— by Janos on Aug. 3, 2012, 11:44 a.m.

Explanation

  • eth0 is the name of the network interface
  • ipadm show-if shows the list of network interfaces
  • staticaddr is a name you can choose

More details here: http://docs.oracle.com/cd/E19963-01/html/821-1458/gjwiq.html

1

Record audio from microphone or sound input from the console

 $ sox -t ossdsp -w -s -r 44100 -c 2 /dev/dsp -t raw - | lame -x -m s - File.mp3

— by Kleper on July 28, 2012, 8:55 p.m.

Explanation

sox is a software that lets you connect directly to the sound card and send what passes for it in raw format and the system memory through the concatenation of the linux command we can make real-time audio processing is generated by licks and be converted to mp3.

Limitations

Requires a plugin for alsa oss to run on modern distributions.

1

Use vim to pretty-print code with syntax highlighting

 $ vim +'hardcopy > output.ps' +q style.css 

— by Janos on July 21, 2012, 12:13 a.m.

Explanation

If you have syntax highlighting properly setup in vim, this command will pretty-print the specified file with syntax highlighting to output.ps.

If you prefer PDF, you can convert using ps2pdf output.ps.

1

Log and verify files received via FTP

 $ for i in $(cat /var/log/vsftpd.log | grep $DATE_TIME | grep UPLOAD | grep OK); do ls /FTP/HOME/$i >> /dev/null 2> \&1; if \[ $? = 0 \]; then echo "$i" >> $FILES_OK_UPLOADS.log; else  echo "$DATE ERROR: File $i not found" >> $FTP_FILES_NOTOK_$DATE_TIME.log; fi; done

— by dark_axl on July 10, 2012, 8:54 p.m.

Explanation

This one-liner checks and validates the received files via ftp, and generates a log of these files. To have a record of files received and be able to process, based on the successful transfer and the existence of the files.

1

Edit the Gimp launcher file to disable the splash screen

 $ printf '%s\n' ',s/^Exec=[^ ]*/& -s/' w q | ed /usr/share/applications/gimp.desktop

— by Anon8yhYNaVe on July 1, 2012, 12:57 a.m.

Explanation

sed is designed for editing streams - editing files is what ed is for! You can get consistent behavior on any UNIX platform with the above one-liner.

The printf command sends a series of editing commands to ed, each separated by a newline. In this case, the substitution command ,s/^Exec=[^ ]*/& -s/ is nearly the same as in sed, appending a space and a -s to the line starting with Exec=. The only difference is the comma at the beginning designating the lines to operate on. This is shorthand for 1,$, which tells ed to apply the command to the first through the last lines (i.e., the entire file). w tells ed to write the file, and q to quit.

0

Edit the Gimp launcher file to disable the splash screen

 $ sudo sed -i 's/^Exec=[^ ]*/& -s/' /usr/share/applications/gimp.desktop

— by Janos on June 30, 2012, 9:06 p.m.

Explanation

  • The -i flag of sed means to perform the command "in place", that is, save any changes in the input file. Use this flag with extreme caution, one wrong move and you can completely break the original file.
  • The regex pattern /^Exec=[^ ]*/ will match the line starting with Exec= followed by zero or more non-space characters.
  • In the replacement string, & -s, the & is replaced with whatever was matched, in this example probably something like Exec=gimp-2.8, after which we add a space and the -s flag which will effectively disable the splash screen when starting Gimp.

Limitations

The -i flag of sed works differently in GNU and BSD systems. This example works in GNU systems only. The equivalent in BSD is:

sudo sed -i '' 's/^Exec=[^ ]*/& -s/' /usr/share/applications/gimp.desktop

In any case, always be very careful when using the -i flag of sed.

0

`less` is more convenient with the `-F` flag

 $ less -F FILE1

— by Janos on June 25, 2012, 6:47 p.m.

Explanation

less is a "pager" program like more, with a lot of added features. By default, to exit less you have to press q. This can be annoying when viewing a small file that would fit on the screen.

The -F flag to the rescue! When started with the -F flag, less will quit if the entire input (whether from stdin or a file) fits on a single screen.

It has no effect whatsoever for longer input, so it is safe to add an alias for this:

alias less='less -F'

0

Append to a file text, a blank line, and the last line of another file

 $ { echo some text; echo; tail -n1 /var/log/apache2/error.log; } >> /path/to/file

— by Janos on June 22, 2012, 5:29 p.m.

Explanation

All the standard output from all the commands between the braces will be redirected.

0

Append to a file text, a blank line, and the last line of another file

 $ echo -e "From: me\n\n$(tail -n1 /var/log/apache2/error.log)" >> file

— by kevin on June 21, 2012, 8:18 p.m.

Explanation

  • -e option to echo makes it interpret '\n' as a newline
  • $(command) syntax runs a command, then uses its output in place

Limitations

The -e flag of echo doesn't work on all systems. In that case you can use printf instead.

-1

Run a local shell script on a remote server without copying it there

 $ ssh user@server bash < /path/to/local/script.sh

— by Janos on June 21, 2012, 12:06 a.m.

Explanation

Yes this is almost trivial: a simple input redirection, from a local shell script to be executed by bash on the remote server.

The important point being, if you have a complex and very long chain of commands to run on a remote server, it is better to put the commands in a shell script, break the long one-liner to multiple lines for readability and easier debugging.

Replace bash accordingly depending on the language of the script, for example for python:

ssh user@server python < /path/to/local/script.py

0

Convert a list of terms in slug format to capitalized words

 $ sed -e 's/^./\U&/' -e 's/_./\U&/g' -e 's/_/ /g' /path/to/input

— by Janos on June 17, 2012, 7:54 a.m.

Explanation

The goal here is to take an input like this:

police_station
post_office
real_estate_agency

... and convert it to an output like this:

Police Station
Post Office
Real Estate Agency
  • -e ... the sed command can take several -e parameters, which will be executed one by one when processing each line in the input
  • The s/// command is a pattern replacement, and has the general format s/pattern/replacement/flags
  • s/^./\U&/ - replace the first letter of the line with uppercase version of the letter: \U means convert to uppercase, & is the matched string
  • s/_./\U&/g- replace _ and any letter followed by it. The g flag at the end means a "global" replacement, so all occurrences of the pattern _. will be replaced
  • s/_/ /g - replace all underscores with spaces
  • Input to sed can come from a list of files, or input redirection with <, or from a pipe.

0

Execute different commands with find depending on file type

 $ find /path/to/dir -type d -exec chmod 0755 '{}' \; -o -type f -exec chmod 0644 '{}' \;

— by Janos on June 17, 2012, 12:01 a.m.

Explanation

  • -type d -exec chmod 0755 '{}' \; for each directory, run chmod 0755
  • \; is to mark the end of the -exec
  • {} is replaced with the filename, we enclosed it in single quotes like this '{}' to handle spaces in filenames
  • -ological OR operator
  • -type f -exec chmod 0644 '{}' \; for each regular file, run chmod 0644

0

Convert m4a files to mp3 using faad and lame

 $ faad -o tmp.wav music.m4a && lame -b 192 tmp.wav music.mp3

— by Janos on June 14, 2012, 9:29 a.m.

Explanation

  • Step 1: convert m4a to wav using faad
  • Step 2: convert wav to mp3 using lame
  • -b 192 is the bitrate

Limitations

Neither faad nor lame are standard commands.

0

Write both stdout and stderr to the same file

 $ do_something.sh &> out.log

— by Janos on June 5, 2012, 11:34 a.m.

Explanation

Note: this is not always the same as:

do_something.sh >out.log 2>out.log

Using &> will ensure that the order of lines coming from stdout and stderr is preserved.

Limitations

This works in /bin/bash and may not work in /bin/sh or other stripped down variants of bash.

0

Create or mount an encrypted directory using encfs

 $ encfs -i 5 $PWD/raw $PWD/content

— by Janos on May 22, 2012, 12:21 a.m.

Explanation

The first time you run this, it will create an encrypted directory raw in the current working directory, and mount it on directory content in the same directory. It will ask a couple of questions:

  1. Create directory "raw" ? -- answer "y"
  2. Create directory "content" ? -- answer "y"
  3. Choose configuration mode -- press enter, or "p" for "paranoia mode"

The content directory will be visible and editable only by your user, and all the files you put inside will be saved encrypted under raw. The encrypted data is only visible when raw is mounted somewhere.

The -i 5 means the content directory will be automatically unmounted after being idle for 5 minutes.

You can manually unmount with umount ./content in Linux, and fusermount -u ./content in Mac OS.

To mount raw again, simply re-run the same command. This time it will only ask for the password.

Limitations

encfs is not a standard command/package. Major Linux distros have it in their repos, on Mac OS you can install using ports. (sudo port install encfs)

1

Faster disk imaging with dd

 $ dd if=/dev/sda bs=$(hdparm -i /dev/sda | grep BuffSize | cut -d ' ' -f 3 | tr [:lower:] [:upper:] | tr -d BUFFSIZE=,) conv=noerror | dd of=image.dd conv=noerror

— by austindcc on May 19, 2012, 3:28 a.m.

Explanation

GNU dd (disk dump) copies any block device to another block device or file. It's really useful for disk cloning, but its usual invocation isn't as fast as it could be. These settings, or settings like them, often improve copying speed by more than double.

  • Piping the input of dd into the output of another instance seems to always improve copying speed.
  • /dev/sda refers to your input device, which may vary. Check yours with fdisk -l.
  • image.dd refers to the copy stored in the current working directory. You can also use another block device, such as /dev/sdb. WARNING! Be sure you know what you set the output file to! A mistake here could do irreparable damage to your system.
  • The entire hdparm subshell sets dd's input block size to the buffer size of the source medium. This also usually improves copy speed, but may need adjustment (see limitations below).
  • conv=noerror tells dd to ignore read errors.

Check dd's progress with: kill -USR1 $(pidof dd)

Limitations

The hdparm subshell is not appropriate for block devices without buffers, like flash drives. Try block sizes from 512 bytes to 1 or 2MiB to get the best speed. dd usually requires root privileges to run, because it is very powerful and dangerous, and will not prompt when overwriting!. If you're not careful where dd outputs, you may permanently destroy all or part of your system. Use with care; double-check all parameters, especially the of file/device!

0

Run a never-ending process in the background in a way that you can check on its status anytime

 $ screen -d -m -S some_name ping my_router

— by Janos on May 17, 2012, 11:03 p.m.

Explanation

  • -d -m start screen in "detached" mode. This creates a new session but doesn't attach to it.
  • -S some_name gives the screen session a label, so you can easily re-attach to it with screen -R some_name
  • The shell prompt returns immediately, and of course you can logout too, the screen session will continue to run happily.
  • When the specified program exits (in this example ping), its screen window will also exit, and if there are no other windows in the screen session, then the screen session itself will also exit.

0

Make a hexdump or do the reverse with the xxd command

 $ xxd /path/to/binary/file

— by Janos on May 16, 2012, 10:22 a.m.

Explanation

This shows a very nice hexdump of the specified file. You can edit the output and convert it back to binary with xxd -r. But the best part is that you can configure vim to do all the converting back and forth for you, effectively turning vim into a binary editor, by adding this to your .vimrc:

augroup Binary
    au!
    au BufReadPre  *.bin let &bin=1
    au BufReadPost *.bin if &bin | %!xxd
    au BufReadPost *.bin set ft=xxd | endif
    au BufWritePre *.bin if &bin | %!xxd -r
    au BufWritePre *.bin endif
    au BufWritePost *.bin if &bin | %!xxd
    au BufWritePost *.bin set nomod | endif
augroup END

This will work for editing .bin files. To use it for other file extensions too, duplicate the lines within augroup and replace *.bin with *.exe for example.

This tip is from vim's :help xxd.

0

Really lazy way to print the first instance of $foo that occurs after $bar

 $ ifconfig | grep ^en1 -A5 | grep inet | head -n 1

— by Janos on May 12, 2012, 12:30 p.m.

Explanation

This is just for the sake of an example of finding $foo that occurs after $bar. Substitute ifconfig and the arguments of grep appropriately for your use case.

  • In the output of ifconfig there are several lines with inet. We want to get to the first one that comes after a line starting with en1
  • grep ^en1 -A5 will print the line starting with en1 and the next 5 lines that follow it
  • grep inet will print only the lines matching inet
  • head -n 1 will print only the first line

The value 5 in -A5 is really just a guess that the line we're interested in will be within the next 5 lines, the appropriate number depends on your use case.

Kind of a dumb technique, but it's easy to remember.

0

Print the first instance of $foo that occurs after $bar

 $ sed -n '\@interface Ethernet3/1@,/!/ s/ip address/&/p' file...

— by Anon5DuJaBeh on May 12, 2012, 6:27 a.m.

Explanation

Should have realized this can be done with sed, too - it's even shorter!

-n suppresses the normal printing of lines.

\@interface Ethernet3/1@,/!/ specifies the beginning and end of the stanza we want to work on, and addresses all lines in between. Because the string we want to match at the beginning contains a slash, we need to designate a different character as the delimiter for the regular expression (@ in this case - note the backslash before the first one). We are OK using slashes for the ending regex, which matches an exclamation mark.

While in the stanza of interest, we slightly abuse the s function with s/ip address/&/p. This matches the text "ip address" and replaces it with... exactly the same thing (& just represents the matched text). The reason we bother doing this is to take advantage of the p flag, which will print any line where a replacement has been made.

The result is that only lines which match "ip address" will be printed, and only when we are in the stanza of interest.

Limitations

sed is a little less flexible - it uses basic regular expressions while awk uses extended regular expressions. In addition, awk has a fuller set of functions and operators available, which can be used to create a wider variety of tests and actions.