We collect practical, well-explained Bash one-liners, and promote best practices in Bash shell scripting. To get the latest Bash one-liners, follow @bashoneliners on Twitter. If you find any problems, report a bug on GitHub.

Tags

1

Replace the header of all files found.

 $ find . -type f -name '*.html' -exec sed -i -e '1r common_header' -e '1,/STRING/d' {} \;

— by jam on Oct. 25, 2012, 9:29 a.m.

Explanation

Replaces the lines from 1 to the first occurrence of a line starting with STRING of every file found with find.

  • find . -type f -name '*.html' returns a list of all files (not including directories) ending with .html. The ' ' in name is used to pass the literal wildcard * to the find command, instead of the * interpretation of bash, that is, repeat the command for every file in the current folder.
  • -exec execute the following command in each of the files found, using {} as the filename. the ; termination must be escaped with \;
  • sed -i replaces in file (output is the same file)
  • -e '1r common_header' -e '1,/STRING/d' {} reads common_header file, then finds the first occurrence of STRING and replaces it, deleting all the previous lines and putting the contents of common_header.

Limitations

The -i flag of sed requires a parameter in BSD sed, used as the suffix of a backup file. To not use a backup file, you can pass an empty value with -i''.

0

Remove EXIF data such as orientation from images

 $ mogrify -strip /path/to/image.jpg

— by Janos on Oct. 24, 2012, 12:08 a.m.

Explanation

I use this mostly to remove orientation information from images. My problem with orientation information is that some viewers don't support it, and thus do not show the image correctly oriented. Rotating the image doesn't help, because if I make the image look correct in the viewer that doesn't support orientation, that will break it in the viewer that does support orientation. The solution is to remove the orientation information and rotate the image appropriately. That way the image will always look the same in all viewers, regardless of support for the orientation information.

The tool mogrify is part of ImageMagick, an image manipulation software. It manipulates image files and saves the result in the same file. A similar tool in ImageMagick that saves the result of manipulations is convert, you can use it like this:

convert -strip orig.jpg stripped.jpg

Limitations

The tool is part of ImageMagick, an image manipulation software.

0

Get the last modification date of a file in any format you want

 $ date -r /etc/motd +%Y%m%d_%H%M%S

— by Janos on Oct. 17, 2012, 4:42 p.m.

Explanation

The -r flag is a shortcut of --reference and it is used to specify a reference file. Used in this way, the date command prints the last modification date of the specified file, instead of the current date.

The + controls the output format, for example:

  • %Y = 4-digit year
  • %m = 2-digit month
  • %d = 2-digit day
  • %H = 2-digit hour
  • %M = 2-digit minutes
  • %S = 2-digit seconds

So in this example +%Y%m%d_%H%M%S becomes 20121001_171233

You should be able to find all the possible format specifiers in man date.

Limitations

The default date command in Solaris does not support the --reference flag. Modern Solaris systems have the GNU tools installed, so you may be able to find the GNU implementation of date which supports this flag. Look for it in /usr/gnu/bin/date or /usr/local/bin/date, or do search the entire /usr with find /usr -name date.

In Solaris this may be a suitable substitute without using the date command:

ls -Ego /etc/motd | awk '{print $4 "_" $5}' | tr -d :- | sed -e 's/\..*//'

Or you can use good old perl:

perl -mPOSIX -e 'print POSIX::strftime("%Y%m%d_%H%M%S\n", localtime((stat("/etc/motd"))[9]))'

0

Forget all remembered path locations

 $ hash -r

— by Janos on Oct. 14, 2012, 9:46 a.m.

Explanation

bash remembers the full path name of each command you enter, so it doesn't have to lookup in $PATH every single time you run the same thing. It also counts the number of times you used each command in the current session, you can see the list with hash.

Anyway, this behavior can poses a small problem when you reinstall an application at a different path. For example you reinstall a program that used to be in /usr/local/bin and now it is in /opt/local/bin. The problem is that if you used that command in the current shell session, then bash will remember the original location, which of course doesn't work anymore. To fix that, you can either run hash cmd which will lookup the command again, or run hash -r to forget all remembered locations (less efficient, but maybe faster to type ;-)

For more details, see help hash

0

Rename files with numeric padding

 $ perl -e 'for (@ARGV) { $o = $_; s/\d+/sprintf("%04d", $&)/e; print qq{mv "$o" "$_"\n}}'

— by Janos on Oct. 6, 2012, 1:38 p.m.

Explanation

Basically a one-liner perl script. Specify the files to rename as command line parameters, for example:

perl -e '.....' file1.jpg file2.jpg

In this example the files will be renamed to file0001.jpg and file0002.jpg, respectively. The script does not actually rename anything. It only prints the shell commands to execute that would perform the renaming. This way you can check first that the script would do, and if you want to actually do it, then pipe the output to sh like this:

perl -e '.....' file1.jpg file2.jpg | sh

What's happening in the one-liner perl script:

  • for (@ARGV) { ... } is a loop, where each command line argument is substituted into the auto-variable $_.
  • $o = $_ :: save the original filename
  • s/// :: perform pattern matching and replacement on $_
  • print qq{...} :: print the mv command, with correctly quoted arguments

Limitations

The script does not cover all corner cases. For example it will not work with files that have double-quotes in their names. In any case, it is safe to review the output of the script first before piping it to sh.

If your system has the rename command (Linux), then a shortcut to do the exact same thing is with:

rename 's/\d+/sprintf("%04d", $&)/e' *.jpg

It handles special characters better too.

0

Copy or create files with specific permissions and ownership

 $ install -b -m 600 /dev/null NEWFILE

— by Janos on Sept. 25, 2012, 2:20 p.m.

Explanation

This example creates a new (empty) file with permissions 600. You could also specify the owner and group using the -o and -g flags respectively.

Although you could accomplish the same for example by creating the file with touch and then change permissions with chmod and chown, or use umask to control the permissions of newly created files, those methods take multiple steps, while with install it is a single step.

You can also use install to copy multiple files to a directory with specified permissions like this:

install -m 600 -o jack -g wheel file1 file2 /path/to/existing/dir

1

Redirect stdout to a file you don't have write permission on

 $ echo hello | sudo tee -a /path/to/file

— by Janos on Sept. 11, 2012, 9:24 a.m.

Explanation

  • The tee command copies standard input to standard output, making a copy in zero or more files.
  • If the -a flag is specified it appends instead of overwriting.
  • Calling tee with sudo makes it possible to write to files the current user has no permission to but root does.

1

`tail -f` a file until text is seen

 $ tail -f /path/to/file.log | sed '/^Finished: SUCCESS$/ q'

— by Janos on Aug. 22, 2012, 8:29 a.m.

Explanation

tail -f until this exact line is seen:

Finished: SUCCESS

The exit condition does not have to be an exact line, it could just well be a simple pattern:

... | sed '/Finished/ q'

0

Run command multiple times with a for loop and a sequence expression

 $ for i in {1..10}; do date; sleep 1; done

— by Janos on Aug. 19, 2012, 9:27 a.m.

Explanation

This is just a regular for loop with a sequence expression. A sequence expression has the form {x..y[..incr]}, where x and y are either integers or single characters, and incr an optional increment.

More examples:

  • {a..f} = a b c d e f
  • {a..f..2} = a c e
  • {0..1}{0..1} = 00 01 10 11

Limitations

Don't try a large range like {1..10000000000000000}, it may render your computer unusable until killed.

1

Recording SSH sessions

 $ ssh -l USER HOST | tee -a /path/to/file

— by LeandroToledo on Aug. 15, 2012, 5:04 p.m.

Explanation

tee is a command which displays or pipes the output of a command and copies it into a file or a variable.

The -a option appends the output to the end of file instead of writing over it.

You can also create an alias in ~/.bashrc to record your session when using ssh:

function sshlog () { \ssh $@ 2>&1 | tee -a $(date +%Y%m%d).log; }
alias ssh=sshlog

0

Clear the swap space forcing everything back to main memory in Linux

 $ sudo swapoff -a; sudo swapon -a

— by Janos on Aug. 14, 2012, 11:21 a.m.

Explanation

Note: if you don't have enough main memory the swapoff will fail.

Limitations

This works only in Linux.

0

Redirection operator to override the noclobber option

 $ some_command >| output.txt

— by Janos on Aug. 11, 2012, 9:21 a.m.

Explanation

Normally the > operator overwrites the target file.

If the noclobber option is set (using: set -o noclobber), the > operator will fail if the target file exists.

The >| overrides the noclobber setting and overwrites the target file.

If the noclobber option is not set, then >| is equivalent to >, naturally.

0

How to set the ip address in Solaris 11

 $ ipadm create-addr -T static -a 192.168.1.10/24 eth0/staticaddr

— by Janos on Aug. 3, 2012, 11:44 a.m.

Explanation

  • eth0 is the name of the network interface
  • ipadm show-if shows the list of network interfaces
  • staticaddr is a name you can choose

More details here: http://docs.oracle.com/cd/E19963-01/html/821-1458/gjwiq.html

1

Record audio from microphone or sound input from the console

 $ sox -t ossdsp -w -s -r 44100 -c 2 /dev/dsp -t raw - | lame -x -m s - File.mp3

— by Kleper on July 28, 2012, 8:55 p.m.

Explanation

sox is a software that lets you connect directly to the sound card and send what passes for it in raw format and the system memory through the concatenation of the linux command we can make real-time audio processing is generated by licks and be converted to mp3.

Limitations

Requires a plugin for alsa oss to run on modern distributions.

1

Use vim to pretty-print code with syntax highlighting

 $ vim +'hardcopy > output.ps' +q style.css 

— by Janos on July 21, 2012, 12:13 a.m.

Explanation

If you have syntax highlighting properly setup in vim, this command will pretty-print the specified file with syntax highlighting to output.ps.

If you prefer PDF, you can convert using ps2pdf output.ps.

1

Log and verify files received via FTP

 $ for i in $(cat /var/log/vsftpd.log | grep $DATE_TIME | grep UPLOAD | grep OK); do ls /FTP/HOME/$i >> /dev/null 2> \&1; if \[ $? = 0 \]; then echo "$i" >> $FILES_OK_UPLOADS.log; else  echo "$DATE ERROR: File $i not found" >> $FTP_FILES_NOTOK_$DATE_TIME.log; fi; done

— by dark_axl on July 10, 2012, 8:54 p.m.

Explanation

This one-liner checks and validates the received files via ftp, and generates a log of these files. To have a record of files received and be able to process, based on the successful transfer and the existence of the files.

1

Edit the Gimp launcher file to disable the splash screen

 $ printf '%s\n' ',s/^Exec=[^ ]*/& -s/' w q | ed /usr/share/applications/gimp.desktop

— by Anon8yhYNaVe on July 1, 2012, 12:57 a.m.

Explanation

sed is designed for editing streams - editing files is what ed is for! You can get consistent behavior on any UNIX platform with the above one-liner.

The printf command sends a series of editing commands to ed, each separated by a newline. In this case, the substitution command ,s/^Exec=[^ ]*/& -s/ is nearly the same as in sed, appending a space and a -s to the line starting with Exec=. The only difference is the comma at the beginning designating the lines to operate on. This is shorthand for 1,$, which tells ed to apply the command to the first through the last lines (i.e., the entire file). w tells ed to write the file, and q to quit.

0

Edit the Gimp launcher file to disable the splash screen

 $ sudo sed -i 's/^Exec=[^ ]*/& -s/' /usr/share/applications/gimp.desktop

— by Janos on June 30, 2012, 9:06 p.m.

Explanation

  • The -i flag of sed means to perform the command "in place", that is, save any changes in the input file. Use this flag with extreme caution, one wrong move and you can completely break the original file.
  • The regex pattern /^Exec=[^ ]*/ will match the line starting with Exec= followed by zero or more non-space characters.
  • In the replacement string, & -s, the & is replaced with whatever was matched, in this example probably something like Exec=gimp-2.8, after which we add a space and the -s flag which will effectively disable the splash screen when starting Gimp.

Limitations

The -i flag of sed works differently in GNU and BSD systems. This example works in GNU systems only. The equivalent in BSD is:

sudo sed -i '' 's/^Exec=[^ ]*/& -s/' /usr/share/applications/gimp.desktop

In any case, always be very careful when using the -i flag of sed.

0

`less` is more convenient with the `-F` flag

 $ less -F FILE1

— by Janos on June 25, 2012, 6:47 p.m.

Explanation

less is a "pager" program like more, with a lot of added features. By default, to exit less you have to press q. This can be annoying when viewing a small file that would fit on the screen.

The -F flag to the rescue! When started with the -F flag, less will quit if the entire input (whether from stdin or a file) fits on a single screen.

It has no effect whatsoever for longer input, so it is safe to add an alias for this:

alias less='less -F'

0

Append to a file text, a blank line, and the last line of another file

 $ { echo some text; echo; tail -n1 /var/log/apache2/error.log; } >> /path/to/file

— by Janos on June 22, 2012, 5:29 p.m.

Explanation

All the standard output from all the commands between the braces will be redirected.

0

Append to a file text, a blank line, and the last line of another file

 $ echo -e "From: me\n\n$(tail -n1 /var/log/apache2/error.log)" >> file

— by kevin on June 21, 2012, 8:18 p.m.

Explanation

  • -e option to echo makes it interpret '\n' as a newline
  • $(command) syntax runs a command, then uses its output in place

Limitations

The -e flag of echo doesn't work on all systems. In that case you can use printf instead.

-1

Run a local shell script on a remote server without copying it there

 $ ssh user@server bash < /path/to/local/script.sh

— by Janos on June 21, 2012, 12:06 a.m.

Explanation

Yes this is almost trivial: a simple input redirection, from a local shell script to be executed by bash on the remote server.

The important point being, if you have a complex and very long chain of commands to run on a remote server, it is better to put the commands in a shell script, break the long one-liner to multiple lines for readability and easier debugging.

Replace bash accordingly depending on the language of the script, for example for python:

ssh user@server python < /path/to/local/script.py

0

Convert a list of terms in slug format to capitalized words

 $ sed -e 's/^./\U&/' -e 's/_./\U&/g' -e 's/_/ /g' /path/to/input

— by Janos on June 17, 2012, 7:54 a.m.

Explanation

The goal here is to take an input like this:

police_station
post_office
real_estate_agency

... and convert it to an output like this:

Police Station
Post Office
Real Estate Agency
  • -e ... the sed command can take several -e parameters, which will be executed one by one when processing each line in the input
  • The s/// command is a pattern replacement, and has the general format s/pattern/replacement/flags
  • s/^./\U&/ - replace the first letter of the line with uppercase version of the letter: \U means convert to uppercase, & is the matched string
  • s/_./\U&/g- replace _ and any letter followed by it. The g flag at the end means a "global" replacement, so all occurrences of the pattern _. will be replaced
  • s/_/ /g - replace all underscores with spaces
  • Input to sed can come from a list of files, or input redirection with <, or from a pipe.

0

Execute different commands with find depending on file type

 $ find /path/to/dir -type d -exec chmod 0755 '{}' \; -o -type f -exec chmod 0644 '{}' \;

— by Janos on June 17, 2012, 12:01 a.m.

Explanation

  • -type d -exec chmod 0755 '{}' \; for each directory, run chmod 0755
  • \; is to mark the end of the -exec
  • {} is replaced with the filename, we enclosed it in single quotes like this '{}' to handle spaces in filenames
  • -ological OR operator
  • -type f -exec chmod 0644 '{}' \; for each regular file, run chmod 0644

0

Convert m4a files to mp3 using faad and lame

 $ faad -o tmp.wav music.m4a && lame -b 192 tmp.wav music.mp3

— by Janos on June 14, 2012, 9:29 a.m.

Explanation

  • Step 1: convert m4a to wav using faad
  • Step 2: convert wav to mp3 using lame
  • -b 192 is the bitrate

Limitations

Neither faad nor lame are standard commands.