We collect practical, well-explained Bash one-liners, and promote best practices in Bash shell scripting. To get the latest Bash one-liners, follow @bashoneliners on Twitter. If you find any problems, report a bug on GitHub.



Calculate an h index from an EndNote export

 $ MAX=$(NUM=1;cat author.xml |perl -p -e 's/(Times Cited)/\n$1/g'|grep "Times Cited" |perl -p -e 's/^Times Cited:([0-9]*).*$/$1/g'|sort -nr | while read LINE; do if [ $LINE -ge $NUM ]; then echo "$NUM"; fi; NUM=$[$NUM+1]; done;); echo "$MAX"|tail -1

— by openiduser14 on Feb. 4, 2012, 1:06 a.m.


EndNote?! I know but sometimes we have windows users as friends


Cut select pages from a pdf file and create a new file from those pages.

 $  pdftk input.pdf cat 2-4 7 9-10 output output.pdf

— by mmaki on Feb. 3, 2012, 6:50 a.m.


pdftk is the PDF Toolkit

input.pdf is the input file.

cat 2-4 7 9-10 concatenate (combine) pages 2,3,4,7,9,10 of input.pdf.

output output.pdf the resulting pdf file containing the above pages.


Re-compress a gzip (.gz) file to a bzip2 (.bz2) file

 $ time gzip -cd file1.tar.gz 2>~/logfile.txt | pv -t -r -b -W -i 5 -B 8M | bzip2 > file1.tar.bz2 2>>~/logfile .txt

— by DAVEB on Feb. 1, 2012, 6:02 p.m.


*Requires PV (pipe viewer) if you want to monitor throughput; otherwise you can leave out the pv pipe.

Transparently decompresses an arbitrary .gz file (does not have to be a tar) and re-compresses it to bzip2, which has better compression and error recovery. Echoes error messages to a file named logfile.txt in your home directory.

NOTE: The original .gz file will NOT be deleted. If you want to save space, you will have to delete it manually.


Test your hard drive speed

 $ time (dd if=/dev/zero of=zerofile bs=1M count=500;sync);rm zerofile

— by DAVEB on Feb. 1, 2012, 5:35 p.m.


Creates a 500MB blank file and times how long it takes to finish writing the entire thing to disk (sync)

time the entire dd + sync operation, and then remove the temporary file


Works with Bash; not tested in other environments


Recursively remove all empty sub-directories from a directory tree

 $ find . -depth  -type d  -empty -exec rmdir {} \;

— by openiduser16 on Jan. 31, 2012, 11:15 p.m.


Recursively remove all empty sub-directories from a directory tree using just find. No need for tac (-depth does that), no need for xargs as the directory contents changes on each call to rmdir. We're not reliant on the rmdir command deleting just empty dirs, -empty does that.


Will make many calls to rmdir without using xargs, which bunches commands into one argument string, which is normally useful, but -empty /could/ end up being more efficient since only empty dirs will be passed to rmdir, so possibly fewer executions in most cases (searching / for example).


Group count sort a log file

 $ A=$(FILE=/var/log/myfile.log; cat $FILE | perl -p -e 's/.*,([A-Z]+)[\:\+].*/$1/g' | sort -u | while read LINE; do grep "$LINE" $FILE | wc -l | perl -p -e 's/[^0-9]+//g'; echo -e "\t$LINE"; done;);echo "$A"|sort -nr

— by openiduser14 on Jan. 31, 2012, 6:49 p.m.


  • BASH: a temp var for the last sort: $A=$(
  • the file you want: FILE=/var/log/myfile.log
  • dump the file to a stream: cat $FILE |
  • cut out the bits you want to count: perl -p -e 's/.*,([A-Z]+)[\:\+].*/$1/g' |
  • get a unique list: sort -u |
  • for each line/value in the stream do stuff: while read LINE; do
  • dump all lines matching the current value to an inner stream: grep "$LINE" $FILE |
  • count them: wc -l |
  • clean up the output of wc and drop the value on stdout: perl -p -e 's/[^0-9]+//g';
  • drop the current value to stdout: echo -e "\t$LINE";
  • finish per value operations on the outer stream: done;
  • finish output to the temp var: );
  • dump the temp var to a pipe: echo "$A" |
  • sort the list numerically in reverse: sort -nr


Use ghostscript to shrink PDF files

 $ gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/ebook -dNOPAUSE -dQUIET -dBATCH -sOutputFile=output.pdf input.pdf

— by openiduser10 on Jan. 31, 2012, 10:43 a.m.


Replace input.pdf and output.pdf with the original PDF name and the new compressed version's file name respectively. The key to this is the PDFSETTINGS option which can be tuned for various levels of compression. For scanned text documents, I find the ebook setting works well enough for most purposes but you can experiment with the options below.

  • -dPDFSETTINGS=/screen (screen-view-only quality, 72 dpi images)
  • -dPDFSETTINGS=/ebook (low quality, 150 dpi images)
  • -dPDFSETTINGS=/printer (high quality, 300 dpi images)
  • -dPDFSETTINGS=/prepress (high quality, color preserving, 300 dpi imgs) '-dPDFSETTINGS=/default (almost identical to /screen)'


How to find all hard links to a file

 $ find /home -xdev -samefile file1

— by openiduser7 on Jan. 30, 2012, 8:56 p.m.


Note: replace /home with the location you want to search

Source: http://linuxcommando.blogspot.com/2008/09/how-to-find-and-delete-all-hard-links.html


Find all the unique 4-letter words in a text

 $ cat ipsum.txt | perl -ne 'print map("$_\n", m/\w+/g);' | tr A-Z a-z | sort | uniq | awk 'length($1) == 4 {print}'

— by Janos on Jan. 29, 2012, 10:28 p.m.


  • The perl regex pattern m/\w+/g will match consecutive non-word characters, resulting in a list of all the words in the source string
  • map("$_\n", @list) transforms a list, appending a new-line at the end of each element
  • tr A-Z a-z transforms uppercase letters to lowercase
  • In awk, length($1) == 4 {print} means: for lines matching the filter condition "length of the first column is 4", execute the block of code, in this case simply print


Concatenate two or more movie files into one using mencoder

 $ mencoder cd1.avi cd2.avi -o movie.avi -ovc copy -oac copy

— by Janos on Dec. 24, 2011, 3:51 p.m.


  • You can specify as many files as you want on the command line to process them in sequence.
  • -ovc copy simply means to copy the video exactly
  • -oac copy simply means to copy the audio exactly
  • -o movie.avi is the output file, with all the source files concatenated


  • mencoder is usually not a standard package
  • mencoder may be in the same package as mplayer, or maybe not
  • mencoder has binary packages for Linux, Mac and Windows

See the MPlayer homepage for more info: http://www.mplayerhq.hu/


Calculate the average execution time (of short running scripts) with awk

 $ for i in {1..10}; do time some_script.sh; done 2>&1 | grep ^real | sed -e s/.*m// | awk '{sum += $1} END {print sum / NR}'

— by Janos on Dec. 21, 2011, 8:50 a.m.


  • The for loop runs some_script.sh 10 times, measuring its execution time with time
  • The stderr of the for loop is redirected to stdout, this is to capture the output of time so we can grep it
  • grep ^real is to get only the lines starting with "real" in the output of time
  • sed is to delete the beginning of the line up to minutes part (in the output of time)
  • For each line, awk adds to the sum, so that in the end it can output the average, which is the total sum, divided by the number of input records (= NR)


The snippet assumes that the running time of some_script.sh is less than 1 minute, otherwise it won't work at all. Depending on your system, the time builtin might work differently. Another alternative is to use the time command /usr/bin/time instead of the bash builtin.


Rotate a movie file with mencoder

 $ mencoder video.avi -o rotated-right.avi -oac copy -ovc lavc -vf rotate=1

— by Janos on Dec. 2, 2011, 10:30 p.m.


mencoder is part of mplayer.

Other possible values of the rotate parameter:

  • 0: Rotate by 90 degrees clockwise and flip (default).
  • 1: Rotate by 90 degrees clockwise.
  • 2: Rotate by 90 degrees counterclockwise.
  • 3: Rotate by 90 degrees counterclockwise and flip.


View specific column of data from a large file with long lines

 $ cat /tmp/log.data |colrm 1 155|colrm 60 300

— by versorge on Oct. 4, 2011, 9:55 p.m.


  • cat: prints the file to standard output
  • colrm: removes selected columns from a file


Replace a regexp pattern in many files at once

 $ vi +'bufdo %s/pattern/replacement/g | update' +q $(grep -rl pattern /path/to/dir)

— by Janos on Sept. 15, 2011, 11:50 p.m.


  • The inner grep will search recursively in specified directory and print the filenames that contain the pattern.
  • All files will be opened in vi, one buffer per file.
  • The arguments starting with + will be executed as vi commands:
    • bufdo %s/pattern/replacement/g | update = perform the pattern substitution in all files and save the changes
    • q = exit vi


The :bufdo command might not be there in old versions of vim.


Delete all untagged Docker images

 $ docker rmi $(docker images -f "dangling=true" -q)

— by stefanobaghino on April 27, 2018, 2:50 p.m.


docker images outputs all images currently available. By specifying -f "dangling=true" we restrict the list to "dangling" images (i.e. untagged). By specifying the -q option we use quiet mode, which limits the output to the images hash, which is the directly fed into docker rmi, which removes the images with the corresponding hashes.


Get a free shell account on a community server

 $ sh <(curl hashbang.sh | gpg)

— by lrvick on March 15, 2015, 9:49 a.m.


Bash process substitution which curls the website 'hashbang.sh' and executes the shell script embedded in the page.

This is obviously not the most secure way to run something like this, and we will scold you if you try.

The smarter way would be:

Download locally over SSL

curl https://hashbang.sh >> hashbang.sh

Verify integrity with GPG (if available):

gpg --recv-keys 0xD2C4C74D8FAA96F5
gpg --verify hashbang.sh

Inspect source code:

less hashbang.sh


chmod +x hashbang.sh


Run a local shell script on a remote server without copying it there

 $ ssh user@server bash < /path/to/local/script.sh

— by Janos on June 21, 2012, 12:06 a.m.


Yes this is almost trivial: a simple input redirection, from a local shell script to be executed by bash on the remote server.

The important point being, if you have a complex and very long chain of commands to run on a remote server, it is better to put the commands in a shell script, break the long one-liner to multiple lines for readability and easier debugging.

Replace bash accordingly depending on the language of the script, for example for python:

ssh user@server python < /path/to/local/script.py


Find all log file which modified 24 hours ago and compress into zip file

 $ find . -type f -mtime +1 -name "*.log" -exec zip -m {}.zip {} \; >/dev/null &

— by TrongTan124 on Nov. 9, 2018, 10:04 a.m.


-type f: only file

-mtime +n: File's data was last modified n*24 hours ago

-name "*.log": file have extend .log, can replace other word

zip -m {}.zip: compress all file into zip

/dev/null &: skipping print screen.


Below is an Unix command to list all the IP addresses connected to your server on port 80.

 $ netstat -tn 2>/dev/null | grep :80 | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr | head

— by Goeks1 on Sept. 26, 2018, 11:10 p.m.


This command is useful to detect if your server is under attack, and null route those IPs. Read this null route attacker IP story.



Source: https://www.mkyong.com/linux/list-all-ip-addresses-connected-to-your-server/


I think netstat is not a default package on Debian Stretch. You have to install net-tools

apt-get install net-tools


Random 6-digit number

 $ python -c 'import random; print(random.randint(0,1000000-1))'

— by johntellsall on Sept. 19, 2018, 10:42 p.m.


Stackoverflow has a dozen different ways to generate numbers, all of which fall apart after 3-4 digits. This solution requires Python to be installed, but is simple and direct.


Requires Python, it's not a pure-Bash solution


Very fast history search with Ripgrep

 $ rh() { rg "$1" ~/.bash_history }

— by johntellsall on Sept. 18, 2018, 7 p.m.


I search my history a lot. At 65k lines, this can take a while. Recently I've installed Ripgrep, which is much faster than my previous favorite Ack, and faster than good old Grep/Egrep.

After defining the above function, history searches are much faster!

Example: I forgot how to hit my service's endpoint. Instead of grepping through the Bash history file, I use my new Ripgrep-powered function:

rh curl.*health


This function only works if you have Ripgrep installed

It's not really needed unless you have "infinite history" turned on.


While loop to pretty print system load (1, 5 & 15 minutes)

 $ while :; do date; awk '{printf "1 minute load: %.2f\n", $1; printf "5 minute load: %.2f\n", $2; printf "15 minute load: %.2f\n", $3}' /proc/loadavg; sleep 3; done

— by Janos on Sept. 5, 2018, 8:41 p.m.


while :; do ...; done is an infinite loop. You can interrupt it with Control-c.

The file /proc/loadavg contains a line like this:

0.01 0.04 0.07 1/917 25383

Where the first 3 columns represent the 1, 5, 15 minute loads, respectively.

In the infinite loop we print the current date, then the load values nicely formatted (ignore the other values), then sleep for 3 seconds, and start again.


/proc/loadavg is only available in Linux.


Scan entire Git repos for dangerous Amazon Web Service IDs

 $ git ls-tree --full-tree -r --name-only HEAD | xargs egrep -w '[A-Z0-9]{20}'

— by johntellsall on Aug. 31, 2018, 10:29 p.m.


Letting your AWS credentials escape is very dangerous! This simple tool makes sure none of your secrets make into version control and therefore out into the hands of evil robots.

Use Git to quickly list all files in the repos. Then, take this list and search for things that look like AWS IDs: a 20-character uppercase word.


While loop to pretty print system load (1, 5 & 15 minutes)

 $ while [ 1 == 1 ]; do  cat /proc/loadavg | awk '{printf "1 minute load: %.2f\n", $(NF-5)}' && cat /proc/loadavg |awk '{printf "5 minute load: %.2f\n", $(NF-3)}' && cat /proc/loadavg |awk '{printf "15 minute load: %.2f\n", $(NF-2)}'; sleep 3; date; done

— by peek2much3 on Aug. 30, 2018, 8:54 a.m.


top is great but this will make it easier to read and makes it easy to pipe to text file for historical review. kill with ctrl+c


Dump all AWS IAM users/roles to a Terraform file for editing / reusing in another environment

 $ echo iamg iamgm iamgp iamip iamp iampa iamr iamrp iamu iamup | AWS_PROFILE=myprofile xargs -n1  terraforming

— by johntellsall on Aug. 28, 2018, 12:38 a.m.


Amazon Web Services (AWS) use a collection "IAM" resources to create Users and related objects in the system. This oneliner scrapes all the relevant info and puts it into Terraform. This lets us audit our users-groups. And, it lets us re-use them in another environment!