bashoneliners.com
Welcome to bashoneliners.com, a place for practical or just pure awesome bash one-liners or shell script tips and tricks for GNU Linux, UNIX or BSD systems. You can post a bash one-liner to share it with the world, or browse or search the one-liners contributed by others.
You can follow the latest bash one-liners via rss feeds or on twitter @bashoneliners.
Tags
One-Liners
Check that a directory is a parent of another
$ is_parent() { [[ "$2" =~ ${1}['/'?] ]] && return 0 || return 1;}
Explanation
The function expanded would look like this :
T() {
if [[ "$2" =~ ${1}['/'?] ]] ; then
echo "$2 is child of $1"
return 0
else
echo "$2 is NOT child of $1 ($?)"
return 1
fi
}
Limitations
I'd like to know if this works in all cases :).
Creates fattal tone mapped images from a directory of raw images
$ for img in /path/to/rawimages/*.RW2; do pfsin ${img} | pfssize -x 1024 -y 768 | pfstmo_fattal02 -v -s 1 | pfsout /path/to/finished/${img%%}.jpg; done
Explanation
for img in /path/to/rawimages/*.RW2; do Loop through image directory
pfsin ${img} | read the raw image
pfssize -x 1024 -y 768 | resize it to 1024x768 because fattal looks better at low resolutions
pfstmo_fattal02 -v -s 1 | use the fattal tone mapping operator, be verbose and saturation value of 1
pfsout ./path/to/finished/${img%%}.jpg; done output and rename the file as a jpg.
Examples of fattal tone mapped images http://goo.gl/IayQQ
pfstools website http://pfstools.sourceforge.net/
Limitations
Portrait orientation images need to be processed -x 768 -y 1024
Send a file by email as attachment
$ uuencode /var/log/messages messages.txt | mailx -s "/var/log/messages on $HOST" me@example.com
Explanation
uuencode /var/log/messages messages.txt-- the first parameter is the file to attach, the second is the filename to use for the attachmentmailx -s subject emailaddress-- takes standard input as the content of the email
Calculate md5sum from an input string
$ md5sum<<<YOUR_TEXT | cut -f1 -d' ';
Explanation
Will calculate a md5sum from an input string
Wrap it up in a function:
function md5() { md5sum<<<$1 | cut -f1 -d' '; }
Example usage:
md5 "this is a long string test_string"
md5 singleWordExample
Convert a music file (mp3) to a mp4 video with a static image
$ ffmpeg -loop_input -i cover.jpg -i soundtrack.mp3 -shortest -acodec copy output_video.mp4;
Explanation
Can come handy when you'd like to post a song to YT or somethin' :)
Can be easily wrapped up in a function:
function mp3tovidwithimg() {
ffmpeg -loop_input -i $1 -i $2 -shortest -acodec copy $3;
}
and used like that:
mp3tovidwithimg cover.jpeg music_track.mp3 output_vid.mp4
Find all of the distinct file extensions in a folder
$ find . -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort -u;
Explanation
Will find all of the distinct file extensions in a folder hierarchy.
Originally posted at: http://stackoverflow.com/questions/1842254/how-can-i-find-all-of-the-distinct-file-extensions-in-a-folder-hierarchy
Dump network traffic with tcpdump to file with time-stamp in its the filename
$ date +"%Y-%m-%d_%H-%M-%Z" | xargs -I {} bash -c "sudo tcpdump -nq -s 0 -i eth0 -w ./dump-{}.pcap"
Explanation
will dump the traffic into a file with a time-stamp in its name. Example filename:
dump-2013-05-17_15-46-UTC.pcap
Get streamed FLV from Chrome with lsof
$ export psid=$(pgrep -f libflashplayer.so); cp /proc/$psid/fd/$(lsof -p $psid | grep eleted | awk {' print $4 '} | sed -e "s/[a-z]//g") saved.flv
Explanation
first get the process id of the chome browser process which runs the flashplayer export it to a variable to be used later. Then we get in subshell the filedescriptor which is marked deleted and construct the /proc path for the in memory fileimage and copy it to the file named saved.flv
Limitations
IMPORTANT: only one video should be open to play in chrome
Remove files and directories whose name is a timestamp older than a certain time
$ ls | grep '....-..-..-......' | xargs -I {} bash -c "[[ x{} < x$(date -d '3 days ago' +%Y-%m-%d-%H%M%S) ]] && rm -rfv {}"
Explanation
Suppose you have a backup directory with backup snapshots named by timestamp:
$ ls
2013-05-03-103022
2013-05-04-103033
2013-05-05-103023
2013-05-06-103040
2013-05-07-103022
You want to remove snapshots older than 3 days. The one-liner does it:
$ date
Tue May 7 13:50:57 KST 2013
$ ls | grep '....-..-..-......' | sort | xargs -I {} bash -c "[[ x{} < x$(date -d '3 days ago' +%Y-%m-%d-%H%M%S) ]] && rm -rfv {}"
removed directory: `2013-05-03-103022'
removed directory: `2013-05-04-103033'
Limitations
It doesn't work on OS X due to the differences between GNU date and BSD date).
Tree-like output in ls
$ ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'
Explanation
This one-liner initially does a recursive listing of the current directory: ls -R.
Any output other that the directory names, identified by ":" at the very end of each line (hence ":$"), is filtered out: grep ":$".
Finally there's a little of "sed" magic replacing any hierarchy level ("/") with dashes ("-"): sed ....
Works for me with Bash under Linux, Mac OS X, Solaris.
Unhide all hidden files in the current directory.
$ find . -maxdepth 1 -type f -name '\.*' | sed -e 's,^\./\.,,' | sort | xargs -iname mv .name name
Explanation
This will remove the leading dot from all files in the current directory using mv, effectively "unhiding" them.
It will not affect subdirectories.
Limitations
Probably only works on GNU Linux, due to the specific usage of xargs.
Rename all files in a directory to upper case
$ for i in `ls -1`; do mv $i "${i^^}" ; done
Explanation
using for, for looping the files through a directory and using bash build-in case modification expansion to upper case the files
An elegant way to rename all files in a directory to lowercase names
$ for i in `ls -1`; do mv $i "${i,,}" ; done
Explanation
there is a lower/upper case modification build-in bash using for, for looping files through directory
Limitations
none
Compare file owners and permissions of two directory trees
$ find /path/to/dir1 -printf "%U %G %m %p\n" > /tmp/dir1.txt
Explanation
The command simply traverses the specified directory tree and for each file and directory it prints the UID of the owner, GID of the group, the permission bits and the path.
To compare file owners and permissions of two directory trees you can run this command for each directory, save the output in two files and then compare them using diff or similar.
See man find for more explanation of all the possible symbols you can use with -printf
Limitations
The -printf option does not exist in find on Solaris 10.
Get only the latest version of a file from across mutiple directories.
$ find . -name 'filename' | xargs -r ls -tc | head -n1
Explanation
Shows latest file (by last modification of file status information) for the given pattern. So in this example filename = custlist*.xls.
We use ls to do the sorting (-t) and head to pick the top one.
xargs is given the -r option so that ls isn't run if there is no match.
Limitations
The filesystem needs to support ctime. Does not depend on a consistent naming scheme.
Sort and remove duplicate lines from two (or more files). Display only uniq lines from files.
$ uniq -u <(sort file1 file2)
Explanation
eg.
file1:
123456
234567
345678
file2:
234567
345678
456789
We want only the uniq lines from each file:
result
123456
456789
Get only the latest version of a file from across mutiple directories
$ find . -name custlist\* | perl -ne '$path = $_; s?.*/??; $name = $_; $map{$name} = $path; ++$c; END { print $map{(sort(keys(%map)))[$c-1]} }'
Explanation
The purpose of the one-liner is to find the the "latest" version of the custlist_*.xls file from among multiple versions in directories and sub-directories, for example:
./c/custlist_v1.003.xls
./c/custlist_v2.001.xls
./d/b/custlist_v1.001.xls
./d/custlist_v1.002.xls
Let's decompose the one-liner to the big steps:
find . -name custlist\*-- find the files matching the target pattern... | perl -ne '...'-- run perl, with the input wrapped around in awhileloop so that each line in the input is set in the variable$_$path = $_; s?.*/??; $name = $_;-- save the full path in$path, and cut off the subdirectory part to get to the base name of the file and save it in$name$map{$name} = $path;-- build a mapping of$nameto$path++$c;-- we count the elements, to use it later(sort(keys(%map)))[$c-1]-- sort the keys of the map, and get the last element, which iscustlist_v2.001.xlsin this exampleEND { print $map{$last} }'-- at the end of all input data, print the path of the latest version of the file
Limitations
Even if the latest version of the file appears multiple times in the directories, the one-liner will print only one of the paths. This could be fixed though if needed.
Create a thumbnail from the first page of a PDF file
$ convert -thumbnail x80 file.pdf[0] thumb.png
Explanation
convertis part ofImageMagickimage manipulation tool-thumbnail x80means create a thumbnail image of height 80 pixels, the width will be automatically chosen to make the image proportional- The
[0]is to create a thumbnail for the first page only, without that a thumbnail image would be created for each page in the pdf file
Limitations
Requires the ImageMagick image manipulation tool.
Recreate or update an existing zip file and remove files that do not exist anymore
$ zip --filesync -r /path/to/out.zip /path/to/dir
Explanation
zip does not have an explicit option to overwrite/recreate an existing zip file. If the specified destination file already exists, zip updates it. The problem is that files you did not specify to add to the zip but they already existed in the zip, will not be removed.
For example let's say you created a zip file from a directory with the command:
zip -r /path/to/out.zip /path/to/dir
Next you delete some files from the directory and repeat the command to recreate the zip. But that will not recreate the zip, it will only update it, and so the file you deleted from the directory will still be there in the zip.
One way to recreate the zip is to delete the file first. Another, better way is to use the --filesync or -FS flag. With this flag zip will remove files from the zip that do not exist anymore in the filesystem. This is more efficient than recreating the zip.
Remove offending key from known_hosts file with one swift move
$ sed -i 18d .ssh/known_hosts
Explanation
Using sed to remove a specific line
Adding -i parameter for inplace editing
How to expand a CIDR notation to its IPs
$ for j in `seq 0 255`; do for i in `seq 0 255` ; do seq -f "10.$j.$i.%g" 0 255 ; done ; done
Explanation
Using two for loop to create second & third block of IP and finally through a formatted seq to printf the output
More efficient/using less memory than using {..} (range)
Get load average in a more parse-able format
$ python -c 'import os;print os.getloadavg()[0]'
Explanation
In short, it runs a Python one-line script which imports the 'os' module and uses it to get the load average. It then indexes into a tuple, using the index of zero. The tuple is like this: (oneminuteaverage, fiveminuteaverage, fifteenminuteaverage), so you could substitute 0 for 1 to get the five minute load average, or 2 to get the fifteen minute load average. I find this really useful when writing larger bash one-liners which require this information, as I don't have to parse the output of uptime.
Limitations
Won't work unless you have Python installed.
A simple function to conveniently extract columns from an input stream
$ col() { awk '{print $('$(echo $* | sed -e s/-/NF-/g -e 's/ /),$(/g')')}'; }
Explanation
A slightly improved version of the original one-liner to allow negative indexes to extract columns relative to the end of the line, for example:
$ echo a b c | col 1 -0 -1
a c b
In this example the function expands to:
awk '{print $(1), $(NF-0), $(NF-1)}'
Show dd status every so often
$ watch --interval 5 killall dd -USR1
Explanation
Basically, it sends DD the USR1 signal, displaying status, every 5 seconds. It runs watch at an interval of 5, killing dd with -USR1. If you're running DD as root you'll have to run this as root.
Limitations
You have to run it as root if dd is running as root
Make the output of the `time` builtin easier to parse
$ TIMEFORMAT=%R
Explanation
The time builtin prints a summary of the real time, user CPU time and system CPU time spent executing commands, for example:
$ time sleep 1
real 0m1.002s
user 0m0.000s
sys 0m0.002s
If you need to parse this output, it helps to simplify it using the TIMEFORMAT variable. The value %R means "the elapsed time in seconds", for example:
$ TIMEFORMAT=%R
$ time sleep 1
1.004
The complete documentation of the format definition is in man bash, search for TIMEFORMAT.
You should <a href="http://mywiki.wooledge.org/ParsingLs">never parse ls</a>: `for i in *;` is a better approach and will not break on filenames with spaces...
The link that was stupidly removed was to the Bash wiki: mywiki.wooledge.org/ParsingLs
Is only supported in bash version 4.0