Useful Shell Commands

My own sort of commandlinefu.

• File operations
• List and find files
• Folders
• Text files
• CSVs
• Networking
• Images
• Video
• Miscellaneous

File operations

Run a command for multiple files

for i in *; do wc -l $i; done;

Batch prepend filename

for f in *.jpg; do mv "$f" "Image Foo - $f"; done

Batch rename files

for i in * ; do j=`echo $i | sed 's#searchstring#replacestring#g' - ` ; mv "$i" "$j" ; done

Batch rename part of a filename (have to install rename from brew)

rename 's/.txt/.md/i' *

Remove spaces from filenames

find . -depth -name '* *' | while IFS= read -r f ; do mv -i "$f" "$(dirname "$f")/$(basename "$f"|tr ' ' _)" ; done

Copy files by extension to new directory

ls *.jpg | xargs -n1 -i cp {} /directory

Wildcards and tokens

for file in *.src; do mv ${file} ${file%.src}.c; done

List and find files

ls by size

ls -S

ls order by last modified

ls -ltr

ls files in subdirectories

ls *

Find all files with extension

find -name "*.xcf"

Find and copy all files by extension

cp `find / -name "*.jpg"` .

Find files modified in last x mins

find / -mmin -120

Find files larger than a certain size

find /etc -size 100k

Find large directories

du -h /|grep M|sort -nr|head -15

Find total size of directory (including subdirectories)

du -ch directory

Find files owned by specific group

find -group {group_name}

Find and delete files with a specific name

find /home/userA/folderA/* -type f \( -name "delete1.txt" -or -name "delete2.txt" \) -delete

Sort directory by file size

du -h | sort -n

Print the file count per subdirectory

du -a | cut -d/ -f2 | sort | uniq -c | sort -nr

Folders

Create nested folders

mkdir -p dir1/dir2/dir3/dir4/

Create folders based on list of filenames

for file in *.mov; do mkdir -- "${file%.mov}"; mv -- "$file" "${file%.mov}"; done

Move files out of nested folders

find ~/foo/ -type f -print0 | xargs -0 mv -t ~/bar

Differences between folder contents

diff -rq folder1 folder2

Recursive difference between folder contents

diff <(cd dir1 && find | sort) <(cd dir2 && find | sort)

Delete all subdirectories with a specific name

find /home/userA/folderA/* -depth -name "match" -type d -exec rm -rfv "{}" \;

Text files

Insert header row into file

sed '1i FRUITS' file1

Remove first line of file

sed '1d' INPUT_FILE_NAME

Substitute “foo” with “bar” only for lines which contain “baz”

sed '/baz/s/foo/bar/g'

Find human readable strings in a file

strings data.txt

Find lines that occur only once

sort data.txt | uniq -u

Find similarities between two files

grep -Fx -f file1 file2

Remove duplicate lines

sort -u myfile.csv

Grep recursively, return only filenames

grep -rl foo .

Grep for multiple terms

grep 'word1' filename | grep 'word2'

CSVs

Keep certain columns from a csv

cut -d ',' -f 1,2,3 old.csv > new.csv

Remove non-adjacent duplicates

cat test.csv | perl -ne '$H{$_} or print' > test_nodupes.csv

Networking

Check open ports

nmap -sS 127.0.0.1

Get IP address

curl ifconfig.me

Ping every 10 seconds, audible bell if no response

ping -i 10 -A 127.0.0.1

Download all URLs in file

cat url-list.txt | xargs wget –c

Images

All using ImageMagick (brew install imagemagick)

Find dimensions of image

identify -format '%w %h\n' foo.jpg

Convert image format

mogrify -format png *.jpg

Replace color in batch of images; first color is to, second is from

mogrify -path batch -format png -fuzz 10% -fill "#003A70" -opaque "#15ACCF" *.png

Video

Bulk convert using ffmpeg

find . -name '*.avi' -exec ffmpeg -i {} {}.mpg \;

Combine video files using ffmpeg given a list of files

# mylist.txt
file '/path/to/file1'
file '/path/to/file2'
file '/path/to/file3'

ffmpeg -f concat -safe 0 -i mylist.txt -c copy output.mp4

Convert mov to mp4

find . -name '*.mov' -exec ffmpeg -i {} {}.mp4 \;

Resize video

for x in *.mp4; do ffmpeg -i $x -vf scale=iw/2:ih/2 shrunk-$x; done

or

for x in *.mp4; do ffmpeg -i $x -vf scale=320:-1 shrunk-$x; done

Miscellaneous

Sudo run last command

sudo !!

Run a command every 30 seconds

while true; do ls ; sleep 30; done

Another way to run a command every 30 seconds

watch -n 30 ls -l

Create a tar.gz

tar -pczf name_of_your_archive.tar.gz /path/to/directory

Extract first 10 seconds from wav file

sox input.wav output.wav trim 0 10

Alert when a process finishes

ls && tput bel

Open multiple URLs in Chrome

open -a "Google Chrome.app" https://cnn.com https://google.com https://duckduckgo.com