UNIX Commands – Helpful in Hadoop

By | September 3, 2014
Few useful UNIX Commands

Find how many jars name ending with examples you have inside location /usr/lib/

#find /usr/lib/ -name "*hadoop*examples*.jar"

To list all the class name inside jar

#find /usr/lib/ -name "hadoop-examples.jar" | xargs -0 -I '{}' sh -c 'jar tf {}'

To search for specific class name inside jar

#find /usr/lib/ -name "hadoop-examples.jar" | xargs -0 -I '{}' sh -c 'jar tf {}' | grep -i wordcount.class

To compress a file using gzip
Below command replaces the original file with file1.txt.gz

$gzip file1.txt

To compress a file using gzip
To retain the original file after zipping, use below command

$gzip -c file1.txt > file1.txt.gz

To view the contents of gzip file without unzipping it

$zcat file1.txt.gz

To compress the contents of folder using gzip

$gzip -r folder

To print environment variables


Untar the file

$tar -xzf filename.tar.gz

Unzip the file

$unzip filename.zip

List all members of a group called “dummygroup”:

$awk -F':' '/dummygroup/{print }' /etc/group

List all the groups to which user belongs to: (Ex: dummyuser)

$groups dummyuser

Detect encoding format of the file

$file -i <filename>

Leave a Reply

Your email address will not be published. Required fields are marked *