Bash commands to make your life easier

Unless you really wanna cat out that whole file…

watch du -a /var | sort -n -r | head -n 10

In the last few posts I covered system functions and some troubleshooting stuff, but now I want to shift focus in this post and talk about some commands that I use on a regular basis; commands I have stored on my trusty notepad that would definitely have coffee stains on it if it wasn’t on my computer.

We’re not going to cover basic commands like cd, ls, mkdir, chmod and so on. Instead, I want to focus on why piping commands or using a combination of flags for simple commands can be so powerful and can help expose important data. We’ll also briefly cover a few tools that are very useful when it comes to troubleshooting.

I have categorized these into system and networking sections. I’ll explain a few of the commands/tools, but you should have a general feel for linux.

System/directory

Grab top 10 largest files in a directory

du -a /var | sort -n -r | head -n 10

Here are a bunch of variations of ls that I use frequently. It’s all about combining flags!

Sorting a directory by largest file

ls -lsrh

Sorting a directory by most recently modified file

ls -ltrh

Listing just sub-directories

ls -d */

Grab all users in the wheel group

cat /etc/group| grep wheel |cut -d: -f4

Find a specific running process

ps -ef | grep <name of process>

Spitting out live log output

This will display live output of a log till you crtl^c out of it.

tail -f /var/log/<nameoflog>

Watch

Watch is one of my favorite commands that I feel is underutilized. What this does is, well watches a command. For example, I used this today to make sure a service was running as I was troubleshooting an error that kept stopping that service. This would look like this:

watch systemctl status -l stackdriver

What watch will do by default the specified command every 2 seconds and display the output. Other scenarios could be watching filesystem disk space, verifying a system is done writing files to a directory and so on. The applications I’m sure are endless but you get the gist.

Convert putty keys to openssh keys

I added this because some users think that using putty on windows is amazing (do yourself a favor and get mobaxterm), and I have to convert their dumb keys into openssh keys.

ssh-keygen -i -f ~/.ssh/putty_key.pub > ~/.ssh/id_dsa.pub

Yes

Yes, yes. I’m sure any new linux admin has run this before and was always curious what it’s purpose is. Well it’s great/hacky way for dealing with interactive scripts. Here’s some examples:

yes | fsck /foo/bar

This will answer yes to any time fsck asks to fix a broken file system.

yes |rm -r /foo/bar

This will answer yes to all of the “Are you sure you want to delete this file?” when deleting large directories (note that rm -rf /foo/bar will do the same thing). If you specify any word(s) after yes it will repeat what you wrote.

Lastly, you can use yes to test cpu load as it will peg one cpu processor to 100%.

Network

Find all outgoing connections

This runs netstat will show all (-a) tcp (-t) connects by ip (-n). Then to trim all the spaces and only show the column with the ip the host is connecting to. Lastly we run grep to show everything but localhost.

netstat -atn | tr -s ' '| cut -f5 -d ' ' | grep -v '127.0.0.1'

Combine ping and traceroute

This one I just found recently and is a game changer. I don’t really need to explain much, just run it and see.

mtr google.com

TCP Dumps!

What would a network command post be without TCP dumps! This is a very powerful network sniffing tool that can come in handy when troubleshooting network connectivity with applications, or if you’re just curious about seeing what kind of traffic you get on a system.

First make sure you have tcpdump installed

sudo yum install tcpdump
sudo apt-get install tcpdump

Once you have it installed you can do a bunch of different types of sniffing techniques. To list out all traffic on eth0, you can run the following:

tcpdump -i eth0 > eth0_output.txt

Redirecting it to a file makes it easier to search keywords.

You can also make pcap files with tcpdump by running the following:

tcpdump -w eth0_cap.pcap -i eth0 port 27017

This would capture all packets coming in on eth0 that is using port 27017. Once you have a pcap file you can use tools like wireshark to parse and analyze packets.

This is just the tip of the iceberg with tcpdump and we can talk about packet analysis at some point in the future as it’s really interesting.. anyways moving on.

Telnet

Warning before we get into telnet. It’s very un-secure and you will fail audits if telnet is on a system. If you are using it, be sure to un-install it after you are done testing ports. That being said, telnet is fantastic for verifying if ports are open (netstat can do this too but whatever). Anyways to use this first make sure it’s installed.

RHEL/CentOS

yum install telnet

Debian/ubuntu

sudo apt-get install telnet

Now to test a connection all you need to run is:

telnet <hostIP> <port>

So if the port is open you’ll see this:

If it isn’t you’ll get a connection refused and will return you to the shell. This is usually a red flag that firewall ports aren’t opened.

There are certainly more commands and tools that should be on this list (fpm, top, rsync, curl) but I feel they deserve their own write ups as they have a lot to offer. In any case I hope this helps!