EggXpert

Last updated on 05/31/2012 @ 01:56 CST

Goal of post 

Is to get your feet wet with shell scripting and learn the basics.

Audience

Anyone that's new to Linux and wants to know how to effectively use the OS. Some Linux experience in a terminal window is a bonus, but not required.

Legend

Anything bold & underlined is a title of a section and anything in bold is part of the script. If it's surrounded by 'single quotes', it's something you type in (and you ignore the single quotes).

Intro

If you're just getting into Linux, you'll eventually realize that there is a very powerful tool available to you called the shell script. First thing you might be wondering is what the heck a shell is. If you've ever opened up a terminal window (aka CLI or Command Line Interface), that's a shell. You give it commands, and it executes them. Need to list files in a directory (aka folder)? 'ls' command. Need to know where you are currently? 'pwd' command. You get the idea. The most common shell for Linux machines is Bash. So what's a script? A script, or sometimes called a program, is essentially a file that contains a set of instructions/commands to be ran once executed.

OK root, that's just about as vague as you can make it. How does this help me?

That's a perfectly reasonable question my good man/woman.

Like everything else in technology, it all starts with a problem or challenge that you're trying to solve.

The Problem

Let's take the problem I faced just the other day. I recently started beta-testing a friend's VPN service. Said VPN service seemed to work just fine until I realized that my public IP address had changed back to my old one... and the VPN client installed on my Windows machine said I was still connected. That was a big problem.

For those of you that don't know what a VPN is, it's an encrypted tunnel made to another server/computer. VPN's are great for---*cough*---hiding what you're doing from everyone else, including your ISP. So once you're VPN'ed in, your public IP address (not your local LAN IP address) changes to reflect the server that you're connecting to's IP address. I immediately realized that I'd caught this bug by pure luck and I needed an automated way to be notified if my public IP address changed as there was no way I was going to visit http://www.whatsmyip.org/ every day just to check on it.

Setting the Stage

First thing I knew was that if this was going to be automated, I needed a script. Since the client was on a Windows machine, and I didn't want to test my Perl skillz, I needed a Virtual Machine. So I grabbed a Debian (Linux Distro) iso and installed it onto a VM. Since the VM used the Windows machine's internet connection, it meant that it would participate in this lovely VPN tunnel that was failing. Logging into the shell (I didn't install a desktop--but if you have a desktop, you'll need to run a "terminal" window) I found myself in my home directory (read: /home/yourusername unless you're logged into root and then it's /root). Typing 'pwd' command (print working directory) confirmed this. I like to put my scripts into a single folder, so I type 'mkdir scripts' and then 'cd scripts' (you just created a folder/directory called scripts and went into the directory). Typing 'pwd' again should say that you're in the scripts directory now.

Creating a Script 

Now, let's create a script. Using the command 'touch publicip.sh' creates a blank file called "publicip.sh". As you probably know, Linux doesn't look at extension types when running programs, so you can use whatever extension you want. Now you have to change the file to be executable with 'chmod u+x public.sh'. This means the owner of this file (the username you're logged in as right now) can execute this file. 

With the file created and marked as executable, I open the blank file to edit it with a text editor. Personally I use nano, but use whatever suits your fancy. With that said, I type 'nano publicip.sh'.

Technically, right now, this file isn't a shell script until we tell the shell executing the file what program to use to read this file. In our case, we want bash itself, so this is where the pound-bang comes in. The pound-bang, depicted as #!, must be used in the first line or the shell would think this is just any regular text file. So type:

#!/bin/bash

Plan of Attack

Before I write anything else, I need to know what I'm logically trying to accomplish. I know the 'trigger' in all of this is if my public IP address changes from a the static IP address I get with VPN. Once that occurs, I'll need to alert myself via email if the current IP address changes. I also know I'll use cron (a task scheduler) to run this script every 15 minutes so it's always monitoring my IP address--if I didn't do that, I'd have to manually run the script to check my IP address. Sometimes you'll need cron to help run your script, or turn it into a daemon (i.e. constantly running in the background), and other times you just need a script to run once. It all depends on your needs.

Variables

Now that we know what we want to accomplish overall, let's start with a fundamental component of scripts: the variable. A variable is a logical storage space (you can almost think of it as an alias) that you can put anything into it. Variables are great for referencing large strings of text/numbers easily, capturing user input and storing it, and/or great for easily updating a big script if you ever have to change something. A lot of people fill out their variables at the start of the script (so if you ever have to change your variables, it's right at the top).

If that all doesn't make sense, just nod your head and roll with me. You're about to get an example.

I already know that I'll need at least two variables. One for the public IP address that I know it should be, and the other for what it actually is. Let's first focus on the IP address that we know. I know for a fact that my public VPN IP address (when it works) is static (another useful thing about VPN's), so let's throw this IP address into a variable. To protect myself here, I'll use the IP address 5.6.7.8.

#!/bin/bash 

KNOWN_IP="5.6.7.8" #The known "correct" public IP address

In this case, the variable is KNOWN_IP. The equal sign means that the variable KNOWN_IP now contains the string 5.6.7.8. While you don't have to surround the value you're assigning the variable with double quotes in this case, it's a good habit to use. This is because the moment there is a space in there, and you aren't using double quotes, the variable will only contain the first word. Be sure to NOT have any spaces between the variable, the equal sign, and the value... only exception to the value part is if it's surrounded by double quotes. Otherwise it won't work. You'll also note that the variable is in all caps. This isn't a requirement, I just like using all caps so I can easily tell that it's a variable for my own sake.

You might be wondering what the # sign is for. That's tells bash to ignore everything after it. Why would you want that? For commenting of course. If you want to start using good habits, start commenting. BELIEVE me, one day you'll have a need to go back and modify an old script and 9 times out of 10 you won't know what is doing what or why you did it that way without spending a lot of time trying to figure it out. 

Now, the next variable is the public IP address upon the execution of this script. This is a point in time snapshot of what my current IP address is. But how can you get it? This is where Google helped me. A quick google pointed me to the command 'curl -s http://whatismyip.org'. The curl program will most likely have to be downloaded/installed (hint: type 'apt-get install curl' if in a Debian based system like Ubuntu). If you happen to have another terminal window up, you can test this command out in another terminal. It will reply back with your public ip address. This is exactly what I wanted. Remember, a bash script contains commands, so if it works in the terminal, it will work in a script.

#!/bin/bash 

KNOWN_IP="5.6.7.8" #The known "correct" public IP address

CURRENT_IP=$(curl -s http://whatismyip.org) #Current Public IP address

"OK root, why the #$@% did you put a dollar sign and parenthesis around it this time?"

 This is telling bash that everything inside the parenthesis is a command. If you didn't do this, bash would just think you want text in the variable. This is called command substitution. If you don't believe me, or if you don't quiet "get it", add echo $CURRENT_IP in the next line and exit/save out of the script (CTRL+X, y, 'enter') and run the script './publichip.sh'. Go back into the script and try both options and you'll see what I'm talking about. 

Echo is a command that "prints" to the screen whatever your're passing to it (in this case a variable). The dollar sign in front of a variable is how you reference a variable's value. If you didn't have the dollar sign in front of the variable, echo would think you want to print to the screen the text CURRENT_IP. The only time you don't use $ with variables is when you're assigning it a value (like the equal sign).

Now that we have that settled, and you're on board with me (hopefully), let's move on to what we're trying to accomplish. Now that we know what the IP address SHOULD be (KNOWN_IP), and what our IP address REALLY IS (CURRENT_IP), we need to compare these two. Remember the 'trigger' of the alert? If KNOWN_IP doesn't equal CURRENT_IP, then we need to notify someone that it's not equal.

Funny that I should say that...

The If Statement

An if statement is a way of looking for a condition. "if some condition is true, then execute these command(s)". Which leads us to the next couple of additions to our script.

#!/bin/bash 

KNOWN_IP="5.6.7.8" #The known "correct" public IP address

CURRENT_IP=$(curl -s http://whatismyip.org) #Current Public IP address

  TO="My.Email@Address.com" #email address used for alerting

#Check to see if the public IP address has changed 

if [[ "$KNOWN_IP" != "$CURRENT_IP" ]] ; then

  #If it has changed, note it in logs that the IP has changed

logger -t ipcheck "IP address changed to $CURRENT_IP"

#If manually ran, let user know that IP has changed 

echo "IP address has changed to $CURRENT_IP"

#Alert via email of IP address being changed, and include the tail of the log

 tail /var/log/messages | mail -s "IP has changed to $CURRENT_IP" $TO

  else # if IP hasn't changed...

echo "IP address has NOT changed." 

fi 

The first addition you'll note is the TO variable. This is part of the email home function of the script, which is found later on.

The next new line, if [[ "$KNOWN_IP" != "$CURRENT_IP" ]] ; then, says that if Known and Current are NOT equal (! means "not"), then execute the next set of commands until you see either "else" or "fi". In this case we run into else, which really means "if Known does equal Current then print to the screen that the IP address hasn't changed". The fi portion closes the if statement (so if there was anything after it, the script would continue on to the next command).

You'll note that I double quoted the variables. This is another one of those "not required but it would be a good habit to start" moments. This is because if there is a space in the variable, then it would screw up your condition as it wouldn't factor in the space as part of the condition (if that makes sense). You can actually get away with a single bracket, but a good habit is to use double brackets. I won't get into why as it would take a bit to explain. Last thing I'll point out is what ; means, which you can interpreted it as "new line" in a sense. Meaning...

if [[ "$KNOWN_IP" != "$CURRENT_IP" ]] ; then 

Is the same thing as saying...

if [[ "$KNOWN_IP" != "$CURRENT_IP" ]]

then 

It's just short handing it (you can do either one, but you have to do it one of the ways). Also, you MUST have a space between the brackets, the variables, and the operator (in this case !=).  

Next is logger -t ipcheck "IP address changed to $CURRENT_IP" which logger is a command that puts the tag "ipcheck" along with the message "IP address changed to the_current_public_IP" into the Linux system log file called "messages" found here: /var/log/messages 

Using logger is a good way to note the date/time of, in this case, the ip change and what the new IP address is.

A good thing to include in any script, even those that will be ran by cron (ala in the background and out of your sight), is the echo command. This is because you usually run it manually and troubleshoot it (ala make sure it works) before you automate it with cron/daemon. As previously mentioned, echo simply prints the words to the screen which is very helpful when you manually run this from the shell/terminal.  

But the REAL reason for this if statement is the line that says  tail /var/log/messages | mail -s "IP has changed to $CURRENT_IP" $TO 

First, tail is a simple command that outputs the end contents of a file. In this case, we're saying "show us the last couple of lines of the messages file". The reason for this is to tell us if there was anything else going on with the system with the IP address changed (ex: network port bounced, etc). The | or "pipe" basically takes the output of the tail command and puts it into the mail command (in this case, this will put the tail output into the body of an email). The -s switch to the mail command is the subject line of the email. The very last part of the mail command is the address you're emailing this email to. I will add that the mail command requires you to have sendmail installed (hint: 'apt-get install sendmail' )

Common Sense

So right now, the script will complete our minimum requirements. To recap, our requirement was to notify us (most logical thing was email) if the public IP address changed.

We went a little beyond the minimum requirements though as we had included the echo command to aide us in the event we manually ran the command, as well as added logging info to the email alert (and noted the time stamp of the first time we see the address change). This is part of what I meant by common sense. When we get notified, it would be smart of us to take the opportunity to capture additional information (tail of messages, and making a note in the logs when we first noticed the IP address is different). It would also make sense that we'd manually run the command at some point, so we should probably know the outcome of the script (in this case, either "IP address has changed to public_ip_address" or "IP address has NOT changed.").

But we should take this a step further if we want to do this one right.

First, let's say we do run this script with cron, and we check for an IP change every minute. Let's also say that the IP address did indeed change and it was right when you went asleep. Yeahhhhh. Wouldn't that suck? That would mean in the morning you'd have over 480 emails (assuming you sleep for 8 hours--one email every minute). See, you have to think about these things before you turn it over to cron or daemonize it because a script will do EXACTLY what you tell it to do and that can sometimes be a very bad thing |o.O|

So how would we stop this from happening? And remember, each time we run the script, we have no way of knowing what the outcome of the last time we had ran it.

Actually, it's pretty easy.  

#!/bin/bash 

KNOWN_IP="5.6.7.8" #The known "correct" public IP address

CURRENT_IP=$(curl -s http://whatismyip.org) #Current Public IP address

  TO="My.Email@Address.com" #email address used for alerting

STOP_FILE="/root/scripts/stop.it" 

#Has an email alert already been sent out? If so, exit out

if [[ -e "$STOP_FILE" ]]; then 

exit 0

fi 

#Check to see if the public IP address has changed 

if [[ "$KNOWN_IP" != "$CURRENT_IP" ]] ; then

  #If it has changed, note it in logs that the IP has changed

logger -t ipcheck "IP address changed to $CURRENT_IP"

#If manually ran, let user know that IP has changed 

echo "IP address has changed to $CURRENT_IP"

#Alert via email of IP address being changed, and include the tail of the log

 tail /var/log/messages | mail -s "IP has changed to $CURRENT_IP" $TO

  touch "$STOP_FILE"

  else # if IP hasn't changed...

echo "IP address has NOT changed." 

fi  

The new lines included all reference the new STOP_FILE variable, which points to a file (can be anywhere). The new if statement if [[ -e $STOP_FILE ]]; then  checks to see if $STOP_FILE, or in this case /root/scripts/stop.it, file exists (-e is a file test operator which means "does the file exist?"). If it does, then it exits out of the script exit 0. Note that this is BEFORE the main if statement, where we added touch $STOP_FILE, which if you can recall, touch creates a blank file and since we're referencing /root/scripts/stop.it, it's going to create the stop.it file.

Sooooo, what that means is if you get an email, then you'll only get one email until you delete the stop.it file in that directory (presumably after you've corrected the problem). 

OK, we got that covered. Let me throw you one more curve ball. We're assuming in this script that the curl -s http://whatismyip.org command will ALWAYS come back with an IP address. I can say with first hand experience that sometimes it will time out and return nothing. It's rare, but it occasionally happens. Here's how I addressed it.

  #!/bin/bash 

KNOWN_IP="5.6.7.8" #The known "correct" public IP address

CURRENT_IP=$(curl -s http://whatismyip.org) #Current Public IP address

  TO="My.Email@Address.com" #email address used for alerting

STOP_FILE="/root/scripts/stop.it" 

LOOP_COUNTER="0" 

#Has an email alert already been sent out? If so, exit out

if [[ -e "$STOP_FILE" ]]; then 

exit 0

fi 

#If whatismyip.org returns nothing, try again in 10 seconds and keep going until an IP is given

#This can sometimes happen, which would trigger an email alert as a null value doesn't equal $KNOWN_IP.

#If over 100 seconds has passed and still no IP, exit out and prevent a continuous loop

while [[ -z "$CURRENT_IP" ]]; do

  if [[ $LOOP_COUNTER -gt 10 ]]; then

  logger -t ipcheck "Couldn't obtain an IP from whatismyip.org."

echo "Error! Couldn't obtain an IP address from whatismyip.org." 

tail /var/log/messages | mail -s "IP has changed to $CURRENT_IP" $TO

touch $STOP_FILE 

exit 0

fi 

echo "Sleeping for 10 seconds. I have not obtain the public IP address yet."

sleep 10

CURRENT_IP=$(curl -s http://whatismyip.org)

LOOP_COUNTER=$(( LOOP_COUNTER + 1 ))

done  

#If we had to enter the loop, note that we're out of it in the logs. 

#Purpose: for whatever reason whatismyip.org doesn't ever supply an

#IP, we'll know. If they eventually do supply, this will tell us how many iterations it took.

if [[ $LOOP_COUNTER -gt 0 ]]; then

logger -t ipcheck - Got out of loop after $LOOP_COUNTER tries and the IP is currently $CURRENT_IP

fi

#Check to see if the public IP address has changed 

if [[ "$KNOWN_IP" != "$CURRENT_IP" ]] ; then

  #If it has changed, note it in logs that the IP has changed

logger -t ipcheck "IP address changed to $CURRENT_IP"

#If manually ran, let user know that IP has changed 

echo "IP address has changed to $CURRENT_IP"

#Alert via email of IP address being changed, and include the tail of the log

 tail /var/log/messages | mail -s "IP has changed to $CURRENT_IP" $TO

  touch "$STOP_FILE"

  else # if IP hasn't changed...

  #If manually ran, alert user that nothing has changed

echo "IP address has NOT changed." 

fi  

So I added a new variable called LOOP_COUNTER which I'll get into in a bit. The next thing I added was this...

while [[ -z "$CURRENT_IP" ]]; do

  if [[ $LOOP_COUNTER -gt 10 ]]; then

  logger -t ipcheck "Couldn't obtain an IP from whatismyip.org."

echo "Error! Couldn't obtain an IP address from whatismyip.org." 

tail /var/log/messages | mail -s "IP has changed to $CURRENT_IP" $TO

touch $STOP_FILE 

exit 0

fi 

echo "Sleeping for 10 seconds. I have not obtain the public IP address yet."

sleep 10

CURRENT_IP=$(curl -s http://whatismyip.org)

LOOP_COUNTER=$(( LOOP_COUNTER + 1 ))

done  

The "While, do, done" loop--aka While Loop, means "while this is true, do the following commands". The -z conditional test checks to see if $CURRENT_IP has a zero string length (ala nothing came back from whatismyip.org). So to put it this way...until $CURRENT_IP has something in it, it will continuously run--at least for 10 iterations. That's what the nested if statement is saying since LOOP_COUNTER starts with 0 (as stated in the beginning of the script), -gt means "greater than", and LOOP_COUNTER counts up by one with each While Loop in this line: LOOP_COUNTER=$(( LOOP_COUNTER + 1 )) .... And since I also added the email function and touch $STOP_FILE before we exited the script, we'll be notified of this issue and be placed into a holding pattern until we look into why it broke (ala whatismyip.org isn't returning an IP address).

Pretty neat, huh?

For more Conditional Expressions, check out this link: http://www.gnu.org/software/bash/manual/bashref.html#Bash-Conditional-Expressions 

Now you already know what logger and echo does, but sleep is a new one. sleep does exactly what it implies, it sleeps (or pauses) for x amount of seconds (in this case, 10 seconds). We definitely don't want to flood the whatismyip.org site with requests, especially if they are timing out.

The last thing I added was an if statement to see if the LOOP_COUNTER changed and noted that in the logs. While this isn't necessarily needed, I think it's good to have an idea how frequent this loop situation occurs and how many times it took to correct it.

Finishing Touches

Now we need to save the script (CTRL+X, y, enter) and test it out. You can do this by typing in './publicip.sh' and it will run the script. Assuming you have the appropriate variables correctly set (KNOWN_IP, TO, and STOP_FILE), and there aren't any syntax errors, you should get back:

"IP address has NOT changed." 

Now get back into the script and change the KNOWN_IP to something that you know is wrong. Run the script again. You should receive an email and get the following message back:

"IP address has changed to Your_public_IP_address"

Assuming it's working as expected (run 'tail /var/log/messages' to confirm there's a mention of ipcheck, then rerun the script and confirm that you didn't receive a second email), you can add the script to crontab.

To add it to cron, simply type 'nano /etc/crontab'

In this window, append the following line at the end of the file

*/15 *    * * *   root    /root/scripts/publicip.sh

Assuming your publicip.sh lives in /root/scripts directory, this is telling crontab to run /root/scripts/publicip.sh as root every 15 minutes (from the */15 part).

I should probably add here that you should remember to delete the stop.it file (assuming you tested it out before hand) before leaving it to cron.

In Summation... 

Today we learned that a bash script is simply a file that has an executable permission, and contains commands. We also learned about variables, if statements, do while loops, crontab, and using some simple common sense in how you go about your scripts. Google is also your friend here. Use the all mighty Google when in doubt. It's magical like that.

Lastly, we learned how much root can type.

This is a long freaking post.

Any questions, reply away below. Otherwise, good scripting |o.O|