Sunday, November 4, 2012

Site update check script (or how I knew the Viriginia Bar results within 5 seconds)

Joy just passed the Virginia bar (congrats baby!!!). The way that bar results work  in Virginia, is at some random time over the span of a couple days the list of everyone who passed gets posted to http://www.vbbe.state.va.us/bar/barresults.html This results in 1500 people staring at their computer endlessly beating the refresh key. To save Joy the stress on her fingers I wrote up a script to do it for her. I thought of two different ways of doing it.

1. Using the document.lastModified property

2. Checking the filesize of that webpage.

2 seemed like a cleaner/quicker option but did have the extremely unlikely chance that the modified page would be exactly the same size as the unmodified page. To fix this, I decided to do a checksum instead of a file size check. I also had it either dump the page to shell or email a copy of the source code just in case it got changed back quickly. I wrote the script in a way I could reuse it in the future. It should work on any *nix system. However, if you want an email instead of a shell output you will need to have mutt installed and outgoing mail properly configured. It will probably not work correctly on most dynamically generated sites. I’m sure there are better ways of going about this but this worked well for me and let Joy and me know results were up within 5 seconds of them being posted. Example usage and output below.

[harlow]$ ./sitecheck.sh
Usage: sitecheck.sh $site [$email] [$sleep (default 30)]

harlow]$ ./sitecheck.sh betterthanbacon.me/test.html luke@lukeodom.com 5
Sun Nov 4 19:57:13 PST 2012 no change
Sun Nov 4 19:57:18 PST 2012 no change
Sun Nov 4 19:57:23 PST 2012 no change
Sun Nov 4 19:57:28 PST 2012 no change
Change detected. Sending email to luke@lukeodom.com
Sun Nov 4 19:57:38 PST 2012 no change
Sun Nov 4 19:57:43 PST 2012 no change

Script can be downloaded at

http://betterthanbacon.me/sitecheck.sh

Or it’s dumped at:

#! /bin/bash

#Check for at least one variable

if [ $# == 0 ]
then
echo “Usage: sitecheck.sh \$site [\$email] [\$sleep (default 30)]”
exit
fi

#Create temp file

tmp=`mktemp`

#Create initial md5 sum

curl -s $1 > $tmp
md5old=`md5sum $tmp | awk ‘{print $1}’`

#Get site again, compare to inital md5 checksum, rinse, repeat

while true; do
curl -s $1 > $tmp
md5=`md5sum $tmp | awk ‘{print $1}’`
if [ $md5 != $md5old ]
then
if [ -z $2 ]
then
echo “change detected”
cat $tmp
else
echo Change detected. Sending email to $2
echo $1 |  mutt -a $tmp -s “$1 updated” $2
fi
md5old=$md5
else
echo `date` no change
fi
if [ -z $3 ]
then
sleep 30
else
sleep $3
fi
done

posted by admin at 8:02 pm  

No Comments »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress Copyright © 2011 Luke Odom