Sunday, November 4, 2012

Site update check script (or how I knew the Viriginia Bar results within 5 seconds)

Joy just passed the Virginia bar (congrats baby!!!). The way that bar results work  in Virginia, is at some random time over the span of a couple days the list of everyone who passed gets posted to This results in 1500 people staring at their computer endlessly beating the refresh key. To save Joy the stress on her fingers I wrote up a script to do it for her. I thought of two different ways of doing it.

1. Using the document.lastModified property

2. Checking the filesize of that webpage.

2 seemed like a cleaner/quicker option but did have the extremely unlikely chance that the modified page would be exactly the same size as the unmodified page. To fix this, I decided to do a checksum instead of a file size check. I also had it either dump the page to shell or email a copy of the source code just in case it got changed back quickly. I wrote the script in a way I could reuse it in the future. It should work on any *nix system. However, if you want an email instead of a shell output you will need to have mutt installed and outgoing mail properly configured. It will probably not work correctly on most dynamically generated sites. I’m sure there are better ways of going about this but this worked well for me and let Joy and me know results were up within 5 seconds of them being posted. Example usage and output below.

[harlow]$ ./
Usage: $site [$email] [$sleep (default 30)]

harlow]$ ./ 5
Sun Nov 4 19:57:13 PST 2012 no change
Sun Nov 4 19:57:18 PST 2012 no change
Sun Nov 4 19:57:23 PST 2012 no change
Sun Nov 4 19:57:28 PST 2012 no change
Change detected. Sending email to
Sun Nov 4 19:57:38 PST 2012 no change
Sun Nov 4 19:57:43 PST 2012 no change

Script can be downloaded at

Or it’s dumped at:

#! /bin/bash

#Check for at least one variable

if [ $# == 0 ]
echo “Usage: \$site [\$email] [\$sleep (default 30)]”

#Create temp file


#Create initial md5 sum

curl -s $1 > $tmp
md5old=`md5sum $tmp | awk ‘{print $1}’`

#Get site again, compare to inital md5 checksum, rinse, repeat

while true; do
curl -s $1 > $tmp
md5=`md5sum $tmp | awk ‘{print $1}’`
if [ $md5 != $md5old ]
if [ -z $2 ]
echo “change detected”
cat $tmp
echo Change detected. Sending email to $2
echo $1 |  mutt -a $tmp -s “$1 updated” $2
echo `date` no change
if [ -z $3 ]
sleep 30
sleep $3

posted by admin at 8:02 pm  

Powered by WordPress Copyright © 2011 Luke Odom