Managing your automationPosted: September 20, 2007
While I keep stressing the importance and value in automating and scheduling tasks such as backups, nightly reports and cleanups, I never really mentioned what a mess the entire system can become when the automation is not managed carefully.
It starts by the place where you keep the scripts – if you keep all the scripts you need on each machine, each machine will eventually run very different scripts. They will just naturally evolve in different directions – which is not fun to manage and maintain. If you have a central location where you keep all scripts and run them from there – eventually someone will make a change to a script that will work on some machines but will break on others. I recommend working with a central repository, preferably with version control.
Then there is the issue of who owns each script. Unless you believe in complete shared ownership, you will want to record who is responsible for each script and at least notify him when making changes.
There is also the issue of what each script is supposed to be doing. Documentation is extremely important, and good documentation starts with one or two lines at the top of the script explaining what this script is good for. We also document what are the common cases that make the script fail and how to resolve them.
Finally, you will want to record the priority of the script. We have two priorities – 911 and can wait. 911 is for scripts that when they fail you want to wake up the DBA. Backups are our only example for 911 script. Everything else can wait till morning. Before we documented this, we’d get phone calls at 2am about the daily AWR failing. Remember that operators always fail on the side of waking up the DBA, so be extra clear about when not to call you.
If you feel that you are drowning in scripts and you spend most of your day looking for scripts and fixing errors that should never have been there in the first place, you are probably not managing your scripts correctly. Even simple steps towards a more organized automation will significantly improve the situation.