Lessons From OOW09 #1 – Shell Script Tips

During OpenWorld I went to a session about shell scripting. The speaker, Ray Smith, was excellent. Clear, got the pace right, educating and entertaining.

His presentation was based on the book “The Art of Unix Programming” by one Eric Raymond. He recommended reading it, and I may end up doing that.

The idea is that shell scripts should obey two important rules:

  1. Shell scripts must work
  2. Shell scripts must keep working (even when Oracle takes BDUMP away).

Hard to object to that ๐Ÿ™‚

Here’s some of his advice on how to achieve these goals (He had many more tips, these are just the ones I found non-trivial and potentially useful. My comments in italics.)

  1. Document dead ends, the things you tried and did not work, so that the next person to maintain the code won’t try them again.
  2. Document the script purpose in the script header, as well as the input arguments
  3. Be kind – try to make the script easy to read. Use indentation. Its 2009, I’m horrified that “please indent” is still a relevant tip.
  4. Clean up temporary files you will use before trying to use them:

    function CleanUpFiles {
    [ $LOGFILE ] && rm -rf ${LOGFILE}
    [ $SPOOLFILE ] && rm -rf ${SPOOLFILE}
  5. Revisit old scripts. Even if they work. Technology changes. This one is very controversial – do we really need to keep chasing the latest technology?
  6. Be nice to the users by working with them – verify before taking actions and keep user informed of what the script is doing at any time. OPatch is a great example.
  7. Error messages should explain errors and advise how to fix them
  8. Same script can work interactively or in cron by using: if [ tty -s ] …
  9. When sending email notifying of success or failure, be complete. Say which host, which job, what happened, how to troubleshoot, when is the next run (or what is the schedule).
  10. Dialog/Zenity – tools that let you easily create cool dialog screens
  11. Never hardcode passwords, hostname, DB name, path. Use ORATAB, command line arguments or parameter files.I felt like clapping here. This is so obvious, yet we are now running a major project to modify all scripts to be like that.
  12. Be consistent – try to use same scripts whenever possible and limit editing permissions
  13. Use version control for your scripts. Getting our team to use version control was one of my major projects this year.
  14. Subversion has HTTP access, so the internal knowledge base can point at the scripts. Wish I knew that last year.
  15. Use deployment control tool like CFEngine. I should definitely check this one out.
  16. Use getopts for parameters. Getopts looked to complicated when I first checked it out, but I should give it another try.
  17. Create everything you need every time you need it. Don’t fail just because a directory does not exist. Output what you just did.
  18. You can have common data files with things like hosts list or DB lists that are collected automatically on regular basis and that you can then reference in your scripts.
  19. You can put comments and descriptions in ORATAB

15 Comments on “Lessons From OOW09 #1 – Shell Script Tips”

  1. davmac says:

    Please do not forget the all-important double quotes! Spaces in directory and file names can break your scripts otherwise. Eg:

    function CleanUpFiles {
    [ “$LOGFILE” ] && rm -rf “${LOGFILE}”
    [ “$SPOOLFILE” ] && rm -rf “${SPOOLFILE}”

  2. Just installed the collabnet version of subversion. don’t know for sure if it can be accomplished with a regular subversion and apache install, but at least with the collabnet version authentication can be setup to LDAP (what I am doing), but also to microsoft AD.

    cfengine (and puppet, which serves the same purpose) is golden. In my time at an outsourcing company, the sysadmins here and there forgot to do (fully documented) settings. using cfengine, you can make machine templates, which are set by the cfengine agent. this means if a machine is declared linux-redhat-x86-database, it will have correct settings. of course you can make exceptions, but the’ll be at the cfengine server. so one place to look. cfengine is a sysadmin game, though, not really a DBA game.

  3. prodlife says:

    We are currently using Borland’s StarTeam. It is far from perfect but it has the important advantaged of already existing in the organization and being managed by someone who is not me ๐Ÿ™‚

    I want to look at cfengine for post-sysadmin parts: Our sysadmins will give me the server, but they will not manage OPatches, and versions of backup scripts, etc. I understood that cfengine and its cousins can also be used for maintenance and patching, not just for initial provisioning.

    • as far as I see it, it’s build for maintenance (such as the version of scripts). you can let cfengine execute shell scripts and distribute files, so it could be possible.

      mmmh, that I think of it, you can check for the existence of certain lines in files, so theoretical you could check oratab for the existence of a database, and if not, create it.

  4. about 14, exposing your code may help hackers that look for security breaches, like hardcoded passwords ๐Ÿ™‚

    prefer https with authentification ๐Ÿ˜‰

  5. Cd-MaN says:

    Thanks for the tty -s tip!

  6. Another good practice: make sure temp files are removed automatically:

    tmp=”/tmp/`basename “$0″`.$$”
    trap “rm -f ‘$tmp'” 0

    foo >”$tmp”

    grep something “$tmp”


  7. Narendra says:

    # Revisit old scripts. Even if they work. Technology changes. This one is very controversial โ€“ do we really need to keep chasing the latest technology?

    Now, we may not need to keep chasing the latest technology in the context of shell scripts, but it is worth doing this for Oracle SQL (and PL/SQL). You may not end up implementing all new features but it is always nice to know what is new and how, if at all, new stuff can help getting things done in more efficient manner for business.

  8. […] Of course the conference wasn’t short in technical issues and these articles prove it. Jarneil talks about 11 Things about 11gR2. Chen Shapira talks about one of the lessons she learned and shares with us some Shell Script Tips. […]

  9. […] Chen Shapira-Lessons From OOW09 #1 โ€“ Shell Script Tips […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s