Common mistakes in RAC installation

This was supposed to be my OpenWorld Unconference session, which I didn’t give partially due to shyness and partially because I preferred to spend my time listening and learning.

I’m probably the worlds expert on failed RAC installations. I started my career as a DBA by spending four days with a consultant failing to install RAC in our test environment. In the three years that passed since that fatefull week, I’ve probably failed installing RAC over fifty times (I’ve succeeded quite a few times too), so I’m well qualified to tell everyone how to fail installing RAC.

So, how do you completely screw up your RAC installation?

  1. Don’t use the installation guide. Thats a common mistake done by both beginners and experts. If you don’t follow your RAC installation guide closely, your RAC installation will fail. The installation is simply too complicated to do from memory or by hunches. That is the most important thing to remember. The rest of this post will just contain common consequences of not following the installation guide. Also keep in mind to match the version of the installation guide to the version of RAC you are actually installing, because some things change with time.
  2. Your nodes don’t see each other. Huge mistake. Your nodes should be able to connect to each other by name, ip and fully qualified domain name, through public ip and interconnect ip. Verify with pings. Also make sure your host name is spelled the same everywhere – some parts of the installation are case sensitive.
  3. Don’t verify that all your RPMs are installed before beginning the installation. Unfortunately, this is a very easy mistake to make, because the RPM list in the installation guide is somewhat incomplete. There are metalink articles that attempt to correct the mistakes, so look for them. Keep in mind that at least in 10g, the prerequisite check didn’t cover all the required RPMs, so if you mess this step you will end up with a rather random error during the installation.
  4. Ask your network manager to configure the VIP in Linux before your install your cluster ware. Don’t. Just ask him for an IP – Oracle has a VIPCA utility that will configure and manage the VIP for you. If Linux already controls the VIP, RAC installation will fail.
  5. Configure SSH incorrectly. SSH configuration is a somewhat tricky part. Remember that your nodes should be able to ssh each other with user oracle without ssh asking for password or saying anything. ssh remotenode date should just give the date.
  6. Different times for different nodes. All nodes should show the exact date and time.
  7. Bad permissions on shared storage. Verify that root on all nodes has write access to the voting disk.

Thats what I recall right now. I’m sure there are lots more.

Advertisements

5 Comments on “Common mistakes in RAC installation”

  1. dhoogfr says:

    1.Very true, with the exception of solaris 10 where the oracle documentation is just plain wrong (but there seems to be corrections in the readme)

    3. I find metalink note: 169706.1 – Oracle Database on AIX,HP-UX,Linux,MacOSX,Solaris,Tru64 Operating Systems Installation and Configuration Requirements Quick Reference (8.0.5 to 11.1)
    very usefull for this.

    6. True, but be carefull how you configure ntpd (use the -x option or slewalways=Y). If the ntp deamon just corrects the time by putting the time backup a few seconds it could crash your node without any logging (the crs finds a timestamp on the voting disk that is in the future and thinks there is something wrong with its io paths).

  2. prodlife says:

    1. Good to know!
    3. Yes, I use this metalink note a lot too. I think I even recommended it in a previous post.
    6. Ooh! Thats very useful information! Maybe we even lost a node that way and never figured it out. Thank you.

  3. Rudy says:

    I would like to add: (do) configure a cluster listener from the Oracle database home before attempting to configure the ASM cluster instances from the ASM home (in case of different ASM and database homes).
    It’s not going to stop you, it’s only better-looking.

  4. Arul Ramachandran says:

    Nice check list you have here.

    2. I have seen this and I’ve blogged it in detail.
    6. I’ve heard about this, but it was interesting to hear first hand information.

    Thanks.

  5. Arul Ramachandran says:

    P.S: blogged #2 at http://oraclezone.wordpress.com/

    Thanks.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s