NoCoug Spring Conference – May 21 May 8, 2009

I’m into local things these days. I try to buy food at the local farmers market and my books in local book stores. I’m lucky to live in California where the farmer markets contain tons of fresh veggies even in February, and the local book shop carries several copies of “Cost Based Fundamentals” 

I’m also happy when I can get my Oracle Geekery locally. I mean,  I enjoy meeting DBAs from all over the world in the big international conferences. But practically speaking, my next job is more likely to come from the DBA I met at a local user group and tells me about that great company he is working for, and the local OUG is where I meet the guys (and girls) that I can call up when I’m completely stuck on a problem, or if I want to meet someone to have a drink and talk shop for a bit.

Local user groups rule. Especially in California.

After all that chat, here are my reasons for telling my bosses that I’ll be on training on May 21st and they can only call me on emergencies:

1. Cary Millsap will be giving the keynote and a presentation. I’m a huge fan of his
2. Gaja Krishna Vaidyanatha will be presenting “The SAN is Guilty… Until Proven Otherwise!”. He is such a good presenter that you’ll enjoy this even if you don’t have SAN.
3. I’m giving my “Streams Troubleshooting” presentation. If you ever stared at the screen helpless while streams simply did not work, you probably want to attend. I have tons of experience to share.
4. Dave Abercrombie from Convio will talk about session tagging for end-to-end monitoring and troubleshooting. I met him at HotSos – he’s a good speaker with great approach to monitoring.
5. Did I mention how cheap NoCoug conferences are? I still can’t believe that you get to listen to all those great presentations + breakfast + lunch for 25$. I mean, this is 1% from the cost of Oracle Open World, and you get so much more value.
6. Its a good chance to catch up with my local Oracle friends.
7. People go to me and say “Hey! I love your blog!”
8. I’m not in the office. This is almost like vacation, but it doesn’t count as PTO because I’m in a conference.

See you there!

Always Make New Mistakes May 6, 2009

I found a cute fridge magnet with this sentence on it, and I liked it so much that I copied it to a post it and stuck it to the wall next to the monitor.

I love it because I want to always be doing new stuff, and when you do new stuff you naturally make mistakes, and you learn from them, and its great. You just want to avoid doing the same mistake twice, because this may mean that you are not learning. What’s not to like? Oh, and I also get to write blog posts about my new mistakes. Old mistakes are usually not worth writing about.

But this should probably be improved to “Always Make New Mistakes (in staging)”, because when I work on production I want to always see nice old mistakes that I’ve seen before and know exactly how to solve in the 3 minutes of maintenance downtime I have this month while my boss is looking over my shoulder asking questions.

Yay for new mistakes, but not on production.

Streams on RAC May 5, 2009

We had RAC system as streams source for 18 month now. But just today I configured a RAC as streams target.

It was somewhat of an anticlimax since there is absolutely nothing interesting to do.

We do downstream capture, so I had to place the archive logs on a shared drive. Every place where I used the SID before, I now used the DB name.

Capture and Apply processes both started on the same node. When I stopped that node, I saw the other node modifying a service called SYS${streams user}.{streams queue name}.{target db name}’ , changing it to run on the remaining node.

Then the capture and apply processes started on the remaining node and everything continued as usual.

As I said, no big deal. I just wanted to let everyone know that it is no big deal.

When Archive Logs Are Not Copied To Standby May 5, 2009

I debug this same issue at least once a month since January 2008, and I always debug it from scratch because I’m a DBA with bad memory and I never remember how I solved it the last 18 times.

So, if you added a second (or third) archive log destination and then you see errors in the alter log that look like:

————————————————————
Check that the primary and standby are using a password file
and remote_login_passwordfile is set to SHARED or EXCLUSIVE,
and that the SYS password is same in the password files.
returning error ORA-16191
————————————————————

Check:

1. That you are using password files
2. That the name of the password file is $ORACLE_HOME/dbs/orapw${ORACLE_SID}
3. That the password of sys is the same on all the servers involved in the archive copy
4. If one of the databases is 11g, make sure you set sec_case_sensitive_logon to false
5. Also check that you created the password file with ignore_case=y

I think this covers my usual debug process. Except it normally takes me an hour to remember that 11g has this thing with case sensitive passwords, and what the parameter is called, and that there is another thingy to do with the password files… Hopefully, not anymore.