crsd.bin hangs on HP-UX results ASM hang and DB hang with ORA-240

I had a lot of project with Oracle RAC on HP-UX based on Intel Itanuim platform.

We started with version 10.1.0.4, but quickly moved to 10.2.0.2 when it became available.

There were clusters with 2, 4 and 8 nodes and there weren’t stability problems with 2 or 4 nodes cluster, but those one – with 8 nodes … was the real nightmare…

At some point in time there were about 28 databases and ASM… was hanging…  every 2 days!!!

To temporary resolve this hanging issue customer had to shutdown all ASM instances at once and than start them – so it required FULL CUSTER OUTAGEContinue reading

ASM_DISKSTRING parameter MAXIMUM length

Hi.

It was quite interesting situation when one customer contacted me and asked for the maximum length of ASM_DISKGROUP parameter’s value.

I remembered that the maximum number of diskgroups is 63 but stuck with asked question…

So I think this information may be useful to others.

Conclusion:

  • if You have a lot of diskgroups and You’re at ASM version 10.2 then You have one more reason to upgrade your ASM instance to version 11.x

PS.

  • ASM_DISKGROUPS parameter only limits the set of diskgroups that will be automatically mounted on instance startup, so You may have diskgrpous not mentioned with this parameter, but You will need to mount them manually.
  • different instances of RAC may have different values for this parameter, so You may specify just a subset of diskgroups which are used on specific node

Another reason for ORA-240 with ASM & RAC in place

I was looking which search engine queries forwarded people to my blog and was quite surprised that most of them were like this one:

ORA-00240: control file enqueue held for

People doing searches by this phrase almost every day…

So I decided to point to another reason for ORA-240 with ASM & RAC.

The Issue is connected with fact that Unix/Linux processes started through inittab(like CRSD & CSSD) don’t inherit ulimit -n settings specified in profile/login scripts. Continue reading

ASM Performance with HUGE number of files

Once I was contacted by a customer complaining of poor RAC performance, hangs, etc…

It was 4 node RAC with ASM as a storage solution. There were about 10 databases.

Alert.logs were full of messages like:

ORA-00240: control file enqueue held for more than 120 seconds 
ORACLE Instance XXX - Can not allocate log, archival required
Thread 2 cannot allocate new log, sequence NNNN
All online logs needed archiving

One notice about the customer’s system: there was one special 5.6 TB diskgroup collected from 3 LUNs, each of them was a RAID-5 of SATA disks. Don’t laugh…:-) It worked quite good as a place for archive log destination and for disk backups, especially compressed. It wasn’t  supposed for any other types of database files. It’s quite important! Continue reading