I was looking which search engine queries forwarded people to my blog and was quite surprised that most of them were like this one:
ORA-00240: control file enqueue held for
People doing searches by this phrase almost every day…
So I decided to point to another reason for ORA-240 with ASM & RAC.
The Issue is connected with fact that Unix/Linux processes started through inittab(like CRSD & CSSD) don’t inherit ulimit -n settings specified in profile/login scripts.
Let’s look at some details of the problem – just short excerpts:
NOTE: 858279.1 ASM and Database Instance hang when exceeding around 1800 sessions
cssd was stuck due to reached max open files limit at 1024.
Regardless of ulimit value set it to unlimit for oracle and root user.Max open file setting in limits.conf doesn’t take affect for ocssd.
CSSD started via /etc/inittab, then login scripts aren’t executed.
so limits aren’t set.We need to explicitly set them in the init scripts.
Another issue that has the common root with previous one:
NOTE: 416784.1 WARNING:Oracle instance running on a system with low open file descriptor limit
The ‘ulimit -n unlimited’ is not taking effect for processes started via inittab
On linux using something higher to 65536 or unlimited set the actual limit to 1024.
The same problem can happen on 126.96.36.199 and 188.8.131.52
This ulimit problem can result in database or asm hangs (see Note 858279.1)
BUG:8537153 ASM & DB INSTANCE HUNG DUE TO ORA-240
(Base BUG:8429716 DATABASE HANGS AT 1900 USER LOAD)
ORA-16146: standby destination control file enqueue unavailable
ORA-00240: control file enqueue held for more than 120 seconds
- following installation pre-requirements not always enough!
- it’s good practice to check init.crsd & init.cssd for ulimit -n after initial installation and after each CRS patch(set) application
My personal advise for CRS & ASM versions:
- be at version 184.108.40.206 with latest(for now it’s #1) CRS Bundle Patch applied for both products
- be at least at version 10.2.0.4 with at least CRS Bundle Patch #3 applied for both products
- database may be at any version not greater then ASM
Later I’ll point to another issues that may be cause of ORA-00240 errors.
For previous post on this topic read ASM Performance with HUGE number of files.