Production System with Performance Problems


Recently I was involved in performance troubleshooting of one quite expensive system

that had performance problems during business hours.

I’ll not provide all details, but just post some pictures that will show resource usage:

Oracle memory allocation:

Linux memory allocation

or from other side:

cat /proc/meminfo
 MemTotal:     98848220 kB
 MemFree:      75109972 kB
 Buffers:        597460 kB

so, we have Buffer Cache sized as 368MB/(96GB*1024)=0,37% of the Total Memory!!!

To be crystal clear I have to say that this system use ASM as a storage…

So, why do you think  ASM performance is so BAD ?…

Just to note – it’s based system.

PS: sometimes I think that problems like this are based on special DNA structure…


ok, some additional comments on this system – it’s 1/4 Exadata in which ASSM feature is used and pressure from shared pool on the buffer cache was so strong, that at the end buffer cache became 256MB. It was on first node only! On the second node buffer cache was about 3.6BG and it worked like a lightning! 😉

So, SGA target was increased to 50GB and buffer pool was set from the bottom at 40GB limit.


4 thoughts on “Production System with Performance Problems

  1. Andrey,
    grep Huge /proc/meminfo
    HugePages_Total: 4100
    HugePages_Free: 421
    HugePages_Rsvd: 415
    Hugepagesize: 2048 kB

    use_large_pages string TRUE
    use_large_pages string ONLY

  2. Из той же серии – вопрос (реальный) от разработчика: какая разница, сколько джобов я запущу одновременно, 10 или 1000? Ведь они все равно в очередь все встанут, и по очереди выполнятся…

    • Юра,
      вопрос от разработчика практически правомерный,
      потому как возможна ситуация когда так и будет отрабатывать 😉
      просто такое не всегда бывает, а разработчиков это уже не беспокоит…

      на ситуацию с джобами похожа другая:
      cpu_count integer 4
      parallel_max_servers integer 160

      и никого не волнует что снизу лежит один 5-й рейт из 3-х дисков… 😉


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s