Relationship between huge page and OS Page

 

1. Normal OS Page vs HugePage (Fundamentals)

Default Linux memory pages

• Standard page size: 4 KB
• Every memory access uses page tables to translate virtual → physical
• Large SGA → millions of page table entries (PTEs)

HugePages (HugeTLB)

• Larger page size: typically 2 MB (or 1 GB on some systems)
• Reduces:
  • Page table entries
  • TLB (Translation Lookaside Buffer) misses
  • CPU overhead

---

2. How Oracle Uses HugePages Internally

SGA allocation path

When HugePages are enabled (USE_LARGE_PAGES=TRUE|ONLY):

1. Oracle requests SGA memory during startup
2. Kernel checks:
   • Are HugePages available (/proc/meminfo)?
3. If yes:
   • Allocates SGA entirely from HugePages pool
   • Pins memory → not swappable
4. If not:
   • Falls back to normal 4KB pages (unless ONLY)

Key behavior

• Only SGA uses HugePages
• PGA, stack, processes → still use normal pages

---

3. Interaction with OS Page Cache & IO

This is where most confusion happens.

A. Buffered I/O (filesystem)

• Uses OS page cache
• Pages are normal 4KB pages
• Reads:

  Disk → OS Page Cache → Oracle buffer cache
  

✅ HugePages are NOT used in OS page cache

---

B. Direct I/O (O_DIRECT / ASM / Filesystem with DirectIO)

• Bypasses OS page cache
• Data flows:

Disk → Oracle SGA buffer cache (HugePages)

Important insight:

• With HugePages:
  • Buffer cache resides in HugePages
  • Data blocks (8 KB, 16 KB) are packed inside HugePages

👉 HugePages improve:

• Memory lookup efficiency
• Buffer cache access speed

👉 But they do NOT change I/O block size

---

4. Relationship: HugePages vs DB Block vs OS Page

Layer	Typical Size
Oracle DB block	8 KB (default)
OS normal page	4 KB
HugePage	2 MB

Internal packing

A single HugePage (2 MB) holds:

2 MB / 8 KB = 256 Oracle blocks

So:

• Oracle still uses 8 KB blocks
• HugePages just back the memory region

---

5. Does HugePages Increase IO Performance?

Direct Impact:

❌ Does NOT increase disk IO throughput directly
✅ Reduces CPU overhead during memory access

Indirect Impact:

• Faster buffer cache lookups
• Reduced TLB misses
• Better scalability for large SGA
• Less kernel overhead → more CPU for DB work

👉 So overall DB performance improves, but IO bandwidth stays same

---

6. How HugePages Reduce Kernel Overhead

Without HugePages (example 500 GB SGA):

500 GB / 4 KB ≈ 131 million pages

With HugePages (2 MB):

500 GB / 2 MB ≈ 256,000 pages

✅ Reduction:

• Page table size drastically reduced
• Fewer TLB entries needed
• Less CPU spent on memory translation

---

7. Key Kernel Components Involved

1. TLB (Translation Lookaside Buffer)

• Cache of virtual→physical mappings
• HugePages → fewer entries needed → less misses

2. Page Tables

• Shrink significantly with HugePages

3. Buddy Allocator / HugeTLB pool

• Pre-reserved memory (vm.nr_hugepages)
• Prevents fragmentation issues

---

8. Why HugePages Do NOT Affect OS Page Cache

Because:

• Page cache is managed by kernel VM subsystem
• HugePages are:
  • Preallocated
  • Not swappable
  • Not part of regular memory pool

👉 They are isolated memory region

---

9. How HugePages Size Is Increased

Step 1: Calculate requirement

HugePages = (SGA size) / HugePage size

Example:

SGA = 100 GB
HugePage = 2 MB

100 GB / 2 MB = 51200 pages

---

Step 2: Configure kernel

vm.nr_hugepages = 51200

---

Step 3: Reload

sysctl -p

---

Step 4: Verify

cat /proc/meminfo | grep Huge

Important fields:

HugePages_Total
HugePages_Free
HugePages_Rsvd
HugePages_Surp

---

10. Advanced Behavior (Very Important)

Partial allocation

If HugePages are insufficient:

• Oracle may split:
  • Part SGA → HugePages
  • Rest → normal pages ❌ (bad)

👉 Causes:

• Performance inconsistency
• Memory fragmentation

✅ Best practice:

USE_LARGE_PAGES=ONLY

---

11. Summary (Core Concepts)

What HugePages DO

✅ Optimize SGA memory management
✅ Reduce CPU overhead
✅ Improve scalability
✅ Prevent swapping

What HugePages DO NOT DO

❌ Do not change DB block size
❌ Do not increase disk IO speed
❌ Do not affect OS page cache

---

12. Simple Mental Model

Think of it like this:

• Regular pages → many small boxes (slow to manage)
• HugePages → fewer large containers (efficient)

Oracle:

• Stores data blocks inside containers
• Still processes blocks same way

---

13. DBA Insight (Important for you as Architect)

Given your role:

HugePages matter most when:

• SGA > 16–32 GB
• High OLTP concurrency
• CPU contention exists

They matter less when:

• IO bound system (slow storage)
• Small SGA