A - Oracle 19c EXADATA

Introduction to Oracle Exadata

The Oracle Exadata Database Machine is a computing platform optimized for running Oracle Database. Exadata is a combined hardware and software platform that includes scale-out Intel x86-64 compute and storage servers, RoCE or InfiniBand networking, persistent memory, NVMe flash, and specialized software.

Exadata was introduced in 2008, and, since October 2015, is available either as an on-premises product or via the Oracle Cloud as a subscription service, known as the Exadata Cloud Service. Oracle databases deployed in the Exadata Cloud Service are 100% compatible with databases deployed on Exadata on-premises, which enables customers to transition to the Oracle Cloud with no application changes. Oracle Corporation manages this service, including hardware, network, Linux software and Exadata software, while customers have complete ownership of their databases.

Exadata is designed to run Oracle Database workloads, such as an OLTP application running simultaneously with Analytics processing. Historically, specialized database computing platforms were designed for a particular workload, such as Data Warehousing, and poor or unusable for other workloads, such as OLTP. Exadata allows mixed workloads to share system resources fairly with resource management features allowing prioritized allocation, such as always favoring workloads servicing interactive users over reporting and batch, even if they are accessing the same data. Long running requests, characterized by Data Warehouses, reports, batch jobs and Analytics, are reputed to run many times faster compared to a conventional, non-Exadata database server.

The Exadata Database Machine uses a scale-out architecture for both database servers and storage servers. As workloads grow, database CPUs, storage, and networking can be added to an Exadata Database Machine to scale without bottlenecks. The architecture expands from small to extremely large configurations to accommodate workloads of any size.

A brand new high-bandwidth low-latency 100 Gb/sec RDMA over Converged Ethernet (RoCE) Network Fabric connects all the components inside an Exadata Database Machine. Specialized database networking protocols deliver much lower latency and higher bandwidth than is possible with generic communication protocols for faster response time for OLTP operations and higher throughput for analytic workloads. External connectivity to the Exadata Database Machine is via standard 10 Gb/sec or 25 Gb/sec Ethernet.

The Exadata X8M-2 Database Machine uses powerful database servers, each with two 24-core x86 processors and 384 GB of memory (expandable up to 1.5 TB). Exadata also uses scale-out, intelligent storage servers available in two configurations High Capacity (HC) or Extreme Flash (EF). HC Storage Servers have four NVMe PCI Flash cards each with 6.4 TB (raw) Exadata Smart Flash Cache and twelve 14 TB 7,200 RPM disks. EF Storage Servers have an all-flash configuration with eight NVMe PCI Flash drives, each with 6.4 TB (raw) storage capacity. Exadata X8M HC and EF Storage now include persistent memory, further boosting capacity and performance. HC and EF Servers receive twelve 128 GB Intel Optane DC Persistent Memory modules as a new tier between DRAM and flash. Exadata combines persistent memory with innovative RDMA algorithms that bypass the network and I/O stack, eliminating expensive CPU interrupts and context switches, reducing latency by 10x, from 200s to less than 19s.

The minimal configuration of an Exadata Database Machine consists of two database servers and three storage servers, which can be expanded into elastic configurations adding more database and/or storage servers within the same rack. Elastic configurations provide a flexible and efficient mechanism to meet any size business need.

Shared Persistent Memory Acceleration

New with Exadata X8M, storage servers include persistent memory (PMEM) data and commit accelerators in front of flash cache, enabling orders of magnitude lower latency accessing remotely stored data. Persistent memory is a new silicon technology, adding a distinct storage tier of performance, capacity, and price between DRAM and Flash. As the persistent memory is physically present on the memory bus of the storage server, reads perform at memory speed, much faster than flash. Writes are persistent, surviving power cycles, unlike DRAM. By utilizing RDMA to access persistent memory remotely, Exadata Smart PMEM Cache is able to bypass the network, I/O software, interrupts and context switches, achieving more than 10x lower latency than previous Exadata generations, down to less than 19 microseconds. Smart Exadata System Software also ensures data is mirrored across storage servers, which provides additional fault-tolerance. Exadatas unique end-to-end integration between Oracle Database and Exadata Storage automatically identifies the hottest data blocks to store, while ensuring database, persistent memory, and flash cache do not hold the same block multiple times, increasing the efficiency across the storage tiers. Adding persistent memory to the storage tier means the aggregate performance of this new cache tier can be dynamically used by any database on any server. This is a significant advantage over general-purpose storage architectures, which preclude sharing across servers.

Another smart new Exadata System Software feature boosts log write performance. Log write latency is critical for OLTP performance, a faster log write means faster commit times. Inversely, any slowdown of log writes can cause the database to stall. Unique to Exadata X8M, Exadata Smart PMEM Log automatically enables the database to issue a one-way RDMA log write to persistent memory. RDMA and persistent memory technologies allow the log write to occur without acknowledgement, and smart software places the write across multiple servers for resilience. This leads to an 8x performance increase in log writes.

Security and management of this new tier are also automated. Persistent memory is configured automatically at installation time, with no user interaction required. Hardware monitoring is configured out of the box. Persistent memory is only accessible to databases using database access controls, ensuring end to end security of data. Deploying persistent memory in Exadata X8M is so simple, its transparent.

Extreme Flash Storage Server: Record-Breaking I/O Performance

Exadata Extreme Flash (EF) Storage Server, first introduced with Exadata X5, is the foundation of a database-optimized all-flash Exadata Database Machine. Each EF Storage Server contains eight 6.4 TB Flash Accelerator F640v2 NVMe PCI Flash drives, offering 51.2 TB raw flash capacity per EF Storage Server. This state-of-the-art flash memory improves speed and power efficiency, and provides an expected endurance of 8 years or more for typical database workloads. In addition, Exadata delivers ultra-high performance by placing these flash devices directly on the high speed PCI bus rather than behind slow disk controllers and directors. Exadata X8M adds the shared persistent accelerator, can achieve up to 16 Million random 8K database read and 5.17 Million random 8K flash write I/O operations per second (IOPS), which is an industry record for database workloads.

These are real-world end-to-end performance figures measured running SQL workloads with standard 8K database I/O sizes inside a single rack Exadata system, unlike storage vendor performance figures based on small I/O sizes and low-level I/O tools and are therefore many times higher than can be achieved from realistic SQL workloads. Exadatas performance on real database workloads is orders of magnitude faster than traditional storage array architectures, and is also much faster than current all-flash storage arrays, whose architecture bottlenecks flash throughput.