Saturday, May 25, 2019

I/O Issues between DB and Storage tiers in Exadata ?

How storage servers detect and cancel or repair slow I/Os and hung I/Os and confine sick disks..

IOs are pumping between Database and Storage Tiers from time to time. 
Let's see what are the different problems can be handled at storage tier.

1. Slow IO ?    ->  Cell IO Latency Capping

  What happens if we hit with slow I/Os in the storage tier, something called cell I/O latency issues? Well Exadata has a feature called Cell IO Latency Capping, which monitors I/O timings and if any disk is taking too long, it will direct read to a mirror and write to an alternate healthy disk.

2. Hung IO ?      ->  IO Hang detection

  It can be really bad if you face with truly hung I/O that escalates all the way up to like a controller level problem, you can stall your entire system with this hung I/O.. IO Hang detection will help with detection and repair and may even reset a whole cell if the problem is bad to make sure system won't stop.

3. Sick disk?     -> Predictive failure / confinement

If you have a situation where the disk about to die and I/O service timings are really bad..
Predictive failure feature built in the controllers which has heuristics to tell when a disk is going to fail and it will put in a predictive failure mode. This feature monitors metrics of disks and flash are being serviced across all different components. If they aren't then it potentially offline the sick disk.


What happens if there is undiscovered hardware or software issue on the storage tier, probably a bug or a network glitch on InfiniBand network connecting to cells or so..

4. Undiscovered hardware / Software issue?  -> Database tier I/O latency capping

From database tier, it monitors how long I/Os are taking. If there is a problem detected it will cancel them and redirect to a healthy cell.




Tuesday, May 14, 2019

Exadata X8

Technical Specifications:

> Latest Intel Xenon processors
> Latest PCIe NVME flash technology
> 25 Gbps Ethernet for client connectivity


Exadata X8-2 Features:
> Up to 912 CPU cores and 28.5 TB memory per rack for database processing
> Up to 576 CPU cores per rack dedicated to SQL processing in storage
> From 2 to 19 database servers per rack
> From 3 to 18 storage servers per rack
> Up to 920 TB of flash capacity (raw) per rack
> Up to 3.0 PB of disk capacity (raw) per rack
> Hybrid Columnar Compression often delivers 10X-15X compression ratios
> 40 Gb/second (QDR) InfiniBand Network
> Complete redundancy for high availability


Exadata X8-2 Benefits:
> Pre-configured, pre-tested system optimized for all database applications
> Uncompressed I/O bandwidth of up to 560 GB/second per full rack from SQL
> Ability to perform up to 4.8M 8K database read I/O operations, or 4.3M 8K flash write I/O operations per second per full rack
> Easily add compute or storage servers to meet the needs of any size application
> Scale by connecting multiple Exadata Database Machine X8-2 racks or Exadata Storage Expansion Racks. Up to 18 racks can be  connected by simply adding InfiniBand cables and internal switches. Larger configurations can be built with external InfiniBand switches
 
New hardware Extended (XT)
-> Much Lower cost Exadata Storage
       - Used for infrequently accessed, older or regulatory data
-> Better performance:
       - 560 GB/sec I/O throughput
       - 60% more for all-flash storage vs X7
-> 6.57 Million OLTP read IOPS
       - 25% more per storage server vs X7
       - 3.5 million iops under 250 microseconds
-> Dramatically faster than leading all-Flash arrays in every metric


Smart system Software:
Analytics:
Smart scan technology:
-Exadata automatically offloads data intensive SQL operations to storage
 - Unique Smart Scan technology offloads SQL processing to storage delivers:
    - Over 560 GB/sec throughput while offloading database CPUs
 - Unique algorithms offload Data Mining, Decryption, Aggregation and Backups to storage
-Exadata automatically reduces I/o
 - Unique Database-aware flash caching yields speed of PCI flash with capacity of disk
 - Unique storage indexes eliminates I/O that is not relevant to a particular query
-Exadata uses analytics optimized Columnar format
 - Unique Hybrid Columnar compression reduce space and speeds analytics by up to an order of magnitude
Exadata brings In-memory Analytics Performance to Storage:
In- Memory Columnar scans but also In-flash Columnar scans at storage levels
 - As exadata flash throughput approaches memory througput, SQL bottleneck moves from I/O to CPU
 - Exadata storage automatically transforms table data into In-memory DB columnar formats in Exadata Flash Cache
   - Enables fast vector processing for storage server queries
 - Uniquely optimizes next generation flash as memory
   - Now works for both row format OLTP databases and Hybrid Columnar Compressed Analytics databases
 Preview - Intel Optane DC persistent Memory will be enabled for columnar data in DB and Exa servers
for more speed and more columnar storage for anlaytics


OLTP:

Exadata automatically eliminates traditional OLTP bottleneck : random I/O
through use of unique scale-out storage, ultra-fast NVMe flash, ultra-fast IB delivers:
- Unique Smart Flash loging automatically optimizes OLTP logging to flash
Exadata automatically eliminates OLTP stalls from failed or sick components
  - Unique detection of server failures without a long timeout avoids system hangs
  - Unique sub-second redirection of I/Os around sick devices and avoid database hangs
Exadata automatically eliminates inter-node cluster coordination bottlenecks
 - Unique direct-to-wire protocol gives 3x faster inter-node OLTP messaging
 - Unique Smart Fusion Block Transfer eliminates log write on inter-node block move
 - Unique RDMA protocol to coordinate transactions between nodes
Persistent memory for even faster OLTP in storage
 - Exadata storage servers will add persistent memory OLTP accelerator infront of flash memory
   - Using Intel Optance DC persistent Memory
 - RDMA bypasses software stack, giving 20x faster latency to remote persistent memory
 - Persistent Memory mirrored across storage servers for fault-tolerance
 - Persistent Memory used as shared cache increases its value 10x versus using it directly as expensive storage
 - Makes it cost-effective to run multi-TB databases in memory

Consolidation:
Exadata uniquely optimizes Mixed workload and consolidation
Completely Automatic, No management required
 - Exadata automatically prioritizes latency sensitive operations
    - Unique prioritization of critical network messages for locks, cache fusion, logging etc
    - Unique prioritization of OLTP I/O over Analytic or Batch I/O
 - Exadata automatically prioritizes important workloads based on user policies
     - Unique prioritization of CPU and I/O by job,user, service,pdb,session, SQL
 - Exadata automatically provides isolation between multiple tenants
    - Unique prioritization and separation by database, or pluggable database


Software Release:
> Exadata system software 19.1.0.0.0 and 19.2.0.0.0
> OEL linux 7.6
> AIDE (advanced intrusion detection env)
> Automatic monitoring of CPU, Network and Memory using Machine Learning
 - Detects and alerts on stuck process, memory leaks, flaky networks etc


Automated management:
 - Automation and optimization of configuration, updates, performance and management culmination in fully autonomous infrastructure and database