How do On-Demand serving and Dedicated AI Clusters differ for OCI Enterprise AI Models?

Oracle Cloud Infrastructure (OCI) Generative AI offers two distinct hosting modes: On-Demand serving and Dedicated AI Clusters. On-Demand serving utilizes shared multi-tenant infrastructure to host pre-trained baseline models, allowing users to serve inference requests immediately without managing underlying compute hardware. Conversely, Dedicated AI Clusters provide physically isolated GPU resources dedicated exclusively to a single tenancy. These dedicated clusters are required for hosting custom fine-tuned models, guaranteeing dedicated compute capacity, predictable latency, and high throughput. By isolating GPU instances, Dedicated AI Clusters prevent resource contention and satisfy strict enterprise compliance and security requirements while supporting custom fine-tuning and endpoint deployment workflows.

A. On-Demand uses shared multi-tenant resources, whereas Dedicated AI Clusters assign physically isolated GPU hardware exclusively to a single customer.

C. On-Demand serving actively utilizes GPU processors behind shared service endpoints to process incoming inference workloads.

D. On-Demand serving fully supports both generative text models and embedding models across all standard pre-trained offerings.

- Oracle Cloud Infrastructure Documentation, Generative AI Service Concepts: "Dedicated AI Clusters vs. On-Demand Serving". Section: Overview of Generative AI.

- Oracle Cloud Infrastructure User Guide, Managing Dedicated AI Clusters for Custom Models and Fine-Tuning.