A company is developing three specialized NLP models that support a customer service application. One model categorizes each customer’s specific…

This solution correctly addresses all the requirements. Deploying each model to a separate Amazon SageMaker Serverless Inference endpoint allows for independent scaling and resource configuration tailored to each model's specific needs. SageMaker Serverless Inference is ideal for workloads with intermittent traffic, as it automatically scales compute resources up and down, even to zero, which optimizes costs during low-demand periods. To meet the strict low-latency (<500 ms) and high-concurrency (500 requests) requirements during predictable peak periods, configuring provisioned concurrency pre-warms the necessary resources, eliminating cold-start latency and ensuring consistent, fast performance.

A. A multi-model endpoint is not ideal for this low-latency scenario, as it can introduce latency when loading models from S3 into memory on demand, potentially violating the 500 ms response time requirement.
C. Amazon Bedrock is used for foundation models (FMs). The scenario describes custom-developed specialized models, which are typically deployed on Amazon SageMaker, not Bedrock.
D. Asynchronous inference is designed for long-running jobs with large payloads, not for real-time, low-latency (<500 ms) requests. It queues requests for background processing, which fundamentally conflicts with the application's response time requirement.

1. AWS SageMaker Developer Guide, "Provisioned Concurrency for Serverless Inference": This document explains how provisioned concurrency is used for serverless endpoints to handle predictable traffic patterns requiring low latency. It states, "When you configure provisioned concurrency for your endpoint, SageMaker launches and pre-warms a specified number of compute resources so that they are ready to respond to inference requests immediately," which directly addresses the peak load and latency requirements.

2. AWS SageMaker Developer Guide, "When to use Serverless Inference": This section highlights that Serverless Inference is ideal for "workloads with intermittent or infrequent traffic patterns" and that it "scales down to zero when there is no traffic, helping you to minimize your costs." This supports the cost-optimization requirement.

3. AWS SageMaker Developer Guide, "Amazon SageMaker Asynchronous Inference": This guide clarifies the use case for asynchronous inference: "This option is ideal for requests with large payload sizes... and long processing times...". This confirms it is unsuitable for the sub-500 ms latency requirement.

4. AWS SageMaker Developer Guide, "Deploy multiple models in one container behind one endpoint": This page on multi-model endpoints notes that "SageMaker loads models from S3 when they are first invoked and caches them in memory." This initial loading step can introduce latency, making it a risk for strict real-time performance guarantees.