Which deployment option is most cost-effective for infrequent predictions?

Batch endpoints are designed for asynchronous, long-running inference jobs on large amounts of data. The key cost-effective feature is that compute resources are provisioned only when the batch job is submitted and are automatically de-provisioned once the job is complete. This pay-per-use model is ideal for workloads that do not require an immediate response and run infrequently, as you do not incur costs for idle, always-on compute resources, which is typical for real-time or online endpoints.

A. Real-time endpoint: This is an older term for an online endpoint, which requires provisioned, always-on compute, making it costly for infrequent use.

C. Online endpoint: This is designed for low-latency, real-time predictions and requires compute resources to be continuously running, leading to higher costs for sporadic workloads.

D. Streaming endpoint: This is not a standard deployment type in Azure Machine Learning for trained models; services like Azure Stream Analytics handle streaming data.

1. Microsoft. (2023, November 13). What are Azure Machine Learning endpoints? Microsoft Learn. Retrieved from https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints. (See the "Batch endpoints" section, which states they are for "long-running asynchronous inference").

2. Microsoft. (2023, November 13). Use batch endpoints for batch scoring. Microsoft Learn. Retrieved from https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-batch-endpoint. (See the "When to use batch endpoints" section, which highlights cost-effectiveness for non-real-time workloads).