Deploy SageMaker Endpoints with Confidence

This skill covers the full lifecycle of Amazon SageMaker endpoint deployment: model registration and versioning through the SageMaker Model Registry before deployment, endpoint configuration strategy using separate, timestamped endpoint configs per deployment stage with data capture enabled, and two deployment patterns - blue/green deployment with gradual traffic-weight shifting between variants, and multi-model endpoints that share infrastructure across multiple models registered under one endpoint.

It covers auto-scaling configuration (registering a scalable target and a target-tracking scaling policy based on invocations-per-instance, with configurable scale-out and scale-in cooldowns), inference optimization through custom inference handlers (a model_fn/input_fn/predict_fn/output_fn class structure for loading a model, parsing JSON input, running inference, and formatting the response), and monitoring through CloudWatch metric alarms (for example alerting when model latency exceeds a threshold).

Best-practice guidance spans instance selection (compute-optimized C5/M5 for CPU inference, P3/G4 for GPU workloads), using batch transform instead of real-time endpoints for large-scale batch inference, keeping model artifacts under 10GB for faster cold starts, securing endpoints with VPC endpoints, encryption at rest and in transit, and least-privilege IAM, and cost optimization via Serverless Inference for sporadic traffic versus Reserved Instances for predictable workloads. It also covers troubleshooting: cold-start latency (model warming, provisioned concurrency), memory issues (CloudWatch metrics, instance sizing, quantization), timeout errors (raising SAGEMAKER_MODEL_SERVER_TIMEOUT), and scaling issues (reviewing auto-scaling policies and CloudWatch thresholds).

import boto3
from sagemaker import Model, get_execution_role
from sagemaker.model_package import ModelPackage

Use this skill when you need to move a registered model into production on SageMaker and keep it there reliably - picking between a gradual blue/green traffic shift or a shared-infrastructure multi-model endpoint, sizing an auto-scaling policy around invocations-per-instance, or working through a specific failure mode like cold starts, out-of-memory errors, request timeouts, or scaling lag.

It is not a fit for large-scale batch scoring, where the skill itself recommends batch transform over a real-time endpoint, or for model training and artifact packaging - the guidance assumes a model package is already registered and starts from there.

Inputs are a registered SageMaker model package, target instance type and count, deployment-pattern choice (blue/green or multi-model), and any custom inference environment variables such as SAGEMAKER_MODEL_SERVER_WORKERS. Outputs are working boto3/SageMaker SDK code for endpoint creation with data capture enabled (a configurable sampling percentage and a destination S3 URI for captured input and output payloads), traffic-shifting logic between variants, an auto-scaling policy, a custom inference handler module, and CloudWatch alarm definitions for latency monitoring.

The SageMaker Model Registry supplies the versioned model package deployment starts from; AWS Application Auto Scaling watches the SageMakerVariantInvocationsPerInstance metric to drive scale-out and scale-in; CloudWatch alarms watch ModelLatency and other metrics keyed to the endpoint name and variant; IAM execution roles gate what the endpoint can access; and VPC endpoints keep inference traffic off the public internet.

ML engineers and platform teams deploying models to production SageMaker endpoints who need concrete patterns for zero-downtime rollouts and cost-efficient multi-model hosting, and who test with production-like data volumes and request patterns before going live, rather than assembling endpoint configuration, scaling policies, and inference handlers from the SageMaker documentation alone.