SageMaker container caching cut GenAI scale-out latency 51% (2026 guide)

SageMaker container caching cut GenAI scale-out latency 51% (2026 guide) Summary. On 16 June 2026 AWS turned on container image caching for Amazon SageMaker AI inference, and in its own published test