Introducing Kimi K3 on Amazon Bedrock
Kimi K3, Moonshot AI’s 2.8‑trillion‑parameter open‑weight model, is now available on Amazon Bedrock, offering a 1‑million‑token context window, native vision, and an approximate 2.5× scaling‑efficiency gain over Kimi K2. It introduces explicit prompt‑caching to cut latency and input costs, and can be accessed via Bedrock’s OpenAI‑compatible Responses and Chat Completions APIs or the Invoke/Converse APIs. Engineers can route calls through a global cross‑Region inference profile (global.moonshotai.kimi‑k3) for roughly 10 % lower cost versus a geographic profile. This enables secure, high‑throughput inference for long‑running coding and knowledge‑workflows while keeping data within the AWS boundary.
⚡ Key Takeaways
- Model size: 2.8 trillion parameters; context window: 1 million tokens; scaling efficiency: ~2.5× over Kimi K2.
- Prompt‑caching support reduces latency and input‑token cost when reusing context across calls.
- Global cross‑Region inference profile (global.moonshotai.kimi‑k3) costs ~10 % less than a geographic profile.
- Access via Bedrock’s OpenAI‑compatible Responses and Chat Completions APIs, or the Invoke and Converse API endpoints.
- Prerequisites: active AWS account with Bedrock access, Python 3.10+, IAM permissions bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream, bedrock:CallWithBearerToken.
Engineers can now run a very large, open‑weight LLM with massive context and vision capabilities at production scale, benefiting from lower latency, reduced token costs, and strict data‑privacy guarantees—all without changing existing security postures.
✅ Practical Steps
- Install the AWS SDK for Python (boto3) in a Python 3.10+ environment and ensure IAM permissions for bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream, and bedrock:CallWithBearerToken.
- Call Kimi K3 using the `bedrock-runtime` endpoint with the OpenAI‑compatible Responses or Chat Completions API, selecting the global profile `global.moonshotai.kimi-k3` for cost‑effective cross‑Region inference.
- Enable prompt caching in your request payload to reuse context and lower latency/cost.
Want the full story? Read the original article.
Read on AWS ML Blog ↗