← Back
AWS ML Blog

Introducing Kimi K3 on Amazon Bedrock

•8 min read•
#bedrock#llm#amazon
Level:Advanced
For:AI Engineers
✦TL;DR

Kimi K3, Moonshot AI’s 2.8‑trillion‑parameter open‑weight model, is now available on Amazon Bedrock, offering a 1‑million‑token context window, native vision, and an approximate 2.5× scaling‑efficiency gain over Kimi K2. It introduces explicit prompt‑caching to cut latency and input costs, and can be accessed via Bedrock’s OpenAI‑compatible Responses and Chat Completions APIs or the Invoke/Converse APIs. Engineers can route calls through a global cross‑Region inference profile (global.moonshotai.kimi‑k3) for roughly 10 % lower cost versus a geographic profile. This enables secure, high‑throughput inference for long‑running coding and knowledge‑workflows while keeping data within the AWS boundary.

⚡ Key Takeaways

  • Model size: 2.8 trillion parameters; context window: 1 million tokens; scaling efficiency: ~2.5× over Kimi K2.
  • Prompt‑caching support reduces latency and input‑token cost when reusing context across calls.
  • Global cross‑Region inference profile (global.moonshotai.kimi‑k3) costs ~10 % less than a geographic profile.
  • Access via Bedrock’s OpenAI‑compatible Responses and Chat Completions APIs, or the Invoke and Converse API endpoints.
  • Prerequisites: active AWS account with Bedrock access, Python 3.10+, IAM permissions bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream, bedrock:CallWithBearerToken.
💡 Why It Matters

Engineers can now run a very large, open‑weight LLM with massive context and vision capabilities at production scale, benefiting from lower latency, reduced token costs, and strict data‑privacy guarantees—all without changing existing security postures.

✅ Practical Steps

  1. Install the AWS SDK for Python (boto3) in a Python 3.10+ environment and ensure IAM permissions for bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream, and bedrock:CallWithBearerToken.
  2. Call Kimi K3 using the `bedrock-runtime` endpoint with the OpenAI‑compatible Responses or Chat Completions API, selecting the global profile `global.moonshotai.kimi-k3` for cost‑effective cross‑Region inference.
  3. Enable prompt caching in your request payload to reuse context and lower latency/cost.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

Censored vs. Truncated Regression in Python, Explained

Towards Data Science•#llm

How Google Duplex evolved to also call businesses for you in Gemini

Google Research•#agents

Q&A: Enabling more human-like interactions and understanding in multimodal foundation models

MIT News AI•#llm

Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING