Set up OpenAI ChatGPT Codex with LiteLLM on Amazon ECS and Amazon Bedrock
This article details how to deploy a customer‑operated LiteLLM gateway on Amazon ECS (Fargate) to mediate OpenAI Codex requests to Amazon Bedrock, providing centralized authentication, routing, budgeting, rate‑limiting, and telemetry. The architecture routes Codex’s /v1/responses calls through an Application Load Balancer and AWS WAF before reaching LiteLLM, which then invokes the approved Bedrock model using its ECS task role. Supporting services include Amazon RDS PostgreSQL for state, Secrets Manager/KMS for key storage, and CloudWatch for logging and health monitoring, enabling enterprises to enforce consistent model‑access controls while keeping tool execution local.
⚡ Key Takeaways
- LiteLLM is deployed on Amazon ECS (Fargate) as a gateway for OpenAI Codex to Amazon Bedrock.
- An Application Load Balancer and AWS WAF front‑end the gateway, applying network and web‑layer controls.
- Codex interacts with LiteLLM via the /v1/responses endpoint, which enforces budgets, rate limits, and usage telemetry.
- The deployment stores state in Amazon RDS PostgreSQL and secrets in AWS Secrets Manager with encryption via AWS KMS.
- The gateway does not receive a general‑purpose shell and does not replace Codex’s local tool‑approval sandbox.
Engineers can enforce enterprise‑wide model governance and cost controls without altering developers’ local workflows, simplifying secure, scalable adoption of generative AI coding agents.
✅ Practical Steps
- Run the “LiteLLM on AWS quickstart” to provision the LiteLLM gateway on Amazon ECS (Fargate) with the required ALB, WAF, and supporting services.
- Configure OpenAI Codex to send its task context and tool definitions to the gateway’s /v1/responses endpoint.
- Optionally enable AWS WAF managed protections and source‑IP rate limiting for additional security.
Want the full story? Read the original article.
Read on AWS ML Blog ↗