← Back
AWS ML Blog

Accelerating software delivery with agentic QA automation using Amazon Nova Act – Part 3

•10 min read•
#agents#deployment
Level:Advanced
For:AI Engineers
✦TL;DR

The post introduces two scaling features for QA Studio, an Amazon Nova Act‑based QA automation platform: “applications” to group tests by system under test and expose health metrics, and team‑based user management for access control. QA Studio records each test execution as Amazon CloudWatch custom metrics (by application and environment) and surfaces a health score from 0–100 that starts from the pass rate and penalizes flaky tests. Engineers can view pass rate, execution count, failure count, and health score on an application’s dashboard and analyze trends over selectable 7‑day hourly or 30‑day daily windows.

⚡ Key Takeaways

  • Health score is a 0–100 metric derived from pass rate minus a penalty for flaky tests.
  • Applications act as containers for use cases and test suites, each tied to a base URL and optional environments (dev, staging, prod).
  • Metrics are stored as Amazon CloudWatch custom metrics, dimensioned by application and environment, enabling trend charts.
  • QA Studio’s dashboard shows pass rate, total executions, current failures, and health score for quick health assessment.
  • Trend analysis supports a 7‑day window with hourly granularity or a 30‑day window with daily granularity.
💡 Why It Matters

By aggregating test results into actionable health scores and providing scoped ownership, engineers can quickly identify unstable applications and enforce accountability, reducing manual triage in large‑scale CI/CD pipelines.

✅ Practical Steps

  1. Create an application in QA Studio by providing a name, base URL, and optional environments (development, staging, production).
  2. Run tests through QA Studio; execution outcomes are automatically recorded as Amazon CloudWatch custom metrics.
  3. Use the application detail page to monitor pass rate, execution count, failure count, and health score, and select a 7‑day hourly or 30‑day daily window to view trend charts.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

New in AWS Quick: More capable flows, an updated web experience, and agents that work together

AWS ML Blog•#agents

Three insights you may have missed from theCUBE’s coverage of the Dell Technologies New York Symposium

SiliconANGLE AI•#deployment

How Google Duplex evolved to also call businesses for you in Gemini

Google Research•#agents

Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING