Amazon AMET Slashed Test Case Generation from Weeks to Hours: How an AI Agent Accelerated QA Processes

At Amazon.ae, serving approximately 10 million customers monthly across five countries in the Middle East and North Africa, the AMET Payments team releases an average of five new features each month. Each new feature requires comprehensive test case generation, which traditionally consumed one week of manual effort per project. Following the implementation of a multi-agent AI solution, this time was reduced to mere hours, freeing up one QA engineer per year and significantly improving test coverage quality.

In large companies like Amazon, routine tasks silently consume vast resources. Weeks are spent creating test cases, analyzing documentation, and performing routine checks that add no new value but merely maintain existing processes. These tasks divert highly skilled engineers from strategic work, slow down new feature releases, and increase development costs. Today, technologies exist that can alleviate this burden, making QA processes faster and more accurate.

The Problem of Manual Test Case Generation at Amazon AMET

The AMET Payments team at Amazon.ae is responsible for managing payment selections, transactions, and affordability features across several countries. This means an average of five new features are released monthly, each requiring thorough verification. Before the AI agent implementation, generating test scenarios for each feature took approximately one week of manual labor.

Quality Assurance (QA) engineers spent this time analyzing extensive documents: Business Requirement Documents (BRDs), design documents, UI mocks, and historical test preparations. This process was so labor-intensive that it required one full-time QA engineer annually just for test case creation. This not only slowed down the product release cycle but also diverted valuable engineering resources from more strategic testing initiatives.

Why Traditional AI Approaches Didn't Work

Initial attempts at automation involved feeding entire BRDs to a single AI agent for test case generation. However, this approach often produced generic and non-specific outputs, such as "verify payment works correctly." The Amazon AMET team needed highly detailed scenarios, for example: "verify that when a UAE customer selects cash on delivery (COD) for an order above 1,000 AED with a saved credit card, the system displays the COD fee of 11 AED and processes the payment through the COD gateway, with the order state transitioning to 'pending delivery'."

Context length restrictions prevented effective processing of large documents. The lack of specialized processing phases meant the AI couldn't understand testing priorities or employ risk-based approaches. Hallucination issues led to irrelevant scenarios that could mislead the QA team. The root cause was clear: the AI attempted to compress complex business logic without the iterative thinking process that experienced testers employ when analyzing requirements.

The Human-Centric Breakthrough: How the SAARAM AI Agent Was Designed

The breakthrough occurred when the Amazon AMET team shifted its approach, asking the question: "How do experienced human testers think?" Instead of merely optimizing AI algorithms, they focused on studying human cognitive patterns. Research and interviews with senior QA professionals revealed that experienced testers do not process documents holistically. They go through specialized mental phases: first, they analyze by extracting acceptance criteria, identifying customer journeys, UX requirements, and mapping them to product requirements. Then, they develop tests through a systematic process: journey analysis, scenario identification, data flow mapping, test case development, and finally, organization and prioritization.

This insight led to the design of SAARAM (QA Lifecycle App) — a multi-agent AI solution that mimics these expert approaches. Each specialized AI agent focuses on a specific aspect of the testing process, similar to how a human expert mentally compartmentalizes different analysis phases. Thus, the original AI agent was decomposed into sequential "thinking" actions, each becoming a separate step:

• Agent 1: Customer Segment Creator. Uses four subagents for customer segmentation analysis, segment discovery, parameter-based matrix creation, E2E scenarios, and detailed test case development.
• Agent 2: User Journey Mapper. Employs four subagents for comprehensive product journey mapping, creating flow and sequence diagrams, generating E2E scenarios, and detailed test documentation.
• Agent 3: Customer Segment x Journey Coverage. Combines inputs from Agents 1 and 2 to create detailed segment-specific analyses, using flow diagrams, user journeys, sequence diagrams, and corresponding test steps.
• Agent 4: State Transition Agent. Analyzes various product states in user journey flows, creating state diagrams, segment-specific state scenario diagrams, and generating related test scenarios and steps.

Implementation and Evolution

The first iteration of SAARAM processed a document and generated comprehensive test coverage. However, during operation, limitations were identified: context and hallucination issues due to isolated agent operations, inefficient data generation, limited parsing capabilities (Word documents only), and rigid architecture.

The second iteration of SAARAM was a complete reimagining of the agentic workflow architecture. It was rebuilt from the ground up with an emphasis on modularity, context-awareness, and extensibility:

• Agent 1: Intelligent Gateway. Determines the file type (documents, Figma designs, code repositories) and directs data to appropriate agents.
• Agent 2: Specialized Data Extractor. Uses six specialized subagents for parallel data extraction from various domains, ensuring deep coverage and domain-specific knowledge.
• Agent 3: Visualizer. Transforms extracted data into six distinct diagram types (ERD, flow diagrams, requirements, UX, process, Mind Maps), providing multiple perspectives for analysis.
• Agent 4: Data Condenser. Synthesizes data, ensuring that each subsequent agent receives only the information necessary for its specialization, reducing noise and increasing accuracy.

This approach significantly reduced context and hallucination problems, improved data generation efficiency, expanded parsing capabilities, and made the architecture more flexible and adaptable to changes.

Results

Metric	Before AI Agent Implementation	After AI Agent Implementation
Test case generation time for one feature	1 week	Several hours
Human resource cost for test case generation	1 full-time QA engineer per year	Significant resource liberation
Test coverage quality	Baseline level	Improved

The implementation of the SAARAM AI agent allowed the Amazon AMET Payments team to reduce test case generation time from one week to several hours. This resulted in the freeing up of one QA engineer per year for more strategic tasks and significantly improved the quality of test coverage. The solution demonstrated how studying human cognitive patterns, rather than solely optimizing AI algorithms, can create production-ready systems that enhance, rather than replace, human expertise.

How to Implement This in Your Business

The Amazon AMET case clearly demonstrates that AI agents can radically accelerate routine but critically important processes. If your company has tasks that require deep documentation analysis and the creation of detailed scenarios, this is a ready-made case for implementing an AI agent:

• Study the cognitive patterns of your experts. Instead of simply automating a task, understand how your best specialists think and act. Break down their thought process into stages and assign each stage to a separate AI agent.
• Develop a multi-agent architecture. Use multiple specialized AI agents, each responsible for its stage of the process. This will allow for more accurate and efficient processing of complex tasks, reducing the risk of hallucinations.
• Focus on structured output. Define the exact data format you need at each stage and configure the agents to generate only that format. This will significantly increase the accuracy and usefulness of the results.
• Integrate AI agents into existing processes. The less employees have to change their habits, the faster they will adopt new technology. Embed agents into the tools and systems currently in use.

If this case sounds like what's happening in your company, our manager can help: he'll analyze your business and niche for free and point out where an AI agent would bring a real result in your case. Message the manager