menu
{ "item_title" : "Python AI Agent Evals and Regression Testing", "item_author" : [" Marcus C. Lauritsen "], "item_description" : "Can you trust your AI agent when the next update changes how it behaves?An AI agent may give a convincing answer while choosing the wrong tool, mishandling a request, or overlooking a security rule. Knowing how to catch these problems can feel intimidating, especially if you're new to AI testing.Python AI Agent Evals and Regression Testing makes the process approachable. With basic Python knowledge-no prior AI evaluation or advanced machine learning experience required-you'll learn to build repeatable tests, investigate failures, and measure agent reliability.Clear explanations and manageable coding steps guide you through a practical ServiceDesk Agent Evaluation Lab. Every test builds your confidence, and every mistake becomes an opportunity to understand and improve your system.Key FeaturesHands-on Python project: Develop a complete agent testing lab through connected examples.Practical tools: Work with pytest, Pydantic, mocks, evaluation datasets, and GitHub Actions.Realistic testing scenarios: Explore incorrect tool calls, authorization failures, timeouts, retries, and security risks.Step-by-step learning: Progress from basic automated tests to regression comparisons and CI/CD quality gates.What You Will LearnDesign evaluation datasets and measurable success criteria.Write unit, integration, and parameterized tests with pytest.Validate agent tool selection, arguments, permissions, and execution order.Evaluate multi-step workflows, failures, and recovery strategies.Identify prompt injection risks and unsafe information handling.Measure task success, response quality, latency, and cost.Compare agent versions and detect behavioral regressions.Automate testing and release checks using CI/CD pipelines.Who This Book Is ForIdeal for Python developers, aspiring AI engineers, software testers, QA professionals, and self-learners seeking practical AI testing skills.Whether you're exploring AI agent evaluation for the first time or strengthening your software testing knowledge, this guide provides an accessible learning path. Basic Python familiarity is recommended, but advanced AI expertise is unnecessary.Table of ContentsChapter 1: Foundations of AI Agent Evaluation and TestingChapter 2: Building a Reliable Python Agent Testing FrameworkChapter 3: Engineering Evaluation Datasets and Test CasesChapter 4: Testing AI Agent Tool Calls and Execution ContractsChapter 5: Evaluating Multi-Step Agent Workflows and Failure RecoveryChapter 6: Security, Authorization, and Adversarial Agent TestingChapter 7: Measuring Agent Quality and Combining Evaluation MethodsChapter 8: Detecting Regressions and Comparing Agent VersionsChapter 9: Integrating Agent Evaluations into CI/CD and Production WorkflowsStart building AI agents you can confidently test, understand, and improve.Take your first step toward reliable AI development today with a practical guide that turns complex testing challenges into achievable skills.", "item_img_path" : "https://covers2.booksamillion.com/covers/bam/9/79/816/592/9798165923517_b.jpg", "price_data" : { "retail_price" : "38.99", "online_price" : "38.99", "our_price" : "38.99", "club_price" : "38.99", "savings_pct" : "0", "savings_amt" : "0.00", "club_savings_pct" : "0", "club_savings_amt" : "0.00", "discount_pct" : "10", "store_price" : "" } }
Python AI Agent Evals and Regression Testing|Marcus C. Lauritsen

Python AI Agent Evals and Regression Testing : A Hands-On Guide to Evaluation Datasets, Tool-Call Validation, Automated Tests, and CI/CD Quality Gates

local_shippingShip to Me
In Stock.
FREE Shipping for Club Members help

Overview

Can you trust your AI agent when the next update changes how it behaves?

An AI agent may give a convincing answer while choosing the wrong tool, mishandling a request, or overlooking a security rule. Knowing how to catch these problems can feel intimidating, especially if you're new to AI testing.

Python AI Agent Evals and Regression Testing makes the process approachable. With basic Python knowledge-no prior AI evaluation or advanced machine learning experience required-you'll learn to build repeatable tests, investigate failures, and measure agent reliability.

Clear explanations and manageable coding steps guide you through a practical ServiceDesk Agent Evaluation Lab. Every test builds your confidence, and every mistake becomes an opportunity to understand and improve your system.

Key Features
  • Hands-on Python project: Develop a complete agent testing lab through connected examples.

  • Practical tools: Work with pytest, Pydantic, mocks, evaluation datasets, and GitHub Actions.

  • Realistic testing scenarios: Explore incorrect tool calls, authorization failures, timeouts, retries, and security risks.

  • Step-by-step learning: Progress from basic automated tests to regression comparisons and CI/CD quality gates.

What You Will Learn
  • Design evaluation datasets and measurable success criteria.

  • Write unit, integration, and parameterized tests with pytest.

  • Validate agent tool selection, arguments, permissions, and execution order.

  • Evaluate multi-step workflows, failures, and recovery strategies.

  • Identify prompt injection risks and unsafe information handling.

  • Measure task success, response quality, latency, and cost.

  • Compare agent versions and detect behavioral regressions.

  • Automate testing and release checks using CI/CD pipelines.

Who This Book Is For

Ideal for Python developers, aspiring AI engineers, software testers, QA professionals, and self-learners seeking practical AI testing skills.

Whether you're exploring AI agent evaluation for the first time or strengthening your software testing knowledge, this guide provides an accessible learning path. Basic Python familiarity is recommended, but advanced AI expertise is unnecessary.

Table of Contents

Chapter 1: Foundations of AI Agent Evaluation and Testing

Chapter 2: Building a Reliable Python Agent Testing Framework

Chapter 3: Engineering Evaluation Datasets and Test Cases

Chapter 4: Testing AI Agent Tool Calls and Execution Contracts

Chapter 5: Evaluating Multi-Step Agent Workflows and Failure Recovery

Chapter 6: Security, Authorization, and Adversarial Agent Testing

Chapter 7: Measuring Agent Quality and Combining Evaluation Methods

Chapter 8: Detecting Regressions and Comparing Agent Versions

Chapter 9: Integrating Agent Evaluations into CI/CD and Production Workflows

Start building AI agents you can confidently test, understand, and improve.

Take your first step toward reliable AI development today with a practical guide that turns complex testing challenges into achievable skills.

This item is Non-Returnable

Details

  • ISBN-13: 9798165923517
  • ISBN-10: 9798165923517
  • Publisher: Independently Published
  • Publish Date: October 2026
  • Dimensions: 10 x 7 x 0.61 inches
  • Shipping Weight: 1.12 pounds
  • Page Count: 292

Related Categories

You May Also Like...

    1

BAM Customer Reviews