Slack Engineering has introduced Agentic Testing, an AI-driven approach to end-to-end test automation. It uses AI agents that adapt to UI and system changes at runtime, aiming to improve resilience in test automation for distributed systems.
SynthePulse Insight · AI deep reading
Version 1 · 1 source
Faced with the fragility of traditional end-to-end testing in dynamic systems, the Slack engineering team introduces agent-driven testing, allowing AI agents to dynamically execute tests based on target intent, reducing test failures and maintenance costs caused by UI or service changes.
In continuous delivery environments, end-to-end tests frequently fail due to UI or interface changes that are not actual functional regressions. Such changes include modified interface structures or shifted elements, invalidating the assumptions of fixed steps and stable selectors. The Slack engineering team points out that this fragility increases maintenance overhead, consuming significant time to diagnose and fix non-substantive failures.
Agent-driven testing is goal-oriented: tests no longer describe a strict sequence of 'click → click → input → assert,' but rather express high-level intent. After receiving the intent, the AI agent plans, executes actions, observes the application state, and makes dynamic decisions. When encountering minor changes, the agent attempts alternative paths to continue execution rather than failing immediately. The final result is verified against engineer-predefined assertions, while the complete execution trail, including every decision and interaction, is recorded.
The key to this process is the agent's adaptability: it continuously evaluates the current state until the goal is achieved or a stopping condition is met. Constraint mechanisms define allowed operations, exploration scope, and termination conditions, ensuring controllable behavior.
Slack views agent-driven testing as a supplement to existing methods, not a replacement. Deterministic end-to-end tests remain the mainstay for regression verification, ensuring critical logic and contract correctness. Agent tests are placed at the top of the testing pyramid, applied to the end-to-end layer sensitive to UI and structural changes. Their advantages lie in handling complex UI behaviors, debugging flaky workflows, and reproducing production issues, while scripted tests stably cover predefined paths.
Slack engineers emphasize that each mode has its use cases: scripted tests provide fast, repeatable verification; agent tests dynamically adapt to changes in a goal-oriented model, reducing failures caused by superficial changes.
Due to cost issues, the Slack blog indicates that agent-driven testing is currently more suitable for targeted debugging and exploratory testing, rather than frequent execution in CI pipelines. This limits its timeliness in large-scale regression testing but provides a new tool for debugging flaky tests and exploring complex scenarios. The structured design of execution logs gives teams the ability to replay and inspect failures, enhancing observability.
Slack's agent-driven testing reflects a new trend in AI for software testing: shifting from predefined scripts to an intent-driven, dynamically adaptive model. Although still in early stages and with cost as a major bottleneck, it provides a promising resilient solution for systems with frequent UI changes. In the future, it may be applied in broader scenarios, forming a complementary ecosystem with deterministic tests.
This article is based on InfoQ's coverage of the Slack engineering blog. Slack is a credible source, but no third-party verification or quantitative data is provided. The reasoning is based on the report's description.
Agent-driven testing dynamically adapts to UI changes using AI agents, enhancing end-to-end test resilience, but is currently cost-constrained and suitable for targeted debugging, complementing traditional deterministic tests.
Primary report
Primary source