Nordic Testing Days 2026

Writing mobile tests with Appium can be challenging. Many of us have dealt with unstable selectors, differences between Android and iOS, or slow progress. What if we could use artificial intelligence with Appium? And what exactly is Mobile MCP?

Tutorial Overview

This tutorial is for anyone interested in starting mobile testing with a modern approach. We’ll use Python throughout. Together, we will build an end-to-end framework where AI supports us from the beginning. We will see how tools like Mobile MCP can find elements without needing static IDs or XPath.

We’ll focus on building full end-to-end scenarios. We’ll test whether self-healing really works or is just hype. We’ll also look at where AI can speed up our work, and where we still need the Appium Inspector.

Prerequisites & Setup for Attendees:
Please complete the guideline beforehand: https://github.com/paciadawid/NTD2026_mobile_ai/blob/master/workshop-setup.md

Agenda

1. AI in Appium: We’ll talk about the main challenges in classic mobile testing and what AI can help solve.
2. E2E framework architecture: How to set up a project for AI-powered end-to-end testing, going beyond the basic Page Pattern.
3. What is "Mobile MCP"? We’ll give a practical introduction to the tool and show how it works with Appium.
4. AI vs. Appium Inspector: We’ll see how AI can find elements in an app as they change.
5. Building an E2E Scenario: Step by step, we’ll create a full test path, like logging in, searching, and adding to the cart.
6. Intelligent assertions and self-healing: We’ll see how AI helps check the app’s state and what it does when the UI changes.
7. Results analysis: We’ll look at how AI can help us figure out why an end-to-end test failed.
8. Summary: AI in mobile - revolution or evolution? We’ll finish with a Q&A session.

Traditional software is deterministic: the same input yields the same output. Large Language Models (LLMs) and AI Agents have shattered this rule, introducing an inherently probabilistic paradigm. How do we ensure quality when the ground truth is shifting? This tutorial bridges the gap between traditional QA and AI evaluation. We will move beyond simple prompt testing into validating complex multi-agent systems. Participants will learn to build test oracles that evaluate intent and semantics rather than exact matches, evolving the QA role from a code verifier to an evaluation framework architect.

Target Audience: QA Engineers, SDETs, and Developers working with or transitioning to Generative AI systems.

Learning Objectives
By the end of this workshop, participants will be able to:

• Deconstruct AI Architectures: Identify specific testable layers such as Shell (API/UI), Orchestration (Context/Tools), and Inference Core (Probabilistic).
• Build Modern Test Oracles: Implement aggregated and property-based oracles using Python to handle non-deterministic outputs.
• Validate Multi-Agent Systems: Apply a four-level framework to test communication, delegation, and error propagation between AI agents.
• Execute AI Red Teaming: Identify vulnerabilities such as prompt injection, hallucinations, and safety bypasses.
• Automate Quality Metrics: Integrate BERTScore and RAG-specific metrics such as Faithfulness and Relevance into CI/CD pipelines.

Prerequisites for Attendees:

• Basic knowledge of Python and API fundamentals. Participants must bring a laptop with VS Code and Python installed. Alternatively, also Cursor, Claude Code, Codex, OpenCode, or any equivalent tool is suitable. In any case, make sure that the chosen agent is already installed, configured and ready for the session.

Workshop Outline

1. The Paradigm Shift
   • Theory: Deterministic vs. probabilistic testing. Agent taxonomy.
   • Practice: Environment setup and executing your first fuzzy test.
2. Oracles & Orchestration
   • Theory: Atomic vs. aggregated oracles. Testing the orchestration layer.
   • Practice: Writing scripts to validate JSON schemas and output consistency.
3. Semantic Evaluation
   • Theory: RAG metrics such as Faithfulness and Relevance. Introduction to BERTScore.
   • Practice: Building an LLM-as-a-Judge evaluator to grade complex answers.
4. Multi-Agent Testing
   • Theory: Inter-agent communication and task delegation loops.
   • Practice: Debugging a workflow where a Travel Agent delegates to a Finance Agent.
5. Red Teaming & Security
   • Theory: Prompt injection, mutation testing, and metamorphic testing.
   • Practice: Simulated attack scenarios, bypassing safety filters, and implementing guardrail fixes.
6. QA Strategy & Governance
   • Theory: Human-in-the-loop workflows and production monitoring.
   • Practice: Designing a full-scale QA strategy for a real-world GenAI product.