Nordic Testing Days 2026

Writing mobile tests with Appium can be challenging. Many of us have dealt with unstable selectors, differences between Android and iOS, or slow progress. What if we could use artificial intelligence with Appium? And what exactly is Mobile MCP?

Tutorial Overview

This tutorial is for anyone interested in starting mobile testing with a modern approach. We’ll use Python throughout. Together, we will build an end-to-end framework where AI supports us from the beginning. We will see how tools like Mobile MCP can find elements without needing static IDs or XPath.

We’ll focus on building full end-to-end scenarios. We’ll test whether self-healing really works or is just hype. We’ll also look at where AI can speed up our work, and where we still need the Appium Inspector.

Note: This is a hands-on tutorial and requires some preparation upfront. We’ll send you detailed instructions in advance.


Agenda

1. AI in Appium: We’ll talk about the main challenges in classic mobile testing and what AI can help solve.
2. E2E framework architecture: How to set up a project for AI-powered end-to-end testing, going beyond the basic Page Pattern.
3. What is "Mobile MCP"? We’ll give a practical introduction to the tool and show how it works with Appium.
4. AI vs. Appium Inspector: We’ll see how AI can find elements in an app as they change.
5. Building an E2E Scenario: Step by step, we’ll create a full test path, like logging in, searching, and adding to the cart.
6. Intelligent assertions and self-healing: We’ll see how AI helps check the app’s state and what it does when the UI changes.
7. Results analysis: We’ll look at how AI can help us figure out why an end-to-end test failed.
8. Summary: AI in mobile - revolution or evolution? We’ll finish with a Q&A session.

Traditional software is deterministic: the same input yields the same output. Large Language Models (LLMs) and AI Agents have shattered this rule, introducing an inherently probabilistic paradigm. How do we ensure quality when the ground truth is shifting? This tutorial bridges the gap between traditional QA and AI evaluation. We will move beyond simple prompt testing into validating complex multi-agent systems. Participants will learn to build test oracles that evaluate intent and semantics rather than exact matches, evolving the QA role from a code verifier to an evaluation framework architect.

Learning Objectives
By the end of this workshop, participants will be able to:

• Deconstruct AI Architectures: Identify specific testable layers such as Shell (API/UI), Orchestration (Context/Tools), and Inference Core (Probabilistic).
• Build Modern Test Oracles: Implement aggregated and property-based oracles using Python to handle non-deterministic outputs.
• Validate Multi-Agent Systems: Apply a four-level framework to test communication, delegation, and error propagation between AI agents.
• Execute AI Red Teaming: Identify vulnerabilities such as prompt injection, hallucinations, and safety bypasses.
• Automate Quality Metrics: Integrate BERTScore and RAG-specific metrics such as Faithfulness and Relevance into CI/CD pipelines.

Target Audience & Prerequisites

• Target Audience: QA Engineers, SDETs, and Developers working with or transitioning to Generative AI systems.
• Prerequisites: Basic knowledge of Python and API fundamentals. Participants must bring a laptop with VS Code and Python installed.

Workshop Outline

1. The Paradigm Shift
   • Theory: Deterministic vs. probabilistic testing. Agent taxonomy.
   • Practice: Environment setup and executing your first fuzzy test.
2. Oracles & Orchestration
   • Theory: Atomic vs. aggregated oracles. Testing the orchestration layer.
   • Practice: Writing scripts to validate JSON schemas and output consistency.
3. Semantic Evaluation
   • Theory: RAG metrics such as Faithfulness and Relevance. Introduction to BERTScore.
   • Practice: Building an LLM-as-a-Judge evaluator to grade complex answers.
4. Multi-Agent Testing
   • Theory: Inter-agent communication and task delegation loops.
   • Practice: Debugging a workflow where a Travel Agent delegates to a Finance Agent.
5. Red Teaming & Security
   • Theory: Prompt injection, mutation testing, and metamorphic testing.
   • Practice: Simulated attack scenarios, bypassing safety filters, and implementing guardrail fixes.
6. QA Strategy & Governance
   • Theory: Human-in-the-loop workflows and production monitoring.
   • Practice: Designing a full-scale QA strategy for a real-world GenAI product.

Artificial Intelligence is increasingly used to write test code. Today it is estimated that 60% of new test code is written by AI tools and this number will only continue to rise. Surprisingly, however, the testing is now the critical component needed is to ‘test’ the 40% that AI is unable to test itself, that is where the future of testers.

That 40% is not busy work. This is not your typical motivational speech dressed up as another AI testing keynote. What I will talk about in the keynote is the three critical capabilities that are becoming more valuable not less, and the underlying reason for this is that all of them depend on something that current models lack: the human intent behind engineering quality software for other humans to use.

• Engineer the Context Understanding: While AI can greatly amplify the effectiveness of testing, the quality of results is only as good as the input you give it. In 2026, the testers who will rise to the top will not be the ones who came up with the cutest prompt for the AI to answer. Rather, they will be those who can successfully engineer the context of the problem for the AI to solve. We will look at real-world best practices for testing with AI, and I will share an AI Assurance playbook for context engineering that will immediately raise the quality of your AI-infused testing.
• Review with Heuristic Judgment: AI generated test suites look good - until they fail to test what really matters silently. Green pipelines are the most dangerous artefact in organisations today. No one asks: what are we not testing? This capability helps organisations audit test suites generated by AI with a sense of heuristic judgement. I can explain how to design world class AI augmented tests like an experienced navigator reading a map to identify the blank spaces where organisation specific risks reside.
• Orchestrate Trust: Humans trust humans they still need decide when the machine is wrong. That is not a technical skill - it is a quality leadership act. This capability describes the shifting role of quality engineers to trust AI Assurance oracles. This capability explores the new role of orchestrating AI Assurance that trust across teams, tools, and stakeholders, and answers the question: “Who is really responsible for quality?”

By the end of this keynote, you will have an assessment of your skills in planning, architecting, designing and leading AI governance, a practical plan of action to implement in the real world, and the assurance that you will be “ready” to become part of the “Irreplaceable 40%”.

Key Takeaways:

• Context Engineering is our New Superpower: AI-augmented tests are only as valuable as the context a human provides. Attendees will leave with a repeatable framework for curating system, user, and business context that transforms AI output from "technically correct" into "intent driven tests" - a skill that compounds in value as the important of testing of AI infused systems improve.

• Heuristic Auditing Catches the Real Truth: A passing automated test regression suite is not proof of quality; it is proof that the tests you wrote passed. Attendees will gain practical heuristic-led auditing patterns to interrogate value-driven testing, identify dangerous blind spots, and ask the questions that thinking machines are structurally incapable of asking themselves.

• Trust Orchestration is the Quality Engineer's Next Career: The highest-value skill in an AI-augmented team is not technical testing - it is the ability to calibrate, communicate, and own confidence decisions across people, tools, and stakeholders. Attendees will understand how to position themselves as the trust architect oracles in their organisation increasingly needs, turning a perceived threat into their most durable career advantage.

Intro:Artificial Intelligence challenges almost every assumption the testing discipline is built on. Traditional testing depends on fixed inputs and predictable logic, but AI systems are adaptive, probabilistic, and context-dependent. That means our classical test cases are no longer stable reference points.


In this 20-minute talk, Nicole van Gijn explores what testing looks like when your system learns, reasons, and occasionally hallucinates. She introduces the AI Quality Grid, a structured framework co-developed with John Kronenberg, that helps define quality attributes, risks, and validation strategies for AI applications. The session bridges theory and practice through concrete examples from a real AI test project, showing how LLM-evals and risk-based thinking can be combined to test prompt robustness, output consistency, and bias control within modern CI/CD pipelines.


Attendees will walk away with a lightweight but actionable structure for AI quality assessment and a new mindset: understanding quality not as a checklist, but as an intelligent, adaptive discipline. Why this topic is relevant:

1. AI systems are rapidly entering production pipelines, yet testing methods lag behind.
2. Testers and QA leads urgently need practical models to evaluate non-deterministic outputs.
3. The AI Quality Grid offers a bridge between AI model evaluation (LLM-evals) and classical test strategy, providing testers with new tools and thinking patterns to stay relevant in the AI era.

Deploying to production shouldn't require a meeting, three approvals, and a prayer. Yet most teams treat every release like launching a rocket - mission control on standby, everyone watching the countdown, but no one wanting to press the red button.

At Sokos Hotels, our web booking system handles thousands of reservations daily, processing millions monthly across 50 hotels all over Finland and Estonia. One critical bug means lost revenue; one outage means thousands of unhappy guests. We were trapped in weekly releases, manual verification, and the kind of Thursday tension that put everyone in the mission control room. We asked ourselves if Continuous Deployment is just a myth.

In less than a year, we broke the cycle. We went from weekly manual releases to deploying seven times per day with higher confidence than ever. The results? 4.5/5.0 customer effort score, 40% higher conversion rates and 4.3+/5.0 overall team happiness. The secret? A testing strategy that made deployment boring for the last 2 years.

Join my talk to find out how we did it!

What you’ll get from this talk:

1. The three signs your team is not ready for Continuous Deployment, and the technical enabler that breaks the testing tension. 
2. How we shifted from “QA signs off on releases” to "Quality is built-in", and why the cultural change was harder than the technical one. 
3. The valuable failure lessons we learned along the way and what it taught us about green pipeline.

Your takeaway: A practical, battle-tested roadmap for testing-enabled Continuous Deployment. You'll leave knowing which tests to automate first, how to build confidence without sacrificing speed, and how to prove to skeptics that this isn't just another risky experiment. Real patterns, real failures, real results - ready to implement Monday morning.

Who should attend: QA engineers, test automation engineers, developers, engineering managers, and DevOps practitioners who believe testing should accelerate delivery, not slow it down.

In this talk, I show how we run a lot of full end-to-end Cypress web application tests every day. In addition to running the full data set, we do separate feature test runs based on test tags. We also allow everyone from all teams to trigger the tests right from GitHub Actions UI. This lets every group quickly test their feature before merging into the main branch. For pull requests, we employ source code analysis based on data test IDs to run the affected tests first for quicker feedback. The software automation team uses the flake test information to chase the sources of the underlying errors to minimize noise and make every passing test run give us confidence in the released code, and every failing test run useful to quickly diagnose the real underlying issue. The presentation covers test writing, test organization, selecting tests to run based on the source code changes, running tests in different resolutions. I also look into making the tests faster by employing data creation and caching, as well as using API calls to bypass the user interface in some places. Finally, making the tests robust and flake-free and triaging the failed runs is an ongoing activity for the automation team.


Key takeaways:

1. How to run 1000 of end-to-end tests quickly
2. Which tests to run on a pull request
3. How AI is helping us pick tests to run

Every automation engineer knows the "Maintenance Tax." We spend hours fixing fragile XPaths and CSS selectors just to verify if a button is visible or a chart is correct. In fact, our internal code analysis shows that assertion logic is typically five times (5x) larger in code size than action logic, consuming up to three months of dedicated maintenance effort every year.In this session, I introduce ViTO (Visual Testing Oracle), a production-deployed framework that ends the era of DOM-dependent verification. By leveraging multimodal Generative AI (GenAI), ViTO "sees" the application exactly like a human does.I will share our industrial experience of decoupling verification from the underlying code, resulting in a 50% reduction in our assertion codebase. You will see how we replaced thousands of lines of brittle verification logic with resilient, prompt-driven visual oracles that can handle complex data visualisations and unseen UI faults with zero extra effort. If you are tired of your tests breaking because a div changed, it's time to shift from structural selectors to a visual AI oracle.


Key takeaways:

1. Prompt Engineering for Testers: How to write resilient "Assertion Prompts" that replace complex conditional code and handle visual regression automatically.
2. Real-World ROI: Evidence-based results from a production environment, showing a 50% reduction in code maintenance and expanded coverage for rich UI components.
3. Deterministic AI: Practical strategies to control GenAI hallucinations using "concentrated screenshots" and "dynamic HTML filtering."

Teams often rely on test cases executed, bugs reported, and pass rates to measure success. These numbers might look impressive, but do they truly reflect software quality? Vanity metrics can mislead teams, encourage the wrong behaviours, and create a false sense of progress.   This talk introduces a 7-step framework to move beyond superficial KPIs and focus on metrics that drive real value. Inspired by analytical approaches in competitive sports, this model helps teams make better decisions, align testing efforts with business goals, and ensure that data supports meaningful improvements.


Key takeaways:

1. The risks of vanity metrics and how they can mislead decision-making.
2. How to design KPIs that focus on value, not just activity.
3. A practical framework to ensure testing metrics drive meaningful change.

Visual testing is supposed to protect QA teams from the familiar “it looks wrong” bug, yet traditional pixel-diff approaches only show that something changed, not whether that change actually matters. As modern interfaces grow more dynamic and design systems become more complex, teams need smarter ways to detect meaningful visual regressions.

This talk presents a practical approach to automated visual bug detection using multimodal LLMs. Drawing on a real-world implementation, it shows how AI models from providers such as OpenAI, Anthropic, and Google can be orchestrated to analyze screenshots and identify issues that pixel-based tools often cannot interpret on their own. These include layout breaks, missing elements, accessibility concerns, color contrast problems, and platform-specific guideline violations.

The session explores how AI-driven visual analysis can move beyond pixel-perfect comparison toward semantic understanding, helping teams distinguish intentional UI changes from genuine defects. It also addresses one of the biggest challenges in visual testing at scale: false positives, demonstrating how agent-based review systems can reduce noise while still surfacing critical issues.

Attendees will leave with practical ideas for using multimodal AI to strengthen visual testing workflows and make automated UI validation more accurate, scalable, and useful.

Key Takeaways:

1. How to evolve from “pixel diffs” to impact-based automated visual feedback
2. Patterns that turn image feedback into structured results (what changed, where, severity, why it matters)
3. Tips for integrating automated LLM-powered visual feedback into existing automated UI test frameworks