Nordic Testing Days 2026

Traditional software is deterministic: the same input yields the same output. Large Language Models (LLMs) and AI Agents have shattered this rule, introducing an inherently probabilistic paradigm. How do we ensure quality when the ground truth is shifting? This tutorial bridges the gap between traditional QA and AI evaluation. We will move beyond simple prompt testing into validating complex multi-agent systems. Participants will learn to build test oracles that evaluate intent and semantics rather than exact matches, evolving the QA role from a code verifier to an evaluation framework architect.

Target Audience: QA Engineers, SDETs, and Developers working with or transitioning to Generative AI systems.

Learning Objectives
By the end of this workshop, participants will be able to:

• Deconstruct AI Architectures: Identify specific testable layers such as Shell (API/UI), Orchestration (Context/Tools), and Inference Core (Probabilistic).
• Build Modern Test Oracles: Implement aggregated and property-based oracles using Python to handle non-deterministic outputs.
• Validate Multi-Agent Systems: Apply a four-level framework to test communication, delegation, and error propagation between AI agents.
• Execute AI Red Teaming: Identify vulnerabilities such as prompt injection, hallucinations, and safety bypasses.
• Automate Quality Metrics: Integrate BERTScore and RAG-specific metrics such as Faithfulness and Relevance into CI/CD pipelines.

Prerequisites for Attendees:

• Basic knowledge of Python and API fundamentals. Participants must bring a laptop with VS Code and Python installed. Alternatively, also Cursor, Claude Code, Codex, OpenCode, or any equivalent tool is suitable. In any case, make sure that the chosen agent is already installed, configured and ready for the session.

Workshop Outline

1. The Paradigm Shift
   • Theory: Deterministic vs. probabilistic testing. Agent taxonomy.
   • Practice: Environment setup and executing your first fuzzy test.
2. Oracles & Orchestration
   • Theory: Atomic vs. aggregated oracles. Testing the orchestration layer.
   • Practice: Writing scripts to validate JSON schemas and output consistency.
3. Semantic Evaluation
   • Theory: RAG metrics such as Faithfulness and Relevance. Introduction to BERTScore.
   • Practice: Building an LLM-as-a-Judge evaluator to grade complex answers.
4. Multi-Agent Testing
   • Theory: Inter-agent communication and task delegation loops.
   • Practice: Debugging a workflow where a Travel Agent delegates to a Finance Agent.
5. Red Teaming & Security
   • Theory: Prompt injection, mutation testing, and metamorphic testing.
   • Practice: Simulated attack scenarios, bypassing safety filters, and implementing guardrail fixes.
6. QA Strategy & Governance
   • Theory: Human-in-the-loop workflows and production monitoring.
   • Practice: Designing a full-scale QA strategy for a real-world GenAI product.

How do you define yourself as a leader? How do you see your leadership? In this personal pitstop we will go over what being a leader means to you. At work, at home, or at your hobby, your leadership skills matter. They play a huge role in how you perceive the world around you, and how others perceive you. As a leadership coach, I’ve picked up a lot of knowledge on how to help and train people on their leadership skills.

In this workshop I’m sharing my best tips.We’ll look at where in your process you currently are, and where you would like to go. We will do this via small games, assessments, and observations from the group. As a group we will help each other. We will set (achievable) goals for you to work on in your ‘Leadership Plan’ that you will take home.

Key takeaways:

• Assess and identify your leadership styleUnderstand your communication style
• Create an achievable Leadership Plan to take home (that works!)

The software testing profession has been around for approximately 70 years, yet nothing has fundamentally transformed it to deliver on what it was always capable of. The majority of our industry has delivered "glorified clerical work" in the name of testing. Industry reports show that almost 70% of testing capacity is spent on testing-related activities, while only 30% is devoted to actual testing that creates real value.

Organizations have been trying to automate away all things testing for decades. It never worked because the real value of testing comes from the intellectual part i.e. asking the right questions, critical evaluation, risk analysis, deep exploration, and informed decision-making. But mastering this craft requires years of investment that organizations see as overhead. Hence, the widespread acceptance of "testing as artefact-building" - easy to automate, but without substantial value.

What if you could deliver at scale and speed without compromising the value real testing creates? Agentic Quality Engineering gives every tester access to expert-level thinking without years of investment. AI agents built on 47 years of combined practitioner experience based on the award-winning QCSD (Quality Conscious Software Delivery) framework, context-driven approaches, risk-based thinking, deep exploration techniques - all encoded into 41 specialized skills and 30 purpose-built agents. The agents are self-learning, building institutional knowledge over time. They collaborate with other agents, with humans, and with existing systems. This isn't automation replacing testers; it's accumulated wisdom amplifying what testers can do from day one.

Key takeaways:

• Expert Thinking, Accessible: Leverage decades of encoded testing expertise without years of personal skill developmentHands-On Agent Orchestration: Configure, understand and run multi-agent pipelines that involve AI agents to support test activities across the entire SDLC. It includes 6 Core Agents, 2 Performance Agents, 3 Strategic Agents, 4 Advanced and 3 Specialized agents. More yet, 11 purpose-built agents for widespread coverage of important testing activities.
• The PACT Framework: Evaluate agentic quality systems using Proactive, Autonomous, Collaborative, Targeted principles
• Self-Learning & Collaborative Systems: Understand with practical hands-on how these agents build institutional knowledge and collaborate with humans and systemsProduction-Ready Tools: Leave with a configured environment and open-source framework (MIT license) — nothing held backPersonal Adoption Roadmap: Design a concrete plan tailored to your context with clear first steps