Appium Test Automation for Industrial QA at Scale

Short description

Following a QA process assessment, a global industrial automation leader tasked Tietoevry Create to execute a large-scale automation roadmap under a tight timeline. In just 18 months, we achieved over 90% automated regression test coverage across 20+ business-critical desktop and web applications – without disrupting ongoing releases.

Executive Summary

Goal: Achieve ≥90% automated regression coverage across the Main System and 20+ satellite applications within 18 months, replacing outdated Coded UI scripts and ensuring uninterrupted release cycles.

Solution: Delivered a scalable, multi-layered Appium-based automation framework using C#, NUnit, and hybrid drivers, with reusable libraries and enhanced Azure DevOps pipelines for cross-technology support.

Benefits: Reached >90% automation of regression testing, reduced post-release defects by 72%, saved ~9,000 hours of manual testing annually, and reduced release cycles period.

Services delivered: Quality engineering and test automation

About the Client

The client, part of a global leader with a century-long history of success, is a leading supplier of industrial automation solutions. The company’s solutions help businesses automate their processes, maximize production, optimize costs, and protect their personnel and environment. Based in Texas, USA, the company serves numerous industries, including chemical, oil & gas, refining, pulp & paper, power, water & wastewater treatment, metals & mining, food & beverage, pharmaceutical & others.

• Client: Leading Industrial Automation Company
• Location: USA
• Industry: Industrial Automation
• Employees: 10,000+

Business Challenge

After we conducted a successful assessment of QA processes, our client engaged us to execute a strategic automation roadmap. The aim was to achieve ≥90% automated regression coverage across their flagship Main System and 20+ satellite applications – within just 18 months and without disrupting active release cycles. The key challenges included:

• Replacing the obsolete Coded UI scripts with a modern, future-proof framework.
• Harmonizing automation across WPF, Chromium/CEF, and legacy Win32 controls.
• Providing reusable libraries to empower domain engineers (not only SDETs) to create modular, reusable tests.
• Introducing enhanced Azure DevOps pipelines for broad adoption by test automation teams and a gated check-in policy to raise the bar of quality engineering.
• Maintaining execution times acceptable for nightly CI while running on mixed on-premises & cloud VMs of variable performance.

Solution & Business Value

To meet the ambitious automation goals, we engineered a multi-layered Appium Test Automation Platform using C# and NUnit. The solution was designed for modularity and rapid onboarding of new applications within days rather than weeks.

By combining scalability, cross-technology compatibility, and development efficiency, the test automation platform delivered clear gains, including:

• Productivity Boost: Engineers can now build new automated test scenarios in just hours with pre-built business domain specific “Steps”, accelerating feature validation.
• Significant Cost Savings: By using an open-source stack and reusable libraries, our client eliminated proprietary tool licenses, cutting the QA budget by an estimated $350K annually.
• Quality Uplift: With over 90% of regressions test suites now automated, human error in regression testing dropped sharply, enabling the business stakeholders to confidently accelerate release cycles.
• Scalability & Futureproofing: Thanks to modular architecture, new desktop and web applications can be onboarded without altering the framework – two greenfield IoT dashboards have already been added with minimal effort.

Technical Details

Our QA engineers created a robust test automation platform using Appium 2.x and C# with the following capabilities:

1. Architecture-first approach for modularity and rapid onboarding, separating underlying technology concerns into the specific low-level drivers, adding control layers, and business logic abstractions. This structure enables new applications to be integrated within days, not weeks.
2. Rich Reusable Library with 4,259 reusable public methods, referenced across 965 automated end-to-end test scenarios, effectively eliminating script duplication and accelerating test creation.
3. Smart Wait & Cross-Technology Engine – hybrid drivers (Appium + WinAppDriver) with conditional waits handle WPF, CEF, SVG graphics, and even legacy non-DOM controls via OCR fallbacks.
4. Infrastructure as Code – automated pipeline templates handle device health checks, ChromeDriver-to-CEF version matching, database cleanup, and artifact version tagging via Git and DLLs.
5. Developer-Friendly by Design – the framework works with Visual Studio Community, no paid IDE licenses required. Allure reporting and layered logging make debugging and root-cause analysis a one-click task.

Throughout the implementation, the team tackled a range of technical obstacles, each critical to achieving reliable, scalable, and high-coverage automation. Here’s how we solved them:

• Environment volatility: Added pipeline stage that validates VM controllers & licenses before triggering tests, cutting false failures by 38%.
• Slow virtual machines: Test sharding and dynamic agent allocation reduced long test runs from 9 h to 4 h.
• Tightly coupled data: Self-healing test data and automated DB cleanup enabled stable execution of parallel feature branches.
• Non-interactable controls: Used OCR and low-level Win32 API hooks to reach 100% critical-path automation.

The solution delivered measurable results across coverage, efficiency, and quality. Below are the key figures demonstrating the impact:

• Automated Libraries: 4,259 public methods
• Applications Covered: 20+ (including Main System modules and 15+ satellite apps)
• Automated Tests: 965 end-to-end scenarios
• Test Coverage: 90% of regression test suites
• Average Test Duration: 3–4 minutes (fastest < 30s; longest > 30 min)
• Regression Execution Time: ~4 hours (parallelized on 6 agents)
• Manual Effort Saved: ~9,000 hours/year
• Defect Leakage Reduction: –72% within two release cycles

Technologies & Tools

• Automation Framework: Appium 2.x, WinAppDriver hybrid
• Language & Test Framework: C#, NUnit
• CI/CD & Version Control: Azure DevOps (Repos, Pipelines, Releases, Artifacts)
• Reporting & Observability: Allure, custom Azure DevOps dashboards, custom email notifications
• Utilities: Tesseract OCR, SQL/XML helpers, Smart CEF matcher, SVG & HTML style parsers