On Thursday October 1st, Robin Muir and I invited 12 Quality Engineering and Testing leaders across the insurance market for a private roundtable on behalf of Eames Consulting at our offices in London.
The discussion explored the challenges and opportunities shaping the future of quality engineering: how quality roles are evolving, where AI delivers real business value and where it falls short, and how insurers can deliver quality at pace amid rapid technological and regulatory change.
Here are key takeaways:
How quality engineering and mindset need to evolve in insurance
QE leaders say that quality should be brought in early
Heads of testing and programme managers in the room agreed that defects caught early cost less to fix; a core teaching of ISTQB (International Software Testing Qualifications Board) training.
However, attendees agreed that quality teams still struggle to get into the conversations that matter early enough. Leaders noted that early involvement has been a talking point for more than 20 years, yet it is still not standard practice, and quality, testing and assurance are often treated as end-of-process functions.
How mindset needs to shift regarding cost vs quality
Leaders hypothesised that insurers bring in testing at the end of the lifecycle to reduce cost. Yet the room agreed that this can prove ‘penny-wise, pound-foolish’ due to expensive rework or delays at the back end of programmes.
We had noticed that there aren't many C-Suite leaders in insurance that typically come from a Test and QA background. The opportunity to realise the true value of having a strong QA function included early in the 'requirements gathering' phase could therefore be misunderstood.
We have seen that most businesses running larger technology transformation programmes have typically made business decisions around cost reductions, which has resulted in most test delivery capabilities being pushed back further into the latter phases of the programme. This can result in finding defects and bugs late, leading to delays in releases, which ultimately could be avoided.
Why clear reporting lines, ownership, and seeing testing as its own domain of expertise matter
Quality engineering (QE) leaders agreed that testing needs to be respected as its own discipline and area of expertise. As someone shared, when developers think of themselves as testing experts, everybody knows testing, but nobody owns testing.
Senior QE professionals seek clear reporting lines, with some even being reluctant to consider offers that require reporting to a development or engineering director rather than having parity as a co-equal technology function.
The impact of legacy insurance environments on quality
Test leaders say it often surprises peers entering insurance for the first time how much the industry is still constrained by legacy systems and manual process.
Common challenges senior quality engineers and test programme leaders face
Challenges specialists face now include inherited technical debt due to acquisitions, uneven maturity and shaky data foundations:
-
Growth by acquisition can bring inherited technical debt. When insurers grow through acquisition, they buy the book of business or a particular entry opportunity into a new market, not necessarily considering the technology or estate they will inherit. All too often, acquired IT amounts to a legacy system(s) that only “Bob in the corner” understands.
-
Uneven maturity. It is often the case in insurance that some departments are more mature than others, making it challenging for QA to make a material difference until business units are aligned.
-
Shaky data and process foundations. QE and QA leaders say robust continuous integration and continuous delivery (CI/CD) and better data foundations often need to come first. As the room put it, you can’t just sprinkle AI on top and hope it works.
The overarching takeaway for insurers is that there needs to be investment in both people and technology to truly move beyond legacy bottlenecks and constraints.
AI-enabled quality engineering: Where business value is being driven
AI adoption is moving quickly across insurance, but in quality engineering the picture is mixed: some applications are delivering real business value, while others have yet to move beyond pockets of experimentation.
Where AI is solving business problems meaningfully
QE leaders shared several areas where AI is proving useful:
- Early, controlled use. Some QE teams have agents creating test datasets (not from live data), collating reports and reviewing work, with background reverse-engineering and data profiling performed while guardrails are put in place
- Connectivity to code repositories. Use-cases include connecting GitHub and Jira to AI, with multi-prompt workflows for defect analysis and logging leakage rates
- Data mapping. Using AI to check data mapping is seen as freeing up time for teams to pursue higher-value work
Among insurers, discovery and experimentation still appear to be the most common use-cases. Adoption is uneven, with some teams even being told not to use AI due to auditability and compliance risks.
Where senior QE specialists have ongoing challenges concerning AI
Although efficiencies are evident, QE leaders report four main challenges:
Pressure to adopt AI without a clear problem statement. Leaders say the problem to be solved should come first. Without a clear objective, teams risk applying AI where it adds minimal real value.
Defining and proving value. Beyond confirming a system was released and works as expected, it is hard to put a figure on AI’s contribution. Some boards see AI mainly as a route to reducing headcount, while some QE leaders suggest that the better measure is how much more the same teams can achieve.
The need for speed. AI is uncovering more security vulnerabilities, so fix, test and release cycles need to accelerate too.
Token costs and output degradation. Costs can escalate quickly: one leader saw token spend rise roughly tenfold at scale. Leaders also flagged “model drift”. In one case, a prompt framework took three to four months to build, worked well for a month or two, then began to hallucinate.
Despite these challenges, leaders say they’re now able to rewrite testing frameworks that would have taken a month within a day. The challenge is that when test maturity is low, organisations risk not checking output sufficiently to meet standards.
Perspective on the buy vs build question
Discussion also covered buying versus building tooling and resource for automation. The most common approach appears to be a hybrid one. QE leaders say insurers are more likely to buy and customise to their own business needs, rather than following a 'plug & play', off-the-shelf approach.
There is a degree of caution, however, as the relevant tool may be overtaken tomorrow. There was consensus that Microsoft’s Playwright testing platform is currently the strongest tool in the market due to its breadth, elasticity, user friendliness and AI capability.
The idea was raised that ‘off-the-shelf’ is becoming an obsolete concept, as organisations customise frameworks extensively to meet their individual requirements and own the overall IP of that customised system.
Risk and regulation in an emerging technology landscape
On risk and regulation, one view was that testing should push further into AI governance.
Quality professionals already ask the probing questions governance needs, and they understand processes end to end, yet they are rarely involved in shaping responsible AI reviews.
The main risk raised was data security and intellectual property: how much sensitive data is exposed to AI tools, and where it ends up. Leaders contrasted secured internal systems with “shadow AI”, where staff use personal tools for work.
Leaders also flagged the risk for professionals of skills loss through overreliance on AI without foundational understanding. Explainability was another concern: when AI extracts information from requests for proposals (RFPs) or slips, who makes the decision, and how is it explained?
The future of quality engineering: Workforce and operating models
The future of quality engineering: Workforce and operating models
When asked where they see quality engineering evolving in the future, leaders explored several themes:
The impact of AI on junior talent
Leaders wondered what junior members of their teams would do in a world where lower-level tasks are largely automated.
Falling graduate intake at insurers and consultancies was also a concern, with some suggesting that the traditional organisational pyramid (involving a few senior leaders at the top and larger headcount at the junior level) could potentially invert, with teams of seniors managing AI agents and reduced junior headcount.
Senior leaders’ advice to newer entrants to testing includes:
- Using AI to learn. AI in quality engineering can accelerate sector fluency. In one example, a newcomer to testing used AI to support learning the basics and achieved what would have been an eighteen-month milestone within nine months
- Building understanding, not just knowledge. A viewpoint was shared that instant knowledge is not the same as understanding and that people learn through failure
- Staying close to the business. Building strong business relationships through effective communication will present opportunities to be exposed to more niche product lines, and understanding in business areas such as Reinsurance for example
- Testing and challenging AI outputs. Learning to evaluate models, including testing for bias and model drift
Insurers’ hiring priorities, according to quality leaders, should span domain knowledge, curiosity, and building balanced teams.
Balancing onshore, nearshore and offshore QE talent
What delivery models are insurers using when it comes to building QE teams? Leaders noted that sourcing decisions aren’t always in their remit, with this falling more commonly under procurement.
However, trends observed in how insurers are building quality teams include:
- Preference for smaller teams of the right people rather than a 30-strong department
- A blended model: Onshore roles for more specialist business needs and nearshore/offshore people who work closer with the business to understand core support and BAU activities
The need for collaboration on programmes of work between tech and the business is becoming ever more present, which has resulted in businesses requiring higher in-person attendance and the reintroduction of hybrid working for most.
In summary
These events provide an opportunity to get closer to the work of quality engineering and understand where challenges and opportunities lie for senior test specialists and programme managers as well as insurers.
Our sincere thanks to the leaders who joined us and contributed their perspective.
To discuss building senior quality engineering capability, speak to Scott or Robin, or get in touch with our Technology & Digital Enablement team.
