The Problem

Two brands, six regions, a deploy every day, and a smoke suite run by hand

Abercrombie & Fitch Co. is a global specialty retailer operating Abercrombie & Fitch, abercrombie kids, Hollister and Gilly Hicks across North America, EMEA and APAC. Code ships every day, which means the site has to be verified every single morning across six regional storefronts, eight franchise partner sites and more than 40 supported languages.

The smoke suite that gated those deploys was run by hand, overnight, and validation sat with the QE team alone. Franchise and localized sites were checked ad hoc. Analytics faults could sit undetected for a month. And on mobile, iOS regression was one week of fully manual testing, against 14,000–15,000 unit tests per App Store update and monthly releases.

What Changed

A narrow pilot, expanded into five teams

Abercrombie didn't start with the whole estate. The pilot covered two validation types, stage prop validations and analytics events, chosen because they were measurable and low-risk. Agent runtime on the same test fell from 17 minutes to 8 minutes.

Onboarding converted what already existed rather than rewriting it: pre-prod validation steps that lived in a spreadsheet, and Playwright scripts, came across as end-to-end Spur flows. The daily pipeline now makes all P0 tests compulsory before deploy, Abercrombie and Hollister, desktop and mobile, US and EMEA, with results posting to Teams and failures triaged before the release gate.

The smoke suite runs itself at 4AM, fully automated, finished before anyone opens a laptop. From there, validation expanded to five teams: QE, Martech, engineering, mobile QE and franchise. A 3-day mobile POC put the one-week regression cycle on a path to roughly two hours, and one engineer automated four of eight franchise sites at fifteen test cases each, displacing about five manual hours per franchise per week.

What it Unlocked

23 production defects that would have reached customers

In the first 90 days, the ten daily smoke suites caught 23 unique production defects. The specifics show why visual, agent-driven testing matters at this scale: a cart sync failure where remove-from-bag silently failed, traced through network logs; the Qatar storefront serving imperial units instead of metric; a Mexico PDP rendering a 446cm model height; a German translation audit finding seven pages shipping an untranslated header and nav; a Quick View modal failing intermittently from a gap in the EU smoke suite; and Tealium order_confirmation cert failures in EMEA.

Analytics faults that could once sit undetected for a month are now caught the same day. These are exactly the faults that scripted selectors pass over, the page loads, the button exists, and the content is still wrong.

"I got the login and just tried it. Without any training I had a comprehensive report in an hour, it called out every functional difference between the two pages, including translated content I'd never have caught by myself."

The smoke suite that used to run on human hours now runs on a schedule, and success is measured the way leadership wants it measured: defects caught, coverage analyzed continuously, manual hours displaced, and a domain-by-domain rollout plan. The regression marathon didn't get faster; it stopped being a human job at all.