Saving the high street: Using experimentation in-store to improve customer experience

Living in a world on fast forward is no easy thing.

Dramatic transformation takes place in every aspect of our lives, every industry and every sector. Staying in the game is posing new challenges for businesses across the globe. And in-store retail is particularly vulnerable.

This blog focuses on explaining the challenges retailers are facing and why experimentation could be the path for success in the new world of customer experience. 

After the Great Recession in 2008-09, a new era started in the consumer market. A golden age of consumption driven by the dazzling evolution of technology. Anyone with a mobile phone and an internet connection got access to virtually any product or service.

As a consequence, the previously almost non-existent online channel became absolutely crucial for most brands. E-commerce sales rocketed and expectations are that this trend will continue.

However, in the offline world, things became less pink. With online taking up more and more market share, alongside fierce competition, polarising economics, high rents or rising taxes, led to more and more retailers cutting their losses and pulling down the shutters for the last time. 

Despite the headlines showing a gloomy perspective, more than 80% of the sales still come from stores. A FirstInsight report shows that consumers spend more time and money when purchasing in-store compared to online.

As the uprise of the online will continue, there is only one way the stores will be successful –  brands must embrace the paradigm shift that is taking place. Simply distributing products is no longer enough, as consumers today are not relying on stores as their sole means of access to goods. The reason consumers still go to stores is to engage with the brand, its products, and culture in an emotional way that cannot be replicated online. It’s the experience that drives them in.  

With more and more sales being attributed to mobile, social and online channels, continuing to measure success only by the conventional sales per square foot will indicate diminishing productivity for stores. This, in turn, will lead to more closures and missing out on a huge opportunity: capitalising on the in-store strategic importance to delivering powerful, unique experiences that create engagement and loyalty with a brand.

So, how do we measure this new goal of optimizing the customer experience?

Alongside the traditional ROI metrics, brands need to define their ROX (return on experience) KPIs. The first step is mapping consumers’ purchase journey and isolating the touch points and factors that drive experience. Then, invest in the parts of the company that will move the needle on those interactions and yield measurable results. It’s all about the ‘magic moments’.

In their Global Consumer Insights Survey 2019, PwC offer a list of questions to help build the baseline for ROX KPIs.

But, unfortunately there’s not much that has been done so far to define them. The well-known Net Promoter Score, which as many as two-thirds of Fortune 1000 companies use, is based on a single question concerning whether the consumer would recommend a product, service or company. Many companies hire vendors to track customer experiences with call centres and web properties.

Coming up with good KPIs to optimize for ROX will probably be one of the milestones for the new retail industry in the upcoming years.

So, why is experimentation the answer?

In the online world, user experience has increasingly become the focus of brands over recent years. We now have UX design teams trying to create products that provide meaningful and relevant experiences to users, and experimentation programs that are letting us know how a change would impact KPIs before implementing it. More and more online businesses rely on experimentation to inform their decisions and strategy and improve CX (customer experience).

However offline, things are quite different. Management usually introduces ‘innovations’ aimed at improving revenue without analyzing data and understanding what difficulties customers are facing or where the opportunity lies. Sometimes they get it right, at other times, it results in catastrophic failure.

One example of things going wrong is that of J.C.Penney. In 2011 Ron Johnson, Apple’s senior VP, became J.C.Penney’s CEO. Without any testing, he immediately introduced a change that decimated Penney’s revenue. He introduced technology, which took over the cashiers’ duties. This made the checkout experience too formal. All forms of clearance sales and discounts stopped. Also, high-priced branded products became prominent in the stores. It took only a year and five months after these changes for J.C.Penney’s sales revenue to crash to an all-time low. Management reacted, relieved Johnson of his position as CEO and went ahead to reverse the changes.

Blockbuster, the multibillion-dollar video rental giant, is another example. The company faced decreasing popularity in 2000 because of unreasonable ‘late’ fees customers had to pay if they didn’t return their rented movies in time. One of the proposed solutions for the problem  was to run a simple and cheap ($12,000 only) email-as-a-reminder experiment. But this was rejected on the basis that it’s not a ‘grand strategy’. Too simple. Instead, they came up with a transformative, big-budget effort to eliminate fees backed by a huge advertising campaign. The plan proved to be both unpopular and, ultimately, illegal. The estimated legal and business costs of this ‘extended viewing fee’ fiasco exceeded tens of millions of dollars.

And there are plenty of similar examples. The conclusion – this is not a game of chance. As the era of ‘bricks and clicks’ and omnichannel is upon us, it’s time for the online techniques to be reflected offline and experimentation put to good use.

Surely everyone should be experimenting in-store then?

Testing in-store does pose significantly more challenges than online. And the main reason for that is organizational and technical complexities.

Having multiple store branches in several locations adds to the difficulty and complexity of experimentation. While online tests give access to a large pool of consumers from which companies can do a random sampling of any number they want, this type of sampling is not possible with brick and mortar. Carrying out experiments on thousands of store locations is simply not an option.

This challenge usually causes a company to carry out tests on only a small number of its customers which is not representative of the majority.

To make the most out of the in-store experimentation programs, strategy, as well as it’s execution becomes crucial. Strategy identifies the goal of the program, defines how success should be measured and uses data to inform hypotheses. Furthermore, commitment from all stakeholders involved is essential, as well as throughout feasibility and reliability investigations when it comes to experiment design and results analysis.

Where do we stand at the moment?

In-store experimentation, also referred to as business experimentation, is scarcely used nowadays, although it’s potential for driving customer experience, informing business decisions and strategy is massive. There are, however, quite a few examples that can build up a strong case in favour.

One example is Kohl’s – one of the largest department store retail chains in America, with 1,158 locations across the country. Back in 2013, someone suggested a one-hour delay in the opening of stores from Mondays to Saturdays to reduce operational costs. This threw up a strong debate within the company’s management, and they decided the only way to know what was right was to subject the idea to a comprehensive business experiment. After conducting a test across 100 stores, the result showed that a delay in store opening time would not have any serious negative impact on sales.

Another example is Wawa, the convenience store chain in the mid-Atlantic United States. They wanted to introduce a flatbread breakfast item that had done well in spot tests. But the initiative was killed before the launch when a rigorous experiment—complete with test and control groups followed by regression analyzes—showed that the new product would likely cannibalise other more profitable items.

While ROI is everyone’s focus, ROX as a goal is relatively new, and mainly the leading brands are the ones starting to understand it and make it part of their company’s strategy. There are a few factors (here at Conversion.com, we call them levers) that proved driving in-store customers experience as well as ROI:

• • Technology is the one that stands out. In an era of fast technological advancements, there is a plethora of options for retailers to choose from to take their customer experience to the next level.

The first example that comes to mind is Amazon and their Amazon GO shops where the world’s most advanced shopping technology turn lines and checkout into history. Computer vision, sensor fusion, and deep learning automatically detect when products are taken from or returned to the shelves and keep track of them in a virtual cart. When customers are done shopping, they can just leave the store. A little later, a receipt is sent, and customers’ Amazon account charged.

Similarly, Sainsbury’s experimented with improving their customers’ experience by opening a till-free store in London back in April. This way, customers didn’t have to wait in long lines, and could self-checkout simply by scanning the products with their phone, after installing an APP. You can read more about the experiment here. The results of this experiment are still yet to be announced.

• • Employees are a powerful factor to drive ROX. However, research suggests that retailers tend to view store associates as an expense to be controlled rather than as a medium to provide better service for customers.

A randomized controlled experiment  was run in 28 Gap stores in San Francisco Bay Area and Chicago in 2015 by an interdisciplinary team led by Principal Investigator Joan C. Williams from the University of Chicago. For the experiment, retail associates were shifted to more-stable schedules to see how that would impact the sales and work productivity.

The results were striking.

Sales in stores with more stable scheduling increased by 7%, an impressive number in an industry in which companies work hard to achieve increases of 1–2%. Labour productivity increased by 5% in an industry where productivity grew by only 2.5% per year between 1987 and 2014. The estimate is that Gap earned $2.9 million as a result of more-stable scheduling during the 35 weeks the experiment was in the field. All details about the experiment can be found here.

Nevertheless, the best to exemplify the impact employees can have on ROX are the iconic Apple stores. They rely on a very effective communication technique adapted from The Ritz-Carlton – Steps of Service. Every employee is trained to walk a customer through five steps that spell out the acronym A-P-P-L-E:

A – Approach customers with a personalized, warm welcome.

P – Probe politely to understand the customer’s needs.

P – Present a solution for the customer to take home today.

L – Listen for and resolve issues or concerns.

E- End with a fond farewell and an invitation to return.

• • In-store design, fixtures, and facilities also play a significant role in customer experience. Proving that they understand how brick and mortar retail is changing in the age of e-commerce, Nike opened a new five-story, 55,000 square foot store in New York City. There is a mini indoor basketball court, a treadmill, a system that simulates runs in different locations, a small soccer enclosure, a shoe bar where shoppers can personalize a pair of Nike Air Force and coaches who put customers through drills to test out different pairs of shoes. It is as much a place to play as it is a place to shop.

To conclude, in a market with customers’ expectations higher and more dynamic than ever, businesses have a powerful instrument in their toolkit to help them understand and meet these expectations – experimentation.

Putting experimentation at the heart of a business, not only leads to better and more innovative ways of doing things – but actually gives companies the confidence to overturn wrongheaded conventional wisdom, and the faulty business intuition that even seasoned executives still inhabit.

To find out more about our approach to experimentation, get in touch today!