Position – Experience Designer
Contributions – User Experience, Design direction, Client Solutions, asset creation, vendor and partner management, design research, Lidar and Mapping
THE VISION
Integrate Machine Learning, Computer Vision and AI across every endpoint, both in Marcs and Spencer stores and behind the scenes.
Vision Video
UNDERSTANDING M&S NEEDS
Our team of business strategist, user researchers, and designers visited the M&S headquarters in London for a week to fully understand the customer needs. During that time we were able to identify major needs with some key examples below:
As a front end employee I need to easily understand where a safety concern is on the store floor.
As a back end employee I need to know where I can identify and store recent deliveries
As a consumer I need to quickly understand which products are for sale and most relevant to me
DEFINING THE GOALS
Partnering with Marcs and Spencer, the Ambient Intelligence team developed design solutions with AI technologies to help improve customer experiences and store logistics based off their needs. The goal was to bring M&S to a “Digital First” retailer. The ambient technologies range from gesture based analytics to maximizing logistical efficiencies from product delivery to customer insights. Ultimately the goals we settled on were some of the following:
Identify stock shortages to quickly backfill for further sales of a fast moving product
Increase the safety on the store floors by quickly identifying safety hazards for the most efficient and appropriate reactions
Identify and supply recommendations for the best ways to organize existing and new store products at the receiving dock
Create a buying experience that saves customers time and money by tailoring user specific metrics in identifying the most relevant products for consumers to purchase.
CONSTRAINTS
Some of the major constraints we faced in developing the product were Privacy Concerns, High Resolution Data, and Variability Across Stores.
It was critical that we addressed privacy to ensure our customers and workers understood how their data was being used and shared across the stores. Transparency was key in developing trust across our user base. Not only was it important to share how their data was being used, but it was important to allow users to have authority over how their data was used.
Getting high resolution data for machine learning also proved to be a difficult challenge as we would need to create a pipeline that scanned all M&S product across their grocery stores that the ML could use for further instore analysis and recognition.
The other major constraint of Stores being very different from place to place created a challenger for how best to analyze the data across these different environments and how the metrics would then be applied across the space.
THE TECH
In order to make a successful product we needed to nail a few core areas of the tech:
Scales - the shelves across the environment needed an accurate weight metrics to know how much product was to be on the shelves
Object Recognition - High resolution scans of products were necessary to understand when product was being moved
Spatial Understanding - Cameras needed to know when products where empty on shelves or identifying safety concerns
Digital Twins - A digital twin was necessary to have a properly mapped space to compare to real time happenings
Facial Recognition - Was an option to help with identifying and lead consumers to areas of interest based off their shopping past. This could have also been done via QR code
RFID - RFID chips help with the movement of product through logistics
PIVOTING
Through the process we found that the variability in store environments became a critical challenge in how to further approach the problems we came out to solve for. The other challenges we faced with cameras and privacy also proved to be a major challenge for how it was used in consumer facing environments. What this lead to was a system that focused more on manual inputs for communication across the stores for the frontline workers and maintaining our original technological approaches to the back end logistics of the stores such as shipping and receiving.
OUTCOME
Today you can get a further glimpse of how our original goals for the front end of the experience after our pivot were achieved in this video:
The backend for logistics continues to be developed today. Efficiencies across the logistics has improved greatly by reducing mis-placed items, identifying the approaches to storage, helping workers in receiving to quickly identify areas of interest, and finally, helping improve customer satisfaction and M&S’s bottom line.