Understanding the problem(s)
To validate Etsy’s request for agile coaches, our team kicked off a six week discovery to better understand how their teams run and deliver work. We conducted a series of interviews from the individual contributor (IC) level up to the c-level. I facilitated a service blueprinting session, capturing the customer journey as the IC experience, and the supporting layers of the blueprint as leadership and operational teams.
Sample service blueprint segment
I also ran a series of workshops with teams to capture their pains and their corresponding root causes and impacts.
Gathering all of this data was illuminating and helped us see that:
1. Product discovery on teams was very brief, and often was prescribed by a solution or feature idea.
2. Product delivery across teams varied so widely that defining a typical most commonly followed process was not possible.
Ultimately, a lack of clarity and consistency around our product practices caused degraded experiences for Etsy at both a company-level and customer-level.
Root cause workshop synthesis
How we solved it
Our team then worked with Etsy’s directors (VP of Eng, VP of Product, VP of Design) to construct a Lean Value Tree that helped shape Etsy’s strategy for more user-centered product development. Our strategy came in four phases: discovery, process prototyping, validation, and scaling.
With discovery complete, we started generating a backlog of hypotheses to experiment with teams. Some processes we experimented with were: user story writing, acceptance criteria, implementing kick offs and desk checks, user journey mapping, and low fidelity experiment testing. Overall, our process prototypes were measured based on team communication and number of successful experiments getting built by engineering.
Within our validation phase, we picked the most successful process experiments and ran coaching sessions with a few pilot teams. Within these teams I led design direction, and helped them specifically build a stronger experimentation process. Here we used provocations or low fidelity wireframes to test ideas with users prior to having engineering build a line of code. Prior to these experiments, Etsy employees were taking an average of 48 days before getting user feedback. I also introduced a “stoplight” exercise where the designers and product managers come together to assess the success of an experiment to decide if the idea is moved to engineering, stays in discovery, or retires. Ultimately, our product coaches implemented 30+ new capabilities across Etsy pilot teams.
Discovery playbook capabilities
Our final phase was scaling these new capabilities across the organization. Instead of rotating our product coaches across each team, we focused on training Etsy leadership on how to use each new capability. Over the course of six weeks, Etsy teams applied our Product principles on their projects and received feedback from coaches. We were met with some resistance to change, as it was hard for teams to shift from prescriptive roadmaps and features to outcomes and ambiguity, but ultimately most teams saw the value in this new way of working. Rather than focusing on prescriptive features or long timelines, teams began operating with autonomy to provide customer impact.
Sample stoplighting exercise
Sample hypotheses to test
Making an impact
Our team made a huge impact within Etsy and they still operate under the same principles today. Our initiative resulted in Etsy defining the “Etsy way”, developing playbooks for how each practice operates within their organization, and moving away from prescriptive roadmaps to true customer impact. Teams no longer jump straight into usability testing but rather test high level ideas with end users on a weekly basis, a huge shift from getting user feedback once a quarter. UX research is no longer siloed, but rather works in tandem with project teams, staying closer to the problems and connecting them to the right solutions.
Although it’s been a few years since I worked with Etsy, I remember grabbing a beer with a former client recently and I asked him how much of our work was still relevant to Etsy teams. His response was, “it’s like you all never left.” I think that’s a nice testament to the impact we made.