Skip
Current Issue
This Month's Print Issue

Follow Fast Company

We’ll come to you.

What Etsy Does When Things Go Wrong: A 7-Step Guide

An excerpt from Etsy's upcoming debriefing guide outlines how you, too, can adopt the company's blameless approach to understanding how things go wrong.

"The purpose of an investigation is to understand how things usually go right as a basis for explaining how things occasionally go wrong."
—Erik Hollnagel, Safety-I and Safety-II

Most traditional accident investigations tend to focus on discovering things around an event that never actually happened. In an attempt to prevent future accidents, there is an underlying assumption for this somewhat peculiar emphasis, which is:

Someone did not do something they should have, according to someone else.

Through this lens, what generally surfaces in investigations are "findings" about what people did not do (pay attention, make the right decision, etc.) rather than what they actually did. Without anyone really noticing, these items get labeled as "human error" and through a seductive and convenient contortion of logic, an event that never actually happened is deemed after the fact as the "cause" of the accident. Perhaps unsurprisingly, this results in an obvious recommendation for the future:

"Next time, do what you should."

Unfortunately, this approach does not result in the safer and improved future we want.

The perspective now known as the "New View" on accidents and mistakes flips this thinking around, providing a different path to improvement and learning. We wholeheartedly believe in this approach at Etsy. We’ve invested in operationalizing it on an organizational level and have shared our perspective publicly.

Postmortem debriefings hinge on the expertise of the debriefing facilitator. Over the years, we have developed an internal course at Etsy focused on building a strong bench of debriefing facilitators. At the time of writing, this course is composed of:

  • Three seminars covering the theoretical and empirical foundations of accident models, historical case studies, topics of "Just Culture" and Safety-I/II (with readings and written exercises)
  • An interactive workshop for facilitators to practice core skills
  • Shadowing and feedback for new facilitators
  • Regular follow-up discussion groups

This guide is intended to be a lightweight set of techniques and tactics distilled from the full course, designed to help facilitators develop and sharpen their debriefing facilitation skills.

Etsy CTO John Allspaw and Josh Wise, Etsy's director of workplace ecology and design

We believe that organizational learning from accidents comes from gathering data about the accident, constructing a timeline, and bringing people involved into a group discussion led by a skilled and neutral facilitator. While it’s tempting to simply send a form for people to fill out over email, we’ve found that approach to be void of the context and dialogue necessary for real improvement. It’s important to gather in a room together and make the learning dynamic, rather than static.

There are a few key elements of structuring and preparing for your debriefing.

1. Familiarize Yourself With the Timeline in Advance

Before the debriefing, facilitators need to get a sense of the flow of events, without forming opinions about what happened. You’ll want to gather whatever objective data is available and construct a timeline of events.

The initial timeline should be created by the people closest to the event. Generally, they are responsible for coming up with the basic structure of a timeline, which helps determine the boundaries of the event being discussed. You’ll want to review this document to get an initial idea of where the interesting communication patterns, decision points, observations, and actions occurred so you can sketch out where to dive deeper in your debrief.

Prepping is mostly about setting yourself up to best manage your focus and the group’s mental cycles during the debriefing itself.

2. Unearth More Objective and Subjective Data

The timeline provides widely accepted observations to use as a skeleton for the debriefing. Additional data that can help construct the timeline might include chat transcripts capturing the dialogue between people attempting to coordinate and respond to an event, dashboards, graphs or logs that people observed, the time-stamped actions people took during the event, and so on.

As the debriefing progresses, you’ll use the timeline and the artifacts within it to develop questions. We’ll talk more about these questions in the next section, but they represent prompts to gather subjective data that will build context around the actions and decisions that took place. Subjective data includes the opinions, judgments, assumptions, beliefs, and individual statements that will come to light during the debriefing.

The key idea is that when somebody says "X happened" you take the opportunity to ask questions that provide context for X. Not everyone in the discussion will come to the meeting knowing that context, so it’s important not to skip over details as you walk through the timeline in the debriefing.

You’ll want to make sure that everyone involved in this incident, especially representatives with diverse perspectives, are invited to the debriefing and that you talk to key stakeholders in advance.

3. Talking to the People Who Would Usually Be Blamed Is Really Important

It’s crucial to establish a sense of trust with all participants up front. Find out if they’ve been to a debriefing before. Try to gauge how much fear or anxiety they’re feeling. You’ll want to tread with more sensitivity here. Some of this might feel like therapy, and that’s okay. You want to set everyone up for success in the debriefing, which means helping them feel at ease and reminding them that feeling anxious or embarrassed is what makes them the best teachers for their coworkers, who will almost certainly find themselves in similar situations in the future.

Remind them that hindsight and all our after-the-fact knowledge of the outcome is what makes their decisions or actions feel embarrassing. Remind them that before the unexpected occurred, they were just doing normal work on a normal day.

4. Come up With Some Initial Questions

Think a little bit about the sorts of things you might want to talk about (based on your reading of the timeline and your discussions with those involved). Focus on open questions—ones that don’t have "yes" or "no" answers. Make notes in the margin about what seem like important pivot points in the timeline, and jot down questions that might draw out additional context from people involved. Doing this in advance can help you stay on track and attentive during the debriefing itself.

5. Documentation

One of the most important outcomes from any debriefing is the annotated timeline of how the events during the incident happened, as agreed on by everyone in the room. To this end, it is important to have a dedicated notetaker. You as the facilitator will be plenty busy guiding the discussion, digging deep on interesting events, and coming up with the questions that get the room to understand what was really going on.

Etsy has created a tool called Morgue to aid us with the documentation process. It’s a simple web application that allows for adding a lot of metadata about an incident, as well as a timeline, graphs, images, and chat logs related to what was going on. The format that will work for you is largely dependent on what fits your organization best. Wiki pages or Google docs can work just fine. Etsy’s debriefing documentation started as pages on our internal wiki, but over time we built Morgue to better accommodate our unique needs. We even built a "request a facilitator" button, and have added Google Calendar integration so that scheduling the debriefing itself is part of the tool’s workflow.

However you document the debriefing and related artifacts, it must be available and accessible to everyone in the company. You need to make it very clear before, during, and after the facilitation that a debrief is a collaborative process that benefits from everyone’s input.

6. Setting Expectations

In order to make sure everybody in the room understands the importance and goals of a debrief, the introduction you give at the beginning of the meeting is key.

You have the job of simultaneously setting the agenda for the next hour or so, while also getting people into the mind-set for learning, which means making the atmosphere feel as safe and as welcoming as possible. One of the hardest parts is making everybody understand that we are not coming together to prevent a future event from happening. The things you are about to look into and find may very well happen again. Preventing the future is not the goal—learning is. And to that end, learning needs to be the focus of the whole debrief.

As mentioned before, there is a powerful tendency to try to find a single simple solution that explains what happened. So as the facilitator, you need to counter this most human urge to find a single explanation and single fix for the incident. You can do this by giving a brief introduction in the debrief to remind participants what you are all there for: a semi-structured group interview. Nothing more.

7. Structure of the Discussion

One of the best ways to ensure you are following a path of exploration (instead of looking for a single fix) is by sticking closely to the timeline. This makes sure you get a representation of the past that everyone in the room agrees is as close as possible to what really happened. Any remediation items based on a less accurate version of how things went down will be less effective or even obstructive.

Because reconstructing the timeline is your main task, you should focus the majority of time in the debrief on doing just that, addressing remediation items afterward. For a 60-minute meeting, this can easily be 35–40 minutes of discussion.

There will be many people who suggest fixes before the group has finished walking through the timeline. This is inevitable, but these contributions are less effective during this part of the debriefing. Your job as the facilitator is to encourage people to write down any ideas for improvement (sometimes known as "remediations") that come up during the timeline reconstruction, so they can bring them up later, after the timeline is complete. More often than not, the unfolding of events in the timeline render those initial remediation items moot when the full (or at least fuller) context becomes clear. So let the room know there will be a time for remediation items and to hold on to them until then.

After you have established the timeline, you want to make sure everybody agrees that it is accurate and comprehensive. This means asking the room what’s missing, if there’s anything you haven’t talked about, or if anything happened differently than described so far.

Your next step is to pivot the conversation toward what the group learned from going through the timeline, and to discover more perspectives that can clarify or add additional context to what’s been written down. This is also when you’d invite everyone in the room to start thinking about the learning points from the event. These can include recommendations for remediation items, but shouldn’t be limited to them.

When you begin discussing learning points, encourage everyone to speak up if anything comes to mind and to not censor themselves. You want the initial phase to be more of a brainstorm. Nothing more, nothing less. Anything goes, because we are looking for improvements to the current system, and trying to unearth innovative ways to underpin the learnings from the meeting. However, you must resist the urge to turn them into actionable tickets just yet. These learning points are likely to come from all directions and appear haphazard. Some might be very specific, others very broad, and some might apply to different systems altogether.

Ultimately, you want to create remediations that are specific, measurable, achievable, relevant, and time-bound, but don’t dismiss ideas that don’t conform to that criteria right off the bat. Instead collect all the ideas as a brainstorming list. Then choose a small set of people from the meeting who were close to the incident and are familiar with the systems mentioned, and ask them to take the brainstorm list and dwell on it for a bit.

The idea behind dwelling on the recommendations for some time and developing remediation items after the fact is to just be aware of them as suggestions and keep them in the back of your mind to let them marinate. Then two or so (work) days later, bring that group into a room to discuss the viability of those remediation items. This way, what results are tasks that are effective and logical and stand the test of time, rather than action items created in a rush during the final moments of the debrief.

You might also want to remind everyone involved that having remediation items is not a requirement for a good debriefing. If what a group learns during a debriefing results in new ideas on how to make mistakes and accidents less likely, that’s great. However, our experience at Etsy reveals that it can be easy to generate ideas for remediations in an effort to satisfy the urge to "do something" in the wake of a painful incident. But what’s often downplayed is how the recommended fix might needlessly complicate, or even increase, the likelihood of new types of accidents in the future.

This is an adapted excerpt from Etsy's Debriefing Facilitation Guide (John Allspaw, Morgan Evans, Daniel Schauenberg), which Etsy will be publishing in full. Allspaw, Etsy's CTO, gave a presentation on blameless postmortems during the Fast Company Innovation Festival, as pictured above.

loading