Fixing the design interview

For almost a decade now, I’ve been a part of whiteboarding exercises to evaluate the skillset of a designer, both as the interviewer as well as the candidate being interviewed. And every single time, I’ve dreaded them. At no point in my actual work a designer for the past eight or so years have I been given a random prompt and asked to brainstorm a solution for forty-five minutes on the spot. And yet, in 2021, this “whiteboard brainstorm” is still a common part of design interviews everywhere. Why? Why is it still used to evaluate designers if it’s so broken and not representative of the actual work? And is there a better alternative to evaluate the skillset?

Whenever this comes up in a company’s design interview process, the defense from the proponents of the live design exercise is the same — “Well, how else are we supposed to know what it’s like to collaborate and work with this person?” or “We need to get a sense for how they think through the process.” They say that past work and case studies don’t paint enough of a picture of their process under a time constraint or when they aren’t experts in the problem space. They repeatedly say that design exercises aren’t bad because they’re ultimately not looking for a “correct” solution, but are instead just trying to get a sense of their design thinking skills.

Well, I say it’s hot garbage. Having been on both sides of the process, it is a shitshow every single time. As the candidate being interviewed, it feels absurd. You’re in a room with some designer from the company and they give you a prompt that you know nothing about, like how would you design a car interface that needs to have a bunch of features. Almost every time, the prompt is kept vague and open-ended with no context or details. It’s up to the candidate to ask questions and get these answers. So this is where I always start. I ask things like “Why do these features need to be in the car?” and “Is there an existing model that already has these features?,” to which I get hand-wavy responses made up on the spot by the other designer roleplaying the scenario with me. Something along the lines of “Yeah, let’s just assume the features are a requirement and that there’s no previous model.” In my experience, these designers interviewing the candidate are never prepared for the types of questions I ask and ad-lib some response on the spot, completely throwing off my process and exercise.

When I as a designer don’t get the right answers I need, my process isn’t good. It’s terrible. I thrash all over the place with unclear priorities by making vague attempts to keep moving forward, hoping one of the many paths I’m carving leads to something. The people roleplaying for these design exercises with me are never taught how to work with the designer, clarify assumptions, or collaborate on a solution. They’re just told to observe the designer’s process and are given a checklist of things to keep an eye out for, along with some red flags. And a lot of the time, they screw something up and I panic sketch my way through disorganized thoughts and ideas on the whiteboard for a solution that makes absolutely no sense.

In reality, designers almost never work in a silo when starting a new project. At the beginning of any project, there’s quite a bit of collaboration involved with stakeholders from all disciplines so that there’s alignment on what we’re building. I ask data science to give me the metrics that I need, I ask engineering to give me the technical constraints and feasibility that I need to work in, and I ask product managers to tell me what the most crucial product requirements are. In a live whiteboarding exercise on a prompt with no context whatsoever, the interviewer is just making up responses to these questions on the spot when the candidate asks questions. They’re often not thinking too much about the actual data and are blissfully unaware of how their responses are biasing the designer’s process.

The proponents of the exercise will still defend it by saying that it’s okay to not have all the context and details of the prompt. It’s intentionally open-ended so that the candidate feels like they can pursue pie-in-the-sky ideas. They want to see broad, wild, crazy ideas where the designer can drill down and still focus in on the details. But in reality, the company would struggle to implement any of those ideas if the designer tried to push them along in the actual day-to-day work. What’s the point of testing someone’s visionary ideation skills if you’re struggling to make basic CSS changes in your website? Even if the designer somehow nails this exercise and is hired into the team, they’ll likely leave in less than a year due to how frustrated they’d get with the increasingly visible gap between their ideas and the company’s ability to execute on it.

As an interviewer, I have seen candidates absolutely bomb the design exercise but have still been hired and are excellent designers at the day-to-day job. And they’re still good at collaborative design whiteboarding. When you take away the time pressure and the pressure to land a job amidst a dozen or so interviews that you’re doing as a candidate (not to mention the series of back-to-back interview structure typical of an onsite), your mind is more open and willing to pursue creative ideas. You even have a much better knowledge and experience with the problem space after being an employee of the company. You understand how people work and collaborate. You know how to get the team excited about your ideas. None of this context is available to you when you’re interviewing for the position, are meeting everyone for the first time, and are asked to walk up to a whiteboard and start sketching your design process.

On the contrary, I’ve also seen candidates totally ace the design interview but then struggle to do their day-to-day design job. They seemed to have difficulty iterating on designs or breaking down the work or collaborating with PMs. They’d try to sell their visionary ideas, but the company is either in an industry that prioritizes other things over radical innovation or the organization is just stuck and the relevant VPs or Directors just aren’t interested in what the hotshot new designer has to say. As a result, these designers typically leave the company in a few short months to go pursue a role at some agency or start their own freelance venture where they can ideate and work on their process without feeling trapped and chained down by the company they work for.

There’s the typical joke in tech interviews for engineering roles where they’re asked to write a machine learning algorithm in the interview but their day-to-day job involves centering a button on the website. It’s a similar parallel to the design interview, where you’re asked to ideate on what the future vision of this product would be, but your daily work typically consists of dragging components from your design system into a rectangular frame and aligning things. It’s ridiculously unrealistic and doesn’t serve the candidate or the company in terms of setting expectations right for what they’ll be doing in the day-to-day job.

Some companies have realized the downfalls of the design whiteboarding exercise and instead have their candidates do “take-home exercises” where they have an extended period of time to think about the problem and approach it in their own way. But at this point, you’re basically asking them to work on the problem full-time for several days without paying them for the work. A few companies have been known to exploit this and straight up outsource their own product’s UX problems into a design exercise prompt, implementing the results of whatever some candidate hands in irrelevant of whether they hire them or not. This is a terrible (and often illegal) alternative to whiteboard exercise. So is there any other option?

There is. My suggestion is to keep the whiteboarding exercise, but bring in multiple collaborators who have never seen the prompt before to work with the designer. So bring in a data science person, a frontend engineer, a product manager, a content strategist, etc. Ideally, all of them would be from outside the company (to ensure that everyone, including the designer, feels like they’re on the same playing field with their teammates) or from entirely different teams within the company. Make sure that there’s a good mix of personalities and work styles amongst the team. Then, give the prompt to the entire team for the first time. Nobody on the team should have prior knowledge of the problem space. Then observe what the designer’s role is in the collaboration process as the whole team starts to work on it. This is way more representative of the actual design process than any blue-sky whiteboarding exercise.

With this process, you get to see what the designer actually does. When this happens in real life at work today, I typically don’t say much during the first meeting or two. My “process” consists mostly of letting others speak and getting their ideas, thoughts, and concerns documented. I’m more of a synthesizer than a generator in these meetings. I collect insights and try to coalesce them into some statement of goal or purpose. Certain disciplines tend to have a lot of Type-A personalities that love to dominate and take over group discussions, so I like to be cognizant of this and directly ask the less-talkative folks questions that we’d need answered in the spec.

After gathering all of this information from the first sync, I produce high-level conceptual artifacts in the form of wireframes. I see the role of a designer in these syncs as the person who is able to bring the ideas and visions of the team to life. The designer is the only person in this group who’s uniquely qualified to visualize these ideas and translate them from loose sentences and words into interfaces and storyboards. This lets the rest of the team actually see how their ideas would manifest in the real world and lets them course-correct in the user journey as necessary. The designer would present this to the team and ask questions like “Do we need more touchpoints in this part of the journey?” and “How do we ensure the users are getting the help they need if they get stuck on this step?” This, to me, is far more representative of the actual work of a designer than what they come up with in an individual whiteboarding exercise.

So yeah, I’m suggesting that we keep a small part of the 45-minute session devoted to whiteboarding, but with a team comprised of different disciplines that’s seeing the problem for the first time. Let them whiteboard and ideate for ten minutes as a group, and let the designer go off and produce some visual artifacts for another twenty minutes. Then, let the designer present their work and get feedback from the group for another ten to fifteen minutes. Don’t pressure them into doing any of these steps. Just set expectations that you’re looking for how the candidate would collaborate in a group like that and what they see their role as being. Do not judge the quality of the visual artifacts (it is only twenty minutes, after all) but instead pay attention to see if they were able to synthesize all the requirements and come up with a feasible concept to make it all make sense.

Designers don’t always need to be idea generators. In fact, you could easily make the argument that there’s too many “idea generators” in a given organization and too few executors or synthesizers of all the varying thoughts and ideas floating around in people’s heads. When you’re coming into a company as a designer, companies can sometimes set the wrong expectation and set the designer up for failure by telling them things like “We’re so excited to have someone with a fresh pair of eyes take a look at our product!” or “We can’t wait to have you revamp our whole system!,” only to be disappointed when the designer can’t really make much happen.

Again, the reality of the situation is very different. As a new designer at a company, you’re typically coming into a place where the existing employees have already been in the problem space for a while and are acutely aware of the boundaries and constraints of that problem space. The ones that have been there for a while are experts and the designer could learn a lot from them. Instead of joining the company and proposing wild ideas left and right, the best designers are the ones who talk to everyone, listen to what they’re saying, learn from everyone else’s past mistakes, lean on their expertise, and produce something that captures what everybody’s been thinking about in some visual fashion that everyone can grasp and react to. Designers solve problems in a visual context, and that’s what their interview should evaluate.

I have been rejected from many companies for “not performing up to expectations” in the design whiteboarding exercise where I otherwise thought I’d be a really good fit for the job. I have also been offered roles at companies where I aced the design interview, not due to my skillset but instead thanks to dumb luck. I stumbled upon some solution by wriggling my way through a minefield of random ideas that somehow converged into a couple of viable solutions. There was no “design process” here, just a string of strange thoughts and instinctual gut feels that luckily led somewhere. And here the company was, praising me on a pedestal for how wowed they were by my “design process”. I rolled my eyes so hard when one of the designers said that the actual job is “very different” from what I did in the interview.

I’m really tired of seeing companies have design whiteboarding exercises be a part of their interview process just because it’s “industry standard” or because they can’t figure out another way to evaluate that skillset. It’s not uncommon even for the designer running these exercises in an interview to feel uncomfortable because a lot of the time, the constant thought running through their mind is “Wow, if I was given this exercise today, I’d totally bomb it.” Well, yeah. They would. Because it’s not representative of the work they do on a day-to-day basis at the company and because it’s too far-fetched. Heck, a lot of these designers were likely hired early during the company’s startup hypergrowth stage where these exercises weren’t a part of the design interview and they just needed to hire designers fast.

Anyway, I think my proposed solution of having an entire team of cross-disciplinary folks tackle the problem space as a team is a lot better in evaluating the skillset of the designer than having them walk through their own design process for a totally out-of-left-field prompt on a whiteboard with some unhelpful collaborator giving random answers to their questions. What’s the point of seeing the designer’s incredible process if they’re going to be working in a team of people who all have their own ideas of how they should approach the problem? The real skillset evaluation comes in seeing how the designer works with this team of people. It probably wouldn’t be a bad idea to replace the 45-minute whiteboarding exercise with a 90-minute team collaboration session, split into two sessions with maybe a lunch break in the middle. Because let’s face it, a lot of the inter-office relationship building (before COVID) happened in casual interactions and off-the-cuff chats about ideas and solutions. At the end of the day it comes down to this: the more you can replicate the actual work environment of the day-to-day job of a designer in your interview process, the more successful you’ll be at assessing a fit for the role. #KillTheWhiteboardExercise