As product designers and product managers, we spend a lot of time thinking about developers’ needs. We look at the data and analytics on how they’re using the products we build.
But unless we get out of the building, we’ll never get the full picture of how and why developers use our products.
Our product and design teams speak with developers every single week. But with DigitalOcean’s rapid growth over the last few years, we realized we need to ensure that the whole company – not only product teams – had a consistent view of developers’ needs and their Jobs to be Done. An analytics console or SQL queries weren't enough: we needed to get in front of developers, face to face.
So, in 2018, the Product Management, Product Design, and Customer Success teams kicked off a formal research effort called the Developer Landscape project to reach this understanding. The goal was twofold:
By accomplishing these two things, we hoped we’d be in a great position in the coming years to give developers products and solutions that help them make their own products and solutions even better.
Today, we want to share one of the processes we developed as part of the Developer Landscape project. It’s the playbook for how we found users to interview for the project – and how we ran those interviews.
There’s a multibillion-dollar market research industry loaded with for-fee tools, services, and consultancies. But if you’re on a startup budget, don’t worry. There are lots of ways to recruit users for interviews – for little or no cost. Here are a few examples:
If your audience is very broad – maybe your product is a social media app or a photo sharing service – and you have a modest research budget available, then tools like UserTesting.com or Lookback.io are easy ways to find participants. Ethn.io is another useful tool for targeting actual site visitors.
If you’re targeting a hard-to-reach user base – or you’re looking to rapidly ramp up the volume of user interviews you’re doing – recruitment services like respondent.io might be a good choice. We used this service because we needed to conduct a large number of interviews in a short period of time. We found that it worked well when targeting a specialized international audience (like technical-minded software engineers interested in IoT), but not if screening by specific job titles or roles. If you’re mining your customer database, your CRM or LinkedIn might be the best way to find these folks.
What worked best for us? Personal connections and connections of our colleagues ended up making up the bulk of our user interviews. The takeaway is that individual tools and methods are only one part of a larger strategy for recruiting users. Even if you have a research budget, you should still ping friends and colleagues for help to broaden your network of participants.
No matter how you approach recruiting, you’ll need to screen participants if you haven’t prequalified them. Here are some good tips for writing screeners so you can avoid spending time interviewing people who aren’t really your users.
Once you source users, you then need to get prospects to actually agree to your request.
There’s no one way to do this, but we kept these in mind when writing emails:
If you’re not getting the participants you need, try increasing the types and/or amounts of your incentives to better entice users.
You'll have limited time with each user, so careful preparation is essential.
Before each interview, we made sure the user had everything they needed to connect with us on the big day. We issued a lookback.io link via Google Calendar with a reminder configured. Lookback gave us the ability to record the interview, and let our teammates observe and comment in real time.
At the beginning of the project, we also gave our team access to a short discussion guide we prepared. The guide was stored as a Google Doc, allowing the team to make a copy of the file for each interview and easily add notes.
Here’s a snippet from our discussion guide:
* Thanks for joining the call
* Purpose of call – to understand tools/solutions/processes you team is using so that we can better serve our customers
* Reminder recording – internal purposes only, not to be shared
* Reminder DO credit/incentive after filling out the Google form questionnaire
Background & Current Infra
* Company & Role: Tell me about your company and your role. What do you do? Who are your customers?
* Skill: How did you get to where you are today?
* How would you describe your current infrastructure? (Scope/State/Scale)
Beginning the call informally in nature can help establish a level of comfort with the user. And if there are remote colleagues sitting in to listen, be sure they remain in a listen-only state to avoid overcrowding your subject. When an interviewee is comfortable, they are far more likely to answer questions about their current challenges with their role, team, and company – openly and honestly. This is what that process looked like for us:
* We started with warmup questions to help build a rapport and put the users at ease with the process. * We avoided leading questions, and instead focused on open-ended questions to solicit deeper insights than pointed questions would. * We didn’t follow the order of questions verbatim. If the participant started deep diving in a specific area, we went along with them. In some cases, we found that certain questions might not be applicable to an individual. In that case, we skipped the question and moved on.
At every stage, we were careful to ask questions to get the answers we needed.
Again, we started with easy, open-ended questions. We asked them to tell us a little bit about their company and their role. We asked how they ended up in this job at this company. And we asked them to tell us about their users and customers.
We wanted to understand what jobs users are typically trying to do, and how they get those jobs done.
You can learn a lot about a user by honing in on a difficult situation they’ve faced. By asking them to drill into a specific challenge they’ve recently faced, you can follow up:
We learned a lot by asking users to tell us about a tool, solution, or process that worked really well for them. Knowing what excites and makes them happy is important when trying to figure out how to build a product they’ll love. To dive deeper into these happier scenarios, you can use some of the questions from the previous section as well.
Ask "Why?" as a followup question – frequently.
Asking "Why?" is a great way to demonstrate empathy. And it’s a great opportunity to dig deeper and gain insight into how your ideal user thinks about problems and solutions.
Close the interview by giving a user a floor for aspiration
Ask what their dream process or solution would look like if money and resources were no problem.
And, most importantly, ask if there’s anything else you should know.
This is a great final question because it lets the customer share exactly what is on their mind. Maybe there’s something they wished you asked them, or something else they want to share. Either way, you might be surprised by what you learn when you cap off the interview with this open-ended question.
Send the participant a thank-you followup as soon as possible after the interview, and give them instructions on accessing the incentive you promised. If there was anything you wanted to dive deeper on from the interview, this is a great time to ask.
We sent a short survey with simple followup questions to each interviewee, along with a note letting them know we’d send their incentive once they completed the survey (which we informed them about prior to the interview).
Next, share the raw notes from your interview with your team. Don’t worry if they are messy! You might even want to give a tl;dr version in your daily standup.
If you’ve run a good user interview process, you’ve completed several interviews, and have notes and recordings from each. Now you need to identify common themes throughout your interviews and synthesize the data into actionable conclusions for your team.
The synthesis method you choose varies by the type of user research you're running. For instance, if you're running a usability test, a list of observed usability issues and pain points prioritized by severity and frequency will be more helpful and actionable to you than a word cloud.
To get to the next level, present the insights (along with the customer quotes that contributed to them) in your team or company all-hands meeting. And include those insights in the onboarding materials for new hires. In many cases, it’s not hard to convert the customer’s journey into a single slide that’s easy to understand for both new and tenured team members.
We hope this helps you start your own user research project – or provides a few tools and suggestions you can apply to existing projects.
If you have questions or would like to share how you’re incorporating user interviews into your product development process, please leave a comment below. We’d love to hear about your user research needs and will try to answer any followup questions you might have.