This article was published here

Hiring is one of the most difficult and also rewarding parts of building a company. I prioritize hiring as important as building products. This article reflects what has worked for me to find and hire the best Product Engineers across hundreds of hiring interviews. It outlines topics to cover with example questions. You might find it helpful if you are a non-technical founder and looking for a technical co-founder or simply want to get better at interviewing.

Who is a Product Engineer? A person who is capable of figuring out what customers wants and implementing it. The kind of a person who reads product books like Lean Startup, Hooked, Don’t make me think, and technical books like Clean Code, Design Patterns, Pragmatic Programmer at the same time. Aim is to attract an all-round engineer, who could be a founder of their own startup or possibly has been. It’s not the people who just can code, but the people who can code the right things, that make your company successful.

Structure of the interview

As opposed to some people who like to have a generic conversations with a candidate, I structure the interviews into following sections, with a set of benchmark questions.

1. Motivation
2. Background and ambitions
3. Coding exercise
4. Coding questions
5. Product thinking
6. Teamwork and leadership skills

Why follow the same structure every time? This allows me to assess and compare candidates across a spectrum of skillsets and then make a relatively objective decision every time.

How much time does it take to on-board the candidate to become productive? What is candidate’s possible impact to the company? These last two variables directly feed into making a fair offer both ways — to the candidate and to the rest of the team. Compensation is based on impact. People in the team need to know that they or newcomers are not getting a better offer simply because they can negotiate better.

Example schedule and questions

Following is an interview structure with a set of example questions, which could be replaced with any other benchmarkable inquiry.

Motivation

Question: How much do you know about the company already?

Well prepared people go through your product and public information on the company, before the interview. Investors, background story, tech stack, previous blog posts etc. To understand how deep their knowledge goes, you can ask extra questions like “What caught your attention the most?”. You want the candidate to vet your company as much as you vet the candidate. It’s fair to assume candidate has equal responsibility on making sure that there is a good fit.

Question: Why this company?

Subpar candidates say arbitrary reasons like “I like that you use some given technology and your office is nice.” or “I would like to move to your country and work in finance.” You want people to join the company for the right reasons.

A-class people often say “I want to make a difference in the world by building a product to solve a particular problem.” They strive for making an impact and emphasise creating value.

Excellent product engineers get motivated by acquiring feedback from the market — the customers. These are the people who don’t need a Jira ticket to start working, but instead understand what has impact through customer feedback.

Corporate minded candidates might strive for positions or have expectations like“I want to be a manager”. The title is not important in order to get things done. The latter I usually fail straight away. Smart people don’t need managers — they can figure out what to do. Furthermore, smart people usually don’t want to be told what to do. It’s likely that they understand the particular problem and the users more than the rest of the company, because they have spent time and focused on it.

You might have recruiters who sometimes say “This is a headhunted candidate, they don’t have a strong motivation.” This doesn’t mean you should lower the hiring bar. When it happens, I would talk the company values through with the recruiters and make the headhunter pre-screening stronger and get the candidates whose values are aligned.

Background, goals and ambitions

Question: What is interesting to you? Where do you see yourself growing?

People, who are in a growth mindset, are in a habit of targeting specific goals and achieving them.

Mediocre candidates might only mention technical skills — which are the minimum ambitions a product engineer should have. Better candidates go further and talk about becoming better at understanding problem domain complexities. One level even further would be focusing on customer value and understanding the users. It might follow a similar pattern:

Tech skills → problem domain → understanding users → value for users → scaling products → scaling organisations

If you go deep in to one of these areas, then good candidates can talk in detail about the key learnings they have made this far and have an idea about what is still ahead of them.

Coding exercise

Exercise consists of writing actual code and observing. Does the candidates use the principles they talk about? How are they problem solving? Open ended tasks that encompass some domain thinking, system design and coding, give you a good overview on skills. For example: Model a core of a bank, start system, route finder etc. Syntax and compiling are not important, but domain representation, OOP, system design, clean code, TDD are.

Coding questions

Question: Can you talk me through your experience through layers in full stack?

This question helps to get started in getting an idea of the spectrum of technical skills, which directly translates to the ability of getting things done.

I’ll try to get an understanding on some or all of these topics: infrastructure → databases → low level algorithms → service layer frameworks → api layer → communication protocols → front-end → design.

Going deeper into each of those areas and asking a few question about them. Example: What would you use to set up microservice architecture application infrastructure and continuous integration pipeline? What are pros/cons of database procedures? What is the complexity of Java HashMap lookup, Why? How to implement a HashMap? What is dependency injection, why is it useful? Do you prefer MVC or any other type of setup? What do you think about web sockets vs HTTP? Why do we need front-end frameworks at all?

I think it’s important for a candidate to have development layer agnostic mindset. Good candidates treat coding and technologies as a tool to achieve a goal. Some might perceive technology as a thing of its own right. It also comes down to a comfort zone — there are people who have gotten used to their corner of engineering world and are scared to work with anything else. When you’re hiring product engineers or in other worlds technical entrepreneurs then go for the attitude “I do whatever it takes, to get a result”. This includes all the layers of engineering, product work, building teams and anything else that is needed to create a successful product.

Product thinking

Typically a longer product interview is scheduled separately and will be discussed in another post. This section describes basic product screening, which you can do in 15–20 minutes.

Question: If at step 0 you have an idea and in step 7 (or any other number) you have a successful product, then what are the steps that happen in between?

This questions seeks to understand if candidate has a vision on building products and which parts he might be over emphasising or have missing. These topics depend on the company culture, but I listen out for these themes:

Customer focus — learning from people, running user tests and surveys.

Being data driven — setting KPIs, what and how to measure?

Building MVP — prototyping, finding a minimum set of features, being lean.

Iterating — working with feedback, understanding traction and detractors.

Some additional example questions: What are the different ways to gather user feedback? When and how to use it? What is your favourite product and how would you improve it? How to put together a product economics/revenue model?

Question: Can you walk me through an example user journey of adopting a product. For example customer switching from phones to Skype?

This question endeavours to understand if the person has customer empathy.

Good answer might talk you through one or many possible journeys and can explain what and why the customer might feel in every step.

Unaware customer → aware → considering→ converted → advanced customer → promotor

Teamwork and leadership skills

Question: Do you prefer layered or full-stack teams?

This is an easy inquiry to separate the wheat from the (corporate minded) chaff.

Having horizontally layered teams – database, back-end, front-end, design is detrimental to common goals thinking and creates problems like hand-over, unnecessary interfacing, pointing fingers and “not my job” excuses etc. Organisations might go down this slippery slope when they think that they can not find enough talented people, who would encompass skillsets to work throughout the stack and product. This might mean that company doesn’t cultivate any mission-driven teams.

I’m strong believer in teams that are set up around a customer centric KPI. Teams that have all necessary skillsets and autonomy to independently execute their goals.

Common denominator to extract from candidates thinking — is the person taking ownership across all the levels they need to work on.

Question: How to build scalable teams and organisations?

While listening to answers to this question, I would try to spot thinking along the lines of bringing decision making, responsibility and accountability to as close to every individual as possible. When you have only a handful of people in the company making decisions then the company’s innovation bandwidth is limited to only those few people. This is is a failure to scale. Better to build an organisation that can harness the brain power of each individual to the maximum.

This is sometimes called empowering people and teams. There should be thinking of deep ownership on individual level — what is my KPI, how am I going to maximise it, and on a team level — what are the goals of the team and how best achieve them.

Removing dependencies — so that teams would be able to independently execute what is in their plan. For example product teams shouldn’t need to wait for a green light from marketing team in order to test a new customer segment.

Next, see candidates ability to take responsibility. Good candidates can start initiatives and bring people into teams to solve problems. Candidates who need managers to function and to assign people into to groups to solve problems need babysitting — not a good fit.

Question: Can you provide examples of difficult situations with people you were coaching? If former is not applicable then, What makes a good coach?

Endeavouring to understand if candidate is interested in setting people up for success. Does the candidate follow through with enough support and can give adequate feedback? Does the candidate care for people?

Candidates questions

Expecting to have good questions about the mission of the company, product, teams, culture and tech etc. Instant turnoff when the first or most important question is something like “what are the employee benefits?”.

Notes on form

Altogether the interview usually takes 1.5 hours and is better when done with at least two people to compare notes and have a discussion on the candidate. If candidate fails in some part, feel free to finish the interview early.

Active listening and feedbacking

Active listening makes your candidate understand that you are intently listening.

Summarising and reflecting back

Example: “I heard you talk about two things, that you want to learn: Big Data and Machine Learning. Is there anything else you want to add to this list?”

Helpful for checking with candidate that you understood her correctly and summarising a long chat into a few key points for your interview notes.

Giving immediate feedback

Example 1: “It seems that your learning goals are very tech oriented. I find it surprising, that you don’t seek to get better at user testing, product work etc.”

Example 2: “Do I understand correctly that your main motivations are working in finance sector and using Java? This sounds like a weak motivation — seems like you could work in practically any financial institution.”

Giving immediate feedback gives candidate the opportunity to react to what you are saying — either prove your opinion wrong or right or make amendments to how they answer altogether. Giving feedback also provides you information on how the candidate resolves opinions.

Being fully honest might also show you how candidate react to feedback and if they are the kind of person you can work together with. When faced with difficult feedback people can either get defensive or get curious. I like to work with people who get curious. If in doubt about candidates ability to have conflict resolution, you might want to bring about a particularly conflicting topic to see if you and the candidate can problem-solve together efficiently. After all, problem solving is what you hire people for.

Seems like candidates appreciate honesty and feedback. We get emails like the following every once in a while

I would like to let you know my feelings after the technical interview with Jordan and Erko. I really liked it. Not only because it went well (in my opinion), but also because it already influenced my day-to-day routine.

Why did I like it — mostly because of instant feedback. They were very straight regarding my motivation and customer orientation. I thought I had no issues as to those, but now I realise, that I should pay more attention to them. Also, I was able to explain what I really wanted to say and avoid any misunderstanding. Thanks to Jordan I keep in mind that a good colleague will appreciate honest feedback more than anything else.