Technical Interviews Still Suck

This post actually began as a "technical interviews do's and don'ts" (does that need another aposthrophe?), and the bulk of it was written in 2009. Revisiting it, I marveled at how technical interviews still suck… and that became the title of the post. Instead of just refreshing the original article, I've added in additional, recent context to further support the premise.

I've been on both the giving and the receiving end of interviews in my career as a software engineer and engineering manager, and I've been privy to a great many styles of interviews. There are interview questions that completely turn me off as a potential candidate and there are interview styles that make me want to jump through a window. In any event, interviewing programmers can be a tricky process, and I wanted to lay out some of the issues, techniques, and other ideas that companies and human resource departments can utilize to end up with better hires.

As a rule of thumb, here are four primary ideas to keep in mind:

1.) Throw education out the window. Not exactly, but keep a few things in mind. When's the last time you actually called a school to verify that a person attended or graduated? For foreign applicants, have you ever called over to that Indian or Russian university to verify their master’s degree in engineering? No. Why put stock in what's listed on a resume for education in the first place then? Education is usually secondary to experience in most positions, unless you’re interviewing for an entry level position. Education is important for critical thinking skills, problem-solving, and fundamentals like English and math skills, but the difference between somebody with a computer science degree and one with a (ahem) psychology degree might not be that big. It’s the application of skills that is important.

2.) Be careful with what your headhunters list and say. It's bad enough when a job requirement lists "n" number of years of experience in a certain technology that has been out for less than "n" number of years, but then to have a headhunter begin asking technology questions when they don't understand the answers is demoralizing to potential candidates. The headhunter ends up listening for what sounds most impressive, rather than what really qualifies the person. It becomes buzzword bingo.

3.) Don't quiz the candidate. Asking a question like "what are generics?" is basically like saying "I don't trust you, so I'm going to throw random terms out and see if you can answer them to my satisfaction." Technical knowledge means nothing compared to problem solving skills. The best programmers excel when they've reached a problem where they don't know the answer. Nobody knows everything, but the better employees are the ones that, despite not knowing, can easily learn and adapt to new technologies and obstacles. Instead of "what are generics?" ask a question like "Tell me about a time when you've used generics." with a follow-up of "How about a time when you've had to abstract or make a generic method so that it accepted multiple object types?" if the person can't answer the first question about generics.

Instead of quizzing the candidate, ask about their experience. Ask what they've done. Based on their answers, dig deeper into the areas that are important to your company and ask for more specific details of the operation in that area. This shows the candidate interest rather than trust issues, and it gives you a more accurate idea of what the programmer has worked on and how they worked in that environment.

What’s worse than quizzing someone in an interview? Giving them a test. I understand that some places need some form of screening, but job tests are a pet peeve of mine. I had to learn (and get trained in) psychological testing at both the undergraduate and graduate level when I was in college. Our text was the Standards for Educational and Psychological Testing, published by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. This book specifically outlines ethical considerations in psychological and educational testing. There are many ethical considerations that need to be made. For example, are you testing an Indian programmer? Testing standards are explicit that tests need to be given in the language with which the test taker is most comfortable. Ever have somebody take an online personality test? Ethical standards dictate that each test taker needs to be afforded the same testing environment. How can you be certain that each prospective employee has the same bandwidth or the same quiet time at home to take the test? Not to mention the fact that most personality tests (such as the popular Myers-Briggs) are explicit that test results not be used to make determinations on hiring.

More and more corporations and mid-sized companies employ industrial/organizational psychologists for human resources. These individuals are required to adhere to psychological standards, including testing standards. Personality tests would fall under psychological testing standards, while programming tests, would fall under educational testing standards, in my opinion.

Lastly, psychological and educational standards make it very clear that in order to ethically give a test, one must be trained to give the test. Clearly that is not the case with many hiring tests, especially ones that are online.

This is only my opinion, but coming from a psychology background, I feel very strongly about it since it is a part of my training. Furthermore, hiring tests such as these bring to mind the trust issues mentioned above, as well as, again, only really telling you about a prospective employee’s immediate knowledge and not necessarily his or her abilities.

4.) Take hunger over book smarts. At a previous company I worked for, we had interviewed a programmer with an excellent resume and excellent technical knowledge. He was hired, given a larger salary than many already working at the company, and was entrusted with an important project. Despite his education, credentials and technical knowledge, he was a complete bust. He didn't work well in a fast paced environment, nor did he work well with a team. He missed deadlines, spent more time on instant messenger than on work, and was ultimately let go. His replacement? A graphic designer hungry to learn programming, who was enthusiastic to join the team and worked diligently to prove himself. That art school drop-out eventually become my Codepunk co-host.

Now this doesn't mean that Bill was better than the seasoned programmer at the time. This shows that the importance in hiring had less to do with the person's experience and more to do with their character. A person with a good character and a solid skill set will trump a superstar engineer with a poor work ethic any day of the week.

This is why the style of the interview is so important. It is impossible to tell a person's character by their resume or by quizzing them with definitions. You need to get them in story mode by asking them about their work environment and experience. Your questions should nudge them in the direction you want them to go, but at the same time, let them tell their own story. This way, you're guaranteed to get a better understanding of their goals and candidacy, and your new knowledge of their character will help you decide what kind of employee they will be.

I'll leave you with a story from my last time interviewing for a position.

This job was for a "data science" position and I put that in quotes because the job was actually for a scala programming position dealing with geolocation data and application engineering. It certainly wasn't what one traditionally thinks of as data science. In fact, the job that I had applied for was more of a front-end position, but after interviewing with the hiring committee, they referred me to the hiring committee of the engineering team, and I was asked to come in for an in-person interview. The only problem was… they never actually told me that the in-person interview was for a different job than the one I applied for.

The interview itself was cordial and I liked the people on the hiring committee. At the end of the hour long interview, they then decided to give me a technical exercise to accomplish—asking me to write it in pseudo-code on a piece of paper. They didn't inform me of the technical exercise, and after driving an hour for the interview and spending an hour there, I had at least an hour ahead of me before returning home to my wife and newborn twins. I began working on the technical exercise and it was difficult to work through on paper. A few hours into the exercise, they informed me that I could stop if I wanted to, since not many people could finish the exercise (which would have been valuable information at the beginning). At the same time, my wife was blowing up my phone because she was worried that I had gotten into a car accident. This deep into the exercise, I refused to quit and finished it on paper. They then looked at the pseudo-code and said, "Great! Can you now write it in real code?"

You could have asked me that to start…

After making a few changes to the pseudo-code, I handed them the paper. "Great! Can you type it up for us when you get home?"

Beyond frustrated, and roughly 5 hours from leaving my home, I returned to my house, opened up my laptop, and proceeded to build an application that did exactly what they asked instead of simply typing up written code, I built a quick Express application, deployed it to the cloud, and then sent them the link.

After not receiving any response, 2 weeks later I was emailed for a reference—only one reference. I gave them the reference and within an hour they sent me a job offer. Unfortunately, by that time, there were two problems:

1.) I already had a new job.

2.) The style, format, and communication that occurred during the interview was something that completely turned me off.

Yes, technical interviews still suck.