Hire great software engineering interns

My experience as a co-op student from the University of Waterloo

I attended the University of Waterloo years ago which even then had a relatively well known co-op program in Canada. The program is great. While it takes five years to get an undergraduate degree, you alternate school terms with work terms for the entire time. There are no summers off but you graduate with two years of work experience. In my case, my first work term was at Nortel in Calgary programming robots on an assembly line which was manufacturing telephones. I had other fantastic work terms including hardware verification research at the University of Calgary, automated theorem proving at a research company in Ottawa, image processing and pattern recognition in Cork, Ireland, and then subsequently for a company in Ottawa. At graduation I had practical work experience programming in C, C++, SML and Lisp.

Co-op was a great experience for me as a student and I feel like I made great contributions to the companies I worked for. This was a great incentive for me to build a great co-op/intern program at Remind.

Why hire interns/co-ops?

Hiring students is a great way to get access to the best and brightest up and coming engineers. Once they enter the work force it becomes very difficult to hire them from their current employers. Benefits of hiring interns:

• Get productive, driven team members
• Build a great reputation at top schools for your company and engineering organization
• Fill a long-term pipeline for new grad hiring
• Working with somebody for 4 months gives you great insight into what it’s like to work with them. It also helps the student decide if your company is a good fit for them.

Process for screening students

Job Posting

We post a single job for the whole company despite the fact that we hire for all of our teams. i.e. iOS, Android, JavaScript, Backend and DevOps engineers. We tell prospective applicants what our company is about, what each of the teams does and what our tech stack is.

Programming test

Our most recent job posting at the University of Waterloo had 151 applicants. We have all of them do an online coding test with HackerRank to pre-filter. This term 125 people took the test. 13 got a perfect score. 37 people got a score of 150/225 or better.

What we look for

We don’t exclusively look for senior students. We’ll hire somebody with no work experience on their first work term. However, we are looking for raw programming talent and great side projects. Good marks often correlate with good students but not always. Bad marks in programming courses are generally a bad signal but the occasional bad course or even a bad school term are not disqualifications. We won’t hire somebody if they’ve never written any code aside from coursework even if they are top in their class. The students need to have built something substantial - either in their previous work experience or in side projects. You can usualy see this clearly in their github profiles.

Reviewing applicants

We look at all of the following factors for the applicants:

• HackerRank score: Their score doesn’t even have to be close to perfect but we do expect the candidate to take the test and the other factors are just as important.
• Resume:
  • Do they have previous work experience what did they do?
  • What were their big accomplishments?
  • What companies did they work for? If they worked for Facebook or Google and got an Excellent or Outstanding review this is a great signal since these companies have high hiring bars. Working for a consulting company like Pivotal Labs or Thoughtworks is also a great signal as they’ve learned great work habits and they’ve usually worked on multiple stacks and written production code.
  • We look for experience using more than one programming language.
  • Did they work in a team or were they solo? Working on a team usually means they are more likely to be coachable and have improved their software engineering practices.
• Transcript: I generally look for bad marks in programming courses. If they failed “Intro to Data Structures” it’s pretty hard to recover from that.
• Work term evaluations: I look for mostly excellent or outstanding reviews. A bad review can have mitigating circumstances but usually not multiple bad ones.
• Side projects: This is one of the most important items on the list. It demonstrates drive, shows you examples of code they’ve written and also gives you a good sense of what they are interested in.

Once we’ve reviewed all of the applications, we select 8 to 12 students to move to the next phase - a 1 hour live coding session.

Group Session

We do a 30 minute group session for all of the applicants where we sell the company and the experience of working here. If an applicant misses this we can do an abbreviated version of the group session in the interview but I am generally unlikely to extend an offer to a student that doesn’t attend the group session. It is usually an indication that they are either disorganized or lacking motivation. I cover these topics in my group session:

• My background as a Waterloo co-op student.
• Criteria I think are important in selecting a company to work for
• How did the company start?
• Describe the product
• Talk about our growth and success. e.g. number of users, growth chart, deliveries per month
• Provide company details including number of employees, engineers, co-ops/interns
• Funding and investors
• Product roadmap and what a student might be working on
• Overview of tech stack
• Team structure and process
• Examples of what previous students worked on
• Q&A

Live coding session

Each student spends one hour coding with the interviewer. The hiring manager does the screen. Often, I’ll bring in another engineer to sit in on the interview to get a second opinion. We do a Skype video call and use CoderPad for the coding part. The advantages of CoderPad are that it lets multiple people collaboratively work on a problem in most popular programming languages and lets them execute the code. It also lets you record the session so other people can see it after the fact.

At the beginning of the interview I spend 10 minutes asking the candidate about their previous work experience and side projects. I want to know what motivates them and what they are looking for in their next work term.

Next, I introduce the programming exercise. We’ve found that the best types of problems can be solved in any language with minimal dependencies and most importantly where the problem starts out easy and gets progressively harder. We provide lots of hints and expect the candidate to test and refactor their code as they go.

I’ll provide boilerplate for the test code and explain how it works at the beginning since many students have never written automated tests before.

Most candidates should be able to get through the first step of the problem without any major difficulties. After this we make the problem more complex and see how they deal with it. If they are stuck, we provide lots of hints on what to do next.

CoderPad records the full session but I also take detailed notes as I conduct the interview so I can review later when I’m doing my rankings. If you interview 12 people, it can be difficult to remember the specfics of each candidate even a few days later. Don’t forget that you might get a student applying again in a future term and it’s good to compare how they did this time to the previous one. We’ve hired students who previously did poorly in an interview.

Here’s some of what we look for in the live coding interview:

• Great programmer.
• Coding style isn’t imporant although we are looking for coachability. If they are stubborn or unwilling to be coached it’s a bad sign.
• Once you’ve administered a given problem a number of times you can see how far good candidates should get and build a checklist of what to expect.
• Often candidates will start to build very hacked together solutions. We are looking for them to write code that will be more maintainable over time and we coach them on how to do this. Seeing how well they respond to this guidance is a really important determining factor.

Experience during work term

We treat students the same as full-time engineers. They join a team that they are interested in and have some relevant experience with. It’s easy if they’ve done iOS or Android development. We try to keep one intern per team and keep the team composition as stable as possible through the term. We assign a senior developer to each intern as a mentor. The interns participate in all of the company activities such as workations and hackdays. There are no special projects for interns. Our goal is to have interns push code into production in their first day.

Our first interns came in Spring 2014. Since then, we’ve had 13 students. Right now we have 7 students working with us - five from University of Waterloo and two from Stanford. It’s been a great experience for us and the students.