LebGeeks

arithma

Member

Wally Recruitment Programming Questions

I am writing some programming questions to screen Wally.me applicants.
Let me know what you think, especially regarding the clarity of the questions.
Please don't spoil the questions by answering publicly on this board though.

When we'll discontinue the questions, I'll make sure to post the old ones and their solutions if anyone is interested.

Anyway, if you do answer the questions, you pass the screening test, and deserve an interview.

http://istest.co/wally

Last edited by arithma (November 16 2013)

arithma

Member

Re: Wally Recruitment Programming Questions

A link that doesn't require previous registration to the test:
http://istest.co/wally_nopass
Took the test offline to keep the few remaining invites for later after I update the questions for clarity.

Last edited by arithma (November 17 2013)

Joe

Member

Re: Wally Recruitment Programming Questions

Some thoughts:

• The questions are difficult to understand. I would give more samples.

• I don't understand why are the test cases hidden. Are you testing the candidate's ability to decypher your text or writing a working code?

• I think these questions aren't very relevant to a screening test.

That last point deserves some explanations.

We're going through the same thing at work, designing an automated screening test for interviews. The biggest challenge we face is that a false negative is much more costly than a false positive. Interviewing the wrong candidate will cost you 30 minutes, one hour at most. Missing out on a good candidate will cost you a lot more, because good talent is rare.

First of all, I think these are difficult questions. It took me 10 minutes to understand what's expected of me for each question, and I'm not even under the stress of a job interview. There are really good employees who will fail these tests, and you'll miss out on a chance to hire them.

Second of all, I don't think that algorithm skills is the only thing you want to screen on. As a matter of facts, I think it's the last thing you want to screen on. I would rather have a test that makes sure that the candidate knows much more relevant skills like:

• SQL and some database basics.

• Basic unix administration

• Familiarity with source control systems

Even if your screening test ends up being a multiple choice bullet point form, it's still more relevant than complex algo questions. These are fun competition between programmer, but not a good way of determining who would be a good hire.

The goal of the screening test is not to qualify the best (no automated test will ever do that), but to weed out the hopeless cases, the (surprisingly high) percentage of candidates that are a waste of time. In that aspect, I think replacing your questions with something as simple as a fizzbuzz would be enough.

samer

Admin

Re: Wally Recruitment Programming Questions

An interesting side-effect is that you might get someone who is great at solving CS puzzles, but not so good at everything else. This is illustrated in this article which I recommend you check out.

I like rahmu's approach of using something like fizzbuzz to weed out the hopeless cases. After all, I believe the best candidates are the ones who are excited to grow with your company and are eager to learn new things along the way. I think there are more important things to screen for than algorithms (unless, maybe, if you are CERN), including: communication skills, team play, hunger for learning and how self–directed the candidate is, which makes him less prone to being micro-managed, saving you valuable time.

jsaade

Member

Re: Wally Recruitment Programming Questions

Hi arithma, what will the guy you hire end up doing?

I would usually replace the interview questions with a project. Examples:

Option 1:
- Give a real problem you are facing within your project to all the candidates and give them a day or 2 to solve it.

Option2: (which I like)
2- Give a week to all the candidates to suggest and implement a new feature that will enhance your product.

With option 2, you get to know if the candidate is actually the person you need (business, thinking, programming, etc.). As mentionned in all above posts, you do not only need people who solve CS puzzles.

Instead of fizzbuzz, do phone screening (inform candidate beforehand that the phone call will have some basic programming questions). Most of the the applicants would be out within seconds with basic questions (difference between Class and Interface, or trying to explain a delegate, etc.). just keep it very simple, 2-3 questions, if they pass that then give them a project/assignment.

arithma

Member

Re: Wally Recruitment Programming Questions

Thanks for the reviews.

I guess the questions are still hard to understand even though I've iterated through them more than once. I think they're very terse, to the point of it putting the reader at discomfort (when every word is loaded, you just keep on reading the question and hoping you didn't get anything wrong.) I should give it another go and ask for another set of eyes to look soon.
I extended the time to 4 hours so that anyone would be able to solve at leisure. I don't care about coding at stress levels.

I believe the questions are not hard to solve past understanding the problems themselves. We definitely need something beyond FizzBuzz. The first one is practically an SQL question. The second one is a bit more tricky, it is testing specifically for abstraction ability - once you can extract the right abstraction, it becomes really easy. They do not even qualify as medium level questions in most coding competitions.

@joseph:
We're doing this as a wide net to get interesting candidates from the global pool of talent. When it comes to local developers, we're relying on personal recommendations, and we're doing casual interviews for people we trust are good - trust is a net.

Let me tackle the remaining points quickly:
- Those problems are expected to be easy to Computer Science
- Solving FizzBuzz from wide net of candidates will give almost 0 information.
- False positives are more costly to us, especially with the wide net we have; Even Google can't afford False positives.
- I should've put more test cases
- You can't test for SQL/unix/Source control but those are skills that are easier to screen for on CVs
- CERN is not the place I think about when I think Algorithm

We're basically a data company through all current and future services we provide and throughout all the levels of abstraction you might consider.
We're looking for people who can give radical value to our company. We can't afford to hire juniors who are in the learning phase.

arithma

Member

Re: Wally Recruitment Programming Questions

So here we go, another iteration: http://istest.co/wally_test.

I believe it's much more fun the way I worded everything this time. Reviews are what am after.

There's a very slight difference to the second problem that makes it incompatible with the previous version, but it's still pretty much the same.

Last edited by arithma (November 18 2013)