Former Microsoft boss' favourite interview question

[cguy]

That's not what he's looking for. He's looking for one (yes or no) answer based on the median method being the best yield of information. He's ignoring that a gambler would not use the best algorithm for gleaning information but the one that yields the best (or even) odds, that's randomness. He's not considering that the method determined the odds and only that he's likely controlling the outcome based on the odds.

I also don't agree with the last bit. If I use randomness I may lose $5 but my chances for that overall would be the same as winning $5.

I'm not sure exactly what you mean there. He's really looking for a game theoretical answer: Median is as good as it gets with random input, and one should actually play once the stats are worked out (I calculate an average of 5.8 tries for a random selection, which is a slight win of 20c), however, using his adversarial strategy, one would consistently lose by selecting the median. If one can guess the adversarial strategy being used, one can turn the tables and win a lot more than the expected 20c. The overall expectation of winning depends on the likelihood of the second strategy being chosen by the adversary.

So (a) you should play the game with the median method if the input is random for the slightly positive expectation (b) you are correct, that randomizing your pivot would defeat the adversarial strategy outlined, (c) if you anticipate the adversarial strategy, you would get a much higher positive expectation, (d) given that you don't know if this particular adversarial strategy is being used, your overall expectation is the weighted probability of the two underlying expectations, where your prior should lean towards the adversarial strategy, given the context of the question.

 

[cguy]

These brain teaser questions are useless for interviews and are usually asked by people who think too much of themselves. A lot of those questions are actually answered correctly by somebody that got to hear the trick once before.

The issue you outline over here is false positives, and companies that ask these questions are quite aware that they exist. The purpose of this type of test is to filter out true negatives. The majority of interview candidates do not study these types of questions exhaustively to pass job interviews, especially since most companies have a variety of these that they make up themselves, and change up regularly. They also have follow-on questions which typically highlight whether the trick was known, or derived. (E.g., how would you design an algorithm to defeat the adversarial strategy). This is used to reduce the number of false positives.

Real interviewers usually ask questions like "what is the most difficult coding problem you solved and explain how you went about solving it". It reflects your critical thinking, your experiece, the structure you apply to solving problems etc etc and doesnt rely on smarty pants trick questions. Example - A lot of really bright people answer the Monty Hall problem incorrectly and a lot of really dumb people get it correct not because they can solve it but because they read or were told about it.

We certainly would ask something like the above as well, however, between IP concerns, the interviewer not having the right context for the problem, or the visibility into who actually did the work (I've interviewed plenty of people who were part of interesting projects, could explain it in a fair amount of detail, but weren't actually the people who did the heavy lifting).

I don't think this is any less relevant, than the "smarty pants trick question" (it's not a trick), of course. The idea is that all these questions fill in different parts of a persons profile. If they're consistent, the person gets hired.

Nobody really asks the Monty Hall question since it was popularized. If they do, you should bet that there's a hell of a follow up to it.

If these trick questions worked then quiz show contestants would have been the most successful people in the world..

Most quiz show tasks are based on memory and relatively rudimentary trainable exercises. Many of the most successful people in the world are exactly the type who would be able to answer the OP question easily.

A better question is, if these trick questions didn't work, why was it being asked by the 6th wealthiest person in the world? And why are they still asked in interviews in the most successful companies in the world?

 

[Swa]

I'm not sure exactly what you mean there. He's really looking for a game theoretical answer: Median is as good as it gets with random input, and one should actually play once the stats are worked out (I calculate an average of 5.8 tries for a random selection, which is a slight win of 20c), however, using his adversarial strategy, one would consistently lose by selecting the median. If one can guess the adversarial strategy being used, one can turn the tables and win a lot more than the expected 20c. The overall expectation of winning depends on the likelihood of the second strategy being chosen by the adversary.

So (a) you should play the game with the median method if the input is random for the slightly positive expectation (b) you are correct, that randomizing your pivot would defeat the adversarial strategy outlined, (c) if you anticipate the adversarial strategy, you would get a much higher positive expectation, (d) given that you don't know if this particular adversarial strategy is being used, your overall expectation is the weighted probability of the two underlying expectations, where your prior should lean towards the adversarial strategy, given the context of the question.

He's assuming his strategy SHOULD BE KNOWN by the player. This is incorrect because from a player perspective humans aren't that predictable. He could be making a mathematical error. He doesn't know if you're rounding up or down. These are actually factors that would support a gambling stance and a yes to the answer.

It doesn't even consider if I were to change the algorithm I would change the odds. For instance introducing randomness I may select 52 on the first try. Chances are I won't but any randomness I introduce means I may select the number along the way. It's not so much which algorithm is best, he's not asking that. He wants a "no" as the answer as he's assuming the median method is the most logical one. The flaw is that if I were to assume his strategy I would in fact not use it but change the odds so it becomes a "yes" from a gambler perspective.

 

[Swa]

These brain teaser questions are useless for interviews and are usually asked by people who think too much of themselves. A lot of those questions are actually answered correctly by somebody that got to hear the trick once before.

Real interviewers usually ask questions like "what is the most difficult coding problem you solved and explain how you went about solving it". It reflects your critical thinking, your experiece, the structure you apply to solving problems etc etc and doesnt rely on smarty pants trick questions. Example - A lot of really bright people answer the Monty Hall problem incorrectly and a lot of really dumb people get it correct not because they can solve it but because they read or were told about it.

If these trick questions worked then quiz show contestants would have been the most successful people in the world..

A big issue is that after being so long in the industry people start to think on a purely technical level and forget the constraints and parameters of the actual field. This is a prime example where the only consideration is what the odds and outcome would be if the most logical method is employed. It does not consider the usual conditions a person in the field, in this case a gambler, would employ and so change the odds to something they are comfortable with. This is why Einstein wasn't welcomed in school because he had a different way of looking at problems instead of the methods his teachers had been ingrained with.

Not only do these interview questions result in false positives because they are so predictable but they also screen out the candidates with true potential that think outside the box because their answers aren't the standard responses but employ more thought.

 

[cguy]

He's assuming his strategy SHOULD BE KNOWN by the player. This is incorrect because from a player perspective humans aren't that predictable. He could be making a mathematical error. He doesn't know if you're rounding up or down. These are actually factors that would support a gambling stance and a yes to the answer.

It doesn't even consider if I were to change the algorithm I would change the odds. For instance introducing randomness I may select 52 on the first try. Chances are I won't but any randomness I introduce means I may select the number along the way. It's not so much which algorithm is best, he's not asking that. He wants a "no" as the answer as he's assuming the median method is the most logical one. The flaw is that if I were to assume his strategy I would in fact not use it but change the odds so it becomes a "yes" from a gambler perspective.

He's assuming that the player realizes that the median approach is the most obvious, and that if he was to employ a strategy it would be one to defeat the median approach. In practice, in the interview, the candidate would typically say aloud, that the median approach will minimize the potential loss, but it is also predictable and gameable.

If you were to introduce randomness because you state that the median approach is too gameable, you would get points for that. I'm not sure why you think it isn't considered - it's expected! It's a valid counter-strategy, as is selecting the most expensive leaves of the median algorithm as pivots.

The answer doesn't seem to be "no", BTW. I know it says:

Ballmer said candidates with a strong problem-solving ability will not play the game because they are likely to lose money.

“There are far more numbers on which you would lose, than you would win. Secondly, I can pick numbers that are specifically hard for you to get,” Ballmer said.

However, amusingly, there are more number son which you would lose (37% lose, break-even 32%, and win 31%), but the losses are all $1 losses, while the wins include $5, $4, etc., and it turns out to be a net win of 20c in expectation by my calculations. So I believe that Ballmer is actually wrong.

At this point, I am willing to consider @Johand's point again. Am I mistaken, or has this guy been wrong for the last 40 years? (@G.A.S - do you agree? 1% * $5, 2% * $4, etc.)

 

[cguy]

A big issue is that after being so long in the industry people start to think on a purely technical level and forget the constraints and parameters of the actual field. This is a prime example where the only consideration is what the odds and outcome would be if the most logical method is employed. It does not consider the usual conditions a person in the field, in this case a gambler, would employ and so change the odds to something they are comfortable with. This is why Einstein wasn't welcomed in school because he had a different way of looking at problems instead of the methods his teachers had been ingrained with.

Not only do these interview questions result in false positives because they are so predictable but they also screen out the candidates with true potential that think outside the box because their answers aren't the standard responses but employ more thought.

The whole point of this question is to do exactly the opposite of what you describe in the first paragraph. Also, people who come up with novel solutions gets points for originality. It's not nearly as simplistic as you make it out to be.

 

[Arthur]

The whole point of this question is to do exactly the opposite of what you describe in the first paragraph. Also, people who come up with novel solutions gets points for originality. It's not nearly as simplistic as you make it out to be.

Exactly right. It's all about seeing how adept the candidate is at navigating decisions in a world of partial knowledge. It's about seeing how you approach problems where knowledge is partial or incomplete (ie the real world) rather than guessing the right answer. That's why this sort of question is followed by others such as "how many street lights in Joburg?" or "How many sheets of paper in a pile as high as the Empire State building?". The interviewer isn't looking for a number but at how the candidate can relate knowns and unknowns. How they think rather than what they think. Whether they know what they don't know, etc.

 

[cguy]

Exactly right. It's all about seeing how adept the candidate is at navigating decisions in a world of partial knowledge. It's about seeing how you approach problems where knowledge is partial or incomplete (ie the real world) rather than guessing the right answer. That's why this sort of question is followed by others such as "how many street lights in Joburg?" or "How many sheets of paper in a pile as high as the Empire State building?". The interviewer isn't looking for a number but at how the candidate can relate knowns and unknowns. How they think rather than what they think. Whether they know what they don't know, etc.

What always blows my mind is that people miss the point of the question entirely, and then logically conclude that it must be the hiring practices of the largest, most successful technical companies in the world that must be seriously flawed.

 

[Arthur]

What always blows my mind is that people miss the point of the question entirely, and then logically conclude that it must be the hiring practices of the largest, most successful technical companies in the world that must be seriously flawed.

You'd almost certainly fail the interview if you got the "right" answer to these sorts of questions. The last thing the interviewer wants is the right answer. Steveb would far prefer someone saying "Are you serious? I'm not gonna play that sort of game when the odds are heavily stacked against me - I'd rather put my energy into something that has a better chance of success". That sort of answer shows think-on-your-feet nous, and is fertile ground for then digging in to whether the candidate is too risk-averse and whether or not partial knowledge paralyses...

 

[rietrot]

The answer depends on context for me.

Sure if your building a trading algorithm or something to cheat with at the local casino, you'd want run the math and make sure after many attempts the balance of probably is in your favour.

The problem here is if it is presented as a game especially once off were you can lose a few dollars, why not.

My context will probably never be as a professional gambler, so I'll always agree to play the game even though I'm likely to lose. Well because games are supposed to be fun.

 

[Moto Guzzi]

Former Microsoft boss' favourite interview question

Microsoft is one of the world’s most successful companies and it has some of the best software engineers in the industry.

Just asking, if that is true, why must I continuously Reboot, Update, Upgrade, & Replace-? Over time they just seem not to get it pinned down securely so we can forget and just using the product, a good product works flawlessly out of the box without continuous tampering, and if you rarely have a problem, its fixed no questions asked.

 

[Arthur]

Just asking, if that is true, why must I continuously Reboot, Update, Upgrade, & Replace-? Over time they just seem not to get it pinned down securely so we can forget and just using the product, a good product works flawlessly out of the box without continuous tampering, and if you rarely have a problem, its fixed no questions asked.

Because one of the hardest things to do is legacy compatibility, ie run applications from the Mesozoic era of personal computing. The technical reasons are deep and arcane, and lie in old processor architectures and the old APIs needed to support them. Legacy compatibility, with all its insecurities and the convoluted kludges it requires, is the sine qua non reason why Windows continues to dominate on the desktop. There is not a Microsoft engineer or developer for the last thirty years who would not give her eye teeth to bury Real Mode and Win32. But to do so would kill the golden goose.

 

[TelkomUseless]

yea.. these brain teaser type of interviews is a waste of time.

 

[Willie Trombone]

Former Microsoft boss' favourite interview question

Microsoft is one of the world’s most successful companies and it has some of the best software engineers in the industry.

I'd ask him if Microsoft really needs the money and donate $100 to the canteen if so, then say thanks, but no thanks, shake his hand and leave early.

 

[Willie Trombone]

I know from a mathematical point of view to not play that game, but I also know myself and I am willing to take the risk.

This.
It's $99 at worst (if you're terrible at logic of course). You shouldn't lose more than $2 ultimately, but you'll be gambling $2 for the potential $5 win.
Surely this twit would want to see how you would play the game rather than if you would play it?
Besides, he should really have told you he didn't pick it at random, but to deliberately mislead you... that would give you a fair chance. If it was a random number, would he not be interested in knowing you'd chose the best method of calculating it?

...besides, I don't think I could trust him to say yes if I guessed it first time.
He is the epitome of Microsoft Excel autocorrect when it keeps changing 'CNA' when you enter it into a cell on your expenses spreadsheet. Too much overthinking.

 

[cguy]

This.
It's $99 at worst (if you're terrible at logic of course). You shouldn't lose more than $2 ultimately, but you'll be gambling $2 for the potential $5 win.
Surely this twit would want to see how you would play the game rather than if you would play it?
Besides, he should really have told you he didn't pick it at random, but to deliberately mislead you... that would give you a fair chance. If it was a random number, would he not be interested in knowing you'd chose the best method of calculating it?

...besides, I don't think I could trust him to say yes if I guessed it first time.

Why would any play a betting game that they would statistically be expected to lose at? Sure there's the fun factor (gambling), but the idea is to see if you can work out whether or not you're expected to win or lose.

He doesn't have to to you anything about how he picks the number - it's his question, and it's the candidate who should be cognizant of the fact that random vs non-random selection is a factor. In fact, the real question here is whether or not you are sharp/detail-oriented enough to make that consideration. Only at this point the how comes into it, in the form of the secondary question: a strategy for dealing with the potential adversarial selection.

 

[itareanlnotani]

Honestly the 52 answer is a very princess bride answer.

i.e. the i know that you know, that I know that you know...

 

[susefan]

It’s joy reading comments here.

 

[RandomGeek]

Like most interview questions, it's not the final answer that's the most important, rather the thought process in getting there

 

[rietrot]

Why would any play a betting game that they would statistically be expected to lose at? Sure there's the fun factor (gambling), but the idea is to see if you can work out whether or not you're expected to win or lose.

He doesn't have to to you anything about how he picks the number - it's his question, and it's the candidate who should be cognizant of the fact that random vs non-random selection is a factor. In fact, the real question here is whether or not you are sharp/detail-oriented enough to make that consideration. Only at this point the how comes into it, in the form of the secondary question: a strategy for dealing with the potential adversarial selection.

The small amount makes the loss insignificant.
And it is irrelevant how he chooses a number if it is once off.

What you say is only really applicable if you plan to run this exercise 100 times and make a profit.