What do job-application aptitude tests really measure?

It is common practice for companies (including financial companies) to use aptitude tests (mostly numerical and verbal reasoning) in order to assess candidates. I myself have done such tests and though I do not begrudge companies that use them, I have started to question their validity. Do they actually measure something useful? I explore this theme in a blog post on my life-related blog Meditations of Lambchop. Here is an extract from the post

This excellent essay, A Mathematician’s Lament, by Paul Lockhart has convinced me of something I suspected since learning actual mathematics at university (and not even, at first, within official lectures): our schooling ruins mathematics for children. The same kind of thinking that has created the education system appears to have infected human resource departments in most major companies. I refer, of course, to the ubiquitous use of aptitude tests (a subset of psychometric assessments).  It seems to me that our schools are satisfied with teaching arithmetic rather than mathematics and HR is satisfied with testing “skills” that no candidate will ever need. As with schooling it seems hardly anyone questions the current system. Read more here.

Don't spot the pattern

If I write 2 then 4 then 6, then we feel good because we know that next comes 8. We can foresee it. We are not in the hands of destiny. Unfortunately, however, this has nothing to do with truth. – Arthur Seldom, The Oxford Murders (movie)

The series 2, 4, 8, could obviously be followed by 16, but also by 10 or 7004.
It's always possible to find a rule, a justification which allows a series to be continued by any number. It all depends on how complicated the rule is. – Arthur Seldom, The Oxford Murders (movie)

I remember getting questions at school of the form “which number comes next?” At the time I thought these questions were perfectly normal. I now think they are nonsensical. As such it troubles me to see similar questions (with diagrams rather than numbers) are being used in psychometrics assessments. For instance here are ones called diagrammatic reasoning tests. Whether numbers or diagrams, the idea is the same and it should be put to an end.

These questions expect you to extrapolate from a finite set of data. The problem is, as with the above quotes, there are infinitely many ways to do this. The only difference between them is that some “feel” more right than others. They are intuitive, they are “simple”. But both of these things are in fact rather subjective. And so while these questions pretend to have only one right answer, they really do not.

Here is an example. The series 1 2 3 5 …

This could be “all integers with at most one factor”, i.e. all the primes and the number 1 – then the next number is 7. It could also be the Fibonacci sequence, but starting at 1 2 instead of 1 1 – then the next number is 8.  Of course one could think up infinitely many rules for completing this sequence. Another simple rule is to assume it is periodic 1 2 3 5 1 2 3 5…. - then the next number is 1. Of course if you looked only at the first three elements in the series you would probably guess the next number is 4.

The question, then is not really “what is the next number?” it is: Find a function from the natural numbers to the natural numbers which has the given sequence as its first mappings. The function should be “simple”, meaning it should be described (possibly as a recurrence relation) only with addition, subtract, exponentiation, etc. and should be the one function that whoever is marking the question would think is the simplest.

The problem is that the problem is never actually stated like this. The ways in which you are allowed to describe your function are not enumerated and there is no objective means of determining what is
“simple”. Thus for any above mediocre mind, the problem is not to find the next number, but to determine how far beyond the standard set of descriptions for functions they should allow their mind to search.

Thus the question really only does the following: it forces you to confine your search to what is expected already. It hinders the ability to think beyond this and it penalises anyone who happens to think differently from the standard. It creates a false impression of truth and limits human creativity. The only way to ask these questions (if you have to ask them at all) is to give a precise description of the form of function allowed and then make sure only one function in this set satisfies the requirements. The same is true for diagrammatic questions.