Data Biases

Learn to recognise the statistical tricks.

Survivorship bias

Do you base the average achievement level of your school contemporaries on the attendees at the annual reunion? Wrong! The failures don't show up.

And do you judge the success of a group of funds by analysing those currently on offer? Think again.

Data mining

You can test a hypothesis by checking it against a range of past data, and using statistical tests to see if the data supports the hypothesis. But suppose you just picked the data that happened to fit the hypothesis? That would be a) cheating and b) called "data mining".

And do you judge the skill of a fund manager by an analysis of his past performance over a time period picked by him? Or the reliability of a fund family based on one fund chosen by the managers? Think again.

Picking Peaks

A randomly lucky result is presented as though it is representative of the system/scheme/fund as a whole.

We are used to this in advertisements - for tipsheets e.g. But it's very common in journalism (because extremes make a better story than the boring average)

Ordinary extraordinary events

To decide if something is extraordinary, you need to know the size of the sample from which it is drawn.The Law of Large Numbers says that very rare things occur if you try often enough. Or, you can find very special things if you pick from a large enough sample.

If you meet a fund manager at a party and it transpires that he has beaten the average fund every year for the last five, that's quite unusual. In fact the odds are 1 in 32 against. It would be polite to congratulate him. But out of 800 funds in the UK it's likely that 25 would have achieved this using the dart-throwing stock selection technique. Should you congratulate them also?

Paying for randomness

Consider the following proposition. You pay us £100 and we guarantee you can chose the sex of your next child. In fact we will refund your money if we get it wrong.

Get it? We send out random answers. Half of them will be right and we'll keep the money. Half of them will be wrong but we won't lose.

You should now be able to recognise lots of tipsheet ploys.

The hidden rare event

If you played roulette without any knowledge of the numbers on the wheel, you might by observation come to the conclusion that putting equal amounts of money on red and black for each spin was a pretty safe strategy. Your money always seemed to come back.

That's when the zero comes up - wiping you out. It's only once every 37 spins (for a single-zero wheel) but it's good enough. The zero is the "hidden rare event".

You could devise a tailored financial product offering 6% annual income with the investor's whole capital at risk once a year depending on the spin of a roulette wheel. That's 4% risk free return, just under 3% (1 in 37) for the capital risk, less 1% for you. That should work. In fact, that's how "precipice bonds" work.