Four Rules for Effective Data Science

Search Google and you’ll find millions of articles devoted to data science as a mathematical pursuit—some data goes in, you do terribly clever maths things, and truths come out.

This appeals to those practitioners who wish to believe they’re very clever, and those people who love a box full of blinking lights and tensor cores, but it’s only half the story. If someone asks my team to look at some data, they want us to turn information into insight, contextualised and applied. They want their problem to go away.

The history of scientific inquiry is replete with examples of significant mistakes, falsehoods tested rigorously and believed by large groups of people for many decades. It’s hard to defend phrenology, for example. But when we look back at these failings, it’s equally hard to find errors based on method and procedure—getting better at the maths doesn’t mean you’re better at finding the truth. Most of the time people are bad at applying, rather than executing, these methods.

The world of research methods is a complex one: often researchers rely on validation from their community, using common practice to guide their study designs and conforming to recognised best practice whilst executing it. Some fields are lucky enough to have these formally specified due to the extreme costs of mistakes (e.g. medicine).

But you’re a data scientist, knight of the 21st century information age. You don’t have this pressure. You’re paid to get results quickly and rigor be damned. A lone wolf. Move fast. Break things. Shoot first and IPO later.

So which bits of the research world should you shrug off, and which bits do you need to retain the value of your work? How do we apply this in practice without failing to meet deadlines and spending endless time on tiny details of representativeness?

It turns out, when he wasn’t inventing rainbows, René Descartes was off solving this problem back in the 17th century. Yes, he may have been in a coffee shop, but we can be certain he didn’t pen these on a macbook.

Descarte’s philosophy was founded on a distrust of senses and all prior reason, both of which are known to err. In “Discourse on the Method”, he lays out four principles for building up a new system of truths:

The first was never to accept anything for true which I did not clearly know to be such; that is to say, carefully to avoid precipitancy and prejudice, and to comprise nothing more in my judgement than what was presented to my mind so clearly and distinctly as to exclude all ground of doubt.

There’s no shortage of people making conclusions about what is going on in your business. The function of data scientists is to prove these wrong, and offer an objective reason for doing so. Extrapolate too much and you become just another ’expert’ making unfounded claims.

This rule contains another important quality: tenacity. Key to high quality research is a driven curiosity to falsify and test all your supposed findings, to ’exclude all ground of doubt'.

The second, to divide each of the difficulties under examination into as many parts as possible, and as might be necessary for its adequate solution.

At first glance this is obvious: break the problem up. But there’s another big win here. In many contexts, we’re drafted in to make sense of a process or system that has existed for decades, often before computerisation. Ask why the current problem is structured the way it is. Is it better to re-parameterise the data you’ve got, solve a smaller problem, or expand the scope in order to break it down in a way that better suits modern computational capabilities?

Separate problems are usually simpler to solve, and the procedures to do so are less error-prone. Ask if decomposing the problem might cost less accuracy than failing to understand, reason about, or operationalise the resulting models.

The third, to conduct my thoughts in such order that, by commencing with objects the simplest and easiest to know, I might ascend by little and little, and, as it were, step by step, to the knowledge of the more complex; assigning in thought a certain order even to those objects which in their own nature do not stand in a relation of antecedence and sequence.

Start from what is evident, and step slowly enough that you don’t leave room for assumptions. This is great advice for communicating your findings, and structuring your thoughts. Your workflow should be as defensible as your methods, because your intuition is one of your methods.

There are many ways to realise this one, ranging from a strict process through to a more creative and open one. Keep a lab book. Use an outliner for your reports. Store and document your intermediate output.

And the last, in every case to make enumerations so complete, and reviews so general, that I might be assured that nothing was omitted.

I like to think this could be paraphrased as “don’t fuck up”. If you’re systematic about the others, you’ll have all the tools you need to ensure you’ve addressed the initial research questions and described the process clearly, systematically, and completely.

If you’re skeptical of these guidelines for inquiry in a mathematical world, remember that they come from the guy who invented coordinates, analytical geometry, and using a little number above another one to indicate “to the power”. Although a remarkable amount of stuff has happened since, his work remains relevant in a surprising number of ways and his guidelines for effective scientific inquiry hold true.

P.S. I want credit for mentioning Descartes without once saying ‘Cogito ergo sum’.