Humans are not naturally data driven. Our brains aren’t good at it and it takes conscious effort to overcome the cognitive biases preventing us from making data driven decisions. As someone who strives to be a data driven person, I wanted to further explore some of the characteristics that define this type of thinking.

1. They resist the urge to build narratives
As Nassim Taleb explains in his 2007 book, the Black Swan, building a narrative “increases our impression of understanding” without proving causal relationships. The world is an incredibly complex place and to make sense of it we reduce its complexity by creating plausible stories. Journalists frequently associate daily economic market changes with world events, sometimes changing their narrative within the hour.

“One day in December 2003, when Saddam Hussein was captured, Bloomberg News flashed the following headline at 13:01:
US TREASURIES RISE: HUSSEIN CAPTURE MAY NOT CURB TERRORISM.
At 13:31 they issued the next bulletin:
U.S. TREASURIES FALL: HUSSEIN CAPTURE BOOSTS ALLURE OF RISKY ASSETS” (Taleb, pg 74)

The next time you feel the need to provide a reason for why something happened, step back and think about whether the data supports a narrative. Better yet, start by making testable predictions. This forces you to resist the urge to make connections that aren’t there.

2. They don’t fall for the serial position effect
How would you feel if your business acquired 100 new customers this month? The only way to answer this is to compare it with the past. Metrics should always be viewed with historical context, but this presents a critical problem — humans are terrible at remembering things.

In 1885 Hermann Ebbinghaus published the discovery of an interesting phenomenon. He relayed lists of numbers to volunteers and asked them to recite the numbers back. Most did significantly better with the earlier values (primacy effect) and the later values (recency effect), but had trouble recalling the middle values of the sequence. This became known as the serial position effect.

Recently I recreated Ebbinghaus’s experiment on my co workers. I asked them all the to recall the same set of numbers listed below:
52, 84, 50, 32, 35, 36, 93, 72, 56, 81, 37, 19

The results match those predicted by the serial position effect, but we did see higher than expected values for positions 4-6. Can we make a testable prediction for why this happened?

The lesson here for those of us aspiring to be data driven thinkers is to know that our brains are susceptible to this haphazard memory. Whenever possible, rely on data in front of you, not your memory of that data. Data visualizations showing trends over time are invaluable in helping us mitigate recency bias.

3. They don’t fall for evaluability bias
How much should we spend for developer salaries, legal advice, or today’s lunch? Whenever you spend money on things, wait until you have a proper evaluation framework or you risk making choices based on unimportant metrics.

As mentioned in Thinking Fast and Slow, Christopher Hsee’s 1996 experiment asked volunteers to price dictionaries. Some were given brand new 10,000 word dictionaries and others were given ones in poor condition with 20,000 words. Evaluated separately, the average new book received $8 more than the old ones. Hsee’s next experiment asked volunteers to price both dictionaries together and this time the older book with twice as many words was valued $4 higher. In the absence of a meaningful comparison metric, we will rely on what is readily available.

4. They test and measure
Big data relies on gleaning insights from massive data sets. And while it’s definitely on the cutting edge of data technology, it’s frequently not necessary to have big data to make good decisions. By adopting a cycle of testing and measuring, data driven thinkers can unlock enormous value from very small data sets.

For example, if you are redesigning your website, make sure to clearly state your goals. Is it to make it look better or increase conversion rates? Don’t assume these goals are aligned, a mistake we’ve made on our site in the past. It’s only once you have a clear goal in mind that you have something that can be tested against.

5. They don’t fall into analysis paralysis
We will never have all the data we need, but data driven thinkers don’t let incomplete data compromise their ability to move fast.

Analysis paralysis “describes a situation where the opportunity cost of decision analysis exceeds the benefits that could be gained by enacting some decision.” You can avoid analysis paralysis by asking the following questions:

• Can we reverse the decision if it turns out wrong? If so, you may do better with a quick decision and measuring the consequences to see if you should change course.
• Is this worth my time? We can’t reverse the decision of what toppings to order on the pizza, but is it that consequential? There is an optimal solution out there, but the time spent getting there is probably not worth it.

Decisions that impact your life and the lives of others like getting married, hiring employees, and making large financial commitments are worth deliberating over. Data driven thinkers avoid analysis paralysis by sorting out when it’s worth taking action now, and when it’s better to pause and collect more data.

Becoming a data driven thinker
Throughout history people have accomplished amazing things using data. It’s harder, but it consistently gets better results. These five characteristics have heavily influenced my own thinking, I hope they inspire you to continue striving to make more data driven decisions.