How to ensure Data Driven Decision Making does not become Disastrous Decision making?
Data-driven decision making (DDDM) is the new buzzword that is increasingly used inside companies. Apart from the buzz, companies do take in efforts to establish a data driven understanding culture where decisions & strategies are made based on factual data.
Despite this trend, companies must be aware of fallacies which are commonplace and can turn DDDM into disastrous decision making (DDM). Here are the main fallacies in which companies fall into:
- Reliance on HiPPOs (highest paid person in organization) opinion despite data telling otherwise or simply ignoring it if the data does not tell pleasing story.
- There is also another extreme where company solely relies on data and follows it to the bone, where human reason has been eliminated from the whole process.
- Fitting, cooking, manipulating or cherry picking the data to comply with initial gut feeling, hypothesis or beliefs or reaffirm initial though process.
- GIGO data which might be only thought about bad data quality or simply wrong data. but also put improper data that can lead to wrong decision. Here are the main issues:
o Not representative enough (biased sample, “company’s population” rather than the actual population).
o It is backward looking only and only descriptive (trying to predict future only based on past data).
o Too much data which leads to confusion, complexity & decision-making paralysis.
o Lack of data that takes into account global changes, trends, and qualitative data into consideration.
o Finding problems in the data rather than reasoning or hypothesis
o Manipulative methods helping to shape the data story as one pleases (tampering axis, selecting favourable periods, reverting axis, use of colours, etc.).
- Finally, there are also cognitive biases that prevail despite data telling otherwise
o Confirmation bias – company managers tend to favour information that confirms the beliefs they already have and ignoring the rest of it – cherry picking information.
o Cognitive inertia – the inability to adapt to new market or societal conditions and sticking to old preconceived beliefs despite data proving otherwise (e.g. Kodak, RIM, Nokia, Blockbuster).
o Group Thinking – willing to be part of the group (especially senior leaders) by siding with their opinion or opinion of the majority, despite lack of evidence or motives to prove it.
o Optimism Bias –decision making that is based on the ingrained paradigm that the future will be much better than the past (that is why budget will never be lower than last year which in turn creates over production or cyclicality that spills over the system).
These fallacies are commonplace and even the best organizations struggle to fix them, yet there are few suggestions that companies can make to mitigate these issues.
The first thing that DDDM company should do is to get the buy-in and championing from the top management. If top management does not want to transition to DDDM then it will be extremely difficult to create such culture inside the company. Such buy-in immediately reduces the risk of HiPPO where that person will ask data to give the right answer and not just his gut feeling.
Another aspect would be acquiring data of outside the company and being aware of major shifts. This is where top management should be encouraged to think most of their time out side of the business – to seek opportunities, understand competition, trends, ways of working.
For decision making data should use past to understand why the company is where it is now but it should not use it to derive the future, for this, the company must engage in the effort to build predictive models, include information outside company, trends, and be aware of biases in doing so. Another thing is assumptions, they must be stated very clearly instead of a caption note which is in most cases, the assumptions should be tested and checked if they would hold if there was a different result. Assumptions are the most important input for the future where data can only help getting the right answer but it will not tell one.
With all this data it is very easy to drown in it and companies sometimes overdo measuring everything, this leads to measurement of couple of hundred of KPIs which dilute the focus and create the company a reporting factory rather than decision maker. For this a balanced scorecard can help. All KPIs that the company currently has should be measured in terms of financial or balanced scorecard impact, only the ones that actually change company’s performance should be left to track and others removed. I personally had an experience with my company where about 87 KPIs were reduced to 16 meaningful ones. They were structured by departments with about three KPIs per department, clearly indicating focus & what’s important.
In order not to overly rely on data a Centaur approach can help. In a nutshell, it is the idea that computers or data help empower human decisions. Most famous example of this is Freestyle chess where couple of students with an average computer beat IBMs Deep Blue and chess grandmasters. It combines best of both and enables to synthesize the best possible decisions where human gut is blended with computing power & control over data.
Finally, regarding cognitive biases, these are the toughest to beat as they are ingrained into our psyche. What helps the most is being aware of them and putting them clearly in front before any important decision making, then there must be a checklist to validate if the decision does not fall into any of these aspects. Another thing that can help manipulative data is setting a visualizing standard to adhere to and creating a data governor position which ensures integrity of data & checks whether there is any tampering with the data. That person should ask the following questions:
- What is the data source; is it truly representative and unbiased?
- What are the assumptions based on the data, would these assumptions still hold under a different result?
- Would independent variables impact the outcome and how to treat them?
- Could a different analytical approach change the result, if so, which one to choose?
DDDM is the future for companies to obtain competitive advantage, yet it is crucial to ensure that data based on which decisions are formed and the arguments for those decisions are bulletproof, this is why companies should invest a lot of effort, training and establishing checks and balances to ensure data is treated fairly and decisions are based objectively, otherwise it can become a case where DDDM quickly escalates to DDM.