Producing faulty analysis and steering my organization in the wrong direction based on my results has been one of my biggest fears since I started as an analyst. And to be honest, being called out and called out even more for my mess.
My nightmare scenario: A major feature that my team has worked on for months, based on my recommendations, fails to deliver value during testing. People start asking questions and upon investigation it becomes clear that my assumptions were based on incorrect analysis results.
Having spoken to and mentored many analysts since then, I have learned that I am far from alone in feeling this way. On the contrary, This is a very common fear shared by many analysts.
The reasons why one might be concerned about an analysis are obvious: publishing and promoting erroneous results and recommendations both affects the organization as a whole and at the same time poses risks to us personally.
Despite these risks, avoiding sharing analyzes or avoiding results that are controversial or challenge assumptions is not the right approach. In fact, these analyzes are often the most important. Instead, it is crucial to develop strategies to mitigate the risk of producing faulty results. And that starts with appreciating the following:
Making mistakes as an analyst is really easy.
Analysts work in a very complex environment with a high degree of freedom. So there are many opportunities to make mistakes or draw incorrect conclusions.
Performing an analysis involves several different steps:
- Define the research question and approach
- Data collection and cleaning
- Analyze and interpret data
- Visualize and communicate results
At each stage, it is quite easy to make a mistake which will ultimately be reflected in our recommendations and…