I have observed that in most organisations there is a clear delineation between teams which manage data lakes or warehouses and teams which analyse data. The specialists who build BI assets such as reports, dashboards, and cubes will be separated from the engineers who maintain the data platform.
Furthermore, there is sometimes a third, separate function. Between the data platform and data analysis factions there very well could be a team focused on ETL (and other forms of data preparation).
Issues with separating ETL and Analysis functions
However, this is not an efficient way to organise the work. Fact is – data prep and analysis go together. You must begin with the know-how of how precisely the data will be used. Only then can you prep the data for analysis. This approach ensures that the data is in the right format, has the right level of detail and is updated at the right frequency.
Creating artificial boundaries between teams can lead to missed opportunities, and often to the kind of governance problems. In fact, in my 20 years in this industry – it’s been true that approximately 80% of the time is spent on the data part of analytics – this percentage hasn’t changed. A big part of this is caused by the artificial boundary that is created between data prep and analytics.
Improved Analysis by better Data Preparation
Think of it as a simple unified workflow: an improvement in data preparation, will lead to greater accuracy in data analytics, leading to better decisions for better enterprise outcomes.
The ability for your organisation to prepare data will ultimately dictate the type of analysis that can be performed. Business-driven analytics falls on the back of how flexible your data preparation is, and for most cases, the more flexible the process, the simpler it is to adapt to different data sets, resulting in efficiency in leveraging clean data for analysis.
With the right tools on hand, users can maintain control, accelerate data prep/analysis, to build the foundations for an improved data-driven decision-making process.
So how do you approach data prep and data analysis in your business?