We see it all the time, you’re spending more time wrangling data than actually analyzing it. Your laptop crashes every time you put together an Excel-based weekly report. As companies struggle with siloed fragmented data, there are three fundamental steps you can take to start building a scalable on-demand data system. These three tips will help you say goodbye to enormous excel files and hello to on-demand data all living under one roof.
- Who, What, When, Where, Why Your Data
For each data source, ask the 5 W’s to understand why the data is important to your organization and where the data should be catalogued.
Who is the consumer of this data? Which business groups need this data? Which business groups were behind the creation of this data?
What kind of data is this? Is this transactional, marketing, operational, technical data?
When does my company need this data? At what frequency do we need to have fresh data from this source?
Where does this data originate? What is the system that this data comes from?
Why is this data important? What decisions are made using this data?
From this exercise you should end up with an explicit catalog of data organized by source, consumer, content, frequency and business usage.
This question is also one of the most important things you can identify when building a data system. Automation often relies on consistent naming and labeling of data. Since messy data can be difficult to impossible to automate, naming your data consistently across data sources will save you time and money and increase your automation capabilities.
Marketing data should take the same format across marketing channels, product names and attributes should be the same across transactional and inventory systems and so on. You need to do this retroactively for historical data but at a minimum, employ a robust set of business logic for naming your data.
- Automate, Automate, Automate Your Data. The next step is to take the data sources you’ve documented and automate the collection and storage of each of these sources.
Once you have your data saved, plot out how data sources relate to one another and in what, if any, ways they can be joined together. This is where consistent naming will pay dividends by way of reduced development effort. Congratulations, you’re now ready to automate the cleaning and joining of your raw data into a format that’s ready to be consumed by the respective end users for this data.
- Spread The Love, er Data
With rich, high fidelity data available, you’re now able to proliferate the number of people who have access to this data via visualization software, such as, Tableau, Looker or Google Data Studio. Data that once was siloed, raw and required knowledge of SQL to access and join is now available on-demand at your fingertips.
