Tackling data automation, one bite at a time

Putting data automation to work for your organization is an important and challenging task. If you are not a data engineer, or don’t have a deep bench of talent in that area, how do you even begin automating your data? While it’s normal to feel overwhelmed with the task ahead of you, in my experience it helps to break your data automation journey into smaller parts. After all, the only way to become data strong is one bite at a time.

Work backwards from the desired outcome

So, before we even discuss data automation, one great way to get started is to ask yourselves, “Why are we doing this?” The answer to this question is deceptively simple: your team needs unified and normalized data to drive business insights in a timely fashion. If, for example, your Ad Monetization team is looking to understand advertiser performance from 25+ data sources in one place, they can use unified data to find insights that were simply inaccessible in the raw data.

So, you begin your data automation journey by understanding the answers you are seeking, and work backward from there.

Automate according to each department

Additionally, it helps to determine how different departments would benefit most from a particular view of the data. For instance, revenue teams are likely to need a different cut of the data compared with business teams. With the right reports at hand, teams can fully understand where to focus their efforts.

At a very high level, our approach to breaking down the DataOps journey at Switchboard is ETL – extract, transform, and load – with data automation playing a key role at every stage in this process.

• During the extraction phase, pulling data automatically from various sources saves you time, reduces costs, reduces manual errors, enables you to make more accurate decisions, and removes mind-numbing, repetitive work from your data teams.
• Automating transformation helps your organization ingest new data so that up-to-date records are available. If you’re not operating with at least same-day information, you are likely making decisions based on old data, which is not much better than guessing.
• Loading your data into a data warehouse using automation gives you the confidence that the data is in the right format to access the exact insights you need.

One challenge is that, early on in the data extraction process, people learn that every extracted data source exists in a different format. Another issue is that you can’t just use the raw data for automation – it needs to be normalized first. We will discuss these challenges, and solutions to them, in future posts.

Data automation is foundational to your data journey, and by breaking the process down into bite-size chunks, you can be confident of providing your non-technical business teams access to accurate data in a single format that simply wouldn’t be possible without automation.