Plug and Play Data Movement in the Cloud

Plug and Play Data Movement in the Cloud

Any developer who has ever been tasked with consolidating data sources before performing analysis knows the challenge of silos, especially when data is sprawled out across multiple third-party services. But what about if you had to help a client pull application data from 47 different MySQL databases and then consolidate it into an operational data store (ODS) for analysis?

In Episode 7 of The New Builders, Killian and Maston break down how they deployed a cloud-based data warehouse to solve the data silo challenge (3:14), how plug-and-play data movement tools enabled them to reduce an ODS project timeline from three months to three hours (8:03), how data warehousing differs at enterprises vs. startups (22:19), and why “learning the data” is the best step any developer can take, even if they aren’t directly involved in data warehousing (29:22).

Other posts you may be interested in

How to build a Singer tap: infographic

"How do you eat an octopus? One bite at a time…" When I first encountered Singer, I was armed with basic Python skills and experience with REST APIs. I was familiar with data integration and ETL but I had no knowledge of the Singer platform. After learning how to use...

Looker ERD Generator – from an Explore using the Looker API

Have you ever wanted to create an Entity Relationship Diagram (ERD) from your Looker Model Explores? As a Looker partner consultant, I get asked for an ERD or Data Model all of the time. Data Model diagrams provide a concise, visual way to show how the tables (or...

Plug and Play Data Movement in the Cloud

Any developer who has ever been tasked with consolidating data sources before performing analysis knows the challenge of silos, especially when data is sprawled out across multiple third-party services. But what about if you had to help a client pull application data...

Small Changes Can Lead to Big Improvements

The way in which a simple fix to one problem can have unexpected effects in other areas is always interesting. Anyone who has done any coding will know that changing one little thing can affect the larger project in ways that aren’t always apparent at the outset....