Building your one-version-of-the-TRUTH cloud-based data warehouse (OTTER) is the journey of owning your company’s most valuable asset - your data. After learning about what it means to own your data in the previous edition, we now have to roll up our sleeves and dive into the actual hard work of owning our data. There’s no denying it: loading your data is where the real work begins. This is the hardest, most challenging and the most critical stage: getting your data into your cloud-based data warehouse.

If your OTTER is the foundation of your data-driven future, then this step is like pouring the concrete. Do it well, and you’ll have a solid base for all your future data initiatives. Do it poorly, and cracks will emerge that weaken everything built on top. So, let’s roll up our sleeves and discuss how to do this right.

Why Loading Your Data Matters

In our Own Your Data article, we talked about the fundamental steps to owning your data:

1. Identify Key Systems: Start by pinpointing your most critical data sources. These might include ERP systems, CRMs, and financial platforms. Aim for the essentials—quality over quantity.
2. Establish Secure Connections: Connect securely to each system, whether internal databases or SaaS platforms. Internal systems may require VPN access or database views, while SaaS platforms might offer APIs or data exports.
3. Extract and Load: Begin by loading small batches of data into your OTTER. This allows you to identify gaps or issues before scaling to full datasets.
4. Transform for Clarity: As data flows into your warehouse, clean and enrich it. Use this stage to remove duplicates, standardize formats, and ensure accuracy.

What we’re focusing on right now is step 3, extract and load.

At this stage, you’ve already identified and connected to your data—those SaaS systems, legacy databases, and even the occasional rogue spreadsheet. But raw data, sitting in silos, doesn’t create insights or solve problems. To unlock its value, that data must be loaded into your OTTER, which can become the one version of the truth that drives your business forward.

Loading your data is no small task, but with the right approach, you can set your OTTER up for long-term success. Here are some actionable strategies:

1. Start with Iterative Initial Loads: Don’t try to boil the ocean. Instead, load your data in small, iterative batches, testing and refining as you go. This approach helps catch errors early, ensures scalability, and allows your team to build confidence with each success. For example, just start by extracting and loading 10 rows of data from each system. This, in fact, will be the most challenging part. Just make sure you have solid connectivity, that you’re getting good data, and that it is loading successfully. Once you’ve loaded 10 rows of data from each system, you’re well on your way!

2. Address SaaS and Legacy Challenges Head-On: SaaS systems are fantastic—until it’s time to extract data from them. Similarly, old legacy systems can be equally difficult. Have you ever tried to extract and load data from a mainframe? Each system will likely present unique hurdles, from API limitations to proprietary formats. Remember, there is always a way, so don’t give up and just keep asking questions until you find a solution that works. Perseverance is the key to completing the OTTER mission!

3. Load, Load and Load: Once you’ve persevered and successfully loaded 10 rows from every system, now the real fun begins. Start loading 1,000 rows, then 100,000, millions and then billions (however many your systems contain). You’ll have new challenges during this process. API constraints, file size, and file transfer issues, and new errors that did not occur when loading small numbers of rows. The OTTER mission is fraught with challenges, or if you’re a glass-half-full kind person, the OTTER mission is fraught with opportunities to overcome! Once again, the key is to just keep trying and keep searching for new ways to load the data until you find one that works - there is always a way, so don’t give up!

4. Build for the Long Game: Loading your data isn’t a one-time event—it’s the beginning of a continuous process. If you just load the data, your OTTER will be stale, outdated, and worthless in no time. You have to create automated jobs and pipelines that perform all those amazing processes you persevered to figure out to constantly refresh the data into your OTTER. Automation and monitoring tools are your friends here.

Why It’s Worth It

Loading your data is hard, but as John C. Maxwell says, “Everything worthwhile is uphill.” This step tests your leadership, your problem-solving skills, and your perseverance. But when it’s done, the rewards are immense: a unified, trusted, and actionable repository of truth that becomes the foundation for your organization’s future success.

The journey to becoming a data-driven leader and transforming your company into a data-driven business doesn’t end here—but this is the stage where it truly begins to take shape. So, keep climbing your OTTER will be your company’s most valuable asset.

‍