How Litmus Uses Integrate.io to Enable Company-Wide Reporting

A case study showing how Integrate.io is used to connect and report sales data for Litmus between Salesforce and their Redshift data warehouse.
Leonard Linde
Keynote Speaker
Safiya Moran
Senior Manager of Operations & Data Science at Litmus
Areas We Will Cover
Litmus, Data Integration, Integrate.io

Share This Webinar

How Litmus Uses Integrate.io to Enable Company-Wide Reporting

In this brief and informative talk, Safiya Moran, Senior Manager of Operations & Data Science at Litmus, shares how Litmus uses Integrate.io for company-wide reporting. Moran shares how Litmus needed a way to connect sales data from Salesforce into their data warehouse. She details the company’s data challenges and how hard it was to find a tool that easily integrated with Salesforce and didn’t require engineering support. 

Moran shares how Litmus uses Integrate.io to connect and report sales data. In addition, there were unexpected benefits to using Integrate.io, saving time on manual sales ops tasks for example. She concludes with three tips that will help any Salesforce developer or user who needs to maintain quality data connections between a database of record and Salesforce. The short Q&A illustrates Integrate.io’s ease of use. 

This talk is also useful for anyone considering implementing Integrate.io as it provides a customer success story and example of success.

  • Building reports and modeling in their data warehouse. [00:01:44] 
  • Reporting Challenges: Direct sales data (from Salesforce) didn’t exist in their data warehouse and they needed to push data from the warehouse to Salesforce. [00:03:32]
  • Finding a solution: After trialing products they needed engineering support for, they found Integrate.io. [00:05:23]
  • Using Integrate.io to schedule data movement in and out of the Redshift warehouse. [00:07:37]
  • Unexpected bonuses to using Integrate.io, like saving hours on previously manual sales ops tasks. [00:09:17] 
  • What we learned: Continuous communication with your Salesforce admin is essential. [00:10:23]
  • Q&A. [00:12:55]

[00:00:00] Hello, welcome to another S X-Force summit presentation brought to you by Integrate.io. And today we're excited to have one of our customers. Safiya Moran who works for Litmus, which is a company that makes software that enables email marketing. And Safia is going to tell us how, Litmus integrating their Salesforce instance and perhaps other of their data sources, using Integrate.io.

[00:00:40] So without further ado, here’s Safiya.

Thank you. I'll talk a little bit about how Litmus is using Integrate.io to enable company-wide reporting. so first off, I want to kind of talk about what our challenge was we were solving [00:01:00] for, this was unique to our company. But I imagine that other companies and other customers out there have probably experienced something similar.

[00:01:15] So when I joined Litmus, in 2016, it was a pretty small company. We were about 30 employees. And, when I was hired on in 2016, I was the very first data hire. so we were starting very much from scratch in the data world. Over the next couple of years, we built that data and analytics team out. As part of that, we started building out a more robust data warehouse.

[00:01:44] We added on a data translation layer and we started building out a bunch of dashboards and reports and even modeling and built on top of all that data in our data warehouse. And then we got to 2019 and as we were building all this stuff out in 2017 and 2018 there was a [00:02:04] lot of other stuff happening in the company.

[00:02:06] And we got to 2019 and found ourselves at a point where we needed to really overhaul the majority of the assets we had built out in the data and analytics team. So to give you an idea of kind of what happened in the business that brought us to that point in 2019. Starting in 2016, 95% of our revenue was coming from self-serve customers.

[00:02:34] So folks that were just coming directly to our site and signing up for an individual plan. Over the next few years though, Litmus started investing heavily in the direct sales channel and as a result, that part of the business group, However, back in 2016 and 2017 when we were building out our data warehouse and that data translation layer, and creating all of our business metrics from that data, because the direct sales part was such a small portion

[00:03:06] we weren't building that out to scale. We were focused on the self-serve piece because that was where the majority of the revenue was. Well, we got to 2019 and found that 50% of our revenue now came from direct sales and we had no reporting around it. All the reporting that existed, our entire data warehouse, was all focused on self-serve.

[00:03:32] So that brings us to challenge number one we're solving for. We needed to incorporate the direct sales side into our reporting. And the first challenge we came up against was that the data for direct sales did not exist in our data warehouse. We already had our internal backend system and our billing system, which was what we used to build everything out for the self-serve business.

[00:04:02] Those were already in our data warehouse, but we didn't have anything from Salesforce. In our data 

[00:04:08] warehouse, and that's where almost all of the data we needed to be able to report on the direct sales business lived. And then challenge number two that we were facing was, aside from getting data out of Salesforce and into the data warehouse in order to build out that translation layer and those business metrics, we also wanted to push data to the sales team in Salesforce.

[00:04:37] There were a lot of derived fields that we had created in our data warehouse and in that translation layer. Derived fields such as we had done modeling around health score and stuff, plan information for existing self-serve customers, things that would be really helpful for us to be able to pass to the sales team when they're working those leads out of Salesforce.

[00:05:01]and then there were the other data systems that we had already connected into our data warehouse that also weren't talking to Salesforce. So there was also that piece there that [00:05:00] we wanted to pass some of that information into Salesforce as well. These were the two primary challenges that we were faced with when we got to, late 2018, early 2019.

[00:05:23] We went out looking for the solution to this. We started our hunt for a tool that would enable us to incorporate that Salesforce data into our data warehouse. So we were looking for a few different things in this solution. First of all, we wanted something with native Salesforce integration, which at the time that we were looking was a lot harder to find than we thought it would be

[00:05:56] On the data and analytics team, while we all consider ourselves pretty tech-savvy, none of us are engineers and building a REST API connection just wasn't something that was in our wheelhouse. We needed something that was already built in with that Salesforce integration. We also needed the bi-directional data thing.

[00:06:19] So as I mentioned, the two challenges we're solving for, we needed to push data to Salesforce and get data out of Salesforce. We also needed the scheduling capability, so we needed to be able to set it up on a regular cadence so that every night we're pulling down information from Salesforce that is then that translation layer and all those views are being created and cached on the back end of our data warehouse.

[00:06:48] And then something that was intuitive and easy to use. As I mentioned, the data and analytics team, definitely tech savvy, but we're not engineers. And there were several products that we trialed that, quite frankly, we just couldn't figure out how to use without quite a bit of support. So we wanted something that we could get in there so that

[00:07:11] we could set up everything without having to rely on getting engineering resources. So we came across Integrate.io and Integrate.io really checked all these boxes for us. It was definitely easy enough for us to do, easy enough to get that Salesforce connection set up. We've continued to be able to use it easily with just the data and analytics team maintaining that.

[00:07:37] So the solution to the particular challenges we had were, first of all, getting that Salesforce data to our warehouse, Integrate.io makes it really easy for us to copy those Salesforce objects directly into our data warehouse, which is a Redshift warehouse as we could just bring them in as tables. It's very easy.

[00:07:58] We have it run every night. It just overwrites those tables every night and recreates them in our warehouse. And now that we have all those Salesforce objects in the data warehouse, that gives us the missing piece that we needed to have that raw data. And now we can do all the [00:08:15] translation layer, we can convert those to all the business metrics.

[00:08:20] And so now we have all that for the direct sales side as well as on the self-serve side. And then challenge two the solution to that was also easy with Integrate.io. it does have that bi-directional sync. So we are able to pass data back into Salesforce. So some of the things we are doing currently to pass data into Salesforce is for accounts that are existing, self-serve customers.

[00:08:45] We're able to pass information about that account and their current plan information into Salesforce. We're passing different product usage metrics and firmographic fields. as well as we've built out some health score models, that we're able to flag certain accounts based on that, and we can pass that metric over to Salesforce as well. So now it's accessible to the sales team in Salesforce. And then there's a couple 

[00:09:17] kind of added bonuses. These weren't things that we were looking for when we were started, looking for a solution and a tool. But there are things that Integrate.io has allowed us to do 

[00:09:28] that have been really beneficial. So now that we have Salesforce and Redshift data warehouse sync, we've actually been able to automate several of these sales ops tasks. So these were things that our sales ops specialist was doing manually every week, and it would take hours to complete these tasks.

[00:09:52] It was things like just updating data on specific accounts and specific opportunities. And we are able to automate that now using packages and Integrate.io. And we're also doing some quality control reporting now that we have the Salesforce data and our billing system data all within our data warehouse 

[00:10:17] it's easy to do cross-checks to make sure that plan information is matching up between the two and stuff. So we use it also to run some regular QC reporting. So along the way, there were a few things we learned that, hopefully, will help other people setting up something similar. So the number one thing we've learned is that communication with your Salesforce admin is essential.

[00:10:45] You're of course going to be working with them when you do the initial setup and you get the initial connections created. And probably when you're creating the initial packages as well and figuring out what you're writing to, but once you get all that set up and it's just kind of running nightly and you've got it scheduled and it's fairly hands-off; that communication needs to continue.

[00:11:08] We've run into issues where they’ll be making changes to objects in Salesforce and not necessarily considering the downstream impact that might have; and it'll end up causing our job and package to fail. And it's usually a pretty easy fix, but it's never great to not find out about it until something fails.

[00:11:34] Maintaining that line of communication is really important. The second is just considering timing carefully. There are timing aspects to all the steps. Our data warehouse has a translation in the translation views. Things happen at a certain time. And so we want to time the packages carefully so that we're getting the freshest data and passing the freshest data between sources since for a lot of things

[00:12:08] we have it set up only nightly. It is possible to set it up at more frequent intervals, but if you're doing it only at a 24-hour interval you want to make sure that the data you are passing is as fresh as possible at that point in time so it's not a bigger lag. Third is using a designated login or user for both ends.

[00:12:31] So for this, that meant you creating a user called “Integrate.io API user” in Salesforce. This makes the auditing and the troubleshooting just so much easier when you can tell exactly which system created a record in Salesforce or changed a field. This makes that audit trail a lot easier.

That's what I've got. Thank you so much. Hopefully, that will be helpful to some of you. 

Well, thank you, Safiya. I've got a few questions. First off, I just have to underline that's really smart to assign a user as your Integrate.io user, so when somebody sees something happening, Salesforce, it's not under somebody's name, and then it's like, why did so and so do such and such.

[00:13:22] Q: So tell me a little bit about the communication with your Salesforce administrator. When that person makes a change do they write an email or do they call you up? 

Moran: Yeah, so we, our company, because a lot of us are remote, we do a lot of communication through Slack so it's usually just a quick Slack message.

[00:13:43] Just, Hey, heads up, I, you know, remove this field from this object. They usually try to confirm with us first to make sure it's not a field that we were using. In most cases it's not, there's so many fields in our Salesforce instance that most of these are trivial. So confirmation we're not using that field and then just a heads up so we can update the package and the next planning. 

[00:14:15] Q: Who normally writes the Integrate.io transforms or data pipelines? Is it you or is it somebody who works with you?

Moran: It's typically been me. The role I'm in now is actually a new role for me, so I was our senior data scientist up until February of this year. In that role, I was creating most of these and I've kind of maintained that as we've gone on. But we've empowered kind of everyone on our data and analytics team to create packages as they need them. I handle a lot of the Salesforce things, but some of the other data connections we have set up are handled by other folks on the dating analytics team. 

[00:14:58] Q: Wow. So they're able to just pick it up and learn Integrate.io without too much hassle. 

Moran: Yeah, absolutely. It's pretty easy, especially if you have some kind of a template or an existing package to go off of, to kind of just take that, copy it, and tweak it to what you need.

[00:15:21] I'd say the hardest part for any of us to learn is that initial connection set up to any of the different systems. And once you get that right, and get the permissions right, it's very easy to learn and set up. 

[00:15:34] Q: What reporting tool or reporting tools are you using to get your reports?

Moran: It was called Periscope, it's now called Sisense for Cloud Data Teams. We were an early adopter of that tool and they also maintain our Redshift data warehouse as well. 

[00:15:54] Q: And that's, and you're sticking with that. You think that's a good choice. Works well for your company.

Moran: We're pretty invested in. We've built out quite a robust reporting portfolio in it, so we're pretty invested in it at this point. 

Q: So the other, the other system, I guess it's a self-service system is the other system that was already in your data warehouse. 

Moran: Yep. 

Q: And that has its own loading procedure and everything.

[00:16:24] That's not, that wasn't done with Integrate.io. That was a legacy, correct. 

Moran: Yeah. So we initially set up quite a few of the connections where possible through Periscope and they were able to pull those, connect to our other systems, which, like our backend self-serve system is a MySQL database, for example.

So we certainly could have pulled that in through Integrate.io, but we already had that setup.

[00:16:54] Q: Periscope doesn't have a direct Salesforce connection or it wouldn't make any sense to do that anyway. Right. Blow through your governor limits. 

Moran: [laughs] Right.

Moderator: Well, this is a great example of how easy it is really to connect something up with Integrate.io in Salesforce.

And I think you're right. as somebody who used Salesforce your insight that it is strangely hard to find something that's both easy to use and powerful enough and bi-directional with Salesforce. So, well, I'm glad you found Integrate.io and I really appreciate your time today. Thank you so much and wish you success and good health.

Moran: Thank you.

The Evolved Stack
for Tomorrow's Leaders

The no-code pipeline platform for
your entire data journey

Ensure Data Quality