Rightsizing Your Salesforce ETL Solution

Welcome to another X Force Data Summit session. Today, it's going to be me and Mark Smalcom. I'm Leonard Linde. I've got a lot of experience in a lot of different platforms, including Salesforce.

I've been a software developer for thirty years, I think. I've also been the CIO of a small company. I founded my own software company and done a lot of different things. And with me today is Mark Smallcom. He's the Chief Technical Officer at xPlanity. And I'll let Mark tell you a little more about himself.

Yeah, thank you, Len. Yeah, I'm CTO at xPlanity. I head up development support operations at xPlanity. I've worked in Silicon Valley for a number of years, worked at some big companies like Pinterest and Funny or Die and City Search.

And I've moved my way back to Sydney where I live now and manage our global team.

I've been involved in a lot of data projects over my time. I've worked at Deloitte in Sydney, I also worked at News Corp. So I've worked in very, very small companies and very large companies, and have seen a number of things and worked with a number of different Salesforce environments as well during that time. Thank you, Len.

All right, well, let's get this party started and take a look at our agenda today. We've done the introductions.

Our topics are, this is gonna be more of a freewheeling session but the first one is gonna be the unique challenges of the Salesforce platform.

There's quite a lot there, we're just gonna touch the surface of that but the important point there is that Salesforce is a unique platform. It's not like, you know, the database or data warehouse environments that a lot of different integration tools deal with.

Mark's gonna talk a little bit about the integration trends we see in the market and I'll pitch in a bit on that.

How do you select an integration platform? What are the factors you use to select that? We're talk about that and we've got a little bit of experience based knowledge on that.

Integration solutions, we think that total cost of ownership is a real big factor especially in the Salesforce arena. And we're gonna go over a chart that shows you the wide range of integration that's available in Salesforce from free to very expensive and try to give you some insight into how you might wanna right size your integration choice.

Then we're gonna close out with some tips and tricks on integration projects and then we'll finish out with a few final words. So with that, let's start talking about the Salesforce platform.

So Salesforce is interesting. For one thing, I think it's probably lagged a lot of other what you might call database platforms in terms of getting integration tools that are purpose built for the platform simply because Salesforce itself provided a lot of really good enough tools for free with the platform.

We start with the new tool in town which is a Salesforce Data Wizard which basically through a web interface allows you to load data directly into objects in Salesforce. Prior to that and still existing is the Salesforce Data Loader which lets you take data from Excel spreadsheets and push them into Salesforce and also take data out of Salesforce and put it into Excel spreadsheets. And of course you can't forget reporting. Salesforce has always let you export reports into CSV files, and you can actually do quite a lot with that to integrate your data.

Alright. Well, that's the first thing that I'm gonna have to write down three forty eight dot cough.

Interesting challenge of the platform is unlike most relational database platforms, so even if you're running SAP, you're running SAP in Oracle or some other or SQL Server and almost every ETL supports that. Salesforce has a fairly unique object database that isn't supported by every tool. And that's important because even if, and another factor is some of that support is unidirectional. They'll take data out of Salesforce, but they won't put it back in. So that's something to think about when you're looking at ETL tools, especially when your organization is buying an ETL tool.

That brings me to the next dilemma or conundrum that you come up with with Salesforce which is if your organization's buying an ETL tool sometimes you'll be left in the dust because Salesforce is often implemented in what I call a rebel alliance implementation, which means that somebody's been fed up with whatever CRM tool or lack of CRM tool that centralized corporate IT has decreed would be the product for the company. And they went out and bought Salesforce. They got some money and they bought it and they hired an administrator and they ended up and that ends up having, you end up having a DIY culture.

By that I mean that you're not going to be relying on central IT to support you with the software as a service platform like Salesforce, a cloud platform, you can often support yourself and that leads to some more interesting cultural factors about Salesforce and I'm going use Ian Gott's term for this. Ian made a really interesting presentation, which I would recommend to everybody as part of the conference And he came up with two terms, one is accidental administrator. That's not original to Ian but it's really common Salesforce culture, cultural factor which means that people that are Salesforce administrators often grow into the job.

They might've been in sales to start with their sales ops and then they become a Salesforce administrator. They don't have an engineering background and that's not a knock against those administrators.

Can be very effective but what happens to an administrator often is the side effect of not having an engineering background is what Ian calls uncontrolled agility, what I call kid in the candy store, which means that you have this tool that is very powerful. You can do anything with it and you do.

You end up building tons of custom objects, tons of custom fields. So what happens is you have kind of an out of control Salesforce instance that you have to roll back and that makes integration hard because if you have tons of custom objects and tons of custom fields, a lot of which you might have abandoned, if you pull up your integration tool, you point it at Salesforce and you get this giant list of objects and you're wondering which ones of those objects you really want to target or you really want to take data out of. And you might have been the second or third administrator in that instance, that Rebel Alliance instance, and you have no idea which one of those objects are relevant to today's task and which aren't.

Another interesting or problematic part of the Salesforce equation is governor limits. It's very easy to point a tool at Salesforce and just roll through your API limits very quickly. Now Integrate.io, not to plug our tool too much, but Integrate.io like any good Salesforce integration tool uses a Salesforce bulk API. And the Salesforce bulk API, because it allows you to be a good citizen in a multi tenant organization, multi tenant software, software as a service platform like Salesforce, the bulk API is much easier to use. It doesn't take all the, it doesn't use up all your governor limits right away. But if you're right, if you're doing a roll your own Salesforce integration platform, don't do what I did the first time I ever pointed something at Salesforce and blow through your governor limits right away because you don't know what you're doing and you're going and running queries and retrieving one row at a time with every API call.

But if you learn that lesson, it's fine if you learn that doing it once. It's very painful if you do it in a production environment when you're on deadline.

That brings, and the governing limits exist for very good reason that Salesforce is a multi tenant object oriented database. Now a multi tenant object oriented database is going to be optimized for performance in transactional performance. In other words, it's optimized to throw a screen up to a user. It's not optimized to do bulk queries. And it's also as an object oriented database, it doesn't have native adapters.

Again, if you're in a if you're in an environment where you have a traditional relational database or even like a columnar data warehouse database like Amazon Redshift, you can use ODBC or JDBC and JDBC in the case of Redshift to go and query that database.

You don't have that in Salesforce. All Salesforce queries have to go through the API.

So there's no SQL. It's all SOQL, which is certainly not SQL. It's not, it's much more limited.

And I wanna recommend just for pure interest, another presentation that we have in the conference by a guy named Daniel Peter. And Daniel is a guy who has been saddled with doing some very unnatural acts with Salesforce in terms of getting performance out of the object oriented database. And Daniel has really jumped through some hoops for customers who had requirements like, oh, we need to bring down a hundred thousand records and do something with them in almost real time. Now, you had a relational database bring and it's a reasonably functional platform, bringing down a hundred thousand records is nothing really. But in Salesforce, if you watch Daniel's presentation, you'll see what he had to do to get that accomplished.

So I think the take home from this slide or this topic is that when you're looking at an integration tool at Salesforce, there's a lot of things that you don't have when you're integrating other platforms that you might as an integrator, as even experienced integrator that you might not expect when you come to the Salesforce platform.

Okay, thank you.

Now on this slide, we want to talk about four integration trends that we see in the market. So we're seeing this from our customers coming in.

The big one is privacy and compliance. This has been going on for a while.

GDPR and other standards are coming into the market. Next one we want to talk about is total cost of ownership, API platforms, and then decentralization of data. So if we talk about privacy and compliance, there's a number of big standards, as I mentioned GDPR, there's also CCPA, which is a California Consumer Privacy Act that's coming into power, I think that came in this year, actually. So a lot of people are focused on that and figuring out what to do.

There's also Brexit, which I think is pretty interesting. So obviously, the UK was part of Europe, under GDPR. And now with Brexit, that all gets turned on its head, and they've got to figure out what they're doing next.

There is the UK Data Privacy Act, which they have in place. And so the rumor is they're going to almost do like a UK GDPR, which hopefully will have no material difference, but it's certainly something to watch. And the reason why it's important for integration is one of the big things you have to do is keep your data in Europe or protect it in Europe. And so if you want to take it out of Europe, you need to start encrypting or removing data. And so common things we see are people wanting to anonymize data, they want to remove data, they want to mask data, or actually they want to encrypt it.

And we see that for GDPR. And we also see that for customers who have PII data, or even PHI data, which is for healthcare industries. And so they're important trends that we see. And because these governance regulations are, you have very material fines, what we're seeing is this is a trend that goes right the way up to the board level of these companies. And they want to manage the risk of that. And so the decisions that are coming in from the CTO, So it's less about the data engineer making the decision, it's actually being made at the CFO, CTO level, that they want to basically minimize their risk, and want to make sure all the data is compliant with the standards that are applied to their business.

Give you an example, a recent example for us actually, we had a customer come in, had their companies based in Frankfurt. And for compliance reason, I'm not sure what the reason was, maybe it was financial services company but they need to keep all their data in Frankfurt. And the beautiful thing about the cloud now is that it's actually possible. A few years back that would be a complete nightmare. You'd have to set up a data center and hire all the people, have all infrastructure in Frankfurt. We were lucky we could spin that up with Kubernetes and had to create a new AWS region. We had it done within a couple of weeks, which is fantastic and something you couldn't have done probably even five years ago, it wouldn't be possible.

Now with all the cloud vendors like AWS and Google and all the other ones out there, they all have multi regions and makes this much, much easier. So for people who are doing integration work, that is a huge benefit, I think.

The total cost of ownership, I think there's a few things people are doing is they're looking at different options. So there's lots of different solutions for integration, You can pick a low end point to point solution. And that's obviously low cost, it's got low functionality, probably doesn't do many transformations. But if your only goal is to get data maybe out of Salesforce, you know, it's just a single pool, that might be a good fit. And it might be a really good way of doing your ETL.

Another thing we see is people who want to do ETL with transformations. So they want to do all this before the data is stored. And it's often done for compliance reasons. So they need to make sure that the transformations are done.

As we talked about getting, you know, moving data or encrypting data, They want to do that before the data gets into their warehouse. And before it gets dispersed really within the company and before all that data is written to locks. The other trend we see is ELT, where the transformations are done in the warehouse. And that's another valid path to take as well, where you have a very basic extract, and then you push that into warehouse.

And then you run your transformations later on in your warehouse to remove data or to reformat dates and things like that.

And then the other trend we see is people wanting to build it themselves. And this can be a really good fit if you've got a very strong team. So if you have a very large data engineering team, that can be a great way of doing this. And it means that you can centralize everything.

But there's lots of downsides obviously about building yourself. You've got to maintain it, you've got to make sure you can hire all those people for your team. So we see that happening with some very large orgs where they have the ability to hire large engineering teams. And that gives them the flexibility because they can almost do whatever they want, whatever that is out there, and they can choose whatever product they want.

So when we talk about total cost of ownership, there's really not one size fits all. And there's not one size fits all from the vendor point of view. There's lots of great vendors out there, Integrate.io included, and I'm biased on that.

But there's lots of different paths that you can pick. And the paths actually depend on where you are almost on your integration journey. Like I mentioned, you might start off with point to point might be a great solution at the beginning. And then later on, you might decide that you need to do some transformations, or you need to write stuff into Salesforce.

And then you go down that journey. And sometimes people start with in house development, that's another path where they start, and then they find that it gets to be a maintenance problem for them. And they need a vendor. And that can also be a valid path as well. The other thing we see is this, the emergence of API platforms, which I think is quite interesting.

Everyone wants to call themselves an API platform now. And it feels like almost like the new word for enterprise service bus, which is probably an old term and Leonard and I probably have worked with that many times in other projects.

But it's now API platforms is the big thing. And the very large vendors like MuleSoft, I think, and other ones, and Informatica and vendors like that are promoting their API platforms, which is interesting, I think.

And the fourth trend we see is this decentralization of data, which I think is we've seen this with a number of customers. And what's happening is, originally everything was centralized into IT. So all integrations were all done in IT, probably with a big product, like Informatica, or maybe even custom built work. And what we see is that that worked well, but it meant that people in the business, like the business analysts had to always go back to IT to do all their pipelines.

And it makes them slow, it made them less agile, they couldn't get this work done quickly enough. And we have some examples of that. And what they're finding now is they want to almost push all this work out to the data scientists and the business analysts who are working with the business units, because there's been a decentralization of data science as well, and visualization. And so these BAs are actually sitting in the business unit, and they don't want to wait for IT to do this stuff, almost like you mentioned, Leonard about Salesforce, and how there's a culture of getting stuff done and almost getting around IT.

And just I need sales information today.

What they're finding is that VAs want to do stuff immediately, they want to build their own pipelines, they don't want to wait for central IT to do it. They want to have a tool that can do that. And that changes a bit of almost like the tool they choose. So they don't want to have a development platform where they have to write in code.

They're sort of self selecting platforms that are low code, drag and drop type platforms that they can run and maintain and look after, but also fit the IT's needs. They're security, they know how to do encryption, all those sorts of things. So that is a bit of a blend of development as well as decentralization of creation of pipelines as the data teams are being decentralized.

Yeah, I just want to add Mark that in the presentations in the conference from ex Plenty customers, for one, it's all data analysts who are doing a DIY solution.

And I also wanted to plug when you're talking about privacy and compliance HIPAA huge in the United States, mentioned private health insurance, private health information. Matt Gretzko has a presentation where he talks about Salesforce for healthcare that if people are interested in, know, the healthcare and some of the issues around privacy and so forth. He does touch on that.

Oh, that's great. I'll make sure I check that one out as well.

Great. So the next slide is really to go through some of the things that we see about how do you select an integration platform and some of the decisions that you need to make.

So when you look at the core systems, I think the first step is to look at the core systems that you want to integrate to. And basically look deeply at them and figure out opportunities are there.

You often have a business requirement to do an integration and get some data out. But there might actually be other opportunities, other business opportunities that come up because of this new platform or this new integration you're going to do.

And it's important to check the vendor support for the integrations you want to perform. We see a lot of people who have integration platforms that for instance, read out of Salesforce, but they can't write into Salesforce, which is fine at the beginning, you think, that's fine, I only need read, you know, this is the only thing I want at the moment. But you know, two years down the track, you probably find that you actually want to write stuff into Salesforce as well and update it with some information from your ERP system or something like that.

And to do that, you'll actually have to probably change vendors, which is an enormous total cost of ownership issue. Because most of these vendors, it doesn't matter which vendor you pick, the pipelines are normally proprietary, and they need to be rewritten when you switch vendors, which is a big cost. So recommendation there is check almost the systems you want to integrate with, not just the ones you want to do today, but almost the ones you want to do in the future. Just make sure that you have that covered. And they can do those things. Because it be painful if you learn that a year later that you need to do the same right to Salesforce.

And the other thing with of course with Salesforce, like you mentioned was, you need to make sure it can handle or uses the bulk API for the governor limits. That's another important criteria, I think for Salesforce. And not all the vendors do that. So worth checking into that.

And then team capability, think is important. And there's, you know, there's not a one size fits all on this either. But it's who's going to build and maintain these pipelines? You know, is this going to be a developer, you can be someone who's got, you've got a team of Python developers that are really skilled at that.

You happy maybe working in a proprietary language for a vendor?

Or is it going to be a BA or a data scientist that's going to do this work.

And so, you know, looking at your team capability, and also looking at whether this is a centralized IT, or is this going to be, like I mentioned, distributed amongst all the business units, because that can make a huge difference to the platform you choose as well.

And then related to this as well is actually the user experience. And when I mean that it's more the user experience of the integration platform.

So if you are a business analyst, you probably want a low code drag and drop type platform that's easy to maintain and quick to use. If you're a developer, you probably don't want that at all. You want, you know SDKs and workflows and you want integration into airflow and more ******** tech stuff. And neither is one, one isn't better than the other. It's just that there are different user needs. And you have to pick the platform that fits your users, your customers in your company.

And so it's worth thinking about that.

And then on the cost side for these integration platforms, often cost is aligned with breadth of functionality is how I see it. So you know, if you want low functionality, like you want point to point, there's lots of great vendors out there that will do that. And they have a low price point and they have limited functionality. And then on the other extreme, there are really massive companies like MuleSoft, Informatica, Dell Boomi, companies like that, that have tonnes of functionality way beyond ETL and integration, you know, data governance and data lineage and all sorts of things, which is fantastic if you need all that. But it's good to know what you really need, rather than picking this massive thing, when you might only use a slither of the functionality. So again, just figure out like where are you on that continuum? Are you one extreme is point to point, the other extreme is very large enterprise vendors that have everything in one package.

And then the other thing to think about when you pick a vendor is that you probably want to think of like a three to five year timeline, I think maybe even longer. Because when you do these integrations, it's a lot of work to do integrations. These pipelines take time to build. And they, you know, you have to investigate systems, and it's an investment for a company.

So you don't want to switch them out, like you don't want to have a vendor for one year, and then have to replace everything the year after. So when you work with these solutions is find a solution that you think you're going to be comfortable with, for the next, you know, three to five years, and you can grow, the vendor can grow with you, and you can grow into this solution, and you feel comfortable with it. I think that's important.

And then on the data privacy side, we mentioned GDPR, if it's important for you, make sure the vendor can do a data privacy agreement, a DPA, you probably want a vendor that has security certifications like SOC two.

And then if you want to do things like HIPAA, need to have a vendor that can handle that and has the security standards that you feel comfortable giving them your HIPAA data.

So I think data privacy is important. And then the other thing we see is some companies want to actually encrypt data and encrypt fields of data.

And so if you want to do that, again, a vendor that can do things like field level encryption, and you feel comfortable that they can handle that level of sophistication.

And then on the scalability side, this again is something that's worth thinking about right from the start is how big are your data sets? Like how much data do you actually want to move in and out of your systems? And some, again, that can actually impact which vendor you pick. So if you have absolutely enormous data sets, then you need to sort of make that as part of the requirements right up front to make sure they can handle that.

And the other thing is data latency, which is a bit of a subtle one that you might not think about is almost like where are the location of the data centers for the integration. And so if I have an on prem, like I'm on prem in, say in New York, I have a database there.

And I want to do some integration work and maybe the other systems in a different region.

You know, that might not be possible with some vendors, some vendors might not have a data center in New York or not be able to write data locally in New York.

And so understanding the location of the data centers in relation to your data and your warehouses can be important for latency.

And the next one I've got is a real time or batch.

So we see this, this is another trend that we're starting to see a little bit is that people are talking about streaming of data. And you know, there's Kafka and Flink and other solutions like that. And there's some vendors like Aluma that just got bought by, I think Google is one of the vendors in this space. And so the question is, do you really need real time?

Is that critical for your business? And for some it is they need, to move data around within a minute or thirty seconds or something like that. And that can be a business use case. And that reduces the number of vendors that are capable of doing it.

Often, most companies can work in a batch, they can work in probably not a day batch now, I think everything is moving a lot faster. But you know, five to ten minute batch is actually almost like real time for ninety nine percent of all companies. And so again, when you're selecting an integration platform is you know, don't necessarily fall in love with real time if you don't need real time is like pick a platform, pick the tool that you really need. And you might find the batch is actually exactly what you need for your organization.

The other recommendation I have is create a matrix of your requirements before you actually talk to vendors. And so it's very easy to talk to vendors and get swayed on what's important, because that's what the vendor has in their product set. And I think it's much better to independently go through what you need from an integration platform or in fact, whatever technology you're picking is go through that create a spreadsheet, create the columns in the spreadsheet with the different features you want, you think are important and why they're important.

Get buy in from your team that that is actually true. And that's really what you all want and understand that. And then when you go to market, can talk to the vendors and actually see how they rate.

Because unless you're going to go for the really, really huge vendors, most vendors will be a set of trade offs. You know, some will have, you know, it might be data centers versus user experience or even, you know, some other thing might come up. And so good to have all those requirements in a matrix so you can actually go through them together.

I was going to talk about a couple of things I've seen which are quite interesting, just a couple of examples from customer point of view.

So on the agile, I've seen, this is almost like an example of decentralization. So I went and visited one company, and they actually received all these text files from banking customers. And the text files would all come in in different formats. And they found it very hard with these big companies to get them all to agree to a format and stick to a format. And so they'd find these files would come in, know, all sorts of stuff.

And they were a very big global company with a US head office that had a US data team and US data integration team there.

And what they found was it was really hard and time consuming when a new file format came in to work with the US team because of time zones. And this is in Sydney, and actually get the US team to make the changes and then send them back the updated pipeline. And so that cycle probably took two days to do.

And so they actually picked, you know, they picked Xpenseed for this solution. Because what they wanted to do was they needed the team to be empowered to make their own pipeline changes, and make them in real time. So when they got a file in from a bank, they could immediately make that change and do it independent of central IT. And so I thought that was a really interesting use case and very decentralized.

And then on the scale side that I've seen again with XPanty was we have a big customer, big pharmaceutical customer, who's working with one of the big four consulting companies.

And they're rolling out an Asia wide implementation of Salesforce using Integrate.io. And what they needed in that case is they, which is sort of a scale thing is they needed a lot of data out of the ERP system. So they need to do complicated joins across multiple systems. And then they wanted to push that data in near real time into Salesforce. So every five minutes, they're getting data pushed into their Salesforce instances around the world. And I thought that was pretty interesting example of how we've seen customers wanting to do that with Salesforce at scale.

Yeah, I wanted to add on the real time or batch thing, talked to Thomas Skranik, I think his name is.

I just murdered the pronunciation of his name but he gave a presentation about integrating from Segmint. Segmint is a massive data integrator that basically takes data from, he works for Comic Health Bitwise, takes data from all different systems including internal systems and pushes it, basically pushes out a stream of JSON from what I could tell.

And he uses XPlani and his company uses XPlani. He implemented, he's a data analyst. And his implementation runs I think once a day or twice a day. But there's one specific event that he wants to capture in near real time, which is when a customer, when there's an inquiry from a potential customer, he wants to push it into Salesforce as a lead immediately. And so it can be acted on immediately. And his choice to do that was to just write a Python script that uses the segment API and created a lead in the Salesforce API.

And essentially, you know, I don't wanna put words in his mouth, but what it seemed like to me was basically, look, the complexity of having a real time solution for everything was unnecessary. Instead, we'll just write one for this very simple use case and push that in and basically run that.

Don't even think it was triggered. I think it was, you ran it. They're just gonna run it very frequently.

So hybrid approach is possible too in this real time thing because most of your data in any business situation is going to be real time. I agree.

And there's real time in real time, you know. Yeah, yeah.

Real soon time and then real real time.

Yeah, exactly. Yeah I mean if you're a military then yeah you need real time, a lot of most businesses, I mean we used to do everything in you know twenty four hour windows or even longer, and so real time to most businesses is the issue if happens within an hour, I'm happy, I'm thrilled, that's much better than I've got today.

Absolutely.

All right, we're gonna talk a little bit about the total cost of ownership of, and remember these are the bidirectional Salesforce integration solutions.

So if you don't see your favorite on here or one you've heard of, it's probably because it doesn't support a full robust bidirectional Salesforce integration. So this chart goes from left to right from total cost of ownership to essentially zero over and above the Salesforce platform to, you know, infinity and beyond for all I know. But on the left side, we can see Salesforce is there because of the Salesforce tools I mentioned a few minutes ago that come for free data loader and data wizard.

And then there's Zapier. Now Zapier is an interesting example and there might be others because Zapier has what are called Zaps which are very purpose built integrations. For instance, you wanna answer a lead into Salesforce from one of the, you know, forty other, systems that Zapier supports, Zapier can do that. So if that's what you need, if it's if you're if you have, I don't know, I'm gonna pick a system out of the top of my head, Constant Contact or something like that, bad example, But you have some kind of a web form that you need to take a lead and push it into Salesforce and there's a zap for it, that web form embedded web form, fine, you're done.

A little bit over a little further in complexity, DataLoader. Io, which I think many of the Salesforce administrators who are watching this presentation have heard of because it's the number one integration app on the Salesforce app exchange.

DataLoader. Was purchased by MuleSoft. It's a completely different solution than MuleSoft. Essentially, it lets you pull data out from Salesforce and put it into a file, take data from file and put it into Salesforce.

How it's different from the native tools is it allows you to schedule those pushes and pulls. But other than that, it is pretty simple. And what distinguishes X Plenty, which is a little farther down the TCO line here, distinguishes X Plenty from those to the left is that X Plenty, data pipelines let you do transformations on the data for one thing. You bring data in from Salesforce, you do some work on it and you push it into one of the many targets X Plenty supports.

The other thing about XPlanet is it does support a number of different targets, including databases, including REST APIs. So as you go farther to the right in TCO, you tend to get a tool that has more power, has the power to transform data and has many more targets. And that's the case for Talend as we go to the right. And then we get to Jitterbit and MuleSoft, which I'll talk about together because as Mark was saying in that in a prior slide, Jitterbate and MuleSoft, and this is gonna sound a little cynical, but remember, I'm a person that bought software and recommended software purchases.

When you're a software vendor at some point you have to justify your existence. So Jitterbit and MuleSoft have decided that integration isn't enough and where they wanna go from integration is to be an API platform.

So what they mean by API platform is pretty much as Mark said, enterprise service bus. Basically the idea is you're a company, you have many different integration points. Some of them might be home built and you wanna expose an API that other of your systems can use to access the services of those platforms. So not only do MuleSoft and Jitterbit allow you to just push data back and forth between platforms and transform it, they also let you build APIs.

Do you need that? Question mark. Is there a best of breed API builder and a best of breed integration solution that you might wanna buy instead of buying one bundle? That's definitely something you should think about.

And let's now and Boomi I think is similar in their strategy, but I don't know that much about as much about it as Titterbit and MuleSoft. Let's take Informatica because they've taken kind of look at Informatica for a minute because they've taken kind of a different approach. It's very interesting. Instead of going down the API path, they've gone down the MDM path, master data management, and they want to help you build an enterprise data dictionary and do enterprise data cleansing. So you have Salesforce and you have homegrown systems and you have SAP or whatever your ERP is. The Informatica promise is that you can take all the data from all those systems and put that into an Informatica data dictionary and manage it and have a master data management scheme where you don't have siloed systems, have one integrated that even though you might have systems that were once in a silo, you have one integrated data dictionary.

The question I'd raised to people considering Informatica as both an integration tool and an MDM tool is to compare the MDM toolkit or the enterprise data dictionary Informatica with purpose built Salesforce tools.

And the one I'll pick out is demand tools, which I think many administrators listening have heard of. Demand tools is a purpose built data cleansing tool for Salesforce. It's incredibly powerful, I'm not selling it obviously, but I've seen it in action in the hands of a good administrator. You can do really some really powerful transforms on your data and do a lot of data cleansing And demand tools is built just for Salesforce and it handles all the oddities of Salesforce like object, object oriented database, like, you know, like the government limits.

So again, the message I'd give as you get to the right of the TCO line is to get that, to get the cost, to justify the cost of those products, they've added a lot of functionality. And the question I would ask if I were in a procurement situation with those products is, are are all the check marks you're gonna get from that vendor? Because they'll give you a list of marks to check. Are all those check marks things, a, that you need, and b, are they best of breed?

Are they good enough for your situation? If the answer is yes, then those are the tools for you. If the answer is no, looking at a point solution like at going to the left on the TCO graph might make more sense. Also, to note one thing that isn't on this TCO chart is a homegrown API because that can be way over to the right or it can be way over to the left.

If you've made it, if you've built one little integration and that's all the integration you'll ever need, that's very cheap. If you've, I'll talk about my experience. I build a I was integrating Salesforce with a data warehouse and Salesforce as an API, and this was in my defense, this was before there were as many great tools.

I just got out and started writing code to the Salesforce API, worked great, but then they want me want to integrate another object, another object, other object, and we have to write code to do each of them. And I have to rewrite my code to make it more of a little bit of a framework so I can plug in new objects more easily. In the end, I'm ending up writing a, you know, half baked X Plenty or talent or whatever and not doing it as well because I don't have the budget or the, you know, I don't have a big team to do it, it's just me. So that's definitely something you should think about, You know, where if you're thinking about writing an API solution, where are you gonna be in three years? Where are you gonna be in five years? Yeah, you need to think about that. Don't just go and sit down at your keyboard and hack out a piece of code and feel happy with yourself because you might have just bought yourself a headache.

Yeah, or even someone in your team a headache as well, that's the other Hopefully, that person ever leaves. Yeah, like we have calls with that where customers build all their pipelines and then they find it hard to maintain it, find it hard to hire great data engineers or Salesforce administrators to actually maintain that code. And so they want something that removes that maintenance burden from them, but still allows them to keep moving forward and developing.

Okay. What's next?

All right, what are some tips for integration projects?

So some of the things I've seen, in fact we're actually doing an integration project right now for our parent company. And so we're trying to integrate with Salesforce, Google Analytics, Intercom, Stripe for monthly recurring revenue across a whole range of companies.

And the way we're doing it is we're actually of course using Axe Plenty, but we're writing to BigQuery, which is our data warehouse. And then we're using Data Studio to do the visualization.

And the steps we take when we do these type of projects, first of all, you know, figuring out the scope is really dig into that and define the scope, define the objectives. And related to project management is really get by it, like you need to have executive buy into these types of projects. It's very hard to be successful if you you know, it's a skunkworks thing, it's done by one individual engineer, you really need to have like the executive owner of the system or systems involved and passionate about the project. Because there will be speed bumps. And with executive buy in, they help get you over those speed bumps, which may be, you know, more budget or more resources or more time or whatever it is that is causing that speed bump.

And then look at the tech as well, when you're doing the scope is really look at what you need the tech to do, like almost like what are the technical requirements? So how fast you need the data to be integrated? What sort of data volume are you going to pass between different systems, because that can make a significant difference to the solution you pick.

And the size of the scope, you know, if you need, you know, huge data volume, that might mean a completely different technical solution to be able to handle that.

And then on the system side, really understand your integration point capability. So what are these systems, you know, what is their API, which API should you use?

What are the requirements for the API? Maybe on the security side, how you can authenticate? Maybe on the security side, what type of data are you allowed to pull out of that system? Maybe there are data compliance issues that you need to adhere to. Maybe you need to anonymize data or mask it or delete it before it even moves out of a system for security reasons. And most of those are internal and you need to sort of talk to your IT teams to understand their data compliance concerns or requirements for you when you're doing your integration work.

And then we move on to data flows. And that's really mapping out the data flows, understanding what fields are going to be needed, what APIs you should call.

And then related to that is the transformations. During journey, what transformations are you going to need for the data?

How are you going to join the different data sets? What's primary key that you'll use to do that? And also what format do you need to have that data in? So you're probably exporting, it could be to Salesforce, which maybe has one format, but it could also be to a data warehouse or to a data lake, you know, what format of date time do they need?

You know, how do they, you know, what ID do you need to pass across or whatever it is, but really understand the format that's needed, because that might require you to do some transformations along the way.

Most of the time it does, because one system is slightly different than the other for how they handle things like that.

And then the other thing I see is incremental development. I remember those death march waterfall projects of the 80s where they were you know everything was like six months and nothing really came in under a year you know everything was under scoped and everyone promised the world and never delivered it and stuff like that. And I'm a big believer in agile, it's just lots of quick wins, small wins. So if you're going do a big integration project, do it almost like one pipeline at a time, just deliver one thing, one integration into Salesforce, get it out to the business, get feedback on it, deliver it, move on to the next one.

Because every time you do it, every time you get one of these wins, you're getting almost like the benefit from the business, know, they see the advantage that you've done, they support the future work, they get excited about it. But if you do it in isolation over a long period of time, it's likely you'll miss your deadlines and your executive sponsors won't be excited about the project anymore, they would have moved on. And you probably won't understand the requirements, because I think often when you do these quick wins, you get some more learnings about what the scope is. Like you thought it was this and the business guy thought it was this, but when you did the integration it's like, I need to have this other thing as well, or I need this other integration.

And you get that feedback immediately. And then you can move on that and actually give them what they need in the end.

So I recommend launching, you know, pipelines and functionality incrementally as you're doing this work.

And then on the project management side, I've done a lot of projects in big companies and small. And the more I do it, the more I realise you really need a strong project champion. You need someone in the business who's passionate about what you're doing, who will be your backer. Really hard to build to do any project if you're doing it just as from a technical point of view without that sponsor. So I'm a big believer in trying to get almost like the highest person you can get that's passionate about what you're doing to sponsor the project and to be the person who's going to be the protector of the project and salesman for the project.

And then another thing I don't like big company stuff. But the thing I do like is actually steering committees, I'm a bit of a believer in steering committees. And the reason is that you have a very quick one, it can be half an hour every week or something like that. But you can get the executive sponsor in there.

So you're constantly on top of mind for them, they see the deliverables, they see what's happening. If you end up with any roadblocks, they will be there, you know, covering your back and helping you and pushing the project along. And so I really like having a steering committee, even in tiny companies, I still have steering committees, just because I think it's important to get stakeholders together and get them bought into the problems and the solutions as they evolve and emerge out of the project. And then I'm a big believer in constant communication about a project.

So ideally having artifacts about the project. So like, you know, what happened this week, what's going to happen next week, what are the risks, the risks that have changed, and then making sure everyone's aware of the decisions that have been made.

So they can't, you know, down the track, people can't say, well, I didn't know about that, or I never agreed to that thing. If you have it all written down, and you have clear constant communication with everyone, all the stakeholders should be all on the same page and understanding exactly what they're getting. And then I'm a believer as well in clear timelines and deliverables as well. Again, because you just want to, at the end of the project, make sure everyone's happy and things will change and things will come up.

And if you have, you know, clear timelines and deliverables, people will be forgiving if things come up and they're in the loop. But no one wants to be surprised, know, they don't want to come into a meeting in two months time and be told, Oh, yeah, the project is actually three months late. And these are the reasons you know, much better to be involved in it, and maybe have some shared accountability and also have some steps that we agree together to get maybe get things back on track.

Yeah, I just want to underline and you're absolutely right, a strong champion, especially for an integration project because those are generally breaking down the walls of silos and there's a lot of corporate politics around who owns the data in those silos. You have to get somebody above silo owners otherwise it's just going to be a fight.

It is, yeah, exactly.

And often in a very large company often there's an owner for each one of these big systems. And so ideally your project champion, these people either report to them or they're somewhere in the hierarchy and they can break those potential conflicts you might have between teams and stuff like that. So the project champion should be able to create the shared vision for the project and get everyone aligned to help and work towards getting integrations done across the organization.

Absolutely.

Great, well, thank you. I mean, is the end of our brief discussion.

But I wanted to thank everyone for attending the X Force Virtual Summit. And hopefully you found all the talks useful, and you've found our discussion useful today.

What I think is really exciting for all of us is that companies are now leveraging data more and more for their competitive advantage. I see that all the time now. It used to be a thing of the, it was only the very big companies like the Facebooks and the Google that were into data and using data. And now it's really becoming a commodity and it's becoming a point where every company has to do it to remain competitive. So it's a great time to be in the industry for all of us. It's a great time to be in Salesforce, great time to be in the integration space.

And what we're seeing is emerging data strategies and data platforms and storage solutions coming out all the time. Things are evolving And our work is really becoming business critical, which is very exciting. So I'm very happy to be in this space and happy to be involved in this X Force conference. And hopefully next year if COVID-nineteen goes away, we can actually have this in person next year and have a real conference and have speakers there and meet and discuss this one on one.

But I really appreciate everyone for joining and look forward to your questions. If you have any, feel free to reach out anytime. And if anyone wants to learn more about our platform, can reach out through our sales team as well.

Yeah, and I just wanted to plug that I've written a few articles on Salesforce for the x planning blog that you can go out and look at that if you have some more questions, and I invite everybody to attend as many sessions as you can at the conference because and especially the X Plenty customers because as Mark said, it is interesting how these customers are not using X Plenty for nice to haves. They're mission critical. Run their business on integrated systems, which is something that ten, twenty years ago would not be the So thank you for watching and invite you to participate in some of the other conference sessions.

Yeah. Thank you.

‍