New Integrate.io features: processing small files, importing Heroku connections, MongoDB secondary replica sets and more.

Some developers need to process data. Maybe you work in a small startup where people take on several roles, or maybe in an enterprise company where you are asked to...

We’ve processed plenty of JSON data in our blog - from Tweets to GitHub commits - but we’ve never really discussed how to process JSON with Integrate.io’s data integration on...

Although the Internet made the world flat, geography still matters. Knowing which countries your users live in could provide business opportunities to localize your services and increase profits. The only...

Amazon’s CloudTrail is a service that logs AWS activity. However, these logs need some preparation before they can be analyzed. In this post, we’ll see how to parse these log...

Big Data brother is watching - whenever users surf your website, their browser sends an HTTP header called ‘User Agent’. It tells your web server which browser they’re using, in...

Parsing URL query string parameters is easy with Integrate.io. You can take a huge pile of web server logs and analyze them via Integrate.io’s visual interface. Let me show you...

You’ve spent hours tinkering and preparing the perfect dataflow to batch process zillions of web logs. Feeling satisfied, you run the job on one of the clusters and leave your...

In our previous post , we discussed Mad Men and how to design a data warehouse in the space age of Big Data. This post will take another step forward,...