Amazon’s CloudTrail is a service that logs AWS activity. However, these logs need some preparation before they can be analyzed. In this post, we’ll see how to parse these log...

Big Data brother is watching - whenever users surf your website, their browser sends an HTTP header called ‘User Agent’. It tells your web server which browser they’re using, in...

Parsing URL query string parameters is easy with Integrate.io. You can take a huge pile of web server logs and analyze them via Integrate.io’s visual interface. Let me show you...

You’ve spent hours tinkering and preparing the perfect dataflow to batch process zillions of web logs. Feeling satisfied, you run the job on one of the clusters and leave your...

In our previous post , we discussed Mad Men and how to design a data warehouse in the space age of Big Data. This post will take another step forward,...

Integrate.io loves MongoDB. We’ve already guided you how to integrate with MongoLab, and now we’ll show you how to do data integration on the cloud with Integrate.io and MongoHQ.

Papertrail is a cloud-hosted log management service. We use it internally for business insights and eat our own dogfood by processing the logs on Integrate.io, our data integration platform on...