Drake is a new piece of software I discovered that can be very helpful for automating data manipulation and management. The use case for Drake is simple, one of the biggest, and most time consuming, tasks in any data project is cleaning the data so it can be transformed into a format for analysis.
Data often comes in crazy, non-standard, formats which need to be regularized in order to be imported into an analysis program like R or ArcGIS.
When I first read the news reports about internet addiction in the early and mid-2000s I dismissed the idea. How, I thought, could anyone be addicted to the internet?
Again in the mid-2000s I started reading more about information overload and personal productivity. I read about Inbox Zero and the transformative power of Getting Things Done. When confronted with a problem created by technology, namely too much information, many people in the technorati went looking for a technical solution.
So I’ve been thinking about a way to describe the enthusiasm about big data that has captured the business, government, and academic worlds over the past decade. I will admit up front that the rapid advances in deep learning technology that large businesses like Google, Microsoft, Facebook, and Amazon have accomplished in the past few years should be celebrated. The efforts by these companies and the fact that they have open-sourced many of the artificial intelligence frameworks they are creating should be applauded.