Big data lectures

Summary of the big data explosion from the 2011 Strata Online conference sponsored by O’Reilly.

Three interlocking changes

1. exponential economics. The dramatic decrease in storage costs, and the increase in network connections/access points.
2. sensor networks. The ubiquity of data collection. “Instrumented spimes” - term from Bruce Sterling for devices that are streaming information into the cloud/ether.
3. cloud computing. Computational and data resources on demand.

Success on the data stack

1. Persistence - data. Recipe for success = Fast
2. Analytics - intelligence. Big
3. Services - what do you do with it. Focused

Distinguish between exploratory and hypothesis-driven science.

[youtube http://www.youtube.com/watch?v=yOqzJu-umVI]