After catching up on some much-needed sleep, I settled into work Monday and found that several of my colleagues have been busy recapping their IOD experiences. Delaney Turner has a handy list of additional resources, including a reading list from Jason Silva.
But here’s what’s still rattling around in my head:
Making – and Trusting – Predictions
Nate Silver’s Wednesday keynote presentation was fascinating, as was the interview he did with Todd Watson and Scott Laningham (watch it here). Silver spoke about why and how we make predictions, and when we should trust such predictions. He stressed, “Collectively, we’re smarter than anyone is individually” – something we should remember as we all return to our respective jobs and challenges. As Silver explained, when solving problems, we should look at multiple theories and models rather than relying on a single indicator. I picked up his book while at IOD and can’t wait to dig in.
Analytics Are for Everyone
Saturday morning, I rode a train into Chicago. While rolling through a freight yard, I remembered the intriguing Information Management keynote on Monday. Kent Collins, Senior Database Architect with BNSF Railway (go to the 54:00 mark to hear Collins). BNSF has 220,000 active cars on the railroad across 32,000 miles in 28 states and two Canadian provinces. In 2010, BNSF hauled enough grain to make a year’s supply of bread for 900 million people and enough asphalt to lay a single lane road around the equator – four times.
While many may think of railroads as an old-school industry, BNSF is finding tremendous value in using predictive analytics to predict failures and
“We’ve got data everywhere,” Collins explained, noting they have over 1700 servers to track maintenance, weather, scheduling, inventory, safety, deliveries and more. These are mission critical to BNSF – as Collins noted, “The systems we put on PureScale can never go down… In testing, we did everything but pour water into the thing. We tried to kill it, and it wouldn’t die.”
Freeing up resources to really dig into analytics and their implications has proved hugely valuable to BNSF. Collins continued, “If I can do it in my sleep, I want somebody or something else to do it. I don’t want to do the mundane. I want to do is help my business customers meet their objectives. I want to unlock the data and open up the environment.”
But the thing that really struck me was Collins’ ice cream analogy for PureScale. Before enjoying the benefits (sweetness or analytics), you must first buy all the components – the ice cream, bowls, scoops, etc. Then, you can either use some muscle or wait for it to soften a bit before scooping. But PureScale is more like an ice cream shop experience: everything you need is in one place. You can focus on customizing your cone exactly how you want it, and it’s ready to go.
Big Data Needs Context
This year’s IOD theme was “Think BIG,” and many of the sessions at least mentioned big data. But Jeff Jonas’ Wednesday morning talk reminded us that, “Just because you have a big pile of data doesn’t mean there’s gold in the hills” – and that the key is to understand the context, as well as what’s missing. As Jonas explained, “No one writes ‘bomb’ on the manifest!” You have to widen your observation space to make quality predictions. And that means understanding your data’s context.
“Context accumulation is the future of big data,” Jonas stressed before revealing his new skunk works project, G2. Leveraging InfoSphere Streams and SPSS Modeler, G2 is a general purpose context accumulating engine. Basically, G2 helps discern whether entities are unique or the same. Watch Jonas’ demo and listen to his examples, starting at the 32-minute mark.
Jonas shared a couple of examples of G2 in action. He’s used G2 to do genealogy research with the 1880 Census, which has “very weak” data. He also worked with Singapore’s government to predict risk to the Malacca Strait, through which half the world’s oil supply passes.
Watch some of these snippets and tell me what resonates. Or, now that you’re back from IOD, which stories are you sharing with colleagues?