Dark data … dark matter …dark dimensions… the dark side …
what?
I noted in my last post that Bill McDermott’s opening keynote at SapphireNOW on May 14’th made me start to think about a couple of things. One of them was his mention of Dark Data during the keynote and how together it and SAP can help revolutionize business and open the door to the new possible. Dark Data is like that pile of stuff you collect in the garage, usually unsorted and unorganized, and don’t throw out – just in case you find a use. Back in 2008 Byran Heidorn wrote an article about Dark Data in terms of library curation noting how the bulk of scientific data including that from failed experiments was not carefully indexed or stored thus virtually becoming invisible, likely to remain underutilized if not lost. (https://www.ideals.illinois.edu/handle/2142/9127)
Similarly in business Dark Data, is all the data that organizations amass that is not directly part of their day to day operations, but that has been hoarded
just in case. A few months ago Elizabeth Gaines of SAP wrote a good blog – “What is Dark Data?” (http://blogs.sap.com/innovation/big-data/what-is-dark-data-020004) and last month Isaac Sacolick penned a nice blog entitled “Dark Data - A Business Definition” (http://blogs.starcio.com/2013/04/dark-data-business-definition.html) – though I am not sure I fully agree with his last sentence. Like dark matter in physics, dark data cannot be seen directly by the organizations analytics tools and processes, yet it is the bulk of their organizational data universe.
During Dr. Hasso Plattner and Dr. Vishal Sikka’s SapphireNOW keynote May16’th it was pointed out that many opportunities for advancement and associated new hurdles that need to be tackled were and are not evident or considered until advances in technology expose them and make the previously impossible the new possible. Why consider evaluating postponement - moving production closer to the end point -unless the item can be produced further down the chain. In the demonstration around this during the keynote, 3D printing has enabled cost effective production at the end point and SAP HANA
has allowed rapid flexible analysis of data. Now that a product’s production can easily be moved, asking where best to execute the production is no longer a
moot point and something that SAP, with its advances, is able to answer. As a result patients no longer have to wait – a significant impact on how we live –
an impossible now the new possible.
Another example presented in the keynote was the way SAP HANA with its real-time ability to analyze mass data is able to provide eBay with an intelligent analytical platform to automatically separate signals from noise in an actionable time frame. Sorting through 50 PB of data looking for a signal that is different than the trends based on 10’s of 1000’s of variables is time sensitive and has to be done within a day, not a month, if it is to be of value to and make eBay a
differentiated data driven business. The speed of SAP HANA has opened a world of potentials and is helping make the great amazing.
As I noted in my last post it is hard to imagine what you have never imagined. But technology has taken leaps forward and SAP with its SAP Hana platform and cloud options have opened new horizons. Rethink not only how you think but also what data you are evaluating. Like the dark matter of physics, Dark Data may hold the keys to many mysteries. How many amazing scientific ah-ha moments came about when someone stumbled across long forgotten
experimental data and connected it to a modern problem. SAP with its advances has made the use of your Dark Matter a reality. Throw off the shackles and just say what if … it may well be the new possible.
Missed Dr. Hasso Plattner and Dr. Vishal Sikka’s SapphireNOW keynote – watch the replay here: http://www.sap.com/events/
