Digital transformation is a broad term that has various meanings by application, but in general, it means that more and more of what organizations, people, governments do is happening in computers, mobile devices and networks. As a result, the way things are done is changing, especially in the way things are connected. So in this new world of data flying everywhere, being generated and consumed, where does one stop for a second to take a look at what’s going on?
In the old days, maybe 5-10 years ago, the obvious answer was analytical tools, such as BI or Excel, pulling data from a data warehouse. But something happened.
Instaology ≠ Real-Time
I like to call it a new theory of instatology. Everything changes, from why your kids don’t call you, they text you and put pictures on Instagram. Self-driving cars, your GPS system with turn-by-turn instructions knowing exactly where you are, annoying devices in grocery stores that talk to you as you pass based on (hopefully) a profile of what you may be interested in. Or a refrigerator that sends a message to your wife, “Neil is eating ice cream again.” Contrast this with an analyst sitting at a desktop developing a query for a report or a dashboard using information that hasn’t been updated since last night.
Now in the latter case, that really isn’t quite so bad as there are still a multitude of applications that are served by what we call classic Business Intelligence. Classic Business Intelligence isn’t going away, though it is becoming more intelligent. But in the other cases, there are a series of devices and flows that are needed that cannot operate with that sort of latency. Streaming out data is not BI, but when those flows ARE captured by business logic and help to deliver not only data but advice or even decisions, that is BIG BI.
The Two Forms of Big BI
Actually Big BI comes in two forms. The first, as mentioned above, is when BI kinds of logic are embedded in real-time or even near real-time systems. But Big BI can also mean the more traditional peripatetic analysis of interactive examination against the new and massive sources of data that were not part of traditional BI. In a way, you can consider this second type of Big BI of a Red Queen exercise, a concept discussed in the previous blog. It uses existing methods and tools for analysts to do their work, but expands the universe of data available to them. Sort of like keeping up with the Red Queen. It uses data as an asset to be exploited, keeping up with the trends to use the kind of data we see today in “big data.” But it doesn’t, on its own transform anything.
This first form, however, can be as wild as you can imagine. It’s cliché to repeat how massive computing and networking capabilities are today, the question is, how can you exploit decades of learning in tools like OLAP and BI to facilitate the digital transformation? In every area of digitally transforming the organization, such as customer experience, products and services and operations, 'analytics' are a key part of every operation. People with experience in advanced BI applications like market basket analysis, dynamic pricing, or loss prevention - have the skills to embed analytics in BIG BI. BI vendors are smart enough now to provide embeddable tools, exposing their API’s in RESTful interfaces compatible with cloud computing and open source tools. Many organizations overlook this now and engage in expensive Java or Python programming to do what BIG BI could do more easily. And no digitally transformed process can exist without analytics about its own performance.
This is where BI, and especially OLAP analysis is needed to keep an eye on this unattended analytics process and to continually improve them with methods like A/B testing (two versions of a website, which performs better) or even the Champion Challenger method which describes how optimal results are obtained by moving to a best solution through a series of challenges to the current plan. These kinds of analytics are perfect BIG BI.
Digital Transformation Through BI: Not As Hard As You Might Think
How difficult is it to move to Digital Transformation? For those companies, the so-called digital giants, not at all, as their businesses are based on digital technology to begin with. For everyone else, it depends on how far you want to go and how quickly. But to extend the data available to your BI analysts, it’s become rather easy. Access the sources, and apply some handy middleware that can create an abstraction of all the disparate kinds into one semantic model that any or all of your BI tools can consume, and don’t smother them at first with governance and rules, unless you think your employees are up to no good. They usually aren’t.
In the next post, we’ll segue back to take a very close look at OLAP, and how it’s drawbacks in decades past have been largely solved by technology. In addition, with examples, we’ll describe why OLAP “thinking” is still a primary way for people to look at their business.
Until next time... ~ Neil
p.s. And if you haven't already, be sure to catch the first two posts in the series, 'Big BI: Data isn't a Process, It's an Asset' and 'Big BI: Running with the Red Queen: BI is Here to Stay'.
About the Author:Neil Raden is an author, consultant, industry analyst and founder of Hired Brains Research, based in Santa Fe, NM. He has a passion for analytics based on decades of experience and strives to express it through his work in writing, speaking and advising clients. Neil began his career as an actuary with AIG. In 1985, he started Archer Decision Sciences, consulting on analytics projects for Fortune 500 companies. Archer was one of the first to develop large-scale data warehouses and BI environments. In 2003, Neil expanded into a role as an industry analyst, publishing over 50 white papers, hundreds of articles, blogs, keynote addresses and research reports. He is also co-author of the book, Smart (Enough) Systems, about decision automation systems driven by predictive analytics. Please feel free to contact Neil directly at firstname.lastname@example.org