It is not really a surprise this was the dominant topic, as it was predicted to be a major focus leading up to the event. However, when you dig into the details there were some interesting points.
To start, Wikibon released their Worldwide Big Data Market Forecast at the event, which projects big data growth from $18.2B to $92.1B at a 14.5% CAGR. They also published the first-ever forecast on Apache Spark, saying the market will be 16% of the overall big data market by 2022 at $11.5B, signaling this will be one of the major topics in big data for some time. Datanami agrees that the real-time access that Spark provides is leading this big data boom. For the underlying use cases driving this growth, Wikibon cites three—maturing data lakes, self-tuning systems, and intelligent systems of engagement. We also suspect the rise of IoT as a differentiator for consumer goods and industrial equipment, could pull this number even higher.
ESG also highlighted the rise of Spark and machine learning at Strata. They pointed out the shifting markets for data warehousing and cloud computing, explaining that IT leaders are more regularly evaluating a data warehouse replacement by Hadoop, and 35-45% are considering public cloud based big data and data warehouse solutions (not on premise). This is a powerful indicator that Hadoop growth predictions are realistic, as it shows the appetite for on-demand, ubiquitous data is on the rise.
TechTarget also reinforced this prediction that streaming analytics are coming to the big data forefront as their open source roots and real-time capabilities have finally right-sized the economics enough to open up a variety of compelling use cases.
This right use cases was also the single biggest theme for us going into the event, with our own Ian Andrews, VP Products, focused his Strata keynote, “Delivering Information in Context” along with an interview on TheCube.
The point? While we believe BI won’t die, it will live on, in a more integrated, useful way, lingering underneath the covers, and creating more meaningful engagements and dramatically more intelligent systems.
For example, consider Uber—when you order a car, you don’t need monthly reports on the average arrival time. Instead, you need a small little button within the customer experience tells you one really important thing, “The car can be here in 7 minutes.“
That is the power of real-time.
About the Author