In years past, if you asked C-level execs whether they used open source software (OSS) at their company, they would likely have said no. Meanwhile, any developers within earshot would roll their eyes or chuckle, knowing full well that OSS was used to support any number of critical processes.
Today, however, most C-level execs have come to understand the value of OSS, and many enterprises have even implemented “open-source first” policies for new RFPs. According to the 2015 Future of Open Source Survey, nearly 80% of respondents said their company runs “part or all of its operations on OSS and 66% said their company creates software for customers built on open source.” That’s up from just 42% of respondents who reported OSS use in their organizations in 2010.
Note: It is important, of course, to be smart about when and how to apply OSS, particularly as enterprises embark on extensive digital transformation initiatives. In an upcoming webinar, my colleagues Roman Shaposhnik and Dormain Drewitz join Redmonk’s Stephen O’Grady to share tips and best practices for developing Your Open Source Playbook For 2016.
The use of OSS has extended to virtually all areas of IT, from middleware and data integration software to application development tools. In more recent years, we’ve witnessed the rise of open source approaches with Big Data. In fact, without OSS, there would be no Big Data as we know it today. Apache Hadoop, the most well-known open source Big Data framework, arose in part because engineers and developers at Yahoo! and elsewhere recognized it was both technologically and economically impossible to support web-scale distributed applications with expensive, proprietary relational database software and appliances. Instead, these pioneers of Big Data started developing homegrown solutions to their data-intensive challenges and donated the resulting code to the world. And with that, Big Data as we know it was born.
As mentioned, today OSS is in use at the overwhelming majority of companies, and increasingly this includes open source Big Data software. Gartner’s most recent Hadoop adoption survey says that just over a quarter of companies, 26%, have deployed or are experimenting with Hadoop, while another 18% plan to invest in Hadoop-related software and services over the next 12 to 24 months. That may not seem like significant market penetration compared to traditional database technology, which is of course ubiquitous, but considering Hadoop is just a decade or so into its existence, it’s indicative of a maturing technology on its way to mainstream adoption. And when you consider other open source Big Data approaches such as Spark, Cassandra, and Greenplum, open source Big Data adoption is likely higher still.
Why is Big Data OSS winning in the enterprise? There’s simply no way to compete with the likes of Uber, Airbnb, Netflix, Google, Amazon, or other companies that have mastered the art of operationalizing Big Data with outdated, black box technology and closed-source approaches.
Here are three key reasons why OSS is so critical to Big Data success.
- Big Data OSS is simply superior to proprietary software. Would you rather have hundreds or thousands of the smartest minds in the industry working everyday to improve the technology that supports your critical business processes or a handful of engineers at a private company? Of course it’s the former. As a result of active community support, Big Data OSS enjoys significantly faster development cycles than proprietary software, leading to more feature rich, secure and stable software. If you base your Big Data stack on proprietary software, you’re losing before you even start.
- The best minds in Big Data are committed to OSS. Big Data success and digital transformation are as much about people as they are about technology. Proficiency in Big Data—both deep analytics and operationalizing insights – requires a diverse team of individuals that includes experienced data engineers, data scientists, business analysts and application developers. With a well-known shortage of these types of professionals in the market, enterprises that want to attract the top talent must provide their tools of choice. And that means OSS. Among data scientists, to take just one example, R and Python, both OSS, are the most popular languages for analytics. No proprietary language even comes close.
- Agility and transparency are Big Data “must-haves”. The term “vendor lock-in” has never been a popular one with CIOs, but when it comes to Big Data and digital transformation, it’s the equivalent of a four-letter word. As Big Data becomes ever more central to your enterprise’s competitive advantage, you need transparent access to the underlying component parts in order to adapt to new market conditions. The last thing you want is one of the main sources of your competitive differentiation—Big Data-driven insights—being held for ransom by a proprietary vendor. Big Data OSS provides the level of transparency and agility you’ll need to compete.
Of course, Big Data doesn’t live in isolation. Big Data technologies need an extensible platform on which to run, while insights gleaned from Big Data must be operationalized via intelligent applications to actually impact the business. The next step, then, is developing a plan of action to implement Big Data as part of your larger digital transformation initiative with open source software at its heart.
Next week, Pivotal’s Roman Shaposhnik and Dormain Drewitz, along with Redmonk Co-Founder & Principal Analyst Stephen O’Grady, as they discuss Your Open Source Playbook for 2016. In this webinar, which takes place 1/13 at 9am PST/Noon EST, Roman, Dormain and Stephen will expand on why OSS is so critical to digital transformation, provide best practices to follow to avoid pitfalls, and suggest where you should start to make changes and gain quick wins.
Register here for the upcoming event.