This week, EMC is announcing a massive cross-company solution that is in many ways a one-stop-shop, best-of-breed, silver bullet CIOs have been looking for to help them succeed. Called the Federation Business Data Lake, this solution combines the hardware, software and all the connected engineering to give companies the foundation to fulfill their data and development needs.
This comes at a time when businesses are feeling increasing pressure to incorporate real-time analytics and insight into business operations. The uses pervade every industry from precision farming to airplane maintenance to dramatically improving scientific medical research. The results are boosting trillions of dollars in GDP a year.
Yet, despite these early successes whetting the global appetite to handle big data better, the process is still hard for many companies. The Apache Hadoop® ecosystem has blossomed and matured, but a true big data solution takes much more than one piece of software. It takes hardware, software and lots of engineering to build a solution that can be self-service and powerful enough to spur innovation.
Pieces of this solution have been available á la carte from various vendors for some time now, proving each to be successful in their own rights. Some of the pieces have even been engineered to work together already. For instance, Pivotal HD has been engineered to work on Isilon for about a year now. However, constructing a fully functional data lake, the leading architecture for companies serious about big data, takes more than a few pieces from a few companies to construct.
Today, with the announcement of the Federation Business Data Lake, the first end-to-end data lake solution will be for any company to buy. It sports the tailored hardware optimized for Apache Hadoop® and big data, a variety of analytics tools to slice and dice the data however you like, the ability to ingest huge streams of data, the controls to govern it, and, to top it off, comes equipped with the platform to help operationalize that data rapidly into applications, even baking in a variety of ready-made algorithm libraries.
And all of it has been vetted by EMC engineering to make sure all the pieces work together smoothly, and backed with a single call to support to EMC.
- The Data Manager manages and provisions storage from various sources and platforms to fit the application need. It is a purpose-built, software-defined infrastructure as a service (IaaS) for abstraction, aggregation, and pooling. When new applications are deployed, storage is made available to the analytics application so that it can be analyzed.
- The Analytics Manager is an agile platform as a service (PaaS) environment to test, develop, deploy, operate, scale, and manage applications for ingestion, storage, protection, management, analysis, and collaboration of data responsible for the provisioning and deployment of new applications to meet demands of the users.
- The Data and Analytics Catalog allows IT Admins to quickly review and deploy available applications and quickly deploy them. The Catalog serves to manage applications in the BDL and provision data sets so that analysis can be performed.
Additional snap-on components are included in the announcement and planned for 2015, but are subject to change. Consult the Big Data Solutions team with any questions related to roadmap.
- EMC Press Release: New Federation Business Data Lake Solution Paves Way For Big Data to Disrupt Every Industry Around the Globe
- EMC Blog: Big Data Game Changer: Federation Business Data Lake
- Product Page: Pivotal Big Data Suite
- Product Page: Pivotal Cloud Foundry
- How companies struggling to change to be a technology company succeed: Silicon Valley’s Secret Weapon: Pivotal Labs
- Pivotal Big Data Suite release blog: Pivotal Big Data Suite: Open. Agile. Cloud-Ready.
- Understanding more about the cross-industry initiative to accelerate open source big data tool development:
- Launch Event Replay: Federation Big Data Launch
Editor’s Note: Apache, Apache Hadoop, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.
About the Author