Cloud Native Streaming and Event-Driven Microservices

SpringOne Platform 2016 Speaker: Marius Bogoevici; Spring Cloud Stream Lead, Pivotal The future of scalable data processing is event-driven microservices! They provide a powerful paradigm that solves issues typically associated with distributed applications, such as availability, data consistency, or communication complexity, and allows the creation of sophisticated and extensible data processing pipelines, bridging the gap between the big data and more traditional enterprise integration world. Building on the ease of development and deployment provided by Spring Boot, the cloud native capabilities of Spring Cloud, and the messaging model and integration patterns of Spring Integration, the Spring Cloud Stream project provides a simple and powerful framework for event-driven microservices. It defines primitives and abstractions specifically addressing the needs of both event-driven integration, as well as data streaming. For addressing the complexity of deployment, at a higher level of abstraction, Spring Cloud Data Flow is an integrated orchestration layer that provides a highly productive experience for deploying and managing sophisticated data pipelines consisting of standalone microservices. A pluggable runtime SPI allows Spring Cloud Data Flow to coordinate these applications across a variety of distributed runtime platforms such as Apache YARN, Cloud Foundry, Kubernetes, or Apache Mesos. We will demonstrate how to easily create complex data processing pipelines that bridge the world of big data and enterprise integration together, by building Spring Cloud Stream applications from the ground up, and, in the end, how to orchestrate them with Spring Cloud Data Flow.