Название: Streaming Databases: Unifying Batch and Stream Processing (Final Release) Автор: Hubert Dulay, Ralph M. Debusmann Издательство: O’Reilly Media, Inc. Год: 2024 Страниц: 260 Язык: английский Формат: True PDF, True EPUB (Retail Copy) Размер: 15.4 MB
Real-time applications are becoming the norm today. But building a model that works properly requires real-time data from the source, in-flight stream processing, and low latency serving of its analytics. With this practical book, data engineers, data architects, and data analysts will learn how to use streaming databases to build real-time solutions.
Authors Hubert Dulay and Ralph M. Debusmann take you through streaming database fundamentals, including how these databases reduce infrastructure for real-time solutions. You'll learn the difference between streaming databases, stream processing, and real-time online analytical processing (OLAP) databases. And you'll discover when to use push queries versus pull queries, and how to serve synchronous and asynchronous data emanating from streaming databases.
So what is a streaming database? Database systems have many different flavors, from traditional relational databases to XML, graph, object, vector, and NoSQL databases. Many of these are well known and have been established for many decades. Streaming, or stream processing, is much less established, although it has seen a steep adoption rate in the industry over the past decade or so, led by the rise of Apache Kafka as the de facto streaming platform.
Historically, stream processing was considered difficult, and only larger organizations with dedicated teams of streaming experts could master it. The same was true for data processing and computing 50 years ago, before SQL and relational database systems were invented to allow nontechnical users to work with data stored in computer systems. Now, SQL is the lingua franca of data processing.
Streaming databases are the next step in the evolution of stream processing. They unify well-established techniques from database systems with the new paradigms from the streaming world to simplify stream processing and enable nontechnical users to work with data in motion, similar to what we are used to when we query data at rest.
Data streaming, as the second child of the “Big Data” era, followed the same trend: first, stream processing systems were built by experts for experts without the support of SQL. It wasn’t long until SQL and database technologies were introduced to enable nontechnical users to use these new streaming systems. This development led to streaming databases and the waves of innovation that followed.
This guide helps you:
Explore stream processing and streaming databases Learn how to build a real-time solution with a streaming database Understand how to construct materialized views from any number of streams Learn how to serve synchronous and asynchronous data Get started building low-complexity streaming solutions with minimal setup
Whether you’re a seasoned database engineer or a novice developer, this book guides you to unlocking the full potential of streaming databases and embracing the future of data processing.
Preface 1. Streaming Foundations 2. Stream Processing Platforms 3. Serving Real-Time Data 4. Materialized Views 5. Introduction to Streaming Databases 6. Consistency 7. Emergence of Other Hybrid Data Systems 8. Zero-ETL or Near-Zero-ETL 9. The Streaming Plane 10. Deployment Models 11. Future State of Real-Time Data Index