Matteo Merli, Apache Pulsar PMC Chair and CTO @ StreamNative

Matteo Merli is the Apache Pulsar PMC Chair and CTO at StreamNative. He was one of the original co-creators of Pulsar while at Yahoo!, co-founder of Streamlio, committer and PMC of Apache Pulsar and Apache BookKeeper.

Exclusive Producer: Using Pulsar to Build Distributed Applications

Thur Jun 17, 11:00 AM - 11:35 AM, PT

There are several patterns that are very useful when designing and building distributed applications, like leader election or “atomic broadcast”. In practice, implementing these concepts is a very difficult undertaking and it is often offloaded to external services. These services in turn need to be maintained and operated and might even expose very complex APIs or semantics.

“Exclusive Producer” is a new addition to the set of Pulsar’s features, and it allows an application to have the guarantee of exclusive access when writing to a topic and to discard any data from producers who lost the exclusive access.

During this session we are going to show how easy it is, using Exclusive Producer, to build very robust mechanisms of communications across different services or instances of a single service.

These mechanisms, like for example leader election, failover and “total order and atomic broadcast”, are the main building blocks to build consistent, correct and reliable distributed systems.

We will also present how the exclusive producer is implemented internally, and how it can guarantee its properties in the presence of all different sorts of failures.