Member-only story
Event-Driven Dumpsterfire
I started my career on a backend team building services that communicated through queues. While some had SOAP endpoints (if anyone remembers those), and eventually REST and gRPC endpoints, the bulk of my time was spent on all the asynchronous message passing.
I loved it. Async messaging comes with many benefits: your servers don’t need to be highly-available, you can retry messages, and you can easily protect your dependencies by putting backpressure on your message queue or bus. Then of course—my personal favorite—you usually don’t need to wake up at 3am if something breaks because the messages will still be there to process in the morning!
We were using queue technologies like RabbitMQ, and then eventually SQS, to send tasks from one system to another. Since they were intended for one consumer and were telling the consumer what to do, you could think of them like a distributed implementation of the command pattern.
Moving on to my next job, I got the chance to expand my async messaging repertoire. We were using Kafka and SNS to send messages, but with a small twist. Instead of queueing them up for one receiver, we were creating multiple streams to multiple receivers. And along with that, the receivers no longer defined the message shape and schema; instead, the publishers dictated the events, and subscribers chose which ones to listen to. I was…
