Kafka Streams topologies you can draw and run
You have an orders topic. Each record carries a customer, an item, a quantity, and a price. You want the big ones — total over 50 — in a topic of their own, with that total already calculated. In Kafka Streams, that is a mapValues and a filter: a dozen lines of real logic. Getting them to run is a different size of job: a build file, a serde configuration, a jar, somewhere to put it, and a redeploy every time you want to check if your expression was right about the data. The work is small. The apparatus around it is not — and none of it tells you anything until the whole thing is up. That distance is the subject here. Not that Kafka Streams is hard, because it is not — but that the trip from knowing what you want to watching it happen is longer than a dozen lines of logic deserves. Start with what Kafka Streams already gives you — most of the picture is there, and the missing part is smaller than it looks. Deploy a topology and ask it to describe itself; here is a real one: four operators, reading orders and writing orders-enriched: Topologies: Sub-topology: 0 Source: Orders-src (topics: [orders]) --> order-total-enrich Processor: order-total-enrich (stores: []) --> big-orders-bigOnly Big-Orders-sink
This is a summary aggregated from Dev.to. Read the complete article on the original site:
Read full article at Dev.to