Hazelcast Jet Kafka with not serializable event handler

I want to use hazelcast-jet-kafka in my app, because in my case the number of kafka partitions is limited. How I understand jet-kafka parallelism doesn't depend on kafka partitions, it would be nice to find explanations of how jet-kafka achieve independence of the number of kafka partitions.

But my question is how I can handle events in jet when my event handler could not be serializable. For example, I've found a solution - use map sink and add local event listener to this map, but for me, it seems like a crutch, because I don't need to store these events in map. It is possible to set map size to zero in such scheme?

Also, I see in docs new type of sink - observable, it seems what I want, but observable listener could not get only local entries and for me, it is not suitable.

Could you help find the right solution? Or hazelcast-jet-kafka is not a good choice in that case?

Read more here: https://stackoverflow.com/questions/64952553/hazelcast-jet-kafka-with-not-serializable-event-handler

Content Attribution

This content was originally published by Aliksz at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: