Kafka message key is null using Confluent.Kafka consumer library

I am using Debezium to perform CDC against my DB to create messages in Kafka. Using tools like kafdrop and OffsetExplorer I am able to see the message key and value. However, from a .NET framework application, using the Confluent.Kafka libraries, when I consume the messages, the message key is always null.

How can I retrieve the message key using the Confluent.Kafka library?

Here is the code for the VS project:

using System;
using Confluent.Kafka;
using System.Threading;

namespace KafkaConsumer
{
    class Program
    {
        static void Main(string[] args)
        {
            var config = new ConsumerConfig
            {
                BootstrapServers = "kakfa:9092,localhost:9093",
                GroupId = "foo",
                AutoOffsetReset = AutoOffsetReset.Earliest,
                
            };

            var topics = "CAC_connector.dbo.sessionLogs";
            bool cancelled = false;

            // Define the cancellation token.
            CancellationTokenSource source = new CancellationTokenSource();
            CancellationToken cancellationToken = source.Token;

            using (var consumer = new ConsumerBuilder<Ignore, string>(config).Build())
            {
                consumer.Subscribe(topics);

                while (!cancelled)
                {
                    var cr = consumer.Consume(cancellationToken);
                    Console.WriteLine($"Consumed record with key {cr.Message.Key} and value {cr.Message.Value.Substring(0, 96)}");
                }

                consumer.Close();
            }
        }
    }
}

Cheers, Kyley



Read more here: https://stackoverflow.com/questions/68472577/kafka-message-key-is-null-using-confluent-kafka-consumer-library

Content Attribution

This content was originally published by Kyley Jex at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: