Kafka / Redpanda

Kafka is only required if your adapters use the Kafka plugin. If none of your connections involve Kafka, skip this page.

Supported Brokers

Broker	Supported
Apache Kafka 3.0+	Yes
Redpanda	Yes
Confluent Platform 7.0+	Yes

Helm Configuration

Connect the adapter pool to your Kafka cluster under adapter.kafka:

adapter:
  kafka:
    bootstrapServers: "kafka.example.com:9092"

Network Security

For encrypted or authenticated broker connections, configure TLS termination at the load balancer or network layer using your organization's standard approach. Use Kubernetes NetworkPolicies to restrict traffic between adapter pods and the broker.

Topics

Conncentric does not create Kafka topics automatically. Create any topics your adapters will use before enabling them:

kafka-topics.sh --create \
  --bootstrap-server kafka:9092 \
  --topic my.fix.inbound \
  --partitions 3 \
  --replication-factor 2

Offset Tracking

Each adapter instance tracks its own position in a Kafka topic. If a pod is replaced or restarted, it picks up from where it left off. It won't re-process messages that were already handled or skip messages that haven't been processed yet.

Messages are only marked as processed after they have been fully handled and sent to the downstream target. This prevents data loss if a pod crashes mid-processing, but it does mean a message could be delivered twice in an edge case. Design your downstream systems to handle duplicate messages safely where this matters.