Kafka consumer and producer properties

When recording a Kafka script in VuGen, all configuration properties defined by Apache Kafka can be used to configure producers and consumers. However, certain configuration properties are mandatory, while others are ignored. In addition, certain properties are shared between consumer and producer configurations. This topic lists the mandatory, shared, and ignored configuration properties for producers and consumers.

In this topic:

Mandatory consumer properties

The following property keys must be included in the consumer configuration file in order to record a script in VuGen.

• bootstrap.servers
• key.deserializer
• value.deserializer

Mandatory producer properties

The following property keys must be included in the producer configuration in order to replay a script.

• key.serializer
• value.serializer

Shared consumer and producer properties

If a property key listed below is included in the consumer configuration file, VuGen automatically copies the property to the producer configuration during recording.

• client.dns.lookup
• connections.max.idle.ms
• interceptor.classes
• metadata.max.age.ms
• metric.reporters
• metrics.num.samples
• metrics.recording.level
• metrics.sample.window.ms
• receive.buffer.bytes
• reconnect.backoff.max.ms
• reconnect.backoff.ms
• request.timeout.ms
• retry.backoff.ms
• security.providers
• send.buffer.bytes
• socket.connection.setup.timeout.max.ms
• socket.connection.setup.timeout.ms

Ignored properties

The following property keys are ignored by VuGen when initializing a producer or consumer, even if the property appears in the producer or consumer configuration.

• group.id (this value is created automatically and randomly)
• client.id

Kafka serializers/deserializers

The generated script format (Builder pattern or XML deserialization) depends on the deserializer configured in your consumer properties. The recording captures what the deserializer returns at runtime, not the original object.

Supported deserializers

The following deserializers produce primitive or well-known Java types, and are fully compatible with the Builder pattern API.

Kafka deserializer	Returns	Generated script
StringDeserializer	String	.withValue("my-value") / .withKey("my-key")
LongDeserializer	Long	.withValue(42L) / .withKey(42L)
IntegerDeserializer	Integer	.withValue(42) / .withKey(42)
DoubleDeserializer	Double	.withValue(3.14D) / .withKey(3.14D)
FloatDeserializer	Float	.withValue(1.5F) / .withKey(1.5F)
ShortDeserializer	Short	.withValue((short) 1) / .withKey((short) 1)
ByteDeserializer	Byte	.withValue((byte) 0) / .withKey((byte) 0)
ByteArrayDeserializer	byte[]	.withValue(java.util.Base64.getDecoder().decode("...")) / .withKey(java.util.Base64.getDecoder().decode("..."))
UUIDDeserializer	UUID	.withValue(java.util.UUID.fromString("...")) / .withKey(java.util.UUID.fromString("..."))

Unsupported deserializers

The Builder pattern is not supported when the deserializer returns a custom Java object at runtime. For the following deserializers, the script is generated using lr.deserialize(...) (the implementation used in VuGen prior to version 26.3).

Kafka deserializer	Returns	Generated script
Avro deserializer	GenericRecord (custom object)	Falls back to lr.deserialize(...)
Protobuf deserializer	Protobuf message (custom object)	Falls back to lr.deserialize(...)
Custom deserializer	Any custom Java class	Falls back to lr.deserialize(...)

Workaround for custom objects:

If your consumer uses a custom deserializer, but you still want Builder pattern output, you can instead configure StringDeserializer in the consumer properties file for the recording session. The recording will capture the string representation returned by the deserializer.

See also

• Record a Kafka script
• Kafka recording options
• Kafka functions