1. Introduction
In this tutorial, we’ll review the Spring Kafka trusted packages feature. We’ll see the motivation behind it, along with its usage. All with practical examples, as always.
2. Prerequisite
In general, the Spring Kafka module allows us, as users, to specify some metadata about the POJO we’re sending. It usually takes the form of the Kafka message headers. For instance, if we’ll configure the ProducerFactory in this way:
@Bean
public ProducerFactory<Object, SomeData> producerFactory() {
JsonSerializer<SomeData> jsonSerializer = new JsonSerializer<>();
jsonSerializer.setAddTypeInfo(true);
return new DefaultKafkaProducerFactory<>(
producerFactoryConfig(),
new StringOrBytesSerializer(),
jsonSerializer
);
}
@Data
@AllArgsConstructor
static class SomeData {
private String id;
private String type;
private String status;
private Instant timestamp;
}
Then we’ll produce a new message into a topic, for example, using KafkaTemplate configured with producerFactory above:
public void sendDataIntoKafka() {
SomeData someData = new SomeData("1", "active", "sent", Instant.now());
kafkaTemplate.send(new ProducerRecord<>("sourceTopic", null, someData));
}
Then, in this case, we’ll get the following message in the console of the Kafka consumer:
CreateTime:1701021806470 __TypeId__:com.baeldung.example.SomeData null {"id":"1","type":"active","status":"sent","timestamp":1701021806.153965150}
As we can see, the type information of the POJO that is inside the message is in the headers. This is, of course, the Spring Kafka feature recognized by Spring only. Meaning, these headers are just metadata from Kafka or other framework’s points of view. Therefore, we can assume here that both the consumer and the producer use Spring to handle Kafka messaging.
3. Trusted Packages Feature
Having said that, we may say that, in some cases, this is quite a useful feature. When messages in the topic have different payload schema, then hinting at the payload type for the consumer will be great.
However, in general, we know what messages in terms of their schemas can occur in the topic. So, this might be a great idea, to restrict the possible payload schemas consumer will accept. This is what the Spring Kafka trusted packages feature is about.
4. Usages Samples
Trusted packages Spring Kafka feature is configured on the deserializer level. If trusted packages are configured, then Spring will make a lookup into the type headers of the incoming message. Then, it will check that all of the provided types in the message are trusted – both key and value.
It essentially means that Java classes of key and value, specified in the corresponding headers, must reside inside trusted packages. If everything is ok, then Spring passes the message into further deserialization. If the headers are not present, then Spring will just deserialize the object and won’t check the trusted packages:
@Bean
public ConsumerFactory<String, SomeData> someDataConsumerFactory() {
JsonDeserializer<SomeData> payloadJsonDeserializer = new JsonDeserializer<>();
payloadJsonDeserializer.addTrustedPackages("com.baeldung.example");
return new DefaultKafkaConsumerFactory<>(
consumerConfigs(),
new StringDeserializer(),
payloadJsonDeserializer
);
}
It also may be worth mentioning, that Spring can trust all packages if we substitute the concrete packages with star (*):
JsonDeserializer<SomeData> payloadJsonDeserializer = new JsonDeserializer<>();
payloadJsonDeserializer.trustedPackages("*");
However, in such cases, the usage of trusted packages does not do anything and just incurs additional overhead. Let’s now jump into the motivation behind the feature we just saw.
5.1. First Motivation: Consistency
This feature is great because of two major reasons. First, we can fail fast if something goes wrong in the cluster. Imagine that a particular producer will accidentally publish messages in a topic that he is not supposed to publish. It can cause a lot of problems, especially if we succeed at deserializing the incoming message. In this case, the whole system behavior can be undefined.
So if the producer publishes messages with type information included and the consumer knows what types it trusts, then this all can be avoided. This, of course, assumes that the producer’s message type is different from the one that the consumer expects. But this assumption is pretty fair since this producer shouldn’t publish messages into this topic at all.
5.2. Second Motivation: Security
But what is most important is the security concern. In our previous example, we emphasized that the producer has published messages into the topic unintentionally. But that could be an intentional attack as well. The malicious producer might intentionally publish a message into a particular topic in order to exploit the deserialization vulnerabilities. So by preventing the deserialization of unwanted messages, Spring provides additional security measures to reduce security risks.
What is really important to understand here is that the trusted packages feature is not a solution for the “headers spoofing” attack. In this case, the attacker manipulates the headers of a message to deceive the recipient into believing that the message is legitimate and originated from a trusted source. So by providing the correct type headers, the attacker may deceive Spring, and the latter will proceed with message deserialization. But this problem is quite complex and is not a topic of the discussion. In general, Spring merely provides an additional security measure to minimize the risk of the hacker’s success.
6. Conclusion
In this article, we explored the Spring Kafka trusted packages feature. This feature provides additional consistency and security to our distributed messaging system. Still, it is critical to keep in mind, that trusted packages are still vulnerable to header spoofing attacks. Still, Spring Kafka does a great job at providing additional security measures.
As always, the source code for this article is available over on GitHub.
