Check the example here: https://github.com/kgoralski/go-kotlin-java-kafka

Java - demo-java-source with Spring Cloud Stream - localhost:8080
Kotlin - demo-kotlin-sink with Spring Cloud Stream - localhost:8081 (Consumer)
Go aka Golang - demo-go-sink with Shopify/sarama (Kafka Client) - localhost:8082 (Consumer)
Apache Kafka as Message Broker
Go Shopify/sarama - Apache Kafka Client
Go HTTP/2 Push example (just for fun)
Gradle

Running Kafka inside Docker (spotify/kafka) to play with it locally :

docker run --name spotifykafka -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=localhost --env ADVERTISED_PORT=9092 spotify/kafka

to get Shopify/sarama lib - Kafka client:

go get github.com/Shopify/sarama

How to run it

  1. Run docker spotify/kafka
  2. Run Java/Kotlin/Go app from IDE (Prefer IDEA & Gogland)
  3. Send request to Java Source App - POST http://localhost:8080/message {“name”: “Krzysztof”}
  4. Both Go and Kotlin app will print the messages on their consoles, something like this: INFO 22360 --- [afka-listener-1] com.example.SampleSink Kotlin received: Krzysztof 
Golang received: Krzysztof

Content-Type with Go Application Problem

If you have Spring Applications, you can just set content-type inside properties like this:

spring.cloud.stream.bindings.output.content-type=application/json

But Go Shopify/sarama client has problem with it, I used binary format, this way:

spring.cloud.stream.bindings.output.producer.headerMode=raw

And my source in Java looks like this:

	@PostMapping(path = "/message", consumes = MediaType.APPLICATION_JSON_UTF8_VALUE)
	public @ResponseBody void sendMessage(@RequestBody MessageDto message) throws JsonProcessingException {
		byte[] bytes = objectMapper.writeValueAsBytes(message);

		// normally you can use something like this instead of bytes :
		// MessageBuilder.withPayload(message).build()
		output.send(new GenericMessage<>(bytes));
	}

If you got better solution, tell me :)

Avro Content-Type

My example doesn’t include it, but it is a popular Content-Type used with Apache Kafka.

  1. Spring Cloud Stream Docs
  2. BeeWorks Blog: Start Streaming with Kafka and Spring Cloud
  3. Github: Spring Cloud Stream, Kafka, Avro examples

References

  1. Implementing CQRS using Kafka and Sarama Library in Golang
  2. Event Driven Microservices with Spring Cloud Stream
  3. Go 1.8 HTTP/2 Push - in my example it is almost the same

HTTP/2 Go Push Bonus (only for fun)

  1. Go 1.8beta2 required
  2. Run crypto/tls/generate_cert.go to generate cert.pem and key.pem. and put them to the project
  3. Go to https://localhost:8082/ (yes, https)
  4. Resources will be pushed to browser, check Network (devtools)