Home » Kafka

Tag Archives: Kafka

It is never enough of them: enriching Apache Avro generated classes with custom Java annotations

Apache Avro, along with Apache Thrift and Protocol Buffers, is often being used as a platform-neutral extensible mechanism for serializing structured data. In the context of event-driven systems, the Apache Avro‘s schemas play the role of the language-agnostic contracts, shared between loosely-coupled components of the system, not necessarily written using the same programming language. Probably, the most widely adopted reference ...

Read More »

Kafka – Exactly-Once Semantics

java-interview-questions-answers

In distributed environment failure is a very common scenario that can be happened any time. In the Kafka environment, the broker can crash, network failure, failure in processing, failure while publishing message or failure to consume messages, etc. These different scenarios introduced a different kind of data loss and duplication. Failure scenarios A (Ack Failed): Producer published message successfully with retry>1 ...

Read More »

Long Live ETL

Extract transform load is process for pulling data from one datasystem and loading into another datasystem. Datasystem involved are called source system and target system. Shape of data from source system does not match to the target system, so some conversion is required to make it compatible and that process is called transformation. Transformation is made of map/filter/reduce operations. To ...

Read More »

Send your data async on Kafka

spring-interview-questions-answers

For a project, I’m trying to log the basic transactions of the user such as addition and removal of an item and for multiple types of items and sending a message to kafka for each transaction. The accuracy of the log mechanism is not crucial and I don’t want it to block my business code in the case of kafka ...

Read More »

Microservices in Publish-Subscribe communication using Apache Kafka as a Messaging Systems and validated through Integration Test

Publish-Subscribe Messaging systems play an important role in any enterprise architecture as it enables reliable integration without tightly coupling the applications. The ability to share data between decoupled systems is not a problem that is easily tackled. Consider an enterprise with multiple applications that are being built independently, with different languages and platforms. It needs to share data and processes ...

Read More »

Spring Cloud Stream with Kafka

spring-interview-questions-answers

Overview This sample project demonstrates how to build real-time streaming applications using event-driven architecture, Spring Boot,Spring Cloud Stream, Apache Kafka and Lombok. By the end of this tutorial you’ll have a simple Spring Boot based Greetings micro-service running that takes a message from a REST api writes it to a Kafka topic reads it from the topic outputs it to the console Let’s get started! Btw, you’ll ...

Read More »

Getting started with Apache Flink and Kafka

java-interview-questions-answers

Introduction Apache Flink is an open source platform for distributed stream and batch data processing. Flink is a streaming data flow engine with several APIs to create data streams oriented application. It is very common for Flink applications to use Apache Kafka for data input and output. This article will guide you into the steps to use Apache Flink with ...

Read More »