Home » Apache Kafka

Tag Archives: Apache Kafka

Kafka Record Patterns for Data Replication

java-interview-questions-answers

Imagine going down to your local milkshake bar and signing a contract with the owner so that you could purchase bespoke drinks at a set price. Let’s say you agreed on fresh milk with 3.5% fat and one tablespoon of chocolate powder, per 500ml of milk.  Putting that into a table might look like this: PKcontract_numberstartfat_contentchocolate_powder100123456782021-01-013.5%1 tbsp After a few ...

Read More »

Apache Kafka Consumer Rebalance

java-interview-questions-answers

Consumer rebalance decide which consumer is responsible for which subset of all available partitions for some topic(s). For example, you might have a topic with 20 partitions and 10 consumers; at the end of a rebalance, you might expect each consumer to be reading from 2 partitions. If you shut down 10 of those consumers, you might expect each consumer ...

Read More »

Stream Big Data with Apache Kafka Integration

java-interview-questions-answers

Apache Kafka is being increasingly integrated into a variety of systems and solutions, from filtering and processing large amounts of data real-time, to logging and creating metric data into a centralized handler from different sources. Using CData Sync, such solutions can easily be applied to any CRM, ERP or Analytics software. Configuring Apache Kafka Destination for CData Sync Setting up the ...

Read More »

Service Mesh and Cloud-Native Microservices with Apache Kafka, Kubernetes and Envoy, Istio, Linkerd

Microservice architectures are not free lunch! Microservices need to be decoupled, flexible, operationally transparent, data aware and elastic. Most material from last years only discusses point-to-point architectures with tightly coupled and non-scalable technologies like REST / HTTP. This blog post takes a look at cutting edge technologies like Apache Kafka, Kubernetes, Envoy, Linkerd and Istio to implement a cloud-native service ...

Read More »

Deep Learning Example: Apache Kafka & Python & Keras &TensorFlow & Deeplearning4j

I added a new example to my “Machine Learning + Kafka Streams Examples” Github project: “Python + Keras + TensorFlow + DeepLearning4j + Apache Kafka + Kafka Streams“. This blog post discusses the motivation and why this is a great combination of technologies for scalable, reliable Machine Learning infrastructures. For more details about building Machine Learning / Deep Learning infrastructures ...

Read More »

Apache Kafka & MQTT = End-to-End IoT Integration (Github Examples)

Kafka and MQTT are two complementary technologies. Together they allows to build IoT end-to-end integration from the edge to the data center. No matter if on premise or in the public cloud. I talk about this topic at Kafka Summit SF in San Francisco in October 2018: “Processing IoT Data from End to End with MQTT and Apache Kafka“. As ...

Read More »

Spring Apache Kafka Tutorial

java-interview-questions-answers

In this Spring Apache Kafka lesson, we will learn how we can get started with Apache Kafka in a Spring Boot project and start producing and consuming messages on the topic of our choice. Apart from a simple project, we will deep dive into Kafka terminologies and how the concept of partitioning works in Kafka. Let’s get started. 1. Introduction With ...

Read More »

Using Apache Kafka for Asynchronous Communication in Microservices

While microservice architecture might not be a silver bullet for all systems, it definitely has its advantages, especially when building a complex system with a lot of different components. Of course, if you’re considering microservices, you have to give serious thought to how the different services will communicate. In this post, we’ll look at how to set up an Apache ...

Read More »