Home » Apache Kafka » Page 4

Tag Archives: Apache Kafka

Apache Kafka – What Is It And Does It Compare To Amazon Kinesis?

What is Apache Kafka? Apache Kafka is an open-source, distributed, scalable publish-subscribe messaging system.  The organization responsible for the software is the Apache Software Foundation.  The code is written in Scala and was initially developed by the LinkedIn Company.  It was open-sourced in 2011 and became a top-level Apache project. The project has the intention of providing a unified low-latency ...

Read More »

Getting Started with Heron on Apache Mesos and Apache Kafka

Heron has been Open Sourced, woo! Heron is Twitter’s  distributed stream computation system for running Apache Storm compatible topologies in production. A Heron topology is a directed acyclic graph used to process streams of data. Heron topologies consist of three basic components: spouts and bolts, which are connected via streams of tuples. Below is a visual illustration of a simple topology: Spouts are ...

Read More »

Distributed Quasar Actors with Kafka and ZeroMQ

java-interview-questions-answers

So you’ve got a fancy design using actors, you’ve chosen the JVM and Quasar’s powerful, loyal take on the subject. All wise decisions, but then what are your options for distributing them on a cluster? Galaxy Galaxy is a really cool option: a fast in-memory data grid optimized for data locality with replication, optional persistence, a distributed actor registry and even ...

Read More »

Whirlpool: Microservices Using Netty And Kafka

Introduction In my last blog, I introduced Netty being used as a web server. That example worked well… as long as a broadcast server is what was needed. Most of the time that is not very useful. It’s more likely that the need is for each client to receive only the data intended for them, with broadcasts reserved for special circumstances like “The ...

Read More »

Getting Started with Sample Programs for Apache Kafka 0.9

Streaming data is of growing interest to many organizations, and most applications need to use a producer-consumer model to ingest and process data in real time. Many messaging solutions exist today on the market, but few of them have been built to handle the challenges of modern deployment related to IoT, large web based applications and related big data projects. ...

Read More »

Open Source Cloud Formation with Minotaur for Mesos, Kafka and Hadoop

Today I am happy to announce “Minotaur” which is our Open Source AWS based infrastructure for managing big data open source projects including (but not limited too): Apache Kafka, Apache Mesos and Cloudera’s Distribution of Hadoop. Minotaur is based on AWS Cloud Formation. The following labs are currently supported:           Apache Mesos Apache Kafka Apache Zookeeper Cloudera Hadoop ...

Read More »

Kafka Benchmark on Chronicle Queue

java-interview-questions-answers

Overview I was recently asked to compare the performance of Kafka with Chronicle Queue.  No two products are exactly alike, and performing a fair comparison is not easy.  We can try to run similar tests and see what results we get. This test is based on Apache Kafka Performance Results.         What was the test used? One ...

Read More »

Reporting Metrics to Apache Kafka and Monitoring with Consumers

Apache Kafka has been used for some time now by organizations to consume not only all of the data within its infrastructure from an application perspective but also the server statistics of the running applications and infrastructure.  Apache Kafka is great for this. Coda Hale’s metrics’s has become a leading way to instrument your JVM applications capturing what the application is doing ...

Read More »