Home » Author Archives: Joe Stein

Author Archives: Joe Stein

Open Source Cloud Formation with Minotaur for Mesos, Kafka and Hadoop

apache-hadoop-logo

Today I am happy to announce “Minotaur” which is our Open Source AWS based infrastructure for managing big data open source projects including (but not limited too): Apache Kafka, Apache Mesos and Cloudera’s Distribution of Hadoop. Minotaur is based on AWS Cloud Formation. The following labs are currently supported:           Apache Mesos Apache Kafka Apache Zookeeper Cloudera Hadoop ...

Read More »

Resource scheduling and task launching with Apache Mesos and Apache Aurora at Twitter

java-interview-questions-answers

Episode # 23 of the podcast was a talk with Bill Farner Bill explained how Twitter, using Apache Mesos and Apache Aurora, gets more for their money for the hardware and saves engineering time (both development and operations) by utilizing fine grained resources scheduling across their infrastructure. Bill talked a bit how the power of what he saw and experienced at Google with Borg ...

Read More »

Reporting Metrics to Apache Kafka and Monitoring with Consumers

software-development-2-logo

Apache Kafka has been used for some time now by organizations to consume not only all of the data within its infrastructure from an application perspective but also the server statistics of the running applications and infrastructure.  Apache Kafka is great for this. Coda Hale’s metrics’s has become a leading way to instrument your JVM applications capturing what the application is doing ...

Read More »

XML to Avro Conversion

java-interview-questions-answers

We all know what XML is right?  Just in case not, no problem here is what it is all about.                   <root> <node>5</node> </root> Now, what the computer really needs is the number five and some context around it. In XML you (human and computer) can see how it represents context to ...

Read More »

Getting started with Apache Mesos and Apache Aurora using Vagrant

vagrant-logo

Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. Think of it as the “kernel” for your data center. Paco Nathan talked about this on one of the All Things Hadoop Podcasts. Features:             Fault-tolerant replicated master using ZooKeeper Scalability to 10,000s of nodes Isolation between tasks ...

Read More »

Technology Decisions Are About Trade Offs and Solving Problems

software-development-logo

At some point in the last decade we hit the inflection point where distributed systems, and all their complexities, became the common reality. Maybe it was the need to change how we scale since CPU clocks are not getting any faster… Maybe it was the Google MapReduce and/or Amazon Dynamo papers… Or maybe it was just the RedSox winning the ...

Read More »

Big Data Open Source Security

apache-hadoop-logo

In security there has never (IMHO) been enough open source solutions and Bruce Schneier has written about this several times in the past, and there’s no need to rewrite the arguments again. Now with “NoSQL” and “Big Data” Open Source trends in the market place Security finally has an intersection… a union if I may where new solutions to solve ...

Read More »

Distributed System Development Considerations

apache-hadoop-logo

There are a number of factors to take into account while developing distributed software systems. If you don’t even know what I am talking about in the first sentence then let me give you some insight, examples and for instances of what distributed systems are. Overview A distributed system is when multiple physical hardware devices interact with separate and discrete users and collaborate together through these ...

Read More »
Do you want to know how to develop your skillset and become a ...

Subscribe to our newsletter to start Rocking right now!

To get you started we give you our best selling eBooks for FREE!
Get ready to Rock!
To download the books, please verify your email address by following the instructions found on the email we just sent you.

THANK YOU!

Close