Apache Gearpump is a real-time big data streaming engine. It conceived at Intel in mid-2014 as an open source project on GitHub from start and entered Apache incubation on March 8th, 2016. The name Gearpump is a reference to the engineering term “gear pump” which is a super simple pump that consists of only two gears, but is very powerful at streaming water. Different to other streaming engines, Gearpump’s engine is event/message based. Per initial benchmarks we are able to process 18 million messages per second (message length is 100 bytes) with a 8ms latency on a 4-node cluster.
The Highlights
- Extremely high throughput and low latency stream processing
- Configurable message delivery guarantee (at least once, exactly once)
- Application hot re-deployment
- Comprehensive Dashboard for application monitoring
- Native Storm Application compatibility
- Native Samoa Application compatibility
- Friendly and extensible APIs
Before you can submit and run your first Gearpump application, you will need a running Gearpump service. There are multiple ways to run Gearpump Local mode, Standalone mode, YARN mode or Docker mode.
The easiest way is to run Gearpump in Local mode. Any Linux, MacOSX or Windows desktop can be used with zero configuration.
In the example below, we assume your are running in Local mode. If you running Gearpump in one of the other modes, you will need to configure the Gearpump client to connect to the Gearpump service by setting the gear.conf configuration path in classpath. Within this file, you will need to change the parameter gearpump.cluster.masters to the correct Gearpump master(s).
Steps to submit your first Application
Step 1: Submit Application
After the cluster is started, you can submit an example wordcount application to the cluster
Open another shell,
### To run WordCount example bin/gear app -jar examples/wordcount-2.11-0.8.4-assembly.jar org.apache.gearpump.streaming.examples.wordcount.WordCount
Step 2: View Application Status and Metrics.
To view the application status and metrics, start the Web UI services, and browse to http://127.0.0.1:8090 to check the status. The default username and password is “admin:admin”, you can check UI Authentication to find how to manage users.
[1] https://gearpump.apache.org/overview.html
[2] https://www.slideshare.net/manuzhang/apache-gearpump-lightweight-runtime-streaming-engine
[3] http://incubator.apache.org/projects/gearpump.html
[4] https://gearpump.apache.org/releases/latest/introduction/submit-your-1st-application/index.html
| Published on Java Code Geeks with permission by Furkan Kamaci, partner at our JCG program. See the original article here: Apache Gearpump Opinions expressed by Java Code Geeks contributors are their own. |
