Enterprise Java

Implementing Master Slave / Grid Computing Pattern in Akka

Master Slave pattern is a prime example of fault tolerance and parallel computation. The idea behind the pattern is to partition the work into identical sub tasks which are then delegated to Slaves. These slave node or instances will process the work task and send back the result to the master. The master will then compile the resultsreceivedfrom all the slave nodes.Key here is that the Slave nodes are only aware on how to process the task and not aware of what happens to the output.

The Master Slave pattern is analogous to the Grid Computing pattern where a control node distributes the work to other nodes. Idea is to make use of the nodes on the network for their computing power. SETI@Home was one of the earliest pioneers in using this model.

I have build a similar example with difference being that worker nodes get started on Remote Nodes, Worker Nodes register with Master(WorkServer) and then subsequently start processing work packets. If there is no worker slave registered with Master(WorkServer), the master waits the workers to register. The workers can register at any time and will start getting work packets from there on.

The example demonstrates how an WorkerActor system sends a request for registration. The RegisterRemoteWorker recieves the request and forwards the same to JobController where the RoundRobinRouter is updated for the new worker information. The WorkScheduler sends a periodic request to JobController, who then sends packets to all the registered worker actors.

The example does not implement fault tolerance with respect to on how to handle failures when the remote actors die or how to re-process packets that have not been processed. Similarly, there may be cases where the remote worker actors might want to shutdown after processing certain amount of packets, they can then indicate to the master to stop giving them work. I will add fault tolerance soon!

Updated: Code base updated to handle worker shutdowns. If the remote actors die or shut down, the JobController detects thefail-oversusing remote actorlistenersand updates the router.

The code base for the program is available at the following location – https://github.com/write2munish/Akka-Essentials under the GridPatternExample

Reference: Implementing Master Slave / Grid Computing Pattern in Akka from our JCG partner Munish K Gupta at the Akka Essentials blog.

Munish Gupta

Munish K Gupta is a senior architect working in a leading IT services company. His experience is in building Online Portals, SaaS Platforms, CRM Solutions and Transaction Processing Systems. He is author of the book - Akka Essentials.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

1 Comment
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Young
10 years ago

I will fork it.

Back to top button