JavaScript

Readable and Writable Streams in Node.js

Hello. In this tutorial, we will explain readable and writable Streams in a Node.js application.

1. Introduction

Streams in the node are the objects that help you to write and read data to and from the destination source. There are four different types of streams in node.js:

  • Readable – Stream uses for reading operations
  • Writable – Stream used for write operations
  • Duplex – Stream used for both reading and write operations
  • Transform – Type of duplex stream where the output is computed according to the input

Each stream consists of several Event emitter events fired at different times –

  • Data – Event is fired when data is available to read
  • End – Event is fired when there is no more data available to read
  • Error – Event is fired when there is an error receiving or writing the data
  • Finish – Event is fired when all the data has been flushed

1.1 Setting up Node.js

To set up Node.js on windows you will need to download the installer from this link. Click on the installer (also include the NPM package manager) for your platform and run the installer to start with the Node.js setup wizard. Follow the wizard steps and click on Finish when it is done. If everything goes well you can navigate to the command prompt to verify if the installation was successful as shown in Fig. 1.

Writable Streams in Node.js - npm installation
Fig. 1: Verifying node and npm installation

2. Readable and Writable Streams in Node.js

To set up the application, we will need to navigate to a path where our project will reside. For programming stuff, I am using Visual Studio Code as my preferred IDE. You’re free to choose the IDE of your choice.

2.1 Setting up the implementation

Let us write the different files which will be required for practical learning.

2.1.1 Setting up dependencies

Navigate to the project directory and run npm init -y to create a package.json file. This file holds the metadata relevant to the project and is used for managing the project dependencies, script, version, etc. Add the following code to the file wherein we will specify the required dependencies.

package.json

{
  "name": "streamdata",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "nodemon index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "faker": "^5.5.3",
    "nodemon": "^2.0.13"
  }
}

To download the dependencies navigate to the directory path containing the file and use the npm install command. If everything goes well the dependencies will be loaded inside the node_modules folder and you are good to go with the further steps.

2.1.2 Creating controller

In the root folder add the following content to the index file. The file will be exposing 2 endpoints –

  • /write to create a writeable stream and write mock data into a file
  • /read to create a readable stream and read data from the file

index.js

const http = require("http");
const fs = require("fs");
const faker = require("faker");

const file = "data.txt";

const server = http.createServer(function (req, resp) {
  if (req.method === "POST" && req.url === "/write") {
    // http://localhost:5001/write
    console.log("writing data to stream.");
    const writeStream = fs.createWriteStream(file);
    for (let i = 0; i <= 1000; i++) {
      writeStream.write(faker.lorem.words(20) + "\n");
    }

    writeStream.end();
    resp.end();
  } else if (req.method === "GET" && req.url === "/read") {
    // http://localhost:5001/read
    console.log("reading data.");
    // getting data in a single chunk
    /* fs.readFile(file, (err, data) => {
      if (err) throw err;
      resp.end(data);
    }); */

    // doing data streaming
    var data = "";
    const readStream = fs.createReadStream(file);
    readStream.on("data", (chunk) => {
      console.log("chunk data received.");
      data += chunk;
    });
    // method will be called when no more data is available for read.
    readStream.on("end", () => {
      resp.end(data);
    });
  } else {
    throw new Error("NOT_YET_IMPLEMENTED");
  }
});

// start app
const PORT = process.env.PORT || 5001;
server.listen(PORT, () => {
  console.log(`Server started on port ${PORT}`);
});

3. Run the Application

To run the application navigate to the project directory and enter the following command as shown in Fig. 2. If everything goes well the application will be started successfully on a port number read from the .env file or 5001.

Writable Streams in Node.js - starting the app
Fig. 2: Starting the application

4. Demo

You are free to use postman or any other tool of your choice to make the HTTP request to the application endpoints.

Endpoints

// http post - http://localhost:5001/write
// http get - http://localhost:5001/read

That is all for this tutorial and I hope the article served you with whatever you were looking for. Happy Learning and do not forget to share!

5. Summary

In this tutorial, we explained readable and writable Streams in a Node.js application. You can download the source code and the postman collection from the Downloads section.

6. Download the Project

This was a tutorial to implement Streams in the nodejs application.

Download
You can download the full source code of this example here: Readable and Writable Streams in Node.js

Yatin Batra

An experience full-stack engineer well versed with Core Java, Spring/Springboot, MVC, Security, AOP, Frontend (Angular & React), and cloud technologies (such as AWS, GCP, Jenkins, Docker, K8).
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Back to top button