I had the opportunity to optimize my company's product search engine. My goal was to reduce its response time compared to the previous codebase. To ensure this improvement, I needed to conduct load testing. This blog shares my experience and insights from that process.

In my view, load testing assesses how a system handles real-world requests, including traffic spikes beyond normal levels. This allows us to evaluate its performance, scalability, and reliability under real conditions. Based on the results, we can optimize the system by adding more nodes, increasing RAM, or making other improvements to ensure it can handle production traffic effectively.

Why I Chose k6

Before starting, I needed to choose a tool for building load tests. Some popular options included k6, Apache JMeter, and others. After researching various tools, I found that:

• k6 has a visually appealing UI for statistics, making performance analysis easier.

• k6 uses JavaScript for scripting, which is more accessible for developers familiar with modern web technologies.

• k6 is built with Go, a language making it lightweight and efficient (and I love Go).

• k6 has the highest number of GitHub stars compared to other load testing tools, indicating strong community support.

Considering these factors, I decided to use k6 for load testing. Looking back, I believe I made the right choice.

Installation

The first step is to install k6 on your local machine. It supports Linux, macOS, and Windows, and you can install it using a package manager, a Docker container, or a standalone binary.

For detailed installation instructions, refer to: https://k6.io/docs/get-started/installation.

There are multiple ways to install k6, but in my opinion, using a Docker container is the quickest method. However, I also discovered that k6 can be installed as a standalone binary using Go.

To do this, start by creating a new Go workspace and installing the xk6 package with:

$ go install go.k6.io/xk6/cmd/xk6@latest

And then, build the binary file

$ xk6 build latest \
  --with github.com/grafana/[email protected] \
  --with github.com/grafana/xk6-output-prometheus-remote

Write the script

Now, we can begin writing our first load test script in JavaScript. In this script, we'll use the http module to send requests to multiple APIs.

import { SharedArray } from 'k6/data';
import { check, group } from 'k6';
import http from 'k6/http';
import { URL } from 'https://jslib.k6.io/url/1.0.0/index.js';

// Load the JSON file once and share across all VUs
const data = new SharedArray('logs', function () {
    return JSON.parse(open('./logs.json')); // exported from the database on the production.
});

export const options = {
    vus: 4, // Number of virtual users to run concurrently (depend on number of cores).
    iterations: 1000, // Number of total requests
}

export default function () {
    const randomIndex = Math.floor(Math.random() * data.length);
    const log = data[randomIndex];

    group('Request', function () {
        const url = new URL('<URL_FOR_TEST>');

        // add query params
        url.searchParams.append('q', log.keyword);

        // request to url
        const res = http.get(url.toString());

        console.log(`Requesting: ${url.toString()}`);

        // Check the response status code
        check(res, {
            'status is 200': (r) => r.status === 200,
        });
    });
};

You can review the code above along with the comments I’ve added to understand its purpose.

Next, I run the script using the following command:

$ K6_WEB_DASHBOARD=true K6_WEB_DASHBOARD_EXPORT='report.html' .\k6.exe run .\script.js

• K6_WEB_DASHBOARD enables the web dashboard.

• K6_WEB_DASHBOARD_EXPORT automatically saves the test report as an HTML file at the end of the test run.

Analysis load test report

Once the script runs successfully, a report is generated to provide insights into the load test results. To view the report, open the report.html file in the source code.

As part of my role in optimizing the search engine, I created two versions of the load test for comparison. While there are many statistical metrics available to evaluate performance, I primarily focus on the Request Duration metric to assess improvements. However, you can explore other metrics relevant to your specific load testing objectives.

Conclusion

That is it for this article on some really really busy days (T.T). I hope you found this article useful, if you need any help please let me know by directly inbox.

Let's connect on Twitter and LinkedIn.

👋 Thanks for reading, See yaaaaaaaa