ReactJS

State Management with React: Client State and Beyond

Jun 28, 2023

6 min read

Introduction

State management has long been a hotly debated subject in the React world. Best practices have continued to evolve, and there’s still a lot of confusion around the subject. In this article, we are going to dive into what tools we might want to use for solving common problems we face in modern web application development. There are different types of state and different types of ways to store and manage your application state. For example, you might have some local client state in a component that controls a dropdown, and you might also have a global store that contains authenticated user state. Aside from those typical examples, on some pages of your website, you might store some state in the URL.

For quite a while, we’ve been leaning into global client state in single-page app land. This is what state management tools like Redux and other similar libraries help us out with. In this post, we will cover these different strategies in more detail so that we can try to make the best decisions going forward regarding state management in our applications.

Local and global client state

In this section, we'll discuss the differences between local and global client state, when to use each, and some popular libraries that can help you manage them effectively.

When to use local client state

Local client state is best suited for managing state that is specific to a single component or a small group of related components. Examples of local state include form inputs, component visibility, and UI element states like button toggles. Using local state keeps the component self-contained, making it easier to understand and maintain. In general, it's a good idea to start with local state and only use global state when you have a clear need for it. This can help keep your code simple and easy to understand.

When to use global client state

Global client state is useful when you have state that needs to be accessed by multiple unrelated components, or when the state is complex and would benefit from a more carefully designed API. Common examples of global state include user authentication, theme preferences, and application-wide settings. By centralizing this state, you can easily share it across the entire application, making it more efficient and consistent.

If you’re building out a feature that has some very complex state requirements, this could also be a good case for using a “global” state management library. Truth is, you can still use one of these libraries to manage state that is actually localized to your feature. Most of these libraries support creating multiple stores. For example, if I was building a video chat client like Google Meets with a lot of different state values that are constantly changing, it might be a good idea to create a store to manage the state for a video call. Most state management libraries support more features than what you get out of the box with React, which can help design a clean and easy to reason about modules and APIs for scenarios where the state is complex.

Popular libraries for global state management

There are a lot of great libraries for managing state in React out there these days. I think deciding which might be best for you or your project is mostly a matter of preference.

Redux and MobX are a couple of options that have been around for quite a long time. Redux has gained a reputation for being overly complex and requiring a lot of boilerplate code. The experience is actually much improved these days thanks to the Redux Toolkit library. MobX provides an easy-to-use API that revolves around reactive data/variables. These are both mature and battle-tested options that are always worth considering.

Meta also has a state management named Recoil that provides a pretty easy-to-use API that revolves around the concept of atoms and selectors. I don’t see this library being used a ton in the wild, but I think it’s worth mentioning.

A couple of the more popular new players on the block are named jotai and zustand. I think after the Redux hangover, these libraries showed up as a refreshing oasis of simplicity. Both of these libraries have grown a ton in popularity due to their small byte footprints and simple, straightforward APIs.

Context is not evil

The React Context API, like Redux, has also been stigmatized over the years to the point where many developers have their pitchforks out, declaring that you should never use it. We leaned on it for state management a bit too much for a while, and now it is a forbidden fruit. I really dislike these hard all-or-nothing stances though. We just need to be a little bit more considerate about when and where we choose to use it.

Typically, React Context is best for storing, and making available global state that doesn’t change much. Some of the most common use cases are things like themes, authentication, localization, and user preferences. Contrary to popular belief, only components (and their children) that use that context (const context = useContext(someContext);) are re-rendered in the event of a state change, not all of the children below the context provider.

Storing state in the URL

The most underused and underrated tool in the web app state management tool belt is using the URL to store state. Storing state in the URL can be beneficial for several reasons, such as enabling users to bookmark and share application state, improving SEO, and simplifying navigation. The classic example for this is filters on an e-commerce website. A good user experience would be that the user can select some filters to show only the products that they are looking for and then be able to share that URL with a friend and them see the same exact results. Before you add some state to a page, I think it’s always worth considering the question: “Should I be able to set this state from the URL?”.

Tools for managing URL state

We typically have a couple of different tools available to use for managing URL state. Built-in browser APIs like the URL class and URLSearchParams. Both of these APIs allow you to easily parse out parts of a URL. Most often, you will store URL state in the parameters.

In most React applications, you will typically have a routing library available to help with URL and route state management as well. React Router has multiple hooks and other APIs for managing URL state like useLocation that returns a parsed object of the current URL state.

Keeping URL and application state in sync

The tricky part of storing state in the URL is when you need to keep local application state in sync with the URL values. Let’s look at an example component with a simple name component that stores a piece of state called name.

import React, { useState, useEffect } from 'react';
import { useLocation, useHistory } from 'react-router-dom';

function MyComponent() {
	const location = useLocation();
	const history = useHistory();
	const [name, setName] = useState('');

	useEffect(() => {
		// Update the name state when the URL changes
		const searchParams = new URLSearchParams(location.search);
		setName(searchParams.get('name') || '');
	}, [location]);

	function handleNameChange(event) {
		setName(event.target.value);
		// Update the URL when the name changes
		history.push(`/my-component?name=${event.target.value}`);
	}

	return (
		<div>
			<input type="text" value={name} onChange={handleNameChange} />
		</div>
	);
}

The general idea is to pull the initial value off of the URL when the component mounts and set the state value. After that, in our event handler, we make sure to update the URL state as well as our local React state.

Moving state to the server

Moving web application state to the server can be beneficial in several scenarios. For example, when you have a complex state that is difficult to manage on the client-side. By moving the state to the server, you can simplify the client-side code and reduce the amount of data that needs to be transferred between the client and server. This can be useful for applications that have a lot of business logic or complex data structures. In most cases if there’s some logic or other work that you can move off of your client web application and onto the server that is a win.

Conclusion

State management is a crucial aspect of building modern web applications with React. By understanding the different types of state and the tools available for managing them, you can make informed decisions about the best approach for your specific use case. Remember to consider local and global client state, URL-based state, and server-side state when designing your application's state management strategy.

This Dot Labs is a development consultancy that is trusted by top industry companies, including Stripe, Xero, Wikimedia, Docusign, and Twilio. This Dot takes a hands-on approach by providing tailored development strategies to help you approach your most pressing challenges with clarity and confidence. Whether it's bridging the gap between business and technology or modernizing legacy systems, you’ll find a breadth of experience and knowledge you need. Check out how This Dot Labs can empower your tech journey.

Dane Grant

JavaScript developer building things on web, native, and server environments. Free time filled with reading, cooking, dog, sports, comedy, and music.

@danecando @danecando

Building a Multi-Response Streaming API with Node.js, Express, and React

Introduction As web applications become increasingly complex and data-driven, efficient and effective data transfer methods become critically important. A streaming API that can send multiple responses to a single request can be a powerful tool for handling large amounts of data or for delivering real-time updates. In this article, we will guide you through the process of creating such an API. We will use video streaming as an illustrative example. With their large file sizes and the need for flexible, on-demand delivery, videos present a fitting scenario for showcasing the power of multi-response streaming APIs. The backend will be built with Node.js and Express, utilizing HTTP range requests to facilitate efficient data delivery in chunks. Next, we'll build a React front-end to interact with our streaming API. This front-end will handle both the display of the streamed video content and its download, offering users real-time progress updates. By the end of this walkthrough, you will have a working example of a multi-response streaming API, and you will be able to apply the principles learned to a wide array of use cases beyond video streaming. Let's jump right into it! Hands-On Implementing the Streaming API in Express In this section, we will dive into the server-side implementation, specifically our Node.js and Express application. We'll be implementing an API endpoint to deliver video content in a streaming fashion. Assuming you have already set up your Express server with TypeScript, we first need to define our video-serving route. We'll create a GET endpoint that, when hit, will stream a video file back to the client. Please make sure to install cors for handling cross-origin requests, dotenv for loading environment variables, and throttle for controlling the rate of data transfer. You can install these with the following command: ` yarn add cors dotenv throttle @types/cors @types/dotenv @types/throttle ` `typescript import cors from 'cors'; import 'dotenv/config'; import express, { Request, Response } from 'express'; import fs from 'fs'; import Throttle from 'throttle'; const app = express(); const port = 8000; app.use(cors()); app.get('/video', (req: Request, res: Response) => { // Video by Zlatin Georgiev from Pexels: https://www.pexels.com/video/15708449/ // For testing purposes - add the video in you static` folder const path = 'src/static/pexels-zlatin-georgiev-15708449 (2160p).mp4'; const stat = fs.statSync(path); const fileSize = stat.size; const range = req.headers.range; if (range) { const parts = range.replace(/bytes=/, '').split('-'); const start = parseInt(parts[0], 10); const end = parts[1] ? parseInt(parts[1], 10) : fileSize - 1; const chunksize = end - start + 1; const file = fs.createReadStream(path, { start, end }); const head = { 'Content-Range': bytes ${start}-${end}/${fileSize}`, 'Accept-Ranges': 'bytes', 'Content-Length': chunksize, 'Content-Type': 'video/mp4', }; res.writeHead(206, head); file.pipe(res); } else { const head = { 'Content-Length': fileSize, 'Content-Type': 'video/mp4', }; res.writeHead(200, head); fs.createReadStream(path).pipe(res); } }); app.listen(port, () => { console.log(Server listening at ${process.env.SERVER_URL}:${port}`); }); ` In the code snippet above, we are implementing a basic video streaming server that responds to HTTP range requests. Here's a brief overview of the key parts: 1. File and Range Setup__: We start by determining the path to the video file and getting the file size. We also grab the range header from the request, which contains the range of bytes the client is requesting. 2. Range Requests Handling__: If a range is provided, we extract the start and end bytes from the range header, then create a read stream for that specific range. This allows us to stream a portion of the file rather than the entire thing. 3. Response Headers__: We then set up our response headers. In the case of a range request, we send back a '206 Partial Content' status along with information about the byte range and total file size. For non-range requests, we simply send back the total file size and the file type. 4. Data Streaming__: Finally, we pipe the read stream directly to the response. This step is where the video data actually gets sent back to the client. The use of pipe() here automatically handles backpressure, ensuring that data isn't read faster than it can be sent to the client. With this setup in place, our streaming server is capable of efficiently delivering large video files to the client in small chunks, providing a smoother user experience. Implementing the Download API in Express Now, let's add another endpoint to our Express application, which will provide more granular control over the data transfer process. We'll set up a GET endpoint for '/download', and within this endpoint, we'll handle streaming the video file to the client for download. `typescript app.get('/download', (req: Request, res: Response) => { // Again, for testing purposes - add the video in you static` folder const path = 'src/static/pexels-zlatin-georgiev-15708449 (2160p).mp4'; const stat = fs.statSync(path); const fileSize = stat.size; res.writeHead(200, { 'Content-Type': 'video/mp4', 'Content-Disposition': 'attachment; filename=video.mp4', 'Content-Length': fileSize, }); const readStream = fs.createReadStream(path); const throttle = new Throttle(1024 1024 * 5); // throttle to 5MB/sec - simulate lower speed readStream.pipe(throttle); throttle.on('data', (chunk) => { Console.log(Sent ${chunk.length} bytes to client.`); res.write(chunk); }); throttle.on('end', () => { console.log('File fully sent to client.'); res.end(); }); }); ` This endpoint has a similar setup to the video streaming endpoint, but it comes with a few key differences: 1. Response Headers__: Here, we include a 'Content-Disposition' header with an 'attachment' directive. This header tells the browser to present the file as a downloadable file named 'video.mp4'. 2. Throttling__: We use the 'throttle' package to limit the data transfer rate. Throttling can be useful for simulating lower-speed connections during testing, or for preventing your server from getting overwhelmed by data transfer operations. 3. Data Writing__: Instead of directly piping the read stream to the response, we attach 'data' and 'end' event listeners to the throttled stream. On the 'data' event, we manually write each chunk of data to the response, and on the 'end' event, we close the response. This implementation provides a more hands-on way to control the data transfer process. It allows for the addition of custom logic to handle events like pausing and resuming the data transfer, adding custom transformations to the data stream, or handling errors during transfer. Utilizing the APIs: A React Application Now that we have a server-side setup for video streaming and downloading, let's put these APIs into action within a client-side React application. Note that we'll be using Tailwind CSS for quick, utility-based styling in our components. Our React application will consist of a video player that uses the video streaming API, a download button to trigger the download API, and a progress bar to show the real-time download progress. First, let's define the Video Player component that will play the streamed video: `tsx import React from 'react'; const VideoPlayer: React.FC = () => { return ( Your browser does not support the video tag. ); }; export default VideoPlayer; ` In the above VideoPlayer component, we're using an HTML5 video tag to handle video playback. The src attribute of the source tag is set to the video endpoint of our Express server. When this component is rendered, it sends a request to our video API and starts streaming the video in response to the range requests that the browser automatically makes. Next, let's create the DownloadButton component that will handle the video download and display the download progress: `tsx import React, { useState } from 'react'; const DownloadButton: React.FC = () => { const [downloadProgress, setDownloadProgress] = useState(0); const handleDownload = async () => { try { const response = await fetch('http://localhost:8000/download'); const reader = response.body?.getReader(); if (!reader) { return; } const contentLength = +(response.headers?.get('Content-Length') || 0); let receivedLength = 0; let chunks = []; while (true) { const { done, value } = await reader.read(); if (done) { console.log('Download complete.'); const blob = new Blob(chunks, { type: 'video/mp4' }); const url = window.URL.createObjectURL(blob); const a = document.createElement('a'); a.style.display = 'none'; a.href = url; a.download = 'video.mp4'; document.body.appendChild(a); a.click(); window.URL.revokeObjectURL(url); setDownloadProgress(100); break; } chunks.push(value); receivedLength += value.length; const progress = (receivedLength / contentLength) 100; setDownloadProgress(progress); } } catch (err) { console.error(err); } }; return ( Download Video {downloadProgress > 0 && downloadProgress Download progress: )} {downloadProgress === 100 && Download complete!} ); }; export default DownloadButton; ` In this DownloadButton component, when the download button is clicked, it sends a fetch request to our download API. It then uses a while loop to continually read chunks of data from the response as they arrive, updating the download progress until the download is complete. This is an example of more controlled handling of multi-response APIs where we are not just directly piping the data, but instead, processing it and manually sending it as a downloadable file. Bringing It All Together Let's now integrate these components into our main application component. `tsx import React from 'react'; import VideoPlayer from './components/VideoPlayer'; import DownloadButton from './components/DownloadButton'; function App() { return ( My Video Player ); } export default App; ` In this simple App component, we've included our VideoPlayer and DownloadButton components. It places the video player and download button on the screen in a neat, centered layout thanks to Tailwind CSS. Here is a summary of how our system operates: - The video player makes a request to our Express server as soon as it is rendered in the React application. Our server handles this request, reading the video file and sending back the appropriate chunks as per the range requested by the browser. This results in the video being streamed in our player. - When the download button is clicked, a fetch request is sent to our server's download API. This time, the server reads the file, but instead of just piping the data to the response, it controls the data sending process. It sends chunks of data and also logs the sent chunks for monitoring purposes. The React application collects these chunks and concatenates them, displaying the download progress in real-time. When all chunks are received, it compiles them into a Blob and triggers a download in the browser. This setup allows us to build a full-featured video streaming and downloading application with fine control over the data transmission process. To see this system in action, you can check out this video demo. Conclusion While the focus of this article was on video streaming and downloading, the principles we discussed here extend beyond just media files. The pattern of responding to HTTP range requests is common in various data-heavy applications, and understanding it can be a useful tool in your web development arsenal. Finally, remember that the code shown in this article is just a simple example to demonstrate the concepts. In a real-world application, you would want to add proper error handling, validation, and possibly some form of access control depending on your use case. I hope this article helps you in your journey as a developer. Building something yourself is the best way to learn, so don't hesitate to get your hands dirty and start coding!...

Aug 9, 2023

6 mins

NodeJSExpress.jsReact

How to Integrate Mailchimp Forms in a React Project

Intro Today we will cover how to set up an email signup form using React and Mailchimp. This blog will be using the starter.dev cra-rxjs-styled-components template to expedite the process. This article assumes you have a basic understanding of React, and have set up a Mailchimp account. Here is the code repo if you want to review it while reading, or just skip ahead. We will start with setting up our React project using Starter.dev for simplicity, and then finish it up by integrating the two for our signup form. To start, we will be using the command yarn create @this-dot/starter --kit cra-rxjs-styled-components, which can be found here. We’ll go ahead, and give the project a name. I will be calling mine react-mailchimp. Now we will navigate into the project and do a yarn install. Then we can run yarn run dev to get it up and running locally on localhost:3000. This should have us load up on the React App, RxJS, and styled-components Starter kit page. With that all set, we’ll also need to install jsonp by using yarn add jsonp`. We’ll be using jsonp instead of fetch to avoid any CORS issues we may run into. This also makes for an easy and quick process by not relying on their API, which can’t be utilized by the client. Now that we have our project set up, we will go ahead and go and grab our form action URL from MailChimp. This can be found by going to your Audience > Signup Forms > Embedded Forms > Continue and then grabbing the form action URL found in the Embedded Form Code. We need to make a small change to the URL and swap /post? with /post-json?. We can now start setting up our form input, and our submit function. I will add a simple form input and follow it up, and a submit function. Inside the submit function, we will use our imported jsonp to invoke our action URL. ` import { useState } from 'react'; import jsonp from 'jsonp'; export const MailChimp = () => { const [email, setEmail] = useState(''); const onSubmit = (e: any) => { e.preventDefault(); const url = 'insert-mailchimp-action-url-here'; jsonp(${url}&EMAIL=${email}`, { param: 'c' }, (_: any, data: any) => { console.log('data', data); const { msg } = data; alert(msg); }); }; return ( Email setEmail(e.target.value)} > Submit ); }; ` We’ll also add a quick alert to let the user know that it was successful and that’s it! We’ve now successfully added the email to our MailChimp account. Conclusion Today, we covered how to integrate Mailchimp with a react app using the cra-rxjs-styled-components template from starter.dev. I highly recommend using starter.dev to get your project up and running quickly. Here is the code repo again for you to check out....

Jul 5, 2023

3 mins

React

NextJS App Router - Examining the First RSC Implementation

What is the NextJS App Router and why is it important? Why are we talking about the NextJS App Router? What’s the big deal about another application router in the React ecosystem? On the surface level, it doesn’t seem that important or interesting, but it turns out that it’s not just another run-of-the-mill routing library. Until now, React has been a client-side library that concerned itself only with the view layer. It has avoided having opinions on just about everything that isn’t rendering your UI. But with React Server Components (RSC) on the horizon, it’s become more difficult for them to not have a concern with some of our other application layers like routing. If React is now going to have a hand in our server it’s going to need to be more integrated with our stack to be able to coordinate work from the server all the way back to the client side of the application. So what is the plan for this? Instead of shipping a router or an entire full-stack framework, they are going to provide an API that framework authors can integrate with to support Server Components and Suspense. This is the reason why the React team has been working closely with the NextJS team. They are figuring out the API’s that React will provide and what an implementation of it will look like. Queue drumroll… Meet the NextJS App Router. So the App Router isn’t just your grandpa’s router. It’s the reference implementation of an integration with the new RSC architecture. It’s also a LOT more than just a routing library. Glancing at the documentation page on the Next beta docs it appears to span across almost all concerns of the framework. It turns out there’s a lot of pieces involved to support the RSC puzzle. Pieces of the App Router puzzle On the getting started page of the NextJS beta documentation, there’s a summary list of features in the new App Router. Let’s take a look at some of the important pieces. Routing This seems like the most obvious one given the name “App Router”. The routing piece is extremely important to the RSC implementation. It’s still a file-based routing setup like we’re used to with NextJS with some new features like layouts, nested routing, loading states, and error handling. This is truly where the magic happens. The docs refer to it as “server-centric” routing. The routing happens on the server which allows for server-side data fetching and fetching RSC’s. But don’t worry, our application can still use client-side navigation to give it that familiar SPA feel. With nested routing, layouts, and partial rendering a navigation change and page render might only change a small part of the page. Loading states and error handling can be used to apply a temporary loading indicator or an error message nested in your layout to handle these different states. Rendering Since the App Router is RSC based, it needs to know how to render both client and server components. By default, App Router uses server components. Client components are opt-in by placing a use client directive at the top of a file. One of the main selling points of using RSCs is that you don’t have to ship JavaScript code for your RSCs in your client bundles. You can interleave server and client components in your component tree. Your pages and components can still be statically rendered at build time or you have the option for dynamic (server) rendering using either node or edge runtimes. Data Fetching One of the main selling points of RSC is being able to collocate your data-fetching with your components. Components are able to do data fetching using async/await using the fetch API. This will probably end up being a point of controversy since according to the documentation, both React, and NextJS “extend” the built-in fetch primitive to provide request deduping and caching/revalidation. The docs recommend that you do your data-fetching inside server components for several different reasons. Some of the main ones being: - Reducing client-side waterfalls. - Possible direct access to databases. - Aggregating your data-fetching in requests to a single server call (think GraphQL resolvers). This pattern is definitely becoming the norm in newer frameworks. These are similar benefits that you would reap when using something like data loaders in Remix. The big difference is that you will be able to do the fetching directly from your server components which is a nice win for co-location. Caching We touched on part of this in the Fetching__ section. It’s one of the reasons why NextJS is extending the fetch primitive. It’s adding support for caching your data using HTTP. If you’re used to client-side React, and tools like React Query, you can kind of think of this as the server version of that. If the data from a particular fetch request is already available in the cache, it will return right away, instead of making a trip to the origin server to get it. The other piece of the App Router caching story has to do with server components specifically. NextJS App Router stores the result of RSC payloads in an in-memory client-side cache. If a user is navigating around your application using client-side navigation, and encounters a route segment that they have visited previously, and is available in the cache, it will be served right away. This will help to provide a more instantaneous feel to certain page transitions. Tooling (bundler) We still haven’t covered the entire App Router, and RSC story because in order to support RSC, you need a bundler that understands the server component graph. This is where Vercel’s new Webpack replacement Turbopack comes into play. It’s built on a modern low-level language named Rust. This provides improved build times and hot-module-reloading (HMR) times in development, which is fantastic. Since it’s a Webpack replacement, it will be able to handle a lot of different concerns like styles, static files, etc. Goals of RSC and NextJS App Router In this Twitter thread, Vercel CEO Guillermo Rauch highlights what he believes NextJS App Router brings to User Experience. The first one is that less JavaScript code gets shipped to the client. I don’t think anyone is arguing that this is not a good thing at this point. He also mentions fast page/route transitions that feel more like a SPA, and being able to quickly stream and render above-the-fold content like a hero section while the rest of the page below finishes loading. I’ve heard a counter-argument from some RSC critics that aren’t as confident about these gains, and believe that RSC trades off UX for better DX (things like co-locating data fetching with components). Since RSC and NextJS App Router are still largely untested beta software, it’s really hard to say that this new, novel idea will be all that they are hyping it up to be. There’s a major paradigm shift currently occurring in the community though, and there are a lot of new frameworks popping up that are taking different approaches to solving the problems brought on by the proliferation of large client-side JavaScript applications. I, for one, am excited to see if React can once again push some new ideas forward that will really change how we go about building our web applications. Opening the black box I don’t know about you, but I feel like I’ve been hearing about RSC for a long time now, and it’s really just felt like this fictional thing. It's as if nobody knows what it is or how it works. Its secrets are locked away inside these experimental builds that have been released by the React team. NextJS 13 Beta has finally started to give us a glimpse behind the curtain to see that it is a tangible thing, and what it looks like in practice. I’ll be honest, up to this point, I haven’t been interested enough to dig for answers to the half-baked questions and ideas about it swimming in my mind. I know that I’m not the only one that has had this feeling. If you’re keen on learning more about what an RSC implementation looks like, there’s a good Tweet thread from Dan Abramov that highlights a lot of the important pieces and links to the relevant source code files. Some other really curious people have also embarked on a journey to see if they could create an RSC implementation similar to App Router using Vite. The repo is a great reference for understanding what’s involved, and how things work. What’s left? Even if it does feel like a lot of things have been happening behind the scenes, to their credit, NextJS has provided a beta version of the new App Router that is still experimental, and very much a work in progress. We can try out RSC today to get a feel for what it’s like, and how they work. On top of that, the NextJS documentation includes a nice roadmap of the pieces that are completed, and things that are still in progress or not quite fleshed out. As of the time of this writing, some of the major items on the list that look like blockers to a stable release are related to data fetching like use(fetch() and cache(). The most important one that I’m excited to see a solution for is mutations. They currently have a “temporary workaround” for mutations that basically involves re-running all of the data-loading in the component tree where the mutation happens. I think the plan is to have some sort of RPC built into the React core to handle mutations. Final thoughts It’s been a long time coming, but I for one am excited to see the progress and evolution of RSC through the new NextJS App Router. Since it’s still an experimental and incomplete product, I will wait before I do any real application development with it. But I will probably spend some time trying it out and getting more familiar with it before that day comes....

May 24, 2023

7 mins

NextJS

Testing a Fastify app with the NodeJS test runner

Introduction Node.js has shipped a built-in test runner for a couple of major versions. Since its release I haven’t heard much about it so I decided to try it out on a simple Fastify API server application that I was working on. It turns out, it’s pretty good! It’s also really nice to start testing a node application without dealing with the hassle of installing some additional dependencies and managing more configurations. Since it’s got my stamp of approval, why not write a post about it? In this post, we will hit the highlights of the testing API and write some basic but real-life tests for an API server. This server will be built with Fastify, a plugin-centric API framework. They have some good documentation on testing that should make this pretty easy. We’ll also add a SQL driver for the plugin we will test. Setup Let's set up our simple API server by creating a new project, adding our dependencies, and creating some files. Ensure you’re running node v20 or greater (Test runner is a stable API as of the 20 major releases) Overview `index.js` - node entry that initializes our Fastify app and listens for incoming http requests on port 3001 `app.js` - this file exports a function that creates and returns our Fastify application instance `sql-plugin.js` - a Fastify plugin that sets up and connects to a SQL driver and makes it available on our app instance Application Code A simple first test For our first test we will just test our servers index route. If you recall from the app.js` code above, our index route returns a 501 response for “not implemented”. In this test, we're using the createApp` function to create a new instance of our Fastify app, and then using the `inject` method from the Fastify API to make a request to the `/` route. We import our test utilities directly from the node. Notice we can pass async functions to our test to use async/await. Node’s assert API has been around for a long time, this is what we are using to make our test assertions. To run this test, we can use the following command: By default the Node.js test runner uses the TAP reporter. You can configure it using other reporters or even create your own custom reporters for it to use. Testing our SQL plugin Next, let's take a look at how to test our Fastify Postgres plugin. This one is a bit more involved and gives us an opportunity to use more of the test runner features. In this example, we are using a feature called Subtests. This simply means when nested tests inside of a top-level test. In our top-level test call, we get a test parameter t` that we call methods on in our nested test structure. In this example, we use `t.beforeEach` to create a new Fastify app instance for each test, and call the `test` method to register our nested tests. Along with `beforeEach` the other methods you might expect are also available: `afterEach`, `before`, `after`. Since we don’t want to connect to our Postgres database in our tests, we are using the available Mocking API to mock out the client. This was the API that I was most excited to see included in the Node Test Runner. After the basics, you almost always need to mock some functions, methods, or libraries in your tests. After trying this feature, it works easily and as expected, I was confident that I could get pretty far testing with the new Node.js core API’s. Since my plugin only uses the end method of the Postgres driver, it’s the only method I provide a mock function for. Our second test confirms that it gets called when our Fastify server is shutting down. Additional features A lot of other features that are common in other popular testing frameworks are also available. Test styles and methods Along with our basic test` based tests we used for our Fastify plugins - `test` also includes `skip`, `todo`, and `only` methods. They are for what you would expect based on the names, skipping or only running certain tests, and work-in-progress tests. If you prefer, you also have the option of using the describe` → `it` test syntax. They both come with the same methods as `test` and I think it really comes down to a matter of personal preference. Test coverage This might be the deal breaker for some since this feature is still experimental. As popular as test coverage reporting is, I expect this API to be finalized and become stable in an upcoming version. Since this isn’t something that’s being shipped for the end user though, I say go for it. What’s the worst that could happen really? Other CLI flags —watch` - https://nodejs.org/dist/latest-v20.x/docs/api/cli.html#--watch —test-name-pattern` - https://nodejs.org/dist/latest-v20.x/docs/api/cli.html#--test-name-pattern TypeScript support You can use a loader like you would for a regular node application to execute TypeScript files. Some popular examples are tsx` and `ts-node`. In practice, I found that this currently doesn’t work well since the test runner only looks for JS file types. After digging in I found that they added support to locate your test files via a glob string but it won’t be available until the next major version release. Conclusion The built-in test runner is a lot more comprehensive than I expected it to be. I was able to easily write some real-world tests for my application. If you don’t mind some of the features like coverage reporting being experimental, you can get pretty far without installing any additional dependencies. The biggest deal breaker on many projects at this point, in my opinion, is the lack of straightforward TypeScript support. This is the test command that I ended up with in my application: I’ll be honest, I stole this from a GitHub issue thread and I don’t know exactly how it works (but it does). If TypeScript is a requirement, maybe stick with Jest or Vitest for now 🙂...

Nov 29, 2023

5 mins

NodeJSTypeScriptTesting