Javascript

Configure your project with Drizzle for Local & Deployed Databases

Mar 8, 2024

6 min read

Updated April 8, 2024: Thanks to Jassi Bacha for pointing out that the migrate function is not exported from the drizzle-orm/postgres-js/migrator module for Vercel. I've updated the article to reflect the correct import path.

It was a fun Friday, and Jason Lengstorf and I both decided to try and use Drizzle on our respective projects. Jason went the SQLite route and wrote an amazing article about how he got his setup working. My approach was a bit different. I started with Vercel's Postgres + Drizzle Next.js Starter and wanted to use PostgreSQL. If you don't know what Drizzle is, it's a type-safe ORM similar to Prisma. My colleague, Dane Grant, wrote a great intro post on it, so go check his article out if you want to learn more about Drizzle.

Getting my project off the ground took longer than I expected, especially coming from a starter kit, but I figured it out. This is the article I wish I had at the time to help get this project set up with less friction. I will focus on using local and Vercel PostgreSQL, but this same setup should work with other databases and adapters. I'll make some notes about where those places are. While I did use Next.js here, these setup instructions work on other projects, too.

Configuring Drizzle

Every project that leverages Drizzle requires a drizzle.config in the root. Because I'm leveraging TypeScript, I named mine drizzle.config.ts, and to secure secrets, I also installed dotenv. My final file appeared as follows:

import type { Config } from "drizzle-kit";
import * as dotenv from "dotenv";

dotenv.config();

export default {
  schema: "./db/schema.ts",
  out: "./db/migrations",
  driver: "pg",
  dbCredentials: {
    connectionString: process.env.POSTGRES_URL as string,
  },
} satisfies Config;

The schema field is used to identify where your project's database schema is defined. Mine is in a file called schema.ts but you can split your schema into multiple files and use glob patterns to detect all of them.

The out field determines where your migration outputs will be stored. I recommend putting them in a folder in the same directory as your schema to keep all your database-related information together.

Additionally, the config requires a driver and dbCredentials.connectionString to be specified so Drizzle knows what APIs to leverage and where your database lives. For the connectionString, I'm using dotenv to store the value in a secret and protect it. The connectionString should be in a valid connection format for your database. For PostgreSQL, this format is postgresql://<username>:<password>@<hostname>:<port>/<database_name>.

Getting your connection string

Now, you may be wondering how to get that connection string. If you're hosting on Vercel using their Postgres offering, you need to go to your Vercel dashboard and select their Postgres option and attach it to your app. This will set environment variables for you that you can pull into your local development environment. This is all covered in their "Getting Started with Vercel Postgres" guide. If you use a different database hosting solution, they'll offer similar instructions for fetching your connection string. drizzle screenshot However, I wanted a local database to modify and blow away as needed for this project. Out of the box, Drizzle does not offer a database initialization command and I needed something that could be easily and quickly replicated across different machines. For this, I pulled in Docker Compose and set up my docker-compose.yaml as follows:

services:
  postgres:
    image: "postgres:15.2-alpine"
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: my-local-db
    ports:
      - "5432:5432"

The 3 most important values to note here are the values in the environment key and the ports. These are what allowed me to determine my connection key. For this example, it would be: postgresql://postgres:postgres@localhost:5432/my-local-db. With the compose file set, I ran docker-compose up -d to get the container running, which also initializes the database. Now, we can connect and operate on the database as needed.

Creating the database connection

To make operations in our app, we need to get a database connection instance. I put mine in db/drizzle.ts to keep all my related database files together. My file looks like:

import { sql } from "@vercel/postgres";
import postgres from "postgres";
import {
  drizzle as VercelDrizzle,
  type VercelPgDatabase,
} from "drizzle-orm/vercel-postgres";
import {
  drizzle as LocalDrizzle,
  type PostgresJsDatabase,
} from "drizzle-orm/postgres-js";

let db:
  | VercelPgDatabase<Record<string, never>>
  | PostgresJsDatabase<Record<string, never>>;
if (process.env.NODE_ENV === "production") {
  db = VercelDrizzle(sql);
} else {
  const migrationClient = postgres(process.env.POSTGRES_URL as string);
  db = LocalDrizzle(migrationClient);
}

export { db };

This is a bit more complicated because we're using 2 different Drizzle adapters depending on our environment. For local development, we're using the generic PostgreSQL adapter, but for production, we're using the Vercel adapter. While these have different initializers, they have the same output interface which is why this works. The same wouldn't be true if you used MySQL locally and PostgreSQL in production. If we chose a RDS or similar PostgreSQL solution, we could use the same postgres adapter in both cases but change the connection string. That's all this file does at the end - detects which environment and uses the chosen adapter.

If we go to use this exported instance, it won't be able to find our tables or provide type safety. This is because we haven't created our database tables yet.

Creating database tables

To get our database tables created, we're going to leverage Drizzle's Migrations. This allows us to create atomic changes to our database as our schema evolves. To accomplish this, we define the schema changes in our schema files as specified in our config. Then we can run npm run drizzle-kit generate:pg (or whatever script runner you use) to generate the migration SQL file that will be located where we specified in our config. You want to check this file into source! By default, Drizzle doesn't allow you to override migration names yet (they're working on it!) so if you want to make your migration file more descriptive, you need to take both of these steps:

Rename the migration file. Take note of the old name.
Locate _journal.json. It should be in your migration folder in a folder called meta. From here, find the old file name and replace it with the new file name.

Now, we need to run the migrations. I had some issues with top-level awaits and tsx like the Drizzle docs recommend, so I had to go a slightly different route and I'm not thrilled about it still. I made a file called migrate.mts that I stored next to my drizzle.ts. In theory, I should have been able to import my drizzle connection instance here and use that, but I ran out of time to figure it out and ended up repeating myself across files. Here's the file:

import "dotenv/config";
import { migrate as LocalMigrate } from "drizzle-orm/postgres-js/migrator";
import { migrate as VercelMigrate } from "drizzle-orm/vercel-postgres/migrator";
import postgres from "postgres";
import { sql } from "@vercel/postgres";
import { drizzle as VercelDrizzle } from "drizzle-orm/vercel-postgres";
import { drizzle as LocalDrizzle } from "drizzle-orm/postgres-js";

// Could not import from drizzle.ts due to mts v ts compatibility issues
let db;
if (process.env.NODE_ENV === "production") {
  db = VercelDrizzle(sql);
  await VercelMigrate(db, { migrationsFolder: "./db/migrations" });
} else {
  const migrationClient = postgres(process.env.POSTGRES_URL as string, {
    max: 1,
  });
  db = LocalDrizzle(migrationClient);
  await LocalMigrate(db, { migrationsFolder: "./db/migrations" });
  await migrationClient.end();
}

Here, I'm connecting to the correct database pending environment then running the drizzle migrate command. For local development, I set my connection pool to max at 1. This probably isn't necessary for this use case, but when connecting to a cluster, this is a recommended best practice from the Drizzle team. For the local case, I also had to close the connection to the db when I was done. For both cases, though, I had to specify the migrations folder location. I could probably DRY this up a bit, but hopefully, the Drizzle team will eliminate this need and use the config to set this value in the future.

With the above file set and our schema generated, we can now run npm run tsx db/migrate.mts and our database will have our latest schema. We can now use the db client to fetch and store data in our database.

Note: Jason uses the push command here. This is fine for an initial database creation, but it will override tables in the future. The migration path is the recommended pattern for non-destructive database updates.

Conclusion

Congratulations! We can connect to our database and perform CRUD operations against our tables. We can also use Drizzle Studio to modify and inspect our data. To review, we had to:

Setup a local PostgreSQL server via a tool like Docker Compose
Configure the database adapter to work in local mode
Generate a schema
Create a script to execute migrations so our database is aligned with our schema

This was my first experience with Drizzle, and I enjoyed its SQL-like interfaces which made it easy to quickly prototype my project. I noticed in their Discord that they're about to have a full-time maintainer so I'm excited to see what their future looks like. I hope you enjoy it too!

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

Dustin Goodman

Engineering Manager with a passion for web and application development. Speaker and writer on work experiences and software development. Dog dad and musician fueled by coffee.

@dustinsgoodman @dustinsgoodman

Intro to EdgeDB - The 10x ORM

Intro to EdgeDB - The 10x ORM I’ve written a couple of posts recently covering different TypeScript ORMs. One about Prisma, and another about Drizzle. ORM’s are a controversial topic in their own right - some people think they are evil, and others think they are great. I enjoy them quite a bit. They make it easy to interact with your databases. What is more important and magical for an application than data? SQL without an ORM is amazing as well, but there are some pain points with that approach. Today I’m excited to write about EdgeDB, which isn’t _exactly_ an ORM or a database from my perspective (although they call themselves one). It is, however an incredibly impressive piece of technology that solves these common pain points in a pretty novel way. So if it’s not an ORM or a database, what exactly is it? I don’t think I can answer that in one or two sentences, so we will explore the various pieces that make up EdgeDB in this article. From a high-level standpoint, though, an interface/query language that sits in front of PostgreSQL. This may seem like some less important implementation detail, but in my eyes, it’s a feature and one of the most compelling selling points. Data Modeling EdgeDB advertises itself as a “graph-relational database”. The core components of EdgeDB data modeling are the schema, type system, and relationship definitions. A schema will consist of objects that contain various typed attributes and links that connect the objects. In SQL, a table is analogous to an Object, and a foreign-key is associated with a link. Here’s what a simple schema in EdgeDB looks like ` There’s a few things to highlight here * We defined two different objects (tables) User and Post * Each object contains properties with their types defined * str is one of several scalar types (bool, int, float, json, datetime, etc) * the author property is a required link to the User object Defining relations / associations In our example above we defined a one-to-many relationship between a user and posts. All the relation types that you can define in traditional SQL are available. One interesting feature though is called backward links. These can be defined in your schema and it allows you to access related data from both sides of a relationship. ` likes are a many-to-many relationship between Tweet and User. With a backlink defined multi link likers := Tweet.<likes[is User]; - we can access likes from a User and likers from a Tweet. ` That's how we can access these relations in our queries. You might be looking at these queries and thinking it looks a lot like GraphQL. This is why they call it a ‘Graph-relational’ database. We’ve only scratched the surface of EdgeDB schema’s. Hopefully, I’ve at least managed to pique your interest. Computed properties Computed properties are a super powerful feature that can be added to your schema or queries. This example user schema creates a computed discriminator and username property. The discriminator uses an EdgeDB standard library function to generate a discriminator number and the username property is a combination between the name and discriminator properties. ` Globals and Access Policies EdgeDB allows you to define global variables as part of your schema. The most common use case I’ve seen for this is to power the access policy feature. You can define a global variable as part of your schema: global current_user: uuid; With a global variable defined, you can provide the value as a sort of context from your application by providing it into your EdgeDB driver/client. ` You can then add access policies directly to your schema, for example, to provide fine-grained access control to blog posts in your blogging application. ` Aside from access policies, you can use your global variables in your queries as well. For example, if you wanted a query to select the current user. ` Types and Sets EdgeDB is very type-centric. All of your data is strongly typed. We’ve touched on some of the types already. * Scalars - There are a lot of scalar types available out of the box * Custom scalars - Custom scalar types are user-defined extensions of existing types * Enums - Are supported out of the box - enum<Admin, Moderator, Member> * Arrays - Defined by passing the singular value type - array<str>; * Tuples - In EdgeD, tuples can contain more than 2 elements and come in named and unnamed varieties - <str, number>; tuple<name: str, jersey_number: float64, active: bool>; All queries return a Set which is a collection of values of a given type. In the query language, all values are Sets. Sets are a collection of values of a given type. A comma-separated list of values inside a set of {curly braces}. A query with no results will return an empty or singleton set. If we have no User values stored yet - select User returns {}. Paired with the query language types and set provides an incredibly powerful and expressive system for interacting with your data. If you thought TypeScript was cool, wait until you start writing EdgeQL! 🙂 EdgeQL Now to the fun stuff: the query language. We’ll use a schema from the docs and start by looking at some of those queries and build on those. The example schema has an abstract type Person with two sub-types based on it Hero and Villian. This is known as a polymorphic type in EdgeDB. The Movie type includes a 1:m association with Person ` Selecting properties / data Before we dig into some real queries we should just touch on how we select actual data from a query. It’s pretty obvious and GraphQL-like but worth mentioning. To specify properties to select, you attach a shape. This works for getting nested link / association data as well. Based on our schema, here’s how we could select fields from Movie, including data from the collection of related characters. ` There is also a feature called a splats that allows you to select all fields and/or all linked fields without specifying them individually. ` If you don’t specify any properties or splats, only id’s get returned select Movie; . Adding some objects with insert To get started, we can use insert to add objects to our database. We’ll start big by looking at the nested insert example. This example is interesting because it shows the creation of two objects in a single query. You’ll notice the simplicity of the syntax. Even though this is the first EdgeQL query we’re looking at, in my experience, it’s like this across the board. I’ve found EdgeQL queries to be simple and intuitive to the point where I’ve been able to intuit how to accomplish things in my head without having to reference the docs or ask the AI. This example adds a new Villian and a new Hero which gets assigned as a link or association to the nemesis field on our Villian. To accomplish this we see that we can nest queries by wrapping them in (). ` The next example is pretty similar, but instead of creating the linked property, we are select ing and adding several potential objects to the characters list of Movie since it is a multi link. This is a pretty complex query that is doing a lot of different things. It’s deceivingly succinct. To accomplish the same thing with SQL this would probably be about 3 different queries. This query finds the objects to add to the characters multi link by filtering on a collection of different strings to match against the name property. ` The last thing we’ll cover for insert is bulk inserts. This is particularly useful for things like seed scripts. In this example, you can just imagine that you have a JSON array of objects with hero names that gets passed in as an argument to your query ` Querying data with select We’ve already seen subqueries and a select in the last section where we found a collection of Person records with a filter. We’ll build on that and see what tools are available to us when it comes to querying data. This one covers a lot of ground. Very similar to SQL we have order and limit, and offset operators to support sorting and pagination. Also there is a whole standard library of functions and operators like count that can be used in our queries. This example returns a collection of villian names, excluding the first and last result. ` Most commonly, you will want to filter by an id ` Here’s another common example filtering by datetime. Since we’re using a string value here we need to cast it to the EdgeDB datetime type. ` You get a pretty similar toolbox to SQL when it comes to filtering with your common operators and things. Combined with all the tools in the standard library, you can get pretty creative with it. Changing values and links with update The update..filter..set statement is how we can update existing data with EdgeQL. set is followed by a shape with assignments of properties to be updated. ` You can replace links for an object ` or add additional ones ` An even more interesting example is removing links matched on a type. Since Villian is a sub-type of Person , this query will remove all characters linked of the Villian type. ` Deleting objects with delete Deleting is pretty straight forward. Using the delete command you can just filter for the objects that you would like to remove. ` When the EdgeQL pieces fall into place As you become more familiar with the EdgeQL query language chances are you’ll start writing very complex queries fluently because everything just makes sense once you’ve learned the building blocks. Domain and business concerns I don’t think they explicitly mention this as a goal anywhere but it’s something that I picked up on pretty quickly. EdgeDB nudges you to move more of what might have traditionally been application logic into your database layer. This is a topic that can bring a lot of division since even things like foreign keys and constraints in SQL are frowned upon in some circles. EdgeDB goes as far as providing constraints, global variables, contexts, and authorization support built into the database. I think that the ability to bake some of these concerns into your EdgeDB Schema is great. The way you model your schema and database in EdgeDB map to your domain in a much more intuitive way where domain concerns don’t really feel out of place there. Database Clients and Query Builders and Generators We’ve covered a lot so far to highlight what EdgeDB is and how to handle common use cases with the query language. To use it in your project though, you will need a client/driver library. There are clients available in several different languages. The one that they clearly have put the most investment into is the TypeScript query builder. We’ll briefly look at both options: simple driver/client and query builder. Whichever you end up choosing you will need to instantiate a driver and make sure you have a connection to your database instance configured. Basic client Although the TS query builder is very popular and pretty amazing, I couldn’t get away from just writing EdgeQL queries. In my application, I composed queries using template strings, and it worked great. The clients all have a set of methods available for passing in EdgeQL queries and parameters. querySingle is a method for queries where you are only expecting a single result. If your query will have multiple results you would use query instead. There is also a queryRequiredSingle which will throw an error if no results are found. There are some other methods available as well including one for running queries in a transaction ` The first argument is the query, and the second is a map of parameters. In this example we include the title parameter and it is accessed in our query via $title. TypeScript query builder If you have a TypeScript app and type-safety is important, you might prefer using the query builder. It is a pretty incredible feat of TypeScript magic initially developed by the same developer behind the popular library Zod. We can’t cover it in very much depth here but we’ll look at an example just to have an idea of what the query builder looks like in an application. ` The query builder is able to infer the result type automatically. It knows which fields you’ve selected, it knows that the result will be a single item. Query generator There are generators for queries and types. So even if you opt out of using the query builder you can still have queries that are strongly typed. It’s nice to have this option if you want to just write your queries as EdgeQL in .edgeql files. ` We end up with an exported function named getUser that is strongly typed. ` Tools and Utilities The team at EdgeDB puts a big emphasis on developer experience. It shows up all over the place. We’ve already seen some utilities with the generators that are available. There are some other tools available as well that help complete the entire experience. EdgeDB CLI The first and most important tool to mention is the CLI. If you’ve started using EdgeDB then you’ve most likely already installed and used it. The CLI is pretty extensive. It includes commands for things like migrations, managing EdgeDB versions and installations, managing projects and local/cloud database instances, dumps and restores, a repl, and more. The CLI makes managing EdgeDB a breeze. Admin UI The CLI includes a command to launch an admin UI for any project or database. The Admin UI includes a awesome interactive diagram of your database schema, a repl for running queries, and a table to inspect and make changes to the data stored in your database. Summary Adopting newer database technology is a tough sales pitch. Replacing your application’s database technology at any point in its lifecycle is not a problem that anyone wants to have. This is one of the reasons why EdgeDB being a superset of PostgreSQL is a huge feature in my opinion. The underlying database technology is tried and true, and EdgeDB is open-source. Based on this, I would feel confident using EdgeDB if it aligned well from a technical and business perspective. We’ve covered a lot of ground in this post. EdgeDB is feature-packed and powerful. Databases is a tough nut to crack, and I commend the team for all their hard work to help continue pushing forward one of the most important components of almost any application. I’m typically pretty conservative when it comes to databases, but EdgeDB took a great approach, in my opinion. I recommend at least giving it a try. You might catch the EdgeDB bug like I did!...

Apr 26, 2024

11 mins

JavaScriptEdgeDB

JSR - The cross-platform package manager for ESM

JSR - The cross-platform package manager for ESM Move over NPM, there’s a new package manager in town. JSR is a new open-source package registry for JavaScript and TypeScript. This particular release has caught my attention, and I’m excited to explore if and where JSR fits into our overflowing ecosystem. Setting the stage The JSR website has a page called Why JSR? I want to explore the _why_ a little bit further. I think that the _why_ points to JSR potentially being the future of the JavaScript ecosystem. NPM and JavaScript modules If you’ve found yourself confused about publishing and consuming packages and modules in the modern ESM era, you’re not alone. When NodeJS was created, there was no standard for JS modules. Node introduced CommonJS modules, and NPM provided a way to publish and consume these modules in our applications. It’s been years now since ES Modules became a standard and we are still in a weird module purgatory with no clear end in sight. JSR represents a fresh start, free from the baggage of the Node and NPM ecosystems. JavaScript runtimes and interoperability New JavaScript runtimes are popping up every year. WinterCG (_Web-interoperable Runtimes Community Group)_ was started to improve interoperability for the various runtimes with some standard APIs. Node has made steps towards this goal, but it might never get all the way there and NPM is coupled pretty tightly to that ecosystem. JSR supports several runtimes (depending on the package). This is a huge opportunity for it to become the package manager for the ecosystem at large. See cross-runtime support section for more details. NPM Compatibility JSR doesn’t end up being a complete departure from NPM since it includes an NPM compatibility layer. The docs highlight some limitations and provide deeper technical details on how this piece works. The simple overview is that you can install and use packages from JSR to a Node/NPM based project. The packages will get added to your package.json and node_modules directory. About the same as any package installed from NPM. JSR Overview At a high level, JSR is very similar to NPM. You can search for packages on the website. You can publish packages to their registry, and you can download packages from it as well. We touched on the two major features that set it apart from NPM: It only supports ESModules, and it has cross-runtime support. In the rest of the article we’ll explore the essentials (package management) and also some other really great features that feel like are bringing the JS package management ecosystem up to speed with our modern workflows. Native TypeScript I have to say I’m not surprised by this feature, considering JSR was developed by the same people behind Deno (a runtime that supports TS natively). It’s a great feature. If you haven’t figured it out yet TypeScript is a big deal and doesn’t seem to be fading away anytime soon. So what exactly does TS support look like in a package manager, though? If your runtime supports TypeScript natively (like Deno) - it can consume the TS files in your package directly. For environments that don’t, it will automatically compile your code to JavaScript and package it with type declaration files (.d.ts). Pretty, stinking, cool. The docs include a page called “about slow types”. It is probably worth a read on its own ,but the TLDR is that JSR will analyze your TS code and penalize you in certain ways if you have some type code that it considers to be slow. On the page, it outlines exactly what these penalties are, as well as what it considers slow types to be. Slow types will also come into play when we get into package scoring later in the article. Packages The docs have a page dedicated to packages. It covers some important stuff like deleting packages, versioning, documentation, and publishing. We’ll touch on some of it with an emphasis on the things that are done differently or better than with NPM. jsr.json file jsr.json is a configuration file that specifies a name, version, and exports of a package. It doesn’t include a list of dependencies like you would have in a package.json file. This is a configuration file for a package to be published to JSR. JSR dependencies are defined in an import map file or a package.json file, depending on the runtime. Adding packages to a project Adding packages to your project will vary depending on your target runtime. Here’s an example of installing it for Deno and NPM using various package managers. For Deno, an import map entry is created. For NPM, it gets installed as an NPM package using the JSR NPM compatibility layer. ` If the runtime has native JSR support, you don’t need to explicitly install packages. You can important them using the jsr: scheme. ` As you can see semantic versioning is supported and works similarly to NPM. Publishing packages If you want to publish a package to JSR, there are many important rules you need to be aware of. I recommend reading “Publishing packages” before moving forward with this. The main things to note are: ESModules only, npm packages are supported, JSR packages are supported, and node APIs are supported. So there aren’t many constraints. You just need to understand how exactly to do these things. IE: how do I use NPM dependencies, how do the imports work, etc? Once you’ve got all that down, you will be ready to publish your first package to JSR 😃 Documentation JSR emphasizes quality package documentation. It’s one of the factors in their package scoring algorithm, which we will discuss shortly. The important parts of a package’s documentation include: * a README.md file * symbol documentation - functions, interfaces, classes, etc * module documentation - docs for each exported module in a package The symbol and module documentation are written using JSDoc. The JSR website generates some nice documentation based on these 3 pieces on your package page. This means you can understand a package's entire public API without ever leaving jsr.io. Browser support The cross-runtime support is amazing, but I was curious where the original JavaScript runtime fit (the browser). According to the documentation: “You can use JSR with any tool that supports ES modules, and either has native support for JSR or supports npm packages using node_modules.” Since modern web browsers support ES modules, I will count this as JSR support. From what I can tell, though, there is one caveat. To include a module on a web page, we need a URL to download it from. Can we publish our package to JSR and use it to load a module via a script tag? According to the JSR usage policy, no. The “Unacceptable use” section states: “It is not acceptable to use JSR as a CDN for serving assets directly to web applications in a browser, except for development purposes.” So, it seems that this is a no-go for now. We will probably need to wait a bit for CDN services like jsdelivr to pull packages from the JSR registry, similar to what they do with NPM currently. Other notable features We’ve covered the most important bits about package management, TS support, NPM compatibility, etc. But there are still a few unique features that add an extra bit of sparkle to JSR. Scoring We mentioned earlier that documentation plays a part in a scoring algorithm. JSR analyzes all of the packages published to its registry and assigns them a score based on certain criteria: documentation, best practices, discoverability, and compatibility. This score is shown with every package listed on the website. You can also add a shiny badge to your README if you’re particularly proud of something. It’s still early on, so it’s likely this will evolve as time goes on. It’s an interesting gimmick, at the very least! Summary I am excited to have a new cross-platform compatible package manager that only supports ESM. As useful as NPM has been all these years, it carries a lot of baggage from “the olden days”. JSR’s native TypeScript support and reliance on modern standards is a breath of fresh air. It feels like I can publish a simple package without much ceremony. I’m excited to release my first package to JSR soon :)...

Apr 12, 2024

6 mins

JSRJavaScript

Demystifying React Server Components

React Server Components (RSCs) are the latest addition to the React ecosystem, and they've caused a bit of a disruption to how we think about React. Dan Abramov recently wrote an article titled "The Two Reacts" that explores the paradigms of client component and server component mental models and leaves us with the thought on how we can cohesively combine these concepts in a meaningful way. It took me a while to finally give RSCs the proper exploration to truly understand the "new" model and grasp where React is heading. First off, the new model isn't really new so much as it introduces a few new concepts for us to consider when architecting our applications. Once I understood the pattern, I found an appreciation for the model and what it's trying to help us accomplish. In this post, I hope I can show you the progression of the React architecture in applications as I've experienced them and how I think RSCs help us improve this model and our apps. A "Brief" History of React Rendering and Data-Fetching Patterns One of the biggest challenges in React since its early days is "how do we server render pages?" Server-side rendering (SSR) is one of the techniques we can use to ensure users see data on initial load and helps with our site's SEO. Without SSR, users would see blank screens or loading spinners, and then a bunch of content would appear shortly after. An excellent graphic on Remix's website demonstrates this behavior from the end user's perspective and it's a problem we generally try to avoid as developers. This problem is so vast and difficult that we've been trying to solve it since 2015. Rick Hanlon from the React Core Team reminded us just how complicated this problem was recently. If you think react is complicated now, go back to 2015 when you had to configure babel and webpack yourself, you had to do SSR and routing yourself, you had to decide between flow and typescript, you had to learn flux and immutable.js, and you had to choose 1 of 100 boilerplates— Ricky (@rickhanlonii) January 28, 2024 But SSR has its issues too. Sometimes SSR is slow because we need to do a lot of data fetching for our page. Because of these large payloads, we'll defer their rendering using lazy loading patterns. All of a sudden we have spinners again! How we've managed these components has changed too. Over the years, we've seen a variety of patterns emerge for managing these problems. We had server-side pre-fetching where we had to hydrate our frontends application state. Then we tried controller-view patterned components for our lazily loaded client-side components. With the evolution of React, we were able to simplify the controller-view patterns to leverage hooks. Now, we're in a new era of multi-server entry points on a page with RSCs. The Benefits of React Server Components RSCs give us this new paradigm that allows us to have multiple entries into our server on a single page. Leveraging features like Next.js' streaming mode and caching, we can limit what our pages block on for SSR and optimize their performance. To illustrate this, let's look at this small block of PHP for something we might have done in the 2000s: index.php ` For simplicity, the blocks indicate server boundaries where we can execute PHP functionality. Once we exit that block, we can no longer leverage or communicate with the server. In this example, we're displaying a welcome message to a user and a list of posts. If we look at Next.js' page router leveraging getServerSideProps, we would maybe write this same page as follows: pages/posts.jsx ` In this case, we're having our server do a lot to fetch the data we need to render this page and all its components. This also makes the line between server and client much clearer as getServerSideProps runs before our client renders, and we're unable to go back to that function without an API route and client-side fetch. Now, let's look at Next.js' app router with server components. This same component could be rendered as follows: app/posts/page.tsx ` This moves us a bit closer back to our PHP version as we don't have to split our server and client functionality as explicitly. This example is relatively simple and only renders a few components that could be easily rendered as static content. This page also probably requires all the content to be rendered upfront. But, from this, we can see that we're able to simplify how server data fetching is done. If nothing else, we could use this pattern to make our SSR patterns better and client render the rest of our apps, but then we'd lose out on some of the additional benefits that we can get when we combine this with streaming. Let's look at a post page that probably has the content and a comments section. We don't need to render the comments immediately on page load or block on it because it's a secondary feature on the page. This is where we can pull in Suspense and make our page shine. Our code might look as follows: app/posts/[slug]/page.jsx ` Our PostComments are a server component that renders static content again, but we don't want its render to block our page from being served to the client. What we've done here is moved the fetch operation into the comments component and wrapped it in a Suspense boundary in our page to defer its render until its content has been fetched and then stream it onto the page into its render location. We could also add a fallback to our suspense boundary if we wanted to put a skeleton loader or other indication UI. With these changes, we're writing components similarly to how we've written them on the client historically but simplified our data fetch. What happens when we start to progressively enhance our features with client-side JavaScript? Client Components with RSCs All our examples focused on staying in the server context which is great for simple applications, but what happens when we want to add interactivity? If you were to put a useState, useEffect, onClick, or other client-side React feature into a server component, you'd find some error messages because you can't run client code from a server context. If we think back to our PHP example, that makes a lot of sense why this is the case, but how do we work around this? For me, this is where my first really mental challenge with RSCs started. Let's use our PostComments as an example to enhance by in-place sorting the section from the server. ` In this example, we're using a server component the same way as our previous example to do the initial data fetch and stream when ready. However, we're immediately passing the results to a client component that renders the elements and enhances our code with a list of sort options. When we select the sort we want, our code makes a request to the server at a predefined route and gets the new data that we re-render in the new order on screen. This is how we might have done things without RSCs before but without a useEffect for our initial rendering. If you're familiar with the old controller-view pattern, this is relatively similar to that pattern, but we have to relegate client re-fetching to the view (client) component, where we might have had all fetch and re-fetch patterns in the controller (server) component Another way to solve this same problem is leveraging server actions, but that would cause the page to re-render. Given the caching mechanisms in Next.js, this is probably fine, but it's not the user experience people are expecting. Server actions are a topic of their own, so I won't cover them in this post, but they're important for the holistic ecosystem experience in the new mindset. Interleaving Nested Server & Client Components These examples have shown a clean approach where we keep our server components at the top of our rendering tree and have a clear line where we move from server mode to client mode. But one of the advantages of RSCs is that we can interleave where our server component entry points may exist. Let's think about a product carousel on a storefront page for example. We may have built this as follows before: ` Here we're rendering carousels that render their cards and our tree goes server -> client -> server. This is not allowed in the new paradigm because the React compiler cannot detect that we moved back into a server component when we called ProductCards from a client component. Instead, we would need to refactor this to be: ` Here, we've changed ProductCarousel to accept children that are our ProductCards server component. This allows the compiler to detect the boundary and render it appropriately. I'd also recommend adding Suspense boundaries to these carousels but I omitted it for the sake of brevity in this example. Some Suggested Best Practices Our team has been using these new patterns for some time and have developed some patterns we've identified as best practices for us to help keep some of these boundaries clear that I thought worth sharing. 1. Be explicit with component types We've found that being explicit with component types is essential to expressing intent. Some components are strictly server, some are strictly client, and others can be used in both contexts. Our team likes using the server-only package to express this intent but we found we needed more. We've opted for the following naming conventions: Server only components: component-name.server.(jsx|tsx) Client only components: component-name.client.(jsx|tsx) Universal components: component-name.(jsx|tsx) This does not apply to special framework file names but we've found this helps us delineate where we are and to help with interleaving. 2. Defer Fetch when Cache is Available If we're trying to render a component that is a collection of other components, we've found deferring the fetch to the child allows our cache to do more for us in rendering in cases where the same element may exist in different locations. This is similar to how GraphQL's data loaders works and can ideally boost the performance of cached server components. So, in our product example above, we may only fetch the IDs of the products we need for the ProductCards and then fetch all the data per card. Yes, this is an extra fetch and causes an N+1, but if our cache is in play, end users will feel a performance gain. 3. Colocate loaders and queries If you're using GraphQL or need loading states for components for Suspense fallback, we recommend colocating those elements in the same file or directory. This makes it easier to modify and manage related elements for your components in a more meaningful way. 4. Use Suspense to defer non-essential content This will depend on your website and needs. Still, we recommend deferring as many non-essential elements to Suspense as possible. We define non-essential to be anything you could consider secondary to the page. On a blog post, this could be recommended posts or comments. On a product page, this could be reviews and related products. So long as the primary focus of your page is outside a Suspense boundary, your usage is probably acceptable. Conclusion RSCs represent a significant evolution in the React ecosystem, offering new paradigms and opportunities for improving the architecture of web applications. They enable multiple server entry points on a single page, optimizing server-side rendering, and simplifying data fetching. RSCs allow for a clearer separation between server and client functionality, enhancing both performance and maintainability. To make the most of RSCs, developers should consider best practices such as being explicit with component types, deferring fetch when caching is available, colocating loaders and queries, and using Suspense to defer non-essential content. Embracing these practices can help harness the potential of React Server Components and pave the way for more efficient and interactive web applications. We're excited about this development in the React ecosystem and the developments happening across teams and frameworks that are opting into using them. While Next.js offers a great solution, we're excited to see similar enhancements to Remix and RedwoodJS....

Feb 2, 2024

9 mins

ReactNextJSArchitecture

Transforming Auth in an AI World with Rod Boothby

In today's digital landscape, the need for secure identity verification has become paramount. With the increasing risks of personal data exposure and the rise of sophisticated cyber threats aided by Artificial Intelligence, it is crucial to adopt robust verification processes to protect individuals and organizations alike. In a recent discussion with Rod Boothby, CEO of ID Partner Systems, the significance of trusted institutions for identity verification was emphasized, particularly the efficiency of bank-based ID verification over traditional methods. One of the key takeaways from the conversation was the importance of continuous authentication. As technology advances, so do the methods employed by cybercriminals. Deepfake technology, for instance, poses a significant threat to identity verification systems. To combat this, tighter security measures and continuous authentication are essential. By constantly verifying and validating user identities, organizations can stay one step ahead of potential fraudsters. Rod Boothby also highlighted the need for a developer-focused approach to identity verification. By providing developers with the tools and resources they need, companies like ID Partner Systems aim to streamline the verification process and enhance security. This approach not only ensures a more efficient experience for users but also allows for the integration of behavioral biometrics, which can further strengthen the verification process. Download this episode here....

Apr 22, 2024

1 min

Engineering Leadership

Configure your project with Drizzle for Local & Deployed Databases

Configuring Drizzle

Getting your connection string

Creating the database connection

Creating database tables

Conclusion

Dustin Goodman

You might also like

Intro to EdgeDB - The 10x ORM

JSR - The cross-platform package manager for ESM

Demystifying React Server Components

Transforming Auth in an AI World with Rod Boothby

You might also like

Intro to EdgeDB - The 10x ORM

JSR - The cross-platform package manager for ESM

Demystifying React Server Components

Transforming Auth in an AI World with Rod Boothby