GraphQL

Leveraging GraphQL Scalars to Enhance Your Schema

Nov 8, 2023

4 min read

Introduction

GraphQL has revolutionized the way developers approach application data and API layers, gaining well-deserved momentum in the tech world. Yet, for all its prowess, there's room for enhancement, especially when it comes to its scalar types. By default, GraphQL offers a limited set of these primitives — Int, Float, String, Boolean, and ID — that underpin every schema. While these types serve most use cases, there are scenarios where they fall short, leading developers to yearn for more specificity in their schemas.

Enter graphql-scalars, a library designed to bridge this gap. By supplementing GraphQL with a richer set of scalar types, this tool allows for greater precision and flexibility in data representation. In this post, we'll unpack the potential of enhanced Scalars, delve into the extended capabilities provided by graphql-scalars, and demonstrate its transformative power using an existing starter project. Prepare to redefine the boundaries of what your GraphQL schema can achieve.

Benefits of Using Scalars

GraphQL hinges on the concept of "types." Scalars, being the foundational units of GraphQL's type system, play a pivotal role. While the default Scalars — Int, Float, String, Boolean, and ID — serve many use cases, there's an evident need for more specialized types in intricate web development scenarios.

Precision: Using default Scalars can sometimes lack specificity. Consider representing a date or time in your application with a String; this might lead to ambiguities in format interpretation and potential inconsistencies.
Validation: Specialized scalar types introduce inherent validation. Instead of using a String for an email or a URL, for example, distinct types ensure the data meets expected formats at the query level itself.
Expressiveness: Advanced Scalars provide clearer intentions. They eliminate ambiguity inherent in generic types, making the schema more transparent and self-explanatory.

Acknowledging the limitations of the default Scalars, tools like graphql-scalars have emerged. By broadening the range of available data types, graphql-scalars allows developers to describe their data with greater precision and nuance.

Demonstrating Scalars in Action with Our Starter Project

To truly grasp the transformative power of enhanced Scalars, seeing them in action is pivotal. For this, we'll leverage a popular starter kit: the Serverless framework with Apollo and Contentful. This kit elegantly blends the efficiency of serverless functions with the power of Apollo's GraphQL and Contentful's content management capabilities.

Setting Up the Starter:

Initialize the Project:

npm create @this-dot/starter -- --kit serverless-framework-apollo-contentful

When prompted, name your project enhance-with-graphql-scalars.

Welcome to starter.dev! (create-starter)
✔ What is the name of your project? … enhance-with-graphql-scalars
> Downloading starter kit...
✔ Done!

Next steps:
 cd enhance-with-graphql-scalars
 npm install (or pnpm install, yarn, etc)

For a detailed setup, including integrating with Contentful and deploying your serverless functions, please follow the comprehensive guide provided in the starter kit here.
And we add the graphql-scalars package

npm install graphql-scalars

Enhancing with graphql-scalars:

Dive into the technology.typedefs.ts file, which is the beating heart of our GraphQL type definitions for the project. Initially, these are the definitions we encounter:

export const technologyTypeDefs = gql`
	type Technology {
		id: ID!
		displayName: String!
		description: String
		url: URL
	}

	type Query {
		"Technology: GET"
		technology(id: ID!): Technology
		technologies(offset: Int, limit: Int): [Technology!]
	}

	type Mutation {
		"Technology: create, read and delete operations"
		createTechnology(displayName: String!, description: String, url: String): Technology
		updateTechnology(id: ID!, fields: TechnologyUpdateFields): Technology
		deleteTechnology(id: ID!): ID
	}

	input TechnologyUpdateFields {
		"Mutable fields of a technology entity"
		displayName: String
		description: String
		url: String
	}
`;

Our enhancement strategy is straightforward:

Convert the url field from a String to the URL scalar type, bolstering field validation to adhere strictly to the URL format.

Post-integration of graphql-scalars, and with our adjustments, the revised type definition emerges as:

export const technologyTypeDefs = gql`
	type Technology {
		id: ID!
		displayName: String!
		description: String
		url: URL
	}

	type Query {
		"Technology: GET"
		technology(id: ID!): Technology
		technologies(offset: Int, limit: Int): [Technology!]
	}

	type Mutation {
		"Technology: create, read and delete operations"
		createTechnology(displayName: String!, description: String, url: URL): Technology
		updateTechnology(id: ID!, fields: TechnologyUpdateFields): Technology
		deleteTechnology(id: ID!): ID
	}

	input TechnologyUpdateFields {
		"Mutable fields of a technology entity"
		displayName: String
		description: String
		url: URL
	}
`;

To cap it off, we integrate the URL type definition along with its resolvers (sourced from graphql-scalars) in the schema/index.ts file:

import { mergeResolvers, mergeTypeDefs } from '@graphql-tools/merge';
import { technologyResolvers, technologyTypeDefs } from './technology';
import { URLResolver, URLTypeDefinition } from 'graphql-scalars';

const graphqlScalars = [URLTypeDefinition];

export const typeDefs = mergeTypeDefs([...graphqlScalars, technologyTypeDefs]);

export const resolvers = mergeResolvers([{ URL: URLResolver }, technologyResolvers]);

This facelift doesn't just refine our GraphQL schema but infuses it with innate validation, acting as a beacon for consistent and accurate data.

Testing in the GraphQL Sandbox

Time to witness our changes in action within the GraphQL sandbox. Ensure your local server is humming along nicely.

Kick off with verifying the list query:

query {
  technologies {
    id
    displayName
    url
  },
}

Output:

{
  "data": {
    "technologies": [
      {
        "id": "4UXuIqJt75kcaB6idLMz3f",
        "displayName": "GraphQL",
        "url": "https://graphql.framework.dev/"
      },
      {
        "id": "5nOshyir74EmqY4Jtuqk2L",
        "displayName": "Node.js",
        "url": "https://nodejs.framework.dev/"
      },
      {
        "id": "5obCOaxbJql6YBeXmnlb5n",
        "displayName": "Express",
        "url": "https://www.npmjs.com/package/express"
      }
    ]
  }
}

Success! Each url in our dataset adheres to the pristine URL format. Any deviation would've slapped us with a format error.

Now, let's court danger. Attempt to update the url field with a wonky format:

mutation {
  updateTechnology(id: "4UXuIqJt75kcaB6idLMz3f", fields: { url: "aFakeURLThatShouldThrowError" }) {
    id
    displayName
    url
  }
}

As anticipated, the API throws up a validation roadblock:

{
  "data": {},
  "errors": [
    {
      "message": "Expected value of type \"URL\", found \"aFakeURLThatShouldThrowError\"; Invalid URL",
      "locations": [
        {
          "line": 18,
          "column": 65
        }
      ],
      "extensions": {
        "code": "GRAPHQL_VALIDATION_FAILED",
        "stacktrace": [
          "TypeError [ERR_INVALID_URL]: Invalid URL",
          "    at new NodeError (node:internal/errors:399:5)",
          "    at new URL (node:internal/url:560:13)",
          ...
        ]
      }
    }
  ]
}

For the final act, re-run the initial query to reassure ourselves that the original dataset remains untarnished.

Conclusion

Enhancing your GraphQL schemas with custom scalars not only amplifies the robustness of your data structures but also streamlines validation and transformation processes. By setting foundational standards at the schema level, we ensure error-free, consistent, and meaningful data exchanges right from the start.

The graphql-scalars library offers an array of scalars that address common challenges developers face. Beyond the URL scalar we explored, consider diving into other commonly used scalars such as:

DateTime: Represents date and time in the ISO 8601 format.
Email: Validates strings as email addresses.
PositiveInt: Ensures integer values are positive.
NonNegativeFloat: Guarantees float values are non-negative.

As a potential next step, consider crafting your own custom scalars tailored to your project's specific requirements. Building a custom scalar not only offers unparalleled flexibility but also provides deeper insights into GraphQL's inner workings and its extensibility.

Remember, while GraphQL is inherently powerful, the granular enhancements like scalars truly elevate the data-fetching experience for developers and users alike. Always evaluate your project's needs and lean into enhancements that bring the most value.

To richer and more intuitive GraphQL schemas!

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

William Mimura

@mimurawil

How to create and use custom GraphQL Scalars

How to create and use custom GraphQL Scalars In the realm of GraphQL, scalars form the bedrock of the type system, representing the most fundamental data types like strings, numbers, and booleans. As explored in our previous post, "Leveraging GraphQL Scalars to Enhance Your Schema," scalars play a pivotal role in defining how data is structured and validated. But what happens when the default scalars aren't quite enough? What happens when your application demands a data type as unique as its requirements? Enter the world of custom GraphQL scalars. These data types go beyond the conventional, offering the power and flexibility to tailor your schema to precisely match your application's unique needs. Whether handling complex data structures, enforcing specific data formats, or simply bringing clarity to your API, custom scalars open up a new realm of possibilities. In this post, we'll explore how to understand, create, and effectively utilize custom scalars in GraphQL. From conceptualization to implementation, we'll cover the essentials of extending your GraphQL toolkit, empowering you to transform abstract ideas into concrete, practical solutions. So, let's embark together on the journey of understanding and utilizing custom GraphQL scalars, enhancing and expanding the capabilities of your GraphQL schema. Understanding Custom Scalars Custom scalars in GraphQL extend beyond basic types like String or Int, allowing data to be defined, validated, and processed more precisely. They're instrumental when default types don't quite capture the complexity or specificity of the data, such as with specialized date formats or unique identifiers. The use of custom scalars brings several benefits: * Enhanced Clarity: They offer a clearer representation of what data looks like and how it behaves. * Built-in Validation: Data integrity is bolstered at the schema level. * Flexibility: They can be tailored to specific data handling needs, making your schema more adaptable and robust. With this understanding, we'll explore creating and integrating custom scalars into a GraphQL schema, turning theory into practice. Creating a Custom Scalar Defining a Custom Scalar in TypeScript: Creating a custom scalar in GraphQL with TypeScript involves defining its behavior through parsing, serialization, and validation functions. * Parsing: Transforms input data from the client into a server-understandable format. * Serializing: Converts server data back to a client-friendly format. * Validation: Ensures data adheres to the defined format or criteria. Example: A 'Color' Scalar in TypeScript The Color scalar will ensure that every color value adheres to a valid hexadecimal format, like #FFFFFF for white or #000000 for black: ` In this TypeScript implementation: * validateColors: a function that checks if the provided string matches the hexadecimal color format. * parseValue: a method function that converts the scalar’s value from the client into the server’s representation format - this method is called when a client provides the scalar as a variable. See parseValue docs for more information * serialize: a method function that converts the scalar’s server representation format to the client format, see serialize docs for more information * parseLiteral: similar to parseValue, this method function converts the scalar’s value from the client to the server’s representation format. Still, this method is called when the scalar is provided as a hard-coded argument (inline). See parseLiteral docs for more information In the upcoming section, we'll explore how to incorporate and validate these custom scalars within your schema, ensuring they function seamlessly in real-world scenarios. Integrating Custom Scalars into a Schema Incorporating the 'Color' Scalar After defining your custom Color scalar, the next crucial step is effectively integrating it into your GraphQL schema. This integration ensures that your GraphQL server recognizes and correctly utilizes the scalar. Step-by-Step Integration 1. Add the scalar to Type Definitions: Include the Color scalar in your GraphQL type definitions. This inclusion informs GraphQL about this new scalar type. 2. Resolver Mapping: Map your custom scalar type to its resolver. This connection is key for GraphQL to understand how to process this type during queries and mutations. ` 1. Use the scalar: Update your type to use the new custom scalar ` Testing the Integration With your custom Color scalar integrated, conducting thorough testing is vital. Ensure that your GraphQL server correctly handles the Color scalar, particularly in terms of accepting valid color formats and rejecting invalid ones. For demonstration purposes, I've adapted a creation mutation to include the primaryColor field. To keep this post focused and concise, I won't detail all the code changes here, but the following screenshots illustrate the successful implementation and error handling. Calling the mutation (createTechnology) successfully: Calling the mutation with forced fail (bad color hex): Conclusion The journey into the realm of custom GraphQL scalars reveals a world where data types are no longer confined to the basics. By creating and integrating scalars like the Color type, we unlock precision and specificity in our GraphQL schemas, which significantly enhance our applications' data handling capabilities. Custom scalars are more than just a technical addition; they testify to GraphQL's flexibility and power. They allow developers to express data meaningfully, ensuring that APIs are functional, intuitive, and robust. As we've seen, defining, integrating, and testing these scalars requires a blend of creativity and technical acumen. It encourages a deeper understanding of how data flows through your application and offers a chance to tailor that experience to your project's unique needs. So, as you embark on your GraphQL journey, consider the potential of custom scalars. Whether you're ensuring data integrity, enhancing API clarity, or simply making your schema a perfect fit for your application, the possibilities are as vast as they are exciting. Embrace the power of customization, and let your GraphQL schemas shine!...

Apr 10, 2024

5 mins

GraphQLJavaScript

Introducing the New Serverless, GraphQL, Apollo Server, and Contentful Starter kit

Introducing the new Serverless, GraphQL, Apollo Server, and Contentful Starter kit The team at This Dot Labs has released a brand new starter kit which includes the Serverless Framework, GraphQL, Apollo Server and Contentful configured and ready to go. This article will walk through how to set up the new kit, the key technologies used, and reasons why you would consider using this kit. Table of Contents - How to get started setting up the kit - Generate the project - Setup Contentful access - Setting up Docker - Starting the local server - How to Create the Technology Model in Contentful - How to seed the database with demo data - How to work with the migration scripts - Technologies included in this starter kit - Why use GraphQL? - Why use Contentful? - Why use Amazon Simple Queue Service (SQS)? - Why use Apollo Server? - Why use the Serverless Framework? - Why use Redis? - Why use the Jest testing framework? - Project structure - How to deploy your application - What can this starter kit be used for? - Conclusion How to get started setting up the kit Generate the project In the command line, you will need to start the starter.dev CLI by running the npx @this-dot/create-starter command. You can then select the Serverless Framework, Apollo Server, and Contentful CMS kit and name your new project. Then you will need to cd into your new project directory and install the dependencies using the tool of your choice (npm, yarn, or pnpm). Next, you will need to Run cp .env.example .env to copy the contents of the .env.example file into the .env file. Setup Contentful access You will first need to create an account on Contentful, if you don't have one already. Once you are logged in, you will need to create a new space. From there, go to Settings -> API keys and click on the Content Management Tokens tab. Next, click on the Generate personal token button and give your token a name. Copy your new Personal Access Token, and add it to the CONTENTFUL_CONTENT_MANAGEMENT_API_TOKEN variable. Then, go to Settings -> General settings to get the CONTENTFUL_SPACE_ID. The last step is to add those CONTENTFUL_CONTENT_MANAGEMENT_API_TOKEN and CONTENTFUL_SPACE_ID values to your .env file. Setting up Docker You will first need to install Docker Desktop if you don't have it installed already. Once installed, you can start up the Docker container with the npm run infrastructure:up command. Starting the local server While the Docker container is running, open up a new tab in the terminal and run npm run dev to start the development server. Open your browser to http://localhost:3000/dev/graphql to open up Apollo server. How to Create the Technology Model in Contentful To get started with the example model, you will first need to create the model in Contentful. 1. Log into your Contentful account 2. Click on the Content Model tab 3. Click on the Design your Content Modal button if this is your first modal 4. Create a new model called Technology 5. Add three new text fields called displayName, description and url 6. Save your new model How to seed the database with demo data This starter kit comes with a seeding script that pre-populates data for the Technology Content type. In the command line, run npm run db:seed which will add three new data entries into Contentful. If you want to see the results from seeding the database, you can execute a small GraphQL query using Apollo server. First, make sure Docker, and the local server(npm run dev) are running, and then navigate to http://localhost:3000/dev/graphql. Add the following query: ` When you run the query, you should see the following output. ` How to work with the migration scripts Migrations are a way to make changes to your content models and entries. This starter kit comes with a couple of migration scripts that you can study and run to make changes to the demo Technology model. These migration scripts are located in the scripts/migration directory. To get started, you will need to first install the contentful-cli. ` You can then login to Contentful using the contentful-cli. ` You will then need to choose the Contentful space where the Technology model is located. ` If you want to modify the existing demo content type, you can run the second migration script from the starter kit. ` If you want to build out more content models using the CLI, you can study the example code in the /scripts/migrations/01-create-technology-contentType.js file. From there, you can create a new migration file, and run the above contentful space migration command. If you want to learn more about migration in Contentful, then please check out the documentation. Technologies included in this starter kit Why use GraphQL? GraphQL is a query language for your API and it makes it easy to query all of the data you need in a single request. This starter kit uses GraphQL to query the data from our Contentful space. Why use Contentful? Contentful is a headless CMS that makes it easy to create and manage structured data. We have integrated Contentful into this starter kit to make it easy for you to create new entries in the database. Why use Amazon Simple Queue Service (SQS)? Amazon Simple Queue Service (SQS) is a queuing service that allows you to decouple your components and process and store messages in a scalable way. In this starter kit, an SQS message is sent by the APIGatewayProxyHandler using the sendMessage function, which is then stored in a queue called DemoJobQueue. The SQS handler sqs-handler polls this queue, and processes any message received. ` Why use Apollo Server? Apollo Server is a production-ready GraphQL server that works with any GraphQL client, and data source. When you run npm run dev and open the browser to http://localhost:3000/dev/graphql, you will be able to start querying your Contentful data in no time. Why use the Serverless Framework? The Serverless Framework is used to help auto-scale your application by using AWS Lambda functions. In the starter kit, you will find a serverless.yml file, which acts as a configuration for the CLI and allows you to deploy your code to your chosen provider. This starter kit also includes the following plugins: - serverless-offline - allows us to deploy our application locally to speed up development cycles. - serverless-plugin-typescript - allows the use of TypeScript with zero-config. - serverless-dotenv-plugin - preloads function environment variables into the Serverless Framework. Why use Redis? Redis is an open-source in-memory data store that stores data in the server memory. This starter kit uses Redis to cache the data to reduce the API response times and rate limiting. When you make a new request, those new requests will be retrieved from the Redis cache. Why use the Jest testing framework? Jest is a popular testing framework that works well for creating unit tests. You can see some example test files under the src/schema/technology directory. You can use the npm run test command to run all of the tests. Project structure Inside the src directory, you will find the following structure: ` This given structure makes it easy to find all of the code and tests related to that specific component. This structure also follows the single responsibility principle which means that each file has a single purpose. How to deploy your application The Serverless Framework needs access to your cloud provider account so that it can create and manage resources on your behalf. You can follow the guide to get started. Steps to get started: 1. Sign up for an AWS account 2. Create an IAM User and Access Key 3. Export your AWS_ACCESS_KEY_ID & AWS_SECRET_ACCESS_KEY credentials. ` 4. Deploy your application on AWS Lambda: ` 5. To deploy a single function, run: ` To stop your Serverless application, run: ` For more information on Serverless deployment, check out this article. What can this starter kit be used for? This starter kit is very versatile, and can be used with a front-end application for a variety of situations. Here are some examples: - personal developer blog - small e-commerce application Conclusion In this article, we looked at how we can get started using the Serverless, GraphQL, Apollo Server, and Contentful Starter kit. We also looked at the different technologies used in the kit, and why they were chosen. Lastly, we looked at how to deploy our application using AWS. I hope you enjoy working with our new starter kit!...

Jun 2, 2023

7 mins

GraphQL

Building a Multi-Response Streaming API with Node.js, Express, and React

Introduction As web applications become increasingly complex and data-driven, efficient and effective data transfer methods become critically important. A streaming API that can send multiple responses to a single request can be a powerful tool for handling large amounts of data or for delivering real-time updates. In this article, we will guide you through the process of creating such an API. We will use video streaming as an illustrative example. With their large file sizes and the need for flexible, on-demand delivery, videos present a fitting scenario for showcasing the power of multi-response streaming APIs. The backend will be built with Node.js and Express, utilizing HTTP range requests to facilitate efficient data delivery in chunks. Next, we'll build a React front-end to interact with our streaming API. This front-end will handle both the display of the streamed video content and its download, offering users real-time progress updates. By the end of this walkthrough, you will have a working example of a multi-response streaming API, and you will be able to apply the principles learned to a wide array of use cases beyond video streaming. Let's jump right into it! Hands-On Implementing the Streaming API in Express In this section, we will dive into the server-side implementation, specifically our Node.js and Express application. We'll be implementing an API endpoint to deliver video content in a streaming fashion. Assuming you have already set up your Express server with TypeScript, we first need to define our video-serving route. We'll create a GET endpoint that, when hit, will stream a video file back to the client. Please make sure to install cors for handling cross-origin requests, dotenv for loading environment variables, and throttle for controlling the rate of data transfer. You can install these with the following command: ` ` In the code snippet above, we are implementing a basic video streaming server that responds to HTTP range requests. Here's a brief overview of the key parts: 1. File and Range Setup: We start by determining the path to the video file and getting the file size. We also grab the range header from the request, which contains the range of bytes the client is requesting. 2. Range Requests Handling: If a range is provided, we extract the start and end bytes from the range header, then create a read stream for that specific range. This allows us to stream a portion of the file rather than the entire thing. 3. Response Headers: We then set up our response headers. In the case of a range request, we send back a '206 Partial Content' status along with information about the byte range and total file size. For non-range requests, we simply send back the total file size and the file type. 4. Data Streaming: Finally, we pipe the read stream directly to the response. This step is where the video data actually gets sent back to the client. The use of pipe() here automatically handles backpressure, ensuring that data isn't read faster than it can be sent to the client. With this setup in place, our streaming server is capable of efficiently delivering large video files to the client in small chunks, providing a smoother user experience. Implementing the Download API in Express Now, let's add another endpoint to our Express application, which will provide more granular control over the data transfer process. We'll set up a GET endpoint for '/download', and within this endpoint, we'll handle streaming the video file to the client for download. ` This endpoint has a similar setup to the video streaming endpoint, but it comes with a few key differences: 1. Response Headers: Here, we include a 'Content-Disposition' header with an 'attachment' directive. This header tells the browser to present the file as a downloadable file named 'video.mp4'. 2. Throttling: We use the 'throttle' package to limit the data transfer rate. Throttling can be useful for simulating lower-speed connections during testing, or for preventing your server from getting overwhelmed by data transfer operations. 3. Data Writing: Instead of directly piping the read stream to the response, we attach 'data' and 'end' event listeners to the throttled stream. On the 'data' event, we manually write each chunk of data to the response, and on the 'end' event, we close the response. This implementation provides a more hands-on way to control the data transfer process. It allows for the addition of custom logic to handle events like pausing and resuming the data transfer, adding custom transformations to the data stream, or handling errors during transfer. Utilizing the APIs: A React Application Now that we have a server-side setup for video streaming and downloading, let's put these APIs into action within a client-side React application. Note that we'll be using Tailwind CSS for quick, utility-based styling in our components. Our React application will consist of a video player that uses the video streaming API, a download button to trigger the download API, and a progress bar to show the real-time download progress. First, let's define the Video Player component that will play the streamed video: ` In the above VideoPlayer component, we're using an HTML5 video tag to handle video playback. The src attribute of the source tag is set to the video endpoint of our Express server. When this component is rendered, it sends a request to our video API and starts streaming the video in response to the range requests that the browser automatically makes. Next, let's create the DownloadButton component that will handle the video download and display the download progress: ` In this DownloadButton component, when the download button is clicked, it sends a fetch request to our download API. It then uses a while loop to continually read chunks of data from the response as they arrive, updating the download progress until the download is complete. This is an example of more controlled handling of multi-response APIs where we are not just directly piping the data, but instead, processing it and manually sending it as a downloadable file. Bringing It All Together Let's now integrate these components into our main application component. ` In this simple App component, we've included our VideoPlayer and DownloadButton components. It places the video player and download button on the screen in a neat, centered layout thanks to Tailwind CSS. Here is a summary of how our system operates: - The video player makes a request to our Express server as soon as it is rendered in the React application. Our server handles this request, reading the video file and sending back the appropriate chunks as per the range requested by the browser. This results in the video being streamed in our player. - When the download button is clicked, a fetch request is sent to our server's download API. This time, the server reads the file, but instead of just piping the data to the response, it controls the data sending process. It sends chunks of data and also logs the sent chunks for monitoring purposes. The React application collects these chunks and concatenates them, displaying the download progress in real-time. When all chunks are received, it compiles them into a Blob and triggers a download in the browser. This setup allows us to build a full-featured video streaming and downloading application with fine control over the data transmission process. To see this system in action, you can check out this video demo. Conclusion While the focus of this article was on video streaming and downloading, the principles we discussed here extend beyond just media files. The pattern of responding to HTTP range requests is common in various data-heavy applications, and understanding it can be a useful tool in your web development arsenal. Finally, remember that the code shown in this article is just a simple example to demonstrate the concepts. In a real-world application, you would want to add proper error handling, validation, and possibly some form of access control depending on your use case. I hope this article helps you in your journey as a developer. Building something yourself is the best way to learn, so don't hesitate to get your hands dirty and start coding!...

Aug 9, 2023

6 mins

NodeJSExpress.jsReact

Intro to EdgeDB - The 10x ORM

Intro to EdgeDB - The 10x ORM I’ve written a couple of posts recently covering different TypeScript ORMs. One about Prisma, and another about Drizzle. ORM’s are a controversial topic in their own right - some people think they are evil, and others think they are great. I enjoy them quite a bit. They make it easy to interact with your databases. What is more important and magical for an application than data? SQL without an ORM is amazing as well, but there are some pain points with that approach. Today I’m excited to write about EdgeDB, which isn’t _exactly_ an ORM or a database from my perspective (although they call themselves one). It is, however an incredibly impressive piece of technology that solves these common pain points in a pretty novel way. So if it’s not an ORM or a database, what exactly is it? I don’t think I can answer that in one or two sentences, so we will explore the various pieces that make up EdgeDB in this article. From a high-level standpoint, though, an interface/query language that sits in front of PostgreSQL. This may seem like some less important implementation detail, but in my eyes, it’s a feature and one of the most compelling selling points. Data Modeling EdgeDB advertises itself as a “graph-relational database”. The core components of EdgeDB data modeling are the schema, type system, and relationship definitions. A schema will consist of objects that contain various typed attributes and links that connect the objects. In SQL, a table is analogous to an Object, and a foreign-key is associated with a link. Here’s what a simple schema in EdgeDB looks like ` There’s a few things to highlight here * We defined two different objects (tables) User and Post * Each object contains properties with their types defined * str is one of several scalar types (bool, int, float, json, datetime, etc) * the author property is a required link to the User object Defining relations / associations In our example above we defined a one-to-many relationship between a user and posts. All the relation types that you can define in traditional SQL are available. One interesting feature though is called backward links. These can be defined in your schema and it allows you to access related data from both sides of a relationship. ` likes are a many-to-many relationship between Tweet and User. With a backlink defined multi link likers := Tweet.<likes[is User]; - we can access likes from a User and likers from a Tweet. ` That's how we can access these relations in our queries. You might be looking at these queries and thinking it looks a lot like GraphQL. This is why they call it a ‘Graph-relational’ database. We’ve only scratched the surface of EdgeDB schema’s. Hopefully, I’ve at least managed to pique your interest. Computed properties Computed properties are a super powerful feature that can be added to your schema or queries. This example user schema creates a computed discriminator and username property. The discriminator uses an EdgeDB standard library function to generate a discriminator number and the username property is a combination between the name and discriminator properties. ` Globals and Access Policies EdgeDB allows you to define global variables as part of your schema. The most common use case I’ve seen for this is to power the access policy feature. You can define a global variable as part of your schema: global current_user: uuid; With a global variable defined, you can provide the value as a sort of context from your application by providing it into your EdgeDB driver/client. ` You can then add access policies directly to your schema, for example, to provide fine-grained access control to blog posts in your blogging application. ` Aside from access policies, you can use your global variables in your queries as well. For example, if you wanted a query to select the current user. ` Types and Sets EdgeDB is very type-centric. All of your data is strongly typed. We’ve touched on some of the types already. * Scalars - There are a lot of scalar types available out of the box * Custom scalars - Custom scalar types are user-defined extensions of existing types * Enums - Are supported out of the box - enum<Admin, Moderator, Member> * Arrays - Defined by passing the singular value type - array<str>; * Tuples - In EdgeD, tuples can contain more than 2 elements and come in named and unnamed varieties - <str, number>; tuple<name: str, jersey_number: float64, active: bool>; All queries return a Set which is a collection of values of a given type. In the query language, all values are Sets. Sets are a collection of values of a given type. A comma-separated list of values inside a set of {curly braces}. A query with no results will return an empty or singleton set. If we have no User values stored yet - select User returns {}. Paired with the query language types and set provides an incredibly powerful and expressive system for interacting with your data. If you thought TypeScript was cool, wait until you start writing EdgeQL! 🙂 EdgeQL Now to the fun stuff: the query language. We’ll use a schema from the docs and start by looking at some of those queries and build on those. The example schema has an abstract type Person with two sub-types based on it Hero and Villian. This is known as a polymorphic type in EdgeDB. The Movie type includes a 1:m association with Person ` Selecting properties / data Before we dig into some real queries we should just touch on how we select actual data from a query. It’s pretty obvious and GraphQL-like but worth mentioning. To specify properties to select, you attach a shape. This works for getting nested link / association data as well. Based on our schema, here’s how we could select fields from Movie, including data from the collection of related characters. ` There is also a feature called a splats that allows you to select all fields and/or all linked fields without specifying them individually. ` If you don’t specify any properties or splats, only id’s get returned select Movie; . Adding some objects with insert To get started, we can use insert to add objects to our database. We’ll start big by looking at the nested insert example. This example is interesting because it shows the creation of two objects in a single query. You’ll notice the simplicity of the syntax. Even though this is the first EdgeQL query we’re looking at, in my experience, it’s like this across the board. I’ve found EdgeQL queries to be simple and intuitive to the point where I’ve been able to intuit how to accomplish things in my head without having to reference the docs or ask the AI. This example adds a new Villian and a new Hero which gets assigned as a link or association to the nemesis field on our Villian. To accomplish this we see that we can nest queries by wrapping them in (). ` The next example is pretty similar, but instead of creating the linked property, we are select ing and adding several potential objects to the characters list of Movie since it is a multi link. This is a pretty complex query that is doing a lot of different things. It’s deceivingly succinct. To accomplish the same thing with SQL this would probably be about 3 different queries. This query finds the objects to add to the characters multi link by filtering on a collection of different strings to match against the name property. ` The last thing we’ll cover for insert is bulk inserts. This is particularly useful for things like seed scripts. In this example, you can just imagine that you have a JSON array of objects with hero names that gets passed in as an argument to your query ` Querying data with select We’ve already seen subqueries and a select in the last section where we found a collection of Person records with a filter. We’ll build on that and see what tools are available to us when it comes to querying data. This one covers a lot of ground. Very similar to SQL we have order and limit, and offset operators to support sorting and pagination. Also there is a whole standard library of functions and operators like count that can be used in our queries. This example returns a collection of villian names, excluding the first and last result. ` Most commonly, you will want to filter by an id ` Here’s another common example filtering by datetime. Since we’re using a string value here we need to cast it to the EdgeDB datetime type. ` You get a pretty similar toolbox to SQL when it comes to filtering with your common operators and things. Combined with all the tools in the standard library, you can get pretty creative with it. Changing values and links with update The update..filter..set statement is how we can update existing data with EdgeQL. set is followed by a shape with assignments of properties to be updated. ` You can replace links for an object ` or add additional ones ` An even more interesting example is removing links matched on a type. Since Villian is a sub-type of Person , this query will remove all characters linked of the Villian type. ` Deleting objects with delete Deleting is pretty straight forward. Using the delete command you can just filter for the objects that you would like to remove. ` When the EdgeQL pieces fall into place As you become more familiar with the EdgeQL query language chances are you’ll start writing very complex queries fluently because everything just makes sense once you’ve learned the building blocks. Domain and business concerns I don’t think they explicitly mention this as a goal anywhere but it’s something that I picked up on pretty quickly. EdgeDB nudges you to move more of what might have traditionally been application logic into your database layer. This is a topic that can bring a lot of division since even things like foreign keys and constraints in SQL are frowned upon in some circles. EdgeDB goes as far as providing constraints, global variables, contexts, and authorization support built into the database. I think that the ability to bake some of these concerns into your EdgeDB Schema is great. The way you model your schema and database in EdgeDB map to your domain in a much more intuitive way where domain concerns don’t really feel out of place there. Database Clients and Query Builders and Generators We’ve covered a lot so far to highlight what EdgeDB is and how to handle common use cases with the query language. To use it in your project though, you will need a client/driver library. There are clients available in several different languages. The one that they clearly have put the most investment into is the TypeScript query builder. We’ll briefly look at both options: simple driver/client and query builder. Whichever you end up choosing you will need to instantiate a driver and make sure you have a connection to your database instance configured. Basic client Although the TS query builder is very popular and pretty amazing, I couldn’t get away from just writing EdgeQL queries. In my application, I composed queries using template strings, and it worked great. The clients all have a set of methods available for passing in EdgeQL queries and parameters. querySingle is a method for queries where you are only expecting a single result. If your query will have multiple results you would use query instead. There is also a queryRequiredSingle which will throw an error if no results are found. There are some other methods available as well including one for running queries in a transaction ` The first argument is the query, and the second is a map of parameters. In this example we include the title parameter and it is accessed in our query via $title. TypeScript query builder If you have a TypeScript app and type-safety is important, you might prefer using the query builder. It is a pretty incredible feat of TypeScript magic initially developed by the same developer behind the popular library Zod. We can’t cover it in very much depth here but we’ll look at an example just to have an idea of what the query builder looks like in an application. ` The query builder is able to infer the result type automatically. It knows which fields you’ve selected, it knows that the result will be a single item. Query generator There are generators for queries and types. So even if you opt out of using the query builder you can still have queries that are strongly typed. It’s nice to have this option if you want to just write your queries as EdgeQL in .edgeql files. ` We end up with an exported function named getUser that is strongly typed. ` Tools and Utilities The team at EdgeDB puts a big emphasis on developer experience. It shows up all over the place. We’ve already seen some utilities with the generators that are available. There are some other tools available as well that help complete the entire experience. EdgeDB CLI The first and most important tool to mention is the CLI. If you’ve started using EdgeDB then you’ve most likely already installed and used it. The CLI is pretty extensive. It includes commands for things like migrations, managing EdgeDB versions and installations, managing projects and local/cloud database instances, dumps and restores, a repl, and more. The CLI makes managing EdgeDB a breeze. Admin UI The CLI includes a command to launch an admin UI for any project or database. The Admin UI includes a awesome interactive diagram of your database schema, a repl for running queries, and a table to inspect and make changes to the data stored in your database. Summary Adopting newer database technology is a tough sales pitch. Replacing your application’s database technology at any point in its lifecycle is not a problem that anyone wants to have. This is one of the reasons why EdgeDB being a superset of PostgreSQL is a huge feature in my opinion. The underlying database technology is tried and true, and EdgeDB is open-source. Based on this, I would feel confident using EdgeDB if it aligned well from a technical and business perspective. We’ve covered a lot of ground in this post. EdgeDB is feature-packed and powerful. Databases is a tough nut to crack, and I commend the team for all their hard work to help continue pushing forward one of the most important components of almost any application. I’m typically pretty conservative when it comes to databases, but EdgeDB took a great approach, in my opinion. I recommend at least giving it a try. You might catch the EdgeDB bug like I did!...

Apr 26, 2024

11 mins

JavaScriptEdgeDB

Leveraging GraphQL Scalars to Enhance Your Schema

Introduction

Benefits of Using Scalars

Demonstrating Scalars in Action with Our Starter Project

Setting Up the Starter:

Enhancing with graphql-scalars:

Testing in the GraphQL Sandbox

Conclusion

William Mimura

You might also like

How to create and use custom GraphQL Scalars

Introducing the New Serverless, GraphQL, Apollo Server, and Contentful Starter kit

Building a Multi-Response Streaming API with Node.js, Express, and React

Intro to EdgeDB - The 10x ORM

You might also like

How to create and use custom GraphQL Scalars

Introducing the New Serverless, GraphQL, Apollo Server, and Contentful Starter kit

Building a Multi-Response Streaming API with Node.js, Express, and React

Intro to EdgeDB - The 10x ORM