Javascript

Integrating AI Models Locally with Next.js ft. Jesus Padron

Published Sep 3, 2024

Updated Dec 11, 2024

1 min read

Jesus Padron from the This Dot team shows you how to integrate AI models into a Next.js application. Jesus walks through the process of running Meta's Llama 3.1 model locally, leveraging OpenAI's Whisper for speech-to-text conversion, and using OpenAI's TTS model for text-to-speech conversion. By the end of the episode, listeners will know how to create an AI voice assistant that processes voice input, understands the content, and responds audibly.

Chapters:

Introduction to the Episode (00:00:03)
Overview of Llama 3.1 and Setup (00:02:14)
Setting Up the Next.js Application (00:04:40)
Recording Audio with MediaRecorder API (00:11:37)
Integrating OpenAI's Whisper for Speech-to-Text (00:36:46)
Generating Responses with Llama 3.1 (00:48:24)
Implementing Text-to-Speech with OpenAI's TTS (01:03:26)
Final Testing and Demonstration (01:06:37)
Summary and Next Steps (01:09:01)
Closing Remarks (01:14:19)

Follow Jesus on Social Media Twitter: https://x.com/padron4497 Github: https://github.com/padron4497

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

Integrating Next.js with New Relic

Integrating Next.js with New Relic When it comes to application monitoring, New Relic is genuinely one of the veterans in the market. Founded in 2008, it offers a large set of tools designed for developers to help them analyze application performance, troubleshoot problems, and gain insight into user behavior through real-time analytics. We needed to integrate New Relic's application monitoring with a modern Next.js app built on top of the app router. This blog post will explain your options if you want to do the same in your project. Please keep in mind that depending on when you read this blog post, there may be some advances in how New Relic works with Next.js. You can always refer to the official New Relic website for up-to-date documentation. The approach is also different, depending on whether you use Vercel's managed Next.js hosting (or any other provider that is lambda-based) or self-host Next.js on an alternative provider such as Fly.io. We'll start with the typical steps you need to do for both ways of deploying; then we'll continue with the self-hosted option, which uses a complete set of features from New Relic, and finish with Vercel where we recommend the OpenTelemetry integration for now. Common Prerequisite Steps First, let's install the necessary packages. These were built by New Relic and are needed whether you self-host your Next.js app or host it on Vercel. ` The next thing you'll want to do is define your app name. The name will be different depending on the environment the app is running in, such as a local environment, staging, or production. This will ensure that the data you send to New Relic is always tagged with a proper app name and will not be mixed between environments. An environment-changing parameter, such as the app name, should always be included in an environment variable. In your local environment, that would be in .env.local, while staging/production or any other hosted environments would be configured in the provider. ` To get full coverage with New Relic, in Next.js, we need to cover the parts running on the client (in the browser) and on the server. The server-running part is different on Vercel's managed hosting, where all of the logic is executed within lambdas, and self-hosting variants, where the server logic is executed on a Node.js server. The client part is configured by going to the New Relic dashboard and clicking "Add Data" / "Browser Monitoring". Choose "place a JavaScript snippet in frontend code," name it the same name you used in the environment variable above, click "Continue" to the last step, and copy the part between the script tags: ` Then paste it to a variable inside any module of your code. For example, this is how we did it: ` Now, create a component that will use this script and put it to Next's Script component. Then you can use the component in your root layout: ` There are several things to note here. First, we need to set a unique ID for the Script component for Next.js to track and optimize the script. We also need to set the strategy to beforeInteractive so that the New Relic script is added to the document’s head element. Finally, we use some environment helper variables. IS_PRODUCTION and IS_STAGING are just helper variables that read the environment variables to determine if we are in a local environment, staging, or production. Now you can create an app router error page and report any uncaught errors to New Relic: ` With this in place, the client part is covered, and you should start seeing data in the New Relic dashboard under "All Entities" / "Browser Applications." Let's now focus on the server part, with both a self-hosted option and Vercel. Self-Hosting Next.js If you self-host Next.js, It will typically run through a Node server (containerized or not), meaning you can fully utilize New Relic's APM agent. Start by adding this to the Next.js config (next.config.js) to mark the newrelic package as external, otherwise you would not be able to import it. ` The next step that New Relic recommends is to copy the newrelic.js file from the newrelic library to the root of the folder. However, we want to keep our project root clean, and we'll store the file in o11y/newrelic.js (“o11y” is an abbreviation for observability). This will also demonstrate the configuration needed to make this possible. This file, by default, will look something like this. ` As you can see, some environment variables are involved, such as NEW_RELIC_APP_NAME and NEW_RELIC_LICENSE_KEY. These environment variables will be needed when the New Relic agent starts up. The trick here is that the agent doesn't start up with your app but *earlier* by preloading the library through another environment variable, NODE_OPTIONS. On the server, that would be done by modifying the start script to this: ` This ensures the agent is preloaded before starting the app, and the rest of the environment variables (such as NEW_RELIC_APP_NAME and NEW_RELIC_LICENSE_KEY) should already exist. Making sure this works in local development is not that easy due to different stages of loading environment variables. The environment variables from .env.local are loaded by the framework, which is too late for the New Relic agent (which is preloaded by the Node process). However, it is possible with a trick. First, add the following new environment variables to .env.local: ` (We have not mentioned NEW_RELIC_HOME so far, but the New Relic agent implicitly uses it to locate the newrelic.js file. We need to specify that since we moved it to the o11y folder.) Next, install the nodenv package. ` The nodenv package provides a cross-platform way of preloading environment variables before we start the Node process. If we provide it with the .env.local file, this means all of the environment variables from that file will be available to the New Relic agent! Change the scripts in package.json: ` And that's it. If you now start the local server using the dev command, nodenv will make environment variables available to the Node process. The Node process will preload the New Relic agent (because we have NODE_OPTIONS in .local.env), the agent will read all the environment variables needed by newrelic.js, and everything will work as expected. The first thing you might notice is the creation of the newrelic_agent.log. This is just a log file by the agent which you can use to troubleshoot if the agent is not working correctly. You can safely add this file to .gitignore. From this point on, the server-side part of the app can utilize New Relic's API (such as noticeError and recordLogEvent) and the agent will automatically be recording transactions made through the server. The data should be visible in the New Relic dashboard, under "All Entities" / "Services - APM". This is how you would log an error in your server action, for example: ` The full source of this type of integration is available on StackBlitz. Using Vercel's Managed Next.js Hosting Using the New Relic APM agent on Vercel is currently not possible, as the server part of Next.js runs within lambdas, and not within a typical Node server as in the self-hosted option. In the future, this may be possible. At the moment, the only way to integrate Vercel and New Relic is by using the Vercel OpenTelemetry collector, which is only available for some plans. The first step is to enable the integration in Vercel's dashboard and then install the following packages: ` Create an instrumentation.ts file in the root of your project and add the following code: ` In the the Next.js config (next.config.js), add: ` The OpenTelemetry integration does not work locally, hence the above conditional check. With the above configuration, the collected OpenTelemetry data should be visible in the New Relic dashboard under "All Entities" / "Services - OpenTelemetry". Optionally, Logging via New Relic API When using the Vercel OpenTelemetry integration, there is no easy way to submit custom logs to New Relic. New Relic offers a logging API, however, so to send custom logs from your backend, you could use a logger such as Winston, which has good library support for custom transports, such as the one for New Relic. Install the following packages: ` Then create a logger.ts file with the following code: ` NewRelicTransport utilizes the New Relic logging API to send application logs. Now you can use the serverLogger in your backend code, from server components to server actions. Conclusion As you can see, various options exist for integrating New Relic with the modern, app-router-based Next.js. Sadly, at the time of writing, no one-size-fits-all solution works equally for all hosting options, as Vercel's managed Next.js platform is still relatively young, and so is the app router in Next.js. If you liked this blog post, feel free to browse our other Next.js blog posts for more tips and tricks in other areas of the Next.js ecosystem....

Jul 19, 2024

7 mins

NextJS

Improving INP in React and Next.js

Improving INP in React and Next.js In one of my previous articles, I've explained what INP is, how it works, and how it may affect your website. I also promised you to follow up with more concrete advice on how to improve your INP in your favorite framework. This is the follow-up article, where I'll focus on how to improve your INP score in React and Next.js. How to prepare for INP in React and Next.js? The first thing to do is to ensure you're using the latest version of React. The React team has been working on making React more INP-friendly and has already made some improvements in the latest versions. To enhance your INP score, consider fully taking advantage of new features introduced in React 18, such as Concurrent Rendering, Automatic Batching, and Selective Hydration. However, there are also some general areas to focus on, such as SSR and SSG in Next.js, Web Workers, or optimizing your hooks and state management. Concurrent Rendering The Concurrent Mode in React uses an algorithm that breaks rendering down into so-called "fiber nodes" and schedules the renders based on their expiration and priority. This effectively allows the user to interact with the page while the rendering is still in progress. In previous React versions, all updates, such as setState calls were treated as "urgent" and once the re-render had started, there was no way to interrupt it. Concurrent Mode changes this by being able to prioritize the updates and interrupt a non-blocking state update started with startTransition. For a simple explanation of concurrency in React, you can check out Dan Abramov's explanation. As part of the Concurrent Mode, React introduced several lifecycle methods that allow you to prioritize the rendering of certain parts of your UI, such as: - useTransition hook that allows you to update the state without blocking the UI, - useDeferredValue hook that allows you to defer the rendering of certain parts of your UI, - startTransition API that, similarly to the useTransition hook lets you mark a state update as non-blocking. It lacks, however, an indication of whether it's still pending. Automatic Batching Introduced in React 18, Automatic Batching reduces the number of re-renders that happen on state changes even when they happen outside of event handlers, e.g. in a setTimeout or Promise callback. This feature comes out of the box and you don't have to do anything to enable it, and it makes a great argument for upgrading to React 18. Selective Hydration Selective Hydration allows you to take hydration off the main thread by wrapping your components in a Suspense boundary. This way, components can become interactive faster as the browser can do other work on the main thread while the hydration is happening. To fully take advantage of selective hydration, consider the following: - Prioritizing Above-the-Fold Content: Use Suspense boundaries strategically around any parts of your application that may take the server longer to deliver to ensure they don’t block critical content from becoming interactive as soon as possible. - Hydration on Interaction: Implementing hydration upon user interaction for non-critical components can drastically reduce the main thread's workload, enhancing INP. Vercel even has a small case study showing how using selective hydration improved the performance of a Next.js site. Server-Side Rendering (SSR) and Static Site Generation (SSG) in Next.js Not everything has to run client-side. Next.js excels in SSR and SSG capabilities, which can significantly impact INP by delivering content to users faster. Optimizing SSR with techniques like incremental static regeneration (ISR) or leveraging SSG for static pages ensures that users can interact with content faster, improving the perceived performance. Workers Offloading heavy computations to Web Workers can free up the main thread, enhancing the responsiveness of React and Next.js applications. This strategy is especially useful when dealing with third-party scripts. Offloading such scripts in Next.js can be easily done by specifying the "worker" strategy on your Script component. Be aware that this feature is not yet stable and does not work with the app directory, though. If you want to take things one step further, you could use Partytown, which helps you offload any resource-intensive scripts to Web Workers. It comes with a React component that you can use to wrap your third-party scripts and offload them to a Web Worker, and it's compatible with Next.js as well. Hooks and State Management State management in React applications can easily get out of hand, leading to unnecessary re-renders and effectively an increased INP. Sometimes, using a state management library like Redux or MobX can help you consolidate your state and reduce the number of re-renders. However, they are not silver bullets and can also introduce performance issues if not used properly. If you are dealing with a lot of re-renders due to prop changes, make sure you are leveraging memoization. As of now, you may need to work with useMemo and useCallback hooks to memoize your values and functions, respectively. The upcoming React 19’s Forget Compiler, however, will apparently memoize everything under the hood, making these hooks obsolete. Using memoization properly can help you reduce the number of re-renders and improve your INP. To investigate your hook dependencies and re-renders, you can leverage React Developer Tools or use this handy helper hook I found on the internet to trace your re-renders: ` Conclusion Improving INP in React and Next.js is not easy and can require much investigation and fine-tuning. Still, it's worth doing to avoid being penalized by Google in its search results and provide a better experience for your users. Adopting React 18's new features, leveraging SSR and SSG in Next.js, utilizing Web Workers, and optimizing hooks and state management can significantly boost your INP score and deliver a faster application to your users. Remember, INP is just one among many performance metrics emphasizing the need for a comprehensive approach to performance optimization...

Apr 19, 2024

5 mins

Web PerformanceReactNextJSJavaScript

Introduction to Zod for Data Validation

As web developers, we're often working with data from external sources like APIs we don't control or user inputs submitted to our backends. We can't always rely on this data to take the form we expect, and we can encounter unexpected errors when it deviates from expectations. But with the Zod library, we can define what our data ought to look like and parse the incoming data against those defined schemas. This lets us work with that data confidently, or to quickly throw an error when it isn't correct. Why use Zod? TypeScript is great for letting us define the shape of our data in our code. It helps us write more correct code the first time around by warning us if we are doing something we shouldn't. But TypeScript can't do everything for us. For example, we can define a variable as a string or a number, but we can't say "a string that starts with user_id_ and is 20 characters long" or "an integer between 1 and 5". There are limits to how much TypeScript can narrow down our data for us. Also, TypeScript is a tool for us developers. When we compile our code, our types are not available to the vanilla JavaScript. JavaScript can't validate that the data we actually use in our code matches what we thought we'd get when we wrote our TypeScript types unless you're willing to manually write code to perform those checks. This is where we can reach for a tool like Zod. With Zod, we can write data schemas. These schemas, in the simplest scenarios, look very much like TypeScript types. But we can do more with Zod than we can with TypeScript alone. Zod schemas let us create additional rules for data parsing and validation. A 20-character string that starts with user_id_? It's z.string().startsWith('user_id_').length(20). An integer between 1 and 5 inclusive? It's z.number().int().gte(1).lte(5). Zod's primitives give us many extra functions to be more specific about *exactly* what data we expect. Unlike TypeScript, Zod schemas aren't removed on compilation to JavaScript—we still have access to them! If our app receives some data, we can verify that it matches the expected shape by passing it to your Zod schema's parse function. You'll either get back your data in exactly the shape you defined in your schema, or Zod will give you an error to tell you what was wrong. Zod schemas aren't a replacement for TypeScript; rather, they are an excellent complement. Once we've defined our Zod schema, it's simple to derive a TypeScript type from it and to use that type as we normally would. But when we really need to be sure our data conforms to the schema, we can always parse the data with our schema for that extra confidence. Defining Data Schemas Zod schemas are the variables that define our expectations for the shape of our data, validate those expectations, and transform the data if necessary to match our desired shape. It's easy to start with simple schemas, and to add complexity as required. Zod provides different functions that represent data structures and related validation options, which can be combined to create larger schemas. In many cases, you'll probably be building a schema for a data object with properties of some primitive type. For example, here's a schema that would validate a JavaScript object representing an order for a pizza: ` Zod provides a number of primitives for defining schemas that line up with JavaScript primitives: string, number, bigint, boolean, date, symbol, undefined, and null. It also includes primitives void, any, unknown, and never for additional typing information. In addition to basic primitives, Zod can define object, array, and other native data structure schemas, as well as schemas for data structures not natively part of JavaScript like tuple and enum. The documentation contains considerable detail on the available data structures and how to use them. Parsing and Validating Data with Schemas With Zod schemas, you're not only telling your program what data should look like; you're also creating the tools to easily verify that the incoming data matches the schema definitions. This is where Zod really shines, as it greatly simplifies the process of validating data like user inputs or third party API responses. Let's say you're writing a website form to register new users. At a minimum, you'll need to make sure the new user's email address is a valid email address. For a password, we'll ask for something at least 8 characters long and including one letter, one number, and one special character. (Yes, this is not really the best way to write strong passwords; but for the sake of showing off how Zod works, we're going with it.) We'll also ask the user to confirm their password by typing it twice. First, let's create a Zod schema to model these inputs: ` So far, this schema is pretty basic. It's only making sure that whatever the user types as an email is an email, and it's checking that the password is at least 8 characters long. But it is *not* checking if password and confirmPassword match, nor checking for the complexity requirements. Let's enhance our schema to fix that! ` By adding refine with a custom validation function, we have been able to verify that the passwords match. If they don't, parsing will give us an error to let us know that the data was invalid. We can also chain refine functions to add checks for our password complexity rules: ` Here we've chained multiple refine functions. You could alternatively use superRefine, which gives you even more fine grained control. Now that we've built out our schema and added refinements for extra validation, we can parse some user inputs. Let's see two test cases: one that's bound to fail, and one that will succeed. ` There are two main ways we can use our schema to validate our data: parse and safeParse. The main difference is that parse will throw an error if validation fails, while safeParse will return an object with a success property of either true or false, and either a data property with your parsed data or an error property with the details of a ZodError explaining why the parsing failed. In the case of our example data, userInput2 will parse just fine and return the data for you to use. But userInput1 will create a ZodError listing all of the ways it has failed validation. ` ` We can use these error messages to communicate to the user how they need to fix their form inputs if validation fails. Each error in the list describes the validation failure and gives us a human readable message to go with it. You'll notice that the validation errors for checking for a valid email and for checking password length have a lot of details, but we've got three items at the end of the error list that don't really tell us anything useful: just a custom error of Invalid input. The first is from our refine checking if the passwords match, and the second two are from our refine functions checking for password complexity (numbers and special characters). Let's modify our refine functions so that these errors are useful! We'll add our own error parameters to customize the message we get back and the path to the data that failed validation. ` Now, our error messages from failures in refine are informative! You can figure out which form fields aren't validating from the path, and then display the messages next to form fields to let the user know how to remedy the error. ` By giving our refine checks a custom path and message, we can make better use of the returned errors. In this case, we can highlight specific problem form fields for the user and give them the message about what is wrong. Integrating with TypeScript Integrating Zod with TypeScript is very easy. Using z.infer<typeof YourSchema> will allow you to avoid writing extra TypeScript types that merely reflect the intent of your Zod schemas. You can create a type from any Zod schema like so: ` Using a TypeScript type derived from a Zod schema does *not* give you any extra level of data validation at the type level beyond what TypeScript is capable of. If you create a type from z.string.min(3).max(20), the TypeScript type will still just be string. And when compiled to JavaScript, even that will be gone! That's why you still need to use parse/safeParse on incoming data to validate it before proceeding as if it really does match your requirements. A common pattern with inferring types from Zod schemas is to use the same name for both. Because the schema is a variable, there's no name conflict if the type uses the same name. However, I find that this can lead to confusing situations when trying to import one or the other—my personal preference is to name the Zod schema with Schema at the end to make it clear which is which. Conclusion Zod is an excellent tool for easily and confidently asserting that the data you're working with is exactly the sort of data you were expecting. It gives us the ability to assert at runtime that we've got what we wanted, and allows us to then craft strategies to handle what happens if that data is wrong. Combined with the ability to infer TypeScript types from Zod schemas, it lets us write and run more reliable code with greater confidence....

Nov 15, 2024

7 mins

JavaScriptZod

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

Understanding the Difference Between :focus and :focus-visible in CSS I have learned my fair share about the importance of keyboard accessibility, so I know that visual indication of the focused element is very important. But the well-known :focus pseudo-class is not always the best fit for this job. That's where :focus-visible comes in. Let's look at the differences between these two pseudo-classes and explore the best practices for using them effectively. What is the :focus Pseudo-Class? The :focus pseudo-class is a CSS selector that applies styles to any element that receives focus, regardless of how that focus was triggered. This includes focus events from keyboard navigation, mouse clicks, and touch interactions. Example Usage of :focus ` In this example, the button will display a blue outline whenever it is focused, whether the user clicks on it with a mouse, taps it on a touchscreen, or navigates to it using the keyboard. What is the :focus-visible Pseudo-Class? The :focus-visible pseudo-class is more specialized. It only applies styles to an element when the browser determines that the focus should be visible. This typically occurs when the user navigates via the keyboard or assistive technologies rather than through mouse or touch input. Example Usage of :focus-visible ` Here, the button will only show a blue outline when focused through keyboard navigation or another input method that usually requires visible focus indicators. Key Differences Between :focus and :focus-visible :focus - Behavior: Applies to any element that receives focus, regardless of the input method. - Use Cases: Ensures that all interactions with the element are visually indicated, whether by mouse, keyboard, or touch. :focus-visible - Behavior: Applies styles only when the focus should be visible, such as using a keyboard or assistive technology. - Use Cases: Ideal for scenarios where you want to provide focus indicators only to keyboard and assistive technology users while avoiding unnecessary outlines for mouse and touch users, typically required by design. Accessibility Implications :focus - Pros: - Guarantees that all users can see when an element is focused, which is critical for accessibility. - Cons: - Can lead to a suboptimal experience for mouse users, as focus styles may appear unnecessarily during mouse interactions. :focus-visible - Pros: - Enhances user experience by showing focus indicators only when necessary, thus keeping the interface clean for mouse and touch users. - Tailors the experience for keyboard and assistive technology users, providing them with clear visual cues. - Cons: - Additional considerations may be required to ensure that focus indicators are not accidentally omitted, especially in older browsers that do not support :focus-visible. - There may be cases where you want to show focus indicators for all users, regardless of input method. Best Practices for Using :focus and :focus-visible To achieve the best accessibility and user experience, combining both :focus and :focus-visible in your CSS is often a good idea. Combining :focus and :focus-visible ` Here is a Stackblitz example of what such styling could look like for you to try out and play with. Additional Tips - Test with Keyboard and Assistive Technology: Ensure that your web application is navigable using a keyboard (Tab, Shift + Tab, etc.) and that focus indicators are visible for those who rely on them. It's never a bad idea to include accessibility testing in your e2e testing suite. - Provide Clear Focus Indicators: Make sure that focus indicators are prominent and easy to see. A subtle or hard-to-spot focus indicator can severely impact accessibility for users who rely on keyboard navigation. Conclusion The :focus-visible pseudo-class offers a more refined way to manage focus indicators, improving accessibility and user experience, particularly for keyboard and assistive technology users. By understanding the differences between :focus and :focus-visible, and applying best practices in your CSS, you can create more accessible and user-friendly web applications. Remember, accessibility should never be an afterthought. By thoughtfully applying focus styles, you ensure that all users, regardless of how they interact with your site, can easily navigate and interact....

Nov 8, 2024

3 mins

AccessibilityCSSPlaywright

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

Integrating AI Models Locally with Next.js ft. Jesus Padron

You might also like

Integrating Next.js with New Relic

Improving INP in React and Next.js

Introduction to Zod for Data Validation

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

Let's innovate together!

You might also like

Integrating Next.js with New Relic

Improving INP in React and Next.js

Introduction to Zod for Data Validation

Understanding the Difference Between `:focus` and `:focus-visible` in CSS