Javascript

How to build an AI assistant with OpenAI, Vercel AI SDK, and Ollama with Next.js

Published Sep 27, 2024

Updated Oct 1, 2024

8 min read

In today’s blog post, we’ll build an AI Assistant using three different AI models: Whisper and TTS from OpenAI and Llama 3.1 from Meta.

While exploring AI, I wanted to try different things and create an AI assistant that works by voice. This curiosity led me to combine OpenAI’s Whisper and TTS models with Meta’s Llama 3.1 to build a voice-activated assistant.

Here’s how these models will work together:

First, we’ll send our audio to the Whisper model, which will convert it from speech to text.
Next, we’ll pass that text to the Llama 3.1 model. Llama will understand the text and generate a response.
Finally, we’ll take Llama’s response and send it to the TTS model, turning the text back into speech. We’ll then stream that audio back to the client.

Let’s dive in and start building this excellent AI Assistant!

Getting started

We will use different tools to build our assistant. To build our client side, we will use Next.js. However, you could choose whichever framework you prefer.

To use our OpenAI models, we will use their TypeScript / JavaScript SDK. To use this API, we require the following environmental variable: OPENAI_API_KEY—

To get this key, we need to log in to the OpenAI dashboard and find the API keys section. Here, we can generate a new key.

Awesome. Now, to use our Llama 3.1 model, we will use Ollama and the Vercel AI SDK, utilizing a provider called ollama-ai-provider.

Ollama will allow us to download our preferred model (we could even use a different one, like Phi) and run it locally. The Vercel SDK will facilitate its use in our Next.js project.

To use Ollama, we just need to download it and choose our preferred model. For this blog post, we are going to select Llama 3.1. After installing Ollama, we can verify if it is working by opening our terminal and writing the following command:

Notice that I wrote “llama3.1” because that’s my chosen model, but you should use the one you downloaded.

Kicking things off

It's time to kick things off by setting up our Next.js app. Let's start with this command:

npx create-next-app@latest

After running the command, you’ll see a few prompts to set the app's details. Let's go step by step:

Name your app.
Enable app router.

The other steps are optional and entirely up to you. In my case, I also chose to use TypeScript and Tailwind CSS.

Now that’s done, let’s go into our project and install the dependencies that we need to run our models:

npm i ai ollama-ai-provider openai

Building our client logic

Now, our goal is to record our voice, send it to the backend, and then receive a voice response from it.

To record our audio, we need to use client-side functions, which means we need to use client components. In our case, we don’t want to transform our whole page to use client capabilities and have the whole tree in the client bundle; instead, we would prefer to use Server components and import our client components to progressively enhance our application.

So, let’s create a separate component that will handle the client-side logic.

Inside our app folder, let's create a components folder, and here, we will be creating our component:

app
 ↳components
  ↳audio-recorder.tsx

Let’s go ahead and initialize our component. I went ahead and added a button with some styles in it:

// app/components/audio-recorder.tsx
'use client'
export default function AudioRecorder() {
    function handleClick(){
      console.log('click')
    }

    return (
        <section>
		<button onClick={handleClick}
                    className={`bg-blue-500 text-white px-4 py-2 rounded shadow-md hover:bg-blue-400 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2 focus:ring-offset-white transition duration-300 ease-in-out absolute top-1/2 left-1/2 -translate-x-1/2 -translate-y-1/2`}>
                Record voice
            </button>
        </section>
    )
}

And then import it into our Page Server component:

// app/page.tsx
import AudioRecorder from '@/app/components/audio-recorder';

export default function Home() {
  return (
      <AudioRecorder />
  );
}

Now, if we run our app, we should see the following:

Awesome! Now, our button doesn’t do anything, but our goal is to record our audio and send it to someplace; for that, let us create a hook that will contain our logic:

app
 ↳hooks
  ↳useRecordVoice.ts

import { useEffect, useRef, useState } from 'react';

export function useRecordVoice() {
  return {}
}

We will use two APIs to record our voice: navigator and MediaRecorder. The navigator API will give us information about the user’s media devices like the user media audio, and the MediaRecorder will help us record the audio from it. This is how they’re going to play out together:

// apps/hooks/useRecordVoice.ts
import { useEffect, useRef, useState } from 'react';

export function useRecordVoice() {
    const [isRecording, setIsRecording] = useState(false);
    const [mediaRecorder, setMediaRecorder] = useState<MediaRecorder | null>(null);

     const startRecording = async () => {
        if(!navigator?.mediaDevices){
            console.error('Media devices not supported');
            return;
        }

        const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
        const mediaRecorder = new MediaRecorder(stream);
        setIsRecording(true)
        setMediaRecorder(mediaRecorder);
        mediaRecorder.start(0)
    }

    const stopRecording = () =>{
        if(mediaRecorder) {
            setIsRecording(false)
            mediaRecorder.stop();
        }
    }

  return {
    isRecording,
    startRecording,
    stopRecording,
  }
}

Let’s explain this code step by step. First, we create two new states. The first one is for keeping track of when we are recording, and the second one stores the instance of our MediaRecorder.

 const [isRecording, setIsRecording] = useState(false);
    const [mediaRecorder, setMediaRecorder] = useState<MediaRecorder | null>(null);

Then, we’ll create our first method, startRecording. Here, we are going to have the logic to start recording our audio. We first check if the user has media devices available thanks to the navigator API that gives us information about the browser environment of our user:

If we don’t have media devices to record our audio, we just return. If they do, then let us create a stream using their audio media device.

// check if they have media devices
if(!navigator?.mediaDevices){
 console.error('Media devices not supported');
 return;
}
// create stream using the audio media device
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });

Finally, we go ahead and create an instance of a MediaRecorder to record this audio:

// create an instance passing in the stream as parameter
const mediaRecorder = new MediaRecorder(stream);
// Set this state to true to 
setIsRecording(true)
// Store the instance in the state
setMediaRecorder(mediaRecorder);
// Start recording inmediately
mediaRecorder.start(0)

Then we need a method to stop our recording, which will be our stopRecording. Here, we will just stop our recording in case a media recorder exists.

if (mediaRecorder) {
  setIsRecording(false)
  mediaRecorder.stop();
}

We are recording our audio, but we are not storing it anywhere. Let’s add a new useEffect and ref to accomplish this. We would need a new ref, and this is where our chunks of audio data will be stored.

const audioChunks = useRef<Blob[]>([]);

In our useEffect we are going to do two main things: store those chunks in our ref, and when it stops, we are going to create a new Blob of type audio/mp3:

export function useRecordVoice() {   
   const audioChunks = useRef<Blob[]>([]);

   ...
   useEffect(() => {
        if (mediaRecorder) {
            // listen to when data is available and store it as chunks in our ref
            mediaRecorder.ondataavailable = (e) => {
                audioChunks.current.push(e.data);
            }

            mediaRecorder.onstop = () => {
                // Listen to when we stop recording audio 
                // Then, convert our data to a Blob of type audio/mp3 and reset the ref
                const audioBlob = new Blob(audioChunks.current, { type: 'audio/mp3' });
                audioChunks.current = [];
            }
        }

    }, [mediaRecorder]);
    ...
}

It is time to wire this hook with our AudioRecorder component:

'use client'
import { useRecordVoice } from '@/hooks/useRecordVoice';

export default function AudioRecorder() {
    const { isRecording, stopRecording, startRecording } = useRecordVoice();

    async function handleClick() {
        if (isRecording) {
           stopRecording();
         } else {
         await startRecording();
       }
     }

    return (
        <div>

            <button onClick={handleClick}
                    className={`bg-blue-500 text-white px-4 py-2 rounded shadow-md hover:bg-blue-400 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2 focus:ring-offset-white transition duration-300 ease-in-out absolute top-1/2 left-1/2 -translate-x-1/2 -translate-y-1/2`}>
                {isRecording ? "Stop Recording" : "Start Recording"}
            </button>

        </div>
    )
}

Let’s go to the other side of the coin, the backend!

Setting up our Server side

We want to use our models on the server to keep things safe and run faster. Let’s create a new route and add a handler for it using route handlers from Next.js. In our App folder, let’s make an “Api” folder with the following route in it:

app
 ↳api
  ↳chat
    ↳route.ts

Our route is called ‘chat’. In the route.ts file, we’ll set up our handler. Let’s start by setting up our OpenAI SDK.

const openai = getOpenai();

export async function POST(req: Request) {
  // our logic will go here
}

// inside a utils folder apps/utils/get-openai.ts
import OpenAI from 'openai';

const openai = new OpenAI({
    apiKey: process.env.OPENAI_API_KEY,
});

export function getOpenai() {
    return openai;
}

In this route, we’ll send the audio from the front end as a base64 string. Then, we’ll receive it and turn it into a Buffer object.

export async function POST(req: Request) {
    const { audio } = await req.json();
    const audioBuffer = Buffer.from(audio, 'base64');
 }

It’s time to use our first model. We want to turn this audio into text and use OpenAI’s Whisper Speech-To-Text model. Whisper needs an audio file to create the text. Since we have a Buffer instead of a file, we’ll use their ‘toFile’ method to convert our audio Buffer into an audio file like this:

import { toFile } from 'openai';

export async function POST(req: Request) {
    const { audio } = await req.json();
    const audioBuffer = Buffer.from(audio, 'base64');

    try {
        // FileLike object
        const audioFile = await toFile(audioBuffer, 'audio.mp3');

    } catch (err) {
        console.error(err);
        return NextResponse.json(
            {
                err: err,
                error: 'Error converting audio',
            },
            {
                status: 500,
            }
        );
    }
}

Notice that we specified “mp3”. This is one of the many extensions that the Whisper model can use. You can see the full list of supported extensions here: https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-file

Now that our file is ready, let’s pass it to Whisper! Using our OpenAI instance, this is how we will invoke our model:

import { toFile } from 'openai';
const openai = getOpenai();

export async function POST(req: Request) {
        ...
        const audioFile = await toFile(audioBuffer, 'audio.mp3');

        const transcription = await openai.audio.transcriptions.create({
            // here we specify the model
            model: 'whisper-1',
            // our audio file
            file: audioFile,
        });
        ...
}

That’s it! Now, we can move on to the next step: using Llama 3.1 to interpret this text and give us an answer. We’ll use two methods for this. First, we’ll use ‘ollama’ from the ‘ollama-ai-provider’ package, which lets us use this model with our locally running Ollama. Then, we’ll use ‘generateText’ from the Vercel AI SDK to generate the text. Side note: To make our Ollama run locally, we need to write the following command in the terminal:

ollama serve

import { toFile } from 'openai';
// new imports
import { ollama } from 'ollama-ai-provider';
import { generateText } from 'ai';

const openai = getOpenai();

export async function POST(req: Request) {
        ...
        const audioFile = await toFile(audioBuffer, 'audio.mp3');

        const transcription = await openai.audio.transcriptions.create({
            model: 'whisper-1',
            file: audioFile,
        });

        const { text: response } = await generateText({
            // we specify our model running locally in the background
            model: ollama('llama3.1'),
            // we can set initial instructions to our model
            system: 'You know a lot about video games',
            // the text we want the model to interpret
            prompt: transcription.text,
        });
        ...
}

Finally, we have our last model: TTS from OpenAI. We want to reply to our user with audio, so this model will be really helpful. It will turn our text into speech:

import { toFile } from 'openai';
// new imports
import { ollama } from 'ollama-ai-provider';
import { generateText } from 'ai';

const openai = getOpenai();

export async function POST(req: Request) {
        ...
        const audioFile = await toFile(audioBuffer, 'audio.mp3');

        const transcription = await openai.audio.transcriptions.create({
            model: 'whisper-1',
            file: audioFile,
        });

        const { text: response } = await generateText({
            model: ollama('llama3.1'),
            system: 'You know a lot about video games',
            prompt: transcription.text,
        });

        const voiceResponse = await openai.audio.speech.create({
            // Specify here our tts model
            model: 'tts-1',
            // we pass in our response
            input: response,
            // We can choose a variety of different voices
            // I chose 'onyx' but you can pick from this list: <https://platform.openai.com/docs/guides/text-to-speech/quickstart>
            voice: 'onyx',
        });
        ...
}

The TTS model will turn our response into an audio file. We want to stream this audio back to the user like this:

import { toFile } from 'openai';
import { getOpenai } from '@/utils/getOpenai';
import { ollama } from 'ollama-ai-provider';
import { NextResponse } from 'next/server';
import { generateText } from 'ai';

const openai = getOpenai();

export async function POST(req: Request) {
    const { audio } = await req.json();
    const audioBuffer = Buffer.from(audio, 'base64');

    try {
        const audioFile = await toFile(audioBuffer, 'audio.mp3');

        const transcription = await openai.audio.transcriptions.create({
            model: 'whisper-1',
            file: audioFile,
        });

        const { text: response } = await generateText({
            model: ollama('llama3.1'),
            system: 'You know a lot about video games',
            prompt: transcription.text,
        });

        const voiceResponse = await openai.audio.speech.create({
            model: 'tts-1',
            input: response,
            voice: 'onyx',
        });

        // stream back our audio
        return new Response(voiceResponse.body, {
            headers: {
                 // we specify the content type         
                'Content-Type': 'audio/mpeg',
                // we indicate that this is going to be streamed in chunks of data
                'Transfer-Encoding': 'chunked',
            },
        });
    } catch (err) {
        console.error(err);
        return NextResponse.json(
            {
                err: err,
                error: 'Error converting audio',
            },
            {
                status: 500,
            }
        );
    }
}

And that’s all the whole backend code! Now, back to the frontend to finish wiring everything up.

Putting It All Together

In our useRecordVoice.tsx hook, let's create a new method that will call our API endpoint. This method will also take the response back and play to the user the audio that we are streaming from the backend.

// app/hooks/useRecordVoice.tsx
...
export function useRecordVoice() {
// new state to track when our server is loading the response for us
const [loading, setLoading] = useState(false);

async function getResponse(audioBlob: Blob) {
  // We transform our audio to base64 to send it to the endpoint
  const audioBase64 = await transformBlobToBase64(audioBlob);

  try {
            setLoading(true);
            // Calling out "chat" endpoint
            const res = await fetch('/api/chat', {
                method: 'POST',
                // Sending our base64 audio here
                body: JSON.stringify({ audio: audioBase64 }),
                headers: {
                    'Content-Type': 'application/json',
                },
            });

            if (!res.ok) {
                throw new Error('Error getting response');
            }

      } catch (err) {
         console.error(err);
      } finally {
        setLoading(false);
      }

}

    useEffect(() => {
        if (mediaRecorder) {
            mediaRecorder.ondataavailable = (e) => {
                audioChunks.current.push(e.data);
            };

            mediaRecorder.onstop = () => {
                const audioBlob = new Blob(audioChunks.current, {
                    type: 'audio/mp3',
                });
                // we call our method here
                void getResponse(audioBlob);
                audioChunks.current = [];
            };
        }
    }, [mediaRecorder]);

...

// app/utils/transform-blob-to-base64.ts
export function transformBlobToBase64(blob: Blob): Promise<string> {
    return new Promise((resolve, reject) => {
        const reader = new FileReader();
        reader.onloadend= () => {
            resolve(reader?.result?.toString().split(',')[1] || '');
        }
        reader.onerror = reject;
        reader.readAsDataURL(blob);
    })
}

Great! Now that we’re getting our streamed response, we need to handle it and play the audio back to the user. We’ll use the AudioContext API for this. This API allows us to store the audio, decode it and play it to the user once it’s ready:

...
async function getResponse(audioBlob: Blob) {
    const audioBase64 = await transformBlobToBase64(audioBlob);

    try {
        setLoading(true);
        const res = await fetch('/api/chat', {
            method: 'POST',
            body: JSON.stringify({ audio: audioBase64 }),
            headers: {
                'Content-Type': 'application/json',
            },
        });

        if (!res.ok) {
            throw new Error('Error getting response');
        }

        // Create an instance of AudioContext
        const audioContext = new AudioContext();

        // Create a reader to read the streaming response
        const reader = res.body?.getReader();
        if (!reader) {
            throw new Error('Error getting response');
        }

        // Create a buffer source to store the audio
        const source = audioContext.createBufferSource();

        // Array to hold the audio chunks received from the backend
        let audioChunks: Uint8Array[] = [];

        // Flag to check if the audio streaming has finished
        let isDataStreamed = false;

        while (!isDataStreamed) {
            // Start reading the data
            const { value, done } = await reader.read();

            // If true, the stream has finished
            if (done) {
                isDataStreamed = true;
                break;
            }

            // Add each data chunk to our list of audio chunks
            if (value) {
                audioChunks.push(value);
            }
        }

        // Merge all buffer chunks into a single Uint8Array
        const audioBuffer = new Uint8Array(
            audioChunks.reduce(
                (acc, val) => acc.concat(Array.from(val)),
                [] as number[]
            )
        );

        // Decode the audio data and store it in our source buffer
        source.buffer = await audioContext.decodeAudioData(
            audioBuffer.buffer
        );

        // Connect the source to the audio output (speakers or headphones)
        source.connect(audioContext.destination);

        // Start playing the audio
        source.start(0);
    } catch (err) {
        console.error(err);
    } finally {
        setLoading(false);
    }
}

...

return {
    startRecording,
    stopRecording,
    isRecording,
    // Return the loading state
    loading,
};

And that's it! Now the user should hear the audio response on their device. To wrap things up, let's make our app a bit nicer by adding a little loading indicator:

// app/components/audio-recorder.tsx

'use client';
import { useRecordVoice } from '@/hooks/useRecordVoice';

export default function AudioRecorder() {
    const { isRecording, stopRecording, startRecording, loading } =
        useRecordVoice();
    async function handleClick() {
        if (isRecording) {
            stopRecording();
        } else {
            await startRecording();
        }
    }

    // New condition
    if (loading) {
        return <div>Loading...</div>;
    }

    return (
        <div>
            <button
                onClick={handleClick}
                className={`bg-blue-500 text-white px-4 py-2 rounded shadow-md hover:bg-blue-400 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2 focus:ring-offset-white transition duration-300 ease-in-out absolute top-1/2 left-1/2 -translate-x-1/2 -translate-y-1/2`}
            >
                {isRecording ? 'Stop Recording' : 'Start Recording'}
            </button>
        </div>
    );
}

Conclusion

In this blog post, we saw how combining multiple AI models can help us achieve our goals. We learned to run AI models like Llama 3.1 locally and use them in our Next.js app. We also discovered how to send audio to these models and stream back a response, playing the audio back to the user.

This is just one of many ways you can use AI—the possibilities are endless. AI models are amazing tools that let us create things that were once hard to achieve with such quality. Thanks for reading; now, it’s your turn to build something amazing with AI!

You can find the complete demo on GitHub: AI Assistant with Whisper TTS and Ollama using Next.js

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

About the author(s)

Jesús Padrón
@padron4497 @jesus4497

The HTML Dialog Element: Enhancing Accessibility and Ease of Use

The HTML Dialog Element: Enhancing Accessibility and Ease of Use Dialogs are a common component added to applications, whether on the web or in native applications. Traditionally there has not been a standard way of implementing these on the web, resulting in many ad-hoc implementations that don’t act consistently across different web applications. Often, commonly expected features are missing from dialogs due to the complexity of implementing them. However, web browsers now offer a standard dialog element. Why use the dialog element? The native dialog element streamlines the implementation of dialogs, modals, and other kinds of non-modal dialogs. It does this by implementing many of the features needed by dialogs for you that are already baked into the browser. This is helpful as it reduces the burden on the developer when making their applications accessible by ensuring that user expectations concerning interaction are met, and it can also potentially simplify the implementation of dialogs in general. Basic usage Adding a dialog using the new tag can be achieved with just a few lines of code. ` However, adding the dialog alone won’t do anything to the page. It will show up only once you call the .showModal() method against it. ` Then if you want to close it you can call the .close() method on the dialog, or press the escape key to close it, just like most other modals work. Also, note how a backdrop appears that darkens the rest of the page and prevents you from interacting with it. Neat! Accessibility and focus management Correctly handling focus is important when making your web applications accessible to all users. Typically you have to move the current focus to the active dialog when showing them, but with the dialog element that’s done for you. By default, the focus will be set on the first focusable element in the dialog. You can optionally change which element receives focus first by setting the autofocus attribute on the element you want the focus to start on, as seen in the previous example where that attribute was added to the close element. Using the .showModal() method to open the dialog also implicitly adds the dialog ARIA role to the dialog element. This helps screen readers understand that a modal has appeared and the screen so it can act accordingly. Adding forms to dialogs Forms can also be added to dialogs, and there’s even a special method value for them. If you add a element with the method set to dialog then the form will have some different behaviors that differ from the standard get and post form methods. First off, no external HTTP request will be made with this new method. What will happen instead is that when the form gets submitted, the returnValue property on the form element will be set to the value of the submit button in the form. So given this example form: ` The form element with the example-form id will have its returnValue set to Submit. In addition to that, the dialog will close immediately after the submit event is done being handled, though not before automatic form validation is done. If this fails then the invalid event will be emitted. You may have already noticed one caveat to all of this. You might not want the form to close automatically when the submit handler is done running. If you perform an asynchronous request with an API or server you may want to wait for a response and show any errors that occur before dismissing the dialog. In this case, you can call event.preventDefault() in the submit event listener like so: ` Once your desired response comes back from the server, you can close it manually by using the .close() method on the dialog. Enhancing the backdrop The backdrop behind the dialog is a mostly translucent gray background by default. However, that backdrop is fully customizable using the ::backdrop pseudo-element. With it, you can set a background-color to any value you want, including gradients, images, etc. You may also want to make clicking the backdrop dismiss the modal, as this is a commonly implemented feature of them. By default, the <dialog> element doesn’t do this for us. There are a couple of changes that we can make to the dialog to get this working. First, an event listener is needed so that we know when the user clicks away from the dialog. ` Alone this event listener looks strange. It appears to dismiss the dialog whenever the dialog is clicked, not the backdrop. That’s the opposite of what we want to do. Unfortunately, you cannot listen for a click event on the backdrop as it is considered to be part of the dialog itself. Adding this event listener by itself will effectively make clicking anywhere on the page dismiss the dialog. To correct for this we need to wrap the contents of the dialog content with another element that will effectively mask the dialog and receive the click instead. A simple element can do! ` Even this isn’t perfect though as the contents of the div may have elements with margins in them that will push the div down, resulting in clicks close to the edges of the dialog to dismiss it. This can be resolved by adding a couple of styles the the wrapping div that will make the margin stay contained within the wrapper element. The dialog element itself also has some default padding that will exacerbate this issue. ` The wrapping div can be made into an inline-block element to contain the margin, and by moving the padding from the parent dialog to the wrapper, clicks made in the padded portions of the dialog will now interact with the wrapper element instead ensuring it won’t be dismissed. Conclusion Using the dialog element offers significant advantages for creating dialogs and modals by simplifying implementation with reasonable default behavior, enhancing accessibility for users that need assistive technologies such as screen readers by using automatic ARIA role assignment, tailored support for form elements, and flexible styling options....

Oct 25, 2024

4 mins

HTMLJavaScript

Making Seamless Page Transitions with the View Transitions API

Make Seamless Page Transitions using the View Transitions API Traditionally web applications have always had a less polished experience, both functionally and visually speaking, compared to native applications on mobile and other platforms. This has been improving over the years with the introduction of new web APIs and standards. One such new API that bridges that gap is the View Transitions API. What is the View Transitions API? The View Transitions API is a new browser feature that allows developers to more easily create animated transitions between pages and views, similar to how mobile apps have transitions when navigating between pages. Adding view transitions to your application can reduce the cognitive load on your users and make the experience feel less inconsistent. One great thing about view transitions is that they can be used by both SPAs (single-page applications) and MPAs (multi-page applications) alike! Using the View Transitions API for SPAs Let’s see how we can use view transitions ourselves. We will use examples adapted from demos created by the wonderful Jake Archibald and Bramus. There are some alterations to a few of the examples to make them work with the newest version of the View Transitions API, but otherwise, they’re mostly the same. The process of getting view transitions working for SPAs is as simple as calling document.startViewTransition right before starting navigation and replacing your page’s content immediately after that. You need to make this call in your router or in an event listener that triggers your page navigation. The following is an example of how you can hook into page navigation in a vanilla JavaScript application. This example uses the Navigation API if it’s available. As of the time of writing this article only Chromium-based browsers support the Navigation API, but there is a fallback to using a click listener on link elements so that this works with Firefox and Safari. ` If you’re using a framework, then there’s probably a better way to do this specific to that framework than by using this contrived example. For example, if you are implementing usage of this API with a Next application, you may opt for something like next-view-transitions. For this article, we’re going to focus on the way this is done in vanilla JavaScript so that the fundamentals of using the API are understood. Knowing how it’s implemented in vanilla JavaScript should give you enough understanding to implement this in the framework of your choice if there isn’t already a library that does it for you. Now we have a way to know when a navigation is being requested. Using the above utility function we can start the animation and replace our content as follows: ` getMyPageContentSomehow can be any function that returns HTML. How that is implemented will depend on your application and framework of choice. If you want to run these examples on your machine, the demo repository has instructions in its README. In essence, it’s as simple as hosting an HTTP server at the root of the project and visiting localhost. No fancy build system is required! By default, a fade is done that only lasts for a moment. The animation that happens can be configured using CSS with the ::view-transition-group pseudo-element on the element you want to animate, which in our case is the entire page, and we pass in root. ` Given that the animation properties are used, you have much control over what kind of animation to do. You can change the duration and the interpolation, add keyframes and gradients, and even add image masks. You can learn more about the animation properties in CSS on MDN. Using the View Transitions API for MPAs As mentioned, view transitions also work for MPAs with no client-side routing. Setting up view transitions for MPAs is much easier than for SPAs. All you have to do is include the CSS rule on all pages where you want the animations. ` Of course, like how it works with SPAs, you can customize the animations to your heart’s content using the view transition pseudo-elements. Conclusion I hope this article helped you understand the fundamentals of using the View Transitions API. Now that most browsers support it and it’s not too difficult to add it to existing applications, it is a great time to learn how to use it. You can view a full list of examples here (made by Jake Archibald and Bramus) if you want to see how advanced the animations can get. I like the Star Wars ones especially. The README explains how to get all the examples up and running (a total of 43!)....

Aug 2, 2024

4 mins

JavaScript

How Vim Transformed My Workflow for the Better

> Before getting into this article, I want to say that this is just my personal experience. I don’t think Vim is the best editor for everyone, but it worked well for me. What is Vim for you who don’t know? Well, Vim is a text editor, which is a tool you use to write and edit text or code on your computer. What makes Vim unique is that it has lots of keyboard shortcuts. Instead of using the mouse to click around, you can keep your fingers on the keyboard, making you faster once you get the hang of it. I've always heard good things about Vim. People say it’s great and that once you learn it, it can speed up your coding. Vim users sometimes act like they’re in a special club, which is funny. But jokes aside, I was curious because of all the good things I heard, especially about how comfy and efficient it is for coding. I knew learning Vim wouldn’t be easy. From past experiences (like learning touch typing), I knew it would be like climbing a steep hill. It's akin to learning to ride a bike with special gear. At first, it might feel tricky, but once you get used to it, you can ride faster and smoother. There’s a stage I like to call the "Hybrid" point. It's when you’re getting to grips with something new, and you're not proficient at it yet, but you also start losing comfort with the old way of doing things. This phase is a bit frustrating, and I wasn’t sure I wanted to go through this again as it could have led to a decrease in productivity, which was the opposite of what I was aiming for. However, the prospect of a smoother and faster coding experience kept me excited to try Vim. Discovering a New Coding Comfort Zone So, I began my journey with Vim. I started learning the basic moves by chatting with GPT and playing Vim Snake – A fun game that's like the classic snake game, but with Vim controls – Slowly but surely, I got the hang of it. Once I felt more at ease, I decided to add a Vim plugin to my IDE. Right now, I'm using IntelliJ. This plugin let me use Vim right inside IntelliJ. It was the best of both worlds—I could use Vim’s quick commands and still have my IDE’s features, plus the option to use my mouse when I wanted to. In the beginning, the transition was rough. I felt out of my element, constantly reaching for my mouse. My coding speed took a hit as I was making more mistakes. For instance, I’d try to edit a line of code, and end up deleting it or accidentally switching to another tab. It was frustrating. But I pushed through the initial hurdles, working in this odd hybrid mode with both my mouse and Vim commands. Gradually, things started to click and my workflow began to improve. As the weeks went by, I got more and more comfortable. There were times I’d code without touching the mouse, and let me tell you, it felt AMAZING. Here's why: Just using your keyboard to zip around your code is a total game-changer. Here's a simple example to illustrate this. Let's say you want to move the line of code you're on to the line below. Normally, you'd grab your mouse, highlight the code, cut it, hit enter to create a new line, paste the code, then put the mouse down to continue coding. With Vim, you just press the "dd" keys to cut and copy the line, then “p” to paste it below. And just like that, you're done! It may seem small, but when all these little time-savers add up, you end up saving a TON of time. Without Vim With Vim --- But the magic of Vim doesn't end there. You can tweak Vim in numerous ways by adding your own commands, modifying existing ones, or loading up on plugins to extend its functionality. For instance, I tried out the easy-motion plugin and it’s been a game-changer for navigating through files. I also set up shortcuts to switch between tabs, which has made things even smoother. With all these enhancements, I found myself breezing through code like a pro and seeing what I needed in no time. And if I ever need a new command, a quick chat with GPT sets me right back on track! The more I explored Vim, the more I discovered its immense potential. It's not just a text editor; it's a powerful tool that can be molded to fit your coding style. From creating custom key mappings to defining your own text objects, Vim allowed me to tailor my editing environment exactly how I wanted. And the community around Vim is incredible. There’s a plethora of resources: tutorials like vimtutor, courses such as Vim Basics Course, Learning Vim in a Week, interactive apps, blogs, and plugins available that can help overcome any challenge I faced. It's been a rewarding journey that not only improved my efficiency but also broadened my perspective on what's possible with the right tools. The journey with Vim has also opened up a new world of exploration for me. I’ve started checking out other text editors and tools that prioritize keyboard use, seeking to refine my setup further. Each new discovery, whether it's a handy plugin or a clever command, feels like unlocking a new level in a game. It’s exciting to think that there's so much more to learn and that I can continue to fine-tune my workflow. I know there's still much to explore, but the excitement keeps growing—it's genuinely enjoyable! Up next on my adventure is diving into Neovim, and I can't wait to share my experiences with it. Conclusion It's clear that stepping out of my comfort zone and delving into Vim has been nothing short of transformative. But it revamped my coding workflow and ignited a newfound curiosity to explore and learn. The hurdles encountered along the way were just stepping stones to achieving a level of proficiency that I now enjoy. Vim, with its simplicity yet profound flexibility, has proven to be more than just a text editor—it's a coder's companion on the journey towards efficiency and mastery. As I venture further, experimenting with Neovim and other tools, the horizon of possibilities keeps expanding. In the realm of coding, time is of the essence, and Vim has gifted me the ability to reclaim it, one keystroke at a time. It's not just about the shortcuts or the plugins; it's about the mindset of continuous learning and improvement that Vim instills. This blog post is not just an endorsement of Vim but a testament to the boundless exploration that awaits when one is willing to embrace new tools and methodologies....

Dec 20, 2023

5 mins

VimToolingSoftware EngineeringTips

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

Understanding the Difference Between :focus and :focus-visible in CSS I have learned my fair share about the importance of keyboard accessibility, so I know that visual indication of the focused element is very important. But the well-known :focus pseudo-class is not always the best fit for this job. That's where :focus-visible comes in. Let's look at the differences between these two pseudo-classes and explore the best practices for using them effectively. What is the :focus Pseudo-Class? The :focus pseudo-class is a CSS selector that applies styles to any element that receives focus, regardless of how that focus was triggered. This includes focus events from keyboard navigation, mouse clicks, and touch interactions. Example Usage of :focus ` In this example, the button will display a blue outline whenever it is focused, whether the user clicks on it with a mouse, taps it on a touchscreen, or navigates to it using the keyboard. What is the :focus-visible Pseudo-Class? The :focus-visible pseudo-class is more specialized. It only applies styles to an element when the browser determines that the focus should be visible. This typically occurs when the user navigates via the keyboard or assistive technologies rather than through mouse or touch input. Example Usage of :focus-visible ` Here, the button will only show a blue outline when focused through keyboard navigation or another input method that usually requires visible focus indicators. Key Differences Between :focus and :focus-visible :focus - Behavior: Applies to any element that receives focus, regardless of the input method. - Use Cases: Ensures that all interactions with the element are visually indicated, whether by mouse, keyboard, or touch. :focus-visible - Behavior: Applies styles only when the focus should be visible, such as using a keyboard or assistive technology. - Use Cases: Ideal for scenarios where you want to provide focus indicators only to keyboard and assistive technology users while avoiding unnecessary outlines for mouse and touch users, typically required by design. Accessibility Implications :focus - Pros: - Guarantees that all users can see when an element is focused, which is critical for accessibility. - Cons: - Can lead to a suboptimal experience for mouse users, as focus styles may appear unnecessarily during mouse interactions. :focus-visible - Pros: - Enhances user experience by showing focus indicators only when necessary, thus keeping the interface clean for mouse and touch users. - Tailors the experience for keyboard and assistive technology users, providing them with clear visual cues. - Cons: - Additional considerations may be required to ensure that focus indicators are not accidentally omitted, especially in older browsers that do not support :focus-visible. - There may be cases where you want to show focus indicators for all users, regardless of input method. Best Practices for Using :focus and :focus-visible To achieve the best accessibility and user experience, combining both :focus and :focus-visible in your CSS is often a good idea. Combining :focus and :focus-visible ` Here is a Stackblitz example of what such styling could look like for you to try out and play with. Additional Tips - Test with Keyboard and Assistive Technology: Ensure that your web application is navigable using a keyboard (Tab, Shift + Tab, etc.) and that focus indicators are visible for those who rely on them. It's never a bad idea to include accessibility testing in your e2e testing suite. - Provide Clear Focus Indicators: Make sure that focus indicators are prominent and easy to see. A subtle or hard-to-spot focus indicator can severely impact accessibility for users who rely on keyboard navigation. Conclusion The :focus-visible pseudo-class offers a more refined way to manage focus indicators, improving accessibility and user experience, particularly for keyboard and assistive technology users. By understanding the differences between :focus and :focus-visible, and applying best practices in your CSS, you can create more accessible and user-friendly web applications. Remember, accessibility should never be an afterthought. By thoughtfully applying focus styles, you ensure that all users, regardless of how they interact with your site, can easily navigate and interact....

Nov 8, 2024

3 mins

AccessibilityCSSPlaywright

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

How to build an AI assistant with OpenAI, Vercel AI SDK, and Ollama with Next.js

Getting started

Kicking things off

Building our client logic

Setting up our Server side

Putting It All Together

Conclusion

Jesús Padrón

You might also like

The HTML Dialog Element: Enhancing Accessibility and Ease of Use

Making Seamless Page Transitions with the View Transitions API

How Vim Transformed My Workflow for the Better

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

Let's innovate together!

You might also like

The HTML Dialog Element: Enhancing Accessibility and Ease of Use

Making Seamless Page Transitions with the View Transitions API

How Vim Transformed My Workflow for the Better

Understanding the Difference Between `:focus` and `:focus-visible` in CSS