General

ChatGPT can't solve these problems! with Chris Gardner

Published Jan 16, 2024

Updated Jan 16, 2024

1 min read

Chris Gardner talks neural networks, covering fundamental concepts, historical development, and practical applications. It’s important to understand the difference between artificial intelligence (AI) and machine learning, and the role of neural networks in solving intricate problems with vast datasets.

This conversation centers around the intricacies of training neural networks, particularly as they become more complex with multiple layers. Chris and Rob touch on the fascinating yet daunting nature of neural networks, discussing their ability to learn beyond human comprehension.

Turning to the practical side of using neural networks, Chris shares the existence of libraries that exist to simplify the process of building a network, enabling users to input data, specify parameters, and entrust the system with the training.

Both caution about the biases inherent in the data and the responsibility associated with working on machine learning models. They address challenges related to ethics, highlighting the difficulties in identifying biases and emphasizing the delicate balance between excitement and caution in the evolving field of machine learning.

Listen to the podcast here: https://modernweb.podbean.com/e/chris-gardner/

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

Let Purpose Drive Your Artificial Intelligence Transformation

From administrative and analytics tasks present in nearly every industry, to niche processes that serve only a handful of businesses, the diversity of how we apply Artificial Intelligence grows, seemingly by the moment. And with it, the size of the market is exploding. In fact, less than half a decade ago, this market was valued at $644 million, and half a decade from now, it is expected to be worth nearly $118 billion. When discussing the growth of AI, many have a hard time separating it from the advancement of AI. While the latter will naturally follow the former, the vast majority of companies that are integrating, and expanding the size of this market, will be in a constant state of catching up to some of the world’s foremost leaders in AI advancement. And I’m here to tell you, that’s okay. Because, as the growth of the AI market continues, the diversity in humans that will use it, and human demand that will be met by it, will grow too. And your responsibility as an executive or leader is not necessarily to push the envelope of AI, or discover the next game changing function at which the world can marvel; it is to meet the needs of the humans, be they your employees or customers, who will benefit from your integration. Of course, this is a very exciting time in the history of advanced digital technologies. Despite how far we have come, when we consider the potentially unending future of this transformative technology, we realize that AI is still in its infancy. As such, we often approach from a performance advancement mindset. However, if you confine your definition of performance to objective technical metrics, you may find yourself perpetually pursuing faster load times, smaller bundle sizes, and more impressive features. This may work out for your team if you have an unending line of investors, or get incredibly lucky. But even some of the biggest companies that neglected to first think of their user experience above all else have found themselves victims of Icarian pitfalls that have shelved their projects, depleted their financial resources, and denied users of products that they need. I do want to note, however, that the focus of this article isn’t to suggest that we shouldn’t strive to advance our understanding of the “nuts and bolts” of AI and other transformative digital technologies. But it’s time to make tough decisions about your company’s place in the AI marketplace. Do you have the budget, the demand, or a responsibility that requires you to implement the most cutting edge technologies available to the market? The answer for most companies is, no. For many, their future with AI is a continual, and incremental process of meeting the minimal needs of their customers, employees, or processes. And I am here to say that not only is this okay, but it’s a good thing. In this article, I implore you to look at AI as a tool for promoting human excellence and experience, and rethink what it means for AI technology to be “performant”. Ultimately, I want you to feel inspired and empowered to begin your digital transformation voyage by placing people and purpose at the helm. RETHINKING PERFORMANCE According to leading educational non-profit Autism Speaks, 1 in every 59 children born today will fall somewhere on the autism spectrum. Although symptoms are diverse, many individuals with this diagnosis struggle with normative social conventions in a manner that impacts their daily lives. Due to advancements in our understanding of autism, it’s now typically diagnosed in early childhood, tasking many parents with the responsibility of helping their children navigate a world that better accommodates neurotypical behaviors. Laura Krieger is one of these parents. Matthew, her eight year old son, has autism, and struggles to read others’ emotions. The pair were featured in a PBS Newshour segment on Brain Power, a company founded by award winning neuroscientist and entrepreneur, Ned Sahin. Brain Power’s product is a Google Glass aided software that utilizes both AI and augmented reality technologies to help kids with learning differences build skills such as identifying emotions, and maintaining eye contact, through play. When Krieger plays one of these skill building games with her son, she can’t help but break out in tears, saying she feels like he can truly see her for the first time. Krieger is not responding to just how low the latency of the codebase powering the software is, or whether its deploying the latest architectural concepts. She likely doesn’t care whether the software was created using Amazon Web Services, TensorFlow, or a proprietary system. What she does care about is getting the chance to connect with her son in a way she never thought possible. And when we look at Brain Power through the eyes of children who use it, we see its strength in its mode of education. Rather than running users through drills or lesson plans, it rewards them through collaboration, natural interaction, and a gamified points system. When you see the children using the system, they aren’t marveling at the natural feel of the interface that betrays its complexity. They’re simply having fun. This is the type of performance for which we should strive. THE CASE OF WATSON FOR ONCOLOGY In 2013, IBM launched a partner project with The University of Texas MD Anderson Cancer Center to create a “Watson for Oncology” software that would help doctors identify and prescribe courses of action for cancer treatment. After pumping $62 million dollars into the project, MD Anderson officially shelved it in 2017, halting their pursuit of a cure for cancer. But the problem wasn’t a technical one. That’s to say, it was not found within the codebase. It was with the data being processed by it. The reason that MD Anderson pulled the plug on the project, which would eventually be revealed by StatNews after reviewing internal slide decks from MD Anderson, was that the program was prescribing “unsafe and incorrect” treatment plans to real patients after the product had been sold to hospitals around the world. Anderson sourced the problem back to IBM engineers and New York City-based Memorial Sloan Kettering Cancer Center, who were responsible for training Watson for Oncology. It was later discovered that their ML training process relied on a relatively small collection of hypothetical oncological cases rather than using actual patient data. Of course, we cannot presuppose intent. I am of the opinion that IBM and Kettering truly wanted to provide a product that would revolutionize oncology treatment, and save lives. But it is hard to imagine a reason why a company creating a product meant to help doctors treat their patients in the field, would not have trained their AI software with data produced by doctors working with actual cancer patients. So after indefinitely stopping this project, MD Anderson had exchanged $62 million, and four years of its time, to create a product that may have reflected some of the most advanced technical concepts available at that time, but is completely useless, in its current form, for the purpose it was meant to serve. Imagine where we could be, how many lives could have been saved or prolonged, and how much money would have been saved, in the nearly three years since this product was pulled, if developers had placed equal focus on their users and the purpose of their product as they did the technical elements. MEETING NEEDS *“As researchers, we make decisions about what our AI systems can do. It may not necessarily be optimal. It may not be necessarily efficient if you look at all of the metrics… but it may be optimal with respect to the human… which means that the system works.”* - Ayanna Howard, Ph.D Chair, School of Interactive Computing Georgia Institute of Technology Kaden Bowen of Lincoln, Nebraska shares his father’s passion for cars. Though he is non-verbal due to cerebral palsy, he asks his father, James, to “go for a ride”, multiple times a day with the help of a digital talkboard. James dreamed of taking his son on a roadtrip in a vehicle that was more stylish and fun than their wheelchair-accessible van. He searched high and low for a sporty car that he could modify to at least allow him to store a foldable wheelchair, until he realized that the hatchback of a standard Corvette might just be big enough for one. Two weeks later, the duo took a 728 mile round trip to the Corvette museum in Bowling Green, Kentucky in their brand-new (to them) Corvette, which came standard with a trunk big enough for Kaden’s chair. The Bowens don’t seem to over-complicate the accommodations that they have put in place for their son. To help increase his level of independence, they outfitted their home with Amazon Echo devices that are sensitive enough to understand the commands that are programmed into Kaden’s talkboard. Using a combination of the two technologies, Kaden can do a lot on his own. He can call his parents on the phone, stream videos on Netflix, and operate lights. It seems so obvious, but it really is quite inventive. Sure, one day we will have widely accessible neural link technology that will allow Kaden to circumvent the talkboard, and do so much more than what is permitted by an Echo. But just like his dad’s Corvette, that comes standard with enough storage space, sometimes, the best fixes are the best fixes because they are available, and they work. The technologies that Kaden uses may not boast the best metrics, feature the most jaw dropping functions, or offer the most direct route to helping him achieve increased independence. I’m sure that the communication between the talkboard and the Echo device is not always perfect. But the combination of these two relatively affordable, and accessible technologies have given Kaden more autonomy with reliable, usable services, and that, in and of itself, *is* high performance. AI INITIATIVES ARE FAILING These past few years have proven to be extremely exciting for AI technologies, and businesses are responsive, with a 2019 Gartner report showing that 37% of the nation’s leading enterprises are or are shortly planning to integrate some form of AI into their products or processes. This percentage reflects a 270% growth in that statistic when compared to research conducted in 2015. And this proportion bumps up to 62% when we look at Supply Chain and Logistics, and reaches nearly 80% when discussing the Healthcare industry. But these awe-inspiring stats come with troubling predictions that, through 2020, roughly 80% of enterprise AI programs will remain in a limbotic state of development due to their inability to properly scale with their organizations. In essence, we may be creating fabulous algorithms, with wildly impressive features and functions that may become little more than multi-million dollar proof-of-concepts. And this might be okay for some businesses who have the resources to pursue multiple avenues and digital transformation. But I am of the belief that most companies that have yet to enter AI space by now will depend on a significant ROI from the programs that they start over the next few years. For these companies, it is imperative not only to balance resources, and create realistic time-frames for integrating and/or shipping their products, but to stay diligent against the propensity for AI programs to become isolated within an organization. With all of the buzz around AI, and the seemingly endless supply of jaw dropping products and services coming out of the world’s foremost information technology companies, it’s natural that smaller programs want to keep up with the Joneses. It is, therefore, the responsibility of executives, and others in business development roles, to remain involved with their AI development programs to ensure that products and services do not outpace the demands and capacities of their customers or businesses. WORKING SYSTEMS *“The current wave of Artificial Intelligence is going to hit a peak inside of the enterprise. But when it does, it’s not going to be a monumental revolution of technology, but rather a monumental revolution of people.”* - Traci Gusher Partner, US Leader- Artificial Intelligence Analytics and Engineering, KPMG There is a lot of chatter about the need for enterprises to integrate AI powered technologies into their workflows, and products. Anxiety about the need to introduce this transformative technology is warranted. Products like Alexa, Siri, and Echo Dot are changing consumer expectations, while Forrestor predicts that, in 2020, 25% of Fortune 500 companies will include AI building blocks in their automated processes. From product to process, AI is infiltrating nearly every sect of business. That being said, it is a mistake for companies to rush an AI program without first internalizing and developing concepts of how the resulting products will better the lives or capacities of those who will use it. I get it. It’s so tempting to want to patent the next game changing system or algorithm, but if your customers, users, or employees, could be equally, if not better served by a simpler system, or another company’s proprietary tools, what is the point? Is it 100% necessary that your business be on the cutting edge of Artificial Intelligence? Will it truly better your products, services, workflows, or customers? AI has such an amazing capacity to enrich the human experience, be that personal, or professional. We overemphasize minute technical details without giving that same attention to the aspects of user needs, and experience at our own peril. Don’t feel the need to push the boundaries of our understanding of transformative technologies simply to implement your own products and services. All you need is something that sees your user, empowers them, strengthens their skills, and most importantly, works. Ready to begin your digital transformation journey, but don’t know how to start? Don’t hesitate to reach out to the team at This Dot Labs by emailing hi@thisdot.co....

Feb 21, 2020

10 mins

How to build an AI assistant with OpenAI, Vercel AI SDK, and Ollama with Next.js

In today’s blog post, we’ll build an AI Assistant using three different AI models: Whisper and TTS from OpenAI and Llama 3.1 from Meta. While exploring AI, I wanted to try different things and create an AI assistant that works by voice. This curiosity led me to combine OpenAI’s Whisper and TTS models with Meta’s Llama 3.1 to build a voice-activated assistant. Here’s how these models will work together: * First, we’ll send our audio to the Whisper model, which will convert it from speech to text. * Next, we’ll pass that text to the Llama 3.1 model. Llama will understand the text and generate a response. * Finally, we’ll take Llama’s response and send it to the TTS model, turning the text back into speech. We’ll then stream that audio back to the client. Let’s dive in and start building this excellent AI Assistant! Getting started We will use different tools to build our assistant. To build our client side, we will use Next.js. However, you could choose whichever framework you prefer. To use our OpenAI models, we will use their TypeScript / JavaScript SDK. To use this API, we require the following environmental variable: OPENAI_API_KEY— To get this key, we need to log in to the OpenAI dashboard and find the API keys section. Here, we can generate a new key. Awesome. Now, to use our Llama 3.1 model, we will use Ollama and the Vercel AI SDK, utilizing a provider called ollama-ai-provider. Ollama will allow us to download our preferred model (we could even use a different one, like Phi) and run it locally. The Vercel SDK will facilitate its use in our Next.js project. To use Ollama, we just need to download it and choose our preferred model. For this blog post, we are going to select Llama 3.1. After installing Ollama, we can verify if it is working by opening our terminal and writing the following command: Notice that I wrote “llama3.1” because that’s my chosen model, but you should use the one you downloaded. Kicking things off It's time to kick things off by setting up our Next.js app. Let's start with this command: ` After running the command, you’ll see a few prompts to set the app's details. Let's go step by step: * Name your app. * Enable app router. The other steps are optional and entirely up to you. In my case, I also chose to use TypeScript and Tailwind CSS. Now that’s done, let’s go into our project and install the dependencies that we need to run our models: ` Building our client logic Now, our goal is to record our voice, send it to the backend, and then receive a voice response from it. To record our audio, we need to use client-side functions, which means we need to use client components. In our case, we don’t want to transform our whole page to use client capabilities and have the whole tree in the client bundle; instead, we would prefer to use Server components and import our client components to progressively enhance our application. So, let’s create a separate component that will handle the client-side logic. Inside our app folder, let's create a components folder, and here, we will be creating our component: ` Let’s go ahead and initialize our component. I went ahead and added a button with some styles in it: ` And then import it into our Page Server component: ` Now, if we run our app, we should see the following: Awesome! Now, our button doesn’t do anything, but our goal is to record our audio and send it to someplace; for that, let us create a hook that will contain our logic: ` We will use two APIs to record our voice: navigator and MediaRecorder. The navigator API will give us information about the user’s media devices like the user media audio, and the MediaRecorder will help us record the audio from it. This is how they’re going to play out together: ` Let’s explain this code step by step. First, we create two new states. The first one is for keeping track of when we are recording, and the second one stores the instance of our MediaRecorder. ` Then, we’ll create our first method, startRecording. Here, we are going to have the logic to start recording our audio. We first check if the user has media devices available thanks to the navigator API that gives us information about the browser environment of our user: If we don’t have media devices to record our audio, we just return. If they do, then let us create a stream using their audio media device. ` Finally, we go ahead and create an instance of a MediaRecorder to record this audio: ` Then we need a method to stop our recording, which will be our stopRecording. Here, we will just stop our recording in case a media recorder exists. ` We are recording our audio, but we are not storing it anywhere. Let’s add a new useEffect and ref to accomplish this. We would need a new ref, and this is where our chunks of audio data will be stored. ` In our useEffect we are going to do two main things: store those chunks in our ref, and when it stops, we are going to create a new Blob of type audio/mp3: ` It is time to wire this hook with our AudioRecorder component: ` Let’s go to the other side of the coin, the backend! Setting up our Server side We want to use our models on the server to keep things safe and run faster. Let’s create a new route and add a handler for it using route handlers from Next.js. In our App folder, let’s make an “Api” folder with the following route in it: We want to use our models on the server to keep things safe and run faster. Let’s create a new route and add a handler for it using route handlers from Next.js. In our App folder, let’s make an “Api” folder with the following route in it: ` Our route is called ‘chat’. In the route.ts file, we’ll set up our handler. Let’s start by setting up our OpenAI SDK. ` In this route, we’ll send the audio from the front end as a base64 string. Then, we’ll receive it and turn it into a Buffer object. ` It’s time to use our first model. We want to turn this audio into text and use OpenAI’s Whisper Speech-To-Text model. Whisper needs an audio file to create the text. Since we have a Buffer instead of a file, we’ll use their ‘toFile’ method to convert our audio Buffer into an audio file like this: ` Notice that we specified “mp3”. This is one of the many extensions that the Whisper model can use. You can see the full list of supported extensions here: https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-file Now that our file is ready, let’s pass it to Whisper! Using our OpenAI instance, this is how we will invoke our model: ` That’s it! Now, we can move on to the next step: using Llama 3.1 to interpret this text and give us an answer. We’ll use two methods for this. First, we’ll use ‘ollama’ from the ‘ollama-ai-provider’ package, which lets us use this model with our locally running Ollama. Then, we’ll use ‘generateText’ from the Vercel AI SDK to generate the text. Side note: To make our Ollama run locally, we need to write the following command in the terminal: ` ` Finally, we have our last model: TTS from OpenAI. We want to reply to our user with audio, so this model will be really helpful. It will turn our text into speech: ` The TTS model will turn our response into an audio file. We want to stream this audio back to the user like this: ` And that’s all the whole backend code! Now, back to the frontend to finish wiring everything up. Putting It All Together In our useRecordVoice.tsx hook, let's create a new method that will call our API endpoint. This method will also take the response back and play to the user the audio that we are streaming from the backend. ` Great! Now that we’re getting our streamed response, we need to handle it and play the audio back to the user. We’ll use the AudioContext API for this. This API allows us to store the audio, decode it and play it to the user once it’s ready: ` And that's it! Now the user should hear the audio response on their device. To wrap things up, let's make our app a bit nicer by adding a little loading indicator: ` Conclusion In this blog post, we saw how combining multiple AI models can help us achieve our goals. We learned to run AI models like Llama 3.1 locally and use them in our Next.js app. We also discovered how to send audio to these models and stream back a response, playing the audio back to the user. This is just one of many ways you can use AI—the possibilities are endless. AI models are amazing tools that let us create things that were once hard to achieve with such quality. Thanks for reading; now, it’s your turn to build something amazing with AI! You can find the complete demo on GitHub: AI Assistant with Whisper TTS and Ollama using Next.js...

Sep 27, 2024

8 mins

AINextJSJavaScript

Keeping Costs in Check When Hosting Next.js on Vercel

Vercel is usually the go-to platform for hosting Next.js apps, and not without reason. Not only are they one of the sponsors of Next.js, but their platform is very straightforward to use, not just for Next.js but for other frameworks, too. So it's no wonder people choose it more and more over other providers. Vercel, however, is a serverless platform, which means there are a few things you need to be aware of to keep your costs predictable and under control. This blog post covers the most important aspects of hosting a Next.js app on Vercel. Vercel's Pricing Structure Vercel's pricing structure is based on fixed and usage-based pricing, which is implemented through two big components of Vercel: the Developer Experience Platform (DX Platform) and the Managed Infrastructure. The DX Platform offers a monthly-billed suite of tools and services focused on building, deploying, and optimizing web apps. Think of it as the developer-focused interface on Vercel, which assists you in managing your app and provides team collaboration tools, deployment infrastructure, security, and administrative services. Additionally, it includes Vercel support. Because the DX Platform is developer-focused, it's also charged per seat on a monthly basis. The more developers have access to the DX Platform, the more you're charged. In addition to charging per seat, there are also optional, fixed charges for non-included, extra features. Observability Plus is one such example feature. The Managed Infrastructure, on the other hand, is what makes your app run and scale. It is a serverless platform priced per usage. Thanks to this infrastructure, you don't need to worry about provisioning, maintaining, or patching your servers. Everything is executed through serverless functions, which can scale up and down as needed. Although this sounds great, this is also one of the reasons many developers hesitate to adopt serverless; it may have unpredictable costs. One day, your site sees minimal traffic, and the next, it goes viral on Hacker News, leading to a sudden spike in costs. Vercel addresses this uncertainty by including a set amount of free serverless usage in each of its DX Platform plans. Once you exceed those limits, additional charges apply based on usage. Pricing Plans The DX Platform can be used in three different pricing plans on a team level. A team can represent a single person, a team within a company, or even a whole company. When creating a team on Vercel, this team can have one or more projects. The first of the three pricing plans is the Hobby plan, which is ideal for personal projects and experimentation. The Hobby plan is free and comes with some of the features and resources of the DX Platform and Managed Infrastructure out of the box, making it suitable for hosting small websites. However, note that the Hobby plan is limited to non-commercial, personal use only. The Pro plan is the most recommended for most teams and can be used for commercial purposes. At the time of this writing, it costs $20 per team member and comes with generous resources that support most teams. The third tier is the Enterprise plan, which is the most advanced and expensive option. This plan becomes necessary when your application requires specific compliance or performance features, such as isolated build infrastructure, Single Sign-On (SSO) for corporate user management or custom support with Service-Level Agreements. The Enterprise plan has a custom pricing model and is negotiated directly with Vercel. What Contributes to Usage Costs and Limiting Them Now that you've selected a plan for your team, you're ready to deploy Next.js. But how do you determine what contributes to your per-usage costs and ensure they stay within your plan limits? The Vercel pricing page has a detailed breakdown of the resource usage limits for each plan, which can help you understand what affects your costs. However, in this section, we've highlighted some of the most impactful factors on pricing that we've seen on many of our client projects. Number of Function Invocations Each time a serverless function runs, it counts as an invocation. Most of the processing on Vercel for your Next.js apps happens through serverless functions. Some might think that only API endpoints or server actions count as serverless function invocations, but this is not true. Every request that comes to the backend goes through a serverless function invocation, which includes: - Invoking server actions (server functions) - Invoking API routes (from the frontend, another system, or even within another serverless function) - Rendering a React server component tree (as part of a request to display a page) To give you an idea of the number of invocations included in a plan, the Pro plan includes 1 million invocations per month for free. After that, it costs $0.60 per million, which can total a significant amount for popular websites. To minimize serverless function invocations, focus on reducing any of the above points. For example: - Batch up server actions: If you have multiple server actions, such as downloading a file and increasing its download count, you can combine them into one server action. - Minimize calls to the backend: Closely related to the previous point, unoptimized client components can call the backend more than they need to, contributing to increased function invocation count. If needed, consider using a library such as useSwr or TanStack Query to keep your backend calls under control. - Use API routes correctly: Next.js recommends using API routes for external systems invoking your app. For instance, Contentful can invoke a blog post through a webhook without incurring additional invocations. However, avoid invoking API routes from server component tree renders, as this counts as at least two invocations. Reducing React server component renders is also possible. Not all pages need to be dynamic - convert dynamic routes to static content when you don’t expect them to change in real-time. On the client, utilize Next.js navigation primitives to use the client-side router cache. Middleware in Next.js runs before every request. Although this doesn't necessarily count as a function invocation (for edge middleware, this is counted in a separate bucket), it's a good idea to minimize the number of times it has to run. To minimize middleware invocations, limit them only to requests that require it, such as protected routes. For static asset requests, you can skip middleware altogether using matchers. For example, the matcher configuration below would prevent invoking the middleware for most static assets: ` Function Execution Time The duration your serverless function takes to execute counts as the execution time, and it impacts your end bill unless it's within the limits of your plan. This means that any inefficient code that takes longer to execute directly adds up to the total function invocation time. Many things can contribute to this, but one common pattern we've seen is not utilizing caching properly or under-caching. Next.js offers several caching techniques you can use, such as: - Using a data cache to prevent unnecessary database calls or API calls - Using memoization to prevent too many API or database calls in the same rendering pass Another reason, especially now in the age of AI APIs, is having a function run too long due to AI processing. In this case, what we could do is utilize some sort of queuing for long-processing jobs, or enable Fluid Compute, a recent feature by Vercel that optimizes function invocations and reusability. Bandwidth Usage The volume of data transferred between users and Vercel, including JavaScript bundles, RSC payload, API responses, and assets, directly contributes to bandwidth usage. In the Pro plan, you receive 1 TB/month of included bandwidth, which may seem substantial but can quickly be consumed by large Next.js apps with: - Large JavaScript bundles - Many images - Large API JSON payloads Image optimization is crucial for reducing bandwidth usage, as images are typically large assets. By implementing image optimization, you can significantly reduce the amount of data transferred. To further optimize your bandwidth usage, focus on using the Link component efficiently. This component performs automatic prefetch of content, which can be beneficial for frequently accessed pages. However, you may want to disable this feature for infrequently accessed pages. The Link component also plays a role in reducing bandwidth usage, as it aids in client-side navigation. When a page is cached client-side, no request is made when the user navigates to it, resulting in reduced bandwidth usage. Additionally, API and RSC payload responses count towards bandwidth usage. To minimize this impact, always return only the minimum amount of data necessary to the end user. Image Transformations Every time Vercel transforms an image from an unoptimized image, this counts as an image transformation. After transformation, every time an optimized image is written to Vercel's CDN network and then read by the user's browser, this counts as an image cache read and an image cache write, respectively. The Pro plan includes 10k transformations per month, 600k CDN cache reads, and 200k CDN cache writes. Given the high volume of image requests in many apps, it's worth checking if the associated costs can be reduced. Firstly, not every image needs to be transformed. Certain types of images, such as logos and icons, small UI elements (e.g., button graphics), vector graphics, and other pre-optimized images you may have optimized yourself already, don't require transformation. You can store these images in the public folder and use the unoptimized property with the Image component to mark them as non-transformable. Another approach is to utilize an external image provider like Cloudinary or AWS CloudFront, which may have already optimized the images. In this case, you can use a custom image loader to take advantage of their optimizations and avoid Vercel's image transformations. Finally, Next.js provides several configuration options to fine-tune image transformation: - images.minimumCacheTTL: Controls the cache duration, reducing the need for rewritten images. - images.formats: Allows you to limit eligible image formats for transformation. - images.remotePatterns: Defines external sources for image transformation, giving you more control over what's optimized. - images.quality: Enables you to set the image quality for transformed images, potentially reducing bandwidth usage. Monitoring The "Usage" tab on the team page in Vercel provides a clear view of your team's resource usage. It includes information such as function invocation counts, function durations, and fast origin transfer amounts. You can easily see how far you are from reaching your team's limit, and if you're approaching it, you'll see the amount. This page is a great way to monitor regularity. However, you don't need to check it constantly. Vercel offers various aspects of spending management, and you can set alert thresholds to get notified when you're close to or exceed your limit. This helps you proactively manage your spending and avoid unexpected charges. One good feature of Vercel is its ability to pause projects when your spending reaches a certain point, acting as an "emergency break" in the case of a DDoS attack or a very unusual spike in traffic. However, this will stop the production deployment, and the users will not be able to use your site, but at least you won't be charged for any extra usage. This option is enabled by default. Conclusion Hosting a Next.js app on Vercel offers a great developer experience, but it's also important to consider how this contributes to your end bill and keep it under control. Hopefully, this blog post will clear up some of the confusion around pricing and how to plan, optimize, and monitor your costs. We hope you enjoyed this blog post. Be sure to check out our other blog posts on Next.js for more in-depth coverage of different features of this framework....

Mar 28, 2025

10 mins

VercelNextJS

Incremental Hydration in Angular

Incremental Hydration in Angular Some time ago, I wrote a post about SSR finally becoming a first-class citizen in Angular. It turns out that the Angular team really treats SSR as a priority, and they have been working tirelessly to make SSR even better. As the previous blog post mentioned, full-page hydration was launched in Angular 16 and made stable in Angular 17, providing a great way to improve your Core Web Vitals. Another feature aimed to help you improve your INP and other Core Web Vitals was introduced in Angular 17: deferrable views. Using the @defer blocks allows you to reduce the initial bundle size and defer the loading of heavy components based on certain triggers, such as the section entering the viewport. Then, in September 2024, the smart folks at Angular figured out that they could build upon those two features, allowing you to mark parts of your application to be server-rendered dehydrated and then hydrate them incrementally when needed - hence incremental hydration. I’m sure you know what hydration is. In short, the server sends fully formed HTML to the client, ensuring that the user sees meaningful content as quickly as possible and once JavaScript is loaded on the client side, the framework will reconcile the rendered DOM with component logic, event handlers, and state - effectively hydrating the server-rendered content. But what exactly does "dehydrated" mean, you might ask? Here's what will happen when you mark a part of your application to be incrementally hydrated: 1. Server-Side Rendering (SSR): The content marked for incremental hydration is rendered on the server. 2. Skipped During Client-Side Bootstrapping: The dehydrated content is not initially hydrated or bootstrapped on the client, reducing initial load time. 3. Dehydrated State: The code for the dehydrated components is excluded from the initial client-side bundle, optimizing performance. 4. Hydration Triggers: The application listens for specified hydration conditions (e.g., on interaction, on viewport), defined with a hydrate trigger in the @defer block. 5. On-Demand Hydration: Once the hydration conditions are met, Angular downloads the necessary code and hydrates the components, allowing them to become interactive without layout shifts. How to Use Incremental Hydration Thanks to Mark Thompson, who recently hosted a feature showcase on incremental hydration, we can show some code. The first step is to enable incremental hydration in your Angular application's appConfig using the provideClientHydration provider function: ` Then, you can mark the components you want to be incrementally hydrated using the @defer block with a hydrate trigger: ` And that's it! You now have a component that will be server-rendered dehydrated and hydrated incrementally when it becomes visible to the user. But what if you want to hydrate the component on interaction or some other trigger? Or maybe you don't want to hydrate the component at all? The same triggers already supported in @defer blocks are available for hydration: - idle: Hydrate once the browser reaches an idle state. - viewport: Hydrate once the component enters the viewport. - interaction: Hydrate once the user interacts with the component through click or keydown triggers. - hover: Hydrate once the user hovers over the component. - immediate: Hydrate immediately when the component is rendered. - timer: Hydrate after a specified time delay. - when: Hydrate when a provided conditional expression is met. And on top of that, there's a new trigger available for hydration: - never: When used, the component will remain static and not hydrated. The never trigger is handy when you want to exclude a component from hydration altogether, making it a completely static part of the page. Personally, I'm very excited about this feature and can't wait to try it out. How about you?...

Mar 14, 2025

3 mins

AngularJavaScript

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.