Javascript

How to Run End-to-End Tests on Vercel Preview Deployments

Published Sep 20, 2024

Updated Sep 20, 2024

3 min read

How to Run End-to-End Tests on Vercel Preview Deployments

End-to-end (E2E) tests are an essential part of modern web development. They simulate real user interactions and ensure your application works as expected in a live environment. When you're working with platforms like Vercel, which provide seamless deployment experiences, running E2E tests on preview deployments makes sense to catch any issues before merging your changes into the main branch.

However, running E2E tests on preview deployments can be challenging as if the deployment isn’t ready when the tests start running, you’ll get errors when they should have succeeded.

The Challenge: Timing Your E2E Tests

Imagine you're working on a feature and pushing your changes to a branch. Vercel starts deploying the preview, and once that's done, you want to run your E2E tests automatically. But if the deployment isn't ready when the tests start, they might fail, and you'll be left wondering if the issue is with your code or the deployment process.

Github Actions allow us to leverage a combination of actions that work together to ensure that your E2E tests only start once the Vercel deployment is ready.

The Solution: Combining GitHub Actions for Seamless Testing

To solve this problem, we can use two GitHub Actions in tandem:

vercel-preview-url: This action retrieves the preview URL of the Vercel deployment. It runs on push and pull request events, ensuring you get the exact URL of your deployment, which can then be used for further tests or other integrations.
github-action-await-vercel: This action waits until the Vercel deployment is marked as "READY". It continuously checks the deployment status, ensuring your tests only run once the deployment is fully prepared.

jobs:
  e2e-tests:
    runs-on: ubuntu-latest
    steps:
      # Step 1: Retrieve the Vercel Preview URL
      - name: vercel-preview-url
        uses: zentered/vercel-preview-url@v1.1.9
        if: github.event_name != 'pull_request'
        id: vercel_preview_url
        env:
          VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
          GITHUB_REF: "main"
          GITHUB_REPOSITORY: "your-repo/your-project"
        with:
          vercel_project_id: ${{ secrets.VERCEL_PROJECT_ID }}

      # Step 2: Wait for the Vercel deployment to be ready
      - uses: UnlyEd/github-action-await-vercel@v1
        id: await-vercel
        if: github.event_name != 'pull_request'
        env:
          VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
        with:
          deployment-url: ${{ steps.vercel_preview_url.outputs.preview_url }}
          timeout: 420
          poll-interval: 15

      # Step 3: Run your end-to-end tests
      - name: Run E2E Tests
        run: npm run test:e2e

Breaking Down the Workflow

Fetch the Preview URL: The vercel-preview-url action first retrieves the URL of the Vercel deployment. This URL is unique to the deployment, and we need to know it to run our E2E tests against the correct environment. This step ensures that we have the right URL before proceeding to the next steps.
Await Deployment Readiness: The github-action-await-vercel action waits for the deployment to be fully ready. It checks the deployment status at intervals (e.g., every 15 seconds) and only proceeds once the deployment is marked as "READY." This step ensures that your E2E tests don't run prematurely, preventing false failures.
Run E2E Tests: Once the deployment is ready, your E2E tests are executed against the live preview. By this point, you can be confident that the environment is fully set up, and any failures will likely be due to actual issues in the code rather than deployment timing.

Why This Workflow Matters

This workflow is essential for teams that rely on automated testing as part of their CI/CD pipelines. By waiting for the Vercel deployment to be fully ready, you minimize the risk of flaky tests that fail due to timing issues rather than code problems. This approach saves time and effort, as developers can focus on fixing real bugs rather than troubleshooting deployment issues.

Moreover, this setup is flexible and can be adapted to various project needs. Whether you're running tests on a simple static site or a complex application, ensuring that your deployment is ready before running tests can dramatically improve the reliability of your CI/CD pipeline.

Conclusion

In modern web development, ensuring the reliability of your CI/CD pipeline is crucial. By leveraging GitHub Actions like vercel-preview-url and github-action-await-vercel, you can confidently run E2E tests on Vercel preview deployments.

However, remember that the Vercel deployment can take some time to deploy, especially for larger projects, so use this approach wisely and ensure you only run it when necessary.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

Jan Kaiser

Software engineer passionate about everything web-related. Particularly loves Angular, SCSS, TypeScript, soccer and dachshunds

@honzikec @honzikec

Enhancing Your Playwright Workflow: A Guide to the VSCode Extension

Introduction In my last post, Quick Guide to Playwright Fixtures: Enhancing Your Tests, I delved into some of the enhancements we've been implementing in our end-to-end (E2E) tests using Playwright. As I refine our testing strategies, I've come across a tool that has quickly become an essential part of my workflow: the Playwright VSCode extension. If you're like me and constantly looking for ways to streamline testing and debugging, you'll appreciate any tool that can make the process more efficient and enjoyable. That's where this extension comes in. It's not just about writing tests - it's about enhancing the entire development experience. In this post, I'll walk you through getting started with the Playwright VSCode extension, sharing some tips and tricks that have made a real difference in my day-to-day work. Installing the Extension & Basic Setup Before diving into the Playwright VSCode extension, it's essential to have Playwright installed on your machine. If you haven't done so already, you can quickly install it by running: ` This command will set up Playwright and ensure all necessary dependencies are installed. Once Playwright is ready, the VSCode extension will be installed next. Open Visual Studio Code, navigate to the Extensions view by clicking on the Extensions icon in the Activity Bar on the side of the window, and search for “Playwright”. The official extension, ID: ms-playwright.playwright / named: “Playwright Test for VSCode” by Microsoft, should appear at the top of the list. Click "Install," and you're all set. With the extension installed, you can start leveraging its powerful features to enhance your Playwright testing workflow within VSCode. Running the Tests and Identifying Outputs To run a test, simply open the test file in VSCode. The Playwright extension will automatically detect test files and display a "Run" icon next to each test and test suite. You can click on this icon to run individual tests or test suites. Alternatively, you can run all the tests in your project using the Playwright Test Explorer, accessible from the sidebar. Once you start running your tests, the extension provides real-time feedback within the editor. You'll see the status of each test - whether it passes, fails, or is skipped - right next to the corresponding test in your code. This immediate feedback loop is incredibly helpful for catching issues as you write your tests. The output of your tests will be displayed in the VSCode terminal. You'll see detailed information about each test run. Debugging Step-by-Step I find debugging particularly useful when a test fails unexpectedly or when I want to verify that certain actions are being performed as intended. Instead of guessing what might be wrong, I can see exactly what's going on in each test step, making debugging a much more straightforward and less frustrating process. To start debugging, you can easily set a breakpoint in your test file by clicking on the left margin next to the line number where you'd like the execution to pause. Once your breakpoints are in place, you can initiate the debug process by selecting the "Debug" option next to the test you'd like to investigate. Once the debugger is running, the extension allows you to step through your code, inspect variables, and evaluate expressions - all within VSCode. This real-time insight into your test execution is a game-changer, enabling you to pinpoint issues more effectively and confidently refine your tests. Using the Pick Locator Tool Another handy feature is the "Pick Locator" tool. If you've ever struggled with selecting the right element in your tests, this tool can be a time saver. It helps you generate reliable locators by letting you interact directly with the webpage elements you want to target. To use the Pick Locator tool, click the "Pick Locator" button in the Playwright Test Explorer. This will open a new window where you can navigate to the site you're testing. As you hover over elements on the page, the tool will suggest locators, allowing you to select the most appropriate one for your test. While the Pick Locator tool is handy, it’s important to ensure that the locators you generate are robust and maintainable. This is especially true when integrating them with the fixtures I discussed in my previous blog post. Combining the proper locators with well-designed fixtures can create more reliable and reusable test setups, ultimately making your E2E tests more efficient. Conclusion The Playwright VSCode extension has quickly become indispensable in my development workflow. It significantly enhanced my experience of writing and running Playwright tests. Whether you’re just starting with Playwright or looking to optimize your existing tests, this extension offers a range of features that can save you time and effort. Combining these tools with thoughtful test design, such as leveraging fixtures, you can create a more efficient and effective testing process. I hope this guide has given you a good overview of what the Playwright VSCode extension can do and how it can benefit your work. If you haven’t tried it yet, I highly recommend giving it a go. And as always, feel free to explore further and experiment with the features that best suit your needs....

Sep 6, 2024

5 mins

VSCodeTestingPlaywright

Tim Neutkens, Co-Author of Next.js on the State of Next

Watch this episode! Tim Neutkens, Co-author and Tech Lead for Next.js, discusses how open source maintainers are simplifying the web, and covers the challenges faced with the current Next.js setup. Tim talks about TurboPack, a solution that optimizes bundling, improves parallelism, caching, and module graph calculations. He also talks about TurboAC, which focuses on addressing performance and compatibility issues, providing seamless transitions for Next.js users. Tim highlights the importance of efficient bundling processes to avoid excessive recompilation and discusses the updates in Next.js versions to enhance caching, rendering behavior, and client-side caching. Tim also discusses some exciting upcoming features in Next.js 15. Tim Neutkens' Socials - Twitter: @timneutkens - GitHub: timneutkens - Bluesky: timneutkens.bsky.social - Website: https://timn.tech/ Links - Vercel on Twitter, LinkedIn, Facebook, Instagram, YouTube, GitHub and Vercel's website - Next.js on Twitter, GitHub, LinkedIn, YouTube, Instagram, Facebook, official Next.js website - Turbopack on Twitter, GitHub, YouTube, LinkedIn, Instagram, Official Turbopack Docs - Webpack on GitHub, Twitter, YouTube, and Official Webpack Website Show Notes - [00:00:02] Podcast episode featuring Tim Neutkens discussing Next.js and the upcoming release of TurboPack. - [00:04:27] JavaScript bundlers evolving to handle growth. - [00:07:58] TurboPack solves Webpack limitations efficiently. - [00:12:12] Bundler compatibility for optimal app performance. - [00:16:50] Client components separated in webpack instance. Turbo pack for better parallelism and stability. Industry moving towards server-side. Feed and rollup still relevant. Collaboration between tools for future. - [00:20:57] Replacing part with roll down, similar to Webpack. Overlapping ecosystem with Avonetic Conference. Limits with unbundling and loading on demand. Cycle of building frameworks and hitting limits. History of using Webpack for client-side code. Two compiler architecture for server and client. Coordination between server and client with Webpack. - [00:25:38] Server action imports, turbo pack improves performance. - [00:30:04] Next.js is popular for websites. - [00:34:18] Chipotle using Next in Vercel, exciting improvements. - [00:38:51] Next.js 15 release candidate with changes. This episode is sponsored by Wix Studio....

Aug 7, 2024

2 mins

VercelNextJS

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

Understanding the Difference Between :focus and :focus-visible in CSS I have learned my fair share about the importance of keyboard accessibility, so I know that visual indication of the focused element is very important. But the well-known :focus pseudo-class is not always the best fit for this job. That's where :focus-visible comes in. Let's look at the differences between these two pseudo-classes and explore the best practices for using them effectively. What is the :focus Pseudo-Class? The :focus pseudo-class is a CSS selector that applies styles to any element that receives focus, regardless of how that focus was triggered. This includes focus events from keyboard navigation, mouse clicks, and touch interactions. Example Usage of :focus ` In this example, the button will display a blue outline whenever it is focused, whether the user clicks on it with a mouse, taps it on a touchscreen, or navigates to it using the keyboard. What is the :focus-visible Pseudo-Class? The :focus-visible pseudo-class is more specialized. It only applies styles to an element when the browser determines that the focus should be visible. This typically occurs when the user navigates via the keyboard or assistive technologies rather than through mouse or touch input. Example Usage of :focus-visible ` Here, the button will only show a blue outline when focused through keyboard navigation or another input method that usually requires visible focus indicators. Key Differences Between :focus and :focus-visible :focus - Behavior: Applies to any element that receives focus, regardless of the input method. - Use Cases: Ensures that all interactions with the element are visually indicated, whether by mouse, keyboard, or touch. :focus-visible - Behavior: Applies styles only when the focus should be visible, such as using a keyboard or assistive technology. - Use Cases: Ideal for scenarios where you want to provide focus indicators only to keyboard and assistive technology users while avoiding unnecessary outlines for mouse and touch users, typically required by design. Accessibility Implications :focus - Pros: - Guarantees that all users can see when an element is focused, which is critical for accessibility. - Cons: - Can lead to a suboptimal experience for mouse users, as focus styles may appear unnecessarily during mouse interactions. :focus-visible - Pros: - Enhances user experience by showing focus indicators only when necessary, thus keeping the interface clean for mouse and touch users. - Tailors the experience for keyboard and assistive technology users, providing them with clear visual cues. - Cons: - Additional considerations may be required to ensure that focus indicators are not accidentally omitted, especially in older browsers that do not support :focus-visible. - There may be cases where you want to show focus indicators for all users, regardless of input method. Best Practices for Using :focus and :focus-visible To achieve the best accessibility and user experience, combining both :focus and :focus-visible in your CSS is often a good idea. Combining :focus and :focus-visible ` Here is a Stackblitz example of what such styling could look like for you to try out and play with. Additional Tips - Test with Keyboard and Assistive Technology: Ensure that your web application is navigable using a keyboard (Tab, Shift + Tab, etc.) and that focus indicators are visible for those who rely on them. It's never a bad idea to include accessibility testing in your e2e testing suite. - Provide Clear Focus Indicators: Make sure that focus indicators are prominent and easy to see. A subtle or hard-to-spot focus indicator can severely impact accessibility for users who rely on keyboard navigation. Conclusion The :focus-visible pseudo-class offers a more refined way to manage focus indicators, improving accessibility and user experience, particularly for keyboard and assistive technology users. By understanding the differences between :focus and :focus-visible, and applying best practices in your CSS, you can create more accessible and user-friendly web applications. Remember, accessibility should never be an afterthought. By thoughtfully applying focus styles, you ensure that all users, regardless of how they interact with your site, can easily navigate and interact....

Nov 8, 2024

3 mins

AccessibilityCSSPlaywright

How to Create a Memorable Conference Experience with Vincent Mayers

In this episode of the Modern Web Podcast, Danny Thompson, Director of Technology at This Dot Labs, sits down with Vincent Mayers, a seasoned tech conference organizer with over 15 years of experience. They discuss the intricacies of running successful conferences, including the challenges of selecting event locations, building community engagement, and creating memorable experiences for attendees. Vincent also shares insights into the evolution of tech conferences, from the importance of shorter talks to the value of the "hallway track" for networking. Tune in for an inside look at how these events shape the tech ecosystem and tips for organizing your own conferences! Chapters - 00:00 - Introduction - 01:45 - Vincent Mayers' Background - 03:50 - Choosing Conference Locations - 06:10 - Building Community and Spreading the Word - 08:40 - Sponsorship and Funding Challenges - 11:00 - Securing Speakers for Tech Conferences - 14:20 - Improving the Conference Experience - 16:30 - Badge Design and the Attendee Experience - 18:50 - Engaging Attendees Beyond Talks - 21:00 - The Role of Tech Conferences in the Java Ecosystem - 23:12 - Attendees Still Using Older Java Versions - 26:00 - Balancing Cutting-Edge Tech with Fundamentals - 28:15 - Evolving Attention Spans in Tech Conferences - 30:00 - The Importance of the Hallway Track - 33:19 - Closing Remarks Follow Vincent Mayers on Social Media Twitter: https://x.com/vincentmayers Linkedin: https://www.linkedin.com/in/vincentmayers/ Github: https://github.com/vincentmayers Sponsored by This Dot....

Nov 6, 2024

1 min

Software EngineerSoftware Engineering

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

How to Run End-to-End Tests on Vercel Preview Deployments

How to Run End-to-End Tests on Vercel Preview Deployments

The Challenge: Timing Your E2E Tests

The Solution: Combining GitHub Actions for Seamless Testing

Breaking Down the Workflow

Why This Workflow Matters

Conclusion

Jan Kaiser

You might also like

Enhancing Your Playwright Workflow: A Guide to the VSCode Extension

Tim Neutkens, Co-Author of Next.js on the State of Next

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

How to Create a Memorable Conference Experience with Vincent Mayers

Let's innovate together!

You might also like

Enhancing Your Playwright Workflow: A Guide to the VSCode Extension

Tim Neutkens, Co-Author of Next.js on the State of Next

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

How to Create a Memorable Conference Experience with Vincent Mayers