Javascript

Our Journey from Cypress to Playwright for E2E Testing

William Mimura

Published May 31, 2024

Updated May 31, 2024

3 min read

Our Journey from Cypress to Playwright for E2E Testing

Introduction

Adapting to unexpected technical challenges often leads to innovation. Our team faced such a moment when we discovered that our primary end-to-end (E2E) testing framework, Cypress, was incompatible with an essential component of our tech stack. This discovery led us to consider alternatives to resolve the compatibility issue and enhance our testing processes.

Our choice was Playwright, a modern testing framework acclaimed for its robust features and compatibility with multiple browsers. The shift to Playwright was swift and efficient: all our E2E tests were successfully migrated within a few hours. This rapid transition highlighted our team's ability to adapt and integrate new technologies quickly.

In this blog post, we will share the hands-on process we followed during the migration to Playwright, the challenges we encountered along the way, and the solutions we implemented to overcome them.

Using a Test Conversion Tool

We utilized a third-party test conversion tool to facilitate our migration from Cypress to Playwright, which significantly streamlined the process. Among several options available, we chose Ray's Cypress to Playwright Conversion Tool for its ease of use and effectiveness. This tool allowed us to automatically transpose our existing Cypress tests into the Playwright format.

The tool processes each test script by identifying equivalent functions and commands in Playwright, translating our entire suite with minimal input. Below are some before and after images of our test scripts, demonstrating the tool's conversion capabilities.

One test script written in Cypress:

describe("Newsletter page", () => {
  it("subscribing to email newsletter should show success message", () => {
    cy.visitWithTimeout("/newsletter");

    cy.getDataCy("newsletter-email-input").type(SIGN_UP_EMAIL_TEST);
    cy.getDataCy("newsletter-submit-button").wait(100).click();

    cy.contains("You subscribed to the newsletter successfully!");
  });

  it("home page should contain button to navigate to newsletter page", () => {
    cy.visitWithTimeout("/");

    // email registration signup is located at the bottom of the home page
    cy.scrollTo("bottom");

    cy.getDataCy("newsletter-sign-up-link").contains(
      "Sign up for our newsletters",
    );

    // adding a 100ms wait for safety, otherwise this command runs too fast and the click is not registered by the website
    cy.getDataCy("newsletter-sign-up-link").wait(100).click();

    cy.url().should("include", "/newsletter");
  });
});

And its result converted in Playwright:

import { test, expect } from "@playwright/test";

test.describe("Newsletter page", () => {
  test("subscribing to email newsletter should show success message", async ({
    page,
  }) => {
    page.FIXME_visitWithTimeout("/newsletter");
    page.FIXME_getDataCy("newsletter-email-input");
    await page.fill(SIGN_UP_EMAIL_TEST);
    page.FIXME_getDataCy("newsletter-submit-button");
    await page.waitForTimeout(100);
    await page.click();
    await expect(
      page.getByText(/You subscribed to the newsletter successfully!/).first()
    ).toBeVisible();
  });

  test("home page should contain button to navigate to newsletter page", async ({
    page,
  }) => {
    page.FIXME_visitWithTimeout("/");

    // email registration signup is located at the bottom of the home page
    await page.FIXME_scrollTo("bottom");
    page.FIXME_getDataCy("newsletter-sign-up-link");
    await expect(
      page.getByText(/Sign up for our newsletters/).first()
    ).toBeVisible();

    // adding a 100ms wait for safety, otherwise this command runs too fast and the click is not registered by the website
    page.FIXME_getDataCy("newsletter-sign-up-link");
    await page.waitForTimeout(100);
    await page.click();
    await expect(page).toHaveURL(/\/newsletter/);
  });
});

While the tool performed impressively, converting most of our tests accurately had limitations. Certain custom helper functions and test aliases did not translate directly, and the tool clearly marked these for manual intervention. This feature was beneficial as it allowed us to quickly identify and address the segments that required our attention.

Small Fixes and Fine-Tuning

The initial output from the conversion tool was highly useful, yet it required some tweaks to align with Playwright's capabilities perfectly. The tool conveniently marked each unconverted function with the identifier FIXME_<unconverted function>, making it straightforward to address these specific areas. Here’s how we tackled some of these functions:

FIXME_visitWithTimeout: Originally a custom Cypress command designed to visit a URL with a specified timeout (cy.visit(url, { timeout: 30000 })), this was straightforwardly converted to Playwright's page.goto(url) FIXME_getDataCy: Another custom command (cy.get('[data-cy="<a_selector>"]')) was efficiently translated to Playwright’s page.locator('[data-testid="<a_selector>"]') FIXME_scrollTo: This native Cypress command, which scrolls the window to a specific position, proved unnecessary in Playwright. Given Playwright’s automatic detection of elements on the page, we opted to remove this command entirely

import { test, expect } from "@playwright/test";

test.describe("Newsletter page", () => {
  test("subscribing to email newsletter should show success message", async ({
    page,
  }) => {
    await page.goto("/newsletter");
    await page
      .locator(`[data-testid="newsletter-email-input"]`)
      .fill(SIGN_UP_EMAIL_TEST);
    await page.locator(`[data-testid="newsletter-submit-button"]`).click();
    await expect(
      page.getByText(/You subscribed to the newsletter successfully!/).first(),
    ).toBeVisible();
  });

  test("home page should contain button to navigate to newsletter page", async ({
    page,
  }) => {
    await page.goto("/");

    await expect(
      page
        .locator(`[data-testid="newsletter-sign-up-link"]`)
        .getByText(/Sign up for our newsletters/)
        .first(),
    ).toBeVisible();

    await page.locator(`[data-testid="newsletter-sign-up-link"]`).click();
    await expect(page).toHaveURL(/\/newsletter/);
  });
});

Further refinements were made to the converted code, such as removing some "wait/delay" commands. Playwright’s robust handling of asynchronous events allowed us to streamline our tests by eliminating unnecessary pauses enhancing test execution speed and reliability.

Conclusion

The transition from Cypress to Playwright in our testing framework was driven by necessity but has significantly improved our testing practices. Utilizing a test conversion tool allowed us to migrate our entire suite of E2E tests efficiently, minimizing manual effort and accelerating the adoption of Playwright.

Our experience highlights the importance of flexibility and the willingness to embrace new technologies in the face of unexpected challenges. The marked improvements in test execution speed and reliability, along with Playwright's advanced features, have made this transition a resounding success for our team.

As we continue to refine our tests and explore the full capabilities of Playwright, we encourage other teams facing similar challenges to consider this approach. The tools and processes outlined here can serve as a roadmap for those looking to enhance their automated testing solutions.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

About the author(s)

William Mimura
@mimurawil

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

Understanding the Difference Between :focus and :focus-visible in CSS I have learned my fair share about the importance of keyboard accessibility, so I know that visual indication of the focused element is very important. But the well-known :focus pseudo-class is not always the best fit for this job. That's where :focus-visible comes in. Let's look at the differences between these two pseudo-classes and explore the best practices for using them effectively. What is the :focus Pseudo-Class? The :focus pseudo-class is a CSS selector that applies styles to any element that receives focus, regardless of how that focus was triggered. This includes focus events from keyboard navigation, mouse clicks, and touch interactions. Example Usage of :focus ` In this example, the button will display a blue outline whenever it is focused, whether the user clicks on it with a mouse, taps it on a touchscreen, or navigates to it using the keyboard. What is the :focus-visible Pseudo-Class? The :focus-visible pseudo-class is more specialized. It only applies styles to an element when the browser determines that the focus should be visible. This typically occurs when the user navigates via the keyboard or assistive technologies rather than through mouse or touch input. Example Usage of :focus-visible ` Here, the button will only show a blue outline when focused through keyboard navigation or another input method that usually requires visible focus indicators. Key Differences Between :focus and :focus-visible :focus - Behavior: Applies to any element that receives focus, regardless of the input method. - Use Cases: Ensures that all interactions with the element are visually indicated, whether by mouse, keyboard, or touch. :focus-visible - Behavior: Applies styles only when the focus should be visible, such as using a keyboard or assistive technology. - Use Cases: Ideal for scenarios where you want to provide focus indicators only to keyboard and assistive technology users while avoiding unnecessary outlines for mouse and touch users, typically required by design. Accessibility Implications :focus - Pros: - Guarantees that all users can see when an element is focused, which is critical for accessibility. - Cons: - Can lead to a suboptimal experience for mouse users, as focus styles may appear unnecessarily during mouse interactions. :focus-visible - Pros: - Enhances user experience by showing focus indicators only when necessary, thus keeping the interface clean for mouse and touch users. - Tailors the experience for keyboard and assistive technology users, providing them with clear visual cues. - Cons: - Additional considerations may be required to ensure that focus indicators are not accidentally omitted, especially in older browsers that do not support :focus-visible. - There may be cases where you want to show focus indicators for all users, regardless of input method. Best Practices for Using :focus and :focus-visible To achieve the best accessibility and user experience, combining both :focus and :focus-visible in your CSS is often a good idea. Combining :focus and :focus-visible ` Here is a Stackblitz example of what such styling could look like for you to try out and play with. Additional Tips - Test with Keyboard and Assistive Technology: Ensure that your web application is navigable using a keyboard (Tab, Shift + Tab, etc.) and that focus indicators are visible for those who rely on them. It's never a bad idea to include accessibility testing in your e2e testing suite. - Provide Clear Focus Indicators: Make sure that focus indicators are prominent and easy to see. A subtle or hard-to-spot focus indicator can severely impact accessibility for users who rely on keyboard navigation. Conclusion The :focus-visible pseudo-class offers a more refined way to manage focus indicators, improving accessibility and user experience, particularly for keyboard and assistive technology users. By understanding the differences between :focus and :focus-visible, and applying best practices in your CSS, you can create more accessible and user-friendly web applications. Remember, accessibility should never be an afterthought. By thoughtfully applying focus styles, you ensure that all users, regardless of how they interact with your site, can easily navigate and interact....

Nov 8, 2024

3 mins

AccessibilityCSSPlaywright

How to Run End-to-End Tests on Vercel Preview Deployments

How to Run End-to-End Tests on Vercel Preview Deployments End-to-end (E2E) tests are an essential part of modern web development. They simulate real user interactions and ensure your application works as expected in a live environment. When you're working with platforms like Vercel, which provide seamless deployment experiences, running E2E tests on preview deployments makes sense to catch any issues before merging your changes into the main branch. However, running E2E tests on preview deployments can be challenging as if the deployment isn’t ready when the tests start running, you’ll get errors when they should have succeeded. The Challenge: Timing Your E2E Tests Imagine you're working on a feature and pushing your changes to a branch. Vercel starts deploying the preview, and once that's done, you want to run your E2E tests automatically. But if the deployment isn't ready when the tests start, they might fail, and you'll be left wondering if the issue is with your code or the deployment process. Github Actions allow us to leverage a combination of actions that work together to ensure that your E2E tests only start once the Vercel deployment is ready. The Solution: Combining GitHub Actions for Seamless Testing To solve this problem, we can use two GitHub Actions in tandem: 1. vercel-preview-url: This action retrieves the preview URL of the Vercel deployment. It runs on push and pull request events, ensuring you get the exact URL of your deployment, which can then be used for further tests or other integrations. 2. github-action-await-vercel: This action waits until the Vercel deployment is marked as "READY". It continuously checks the deployment status, ensuring your tests only run once the deployment is fully prepared. ` Breaking Down the Workflow 1. Fetch the Preview URL: The vercel-preview-url action first retrieves the URL of the Vercel deployment. This URL is unique to the deployment, and we need to know it to run our E2E tests against the correct environment. This step ensures that we have the right URL before proceeding to the next steps. 2. Await Deployment Readiness: The github-action-await-vercel action waits for the deployment to be fully ready. It checks the deployment status at intervals (e.g., every 15 seconds) and only proceeds once the deployment is marked as "READY." This step ensures that your E2E tests don't run prematurely, preventing false failures. 3. Run E2E Tests: Once the deployment is ready, your E2E tests are executed against the live preview. By this point, you can be confident that the environment is fully set up, and any failures will likely be due to actual issues in the code rather than deployment timing. Why This Workflow Matters This workflow is essential for teams that rely on automated testing as part of their CI/CD pipelines. By waiting for the Vercel deployment to be fully ready, you minimize the risk of flaky tests that fail due to timing issues rather than code problems. This approach saves time and effort, as developers can focus on fixing real bugs rather than troubleshooting deployment issues. Moreover, this setup is flexible and can be adapted to various project needs. Whether you're running tests on a simple static site or a complex application, ensuring that your deployment is ready before running tests can dramatically improve the reliability of your CI/CD pipeline. Conclusion In modern web development, ensuring the reliability of your CI/CD pipeline is crucial. By leveraging GitHub Actions like vercel-preview-url and github-action-await-vercel, you can confidently run E2E tests on Vercel preview deployments. However, remember that the Vercel deployment can take some time to deploy, especially for larger projects, so use this approach wisely and ensure you only run it when necessary....

Sep 20, 2024

3 mins

InfrastructureVercelPlaywrightCypress

Quick Guide to Playwright Fixtures: Enhancing Your Tests

Introduction Following our recent blog post on migrating E2E tests from Cypress to Playwright, we've identified opportunities to enhance our test scripts further. In this guide, we'll delve into the basics of Playwright fixtures, demonstrating their utility and flexibility in test environments. Playwright fixtures are reusable components that set up and tear down the environment or conditions necessary for tests. They are crucial for writing clean, maintainable, and scalable tests. Fixtures can handle tasks like opening a browser, initializing a database, or logging into an application—actions you might need before running your tests. As a practical example, we'll revisit one of the tests from our previous post, enhancing it with a new fixture to streamline the testing process and significantly improve maintainability. This post is designed to provide the foundational skills to integrate fixtures into your testing workflows effectively, giving you the confidence to manage and maintain your test scripts more efficiently. Creating Our First Fixture To illustrate the power of fixtures in Playwright, let’s consider a practical example from a test scenario in our project. Below is a snippet of a test case from our newsletter page: ` This test scenario navigates to the newsletter page, fills in an email, submits the form, and checks for a success message. To optimize our test suite, we'll refactor common actions like navigation, form completion, and submission into reusable fixtures. This approach makes our tests cleaner and more maintainable and reduces redundancy across similar test scenarios. Implementing the Fixture Here’s how the fixture looks: ` This fixture encapsulates the actions of navigating to the page, filling out the email field, and submitting the form. By abstracting these actions, we simplify and focus our test cases. Refactoring the Test With the fixture in place, let’s see how it changes our original test file: ` A beforeEach method to reset the state of the NewsletterPage fixture ensures a clean and consistent environment for each test scenario. This practice is crucial for maintaining the integrity and reliability of your tests. By leveraging the NewsletterPage fixture, each test within the "Newsletter page" suite starts with a clean and pre-configured environment. This setup improves test clarity and efficiency and aligns with best practices for scalable test architecture. Conclusion As we've seen, fixtures are powerful tools that help standardize test environments, reduce code redundancy, and ensure that each test operates in a clean state. By abstracting common setup and teardown tasks into fixtures, we can focus our testing efforts on what matters most, verifying the behavior and reliability of the software we're developing. Remember, the key to successful test management is choosing the right tools and using them wisely to create scalable, maintainable, and robust testing frameworks. Playwright fixtures offer a pathway towards achieving these goals, empowering teams to build better software faster and more confidently....

Jul 26, 2024

2 mins

Testing

Building a Stripe App: A Step-by-Step Guide to QR Code Generation

Building a Stripe App: A Step-by-Step Guide to QR Code Generation Why Build a Stripe App? I recently participated in an audio space with the Stripe team, and something they said really stuck with me: the Stripe app store is a growing area that isn't overly saturated yet. There's a lot of potential for new apps, and companies can use this opportunity to grow. I work at a company called This Dot Labs, and we've created several Stripe apps and even own one. After looking at the data, I can confirm that the Stripe team was right! Creating a QR Code Generator App For this tutorial, we'll build a QR code app that can take a URL and generate a code for it. This is a good use case to help you understand the ins and outs of Stripe's developer tools. Why QR Codes? QR codes are useful tools that have become common in e-commerce, restaurants, and other industries. While Stripe already has a QR code tool, we'll make our own to familiarize ourselves with their syntax and problem-solving approaches. Project Structure Before we dive into the implementation, let's look at the structure of our Stripe App QR Code project: * .vscode: Contains settings for Visual Studio Code * source/views: Holds the main application views * .gitignore: Specifies files to ignore in version control * stripe-app.json: Defines the Stripe app configuration * ui-extensions.d.ts: TypeScript declaration file for UI extensions * .build: this is where the built Stripe app gets placed. Step-by-Step Implementation 1\. Install Stripe Locally First, you need to install Stripe on your local machine. The documentation provides great instructions for this: * For Mac users: Use Brew to install * For Windows users: Download the package and add it to your environment variables You can find the details here in the stripe docs to install the Stripe CLI https://docs.stripe.com/stripe-cli When using Windows, you must do stripe login from Powershell, NOT from Git bash or any other tool. After the server is up, then you can continue using git bash for everything else. After stripe login, you need to enter stripe apps start. Once you do that, the server is up and running and you can go back to using git bash or any other tool. 2\. Install Dependencies We'll be using an extra package for QR code generation. Install it using npm: ` 3\. Set Up the Main Component Let's look at the home.tsx file, where we'll use Stripe's UI components: ` These components are similar to other UI libraries like Bootstrap or Tailwind CSS. 4\. Create the UI Structure Our app will have: * An input field for the URL * Validation using a regex pattern * Error handling for invalid URLs * QR code generation and display Here is the Home.tsx file that is located in the src/views folder ` * ContextView` is at the top level of the app where we see the Title and the link to the Stripe Docs that we placed in our Context View. * Box is how you use Divs. * Banners can be used to show notification errors or any other item you wish to display. * Textfields are input fields. * Everything else is pretty self-explanatory. 5\. Handle Content Security Policy One problem I personally ran into was when I tried to redirect users, the Stripe policies would block it since I did not express that I knew what it was doing. I had to go into the stripe-app.json file and mention the specific security policies. For this particular exercise, I kept these as null. This is my stripe-app.json file. ` 6\. Configure App Views As you can see here, the stripe-app.json file shows the views for each file I have. The Home.tsx file and the Invoice.tsx are also included This is our way of saying that for each view we have, show the app functionality on that page. Our stripe-app.json file will show it but also, the manifest.js file in our .build folder will also show the same. Any view that doesn't have a file will not show the application's functionality. So, if I were to go to transactions, the app would not show the same logic as the home or invoices page. By following these steps, you'll have a fully functional QR code generator app for Stripe. This is just a simple example, but the potential for Stripe apps is massive, especially for businesses serving e-commerce customers. If you need help or get stuck, don't hesitate to reach out, danny.thompson@thisdot.co. The Stripe team is also very active in answering questions, so leverage them as a resource. Happy coding!...

Dec 18, 2024

4 mins

stripe

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

Our Journey from Cypress to Playwright for E2E Testing

Our Journey from Cypress to Playwright for E2E Testing

Introduction

Using a Test Conversion Tool

Small Fixes and Fine-Tuning

Conclusion

William Mimura

You might also like

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

How to Run End-to-End Tests on Vercel Preview Deployments

Quick Guide to Playwright Fixtures: Enhancing Your Tests

Building a Stripe App: A Step-by-Step Guide to QR Code Generation

Let's innovate together!

You might also like

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

How to Run End-to-End Tests on Vercel Preview Deployments

Quick Guide to Playwright Fixtures: Enhancing Your Tests

Building a Stripe App: A Step-by-Step Guide to QR Code Generation