Running Playwright or any e2e test suite on a Netlify preview URL

We're assuming a couple of things in this example: it is valid for running an application build hosted on Github and that the Netlify integration is configured (meaning that you'll get preview deployments on any pull request you open). Also, in this example I use Playwright as the test suite of choice, while you should be able to apply the concept to other suites.

The bulk of the feature lies in a properly configured `yaml` file, here's the example:

# # Filename: # .github/workflows/playwright.yml # name: Playwright Tests env: ## Sets environment variables NETLIFY_SITE_NAME: 'sitename' # This is the "https://deploy-preview-1--${NETLIFY_SITE_NAME}.netlify.app" GITHUB_PR_NUMBER: ${{github.event.pull_request.number}} on: push: branches: [ main, master ] pull_request: branches: [ main, master ] jobs: tests_e2e_netlify_prepare: name: Wait for deployment on Netlify runs-on: ubuntu-latest steps: - name: Waiting for Netlify Preview uses: josephduffy/wait-for-netlify-action@v1 id: wait-for-netflify-preview with: site_name: env.NETLIFY_SITE_NAME max_timeout: 60 tests_e2e_netlify: needs: tests_e2e_netlify_prepare name: Run end-to-end tests on Netlify PR preview runs-on: ubuntu-latest timeout-minutes: 5 steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 - name: install playwright browsers run: npx playwright install --with-deps - name: '@playwright/test' run: npm install -D @playwright/test - name: run playwright run: npx playwright test # Here we provide the preview domain, based on the GITHUB_PR_NUMBER combined with the NETLIFY_SITE_NAME env: PLAYWRIGHT_TEST_BASE_URL: "https://deploy-preview-${{env.GITHUB_PR_NUMBER}}--${{env.NETLIFY_SITE_NAME}}.netlify.app" - uses: actions/upload-artifact@v3 if: always() with: name: playwright-report path: playwright-report/ retention-days: 30

Break it down

Let's break it down. To provide the Netlify url we need two things, because the preview URLs have a specific pattern where the PR number identifies the preview. We can grab that using information that Github provides in the pipeline, the `github.event.pull_request.number`. The other part is your configured domains' name. Knowing this, we can combine it in the preview URL to use it for our tests! 🧪

But wait!

But before we start, we need to make sure that the domain is up and running, because it gets spinned up and it's not available instantaneously. We can set a timeout for the tests, but that's flaky. So we need find out when the preview URL is available. For that, we execute the first job where we use a predefined action (josephduffy/wait-for-netlify-action@v1) that waits for the deploy environment to be available, so that we don't execute against a non existent URL! 🙌

If you're not using Playwright, feel free to take it from here. Just connect a second job to follow the first.

⚠️ Spoiler alert: the preview URL looks like this now: `https://deploy-preview-${{env.GITHUB_PR_NUMBER}}--${{env.NETLIFY_SITE_NAME}}.netlify.app`.

The next step 🎭

The second part is where we basically configured an execution of Playwright (so results may vary depending your test suite of choice).

In the pipeline, we make sure to provide the preview URL to the env, using this line:

env: PLAYWRIGHT_TEST_BASE_URL: "https://deploy-preview-${{env.GITHUB_PR_NUMBER}}--${{env.NETLIFY_SITE_NAME}}.netlify.app"

Lastly, we need to make sure Playwright supports injecting a custom domain (rather than running on localhost for instance). You can modify the `playwright.config.ts` file. At the bottom of the generated file, there's webServer property, which I changed to this:

webServer: { command: 'npm run dev', reuseExistingServer: true, url: process.env.PLAYWRIGHT_TEST_BASE_URL || 'http://localhost:3000', },

This way we let Playwright run on a provided `PLAYWRIGHT_TEST_BASE_URL` if it's available, with a fallback to localhost.

That's it! This allows you to deploy with confidence!