Table of contents

Vercel

If you are using Vercel to develop, preview, and ship your application, you can natively integrate with Checkly by installing the official integration from the Vercel Marketplace.

The Vercel integration will help you by:

1. Automatically creating a pre-scripted browser check to catch any errors and failed requests as your web page loads.
2. Enabling you to run checks against preview and production deployments on Vercel.

To install Checkly’s native Vercel integration, navigate to Integratons, under your account’s dropdown menu.

After scrolling down, click the Vercel marketplace button.

On the marketplace page for Checkly, click Add Integration.

Next, follow the installation wizard to grant the integration access to the right Vercel scope and projects.

You can choose to map your Vercel projects to existing checks on your Checkly account, to have them run on production and/or preview deployments.

Additionally, you can have new checks automatically generated for existing Vercel projects, and set them up to automatically run on preview and/or production deployments.

If you already have the Vercel integration set up, you might still want to connect new checks or groups. The procedure is the same for both: edit the check or group and select Link Vercel project in the CI/CD triggers tab.

You will just need to select the right project to link to your check or group.

Once the project and the check/group have been linked, you are able to specify whether a new preview or production deployment should trigger a new execution. You will also have the chance to choose from which location this check will run.

Should you wish to unlink the Vercel project, simply click Unlink this project.

On each deployment triggered by Vercel, we inject the Checkly runtime with two variables:

1. ENVIRONMENT_NAME: This name is preview for preview deployments and production for production deployments.
2. ENVIRONMENT_URL: For Preview deployments this is the preview URL alias, e.g. https://ecommerce-1wabp0sqy-acme.vercel.app, for Production deployments it is the equivalent production URL alias.
3. DEPLOYMENT_ID: The unique deployment ID generated by Vercel.

You can access all these variables with the familiar process.ENV syntax.

This means you can write a Browser check using Playwright Test or Playwright that automatically targets the Preview deploy whenever this URL is available, but otherwise just defaults to the stable production URL.

Here is a full example that we use ourselves to monitor checklyhq.com which is actually also deployed to Vercel.

import { expect, test } from '@playwright/test'

test('assert response status of page', async ({ page }) => {
  const targetUrl = process.env.ENVIRONMENT_URL || 'https://www.checklyhq.com'
  const response = await page.goto(targetUrl)

  expect(response.status()).toBeLessThan(400)
})

const { expect, test } = require('@playwright/test')

test('assert response status of page', async ({ page }) => {
  const targetUrl = process.env.ENVIRONMENT_URL || 'https://www.checklyhq.com'
  const response = await page.goto(targetUrl)

  expect(response.status()).toBeLessThan(400)
})

This way we are setting the targetUrl variable to either the ENVIRONMENT_URL or just our main production URL.

Whenever a Preview deploy happens on Vercel, this check gets called and runs the script against the preview environment. This check also runs on a 5 minute schedule, and checks our production environment.

This way, we kill two birds with one stone and don’t need separate checks for separate environments.

With API checks, we automatically replace the hostname part of your request URL with the host in the environment URL. For example:

• Your configured URL: https://api.acme.com/v1/customers?page=1
• Environment URL: https://now.customer-api.qis6va2z7.now.sh
• Replaced URL: https://now.customer-api.qis6va2z7.now.sh/v1/customers?page=1

Notice we only replace the host part, not the URL path or any query parameters.

Just like for browser checks, the check will run on deploy against our preview environment, while also still running on a schedule against production.

Vercel recently released their Checks functionality and Checkly integrates deeply into this new API. Vercel uses a slightly different way in representing checks than Checkly does, specifically splitting individual check results into either:

1. Reliability checks
2. Performance checks

This is how a Checkly check maps to a Vercel check:

• API checks are always mapped 1:1 and marked as a Reliability check.
• Browser checks are split into a Performance and Reliability check, where the Performance part is populated with Web Vitals data*.
• Groups of checks are “unfolded” in the Vercel user interface and all checks are shown individually.

* Web Vitals are only recorded for Playwright-based Browser checks

You can block your Vercel deployments if any of your checks fail by enabling the checkboxes highlighted in the screenshot below.

We have two flavours here…

1. Block my deployment when a check fails does what it says on the tin and only applies to Vercel Reliability checks.
2. Block my deployment when Web Vitals degrade is a bit special and only applies to Vercel Performance checks for our Browser checks.

This table below should help you determine how blocking checks works. Note — again — that Browser checks, can fail a Vercel deployment in two ways because the Reliability part and Performance part are evaluated separately.

Together with the team at Vercel, we developed the Virtual Experience Score (VES) that gives you one simple KPI to track.

The Virtual Experience Score does three things:

1. It aggregates your Web Vitals metrics into one number from 0 to 100, just like you might have seen with Lighthouse scoring.
2. It assigns a “weight” to each score, marking its impact on your UX.
3. It chops up the score into the top 10% (p10), the top 50% (median) and the rest.

In the table below you can see the exact numbers and their weight in the Virtual Experience Score.

So, what does this mean for the blocking heuristics? It means that we will mark a check as blocking when:

1. A check reports web vitals, and…
2. The VES has degraded in relation to the last available Production deploy. Degraded means they passed the range threshold, i.e from p10 to median.

For more info on the Virtual Experience Score check the documentation on the Vercel site.

In some cases, Checkly will completely skip performance checks. You will see the “skipped” status in your Vercel deployment overview. Checkly skips performance checks when…

1. A check doesn’t report any Web Vitals. They are all null. For example, this happens when using Puppeteer based checks.
2. Or, the domain of the visited URL in the script does not match the domain of the deployment URL. In 9 out of 10 cases this should be the URL for your Preview and Production deployments.

When selecting a check which is linked to a Vercel project, any results triggered by a deployment on that project will show at the bottom of the check page, under the tab CI/CD triggered check results.

On Vercel, you will also see a breakdown of checks that were executed on a given deployment, together with a breakdown of various key web vitals.

The Deployments tab lists all deployments on linked projects, including Vercel-based ones, together with the check executions they triggered.

Selecting a deployment enables you to quickly determine whether it caused any check failures, and to drill into the relevant information in case it did.

If you’re interested in using Checkly to bypass Vercel’s password-protected deployments. You can accomplish this using the resources referenced in the docs for browser checks or API requests.

GitLab CI Command Line triggers

You can contribute to this documentation by editing this page on Github