Visual Regression Testing with Screenshot API

Visual regression testing is a powerful technique that helps catch unintended visual changes in your user interface. By comparing screenshots of your application before and after code changes, you can identify visual bugs that traditional functional tests might miss. This guide will show you how to implement visual regression testing using our Screenshot API.

Why Visual Regression Testing Matters

Traditional unit and integration tests verify that your code functions correctly, but they don't catch visual issues like:

Misaligned elements
Overlapping text
Incorrect colors or styling
Missing images or icons
Responsive design breakages
Cross-browser rendering differences

Setting Up Your Testing Pipeline

1. Capture Baseline Screenshots

First, you need to establish baseline screenshots of your application in a known good state:

javascript

01const fs = require('fs').promises;
02const path = require('path');
03const fetch = require('node-fetch');
04
05async function captureBaselines(pages) {
06  const baselineDir = path.join(__dirname, 'baselines');
07  await fs.mkdir(baselineDir, { recursive: true });
08  
09  for (const page of pages) {
10    console.log(`Capturing baseline for ${page.name}...`);
11    
12    const response = await fetch('https://api.screenshotapi.com/capture', {
13      method: 'POST',
14      headers: {
15        'Content-Type': 'application/json',
16        'Authorization': `Bearer ${process.env.SCREENSHOT_API_KEY}`
17      },
18      body: JSON.stringify({
19        url: page.url,
20        width: 1280,
21        height: 800,
22        fullPage: true,
23        format: 'png'
24      })
25    });
26    
27    const data = await response.json();
28    
29    // Download the screenshot
30    const imageResponse = await fetch(data.screenshotUrl);
31    const imageBuffer = await imageResponse.buffer();
32    
33    // Save as baseline
34    await fs.writeFile(
35      path.join(baselineDir, `${page.name}.png`),
36      imageBuffer
37    );
38  }
39}
40
41// Define pages to test
42const pages = [
43  { name: 'homepage', url: 'https://your-app.com/' },
44  { name: 'login', url: 'https://your-app.com/login' },
45  { name: 'dashboard', url: 'https://your-app.com/dashboard' }
46];
47
48captureBaselines(pages);

2. Implement Comparison Logic

Next, create a function to compare new screenshots against the baselines:

javascript

01const { PNG } = require('pngjs');
02const pixelmatch = require('pixelmatch');
03
04async function compareWithBaseline(pageName, currentScreenshotBuffer) {
05  const baselineDir = path.join(__dirname, 'baselines');
06  const diffDir = path.join(__dirname, 'diffs');
07  
08  await fs.mkdir(diffDir, { recursive: true });
09  
10  // Read baseline image
11  const baselineBuffer = await fs.readFile(
12    path.join(baselineDir, `${pageName}.png`)
13  );
14  
15  // Parse PNG images
16  const baseline = PNG.sync.read(baselineBuffer);
17  const current = PNG.sync.read(currentScreenshotBuffer);
18  
19  // Create output image
20  const { width, height } = baseline;
21  const diff = new PNG({ width, height });
22  
23  // Compare pixels
24  const numDiffPixels = pixelmatch(
25    baseline.data,
26    current.data,
27    diff.data,
28    width,
29    height,
30    { threshold: 0.1 }
31  );
32  
33  // Calculate difference percentage
34  const totalPixels = width * height;
35  const percentageDifference = (numDiffPixels / totalPixels) * 100;
36  
37  // Save diff image if there are differences
38  if (numDiffPixels > 0) {
39    await fs.writeFile(
40      path.join(diffDir, `${pageName}-diff.png`),
41      PNG.sync.write(diff)
42    );
43  }
44  
45  return {
46    hasDifferences: numDiffPixels > 0,
47    diffPixels: numDiffPixels,
48    percentageDifference,
49    diffPath: numDiffPixels > 0 ? path.join(diffDir, `${pageName}-diff.png`) : null
50  };
51}

3. Integrate with CI/CD Pipeline

Now, integrate the visual testing into your CI/CD pipeline:

javascript

01const fetch = require('node-fetch');
02
03async function runVisualTests(pages) {
04  let failedTests = 0;
05  
06  for (const page of pages) {
07    console.log(`Testing ${page.name}...`);
08    
09    // Capture current state
10    const response = await fetch('https://api.screenshotapi.com/capture', {
11      method: 'POST',
12      headers: {
13        'Content-Type': 'application/json',
14        'Authorization': `Bearer ${process.env.SCREENSHOT_API_KEY}`
15      },
16      body: JSON.stringify({
17        url: page.url,
18        width: 1280,
19        height: 800,
20        fullPage: true,
21        format: 'png'
22      })
23    });
24    
25    const data = await response.json();
26    
27    // Download the screenshot
28    const imageResponse = await fetch(data.screenshotUrl);
29    const imageBuffer = await imageResponse.buffer();
30    
31    // Compare with baseline
32    const result = await compareWithBaseline(page.name, imageBuffer);
33    
34    if (result.hasDifferences) {
35      console.error(`❌ ${page.name} has visual differences!`);
36      console.error(`   ${result.percentageDifference.toFixed(2)}% of pixels differ`);
37      console.error(`   Diff image saved to: ${result.diffPath}`);
38      failedTests++;
39    } else {
40      console.log(`✅ ${page.name} looks identical to baseline`);
41    }
42  }
43  
44  if (failedTests > 0) {
45    console.error(`\n${failedTests} tests failed with visual differences`);
46    process.exit(1); // Fail the CI build
47  } else {
48    console.log('\nAll visual tests passed!');
49  }
50}

GitHub Actions Integration

Here's how to integrate this into a GitHub Actions workflow:

yaml

01name: Visual Regression Tests
02
03on:
04  pull_request:
05    branches: [ main ]
06
07jobs:
08  visual-test:
09    runs-on: ubuntu-latest
10    
11    steps:
12    - uses: actions/checkout@v2
13    
14    - name: Setup Node.js
15      uses: actions/setup-node@v2
16      with:
17        node-version: '16'
18        
19    - name: Install dependencies
20      run: npm install
21      
22    - name: Run visual regression tests
23      run: node visual-regression-tests.js
24      env:
25        SCREENSHOT_API_KEY: ${{ secrets.SCREENSHOT_API_KEY }}
26        
27    - name: Upload diff images if tests fail
28      if: failure()
29      uses: actions/upload-artifact@v2
30      with:
31        name: visual-diffs
32        path: diffs/

Handling Dynamic Content

Dynamic content can cause false positives in visual regression tests. Here are strategies to handle it:

1. Element Masking

Hide dynamic elements before capturing:

javascript

01// Add executeBeforeScreenshot to hide dynamic elements
02body: JSON.stringify({
03  url: page.url,
04  executeBeforeScreenshot: `
05    // Hide date/time elements
06    document.querySelectorAll('.timestamp, .date').forEach(el => {
07      el.style.visibility = 'hidden';
08    });
09    
10    // Hide user-specific content
11    document.querySelectorAll('.user-avatar, .username').forEach(el => {
12      el.style.visibility = 'hidden';
13    });
14  `
15})

2. Region-Based Comparison

Only compare specific regions of the page:

javascript

01// Modify the comparison function to only compare specific regions
02const regions = [
03  { x: 0, y: 0, width: 1280, height: 200 },  // Header
04  { x: 200, y: 300, width: 800, height: 400 } // Main content
05];
06
07// For each region, extract and compare that portion of the image
08for (const region of regions) {
09  // Extract region from both images and compare
10  // ...
11}

Best Practices

Run tests in a consistent environment: Use containerization to ensure consistent rendering
Update baselines intentionally: Don't automatically update baselines; review changes first
Set appropriate thresholds: Allow for minor pixel differences (0.1-1%) to avoid false positives
Test responsive breakpoints: Capture screenshots at different viewport sizes
Include cross-browser testing: Test in Chrome, Firefox, Safari, and Edge

By implementing visual regression testing with our Screenshot API, you can catch visual bugs early in your development process, ensuring a consistent and polished user experience.

Visual Regression Testing with Screenshot API

Why Visual Regression Testing Matters

Setting Up Your Testing Pipeline

1. Capture Baseline Screenshots

2. Implement Comparison Logic

3. Integrate with CI/CD Pipeline

GitHub Actions Integration

Handling Dynamic Content

1. Element Masking

2. Region-Based Comparison

Best Practices

Ready to Get Started?