Hiding in Plain Sight

When a website is heavily protected by Datadome or Akamai, simple HTTP requests (curl_cffi) are often insufficient. The site will force you to execute complex JavaScript challenges or solve invisible Turnstile CAPTCHAs that require a full DOM environment to evaluate.

To proceed, you must use a Headless Browser (Playwright, Puppeteer, Selenium). However, out-of-the-box headless browsers are incredibly easy for WAFs to detect.

The Leaks of Standard Headless Browsers

When you launch playwright or selenium in headless mode, the Chromium engine leaves dozens of blatant "I am a bot" flags scattered throughout the JavaScript environment: 1. navigator.webdriver === true (The most obvious flag). 2. The window.chrome object is missing or malformed. 3. The Permissions API behaves abnormally in headless mode. 4. The Plugins array is empty (real browsers always have default PDF plugins). 5. The User-Agent string contains the literal substring HeadlessChrome. 6. navigator.languages comes back as a single default instead of a realistic ordered list. 7. document.hidden never fires and Page Visibility always reports visible, because headless reverting tabs never actually happens.

The Solution: Stealth Plugins

To survive, you must patch the browser binary in real-time, injecting JavaScript before the page loads to overwrite and mock these exposing variables.

1. Puppeteer Extra Stealth (Node.js) The industry standard in the JavaScript ecosystem. It applies a suite of evasion techniques automatically.

const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());

puppeteer.launch({ headless: true }).then(async browser => {
  const page = await browser.newPage();
  await page.goto('https://bot.sannysoft.com'); // Bot detection test site
  await page.screenshot({ path: 'stealth.png' });
  await browser.close();
});

2. Undetected Chromedriver (Python) For Python developers, undetected-chromedriver is an optimized Selenium wrapper that aggressively patches the ChromeDriver executable binary at the hex level, preventing it from passing the cdc_ string variables that WAFs actively hunt for.

import undetected_chromedriver as uc
import time

options = uc.ChromeOptions()
# uc automatically handles binary patching and JS evasion
driver = uc.Chrome(options=options)

driver.get("https://nowsecure.nl")
time.sleep(5)
driver.quit()

The List Is Longer Than You Think

Complete stealth means patching far more than navigator.webdriver. A hardened setup also fakes: the window.chrome API surface, a non-empty navigator.plugins array, a realistic navigator.languages, the HTMLMediaElement permission behavior, the chrome://media-engagement ID, WebGL renderer strings, and the crypto.getRandomValues entropy behavior. Many WAFs probe a dozen or more properties at once and compare the combination rather than any single value.

The Sandboxing Trap

Default Chromium sandboxing and GPU flags are another giveaway. Real browsers enable hardware acceleration, WebGL, and audio output. Launching a headless browser takes 30–60 seconds to warm up the renderer, initialize the media stack, and fetch nothing. Loading the first page cold—a famously detectable signature—can be hidden by warming the browser (loading about:blank, then a neutral page) before touching the target.

The Future: Anti-Detect Browsers

For enterprise-scale scraping, managing Playwright stealth scripts becomes fragile. Companies now use Anti-Detect Browsers (like AdsPower, Multilogin, or Dolphin Anty). These are custom-compiled forks of Chromium where the fingerprinting APIs (Canvas, WebGL, Audio) have been rewritten in C++ to allow perfect, mathematical spoofing directly from an API payload, completely bypassing JavaScript detection vectors.