Data Analysis

Website Content Scraper

General-purpose website scraper handling JavaScript rendering and various page structures.

Overview

The Website Scraper Agent is a general-purpose scraper handling various page structures and JavaScript-rendered content. Supports sitemap parsing, screenshot capture, and PDF extraction.

Key Capabilities:

  • Page fetching with proper headers
  • Content extraction (text, links, metadata)
  • JavaScript rendering (Puppeteer)
  • Sitemap parsing
  • Screenshot capture
  • PDF extraction
  • robots.txt compliance

Tools & Integrations:

  • Puppeteer - JavaScript rendering
  • Content Extractor - HTML parsing
  • PDF Parser - Document extraction

Instructions preview

No instructions preview available.

Your privacy choices

We use optional assistant personalization, analytics, and advertising technologies only when you allow them. Necessary site functions remain active. Cookie Policy