Open playground Start free

Keyword: url to markdown API · Updated July 31, 2026

URL to Markdown API: Extract Rendered Pages for AI Agents and RAG

AI agents and RAG pipelines often need page text, not pixels. Sending a screenshot to a model is useful when layout matters, but it is wasteful when the task is "summarize this pricing page," "compare these docs," or "extract the current terms from this URL." A URL to Markdown API gives the agent clean, bounded text from a rendered web page while keeping screenshot capture available for visual proof.

Why this is timely

Current AI platforms are moving URL and search retrieval closer to the model runtime. Google documents URL Context as a tool for extracting data, comparing documents, synthesizing sources, and analyzing docs from direct URLs. Anthropic's current web-search API docs describe cited real-time web content and response controls for agentic workflows. Cloudflare's Browser Run docs also now expose browser-backed Markdown, content, and crawl endpoints, with the crawl page last updated July 20, 2026.

That trend does not remove the need for product-owned extraction. Teams still need predictable output formats, auth boundaries, logging, quota control, and a fallback path to pixels when the model needs to inspect layout. SnapshotFlow fits that middle layer: GET /extract returns rendered page content as Markdown, HTML, or text, while GET /screenshot can return an image plus extracted content in the same JSON response.

Scope: this guide covers a single-URL extraction workflow for pages you are allowed to fetch. For image-based analysis, use the AI vision pipeline guide. For MCP setup, use the SnapshotFlow MCP server guide.

When to use URL to Markdown instead of a screenshot

Start with text extraction when the next step is semantic: search, summary, classification, embedding, diffing copy, or feeding an agent context. Ask for a screenshot only when visual state matters.

TaskUseWhy
Summarize a docs pageGET /extract?format=markdownMarkdown keeps headings, links, lists, and readable structure for an LLM or editor.
Build a lightweight RAG ingestion jobGET /extract?format=textPlain text is easy to chunk and embed when formatting is not important.
Audit title, Open Graph, or favicon values/screenshot?metadata=trueMetadata extraction returns structured page fields in the JSON response.
Review a page that may visually differ from its text/screenshot?extract_content=trueOne response can include rendered evidence and Markdown context.
Compare layout, banners, or above-the-fold UIGET /screenshotUse pixels when spacing, overlays, or visual regressions are the subject.

1. Extract rendered page Markdown with cURL

Use GET /extract when you do not need an image. SnapshotFlow validates the URL, renders the page through its browser pool, converts the body to Markdown, caps output with max_chars, and returns JSON.

curl --get "https://api.snapshotflow.com/extract" \ -H "X-Api-Key: $SNAPSHOTFLOW_API_KEY" \ --data-urlencode "url=https://example.com/docs" \ --data-urlencode "format=markdown" \ --data-urlencode "max_chars=12000"

Expected JSON shape:

{ "url": "https://example.com/docs", "format": "markdown", "content": "# Example Docs\n\nUse this guide...", "chars": 12000, "truncated": true, "took_ms": 842 }

Use format=html when your parser needs the rendered DOM, and format=text when the output will go straight into chunking or embeddings. The backend allows max_chars from 100 to 500,000 characters.

2. Add a small Node extraction helper

The official Node SDK covers screenshot, batch, diff, jobs, and screenshot JSON helpers. It does not currently expose a dedicated extract() convenience method, so call the /extract endpoint with fetch for this specific workflow.

async function extractMarkdown(url) { const params = new URLSearchParams({ url, format: "markdown", max_chars: "12000", }); const res = await fetch(`https://api.snapshotflow.com/extract?${params}`, { headers: { "X-Api-Key": process.env.SNAPSHOTFLOW_API_KEY }, }); const data = await res.json(); if (!res.ok) { throw new Error(`${data.error || res.status}: ${data.message || "Extraction failed"}`); } return { url: data.url, markdown: data.content, chars: data.chars, truncated: data.truncated, tookMs: data.took_ms, }; } const page = await extractMarkdown("https://example.com/pricing"); if (page.truncated) { console.warn("The page exceeded max_chars; store a truncation marker with the source."); } console.log(page.markdown);

Store url, format, chars, truncated, and capture time next to the extracted content. That metadata makes downstream summaries easier to audit.

3. Pair Markdown with a screenshot when reviewers need proof

Some workflows need both: text for the model and an image URL for a human reviewer. Use /screenshot with response_type=json, extract_content=true, and content_format=markdown. The Node SDK can do this through takeJson().

import { SnapshotFlow } from "snapshotflow"; const client = new SnapshotFlow({ apiKey: process.env.SNAPSHOTFLOW_API_KEY, baseUrl: "https://api.snapshotflow.com", }); const result = await client.takeJson({ url: "https://example.com/pricing", responseType: "json", format: "webp", width: 1440, height: 900, waitUntil: "networkidle2", waitForSelector: "main", extractContent: true, contentFormat: "markdown", metadata: true, blockAds: true, blockCookieBanners: true, }); console.log(result.metadata?.title); console.log(result.content); console.log(result.storagePath);

Expected behavior: SnapshotFlow captures the rendered page, returns JSON with page metadata and Markdown content, and includes screenshot history/storage fields when the configured storage path is available. Content and metadata requests bypass cache lookup because page text and metadata can change even when a cached image exists.

Operational advice and edge cases

  • Use extraction for allowed pages only. SnapshotFlow validates URLs and blocks private-network targets. Do not use it to fetch content you are not authorized to access.
  • Set a useful max_chars. Start with 8,000 to 20,000 characters for agent prompts. Increase only when the workflow actually needs long-form context.
  • Treat truncated=true as a data-quality flag. Do not ask the model to make whole-page claims when the response was capped. Store the flag and either fetch a narrower URL or raise the cap deliberately.
  • Prefer Markdown for humans and agents. Markdown preserves headings and links better than raw text. Use raw HTML only when your pipeline needs DOM-level parsing.
  • Waits still matter. For screenshot-plus-text workflows, use wait_until and wait_for_selector to avoid capturing an app shell before the content loads.
  • Do not treat extracted text as a citation by itself. If a model uses the output in a user-facing answer, keep the source URL and display citations according to your product policy.
  • Do not leak secrets into generated URLs. Keep API keys in headers for server-side calls. Use query-param keys only for controlled server-side URL generation paths.
Security note: authenticated pages should be extracted only from a trusted server environment. Keep cookies, bearer tokens, and API keys out of browser JavaScript, public logs, and shared tickets.

FAQ

Is URL to Markdown the same as AI web search?

No. Web search helps a model find sources. A URL to Markdown API extracts a specific URL that your product, agent, or queue already selected. Use search for discovery and extraction for deterministic ingestion of known pages.

Can SnapshotFlow extract JavaScript-rendered pages?

Yes. The extraction service uses the same browser pool architecture as the screenshot service, so it can navigate pages that render content in Chromium. If the page loads slowly, use the screenshot JSON path with explicit wait parameters.

Should I use /extract or /screenshot?extract_content=true?

Use /extract when text is enough. Use /screenshot?response_type=json&extract_content=true when the workflow needs both machine-readable content and visual evidence for review.

Sources and further reading

Give your agent text before pixels

Start with SnapshotFlow examples or the content extraction parameters, then choose the lightest response your workflow can safely use.

Start free