Web Scraping is Live on Venice: Turn any URL into AI context 

Web Scraping is Live on Venice: Turn any URL into AI context 

Venice now automatically detects, scrapes, and processes URLs in your messages. Get direct content from any page with seamless integration in app and API.

Venice.ai

Web Scraping is now widely available across the platform with seamless integration into our API. Simply include any URL in your API request or conversation, and Venice automatically detects, scrapes, and processes that content to provide you with comprehensive, context-aware responses.

How web scraping works

When you include a URL in your message or API request, Venice automatically:

  1. Detects the URLs in your input (up to 3 URLs are processed per request)

  2. Scrapes the content using our web crawling infrastructure

  3. Converts to markdown for clean, structured text extraction

  4. Augments your conversation by adding the scraped content into the model's context

  5. Generates a response that draws from both the scraped content and the model's knowledge

The entire process happens automatically in the background, requiring no special configuration or setup beyond including the URL in your message and your data remains private throughout the entire process.

When you include URLs in your message, Venice automatically switches from search mode to scraping mode. This means you get content directly from the pages you specify rather than search results about those pages. No redundant processing, no mixed results, just the exact sources you're asking about.

Using web scraping in the UI

In the chat interface, just paste a URL directly into your message:

"Summarize the key updates https://venice.ai/blog/venice-development-update-october-2025”

Venice detects the URL, scrapes it, and your selected model responds with insights drawn from that page and it works with any model in the selector.

Using web scraping via API

For developers, web scraping integrates seamlessly into the Chat Completions endpoint. Include URLs in your message content and enable the web scraping parameter:

{
  "model": "venice-uncensored",
  "messages": [
    {
      "role": "user",
      "content": "Summarize the key updates from https://venice.ai/blog/venice-development-update-october-2025"
    }
  ],
  "venice_parameters": {
    "enable_web_scraping": true
  }
}

When enable_web_scraping is set to true, Venice automatically detects URLs in your messages, scrapes the content, and feeds it into the model's context. The parameter defaults to false if not specified.

When to use web scraping

Web scraping excels when you need specific content from known sources:

  • Analyzing specific documents – Point directly at research papers, articles, or reports rather than searching for them

  • Extracting technical documentation – Pull API references, implementation guides, or specs directly into context

  • Verifying claims with sources – Cross-reference statements by scraping the actual URLs being cited

  • Tracking competitor changes – Monitor updates to pricing pages, feature lists, or marketing materials

  • Processing fresh content – Access breaking news or recently published material before it's widely indexed

Unlike web search, web scraping provides direct content extraction without algorithmic ranking or filtering. You have full control over which sources reach the model.

Pricing structure - API

Web search and web scraping requires heavy infrastructure to run reliably at scale, so starting October 30th we're introducing usage-based pricing to those features API:

  • $10/1K calls for venice-uncensored, qwen3-4b, mistral-31-24b, and qwen3-235b

  • $25/1K calls for all other models

These four models (Venice Uncensored 1.1, Venice Small, Venice Medium, and Venice Large 1.1) are our core models with dedicated infrastructure that we've scaled specifically to handle high-volume operations efficiently. That additional capacity means we can offer more competitive pricing while maintaining reliability.

These charges apply to any API call where web scraping is enabled and URLs are detected. Search or crawl content that’s injected into the prompt is metered as normal input tokens for the model you pick.

What to expect (and what won't work)

Some pages resist scraping. Paywalls, heavy JavaScript rendering, CAPTCHAs, and aggressive bot protection can block our crawlers. When that happens, you'll get a response based on successfully scraped content, minus the blocked URLs.

Large pages get truncated to fit within model context windows. We prioritize the most relevant sections, but if you're scraping massive documentation sites, expect some content to be trimmed.

The 3-URL limit per request is intentional, processing more creates latency problems and risks context overflow. To scrape more than 3 URLs, partition your target URL set and either batch separate API requests or submit multiple messages sequentially within the same conversation context.

FAQ

Does this change UI pricing
- No. This update applies to API calls that enable web search or web scraping.

Which models support web scraping?

- All models support web scraping. The feature works identically across the entire model catalog

What happens if a URL fails to scrape?

- Failed scrapes don't break your request, the conversation continues with whatever content was successfully retrieved from other URLs.

Do I get charged if scraping fails?

- If a URL fails at the network layer (cannot connect, DNS error, timeout), no charge is applied for that URL. However, if the page is accessed but content extraction is incomplete (paywalled content, JavaScript-rendered pages, etc.), the scraping attempt is still billable since server resources were used.

Can I use web search and web scraping together?

- No. When Venice detects URLs in your message, it automatically bypasses traditional web search to avoid redundant processing.

Back to all posts
Room