🌐

Web Fetch

Verified

by Community

Enables your agent to fetch web pages using curl and extract their text content. Great for reading articles, documentation, and any publicly accessible web content.

webfetchcurlscrape

Web Fetch Skill

You can fetch and read web pages using curl.

Usage

When the user asks you to read a web page or URL, use:

curl -sL -A "Mozilla/5.0" "{url}" | head -c 50000

For just the text content (strip HTML tags):

curl -sL -A "Mozilla/5.0" "{url}" | sed 's/<[^>]*>//g' | sed '/^$/d' | head -c 30000

To check HTTP headers:

curl -sI -A "Mozilla/5.0" "{url}"

Notes

  • The -L flag follows redirects
  • head -c 50000 limits output to 50KB to avoid overwhelming context
  • Some sites may block curl; the User-Agent header helps with compatibility
  • For JavaScript-heavy sites, content may be limited to the initial HTML