DEV Community

# webscraping

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Give Your AI Agent a Web-Fetch Tool: a 60-Line MCP Server (Free, Self-Hosted)
0012303 profile

Give Your AI Agent a Web-Fetch Tool: a 60-Line MCP Server (Free, Self-Hosted)

#webscraping #python #ai #mcp
Comments
10 min read
How We Optimized a Django Playwright Scraper to Save 60% on Rotating Proxy Bandwidth
proxyvero profile

How We Optimized a Django Playwright Scraper to Save 60% on Rotating Proxy Bandwidth

#python #django #webscraping #playwright
Comments
4 min read
Building a Lean, Single-Worker Broken URL Monitor for Data Pipelines
datawinder profile

Building a Lean, Single-Worker Broken URL Monitor for Data Pipelines

#webscraping #datapipeline #devtools #apify
Comments
6 min read
How Paywalls Actually Work: The Engineering Behind Them
tonywangca profile

How Paywalls Actually Work: The Engineering Behind Them

#webscraping #seo #webdev #tutorial
2
Comments
14 min read
Why Cloudflare Breaks Proxy-Only Scrapers
anakin_writers profile

Why Cloudflare Breaks Proxy-Only Scrapers

#webscraping #cloudflare #python #automation
Comments
4 min read
How to track Weibo hot-search velocity with Python in 2026 — the trending-delta problem and how to handle it
sami_8858131362756585e4f4 profile
Sami

How to track Weibo hot-search velocity with Python in 2026 — the trending-delta problem and how to handle it

#python #webscraping #datascience #china
Comments
4 min read
How to Scrape E-Commerce Sites for AI Agents Using Playwright and LLMs
alterlab profile

How to Scrape E-Commerce Sites for AI Agents Using Playwright and LLMs

#webscraping #aiagents #playwright #llm
Comments
6 min read
Your AI Agent Is Paying for HTML It Never Reads — I Measured the 7x Token Tax
0012303 profile

Your AI Agent Is Paying for HTML It Never Reads — I Measured the 7x Token Tax

#ai #python #webscraping #llm
Comments
5 min read
What is Web Scraping? A Beginner's Guide with Real Python Code
raagawrites profile

What is Web Scraping? A Beginner's Guide with Real Python Code

#python #webscraping #beginners #programming
2
Comments
2 min read
Your scraper says 200 OK. I measured how often it's lying.
san64777 profile

Your scraper says 200 OK. I measured how often it's lying.

#showdev #python #webscraping #opensource
1
Comments
7 min read
How Scraping API Pricing Changes Once You Need Browser Sessions
anakin_writers profile
Anakin

How Scraping API Pricing Changes Once You Need Browser Sessions

#webscraping #api #javascript #automation
Comments
3 min read
How to Get Google Search Results in JSON for an AI Agent
cecilia_hill_d7b1b8d510e7 profile

How to Get Google Search Results in JSON for an AI Agent

#ai #api #python #webscraping
3
Comments
6 min read
Building Anti-Bot Detection Systems: How RepoHunter Scrapes GitHub Trending Repositories
axthonyv profile

Building Anti-Bot Detection Systems: How RepoHunter Scrapes GitHub Trending Repositories

#python #webscraping #automation #architecture
Comments
4 min read
Get any Instagram profile data in 10 lines of Python
pouria_anushiravani_42b89 profile

Get any Instagram profile data in 10 lines of Python

#python #api #webscraping #beginners
1
Comments
2 min read
How I built a self-healing, robots-respecting web scraper (and put it on the Apify Store)
teskerton profile

How I built a self-healing, robots-respecting web scraper (and put it on the Apify Store)

#showdev #webscraping #python #api
2
Comments
3 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.