web-scraper-api

1# Oxylabs Web Scraper API
2 
3## Authentication
4 
5Requires HTTP Basic Auth with credentials from environment variables:
6 
7```bash
8curl -u "$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD" ...
9```
10 
11## Endpoint
12 
13```
14POST https://realtime.oxylabs.io/v1/queries   # immediate response
15POST https://data.oxylabs.io/v1/queries       # Push-Pull jobs, callbacks, storage
16Content-Type: application/json
17```
18 
19## Core Parameters
20 
21| Parameter | Required | Description |
22|-----------|----------|-------------|
23| `source` | Yes | Target scraper (e.g., `universal`, `amazon_product`, `google_search`) |
24| `url` | Conditional | URL to scrape (for `universal` and `*_url` sources) |
25| `query` | Conditional | Search query or product ID (for `*_search` and `*_product` sources) |
26| `parse` | No | Enable structured data parsing (recommended for supported sources) |
27| `render` | No | JavaScript rendering: `html` or `png` |
28| `geo_location` | No | Geographic targeting: country/state/city, ZIP/postcode, coordinates, or Criteria ID where supported |
29| `session_id` | No | Reuse the same proxy IP across multiple jobs |
30| `content_encoding` | No | Set to `base64` when downloading image files via Realtime or Push-Pull |
31| `user_agent_type` | No | Device/browser preset, e.g., `desktop_chrome`, `mobile_ios`, `tablet_android` |
32| `locale` | No | Interface language / `Accept-Language`, e.g., `de-DE` |
33| `callback_url` | No | Push-Pull callback endpoint |
34…