# TomsIndex

> Page extraction API for AI agents: POST a URL, get LLM-ready markdown, links and metadata. Every extract is saved to a private, full-text searchable index. Reads server-rendered HTML and text/markdown URLs; GitHub files/issues/folders, npm, PyPI and Stack Exchange are read via their APIs. JavaScript pages are rendered in a headless browser when needed. No crawling, PDFs or screenshots. Free in beta, 10 requests/minute per key.

## Auth

Send `X-API-Key: srch_...` (create keys at https://tomsindex.com/dashboard).

## Endpoints

- POST https://tomsindex.com/v1/extract — body `{"url": "...", "format": "markdown"|"text", "css_selector": "...", "query": "...", "chunks_per_source": 1-5, "include_images": bool, "save": bool}`. Returns `{ results: [{ id, url, markdown, raw_content, metadata, links, warnings? }], failed_results: [{ url, error }], response_time }`. Failures are HTTP 200 with `failed_results`.
- GET https://tomsindex.com/v1/index?q=&limit=&offset= — your saved pages (id, url, title, fetched_at); `q` is full-text search ranked by relevance.
- GET https://tomsindex.com/v1/index/:id — one saved page with its markdown.
- DELETE https://tomsindex.com/v1/index/:id — remove a page from your index.
- GET https://tomsindex.com/v1/index/export — your whole index as JSON Lines.

## MCP

Remote MCP server (Streamable HTTP) at https://tomsindex.com/mcp with header `X-API-Key: srch_...` (or keyless as a trial: extract plus index_get for paging, 20/day per IP, nothing saved). Tools: extract (returns the saved page's index id; long pages are cut at max_chars, default 25000), index_search (with snippets), index_get (read on with id + offset; no re-fetch). MCP extract calls are limited to 10/min per key.
Claude Code: `claude mcp add --transport http tomsindex https://tomsindex.com/mcp --header "X-API-Key: srch_..."`

## Docs

- [API reference](https://tomsindex.com/docs)