Stealtheir
knowledge
1 api call.

One API to scrape, index, and search any website, GitHub repo, or YouTube channel. Sub-100ms. Bypass anti-bot protection. No vector ops. No infra.

GET FREE API KEY →

SCRAPE ANY URL·SUB-100MS SEARCH·NO VECTOR OPS·AUTO-INDEX·DAILY REFRESH·SEMANTIC SEARCH·NODE SDK·PYTHON SDK·MCP SERVER·CLI TOOL·ASYNC MODE·WEBHOOKS·GITHUB INDEXER·YOUTUBE INDEXER·PUBLISHABLE KEYS·SCRAPE ANY URL·SUB-100MS SEARCH·NO VECTOR OPS·AUTO-INDEX·DAILY REFRESH·SEMANTIC SEARCH·NODE SDK·PYTHON SDK·MCP SERVER·CLI TOOL·ASYNC MODE·WEBHOOKS·GITHUB INDEXER·YOUTUBE INDEXER·PUBLISHABLE KEYS·

HOW IT WORKS

Three primitives.
Infinite use cases.

Everything runs on three core operations. Mix and match to build exactly what you need.

Extract their knowledge

Point at any URL and we crawl every page — JS rendering, pagination, anti-bot protection. All handled.

pages indexed847

markdown extracted✓

links resolved✓

js rendered✓

Search with natural language

Ask questions in plain English. Our hybrid semantic + keyword engine returns the exact answer in under 100ms.

> how does rate limiting work?

Found in stripe.com/docs

429 Too Many Requests

Retry-After header

Exponential backoff

Receive webhooks with markdown

Subscribe to knowledge updates. When content changes, we send a webhook with clean markdown and structured data.

POST /your-endpoint

{

"event": "knowledge.updated",

"url": "stripe.com/docs",

"markdown": "...",

"data": { ... }

}

copy variant

Integrate this morning

One SDK call to extract, search, and index any website. Fits right into your stack with SDKs, CLI, and MCP.

Node.js

1import KnowledgeSDK from '@knowledgesdk/node';
2 
3const knowledge = new KnowledgeSDK('sk_ks_xxxxxxxxxx');
4 
5const result = await knowledge.extract({
6  url: 'https://docs.stripe.com',
7  refresh: 'DAILY', // 'HOURLY' | 'DAILY' | 'WEEKLY'
8  maxPages: 500,
9});
10 
11// Now search with AI semantic search
12const searchResult = await knowledge.search({
13  term: "How do i setup stripe in my node.js project?"
14});
15 
16// Optional: Setup webhooks to receive content changes
17const webhook = await knowledge.webhooks.create({
18  events: ['EXTRACTION_COMPLETED', "PAGE_UPDATED"],
19  url: 'https://webhook.site/handle-webhook',
20});
21

Extract their knowledge

Point at any URL and we crawl every page — JS rendering, pagination, anti-bot protection. All handled.

pages indexed847

markdown extracted✓

links resolved✓

js rendered✓

Search with natural language

Ask questions in plain English. Our hybrid semantic + keyword engine returns the exact answer in under 100ms.

how does rate limiting work?

Found in stripe.com/docs
→ 429 Too Many Requests
→ Retry-After header
→ Exponential backoff

Receive webhooks with markdown

Subscribe to knowledge updates. When content changes, we send a webhook with clean markdown and structured data.

POST /your-endpoint
{
  "event": "knowledge.updated",
  "url": "stripe.com/docs",
  "markdown": "...",
  "data": { ... }
}

Source connected

847 pages scraped

Extracting knowledge…

Building search index…

Zero infrastructure

No Pinecone. No pipelines.
No vector DB ops.

Plug the API, schedule a refresh—hourly, daily, weekly, or monthly—and get instant AI semantic search. We handle the scraping, chunking, embeddings, and re-indexing. You handle the fun stuff.

Without KnowledgeSDK

✕Pick and provision a vector DB (Pinecone, Weaviate, Qdrant...)
✕Build a web scraper that handles JS rendering & auth
✕Write chunking, preprocessing & embedding pipelines
✕Tune search ranking, relevance scores & re-rankers
✕Schedule crawl jobs with cron, deal with stale data
✕Handle rate limits, retries & partial crawl failures
✕Monitor uptime, rebuild indexes when models change

With KnowledgeSDK

import KnowledgeSDK from '@knowledgesdk/node';
 
const knowledge = new KnowledgeSDK('sk_ks_...');
 
// Extract + schedule automatic refresh
await knowledge.extract({
  url: 'https://docs.stripe.com',
  schedule: 'daily', // hourly | daily | weekly | monthly
});
 
// Instant AI semantic search
const result = await knowledge.search({
  term: "How do I setup Stripe payments?"
});

✓AI semantic search, zero config
✓Auto-refreshes on your schedule
✓Handles JS rendering & deep pages
✓No infra, no models, no devops

Sample results from an indexed YouTube channel

Demo mode

MrBeast — Every Chocolate in a 5 Mile Radius

YouTube

In this video, MrBeast and his team bought every chocolate bar from every store within a 5 mile radius. They collected over 10,000 chocolate bars and gave them all away to random strangers at a park.

https://youtube.com/@MrBeast

MrBeast — I Spent 50 Hours Buried Alive

YouTube

MrBeast spent 50 hours buried alive in a custom underground chamber. The challenge tested his physical and mental limits. He emerged after the full duration and donated $10,000 to charity.

https://youtube.com/@MrBeast

MrBeast — Squid Game in Real Life

YouTube

MrBeast recreated the popular Netflix show Squid Game with 456 real contestants competing for a $456,000 prize. The production spanned multiple days and cost millions to produce.

https://youtube.com/@MrBeast

MrBeast — I Gave $1,000,000 to Random People

YouTube

Walked up to random strangers and handed out a total of $1,000,000. Each person received between $1 and $100,000. The reactions ranged from disbelief to tears of joy.

https://youtube.com/@MrBeast

WHAT YOU CAN BUILD

01 — GITHUB

Index GitHub Repos

Index any public GitHub repo — README, releases, issues — and make it searchable in seconds.

Learn more →

02 — YOUTUBE

Index YouTube Channels

Extract transcripts from entire YouTube channels and build a searchable video knowledge base.

Learn more →

03 — SEARCH API

Build AI Search Agents

Give your AI agent real-time access to any website's content through our search API.

Get started →

PUBLISHABLE KEYS

SEARCH FROM
THE BROWSER.

Use publishable keys (pk_ks_*) to call the public search endpoint directly from your frontend. No server required. Zero CORS issues.

—Read-only — safe to expose in the browser
—Per-project scope — search only your data
—Rotate anytime from your dashboard
—Works with React, Vue, Svelte, plain JS

// Frontend search — no server needed
const res = await fetch(
  'https://api.knowledgesdk.com/v1/projects/my-project/search/public?
    q=react+hooks&
    pk=pk_ks_...',
);
const data = await res.json();
// { results: [...], total: 12, ms: 42 }

FULL API

METHOD	ENDPOINT	DESCRIPTION
POST	/v1/extract	Full knowledge extraction (~1-3min)
POST	/v1/extract/async	Async extraction, returns jobId
POST	/v1/scrape	URL → clean markdown (~5s)
POST	/v1/screenshot	URL → base64 PNG (~3s)
POST	/v1/classify	AI business classification (~10s)
POST	/v1/sitemap	Discover all URLs on a domain
POST	/v1/search	Semantic search over your knowledge
POST	/v1/projects/{id}/index	Index a URL, GitHub repo, or YouTube channel
GET	/v1/projects/{id}/search/public	Public search with publishable key
GET	/v1/jobs/{jobId}	Poll async job status

MCP SERVER

WORKS IN
CLAUDE.
CURSOR. WINDSURF.

Drop KnowledgeSDK into any MCP-compatible AI tool. Your knowledge base becomes a searchable AI tool instantly.

// ~/.claude/claude_desktop_config.json
{
  "mcpServers": {
    "knowledgesdk": {
      "command": "npx",
      "args": ["@knowledgesdk/mcp"],
      "env": {
        "KNOWLEDGESDK_API_KEY": "sk_ks_..."
      }
    }
  }
}

ECOSYSTEM

JS / TS

NODE SDK

npm install @knowledgesdk/node

PYTHON

PYTHON SDK

pip install knowledgesdk

AI TOOLS

MCP SERVER

npx @knowledgesdk/mcp

TERMINAL

CLI

npx @knowledgesdk/cli

INDEX.SEARCH.SHIP.

Free tier available. No credit card required.

GET ACCESS →

Stealtheirknowledge1 api call.

Three primitives.Infinite use cases.

Extract their knowledge

Search with natural language

Receive webhooks with markdown

Integrate this morning

Three primitives.Infinite use cases.

Extract their knowledge

Search with natural language

Receive webhooks with markdown

No Pinecone. No pipelines.No vector DB ops.

MrBeast — Every Chocolate in a 5 Mile Radius

MrBeast — I Spent 50 Hours Buried Alive

MrBeast — Squid Game in Real Life

MrBeast — I Gave $1,000,000 to Random People

Index GitHub Repos

Index YouTube Channels

Build AI Search Agents

SEARCH FROMTHE BROWSER.

WORKS INCLAUDE.CURSOR. WINDSURF.

INDEX.SEARCH.SHIP.

Stealtheir
knowledge
1 api call.

Three primitives.
Infinite use cases.

Three primitives.
Infinite use cases.

No Pinecone. No pipelines.
No vector DB ops.

SEARCH FROM
THE BROWSER.

WORKS IN
CLAUDE.
CURSOR. WINDSURF.