The Foundation Update: From Theory to Working Federation

TL;DR: Last week I shipped The Foundation with clipboard capture and security theater. Today it's a real ActivityPub server with automatic conversation capture and passage-level search precision. Here's what actually works now.

What Changed Since Launch

Before (Feb 9):

• Clipboard scraping with Ctrl+C
• Security scanner blocking legitimate content
• Theoretical ActivityPub support
• Search returned chunks, not conversations

After (Feb 18):

• Browser extension auto-captures via Claude's internal API
• Real ActivityPub federation (discoverable on Mastodon)
• Passage-level search with scroll-to-highlight
• Clean UI with mobile support

The Clipboard Capture Mistake

My original approach was fundamentally broken. I tried to capture conversations by listening for copy events:

document.addEventListener('copy', async (e) => {
  const copiedText = window.getSelection().toString();
  // Parse timestamps to detect Claude vs user...
});

Problems:

• Required manual Ctrl+A then Ctrl+C
• Missed artifacts (separate panel, not selected)
• Missed uploaded files (not in DOM)
• Only captured partial conversations (virtual scrolling)
• Timestamp regex failed → only got user messages

It was clever but fragile. Every Claude UI update broke it.

The Real Solution: Claude's Internal API

Claude Exporter figured this out first. Instead of scraping the DOM, use the API that Claude.ai itself uses:

// 1. Get org ID (one-time setup)
const orgs = await fetch('https://claude.ai/api/organizations', {
  credentials: 'include'
}).then(r => r.json());

const orgId = orgs[0].uuid;

// 2. Get conversation ID from URL
const convId = window.location.pathname.split('/').pop();

// 3. Fetch complete conversation
const conv = await fetch(
  `https://claude.ai/api/organizations/${orgId}/conversations/${convId}`,
  { credentials: 'include' }
).then(r => r.json());

What you get:

• All 44 messages (not 22 user messages)
• Artifacts included
• File metadata (names, types)
• Timestamps on every message
• Claude's auto-generated summary (free)
• Zero truncation

The extension now injects a "Share to Foundation" button. Click it → conversation captured → sent to Worker → embedded → searchable.

Real ActivityPub Federation

The original article mentioned federation but never built it. Now it's real:

Working endpoints:

• /.well-known/nodeinfo → instance metadata
• /.well-known/webfinger → user lookup
• /federation/actor → ActivityPub identity with RSA keypair
• /federation/inbox → receives Follow activities
• /federation/followers → who's following this instance

What this means:

Mastodon users can search for @knowledge@chat-knowledge-api.fpl-test.workers.dev and follow The Foundation.

✅ Federation tested and working: Successfully followed from Mastodon (techhub.social). Follow activity received and processed at inbox, follower stored in database. The Foundation is live on the fediverse. When someone follows, the inbox handler:

1. Receives Follow activity
2. Stores follower in federated_instances table
3. Sends Accept activity back
4. They see "Follow request accepted"

It's not theoretical. The protocol works. Other instances can discover you.

⚠️ Note: HTTP signature implementation is still pending. This means The Foundation can receive Follow activities and send Accept responses, but cannot yet send signed activities to other instances. Full two-way federation is the next step.

Passage-Level Search Precision

Search used to return chunk IDs. Click a result → land somewhere in a 300-message conversation. Frustrating.

Now it's surgical:

Search query: "scroll-to-passage"
Result: "Passage 2 of 4" in conversation X
Click it: ?msg=92 in URL → scrolls to message 92 → amber highlight pulse

// search.ts extracts message_index from metadata
var msgIndex = r.metadata.message_index;
var url = '/view/' + r.chatId + '?msg=' + msgIndex;

// chat.ts scrolls to exact message
var target = document.querySelector('.message[data-index="' + msgIndex + '"]');
target.scrollIntoView({ behavior: 'smooth' });
target.classList.add('highlighted'); // 2.4s amber pulse

No approximation. No "scroll to top of chunk and hope." You land on the exact message that matched your query.

What I Removed

Security scanning: The original version tried to block API keys before storage. Noble goal, terrible execution. It blocked legitimate code examples, created friction, and users just worked around it.

Better approach: Don't try to be smart. Let people save what they want. If they share publicly, they'll review it first. Trust users.

Clipboard hacks: All gone. Extension uses Claude's API like a normal application.

Complexity: The codebase is smaller and more maintainable now. Fewer edge cases, fewer brittle parsers.

The Stack That Actually Works

Browser Extension
  ├─ Auto-detects org ID (zero config)
  ├─ Fetches conversation from Claude's API
  └─ Sends to Worker

Cloudflare Worker
  ├─ Stores in D1 (chats + messages + chunks)
  ├─ Generates embeddings (Workers AI)
  └─ Indexes in Vectorize

Search
  ├─ Semantic search via Vectorize
  ├─ Returns passage snippets with message_index
  └─ UI scrolls to exact message

ActivityPub
  ├─ NodeInfo discovery
  ├─ WebFinger lookup
  ├─ Actor with RSA keypair
  └─ Inbox processes Follow activities

Cost: ~$5/month on Cloudflare's free tier
Latency: <100ms search response
Uptime: 99.9% (Cloudflare SLA)

Current Limitations

This is infrastructure, not a finished product:

• ⚠️ HTTP signatures not implemented — The Foundation can receive Follow activities and accept them, but cannot yet send signed activities to other instances. Full two-way federation is still pending.
• ⚠️ No visibility controls yet — All conversations are private by default. There's no per-conversation public/private toggle yet.
• ⚠️ Single-user instance — Not multi-tenant. Built for personal use; self-hosters each run their own instance.
• ⚠️ Federated Q&A is a separate future project — Mentioned in the roadmap below, but not part of this release.

The goal isn't perfection — it's proving the protocol works.

What's Next

Immediate:

• ✅ Test Mastodon follow flow (Done - tested with techhub.social)
• Implement HTTP signatures for two-way federation
• Add visibility controls (public/private per conversation)
• Write self-hosting documentation

Next Month:

• Multi-tenant support (multiple users per instance)
• Cross-instance search
• Collections feature (curate conversations by topic)
• Chrome Web Store submission

Long-term Vision:

• Federated Q&A as separate product
• Same protocol, different content type (questions/answers instead of conversations)
• Let developers own their knowledge infrastructure

The Real Vision:

The Foundation proves federated knowledge commons work. Federated Q&A proves they scale. Same infrastructure, same protocol, different use case.

You already have 70% of the code. Just change:

• Content type: conversations → questions/answers
• UI: chat viewer → Q&A forum
• Interaction: search → ask + vote

Try It Yourself

Backend (Worker):

git clone https://github.com/dannwaneri/chat-knowledge.git
cd chat-knowledge
npm install
wrangler d1 create chat-knowledge-db
# Update wrangler.toml with database ID
wrangler d1 execute chat-knowledge-db --remote --file=schema.sql
npm run deploy

Extension:

1. Load unpacked from extension/ folder
2. Navigate to claude.ai/chat
3. Click "Share to Foundation" button
4. Conversation captured automatically

Search:
Visit your Worker URL → semantic search across all captured conversations

The Hard Truth

I over-engineered the first version. Security scanning, clipboard hacks, complex parsers — all solving problems users didn't have.

The new version does less, but does it right:

• Capture: just works (uses Claude's API)
• Search: actually precise (passage-level)
• Federation: real protocol (ActivityPub)

Ship working code. Iterate based on real usage. Stop building features nobody asked for.

The Foundation is live. The protocol works. The search is precise. The federation layer is ready — with the known limitations above clearly documented.

What comes next isn't more features. It's getting people to use it.

Links:

• Live demo: https://chat-knowledge-api.fpl-test.workers.dev
• Source: https://github.com/dannwaneri/chat-knowledge
• Previous article: I Built Federated AI Knowledge Commons