37labsai manager

BoterVlaai Wikipedia fleet

Every BoterVlaai device checks this ledger before scraping. If a page is already available, skip it. If it still needs work, claim it, scrape it, then send the processed botervlaai.wiki.v1 package back. Many devices, no duplicated pages.

Loading coverage…

Check URL first

Paste a Wikipedia URL or article title. The app does this automatically before any scrape.

 
 
 
Still need scraping (sample)
 
Pending work the fleet can claim. Devices take a lease so two phones never fetch the same URL.
   
 
 
Packages waiting for a Mac
 

Compact botervlaai.wiki.v1 files on the VPS. A Studio or MacBook imports them
				into SwiftData with bv-tool fleet-pages pull --apply.

   
 
 
Device contract
 
     
  1. GET /api/wiki-fleet/check?url=… — if available: true, stop.
    2.  
  2. POST /api/wiki-fleet/claim with header X-37Labs-Device-Id — only scrape granted URLs.
    3.  
  3. Process locally into BoterVlaai format (chunks + page block).
    4.  
  4. POST /api/wiki-fleet/ingest with format: "botervlaai.wiki.v1" (include links[] when you have them).
    5.  
  5. A Mac pulls those packages into pv-knowledge-3: bv-tool fleet-pages pull --apply.
    6.  
 
# 1. is it already available?
curl -sS "https://37labsai.com/api/wiki-fleet/check?url=https://en.wikipedia.org/wiki/DNA"

# 2. claim work for this device (only scrape granted URLs)
curl -sS -X POST "https://37labsai.com/api/wiki-fleet/claim" \
  -H "content-type: application/json" \
  -H "X-37Labs-Device-Id: $(hostname)-wiki" \
  -d '{"source":"wikipedia","count":5}'

# 3. after processing, send the BoterVlaai package
curl -sS -X POST "https://37labsai.com/api/wiki-fleet/ingest" \
  -H "content-type: application/json" \
  -H "X-37Labs-Device-Id: $(hostname)-wiki" \
  -d '{"format":"botervlaai.wiki.v1","url":"https://en.wikipedia.org/wiki/DNA","title":"DNA","chunks":[{"index":0,"text":"…"}]}'