Documentation

Install

The distribution is raft-ft (raft was taken on PyPI); the import and the CLI are raft. Python 3.11 or later.

pip install -U raft-ft            # the pipeline: sources, prep, OpenAI finetunes, serve
pip install -U 'raft-ft[hf]'      # + training any huggingface model on a GPU pod via opbdh
pip install -U 'raft-ft[local]'   # + training on this machine's accelerator (mps / cuda)
pip install -U 'raft-ft[pdf]'     # + PDF sources
pip install ariadne-x             # + tweet sources (X / Bluesky)

A project

raft init my-persona
cd my-persona

Inside the directory every command finds the project by itself. raft.json names it; the rest is filled by the pipeline:

path	what
corpus/documents.jsonl	grounding documents: one {title, link, date, content} per line, deduplicated by link
corpus/chunks.jsonl, corpus/chroma/	the chunks and their embeddings (the memory store)
conversations/transcript-NNNN.json	one dated conversation each: participants, date, url, setting, exchanges
conversations/finetune.json, finetune.openai.jsonl	the generated training examples, generic and chat-format
conversations/benchmark.json	a held-out conversation for evaluation, if you add one
metadata/state.json	who the target is, the finetuned models, test questions, source plan
fetch/, blobs/	caches (ariadne's reply reconstruction, downloads)

The pre-2.5 layout, data/{name}* files with the name passed to every command, still works.

Commands

command	does
raft init [dir]	create (or open) a persona project
raft interactive	the guided session in five phases: gather, prep, train, eval, serve; resumable, it suggests the next phase from what is on disk
raft fetch lesswrong --user <handle>	import a LessWrong / EA Forum user; --forum, --min-karma, --no-older-comments; --limit defaults to 200 conversations (0 = all)
raft fetch tweets --user <handle>	a public X account, live; a dotted handle (alice.bsky.social) is Bluesky, --network both merges the two; --archive <export> reads an X export instead (narrowed to --user when given). Without either, the wizard's questions
raft fetch substack --blog <subdomain>	a whole Substack into the corpus
raft fetch rss --url <feed|site>	a blog, RSS or Atom feed into the corpus; --no-full-pages keeps teaser entries as they are
raft fetch url --url <page>	one web page into the corpus
raft fetch pdf --file <path>	PDFs into the corpus (repeat --file)
… --since / --until / --limit	shared by every fetch: only what was written in the window (YYYY-MM-DD, either end optional), and the newest N of it — conversations for lesswrong and tweets, documents for substack and rss. --role auto|corpus|conversation for the two sources that feed both. Every fetch asks for what its flags leave out
raft chunk	cut the corpus into embedding-sized chunks
raft embed	embed the chunks into the store (an upsert; re-run after adding documents)
raft ft:gen [--thinking]	generate the training examples, each reply augmented with what the target had written before that conversation; --thinking puts recall and a reasoning trace in a <think> block; --recheck-traces / --rewrite-traces re-judge or rewrite the traces
raft ft:run --model <id>	train: an OpenAI-finetunable id goes to the OpenAI API, a huggingface org/name to opbdh (a pod, or --target mps|cuda); flags after the model configure the recipe
raft bench:setup	build the benchmark files from the held-out conversation
raft serve	chat with the persona, retrieval-augmented, showing what landed in context
raft comment --source <url|file>	the persona's comment on a post; --web 8090 serves a page that does the same
raft ask --question "..."	ask the memory store a question, no persona

Builtin sources

Two destinations. Conversations teach the persona how it replies: real exchanges, dated, with the target's actual words as the answers. Grounding is everything it wrote that the persona may recall while thinking. A source feeds one or the other; two can feed both at once with a split. Grounding is optional; with conversations alone, prep skips the memory store and trains on the exchanges as they are.

source	feeds	how
LessWrong / EA Forum	both	raft fetch lesswrong --user gwern, or the wizard. Any ForumMagnum forum by URL; the public GraphQL API, no key. For each comment by the target, whatever it replied to is the questioner's side (the post's title, author and opening, or the chain of parent comments) and the comment is the answer; a back-and-forth branch is one multi-turn conversation. Posts and quick takes are grounding; so, by default, are the comments beyond the conversations you asked for, each dated and opening with a line of what it replied to.
X / Bluesky	both (split)	raft fetch tweets --user <handle> (or --archive, or the wizard), needs pip install ariadne-x. An archive export, a CSV/JSON dump, or a public handle; the Community Archive completes reply threads with other people's tweets and needs no key; twitterapi.io is optional (TWITTERAPI_IO_KEY). Bluesky needs only a handle. Replies become conversations, other posts grounding.
Substack	grounding	raft fetch substack or the wizard: the publication's archive, one dated document per post.
Blog / RSS / Atom	grounding	raft fetch rss or the wizard. A feed URL, or the site URL (raft follows its rel=alternate link); teaser-only entries get their full page fetched.
Web page	either	One URL. As grounding (raft fetch url, or the wizard), its text becomes a document; as a conversation (an interview, a transcript), the configured LLM structures it into exchanges.
PDF	grounding	raft fetch pdf or the wizard. Text per file; pip install 'raft-ft[pdf]'.
Local files	either	Text or JSONL. Structured inputs (raft transcripts, chat-message JSON, {title, link, date, content} JSONL) are imported as-is.
Chat logs	conversations	Raw logs, podcast transcripts, message exports: unstructured ones are turned into dated transcripts by the LLM behind RAFT_LLM_MODEL.

Re-adding a source only imports what is new; documents are deduplicated by link. The forum and tweet importers append conversations, so re-run them into a fresh project.

Prep

raft chunk && raft embed
raft ft:gen --thinking

Chunks carry the document's date, and each exchange's recall is limited to writing dated strictly before that conversation: nothing the persona recalls may postdate a reply it is trained on. A recollection must cite a verbatim sentence of the material it comes from, or it is dropped. With --thinking, each training reply opens a <think> block with the recall, continues with a reasoning trace written after the fact from the reply the target actually gave, judged against that reply and rewritten when it does not lead there, and then gives the reply. Without it, the recall is a system note between question and reply, for the OpenAI API and models without a thinking phase. Every exchange the pipeline processes is itself remembered, dated, so later conversations can recall earlier ones.

Train

raft ft:run --model gpt-4o-mini-2024-07-18                       # the OpenAI finetuning API
raft ft:run --model Qwen/Qwen3.8-27B --provider runpod --method qlora --max-spend 6
raft ft:run --model Qwen/Qwen3.8-27B --target mps --epochs 1     # this machine

Huggingface models train through opbdh's native LoRA / QLoRA recipe: on RunPod or Prime Intellect's multi-cloud marketplace (opbdh config wizard once; --gpu-count, --vram-gb, --max-dollars-per-hour, --max-spend), or locally with --target mps|cuda and raft-ft[local]. Recipe flags: --epochs, --learning-rate, --max-length, --batch-size, --gradient-accumulation. --dry-run previews the launch. The adapter (or the OpenAI model id) is recorded in metadata/state.json.

Talk

raft serve                                   # chat; answers on stdout, chrome on stderr
raft comment --source https://www.lesswrong.com/posts/...
raft comment --web 8090                      # paste a URL or text, get recall / thinking / reply

OpenAI finetunes are served directly. A local adapter runs in-process through RAFT_MLX_MODEL (an MLX conversion of the merged model) or through any OpenAI-compatible /v1/completions endpoint (OPENAI_BASE_URL plus --model), with the recall prefilled into the think block exactly as in training. Recall lines the model invents while thinking are flagged, never shown as memories.

Endpoints and knobs

Three roles, three endpoints; unset, each falls back to OpenAI, so OPENAI_API_KEY alone runs the whole pipeline. Point them at ollama, mlx_lm.server or anything OpenAI-compatible and no account is needed end to end.

variable	role
OPENAI_API_KEY, OPENAI_BASE_URL	the persona model (finetunes, serve, comment)
RAFT_LLM_BASE_URL, RAFT_LLM_MODEL, RAFT_LLM_API_KEY	the helper LLM: recall summaries, conversation structuring (default gpt-4o)
RAFT_REASONING_MODEL	a stronger model for the reasoning traces, same endpoint as the helper
RAFT_EMBEDDING_BASE_URL, RAFT_EMBEDDING_MODEL, RAFT_EMBEDDING_API_KEY	embeddings (default text-embedding-ada-002); one model per store
RAFT_MLX_MODEL	an MLX-converted persona for raft comment, in-process
RAFT_MAX_EXAMPLE_TOKENS	how much prior conversation a training example packs (default 8192)
RAFT_WORKERS	parallel requests while generating examples
RAFT_PACE	seconds between requests, for rate-limited endpoints (default 0)

The example: gwern

A persona from one source. gwern's LessWrong account: the 200 newest comment threads as dated conversations, the posts, quick takes and about 11,700 older comments as grounding. No gwern.net scrape; fifteen years of comments give recall something to cite for almost any topic. The script and the full recipe live in examples/gwern.

examples/gwern/bootstrap.sh          # = raft init gwern; raft fetch lesswrong --user gwern --limit 200
cd gwern
export RAFT_EMBEDDING_BASE_URL=http://localhost:11434/v1 RAFT_EMBEDDING_MODEL=nomic-embed-text
export RAFT_LLM_BASE_URL=http://localhost:11434/v1 RAFT_LLM_MODEL=qwen2.5:14b
raft chunk && raft embed             # ~12k chunks
raft ft:gen --thinking               # recall + reasoning in <think>
RAFT_LLM_BASE_URL=... RAFT_REASONING_MODEL=... raft ft:gen --rewrite-traces   # a stronger reasoner
raft ft:run --model Qwen/Qwen3.8-27B --provider runpod --method qlora \
  --gpu-count 2 --vram-gb 94 --gradient-accumulation 4 --epochs 2 \
  --max-length 8192 --max-spend 6 --max-dollars-per-hour 8.2
raft comment --web 8090

What it yields (September 2026): 11.8k grounding documents, 200 conversations, 247 exchanges dated 2021 to 2026; a Qwen3.8-27B QLoRA trained on two H100s in about twelve minutes, under two dollars. A 14B helper writes usable recall but not reasoning traces that survive the judge, hence the rewrite with a stronger model. Qwen3.5/3.8 are hybrid-attention models and need the fused kernels raft ships for them on pods; on Apple Silicon the same run takes hours.