# Training Ares' Brain on Wikipedia, ML Processes, and Roleplay

This repo now includes a dedicated brain-training path for Ares. The goal is to produce an actual Ares checkpoint that generates responses from its own trained weights, not from an external AI API.

## What this adds

- Wikipedia streaming from Hugging Face via `wikimedia/wikipedia`.
- Deterministic machine-learning process curriculum.
- A standalone generated roleplay JSONL dataset with thousands of scenarios.
- Conversion of roleplay JSONL into Ares SFT text.
- Mixed pretraining on Wikipedia + ML processes + roleplay.
- Optional roleplay-focused SFT resumed from the mixed checkpoint.
- A Colab notebook for training and generating with the resulting checkpoint.

## Reality check

The Static Space itself cannot run PyTorch inference or GPU training. Ares will generate from its own brain in Colab or another Python/GPU runtime after training a checkpoint. The public Static Space remains a control deck and documentation interface.

## Recommended Colab notebook

Open:

```text
colab/Ares_Wikipedia_Roleplay_Brain_Training.ipynb
```

Then select:

```text
Runtime → Change runtime type → GPU
```

The notebook now creates:

```text
roleplay_dataset.jsonl
roleplay_sft.txt
corpus_full_wiki_ml_roleplay.txt
corpus_train.txt
corpus_val.txt
ckpt_best.pt / ckpt_last.pt
training_report.html
```

## Corpus mixture command

```bash
python -m ares_core.mixture_build \
  --output artifacts/data/ares_brain_corpus.txt \
  --local data/sample_corpus.txt \
  --wikipedia-records 8000 \
  --wikipedia-config 20231101.simple \
  --ml-records 4000 \
  --roleplay-records 12000
```

Use `20231101.simple` for a Colab smoke run. Use `20231101.en` for a larger English Wikipedia run when you have more time and storage.

## Train/generate command summary

```bash
python -m ares_core.split_corpus \
  --input artifacts/data/ares_brain_corpus.txt \
  --train-output artifacts/data/ares_brain_train.txt \
  --val-output artifacts/data/ares_brain_val.txt

python -m ares_core.tokenizer_train \
  --input artifacts/data/ares_brain_train.txt \
  --output artifacts/tokenizers/ares_brain_tokenizer.json \
  --vocab-size 8192

python -m ares_core.train \
  --config configs/ares_30m.json \
  --tokenizer artifacts/tokenizers/ares_brain_tokenizer.json \
  --train artifacts/data/ares_brain_train.txt \
  --val artifacts/data/ares_brain_val.txt \
  --out artifacts/checkpoints/ares-brain-30m \
  --steps 1000 \
  --batch-size 4 \
  --grad-accum 2 \
  --eval-every 100 \
  --device auto

# Optional roleplay-focused SFT from the mixed checkpoint
python -m ares_core.train \
  --config configs/ares_30m.json \
  --tokenizer artifacts/tokenizers/ares_brain_tokenizer.json \
  --train artifacts/data/roleplay_sft_train.txt \
  --val artifacts/data/roleplay_sft_val.txt \
  --out artifacts/checkpoints/ares-brain-30m-roleplay-sft \
  --resume artifacts/checkpoints/ares-brain-30m/ckpt_best.pt \
  --resume-model-only \
  --reset-step-on-resume \
  --steps 300 \
  --device auto

python -m ares_core.agent_chat \
  --checkpoint artifacts/checkpoints/ares-brain-30m-roleplay-sft/ckpt_best.pt \
  --tokenizer artifacts/tokenizers/ares_brain_tokenizer.json \
  --plan \
  --prompt "Roleplay as a machine-learning tutor and explain KV caches."
```

## Scaling guidance

- Start with 10M or 30M parameters.
- Keep validation loss visible.
- Increase Wikipedia and roleplay records before increasing model size.
- A 1B model is a distributed-training target, not a free-Colab target.