Ares Static Lab

Approx tokens: 0/2048

Trainable Ares is here: this Static Space now ships the full Colab brain notebook, Ares source code, model configs, tokenizer trainer, Wikipedia/ML/roleplay dataset builder, validation-aware trainer, and generation scripts. Training and inference run in Colab because Static Spaces cannot run PyTorch GPUs.

Brain notebook

Runs the complete path: Wikipedia streaming → roleplay JSONL → tokenizer → Ares training → optional roleplay SFT → generation.

Actual Ares responses

After Colab training, ares_core.generate and ares_core.agent_chat sample from Ares checkpoint weights.

Model configs

Start with ~10M or ~30M parameters, then scale only after validation loss and generation quality improve.

Safety boundary

The UI does not hide an external AI model. The brain comes from your trained checkpoint.

Official training files

No-terminal train + connectClick-only Colab notebook: train Ares, start API, and open the UI connected to the checkpoint. Serious dataset notebookRecommended: Wikipedia EN, FineWeb-Edu, OpenWebMath, code, UltraChat, 50k roleplay, 100M/2048 profile. Serious training guideExplains the non-toy dataset mixture and run steps. Official real-generation notebookSmaller default run: Wikipedia, TinyStories, 12k roleplay, tokenizer, training, SFT, and real checkpoint generation. Clear Colab instructionsStep-by-step guide for the smaller official Ares training loop. Earlier brain notebookPrevious Colab notebook for Wikipedia + roleplay training. Brain training guideReadable command guide and scaling notes. 30M configSmall trainable Ares brain target for Colab. Trained seed 1M1.05M Ares checkpoint trained from scratch to 5,000 steps. Trained seed 6M6.23M increased-parameter checkpoint; best val loss ~3.918. 1M seed notesTraining summary and limitations. 6M seed notesParameter increase and validation comparison. Trained seed 30M28.3M checkpoint; needs more data/GPU training to beat 6M. 30M seed notesOfficial 30M architecture and validation comparison. Ares model codeDecoder-only Transformer implementation.

Colab bootstrap

!git clone https://huggingface.co/spaces/jacmor64/ares-static-lab /content/ares-static-space
%cd /content/ares-static-space
!pip -q install -U tokenizers datasets safetensors huggingface_hub tqdm numpy
# Then open/run: colab/OFFICIAL_Ares_Colab_Training_Run.ipynb
# Real generation happens in notebook cells 20 and 22 after checkpoint training.

Generate with trained brain

python -m ares_core.agent_chat \
  --checkpoint /content/drive/MyDrive/ares_brain_artifacts/checkpoints/ares-complete-wiki-roleplay-30m-roleplay-sft/ckpt_best.pt \
  --tokenizer /content/drive/MyDrive/ares_brain_artifacts/tokenizers/ares-complete-wiki-roleplay-30m_tokenizer.json \
  --rag-db /content/drive/MyDrive/ares_brain_artifacts/rag/ares-complete-wiki-roleplay-30m.sqlite \
  --plan \
  --interactive \
  --device auto

Starter data is preview-only: do not use the 1,000-roleplay / 500-ML files as the real Ares corpus. For serious training, use the Serious Dataset notebook, which streams Wikipedia EN, FineWeb-Edu, OpenWebMath, code, dialogue data, and generates 50,000 roleplay scenarios by default.

Roleplay starter JSONL1,000 chat scenarios for Ares roleplay/SFT format. ML-process starter text500 records about tokenizers, training, validation, AdamW, RAG, and debugging. Dataset cardExplains dataset contents and full Colab generation. HF dataset repoOfficial public starter dataset on Hugging Face. Generate serious corpusRecommended: multiple official datasets + 50k roleplay + 100M/2048 profile.

Dataset mixer

Wikipedia records 50000

ML-process records 20000

Roleplay records 50000

Wikipedia source Simple

Dataset preview

roleplay

Machine-learning tutor on a lunar base: explain KV caches while staying calm, honest, and practical.

Wikipedia

Article records: title + cleaned text are streamed from wikimedia/wikipedia in Colab.

ML process

Training troubleshooting: loss spikes, tokenizer mismatch, validation overfitting, GPU OOM, RAG grounding, and checkpointing.

Pipeline status: the repository now includes dataset building, train/validation split, BPE tokenizer training, decoder-only Transformer training, validation loss/perplexity, checkpointing, HTML reports, SQLite RAG, and a RAG/planning chat wrapper.

1. Corpus builder

Streams bounded public data, filters short/long examples, deduplicates, and writes an Ares corpus.

2. Tokenizer

Trains byte-level BPE from scratch with Ares special tokens.

3. Transformer

RoPE, RMSNorm, GQA attention, KV cache, SwiGLU, unembedding, AdamW.

4. Evaluation

Validation loss, perplexity, best checkpoint, and offline HTML report.

Core training command

python -m ares_core.train \
  --config configs/ares_8m.json \
  --tokenizer artifacts/tokenizers/ares_tokenizer.json \
  --train artifacts/data/corpus_train.txt \
  --val artifacts/data/corpus_val.txt \
  --out artifacts/checkpoints/ares-10m \
  --steps 300 \
  --batch-size 8 \
  --eval-every 50 \
  --device auto

Generate report

python -m ares_core.train_report \
  --log artifacts/checkpoints/ares-10m/train_log.jsonl \
  --out artifacts/checkpoints/ares-10m/training_report.html

Public safety: keep Hugging Face tokens, checkpoints, SQLite databases, Google Drive private links, and raw datasets out of the Static Space repo. Colab artifacts should live in Google Drive or a separate model repo.

Step A: Open notebook

Use colab/Ares_Wikipedia_Roleplay_Brain_Training.ipynb for the official trainable Ares brain path.

Step B: Enable GPU

Runtime → Change runtime type → GPU. Free Colab is for smoke tests, not 1B training.

Step C: Run top-to-bottom

The notebook creates Drive artifacts, trains, evaluates, and generates a report.

Step D: Scale carefully

Increase data and steps only after validation loss behaves correctly.

Artifact rule

Do not commit artifacts/, checkpoints/, *.pt, *.safetensors, *.sqlite, .env, kaggle.json, or access tokens to the public Space.