# Ares Seed 30M Model

A 30M-class Ares checkpoint has been trained from scratch as the next parameter increase after the 1M and 6M seeds.

Model repo:

```text
https://huggingface.co/jacmor64/ares-seed-30m
```

## What this model is

- About 28.3M parameters.
- Official `configs/ares_30m.json` architecture.
- 512-token context window.
- 8192-token BPE tokenizer trained from scratch.
- Trained from scratch for 1,500 CPU steps.
- Uses a small seed mixture: Simple Wikipedia, ML-process curriculum, and roleplay curriculum.

## Result

Best validation loss:

```text
4.634 at step 1249
```

Comparison:

```text
Ares Seed 1M:  ~1.05M params, best val loss ~4.422
Ares Seed 6M:  ~6.23M params, best val loss ~3.918
Ares Seed 30M: ~28.3M params, best val loss ~4.634
```

The 30M model has more capacity, but this small seed corpus and short CPU run were not enough for it to beat the 6M model. It needs the full Colab/GPU training notebook with more Wikipedia and roleplay data.

## Next step

Run:

```text
colab/Ares_Wikipedia_Roleplay_Brain_Training.ipynb
```

with larger settings:

```text
WIKIPEDIA_RECORDS = 8000+
ML_PROCESS_RECORDS = 4000+
ROLEPLAY_RECORDS = 12000+
CONFIG_PATH = configs/ares_30m.json
```