# Ares Serious Training Loop

The starter dataset is **not** the real training corpus. It is only a small preview/smoke artifact.

Use this notebook for the serious Ares loop:

```text
colab/OFFICIAL_Ares_Serious_Dataset_Training_Run.ipynb
```

A downloadable copy is also available as:

```text
/home/user/OFFICIAL_Ares_Serious_Dataset_Training_Run.ipynb
```

## Serious default corpus

The serious notebook builds a multi-source corpus in Colab:

| Source | Default records | Purpose |
|---|---:|---|
| `wikimedia/wikipedia`, `20231101.en` | 50,000 | encyclopedic knowledge |
| `HuggingFaceFW/fineweb-edu`, `sample-10BT` | 50,000 | educational web text |
| `open-web-math/open-web-math` | 20,000 | math/scientific text |
| `jqop/python-code-dataset`, Python | 20,000 | code and documentation |
| `HuggingFaceH4/ultrachat_200k` | 20,000 | dialogue/instruction format |
| Ares ML-process curriculum | 20,000 | training/debug/process reasoning |
| Ares roleplay curriculum | 50,000 | roleplay scenarios and conversational behavior |

## Serious default architecture

Default:

```python
PROFILE = "serious_124m_2048"
```

This uses:

```text
configs/ares_124m.json
~100M parameters
2048-token context
32K tokenizer vocabulary
```

If Colab runs out of memory, switch to:

```python
PROFILE = "practical_30m_1024"
```

That keeps the serious dataset but uses a smaller architecture.

## Clear run steps

1. Open Colab.
2. Upload/open `OFFICIAL_Ares_Serious_Dataset_Training_Run.ipynb`.
3. Set `Runtime → Change runtime type → GPU`.
4. Run cell 1 and verify `cuda available: True`.
5. Keep `PROFILE = "serious_124m_2048"` unless it OOMs.
6. Run top-to-bottom.
7. Watch validation loss.
8. Use the final real-generation cells.
9. Do not judge Ares by the starter dataset.
10. Scale data before scaling parameters again.

## Why this fixes the toy-data issue

The serious notebook no longer depends on the 1,000-roleplay starter dataset. Instead it builds a large, mixed corpus from official/public datasets and Ares-generated curriculum. The starter dataset remains in the UI only as a preview download.