Upload sft_train.log with huggingface_hub
Browse files- sft_train.log +82 -0
sft_train.log
ADDED
|
@@ -0,0 +1,82 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
============================================================
|
| 2 |
+
PyCraft-1 Post-Training: SFT + Preference Alignment
|
| 3 |
+
============================================================
|
| 4 |
+
Base : checkpoints/step_0004000
|
| 5 |
+
Device : cuda
|
| 6 |
+
Dropout : disabled throughout fine-tuning
|
| 7 |
+
Precision: float32 (no BF16)
|
| 8 |
+
Params : 55.3M
|
| 9 |
+
|
| 10 |
+
Verifying base model loss (should be ~1.2-2.0)...
|
| 11 |
+
Base model loss on Python code: 0.4318
|
| 12 |
+
Base model verified OK.
|
| 13 |
+
|
| 14 |
+
============================================================
|
| 15 |
+
STAGE 1 — Supervised Fine-Tuning (SFT)
|
| 16 |
+
============================================================
|
| 17 |
+
Examples : 40,000
|
| 18 |
+
Batch size : 4 x 16 = 64 effective
|
| 19 |
+
LR : 0.0001
|
| 20 |
+
Steps : 400
|
| 21 |
+
Dropout : disabled (0.0)
|
| 22 |
+
Precision : float32 (no BF16 for stability)
|
| 23 |
+
|
| 24 |
+
Step 10 sanity check: loss=1.3601 -- OK
|
| 25 |
+
sft step 10 | loss 1.3601 | ppl 3.90 | lr 1.00e-05 | grad 2.335
|
| 26 |
+
sft step 20 | loss 1.3061 | ppl 3.69 | lr 2.00e-05 | grad 2.274
|
| 27 |
+
sft step 30 | loss 1.3067 | ppl 3.69 | lr 3.00e-05 | grad 2.229
|
| 28 |
+
sft step 40 | loss 1.2514 | ppl 3.50 | lr 4.00e-05 | grad 3.011
|
| 29 |
+
sft step 50 | loss 1.2427 | ppl 3.46 | lr 5.00e-05 | grad 2.227
|
| 30 |
+
sft step 60 | loss 1.2233 | ppl 3.40 | lr 6.00e-05 | grad 2.230
|
| 31 |
+
sft step 70 | loss 1.2386 | ppl 3.45 | lr 7.00e-05 | grad 2.269
|
| 32 |
+
sft step 80 | loss 1.2378 | ppl 3.45 | lr 8.00e-05 | grad 2.297
|
| 33 |
+
sft step 90 | loss 1.2648 | ppl 3.54 | lr 9.00e-05 | grad 2.298
|
| 34 |
+
sft step 100 | loss 1.2663 | ppl 3.55 | lr 1.00e-04 | grad 2.281
|
| 35 |
+
sft step 110 | loss 1.2966 | ppl 3.66 | lr 9.98e-05 | grad 2.227
|
| 36 |
+
sft step 120 | loss 1.2263 | ppl 3.41 | lr 9.90e-05 | grad 2.286
|
| 37 |
+
sft step 130 | loss 1.2827 | ppl 3.61 | lr 9.78e-05 | grad 2.310
|
| 38 |
+
sft step 140 | loss 1.2585 | ppl 3.52 | lr 9.61e-05 | grad 2.223
|
| 39 |
+
sft step 150 | loss 1.2534 | ppl 3.50 | lr 9.40e-05 | grad 2.070
|
| 40 |
+
sft step 160 | loss 1.2354 | ppl 3.44 | lr 9.14e-05 | grad 2.049
|
| 41 |
+
sft step 170 | loss 1.2277 | ppl 3.41 | lr 8.84e-05 | grad 2.076
|
| 42 |
+
sft step 180 | loss 1.2714 | ppl 3.57 | lr 8.51e-05 | grad 2.075
|
| 43 |
+
sft step 190 | loss 1.2685 | ppl 3.56 | lr 8.15e-05 | grad 2.257
|
| 44 |
+
sft step 200 | loss 1.2621 | ppl 3.53 | lr 7.75e-05 | grad 2.199
|
| 45 |
+
SFT checkpoint saved at step 200
|
| 46 |
+
sft step 210 | loss 1.2853 | ppl 3.62 | lr 7.33e-05 | grad 2.122
|
| 47 |
+
sft step 220 | loss 1.2623 | ppl 3.53 | lr 6.89e-05 | grad 2.047
|
| 48 |
+
sft step 230 | loss 1.2691 | ppl 3.56 | lr 6.44e-05 | grad 1.974
|
| 49 |
+
sft step 240 | loss 1.2495 | ppl 3.49 | lr 5.97e-05 | grad 2.051
|
| 50 |
+
sft step 250 | loss 1.2426 | ppl 3.46 | lr 5.50e-05 | grad 1.973
|
| 51 |
+
sft step 260 | loss 1.2013 | ppl 3.32 | lr 5.03e-05 | grad 1.970
|
| 52 |
+
sft step 270 | loss 1.2188 | ppl 3.38 | lr 4.56e-05 | grad 1.927
|
| 53 |
+
sft step 280 | loss 1.2244 | ppl 3.40 | lr 4.11e-05 | grad 2.000
|
| 54 |
+
sft step 290 | loss 1.1766 | ppl 3.24 | lr 3.67e-05 | grad 2.091
|
| 55 |
+
sft step 300 | loss 1.2086 | ppl 3.35 | lr 3.25e-05 | grad 2.031
|
| 56 |
+
sft step 310 | loss 1.2127 | ppl 3.36 | lr 2.85e-05 | grad 2.041
|
| 57 |
+
sft step 320 | loss 1.1476 | ppl 3.15 | lr 2.49e-05 | grad 1.968
|
| 58 |
+
sft step 330 | loss 1.1834 | ppl 3.27 | lr 2.16e-05 | grad 2.018
|
| 59 |
+
sft step 340 | loss 1.1942 | ppl 3.30 | lr 1.86e-05 | grad 2.044
|
| 60 |
+
sft step 350 | loss 1.1894 | ppl 3.29 | lr 1.60e-05 | grad 1.863
|
| 61 |
+
sft step 360 | loss 1.2079 | ppl 3.35 | lr 1.39e-05 | grad 1.992
|
| 62 |
+
sft step 370 | loss 1.2100 | ppl 3.35 | lr 1.22e-05 | grad 1.958
|
| 63 |
+
sft step 380 | loss 1.1824 | ppl 3.26 | lr 1.10e-05 | grad 2.060
|
| 64 |
+
sft step 390 | loss 1.1853 | ppl 3.27 | lr 1.02e-05 | grad 2.065
|
| 65 |
+
sft step 400 | loss 1.1465 | ppl 3.15 | lr 1.00e-05 | grad 1.905
|
| 66 |
+
SFT checkpoint saved at step 400
|
| 67 |
+
|
| 68 |
+
SFT complete. Best loss: 1.1465
|
| 69 |
+
|
| 70 |
+
============================================================
|
| 71 |
+
STAGE 2 — Preference Alignment (ORPO-inspired margin loss)
|
| 72 |
+
============================================================
|
| 73 |
+
Novel contribution: lightweight preference alignment
|
| 74 |
+
on a 55M from-scratch model, no reference model needed.
|
| 75 |
+
|
| 76 |
+
Pairs : 20,000
|
| 77 |
+
Margin : 0.5
|
| 78 |
+
LR : 2e-05
|
| 79 |
+
Steps : 200
|
| 80 |
+
|
| 81 |
+
pref step 10 | sft 10.7388 | pref 3.0770 | gap +2.308 | lr 4.00e-06 | grad 1.802
|
| 82 |
+
pref step 20 | sft 10.2856 | pref 2.9803 | gap +2.441 | lr 8.00e-06 | grad 1.824
|