2026-08-30 20:41:55.465 | WARNING | acestep.training.trainer::40 - bitsandbytes not installed. Using standard AdamW. ░▒▓███████▓▒░▒▓█▓▒░▒▓███████▓▒░░▒▓████████▓▒░░▒▓███████▓▒░▒▓████████▓▒░▒▓████████▓▒░▒▓███████▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░ ░▒▓██████▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓██████▓▒░ ░▒▓██████▓▒░ ░▒▓█▓▒░ ░▒▓██████▓▒░ ░▒▓███████▓▒░ ░▒▓█▓▒░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓███████▓▒░░▒▓█▓▒░▒▓███████▓▒░░▒▓████████▓▒░▒▓███████▓▒░ ░▒▓█▓▒░ ░▒▓████████▓▒░▒▓█▓▒░ "Talk to an LLM for 10h? Hell yeah." Side-Step v2.0.0 -- Adapter Fine-Tuning CLI (LoRA + LoKR) Standalone: github.com/koda-dernet/Side-Step Mode : fixed (corrected timesteps + CFG dropout) Stack : Python 3.12.14 | PyTorch 2.10.0+cu128 | CUDA 12.8 | bf16 GPU : a 96 GB workstation-class card (95.0 GiB) ============================================================ Training Configuration ============================================================ [Model] Model variant........... turbo Checkpoint dir.......... music/checkpoints * Dataset dir............. music/tensors/mnml-shakedown/20260831T023450Z-mnml0 [Device] Device.................. cuda:0 * Precision............... bf16 * [LoRA] Rank (r)................ 64 Alpha................... 128 Dropout................. 0.1 Target modules.......... q_proj, k_proj, v_proj, o_proj Bias.................... none [Training] Learning rate........... 1.0e-04 Batch size.............. 1 Grad accumulation....... 4 Effective batch......... 4 Max epochs.............. 10 * Warmup steps............ 100 Weight decay............ 0.01 Max grad norm........... 1 Seed.................... 42 [Corrected Training] CFG dropout ratio....... 0.15 Timestep mu............. -0.4 Timestep sigma.......... 1 Data proportion......... 0.5 [Checkpointing] Output dir.............. music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64 * Save every N epochs..... 5 * Resume from............. (auto) [Logging] TensorBoard dir......... (auto) Log every N steps....... 10 Grad norms every N steps 50 ============================================================ (* = non-default value) ============================================================ [INFO] Loading model (variant=turbo, device=cuda:0) 2026-08-30 20:41:55 [INFO] acestep.training_v2.model_loader: [INFO] Loading model from music/checkpoints/acestep-v15-turbo (variant=turbo, dtype=torch.bfloat16) 2026-08-30 20:41:57 [INFO] acestep.training_v2.model_loader: [OK] Model on cuda:0 (torch.bfloat16), all params frozen 2026-08-30 20:41:57.767 | INFO | acestep.training.lora_injection:inject_lora_into_dit:232 - LoRA injected into DiT decoder: 2026-08-30 20:41:57.767 | INFO | acestep.training.lora_injection:inject_lora_into_dit:233 - Total parameters: 2,437,912,710 2026-08-30 20:41:57.767 | INFO | acestep.training.lora_injection:inject_lora_into_dit:234 - Trainable parameters: 44,040,192 (1.81%) 2026-08-30 20:41:57.767 | INFO | acestep.training.lora_injection:inject_lora_into_dit:237 - LoRA rank: 64, alpha: 128 2026-08-30 20:41:57 [INFO] acestep.training_v2.fixed_lora_module: [OK] LoRA injected: 44,040,192 trainable params 2026-08-30 20:41:57.776 | INFO | acestep.training.data_module:__init__:90 - PreprocessedTensorDataset: 159 samples from music/tensors/mnml-shakedown/20260831T023450Z-mnml0 INFO: Using bfloat16 Automatic Mixed Precision (AMP) 2026-08-30 20:41:57 [INFO] lightning.pytorch.utilities.rank_zero: Using bfloat16 Automatic Mixed Precision (AMP) 2026-08-30 20:41:57 [INFO] acestep.training_v2.tensorboard_utils: [OK] TensorBoard logger initialised at music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64/runs 2026-08-30 20:41:57 [INFO] acestep.training_v2.optim: [Side-Step] Using AdamW optimizer 2026-08-30 20:41:57.793 | INFO | acestep.training.lora_injection:_safe_enable_input_require_grads:50 - Skipping enable_input_require_grads for decoder: get_input_embeddings is not implemented (expected for DiT) [INFO] Loading model from music/checkpoints/acestep-v15-turbo (variant=turbo, dtype=torch.bfloat16) [OK] Model loaded with attn_implementation=sdpa [OK] Loaded 159 preprocessed samples [INFO] Starting training (devices: 1, strategy: auto, precision: bf16-mixed) [INFO] Training 44,040,192 parameters [INFO] Optimizer: adamw [INFO] Scheduler: cosine [INFO] Gradient checkpointing enabled (use_cache=False, input_grads=False) 2026-08-30 20:46:59 [INFO] acestep.training_v2.trainer_helpers: [OK] LoRA adapter saved to music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64/checkpoints/epoch_5_loss_0.6603 2026-08-30 20:47:00 [INFO] acestep.training_v2.trainer_helpers: Training checkpoint saved to music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64/checkpoints/epoch_5_loss_0.6603 (epoch 5, step 200) 2026-08-30 20:52:04 [INFO] acestep.training_v2.trainer_helpers: [OK] LoRA adapter saved to music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64/checkpoints/epoch_10_loss_0.6514 2026-08-30 20:52:05 [INFO] acestep.training_v2.trainer_helpers: Training checkpoint saved to music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64/checkpoints/epoch_10_loss_0.6514 (epoch 10, step 400) 2026-08-30 20:52:05 [INFO] acestep.training_v2.trainer_helpers: [OK] LoRA adapter saved to music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64/final 2026-08-30 20:52:05 [INFO] acestep.training_v2.trainer_helpers: [OK] Adapter verified: 44,040,192 params, 44,040,192 non-zero (100.0%), max|w|=0.031738 ============================================================ Training Complete ============================================================ Total time .......... 10m 08s Epochs .............. 0 / 10 Total steps ......... 400 Loss ................ 0.9573 -> 0.3830 Best loss ........... 0.3830 Peak VRAM ........... 5.7 GiB Saved Checkpoints: Epoch 5 Loss: 0.6603 Epoch 10 Loss: 0.6514 Output dir .......... music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64 Final weights ....... music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64/final (84.1 MiB) TensorBoard ......... music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64/runs ============================================================ Epoch 1/10, Step 10, Loss: 0.9573 Epoch 1/10, Step 20, Loss: 0.9776 Epoch 1/10, Step 30, Loss: 0.9322 Epoch 1/10, Step 40, Loss: 0.6496 [OK] Epoch 1/10 in 59.9s, Loss: 0.9872 Epoch 2/10, Step 50, Loss: 0.8436 Epoch 2/10, Step 60, Loss: 0.7810 Epoch 2/10, Step 70, Loss: 0.8334 Epoch 2/10, Step 80, Loss: 0.5864 [OK] Epoch 2/10 in 60.1s, Loss: 0.7319 Epoch 3/10, Step 90, Loss: 0.7168 Epoch 3/10, Step 100, Loss: 0.8150 Epoch 3/10, Step 110, Loss: 0.7467 Epoch 3/10, Step 120, Loss: 0.7195 [OK] Epoch 3/10 in 60.3s, Loss: 0.6880 Epoch 4/10, Step 130, Loss: 0.8578 Epoch 4/10, Step 140, Loss: 0.8541 Epoch 4/10, Step 150, Loss: 0.6495 Epoch 4/10, Step 160, Loss: 0.7541 [OK] Epoch 4/10 in 60.6s, Loss: 0.6614 Epoch 5/10, Step 170, Loss: 0.6447 Epoch 5/10, Step 180, Loss: 0.5692 Epoch 5/10, Step 190, Loss: 0.5954 Epoch 5/10, Step 200, Loss: 0.6142 [OK] Epoch 5/10 in 60.9s, Loss: 0.6603 [OK] Checkpoint saved at epoch 5 Epoch 6/10, Step 210, Loss: 0.6253 Epoch 6/10, Step 220, Loss: 0.7351 Epoch 6/10, Step 230, Loss: 0.5043 Epoch 6/10, Step 240, Loss: 0.4950 [OK] Epoch 6/10 in 60.7s, Loss: 0.6432 Epoch 7/10, Step 250, Loss: 0.7555 Epoch 7/10, Step 260, Loss: 0.8002 Epoch 7/10, Step 270, Loss: 0.5297 Epoch 7/10, Step 280, Loss: 0.4582 [OK] Epoch 7/10 in 60.8s, Loss: 0.6418 Epoch 8/10, Step 290, Loss: 0.5713 Epoch 8/10, Step 300, Loss: 0.6420 Epoch 8/10, Step 310, Loss: 0.6023 Epoch 8/10, Step 320, Loss: 0.5983 [OK] Epoch 8/10 in 61.0s, Loss: 0.6507 Epoch 9/10, Step 330, Loss: 0.5866 Epoch 9/10, Step 340, Loss: 0.5507 Epoch 9/10, Step 350, Loss: 0.7546 Epoch 9/10, Step 360, Loss: 0.8001 [OK] Epoch 9/10 in 60.9s, Loss: 0.6334 Epoch 10/10, Step 370, Loss: 0.6923 Epoch 10/10, Step 380, Loss: 0.5989 Epoch 10/10, Step 390, Loss: 0.5540 Epoch 10/10, Step 400, Loss: 0.6400 [OK] Epoch 10/10 in 60.8s, Loss: 0.6514 [OK] Checkpoint saved at epoch 10 [OK] Training complete! LoRA saved to music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64/final For inference, set your LoRA path to: music/adapters/mnml-shakedown/20260831T023450Z-mnml0-r64/final