Been experimenting with CRMA — a stability layer for LoRA fine-tuning that handles sequential training without catastrophic forgetting. Tested on TinyLlama and Mistral-7B. Full results dropping soon.
Laimleon
2
Thanks for this post.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| CRMA: Drop-in adapter for fine-tuning + continual learning — zero catastrophic forgetting at 7B scale | 0 | 50 | February 27, 2026 | |
| CRMA: Stable Fine-Tuning + Continual Learning for Small LLMs | 0 | 65 | February 27, 2026 | |
| Catastrophic Forgetting of language models | 2 | 33 | February 28, 2026 | |
| Catastrophic Forgetting by Language models | 0 | 59 | February 27, 2026 | |
| CRMA: A low-rank stability adapter for QLoRA fine-tuning — ablation results on TinyLlama, Gemma, Mistral | 0 | 78 | February 22, 2026 |