Exploring self-distilled reasoning for supervised fine-tuning with Amazon Nova

The Amazon Nova 2 model's performance can be enhanced by incorporating self-distilled reasoning (SDR) into supervised fine-tuning (SFT), mitigating catastrophic forgetting and improving target performance. SDR re-uses the chain of thought from the base Amazon Nova 2 Lite model, allowing for the retention of general performance while achieving similar or better target performance as pure SFT. This approach eliminates the need for human annotation and is applicable to existing SFT datasets. The practical implication for engineers building AI systems is that SDR can be used to improve the performance of models in various domains without requiring additional labeled data.

⚡ Key Takeaways

• Self-Distilled Reasoning (SDR) improves target performance by over 6.5 percent compared to model merging.
• SDR mitigates catastrophic forgetting, retaining most of the general performance, with math performance recovering from 6 percent to nearly 70 percent.
• The Amazon Nova 2 Lite model's chain of thought is re-used as a stand-in for non-reasoning datasets in SDR.
• SDR provides in-training regularization by augmenting the dataset with the model's own reasoning traces, requiring no human annotation.
• SDR is more effective than model merging at preserving both target and general performance.

The use of SDR in SFT can significantly enhance the performance of AI models, particularly in domains where reasoning is a key capability, such as coding and math. By mitigating catastrophic forgetting, SDR allows engineers to fine-tune models without compromising their general performance.

✅ Practical Steps

1. Implement Self-Distilled Reasoning (SDR) in supervised fine-tuning (SFT) to improve target performance and mitigate catastrophic forgetting.
2. Use the Amazon Nova 2 Lite model's chain of thought as a stand-in for non-reasoning datasets in SDR.
3. Apply SDR to existing SFT datasets to retain general performance and improve target performance.

Want the full story? Read the original article.