← Back
Hugging Face Blog

The Definitive Fine-Tuning Guide for 3D Generation Models

•14 min read•
#deployment
Level:Intermediate
For:AI Engineers
✦TL;DR

This guide releases fine‑tuning scripts and container images for adapting the TRELLIS.2 3‑D generation model to custom assets, using LoRA adapters via Hugging Face’s PEFT library. In experiments on a 188‑item footwear subset of the Google Scanned Objects (GSO) dataset, LoRA improves sole geometry that the pretrained model gets wrong. Engineers can run the provided Docker‑based workflow locally or on Hugging Face Jobs to fine‑tune only the structure and geometry components while freezing the rest of the pipeline.

⚡ Key Takeaways

  • Fine‑tuning was evaluated on 188 footwear objects from the GSO dataset.
  • LoRA adapters are inserted into the structure and detailed‑geometry stages of TRELLIS.2, leaving background removal, encoding, and material generation frozen.
  • Using PEFT’s low‑rank adapters reduces trainable parameters and GPU memory compared with full‑model fine‑tuning.
  • The workflow integrates BiRefNet for background removal and DINOv3‑L for image encoding, both required to be accessible via a HF_TOKEN.
  • The pretrained DINOv3 model is gated; access must be granted to the Hugging Face account before training.
💡 Why It Matters

By targeting only the most task‑relevant components, engineers can quickly specialize a powerful 3‑D generator for niche domains (e.g., accurate furniture or footwear geometry) without the cost of full model retraining.

✅ Practical Steps

  1. Upload your mesh dataset to the Hugging Face Hub, then run the provided fine‑tuning script with your HF_TOKEN and the dataset identifier as arguments.
  2. Choose either the local Docker command or a Hugging Face Job to launch the containerized training, ensuring DINOv3‑L access is enabled.
  3. After training, replace the original TRELLIS.2 components with the saved LoRA adapter weights for inference.

Want the full story? Read the original article.

Read on Hugging Face Blog ↗

More like this

Introducing workload recovery and expanded infrastructure checks on Amazon SageMaker HyperPod

AWS ML Blog•#deployment

Introducing Unity Gateway Insights

Databricks Blog•#deployment

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING