How ONESTRUCTION built the Ishigaki-IDS foundation model with AWS GenAIIC

ONESTRUCTION built the Ishigaki-IDS foundation model with technical advisory from AWS GenAIIC, addressing data scarcity and specialized knowledge requirements in the construction industry. The model was trained using a three-stage pipeline (CPT, SFT, RLVR) and synthetic data generation to overcome data scarcity. The model's performance was improved by injecting an IFC vocabulary and using verifiable rewards for structured output generation. This approach has practical implications for engineers building AI systems in data-scarce domains, enabling them to develop specialized models with limited training data.

⚡ Key Takeaways

• The Ishigaki-IDS foundation model was built using a three-stage training pipeline (CPT, SFT, RLVR) to overcome data scarcity in the construction industry.
• Synthetic data generation was used to supplement limited training data, allowing the model to learn from a larger dataset.
• The model was trained on Amazon Elastic Compute Cloud (Amazon EC2) P5en instances with AWS ParallelCluster for distributed training.
• Verifiable rewards were used for structured output generation, enabling the model to produce high-quality output.
• The model's performance was improved by injecting an IFC vocabulary of several thousand terms, allowing it to learn domain-specific terminology.

The development of the Ishigaki-IDS foundation model demonstrates the potential for AI to address industry-specific challenges in data-scarce domains. By leveraging synthetic data generation, domain adaptation, and specialized training pipelines, engineers can build high-performing models with limited training data, enabling them to drive innovation and improvement in their respective industries.

✅ Practical Steps

1. Use synthetic data generation to supplement limited training data in data-scarce domains.
2. Implement a three-stage training pipeline (CPT, SFT, RLVR) for domain specialization.
3. Utilize verifiable rewards for structured output generation to improve model performance.
4. Leverage distributed training on Amazon Elastic Compute Cloud (Amazon EC2) P5en instances with AWS ParallelCluster to scale model training.

Want the full story? Read the original article.