Software Development

Kermt Add cMIM Pretrain

The skill transforms a grover_base checkpoint, whether encoder-only or encoder with vocab heads, into a hybrid checkpoint by appending a randomly initialized cMIM decoder and latent distribution. It then continues pretraining on the user’s corpus using a hybrid objective that combines vocabulary and contrastive learning. This enables data scientists and NLP engineers to adapt existing models to new domains efficiently, with minimal conversion overhead.

Publisher / source

NVIDIA Skills (Verified)

Upstream record

NVIDIA/skills

Version

Version 2

Published

2026-09-15

Publication review

Approved for publication

Content fingerprint

eed773f29585c8b8ad9386831dcce9659d864c4c690320d812d4324e51fab6e2