An open-source framework for distributed reinforcement learning from human feedback (RLHF) fine-tuning of large language models.

A 55 billion token dataset of mathematical and scientific documents for training language models focused on mathematical applications.