Scaling Marin's Open LLM Pretraining
Tech & Startups
In person
Wednesday 18 November 2026, 02:00 – 04:00 · San Francisco, United States
Details
- Starts
- Wed 18 Nov 2026, 02:00
- Ends
- Wed 18 Nov 2026, 04:00
- Type
- In person
- Category
- Tech & Startups
- City
- San Francisco
- Country
- United States
About this event
Marin started as an academic project with one full-time researcher, training medium-sized language models on Google’s TPU Research Cloud. This month, the team finished pre-training a 535B-parameter mixture of experts model on 18T tokens, using GPUs from the Huang Foundation, in the largest live-streamed pre-training run to date.
This required scaling everything: compute, data, focus, and people. We ran thousands of smaller models to understand scaling, built systems for turning trillions of raw tokens into training mixtures, learned how to train sparse models efficiently, built infrastructure…
Hosted by Open Athena & 5 others
This required scaling everything: compute, data, focus, and people. We ran thousands of smaller models to understand scaling, built systems for turning trillions of raw tokens into training mixtures, learned how to train sparse models efficiently, built infrastructure…
Hosted by Open Athena & 5 others
Links and contact
- Website
- https://luma.com/8rz1myft
- Registration
- https://luma.com/8rz1myft
Source
- Imported from
- Luma
- Source city
- San Francisco
- Original event
- Open source event https://luma.com/8rz1myft