Nomad Watch

Scaling Marin's Open LLM Pretraining

Tech & Startups In person

Wednesday 18 November 2026, 02:00 – 04:00 · San Francisco, United States

Details

Starts
Wed 18 Nov 2026, 02:00
Ends
Wed 18 Nov 2026, 04:00
Type
In person
Category
Tech & Startups
Country
United States

About this event

Marin started as an academic project with one full-time researcher, training medium-sized language models on Google’s TPU Research Cloud. This month, the team finished pre-training a 535B-parameter mixture of experts model on 18T tokens, using GPUs from the Huang Foundation, in the largest live-streamed pre-training run to date.
This required scaling everything: compute, data, focus, and people. We ran thousands of smaller models to understand scaling, built systems for turning trillions of raw tokens into training mixtures, learned how to train sparse models efficiently, built infrastructure…

Hosted by Open Athena & 5 others

Links and contact

Source

Imported from
Luma
Source city
San Francisco
Original event
Open source event https://luma.com/8rz1myft

Similar events

  1. Fintech Run Club ft. Rain + Monk + JP Morgan
    US New York Tech & Startups In person Marsha P. Johnson Memorial Fountain

Continue exploring events

Free views Sign up free