CS336: Language Modeling from Scratch

133 points - today at 2:10 PM

Source

Comments

skerit today at 4:07 PM
> GPU compute for self-study

Those suggestions they make for a B200 start at $4.99 an hour.

Is that really required, for starting out? I've been tinkering with my own from-scratch LLM, but in the early phases I don't need anything more than a 4090 on Vast.ai

meken today at 3:38 PM
I have fond memories of cs224d [1] taught by richardsocher. It’s a bit dated now as it was created in the pre-transformer era, but it was very cool introduction to applying deep learning to nlp at the time.

[1] https://cs224d.stanford.edu

airstrike today at 4:25 PM
I wonder if people prefer to learn this on their own or if building a community around open learning is something that others are interested in
storus today at 3:24 PM
Thanks for releasing this again! What are this year's changes to prior offerings?
tmule today at 3:39 PM
Are video lectures available online?