blogs
longer technical writeups
Optimizing Layer Normalization Kernel with CUDA
memory coalescing, shared mem, warp shuffling, and vectorized loading
Learning Efficient Data Selection
worklog: can we learn a policy for pretraining data selection
GRPO's Limitations for Reasoning Tasks
a few grpo limitations
Train Qwen3-1.7B to play Wordle with GRPO
training a model to play wordle with grpo on modal for ~$5-6
How to Build a Brain (without losing yours)
intro to neuromorphic computing
stuff im learning, reading, paper dumps, etc.
Optimizer study
Comparing optimizers — order, preconditioners, momentum, EMA, and curvature.
CS336
Notes from Stanford's Language Modeling from Scratch course.
Tensor memory accelerator
Using TMA in the Hopper architecture to load data from global to shared memory.
AdamC
Weight decay tied to the square of the learning rate
some saved links
learning something new everyday
SunMonTueWed
ThuFriSat