writing

I built a decoder-only transformer from scratch this week — architecture, training loop, sampling, and a KV cache for fast generation. Code's on GitHub: paper2code/transformerhttps://github.com/rkj26/paper2code/tree/master/transformer. These are my notes on it, written the way I wish someone had exp...

In reinforcement learning, policy gradient methods directly parameterize the policy and optimize it to maximize the expected cumulative reward . While mathematically elegant, policy gradient methods are notoriously sensitive to hyperparameters, prone to high-variance gradient estimates, and vulnera...

A few weeks ago I started working through Stanford's CS234 course on Reinforcement Learning. I have been meaning to get into RL properly for a while and decided that working through a real course with real problem sets was the best way to do it. This post covers my solutions to Problems 1 and 2 from...

I have been working through Stanford's CS234 course on Reinforcement Learning, taught by Professor Emma Brunskill, as part of building a solid theoretical foundation in RL. These are my notes from the first two lectures. Lecture 1 covers the framing of RL and builds up to Markov Reward Processes. Le...

I've been trying to get more serious about reinforcement learning, not just the conceptual side but actually being able to implement things from papers. One thing I kept running into is the gap between reading an equation in a paper and knowing what to do with it in code. You see a summation with so...