Fundamentals on Sandeep Kumarhttps://sandeep007734.github.io/tags/fundamentals/Recent content in Fundamentals on Sandeep KumarHugoen-usWed, 12 Aug 2026 00:00:00 +0530Activations, Normalization and Optimizers, in Plain Termshttps://sandeep007734.github.io/posts/training_building_blocks/Wed, 12 Aug 2026 00:00:00 +0530https://sandeep007734.github.io/posts/training_building_blocks/A plain-language guide to the standard parts of a modern transformer and its training loop: ReLU, GELU, SiLU and SwiGLU, LayerNorm and RMSNorm, and the road from SGD through momentum and Adam to AdamW.