https://scale-ml.org/ Recent content on Hugo -- gohugo.io en Wed, 01 Nov 2023 11:30:03 +0000 https://scale-ml.org/posts/critical-batch-size.html Wed, 01 Nov 2023 11:30:03 +0000 https://scale-ml.org/posts/critical-batch-size.html What batch-size should you use for your model? What does the batch-size tell you about your task? This post discusses one main aspects of the scaling law. https://scale-ml.org/posts/moe.html Wed, 04 Oct 2023 11:30:03 +0000 https://scale-ml.org/posts/moe.html MoE are rumored to be a critical components in scaling up to a trillion parameter model. By routing tokens to specialized modular functions, it enables models to have representational power of a much larger model than what is used for prediction. We will be discussing how MoE works and its recent advances in this literature https://scale-ml.org/posts/speculative-decoding.html Wed, 13 Sep 2023 00:00:00 +0000 https://scale-ml.org/posts/speculative-decoding.html A brief overview of speculative decoding, detailing the roots of LLM inference slowdowns and how algorithmic level changes can improve generation speed!