[go: up one dir, main page]

Efficient AI Computing,
Transforming the Future.

Blog Posts

Bet Small to Win Big: Efficient 2K Video Generation via Deeper-compression AutoEncoder, Linear Attention and Two-Stage Refiner

Feb 11, 2026

This blog details SANA-Video transition to a Two-Stage Inference Paradigm. By leveraging a high-compression base model for structural discovery (Stage 1) and a step-distilled refiner for high-frequency detail injection (Stage 2), we achieve 2K resolution output without compromising the latency profile of the original 720p generation.

Read Article

Infinite Context Length with Global but Constant Attention Memory

Nov 24, 2025

By reducing complexity from O(N^2) to O(N), Linear Attention is the key to processing ultra-long sequences. This post explores its mathematical core—"state accumulation"—and how it unlocks infinite context for LLMs and long video generation.

Read Article

Why Stacking Sliding Windows Can't See Very Far

Aug 25, 2025

A mathematical explanation of why sliding window attention's effective receptive field is O(W) rather than the theoretical O(LW), regardless of depth, due to information dilution and exponential decay from residual connections.

Read Article

Statistics behind Block Sparse Attention

Aug 22, 2025

A statistical model revealing how block sparse attention achieves efficiency and accuracy through learned similarity gaps.

Read Article

Latest Posts

Bet Small to Win Big: Efficient 2K Video Generation via Deeper-compression AutoEncoder, Linear Attention and Two-Stage Refiner
Infinite Context Length with Global but Constant Attention Memory
Why Stacking Sliding Windows Can't See Very Far
Statistics behind Block Sparse Attention
How Attention Sinks Keep Language Models Stable

Categories

Efficient AI Algorithm
Efficient AI Hardware & System

Topics

Large Language Models (LLMs)
Generative AI
TinyML
Autonomous Driving
Training
Quantum Computing

Techniques

Pruning & Sparsity
Data Augmentation
LLM Post Training
Quantization
Neural Architecture Search (NAS)
Distillation
New Architecture
ML for Hardware & System
USEFUL LINKS
Homepage of Prof. Song HanHomepage of MIT HAN LabMIT Accessibility
Copyright © MIT Han Lab.
Designed and Developed by Yujun Lin.