[go: up one dir, main page]

Forem

# vllm

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
From one model to seven — what it took to make TurboQuant model-portable
albertocodes profile

From one model to seven — what it took to make TurboQuant model-portable

#python #vllm #gpu #triton
Comments
3 min read
Compressed VLM inference from a single Containerfile — turboquant-vllm v1.1
albertocodes profile

Compressed VLM inference from a single Containerfile — turboquant-vllm v1.1

#python #vllm #gpu #containers
1
Comments
2 min read
vLLM On-Demand Gateway: Zero-VRAM Standby for Local LLMs on Consumer GPUs
soytuber profile
soy

vLLM On-Demand Gateway: Zero-VRAM Standby for Local LLMs on Consumer GPUs

#vllm #llm #gpu #python
1
Comments
4 min read
vLLM Request Lifecycle (Where TTFT is measured)
pilgrim2go profile

vLLM Request Lifecycle (Where TTFT is measured)

#vllm #monitoring
Comments
2 min read
Gemma-SRE: Self-Hosted vLLM Infrastructure Agent
xbill profile

Gemma-SRE: Self-Hosted vLLM Infrastructure Agent

#gemma #mcpserver #tpusprint #vllm
1
Comments
18 min read
I Pushed Local LLMs Harder. Here's What Two Models Actually Did.
dcruver profile

I Pushed Local LLMs Harder. Here's What Two Models Actually Did.

#claudecode #vllm #selfhosted #amd
1
Comments
8 min read
The Ghost in the Batch: How vLLM Silently Switches Algorithms
mketkar profile

The Ghost in the Batch: How vLLM Silently Switches Algorithms

#vllm #machinelearning #gpu #determinism
Comments
5 min read
Compiling the Vision Encoder: Squeezing 3% More Throughput from Qwen3-VL on Hopper GPUs
mketkar profile

Compiling the Vision Encoder: Squeezing 3% More Throughput from Qwen3-VL on Hopper GPUs

#vllm #pytorch #gpu #machinelearning
Comments
11 min read
Session 1: vLLM Overview and the User API
c2sea profile
Ben

Session 1: vLLM Overview and the User API

#vllm #llm #python #machinelearning
Comments
12 min read
vLLM — Session 2: The Engine Layer — Request Management
c2sea profile
Ben

vLLM — Session 2: The Engine Layer — Request Management

#vllm #llm #python #machinelearning
Comments
13 min read
Pare de Brincar com LLMs Locais: Leve a IAG Open Source para a Produção na Magalu Cloud
Cover image for Pare de Brincar com LLMs Locais: Leve a IAG Open Source para a Produção na Magalu Cloud
glaucioripol profile

Pare de Brincar com LLMs Locais: Leve a IAG Open Source para a Produção na Magalu Cloud

#ai #llm #vllm #docker
1
Comments 3
22 min read
Running Claude Code with Local LLMs via vLLM and LiteLLM
dcruver profile

Running Claude Code with Local LLMs via vLLM and LiteLLM

#claudecode #vllm #selfhosted #ai
2
Comments
6 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.