Skip to content

Cryptic News

Higher RAII, and the Seven Arcane Uses of Linear Types

2024-05-14 by Evan Ovadia - (0 min read)

CodeAid: A classroom deployment of an LLM-based programming assistant

Layer-wise inferencing + batching: Small VRAM doesn't limit LLM throughput anymore