K
Research Engineer - LLM Architecture
Kog
On-siteEngineering
From the original posting
Paris, France ABOUT KOG Kog builds a co-designed inference stack for real-time AI agents on standard NVIDIA and AMD datacenter GPUs. We develop model architectures and inference systems together. Our work includes Laneformer 2B and Delayed Tensor Parallelism (DTP), a Transformer architecture that overlaps communication with useful computation and weight streaming. Our inference stack generates 3,500 tokens/s per request on 8 AMD MI300X GPUs and 2,100 tokens/s per request on 8 NVIDIA H200 GPUs, in FP16 at batch size 1, without quantization or speculative decoding. Our next challenge is to…
Excerpt from the original posting (English). Full details are on the employer site.m3rad Jobs does not hire and does not represent the employer. The original posting is the reference.