m3rad jobs

Inference Engineer

Adaption
On-siteEngineering
Location
United States
Work type
On-site
Salary
Not listed in the posting
Posted
1 month ago
Source
jobs.ashbyhq.com
Original language
English

From the original posting

San Francisco THE ROLE You'll own the cost and performance of our inference stack. Your work will determine how efficiently we serve models as workloads, traffic, and hardware change. You'll work closely with the engineers operating the serving fleet while owning the core performance levers: caching, batching, quantization, decoding, and kernel-level optimization. Success means improving throughput and latency without compromising reliability or model quality. RESPONSIBILITIES - Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding,…

Excerpt from the original posting (English). Full details are on the employer site.

m3rad Jobs does not hire and does not represent the employer. The original posting is the reference.

Similar jobs