We make AI faster on less energy.
Kestrel finds the fastest way to run the model you already have, on the hardware you already have. Same quality, in a fraction of the time and energy.
- Faster LLM serving than a default setup
- 6.1×
- Less GPU energy per token
- 86%
- Faster on 36 models it had never seen
- 1.99×
- Faster than the best an engineer can build from public tools
- 25%
Beyond the best that public tools can do.
We set the bar at the fastest setup an inference engineer can build today: the best public engine, quantized checkpoint and draft model for each LLM. Kestrel beat it on five of six LLMs and matched it on the sixth, by up to 25%, at the same quality.
See the resultsLLM serving
Up to 6× the tokens per second from the same GPUs, so the same traffic needs fewer of them.
On devices
On laptops and devices, Kestrel picks the CPU, integrated GPU or NPU for each model, within the energy budget you set.
Your model, tuned to your hardware.
Bring a model
Any model you run today, from a small vision model to an LLM, and the hardware it runs on.
Kestrel searches
It tries engines, precisions, devices and ways of decoding, and times each on your exact hardware. Unattended, in under an hour for an LLM.
Quality holds
A plan ships only if it keeps the original model's quality. You get the fastest one, or the fastest within your energy budget.
Also from llocus: Sequoia.
Our gaze model. Eye tracking that learns each person, on the camera a product already has. Built for the car first.
About SequoiaTry it on your model. Free.
Choose a model you run today. We run Kestrel on it and show you what changes: speed, energy and accuracy.
Start a free pilotKestrel photo: Thibaud Aronson, CC BY-SA 4.0, via iNaturalist, mirrored, sky extended
