How many more GPUs fit on the feed you already have.
Every card on a node is provisioned at its factory limit, 700 W for an H100, whether it is pinned or sitting idle with a model loaded. The cards never all peak at once, so the feed is sized for a moment that rarely happens. DMS Core measures the real coincident peak at the card, ten times a second, and turns 48 hours of it into two numbers: how many more cards the same feed can carry, and what a cap would cost in capped seconds on your actual jobs.
It is one Python file run as an ordinary user, and nothing leaves the node. Every figure on this page came from a real run in read-only mode, and the card it came from is named.
Forty-eight hours on one node, nothing touched.
You run it on one host you choose, and we never log in. The daemon reads NVML, writes to a folder on local disk, and makes no network calls. When the window closes you send the folder and the headroom page comes back within three days with your figures on it. If you already keep a day of nvidia-smi output, the same page can be produced from that without installing anything.
What you do
Pick one node. Put the file in a folder, run one command as an ordinary user, and leave it for 48 hours. Then send the folder. The audit costs nothing.
What comes back
The node's coincident peak against its nameplate, with a timestamp. A table of caps from 90 down to 50 percent of the factory limit, each with the share of the window it would have bound and the energy above it, per card and for the node. The count of extra cards the same feed would carry at each cap. Seconds the daemon did not observe are counted separately, never as zero watts. If every card ran pinned the whole time, the page says so.
Read the sample headroom page from our own H100 run to see what comes back, or write to liam@dynamicmarginsystems.com.
Request the auditIt measures first. Nothing is enforced until you turn it on.
One userspace process per host, with no kernel module, no reboot, and no change to tenant code. Shadow mode is the default and the daemon enforces it on itself: enforcement cannot be switched on without root, and if that is attempted the daemon forces itself back to shadow. Every decision it would have made is logged before it is ever trusted to make one.
It reads the silicon directly
Every hundred milliseconds it asks NVML, the same interface nvidia-smi uses, for the instantaneous power register, utilization, clocks, throttle reasons and junction temperature on each device, keyed by UUID and timestamped.
It counts its own gaps
A health file tracks execution gaps and stale frames. Seconds the daemon did not see are reported as unobserved. The eleven-day run we publish had 31 gaps in 267.9 hours, and we would rather you see that than a log that claims coverage it did not have.
It works from a log you already have
The headroom engine runs over the daemon's own log or over plain nvidia-smi CSV output. The sample page on this site was produced by replaying a July log through the same code that runs live.
It labels every number
Each reading carries a note saying whether it was measured from the register or derived from measured values. You always know which kind you are looking at.
We found a serious bug in our own gating logic, and we published it.
Most vendors tell you their software works. We would rather show you what a real card surfaced, including in our own code.
The H100 run
Shadow mode, 10,848 samples at ten per second, on an H100 SXM 80GB with driver 570.211.01. NVML reported under 5% utilization while the card drew up to 529 W. An idle H100 holding a live context draws 124 W. Our first gate keyed on utilization, which means it would have clock-gated a working GPU on 4,024 of those 10,848 samples. Thirty-seven percent.
We rebuilt the gate to require both signals to read idle, with power able to veto utilization and never the reverse. Re-run on the same card, false gates went from 4,024 to zero while 23.4% of samples were still caught as idle. Gated cards drew between 124.8 and 134.9 W. Released cards drew between 135.0 and 720.5 W. No overlap.
Utilization is not a safe idle signal on this silicon. Power draw is.
The same log, replayed for headroom
In October we ran that July log back through the daemon's headroom mode. Mean draw was 332 W. The card sat at its 700 W limit for 11 percent of the window and under 150 W for 24 percent of it. A cap anywhere between 630 and 490 W would have bound during that same 11 percent and at no other time, because nothing ran between those levels. At 420 W the cap starts catching real work. The full table, with the energy above each cap, is on the sample headroom page.
The endurance run
Eleven days unattended, 267.9 hours in a single process, shadow mode, on one consumer GPU. The daemon logged 31 execution gaps and 21,781 stale frames skipped, counted by itself, and wrote a health file every cycle. The run ended when an operating system update rebooted the machine. This is the health file as it was at that moment.
"status": "healthy", "reason": "telemetry fresh; control loop live", "incidents": 104, "execution_gaps": 31, "last_gap_sec": 5.416, "stale_frames_skipped": 21781, "gpu_count": 1, "shadow_mode": true, "uptime_sec": 964437.626057
Measure first. Then hold the number.
Everything above runs today in read-only mode. What follows is built under the same rule: validated on real hardware before we quote a number.
The budget controller
Enforcement mode, written and running in dry run. You give it a node budget in watts. It holds the sum of the cards under that line through NVIDIA's own power limit, one small step at a time, and raises the caps back as soon as draw falls. Every cap it holds is written to a manifest so one command undoes all of them, and everything is released when the process exits. The cost of each cap in throughput will be measured on rented H100s before it is offered on a customer's node.
Fleet aggregation
Coincident peak across every host, aligned to the same timestamps, so you can see how much of a provisioned envelope is load and how much is margin nothing has ever used.