Skip to content

Pack mixed-size vGPU placements to keep room for larger profiles - #513

Draft
yummybomb wants to merge 19 commits into
mainfrom
hypeship/vgpu-placement-packing
Draft

yummybomb wants to merge 19 commits into
mainfrom
hypeship/vgpu-placement-packing

Conversation

@yummybomb

Copy link
Copy Markdown
Contributor

Stacked on #511. Review that first; this PR only adds the last commit.

Summary

In mixed-size (heterogeneous) mode, each vGPU occupies a fixed run of its GPU's placement slots, and each profile may only start at the driver's heterogeneous placement IDs. The driver puts a new vGPU at the lowest free placement for its type. On an L40S that scatters 2Q across the runs 4Q needs:

  • Of the 16 2Q placements, 8 (0, 10, 12, 22, 24, 34, 36, 46) each overlap one 4Q placement, and 8 (4, 6, 16, 18, 28, 30, 40, 42) overlap none.
  • Lowest-free puts every other 2Q on an overlapping placement, so eight 2Q leave room for four 4Q instead of eight.

Change

  • After configure writes and verifies current_vgpu_type, and before any VM attaches, packPlacement moves the vGPU by writing placement_id.
  • Choice: the free placement that blocks the fewest free placements of larger profiles. Sizes are compared smallest first. A tie keeps the driver's placement, so a write only happens for a strictly better one.
  • Placement data:
    • Free placements are computed from the current_vgpu_type and placement_id of the GPU's other configured VFs.
    • The table and mode come from nvidia-smi vgpu -s -v -i <GPU> and nvidia-smi -q -i <GPU>, read the same way as the infra role's heterogeneous-mode check.
    • Both are cached per GPU for 10 minutes. A failed load is cached too, so a host without nvidia-smi doesn't run it on every claim.
  • Best effort: packing never fails a claim. Equal-size mode, nvidia-smi failing, an unreadable sibling or one with an unknown type, a refused placement_id write, or a read-back mismatch each log and keep the driver's placement.
  • This only runs when the type is actually written. In Place vGPU profiles the driver hides from creatable lists #511 that happens inside claimVGPU under the allocation lock, so siblings can't change underneath it.

Validation

  • Unit tests (the parser reads the real nvidia-smi vgpu -s -v output from a dev L40S, in testdata):
    • Choice: simulating the driver's lowest-free rule plus packing, eight 2Q land on 4, 6, 16, 18, 28, 30, 40, 42 and leave all eight 4Q placements free. Without packing only four stay free. Each further 2Q then costs exactly one 4Q placement. A tie keeps the driver's placement.
    • configure: writes placement_id and loads the table once per GPU. It keeps the driver's placement in equal-size mode, when nvidia-smi fails, when a sibling's type is unknown, and when the driver refuses the placement write.
  • Mutation checks: removing each guard makes at least one test fail. The guards are the call in configure, keep-on-tie, the mode check, the unknown-sibling check, the cache, counting only free larger placements, excluding the VF itself, and only choosing free targets.
  • Test runs: go test ./lib/devices/ and go test -race ./lib/instances/ -run 'VGPU|VendorVFIO' pass. lib/devices and lib/instances build for darwin.
  • On a dev L40S (driver 580.178.05), I ran this branch's devices.ConfigureVGPU and DestroyVGPU against an empty card, with no VMs attached:
    • Eight 2Q writes: the driver placed each at 0, and packing moved them to 16, 18, 28, 30, 4, 6, 40, 42.
    • Eight 4Q writes then landed at 0, 8, 12, 20, 24, 32, 36 and 44, so the card held 16 vGPUs (all 48 GB).
    • A 17th write was refused with EIO, which is the errno a full card returns for current_vgpu_type.
    • All VFs were reset afterwards.
  • Not tested: booting a VM on a VF whose placement was moved. The write happens before attach, and the driver accepted every move, but a boot on dev should confirm it before merging.

Notes

  • No extra 2Q room: packing changes where vGPUs go, not how many 2Q a card holds. Mixed mode still allows at most 16 2Q per card.
  • Placement choice beyond 4Q: the score also counts larger profiles nothing uses today, such as 8Q and 16Q. They only break ties among placements that are equal for 3Q and 4Q, which is why the dev run moved the first 2Q to 16 rather than 4.

@yummybomb
yummybomb added this pull request to stack #522 October 16, 2026 15:24
In mixed-size mode the driver drops a profile from every VF's
creatable_vgpu_types once its GPU holds as many vGPUs as that profile's
equal-size instance limit, even while a placement remains. Remember the
profiles each GPU has advertised, and when none advertises the requested
one, let the driver decide by configuring a clean VF per supporting GPU
under the allocation lock.
Merge the dirty-VF repair and hidden-profile probe into one loop so a VF
reselected after the driver refuses a GPU is reset like any other dirty
VF, instead of being claimed while it may still hold open VFIO handles.

When the driver rejects a hidden profile, reset unclaimed dirty siblings
on that GPU and retry once, since one of them may hold the last
placement. Only a rejected current_vgpu_type write, now reported as
devices.ErrVGPUTypeRejected, moves placement to the next GPU; any other
configure error fails the claim.

Remember profiles listed after the dirty-VF reset retry, and only offer a
hidden profile on a VF whose own creatable list was read and is non-empty.
Advertised profiles were configured after claimVGPU released the lock,
so a later claim could take the placement a saved claim had picked, and
its configure then failed with no fallback. Write the type for every
claim inside the claim loop, so a driver rejection moves to the next GPU
whether or not the profile is advertised. Reset the VF if saving the
claim fails, since it is now configured before the claim exists.
…le check

When every GPU that advertised a profile refused its write, the selector
still counted their creatable lists and reported "no available VF".
Count only VFs still in the candidate set, so the claim fails with the
not-creatable error as it does for hidden profiles.

Count driver rejections per GPU in
hypeman_instances_vgpu_driver_rejections_total, return an error instead
of saving an unconfigured claim if the selected VF is somehow not a
candidate, and note that the later configure call in create and start
only re-checks quarantine and the type.
The driver puts each new vGPU at the lowest free placement for its type,
which on an L40S lands half of the 2Q vGPUs on placements a 4Q needs.
After configuring a type, move the vGPU to the free placement that blocks
the fewest free placements of larger profiles, read from nvidia-smi and
cached per GPU. Ties and any failure keep the driver's placement.
@yummybomb
yummybomb force-pushed the hypeship/vgpu-placement-packing branch from d934a3c to f87d76f Compare October 16, 2026 21:59
Base automatically changed from hypeship/vgpu-place-hidden-profiles to main October 21, 2026 21:12

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant