The nodeGroup that will be selected depends on number of GPUs you’re requested?

Yes, you got that right.

If so, where is it defined in config?

That’s beauty Kubernetes and the Cluster Autoscaler, we don’t need to define it in the config.

Once the Nvidia device plugin gets installed, a new field gets populated in the node spec (similar to CPU/Memory).

So when a notebook server spins up and it requests for 1 GPU, it scales up the 1 GPU nodegroup.

Although if there’s a 4 GPU instance already running, of which only 2 GPUs are being consumed, Cluster Autoscaler will schedule the new 1 GPU notebook server to the same instance, there by saving further costs.

Checkout this article to go more in detail on how the cluster autoscaler works.

“ I setup autoscaling on Kubernetes and you can too!” by Arjun Sunil https://link.medium.com/eDbVDEofuab