GPU machine types  |  Compute Engine  |  Google Cloud Documentation (https://docs.cloud.google.com/compute/docs/gpus) citeturn132view2 [wordlim: 200] Crawled: today; Content type: text/html; Source: open({"ref_id":"https://docs.cloud.google.com/compute/docs/gpus","lineno":684}); Total lines: 1043 L657: ### A3 Ultra machine type (NVIDIA H200) L658: L659: 【66†A3 Ultra】 machine types have 【93†NVIDIA H200 SXM GPUs†www.nvidia.com】 (`nvidia-h200-141gb`) attached and provide the highest network performance in the A3 series. A3 Ultra machine types are ideal for foundation model training and serving. For information about machine type limitations, see 【94†A3 Ultra limitations】. L660: Note: When provisioning A3 Ultra machine types, you must reserve capacity to create instances or clusters, use Spot VMs, use Flex-start VMs, or create a resize request in a MIG. For more information about the parameters to set when creating an A3 Ultra instance, see 【91†Create an A3 Ultra or A4 instance】. L661: Machine type | vCPU count | Instance memory (GB) | Attached Local SSD (GiB) | Physical NIC count | Maximum network bandwidth (Gbps) | NVIDIA H200 GPU count | GPU memory (GB HBM3e) L662: --- | --- | --- | --- | --- | --- | --- | --- L663: `a3-ultragpu-8g` | 224 | 2,952 | 12,000 | 10 | 3,600 | 8 | 1128 L664: #### A3 Ultra machine type notes L665: L666: The following notes apply to the preceding table: L667: * vCPU count: a vCPU is implemented as a single hardware hyper-thread on one of the available 【82†CPU platforms】. L668: * Maximum network bandwidth: maximum egress bandwidth can't exceed the number given. Actual egress bandwidth depends on the destination IP address and other factors. For more information about network bandwidth, see 【83†Network bandwidth】. L669: * GPU memory: GPU memory is the memory on a GPU device that can be used for temporary storage of data. It is separate from the instance's memory and is specifically designed to handle the higher bandwidth demands of your graphics-intensive workloads. L670: ### A3 Mega, High, and Edge machine types (NVIDIA H100) L671: L672: To use 【95†NVIDIA H100 SXM GPUs†www.nvidia.com】, you have the following options: L673: * 【67†A3 Mega】: these machine types have H100 SXM GPUs (`nvidia-h100-mega-80gb`) and are ideal for large-scale training and serving workloads. L674: * 【68†A3 High】: these machine types have H100 SXM GPUs (`nvidia-h100-80gb`) and are well-suited for both training and serving tasks. L675: * 【69†A3 Edge】: these machine types have H100 SXM GPUs (`nvidia-h100-80gb`), are designed specifically for serving, and are available in a 【96†limited set of regions】. L676: ### A3 Mega L677: L678: Note: When provisioning `a3-megagpu-8g` machine types, we recommend using a cluster of these instances and deploying with a scheduler such as Google Kubernetes Engine (GKE) or Slurm. For detailed instructions on either of these options, review the following: L679: * To create Google Kubernetes Engine cluster, see 【97†Deploy an A3 Mega cluster with GKE】. L680: * To create a Slurm cluster, see 【98†Deploy an A3 Mega Slurm cluster】. L681: Machine type | vCPU count | Instance memory (GB) | Attached Local SSD (GiB) | Physical NIC count | Maximum network bandwidth (Gbps) | NVIDIA H100 GPU count | GPU memory (GB HBM3) L682: --- | --- | --- | --- | --- | --- | --- | --- L683: `a3-megagpu-8g` | 208 | 1,872 | 6,000 | 9 | 1,800 | 8 | 640 L684: ### A3 High L685: L686: Note: When provisioning `a3-highgpu-1g`, `a3-highgpu-2g`, or `a3-highgpu-4g` machine types, you must create instances by using Spot VMs or Flex-start VMs. For detailed instructions on these options, review the following: L687: * To create Spot VMs, set the provisioning model to `SPOT` when you 【99†create an accelerator-optimized VM】. L688: * To create Flex-start VMs, you can use one of the following methods: L689: * Create a standalone VM and set the provisioning model to `FLEX_START` when you 【99†create an accelerator-optimized VM】. L690: * Create a resize request in a managed instance group (MIG). For instructions, see 【100†Create a MIG with GPU VMs】. L691: Machine type | vCPU count | Instance memory (GB) | Attached Local SSD (GiB) | Physical NIC count | Maximum network bandwidth (Gbps) | NVIDIA H100 GPU count | GPU memory (GB HBM3) L692: --- | --- | --- | --- | --- | --- | --- | --- L693: `a3-highgpu-1g` | 26 | 234 | 750 | 1 | 25 | 1 | 80 L694: `a3-highgpu-2g` | 52 | 468 | 1,500 | 1 | 50 | 2 | 160 L695: `a3-highgpu-4g` | 104 | 936 | 3,000 | 1 | 100 | 4 | 320 L696: `a3-highgpu-8g` | 208 | 1,872 | 6,000 | 5 | 1,000 | 8 | 640 L697: ### A3 Edge L698: L699: Note: To get started with A3 Edge instances, see 【101†Create an A3 VM with GPUDirect-TCPX enabled】. L700: Machine type | vCPU count | Instance memory (GB) | Attached Local SSD (GiB) | Physical NIC count | Maximum network bandwidth (Gbps) | NVIDIA H100 GPU count | GPU memory (GB HBM3) L701: --- | --- | --- | --- | --- | --- | --- | --- L702: `a3-edgegpu-8g` | 208 | 1,872 | 6,000 | 5 | 600 (`asia-south1` and `northamerica-northeast2`) or 400 (all other 【102†A3 Edge regions】) | 8 | 640 L703: #### A3 Mega, High, and Edge machine type notes L704: L705: The following notes apply to the preceding table: L706: * vCPU count: a vCPU is implemented as a single hardware hyper-thread on one of the available 【82†CPU platforms】. L707: * Maximum network bandwidth: maximum egress bandwidth can't exceed the number given. Actual egress bandwidth depends on the destination IP address and other factors. For more information about network bandwidth, see 【83†Network bandwidth】. L708: * GPU memory: GPU memory is the memory on a GPU device that can be used for temporary storage of data. It is separate from the instance's memory and is specifically designed to handle the higher bandwidth demands of your graphics-intensive workloads. L709: ## A2 machine series (NVIDIA A100) L710: L711: 【103†A2 accelerator-optimized】 machine types have 【104†NVIDIA A100 GPUs†www.nvidia.com】 attached and are ideal for model fine tuning, large model and cost optimized inference. L712: L713: The A2 machine series offers two types: L714: * A2 Ultra: these machine types have A100 80GB GPUs (`nvidia-a100-80gb`) and Local SSD disks attached. L715: * A2 Standard: these machine types have A100 40GB GPUs (`nvidia-tesla-a100`) attached. You can also add Local SSD disks when creating an A2 Standard instance. For the number of disks you can attach, see 【105†Machine types that require you to choose a number of Local SSD disks】. L716: ### A2 Ultra L717: L718: Machine type | vCPU count | Instance memory (GB) | Attached Local SSD (GiB) | Maximum network bandwidth (Gbps) | NVIDIA A100 80GB GPU count | GPU memory (GB HBM2e) L719: --- | --- | --- | --- | --- | --- | --- L720: `a2-ultragpu-1g` | 12 | 170 | 375 | 24 | 1 | 80 L721: `a2-ultragpu-2g` | 24 | 340 | 750 | 32 | 2 | 160 L722: `a2-ultragpu-4g` | 48 | 680 | 1,500 | 50 | 4 | 320 L723: `a2-ultragpu-8g` | 96 | 1,360 | 3,000 | 100 | 8 | 640 L724: ### A2 Standard L725: L726: Machine type | vCPU count | Instance memory (GB) | Local SSD supported | Maximum network bandwidth (Gbps) | NVIDIA A100 40GB GPU count | GPU memory (GB HBM2) L727: --- | --- | --- | --- | --- | --- | --- L728: `a2-highgpu-1g` | 12 | 85 | Yes | 24 | 1 | 40 L729: `a2-highgpu-2g` | 24 | 170 | Yes | 32 | 2 | 80 L730: `a2-highgpu-4g` | 48 | 340 | Yes | 50 | 4 | 160 L731: `a2-highgpu-8g` | 96 | 680 | Yes | 100 | 8 | 320 L732: `a2-megagpu-16g` | 96 | 1,360 | Yes | 100 | 16 | 640 L733: ### A2 machine type notes L734: L735: The following notes apply to the preceding table: L736: * vCPU count: a vCPU is implemented as a single hardware hyper-thread on one of the available 【82†CPU platforms】. L737: * Maximum network bandwidth: maximum egress bandwidth can't exceed the number given. Actual egress bandwidth depends on the destination IP address and other factors. For more information about network bandwidth, see 【83†Network bandwidth】. L738: * GPU memory: GPU memory is the memory on a GPU device that can be used for temporary storage of data. It is separate from the instance's memory and is specifically designed to handle the higher bandwidth demands of your graphics-intensive workloads. L739: ## G4 machine series (NVIDIA RTX PRO 6000) L740: L741: 【72†G4 accelerator-optimized】 machine types use 【106†NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs†www.nvidia.com】 (`nvidia-rtx-pro-6000`) and are suitable for NVIDIA Omniverse simulation workloads, graphics-intensive applications, video transcoding, and virtual desktops. G4 machine types also provide a low-cost solution for performing single host inference and model tuning compared with A series machine types. L742: You can also add Titanium SSD disks when you create a G4 instance. For the number of disks you can attach, see 【105†Machine types that require you to choose a number of Local SSD disks】. For information about machine type limitations, see 【107†G4 limitations】. L743: A key feature of the G4 series is support for direct GPU peer-to-peer (P2P) communication on multi-GPU machine types (`g4-standard-96`, `g4-standard-192`, `g4-standard-384`). This allows GPUs within the same instance to exchange data directly over the PCIe bus, without involving the CPU host. For more information about G4 GPU peer-to-peer communication, see 【108†G4 GPU peer-to-peer communication】. L744: Note: To get started with G4 instances, see 【109†Create a G4 instance】. L745: Machine type | vCPU count | Instance memory (GB) | Maximum Titanium SSD supported (GiB) | Physical NIC count | Maximum network bandwidth (Gbps) | NVIDIA RTX PRO 6000 GPU count | GPU memory (GB GDDR7) L746: --- | --- | --- | --- | --- | --- | --- | --- L747: `g4-standard-6` | 6 | 22 | 0 | 1 | 20 | 1/8 | 12 L748: `g4-standard-12` | 12 | 45 | 375 | 1 | 20 | 1/4 | 24 L749: `g4-standard-24` | 24 | 90 | 750 | 1 | 20 | 1/2 | 48