Amazon EC2 P5 Instances – AWS (https://aws.amazon.com/ec2/instance-types/p5/) citeturn130view0 [wordlim: 200] Crawled: last week; Content type: text/html; Source: open({"ref_id":"https://aws.amazon.com/ec2/instance-types/p5/","lineno":235}); Total lines: 327 L228: You can also use these instances to deploy HPC applications at scale for pharmaceutical discovery, seismic analysis, weather forecasting, and financial modeling. L229: To deliver these performance improvements and cost savings, P5 and P5e instances complement NVIDIA H100 and H200 Tensor Core GPUs with 2x higher CPU performance, 2x higher system memory, and 4x higher local storage as compared to previous-generation GPU-based instances. P5en instances pair NVIDIA H200 Tensor Core GPUs with high performance Intel Sapphire Rapids CPU, enabling Gen5 PCIe between CPU and GPU. L230: P5en instances provide up to 4x the bandwidth between CPU and GPU and lower network latency compared to P5e and P5 instances thereby improving distributed training performance. P5 and P5e instances support provide up to 3,200 Gbps of networking using second-generation Elastic Fabric Adapter (EFA). P5en, with third generation of EFA using Nitro v5, shows up to 35% improvement in latency compared to P5 that uses the previous generation of EFA and Nitro. L231: This helps improve collective communications performance for distributed training workloads such as deep learning, generative AI, real-time data processing, and high-performance computing (HPC) applications. To deliver large-scale compute at low latency, these instances are deployed in 【71†Amazon EC2 UltraClusters】 that enable scaling up to 20,000 H100 or H200 GPUs interconnected with a petabit-scale nonblocking network. L232: P5, P5e, and P5en instances in EC2 UltraClusters can deliver up to 20 exaflops of aggregate compute capability—performance equivalent to a supercomputer. L233: ## Amazon EC2 P5 Instances L234: L235: Play L236: L237: 【72†Image: Missing alt text value†d1.awsstatic.com】 L238: L239: ## Benefits L240: L241: ### Train 100B+ parameter models at scale L242: L243: P5, P5e, and P5en instances can train large generative AI models at scale and deliver up to 4x the performance of previous-generation GPU-based EC2 instances. L244: ### Reduce time to solution and iterate faster L245: L246: P5, P5e, and P5en instances reduce training times and time to solution from weeks to just a few days. This helps you iterate at a faster pace and get to market more quickly. L247: L248: ### Lower your DL and HPC infrastructure costs L249: L250: P5, P5e, and P5en instances deliver up to 40% savings on DL training and HPC infrastructure costs compared to previous-generation GPU-based EC2 instances. L251: ### Run distributed training and HPC with exascale compute L252: L253: P5, P5e, and P5en instances provide up to 3,200 Gbps of EFA networking. These instances are deployed in EC2 UltraClusters and deliver 20 exaflops of aggregate compute capability. L254: L255: ## Features L256: ### NVIDIA H100 and H200 Tensor Core GPUs L257: L258: P5 instances provide up to 8 NVIDIA H100 GPUs with a total of up to 640 GB HBM3 GPU memory per instance. P5e and P5en instances provide up to 8 NVIDIA H200 GPUs with a total of up to 1128 GB HBM3e GPU memory per instance. Both instances support up to 900 GB/s of NVSwitch GPU interconnect (total of 3.6 TB/s bisectional bandwidth in each instance), so each GPU can communicate with every other GPU in the same instance with single-hop latency. L259: ### New transformer engine and DPX instructions L260: L261: NVIDIA H100 and H200 GPUs have a transformer engine that intelligently manages and dynamically chooses between FP8 and 16-bit calculations. This feature helps deliver faster DL training speedups on LLMs compared to previous-generation A100 GPUs. For HPC workloads, NVIDIA H100 and H200 GPUs have new DPX instructions that further accelerate dynamic programming algorithms as compared to A100 GPUs. L262: ### High-performance networking L263: L264: P5, P5e, and P5en instances deliver up to 3,200 Gbps of EFA networking. EFA is also coupled with NVIDIA GPUDirect RDMA to enable low- latency GPU-to-GPU communication between servers with operating system bypass. L265: ### High-performance storage L266: L267: P5, P5e and P5en instances support Amazon FSx for Lustre file systems so you can access data at the hundreds of GB/s of throughput and millions of IOPS required for large-scale DL and HPC workloads. Each instance also supports up to 30 TB of local NVMe SSD storage for fast access to large datasets. You can also use virtually unlimited cost-effective storage with Amazon Simple Storage Service (Amazon S3). L268: ## Customer testimonials L269: L270: Here are some examples of how customers and partners have achieved their business goals with Amazon EC2 P4 instances. L271: ## Anthropic L272: > At Anthropic, we are working to build reliable, interpretable, and steerable AI systems. While the large general AI systems of today can have significant benefits, they can also be unpredictable, unreliable, and opaque. Our goal is to make progress on these issues and deploy systems that people find useful. Our organization is one of the few in the world that is building foundational models in DL research. L273: These models are highly complex, and to develop and train these cutting-edge models, we need to distribute them efficiently across large clusters of GPUs. We are using Amazon EC2 P4 instances extensively today, and we are excited about the launch of P5 instances. We expect them to deliver substantial price-performance benefits over P4d instances, and they'll be available at the massive scale required for building next-generation LLMs and related products.