Learn how to set up, install, and optimize GPU-enabled instances for high-performance computing, machine learning, and graphics-intensive workloads in the AWS cloud. Ideal for system administrators, DevOps engineers, and cloud architects looking to harness the full potential of GPU computing in a scalable, flexible cloud environment. Covers essential steps from IAM permissions to driver installation and post-setup optimization.

Introduction

This article outlines the implementation of NVIDIA GRID drivers on Amazon EC2 instances. The topic's significance stems from several key factors:

1. NVIDIA GRID Drivers: These drivers enable GPU virtualization, which is crucial for connecting virtual machines to physical GPU hardware. They manage resource allocation, scheduling, and communication between VMs and GPUs.

2. Amazon EC2: EC2 provides virtual servers in the cloud. Implementing NVIDIA GRID drivers on EC2 instances enables GPU-accelerated workloads in a cloud environment. This combination allows for scalable, on-demand access to GPU resources.

3. GPU Virtualization: This technology allows multiple virtual machines to share a single physical GPU. Benefits include:

   • Improved resource utilization: Multiple VMs can use GPU resources simultaneously.

   • Cost reduction: Fewer physical GPUs are needed to support multiple workloads.

   • Broader application support: GPU acceleration becomes available to more cloud-based applications.

4. High-Performance Graphics Processing: NVIDIA GRID drivers on EC2 instances support graphics-intensive applications in the cloud, including:

   • 3D rendering: For animation, visual effects, and architectural visualization.

   • CAD (Computer-Aided Design): For engineering and product design.

   • Video encoding/decoding: For streaming services and video processing.

   • Machine learning and AI: For training and inference of deep learning models.

5. Cloud-Based GPU Computing: Implementing NVIDIA GRID in the cloud enables organizations to access powerful GPU capabilities without large upfront hardware investments. This approach offers flexibility in scaling resources based on demand.

This guide will explain the technical aspects of implementing NVIDIA GRID drivers on EC2.

Connect to your Linux instance.

When deploying GPU-accelerated workloads on Amazon EC2, selecting the appropriate instance type is crucial. This allows you to configure and manage your GPU-enabled environment effectively. Let's explore three robust methods for connecting to your EC2 instance, each offering unique advantages depending on your operational needs and security requirements.

SSH

The Classic Approach Secure Shell (SSH) remains a stalwart in the system administrator's toolkit. To connect via SSH, you'll use a command like this:

ssh -i /path/to/key.pem ec2-user@instance-public-dns

This method requires a key pair and an open inbound SSH port (typically 22) in your security group.

Optional: You can also enhance security by implementing key-based authentication and disabling password login:

sudo sed -i 's/PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo systemctl restart sshd

EC2 Instance Connect

For those seeking a streamlined approach, EC2 Instance Connect offers a browser-based SSH connection directly through the AWS Management Console. This method eliminates the need for managing key pairs or configuring SSH ports, as AWS handles these details behind the scenes with temporary SSH keys.

AWS Systems Manager Session Manager

Advanced Control and Auditing Systems Manager Session Manager takes security and management a step further. It provides secure shell access without requiring open inbound ports, supporting both Windows and Linux instances. An added benefit is the automatic logging of session data, which proves invaluable for auditing and compliance purposes.

GPU Instances

Choosing Your GPU-Enabled Instance Before diving into connection methods, it's crucial to select an EC2 instance type that includes GPU capabilities. AWS offers a range of GPU-enabled instances to suit various workloads[1]:

• G instances (e.g., g4dn, g5) for general-purpose GPU computing

• P instances (e.g., p3, p4d) optimized for machine learning tasks

• Inf1 instances featuring AWS Inferentia chips for efficient ML inference

These instances can run on various operating systems, including Amazon Linux 2, Ubuntu, CentOS, and Windows Server. However, it's important to note that this guide focuses specifically on setting up NVIDIA GPUs on Amazon Linux. While the core concepts remain similar across platforms, some commands and file paths may differ on other operating systems.

GPU Options

Power for Every Workload EC2 provides access to cutting-edge GPU hardware. Popular options include:

1. NVIDIA Tesla series (V100, T4, A100): These powerhouses excel in diverse compute-intensive tasks.

2. NVIDIA A10G: A balanced option for graphics and compute workloads.

3. NVIDIA K80: An older but still capable GPU for various applications.

Each GPU architecture has distinct characteristics[2]:

Selecting the right GPU depends on your specific application requirements and performance needs. Whether you're running complex simulations, training machine learning models, or rendering high-fidelity graphics, there's a GPU instance tailored to your workload.

Install and configure the AWS CLI on your Linux instance.

IAM Policy

IAM (Identity and Access Management) is AWS's system for controlling access to various resources and services. Proper IAM configuration is essential for security and functionality when working with AWS services, including EC2 and S3.

AmazonS3ReadOnlyAccess Policy: This specific policy grants read-only access to Amazon S3 (Simple Storage Service) resources. It's required in this context because:

1. Driver Storage: NVIDIA GRID drivers are often stored in S3 buckets. AWS maintains these drivers in their own S3 locations for easy distribution.

2. Automated Installation: During the GRID driver installation process, your EC2 instance may need to download the driver files from S3.

3. Version Control: S3 allows AWS to manage and update driver versions efficiently. Your instance needs access to read these files to ensure it's using the latest compatible drivers.

4. Reduced Local Storage: By accessing drivers from S3, you don't need to store large driver files on your EC2 instance, saving local storage space.

Implementing the Permission: To apply this permission, you have two main options:

1. User-based: If you're using an IAM user, attach the AmazonS3ReadOnlyAccess policy to that user.

2. Role-based: For EC2 instances, it's generally better to use an IAM role. Create a role with the AmazonS3ReadOnlyAccess policy and attach it to your EC2 instance.

   Policy Json

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "s3:Get*",
                    "s3:List*",
                    "s3-object-lambda:Get*",
                    "s3-object-lambda:List*"
                ],
                "Resource": "*"
            }
        ]
    }
   

Using roles is preferred because:

• It's more secure (no long-term credentials stored on the instance)

• It's easier to manage (you can modify permissions without updating instances)

• It works seamlessly with EC2 (AWS handles credential management)

Principle of Least Privilege: Note that we're using a read-only policy. This adheres to the principle of least privilege, granting only the minimum permissions necessary for the task. Your instance needs to read from S3, but it doesn't need write or delete permissions, enhancing security.

Verification: After setting up the permission, you can verify it using the AWS CLI:

aws iam get-user-policy --user-name YourUserName --policy-name AmazonS3ReadOnlyAccess

Or for a role:

aws iam get-role-policy --role-name YourRoleName --policy-name AmazonS3ReadOnlyAccess

By ensuring these permissions are in place, you're setting the foundation for a smooth NVIDIA GRID driver installation process, allowing your EC2 instance to securely access the necessary resources while maintaining proper access control.

NVIDIA Driver Installation

1. Install required packages and update the system:

[ec2-user ~]$ sudo yum install gcc make
[ec2-user ~]$ sudo yum update -y

Why gcc and make?

• gcc (GNU Compiler Collection):

  • It's needed to compile the NVIDIA driver source code.

  • Some driver components may require compilation during installation.

  • It allows for potential custom modifications or optimizations if needed.

• make:

  • It automates the build process for the driver.

  • It manages dependencies and compilation order efficiently.

  • It's used in the driver's installation scripts.

• Reboot the instance to load the latest kernel:

[ec2-user ~]$ sudo reboot

5. After reconnecting, install the kernel headers package:

[ec2-user ~]$ sudo yum install -y kernel-devel-$(uname -r)

• kernel-devel: This package contains the kernel headers and build files.

• $(uname -r): This is a command substitution that returns the current running kernel version. It’s good practice because kernel modules, which the NVIDIA driver installation process compiles, must be built against the specific kernel version in use to maintain system stability and functionality. Any version discrepancy can result in compilation errors or driver instability. These kernel modules serve as the interface between the NVIDIA driver and the kernel, facilitating crucial operations such as GPU memory management and interrupt handling. While not explicitly utilized in this context, the presence of matching kernel headers also enables potential integration with Dynamic Kernel Module Support (DKMS). DKMS can automate the process of rebuilding the NVIDIA kernel module when new kernel versions are installed, enhancing system maintainability. Furthermore, this approach provides a degree of future-proofing; in the event of a kernel update, simply re-executing this command will ensure the retrieval of the corresponding headers, maintaining system coherence and driver compatibility.

  • uname: Prints system information

  • -r: Specifies that we want only the kernel release version

6. Download the GRID driver installation utility:

[ec2-user ~]$ aws s3 cp --recursive s3://ec2-linux-nvidia-drivers/latest/ .

This command uses the AWS CLI to copy the latest NVIDIA GRID driver from an AWS-managed S3 bucket to your current directory.

• The --recursive flag ensures all files in the 'latest' directory are copied.

• This method ensures you're always getting the most recent driver version compatible with EC2.

7. Set permissions and run the self-install script:

[ec2-user ~]$ chmod +x NVIDIA-Linux-x86_64*.run
[ec2-user ~]$ sudo /bin/sh ./NVIDIA-Linux-x86_64*.run

8. Verify the driver installation:

[ec2-user ~]$ nvidia-smi -q | head

• nvidia-smi (NVIDIA System Management Interface) is a command-line utility that provides monitoring and management capabilities for NVIDIA GPUs.

• The -q flag requests a query for all available information.

• | head pipes the output to the head command, showing only the first few lines, which typically include version and driver information.

9. For NVIDIA vGPU software version 14.x or greater on specific instances, disable GSP:

[ec2-user ~]$ sudo touch /etc2-user ~]$ echo "options nvidia NVreg_EnableGpuFirmware=0" | sudo tee --append /etc/modprobe.d/nvidia.conf

This configuration step is tailored specifically for more recent iterations of NVIDIA vGPU software deployed on particular Amazon EC2 instance types. Its primary function is to disable the GPU System Processor (GSP), a component that, while generally beneficial, can introduce complications in certain virtualized environments. The command in question either generates a new NVIDIA configuration file or appends to an existing one, effectively setting a crucial kernel module parameter. This parameter modification is implemented at the system level, ensuring that the GSP remains inactive across reboots and driver updates. By disabling the GSP, potential conflicts between the virtualization layer and NVIDIA's hardware-level optimizations are mitigated, thereby enhancing stability and performance consistency in virtualized GPU scenarios. This adjustment exemplifies the nuanced configuration often required when deploying high-performance graphics solutions in cloud-based, virtualized infrastructures, where the interplay between hardware acceleration mechanisms and virtualization technologies necessitates fine-tuned optimizations.

10. Reboot the instance again:

[ec2-user ~]$ sudo reboot

Optional steps for specific use cases, such as setting up NICE DCV for high-resolution displays or activating GRID Virtual Applications, may be required depending on your particular needs.

By following this guide, system administrators and developers can transform standard EC2 instances into powerful GPU-enabled computing environments. The careful sequence of steps – from setting up the necessary permissions and preparing the system environment, to installing and configuring the NVIDIA drivers – ensures a robust and reliable setup.

References:

1. https://docs.aws.amazon.com/dlami/latest/devguide/gpu.html

2. https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units#Data_Center_GPUs

3. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-nvidia-driver.html