Colorado State University Bighorn Research Computing Collaborative: Data Science Research Institute 

Standard CPU Compute Resources

The cluster contains twenty-three (23) production CPU compute nodes configured for high-throughput and parallel computing workloads.

Each node includes:

• Two (2) AMD EPYC™ 7763 processors
• 512 GiB system memory

Aggregate standard CPU resources:

• 23 compute nodes
• 2,944 CPU cores
• 11.5 TiB aggregate memory

These resources support a wide range of computational applications, including numerical modeling, simulation, data analysis, and parallel computing workflows.

CPU Development and Interactive Access Node

A dedicated development node provides interactive access to cluster resources and hosts the Open OnDemand portal environment.

Configuration:

• Two (2) AMD EPYC™ 7763 processors
• 512 GiB system memory

This resource supports software development, code compilation, testing, workflow development, and interactive research computing activities.

High-Memory Compute Nodes

The HPC environment includes two (2) large-memory compute nodes designed for applications requiring substantial memory footprints.

Each node includes:

• Two (2) AMD EPYC™ 7763 processors
• 2,036 GiB (~2 TB) system memory

Aggregate high-memory resources:

• 256 CPU cores
• Approximately 4 TiB total memory

These resources support memory-intensive applications, including large-scale graph analytics, genomics, machine learning, optimization, in-memory databases, and advanced scientific simulations.

GPU-Accelerated Computing Resources

Production GPU Compute Nodes

The cluster includes four (4) GPU-accelerated compute nodes supporting artificial intelligence, machine learning, deep learning, visualization, and GPU-enabled scientific computing applications.

Each node includes:

• Two (2) AMD EPYC™ 7763 processors
• 512 GiB system memory
• Four (4) NVIDIA A100 Tensor Core GPUs
• 80 GB GPU memory per accelerator

Aggregate GPU resources:

• 1.28 TB total GPU memory

These resources are optimized for large-scale AI model development, training, inference, and accelerated computational workflows.

GPU Development Node

A dedicated GPU-enabled development node supports interactive GPU application development through Open OnDemand.

Configuration:

• Two (2) AMD EPYC™ 7763 processors
• 512 GiB system memory
• Two (2) NVIDIA A100 GPUs
• 80 GB GPU memory per accelerator

This resource enables application development, software testing, and interactive access to GPU computing capabilities.

Next-Generation AI and Accelerated Computing Node

The cluster incorporates a dedicated NVIDIA Grace Hopper compute node designed to support emerging artificial intelligence and advanced scientific computing workloads.

Configuration:

• One (1) NVIDIA GH200 Grace Hopper Superchip
• 120 GB HBM3 GPU memory
• 480 GB LPDDR5X coherent system memory

The Grace Hopper architecture provides tightly integrated CPU-GPU memory access and is particularly well suited for large language models, foundation models, advanced AI workflows, and data-intensive research applications.

Storage Infrastructure

BeeGFS Parallel File System

The HPC Cluster utilizes a BeeGFS parallel file system that provides high-performance, shared storage accessible from all compute resources.

The storage environment is designed to deliver high throughput and scalable performance for large-scale scientific workloads requiring simultaneous access from multiple nodes.

Capabilities include:

• Shared high-performance parallel storage
• Support for concurrent multi-node access
• High-bandwidth I/O operations
• Storage for active research datasets, software, and computational results

Data Transfer Resources

Dedicated Data Transfer Node

A dedicated Data Transfer Node (DTN) facilitates efficient movement of research data between the HPC environment and external systems.

Capabilities include:

• 100 Gigabit Ethernet connectivity
• High-speed data import and export
• Secure transfer of large research datasets
• Support for collaborative and multi-institutional research projects

Network Infrastructure

All computational and storage resources are interconnected through a high-performance networking fabric designed to support large-scale parallel applications and data-intensive workloads.

Network Specifications

• 100 Gb/s internal cluster fabric
• High-bandwidth connectivity between compute and storage resources
• Low-latency communications for distributed computing
• Dedicated 100 Gb/s data transfer infrastructure

This network architecture enables efficient execution of parallel applications and high-speed access to shared storage resources.

Resource Summary

Compute Resources

Aggregate System Capacity

• 3,968 AMD EPYC CPU cores
• 18 NVIDIA A100 GPUs (80 GB each)
• 1 NVIDIA GH200 Grace Hopper Superchip
• Approximately 20 TiB of aggregate system memory
• BeeGFS parallel file system
• 100 Gb/s internal network fabric
• 100 Gb/s external data transfer capability

Research Applications and Capabilities

This HPC infrastructure supports a broad spectrum of computational research activities, including:

• Artificial intelligence and machine learning
• Large language model development and inference
• Data science and advanced analytics
• Computational physics and chemistry
• Engineering simulation and modeling
• Bioinformatics and genomics
• Image and signal processing
• Digital engineering and defense modeling
• Large-scale parallel computing
• High-performance scientific visualization

The combination of CPU, GPU, large-memory, and next-generation accelerated computing resources provides researchers with a comprehensive advanced computing environment capable of supporting both current and emerging computational research requirements.

Executive Summary: Bighorn Research Computing Collaborative (BRCC)

High-Performance Computing (HPC) Infrastructure

The Data Science Research Institute as part of the Office of the Vice President for Research at Colorado State University maintains a modern, research-focused High Performance Computing (HPC) environment that provides advanced computational, data-intensive, and artificial intelligence (AI) capabilities to support faculty, students, and research collaborators across a broad range of scientific and engineering disciplines. The system is designed to accommodate traditional high-performance computing workloads, large-scale data analytics, machine learning, deep learning, simulation and modeling, and emerging AI applications.

The HPC cluster consists of a heterogeneous computing architecture totaling 3,968 AMD EPYC CPU cores, 18 NVIDIA A100 GPUs, and a dedicated NVIDIA Grace Hopper GH200 Superchip resource. Standard computational capacity is provided by 23 production CPU compute nodes, each containing dual AMD EPYC 7763 processors and 512 GiB of memory, delivering a robust platform for parallel and high-throughput computing applications.

To support memory-intensive research, the system includes two high-memory nodes, each equipped with 2,036 GiB of RAM, enabling analysis of exceptionally large datasets and execution of applications requiring substantial in-memory processing. Interactive and software development activities are supported through dedicated CPU and GPU development nodes that provide web-based researcher access through the Open OnDemand platform.

GPU-accelerated computing capabilities are provided through four production GPU nodes, each equipped with four NVIDIA A100 80 GB Tensor Core GPUs, providing a total of 16 production GPUs and approximately 1.28 TB of aggregate GPU memory. An additional GPU development node containing two NVIDIA A100 GPUs supports application development, workflow testing, and interactive AI research. These resources enable state-of-the-art research in artificial intelligence, machine learning, computer vision, large language models, data analytics, and computational science.

To support next-generation AI and data-intensive workflows, the infrastructure includes a dedicated NVIDIA Grace Hopper GH200 compute node that combines advanced CPU-GPU integration with 120 GB of HBM3 memory and 480 GB of coherent LPDDR5X memory. This architecture is particularly well suited for foundation models, large language models, and advanced scientific computing applications requiring high-bandwidth memory access and accelerated performance.

The computational environment is supported by a high-performance BeeGFS parallel file system, providing shared storage accessible by all compute resources. A dedicated Data Transfer Node (DTN) enables secure movement of large research datasets into and out of the computing environment via 100 Gb/s connectivity. All compute, storage, and transfer resources are interconnected through a 100 Gb/s internal high-speed network fabric, ensuring efficient communication among nodes and sustained performance for distributed applications.

Collectively, this integrated cyberinfrastructure provides researchers with a comprehensive computing environment capable of supporting computationally intensive research in engineering, physical sciences, artificial intelligence, data science, computational biology, national security applications, and other data-driven disciplines. The combination of advanced CPU resources, large-memory systems, GPU accelerators, next-generation AI hardware, and high-performance storage enables investigators to conduct cutting-edge research while supporting future growth in computational and AI-enabled scholarship.