HomeGuidesRecipesAPI EndpointsRelease NotesCommunity
Log In

RHEL 8 Helm Chart

In this guide we will go from a basic RHEL 8 operating system to a fully functioning Layar installation. This guide is intended as an example installation only and makes configuration decisions that may not be best suited for your environment.



  • 256 GB RAM
  • 16+ CPUs
  • 1TB SSD
  • 4x NVIDIA A100 generation GPUs (or later)

Additional Requirements

  • The system must have an external DNS entry (not relying on /etc/hosts) with the corresponding IP assigned to the host.
  • Internet access from the installation system is a requirement.
  • Swap must be disabled.
  • SELinux must be disabled or set to Permissive.

LLM Requirements

Certara AI utilizes two LLMs that are available for use. The model is determined by the version of Layar you are installing.

  • Layar 1.6 - Utilizes Mistral LLM ~8k tokens
    • VRAM Requirements - 40GB
  • Layar 1.7 - Utilizes Mixtral LLM - 32k tokens
    • Quantized
      • VRAM Requirements - 40GB
    • Unquantized
      • VRAM Requirements - 80GB


Commands should be run as root or prefixed with sudo.

Install docker

yum install -y yum-utils
yum-config-manager \
    --add-repo \
yum -y install docker-ce
systemctl --now enable docker

Increase operating system vm.max_map_count

echo "vm.max_map_count=262144" > /etc/sysctl.d/99-vyasa.conf 
sysctl -p /etc/sysctl.d/99-vyasa.conf

Configure NetworkManager to ignore Calico interfaces

If using NetworkManager, edit the file /etc/NetworkManager/conf.d/calico.conf and set the following:


Restart NetworkManager

systemctl restart NetworkManager

Install kernel development packages and compiler

dnf -y install kernel-devel-$(uname -r) kernel-headers-$(uname -r) gcc

Install and enable the EPEL and CUDA repository

dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo
dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
dnf-config-manager --enable epel

Install CUDA 12

dnf module install nvidia-driver:latest-dkms
dnf install -y cuda-12

Install NVIDIA Docker toolkit

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.repo | tee /etc/yum.repos.d/nvidia-docker.repo
yum install -y nvidia-docker2 nvidia-container-runtime

Set docker default runtime

Edit the file /etc/docker/daemon.json and replace its contents with:

"default-runtime": "nvidia",
"runtimes": {
   "nvidia": {
      "path": "/usr/bin/nvidia-container-runtime",    
      "runtimeArgs": []

Restart docker

systemctl restart docker

Add kubernetes repo

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

Install Kubernetes and initialize

dnf install -y kubeadm-1.23.15-00 kubelet-1.23.15-00 kubectl-1.23.15-00
/usr/bin/kubeadm init --kubernetes-version=1.23.15 --token-ttl 0 --pod-network-cidr= -v 5
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

Install NGINX Ingress controller

kubectl apply -f https://vyasa-static-assets.s3.amazonaws.com/layar/nginx-ingress.yaml

Install Calico CNI

kubectl create -f https://vyasa-static-assets.s3.amazonaws.com/layar/tigera-operator.yaml
kubectl create -f https://vyasa-static-assets.s3.amazonaws.com/layar/custom-resources.yaml

Let head node run pods

/usr/bin/kubectl taint nodes --all node-role.kubernetes.io/master:NoSchedule-

Install local volume provisioner

/usr/bin/kubectl apply -f https://vyasa-static-assets.s3.amazonaws.com/layar/local-storage-provisioner.yml

Set default storage class

kubectl patch storageclass 'local-path' -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Install Helm

wget -P /usr/local/bin/ https://vyasa-static-assets.s3.amazonaws.com/helm/helm
chmod +x /usr/local/bin/helm

Add the Layar helm repository

helm repo add vyasa https://helm.vyasa.com/charts/ --username vyasahelm --password "sail#away()"
helm repo update

Install Layar

Replace MY_URL with the IP address or DNS name of your system and setting TRITON_GPU_COUNT to n-1 of available GPUs.

helm install layar vyasa/layar --set APPURL=MY_URL --set TRITON_GPU_COUNT=MY_GPU_COUNT