Quobyte runs on any public clouds, and Kubernetes. You can install it on your Kubernetes cluster just like any other application, and can start provisioning persistent volumes with scalable, high performance storage in minutes. Its distributed file system provides shared read-write-many (RWX) persistent volumes with scalable high performance storage for any workload.
Quobyte runs on Amazon Elastic Kubernetes Services (EKS), as well as other clouds and also your own hardware. This allows you to have the same storage system anywhere, making it easy to migrate workloads, meaning you get a true multi-cloud and hybrid cloud system.
In addition, Quobyte offers you a scale-out shared file system with high performance for workloads like AI/ML, VFX, 3D rendering, image analysis or traditional HPC.
In this article, we will walk you through the installation of Quobyte and the Quobyte CSI plugin on a EKS cluster. And within just a few minutes, you will be able to provision persistent volumes to your k8s clusters with Quobyte.
Quobyte Installation on EKS Cluster
We will install Quobyte using a Helm chart, but first, you need to add the Quobyte repo to your environment.
helm repo add quobyte https://quobyte.github.io/quobyte-k8s-resources/helm-charts
Before moving on to the installation, we need to make a few changes to the values.yaml file found in the quobyte-cluster directory. You can view this file as follow:
helm show values quobyte/quobyte-cluster
The last few lines of the file would look like this:
Let’s generate a file called myvalues.yaml from the values.yaml which we will be able to edit using vi:
helm show values quobyte/quobyte > myvalues.yaml
vi myvalues.yaml
By default, the helm chart has the parameters set to install Quobyte on Google Kubernetes Engine (GKE), so we will need to change that. The parameters that you need to change are storageProvisioner and flashStorage. Make sure to uncomment the parameters for an EKS installation:
storageProvisioner:“kubernetes.io/aws-ebs”
flashStorage:“gp2”
Also, make sure to comment out the parameters used for GKE:
storageProvisioner:“kubernetes.io/gce-pd”
flashStorage:“pd-ssd”
Your file should look like this:
Save the changes and let’s proceed with the installation
helm install my-storage-cluster quobyte/quobyte-cluster -f myvalues.yaml --wait=true
Note that this starts the core Quobyte services in containers in your k8s cluster.
On bare metal servers, Quobyte uses the disks (HDD, NVMes) to store data, on VMs it’s block storage (like EBS). In a bare metal k8s cluster, Quobyte services can use the disks in the servers. On VM based clusters – like EKS – we need block storage and Quobyte requests it through persistent volume requests.
Later, Quobyte provides persistent volumes itself – but these are scale-out share file system volumes vs. the block storage persistent volumes provided by the cloud platform.
Let’s take a look at the persistent volume claims that were created by the services:
kubectl get pvc -A
You will be able to see the PVCs for the data, metadata, and registry services. Please note that you can adjust the type and size of storage using the values.yaml file that we previously generated.
Next, we will set up the Quobyte webconsole account. Let’s retrieve the IP address:
kubectl get svc/quobyte-webui
Copy the external IP address, paste it into a new browser window and add port 8080 to it. The Quobyte webconsole should load with a welcome message. After you click “START” you will need to agree to the Quobyte Terms and Conditions, and set up an admin account.
Please note that you will need to add your credentials base64 encoded to the 02_secret_quobyte.yaml file under quobyte-k8s-resources>usage-examples>01_getting_started.
You can also configure which hosts should be trusted to run Quobyte services. Let’s retrieve the IP address:
kubectl get pods -o wide
Copy the IP address and change the last number to zero. For the subnet you can choose 16, which will take half of the network.
You can skip the SMTP configuration, and the license step. However, if you have a license, you can add it here. If you would like to get a free license visit the Quobyte website.
You now have a Quobyte cluster up and running.
The next step to access the Quobyte cluster from Kubernetes is to deploy the quobyte-client and quobyte-csi. The Quobyte client runs on all worker nodes on your k8s cluster; it automatically mounts new volumes and makes them available to the host via a bi-directional volume mapping. On the other hand, the Quobyte CSI is responsible for fulfilling all the PVCs that have a StorageClass configured to use Quobyte. For PVCs that have a size, the Quobyte CSI plugin also creates a Quobyte volume quota.
helm install my-storage-client quobyte/quobyte-client
helm install my-storage-csi quobyte/quobyte-csi
Provisioning Persistent Volumes With Quobyte
Next, we will do a quick example on how to provision a persistent volume, and mount it on a container deployed as a single pod.
For the next part, you will need to access quobyte-k8s-resources repo and cd into the usage_examples>01_getting_started directory:
cd quobyte-k8s-resources/usage-examples/01_getting_started
Add your credentials to the 01_secret_quobyte.yaml. Please remember that your credentials need to be base64 encoded.
Next, apply the examples declarations
kubectl apply -f 01_namespace-quobyte.yaml -f 02_secret_quobyte.yaml -f 03_storageClass.yaml -f 04-testpvc.yaml -f 05_testpod.yaml
These declarations will create the following:
- Quobyte namespace
- The credentials to authenticate against Quobyte
- A Quobyte default storage class to enable access to the Quobyte cluster
- A persistent volume claim that provisions volumes
- A example pod deployment of a container that uses that volume
Now, we can check if the PVC was successfully created
kubectl get pvc quobyte-default-pvc
We can get more details of our PVC:
kubectl describe pvc quobyte-default-pvc
Next, we can check if the persistent volume was successfully mounted inside a container:
kubectl exec -it consumer-pod -- mount | grep quobyte
Lastly, we can check if the Quobyte mount is readable and writable from within the container:
kubectl exec -it consumer-pod -- /bin/bash
cd /usr/share/nginx/html/
Now, let’s write “Hello World!” to an html file:
echo "Hello World!" > index.html
And lastly, read the content of the index.html file:
cat index.html
As you can see, you can now provision scalable persistent storage with Quobyte to any application on your EKS cluster.
