Comment on page
Deploying on GCP with Terraform

Replicated 3 Node Cluster
Prerequisites
VPC, Subnet, and NAT in GCP, to allow pods to pull images from the Granite repository.
An example can be found at Example GKE setup | Cloud NAT | Google Cloud. Steps 1, 3, 4 & 6 from this link need to be completed. There is no need to perform step 2 because Grainite script creates the kubernetes cluster. Make sure for Step 6 to create a NAT per region in case your cluster and VM are in different regions.
​gcloud CLI​
​Terraform​
zip
​Kubectl​
grepcidr - sudo apt install -y grepcidr
Helm
curl https://baltocdn.com/helm/signing.asc | sudo apt-key add -
sudo apt-get install apt-transport-https --yes
echo "deb https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update
sudo apt-get install helm
Tokens:
Helm deploy token (same as GitLab deploy token)
Helm username
Quay username
Quay password
The following permissions are required for the service account:
Compute Admin
Kubernetes Engine Admin
Service Account User
compute.subnetworks.delete (Compute Network Admin)
Recommendation: All of the steps below should be performed from a linux VM running within the same virtual private cloud as the target cluster.
Download scripts
The scripts package contains scripts that make it easier to deploy and manage Grainite clusters with scripts for creating roles, VPCs, Grainite clusters, etc.
1.Run the following to download the Terraform GCP and CloudFormation AWS scripts package tar:
curl -H "DEPLOY-TOKEN: <token>" https://gitlab.com/api/v4/projects/26443204/packages/generic/scripts/<version>/cluster_creation.tar -o cluster_creation.tar
Replace <token> with the deploy token provided to you.

Also, replace <version> with the desired version of Grainite (e.g. 2316.1) that needs to be deployed or latest for the latest available version of Grainite.
2. Run the following to extract the script package tar:
tar xvf cluster_creation.tar
Initialize Terraform
Initialize Terraform by running the following command under grainite/scripts/package/k8s/tf/gcp:
terraform init
If successful, the following output will be shown:
Terraform has been successfully initialized!
Initialize gcloud CLI
Run the following command to Initialize gcloud by picking a project, region, and authenticating:
gcloud init
Create an unsecured 3 node GKE cluster
Create a 3-node cluster in an existing VPC by running the following script:
./grainite/scripts/bin/gcp-grainite cluster-create -h
​
Creates a new unsecured grainite cluster with the given cluster name
​
Usage:
    cluster-create CLUSTER-NAME [flags]
​
Flags:
    -R:         region
​
    -i:         grainite image name
    -j:         jetty image name
    -r:         image repository URL
    -U:         image repository user name
    -T:         image repository access token
​
    -H:         helm repo URL
    -u:         helm repo user name
    -K:         helm repo access token
    -C:         helm chart version
​
    -S:         num servers in cluster, default: 3
    -n:         create the infrastructure without deploying Grainite workload
​
    -D:         type of persistent disks. default: pd-standard
​
    -m:         meta volume size suffixed with {Gi|Ti}, e.g. 1Ti
    -d:         dat volume size suffixed with {Gi|Ti} , e.g. 1Ti
​
    -N:         subnet CIDR of the VM invoking the deployment
    -P:         project name in gcp
    -v:         vpc name
    -p:         use pre-emptible worker nodes or not, true/false
    -E:         use external LB for service end points with allowed source IP CIDR's, e.g., 20.21.22.0/24(,20.21.23.0/24)
​
    -h|--help   help
NOTE:
Grainite helm chart version : The Grainite release version for the helm chart.
Example: for release 2316.1, specify -C 23.16.1 or for release 2317, specify -C 23.17.0
-m: meta size default value is 1Ti.
-d: dat size default value is 1Ti.
-p: default value is false.
Example (Need to replace username and passwords):
./grainite/scripts/bin/gcp-grainite cluster-create toms-cluster
    -N 172.30.0.0/16 
    -P toms-project   
    -v toms-vpc 
    -R us-west1  
    -i cluster:2316.1  
    -r quay.io/grainite 
    -U QUAY_USERNAME 
    -T QUAY_PASSWORD 
    -H https://gitlab.com/api/v4/projects/26443204/packages/helm/stable 
    -u gitlab+deploy-token-123456 
    -K 4354QeeGqdfdfEt454
    -C 23.16.1
Deploy a cluster with TLS and encryption enabled
Encryption can also be enabled on an existing cluster, see the following page for details:
Enabling Disk Encryption
Optionally, the following script can be used to create a cluster with encryption and TLS enabled directly:
Note: This will not create the client certificates necessary for TLS. To create these, follow Step 2 under Enabling TLS.
./grainite/scripts/bin/gcp-grainite cluster-create-secure -h
​
Creates a new secure grainite cluster with the given name
​
Usage:
    cluster-create-secure CLUSTER-NAME [flags]
​
Env:
    GXS_CERTS_KEYS_DIR:
        Default GXS_CERTS_KEYS_DIR=${HOME}/.gxscerts/
​
        For secure setup of grainite cluster and enabling disk encryption export
        this env. The server CA CERTS and encryption KEYS will be generated under
​
            ${GXS_CERTS_KEYS_DIR}/<cluster-name>/{server/, client/, keys/}
​
Flags:
    -R:         region
    -e:         optional, setup disk encryption also
                keys will be generated under ${GXS_CERTS_KEYS_DIR}/<cluster-name>
​
    -i:         grainite image name
    -j:         jetty image name
    -r          image repository URL
    -U          image repository user name
    -T          image repository access token
​
    -H          helm repo URL
    -u          helm repo user name
    -K          helm repo access token
    -C          helm chart version
​
    -S          num servers in cluster, default 3
​
    -D:         type of persistent disks. default: pd-standard
​
    -m:         meta volume size suffixed with {Gi|Ti}, e.g. 1Ti
    -d:         dat volume size suffixed with {Gi|Ti} , e.g. 1Ti
​
    -N:         subnet CIDR of the VM invoking the deployment
    -P:         project name in gcp
    -v:         vpc name
    -p:         use pre-emptible worker nodes or not, true/false
    -x          service account name for disk encryption with KMS
    -y          key ring name
    -z          key name
    -E:         use external LB for service end points with allowed source IP CIDR's, e.g., 20.21.22.0/24(,20.21.23.0/24)
​
    -h|--help   help
Where:
All flags are the same as those passed in Create a 3 node GKE Cluster except for -e which when passed enables encryption at desk.
The -x, -y and -z options are needed for at-rest encryption using a key provided via Google Cloud KMS. More details here.
Access the Kubernetes cluster
The Kubernetes config is set up as part of cluster creation using prep_cluster.sh. However, you can also switch it, to use another cluster.
Use the following script to list all available clusters:
./grainite/scripts/bin/gcp-grainite cluster-list
Use the following script to connect to an existing cluster:
./grainite/scripts/bin/gcp-grainite cluster-switch CLUSTER-NAME
Use the following script to get the IP address of a cluster after connecting to it:
./grainite/scripts/bin/gcp-grainite cluster-ip 
Use the following script to get the name of the cluster that is being used:
./grainite/scripts/bin/gcp-grainite cluster-current
Destroy the cluster
Running the cluster-delete will clean up the PVCs used in the cluster. To preserve the volumes after cluster deletion, delete the cluster from the GCP console instead.
./grainite/scripts/bin/gcp-grainite cluster-delete CLUSTER-NAME
Deploying on Azure with Terraform
Deploying on VMWare Tanzu
Last modified 4mo ago