Skip to the content.

Home

File Storage Service With Kubernetes (OKE) on Oracle Cloud Infrastructure (OCI)

Design

Implementation

Create File System

Follow the instructions here to create one

Here is my configuration, note file system is being created on same subnet, and hence security list modification not required, otherwise we would have to modify security list

Create StorageClass

Get the OCID of the mount target

Create the storage class that references the mount target

storageClass.yml

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: oci-fss
provisioner: oracle.com/oci-fss
parameters:
  mntTargetId: ocid1.mounttarget.oc1.phx.aaaaaa4np2snz72kobuhqllqojxwiotqnb4c2ylefuzqaaaa
D:\practices\kubernetes>kubectl apply -f storageClass.yml
storageclass.storage.k8s.io "oci-fss" created
 
D:\practices\kubernetes>kubectl get storageclass
NAME            PROVISIONER          AGE
oci (default)   oracle.com/oci       9d
oci-fss         oracle.com/oci-fss   14s
 
D:\practices\kubernetes>

Create a Persistent Volume (PV)

Make a note of ip address and export path

oke-pv.yml

apiVersion: v1
kind: PersistentVolume
metadata:
 name: oke-fsspv
spec:
 storageClassName: oci-fss
 capacity:
  storage: 100Gi
 accessModes:
  - ReadWriteMany
 mountOptions:
  - nosuid
 nfs:
  server: 10.0.2.10
  path: "/oke-file-system"
  readOnly: false
D:\practices\kubernetes>kubectl apply -f oke-pv.yml
persistentvolume "oke-fsspv" created

pv created

D:\practices\kubernetes>kubectl get pv -o wide
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM     STORAGECLASS   REASON    AGE
oke-fsspv   100Gi      RWX            Retain           Available             oci-fss                  37s
 
D:\practices\kubernetes>

Create a Persistent Volume Claim (PVC)

oke-pvc.yml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: oke-fsspvc
spec:
 storageClassName: oci-fss
 accessModes:
 - ReadWriteMany
 resources:
  requests:
    storage: 100Gi
 volumeName: oke-fsspv
D:\practices\kubernetes>kubectl apply -f oke-pvc.yml
persistentvolumeclaim "oke-fsspvc" created

pvc created

D:\practices\kubernetes>kubectl get pvc -o wide
NAME         STATUS    VOLUME      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
oke-fsspvc   Bound     oke-fsspv   100Gi      RWX            oci-fss        21s
 
D:\practices\kubernetes>

Verify that PVC is bound

D:\practices\kubernetes>kubectl get pvc oke-fsspvc
NAME         STATUS    VOLUME      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
oke-fsspvc   Bound     oke-fsspv   100Gi      RWX            oci-fss        1m
 
D:\practices\kubernetes>

Consume the PVC

consume-pvc.yml

apiVersion: v1
kind: Service
metadata:
  name: oke-fss-pvc-svc
spec:
  type: LoadBalancer
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: oke-fss-pvc
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: oke-fss-pvc
  name: oke-fss-pvc
spec:
  selector:
    matchLabels:
      app: oke-fss-pvc
  replicas: 3
  template:
    metadata:
      labels:
        app: oke-fss-pvc
    spec:
      containers:
      - name: oke-fss-pvc 
        image: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nfs-mount
          mountPath: "/usr/share/nginx/html/"
      volumes:
      - name: nfs-mount
        persistentVolumeClaim:
          claimName: oke-fsspvc
          readOnly: false
D:\practices\kubernetes>kubectl apply -f consume-pvc.yml
service "oke-fss-pvc-svc" created
deployment.apps "oke-fss-pvc" created
 
D:\practices\kubernetes>
D:\practices\kubernetes>kubectl get pods
NAME                               READY     STATUS    RESTARTS   AGE
oke-fss-pvc-6fdf5c767b-4g6gx       1/1       Running   0          23s
oke-fss-pvc-6fdf5c767b-npsdl       1/1       Running   0          23s
oke-fss-pvc-6fdf5c767b-wrwwf       1/1       Running   0          23s
D:\practices\kubernetes>kubectl get services
NAME                  TYPE           CLUSTER-IP      EXTERNAL-IP                                    PORT(S)        AGE
oke-fss-pvc-svc       LoadBalancer   10.96.149.248   129.146.147.113                                80:30647/TCP   32s

Testing

no index.html file as of now

Lets connect to all the pods and write some content from each pods.

Lets write from pod oke-fss-pvc-6fdf5c767b-4g6gx

D:\practices\kubernetes>kubectl exec -it oke-fss-pvc-6fdf5c767b-4g6gx bash
root@oke-fss-pvc-6fdf5c767b-4g6gx:/# cd /usr/share/nginx/html/
root@oke-fss-pvc-6fdf5c767b-4g6gx:/usr/share/nginx/html# echo oke-fss-pvc-6fdf5c767b-4g6gx >> index.html
root@oke-fss-pvc-6fdf5c767b-4g6gx:/usr/share/nginx/html#

able to see the content written

Lets write some content from pod oke-fss-pvc-6fdf5c767b-npsdl

D:\practices\kubernetes>kubectl exec -it oke-fss-pvc-6fdf5c767b-npsdl bash
root@oke-fss-pvc-6fdf5c767b-npsdl:/# cd /usr/share/nginx/html/
root@oke-fss-pvc-6fdf5c767b-npsdl:/usr/share/nginx/html# echo oke-fss-pvc-6fdf5c767b-npsdl >> index.html
root@oke-fss-pvc-6fdf5c767b-npsdl:/usr/share/nginx/html#

able to see the content written

Lets write some content from pod oke-fss-pvc-6fdf5c767b-wrwwf

D:\practices\kubernetes>kubectl exec -it oke-fss-pvc-6fdf5c767b-wrwwf bash
root@oke-fss-pvc-6fdf5c767b-wrwwf:/# cd /usr/share/nginx/html/
root@oke-fss-pvc-6fdf5c767b-wrwwf:/usr/share/nginx/html# echo <br/> oke-fss-pvc-6fdf5c767b-wrwwf >> index.html
bash: br/: No such file or directory
root@oke-fss-pvc-6fdf5c767b-wrwwf:/usr/share/nginx/html# echo "<br/> oke-fss-pvc-6fdf5c767b-wrwwf" >> index.html
root@oke-fss-pvc-6fdf5c767b-wrwwf:/usr/share/nginx/html#

able to see the content written

Lets delete all the pods

D:\practices\kubernetes>kubectl delete pods --all
pod "oke-fss-pvc-6fdf5c767b-4g6gx" deleted
pod "oke-fss-pvc-6fdf5c767b-npsdl" deleted
pod "oke-fss-pvc-6fdf5c767b-wrwwf" deleted
D:\practices\kubernetes>kubectl get pods
NAME                               READY     STATUS              RESTARTS   AGE
oke-fss-pvc-6fdf5c767b-6676n       1/1       Running             0          29s
oke-fss-pvc-6fdf5c767b-cqhwz       0/1       ContainerCreating   0          30s
oke-fss-pvc-6fdf5c767b-gmgq8       1/1       Running             0          28s
 
D:\practices\kubernetes>

Data is persisted

Also See