Custom Networking
Pharos allows users to plug-in any CNI networking implementation on the cluster.
Configuring Custom Networking
Custom networking can be enabled by setting the network provider to custom
and configuring the deployment manifest path properly. This will make Pharos to deploy all the needed networking components during the cluster creation.
network:
provider: custom
pod_network_cidr: 172.30.0.0/16
service_cidr: 172.31.0.0/16
custom:
manifest_path: ./network-manifests/
options: {}
Options
Pharos does not define any structure for the options
key, it can have any yaml conformant structure. These options, as well as the full cluster configuration yaml, can be used in the manifest templates.
Manifests
As with addons the manifests can be either normal yaml files or templated with Erb. Templating allows easy customizability of the deployment and the ability to match the deployment configuration with the cluster level configuration options.
Example configuration for Flannel
As an example, let's configure the cluster using Flannel as the networking implementation.
In the cluster configuration, you set the provider into custom
and point Pharos to use correct set of deployment manifests:
network:
provider: custom
pod_network_cidr: 172.30.0.0/16
service_cidr: 172.31.0.0/16
custom:
manifest_path: ./flannel/
There's no configuration options defined as Flannel needs only the pod network CIDR to be configured and that it can get from cluster configuration.
The manifests are told to locate in ./flannel
folder. So in that folder, you need to have all the deployment descriptors available. For a simple example Flannel deployment, you'd have following:
01-cluster-role.yml
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: flannel
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- apiGroups:
- ""
resources:
- nodes
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
02-cluster-role-binding.yml
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: flannel
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: flannel
subjects:
- kind: ServiceAccount
name: flannel
namespace: kube-system
03-service-account.yml
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: flannel
namespace: kube-system
04-config-map.yml.erb
---
kind: ConfigMap
apiVersion: v1
metadata:
name: kube-flannel-cfg
namespace: kube-system
labels:
tier: node
app: flannel
data:
cni-conf.json: |
{
"name": "cbr0",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "<%= cluster_config.network.pod_network_cidr %>",
"Backend": {
"Type": "vxlan"
}
}
Notice how the full cluster configuration is available for the templates using cluster_config
variable. If there's specific options
provided for custom network you'd access them using cluster_config.networking.custom.options.option_name
syntax. In this example we set the pod network CIDR for Flannel.
03-daemon-set.yml
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: kube-flannel-ds
namespace: kube-system
labels:
tier: node
app: flannel
spec:
template:
metadata:
labels:
tier: node
app: flannel
spec:
hostNetwork: true
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay.io/coreos/flannel:v0.10.0-amd64
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.10.0-amd64
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: true
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg