Deployment
Minds deploys with GitLab, making use of Docker & Kubernetes.
Docker Containers
FPM
docker build -t minds/fpm:latest -f containers/php-fpm/Dockerfile .
Runners
docker build -t minds/runners:latest -f containers/php-runners/Dockerfile .
Getting connected to the staging environment
Prerequisties
- Follow the steps at https://developers.minds.com/docs/handbook/security-practices/#cli
- Helm and Kubernetes installed
Connecting
aws eks --profile=saml --region us-east-1 update-kubeconfig --name sandbox --role-arn=arn:aws:iam::324044571751:role/minds_eks_k8s_developers
export KUBECONFIG=$HOME/.kube/config
(optional)
Then you should be able to see all available pods with:
kubectl get pods
Review Apps
The review apps make use of Helm and Kubernetes. Our helm charts can be found here and you can inspect the review app auto deployment in our .gitlab-ci.yml file.
A kubernetes environment can be created by running:
helm upgrade \
--install \
--reuse-values \
--set cassandra.deploy=true \
--set elasticsearch.deploy=true \
--set redis.deploy=true \
--wait \
my-minds-stack \
./helm-charts/minds
If you wish to use shared databases, due to resource constraints, first deploy your services and then set deploy=false
like so:
helm upgrade \
--install \
--reuse-values \
--set cassandra.deploy=false \
--set elasticsearch.deploy=false \
--set redis.deploy=false \
--wait \
my-mini-minds-stack \
./helm-charts/minds
Deploying changes to runners
To deploy changes to runners, first deploy your changes to a sandbox as normal to generate an image that we can use to update the runners.
To get the image name once deployed, in your command line run:
kubectl describe deployment/{deployment name for changes} | grep Image
In another terminal, run:
kubectl edit deployment/{runner deployment name}
From there, edit the image name in the file to use the relevant image outputted by the first command.
Managing review app settings
When a pod gets deployed, Helm charts writes in the values by parsing configMap.yaml.
To have your values persist across builds, you must extend the settings.php script in configMap.yaml.
Do not hard code the values in the configMap, reference them via .Values.key.subkey
:
// Twillio configuration
$CONFIG->set('twilio', [
'account_sid' => '{{ .Values.twilio.sid }}',
'auth_token' => '{{ .Values.twilio.token }}',
'from' => '{{ .Values.twilio.from }}'
]);
And add the values to the correponding yaml files:
Why so many values and templating?
Because it enables us to do something really cool, like dynamically override configuration values for your staging environment - useful for turning on your feature flags.
You can test your local changes and manipulate the staging environments by using the helm-charts repo. Create your branch, make your changes and then, inside your helm-charts repository branch, run:
helm upgrade --reuse-values --recreate-pods --set new.key='new value' --wait
{your.staging.site.subdomain} ./minds
So changing an existing configuration value is as simple as:
helm upgrade --install --recreate-pods --reuse-values --set max_video_length=12600 feat-max-video-size-1506 --wait ./minds/
Adding a new value requires a bit more leg work. Create a new branch in helm charts and add your values to configMap.yaml and values.yaml. Then, you can:
helm upgrade --install --recreate-pods --reuse-values feat-max-video-size-1506 --wait ./minds/
We don't need to specify a set value here because the value doesn't exist. However, once you set it, it will continue to be inherited from the last release because of --reuse-values and you'll need to update it with set.
If you hose anything, you can always re-run the pipeline which will rebuild the pods with the latest configuration in master and start over.
Managing Configuration Locally
Engine supports loading environment variables whenever php-fpm starts.
Our environment variables follow a specific naming scheme. They prefixed with MINDSENV so we don't accidentally load a non Minds value.
Beyond just loading key values, our env parser also supports setting nested arrays. You do this by specifying each array key separated by a double underscore.
Rather than deal with setting environment variables and mapping them in docker compose, engine ships with a .env
file. Just add you key value pairs there and they will be loaded with each request.
Note, this file is ignored by gitlab, so your changes won't get committed.
MINDS_ENV_iframely__key=mykey
MINDS_ENV_payments__stripe_apikey=mystripekey
...
Managing Secrets on Review Sites
You can manage secrets in a two step process.
Extend helm to load your secrets as environment variables
Our helm charts have a template called _secrets-env.tpl. This yaml template loads specific secrets as environment variables in the pods.
- name: MINDS_ENV_iframely__key
valueFrom:
secretKeyRef:
name: iframely-key
key: iframely-key
name is the environment variable name.
So, MINDS_ENV_iframely__key becomes:
[
'iframely': [
key: 'supersecretgoeshere'
]
]
Note that single scores after the prefix are treated as keynames.
Setting your secrets in kubernetes
The name and key of the secretKeyRef are the lower-case kebabed secrets set in kubernetes
Setting a key in kubernetes that then gets shared with everyone connected to that cluster:
kubectl create secret generic iframely-key --from-literal=iframely-key=supersecretgoeshere
Interacting with the staging environment
You can get access to the pods by using kubectl. Note, pods are read-only and ephemeral, so you can't go hacking things on the container.
Get a list of all pods
kubectl get pods
Shell into a pod using the name from get pods
(read only)
kubectl exec -it {your.staging.site.subdomain}-{pod.type}-{kubernetes-suffix} sh
What variables am I actually running?
Helm charts write settings.php and interpolate whatever values are defined in the values.yml. The secrets loaded via environment values.
You can tell what values are currently loaded for any given Config value by connecting to the php-fpm container and running:
kubectl exec -it {app-pod-name} sh
cd engine
cli.php Config --key={Minds config key}
This will dump out the current value of that config value.