Skip to main content

Deployment

Minds deploys with GitLab, making use of Docker & Kubernetes.

Docker Containers​

FPM​

docker build -t minds/fpm:latest -f containers/php-fpm/Dockerfile .

Runners​

docker build -t minds/runners:latest -f containers/php-runners/Dockerfile .

Getting connected to the staging environment​

Prerequisties​

Connecting​

  • aws eks --profile=saml --region us-east-1 update-kubeconfig --name sandbox --role-arn=arn:aws:iam::324044571751:role/minds_eks_k8s_developers
  • export KUBECONFIG=$HOME/.kube/config (optional)

Then you should be able to see all available pods with:

kubectl get pods

Review Apps​

The review apps make use of Helm and Kubernetes. Our helm charts can be found here and you can inspect the review app auto deployment in our .gitlab-ci.yml file.

A kubernetes environment can be created by running:

helm upgrade \
    --install \
    --reuse-values \
    --set cassandra.deploy=true \
    --set elasticsearch.deploy=true \
    --set redis.deploy=true \
    --wait \
    my-minds-stack \
    ./helm-charts/minds

If you wish to use shared databases, due to resource constraints, first deploy your services and then set deploy=false like so:

helm upgrade \
    --install \
    --reuse-values \
    --set cassandra.deploy=false \
    --set elasticsearch.deploy=false \
    --set redis.deploy=false \
    --wait \
    my-mini-minds-stack \
    ./helm-charts/minds

Deploying changes to runners​

To deploy changes to runners, first deploy your changes to a sandbox as normal to generate an image that we can use to update the runners.

To get the image name once deployed, in your command line run:

kubectl describe deployment/{deployment name for changes} | grep Image

In another terminal, run:

kubectl edit deployment/{runner deployment name}

From there, edit the image name in the file to use the relevant image outputted by the first command.

Managing review app settings​

When a pod gets deployed, Helm charts writes in the values by parsing configMap.yaml.

To have your values persist across builds, you must extend the settings.php script in configMap.yaml.

Do not hard code the values in the configMap, reference them via .Values.key.subkey:

 // Twillio configuration
    $CONFIG->set('twilio', [
        'account_sid' => '{{ .Values.twilio.sid }}',
        'auth_token' => '{{ .Values.twilio.token }}',
        'from' => '{{ .Values.twilio.from }}'
    ]);

And add the values to the correponding yaml files:

Why so many values and templating?​

Because it enables us to do something really cool, like dynamically override configuration values for your staging environment - useful for turning on your feature flags.

You can test your local changes and manipulate the staging environments by using the helm-charts repo. Create your branch, make your changes and then, inside your helm-charts repository branch, run:

  helm upgrade --reuse-values --recreate-pods --set new.key='new value' --wait
  {your.staging.site.subdomain} ./minds

So changing an existing configuration value is as simple as:

helm upgrade --install --recreate-pods --reuse-values --set max_video_length=12600 feat-max-video-size-1506 --wait ./minds/

Adding a new value requires a bit more leg work. Create a new branch in helm charts and add your values to configMap.yaml and values.yaml. Then, you can:

helm upgrade --install --recreate-pods --reuse-values feat-max-video-size-1506 --wait ./minds/

We don't need to specify a set value here because the value doesn't exist. However, once you set it, it will continue to be inherited from the last release because of --reuse-values and you'll need to update it with set.

If you hose anything, you can always re-run the pipeline which will rebuild the pods with the latest configuration in master and start over.

Managing Configuration Locally​

Engine supports loading environment variables whenever php-fpm starts.

Our environment variables follow a specific naming scheme. They prefixed with MINDSENV so we don't accidentally load a non Minds value.

Beyond just loading key values, our env parser also supports setting nested arrays. You do this by specifying each array key separated by a double underscore.

Rather than deal with setting environment variables and mapping them in docker compose, engine ships with a .env file. Just add you key value pairs there and they will be loaded with each request.

Note, this file is ignored by gitlab, so your changes won't get committed. 

MINDS_ENV_iframely__key=mykey
MINDS_ENV_payments__stripe_apikey=mystripekey
...

Managing Secrets on Review Sites​

You can manage secrets in a two step process.

Extend helm to load your secrets as environment variables​

Our helm charts have a template called _secrets-env.tpl. This yaml template loads specific secrets as environment variables in the pods.

- name: MINDS_ENV_iframely__key
  valueFrom:
    secretKeyRef:
      name: iframely-key
      key: iframely-key

name is the environment variable name.

So, MINDS_ENV_iframely__key becomes: 

[
  'iframely': [
    key: 'supersecretgoeshere'
  ]
]

Note that single scores after the prefix are treated as keynames.

Setting your secrets in kubernetes​

The name and key of the secretKeyRef are the lower-case kebabed secrets set in kubernetes

Setting a key in kubernetes that then gets shared with everyone connected to that cluster:

kubectl create secret generic iframely-key --from-literal=iframely-key=supersecretgoeshere

Interacting with the staging environment​

You can get access to the pods by using kubectl. Note, pods are read-only and ephemeral, so you can't go hacking things on the container.

Get a list of all pods

kubectl get pods

Shell into a pod using the name from get pods (read only)

kubectl exec -it {your.staging.site.subdomain}-{pod.type}-{kubernetes-suffix} sh

What variables am I actually running?​

Helm charts write settings.php and interpolate whatever values are defined in the values.yml. The secrets loaded via environment values.

You can tell what values are currently loaded for any given Config value by connecting to the php-fpm container and running:

kubectl exec -it {app-pod-name} sh
cd engine
cli.php Config --key={Minds config key}

This will dump out the current value of that config value.