Content translation/Apertium/Deployment

This document describes deployment procedure Apertium in production, which is almost similar to cxserver now.

Testing

Note image tag version from Gerrit patchset to be deploy. For eg: 2020-12-01-140153-production.

Run:

docker pull docker-registry.wikimedia.org/wikimedia/mediawiki-services-apertium:2020-12-01-140153-production

to get Apertium docker image, which can be tested locally.

TODO: Add: How to test it with cxserver.

Apertium on Production config stays in: helmfile.d/services/apertium/values.yaml

Once list of packages is changed, we need to do deploy new image in production. Config change can be directly deployed without updating image.

Clone deployment-charts repository (for first time).
Do needful changes in config (update image or other configuration changes as needed).
Make a CR (Example: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/623475) and after a successful review, merge it.
After merge, log in in a deployment server (eg: deploy1001), there is a cron (1 minute) that will update the /srv/deployment-charts directory with the contents from git.
Go to /srv/deployment-charts/helmfile.d/services/apertium.
Execute: helmfile -e ${CLUSTER} diff This will show the changes that it will be applied on the cluster.
Execute: helmfile -e ${CLUSTER} -i apply This will materialize the previous diff in the cluster and also will log into SAL the change.

This is done using helmfile:

Change directory to /srv/deployment-charts/helmfile.d/services/${CLUSTER}/apertium on a deployment server
Unless you are mid un-applied changes the current values files should reflect the deployed values
You can check for un-applied changes with: helmfile -e ${CLUSTER} diff
You can see the status with helmfile -e ${CLUSTER} status

Logs can be access from deploy1001 if needed:

cd /srv/deployment-charts/helmfile.d/services/${CLUSTER}/apertium

source .hfenv

kubectl logs apertium-production-8649f66884-znfgl

apertium-production-8649f66884-znfgl is pod name.

To see all logs:

kubectl logs -l app=apertium -c apertium-production

If you need to roll back a change because something went wrong:

Revert the git commit to the deployment-charts repo
Merge the revert (with review if needed)
Wait one minute for the cron job to pull the change to the deployment server
Change directory to /srv/deployment-charts/helmfile.d/services/apertium
Execute helmfile -e ${CLUSTER} diff to see what you'll be changing.
Execute helmfile -e ${CLUSTER} -i apply where CLUSTER is one of (staging, eqiad, codfw).