autoscale: true build-lists: true code: line-height(1.2) header-emphasis: #53B0EB text: alignment(left) theme: simple, 8 # [fit] **Deploying PHP applications**
using Ansible, Ansible Vault
and Ansistrano ^ I work primarily with PHP, and there will be some PHP-isms in this talk (LAMP stack, Composer). Will be using a Drupal 8 application as the example, but the tools are tool and language agnostic. --- # [fit] **Deploying ~~PHP~~ applications**
using Ansible, Ansible Vault
and Ansistrano --- ## **What we'll be looking at** * **Ansible** crash course * Keeping secrets with **Ansible Vault** * Deployments with **Ansistrano** --- [.build-lists: false] [.header: #111111] ![right 500%](../images/me-phpnw-inviqa.jpg) - Full Stack Software Developer & System Administrator - **Senior Software Engineer** at **Inviqa** - Acquia certified **Drupal 8 Grand Master** and **Cloud Pro** - Open sourcer - Drupal 7 & 8 **core contributor** - @opdavies - www.oliverdavies.uk ^ Maintain Drupal modules, PHP CLI tools and libraries, Ansible roles Blog on my website I work primarily with Drupal and Symfony I work for Inviqa, but this based on my personal and side projects. I've been using Ansible for a number of years, initially only for provisioning and setting up my laptop, and later for application deployments --- ![350% inline](images/logo-platformsh.png) ![150% inline](images/logo-acquia.png) ![130% inline](images/logo-pantheon.png) ^ Large, well-known managed hosting companies Optimised servers for PHP/Drupal applications Include some sort of deployment system This workflow doesn't apply to this scenario --- ![100%](images/logo-digital-ocean.png) ![30%](images/logo-linode.png) ![30%](images/logo-vultr.png) ^ More applicable to virtual or dedicated servers with no existing deployment process Not enough budget for fully-managed, or using internal infrastructure This is where the this workflow would be useful --- # **What is Ansible?** --- ## Ansible is an open-source **software provisioning, configuration management, and application-deployment** tool. ![10% right](images/ansible.png) [.footer: https://en.wikipedia.org/wiki/Ansible_(software)] --- ![10% right](images/ansible.png) ### **What is Ansible?** * CLI tool * Written in Python * Configured with YAML * Executes ad-hoc remote commands * Installs software packages * Performs deployment steps * Batteries included ^ Written in Python but you don't need to write or know Python to use it Drupal, Symfony and a lot of other projects use YAML First-party modules (SSH keys, file and directory management, package repositories, stopping/starting/restarting services, DO/Linode/AWS integration) --- ![10% right](images/ansible.png) ### **What is Ansible?** * Hosts/Inventories * Commands * Playbooks * Tasks * Roles ^ Hosts: where your managed nodes/hosts are. Can be static or dynamic. Commands: run from a control node onto managed nodes Playbooks and Tasks: YAML representation of a series of commands/steps --- ![10% right](images/ansible.png) ### **Why Ansible?** * Familiar syntax * Easily readable * No server dependencies * Easy to add to an existing project * Includes relevant modules (e.g. Composer) * Idempotency ^ Drupal 8, Symfony, Ansible all use YAML Runs on any server with Python Plugins into Drupal via CLI apps like Drush and Drupal Console Changes are only made when needed (once) --- # **Hosts / Inventories** --- ```ini # hosts.ini [webservers] 192.168.33.10 ``` ^ Supports wildcards and ranges. --- ```yaml # hosts.yml webservers: hosts: 192.168.33.10: ``` --- # **Commands** --- # `ansible all -i hosts.yml -m ping` --- ```json webservers | SUCCESS => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": false, "ping": "pong" } ``` --- # `ansible all `
`-i hosts.yml `
`-m command `
`-a 'git pull `
`--chdir=/app'` --- # `ansible all -i hosts.yml`
`-m git -a 'repo=https://github.com/opdavies/dransible dest=/app`' --- # **Tasks and Playbooks** --- ```yaml # playbook.yml --- - hosts: webservers # or 'all' vars: git_repo: https://github.com/opdavies/dransible tasks: - name: Update the code git: repo: '{{ git_repo }}' dest: /app version: master update: true ``` --- # `ansible-playbook `
`playbook.yml -i hosts.yml` --- # **Roles:
configuring a LAMP stack** ^ Collections of tasks, variables and handlers --- ```yaml # requirements.yml --- - src: geerlingguy.apache - src: geerlingguy.composer - src: geerlingguy.mysql - src: geerlingguy.php - src: geerlingguy.php-mysql ``` ^ Provisioning LAMP stack and Composer --- # `ansible-galaxy -r`
`requirements.yml install` --- ```yaml # provision.yml --- - hosts: webservers roles: - geerlingguy.apache - geerlingguy.mysql - geerlingguy.php - geerlingguy.php-mysql - geerlingguy.composer ``` ^ Role order matters! --- ```yaml # provision.yml --- - hosts: webservers # ... vars: apache_vhosts: - servername: dransible.wip documentroot: /app/web ``` --- ```yaml # provision.yml --- - hosts: webservers # ... vars: # ... php_version: '7.4' php_packages_extra: - libapache2-mod-php{{ php_version }} - libpcre3-dev ``` --- ```yaml # provision.yml --- - hosts: webservers # ... vars: # ... mysql_databases: - name: main mysql_users: - name: user password: secret priv: main.*:ALL ``` --- # `ansible-playbook provision.yml -i hosts.yml` --- ``` PLAY [Provision the webserver machines] ******************************************************************************** TASK [Gathering Facts] ************************************************************************************************* ok: [webservers] TASK [geerlingguy.apache : Include OS-specific variables.] ************************************************************* ok: [webservers] TASK [geerlingguy.apache : Include variables for Amazon Linux.] skipping: [webservers] TASK [geerlingguy.apache : Define apache_packages.] ******************************************************************** ok: [webservers] TASK [geerlingguy.apache : include_tasks] ****************************************************************************** included: /Users/opdavies/.ansible/roles/geerlingguy.apache/tasks/setup-Debian.yml for webservers TASK [geerlingguy.apache : Update apt cache.] ************************************************************************** changed: [webservers] ``` --- ``` TASK [geerlingguy.composer : Ensure composer directory exists.] ******************************************************** ok: [webservers] TASK [geerlingguy.composer : include_tasks] **************************************************************************** skipping: [webservers] TASK [geerlingguy.composer : include_tasks] **************************************************************************** skipping: [webservers] RUNNING HANDLER [geerlingguy.apache : restart apache] ****************************************************************** changed: [webservers] RUNNING HANDLER [geerlingguy.mysql : restart mysql] ******************************************************************** changed: [webservers] RUNNING HANDLER [geerlingguy.php : restart webserver] ****************************************************************** changed: [webservers] RUNNING HANDLER [geerlingguy.php : restart php-fpm] ******************************************************************** skipping: [webservers] PLAY RECAP ************************************************************************************************************* webservers : ok=111 changed=32 unreachable=0 failed=0 skipped=78 rescued=0 ignored=0 ``` --- ![70%](images/after-provision-1.png) --- ![70%](images/after-provision-2.png) --- # **Keeping secrets with
Ansible Vault** --- ```yaml # provision.yml --- - hosts: webservers # ... vars: # ... mysql_databases: - name: main mysql_users: - name: user password: secret priv: main.*:ALL ``` --- [.code-highlight: 11-14] ```yaml # provision.yml --- - hosts: webservers # ... vars: # ... mysql_databases: - name: main mysql_users: - name: user password: secret priv: main.*:ALL ``` --- # `ansible-vault create`
`vault.yml` --- ```yaml # vars/vault.yml --- vault_database_name: main vault_database_user: user vault_database_password: secret ``` ^ Optional, but easier to see where variables are set --- ``` $ANSIBLE_VAULT;1.1;AES256 36656233323539616336393838396137343939623233393338666530313 73037323366326363306531336333353163643063663335396139363762 39383133330a35636566623262353733373066363837393264616134613 16366376266646437373366373738303931616333626332353839353332 32663432346662613438330a38643539343232376138613733373636343 63864666430313866623539333039363138646331326565386263386666 35306264396230633939346532356665306564626431353936643135376 23834346635366637613235656165643361316663396530383263333064 33326264316235396431666262346637366563376330363238373331373 43533386165366531626462643662666266316639306262666539373236 343662313265376261316636623963353933613366353737363435 ``` --- ```yaml # vars/vars.yml --- database_name: "{{ vault_database_name }}" database_user: "{{ vault_database_user }}" database_password: "{{ vault_database_password }}" ``` --- ```yaml # provision.yml --- mysql_databases: - '{{ database_name }}' mysql_users: - name: '{{ database_user }}' password: '{{ database_password }}' priv: '{{ database_name }}.*:ALL' ``` --- # `ansible-vault edit vault.yml` --- # `ansible-playbook`
`-i hosts.yml`
`deploy.yml`
`--ask-vault-pass` --- # **Basic deployment** --- ```yaml # deploy.yml tasks: - name: Creating project directory file: path: /app state: directory - name: Uploading application synchronize: src: "{{ playbook_dir }}/../" dest: /app - name: Installing Composer dependencies composer: command: install working_dir: /app ``` --- # Disadvantages * Single point of failure * No ability to roll back * Sensitive data stored in plain text --- # **Better deployments** --- ![full inline](images/ansistrano.png) ^ Just another role, specifically for deployments Ansible port of Capistrano --- # Features * Multiple release directories * Shared paths and files * Customisable * Multiple deployment strategies * Multi-stage environments * Prune old releases * Rollbacks ^ rsync, Git, SVN etc --- ```yaml # requirements.yml --- # ... - ansistrano.deploy - ansistrano.rollback ``` --- ```yaml # deploy.yml --- - hosts: all roles: - ansistrano.deploy ``` --- ```yaml # deploy.yml --- # ... vars: project_deploy_dir: /app ansistrano_deploy_to: '{{ project_deploy_dir }}' ansistrano_deploy_via: git ansistrano_git_branch: master ansistrano_git_repo: 'git@github.com:opdavies/dransible' ``` --- # `ansible-playbook`
`-i hosts.yml`
`deploy.yml` --- ``` PLAY [webservers] ****************************************************************************************************** TASK [Gathering Facts] ************************************************************************************************* ok: [webservers] TASK [ansistrano.deploy : include_tasks] ******************************************************************************* TASK [ansistrano.deploy : include_tasks] ******************************************************************************* included: /Users/opdavies/.ansible/roles/ansistrano.deploy/tasks/setup.yml for webservers TASK [ansistrano.deploy : ANSISTRANO | Ensure deployment base path exists] ********************************************* ok: [webservers] TASK [ansistrano.deploy : ANSISTRANO | Ensure releases folder exists] ************************************************** ok: [webservers] TASK [ansistrano.deploy : ANSISTRANO | Ensure shared elements folder exists] ******************************************* ok: [webservers] TASK [ansistrano.deploy : ANSISTRANO | Ensure shared paths exists] ***************************************************** ok: [webservers] => (item=web/sites/default/files) ``` --- ``` TASK [ansistrano.deploy : Update file permissions] ********************************************************************* changed: [webservers] TASK [ansistrano.deploy : include_tasks] ******************************************************************************* TASK [ansistrano.deploy : include_tasks] ******************************************************************************* included: /Users/opdavies/.ansible/roles/ansistrano.deploy/tasks/cleanup.yml for webservers TASK [ansistrano.deploy : ANSISTRANO | Clean up releases] ************************************************************** changed: [webservers] TASK [ansistrano.deploy : include_tasks] ******************************************************************************* TASK [ansistrano.deploy : include_tasks] ******************************************************************************* included: /Users/opdavies/.ansible/roles/ansistrano.deploy/tasks/anon-stats.yml for webservers TASK [ansistrano.deploy : ANSISTRANO | Send anonymous stats] *********************************************************** skipping: [webservers] PLAY RECAP ************************************************************************************************************* webservers : ok=33 changed=14 unreachable=0 failed=0 skipped=7 rescued=0 ignored=0 ``` --- ```bash vagrant@dransible:/app$ ls -l total 8 lrwxrwxrwx 1 26 Jul 19 00:15 current -> ./releases/20190719001241Z drwxr-xr-x 5 4096 Jul 22 20:30 releases drwxr-xr-x 4 4096 Jul 19 00:00 shared ``` --- ``` vagrant@dransible:/app/releases$ ls -l total 20 drwxr-xr-x 5 4096 Jul 22 20:30 . drwxr-xr-x 4 4096 Jul 19 00:15 .. drwxr-xr-x 10 4096 Jul 19 00:02 20190719000013Z drwxr-xr-x 10 4096 Jul 19 00:14 20190719001241Z drwxr-xr-x 9 4096 Jul 22 20:30 20190722203038Z ``` --- # `ansible-playbook`
`-i hosts.yml`
`rollback.yml` --- ```yaml # rollback.yml --- - hosts: all roles: - ansistrano.rollback vars: ansistrano_deploy_to: '{{ project_deploy_dir }}' ``` --- # **Customising Ansistrano:
Build Hooks** --- ![inline 140%](images/ansistrano-flow.png) ^ Shared = files directory, logs Before/after symlink shared = run tests Symlink = 'current' symlink, site is live Clean up = remove node_modules, database export, sqlite testing DB --- ```yaml # deploy.yml --- # ... ansistrano_after_symlink_shared_tasks_file: '{{ playbook_dir }}/deploy/after-symlink-shared.yml' ansistrano_after_symlink_tasks_file: '{{ playbook_dir }}/deploy/after-symlink.yml' ansistrano_after_update_code_tasks_file: '{{ playbook_dir }}/deploy/after-update-code.yml' release_web_path: '{{ ansistrano_release_path.stdout }}/web' release_drush_path: '{{ ansistrano_release_path.stdout }}/vendor/bin/drush' ``` ^ Each step has a 'before' and 'after' step Ansistrano allows us to add more things by providing a path to a playbook and adding additional steps. --- ```yaml # deploy/after-update-code.yml --- - name: Install Composer dependencies composer: command: install working_dir: '{{ ansistrano_release_path.stdout }}' ``` --- ```yaml # deploy/after-symlink-shared.yml --- - name: Run database updates command: '{{ release_drush_path }} --root {{ release_web_path }} updatedb' ``` --- ```yaml # deploy/after-symlink.yml --- - name: Clear Drupal cache command: '{{ release_drush_path }} --root {{ release_web_path }} cache-rebuild' ``` --- ![80%](images/drupalcon/site.png) --- # **Managing data
across deployments** --- ```yaml # deploy.yml vars: # ... ansistrano_shared_paths: - "{{ drupal_root }}/sites/default/files" ``` --- ``` vagrant@dransible:/app/shared/web/sites/default/files$ ls -la total 28 drwxrwxrwx 6 4096 Jul 19 00:18 . drwxr-xr-x 3 4096 Jul 19 00:00 .. drwxrwxr-x 2 4096 Jul 22 21:24 css -rwxrwxrwx 1 487 Jul 19 00:02 .htaccess drwxrwxr-x 2 4096 Jul 19 00:19 js drwxrwxrwx 3 4096 Jul 19 00:18 php drwxrwxrwx 2 4096 Jul 19 00:03 styles ``` --- ``` vagrant@dransible:/app/current/web/sites/default$ ls -la total 48 dr-xr-xr-x 2 4096 Jul 19 00:14 . drwxr-xr-x 3 4096 Jan 22 17:30 .. -rw-r--r-- 1 6762 Jul 19 00:14 default.services.yml -rw-r--r-- 1 31342 Jul 19 00:14 default.settings.php lrwxrwxrwx 1 45 Jul 19 00:14 files -> ../../../../../shared/web/sites/default/files -rw-r--r-- 1 35 Jul 19 00:12 settings.php ``` --- # **Generating settings
files per deployment** --- ```yaml # vars/vault.yml --- vault_database_name: main vault_database_user: user vault_database_password: secret vault_hash_salt: dfgiy$fd2!34gsf2*34g74 ``` --- ```yaml # vars/vars.yml --- database_name: "{{ vault_database_name }}" database_password: "{{ vault_database_password }}" database_user: "{{ vault_database_user }}" hash_salt: "{{ vault_hash_salt }}" ``` --- ```yaml # vars/vars.yml --- drupal_settings: - drupal_root: /app/web sites: - name: default settings: databases: default: default: driver: mysql host: localhost database: '{{ database_name }}' username: '{{ database_user }}' password: '{{ database_password }}' hash_salt: '{{ hash_salt }}' config_directories: sync: ../config/sync ``` --- ```php // templates/settings.php.j2 // {{ ansible_managed }} {% for key, values in item.1.settings.databases.items() %} {% for target, values in values.items() %} $databases['{{ key }}']['{{ target }}'] = array( 'driver' => '{{ values.driver|default('mysql') }}', 'host' => '{{ values.host|default('localhost') }}', 'database' => '{{ values.database }}', 'username' => '{{ values.username }}', 'password' => '{{ values.password }}', ); {% endfor %} {% endfor %} {% if item.1.settings.base_url is defined %} $base_url = '{{ item.1.settings.base_url }}'; {% endif %} {# ... #} ``` --- ```yaml # tasks/main.yml --- - name: Ensure directory exists file: state: directory path: '{{ item.0.drupal_root }}/sites/{{ item.1.name|default("default") }}' with_subelements: - '{{ drupal_settings }}' - sites no_log: true - name: Create settings files template: src: settings.php.j2 dest: '{{ item.0.drupal_root }}/sites/{{ item.1.name|default("default") }}/{{ item.1.filename|default("settings.php") }}' with_subelements: - '{{ drupal_settings }}' - sites no_log: true ``` --- # **Multiple environments** ## Dev, test, production --- ```yaml mysql_databases: - name: production - name: staging mysql_users: - name: production password: '{{ live_db_password }}' priv: '{{ live_db_name }}.*:ALL' - name: staging password: '{{ staging_db_password }}' priv: staging.*:ALL ``` --- ```yaml # inventories/live.yml --- all: hosts: webservers: ansible_ssh_host: 192.168.33.10 ansible_ssh_port: 22 project_deploy_path: /app git_branch: master drupal_hash_salt: "{{ vault_drupal_hash_salt }}" drupal_install: true drupal_settings: # ... ``` --- ```yaml # inventories/staging.yml --- all: hosts: webservers: ansible_ssh_host: 192.168.33.10 ansible_ssh_port: 22 project_deploy_path: /app-test git_branch: develop drupal_hash_salt: "{{ vault_drupal_hash_salt }}" drupal_install: true drupal_settings: # ... ``` --- # `ansible-playbook deploy.yml -i inventories/staging.yml` --- # `ansible-playbook deploy.yml -i inventories/live.yml` --- # **Questions?**