Add Drupal Somerset slides

deploying-drupal-fabric/2017-10-26-drupal-somerset/slides.md

@@ -0,0 +1,907 @@
+autoscale: true
+build-lists: true
+theme: next, 9
+
+# [fit] Deploying Drupal <br>with Fabric
+### Oliver Davies
+### bit.ly/deploying-drupal-fabric
+
+---
+
+[.build-lists: false]
+
+- What is Fabric and what do I use it for?
+- How to write and organise Fabric scripts
+- Task examples
+
+---
+
+[.build-lists: false]
+
+![right](../../me-phpnw.png)
+
+- Senior Developer at Microserve
+- Part-time freelance Developer & System Administrator
+- Drupal Bristol, PHPSW, DrupalCamp Bristol organiser
+- Sticker collector, herder of elePHPants
+- @opdavies
+- oliverdavies.uk
+
+
+^ Drupal, Symfony, Silex, Laravel, Sculpin
+Not a Python Developer
+
+---
+
+![](images/fry.jpeg)
+
+## [fit] What is <br>Fabric?
+
+---
+
+## What is Fabric?
+
+Fabric is a Python (2.5-2.7) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.
+
+---
+
+## What is Fabric?
+
+It provides a basic suite of operations for executing local or remote shell commands (normally or via sudo) and uploading/downloading files, as well as auxiliary functionality such as prompting the running user for input, or aborting execution.
+
+---
+
+## I use Fabric to...
+
+- Simplify my build process
+- Deploy code directly to different environments
+- Act as an intermediate step
+
+---
+
+![fit](images/files.png)
+
+---
+
+[.build-lists: false]
+
+## Why Fabric?
+
+- Powerful
+- Flexible
+- Easier to read and write than bash
+
+^ - Can be used for different languages, frameworks
+- Can be used for small, simple deployments or large, complicated ones
+- Not very opinioned
+
+---
+
+## Installing Fabric
+
+```bash
+$ pip install fabric
+
+# macOS
+$ brew install fabric
+
+# Debian, Ubuntu
+$ apt-get install fabric
+$ apt-get install python-fabric
+```
+
+---
+
+## [fit] Writing your <br>first fabfile
+
+---
+
+```python
+# fabfile.py
+
+from fabric.api import env, run, cd, local
+
+env.hosts = ['example.com']
+
+# Do stuff...
+```
+
+^ Callables
+
+---
+
+```python
+# fabfile.py
+
+from fabric.api import *
+
+env.hosts = ['example.com']
+
+# Do stuff...
+```
+
+---
+
+## Operations
+
+- cd, lcd - *change directory*
+- run, sudo, local - *run a command*
+- get - *download files*
+- put - *upload files*
+
+[.footer: http://docs.fabfile.org/en/1.13/api/core/operations.html]
+
+---
+
+## Utils
+
+- warn: *print warning message*
+- abort: *abort execution, exit with error status*
+- error: *call func with given error message*
+- puts: *alias for print whose output is managed by Fabric's output controls*
+
+[.footer: http://docs.fabfile.org/en/1.13/api/core/utils.html]
+
+---
+
+## File management
+
+```python
+from fabric.contrib.files import *
+```
+
+- exists - *check if path exists*
+- contains - *check if file contains text/matches regex*
+- sed - *run search and replace on a file*
+- upload_template - *render and upload a template to remote host*
+
+[.footer: http://docs.fabfile.org/en/1.13/api/contrib/files.html#fabric.contrib.files.append]
+
+^ Allows for jinja2 templates
+
+---
+
+## Tasks
+
+```python
+
+def main():
+  with cd('/var/www/html'):
+    run('git pull')
+    run('composer install')
+```
+
+---
+
+## Task arguments
+
+```python, [.highlight: 1, 4-6]
+def main(run_composer=True):
+  with cd('/var/www/html'):
+    run('git pull')
+
+    if run_composer:
+      run('composer install')
+```
+
+---
+
+## Task arguments
+
+```python, [.highlight: 1, 4-15]
+def main(run_composer=True, env='prod', build_type):
+  with cd('/var/www/html'):
+    run('git pull')
+
+    if run_composer:
+      if env == 'prod':
+        run('composer install --no-dev')
+      else:
+        run('composer install')
+
+      if build_type == 'drupal':
+        ...
+      elif build_type == 'symfony':
+        ...
+      elif build_type == 'sculpin':
+        ...
+```
+
+---
+
+## Calling other tasks
+
+```python, [.highlight: 4-15]
+@task
+def main():
+  with cd('/var/www/html'):
+    build()
+    post_install()
+
+def build():
+  run('git pull')
+  run('composer install')
+
+def post_install():
+  with prefix('drush'):
+    run('updatedb -y')
+    run('entity-updates -y')
+    run('cache-rebuild')
+```
+
+^ Better organised code
+Not everything in one long
+Easier to read and comprehend
+Single responsibilty principle
+
+---
+
+## Running Tasks
+
+```bash
+fab --list
+
+fab <task>
+
+fab <task>:build_number=$BUILD_ID,build_type=drupal
+```
+
+^ fabfile.py in the current directory is found automatically.
+
+---
+
+```
+[production] Executing task 'main'
+[production] run: git pull
+[production] out: Already up-to-date.
+[production] out:
+
+[production] run: composer install
+...
+[production] out: Generating autoload files
+[production] out:
+
+Done.
+Disconnecting from production... done.
+```
+
+---
+
+## Downsides
+
+- Running build tasks on production
+
+---
+
+[.build-lists: false]
+
+## Not Building on Prod
+
+1. Build locally and deploy.
+
+---
+
+## Local tasks
+
+```python
+# Runs remotely.
+
+from fabric.api import run
+
+run('git pull')
+run('composer install')
+
+# Runs locally.
+
+from fabric.api import local
+
+local('git pull')
+local('composer install')
+```
+
+---
+
+## Local tasks
+
+```python
+# Remote.
+
+from fabric.api import cd
+
+with cd('themes/custom/drupalbristol'):
+  ...
+
+# Runs locally.
+
+from fabric.api import lcd
+
+with lcd('themes/custom/drupalbristol'):
+  ...
+```
+
+---
+
+## rsync
+
+```python, [.highlight: 1, 5-11]
+from fabric.contrib.project import rsync_project
+
+...
+
+def deploy():
+  rsync_project(
+    local_dir='./',
+    remote_dir='/var/www/html'
+    default_opts='-vzcrSLh',
+    exclude=('.git', 'node_modules/', '.sass-cache/')
+  )
+```
+
+---
+
+```
+[production] Executing task 'main'
+[localhost] local: git pull
+Current branch master is up to date.
+[localhost] local: composer install
+Loading composer repositories with package information
+Installing dependencies (including require-dev) from lock file
+Nothing to install or update
+Generating autoload files
+
+Done.
+```
+
+^ - The risky steps have been run separate to the live code.
+- Any issues will not affect the live site.
+
+---
+
+[.build-lists: false]
+
+## Not Building on Prod
+
+1. ~~Build locally and deploy.~~
+1. Build in a separate directory and switch after build.
+
+^ Capistrano
+
+---
+
+## Deploying into a different directory
+
+```python
+from fabric.api import *
+from time import time
+
+project_dir = '/var/www/html'
+next_release = "%(time).0f" % { 'time': time() } # Current timestamp
+
+def init():
+  if not exists(project_dir):
+    run('mkdir -p %s/backups' % project_dir)
+    run('mkdir -p %s/shared' % project_dir)
+    run('mkdir -p %s/releases' % project_dir)
+```
+
+---
+
+## Deploying into a different directory
+
+```python
+current_release = '%s/%s' % (releases_dir, next_release)
+
+run('git clone %s %s' % (git_repo, current_release))
+
+def build():
+  with cd(current_release):
+    pre_tasks()
+    build_site()
+    post_tasks()
+```
+
+^ - Clone the repository into a different directory
+- Run any "risky" tasks away from the production code
+
+---
+
+## Deploying into a different directory
+
+```python
+def pre_build(build_number):
+  with cd('current'):
+    print '==> Dumping the DB (just in case)...'
+    backup_database()
+
+def backup_database():
+  cd('drush sql-dump --gzip > ../backups/%s.sql.gz' % build_number)
+```
+
+---
+
+## Deploying into a different directory
+
+```python
+def update_symlinks():
+  run('ln -nfs %s/releases/%s %s/current'
+    % (project_dir, next_release, project_dir))
+
+# /var/www/html/current
+```
+
+---
+
+```
+[production] Executing task 'main'
+[production] run: git clone https://github.com/opdavies/oliverdavies.uk.git
+  /var/www/html/releases/1505865600
+Installing Composer dependencies...
+[production] run: composer install --no-dev
+Update the symlink to the new release...
+[production] run: ln -nfs /var/www/html/releases/1505865600
+  /var/www/html/current
+
+Done.
+```
+
+---
+
+```bash
+# /var/www/html
+
+shared # settings.local.php, sites.php, files etc.
+releases/1502323200
+releases/1505692800
+releases/1505696400
+releases/1505865600
+current -> releases/1505865600 # symlink
+```
+
+---
+
+[.build-lists: false]
+
+## Positives
+
+- Errors happen away from production
+
+## Downsides
+
+- Lots of release directories
+
+^ If the build fails, then your live site is not affected.
+
+---
+
+## Removing old builds
+
+```python
+def main(builds_to_keep=3):
+  with cd('%s/releases' % project_dir):
+    run("ls -1tr | head -n -%d | xargs -d '\\n' rm -fr"
+      % builds_to_keep)
+```
+
+^ - Find directory names
+- 1tr not ltr
+- Remove the directories
+- Additional tasks, removing DB etc
+
+---
+
+
+
+## [fit] Is the site <br>still running?
+
+---
+
+## Checking for failures
+
+```python
+run(command).return_code == 0:
+  # Pass
+
+run(command).return_code == 1:
+  # Fail
+
+run(command).failed:
+  # Fail
+```
+
+^ Works for local and remote.
+
+---
+
+```python
+print 'Checking the site is alive...'
+if run('drush status | egrep "Connected|Successful"').failed:
+  # Revert back to previous build.
+```
+
+^ egrep is an acronym for "Extended Global Regular Expressions Print". It is a program which scans a specified file line by line, returning lines that contain a pattern matching a given regular expression.
+
+---
+
+```bash, [.highlight 3]
+$ drush status
+
+Drupal version                  :  8.3.7
+Site URI                        :  http://default
+Database driver                 :  mysql
+Database hostname               :  db
+Database username               :  user
+Database name                   :  default
+Database                        :  Connected
+Drupal bootstrap                :  Successful
+Drupal user                     :
+Default theme                   :  bartik
+Administration theme            :  seven
+PHP configuration               :  /etc/php5/cli/php.ini
+...
+```
+
+^ Successful
+
+---
+
+```bash, [.highlight 3]
+$ drush status
+
+Drupal version                  :  8.3.7
+Site URI                        :  http://default
+Database driver                 :  mysql
+Database hostname               :  db
+Database username               :  user
+Database name                   :  default
+PHP configuration               :  /etc/php5/cli/php.ini
+...
+```
+
+^ Failed.
+"Database" to "PHP configuration" missing if cannot connect or DB is empty.
+
+---
+
+## [fit] Does the code still <br>merge cleanly?
+
+^ Pre-task
+
+---
+
+```python
+def check_for_merge_conflicts(target_branch):
+  with settings(warn_only=True):
+    print('Ensuring that this can be merged into the main branch.')
+
+    if local('git fetch && git merge --no-ff origin/%s'
+      % target_branch).failed:
+      abort('Cannot merge into target branch.')
+```
+
+^ Define target branch in Jenkins/when Fabric is run.
+
+---
+
+![](images/grumpy.jpg)
+
+## [fit] Do our tests <br>still pass?
+
+[.footer: http://nashvillephp.org/images/testing-workshop-banner-1600x800.jpg]
+
+---
+
+```python
+with settings(warn_only=True):
+  with lcd('%s/docroot/core' % project_dir):
+    if local('../../vendor/bin/phpunit ../modules/custom').failed:
+      abort('Tests failed!')
+```
+
+---
+
+```
+[localhost] run: ../../vendor/bin/phpunit ../modules/custom
+[localhost] out: PHPUnit 4.8.35 by Sebastian Bergmann and contributors.
+[localhost] out:
+[localhost] out: .......
+[localhost] out:
+[localhost] out: Time: 1.59 minutes, Memory: 6.00MB
+[localhost] out:
+[localhost] out: OK (7 tests, 42 assertions)
+[localhost] out:
+
+
+Done.
+```
+
+---
+
+```
+[localhost] run: ../../vendor/bin/phpunit ../modules/custom
+[localhost] out: PHPUnit 4.8.35 by Sebastian Bergmann and contributors.
+[localhost] out:
+[localhost] out: E
+[localhost] out:
+[localhost] out: Time: 18.67 seconds, Memory: 6.00MB
+[localhost] out:
+[localhost] out: There was 1 error:
+[localhost] out:
+[localhost] out: 1) Drupal\Tests\broadbean\Functional\AddJobTest::testNodesAreCreated
+[localhost] out: Behat\Mink\Exception\ExpectationException: Current response status code is 200, but 201 expected.
+[localhost] out:
+[localhost] out: /var/www/html/vendor/behat/mink/src/WebAssert.php:770
+[localhost] out: /var/www/html/vendor/behat/mink/src/WebAssert.php:130
+[localhost] out: /var/www/html/docroot/modules/custom/broadbean/tests/src/Functional/AddJobTest.php:66
+[localhost] out:
+[localhost] out: FAILURES!
+[localhost] out: Tests: 1, Assertions: 6, Errors: 1.
+[localhost] out:
+
+Warning: run() received nonzero return code 2 while executing '../../vendor/bin/phpunit ../modules/custom/broadbean'!
+
+Fatal error: Tests failed!
+
+Aborting.
+```
+
+---
+
+![](images/homer-smart.png)
+
+## [fit] Making Fabric <br>Smarter
+
+---
+
+## Conditional variables
+
+```python
+drupal_version = None
+
+if exists('composer.json') and exists('core'):
+  drupal_version = 8
+else:
+  drupal_version = 7
+```
+
+---
+
+## Conditional tasks
+
+```python
+if exists('composer.json'):
+  run('composer install')
+
+with cd('themes/custom/example'):
+  if exists('package.json') and not exists('node_modules'):
+    run('yarn --pure-lockfile')
+
+  if exists('gulpfile.js'):
+    run('node_modules/.bin/gulp --production')
+  elif exists('gruntfile.js'):
+    run('node_modules/.bin/grunt build')
+```
+
+---
+
+## Project settings file
+
+```yml
+# app.yml
+
+drupal:
+  version: 8
+  root: web
+  config:
+    import: yes
+    name: sync
+    cmi_tools: no
+  tests:
+    simpletest: false
+    phpunit: true
+  theme:
+    path: 'themes/custom/drupalbristol'
+    build:
+      type: gulp
+      npm: no
+      yarn: yes
+
+composer:
+  install: true
+```
+
+---
+
+## Project settings file
+
+```python, [.highlight 3]
+# fabfile.py
+
+from fabric.api import *
+import yaml
+
+config = []
+
+if exists('app.yml'):
+  with open('app.yml', 'r') as file:
+    config = yaml.load(file.read())
+```
+
+---
+
+## Project settings file
+
+```python
+# fabfile.py
+
+if config['composer']['install'] == True:
+  local('composer install')
+```
+
+---
+
+## Project settings file
+
+```python
+# fabfile.py
+
+if build_type == 'drupal':
+  drupal = config['drupal']
+```
+
+---
+
+## Project settings file
+
+```python, [.highlight: 6-9]
+# fabfile.py
+
+if build_type == 'drupal':
+  drupal = config['drupal']
+
+  with cd(drupal['root']):
+    if drupal['version'] == 8:
+
+    if drupal['version'] == 7:
+```
+
+---
+
+## Project settings file
+
+```python, [.highlight: 8-10]
+# fabfile.py
+
+if build_type == 'drupal':
+  drupal = config['drupal']
+
+  with cd(drupal['root']):
+    if drupal['version'] == 8:
+      if drupal['config']['import'] == True:
+        # Import the staged configuration.
+        run('drush cim -y %s' % drupal['config']['name'])
+```
+
+---
+
+## Project settings file
+
+```python, [.highlight: 9-15]
+# fabfile.py
+
+if build_type == 'drupal':
+  drupal = config['drupal']
+
+  with cd(drupal['root']):
+    if drupal['version'] == 8:
+      if drupal['config']['import'] == True:
+        if drupal['config']['cmi_tools'] == True:
+          # Use Drush CMI Tools.
+          run('drush cimy -y %s' % drupal['config']['name'])
+        else:
+          # Use core.
+          run('drush cim -y %s' % drupal['config']['name'])
+```
+
+^ - Less hard-coded values
+- More flexible
+- No need to use different files for different versions or frameworks
+- No forked fabfiles per project or lots of conditionals based on the project
+
+---
+
+## Project settings file
+
+```python
+# fabfile.py
+
+theme = config['theme']
+
+with cd(theme['path']):
+  if theme['build']['gulp'] == True:
+    if env == 'prod':
+      run('node_modules/.bin/gulp --production')
+    else:
+      run('node_modules/.bin/gulp')
+```
+
+---
+
+## Project settings file v2
+
+```yml
+# app.yml
+
+commands:
+  build: |
+    cd web/themes/custom/drupalbristol
+    yarn --pure-lockfile
+    npm run prod
+
+  deploy: |
+    cd web
+    drush updatedb -y
+    drush cache-rebuild -y
+```
+
+^ How do we know when to run Composer or npm?
+How do we know what commands to run?
+Can't tell this just by the presence of files.
+
+---
+
+## Project settings file v2
+
+```python
+# fabfile.py
+
+# Run build commands locally.
+for hook in config['commands'].get('build', '').split("\n"):
+  run(hook)
+
+...
+
+# Run deploy commands remotely.
+for hook in config['commands'].get('deploy', '').split("\n"):
+  run(hook)
+```
+
+^ "run" won't work locally
+
+---
+
+## Other things
+
+- Run Drush commands
+- Run automated tests
+- Verify file permissions
+- Restart services
+- Anything you can do on the command line...
+
+---
+
+[.build-lists: false]
+
+## Fabric has...
+
+- Simplified my build process
+- Made my build process more flexible
+- Made my build process more robust
+
+---
+
+[.build-lists: false]
+
+- https://www.oliverdavies.uk/talks/deploying-drupal-fabric
+- http://fabfile.org
+- https://github.com/opdavies/fabric-example-drupal
+- https://github.com/opdavies/fabric-example-sculpin
+- https://deploy.serversforhackers.com (~~$129~~ $79)
+
+---
+
+## Thanks!
+
+# Questions?
+
+### @opdavies
+### oliverdavies.uk