diff --git a/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/files.png b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/files.png new file mode 100644 index 0000000..a23b62a Binary files /dev/null and b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/files.png differ diff --git a/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/fry.jpeg b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/fry.jpeg new file mode 100644 index 0000000..1e9fe70 Binary files /dev/null and b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/fry.jpeg differ diff --git a/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/grumpy.jpg b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/grumpy.jpg new file mode 100644 index 0000000..f44fa63 Binary files /dev/null and b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/grumpy.jpg differ diff --git a/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/homer-smart.png b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/homer-smart.png new file mode 100644 index 0000000..50c5705 Binary files /dev/null and b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/images/homer-smart.png differ diff --git a/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/slides.md b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/slides.md new file mode 100644 index 0000000..de79570 --- /dev/null +++ b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/slides.md @@ -0,0 +1,893 @@ +autoscale: true +build-lists: true +theme: next, 9 + +# [fit] Deploying Drupal
with Fabric + +### Oliver Davies +### bit.ly/deploying-drupal-fabric + +--- + +[.build-lists: false] + +- What is Fabric and what do I use it for? +- How to write and organise Fabric scripts +- Task examples + +--- + +[.build-lists: false] + +![right](../../me-phpnw.png) + +- Senior Developer at Microserve +- Part-time freelance Developer & System Administrator +- Drupal, Symfony, Silex, Laravel, Sculpin +- Drupal Bristol, PHPSW, DrupalCamp Bristol +- Sticker collector, herder of elePHPants +- @opdavies +- oliverdavies.uk + + +^ Not a Python Developer + +--- + +![](images/fry.jpeg) + +## [fit] What is
Fabric? + +--- + +## What is Fabric? + +Fabric is a Python (2.5-2.7) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks. + +--- + +## What is Fabric? + +It provides a basic suite of operations for executing local or remote shell commands (normally or via sudo) and uploading/downloading files, as well as auxiliary functionality such as prompting the running user for input, or aborting execution. + +--- + +## I use Fabric to... + +- Simplify my build process +- Deploy code directly to different environments +- Act as an intermediate step + +--- + +![fit](images/files.png) + +--- + +[.build-lists: false] + +## Why Fabric? + +- Powerful +- Flexible +- Easier to read and write than bash + +^ - Can be used for different languages, frameworks +- Can be used for small, simple deployments or large, complicated ones +- Not very opinioned + +--- + +## Installing Fabric + +```bash +$ pip install fabric + +# macOS +$ brew install fabric + +# Debian, Ubuntu +$ apt-get install fabric +$ apt-get install python-fabric +``` + +--- + +## [fit] Writing your
first fabfile + +--- + +```python +# fabfile.py + +from fabric.api import env, run, cd, local + +env.hosts = ['example.com'] + +# Do stuff... +``` + +^ Callables + +--- + +```python +# fabfile.py + +from fabric.api import * + +env.hosts = ['example.com'] + +# Do stuff... +``` + +--- + +## Operations + +- cd, lcd - *change directory* +- run, sudo, local - *run a command* +- get - *download files* +- put - *upload files* + +[.footer: http://docs.fabfile.org/en/1.13/api/core/operations.html] + +--- + +## Utils + +- warn: *print warning message* +- abort: *abort execution, exit with error status* +- error: *call func with given error message* +- puts: *alias for print whose output is managed by Fabric's output controls* + +[.footer: http://docs.fabfile.org/en/1.13/api/core/utils.html] + +--- + +## File management + +```python +from fabric.contrib.files import * +``` + +- exists - *check if path exists* +- contains - *check if file contains text/matches regex* +- sed - *run search and replace on a file* +- upload_template - *render and upload a template to remote host* + +[.footer: http://docs.fabfile.org/en/1.13/api/contrib/files.html#fabric.contrib.files.append] + +^ Allows for jinja2 templates + +--- + +## Tasks + +```python + +def main(): + with cd('/var/www/html'): + run('git pull') + run('composer install') +``` + +--- + +## Task arguments + +```python, [.highlight: 1, 4-6] +def main(run_composer=True): + with cd('/var/www/html'): + run('git pull') + + if run_composer: + run('composer install') +``` + +--- + +## Task arguments + +```python, [.highlight: 1, 4-15] +def main(run_composer=True, env='prod', build_type): + with cd('/var/www/html'): + run('git pull') + + if run_composer: + if env == 'prod': + run('composer install --no-dev') + else: + run('composer install') + + if build_type == 'drupal': + ... + elif build_type == 'symfony': + ... + elif build_type == 'sculpin': + ... +``` + +--- + +## Calling other tasks + +```python, [.highlight: 4-15] +@task +def main(): + with cd('/var/www/html'): + build() + post_install() + +def build(): + run('git pull') + run('composer install') + +def post_install(): + with prefix('drush'): + run('updatedb -y') + run('entity-updates -y') + run('cache-rebuild') +``` + +^ Better organised code +Not everything in one long +Easier to read and comprehend +Single responsibilty principle + +--- + +## Running Tasks + +```bash +fab --list + +fab + +fab :build_number=$BUILD_ID,build_type=drupal +``` + +^ fabfile.py in the current directory is found automatically. + +--- + +``` +[production] Executing task 'main' +[production] run: git pull +[production] out: Already up-to-date. +[production] out: + +[production] run: composer install +... +[production] out: Generating autoload files +[production] out: + +Done. +Disconnecting from production... done. +``` + +--- + +## Downsides + +- Running build tasks on production + +--- + +[.build-lists: false] + +## Not Building on Prod + +1. Build locally and deploy. + +--- + +## Local tasks + +```python +# Runs remotely. + +from fabric.api import run + +run('git pull') +run('composer install') + +# Runs locally. + +from fabric.api import local + +local('git pull') +local('composer install') +``` + +--- + +## Local tasks + +```python +# Remote. + +from fabric.api import cd + +with cd('themes/custom/drupalbristol'): + ... + +# Runs locally. + +from fabric.api import lcd + +with lcd('themes/custom/drupalbristol'): + ... +``` + +--- + +## rsync + +```python, [.highlight: 1, 5-11] +from fabric.contrib.project import rsync_project + +... + +def deploy(): + rsync_project( + local_dir='./', + remote_dir='/var/www/html' + default_opts='-vzcrSLh', + exclude=('.git', 'node_modules/', '.sass-cache/') + ) +``` + +--- + +``` +[production] Executing task 'main' +[localhost] local: git pull +Current branch master is up to date. +[localhost] local: composer install +Loading composer repositories with package information +Installing dependencies (including require-dev) from lock file +Nothing to install or update +Generating autoload files + +Done. +``` + +^ - The risky steps have been run separate to the live code. +- Any issues will not affect the live site. + +--- + +[.build-lists: false] + +## Not Building on Prod + +1. ~~Build locally and deploy.~~ +1. Build in a separate directory and switch after build. + +^ Capistrano + +--- + +## Deploying into a different directory + +```python +from fabric.api import * +from time import time + +project_dir = '/var/www/html' +next_release = "%(time).0f" % { 'time': time() } # Current timestamp + +def init(): + if not exists(project_dir): + run('mkdir -p %s/backups' % project_dir) + run('mkdir -p %s/shared' % project_dir) + run('mkdir -p %s/releases' % project_dir) +``` + +--- + +## Deploying into a different directory + +```python +current_release = '%s/%s' % (releases_dir, next_release) + +run('git clone %s %s' % (git_repo, current_release)) + +def build(): + with cd(current_release): + pre_tasks() + build() + post_tasks() +``` + +^ - Clone the repository into a different directory +- Run any "risky" tasks away from the production code + +--- + +## Deploying into a different directory + +```python +def pre_build(build_number): + with cd('current'): + print '==> Dumping the DB (just in case)...' + backup_database() + +def backup_database(): + cd('drush sql-dump --gzip > ../backups/%s.sql.gz' % build_number) +``` + +--- + +## Deploying into a different directory + +```python +def update_symlinks(): + run('ln -nfs %s/releases/%s %s/current' + % (project_dir, next_release, project_dir)) + +# /var/www/html/current +``` + +--- + +``` +[production] Executing task 'main' +[production] run: git clone https://github.com/opdavies/oliverdavies.uk.git + /var/www/html/releases/1505865600 +Installing Composer dependencies... +[production] run: composer install --no-dev +Update the symlink to the new release... +[production] run: ln -nfs /var/www/html/releases/1505865600 + /var/www/html/current + +Done. +``` + +--- + +```bash +# /var/www/html + +shared # settings.local.php, sites.php, files etc. +releases/1502323200 +releases/1505692800 +releases/1505696400 +releases/1505865600 +current -> releases/1505865600 # symlink +``` + +--- + +## Positives + +- Errors happen away from production + +## Downsides + +- Lots of release directories + +--- + +## Removing old builds + +```python +def main(builds_to_keep=3): + with cd('%s/releases' % project_dir): + run("ls -1tr | head -n -%d | xargs -d '\\n' rm -fr" + % builds_to_keep) +``` + +^ - Find directory names +- 1tr not ltr +- Remove the directories +- Additional tasks, removing DB etc + +--- + +## [fit] Does the code still
merge cleanly? + +^ Pre-task + +--- + +```python +def check_for_merge_conflicts(target_branch): + with settings(warn_only=True): + print('Ensuring that this can be merged into the main branch.') + + if local('git fetch && git merge --no-ff origin/%s' + % target_branch).failed: + abort('Cannot merge into target branch.') +``` + +^ Define target branch in Jenkins/when Fabric is run. + +--- + +![](images/grumpy.jpg) + +## [fit] Do our tests
still pass? + +[.footer: http://nashvillephp.org/images/testing-workshop-banner-1600x800.jpg] + +--- + +```python +with settings(warn_only=True): + with lcd('%s/docroot/core' % project_dir): + if local('../../vendor/bin/phpunit ../modules/custom').failed: + abort('Tests failed!') +``` + +--- + +``` +[localhost] run: ../../vendor/bin/phpunit ../modules/custom +[localhost] out: PHPUnit 4.8.35 by Sebastian Bergmann and contributors. +[localhost] out: +[localhost] out: ....... +[localhost] out: +[localhost] out: Time: 1.59 minutes, Memory: 6.00MB +[localhost] out: +[localhost] out: OK (7 tests, 42 assertions) +[localhost] out: + + +Done. +``` + +--- + +``` +[localhost] run: ../../vendor/bin/phpunit ../modules/custom +[localhost] out: PHPUnit 4.8.35 by Sebastian Bergmann and contributors. +[localhost] out: +[localhost] out: E +[localhost] out: +[localhost] out: Time: 18.67 seconds, Memory: 6.00MB +[localhost] out: +[localhost] out: There was 1 error: +[localhost] out: +[localhost] out: 1) Drupal\Tests\broadbean\Functional\AddJobTest::testNodesAreCreated +[localhost] out: Behat\Mink\Exception\ExpectationException: Current response status code is 200, but 201 expected. +[localhost] out: +[localhost] out: /var/www/html/vendor/behat/mink/src/WebAssert.php:770 +[localhost] out: /var/www/html/vendor/behat/mink/src/WebAssert.php:130 +[localhost] out: /var/www/html/docroot/modules/custom/broadbean/tests/src/Functional/AddJobTest.php:66 +[localhost] out: +[localhost] out: FAILURES! +[localhost] out: Tests: 1, Assertions: 6, Errors: 1. +[localhost] out: + +Warning: run() received nonzero return code 2 while executing '../../vendor/bin/phpunit ../modules/custom/broadbean'! + +Fatal error: Tests failed! + +Aborting. +``` + +--- + +## [fit] Is the site
still running? + +--- + +## Checking for failures + +```python +run(command).failed: + # Fail + +run(command).return_code == 1: + # Fail + +run(command).return_code == 0: + # Pass +``` + +^ Works for local and remote. + +--- + +```python +print 'Checking the site is alive...' +if run('drush status | egrep "Connected|Successful"').failed: + # Revert back to previous build. +``` + +^ egrep is an acronym for "Extended Global Regular Expressions Print". It is a program which scans a specified file line by line, returning lines that contain a pattern matching a given regular expression. + +--- + +```bash, [.highlight 3] +$ drush status + +Drupal version : 8.3.7 +Site URI : http://default +Database driver : mysql +Database hostname : db +Database username : user +Database name : default +Database : Connected +Drupal bootstrap : Successful +Drupal user : +Default theme : bartik +Administration theme : seven +PHP configuration : /etc/php5/cli/php.ini +... +``` + +^ Successful + +--- + +```bash, [.highlight 3] +$ drush status + +Drupal version : 8.3.7 +Site URI : http://default +Database driver : mysql +Database hostname : db +Database username : user +Database name : default +PHP configuration : /etc/php5/cli/php.ini +... +``` + +^ Failed. +"Database" to "PHP configuration" missing if cannot connect or DB is empty. + +--- + +![](images/homer-smart.png) + +## [fit] Making Fabric
Smarter + +--- + +## Conditional variables + +```python +drupal_version = None + +if exists('composer.json') and exists('core'): + drupal_version = 8 +else: + drupal_version = 7 +``` + +--- + +## Conditional tasks + +```python +if exists('composer.json'): + run('composer install') + +with cd('themes/custom/example'): + if exists('package.json') and not exists('node_modules'): + run('yarn --pure-lockfile') + + if exists('gulpfile.js'): + run('node_modules/.bin/gulp --production') + elif exists('gruntfile.js'): + run('node_modules/.bin/grunt build') +``` + +--- + +## Project settings file + +```yml +# app.yml + +drupal: + version: 8 + root: web + config: + import: yes + name: sync + cmi_tools: no + tests: + simpletest: false + phpunit: true + theme: + path: 'themes/custom/drupalbristol' + build: + type: gulp + npm: no + yarn: yes + +composer: + install: true +``` + +--- + +## Project settings file + +```python, [.highlight 3] +# fabfile.py + +from fabric.api import * +import yaml + +config = [] + +if exists('app.yml'): + with open('app.yml', 'r') as file: + config = yaml.load(file.read()) +``` + +--- + +## Project settings file + +```python +# fabfile.py + +if config['composer']['install'] == True: + local('composer install') +``` + +--- + +## Project settings file + +```python +# fabfile.py + +if build_type == 'drupal': + drupal = config['drupal'] +``` + +--- + +## Project settings file + +```python, [.highlight: 6-9] +# fabfile.py + +if build_type == 'drupal': + drupal = config['drupal'] + + with cd(drupal['root']): + if drupal['version'] == 8: + + if drupal['version'] == 7: +``` + +--- + +## Project settings file + +```python, [.highlight: 8-10] +# fabfile.py + +if build_type == 'drupal': + drupal = config['drupal'] + + with cd(drupal['root']): + if drupal['version'] == 8: + if drupal['config']['import'] == True: + # Import the staged configuration. + run('drush cim -y %s' % drupal['config']['name']) +``` + +--- + +## Project settings file + +```python, [.highlight: 9-15] +# fabfile.py + +if build_type == 'drupal': + drupal = config['drupal'] + + with cd(drupal['root']): + if drupal['version'] == 8: + if drupal['config']['import'] == True: + if drupal['config']['cmi_tools'] == True: + # Use Drush CMI Tools. + run('drush cimy -y %s' % drupal['config']['name']) + else: + # Use core. + run('drush cim -y %s' % drupal['config']['name']) +``` + +^ - Less hard-coded values +- More flexible +- No need to use different files for different versions or frameworks +- No forked fabfiles per project or lots of conditionals based on the project + +--- + +## Project settings file + +```python +# fabfile.py + +theme = config['theme'] + +with cd(theme['path']): + if theme['build']['gulp'] == True: + if env == 'prod': + run('node_modules/.bin/gulp --production') + else: + run('node_modules/.bin/gulp') +``` + +--- + +## Project settings file v2 + +```yml +# app.yml + +commands: + build: | + cd web/themes/custom/drupalbristol + yarn --pure-lockfile + node_modules/.bin/gulp --production + + deploy: | + cd web + drush cache-rebuild -y +``` + +--- + +## Project settings file v2 + +```python +# fabfile.py + +for hook in config['commands'].get('build', '').split("\n"): + run(hook) + +... + +for hook in config['commands'].get('deploy', '').split("\n"): + run(hook) +``` + +--- + +## Other things + +- Run Drush commands +- Run automated tests +- Verify file permissions +- Restart services +- Anything you can do on the command line... + +--- + +[.build-lists: false] + +## Fabric has... + +- Simplified my build process +- Made my build process more flexible +- Made my build process more robust + +--- + +[.build-lists: false] + +- https://www.oliverdavies.uk/talks/deploying-drupal-fabric +- http://fabfile.org +- https://github.com/opdavies/fabric-example-drupal +- https://github.com/opdavies/fabric-example-sculpin +- https://deploy.serversforhackers.com (~~$129~~ $79) + +--- + +## Thanks! + +# Questions? + +### @opdavies +### oliverdavies.uk diff --git a/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/slides.pdf b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/slides.pdf new file mode 100644 index 0000000..2271ee8 Binary files /dev/null and b/deploying-drupal-fabric/2017-10-20-drupalcamp-dublin/slides.pdf differ diff --git a/me-phpnw.png b/me-phpnw.png new file mode 100644 index 0000000..dfd59af Binary files /dev/null and b/me-phpnw.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/appnovation.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/appnovation.png new file mode 100644 index 0000000..49e432c Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/appnovation.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/collection-class-1.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/collection-class-1.png new file mode 100644 index 0000000..3c1d004 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/collection-class-1.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/collection-class-2.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/collection-class-2.png new file mode 100644 index 0000000..09be34e Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/collection-class-2.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/dcbristol.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/dcbristol.png new file mode 100644 index 0000000..552904f Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/dcbristol.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/deploy-all-the-things.jpg b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/deploy-all-the-things.jpg new file mode 100644 index 0000000..f90028e Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/deploy-all-the-things.jpg differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/files.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/files.png new file mode 100644 index 0000000..a23b62a Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/files.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/homer-smart.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/homer-smart.png new file mode 100644 index 0000000..50c5705 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/homer-smart.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/me.jpg b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/me.jpg new file mode 100644 index 0000000..4b3e031 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/me.jpg differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/phpunit.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/phpunit.png new file mode 100644 index 0000000..d22c405 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/phpunit.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-1.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-1.png new file mode 100644 index 0000000..368de76 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-1.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-2.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-2.png new file mode 100644 index 0000000..f823167 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-2.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-3.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-3.png new file mode 100644 index 0000000..c413758 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-3.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-4.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-4.png new file mode 100644 index 0000000..fcce824 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest-4.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest.png new file mode 100644 index 0000000..76c497c Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/simpletest.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/tdd-circle-of-life.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/tdd-circle-of-life.png new file mode 100644 index 0000000..78e5f34 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/tdd-circle-of-life.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/timmillwood-ono.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/timmillwood-ono.png new file mode 100644 index 0000000..be4eda4 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/timmillwood-ono.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/title.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/title.png new file mode 100644 index 0000000..3110bf9 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/title.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-1.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-1.png new file mode 100644 index 0000000..8378788 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-1.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-2.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-2.png new file mode 100644 index 0000000..4112014 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-2.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-3.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-3.png new file mode 100644 index 0000000..8339887 Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-3.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-button.png b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-button.png new file mode 100644 index 0000000..7f8db5f Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/images/toggle-optional-fields-button.png differ diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/slides.md b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/slides.md new file mode 100644 index 0000000..654585f --- /dev/null +++ b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/slides.md @@ -0,0 +1,1120 @@ +autoscale: true +build-lists: true +theme: next, 9 + +![](images/title.png) + +# [fit] TDD - Test
Driven Drupal + +--- + +- PHP code +- Mixture of D7 and D8 +- SimpleTest (D7) +- PHPUnit (D8) + +--- + +[.build-lists: false] + +- Senior Developer at Microserve +- Contrib module maintainer +- Occasional core contributor +- Sticker collector and elePHPant herder +- @opdavies +- oliverdavies.uk + +![right](../../me-phpnw.png) + +--- + +![inline fit](images/timmillwood-ono.png) + + +^ First experience of testing with a real module. +Used on 11,046 sites (84 D5, 7,094 D6, 3,868 D7). +Currently used on 28,398 (10 D5, 2,207 D6, 23,206 D7, 2,975 D8). +Tests crucial to preventing regressions when adding new features or fixing bugs. + +--- + +## Why Test? + +- Catch bugs earlier +- Piece of mind +- Prevent regressions +- Write less code +- Documentation +- Drupal core requirement - +- More important with regular D8 releases + +^ Dave Liddament talk - better and cheaper to catch bugs earlier (e.g. whilst developing rather than after it's been released) +Refer to tests when writing implementation code +ONO merge conflict + +--- + +## Why Not Test? + +- Don't know how +- No time/budget to write tests + +^ "I'd love to write tests, but I don't have the time to learn." + +--- + +## Core Testing Gate + +New features should be accompanied by automated tests. + +If the feature does not have an implementation, provide a test implementation. + +Bug fixes should be accompanied by changes to a test (either modifying an existing test case or adding a new one) that demonstrate the bug. + +[.footer: https://www.drupal.org/core/gates#testing] + +--- + +## Testing in Drupal - SimpleTest + +- Based on +- In D7 core +- `*.test` files +- All test classes in one file + +--- + +## Testing in Drupal - PHPUnit + +- Used in other PHP projects (e.g. Symfony, Laravel) +- In D8 core, but not default +- `*.php` files +- One test class per file + +--- + +## The PHPUnit Initiative + +- +- D8 core tests to change to PHPUnit +- Deprecate SimpleTest, remove in D9 +- "A big chunk of old tests" converted on Feb 21st + +--- + +## The PHPUnit Initiative + +As part of the PHPUnit initiative __a considerable part of Simpletests will be converted to PHPUnit based browser tests on February 21st 2017__. A backwards compatibility layer has been implemented so that many Simpletests can be converted by just using the new BrowserTestBase base class and moving the test file. There is also a script to automatically convert test files in the conversion issue. + +__Developers are encouraged to use BrowserTestBase instead of Simpletest as of Drupal 8.3.0__, but both test systems are fully supported during the Drupal 8 release cycle. + +The timeline for the deprecation of Simpletest's WebTestBase is under discussion. + +[.footer: https://groups.drupal.org/node/516229] + +--- + +## Types of Tests +### Unit Tests + +- `UnitTestCase` +- Tests PHP logic +- No database interaction +- Fast to run + +--- + +## Types of Tests +### Unit Tests + +Pros: + +- Verify individual parts +- Quickly find problems in code +- Fast execution +- No system setup for the test run + +--- + +## Types of Tests +### Unit Tests + +Cons: + +- Rewrite on every refactoring +- Complicated mocking +- No guarantee that the whole system actually works + +--- + +## Types of Tests +### Kernel Tests + +- Kernel tests are integration tests that test on components. You can install modules. +- `KernelTestBase` + +[.footer: https://www.drupal.org/docs/8/testing/types-of-tests-in-drupal-8] + +--- + +## Types of Tests +### Kernel Tests + +Pros: + +- Verify that components actually work together +- Somewhat easy to locate bugs + +--- + +## Types of Tests +### Kernel Tests + +Cons: + +- Slower execution +- System setup required +- No guarantee that end user features actually work + +--- + + +## Types of Tests +### Web/Functional/FunctionalJavascript Tests + +- `DrupalWebTestCase` (D7) +- `WebTestBase`, `BrowserTestBase`, `JavascriptTestBase` (D8) +- Tests functionality +- Interacts with database +- Slower to run +- With/without JavaScript (D8) + +^ - Use JavascriptTestBase when you need to test how the system works for a user with Javascript enabled. + +--- + +## Test Driven Development (TDD) + +- Write a test, see it fail +- Write code until test passes +- Repeat +- Refactor when tests are green + +![right 100%](images/tdd-circle-of-life.png) + +[.footer: http://www.agilenutshell.com/assets/test-driven-development/tdd-circle-of-life.png] + +^ "Grab for green." +Not the only way +Write code beforehand and test afterwards +Write code first, comment out/reset branch, then TDD + +--- + +## Porting Modules to Drupal 8 + +- Make a new branch + `git checkout --orphan 8.x-1.x` +- Add/update the tests +- Write code to make the tests pass +- Refactor +- Repeat + +--- + +## Writing Tests (SimpleTest) + +--- + +[.hide-footer] + +```ini +# example.info + +name = Example +core = 7.x +files[] = example.test +``` + +--- + +[.hide-footer] + +```php +// example.test + +class ExampleTestCase extends DrupalWebTestCase { + + public static function getInfo() { + return array( + 'name' => 'Example tests', + 'description' => 'Web tests for the example module.', + 'group' => 'Example', + ); + } + +} +``` + +--- + +[.hide-footer] + +```php +class ExampleTestCase extends DrupalWebTestCase { + + ... + + public function testSomething { + $this->assertTrue(TRUE); + } + +} +``` + +--- + +## Writing Tests (PHPUnit) + +- No need to load test classes expicitly. +- Add classes into `tests/src` directory. +- Extend `BrowserTestBase`. +- No `getInfo` method. + +^ Classes get autoloaded PSR-4 + +--- + +[.hide-footer] + +## Creating the World + +```php +public function setUp() { + // Enable any other required modules. + parent::setUp(['foo', 'bar']); + + // Anything else we need to do. +} +``` + +--- + +[.hide-footer] + +## Creating the World + +```php +$this->drupalCreateUser(); + +$this->drupalLogin(); + +$this->drupalCreateNode(); + +$this->drupalLogout(); +``` + +--- + +## Assertions + +- `assertTrue` +- `assertFalse` +- `assertNull` +- `assertNotNull` +- `assertEqual` + `assertEquals` + +--- + +## Assertions + +- `assertRaw` +- `assertResponse` + `assertSession()->statusCodeEquals()` +- `assertField` +- `assertFieldById` +- `assertTitle` + +--- + +## [fit] Running Tests + +--- + +## SimpleTest UI + +--- + +![fit](images/simpletest-1.png) + +--- + +![fit](images/simpletest-2.png) + +--- + +![fit](images/simpletest-3.png) + +--- + +![fit](images/simpletest-4.png) + +--- + +## Running SimpleTest From The Command Line + +[.hide-footer] + +```bash +# Drupal 7 +$ php scripts/run-tests.sh + +# Drupal 8 +$ php core/scripts/run-tests.sh +``` + +--- + +[.hide-footer] + +## Running SimpleTest From The Command Line + +```bash +--color + +--verbose + +--all + +--module + +--class + +--file +``` + +--- + +[.hide-footer] + +## Running PHPUnit From The Command Line + +```bash +$ vendor/bin/phpunit + +$ vendor/bin/phpunit [directory] + +$ vendor/bin/phpunit --filter [method] +``` + +--- + +## Example: Collection Class + +--- + +## Collection Class + +- +- Adds a `Collection` class, based on Laravel’s +- Provides helper methods for array methods +- Drupal 7, uses xautoload + +^ xautoload gives PSR-4 namespaces and autoloading similar to Drupal 8. + +--- + +[.hide-footer] + +```php +$collection = collect([1, 2, 3, 4, 5]); + +// Returns all items. +$collection->all(); + +// Counts the number of items. +$collection->count(); + +// Returns the array keys. +$collection->keys(); +``` + +--- + +[.hide-footer] + +```php +namespace Drupal\collection_class; + +class Collection implements \Countable, \IteratorAggregate { + private $items; + + public function __construct($items = array()) { + $this->items = is_array($items) ? $items + : $this->getArrayableItems($items); + } + + public function __toString() { + return $this->toJson(); + } + + ... +``` + +--- + +[.hide-footer] + +```php +public function all() { + return $this->items; +} + +public function count() { + return count($this->items); +} + + +public function isEmpty() { + return empty($this->items); +} + +public function first() { + return array_shift($this->items); +} +``` + +--- + +[.hide-footer] + +## Testing + +```php +public function setUp() { + $this->firstCollection = collect(['foo', 'bar', 'baz']); + + $this->secondCollection = collect([ + array('title' => 'Foo', 'status' => 1), + array('title' => 'Bar', 'status' => 0), + array('title' => 'Baz', 'status' => 1) + ]); + + parent::setUp(); +} +``` + +--- + +[.hide-footer] + +## Testing + +```php +public function testCollectFunction() { + $this->assertEqual( + get_class($this->firstCollection), + 'Drupal\collection_class\Collection' + ); +} +``` + +--- + +[.hide-footer] + +## Testing + +```php +public function testAll() { + $this->assertEqual( + array('foo', 'bar', 'baz'), + $this->firstCollection->all() + ); +} +``` + +--- + +[.hide-footer] + +## Testing + +```php +public function testCount() { + $this->assertEqual( + 3, + $this->firstCollection->count() + ); +} +``` + +--- + +[.hide-footer] + +## Testing + +```php +public function testMerge() { + $first = collect(array('a', 'b', 'c')); + $second = collect(array('d', 'e', 'f')); + + $this->assertEqual( + array('a', 'b', 'c', 'd', 'e', 'f'), + $first->merge($second)->all() + ); +} +``` + +--- + +![fit](images/collection-class-1.png) + +--- + +![fit](images/collection-class-2.png) + +--- + +[.hide-footer] + +## Example: Toggle Optional Fields + +--- + +## Toggle Optional Fields + +- +- Adds a button to toggle optional fields on node forms using form alters +- Possible to override using an custom alter hook +- Uses unit and web tests + +![right 85%](images/toggle-optional-fields-button.png) + +--- + +[.hide-footer] + +## Example + +```php +// Looping through available form elements... + +// Only affect fields. +if (!toggle_optional_fields_element_is_field($element_name)) { + return; +} + +$element = &$form[$element_name]; + +if (isset($overridden_fields[$element_name])) { + return $element['#access'] = $overridden_fields[$element_name]; +} + +// If the field is not required, disallow access to hide it. +if (isset($element[LANGUAGE_NONE][0]['#required'])) { + return $element['#access'] = !empty($element[LANGUAGE_NONE][0]['#required']); +} +``` + +--- + +## What to Test? + +- **Functional:** Are the correct fields shown and hidden? +- **Unit:** Is the field name check returning correct results? + +--- + +## Unit Tests + +[.hide-footer] + +```php +// Returns TRUE or FALSE to indicate if this is a field. + +function toggle_optional_fields_element_is_field($name) { + if (in_array($name, array('body', 'language'))) { + return TRUE; + } + + return substr($name, 0, 6) == 'field_'; +} +``` + +--- + +[.hide-footer] + +## Unit Tests + +```php +$this->assertTrue( + toggle_optional_fields_element_is_field('field_tags') +); + +$this->assertTrue( + toggle_optional_fields_element_is_field('body') +); + +$this->assertFalse( + toggle_optional_fields_element_is_field('title') +); +``` + +--- + +![fit](images/toggle-optional-fields-1.png) + +--- + +[.hide-footer] + +## Web Tests + +```php +public function setUp() { + parent::setUp(); + + $this->drupalLogin( + $this->drupalCreateUser(array( + 'create article content', + 'create page content' + )); + ); + + // Enable toggling on article node forms. + variable_set('toggle_optional_fields_node_types', array('article')); + + $this->refreshVariables(); +} +``` + +--- + +[.hide-footer] + +## Custom Assertions + +```php +private function assertTagsFieldNotHidden() { + $this->assertFieldByName( + 'field_tags[und]', + NULL, + t('Tags field visible.') + ); +} +``` + +--- + +## Testing Hidden Fields + +[.hide-footer] + +```php +public function testFieldsHiddenByDefault() { + variable_set('toggle_optional_fields_hide_by_default', TRUE); + + $this->refreshVariables(); + + $this->drupalGet('node/add/article'); + + $this->assertShowOptionalFieldsButtonFound(); + $this->assertHideOptionalFieldsButtonNotFound(); + $this->assertTagsFieldHidden(); + + ... +``` + +--- + +[.hide-footer] + +## Testing Hidden Fields + +```php + ... + + $this->drupalPost( + 'node/add/article', + array(), + t('Show optional fields') + ); + + $this->assertHideOptionalFieldsButtonFound(); + $this->assertShowOptionalFieldsButtonNotFound(); + $this->assertTagsFieldNotHidden(); +} +``` + +--- + +![fit](images/toggle-optional-fields-2.png) + +--- + +![fit](images/toggle-optional-fields-3.png) + +--- + +## [fit] Building a new
D8 module
with TDD + +--- + +As a site visitor + +I want to see a list of all published pages at `/pages` + +Ordered alphabetically by title. + +--- + +```yml +# dublintest.yml + +name: DrupalCamp Dublin test +core: 8.x +type: module +``` + +--- + +```php +// tests/src/Functional/ListingPageTest.php + +class ListingPageTest extends BrowserTestBase { + + protected static $modules = ['dublintest']; + + public function testListingPageExists() { + $this->drupalGet('pages'); + + $this->assertSession()->statusCodeEquals(200); + } +} +``` + +--- + +``` +docker@cli:/var/www/core$ ../vendor/bin/phpunit ../modules/dublintest/tests +PHPUnit 4.8.36 by Sebastian Bergmann and contributors. + +Testing ../modules/dublintest/tests/ +E + +Time: 25.94 seconds, Memory: 6.00MB + +There was 1 error: + +1) PageListTest::testListingPage +Behat\Mink\Exception\ExpectationException: Current response status code is 404, +but 200 expected. + +/var/www/vendor/behat/mink/src/WebAssert.php:770 +/var/www/vendor/behat/mink/src/WebAssert.php:130 +/var/www/modules/dublintest/tests/src/PageListTest.php:11 +``` + +--- + +- Add the view. +- Copy the config into `config/install`. + +--- + +``` +docker@cli:/var/www/core$ ../vendor/bin/phpunit ../modules/dublintest/tests +PHPUnit 4.8.36 by Sebastian Bergmann and contributors. + +Testing ../modules/dublintest/tests/ +E + +Time: 19.07 seconds, Memory: 6.00MB + +There was 1 error: + +1) PageListTest::testListingPage +Drupal\Core\Config\UnmetDependenciesException: +Configuration objects provided by dublintest +have unmet dependencies: +node.type.page (node), +views.view.pages (node, views) +``` + +--- + +```yml +name: DrupalCamp Dublin tests +core: 8.x +type: module + +dependencies: + - drupal:node + - drupal:views +``` + +--- + +``` +docker@cli:/var/www/core$ ../vendor/bin/phpunit ../modules/dublintest/tests +PHPUnit 4.8.36 by Sebastian Bergmann and contributors. + +Testing ../modules/dublintest/tests/ +. + +Time: 29.58 seconds, Memory: 6.00MB + +OK (1 test, 1 assertion) +``` + +--- + +```php +public function testOnlyPublishedPagesAreShown() { + // Given I have a mixture of published and unpublished pages, + // as well as other types of content. + + // When I view the pages list. + + // I should only see the published pages. +} +``` + +--- + +```php +public function testOnlyPublishedPagesAreShown() { + $this->drupalCreateContentType(['type' => 'article']); + + $this->drupalCreateNode(['type' => 'page', 'status' => TRUE]); + + $this->drupalCreateNode(['type' => 'article']); + + $this->drupalCreateNode(['type' => 'page', 'status' => FALSE]); + + // When I view the pages list. + + // I should only see the published pages. +} +``` + +--- + +```php +public function testOnlyPublishedPagesAreShown() { + ... + + $results = views_get_view_result('pages'); + + $nids = collect($results)->pluck('nid')->all(); + // [1, 3] + + // I should only see the published pages. +} +``` + +--- + +```php +public function testOnlyPublishedPagesAreShown() { + ... + + $results = views_get_view_result('pages'); + + $nids = collect($results)->pluck('nid')->all(); + // [1, 3] + + $this->assertEquals([1], $nids); +} +``` + +--- + +``` +docker@cli:/var/www/core$ ../vendor/bin/phpunit ../modules/dublintest/tests +--filter=testOnlyPublishedPagesAreShown +PHPUnit 4.8.36 by Sebastian Bergmann and contributors. + +Testing ../modules/dublintest/tests +F + +Time: 26.4 seconds, Memory: 6.00MB + +There was 1 failure: +``` + +--- + +``` +1) PageListTest::testOnlyPublishedPagesAreShown +Failed asserting that two arrays are equal. +--- Expected ++++ Actual +@@ @@ + Array ( +- 0 => 1 ++ 0 => '1' ++ 1 => '3' + ) + +/var/www/core/tests/Drupal/Tests/BrowserTestBase.php:1240 +/var/www/modules/dublintest/tests/src/PageListTest.php:25 + +FAILURES! +Tests: 1, Assertions: 3, Failures: 1. +``` + +--- + +[.build-lists: false] + +- Edit the view +- Add the status filter +- Update the module config + +--- + +``` +docker@cli:/var/www/core$ ../vendor/bin/phpunit ../modules/dublintest/tests +--filter=testOnlyPublishedPagesAreShown +PHPUnit 4.8.36 by Sebastian Bergmann and contributors. + +Testing ../modules/dublintest/tests +. + +Time: 26.53 seconds, Memory: 6.00MB + +OK (1 test, 3 assertions) +``` + +--- + +```php +public function testPagesAreOrderedAlphabetically() { + // Given I have multiple pages with different titles. + + // When I view the pages list. + + // I see the pages in the correct order. +} +``` + +--- + +```php +public function testPagesAreOrderedAlphabetically() { + $this->drupalCreateNode(['title' => 'Page A']); + $this->drupalCreateNode(['title' => 'Page D']); + $this->drupalCreateNode(['title' => 'Page B']); + $this->drupalCreateNode(['title' => 'Page C']); + + $results = views_get_view_result('pages'); + + $nids = collect($results)->pluck('nid')->all(); + + $this->assertEquals([1, 3, 4, 2], $nids); +} +``` + +--- + +``` +docker@cli:/var/www/core$ ../vendor/bin/phpunit ../modules/dublintest/tests +-filter=testPagesAreOrderedAlphabetically +PHPUnit 4.8.36 by Sebastian Bergmann and contributors. + +Testing ../modules/dublintest/tests +F + +Time: 28.03 seconds, Memory: 6.00MB + +There was 1 failure: +``` + +--- + +``` +1) PageListTest::testPagesAreOrderedAlphabetically +Failed asserting that two arrays are equal. +--- Expected ++++ Actual +@@ @@ + Array ( +- 0 => 1 +- 1 => 3 +- 2 => 4 +- 3 => 2 ++ 0 => '1' ++ 1 => '2' ++ 2 => '3' ++ 3 => '4' + ) + +/var/www/core/tests/Drupal/Tests/BrowserTestBase.php:1240 +/var/www/modules/dublintest/tests/src/PageListTest.php:36 +``` + +--- + +- Edit the view +- Remove the default sort criteria (created on) +- Add new sort criteria +- Update the module config + +--- + +``` +docker@cli:/var/www/core$ ../vendor/bin/phpunit ../modules/dublintest/tests +--filter=testPagesAreOrderedAlphabetically +PHPUnit 4.8.36 by Sebastian Bergmann and contributors. + +Testing ../modules/dublintest/tests +. + +Time: 27.67 seconds, Memory: 6.00MB + +OK (1 test, 2 assertions) +``` + +--- + +``` +docker@cli:/var/www/core$ ../vendor/bin/phpunit ../modules/dublintest/tests +PHPUnit 4.8.36 by Sebastian Bergmann and contributors. + +Testing ../modules/dublintest/tests +... + +Time: 1.17 minutes, Memory: 6.00MB + +OK (3 tests, 6 assertions) +``` + +--- + +## Takeaways + +- Testing has made me a better developer +- Testing can produce better quality code +- Writing tests is an investment +- OK to start small, introduce tests gradually +- Easier to refactor +- Tests can pass, but things can still be broken. Tests only report on what they cover. + +^ Made me think about how I'm going to do something more starting to do it +Less cruft, only write code that serves a purpose +Spending time writing tests pays dividends later on +Start by introducing tests for new features or regression tests when fixing bugs +If you know things pass, then you can refactor code knowing if something is broken +Manual testing is still important + +--- + +## Thanks! +# Questions? +### @opdavies +### oliverdavies.uk diff --git a/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/slides.pdf b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/slides.pdf new file mode 100644 index 0000000..e0f910a Binary files /dev/null and b/tdd-test-driven-drupal/2017-10-21-drupalcamp-dublin/slides.pdf differ