Merge branch 'master' into bootswatch

This commit is contained in:
Oliver Davies 2015-09-09 22:04:53 +01:00
commit c8d2aa2f3f
8 changed files with 33 additions and 72 deletions

View file

@ -7,53 +7,37 @@ tags:
- jenkins - jenkins
slug: automating-sculpin-jenkins slug: automating-sculpin-jenkins
--- ---
As part of re-building this site with [Sculpin](http://sculpin.io), I wanted to automate the deployments, as in I As part of re-building this site with [Sculpin](http://sculpin.io), I wanted to automate the deployments, as in I wouldn't need to run a script like [publish.sh](https://raw.githubusercontent.com/sculpin/sculpin-blog-skeleton/master/publish.sh) locally and have that deploy my code onto my server. Not only did that mean that my local workflow was simpler (update, commit and push, rather than update, commit, push and deploy), but if I wanted to make a quick edit or hotfix, I could log into GitHub or Bitbucket (wherever I decided to host the source code) from any computer or my phone, make the change and have it deployed for me.
wouldn't need to run a script like
[publish.sh](https://raw.githubusercontent.com/sculpin/sculpin-blog-skeleton/master/publish.sh) locally and have that
deploy my code onto my server. Not only did that mean that my local workflow was simpler (update, commit and push,
rather than update, commit, push and deploy), but if I wanted to make a quick edit or hotfix, I could log into GitHub or
Bitbucket (wherever I decided to host the source code) from any computer or my phone, make the change and have it
deployed for me.
I'd started using [Jenkins CI](http://jenkins-ci.org) during my time at the Drupal Association, and had since built my I'd started using [Jenkins CI](http://jenkins-ci.org) during my time at the Drupal Association, and had since built my own Jenkins server to handle deployments of Drupal websites, so that was the logical choice to use.
own Jenkins server to handle deployments of Drupal websites, so that was the logical choice to use.
## Installing Jenkins and Sculpin ## Installing Jenkins and Sculpin
If you dont already have Jenkins installed and configured, I'd suggest using [Jeff Geerling](http://jeffgeerling.com/) If you dont already have Jenkins installed and configured, I'd suggest using [Jeff Geerling](http://jeffgeerling.com/) (aka geerlingguy)'s [Ansible role for Jenkins CI](https://galaxy.ansible.com/list#/roles/440).
(aka geerlingguy)'s [Ansible role for Jenkins CI](https://galaxy.ansible.com/list#/roles/440).
I've also released an [Ansible role for Sculpin](https://galaxy.ansible.com/list#/roles/4063) that installs the I've also released an [Ansible role for Sculpin](https://galaxy.ansible.com/list#/roles/4063) that installs the executable so that the Jenkins server can run Sculpin commands.
executable so that the Jenkins server can run Sculpin commands.
## Triggering a Build from a Git Commit ## Triggering a Build from a Git Commit
I created a new Jenkins item for this task, and restricted where it could be run to `master` (i.e. the Jenkins server I created a new Jenkins item for this task, and restricted where it could be run to `master` (i.e. the Jenkins server rather than any of the nodes).
rather than any of the nodes).
### Polling from Git ### Polling from Git
I entered the url to the [GitHub repo](https://github.com/opdavies/oliverdavies.uk) into the **Source Code Management** I entered the url to the [GitHub repo](https://github.com/opdavies/oliverdavies.uk) into the **Source Code Management** section (the Git option *may* have been added by the [Git plugin](https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin) that I have installed).
section (the Git option *may* have been added by the [Git plugin](https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin) that I have installed).
As we dont need any write access back to the repo, using the HTTP URL rather than the SSH one was fine, and I didnt As we dont need any write access back to the repo, using the HTTP URL rather than the SSH one was fine, and I didnt need to provide any additional credentials.
need to provide any additional credentials.
Also, as I knew that Id be working a lot with feature branches, I entered `*/master` as the only branch to build. This Also, as I knew that Id be working a lot with feature branches, I entered `*/master` as the only branch to build. This meant that pushing changes or making edits on any other branches would not trigger a build.
meant that pushing changes or making edits on any other branches would not trigger a build.
![Defining the Git repository in Jenkins](/assets/images/blog/oliverdavies-uk-jenkins-git-repo.png) ![Defining the Git repository in Jenkins](/assets/images/blog/oliverdavies-uk-jenkins-git-repo.png)
I also checked the **Poll SCM** option so that Jenkins would be routinely checking for updated code. This essentially I also checked the **Poll SCM** option so that Jenkins would be routinely checking for updated code. This essentially uses the same syntax as cron, specifying minutes, hours etc. I entered `* * * * *` so that Jenkins would poll each minute, knowing that I could make this less frequent if needed.
uses the same syntax as cron, specifying minutes, hours etc. I entered `* * * * *` so that Jenkins would poll each
minute, knowing that I could make this less frequent if needed.
This now that Jenkins would be checking for any updates to the repo each minute, and could execute tasks if needed. This now that Jenkins would be checking for any updates to the repo each minute, and could execute tasks if needed.
### Building and Deploying ### Building and Deploying
Within the **Builds** section of the item, I added an *Execute Shell* step, where I could enter a command to execute. Within the **Builds** section of the item, I added an *Execute Shell* step, where I could enter a command to execute. Here, I pasted a modified version of the original publish.sh script.
Here, I pasted a modified version of the original publish.sh script.
#!/bin/bash #!/bin/bash
@ -65,15 +49,11 @@ Here, I pasted a modified version of the original publish.sh script.
rsync -avze 'ssh' --delete output_prod/ prodwww2:/var/www/html/oliverdavies.uk/htdocs rsync -avze 'ssh' --delete output_prod/ prodwww2:/var/www/html/oliverdavies.uk/htdocs
if [ $? -ne 0 ]; then echo "Could not publish the site"; exit 1; fi if [ $? -ne 0 ]; then echo "Could not publish the site"; exit 1; fi
This essentially is the same as the original file, in that Sculpin generates the site, and uses rsync to deploy it This essentially is the same as the original file, in that Sculpin generates the site, and uses rsync to deploy it somewhere else. In my case, `prodwww2` is a Jenkins node (this alias is configured in `/var/lib/jenkins/.ssh/config`), and `/var/www/html/oliverdavies.uk/htdocs` is the directory from where my site is served.
somewhere else. In my case, `prodwww2` is a Jenkins node (this alias is configured in `/var/lib/jenkins/.ssh/config`),
and `/var/www/html/oliverdavies.uk/htdocs` is the directory from where my site is served.
## Building Periodically ## Building Periodically
There is some dynamic content on my site, specifically on the Talks page. Each talk has a date assigned to it, and There is some dynamic content on my site, specifically on the Talks page. Each talk has a date assigned to it, and within the Twig template, the talk is positoned within upcoming or previous talks based on whether this date is less or greater than the time of the build.
within the Twig template, the talk is positoned within upcoming or previous talks based on whether this date is less or
greater than the time of the build.
The YAML front matter: The YAML front matter:
@ -97,28 +77,19 @@ The Twig layout:
{% endfor%} {% endfor%}
{%- endraw %} {%- endraw %}
I also didnt want to have to push an empty commit or manually trigger a job in Jenkins after doing a talk in order for I also didnt want to have to push an empty commit or manually trigger a job in Jenkins after doing a talk in order for it to be positioned in the correct place on the page, so I also wanted Jenkins to schedule a regular build regardless of whether or not code had been pushed, so ensure that my talks page would be up to date.
it to be positioned in the correct place on the page, so I also wanted Jenkins to schedule a regular build regardless of
whether or not code had been pushed, so ensure that my talks page would be up to date.
After originally thinking that I'd have to split the build steps into a separate item and trigger that from a scheduled After originally thinking that I'd have to split the build steps into a separate item and trigger that from a scheduled item, and amend my git commit item accordingly, I found a **Build periodically** option that I could use within the same item, leaving it intact and not having to make amends.
item, and amend my git commit item accordingly, I found a **Build periodically** option that I could use within the same
item, leaving it intact and not having to make amends.
I set this to `@daily` (the same `H H * * *` - `H` is a Jenkins thing), so that the build would be triggered I set this to `@daily` (the same `H H * * *` - `H` is a Jenkins thing), so that the build would be triggered automatically each day without a commit, and deploy any updates to the site.
automatically each day without a commit, and deploy any updates to the site.
![Setting Jenkins to periodically build a new version of the site.](/assets/images/blog/oliverdavies-uk-jenkins-git-timer.png) ![Setting Jenkins to periodically build a new version of the site.](/assets/images/blog/oliverdavies-uk-jenkins-git-timer.png)
## Next Steps ## Next Steps
This workflow works great for one site, but as I roll out more Sculpin sites, I'd like to reduce duplication. I see this This workflow works great for one site, but as I roll out more Sculpin sites, I'd like to reduce duplication. I see this mainly as Ill end up creating a separate `sculpin_build` item thats decoupled from the site that its building, and instead passing variables such as environment, server name and docroot path as parameters in a parameterized build.
mainly as Ill end up creating a separate `sculpin_build` item thats decoupled from the site that its building, and
instead passing variables such as environment, server name and docroot path as parameters in a parameterized build.
I'll probably also take the raw shell script out of Jenkins and save it in a text file that's stored locally on the I'll probably also take the raw shell script out of Jenkins and save it in a text file that's stored locally on the server, and execute that via Jenkins. This means that Id be able to store this file in a separate Git repository with my other Jenkins scripts and get the standard advantages of using version control.
server, and execute that via Jenkins. This means that Id be able to store this file in a separate Git repository with
my other Jenkins scripts and get the standard advantages of using version control.
## Update ## Update
@ -130,8 +101,7 @@ Since publishing this post, I've added some more items to the original build scr
/usr/local/bin/composer update /usr/local/bin/composer update
fi fi
Updates project dependencies via [Composer](https://getcomposer.org/doc/00-intro.md#introduction) if composer.json Updates project dependencies via [Composer](https://getcomposer.org/doc/00-intro.md#introduction) if composer.json exists.
exists.
### Updating Sculpin Dependencies ### Updating Sculpin Dependencies
@ -139,8 +109,7 @@ exists.
sculpin update sculpin update
fi fi
Runs `sculpin update` on each build if the sculpin.json file exists, to ensure that the required custom bundles and Runs `sculpin update` on each build if the sculpin.json file exists, to ensure that the required custom bundles and dependencies are installed.
dependencies are installed.
### Managing Redirects ### Managing Redirects
@ -148,7 +117,4 @@ dependencies are installed.
/usr/bin/php scripts/redirects.php /usr/bin/php scripts/redirects.php
fi fi
I've been working on a `redirects.php` script that generates redirects from a .csv file, after seeing similar things in I've been working on a `redirects.php` script that generates redirects from a .csv file, after seeing similar things in the [Pantheon Documentation](https://github.com/pantheon-systems/documentation) and [That Podcast](https://github.com/thatpodcast/thatpodcast.io) repositories. This checks if that file exists, and if so, runs it and generates the source file containing each redirect.
the [Pantheon Documentation](https://github.com/pantheon-systems/documentation) and
[That Podcast](https://github.com/thatpodcast/thatpodcast.io) repositories. This checks if that file exists, and if so,
runs it and generates the source file containing each redirect.

View file

@ -12,19 +12,17 @@ meta:
## PHP ## PHP
* A good knowledge of procedural and object-orientated PHP (5+). * A good knowledge of procedural and object-orientated PHP (5+).
* Knowledge of using CMSes and frameworks such as [Drupal](#drupal), WordPress, [Sculpin](#static-sites) and * Knowledge of using CMSes and frameworks such as [Drupal](#drupal), WordPress, [Sculpin](#static-sites) and [Symfony](#symfony).
[Symfony](#symfony).
## Drupal ## Drupal
* Website development using Drupal 6 and 7. * Website development using Drupal 6 and 7.
* Use of Drupal distributions such as [Commerce Kickstart](https://www.drupal.org/project/commerce_kickstart), * Use of Drupal distributions such as [Commerce Kickstart](https://www.drupal.org/project/commerce_kickstart),
[Drupal Commons](https://www.drupal.org/project/commons), [Open Atrium](https://www.drupal.org/project/openatrium) and [Drupal Commons](https://www.drupal.org/project/commons), [Open Atrium](https://www.drupal.org/project/openatrium) and
[COD](http://usecod.io) (the conference organising distribution). [COD](http://usecod.io) (the conference organising distribution).
* Custom module development including integration with third party APIs. * Custom module development including integration with third party APIs.
* Front-end and theme development using [SASS](http://sass-lang.com) and [Compass](http://compass-style.org). * Front-end and theme development using [SASS](http://sass-lang.com) and [Compass](http://compass-style.org).
* eCommerce using [Ubercart](http://ubercart.org) in Drupal 6, and [Drupal Commerce](http://drupalcommerce.org) in * eCommerce using [Ubercart](http://ubercart.org) in Drupal 6, and [Drupal Commerce](http://drupalcommerce.org) in Drupal 7.
Drupal 7.
* CRM integration including [CiviCRM](http://civicrm.org) and [SugarCRM](www.sugarcrm.com). * CRM integration including [CiviCRM](http://civicrm.org) and [SugarCRM](www.sugarcrm.com).
* Content and data migration using the [Migrate framework](http://www.drupal.org/migrate). * Content and data migration using the [Migrate framework](http://www.drupal.org/migrate).
* Ad-hoc and ongoing support and maintenance of Drupal sites, including security updates. * Ad-hoc and ongoing support and maintenance of Drupal sites, including security updates.
@ -33,10 +31,8 @@ meta:
##Symfony ##Symfony
* An intermediate knowledge of object-oriented PHP. * An intermediate knowledge of object-oriented PHP.
* An introductory knowledge of [Symfony](https://symfony.com/) full-stack framework and * An introductory knowledge of [Symfony](https://symfony.com/) full-stack framework and [Silex](http://silex.sensiolabs.org/).
[Silex](http://silex.sensiolabs.org/). * Use of [Composer](https://getcomposer.org/doc/00-intro.md#introduction) and Symfony components such as [Guzzle](https://packagist.org/packages/guzzlehttp/guzzle) in non-Drupal projects.
* Use of [Composer](https://getcomposer.org/doc/00-intro.md#introduction) and Symfony components such as
[Guzzle](https://packagist.org/packages/guzzlehttp/guzzle) in non-Drupal projects.
* Use of [Twig](http://twig.sensiolabs.org/) in Symfony, Drupal 8 and Sculpin projects. * Use of [Twig](http://twig.sensiolabs.org/) in Symfony, Drupal 8 and Sculpin projects.
##Static sites ##Static sites
@ -47,10 +43,9 @@ meta:
## System Administration ## System Administration
* [CentOS](http://www.centos.org), [RHEL](http://www.redhat.com/en/technologies/linux-platforms/enterprise-linux) and * [CentOS](http://www.centos.org), [RHEL](http://www.redhat.com/en/technologies/linux-platforms/enterprise-linux) and
[Ubuntu](http://www.ubuntu.com/server) server configuration, including [Apache](http://apache.org), [Ubuntu](http://www.ubuntu.com/server) server configuration, including [Apache](http://apache.org),
[Nginx](http://nginx.org), [PHP](http://php.net), [MySQL](http://mysql.com) and [MariaDB](https://mariadb.org). [Nginx](http://nginx.org), [PHP](http://php.net), [MySQL](http://mysql.com) and [MariaDB](https://mariadb.org).
* Server configuration and management using [Puppet](http://puppetlabs.com) or [Ansible](http://www.ansible.com) * Server configuration and management using [Puppet](http://puppetlabs.com) or [Ansible](http://www.ansible.com) (preferred).
(preferred).
* Continuous integration using [Jenkins](http://jenkins-ci.org). * Continuous integration using [Jenkins](http://jenkins-ci.org).
* Virtual machine setup using [Vagrant](http://vagrantup.com) for development or server testing. * Virtual machine setup using [Vagrant](http://vagrantup.com) for development or server testing.
* Domain name and DNS management. * Domain name and DNS management.

View file

@ -57,7 +57,7 @@ talks:
{% set new_year = true %} {% set new_year = true %}
{% endif %} {% endif %}
{% include 'talk-listing-item' %} {% include 'talk_listing_item' %}
{% endfor %} {% endfor %}
{% if found %} {% if found %}
@ -90,7 +90,7 @@ talks:
{% set new_year = true %} {% set new_year = true %}
{% endif %} {% endif %}
{% include 'talk-listing-item' %} {% include 'talk_listing_item' %}
{% endfor %} {% endfor %}
{% if found %} {% if found %}

View file

@ -1,10 +1,10 @@
{% extends 'default.html.twig' %} {% extends 'default' %}
{% block body_classes 'page--blog page--blog__post' %} {% block body_classes 'page--blog page--blog__post' %}
{% block content_wrapper %} {% block content_wrapper %}
<main class="col-md-9"> <main class="col-md-9">
{% include 'post-header' %} {% include 'post_header' %}
{% block content %}{% endblock content %} {% block content %}{% endblock content %}