341 lines
18 KiB
HTML
341 lines
18 KiB
HTML
<!DOCTYPE html>
|
||
<html class="no-js" lang="en-GB">
|
||
<head>
|
||
<title>Automating Sculpin Builds with Jenkins CI | Oliver Davies</title>
|
||
|
||
<meta charset="UTF-8">
|
||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||
|
||
|
||
<meta property="og:url" content="https://opdavies.github.io/oliverdavies.uk/blog/2015/07/21/automating-sculpin-jenkins">
|
||
<meta property="og:title" content="Automating Sculpin Builds with Jenkins CI"/>
|
||
|
||
|
||
|
||
<meta property="og:image" content="https://opdavies.github.io/oliverdavies.uk/assets/images/me-precedent.jpg"/>
|
||
<meta property="og:image:height" content="327"/>
|
||
<meta property="og:image:type" content="image/jpg">
|
||
<meta property="og:image:width" content="327"/>
|
||
|
||
|
||
|
||
<link rel="stylesheet" href="https://opdavies.github.io/oliverdavies.uk/assets/css/main.css">
|
||
<link rel="stylesheet" href="https://opdavies.github.io/oliverdavies.uk/assets/css/blog-post.css">
|
||
|
||
<link rel="apple-touch-icon" href="/assets/images/me-precedent.jpg?s=57" sizes="57x57">
|
||
<link rel="apple-touch-icon" href="/assets/images/me-precedent.jpg?s=114" sizes="114x114">
|
||
<link rel="apple-touch-icon" href="/assets/images/me-precedent.jpg?s=72" sizes="72x72">
|
||
<link rel="apple-touch-icon" href="/assets/images/me-precedent.jpg?s=144" sizes="144x144">
|
||
<link rel="apple-touch-icon" href="/assets/images/me-precedent.jpg?s=60" sizes="60x60">
|
||
<link rel="apple-touch-icon" href="/assets/images/me-precedent.jpg?s=120" sizes="120x120">
|
||
<link rel="apple-touch-icon" href="/assets/images/me-precedent.jpg?s=76" sizes="76x76">
|
||
<link rel="apple-touch-icon" href="/assets/images/me-precedent.jpg?s=152" sizes="152x152">
|
||
|
||
<link rel="icon" href="/assets/images/me-precedent.jpg?s=160" sizes="160x160">
|
||
<link rel="icon" href="/assets/images/me-precedent.jpg?s=96" sizes="96x96">
|
||
<link rel="icon" href="/assets/images/me-precedent.jpg?s=32" sizes="32x32">
|
||
<link rel="icon" href="/assets/images/me-precedent.jpg?s=16" sizes="16x16">
|
||
</head>
|
||
<body class="">
|
||
<nav class="navbar navbar-inverse navbar-fixed-top">
|
||
<div class="container">
|
||
<div class="navbar-header">
|
||
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
|
||
<span class="sr-only">Toggle navigation</span>
|
||
<span class="icon-bar"></span>
|
||
<span class="icon-bar"></span>
|
||
<span class="icon-bar"></span>
|
||
</button>
|
||
<a class="navbar-brand" href="https://opdavies.github.io/oliverdavies.uk/">Oliver Davies</a>
|
||
</div>
|
||
|
||
<div id="navbar" class="collapse navbar-collapse" role="navigation">
|
||
<ul class="nav navbar-nav">
|
||
<li class="">
|
||
<a href="/">About</a>
|
||
</li>
|
||
|
||
<li class="">
|
||
<a href="/experience">Experience</a>
|
||
</li>
|
||
|
||
<li class="">
|
||
<a href="/testimonials">Testimonials</a>
|
||
</li>
|
||
|
||
<li class="">
|
||
<a href="/talks">Talks</a>
|
||
</li>
|
||
|
||
<li class="active">
|
||
<a href="/blog">Blog</a>
|
||
</li>
|
||
|
||
<li class="">
|
||
<a href="/contact">Contact</a>
|
||
</li>
|
||
</ul>
|
||
</div> </div>
|
||
</nav>
|
||
|
||
<div class="container">
|
||
<div class="row">
|
||
<main class="col-md-9">
|
||
<h1>Automating Sculpin Builds with Jenkins CI</h1>
|
||
|
||
<p class="posted">21st July 2015</p>
|
||
|
||
<p>As part of re-building this site with <a href="http://sculpin.io">Sculpin</a>, I wanted to automate the deployments, as in I wouldn't need to run a script like <a href="https://raw.githubusercontent.com/sculpin/sculpin-blog-skeleton/master/publish.sh">publish.sh</a> locally and have that deploy my code onto my server. Not only did that mean that my local workflow was simpler (update, commit and push, rather than update, commit, push and deploy), but if I wanted to make a quick edit or hotfix, I could log into GitHub or Bitbucket (wherever I decided to host the source code) from any computer or my phone, make the change and have it deployed for me.</p>
|
||
|
||
<p>I'd started using <a href="http://jenkins-ci.org">Jenkins CI</a> during my time at the Drupal Association, and had since built my own Jenkins server to handle deployments of Drupal websites, so that was the logical choice to use.</p>
|
||
|
||
<h2 id="installing-jenkins-and-sculpin">Installing Jenkins and Sculpin</h2>
|
||
|
||
<p>If you don’t already have Jenkins installed and configured, I'd suggest using <a href="http://jeffgeerling.com/">Jeff Geerling</a> (aka geerlingguy)'s <a href="https://galaxy.ansible.com/list#/roles/440">Ansible role for Jenkins CI</a>.</p>
|
||
|
||
<p>I've also released an <a href="https://galaxy.ansible.com/list#/roles/4063">Ansible role for Sculpin</a> that installs the executable so that the Jenkins server can run Sculpin commands.</p>
|
||
|
||
<h2 id="triggering-a-build-from-a-git-commit">Triggering a Build from a Git Commit</h2>
|
||
|
||
<p>I created a new Jenkins item for this task, and restricted where it could be run to <code>master</code> (i.e. the Jenkins server rather than any of the nodes).</p>
|
||
|
||
<h3 id="polling-from-git">Polling from Git</h3>
|
||
|
||
<p>I entered the url to the <a href="https://github.com/opdavies/oliverdavies.uk">GitHub repo</a> into the <strong>Source Code Management</strong> section (the Git option <em>may</em> have been added by the <a href="https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin">Git plugin</a> that I have installed).</p>
|
||
|
||
<p>As we don’t need any write access back to the repo, using the HTTP URL rather than the SSH one was fine, and I didn’t need to provide any additional credentials.</p>
|
||
|
||
<p>Also, as I knew that I’d be working a lot with feature branches, I entered <code>*/master</code> as the only branch to build. This meant that pushing changes or making edits on any other branches would not trigger a build.</p>
|
||
|
||
<p><img src="/assets/images/blog/oliverdavies-uk-jenkins-git-repo.png" alt="Defining the Git repository in Jenkins" /></p>
|
||
|
||
<p>I also checked the <strong>Poll SCM</strong> option so that Jenkins would be routinely checking for updated code. This essentially uses the same syntax as cron, specifying minutes, hours etc. I entered <code>* * * * *</code> so that Jenkins would poll each minute, knowing that I could make this less frequent if needed.</p>
|
||
|
||
<p>This now that Jenkins would be checking for any updates to the repo each minute, and could execute tasks if needed.</p>
|
||
|
||
<h3 id="building-and-deploying">Building and Deploying</h3>
|
||
|
||
<p>Within the <strong>Builds</strong> section of the item, I added an <em>Execute Shell</em> step, where I could enter a command to execute. Here, I pasted a modified version of the original publish.sh script.</p>
|
||
|
||
<pre><code class="language-bash">#!/bin/bash
|
||
|
||
set -uex
|
||
|
||
sculpin generate --env=prod --quiet
|
||
if [ $? -ne 0 ]; then echo "Could not generate the site"; exit 1; fi
|
||
|
||
rsync -avze 'ssh' --delete output_prod/ prodwww2:/var/www/html/oliverdavies.uk/htdocs
|
||
if [ $? -ne 0 ]; then echo "Could not publish the site"; exit 1; fi
|
||
</code></pre>
|
||
|
||
<p>This essentially is the same as the original file, in that Sculpin generates the site, and uses rsync to deploy it somewhere else. In my case, <code>prodwww2</code> is a Jenkins node (this alias is configured in <code>/var/lib/jenkins/.ssh/config</code>), and <code>/var/www/html/oliverdavies.uk/htdocs</code> is the directory from where my site is served.</p>
|
||
|
||
<h2 id="building-periodically">Building Periodically</h2>
|
||
|
||
<p>There is some dynamic content on my site, specifically on the Talks page. Each talk has a date assigned to it, and within the Twig template, the talk is positoned within upcoming or previous talks based on whether this date is less or greater than the time of the build.</p>
|
||
|
||
<p>The YAML front matter:</p>
|
||
|
||
<pre><code class="language-yaml">---
|
||
...
|
||
talks:
|
||
- title: Test Drive Twig with Sculpin
|
||
date: 2015-07-24
|
||
location: DrupalCamp North
|
||
---
|
||
</code></pre>
|
||
|
||
<p>The Twig layout:</p>
|
||
|
||
<pre><code class="language-twig">{% for talk in talks|reverse if talk.date >= now %}
|
||
{# Upcoming talks #}
|
||
{% endfor %}
|
||
|
||
{% for talk in talks if talk.date < now %}
|
||
{# Previous talks #}
|
||
{% endfor%}
|
||
</code></pre>
|
||
|
||
<p>I also didn’t want to have to push an empty commit or manually trigger a job in Jenkins after doing a talk in order for it to be positioned in the correct place on the page, so I also wanted Jenkins to schedule a regular build regardless of whether or not code had been pushed, so ensure that my talks page would be up to date.</p>
|
||
|
||
<p>After originally thinking that I'd have to split the build steps into a separate item and trigger that from a scheduled item, and amend my git commit item accordingly, I found a <strong>Build periodically</strong> option that I could use within the same item, leaving it intact and not having to make amends.</p>
|
||
|
||
<p>I set this to <code>@daily</code> (the same <code>H H * * *</code> - <code>H</code> is a Jenkins thing), so that the build would be triggered automatically each day without a commit, and deploy any updates to the site.</p>
|
||
|
||
<p><img src="/assets/images/blog/oliverdavies-uk-jenkins-git-timer.png" alt="Setting Jenkins to periodically build a new version of the site." /></p>
|
||
|
||
<h2 id="next-steps">Next Steps</h2>
|
||
|
||
<p>This workflow works great for one site, but as I roll out more Sculpin sites, I'd like to reduce duplication. I see this mainly as I’ll end up creating a separate <code>sculpin_build</code> item that’s decoupled from the site that it’s building, and instead passing variables such as environment, server name and docroot path as parameters in a parameterized build.</p>
|
||
|
||
<p>I'll probably also take the raw shell script out of Jenkins and save it in a text file that's stored locally on the server, and execute that via Jenkins. This means that I’d be able to store this file in a separate Git repository with my other Jenkins scripts and get the standard advantages of using version control.</p>
|
||
|
||
<h2 id="update">Update</h2>
|
||
|
||
<p>Since publishing this post, I've added some more items to the original build script.</p>
|
||
|
||
<h3 id="updating-composer">Updating Composer</h3>
|
||
|
||
<pre><code class="language-bash">if [ -f composer.json ]; then
|
||
/usr/local/bin/composer install
|
||
fi
|
||
</code></pre>
|
||
|
||
<p>Updates project dependencies via <a href="https://getcomposer.org/doc/00-intro.md#introduction">Composer</a> if composer.json exists.</p>
|
||
|
||
<h3 id="updating-sculpin-dependencies">Updating Sculpin Dependencies</h3>
|
||
|
||
<pre><code class="language-bash">if [ -f sculpin.json ]; then
|
||
sculpin install
|
||
fi
|
||
</code></pre>
|
||
|
||
<p>Runs <code>sculpin install</code> on each build if the sculpin.json file exists, to ensure that the required custom bundles and dependencies are installed.</p>
|
||
|
||
<h3 id="managing-redirects">Managing Redirects</h3>
|
||
|
||
<pre><code class="language-bash">if [ -f scripts/redirects.php ]; then
|
||
/usr/bin/php scripts/redirects.php
|
||
fi
|
||
</code></pre>
|
||
|
||
<p>I've been working on a <code>redirects.php</code> script that generates redirects from a .csv file, after seeing similar things in the <a href="https://github.com/pantheon-systems/documentation">Pantheon Documentation</a> and <a href="https://github.com/thatpodcast/thatpodcast.io">That Podcast</a> repositories. This checks if that file exists, and if so, runs it and generates the source file containing each redirect.</p>
|
||
|
||
<p class="tags">
|
||
Tags:
|
||
<a href="https://opdavies.github.io/oliverdavies.uk/blog/tags/sculpin">sculpin</a>, <a href="https://opdavies.github.io/oliverdavies.uk/blog/tags/jenkins">jenkins</a> </p>
|
||
<div class="post-pager is-flex">
|
||
<div class="is-half">
|
||
<a href="/blog/2015/07/19/sculpin-twig-resources">
|
||
« Sculpin and Twig Resources
|
||
</a>
|
||
</div>
|
||
|
||
<div class="is-half text-right">
|
||
<a href="/blog/2015/12/22/programmatically-load-an-entityform-in-drupal-7">
|
||
Programmatically Load an Entityform in Drupal 7 »
|
||
</a>
|
||
</div>
|
||
</div>
|
||
<div class="about-author">
|
||
<h2>About the Author</h2>
|
||
|
||
<img src="https://opdavies.github.io/oliverdavies.uk/assets/images/me-precedent.jpg" alt="Picture of Oliver" class="img-circle">
|
||
|
||
<p>Oliver Davies is a Web Developer, System Administrator and Drupal specialist based in the UK. He is a Senior Developer at <a href="https://microserve.io">Microserve</a> and also provides freelance consultancy services for Drupal websites, PHP applications and Linux servers.</p>
|
||
</div>
|
||
</main>
|
||
|
||
<div class="col-md-3">
|
||
<div class="panel badges text-center">
|
||
<a class="badge--da-member" href="https://assoc.drupal.org/membership" title="I’m a Drupal Association member.">
|
||
<img
|
||
src="https://opdavies.github.io/oliverdavies.uk/assets/images/da-individual-member.png"
|
||
alt="Drupal Association Individual Member"
|
||
width="152"
|
||
>
|
||
</a>
|
||
|
||
<a href="http://drupalcores.com/#opdavies">
|
||
<img
|
||
alt="I built Drupal 8 with hand holding a wrench on blue background"
|
||
src="https://opdavies.github.io/oliverdavies.uk/assets/images/drupal-8.jpg"
|
||
/>
|
||
</a>
|
||
|
||
<img
|
||
src="https://opdavies.github.io/oliverdavies.uk/assets/images/badges/acquia-certified-developer-drupal-8.png"
|
||
alt="Acquia Certified Developer - Drupal 8 Exam Badge"
|
||
height="147" width="147"
|
||
/>
|
||
|
||
<a href="http://conference.phpnw.org.uk/phpnw17">
|
||
<img src="https://opdavies.github.io/oliverdavies.uk/assets/images/badges/phpnw17.png" alt="">
|
||
</a>
|
||
</div>
|
||
<div class="availability panel panel-default">
|
||
<div class="panel-heading">Availability</div>
|
||
|
||
<div class="panel-body">
|
||
<p>
|
||
<i class="fa fa-thumbs-o-up text-warning"></i>
|
||
|
||
|
||
Currently have limited part-time capacity
|
||
</p>
|
||
<p>
|
||
<i class="fa fa-thumbs-o-down text-danger"></i>
|
||
|
||
Currently no spare full-time capacity.
|
||
|
||
</p>
|
||
</div>
|
||
</div>
|
||
<div class="latest-posts panel panel-default">
|
||
<div class="latest-posts__heading panel-heading">Latest blog posts</div>
|
||
|
||
<ul class="list-group">
|
||
<li class="post list-group-item">
|
||
<span class="post__title">
|
||
<a href="/blog/2017/06/09/introducing-the-drupal-meetups-twitterbot">
|
||
Introducing the Drupal Meetups Twitterbot
|
||
</a>
|
||
</span> -
|
||
<span class="post__date">9th June, 2017</span>
|
||
</li>
|
||
<li class="post list-group-item">
|
||
<span class="post__title">
|
||
<a href="/blog/2017/05/20/turning-drupal-module-into-feature">
|
||
Turning Your Custom Drupal Module into a Feature
|
||
</a>
|
||
</span> -
|
||
<span class="post__date">20th May, 2017</span>
|
||
</li>
|
||
<li class="post list-group-item">
|
||
<span class="post__title">
|
||
<a href="/blog/2017/05/15/drupalcamp-bristol-early-bird-tickets-sessions-sponsors">
|
||
DrupalCamp Bristol 2017 - Early Bird Tickets, Call for Sessions, Sponsors
|
||
</a>
|
||
</span> -
|
||
<span class="post__date">15th May, 2017</span>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
|
||
</div> </div>
|
||
<footer class="container">
|
||
<p class="copyright">
|
||
© 2010-2017 Oliver Davies. Built with <a href="https://sculpin.io">Sculpin</a>.
|
||
</p>
|
||
|
||
<div class="meetups">
|
||
<h2>Things that I organise</h2>
|
||
<ul>
|
||
<li class="meetups--drupal-bristol">
|
||
<a href="http://www.drupalbristol.org.uk" title="Drupal Bristol">
|
||
<img src="https://opdavies.github.io/oliverdavies.uk/assets/images/meetups/drupal-bristol.jpeg" alt="Drupal Bristol">
|
||
</a>
|
||
</li>
|
||
<li class="meetups--drupalcamp-bristol">
|
||
<a href="http://www.drupalcampbristol.co.uk" title="DrupalCamp Bristol">
|
||
<img src="https://opdavies.github.io/oliverdavies.uk/assets/images/meetups/drupalcamp-bristol.png" alt="DrupalCamp Bristol">
|
||
</a>
|
||
</li>
|
||
<li class="meetups--phpsw">
|
||
<a href="http://phpsw.uk" title="PHPSW">
|
||
<img src="https://opdavies.github.io/oliverdavies.uk/assets/images/meetups/phpsw.jpeg" alt="PHPSW">
|
||
</a>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
</footer>
|
||
|
||
<script src="https://opdavies.github.io/oliverdavies.uk/assets/js/site.js"></script>
|
||
|
||
<script>(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-11967257-1', 'auto'); ga('send', 'pageview');</script>
|
||
|
||
</body>
|
||
</html>
|