Lean Drupal Repositories with Composer and Drush
-
Upload
pantheon -
Category
Technology
-
view
835 -
download
1
Transcript of Lean Drupal Repositories with Composer and Drush
Lean Drupal Repositories with
Composer and Drush
by Greg Anderson
Photo by Adam Wyles2016
Pantheon.io 2
Session Description
Composer is the industry-standard PHP dependency manager that is now in use in Drupal 8 core. This session will show the current best practices for
using Composer, drupal-composer, drupal-scaffold, Drush, Drupal Console and Drush site-local aliases to streamline your Drupal 7 and Drupal 8 site
repositories for optimal use on teams.
We will answer such gripping questions as:
● How do I avoid placing a copy all of the Drupal core and contrib modules into my repository?
● How do I keep my core and contrib modules up-to-date?
● What if I need to customize .htaccess, or some other file?
● Can I work with a "lean" repository, and still seamlessly use a full repository to deploy?
● How do I share Drush aliases with team members, without using another repository, and without making my alias list too long?
● What is a Drush wrapper script, and how can it help me?
● How does Drupal Console fit in to all of this?
● Should I use test fixtures in my Behat tests, or make a copy of the production site database?
These techniques will also be helpful for solo site developers--you never know, the next person who needs to check out, build and test your Drupal site
from scratch might be you!
Pantheon.io
Then create a Git repository:
cd myproject
git init
git add .
git commit -m “My new project”
3
Getting Started with Drupal and Composer
One easy step in Composer:
composer create-project drupal-composer/drupal-project myproject --stability=dev
Install and run with Drush:
cd web
drush qd --db-url=mysql://root@localhost/myprojectdb
Answer “yes” when Composer
asks to remove VCS files, or run
with --no-in
teractio
n option
Pantheon.io 4
Preferred Project Structure
webcomposer.json
drupal console
circle.yml
external libraries
behat.yml
behat drush
drupal/core
index.phpcustom-installers
drupal-scaffoldmodules
contrib
custom
vendor
Pantheon.io 5
Specifying Project Layout
{
"name": "drupal-composer/drupal-project",
"scripts": {
"drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold",
"post-install-cmd": "sh ./scripts/composer/post-install.sh"
},
"extra": {
"installer-paths": {
"web/core": ["type:drupal-core"],
"web/modules/contrib/{$name}": ["type:drupal-module"],
"web/profiles/contrib/{$name}": ["type:drupal-profile"],
"web/themes/contrib/{$name}": ["type:drupal-theme"],
"drush/contrib/{$name}": ["type:drupal-drush"]
}
}
}
Pantheon.io 6
Renaming Document Root
# Ignore directories generated by Composer
drush/contrib
vendor
docroot/core
docroot/modules/contrib
docroot/themes/contrib
docroot/profiles/contrib
# Ignore Drupal's file directory
docroot/sites/default/files
.gitignore
#!/bin/sh
DOCUMENTROOT=docroot
# Prepare the scaffold files if they are not
if [ ! -f $DOCUMENTROOT/autoload.php ]
then
composer drupal-scaffold
mkdir -p $DOCUMENTROOT/modules
mkdir -p $DOCUMENTROOT/themes
mkdir -p $DOCUMENTROOT/profiles
post-install.sh
{ "extra": { "installer-paths": { "docroot/core": ["type:drupal-co "docroot/modules/contrib/{$name} "docroot/profiles/contrib/{$name "docroot/themes/contrib/{$name}" "drush/contrib/{$name}": ["type: } }}
composer.json
Pantheon.io 7
Scaffold Files
web
drupal/core
scaffold files
modules
contrib
custom
.htaccess
autoload.php
index.php
robots.txt
updates.php
sites
examples.sites.php
default
default.services.yml
default.settings.php
Drupal.org
drupal-scaffold
Pantheon.io 8
Custom Scaffold Configuration
{
"name": "pantheon-systems/drupal-project",
"scripts": {
"drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold",
"post-install-cmd": "sh ./scripts/composer/post-install.sh"
},
"extra": {
"drupal-scaffold": {
"source": "https://github.com/pantheon-systems/drops-8/archive/{version}.tar.gz",
"includes": [
"sites/default/settings.php",
"sites/default/settings.pantheon.php"
]
}
}
}
Pantheon.io 9
Document Root at Project Root
composer.json
drupal console
external libraries
drush
core
index.phpcustom-installers
drupal-scaffoldmodules
contrib
custom
vendor
circle.yml
private
Privatesubfolder
Pantheon.io 10
composer.json Relocated to Private Directory
{
"name": "pantheon-systems/drupal-project",
"scripts": {
"drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold",
"post-install-cmd": "sh ./scripts/composer/post-install.sh"
},
"extra": {
"installer-paths": {
"../core": ["type:drupal-core"],
"../modules/contrib/{$name}": ["type:drupal-module"],
"../profiles/contrib/{$name}": ["type:drupal-profile"],
"../themes/contrib/{$name}": ["type:drupal-theme"],
"../drush/contrib/{$name}": ["type:drupal-drush"]
}
}
}
Pantheon.io 11
Updating Code
web
drupal/core
scaffold files
modules
contrib
custom
.htaccess
autoload.php
index.php
robots.txt
updates.php
sites
examples.sites.php
default
default.services.yml
default.settings.php
Drupal.org
drupal-scaffold
composer update
composer drupal-scaffoldgit pull
Project FilesOnly
Scaffold FilesOnly
Composer and Scaffold
Files
Pantheon.io 12
Customizing Scaffold Files
{
"name": "pantheon-systems/drupal-project",
"scripts": {
"drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold",
"post-install-cmd": "sh ./scripts/composer/post-install.sh",
"post-drupal-scaffold-cmd": "cat htaccess-append.txt >> ../.htaccess"
}
}
To patch non-scaffold files, use:
cweaga
ns/com
poser-
patche
s
Pantheon.io 13
Checking for Security Updates
With Drush
With Composer
$ drush pm-updatestatusChecking available update data ... [ok]Checking available update data for Drupal. [ok]Checking available update data for Token (token). [ok] Name Installed Version Proposed version Message Token (token) 7.x-1.2 7.x-1.6 SECURITY UPDATE available
$ composer require roave/security-advisories:dev-master$ composer require drupal-composer/drupal-security-advisories:7.x-dev./composer.json has been updatedLoading composer repositories with package informationUpdating dependencies (including require-dev)Your requirements could not be resolved to an installable set of packages.
Problem 1 - drupal-composer/drupal-security-advisories 7.x-dev conflicts with drupal/token[7.1.2]. - Installation request for drupal-composer/drupal-security-advisories 7.x-dev -> satisfiable by drupal-composer/drupal-security-advisories[7.x-dev]. - Installation request for drupal/token 7.1.2 -> satisfiable by drupal/token[7.1.2].
n.b. Requires git_deploy module if using Composer with --prefer-source, or if dev modules are used.
Pantheon.io 14
Manage Custom Modules with Composer
webcomposer.json
drupal console
circle.yml
external libraries
behat.yml
behat drush
drupal/core
index.phpcustom-installers
drupal-scaffoldmodules
contrib
custom
vendor
Variant
WHY?
Use private modules in multiple projects.
Pantheon.io 15
Let’s Make a Module!
$ drupal generate:module $ drupal generate:controller
Pantheon.io 16
Customize Module’s composer.json
{ "name": "greg-1-anderson/snazzy", "type": "drupal-module", "description": "A Snazzy Module", "keywords": ["Drupal"], "license": "GPL-2.0+", "homepage": "https://github.com/greg-1-anderson/snazzy", "minimum-stability": "dev", "support": { "issues": "https://github.com/greg-1-anderson/snazzy/issues", "source": "https://github.com/greg-1-anderson/snazzy" }, "require": { }}
Pantheon.io 17
Add Our Custom Module to Drupal Project
{ "repositories": [ { "type": "vcs", "url": "https://github.com/greg-1-anderson/snazzy.git" } ], "require": { "composer/installers": "^1.0.20", "drupal-composer/drupal-scaffold": "^1.0", "greg-1-anderson/snazzy": "dev-master" }, "extra": { "installer-paths": { "web/modules/custom/{$name}": ["greg-1-anderson/snazzy"], "web/modules/contrib/{$name}": ["type:drupal-module"] } }}
Pantheon.io 18
Generate require Section from Existing Site
composer create-project drupal-composer/drupal-project myproject --stability=dev
cd myproject
drush composer-generate @remote
Look up the modules and themes
used on @remot
e and add them to
the require section of the
compos
er.jso
n in the cwd.
Pantheon.io 19
Lean Repository Deployment
Push
Request Test
Request Build
Behat
Deploy
Pull Lean Repo
Install
Update
Pantheon.io 20
Add a Project to Circle
A couple of clicks will get Circle CI building your project.
There are a few credentials that should also be set up:
● GitHub OAuth Token● Terminus Machine Token● SSH Key Pair
Pantheon.io 21
Generate OAuth Tokens for all Services
GitHub Personal Access Tokens Pantheon Machine Tokens
OAuth tokens are getting to be very common; many services provide them.They work like passwords, but can have limited permissions, and may be revoked.
Pantheon.io 22
Copy OAuth Tokens into CI Envrionment Variables
OAuth Tokens:
Pantheon.io
Prevents ‘composer install’
from failing due to hitting the
GitHub rate limit.
23
Use OAuth Environment Variables in CI
dependencies: pre: - composer config -g github-oauth.github.com $GITHUB_OAUTH - terminus auth login --machine-token=$PANTHEON_MACHINE_TOKEN
circle.yml
Log in to Terminus via machine token, e.g. to later deploy from
dev to test.
Pantheon.io 24
Create SSH Key Pair Specifically for CI
$ ssh-keygen -C [email protected] -f my-ci-keyfile
Add Private Key to Circle SSH Permissions Add Public Key to Provider SSH Keys
For linux servers: ssh-copy-id [email protected]
Pantheon.io 25
Dependency Hell
Bootstrapping Drupal 8 via a global Drush will ONLY WORK if the dependencies of the two projects are in perfect alignment. Upgrading one without upgrading the other is dangerous.
Drush Drupal 8
require autoload.php
Bootstrap require autoload.php
Pantheon.io 26
Simple Example of Dependency Hell
class Sub extends Base{public function foo(){return $this->bar();
}}
class Base{private function bar(){…
}}
class Sub extends Base{public function foo(){return $this->boz();
}}
class Base{private function boz(){…
}}
FancyLib v1.0.1 FancyLib v1.0.2
Sub.php Sub.php
Base.php Base.php
Semantic Versioning will not save
you!
Two autoloaders that contain multiple
copies of the same library can still fail,
even if both conform to the same
public API.
Pantheon.io 27
Dependency Hell Affects Drupal Console Too
Drupal Console, like Drush, keeps dependencies in sync with the Drupal 8 release with the same version number. Problems can still occur if versions become mismatched.
Drupal Console Drupal 8
require autoload.php
Bootstrap require autoload.php
Pantheon.io
Place Drush and Drupal Console in Drupal’s composer.json file:
cd /path/to/drupal-8-root
composer require drush/drush:8.*
composer require drupal/console
28
Solution is to Use a Single Autoloader
CRITICAL
Pantheon.io 29
Gentoo’s “Composer Problem”
“ As long as the necessary require statements are left in the code (where they belong), we can ignore Composer entirely and install the package with the system package manager. We set PHP's include directory for our users, so require('Class.php'); already looks in the right place. https://wiki.gentoo.org/wiki/Project:PHP/The_Composer_problem
Pantheon.io 30
How Could We Fix Things for Gentoo?
One Class.php shared by every application is not going to work!
Make an autoload.php that loaded versioned Class.php files from a global location?
How would all of the different versions of Class.php be managed?
How would you apply a security update for Class.php?
By the time we have solved all of
these issues, we have pretty much re-
invented Composer.
Pantheon.io
Drush startup now happens in four phases:
31
Drush Startup
Drush Finder Drush Wrapper Drush Launcher Drush Application
Pantheon.io 32
Drush Finder
Responsible for finding the correct Drush to run
- Checks to see if it can find a Drupal site
- If the Drupal site contains a Drush script, use it
- If no Site-Local Drush is found, use global Drush
Does not consider alias files.
PHP Scriptnamed
“drush”
Pantheon.io 33
Drush Wrapper
Optional. Located at the Drupal Root if it exists.
User may customize this script to:
- Add site-specific options (e.g. commandfile locations)- Turn off global search locations with --local option- Select the location of the Drush launcher (if non-standard)
If there is no Drush Wrapper, then the Drush Finder will find and execute the Drush Launcher.
Shell Scriptnamed
“drush.wrapper”
Pantheon.io 34
Drush Launcher
Sets up the PHP operating environment.
- Select php.ini file to use
- Select php executable to use
- Passes info about environment to Drush
The launcher will always use the Drush application located in the same directory.
Shell Scriptnamed
“drush.launcher”
Pantheon.io 35
Drush Application
Contains all the code that is Drush.
- Load configuration and command file
- Parse site alias files
Might dispatch again, e.g. if site alias is remote.PHP
Applicationnamed
“drush.php”
Pantheon.io 36
Drush Phar
Bundles all of the code from the Drush application into a single file.
- Does not use Drush Launcher (no PHP executable selection).
- Will run a Drush Wrapper script if one is available.
If a Drush Wrapper or site-local Drush is found, they will be executed via redispatch. The site-local Drush may be a Phar or a regular Drush application.
Phar file named
“drush.phar”
Phar
Pantheon.io 37
Sharing Drush Aliases
cd "`dirname $0`"
private/vendor/bin/drush.launcher \
--local \
--include=../drush/commands \
--alias-path=../drush/aliases \
--config=../drush/config "$@"
drush.wrapper
drush
web
drush.wrapper
aliases
commands
index.php
aliases.drushrc.php
$ cd web
$ drush sa @live
$aliases["live"] = array (
'remote-host' => 'server.isp.com',
'remote-user' => 'www-admin',
'root' => '/srv/www/live.example.com',
'uri' => 'http://example.com',
);
Pantheon.io 38
Behat Driver Enhancement
cd /path/to/drupal-projectcomposer require drush-ops/behat-drush-endpoint
YES!
Adds behat Drush command to your site; used to remotely create content.
Pantheon.io 39
Behat Fixtures
Background: Given "places" terms: | name | | Kingdom of Imaginarium | | Empire of Fabrication | And "offices" terms: | name | | Grand Poohbah | | Undersecretary of Things | | Minister of Ministering |
$ drush behat import --file=fixtures.yml
create_term: places: - name: Kingdom of Imaginarium - name: Empire of Fabrication offices: - name: Grand Poohbah - name: Undersecretary of Things - name: Minister of Ministering
Option 1: Use Behat’s built-in “Background”
Option 2: Create fixtures with behat-drush-endpoint
Deletes and re-creates fixtures for every test; useful for testing operations that modify or delete terms.
Taxonomy IDs stay constant for every test run.
ALSO (for Drupal 8): drush site-install --config-dir=export-dir
Pantheon.io 40
That’s a Wrap!
Follow me on Twitter for slides and other Composer + Drush updates:
@greg_1_anderson