A new tool for measuring performance in Drupal 8

Luca LussoSenior Drupal Developer @ Wellnet

• www.drupal.org/u/lussoluca• @lussoluca

Agenda

• Overview of existing profilers

• Introducing Webprofiler

• Drupal Console integration

• Future developments

Overview of existing profilers

How to measure performance

• Hardware/OS profilers

• Function-level profilers

• Application profilerswe focus on these

Overview of function-level profilers

Open source

• xDebug

• XHProf / uprofiler

• …

Commercial

• NewRelic

• Blackfire.io (free for open source projects)

• Zend z-ray (available in Zend Server 8)

• …

Strengths

• Very detailed analysis

• Useful to find out which functions spends a lot of time or memory

• can be used on production (sampling)

Weakness

• Difficult to install / commercial

• Difficult to analyze data (too much and too low level)

Overview of function-level profilers

Overview of application profilersIn Drupal we have the great Devel module

Query logger / Time and memory logger

The problem with Drupal (8)• A simple page with a couple of views can do some expensive queries

• Requesting a service causes the loading of all dependent services (if not declared as lazy)

• We can trigger of a lot of events

• Poorly implemented cache can cause heavy operations to be executed for every request

• A lot of css and js can slows down the page render

• Builded but not rendered blocks

• …

All of those problems are difficult to discover at PHP level, we need to profile the internals of Drupal.

Introducing Webprofiler

Symfony2 WebprofilerBuilt-in in Symfony2 full stack distribution

Code available in Drupal 8 and missing pieces

A lot of code was already available into the core. All the infrastructure for collecting and storing profiles were in place and there was an initial set of data collectors available.

• Symfony\Component\HttpKernel\Profiler (in Drupal 8)

• Symfony\Component\HttpKernel\DataCollector (in Drupal 8)

Code available in Drupal 8 and missing pieces

The missing piece was the WebProfilerBundle. In a Symfony full stack application a bundle is the equivalent of a Drupal module, the problem here is that Drupal 8 isn’t a Symfony full stack application. Drupal has its own structures and internals and uses some of the Symfony components to replace legacy code. Unfortunately a bundle doesn’t work on Drupal, we need a module.

• Symfony\Bundle\WebProfilerBundle (not in Drupal 8)

Drupal 8 WebprofilerAvailable as contrib module

http://www.drupal.org/project/webprofiler https://github.com/lussoluca/webprofiler

Implemented DataCollectors

PHP ConfigRequest

Timeline FrontendDatabase

UserViews

Block

Http

Extensions

EventsState

ConfigAssets

CacheRouting

ServiceForms

Every widget shows data about a metric measured by a data collector on the current rendered page

Workflow

Instrumented(collect data and

store profile)

Request Response

<div id="webprofilerefa2f1"></div> <script> Webprofiler = (function () { […] } </script>

Response

• Add HTTP header -> X-Debug-Token: efa2f1

• Inject some javascript in the page:

Workflow

Profiling turned off

Ajax Request Response

/profiler/efa2f1Returns only the

toolbar DOM (with embedded

javascript, css and icons)

Instrumentation• To profile our application we need to inject some

instrumentation code

• We have to replace the original code with a version that do the same thing but, at the same time, saves runtime data. In Drupal 8 this is fairly simple because we have services and a Dependency Injection Container

• These data must be saved somewhere to be used later for toolbar rendering and for further analysis

• We need to find the service that is in charge of manage the structure we want to profile (configurations reads from the Configuration Management in this case)

• All core services are defined in core/core.services.yml, contrib services are in modulename.services.yml

• In this example we instrument the “config.factory” service

config.factory: class: Drupal\Core\Config\ConfigFactory tags: - { name: event_subscriber } - { name: service_collector, tag: 'config.factory.override', call: addOverride }

Instrumenting the config.factory service

core/core.services.yml

Instrumenting the config.factory service

$container->setDefinition('config.factory.default', $container->getDefinition('config.factory'));$container->register('config.factory', 'Drupal\webprofiler\Config\ConfigFactoryWrapper') ->addArgument(new Reference('webprofiler.config')) ->addArgument(new Reference('config.factory.default'));

• Rename the original service

• Register a new service with the original name (config.factory) but with a different class (ConfigFactoryWrapper) and inject both the specific data collector service and the renamed service

Drupal\webprofiler\WebprofilerServiceProvider.php

Instrumenting the config.factory serviceclass ConfigFactoryWrapper implements ConfigFactoryInterface { protected $configFactory; protected $configDataCollector; public function __construct(ConfigDataCollector $configDataCollector, ConfigFactoryInterface $configFactory) { $this->configFactory = $configFactory; $this->configDataCollector = $configDataCollector; } public function get($name) { $result = $this->configFactory->get($name); $this->configDataCollector->addConfigName($name); return $result; }[..]}

Forward to the original method

Collect datas

Drupal\webprofiler\Config\ConfigFactoryWrapper.php

Instrumentation

Note that instrumenting a program can cause performance issues, and may in some cases lead to inaccurate results.

Webprofiler should not be used in a production environment.

Demo time

Integration with XHProf/UProfilerDrupal 8 version of XHProf module is compatibile with

Webprofiler (http://www.drupal.org/project/xhprof)

admin/config/development/profiler/configureadmin/config/development/xhprof

Storage backends

• Database (done)

• Filesystem (done)

• Elasticsearch (work in progess https://www.drupal.org/node/2356123)

Drupal Console integration

Drupal Console

• Leverages the Symfony Console Component to provide a powerful CLI

• Started as scaffolding generator now expose an increasing number of commands to interact with a Drupal 8 installation

• Modules can contribute implementing new commands

Implemented commands

• webprofiler:list -> list stored profiles

• webprofiler:export -> export stored profiles

• webprofiler:benchmark -> benchmark a site running a huge number of request and computing statistics:

$> console webprofiler:benchmark http://d8 --runs=10

Future developments

Open issues• Collect initializer time/amount of dependencies for container services

(#2353253)

• Collect performance metrics for services over time (#2349935)

• Show database performance metrics commonly found in MySQL-Tuner.pl (#2349923)

• Collect slow queries and report them over time (#2349911)

• Collect rebuilding time of important caches (e.g. menu router, container, etc.) (#2349867)

• Collect cache / DB services timing metrics (#2349845)

• …

Better UI/UX

• ajax

• backbone

Profiles DiffCompute and show diff of two different profiles

More offline statistics via Drupal Console

• Queries

• Cache hit/miss

• Events

• …

Thanks!