Drupal and Cache

Olivier Briat

@Eiffel_Alec

THANKS TO THE SPONSORS

Diamond Sponsor

Platinum Sponsor

Gold Sponsor

Olivier Briat

Mugface
Capgemini logo

Inspirations

Proudly found elsewhere

  • Drupal 8 Caching overview / Berdir / Drupal Dev Days Seville 2017
  • Render API & Cache API / Artusamak / Drupal Camp Nantes 2016
  • Drupal 8 Caching - A Developer’s Guide / Peter Sawczynec / Pacific Northwest Drupal Submit 2016

Topics

  • Caches upstream of Drupal
    • Browser
    • Network caches
  • Internal Page Cache (aka "Anonymous cache")
  • Dynamic Page Caching
  • Render API cache system
  • Using Drupal cache mechanisms in your custom code
  • Cache storage
  • Questions ?

What is cache?

Store the result of slow calculations so that they do not need to be executed again and so improve performances

Cache mecanisms upstream of Drupal

The first cache is the browser one

  • The url is the browser cache key
  • The browser will only check for updates after max-age seconds or at expires
  • The checksum used is usualy ETag or Last-Modified
  • It will get a new content or a 304 response
  • Vary indicates alternate version of cache content : encoding, cookies, ...
  • We'll see later the X-Drupal headers

Drupal settings

Assets cache settings

  • For files other than a Response object, max-age could be tune in .htaccess
# Requires mod_expires to be enabled.
<IfModule mod_expires.c>
  # Enable expirations.
  ExpiresActive On

  # Cache all files for 2 weeks after access (A).
  ExpiresDefault A1209600

  <FilesMatch \.php$ >
    # Do not allow PHP scripts to be cached unless they explicitly
    # send cache headers themselves(...)
    ExpiresActive Off
  </FilesMatch>
</IfModule>

Before going any further

Unlike Drupal 7, cache is enable by default in Drupal8, so...

it is strongly recommended to disable it when coding

You just have to uncomment the following lines in settings.php

#Copy sites/example.settings.local.php to sites/default/settings.local.php
if (file_exists(__DIR__ . '/settings.local.php')) {
  include __DIR__ . '/settings.local.php';
}

or use drupal console: drupal site:mode dev @see https://www.drupal.org/node/2598914

settings.local.php
# Load development services
$settings['container_yamls'][] = DRUPAL_ROOT . '/sites/development.services.yml';
# Display verbose errors
$config['system.logging']['error_level'] = 'verbose';
# Do not minify css & js
$config['system.performance']['css']['preprocess'] = FALSE;
$config['system.performance']['js']['preprocess'] = FALSE;
# Disable render caching
# $settings['cache']['bins']['render'] = 'cache.backend.null';
# Disable dynamic page caching
# $settings['cache']['bins']['dynamic_page_cache'] = 'cache.backend.null';
development.services.yml
parameters:
  # Enable cache debug headers
  http.response.debug_cacheability_headers: true
services:
  # Enable a "null" cache storage
  cache.backend.null:
    class: Drupal\Core\Cache\NullBackendFactory
  

But, must of all

Don't forget to re-enable cache before testing your code!

Network caches

On their way to Drupal, requests could pass through:

  • Proxies: which respect max-age & s-max-age headers
  • Content Delivery Network (CDN) : Akamaï, Cloudflare, ...
  • Varnish:
    • Drupal best buddy
    • Works like a proxy (default cache ttl is 2mn)
    • Swiss Army knife for HTTP headers and requests
    • Could serve your website even if it's down (grace mode)
    • Could replace Drupal Internal Page Cache
  • Must of them could use X-Drupal-Cache-Tags headers
  • They also can be managed with the purge module and its sub-modules (varnish_purge, akamai, ...) and the HTTP Cache Control module

Internal Page Cache

Anonymous cache

  • Managed by the Internal Page Cache module
  • It caches fully rendered HTML responses
  • Only for anonymous visitors (no session)
  • Really fast
  • It also uses URL as the cache key
  • It's based only on the cache tags metadata send by Drupal
  • It respects the Expires header on the response Object
  • As seen before, it could be disabled if an external page cache system is used (like Varnish)
  • Debugging HTTP Header: X-Drupal-Cache: HIT #or MISS

Dynamic Page Caching

Dynamic Page Cache

  • Managed by the Dynamic Page Cache module
  • Much slower than Anonymous cache
  • Works for anonymous and authenticate users!
  • Is invalidated by cache metadata that come from page contents
  • It does not cache page contents that are too dynamic, they are handle by "Lazyloading"

Render API cache system

Render API cache system

  • The page content are built with the render API, it's where most of the cache meta-data come from
  • They are stored in the #cache key of render arrays:
    • keys: Name this cache part and marks it as cacheable, e.g.: ['node', 5, 'teaser']
    • max-age*:
      • -1: Permanent
      • X: Age in seconds
      • 0: Do not cache
    • context: Allows cache variations according to: theme, language, user roles, permissions, URL, QS, timezone,...
    • tags: Allows to invalidate all caches tagged with them: node:x, config:, user:x, library_info, route_match, node_list, ...
  • * This will not change the max-age HTTP header or the anonymous cache duration, see: https://www.drupal.org/node/2352009 and https://www.drupal.org/project/cache_control_override

Bubbling

They bubble from bottom to top arrays


Child element render array

Parent element render array

Bubbling

  • These cache metadata aggregate to form the whole page cache metadata
  • As seen before they are used to invalidate Dynamic Page Cache
  • Internal Page Cache is only invalidated by tags
  • External caches like Varnish could use X-Drupal-Cache-Tags HTTP headers
  • These headers are also useful for debugging

Here's a page with two blocks containing each a nodes list (cache tags are displayed between brackets, notice the bubbling)


Diagram by @berdir

Let's modify node n°5 and create a new sport node (n°7).
These actions will invalidate cache tags (red)

Here in red all the content parts that should be rebuild, in green the ones that will come from Render caching

Code example: setting cache metadata on a Render Array

$config = \Drupal::config('system.site');
$current_user = \Drupal::currentUser();

$build = [
  '#markup' => t('Hi, %name, welcome on @site!', [
    '%name' => $current_user->getUsername(),
    '@site' => $config->get('name'),
  ]),
  '#cache' => [
    'contexts' => [
      'user', // So this block will be processed by #lazybuilding
    ],
    'tags' => $config->getCacheTags(), // Same as 'tags' => ['config:system.site']
  ],
];
// Another way to add cache depency.
$renderer = \Drupal::service('renderer');
$renderer->addCacheableDependency(
  $build,
  \Drupal\user\Entity\User::load($current_user->id())
);
// Merge cache tags
$tags = Cache::mergeTags($conf_one->getCacheTags(),$conf_two->getCacheTags());

Objects that render content (like plugins) should implement the CacheableDependencyInterface and its methods:

\Drupal\Core\Cache

public function getCacheContexts() {
  return Cache::mergeContexts(
    parent::getCacheContexts(),
    ['user.node_grants:view']
  );
}

public function getCacheTags() {
  return Cache::mergeTags(parent::getCacheTags(), ['node_list']);
}

public function getCacheMaxAge() {
  return 0;
}
About blocks, they have to render content through twig template or these method will not be taken into account:
https://www.previousnext.com.au/blog/ensuring-drupal-8-block-cache-tags-bubble-up-page

Plugins could also inherit cache by using annotation context

Here this block will inherit the node cache metadata

*   context = {
*     "node" = @ContextDefinition("entity:node", label = @Translation("Node"))
*   }
...
$this->getContextValue('node');

See:
https://drupal.stackexchange.com/questions/199527/how-do-i-correctly-setup-caching-for-my-custom-block-showing-content-depending-o
https://api.drupal.org/api/drupal/core!core.api.php/group/annotation/8.5.x
https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Annotation!ContextDefinition.php/group/plugin_context/8.5.x

#lazy_builder (Auto-placeholdering)

  • To prevent highly dynamics content to systematically invalid page cache, Dynamic Page Cache replaces them with placeholders before writing cache.
  • Each time the cache is requested their content will be rendered on the fly.
  • This mechanism is also used by Bigpipe (now enable by default in 8.5)

https://www.drupal.org/docs/8/api/render-api/auto-placeholdering
Bigpipe demo

Lazybuilder triggering:

  • Could be set manually with a #lazy_builder key:
    // Callback : class:method or (better) service:method
    return [
      '#lazy_builder' => ['hello_world.lazy_builder:renderSalutation', []],
      '#create_placeholder' => TRUE,
    ];
  • be automatically detected by Drupal, following the rules defined into core/core.services.yml:
    renderer.config:
      auto_placeholder_conditions:
        max-age: 0
        contexts: ['session', 'user']
        tags: []
  • you could change them or add your own ones.

Drupal already handles all these cache mechanisms, so "everything is cool"

But you will certainly need to cache your own custom data, so here's the cache API

Static variables

  • Simple "caching" method using the fact that php static variables aren't destroyed at the end of a function execution
  • drupal_static() could still be used for procedural code
  • As for OO, don't forget that "services" are "singletons" and therefore their properties are persistents

Drupal cache Getter / Setter

function my_custom_data_processing();
  // Define my cache key
  $key = 'my_module' . ':' . __FUNCTION__;
  // Does the cache exists ?
  if ($cache = \Drupal::cache()->get($key)) {
    $data = $cache->data;
  }
  // No cache: process the data and set the cache.
  else {
    $data = my_slow_data_process();
    \Drupal::cache()->set($key, $data);
  }
  return $data;
}

Multiples

\Drupal::cache()->getMultiple(['key1','key2',...]);
\Drupal::cache()->setMultiple(['key1' => ['data' => 1],'key2' =>...]);

Delete cache

// Deleting cache (fast)
\Drupal::cache()->delete('my_module:cached_data');
\Drupal::cache()->deleteMultiple([
  'my_module:key1',
  'my_module:key2',
  ...
]);
\Drupal::cache()->deleteAll();;

Set an expiration

// Add  a cache lifetime to my cache (timestamp)
\Drupal::cache()->set($key, $data, REQUEST_TIME + 600);

Add tags

// Add tags to my cache
\Drupal::cache()->set(
  $key,
  $data,
  Cache::PERMANENT,
  [
    'tag1',
    'node:1',
    'config:system.menu',
    'config:my_module',
  ]
);

Get tags

// Get tags from an object that implements CacheableDependencyInterface:
$node->getCacheTags();
\Drupal\views\Entity\View::load('front')->getCacheTags();

// Get "List" cache tags associated with this entity type
// Will detects newly created entities
$em = \Drupal::entityTypeManager();
//config:taxonomy_vocabulary_list
$em->getDefinition('taxonomy_vocabulary')->getListCacheTags();
// taxonomy_term_list
$em->getDefinition('taxonomy_term')->getListCacheTags();

Invalidate tags

// Should implements CacheTagsInvalidatorInterface
// my_tag invalidation counter = 0
$cache_tag_invalidator->invalidateTags(['my_tag']);
// my_tag invalidation counter = 1
Cache::invalidateTags(['my_tag']);
// my_tag invalidation counter =2

On cache writing, all tags invalidation counter are sum up to provide the cache checksum.

On cache reading, the checksum is computed again, if it differs, the cache is rebuilt.

Return stale cache while building the new one

$cache = \Drupal::cache()->get('my-key', TRUE);
if ($cache && $cache->valid) {
  return $cache->data;
} elseif (\Drupal::lock()->acquire('my-key')) {
  // Lock the cache and rebuild it.
  $data = my_slow_data_process();
  \Drupal::cache()->set('my-key', $data);
  return $data;
} elseif ($cache) {
  // Someone else is rebuilding, work with stale data.
  return $cache->data;
} else {
  // Wait or rebuild.
}

Cache Storages

Cache Bins

Cache is split into containers:

  • default: Default, for small-ish, few key variations
  • data : Bigger caches, many key variations
  • discovery: Small, frequently used, usually for plugins and similar discovery processes
  • bootstrap: Drupal bootstraping
  • render: HTML rendering cache
  • config: Used for caching configuration
  • static: Memory only, when persistence is not desired

Each could be linked to a specific storage

You could define your own bin as a service

#yourmodule.services.yml
  cache.your_bin:
    class: Drupal\Core\Cache\CacheBackendInterface
    tags:
      - { name: cache.bin }
    factory: cache_factory:get
    arguments: [your_bin]

and enable it in settings.php

// Use redis by default.
$settings['cache']['default'] = 'cache.backend.redis';

// Use the null backend to disable caching for certain bins.
$settings['cache']['bins']['render'] = 'cache.backend.null';

Cache Backends

Where cache are actually stored

  • Core backends (core/core.services.yml):
    • Memory: not persisted across requests
    • Database: Default storage in the (SQL) database (invalidated, never deleted*)
    • APCu: Shared-Memory inside the PHP process, not shared with drush/CLI/multiple servers
    • Null: for disabling cache (dev)
  • Contributed backends:
    • Slushi Cache: A database backend with a configurable max lifetime
    • Memcache : RAM object database server cache.backend.memcache_storage
    • Redis: RAM Key/value database server cache.backend.redis

ChainedFast Backend (cache.backend.chainedfast) allow to "chained" a fast backend on top a slow but shared one.

* : Since 8.4 SQL bins have a max size.

Questions / Feedback?

@Eiffel_Alec

Links & Resources

Refrences

## Conferences - https://md-systems.github.io/drupal-8-caching/ - https://nantes2016.drupalcamp.fr/programme/sessions/render-api-cache-api (fr) - https://pnwdrupalsummit.org/sites/default/files/slides/Drupal%208%20Caching.pdf - Drupal 8 cache for developers - José Jiménez Carrión #DrupalCampES @jjca : https://youtu.be/kfy_JAKudnw - Drupal 8 Caching: A Developer’s Guide : https://youtu.be/eB4NWo5XwMY - BigPipe : https://youtu.be/JwzX0Qv6u3A
## APIs & documentation - https://api.drupal.org/api/drupal/core!core.api.php/group/cache/8.5.x - https://www.drupal.org/docs/8/administering-drupal-8-site/internal-page-cache - https://www.drupal.org/docs/8/core/modules/dynamic-page-cache/overview - https://www.drupal.org/docs/8/api/render-api/auto-placeholdering - https://www.drupal.org/docs/8/api/render-api/cacheability-of-render-arrays - https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching - https://www.drupal.org/project/renderviz

Book

  • Drupal 8 Module development de Daniel Sipos (upchuk)
## Servers - https://varnish-cache.org - https://redis.io/documentation