A Study on a Custom Laravel CMS's Caching System

Posted by Adam Engebretson on September 23, 2013

The CMS I have been working on for an insurance company in Southern New England has proven to be quite powerful, including many facets of the company's web CMS needs. Being it compiles many different items to render a front-end web page, the load time can be quite high. For our home-page to render (after a cache flush), a total of 1776 MySQL database queries, 162 files, and 26.25MB of memory are consumed over 3112.96 milliseconds. Obviously, this won't cut it for a production website.

I've integrated a cache system that off-puts a lot of the rendering on most pages. After the cache system is in place, the 2nd page load of the home page only requires 18 MySQL database queries, 152 files, and 11.75 MB of memory over 986.55 milliseconds. This powerful feature of our CMS is intrinsic to the functionality of our site. Let's take a look at how it works.

How I Did It

I'm using a FrontEndController to handle all of the requests. Multiple routes for multiple URL schemes all point to a specific method inside the controller, all of which then call a parseContent($bypassCache = false) method. Here's the full parseContent() method, but we'll break it down and go through it together.

private function parseContent($bypassCache = false) {
  if(!($this->content instanceof Content))
    App::abort(404);

  if(!$this->content->status) {
    if($this->content->redirectOn404 instanceof Content)
      return Redirect::to($this->content->redirectOn404->slug);
    else
      App::abort(404);
  }

  $this->club = $this->content->club();
  $cacheKey = $this->content->club()->sub_domain.$this->content->category->slug.$this->content->slug.'page'.Input::get('page');

  if($this->content->symlinkedContent instanceof Content)
    $this->content = $this->content->symlinkedContent;

  $viewData = [
    'club' => $this->club,
    'content' => $this->content,
    'breadcrumbs' => $this->bc
  ];

  if(Cache::has($cacheKey) && !$bypassCache) {
    Log::info("Pulled response from cache: {$cacheKey}.");
    return Blade::fromString(Cache::get($cacheKey), $viewData);
  }

  $dom = str_get_html(View::make($this->club->viewPrefix().$this->content->layout->view_string, $viewData));

  $trackers = Tracker::where(function($query){
    $query->where(function($query){
      $query->whereModel('club')
        ->whereModelId($this->content->club()->id);
    })->orWhere(function($query){
      $query->whereModel('category')
        ->whereModelId($this->content->category->id);
    })->orWhere(function($query){
      $query->whereModel('content')
        ->whereModelId($this->content->id);
    });
  })->where(function($query) {
    $query->where('start_at', '<', DB::raw('NOW()'))->where('start_at', '!=', '0000-00-00 00:00:00');
  })->where(function($query) {
    $query->where('end_at', '>', DB::raw('NOW()'))->orWhere('end_at', '=', '0000-00-00 00:00:00');
  })->get();

  foreach($dom->find('link') as $element) {
    $element->href = $this->club->assetPrefix().$element->href;
  }
  foreach($dom->find('script[src]') as $element) {
    if(!starts_with($element->src, 'http'))
      $element->src = $this->club->assetPrefix().$element->src;
  }
  foreach($dom->find('comment') as $comment) {
    $new = str_replace('src="', 'src="'.$this->club->assetPrefix(), $comment->outertext);
    $new = str_replace('href="', 'href="'.$this->club->assetPrefix(), $new);
    $comment->outertext = $new;
  }
  foreach($dom->find('img[src]') as $element) {
    if(!starts_with($element->src, 'http') && !starts_with($element->src, $this->club->assetPrefix()))
      $element->src = $this->club->assetPrefix().$element->src;
  }

  foreach($trackers as $tracker) {
    $tag = $dom->find($tracker->tag, 0);
    if($tracker->pend == 'prepend')
      $tag->innertext = $tracker->tracker.$tag->innertext;
    else
      $tag->innertext = $tag->innertext.$tracker->tracker;
  }

  if($bypassCache)
    return $dom;

  Cache::add($cacheKey, (string)$dom, 999);
  return Blade::fromString(Cache::get($cacheKey), $viewData);
}

Alrighty, the first thing we do is make sure that the router method from earlier did, in fact, define $this->content with a valid Content instance.

if(!($this->content instanceof Content))
    App::abort(404);

Easy enough, right? Next, we'll check to see whether or not said Content is published. If not, we'll either redirect (configured by the end-user in the CMS) or throw a 404.

if(!$this->content->status) {
  if($this->content->redirectOn404 instanceof Content)
    return Redirect::to($this->content->redirectOn404->slug);
  else
    App::abort(404);
}

Next, I'll define $this->club and $cacheKey. Don't worry about the former; the latter is where the magic begins to happen. $cacheKey is a unique string that's associated with any particular piece of content that will be used to set and access it's cached content.

$this->club = $this->content->club();
$cacheKey = $this->content->club()->sub_domain.$this->content->category->slug.$this->content->slug.'page'.Input::get('page');

The next couple lines are an attempt at a "symlinking" feature that, in theory, works great, but in practice, doesn't. I haven't gotten around to cleaning this up yet, but for now, I simply replace $this->content with the symlinked piece of content, effectively displaying the referenced Content's parameters.

if($this->content->symlinkedContent instanceof Content)
    $this->content = $this->content->symlinkedContent;

Next, we define $viewData. You'll see why I wanted to do this before the actual View::make() call later. (Hint: it's because there are multiple View::make() calls).

$viewData = [
  'club' => $this->club,
  'content' => $this->content,
  'breadcrumbs' => $this->bc
];

Given all of this setup, we can now see whether or not we already have a cached vaule for this particular piece of content. We have the ability to bypass the cache by setting $bypassCache when parseContent() is called. This, of course, defaults to false. We throw a log, for debugging purposes, and then... wait... wtf is Blade::fromString()? Stand by, and I'll tell you later! Just pretend it's similar to View::make() for now. Notice how we used $viewData?

if(Cache::has($cacheKey) && !$bypassCache) {
  Log::info("Pulled response from cache: {$cacheKey}.");
  return Blade::fromString(Cache::get($cacheKey), $viewData);
}

Now, assuming that we don't have the cached version of this page, and/or we are using $bypassCache, let's go to render the page. We are using str_get_html(), which is a simple DOM parser for PHP (found here) to manipulate path names and insert HTML Trackers throughout the page. The thing to note here is the use of $viewData, but other than that, you can skip through most of this.

$dom = str_get_html(View::make($this->club->viewPrefix().$this->content->layout->view_string, $viewData));

$trackers = Tracker::where(function($query){
  $query->where(function($query){
    $query->whereModel('club')
      ->whereModelId($this->content->club()->id);
  })->orWhere(function($query){
    $query->whereModel('category')
      ->whereModelId($this->content->category->id);
  })->orWhere(function($query){
    $query->whereModel('content')
      ->whereModelId($this->content->id);
  });
})->where(function($query) {
  $query->where('start_at', '<', DB::raw('NOW()'))->where('start_at', '!=', '0000-00-00 00:00:00');
})->where(function($query) {
  $query->where('end_at', '>', DB::raw('NOW()'))->orWhere('end_at', '=', '0000-00-00 00:00:00');
})->get();

foreach($dom->find('link') as $element) {
  $element->href = $this->club->assetPrefix().$element->href;
}
foreach($dom->find('script[src]') as $element) {
  if(!starts_with($element->src, 'http'))
    $element->src = $this->club->assetPrefix().$element->src;
}
foreach($dom->find('comment') as $comment) {
  $new = str_replace('src="', 'src="'.$this->club->assetPrefix(), $comment->outertext);
  $new = str_replace('href="', 'href="'.$this->club->assetPrefix(), $new);
  $comment->outertext = $new;
}
foreach($dom->find('img[src]') as $element) {
  if(!starts_with($element->src, 'http') && !starts_with($element->src, $this->club->assetPrefix()))
    $element->src = $this->club->assetPrefix().$element->src;
}

foreach($trackers as $tracker) {
  $tag = $dom->find($tracker->tag, 0);
  if($tracker->pend == 'prepend')
    $tag->innertext = $tracker->tracker.$tag->innertext;
  else
    $tag->innertext = $tag->innertext.$tracker->tracker;
}

Now that we've effectively manipulated $dom, let's ask ourselves: 'Are we bypassing the cache with $bypassCache?' If so, we'll just dump that $dom variable, and ta-da. We have a webapge! Otherwise, let's move forward to cache it.

if($bypassCache)
  return $dom;

This is where the magic happens. Now that we know that we would, in fact, like to cache this content, we add it to the cache using its $cacheKey, and then return it. Again with this illusive Blade::fromString() method? Yeah, let's talk about that....

Cache::add($cacheKey, (string)$dom, 999);
return Blade::fromString(Cache::get($cacheKey), $viewData);

Some Weird Stuff

The client has made a request for two features:

  • The ability to use blade syntax (i.e. {{Form::open()}}) inside the Content model's parameters, and
  • The ability to have a certain part of the page not be cached, whie others are (i.e. "Welcome {logged-in-user-name}.").

Obviously, since we're caching the entire response from the parsed view, it'd be difficult to just snip out a part of that and tell it not to cache. So we had to manipulate the Blade Compiler. To do this, we first had to overwrite a couple things in the ViewServiceProvider. Here's what we did:

<?php namespace AAACMS;

use Illuminate\Support\MessageBag;
use Illuminate\View\Engines\PhpEngine;
use Illuminate\Support\ServiceProvider;
use Illuminate\View\Engines\BladeEngine;
use Illuminate\View\Engines\CompilerEngine;
use Illuminate\View\Engines\EngineResolver;
use AAACMS\BladeCompiler;

class ViewServiceProvider extends \Illuminate\View\ViewServiceProvider {

  /**
   * Register the Blade engine implementation.
   *
   * @param  \Illuminate\View\Engines\EngineResolver  $resolver
   * @return void
   */
  public function registerBladeEngine($resolver)
  {
    $app = $this->app;

    $resolver->register('blade', function() use ($app)
    {
      $cache = $app['path.storage'].'/views';

      // The Compiler engine requires an instance of the CompilerInterface, which in
      // this case will be the Blade compiler, so we'll first create the compiler
      // instance to pass into the engine so it can compile the views properly.
      $compiler = new BladeCompiler($app['files'], $cache);

      return new CompilerEngine($compiler, $app['files']);
    });
  }

}

You might notice that this is almost identical to the ViewServiceProvider provided by Illuminate, but there are a couple slight changes.

First of all, we namespaced it differently. This is so that we can call it instead of that provided by Illuminate in app/config/app.php. Additionally, we are using AAACMS\BladeCompiler instead of Illuminate\View\Compilers\BladeCompiler.

Next, we extend Illuminate's ViewServiceProvider, so that we only need to write code for the methods that we'd like to change. We then define public function registerBladeEngine(). This code is identical to that in Illuminate's ViewServiceProvider, but since we've redefined the namespace for BladeCompiler, this will use our namespaced compiler instead.

Finally, we updated app/config/app.php to use our new namespaced service provider.

We had to do all of this so that we could make some minor changes to the BladeCompiler. Specifically, to manipulate protected $compilers. This was necessary, because wanted to put Extensions at the bottom of this array, rather than at the top. You'll see why in a minute.

Now that we have control over which BladeCompiler is used around the install, we'll overwrite it.

<?php namespace AAACMS;

use Closure;
use Illuminate\Filesystem\Filesystem;
use Illuminate\Support\Facades\File;
use Illuminate\Support\Facades\View;

class BladeCompiler extends \Illuminate\View\Compilers\BladeCompiler implements \Illuminate\View\Compilers\CompilerInterface {

  /**
   * All of the available compiler functions.
   *
   * @var array
   */
  protected $compilers = array(
    'Extends',
    'Comments',
    'Echos',
    'Openings',
    'Closings',
    'Else',
    'Unless',
    'EndUnless',
    'Includes',
    'CachedIncludes',
    'Each',
    'Yields',
    'Shows',
    'Language',
    'SectionStart',
    'SectionStop',
    'SectionOverwrite',
    'Extensions',
  );

  public function fromString($string, $data) {
    $path = app_path().'/views/tmp/';
    $filename = md5($string);

    File::put($path.$filename.".blade.php", $string);
    return View::make('tmp.'.$filename, $data);
  }

}

Again, not a lot has changed here. We are simply extending Illuminate\View\Compilers\BladeCompiler in our own namespace. We re-arranged the $compilers array, and here's why.

We created a Blade extension to replace {! and !} tags with {{ and }} tags. If the Extension was compiled first, the new {{ and }} tags would then be compiled into PHP echos. We didn't want this.

Blade::extend(function($value) { return preg_replace('/\{\!(.+)\!\}/', '{{ ${1} }}', $value); });

Why didn't we want this? Because of what the fromString method does. What we do is take the $string parameter, save it to a 'tmp' directory in our views folder, and then pass that into View::make(). This is the accomplishment of goal #1 (outlined above). Now, when the 'content' attribute of Content is called, and it includes blade syntax, it gets rendered here!

public function fromString($string, $data) {
  $path = app_path().'/views/tmp/';
  $filename = md5($string);

  File::put($path.$filename.".blade.php", $string);
  return View::make('tmp.'.$filename, $data);
}

Moreover, now that we replace {! and !} tags with {{ and }} tags AFTER the compileEchos() method is called on Illuminate\View\Compilers\BladeCompiler (link), the {{ and }} tags are saved into the temporary view file, and then send through the View::make() compiler AFTER we define its value in the cache. This is the accomplishment of goal #2 (outlined above).

Recap

In order to accomplish this task, we simply put the output of our pages in a cached string with a unique $cacheKey, and check to see whether it exists when we call the page.

We had to extend the BladeCompiler class, and in order to do so, the ViewServiceProvider as well. This allowed us to customize the way that the BladeCompiler handles our extension to replace {! and !} tags with {{ and }} tags, as well as allow users to add blade syntax in the CMS.

Thanks for reading! If you liked this article, please make sure to give it a point on HackerNews and follow me on twitter @artisangoose.