You are here

Speed up drush make

When using drush make on projects where each repository (or module) have their own makefiles and these declares the same modules over and over again drush will download the same data more than once. The code here will utilize drush's cache to only download a module or library once.

Start by creating a new drush include PHP file in your .drush folder.

~$ nano -w ~/.drush/make_utils.drush.inc

Add the code below to the file. The code uses hook_validate_info() to filter the information parsed by drush make, hence remove modules and libraries seen before. It uses drush's cache to store seen modules/libraries for 10 minutes, so if you build takes more than this you may want to change the value (600) in the code. It uses drush's cache because make uses threads and a static variable will not be available across threads.

<?php
/**
* Implements hook_drush_make_validate_info
*/
function make_utils_validate_info($info) {
  // This options help speed up the build process by only download a given
  // project the first time it's referenced in a makefile.
  if (drush_get_option('only-once', FALSE)) {
    // The list of seen projects are cached in drush's cache due to the fact
    // that drush creates a new process for each makefile, hence static will not
    // work. So to store the information across processes we use the cache with
    // a 10 minute expire.
    $project_seen = array();
    $cache = drush_cache_get('drush_make_project_seen', 'default');
    if ($cache && $cache->expire > time()) {
      $project_seen = $cache->data;
    }

    // Get names of projects and libraries.
    $names = array();
    if (isset($info['projects'])) {
      $names += array_keys($info['projects']);
    }
    if (isset($info['libraries'])) {
      $names += array_keys($info['libraries']);
    }

    // Loop over the names found in current makefile.
    foreach ($names as $name) {
      if (in_array(trim($name), $project_seen)) {
        // Project seen before so simply remove it form the list.
        drush_print('Project seen before: ' . $name);
        if (isset($info['projects'][$name])) {
          unset($info['projects'][$name]);
        }
        else {
          unset($info['libraries'][$name]);
        }
      }
      else {
        // Add project to the seen list.
        $project_seen[] = trim($name);
      }
    }
    // Store information in the cache for 10 minutes.
    drush_cache_set('drush_make_project_seen', $project_seen, 'default', 600);
  }

  return $info;
}

Usage

Simply add "--only-once --strict=0" to your normal drush make command before the make command or simply use the command below.

drush --only-once --strict=0 make --concurrency=1 drupal.make

My own build process have gone from 45-50 minutes to around 5 minutes. If you need to run the build process within 10 minutes of each other, you will need to clear drush cache.

~$ drush cc drush

Tags

drush make makefiles Drupal 7

Add new comment