How to migrate content from drupal 6 to 7 by using Migrate_d2d - Part 3 - files

This is the third part in a series of blog posts about how to use of migrate and migrate_d2d to migrate content and files from Drupal 6 to 7. In the third part will be about files. Part 1 - Some background information, setup and user migration. contains information about how migration in general works and the needed configuration. Please make sure you read, and understand it, before attempting to follow this example.

Blog post in this series

Migrating files from Drupal 6

Files added to Drupal is also tracked by drupal. In drupal 6 is this information stored in in a database table named files. Files you have manually upload to the drupal file system area (normally sites/default/files) with scp/ftp and so on, are not tracked by drupal. The files cannot be migrated, at least not in a easy way.

Migrating files have to be done before migrating nodes. Migrate will keep track of what nodes the files are coming from. During the node migrating will Migrate try to reassign the files to the new nodes. Those nodes can also have new node id, Migrate will address this also.

Drupal 6

You should already have created the needed nodes with with devel generate back in Part 1. If not, please create them.

Drupal 7

You should already have created the needed field back in Part 1.

Example 3 - Migrate files from Drupal 6

Grab the code from github and clone it into your Drupal 7 sites/all/modules/ directory.

git clone --branch example3 https://github.com/steinmb/smb_migrate.git

Or if you already have cloned it by following example 1, simply switch branch:

git checkout example3

Register and de-register statically defined classes

After the module is enabled does Migrate require you might have to register your migration task (class) (Introduced by Migrate 7.x-2.6). Running drush ms should list your taxonomy migration tasks if not you have to register them by running the following drush command:

drush migrate-register

You should see something like "All statically defined migrations have been (re)registered". If UI is your thing visit admin/content/migrate/configure and press the button labeled "Register statically defined classes". This should take you to the migration overview page that should list this migration.

You can also remove registered import tasks and groups with migrate-deregister

drush migrate-deregister --help

lists available options. Notice this is also how you remove orphans. They are normally migrations groups and tasks left behind after you alter and/or remove migration tasks during development.

Migrate status

To view the state and verify that the new migration task was registered run migrate status:

drush ms

It should return something along these lines:

Group: content_migration  Total  Imported  Unprocessed  Status
User                      51     0         0            Idle
File                      1601   0         1601         Idle

Group: taxonomy           Total  Imported  Unprocessed  Status
Tags                      14     0         14           Idle
Tags2                     17     0         17           Idle
Tags3                     19     0         19           Idle

My Drupal 6 system seems to have 1601 files waiting to be migrated.

Migrating files and/or images

Start the file migration as you would start any migration task:

drush mi File

Migrate should give something along these lines.

Processed 1601 (1601 created, 0 updated, 0 failed, 0 ignored) in 12.6 sec (7616/min)

Code breakdown

I will here only review the code that have been added/changed since part 1 and 2.

smb_migrate.migrate.inc

<?php
 
// Register the file migration.
  // source_dir needs to be adjusted to match your file system.
 
$api['migrations']['File'] = $common_arguments + array(
   
'machine_name' => 'filemigration',
   
'description' => t('Import Drupal 6 files'),
   
'class_name' => 'DrupalFile6Migration',
   
'user_migration' => 'User',
   
'default_uid' => 1,
   
'source_dir' => DRUPAL_ROOT . '/../drupal-6.x-dev',
   
'destination_dir' => 'public://legacy_files',
  );
?>

Most of this is have you already seen created in code mention in part 1 and 2 in this series, though there is a few new setting worth mentioning.

  • user_migration would like to know the name of your user migration. In part 1 did we cover this, we named it User. File migration depend on that Users first have been migrated.
  • default_uid this is the user id the file will owned by if the user owning the file in Drupal 6 no longer exists.
  • source_dir this must point to the base directory where drupal 6 lives. On my development machine is this drupal-6.x-dev. Alter the code or create a symlink in your system to match this.
  • destination_dir this is where migrate should put your migrated files. I decided to store them all inside a directory called legacy_files. If your files system permission is correct will migrate create this directory on the first file migration.

Example 3-1 Optional step. Alter file migration destination class

The example above is the bare minimum you need. This entire series is about trying to demonstrate how little code you need to get migration started. Still there is time when you need and want to alter default behavior and use your own migration class.

<?php
 
// Register the file migration.
  // source_dir needs to be adjusted to match your file system.
 
$api['migrations']['File'] = $common_arguments + array(
   
'machine_name' => 'filemigration',
   
'description' => t('Import Drupal 6 files'),
   
'class_name' => 'DrupalFile6Migration',
   
'user_migration' => 'User',
   
'default_uid' => 1,
   
'source_dir' => DRUPAL_ROOT . '/../drupal-6.x-dev',
   
'destination_dir' => 'public://legacy_files',
  );
?>
'class_name' => 'DrupalFile6Migration'

define what destination class we are using. Migrate D2D comes with it's own versions of these classes, DrupalFile6Migration is one of them (located in: migrate_d2d/d6/file.inc:class DrupalFile6Migration). This class define the among many thing the standard behavior we are aiming to alter in this example.

Code

You might experience some problems getting your new migration class registered. It might be the safest thing to de-register the exiting File migration.

drush migrate-deregister File

I have some ready baked code that demonstrate how to alter the default behavior in your own class.

git checkout example3-1

smb_migrate.info

Now contain a new entry telling the autoloader about file.inc.

files[] = file.inc

smb_migrate.migrate.inc

Tell Migrate that we are no longer using DrupalFile6Migration.

'class_name' => 'SMBFile6Migration'

In smb_migrate.migrate.inc we alter the DrupalFile6Migration to SMBFile6Migration. This PHP class contains the settings we are about to alter.

file.inc

A new PHP class that extend the previous used DrupalFile6Migration class.

<?php
class SMBFile6Migration extends DrupalFile6Migration {
  public function
__construct(array $arguments) {
   
parent::__construct($arguments);

   
$this->addFieldMapping('preserve_files', NULL, FALSE)
      ->
defaultValue(FALSE);
   
$this->addFieldMapping('file_replace', NULL, FALSE)
      ->
defaultValue(FILE_EXISTS_REPLACE);
  }
}
?>

Lets have a look at what gets defined inside the constructor function.

  • preserve_files - With this setting FALSE mr Files will remove all from the legacy_files directory.
  • file_replace - FILE_EXISTS_REPLACE tell migrate to remove

Blog post in this series

Comments

Unfortunately, migrate_d2d currently does not support node revisions: https://www.drupal.org/node/1834016

Thanks a lot for this post. Just started reading about/practicing migration scripts this weekend, for a Drupal 6 - Drupal 7 migration I am going to be working on. This blog post couldn't have come at a better time. Looking forward to reading Parts 1 and 2.

Thank you for mentioning that. There is a rough start in the issue you posted but by the look of it, its a start. I think in most use cases do we not need to migrate revisions. Personally I'm fine with dropping them (they should be trimmed anyway (performance issues)) and then just leave the old system running for a while and tell content users to go there if they would need to page back through revisions. Not ideal but should be OK, and then drop the old site after a few months in production.

I've tried many way and the file are not migrated. I did: [code] $api['migrations']['File'] = $common_arguments + array( 'machine_name' => 'filemigration', 'description' => t('Import Drupal 6 files'), 'class_name' => 'DrupalFile6Migration', 'user_migration' => 'User', 'default_uid' => 1, 'source_dir' => '/Users/webapp/Sites/dm6/sites/example/files/newimages', 'destination_dir' => 'public://files/newimages', ); [/code] I have already uploaded the files to D7 at ../files/newimages. I got the following message: Processed 357 (0 created, 0 updated, 357 failed, 0 ignored) in 1.1 sec (18751/min) - done with 'File' I appreciate any helps. Thanks

@Arujo - I do not think will work. Look to me that you are importing and writing to the same directory. That seems rather messy to me. Try keeping them apart and then roll back your file migration and try again. If it still fails, look in the migration log (migrate keep it own log outside dblog).

Add new comment