Using the TFS integration tools to migrate Work Items from multiple projects

I’ll just be continuing my post series on Team project merging and write about the TFS migration toolkit.

Using the migration toolkit

The core of the migration has been performed by this migration toolkit engine and this where I have spent most of my time. The toolkit is decently documented, maybe not as much as I may have wanted, but there are samples, a cool training kit, a very good documentation on how different scenarios can be solved, what kind of migrations are available. Add on top many blog posts by Willy Peter-Schaub and a cool support on a forum, I really had the feeling I had my bang for my buck…

Work Item migration basics

You launch the TFS integration tools, and create a so called “migration” between a source TP to a destination TP. You’ll be able to choose multiple modes and I’ll forward you to the manual part which explains which one to use according to your business scenario. In my case, the one way migration was all indicated. I needed a one shot migration, I didn’t need to keep work items in sync between projects or any bi-directional fancy stuff. And above all, I needed performance.

Note : even one way migration keeps track of source WIs and dest WIs Ids, you may start the same migration twice, changes in the source Work Items (or any new stuff) would be reflected, that is just magic !

Next, you may specify a Query to filter the bunch of work items to be migrated. In my first experiments I tried to move the WIs project by project, but I quickly realized that links among work items were not reproduced in the destination project.

When are links not preserved ?

You lose work items links when you perform multiple Migrations with the TFS integration tools. The only way to preserve all links to use a “large query”, the opposite of what Willy Peter-Schaub calls a narrow query. His posts explain the problem well.

In other words, the TFS integration engine has been implemented in order to preserve the WI links of your current migration, but does not take into account anything existing before your current migration.

image_thumb

Why do I bother about links if have many different projects ? Well, actually, there are really many links between work items of my different Team Projects! Bugs from A are linked to tasks from B, and the tasks trees we use span over many projects.

Yes, although this works and is supported, it was a bit messy…

Unsuccessful first attempts

Ok let’s use very large queries then…But wait, oh no, big surprise, it was not possible to use queries that span over multiple Team Projects : the tool adds a hard coded filter to the queries ! First bad news for my big migration Sad smile

So, why not after all, let’s do it: let’s modify the tool code and recompile it. I changed the following lines :

// VLA : original code
//var c = new StringBuilder("[System.TeamProject]=@project");
//if (!string.IsNullOrEmpty(m_core.Config.Filter))
//{
//    c.AppendFormat(CultureInfo.InvariantCulture, " AND ({0})", m_core.Config.Filter);
//}

// VLA : select WIs from the filter if defined, otherwise from the team project
var c = new StringBuilder();
if (!string.IsNullOrEmpty(m_core.Config.Filter))
{
    c.AppendFormat(CultureInfo.InvariantCulture, "({0})", m_core.Config.Filter);
}
else
{
    c.Append("[System.TeamProject]=@project");
}

Now I was ready to launch very big queries. Sadly, when I started testing the migrations, I ended up with strange error messages :

[19/01/2012 11:54:57] TfsMigrationShell.exe  Information: 0 : WorkItemTracking: Area ‘Mycompany\ADS\Mycompany\ADSWebShop’ does not exist in the TFS work item store ‘ads-tfs\DefaultCollection (Newtp)’ or access is denied.

[19/01/2012 11:55:04] TfsMigrationShell.exe Information: 0 : WorkItemTracking: Wake up from 100 millisec sleep for polling CSS node Id

Not sure why I had those, I apologize I may not be able to explain this clearly since myself I could really understand what was happening. The good news is that after a couple of unsuccessful attempts, I ended up recreating the plan, along with the destination Team Projects and the errors finally disappeared. In order to test my migrations, I had to often create a new destination TPs…

One big query is not the solution

In order to make sure I miss no link in the migration process, I’ll just have to run one single very very big that selects *all* of the work items I need to migrate… in one single query… How long would it take ? And if it fails, I would have to restart the whole query ? Is is the right way ?

Surely not, this operation would take weeks! The TFS integration tools engine is powerful, but relies on TFS API, and mass operations are costly in time. Remember now the constraints for this operation : the final migration should not take more than a Week-End…

That means I’ll have to pre-migrate the vast majority of work items, and I start feeling I’ll have to write a tool myself to reflect links because I won’t be able to do all this in one shot. Even with two shots, I would potentially miss links…

I’ll leave this for the next posts… stay tuned!