2

TABLE OF CONTENTS

ABOUT THE AUTHOR ....................................................................................................... 3

Preface ................................................................................................................................... 4

Migration of TFS 2017 to VSTS with TFS Integration Platform tool ................ 6

Introduction ................................................................................................................................. 6

Understanding TFS Artifacts Association and Relations .............................................. 6

Step 1: Getting Environment Ready ..................................................................................... 7

Step 2: Migrate ............................................................................................................................. 8

Conclusion .................................................................................................................................. 11

TFS 2017 to VSTS migration with Sync Migrator .................................................13

Introduction .............................................................................................................................. 13

Step 1: Getting Environment Ready .................................................................................. 13

Step 2: Preparing TFS ............................................................................................................. 14

Step 4: Preparing JSON File .................................................................................................. 17

Step 5: Migrate Work Iterations ......................................................................................... 19

Step 6: Migrating Work Items ............................................................................................. 21

Step 7: Migrating Links between Work Items ............................................................... 24

Step 8: Migrating Attachments ........................................................................................... 26

Conclusion .................................................................................................................................. 29

Migration Team Foundation Server to Visual Studio Team Services using

Database Import Service ...............................................................................................30

Introduction .............................................................................................................................. 30

Step 1. Get started.................................................................................................................... 30

Step 2. Prerequisites.............................................................................................................. 30

Step 3. Upgrade TFS to the latest version ....................................................................... 31

Step 4. Decision and Validate .............................................................................................. 31

Step 5. Prepare ......................................................................................................................... 33

.1. Prepare – First collection dacpac .................................................................................... 33

.2. Prepare – Second collection Azure VM ......................................................................... 34

Step 6. Import ............................................................................................................................ 34

.1. Import – Dry-run .................................................................................................................... 34

.2. Import – Production ............................................................................................................. 35

Conclusion .................................................................................................................................. 36

3

ABOUT THE AUTHOR

Mohamed Radwan is a Visual Studio ALM MVP and

DevOps practice lead at testhouse; he focuses on

providing solutions for the various roles involved in

software development and delivery through Agile

Methodologies and utilization of Microsoft Visual

Studio ALM/DevOps Tools & Technologies.

Mohamed excels in software practices and automation with 15+ years of

professional experience spanning the different stages and phases of

Software Development Lifecycle. He has helped various customers around

the globe in the UK, Denmark, USA, Egypt, Oman, KSA, Kuwait, and Libya,

amongst others.

Mohamed specialized in Agile Methodologies and is a frequent speaker at

many national and regional technology events and user groups for various

countries, such as: Campus Days in Denmark, Copenhagen user group, Open

Days in Egypt, KSA, Kuwait, and Oman. He has authored several

frameworks and extensions including DevMagicFake and TestingConf

Utilities, as well as many technical articles & guides. He is the founder of the

TFSEG and co-founder of the MEAALM Community.

Mohamed also holds a number of Microsoft certifications including MCT,

MCPD, MCITP in EPM, MCTS (7), MCSD, MCAD, and CIW.

http://mohamedradwan.com/

4

PREFACE

There are several advantages of migrating from TFS to Visual Studio Team

Services (VSTS) to being able to use great capabilities which VSTS has to

offer.

VSTS is cloud based version of TFS and with every update your account will

get all the new features in three-week update cycles and besides that the

cloud first innovation releases first all the updates to the VSTS before

making them available on premises on TFS. Another key advantage of such

migration are also the flexibility and accessibility from anywhere as the

team can access the account from different devices and locations. While

with TFS you need to take good care of infrastructure, this is no longer

needed with VSTS.

It’s important to note here also that migration will not necessarily happen

only between TFS and VSTS, we can use many of those ways to migrate from

older version of TFS to newer version of it, or migration between different

projects on VSTS or even the migration from VSTS to TFS. There is many

ways and directions in which migration can happen.

However, the migration can be done with many different tools and with

different fidelity. Before starting the migration, it’s important to define how

deep we want to go with migration and do we want to migrate everything

or just some sets or subsets of our project.

There are many tools currently on the market available for migration in

many directions and the main difference between them is the usage of the

tool and the fidelity which they can support.

Some of those tools are: TFS Integration Platform, VSTS Sync Migrator,

OpsHub, Git TFS, Timely Migration, TFS Office Integration and TFS

Database Import Service. While some of these tools will be able to migrate

Team Projects (OpsHub), others will be able to migrate work items (TFS

Office Integration, VSTS Sync Migrator, TFS Integration Platform). There is

also Git TFS which will migrate the TFVC repositories to Git and Timely

Migration tool for migration of TFVC source.

5

This guide also includes how to use TFS Database Import Service, which

will enable the migration with very high fidelity. The tool is designed to

migrate source code, work items, links, history and also other artefacts such

as test plans, build definitions and so on.

It’s very important to mention here that TFS Database Import Service tool

supports only the last two TFS releases (6 months window), including

updates and major releases. In this moment, the two last supported

versions of TFS for migration are TFS 2017 Update 2 and TFS 2018 in the

date I am releasing this guide. Furthermore, this tool does not support the

migration from Release Candidates, which means that if you have upgraded

your production database to a Release Candidate (RC) release, you will

have to wait first for an upgrade to (RTM). However, there is also a

possibility for reverting back to a previous version of TFS in order to have

the pre-requisites for migration with TFS Database Import Service tool.

This guide will walk you through step by step migration process with usage

of different tools, where you will be able to understand the complexity of

different tools and the fidelity that they are able to provide.

6

MIGRATION OF TFS 2017 TO VSTS WITH TFS INTEGRATION PLATFORM TOOL

Introduction

Migrating TFS (On-Premise) To VSTS makes it a lot easier to work

efficiently, there is a lot of benefits we can get, including:

• Global Availability.

• Immediate Access to updates as soon as they’re available.

• Higher Security.

• Easy Deployment to Azure.

Migrating TFS to VSTS can be done using several tools, Microsoft Excel,

Microsoft Project or a Migration tool (which we will use)

Note: Using Excel and/or Microsoft Project has limitations for images,

attachment, history, and other things.

I’ll demonstrate another tool, which is TFS Integration Platform. Using TFS

Integration Platform requires SQL Server 2012 first, as the tool is a little bit

old. This tool migrates everything except the relations between work items.

Note: This tool is deprecated, but you can still use it at your own risk.

Migration with TFS Integration Tool is very easy, but again, we have a

limitation that relations between work items can’t be migrated.

Before getting into the real work, you should have access to both TFS &

VSTS with full administrative privileges. TFS Integration Platform is only

helpful when You want only to migrate work items, with no need to migrate

links & relations between them.

Understanding TFS Artifacts Association and Relations

TFS doesn’t only provide work items or test cases, TFS is simply everything

needed to manage work, the following diagram provides some general

explanation for that:

7

It’s clearly shown that relations in TFS make the logic for work, therefore

it’s very important to maintain the relations between work items, not only

the work items itself.

Step 1: Getting Environment Ready

Before getting started, we will need to install Microsoft SQL Server 2012.

Additional help for SQL Server 2012 installation is available here. Please

note SQL Server 2012 is essential for compatibility, as the TFS

Integration tool is deprecated and are is supported for the newer version.

So, make sure that you have connected to the SQL Server 2012. You can do

this using a later version of SQL Server Management Studio.

1 Make sure you’ve selected SQL Server 2012.

2 Click on Connect.

8

• Download & Install Visual Studio 2010 (Required to provide Object

Model for the tool) and Visual Studio 2012 (Required to provide

TFS Connectivity for the tool).

• Download & Install TFS Integration Platform.

Note: TFS Integration Platform must be installed after SQL Server 2012 and

Visual Studio 2010 and Visual Studio 2012, otherwise it won’t work.

Step 2: Migrate

1. Open TFS Integration Platform Tool and Click Create New

2. Click on Team Foundation Server Folder

3. Choose Migration Options, you can Migrate Source Control, Work Items

or Both, I choose Work Items Only.

4. Click Open

5. As a workflow type, from drop down menu select One Way Migration

6. Click on Configure

7. In a drop-down menu, select TFS 11 Migration WIT Provider.

9

8. Select the right TFS Server to connect to the original TFS.

9. Under Team Project Collection, select the Team Project Collection

which you want to use for this migration.

10. Under Team Projects, navigate to the Team Project that you will use

and click on it.

11. Click on Connect to connect to TFS.

12. Click on Configure to configure the targeted Source and select TFS 11

Migration WIT Provider again.

13. Under menu Select the Team Foundation Server to select the targeted

VSTS to connect to it.

10

14. Under Team Project Collection, select the Team Project

Collection which you want to use for this migration.

15. Under Team Projects, navigate to the Team Project that you will use

and click on it.

16. Click on Connect to connect to VSTS.

17. Click Save to Database

18. Click Start Migration

19. Set back and relax, Migration can take few hours if you have

thousands of work items. When the migration will be finished, you’ll be able

to see the notification in the window of TFS Integration tool.

11

20. Additional Note, here is how your SQL Server 2012 Database looks like

for Migration.

Conclusion

The TFS Integration Platform Tool is a pretty straight forward tool to use

for migrating items from TFS to VSTS. The tool enables the migration of

everything, including the acceptance criteria, discussions, history, and

attachments, but not the links between the work items.

However, as already mentioned in the beginning of this section, the

tool is deprecated, which means that no further development will be

performed and there is no current support available for it. The previous

documentation for the tool is currently still available on the Codeplex.

12

In the next section, I will present a different tool, which can be also used for

migration from TFS to VSTS. This tool, VSTS Sync Migrator was developed

by fellow MVP Martin Hinshelwood.

13

TFS 2017 TO VSTS MIGRATION WITH SYNC MIGRATOR

Introduction

In the previous section I’ve described the migration between TFS and VSTS,

with usage of TFS Integration Platform. As described in that section, the

tool is pretty simple and strait forward to use, but it doesn’t support the

migration of all artefacts between work-items, such as links between the

work items. Besides that, I think it’s worth mentioning again, that this

migration tool is deprecated. If your migration demands a high fidelity than

this tool would not be appropriate.

For such cases, demanding the high fidelity, VSTS Sync Migrator is

appropriate solution. This tool is a very powerful and nice tool developed

by Martin Hinshelwood who is a fellow ALM MVP. The tool strength

comes from its coverage for many migration options for work-items.

Migration with VSTS Sync Migration Tool is a bit complicated, however it’s

a powerful option as we don’t have limitations for migrating links,

attachments or work items.

Step 1: Getting Environment Ready

Before starting the migration, make sure to have

3 TFS Process Template Editor extension for Visual Studio installed.

4 Open the Process Editor Extension in Visual Studio

5 Download & Install VSTS Sync Migrator.

6 Open Command Prompt or PowerShell

7 Navigate with CMD or PS to a new folder (Recommended)

8 Run the command vstssyncmigrator init to create a default

configuration

9 Open “vstsbulkeditor.json” file from the same directory.

14

Step 2: Preparing TFS

For Migration to succeed, we need to have an additional field in both TFS &

VSTS for tracking purpose (This is essential for the tool to be able to link

work items to each other).

1. To do that, Open Visual Studio

2. Open the Tools Menu

3. Open Process Editor Menu

4. Open Work Item Types

5. Click Open WIT from Server

For each of the Work Item Types in your Project, do the following

6. Click on New

7. Set the Field Name to ReflectedWorkItemId

8. Make sure it’s of type String

9. Set the Reference Name to TfsMigrationTool.ReflectedWorkItemId

10. Click OK

11. The Field is now added.

15

Important Note: Make sure to add the field to all Work Item Types.

Step 3: Preparing VSTS

We need also to modify Work Items in VSTS to match with TFS, so the Tool

can complete the Migration Process

1. Open your VSTS Account Settings

2. Open Process Settings

3. Open Your Inherited Process Settings

16

4. For Each Work Item Type, open it and repeat the following steps

5. Click Add New

6. In the New Field Dialog, Set the New Field name to be

ReflectedWorkItemId

7. Make sure it’s of Type Text Single Line

8. Click Add Field

17

Make sure you have added the field to all Work Item Types, you can

check Work Items (to make sure you have added the field) by Going to Your

Inherited Process and check the work Item.

Step 4: Preparing JSON File

Do you remember the file we opened in the step 2.1.7? Let’s get back to this

file to adjust few things before starting the migration.

1. In the source we add information for the old TFS, we will add the URL for

the old TFS under the Collection option and the old project name under

the Name Option.

2. In the target we add information for VSTS, we will add the URL for the

VSTS under the Collection option and the VSTS project name under the

Name option.

18

3. In the ReflectedWorkItemIdFieldName option, but the Name of your

inherited Process followed by a period and “ReflectedWorkItemId”, the

should look finally like this

MyInheritedProcessName.ReflectedWorkItemId

4. Put the same value for step 3 in the targetField Option.

5. Optional: You can use the valueMapping to change states of work items

being migrated, this can be helpful if you’re migrating between Scrum and

Agile (or vice versa).

6. Optional, WorkItemTypeDefinition is used to map Work Item Names, this

also should be used if you’re migrating between Scrum and Agile.

19

7. Processors is the control units for migration, each processor controls

migration of a specific part of TFS. If you don’t have any processor

enabled, the tool won’t work at all.

8. Each processor has an ObjectType, which is the name for Object being

migrated.

9. Each processor has an Enabled flag, which is either true or false, if it’s

enabled (true) then this processor will be executed during migration.

Step 5: Migrate Work Iterations

The First Thing we should migrate is the work iterations, this is known in

the file with Object Type

VSTSSyncMigrator.Engine.Configuration.Processing.NodeStructureMigrati

onConfig

As we can see on VSTS, we have no iterations (Except the parent node):

20

2. Now to Migrate Iterations and its structure, let’s get back to the JSON

File, Set the Enabled flag for the Object

VSTSSyncMigrator.Engine.Configuration.Processing.NodeStructureM

igrationConfig to be true, otherwise the tool won’t run this part.

3. Run the following command vstssyncmigrator.exe execute –config

vstsbulkeditor.json

21

4. This might take a minute or less, once finished you’ll see something like

this:

Step 6: Migrating Work Items

Now Let’s move to the Migrating the Work Items itself, Here the VSTS

before migrating any work items.

So here is a query on the new VSTS (Lists all items in tree view) which

shows that we have nothing yet.

22

Now Let’s start Migrating Work Items,

1. Open the file again, make sure to set the Object

VSTSSyncMigrator.Engine.Configuration.Processing.NodeStructureM

igrationConfig to be False, this is very important.

2. We have Two Options for Migrating Work Items, as described below

3. The First Option is to Migrate work items with recent state, which

means that the history of work items will be lost

(VSTSSyncMigrator.Engine.Configuration.Processing.WorkItemMigration

Config)

4. The Second Option is to Migrate work items with its history

(VSTSSyncMigrator.Engine.Configuration.Processing.WorkItemRevisionR

eplayMigrationConfig)

5. I’ll choose migration with history, so we’ll set the Enabled flag for it to

be true.

23

6. Open the command prompt again, and run the same command again

(vstssyncmigrator.exe execute –config vstsbulkeditor.json).

7. The last step might take some time depending on how many work items

are there, in my case it took around 17 minutes.

24

Here is how VSTS Looks after Migration, remember that so far, we did only

migrate Work Items, but with no links between them and with no

Attachments.

Step 7: Migrating Links between Work Items

Now is the time to Migrate the links between work items, I added one more

column to demonstrate that, as we can see there is links between work

items count is zero.

25

So, let’s get back to file:

1. Open the file again, make sure to set the Object

VSTSSyncMigrator.Engine.Configuration.Processing.WorkItemRevisi

onReplayMigrationConfig to be False, This is very important.

2. Find the Object

VSTSSyncMigrator.Engine.Configuration.Processing.LinkMigrationCo

nfig

3. Set the Enabled flag for this object to be true

4. Open the command prompt again, and run the same command again

(vstssyncmigrator.exe execute –config vstsbulkeditor.json)

5. This could take few minutes.

26

6. If we check VSTS again, we’ll see that links have been migrated.

Step 8: Migrating Attachments

Now attachments for Work Items is the only remaining thing, as we can see

the attachments count is zero.

Let’s get back to the file

27

1. Open the file again, make sure to set the Object

VSTSSyncMigrator.Engine.Configuration.Processing.LinkMigrationCo

nfig to be False, this is very important.

2. For Migrating attachments, we need two options to be enabled (true),

not only one, the options are

• The first one is

VSTSSyncMigrator.Engine.Configuration.Processing.Attachmen

tExportMigrationConfig

• And the second one is

VSTSSyncMigrator.Engine.Configuration.Processing.Attachmen

tExportMigrationConfig

Both of them should be set to true

3. Open the command prompt again, and run the same command again

(vstssyncmigrator.exe execute –config vstsbulkeditor.json).

28

4. This might take few minutes.

5. Now Let’s check the Attachment Count IN VSTS Again.

6. And Finally, here is the Work Items in the VSTS, with Attachments

migrated

29

Conclusion

VSTS Sync Migrator is a very powerful tool Developed by Martin

Hinshelwood. The Tool allows migrating between TFS and VSTS, including

Work Items, Iterations, Attachments, Links between work Items and even

more options can be found in the official documentation.

The tool still requires a bit of experience and understanding for both TFS

and VSTS, but it’s really covering migration of all project artefacts, which

adds up a big value to the tool.

30

MIGRATION TEAM FOUNDATION SERVER TO VISUAL STUDIO TEAM SERVICES USING DATABASE IMPORT SERVICE

Introduction

In this part also, I am going to share my personal experience of the

Migration Team Foundation Server to Visual Studio Team Services using

Database Import Service or TFS Migrator. The TFS Database Import

Service, also known shorthand as the Import Service, provides a high-

fidelity way to migrate collection databases from TFS to VSTS. If you are

looking to use this service to import your collection(s), there is a

migration guide provided by Microsoft which I strongly recommend to use

it during your migration as a walkthrough (you can download it from the

following link).

There are six phases of the migration timeline shown in the following

image. We will go through all of them.

Step 1. Get started

In this step, let us first understand why we want to migrate from Team

Foundation Server to Visual Studio Team Services.

Step 2. Prerequisites

- The main task for this second step is to make sure that we have a

working Azure Active Directory that will be used for authenticating

the team members in our VSTS account.

31

- We cannot migrate to an existing VSTS account since the tool is

creating a new one. In case you want to migrate to an existing

account, you cannot use this tool.

- VSTS support two different process models:

o Inherited (default model)

o Hosted XML (by default is turned off)

- Each Team Project Collection will be mapped to separate VSTS

account, for which later Microsoft will provide an entity where we

can keep multiple VSTS accounts.

- Always download the latest version of TFS Migrator Tool

Step 3. Upgrade TFS to the latest version

a. 6 months support window

Step 4. Decision and Validate

b. If you have any customization, you need to take decision. Do

you want to keep it or not?

c. Note that there are customizations that are not supported. If

you want to keep any of the supported customizations, you

will need to use Hosted XML process model. Also note that

you will not be able to move to the Inherited process model

before 2019, as it is planned.

d. In my process of migration, I was using inherited process

model. First, I removed all customization, then checked the

log and fixed all errors, then revalidate again until no errors

appeared.

e. In order to start the TFS Migrator validate, from the

command line on the application tier, navigate to the tool

directory and type the following:

TfsMigrator validate /collection:http://TFSServer:8080/tfs/CollectionName/

Now we should review validation warnings and errors. I got

the following messages displayed in the image below:

32

As we can see on the image, I have one error that I have existing

table with size of 45GBs, which is above recommended size of 20GBs,

meaning that I am not able to use DACPAC import method.

Here is the list of the some of the customizations, which I have removed

during my process of migration:

• List link types (listlinktypes)

witadmin listlinktypes /collection:http://TFSServer:8080/tfs/CollectionName/

• Delete link type (deletelinktype)

witadmin deletelinktype /collection:http://TFSServer:8080/tfs/CollectionName/

/n:xxx.LinkTypes.Blocked

33

• Change work item field (changefield)

witadmin changefield /collection:http://TFSServer:8080/tfs/CollectionName/ /n:xxx.Blocked

/name:"Blocked (Custom)"

• Destroy work item definitions (destroywitd)

witadmin destroywitd /collection:http://TFSServer:8080/tfs/CollectionName/

/p:“ProjectName" /n:"featureset"

Step 5. Prepare

• Once the validation is ok, we are ready to import

• We need to run prepare command in order to generate import

settings and related files:

TfsMigrator prepare /collection:http://TFSServer:8080/tfs/CollectionName/

/tenantDomainName:mydomain.onmicrosoft.com /accountRegion:WEU

Please note that in order to run this command you need Azure

Active Directory tenant. The prepare command will contact your

Azure Active Directory tenant so it will prompt you to login with a

user from the tenant with permissions to read information about all

of the users in the Azure Active Directory tenant.

• It generates the default configuration file (settings.json, which is an

input to the import service)

• In my case I have two collections. One of them is that I used dacbac,

and the second one is that I have table that exceed 45GBs so I had to

use Azure VM with SQL Server

.1. Prepare – First collection dacpac

• Prepare Azure storage container for the dacpac

• Detach collection

• Create the dacpac file from the collection using the following

command:

SqlPackage.exe /sourceconnectionstring:"Data Source=sqlserverName/instase;Initial

Catalog=Tfs_CollectionDBName;Integrated Security=True"

/targetFile:C:\MyCollection.dacpac /action:extract /p:ExtractAllTableData=true

/p:IgnoreUserLoginMappings=true /p:IgnorePermissions=true /p:Storage=Memory

• Upload (Azure Copy) the dacpac file to the Azure storage container

34

AzCopy /Source:"C:\myfolder" /Dest:https://mystorgeurl.blob.core.windows.net/tfsimport

/DestSAS:"?st=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxx"

• Change the import setting in the configuration file (setting.json) to

point to the url of the storage container and you are ready to go

.2. Prepare – Second collection Azure VM

• Prepare the Azure VM with SQL Server

• Detach collections

• Backup the DB from the collection

• Upload (using the Azure Copy) the DB backup file to the Azure

storage container

• Download DB backup to the SQL VM and restore it (using Download

manager)

• Change the configuration file (generated from the preparation first

step) to point to the VM IP and also the SQL connection string and

you are ready to go

Step 6. Import

.1. Import – Dry-run

Till now, I have Azure storage container with dacpac file and a database

backup restored on a SQL Server on a virtual machine. This is my source,

one tool settings file that have all the information needed for the

migration.

The next step is to running the tool with the import as a dry-run (trial

migration). Run the TfsMigrator import command and define the path to

the configuration file which contains all the parameters for the migration,

as following:

TfsMigrator import /importFile:"C:\myfile.json"

35

Next, we are going to see the progress of the import, as shown in the

image bellow:

Next on the screen we see a warning with the following message:

This dry run account will expire and be deleted shortly after 7:20 PM on

2/1/2018. To continue testing beyond this date you will need to repeat the

dry run import.

.2. Import – Production

• Delete dry-run

• Change the configuration into “production”

• Re-run the import in the same way

• Rename your account with the reserved name

• Done

36

Conclusion

Here is the quick overview of the all steps of the migration process. Real

migration process took about twelve continuous hours.

1. Install and Upgrade from TFS 2015 to TFS 2018

2. Remove all customization of all the projects

3. Validate the process is ready

4. Generate migration settings files for (preparation)

5. Detach collections

6. Dacpac The 1st collection

7. Take backup 2nd collection

8. Azure Copy the dacpac to the Azure storage container

9. Azure Copy the DB backup to the Azure storage container

10. Download DB backup to SQL VM and restore it

11. Change the import setting and run the import for both collections

Your team is now in VSTS and will receive now updates frequently and

have many opportunities to implement tools and processes that will help

to be more effective in building quality software.