TfsWorkItemTypeValidatorTool – improve validation

[satano]

TfsWorkItemTypeValidatorTool validates all work item types before the migration starts. This is good, because it really can catch some possible data loss if migration will be performed. But is makes minor problems for user, because it needs some configuration to work properly. And validation can be emproved to count with field mappings. I have theese suggestion, we could make into it (I can do them myself).

• Mandatory Reflected work item ID field validation
  • Validation of reflected work item ID is mandatory #2963
• Excluded some work item types by default
  • Implement default exclude list for work item type validation #2971
  • TfsWorkItemTypeValidatorTool: fix default excluded WITs handling #2973
• Work with existing field mapping
• Work with existing field value mapping

Mandatory Reflected work item ID field validation

ReflectedWorkItemId validation was mandatory before TfsWorkItemTypeValidatorTool. If reflected work item ID field did not exist, migration process stopped immediately. Now, when it is part of the validator, if someone disables this validator, this filed will not be validated at all. I would still keep this validation in TfsWorkItemTypeValidatorTool, but in migration process, I would make it mandatory again. Even if the validator itself will be disabled.

Excluded some work item types by default

There are some work item types, which are (usually) not migrated at all, because it does not make sense to migrate them. For example _
Code Review Request, Code Review Response, Feedback Request, Feedback Response, Shared Parameter… We should exclude them from migration by default. In validator configuration, there is IncludeWorkItemtypes. We can add ExcludeWorkItemtypes as a list. It it will be empty, we will consider it as use default exclude list and some work item types will not be validated. Id the user will add something into it, just this will be excluded. It means, that if the user will addd something into it, our default list will not be excluded, unless explicitly added by user to exclusion list. Or we can make some ohter configuration property bool UseDefaultExclusionList, which will jus tur on/off default excluded work item types. Explicit ExcludeWorkItemtypes will still be there, but when the user will add something into it, it will not disable default exclusions.

Work with existing field mapping

Sometimes the validator reports missing fields. This is OK, but there is FieldMappingTool and the user can already have mapping for theese fields. So the validator can check if there is suitable mapping and if so, it will not report the field as missing. This is applicable to:

• FieldToFieldMap
• FieldToFieldMultiMap
• FieldSkipMap

Work with existing field value mapping

We can also check not just field mapping, but also field value mapping. It is good for fields which have allowed values defined, as noted in this discussion._ For allowed values, we can check the mapping and if there is suitable mapping we will not report that the source and target fields are different. This is applicable to:

• FieldValueMap

Other thoughts

Of course, for field (value) mappings, there are some more mapping types, but they are not easy (even possible) to check if the work, so there still may be fields reported as missing/not matched in validator. I believe, this will be just minor cases and will be fixed by FixedTargetFields in validator configuration.

So @MrHinsh (or anybody) what do you think? Any ideas to this?

Thanks.

[dougkl007]

In short, the biggest challenge with the current default configuration file generated by the devopsmigrator command is that it would appear to be excluding work item types that are not visible with the WIQLQuery line, however, the validator still reviews all work item types for the Custom.ReflectedWorkItemId. Because these work item types are not visible in the process management, but only modified with XML layout files, and because the validator assess them and throws are error stopping the migration process - hence causing me to stop attempting to get the tool to work. It is by far the most complete tool I've ever seen made to handle many complex scenarios, but it I can use it to migrate a project with 135 work items because the validator is dropping errors, it's not all that useful to me.

[MrHinsh]

@satano just added the ability to exclude by default! Check out https://blue-river-093197403-preview.westeurope.5.azurestaticapps.net/docs/reference/tools/work-item-type-validator-tool/#defaults

[bbct]

Some further thoughts/suggestions that might get some creative juices flowing. I've just recently found a need for this tool/processor, downloaded it, and tried to make it run, with similar results as dougkl007, that is, validations are failing, and I don't see how to circumvent these issues to get my data migrated. I cannot get it to work.

Observation 1: Related to which WorkItemType compatibility validations get executed in advance, before getting any data. If you would like to know in advance what Work Item types are to be migrated, might I suggest the following:
a) create a JSON attribute of Processor called "WorkItemTypes" whose value would be an array of strings, perhaps like this: ["Task", "User Story", "Bug", "Feature"].

b) Replace the WIQLQuery with WIQLQueryWhereClauseAddition and WIQLQueryOrderByClause

c) Now the code must generate the full query, rather than simply getting the entire query string from the configuration file as you do now, concatenate the following:
"SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @teamproject "
Then take the WorkItemTypes array and concatenate "AND System.WorkItemType IN (" followed by the list of strings, followed by ")"
Then concatenate the value in WIQLQueryWhereClauseAddition, followed by the value in WIQLQueryOrderByClause

This would be a little work, so I have some other options to consider that would be much simpler to implement:

First, for backward compatibility you might actually still use the WIQLQuery value if it is specified, but if not specified, do what I outlined above to generate the query.
Perhaps even better: the code could continue to use this query as-is, but use the new WorkItemTypes list to choose which ones to validate, and skip validation of the others. Perhaps if this is the preferred method, name the new attribute "WorkItemTypesToPrevalidate", and do just that - only prevalidate the source/target compatibility of the types listed by the user. And to go further, if you want to lean towards more validation by default, name this the "WorkItemTypesToSkipPrevalidation" so if it is empty, do what you do now, but if types are listed, the user must have determined that the validation is just blocking their use of the tool, so they added the types they wish skipped into their configuration file. The default configuration might add the work item types it is already specifying to skip in the default WIQLQuery to this parameter for the user so it is ideally configured by default.

Observation 3: As stated by satano in his initial post, when, during this initial validation of WorkItemType compatibility, you find an issue with a target field of a WorkItemType that is missing or configured differently in the target, before flagging it as a warning or error, first check the FieldMappingTool configuration to see if it is already mapped to be Skipped or has been mapped to another field that actually does exist in the target. The fact that I cannot "fix" these prevalidation issues by adding to the FieldMappingTool is frustrating.

[satano]

Observation 3

Yes, I will (eventually) implement this, as stated at the begining.

Observation 1

I do not understand it at all. Validation of work item types does not use any query. It just directly validates all work item types. So there is no point to specify some condition, because there is no place for using such a condition. But this validation can be configured. Maybe now the configuration is a bit clumsy, but I believe things will get a lot better when all the points at the begining will be implemented, together with issue #2952.

Configuration exaple:

"CommonTools": {
  "TfsWorkItemTypeValidatorTool": {
    "Enabled": true,
    "IncludeWorkItemtypes": [
      "Epic",
      "Feature",
      "Product Backlog Item",
      "Task",
      "Bug"
    ],
    "FixedTargetFields": {
      "Bug": [
        "System.Reason"
      ]
    }
  }
}

• The IncludeWorkItemtypes is what it means – just a list of work item types which will be validated. Basically the list you wanted to add to WHERE condition.
• The FixedTargetFields is a list of fields which are considered OK (fixed) and so they will not trigger error when they do not match between source and target. In this configuration example it says, "do not report field System.Reason in Bug work item type as invalid".

[MrHinsh]

@satano, I like your current iteration, and some of the issues may still stem from the lack of documentation and not referencing where to look in the logs. Both of which I fixed in the current preview version.

[bbct]

@satano, Aha, I had not discovered the TfsWorkItemTypeValidatorTool configuration yet. Your discussion above about adding "ExcludeWorkItemtypes" to the validator makes so much more sense now!. Yes, ignore my uninformed suggestions in Observation 1 above. Observation 2 shall remain a mystery.
Perhaps a place holder for this tool configuration should be included in the default configuration file created, even if turned off, so noobies like me would see it and look for documentation on it? I don't recall any reference to it in the brief "getting started" section.

FixedTargetFields is not yet in the documentation, is this only in the preview version, perhaps I should use that instead of 16.2?

I cloned the source, built the tool, and the FixedTargetFields in the config resolved the issue. All user stories and tasks have been migrated.

@dougkl007 You may find success doing this as well.
Here is the tool configuration I added to the "CommonTools" object configuration: Note that I also added rules in the FieldMappingTool configuration to either ignore or remap these. Some are custom fields I don't need migrated.

			"TfsWorkItemTypeValidatorTool": {
				"Enabled": true,
				"IncludeWorkItemtypes": [
					"User Story",
					"Task",
					"Bug"
				],
				"FixedTargetFields": {
					"Bug": [
						"Microsoft.VSTS.Common.ResolvedReason"
					],
					"User Story": [
						"xxx.VSTS.Common.SupportTicketId",
						"xxx.VSTS.Scheduling.PlannedReleaseDate",
						"Microsoft.VSTS.Common.ResolvedReason",
						"Microsoft.VSTS.Common.Activity"
					],
					"Task": [
						"xxx.VSTS.Common.DevelopedBy",
						"xxx.VSTS.Common.SupportTicketId",
						"xxx.VSTS.Scheduling.PlannedReleaseDate",
						"Microsoft.VSTS.Common.Activity"
					]
				}
			}

@satano Thank you for your contributions to this tool!

[bbct]

@satano I have two more thoughts:
First you mention Work with existing field mapping, and that you should check specifically mappings of types "FieldToFieldMap" and "FieldToFieldMultiMap". Please also include "FieldSkipMap", as skipping a source field altogether should resolve any issues with mismatches.

Second, and this is perhaps a minor point, you mention that the Reflected work item ID field is required. Perhaps report it missing as an Error rather than a Warning. When I ran the tool the first time, I got info and warning messages, but the final error message implied that some or all of the warning messages were actually to be considered an error, blocking the tool from running. It was not clear if all or just some of them needed to be resolved in order for the tool to work.

[satano]

Yes, validating ReflectedWorkItemId is my first point there. I want to make it separate (first) validation and it will be validate always, even if the tool is disabled (this was the state before my changes to this validation).

FieldSkipMap is good addition. I added it to the list at the discussion, thanks.

[MrHinsh]

Excluded some work item types by default

This list should be saved in the appsettings.json file so that it can be configured by advanced users if needed.

[MrHinsh]

Ensure that you use the Work Item Type mapping tool to map ay work items types that have known naming diferences between teh source and target