Partially automating TC22 (No, that’s not a new designer drug)

Download the source for this tool here
kick it on DotNetKicks.com

Some years ago, a short while before I started stress testing my first set of diapers, Lennon was singing “Life is just what happens to you when you’re busy making other plans“. That was just the the first phrase that went through my head when I was told, last week, that another product would be pushed through the Certified For Windows Vista process. The phrase that followed was (reproduced here in a highly sanitized form) “Oh dear, we’ll have to run Test Case 22 on every installer”.

For those lucky people who have not yet had the dubious pleasure of running this test, it goes something like this: you open two instances of Orca, once containing your installer and one containing the reference schema. Then, you go through 80-odd tables, making sure that no custom fields have been added to the standard tables, and no custom tables (and their fields) have names starting with the “MSI” prefix. It is, in short, a drag, and a necessary one at that. You can read more about it in the Certified For Windows Vista Test Cases document.

Having a low boredom threshold, I know that if we were to do such a test manually, chances are that I’d miss something, with all the ensuing hilarity. This sounded like a job for [dramatic pause] a hastily clobbered together script! [fanfare]

A quick (and I mean real quick) browse through the documentation for Orca didn’t bring up any way to export the list of tables, so I googled it up for a while. The incredible Mr. Hanselman, of Computerzen.com fame, had already blogged about How to list all files in an MSI Installer using VBScript. Using this as a base, I modified the script so it would extract the table name and column name from the _Columns table in the MSI. This proved to be a bit tougher than I expected – the script broke every time I tried to refer to the Table field by name. That’s why the script extract.vbs selects everything in the table – something I don’t like to do usually. But then again, heck, I don’t like using VBScript either. The things you do when you’re stuck between the devil and the deep blue sea…

Validation

Once we have the list of tables and columns exported from our msi, and the list of tables and columns exported from schema.msi (the reference implementation), we can run them through a small command line application that we will use to root out any problems. Looking at class TC22.cs, I specified the following rules:

  • If the given table is not present in the reference schema
    • The table will be considered invalid if the name of the table starts with “MSI”, ignoring case.
      • The only exception to this will be table “MsiSFCBypass”, which is explicitly permitted in the test case specification.
    • No column in a custom table may contain a field with the MSI prefix, again ignoring case.
  • If the given table is present in the reference schema
    • All fields in the table must also be present in the reference schema.
      • The table in the installer may have less fields, but may not have any custom fields.

Any items which break these rules will be written to the command line as an Error. All custom items will report a warning, but this is for information purposes only, and does not necessarily mean that the item breaks the rules.

Note that TC22 also states that “The application’s Windows installer must not call Gacutil through a custom action.” This part of the test must still be verified manually.

Usage

Download the source for this tool here

To extract the table and column information from an MSI file, run extract.vbs on the MSI from the command line, for example:

extract.vbs myInstaller.msi

This will create a text file in the same folder. The text file will be called [installer file name].msi.txt. To verify the contents of this file, use the TestCase22 application from the command line:

TestCase22 schema.msi.txt myInstaller.msi.txt

You may want to redirect the output to a text file for easier reading:

TestCase22 schema.msi.txt myInstaller.msi.txt > result.txt

Please remember that this tool was put together fairly quickly, and while it appears to be working, there is no guarantee that it will be spot-on all of the time. Hope this helps.

kick it on DotNetKicks.com