Application SHIMS

4. Resolving Application Compatibility Issues with Shims

Most application incompatibilities occur when the application makes a request to the operating system. A very basic one is when an application asks, “Hey! What operating system are you?” Windows XP would respond, “I am 5.1.” Windows Vista would respond, “I am 6.0,” and Windows 7 would respond “I am 6.1.” Applications that are coded to only accept 5.1 would fail to launch.

Windows 7 Version

Did you notice that Windows 7 is actually version 6.1? There’s a good reason for that. Microsoft was the recipient of a lot of negative thoughts when people couldn’t get their applications to work on Windows Vista. They wanted to avoid the same issues with Windows 7. They aimed to ensure that as many applications as possible worked on Windows 7 if they worked on Windows Vista. And you know what? They were successful at doing that. One trick they used was to only increase the minor version number instead of the major version number of the operating system.

So Windows 7 jumped to 6.1 from version 6 of Windows Vista. Why would they do that? Many applications only check the major version when they install or run. They simply might not care about the minor version. This little solution prevents many compatibility issues.

 

ACT allows you to create mitigations or workarounds in the form of a shim. This shim…well, there’s no other way to put this…it’s a liar. You instruct Windows to lie to the application via the shim. Alternatively, you can think of it as an interpreter that sits between specified applications and the operating system. So in the case of “What operating system are you?” you can create a shim that instructs Windows to lie to that particular application and respond with “5.1” instead of “6.1.” The application might have no other compatibility issue and you end up with a fully working LOB program. Your project team will attempt to create these shims whenever an application has compatibility issues and a newer version or developer fix isn’t available. We’re going to focus on this process for while.

4.1. Using the Standard User Analyzer and Standard User Analyzer Wizard

Many legacy applications require administrative access to run. This is a problem with most business networks. That’s because we like to restrict users to standard users so that we can control security, configuration, and compliance systems. Typical of IT security, we need to find a bottom ground between security and the business operations. Windows 7 has some virtualization features that can fool some operations. For example, attempts to write to restricted parts of the Registry or file system will be redirected to more suitable locations in the user’s profile. But we still have issues where an application checks to see if the user is an administrator. The application may fail to start because of this hard check, even though no functionality may be affected.

Both the Standard User Analyzer (SUA) and the Standard User Analyzer Wizard allow you to do this testing and develop a fix to make the application work in this scenario. You will need to download and install the Microsoft Application Verifier from www.microsoft.com/downloads/en/details.aspx?familyid=c4a25ab9-649d-4a1b-b4a7-c9d8b095df18&displaylang=en to use the SUA tools.

Your shim creation and application testing process will probably start with the SUA tools. This administrative issue is just the tip of the compatibility iceberg. The rapid process of using SUA to fix application administrator access problems may end up providing a quick win for many compatibility problems. That makes it a good place to start.

Let’s delve into working with Microsoft’s sample Stock Viewer application and you can see the problem and the tools in action.

The best way to see the problem is to log in as a user that does not have administrative rights on the test PC. Launch Stock Viewer in this scenario and you’ll be greeted by a Permission Denied dialog box (Figure 23). You are correctly informed that you are not logged in as administrator. IT has decided that no users should have administrative rights on their PC. IT also believes that Stock Viewer will be able to run with no issues on Windows 7—once we sort out that administrator rights check.

The application does start up but it does have problems. We will have to fix this issue using the SUA tools. We’re going to start with the SUA Wizard. The wizard is a simple next-next-next tool for creating a fix for the application’s issue. You can find the tool on the Start menu under All Programs => Microsoft Application Compatibility Toolkit => Developer And Tester Tools. Launch it and you’ll see the screen shown in Figure 24.

You will tell the tool to shadow a particular executable, that is, the application that you will run. You can supply any parameters that would normally be used with the executable to configure the program execution. You can then click the Launch button to run the program you are shadowing or fixing.

Figure 23. The application requires administrative rights.

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_23.jpg


Figure 24. Launch the Standard User Analyzer Wizard.

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_24.jpg

 

At this point, the program will run and experience all the problems you should now be already familiar with. That allows the SUA Wizard to see the issues. Run the program, test it thoroughly, and then exit it. You’ll now be asked by the wizard if the program ran without having any problems. If you click Yes, then the wizard will exit without making changes. If your program complained about administrative rights, then click No to create a fix. That brings up a screen (Figure 25) showing a potential fix for your application. This screen contains one or more shims that will be associated with your program’s executable.

Three interesting shims have been configured. LocalMappedObject resolves a problem where an application tries to create a file mapping object in the Global namespace. The Global namespace hides the complexities of physical storage. This is not possible without administrative rights. With this shim enabled, the application will create the file mapping object in the Local namespace. Take a look at CorrectFilePaths. That will use virtualization to prevent Stock Viewer from trying to write into a location that the user has no access to. ForceAdminAccess is also an interesting one. That’s the one that will fix the core issue at hand. It won’t run the program as an administrator. Instead, it will trick the application into believing that the user is an administrator.

Figure 25. Potential shims in the SUA Wizard

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_25.jpg

 

You now must launch the program again. This time it will temporarily use the shims to run. This is a chance to test the potential fix before saving it. This is a good idea because there’s a good chance the SUA fix won’t be enough to get all functionality in your program running. Click the Launch button again and thoroughly test your application. The SUA Wizard will again sit back and wait for the program to exit.

When you do exit the tested program, you are again asked if the application ran without any issues. If your answer is No, you will be returned to the previous Potential Mitigations screen. You can enable or disable mitigations. If you skip this, then you are asked if you want to try something else:


Run The Standard User Analyzer

That is, run the full tool, not the wizard.


Mark The Application As ‘RunAsAdmin’

This option tells Windows the program should always be run as an administrator.


Export The Mitigations So Far

We’ll come back to this option in a moment.


Exit

Abandon all hope of the SUA tools helping this application!

If you responded Yes to the mitigations fixing your problem, then you will get the option to export your mitigation. That will create an MSI file, which you can install on every PC that will have this application. MSI files can be easily deployed using MSIEXEC via startup scripts or Group Policy, or by using advance software deployment solutions.

Deploying a Mitigation with Configuration Manager

Medium and large organizations may use products such as System Center Configuration Manager or System Center Essentials for deploying applications. To eliminate mistakes, you can automate the deployment of your Standard User Analyzer-generated mitigations with your applications. This is possible by creating a package and program from the mitigation MSI. You can then specify the mitigation’s package installer program as a dependency for your fixed application. Any advertisement of that application will preinstall the mitigation if it is not already present on the client computer.

 

The SUA will produce the same sort of fix as the SUA Wizard, but it gives you a look under the covers to see what is going on. With more experience of application compatibility engineering, you’ll be able to interpret this data to figure out complex problems.

Just like with the wizard, SUA requires that you specify the executable for the application and any required parameters. The Launch Options area on the right contains two configurable options. The Elevate check box is enabled by default. According to Microsoft, this option enables the application to run either as a standard user or an administrator. Clearing the check box will cause the application to run as administrator. That will give SUA the opportunity to identify issues that might occur when a normal user runs the program. The Disable Virtualization check box is grayed out when Elevate is enabled. Microsoft recommends disabling it if a lot of data will be created in virtualized locations on the PC by the application. In Figure 26 we have configured SUA to shadow Stock Viewer with the Elevate option turned off. Clicking Launch starts the program that you want to test and fix.

Figure 26. Standard User Analyzer

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_26.jpg

 

As before with the wizard, you test the program and see all the faults, allowing SUA the opportunity to create fixes. You can see in Figure 27 that a lot more information about the analysis is there for you to check out. Debug Info shows you actions performed. Each of the tabs, such as File, Token, Name Space, and Other Objects, reveals more information.

Figure 27. Testing the application with the Standard User Analyzer

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_27.jpg

 

Click to open the Mitigation menu, where you have three options. As it stands right now, SUA has generated a possible fix but has done nothing with it. You can test that mitigation by choosing Apply Mitigations. Then rerun and retest the application. You can then choose Undo Mitigations to uninstall the possible fix. If you’re happy with the fix, you can choose Export Mitigations As MSI.

So there you go; that’s the Standard User Analyzer tools. It’s quite possible that you can quickly fix issues for applications associated with administrator rights very quickly with either one of these tools. If you took the time to try out more of the Stock Viewer functions, then you’ll already know that it is a very “broken” application. SUA isn’t a panacea, either, as it doesn’t fix enough of the problems. Make sure you have undone any mitigations created in SUA. It’s now time to move on to the big gun.

4.2. Creating Shims

You saw a few shims being created when we looked at the Standard User Analyzer. There are over 300 shims that you can use in ACT 5.6. An automated tool like SUA can only do so much for you. Sometimes you need to get under the hood yourself and get some grease on your hands.

A lot of things can prevent an application from working correctly:

  • The operating system version is different and the application check is very specific.

  • A verification of the version of Internet Explorer finds a newer, unsupported version.

  • File or Registry paths have changed or there is a lack of access rights.

  • The Windows GUI, Aero, is quite advanced and can cause trouble for some legacy applications.

  • An application may need to spawn off another program that requires an elevated UAC token to execute but it doesn’t know how to.

  • Some applications can develop minor issues that could be safely ignored (after testing) but cause the program to crash.

  • Annoying error boxes can repeatedly appear because of minor issues that can be ignored.

Now you are probably wondering, “I can test an application but how do I really know what’s wrong with it?” Tools are available that you can use to help identify the root causes of any failures. If you use the SUA, then you’ve already installed the Application Verifier (Figure 28). You can configure this tool to shadow an application, and it will generate logs of faults that were detected. Another tool that will prove to be very helpful is Process Monitor (ProcMon), a free download from the Microsoft Sysinternals site at http://technet.microsoft.com/sysinterna1s/bb545027.aspx. This tool allows you to watch how an application interacts with the file system and the Registry while it runs. When you see an application failure, you can look back through recent activity and see exactly what it tried to do. And there is always the good old-fashioned eyeball test. Sometimes the application will tell you exactly what is wrong; for example, it will work only with Internet Explorer 6 (IE6).

Figure 28. Application Verifier

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_28.jpg

 

Very often an application, such as a setup.exe, will do a basic check of the operating system. This is typical of driver installers that were written specifically for Vista. The OS check will fail but the driver or program will work fine on Windows 7 if they can only get installed. As you can see in Figure 29, you can right-click on an executable file and create shims for it using the Compatibility tab. You can pick an operating system that the shim should pretend to be, such as Windows XP SP2, for administrative access, and control the screen resolution or colors. This quick solution may provide a handy fix.

Figure 29. Application compatibility properties

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_29.jpg

 

We’re going to continue to work with Stock Viewer because it contains a number of problems that are quite common. Start up Stock Viewer on a PC with no shims or mitigations installed. You’ll get the usual administrator access warning. On the Tools menu there is a Show Me A Star option. That will also fail because the PC has Windows 7, not Windows XP. Clicking the Browse button reveals that Stock Viewer will only work with IE6. If you click Options => Save Preferences, then you’ll get a failure because a standard user cannot write to the default location. If you click Tools => Show Me A Star, the tool will fail to launch because the operating system is Windows 7, not Windows XP. Finally, if you click File => Update Kernel, you will see that this fails because you are trying to update a protected system file. Even with administrator access, you won’t be able to do this on Windows 7. We’re going to do our best to fix all of these issues by creating shims in the ACT Compatibility Administrator.

Compatibility Administrator Requires Admin Rights

Strangely enough, you are going to want to do some compatibility fixing on the Compatibility Administrator application. It requires that you run the program with your elevated UAC token or administrator access instead of with a standard user token/rights. It’s all too easy to click the Start menu entry for the program without right-clicking and picking Run As Administrator. The program will start but it won’t work correctly. A handy fix is to right-click on the menu entry, select the Compatibility tab, and check the Run As Administrator check box. Now the program will always seek administrator access when you run it.

 

Launch the Compatibility Administrator and you’ll get the window shown in Figure 30. The navigation pane on the left shows a number of things. The System Database has three entries:


Applications

This is an extremely useful place to start with any application you find a problem with. Microsoft maintains a database of shims for existing applications. This one contains over 6,500 fixed applications. There is a possibility that the fix you need is already here if you’re using a well-known off-the-shelf application with compatibility issues. We’ll come back to that in few moments.


Compatibility Fixes

These are the potential shims that can be used to change how things work. An example might be a Registry redirect.


Compatibility Modes

There is a subtle difference between compatibility modes and fixes. A mode is something like a setting, such as Windows XP SP2 or a 640×400 screen resolution.

Figure 30. The Compatibility Administrator

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_30.jpg

 

Per User Compatibility Settings shows the currently installed shims for the currently logged-in user. You can see the fixes for the setup.exe driver installer and for the Compatibility Administrator.

Custom Databases is where all the magic happens. This is where you can create a database of shims for each of your applications that will require a compatibility fix. Compatibility Administrator takes a different approach to deployment than you found in SUA. Instead of creating one MSI for each application, it is possible to create a single database file with an SDB extension for your entire organization. You can then deploy that SDB to every PC on your network and fix all possible compatibility issues. Of course, you could go with an alternative solution and have smaller SDB files for departments or offices, or even per-application SDB files.

Deploying an SDB File

Every Windows 7 PC has a program in C:\Windows\System32 called sdbinst.exe. You can use this program to install the fixes contained in an SDB file. You can install an SDB file with:


You can uninstall an SDB file with:


This means you can use solutions like startup scripts and advanced software deployment solutions such as Configuration Manager to run the program using the SDB file as a parameter. Have a look at any package program in ConfigMgr; all it does is run an executable with a parameter. There’s nothing to stop you from doing that with the sdbinst.exe that’s found on the client computer using an SDB file that’s located on the network as the parameter.

You can include your SDB files in a Windows 7 deployment image. Doing so reduces the work required to deploy legacy applications. However, you might want to limit what you include in the image for troubleshooting reasons. You might also want to maintain a centralized library with complete control in case you need to change the SDB file at a later point.

 

A default custom SDB file is present in the Compatibility Administrator. You can either rename it to suit your needs or open a preexisting one by clicking the Open button.

Two tools will help you navigate the database and fixes on your ACT PC:


Search

Using this tool, you can point at a folder on the PC to search for programs with a compatibility fix. You can then double-click on the found programs to view the details of the fix.


Query

This tool allows you to search your database based on application name, filename, and various other properties.

Let’s look at creating a fix for an application that already exists. Start by renaming the custom database. Right-click on Custom Database in the console and choose Rename. It’s good to follow naming standards. For our example, rename this database to deploy.com (to match our domain). We have identified that Photoshop CS2 is on the network and is having issues with Windows 7 There isn’t a budget to upgrade to a newer version, so we will have to deploy a fix. Click Query and enter the name of the program. You can use % as a wildcard. So you can enter either Photoshop CS2 to get a precise result or Photoshop% to view everything starting with the word Photoshop. Note that the search tool is not case sensitive. Our search brought up the results in Figure 31.

Figure 31. Querying for an application fix

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_31.jpg

 

Two programs were tagged as Photoshop CS2. It turns out that there is another program in the package that also needs to be fixed. Double-click one of them and you are brought to the application in the database, as shown in Figure 32.

Each application is boldface in the detail pane. You can now right-click an application (or even the fix in the detail pane) and select Copy. Browse back to your custom database and paste the application. Repeat this for each required application. You can see in Figure 33 that you can add many applications into the database. There are even noncommercial applications like old games that you thought might never see the light of day again after your upgrade to Windows 7.

Figure 32. The application in the database

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_32.jpg


Figure 33. A completed SDB

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_33.jpg

 

Before you go any further with testing this database, you need to save it. This will create the SDB file in the location of your choosing. If you use multiapplication SDB files, you need a strict testing and change control process to avoid nasty calls at 3:00 a.m., angry “conversations” with the boss, and searches of employment websites.

Now you can right-click on the custom database and install it onto your PC for testing. A new node will appear in the navigation pane on the left called Installed Databases. Expand it to reveal your fixes. Now you can test those applications on this PC. Note that an application only loads the associated shims when it starts up. A running application will not dynamically load any associated shims until you restart it. Remember to uninstall the database from the PC when you have finished by right-clicking on it under Installed Databases and selecting Uninstall.

That’s all well and good if your application is one of those in the Applications database. But what if you have a lesser-known or a custom application that cannot be fixed or upgraded? Stock Viewer comes to mind. You will have to create shims for it by hand based on the faults you identified.

First, create a new application in the custom database. Do so by right-clicking on the database and selecting Create New => Application Fix. The Create New Application Fix wizard appears (Figure 34). You should populate the fields with the name of the application, the vendor, and the location of the application executable.

Figure 34. Create New Application Fix wizard

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_34.jpg

 

The next screen allows you to choose operating system modes (for example, Windows XP SP2 or Windows 2000) or compatibility modes for screen resolution and colors. Normally you will not choose any of these options and move ahead to the next screen.

The first problem we will deal with is the check for administrator access. You saw earlier with SUA that the ForceAdminAccess compatibility fix (shim) fooled the application into thinking that it was running with an administrator’s rights. Scroll down through the available shims in the Compatibility Fixes list box (Figure 35) and check that shim. If you click Next you will see all the detail involved in creating the application fix. Click Finish and you get a new application fix in your database. If you save and install that database, you should now be able to run Stock Viewer without the annoying warning about administrator rights.

Figure 35. Selecting an application compatibility fix

image: http://tutorial.programming4.us/image/1305/The%20Application%20Compatibility%20Toolkit_35.jpg

 

That’s the first fix done. You can go back and edit the application fix by right-clicking on it in the detail pane and selecting Edit Application Fix. You’ll return to the wizard’s compatibility fix screen.

There are four fixes in Stock Viewer that we can use to solve problems that we encountered when we first discussed Stock Viewer:


VirtualRegistry

This fix allows you to redirect requests by the application from one Registry location to another.


CorrectFilePaths

This fix allows you to redirect file operations from an unavailable location (it’s simply not there or a standard user has no permissions) to a valid location where the user has permissions to read and/or write.


WRPMitigation

Windows Resource Protection (WRP) only allows “trusted installers” to modify system files and Registry items. This fix lies to an application every time it tries to modify a protected resource that it has no rights to modify.


OS Version Lie

There is a compatibility fix for each lie you wish to present to an application—for example, Win2000SP3VersionLie and WinXPSP2VersionLie.

Let’s configure each of the compatibility fixes now to get the aforementioned features of Stock Viewer working. When you select a compatibility fix, you can configure the command line for it by clicking parameters. This step is required for some fixes, such as when you specify a location to protect and an alternative location for the application to use. Table 1 shows the syntax of the fixes we will employ.

Table 1. Stock Viewer compatibility fixes syntax
Compatibility fix Parameters in the command Line
VirtualRegistry IE60 or ADDREDIRECT(OldPath^NewPath)
CorrectFilePaths “OldPath:NewPath”
WRPMitigation None required
WinXPSP2VersionLie None required


Make Changes One by One

When you are dealing with complex technology, making many changes at once can make it difficult to understand what caused the results. As with any other complex IT operations, you should try to make one application fix at a time and test it. You can then undo that fix, tweak it, or add another fix, all in a predictable manner.

 

Let’s go through these parameters and see how to use them to fix our problem. The first one is VirtualRegistry, which we’ll use to fix our IE6 issue. Windows 7 comes with IE8 and you cannot downgrade it. You can see two solutions in Table 1. The first one, IE60, is a prepackaged cheat from Microsoft to quickly tell any associated application that Internet Explorer 6 is installed. You can also use IE401 for Internet Explorer 4.01, or IE55 for Internet Explorer 5.5. There are lots more of these, which you can find on Microsoft TechNet at http://technet.microsoft.com/1ibrary/cc749368(WS.10).aspx.

That’s fine for Internet Explorer. But what if there’s some other Registry cheat you need to create? The solution is to use ADDREDIRECT(OldPath^NewPath) to create a new value with a setting in an alternative location. For example, create a REG_SZ called Version with a value of 6.0.2404.0 in HKEY_LOCAL_MACHINE\Software\Microsoft\Internet Explorer 6. You need to create a new one for this example because the Version value already exists in the Internet Explorer key. The value of the new version key will be 6.0.2404.0.

Now you can set the parameter of VirtualRegistry to tell the application to go to the new Internet Explorer 6 key instead of the original Internet Explorer key:

 

There are two things to note with this ADDREDIRECT(OldPath^NewPath) solution. The command will use HKLM and not HKEY_LOCAL_MACHINE. You will also need to use %20 to replace space characters.

Next you have to deal with the pesky Save Preferences menu option. That will use the CorrectFilePaths fix with this command:

 

What you have done there is fool the application into thinking it will use the old file location in C:\Windows that the user has no access to. Instead, you are using an alternative location. %appdata% is an environment variable that points to the user’s own profile where they can save their personal settings.

WRPMitigation is pretty simple. You just check the box and there are no parameters to set. Any attempt by the application to modify protected files will, of course, not succeed. But the application will be told that the actions did succeed, thus preventing unwanted failures. Be careful with this fix in case there are other dependencies on those updated files within the application.

The last one you will use for Stock Viewer applies to another executable that Stock Viewer starts on your behalf: the Show Me A Star tool. This is just a demo tool that uses features of Windows XP to display a star on the screen. This won’t work on Windows 7 without a fix. You can find that program, DWM Composting Rendering Demo.exe, alongside StockViewer.exe in C:\Program Files\StockViewer. (It may be Program Files (x86) if you are using a 64-bit Windows 7 installation.) You will need to add another application to fix your customer database in the Compatibility Administrator for that executable. Then you can add the OSVersionLie (WinXPSP2VersionLie, in this case) compatibility fix.

At this point you should have all the aforementioned features of Stock Viewer working. Of course, there are more broken features in there, and there are hundreds of compatibility fixes. You can find a lot more information in the help file and on Microsoft TechNet at http://technet.microsoft.com/library/cc722305(WS.10).aspx. Here’s a few fixes that you might find useful:


DisableDWM

This fix will prevent the application from using the Aero interface of Windows Vista and Windows 7.


FakeLunaTheme

Color may be important in some applications. A few legacy applications may not have the vivid colors they should have on Windows Vista or Windows 7.


IgnoreAltTab

This fix tells the application to ignore the following key sequences or keys: Alt+Tab, Alt+Esc, Ctrl+Esc, the Windows logo key, and the Application Key (Microsoft Natural Keyboard only).


IgnoreException

Microsoft warns that you should use this fix only if you have to. That’s because IgnoreException doesn’t have a filter for valid or invalid exceptions.


IgnoreMessageBox

Using the caption and text from an identified message box, you can prevent the message box from being displayed to a user.


RunAsAdmin

With this fix, the program requires administrative access and must ask for it if it doesn’t already have it.


RunAsInvoker

The program should run as the current standard user but it currently asks for administrative rights. This fix will prevent that.


VirtualizeDeleteFile

Pretend that a file was successfully deleted by an application in scenarios where it cannot be deleted.

By now, you should have the tools and know-how to fix most applications that have compatibility issues. But there are always a few exceptions to the rule. For example, what if you need to use Internet Explorer 6 for a LOB website application that you can’t fix? You can’t install IE6 onto Windows 7. We’re going to look at the last solution in our application compatibility catalog now.

Read more at http://tutorial.programming4.us/windows_7/Evaluating-Applications-for-Windows-7-Compatibility—The-Application-Compatibility-Toolkit-(part-4)—Resolving-Application-Compatibility-Issues-with-Shims.aspx#huDKAg8tcUE35jbo.99