Unable to access OWA: Corrupt restriction filter data

In the past week, we encountered a strange error in OWA for only one user in a large Exchange 2007 organization. Our user was using only OWA to access its mailbox and every time he tried to log in was not able to do it due to the following error:

Request
Url: https://webmail.contoso.com:443/owa/forms/premium/StartPage.aspx
User host address: 192.168.100.100
User: Andrea Tedeschi
EX Address: /o=Contoso/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=AndreaT
SMTP Address: andreat@contoso.com
OWA version: 8.1.359.2
Mailbox server: EX-CAS-A01.corp.contoso.com

Exception
Exception type: Microsoft.Exchange.Data.Storage.CorruptDataException
Exception message: Corrupt restriction filter data.


Call stack
Microsoft.Exchange.Data.Storage.FilterRestrictionConverter.

GetPropertyDefinition(MapiProp mapiProp, PropTag propTag)
Microsoft.Exchange.Data.Storage.FilterRestrictionConverter.

InternalCreateNativeFilter(ExchangeTimeZone timeZone, MapiProp mapiProp,
Restriction restriction, Int32 depth)
...
Microsoft.Exchange.Data.Storage.SearchFolder.GetSearchCriteria()
Microsoft.Exchange.Data.Storage.DefaultFolderManager.
GetRemindersSearchFolderState(SearchFolder reminders)
Microsoft.Exchange.Data.Storage.DefaultFolderManager.

VerifyAndFixRemindersSearchFolder(StoreId reminderSearchFolderId)
Microsoft.Exchange.Data.Storage.DefaultFolderManager.GetDefaultFolderId(DefaultFolderType defaultFolderType)
Microsoft.Exchange.Clients.Owa.Core.UserContext.GetDefaultFolderId

(DefaultFolderType defaultFolderType, String folderName)
Microsoft.Exchange.Clients.Owa.Core.UserContext.get_RemindersSearchFolderId()
Microsoft.Exchange.Clients.Owa.Premium.StartPage.OnInit(EventArgs e)
System.Web.UI.Control.InitRecursive(Control namingContainer)
System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint,

Boolean includeStagesAfterAsyncPoint)

Inner Exception
Exception type: Microsoft.Exchange.Data.Storage.InvalidPropertyTypeException
Exception message: Unable to create custom property with property key

[{00062002-0000-0000-c000-000000000046}:0x8232] type [Boolean].


The above error description ponts to some kind of problem that could be related to an Appointment Reminder object in the mailbox and with a property with the following code: [{00062002-0000-0000-c000-000000000046}:0x8232]

Searching on the net and on MSDN we found a page under the Exchange Protocol Documentation (http://msdn.microsoft.com/en-us/library/ee217690.aspx) explaining that our code was pointing to a structure named PidLidRecurrencePattern.

Then we encountered a page on LATAM Team Blog that was explaining the same issue (http://blogs.technet.com/latam/archive/2008/09/24/owa-2007-exception-corrupt-restriction-filter-data.aspx) and, as outlined in the article, we tried to solve the problem by running OUTLOOK.EXE with /cleanreminders switch on a workstation against the affected user, but we had no chance.

Later we launched MFCMapi and we entered the user mailbox to check for possible corrupted items on calendar, old freebusy or reminders items.
To access the mailbox root directory select Session, Logon and Display Store Table, Select your MAPI profile, Select Mailbox and expand the root container.

We tried the following procedure:

Checked on the Calendar folder and nothing was present on it.
Checked under the Reminders folder and found some old items.
Right clicked "Reminders" folder and selected "Delete Folder".
Deleting, we ensured the chekbox "Hard Deletion" was selected.
Then closed MFCMAPI, launched OUTLOOK.EXE with /resetfolders, then closed it.

Again, this action plan was unlucky. So our last attempt was to check the FreeBusy folder for old items and we found one. We deleted it with "Hard Deletion" and closed MFCMapi tool.

With the above action the user was again able to access its mailbox without problems.

MFCMapi Download
http://mfcmapi.codeplex.com/



--

Andrea Tedeschi \ Senior Systems Engineer \ andrea (at) andreatedeschi (dot) com

Why Outlook prompts for credentials at startup?

Recently I was engaged various times for an issue related to the Outlook 2003 and 2007 client when using an Exchange 2007 hosted mailbox. The scenario to be analyzed was the unexpected prompt for user's credentials trough Credential UI at the Outlook startup on a domain joined Windows XP/Vista client. Generally speaking, on a common similar domain environment, you should not enter your credentials when opening Outlook because the process will run on the user security context. I made an assessment on this behavior and reproduced the same issue on my lab so I will post some useful considerations.

Some common causes for this behavior are described in the following KBs:

You receive error message and are continuously prompted for your Windows NT domain credentials in Outlook 2000, Outlook 2002, or Outlook 2007
http://support.microsoft.com/kb/321652/en-us

You are prompted for your user credentials when you send or receive new e-mail messages in Outlook 2007 or in Outlook 2003
http://support.microsoft.com/kb/825230/en-us

You are unexpectedly prompted for your user credentials in Outlook 2003
http://support.microsoft.com/kb/910346/en-us

Outlook 2007 prompts you repeatedly for your e-mail account credentials
http://support.microsoft.com/kb/923579/en-us

You are repeatedly prompted to enter your credentials when you try to connect to an Exchange mailbox by using Outlook 2007
http://support.microsoft.com/kb/927612/en-us

You are prompted unexpectedly for credentials after you start Outlook 2003
http://support.microsoft.com/kb/954882/en-us

Outlook 2007 prompts you repeatedly for a password under certain network conditions
http://support.microsoft.com/kb/956531/en-us


After analyzing, trying and excluding the above common causes and after confirming the "Always prompt for username and password" option in Outlook is not in place, we assume the user's credentials for mailbox access trough Outlook client should not be asked if the workstation is a domain-joined machine, the user was correctly logged-on with valid credentials, the password is not expired and the required ports to allow client, DC/GC's and Exchange authentication processes are opened: DNS, LDAP, LSA RPC, Kerberos and NTLM ports.

The credential management and the Credential UI are completely indipendent from Exchange 2003 and/or 2007 and the same Outlook 2003 and/or 2007. The authentication and authorization process is completely delegated to the Windows operating system.



The prompts for credentials trough Credential UI should be expected for non-domain joined machines running Outlook client and for some roaming scenarios, for example roaming users with laptops configured to use RPC over HTTPS, as described in the KB's:

You must provide Windows account credentials when you connect to Exchange Server 2003 by using the Outlook 2003 RPC over HTTP feature
http://support.microsoft.com/kb/820281/en-us

In this first case the cause could be:

> You are using Basic authentication to the proxy server for Exchange.
> You are using NTLM authentication to the proxy server for Exchange, but Windows does not automatically send the NTLM challenge/response data. Windows does not do this because the older LANMAN challenge/response password is included in the authentication data.


When you use Outlook to connect to your Exchange server mailbox by using RPC over HTTP, you receive messages that continuously prompt you for your network password
http://support.microsoft.com/kb/895276/en-us

In this second case:

> Your password has expired.
> Your administrator has set the User must change password at next logon option for your user account.



To understand the behavior observed in this case we should do a brief explanation.

In the case you will asked from credentials by Credential UI and you choose to flag the "Remember my password" option, the process of saving and using saved credentials will be managed by credential manager or Windows "Stored User Name and Password" functionality, as described in the following KB:

Behavior of stored user names and passwords
http://support.microsoft.com/kb/281660/en-us

Stored User Names and Passwords is a mechanism that dynamically and manually creates credential sets (a user name and password) for resources. This functionality is available from the graphical user interface (GUI) and from the command line.

The saved credentials are stored in a portion of the user profile, in the following locations:

> For Windows XP and Windows Server 2003:
%USERPROFILE%\Application Data\Microsoft\Credentials\{UserSID}\Credentials
%USERPROFILE%\Local Settings\Application Data\Microsoft\Credentials\{UserSID}\Credentials

> For Windows Vista and Windows 7:
%USERPROFILE%\AppData\Roaming\Microsoft\Credentials\{GUID}
%USERPROFILE%\AppData\Local\Microsoft\Credentials\{GUID}

You may have to uncheck "Hide protected operating system files" to see the folder content.

To review currently stored credentials you can check via Control Panel then User Accounts, or call directly the UI by running the following command:

rundll32.exe keymgr.dll, KRShowKeyMgr

There is also a command-line tool on Windows Server 2003, Vista and Windows 7 that allows managing of stored credentials. It's not available in Windows XP but works with, you simply should copy the Windows 2003 version.

cmdkey.exe command
http://technet.microsoft.com/en-us/library/dd996689(WS.10).aspx


After this explanation, let's focus on the issue.

If, for whatever reason, logging into your Outlook client you are on one of the previous password prompts scenarios and you choose to save, even only one time, your credentials, the system will create an entry in “Stored User Name and Password” on your user profile, on you actual machine. For example, you may have to connect Exchange from the Internet using RPC over HTTP, your password is expired and the credential UI shows up (as expected). Then you enter your credentials and check "Remember my password".

Now you have saved the credentials, every time you will open Outlook the system will retrieve the stored credentials and use it for access Exchange. You can observe this behavior by doing some test and taking traces with Network Monitor and checking kerberos tickets with klist tickets command.

For example, once you saved your credentials, the system will never get a Kerberos ticket to access the resource. This is valid not only for Exchange, but for all other resources and protocols (i.e. CIFS, LDAP) you generally access trough NTLM/Kerberos authentication.

You can observe this behavior by following this steps:

1. Logon to a domain workstation and check for Kerberos tickets:

C:\>klist tickets
Cached Tickets: (2)
Server: krbtgt/CORP.CONTOSO.COM@CORP.CONTOSO.COM
Server: cifs/DC01.CORP.CONTOSO.COM@CORP.CONTOSO.COM

2. Open Outlook client and check again for Kerberos tickets:

C:\>klist tickets
Cached Tickets: (3)
Server: krbtgt/CORP.CONTOSO.COM@CORP.CONTOSO.COM
Server: cifs/DC01.CORP.CONTOSO.COM@CORP.CONTOSO.COM
Server: exchangeAB/EX-MBX01.CORP.CONTOSO.COM@CORP.CONTOSO.COM

You will see tickets for other Exchange Servers if you have more (ie. Public Folders, etc).

3. Purge existing Kerberos tickets and put your credentials on the store:

C:\>klist purge
C:\>cmdkey /add:EX-MBX01.CORP.CONTOSO.COM /user:CONTOSO\andreat /pass
CMDKEY: Credential added successfully.
C:\>klist tickets
Cached Tickets: (0)

4. Logoff and logon again, then open Outlook. You will access correctly your mailbox but no Kerberos ticket were issued for your Exchange logon.

C:\>klist tickets
Cached Tickets: (2)
Server: krbtgt/CORP.CONTOSO.COM@CORP.CONTOSO.COM
Server: cifs/FILESRV01.CORP.CONTOSO.COM@CORP.CONTOSO.COM


This behavior is partially outlined in the following article:

How to manage stored user names and passwords on a computer in a domain in Windows XP
http://support.microsoft.com/kb/306992/en-us

When you type your user name and password for the resource, and then click to select the Remember my password check box, your logon information is stored with your user account. When you next connect to the same resource, these stored credentials are used to automatically authenticate you.…When you access a resource, the authentication package searches in Stored User Names and Passwords for the most specific credential that matches that resource.

Following is the key point of the problem I was engaged for.

After saving credentials, if you change your account password on the same workstation where credentials are stored and only after logging on (trough CTRL+ALT+CANC, Change Password) the system will automatically check for credentials presence on the store and will update it. At the next Outlook use you will not asked for credentials as expected and the stored set will be used with success because was updated during the password change.

Now suppose you make use of multiple workstatsions and log onto another machine.
If you change your account password on another workstation respect on where credentials are stored, or trough OWA Change Password (so, on a different machine because the change process will happens on Exchange Server) or, again, if you change your password on the same workstation before loggin on, the previous stored credentials will not be updated to reflect the change. Here is the point: at the next opening of Outlook on your machine, you will be asked for username and password because the credential manager will try to access Exchange with the saved (old) password. This will happen until you will check again "Remember my password" option. You can reproduce this behavior in lab and is partially outlined on the following KB:

Logging On with New Password Does Not Update Matching Stored User Names and Passwords Credentials
http://support.microsoft.com/kb/281246/en-us

After you change a password on one computer and then log on to another computer, the matching credentials in Stored User Names and Passwords are not updated.
This problem occurs because Stored User Names and Passwords is not able to update the stored passwords...
These credentials cannot be updated during authentication because the credentials are stored in the profile which is not yet loaded. Because the profile is not available, there is no way to update it.

In my case some users were complaining about the unexpected Credential UI and generating calls to the HelpDesk. If you want to address this issue there is some workarounds to adopt, but some could have some implications to evaluate. By adopting one of the following methods the user will be unable to store its password: in normal conditions the user will not see the Credential UI, but in the case it will appear, there will be not option to save credentials and this will prevent the issue of the unexpected popup.

> Completely disable "Stored User Name and Password" on Windows trough GPO (Network access: Do not allow storage of credentials or .NET passports for network authentication). This will preclude storing all creentials type not only for Outlook. The following Registry value removes the "Remember" option from all authentication UIs:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\LsaValue
Name: disabledomaincreds
Value Type: REG_DWORD
Values: 1 = do not store domain credentials



> Configure laptops to use the old Outlook Credential UI. This UI asks for username and password but will not permit to save it trough "Stored User Name and Password". In addition you have to remove any eventually present credential mapping for Exchange on the store. The operating system credentials UI is disabled for Outlook and the Outlook Credentials UI will be used instead.

For Office 2003:
HKEY_CURRENT_USERS\Software\Microsoft\Office\11.0\Outlook\RPC
For Office 2007:
HKEY_CURRENT_USERS\Software\Microsoft\Office\12.0\Outlook\RPC
Value Name: DisableCredUI
Value Type: REG_DWORD
Values: 1 = Operating system credentials UI is disabled





--

Andrea Tedeschi \ Senior Systems Engineer \ andrea (at) andreatedeschi (dot) com

Speeding up the performance counters analysis

Analyzing performance counters of your machine may be a very time consuming operation. Also there is always something you can miss in reviewing the logs. I found a very powerful and cool tool on Codeplex site to speed up the reporting operations. You can use this to rapidly identify the most evident problems and then perform a more in-depth analysis beginning from its output.

Performance Analysis of Logs (PAL) Tool is based on Relog command. "The tool generates an HTML based report which graphically charts important performance counters and throws alerts when thresholds are exceeded. The thresholds are originally based on thresholds defined by the Microsoft product teams and members of Microsoft support, but continue to be expanded by this ongoing project. This tool is not a replacement of traditional performance analysis, but it automates the analysis of performance counter logs enough to save you time. This is a VBScript and requires Microsoft LogParser."

In the past I had performance analysis with other tools and script I made myself, but PAL is actually my preferred solution. Four things i love from this tools:

- Job queue. You can create your analysis job queue and automate multiple analysis.
- You can set the priority of the reporting process.
- The tool is able to manage counters from Server products like Exchange and SQL.
- You can customize your template trough XML and model your analysis based on your needs.

Really a good job from the authors.

Performance Analysis of Logs (PAL) Tool
http://www.codeplex.com/PAL

Download link
http://www.codeplex.com/PAL/Release/ProjectReleases.aspx
--
Andrea Tedeschi Blog Linkedin Profile andrea (at) andreatedeschi (dot) com

Windows 2008 Server Core Configurator

Windows 2008 Server Core Configurator tool allows you to simplify the configuration of a new installation of Windows 2008 Server Core. This utility is made of a script collection and will be able to manage the following settings:

Product Licensing
Networking Features
DCPromo Tool
ISCSI Settings
Server Roles and Features
User and Group Permissions
Share Creation and Deletion
Firewall Settings
Display Settings
Add & Remove Drivers
Screen saver Settings
Windows Updates (Including WSUS)



You can access the project on Codeplex at http://www.codeplex.com/CoreConfig and download the tool at http://www.codeplex.com/CoreConfig/Release/ProjectReleases.aspx


--

Andrea Tedeschi Blog Linkedin Profile andrea (at) andreatedeschi (dot) com

Engineering Windows 7

Are you finding informations about Windows 7?
There is a new blog on msdn.com called "Engineering Windows 7". E7 blog is made by two senior engineering managers for Windows 7, Jon DeVaan and Steven Sinofsky. Blog address: http://blogs.msdn.com/e7

--

Andrea Tedeschi Blog Linkedin Profile andrea (at) andreatedeschi (dot) com

Joining the FoolDNS beta program

Yesterday I posted about the FoolDNS Project here and now I joined the beta testing program.
In the next days I will try FoolDNS with IE7, IE8 Beta on Windows XP and Windows Vista and will install also some third party browsers on a XP based virtual machine. I will post soon more (public) information after my testing.

--

Andrea Tedeschi Blog Linkedin Profile andrea (at) andreatedeschi (dot) com

Linux Integration Components for Hyper-V are available!

After the retirement of IC's RC2, Mike Sterling posted about the availability on Microsoft Connect site of the final version of Linux Integration Components for Hyper-V here:
http://blogs.msdn.com/mikester/archive/2008/09/10/linux-integration-components-now-posted.aspx

The actual release is localized in 12 languages and will provide the following features:

Driver support for synthetic devices: The Linux integration components include support for both the synthetic network controller and synthetic storage controller that have been developed specifically for Hyper-V. These components take advantage of the new high-speed bus, VMBus, which was developed for Hyper-V.
Hypercall adapter: The Hypercall adapter is a thin layer of software that translates the Xen-specific virtualization function calls from a Xen-enabled Linux kernel to Microsoft Hyper-V hypercalls. This results in improved performance for the Linux virtual machine.
Fastpath Boot Support: Boot devices now take advantage of the storage VSC to provide enhanced performance.


To download Linux Integration Components for Hyper-V you should visit the following address at Microsoft Connect Site:
https://connect.microsoft.com/Downloads/Downloads.aspx?SiteID=495

If you have not access to the project you should search it on the Connection Directory, or, if you previously declined the project you can ask the Connect HelpDesk to give you access again trough this page and selecting "Program Help".

--
Andrea Tedeschi Blog Linkedin Profile andrea (at) andreatedeschi (dot) com

FoolDNS project to protect your online privacy

Let me point out the following project, currently in beta testing: FoolDNS.

The project is led by Matteo Flora (his blog is http://www.lastknight.com/).

The project aims to provide a service that will not store IP addresses querying the DNS: IP's would be removed from the logs after 90 minutes or when the log size reaches 1MB.

FoolDNS also will block a number of DNS names (shared at this link http://fooldns.com/rules.txt) corresponding to the advertising domains addresses (including, for example, Google AdSense), in this way the user should be able to browse free from banners, profiling and tracking. More details are available on the project site: http://fooldns.com/

--
Andrea Tedeschi Blog Linkedin Profile andrea (at) andreatedeschi (dot) com

Hyper-V Server 2008 Website

Microsoft has released the Hyper-V Server 2008 Website at the following link:
http://www.microsoft.com/servers/hyper-v-server

Microsoft Hyper-V Server, Microsoft Application Virtualization 4.5 and System Center Virtual Machine Manager 2008 will be available within 30 days:
http://www.microsoft.com/Presspass/press/2008/sep08/09-07GetVirtualNowPR.mspx

--
Andrea Tedeschi Blog Linkedin Profile andrea (at) andreatedeschi (dot) com

Microsoft Exchange Server 2007 Protocol Documentation

From Microsoft Download Site:

Microsoft Exchange Server 2007 Protocol Documentation

The Microsoft Exchange Server 2007 Protocol technical documentation set provides detailed technical specifications for Microsoft protocols and extensions to industry-standard or other published protocols that are implemented and used in Microsoft Exchange Server 2007 to interoperate or communicate natively with Microsoft Office client and other server products. The documentation set includes a set of companion overview and reference documents that supplement the technical specifications with conceptual background, overviews of inter-protocol relationships and interactions, and technical reference information.

http://www.microsoft.com/downloads/details.aspx?FamilyID=a9aae935-ad23-4117-a3d9-9ed6093d6d7f&DisplayLang=en

--
Andrea Tedeschi Blog Linkedin Profile andrea (at) andreatedeschi (dot) com