Exchange 2013 CU8 Application Event Warning 1310 – Microsoft.Exchange.Security Assembly

I ran into a new issue that isn’t well documented publicly earlier this week.  I installed two new Exchange 2013 CU8 servers as “Hybrid” servers for an Office 365 integration deployment.  After configuring the servers, I noticed that none of the web based services on one of the servers was functioning properly and the following Application Event Log entry was reported for all web services (EWS, ActiveSync, MAPI, RPC, OWA, etc…).

Log Name:      Application
Source:        ASP.NET 4.0.30319.0
Date:          6/10/2015 11:04:59 AM
Event ID:      1310
Task Category: Web Event
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      Server1.company.com
Description:
Event code: 3008 Event message: A configuration error has occurred. Event time: 6/10/2015 11:04:59 AM Event time (UTC): 6/10/2015 4:04:59 PM Event ID: 9aca57c1b125421aa3d7faf91155a099 Event sequence: 1 Event occurrence: 1 Event detail code: 0  Application information:     Application domain: /LM/W3SVC/2/ROOT/EWS-290-130784258993945463     Trust level: Full     Application Virtual Path: /EWS     Application Path: C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\exchweb\EWS\     Machine name: Server1  Process information:     Process ID: 12184     Process name: w3wp.exe     Account name: NT AUTHORITY\SYSTEM  Exception information:     Exception type: ConfigurationErrorsException     Exception message: Could not load file or assembly ‘Microsoft.Exchange.Security, Version=15.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35′ or one of its dependencies. The system cannot find the file specified. (C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\exchweb\EWS\web.config line 94)
at System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective)
at System.Web.Configuration.AssemblyInfo.get_AssemblyInternal()
at System.Web.Compilation.BuildManager.GetReferencedAssemblies(CompilationSection compConfig)
at System.Web.Compilation.BuildManager.CallPreStartInitMethods(String preStartInitListPath, Boolean& isRefAssemblyLoaded)
at System.Web.Compilation.BuildManager.ExecutePreAppStart()
at System.Web.Hosting.HostingEnvironment.Initialize(ApplicationManager appManager, IApplicationHost appHost, IConfigMapPathFactory configMapPathFactory, HostingEnvironmentParameters hostingParameters, PolicyLevel policyLevel, Exception appDomainCreationException)

Could not load file or assembly ‘Microsoft.Exchange.Security, Version=15.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35′ or one of its dependencies. The system cannot find the file specified.
at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean forIntrospection)
at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
at System.Reflection.Assembly.Load(String assemblyString)
at System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective)

  Request information:     Request URL: https://localhost:444/ews/exchange.asmx     Request path: /ews/exchange.asmx     User host address: 127.0.0.1     User:      Is authenticated: False     Authentication Type:      Thread account name: NT AUTHORITY\SYSTEM  Thread information:     Thread ID: 26     Thread account name: NT AUTHORITY\SYSTEM     Is impersonating: False     Stack trace:    at System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective)
at System.Web.Configuration.AssemblyInfo.get_AssemblyInternal()
at System.Web.Compilation.BuildManager.GetReferencedAssemblies(CompilationSection compConfig)
at System.Web.Compilation.BuildManager.CallPreStartInitMethods(String preStartInitListPath, Boolean& isRefAssemblyLoaded)
at System.Web.Compilation.BuildManager.ExecutePreAppStart()
at System.Web.Hosting.HostingEnvironment.Initialize(ApplicationManager appManager, IApplicationHost appHost, IConfigMapPathFactory configMapPathFactory, HostingEnvironmentParameters hostingParameters, PolicyLevel policyLevel, Exception appDomainCreationException)
Custom event details:
Event Xml:
<Event xmlns=”http://schemas.microsoft.com/win/2004/08/events/event”&gt;
<System>
<Provider Name=”ASP.NET 4.0.30319.0″ />
<EventID Qualifiers=”32768″>1310</EventID>
<Level>3</Level>
<Task>3</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime=”2015-06-10T16:04:59.000000000Z” />
<EventRecordID>548169</EventRecordID>
<Channel>Application</Channel>
<Computer>Server1.company.com</Computer>
<Security />
</System>
<EventData>
<Data>3008</Data>
<Data>A configuration error has occurred.</Data>
<Data>6/10/2015 11:04:59 AM</Data>
<Data>6/10/2015 4:04:59 PM</Data>
<Data>9aca57c1b125421aa3d7faf91155a099</Data>
<Data>1</Data>
<Data>1</Data>
<Data>0</Data>
<Data>/LM/W3SVC/2/ROOT/EWS-290-130784258993945463</Data>
<Data>Full</Data>
<Data>/EWS</Data>
<Data>C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\exchweb\EWS\</Data>
<Data>Server1</Data>
<Data>
</Data>
<Data>12184</Data>
<Data>w3wp.exe</Data>
<Data>NT AUTHORITY\SYSTEM</Data>
<Data>ConfigurationErrorsException</Data>
<Data>Could not load file or assembly ‘Microsoft.Exchange.Security, Version=15.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35′ or one of its dependencies. The system cannot find the file specified. (C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\exchweb\EWS\web.config line 94)
at System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective)
at System.Web.Configuration.AssemblyInfo.get_AssemblyInternal()
at System.Web.Compilation.BuildManager.GetReferencedAssemblies(CompilationSection compConfig)
at System.Web.Compilation.BuildManager.CallPreStartInitMethods(String preStartInitListPath, Boolean&amp; isRefAssemblyLoaded)
at System.Web.Compilation.BuildManager.ExecutePreAppStart()
at System.Web.Hosting.HostingEnvironment.Initialize(ApplicationManager appManager, IApplicationHost appHost, IConfigMapPathFactory configMapPathFactory, HostingEnvironmentParameters hostingParameters, PolicyLevel policyLevel, Exception appDomainCreationException)

Could not load file or assembly ‘Microsoft.Exchange.Security, Version=15.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35′ or one of its dependencies. The system cannot find the file specified.
at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark&amp; stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark&amp; stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark&amp; stackMark, IntPtr pPrivHostBinder, Boolean forIntrospection)
at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark&amp; stackMark, Boolean forIntrospection)
at System.Reflection.Assembly.Load(String assemblyString)
at System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective)

</Data>
<Data>https://localhost:444/ews/exchange.asmx</Data&gt;
<Data>/ews/exchange.asmx</Data>
<Data>127.0.0.1</Data>
<Data>
</Data>
<Data>False</Data>
<Data>
</Data>
<Data>NT AUTHORITY\SYSTEM</Data>
<Data>26</Data>
<Data>NT AUTHORITY\SYSTEM</Data>
<Data>False</Data>
<Data>   at System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective)
at System.Web.Configuration.AssemblyInfo.get_AssemblyInternal()
at System.Web.Compilation.BuildManager.GetReferencedAssemblies(CompilationSection compConfig)
at System.Web.Compilation.BuildManager.CallPreStartInitMethods(String preStartInitListPath, Boolean&amp; isRefAssemblyLoaded)
at System.Web.Compilation.BuildManager.ExecutePreAppStart()
at System.Web.Hosting.HostingEnvironment.Initialize(ApplicationManager appManager, IApplicationHost appHost, IConfigMapPathFactory configMapPathFactory, HostingEnvironmentParameters hostingParameters, PolicyLevel policyLevel, Exception appDomainCreationException)
</Data>
</EventData>
</Event>

Based upon this event as well as the error shown in the browser when attempting to use one of these services.

EWS Error shown in browser representing Event ID 1310

EWS Error shown in browser representing Event ID 1310

After checking the web.config files on each web service directory and not finding any issue, we opened a PSS case and found an undocumented issue with installation of the server.  For an unknown reason, the SharedWebConfig.Config file was missing on the server altogether. It should be located at C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess, but in this case the file was not there.  This file contains a listing of all DDL Assembly locations including the Microsoft.Exchange.Security DLL that was in the error.

Example of SharedWebConfig.config file contents

Example of SharedWebConfig.config file contents

SharedWebConfig.config File location

SharedWebConfig.config File location

The solution was to simply copy this config file from a working server with the same install path and restart IIS.  Hopefully, this will assist anyone else facing this issue.

Posted in Uncategorized | Leave a comment

Microsoft Releases the Hybrid Migration Troubleshooter

Microsoft has introduced the all new Hybrid Migration Troubleshooter, which was first introduced during Microsoft Ignite in Chicago.

 Why did they create this troubleshooter?

The Hybrid style of Mailbox move is one of the most popular and complex types of migration strategies.  This option gives you a way to move some of their mailboxes to Exchange Online, while keeping some on-premises. It is also the only option that allows for your mailboxes to be off boarded using the native toolset.  It is because of that complexity that Microsoft have seen many customers confused when trying to determine why there migration is failing, appears slow, or stalled.

The solution to this problem is for you to run the Hybrid Migration Troubleshooter from Http://aka.ms/HMTSEmail

Thank you goes to the following folks that made this possible:

Jeremy Kelly, Brad Hughes, Brad Craig, Scott Roberts, Padmini Palanki, and Timothy Heeney

Posted in Uncategorized | Leave a comment

Microsoft Ignite session catalog has been posted!

This morning, Microsoft announced the availability of the session catalog for Ignite, Microsoft’s new, premier enterprise technology event. This provides the first in-depth look at the hundreds of sessions that will be available across Office 365, SharePoint, Exchange, Lync and also watch as GM Julia White breaks down why you should be at Ignite this May.

On February 3rd at 9:00 a.m. PT, they’ll have the whole event team and lots of speakers ready to chat with you on Twitter. They’ll be ready to answer your questions about the event and hear what you’re excited about in terms of community experiences and things to do in Chicago. Add the event to your calendar with this link.

Posted in Uncategorized | Leave a comment

Installing .NET 3.5.1 on Windows 2012 – Error 0x800F081F

Recently, when trying to install Office 365 Dirsync, I ran into an issue with the installation of .NET 3.5.1 as a feature.  I received the error:

Error: 0x800F081F

The Source files could not be found

Since this was a newly installed server, it was baffling as to what the issue was.  After some troubleshooting, I determined there were two causes:

  1. The windows Component Store was damaged
  2. A hotfix KB2966827 had already been installed that causes this issue.

To resolve these issues, I did the following:

  1. I ran DISM /Online /Cleanup-Image /RestoreHealth to repair the Component Store. (http://blogs.technet.com/b/joscon/archive/2012/11/30/how-to-troubleshoot-error-0x800f081f-when-installing-roles-and-features.aspx)
  2. I determined if KB2966826, KB2966827, or KB2966828 were installed, and uninstalled them until after .NET 3.5.1 was installed. (http://www.askvg.com/fix-0x800f0906-and-0x800f081f-error-messages-while-installing-net-framework-3-5-in-windows-8/) After installing .Net 3.5.1, you can safely re-install the hotfixes.
Posted in Uncategorized | Leave a comment

Exchange 2013 – Outlook Online Mode – Search Folders Break!

I ran into a new issue that I wasn’t aware of until now.  When Exchange 2013 was RTM’ed, it was known that OWA no longer supported Search Folders.  Those are the folders you can create in Outlook that show a specific filtered view of your mailbox.  Those can contain items like UnRead Items, Missed Conversations, Call Logs, Flagged Email, Categorized Email, etc…  I personally don’t use Search Folders, but I know that many people do.  I also know a few organizations that have standardized their desktop infrastructure using one brand or another of VDI.  Within their VDI solution, they have chosen to deploy Outlook 2010 or 2013 in Online mode and their internal Exchange 2013 servers have been scaled to support Online Mode.  

This lays the groundwork for the problem.  Outlook cached mode, along with many other features, is this mode that maintains the finder info on the local PC. Since the finder information is no longer synchronized to the Exchange 2013 server store, the server doesn’t store finder information any longer (which indecently is one of the reasons why Search Folders are no longer offered in OWA).  In Online Mode, the users still can try to use Search Folders, and they appear to work until they try to interact with the messages shown in the Search Folder.  Actions like Delete will produce an error message that tells the user (depending on Outlook version) that either the message can’t be found, or better yet, “Unknown Error”.  

The solution is to enable Cache Mode, full stop.  I have not yet confirmed this with the Exchange Product Team but it is my belief that since Office 365 utilizes Cache Mode exclusively and since cache mode is now supported in VDI deployments, that this is by design.  

Posted in Cache Mode, Exchange 2013, Online Mode, Outlook, Search Folders | Tagged , , , , , | Leave a comment

Exchange 2013 – Outlook Online Mode – Search Folders

I ran into a new issue that I wasn’t aware of until now.  When Exchange 2013 was RTM’ed, it was known that OWA no longer supported Search Folders.  Those are the folders you can create in Outlook that show a specific filtered view of your mailbox.  Those can contain items like UnRead Items, Missed Conversations, Call Logs, Flagged Email, Categorized Email, etc…  I personally don’t use Search Folders, but I know that many people do.  I also know a few organizations that have standardized their desktop infrastructure using one brand or another of VDI.  Within their VDI solution, they have chosen to deploy Outlook 2010 or 2013 in Online mode and their internal Exchange 2013 servers have been scaled to support Online Mode.  

This lays the groundwork for the problem.  Outlook cached mode, along with many other features, is this mode that maintains the finder info on the local PC. Since the finder information is no longer synchronized to the Exchange 2013 server store, the server doesn’t store finder information any longer (which indecently is one of the reasons why Search Folders are no longer offered in OWA).  In Online Mode, the users still can try to use Search Folders, and they appear to work until they try to interact with the messages shown in the Search Folder.  Actions like Delete will produce an error message that tells the user (depending on Outlook version) that either the message can’t be found, or better yet, “Unknown Error”.  

The solution is to enable Cache Mode, full stop.  I have not yet confirmed this with the Exchange Product Team but it is my belief that since Office 365 utilizes Cache Mode exclusively and since cache mode is now supported in VDI deployments, that this is by design.  

Posted in Cache Mode, Exchange 2013, Online Mode, Outlook, Search Folders | Tagged , , , , , | Leave a comment

Exchange Server 2010 SP1 RTM!!

So, I have been working with the Exchange TAP and Product Group for months now waiting to make this announcement.  Finally, after months of testing and Microsoft making many improvements, Service Pack 1 for Exchange Server 2010 is officially here and publically available! 

You can see the RTM blog post here.

You can get more information what is in SP1 on my blog as well as TechNet.

The official SP1 bits can be downloaded here.

All I can say is go forth and download as this is the icing on the Exchange 2010 cake…

Posted in Exchange 2010 | Leave a comment