Marc Lognoul's IT Infrastructure Blog

Cloudy with a Chance of On-Prem

Category Archives: Troubleshooting


by 2 Comments

SharePoint: SNI vs. Windows WebDAV Client

Introduction

SNI stands for Server Name Indication, an improvement to the SSL/TLS protocol recently added to Windows (from Windows Server 2012/IIS 8). Its purpose is to allow using multiple SSL certificates on the same web server’s IP address and port. In a certain sense, you could say it is to HTTPS what host header is to HTTP. Similarly to host header, this feature must be implemented in both client and server sides because it relies in additional information’s passed as part of the SSL handshake process initiated by the client. As consequence, older browsers (as well as older client applications in general) are not compatible.

The Problem

Now SharePoint comes into the picture. One key client functionality is the Explorer View exposed by Windows WebDAV Client. Although Microsoft continuously updates its browser, the WebDAV client did not recently receive any update to support SNI. Therefore, if you configure SharePoint together with IIS to use HTTPS using SNI, Windows Explorer browsing SharePoint will simply stop functioning displaying an error such as “A device attached to the system is not functioning”. The problem will sadly occur with Windows 8 as well but is fixed from Windows 8.1.

Workarounds

There is currently no real solution and very few workarounds:

  • On IIS: Use unique combination of web application, certificate and IP address and/or port. Every time a new web application is created on SharePoint, you will have to reconfigure it on each server in the IIS configuration in order to use another IP address or another port.
  • On Windows/HTTP.sys driver: Use a fallback certificate. This blog post details the procedure to do so: How to support non-SNI capable Clients with Web Application Proxy and AD FS 2012 R2
  • On a hardware load balancer: identically to what can be done on IIS, a unique virtual IP address for each web application together with its own certificate can be used. On the SharePoint side, you can whether use no certificate at all or a used self-signed one.

More Information


by Leave a comment

SharePoint: Old Workflow Issues Strike Back

An incident on an old SharePoint 2007 reminded me of 2 workflow-related issues affecting SharePoint when the column “Assigned To” is configured to allow multiple selections.

While one of them is fixed from SharePoint 2010, the other persists until SharePoint 2013.

Here are the matching KB Articles:

Thanks to @jlebutte and MS Support for their help!


by

UAG2010: Configuration Lost after Installing Service Pack 4

ForeFront UAG

Introduction

In this post I will modestly relay a problem experienced by a customer that was hopefully resolved by MS product Support.

After Installing the Service Pack 4 of UAG 2010, the complete configuration was missing. It was impossible to restore a backup taken prior installing the Service Pack because the schema of the database storing the configuration evolved. We were in a rather uncomfortable position…

Solution

Hopefully, the solution to this problem is rather simple:

  1. From the Control Panel, uninstall the Service Pack 4 and reboot once prompted
  2. After rebooting, open the UAG console and verify the current build version of the product. Due to the prerequisites for SP4 installation, it should be 4.0.3206.10100, which means Service Pack 3 + Update Rollup 1. You can refer to this page if you need MS product build versions, it covers UAG2010 as well
  3. “Normally”, the configuration should be back as well. If not, restore it from a previous backup. Since build versions are identical, it should work fine
  4. Activate the configuration then reboot the server again. The situation should be stable from this point. For safety, take a new backup before going any further
  5. From now you can attempt to install the Service Pack 4 again
  6. After successful installation, activate the configuration again and take a backup

Proactive Action

In order to prevent this problem from occurring on other Service Pack 4 deployments, proceed as follows:

  1. Make sure Service Pack 3 and Update Roll-up 1 are installed
  2. Activate the configuration with the current build level
  3. Reboot the server
  4. Take a backup
  5. Deploy Service Pack 4


by

SharePoint 2010: Event ID 5586 and 6398: Could not find stored procedure 'dbo.Search_GetRecentStats' Revisited

SharePoint 2010

Introduction

Recently I have been involved in the troubleshooting of the issue stated in the title. While it’s not a very recent one, I was somewhat disappointed by the misinformation returned by googeling for some kind of resolution.

Description of the Issue

Those two related errors come up in the following situation: You configure usage and health data collection (as well as its matching database) and afterwards, you create a Search Service Application. This should create and enable a timer job responsible for updating the search-related health information to the database created upfront.

Unfortunately, this does not always run as smoothly as it should leading to the errors hereunder popping-up every minute or so in the Application log:

  • Event ID: 6398. The Execute method of job definition Microsoft.Office.Server.Search.Monitoring.HealthStatUpdateJobDefinition (ID 9cb6be54-0384-4c6e-abfc-c2f25621a3ed) threw an exception. More information is included below. Could not find stored procedure ‘dbo.Search_GetRecentStats’.
  • Event ID: 5586. Unknown SQL Exception 2812 occurred. Additional error information from SQL Server is included below. Could not find stored procedure ‘dbo.Search_GetRecentStats’.

Possible Cause: The Timer Job is Not Enabled

To check the status of the timer job, proceed as follows:

    1. Open the Central Administration Web Site
    2. Click on the Menu Monitoring
    3. Under Timer Jobs click on Review Job Definitions
    4. Locate the job named Search Health Monitoring – Trace Events then click on it. The screen capture below depicts a timer job disabled that therefore never ran.

Screen Capture

  1. Click on Enable
  2. Return to the definition of the same time job then click on Run Now
  3. Return to the definition of the same timer job and refresh the page until the value of Last run time increments (about every minute)
  4. Wait for a few minutes
  5. Open the Event Viewer and go to the log Application
  6. Verify that the Event ID 5586 and 6398 do not appear anymore since the execution of the timer job. If this is not the case, jump to the next section.

Automation freaks might want to use the PowerShell interpretation: To get the status of the timer job:
Get-SPTimerJob “Search Health Monitoring – Trace Events”|Select Name,Enabled,LastRunTime

If the command above return False for the property Enabled, execute the following:
Enable-SPTimerJob “Search Health Monitoring – Trace Events”
Start-SPTimerJob “Search Health Monitoring – Trace Events”

Possible Cause: The Stored Procedure does not exist or is not accessible

To check the presence of the Stored Procedure, proceed as follows:

  1. Open a SQL Server Management Studio
  2. Connect to the SQL Server/Instance hosting the SharePoint database and locate the Usage database. Expand it and under Programmability, look after the Stored Procedure named Search_GetRecentStats.
  3. If it’s missing, go to the Central Administration, go to Manage Service Application, Delete the Usage Service Application (including its data) and create a new one. This will force the creation of the missing stored procedur(s) and set permissions appropriately. Note: Make sure the user you’re creating the Service Application with has sufficient permissions on the SQL as well.

Additional Information


by

Networking: Microsoft has Released Message Analyzer

Message Analyzer Logo

Yesterday, Microsoft has released the successor to Network Monitor: Message Analyzer.

Beyond the name change, Message Analyzer comes with a brand new way of capturing and analyzing network traffic: Instead of capturing at a very low level and filtering the flows to identify useful one, it allows to capture closer to the protocols or to the OSI-layer you are interested in. As the screenshot show hereunder: there are plenty of pre-configured layer or protocols (HTTP, Windows Firewall, File & Print Sharing, network adapter…). This greatly simplifies analysis and reduces the impact on system resources as well.

Message Analyzer Screen Capture

The capture’s details are also much easier to read, as depicts the screenshot hereunder.

Message Analyzer Screen Capture

Finally, the footprints is also reduced and the whole application is less intrusive since it does not requires to install a filtering driver. Instead, it leverages the Event Tracing for Windows (ETW) infrastructure. Unfortunately, this also means that the minimal OS requirement is Windows 7/Windows Server 2008.

More Information’s


by

SharePoint: Web Parts vs. MS13-052 with a Fix!

SharePoint 2013

Description of the Problem

After installing the security update KB2844286 for the .Net Framework 3.5.1 on Windows 7 SP1/Windows Server 2008 R2 SP1, some web parts will cease functioning and display the error message hereunder:

Error while executing web part: System.NullReferenceException: Object reference not set to an instance of an object.

If you enable debug mode, then the output will be:

Error while executing web part: System.NullReferenceException: Object reference not set to an instance of an object.
at System.Xml.Xsl.XslCompiledTransform.Load(MethodInfo executeMethod, Byte[] queryData, Type[] earlyBoundTypes)
at Microsoft.Xslt.STransform.GetCompiledTransform()
at Microsoft.SharePoint.WebPartPages.BaseXsltListWebPart.LoadXslCompiledTransform(WSSXmlUrlResolver someXmlResolver)
at Microsoft.SharePoint.WebPartPages.DataFormWebPart.GetXslCompiledTransform()
at Microsoft.SharePoint.WebPartPages.DataFormWebPart.PrepareAndPerformTransform(Boolean bDeferExecuteTransform)

Solutions

  • Install the Update from this KB2872441 on the SharePoint server(s). Note: This will require a restart

or

  • Uninstall the KB2844286 from the SharePoint(s)

More Information

Happy patching! (?)

Marc


by

SharePoint 2013: AppFabric Caching service crashed. Lease with external store expired

SharePoint 2013

Description of the Problem

You experience poor performances when browsing SharePoint 2013-based sites or when consuming User Profile Service? Take a look at the SharePoint server’s event viewer, they might be full of the error hereunder:

AppFabric Caching service crashed.{Lease with external store expired:
Microsoft.Fabric.Federation.ExternalRingStateStoreException: Lease already expired
at Microsoft.Fabric.Data.ExternalStoreAuthority.UpdateNode(NodeInfo nodeInfo, TimeSpan timeout)
at Microsoft.Fabric.Federation.SiteNode.PerformExternalRingStateStoreOperations(Boolean& canFormRing, Boolean isInsert, Boolean isJoining)}

Similarly to many distributed services and applications, AppFabric (alone or packaged inside SharePoint 2013) heavily depends on perfect time synchronization between servers, therefore, a discrepancy of seconds to a minute may lead to AppFabric service repeated crashes.

Possible Causes

To name but a few possible causes of time discrepancies between Windows hosts:

  • Incorrect Windows Time Configuration on domain Controllers and/or Member Servers
  • Network Connectivity issues between member servers and their authenticating domain controllers
  • External mechanisms interfering with Windows Time such as VMWare, Hyper-V, OS deployment solutions…

Solution (or at least, some guidance)

  1. Fix the external causes preventing correct time sync (Network issues, 3rd party software…). Be particularly careful with virtual machines
  2. Make sure the Domain Controller holding the PDC Emulator role is configured to acquired its time from an authoritative source AND members server are configured to use the Windows domain hierarchy (NOT an authoritative time source). This configuration suits 99% of the implementations.
  3. Once 1 and 2 are fixed, run the command w32tm /resync /rediscover or restart the Windows service Windows Time (w32time). In both case this will force an time sync
  4. Make sure time is also valid on the SQL Server used for SharePoint because some sored procedure requires time accuracy to work properly as welll

Other SharePoint-Related Service Impacted by Windows Time

  • All timer jobs but in particular the one responsible for refreshing configuration. improper time sync may lead to stale timer job cache
  • Same applies to timer jobs responsible for (un)deploying solutions (WSP) in a multi-server farm. Out-of-sync servers may prevent proper WSP handling
  • Incorrect time may also prevent SharePoint Alerts from being sent
  • Customer timer jobs might not be started at the correct time
  • And finally, if you use Kerberos, the “clock skew” issue is one of the most common, preventing pre-authentication

Goodies

Here’s a PowerShell snippet to query every SharePoint server in a farm in order to retrieve their local time and time zone.

$SPServerInFarm = Get-SPServer | Where { $_.Role -eq "Application" }

Foreach ($Server in $SPServerInFarm)

{

    $DateTime = (Get-WmiObject -ComputerName $Server.Name -Query "select LocalDateTime from Win32_OperatingSystem").LocalDateTime

    $DateTimeFormatted = ([wmi]”).ConvertToDateTime($DateTime).tostring("MM/dd/yyyy HH:mm:ss")

    $TimeZoneOffset = (Get-WmiObject -ComputerName $Server.Name -Query "select CurrentTimeZone from Win32_OperatingSystem").CurrentTimeZone.ToString()

   Write-Host $Server.Name $DateTimeFormatted $TimeZoneOffset

}

Note: since servers are note queried exactly at the same time, there might by a small time difference. This is obviously harmless in this case.

Note2: While a difference in the time zone will not influence AppFabric, it is not recommended to operate a SharePoint farm on servers operating across different time zones.

Additional Information’s

Conclusions

SharePoint requires a healthy underlying Windows to run smoothly. Keep your Windows Server and AD in good shape and if you’re not in charge of them, make sure you colleague in charge of them does the job right.

Happy caching!

Marc


by

SharePoint: Resetting the SkyDrive Pro Configuration

SkyDrive Pro 2013

While recently released updates are making SkyDrive Pro Client for Windows more and more resilient, there are situations where the sync status will sadly report the generic error An error occurred while attempting to synchronize this tool.

Here is how to reset the configuration (and therefore lose it entirely):

  1. Quit/Stop/Kill all Applications related to SkyDrive Pro (aka Groove aka SharePoint WorkSpace): groove.exe, Office Applications, Windows Explorer and Internet Explorer
  2. Delete the content of the folders C:Users%username%AppDataLocalMicrosoftOfficeSpw and C:Users%username%AppDataLocalMicrosoftOffice15.0OfficeFileCache
  3. Start again the SkyDrive Pro Client () and other Applications previsouly closed at step 1
  4. Re-enter the SkyDrive Pro Configuration manually…

Note: I’ve notices 2 things breaking the SkyDrive Pro sync: changing/renaming the final download folder (usually C:Users%username%SharePoint) and attempting to sync with site whose name and URL contain special characters.

You might want to have a look at the updates already made available and specific to the SkyDrive Pro Client for Windows: http://www.marc-lognoul.me/itblog-en/page/SharePointSkyDrive-Pro-Client-Updates-and-Related.aspx.

Note 2: For those still confused with SkyDrive and SkyDrive Pro, this applied exclusively to the Pro

Marc


by

Synology: Resource Monitor not Loading

Synology

Introduction

On one of my Synology NAS (I call them NAS but they would deserve a more flattering name), the Resource Monitor widget refused to load and remained stuck on Waiting like depicted on the screen capture below:

While I thought upgrading to DSM 4.2 would solve the problem it did unfortunately not, therefore I started playing around and found the culprit: SNMP. I actually found two possible causes:

The built-in SNMP Module

  1. Log on as admin on the web interface
  2. Go to the Control Panel
  3. Under Network Services, click on SNMP
  4. Uncheck Enable SNMP Service
  5. Reboot (yes reboot IS needed)
  6. Go back to the Control Panel and check Enable SNMP Service again

The NET-SNMP Daemon

Months ago I had installed the NET-SNMP using IPKG in order to get Cacti working. It appears that it may also prevent the Resource Monitor from load. Here is how to remove it:

  1. Open an SSH or Telnet session to the Synology as root
  2. Verify net-snmp is installed by executing
    ipkg list_installed
  3. If present, execute the command hereunder to uninstall it
    ipkg remove net-snmp
  4. Reboot the box (yes reboot IS needed too)

Happy Synologying!

Marc