Marc Lognoul's IT Infrastructure Blog

Cloudy with a Chance of On-Prem


SharePoint 2013: Things to Take into Account when Determining the Version of SQL Server to be Used

SharePoint 2013
Still hesitating between SQL Server 2008 R2 SP1 and 2012 for your coming SharePoint 2013 deployment? The rule-of-thumb is simple: If you need one of the following feature, go for SQL 2012 , otherwise, you can still delay the leap:

In a nutshell, if the SharePoint 2013 you’re planning is BI-oriented, you’ll definitely have to go for SQL Server 2012 with SP1.

Obviously, even if the SharePoint infrastructure is debuting with SQL 2008 R2 SP1, nothing prevents its upgrade to 2012 later in its life-cycle.

More Information


Windows: The Confusion over DisableLoopBackCheck, DisableStrictNameChecking and Kerberos

Windows Logo

Introduction

I’ve been answering technical questions in Forums and at customers for a while now and in the recent years there were many related to issue related to DisableLoopBackCheck and DisableStrictNameChecking security features from Windows. I also regularly noticed a lot of confusion and misinformation about these. This post is a modest attempt to explain them more in-depth.

DisableLoopBackCheck

LoopBackCheck is, like its name says, a security feature applicable to connection established to the loopback address (127.0.0.1). It applies to NTLM authentication only. It allows protecting a Windows computer against threat exploiting connections to the loop back adapter specifically. This extra protection level applies to all incoming connections and protocols

What is often misleading with LoopBackCheck is the error message making believe invalid credentials have been provided while it is actually not the case. Example with IIS: HTTP 401.1 – Unauthorized: Logon Failed.

Why isn’t it applicable to Kerberos authentication? Simply because Kerberos authentication is so strongly linked to names (host and service names) that is does actually need this security feature.

When will you experience this issue? Usually, in a test/dev environment when you redirect services such as IIS web sites or SharePoint to the loopback address. It might also affect production SharePoint when the crawler process is configured to crawl from the locally-deployed WFE role thanks to a modification of the HOSTS file. You may also experience this problem when the server’s are part of an NLB cluster or when an IIS-based application accesses a SQL Server instance located on the same server using the loopback address together with Windows Authentication.

This feature was originally implemented with Windows Server 2003 Service Pack 1 and is therefore present in all recent versions of Windows

Solutions/Workarounds

Note: MS KB Article states you have to disable Strict Name Checking as well; this is not true or at least not true if you don’t plan to use file & print sharing over the loopback adapter (see below)

Note2: My field experience tells not to use the loopback adapter anymore for SharePoint crawler because it may also generate other issue related on security software (anti-virus, local firewall…) adding their load of security checks to the communication channel to the loopback adapter.

DisableStrictNameChecking

Strict Name checking is a security feature specific to the Windows implementation of the SMB protocol (File & Print sharing). It will prevent connections from being stabled to a network share or to a shared printer if the host name used is not the server’s real one. The error message might also be considered as misleading: System error 52 has occurred. A duplicate name exists on the network.

The feature has been originally brought by Windows 2000 and is implemented is all subsequent versions of Windows.

Solutions/Workarounds

Kerberos and how it is related to Names

Like I stated upper in this post, Kerberos authentication protocol integrates name checking in its foundation since the secret exchanged between the client and the service are partially based on the name the service is accessed by. If the name of the service is missing or incorrectly configured in the Kerberos database (Active Directory in the Windows world), the authentication will fail with the internal error KRB5KDC_ERR_S_PRINCIPAL_UNKNOWN and is likely to fall back in NTLM authentication, which will ultimately lead to a successful authentication with a less secure protocol

Therefore, if one or multiple alternate names are used to access a service, the appropriate configuration must be associated with the user account (or computer account) running the service, using, for example SETPSPN or NETDOM like in the example above. Note: While some software offer the possibility to auto-magically register Service Principal Names in AD, very few do that actually)

Marc


Windows: An existing connection was forcibly closed by the remote host

Two different projects complaining about the same issue: nice troubleshooting challenge! One is SharePoint-based and the second is a, let’s say “entertaining” .Net-based application. They both make use of SQL Server as back-end data store and both complain about having “existing connection forcibly closed” reported in they stack trace when then attempt to connect to SQL.

This happens when a client application is trying to re-sued an existing TCP connection to a remote host while it closes it, making connection reuse impossible. There are actually multiple possible root causes which do no seem to be mutually exclusive:

Limit set on the number of connection allowed by SQL Server on a given instance

For a given SQL instance, you can set the maximum number of connections that can be used by applications. Depending on the way your application is written, multiple connection might be used for a single transaction… Raise the limit or set it to unlimited as necessary.

The (infamous) Scalable Networking Pack

The Scalable Networking Pack is a set of improvements brought to the Windows Networking stack. It is available as an add-on for Windows Server 2003 but is included from Service Pack 2 as well as from Windows Vista/2003.

This update greatly modifies the way Windows handles network connectivity at TCP-level and might therefore provoke the error. In short, the following settings should be modified on the SQL Server (or on any server acting as the server component):

In the registry, under HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters

  • EnableTCPChimney (REG_DWORD) set to 0 (disabled)
  • EnableRSS (REG_DWORD) 0 (disabled)
  • EnableTCPA (REG_DWORD) 0 (disabled))

Applying the change requires a reboot.[UPDATE] Some MS sources report that a reboot is not necessary for some settings so I switch my statement to *might* require a reboot.

You’ll find a lot of trustworthy online resources recommending to disable the SMP…

On the other hand, recent NIC drivers may allow your system to work properly with these options set to enabled… Look at this page to get a list of SMP “partners”: http://technet.microsoft.com/en-us/network/cc984184.aspx.

Faulty NIC, NIC driver or driver settings 

Some NIC include a TCP Offload Engine (TOE). Incorrectly configured or running an out-dated, they will generate error at TCP-level.

In some cases, the TOE simply does not work, so you also might want to test with this function completely disabled. When editing you driver’s parameters, look for “Large Send Offload”, “Checksum offload”…

Import to note, you might also want to check the link speed and duplex at NIC level AND at switch port level, they might also cause the problem. Remember,: they must be identical on BOTH sides

Applying the change *might* requires a reboot.

Windows TCP/IP Stack Custom Configuration or Hardening

There are plenty of resources describing how to “harden” the Windows TCP/IP stack. Unfortunately most of them simply show the “how to”, not its consequences. Of of them being the performance decrease implied by hardening. You’ll also find those parameters under HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParameters

In our case, the parameter “SynAttackProtect” set to 1 instead of 0 (disabled) will force Windows to be more restrictive regarding the incoming or TCP connection requests and well as more aggressive with the re)use of existing one. If the parameter is  enabled, the following additional parameters will also be taken into account:

  • TcpMaxPortsExhausted: Determines the maximum number of connections that can be opened before enabling protection against SYN attacks
  • TCPMaxHalfOpen: Determines the maximum number of connections that can be left “half-open” (waiting for re-use)
  • TCPMaxHalfOpenRetried: Same as above BUT applicable to connections that were effectively re-used by the original client

The parameters above are thresholds used by Windows to determine if a TCP-based (SYN) attack is in progress or not. They should only be used if the server is put in a high risk situation (DMZ or internet-facing) while there is not other security device put in place (Firewall…).

Note that, before Windows 2003 SP2, this SynAttackProtect is set to 0 while with SP2, it is set to 1 then with the latest versions of Windows, it returns to 0…

Automatic adjustment for the TCP window size (From Vista or 2008 only)

On the client side, Windows, starting from Vista, comes with a feature that allows to dynamically set the TCP windows size depending upon the network (remote host) conditions. See http://support.microsoft.com/kb/929868. But I frankly doubt it can be the root cause, I just documented it for comprehensiveness.

If your application is affected by those problem, I hope you’ll find the culprit amongst one of those.

Any network device catching the traffic at TCP-level

If there is any firewall in place, look at their logs, they might reveal that some connections are refused when the client attempts to re-use them.

More Information

Thanks to Tim B (MSFT) and Pascal B (MSFT) for the hints and guidance.

And cut!

The French Connection Poster


Leave a comment

SharePoint 2007: Search Administration, index propagation and .Net 3.5 install fail

Last week the road was rockier than ever in order to get MOSS 2007 properly deployed in a relatively strict security environment. We experienced 4 funny, though show stopping issues, that were hopefully relatively easy to solve!

.Net Framework 3.5 Installation report error message “The application has requested the Runtime to terminate it in an unusual way”

This is an ‘”old” issue caused by the fact that in a security-hardened environment, the Print spooler service might be set to disabled while the .Net Framework setup needs it in order to install the XPS print driver… Yeah, dependencies show up where you don’t really expect them ;).

Take a look at these resources which are really helpful though a little outdated:

 

MOSS Search Administration Page fails with error “Authentication failed because the remote party has closed the transport stream”

I already met this error under different circumstances: it usually means that the remote web server you’re trying to access requires SSL communications but the server certificate it uses is corrupt or invalid.

My first reaction was: but we don’t use SSL for our Central Admin or SSP so why would it require it then?

Simply because the Search Administration console uses it to communicate with Web Services hosted on each SharePoint server in the farm and in this case, with the query servers in particular.

After some investigations, the excellent tool SSL Diag reported that the server certificate was corrupted on nearly all servers in our farm… How could this happen?

Apparently, this is the consequence on combining the following elements:

  • The server that is running Office SharePoint Server 2007 has the query role in a server farm
  • The server farm was updated with Microsoft .NET Framework 3.5 Service Pack 1 (SP1)
  • The roles of the query server and the index server are not on the same server in the server farm.

And the issue as well as its solution are documented at MS Support under the KB article 962928: http://support.microsoft.com/kb/962928.

Note: although MS says it only affects the Query servers, the server certificate was corrupt on all servers except the Index… So we decided to fix it everywhere for optimal farm sanity.

 

MOSS Search Administration is unable to retrieve the status of index propagation on Query Servers

The real issue is that actually, the Index server is unable to propagate (read ‘”copy”) the index files and configuration to the query servers because the shared folder “searchindexpropagation” is missing since it could not be created at setup time.

Martin Kearn (MS MCS Consultant in UK) also reports the issue in locked-down environment (Windows Server 2008 in particular): http://blogs.msdn.com/martinkearn/archive/2008/10/29/manually-configuring-search-propagation-location-using-stsadm.aspx.

Talking about Martin Kearn, make sure you update your RSS Reader with the address of the new blog run conjointly with all UK SharePoint MCS Consultants: http://blogs.msdn.com/uksharepoint/

 

And finally, even the SQL Server cluster decided to bring its own set of issues, consequently bringing our SharePoint farm down…

Fortunately this one was easier to troubleshoot since it is almost and “old friend”: Cannot generate SSPI context.

The 3 resources hereunder will help tracking the root cause of the problem:

 

This interesting troubleshooting day reminded me Mr Spock saying “Logic, logic, logic. Logic is the beginning of wisdom, not the end”. This was is Start Trek VI: The Undiscovered Country. My favorite ST movie with the TOS crew. Christopher Plummer playing General Chang was simply great!

Star Trek VI: The Undiscovered Country Poster

And cut!


Leave a comment

SharePoint 2007: Relocating data and log files

I usually work in environment where WSS/MOSS is deployed with a remote SQL server. While reading the NG’s, my attention was caught by a person complaining about the lack of disk space on system partition after a few weeks of WSS usage. This reminded me that during the setup of WSS v3, if you chose for a “Basic” installation, the setup process will install and automatically configure Microsoft SQL Embedded Edition (MSEE) as database and set the default path for databases to %SystemRoot%SYSMSISSEEMSSQL.2005MSSQLData, which is obviously not ideal in the long run.

Since I don’t like to work the half, you’ll find hereunder method to relocate all WSS-related files that may grow with time and become therefore problematic on a system partition:

  • All SharePoint databases
  • The SQL “tempdb” (trust me, it may sometimes become really problematic, isn’t it G. N.?)
  • The IIS log files
  • The WSS Search Index files (and related)
  • The trace log files

Now let’s say you’re running WSS with the default configuration, everything on the system partition, here is how to do the relocation job:

0. Download & Install the necessary software if they are not from the page http://www.microsoft.com/downloads/details.aspx?FamilyID=df0ba5aa-b4bd-4705-aa0a-b477ba72a9cb&DisplayLang=en, download the software named “Microsoft SQL Server Native Client” and “Microsoft SQL Server 2005 Command Line Query Utility” and install them on your server. Make sure you chose the appropriate version (x86, x64…)

1. Create data folders on another partition such as
D:SQL_DATA
D:WSS_DATA
D:IIS_LOGS
D:WSS_INDEX
D:WSS_TRACE_LOGS

2. List the SharePoint databases and keep the results at hand. You can simply use the dir command such as:
dir /b “%SystemRoot%SYSMSISSEEMSSQL.2005MSSQLDataWSS_*.*”
dir /b “%SystemRoot%SYSMSISSEEMSSQL.2005MSSQLDataSharePoint_*.*”

3. Stop all WSS-related services. Using command-line, it would give something like:
net stop “Windows SharePoint Services Search”
net stop “Windows SharePoint Services Timer”
net stop “Windows SharePoint Services Tracing”
iisreset /stop

4. Detach databases and physically move the files to the new location
Open a command prompt and set its path to %PROGRAMFILES%Microsoft SQL Server80ToolsBinn
Execute the command sqlcmd -S
\.pipemssql$microsoft##sseesqlquery -E. This will start the SQL command-line utility
For each databases noted at step 2, execute the commands hereunder taking care not writing the extension (.mdf or .mdf) or the name of the log file (_log.ldf):
EXEC sp_detach_db WSS_Content
GO

This should normally look like:
EXEC sp_detach_db WSS_Content
GO
EXEC sp_detach_db WSS_Search_SRV01
GO
EXEC sp_detach_db ‘SharePoint_AdminContent_8c523776-d06c-4460-b7ec-f321aa2032ba’
GO
EXEC sp_detach_db ‘SharePoint_Config_9aae4bcb-37e4-4a88-b98b-40fc180095f0’
GO
Open another command-prompt and execute the command below to move the files:
move %SystemRoot%SYSMSISSEEMSSQL.2005MSSQLDataWSS_*.* D:WSS_DATA
move %SystemRoot%SYSMSISSEEMSSQL.2005MSSQLDataSharePoint_*.* D:WSS_DATA

Return to the SQL command-line utility and execute the instructions hereunder to re-attach database. Make sure you repeat it for each database.
EXEC sp_attach_db @dbname = N’DBNAME’, @filename1 = N’D:WSS_DataDBNAME.mdf’, @filename2 = N’D:WSS_DataDBNAME_log.LDF’
GO

This should normally look like:
EXEC sp_attach_db @dbname = N’WSS_Content’, @filename1 = N’D:WSS_DataWSS_Content.mdf’, @filename2 = N’D:WSS_DataWSS_Content_log.LDF’
GO
EXEC sp_attach_db @dbname = N’WSS_Search_SRV01′, @filename1 = N’D:WSS_DataWSS_Search_SRV01.mdf’, @filename2 = N’D:WSS_DataWSS_Search_SRV01_log.LDF’
GO
EXEC sp_attach_db @dbname = N’SharePoint_AdminContent_8c523776-d06c-4460-b7ec-f321aa2032ba’, @filename1 = N’D:WSS_DataSharePoint_AdminContent_8c523776-d06c-4460-b7ec-f321aa2032ba.mdf’, @filename2 = N’D:WSS_DataSharePoint_AdminContent_8c523776-d06c-4460-b7ec-f321aa2032ba_log.LDF’
GO
EXEC sp_attach_db @dbname = N’SharePoint_Config_9aae4bcb-37e4-4a88-b98b-40fc180095f0′, @filename1 = N’D:WSS_DataSharePoint_Config_9aae4bcb-37e4-4a88-b98b-40fc180095f0.mdf’, @filename2 = N’D:WSS_DataSharePoint_Config_9aae4bcb-37e4-4a88-b98b-40fc180095f0_log.LDF’
GO

Now relocate the tempdb database by executing the following command in the SQL command-line utility:
USE master
GO
ALTER DATABASE tempdb modify file (name = tempdev, filename = ‘D:SQL_DATAtempdb.mdf’)
GO
ALTER DATABASE tempdb modify file (name = templog, filename = ‘D:SQL_DATAtemplog.ldf’)
GO

Exit SQL command-line utility by typing “Exit”
Restart the SQL Server Instance to apply change by executing the following commands:
net stop “SQL Server 2005 Embedded Edition (MICROSOFT##SSEE)”
net start “SQL Server 2005 Embedded Edition (MICROSOFT##SSEE)”

5. Reconfigure IIS and Move Logs
Return to the command-prompt and execute the command taking care of replacing MYWSSITEID with the actual site id of your WSS site. To find it out, you can use the IIS Console and click on “Web Sites”, the is will appear on the right pane under the column name “identified” (second one)
cscript.exe /nologo c:InetpubAdminScriptsadsutil.vbs Set W3SVC/MYWSSITEID/LogFileDirectory D:IIS_LOGS
Note: if you wish to use centralized logging and therefore log all web site activity to the same folder, execute the command hereunder:
cscript.exe /nologo c:InetpubAdminScriptsadsutil.vbs Set W3SVC/MYWSSITEID/LogFileDirectory D:IIS_LOGS
Move the file from %windir%system32logfiles  to D:IIS_LOGS
Restart IIS by executing
iisreset /start
Then time to restart the WSS services by executing
net start “Windows SharePoint Services Search”
net start “Windows SharePoint Services Timer”
net start “Windows SharePoint Services Tracing”

Optionally, look into the Application event logs for possible issues

6. Relocating the WSS Search Files
Return to the command-prompt, set the path to %programfiles%Common FilesMicrosoft Sharedweb server extensions12BIN and execute the command hereunder:
stsadm -o spsearch -indexlocation D:WSS_INDEX

7. Relocating the WSS Trace Logs
For this one, stsadm does not offer any command-line-based solution so we’ll need to go back to the GUI.
Open the SharePoint Central Administration Web Site
Click on the tab “Operations”
Under the section “Logging and Reporting”, click on the link “Diagnostic logging”
Scroll down to the section “Trace Log” and on the right, enter “D:WSS_TRACE_LOGS” as path then click “OK”
Optionally, move the files under %programfiles%Common FilesMicrosoft Sharedweb server extensions12LOGS to D:WSS_TRACE_LOGS

If you wish to the the Trace Log fiels location using STSADM anyway, you may consider using Gary Lapointe’s valuable extensions for STSADM: http://stsadm.blogspot.com/2008/06/trace-log-settings.html