LiquidObject

IIS website performance tuning

website-speed

After having some free time, to upgrade the underlying hardware running this site along with a few other things. The drive upgrades in particular helped a fair amount on the processing time, however going back and remembering to configure the output caching for IIS was a bigger help. In any event the site should be significantly faster loading for everyone. As IIS output caching is not new by any means, below are some links going over the feature within IIS.

IIS.Net – Configuring IIS 7 Output Caching

IIS.Net Dynamic Content caching

Technet – Kernel-Mode caching

While this feature has been available for years, many IIS websites still haven’t taken advantage of any of the newer features.

, , ,
July 2, 2013 at 7:18 pm Comments (0)

Runaway process checking

Recently I ran into an issue with PHP exhaustion on a Windows Server running IIS. In this scenario the PHP-CGI.exe process would continue to spawn additional instances as load on the server would increase but over time the application pool would struggle and begin to slow to a crawl. In the past I have seen other applications during various iterations of development run into the same issue where if you run into more than “x” instances of an application it is unhealthy or less than “y” instances it is not running properly.

 

$myprocess = "php-cgi"
$myserver = "WebServer"
$mydomain = "liquidobject.com"
$mail_server = "mail.liquidobject.com"
$mail_recipient = "my_support_team@liquidobject.com"
$toomany = 40
$waytoomany = 80

$mail_sender = "$myserver@$mydomain"
$mailreport_subject = "Script: $myserver $myprocess count"
$body = " "

function SendEmailReport
{
    $body = [string]::join([environment]::NewLine, ($body)) 
    $msg = New-Object System.Net.Mail.MailMessage $mail_sender, $mail_recipient, $mailreport_subject, $body
    $client = New-Object System.Net.Mail.SmtpClient $mail_server
    $client.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
    $client.Send($msg)
}


$mycount =  (Get-Process -Name $myprocess).count

if($toomany -lt $mycount)
{
	$body = "We have $mycount $myprocess processes, something is unusual."
	if($waytoomany -lt $mycount)
        {
             IISRESET /STOP
             IISRESET /START
             $body = "We have $mycount $myprocess processes, IIS has been reset."
        }
        SendEmailReport
}

In this case we are sending an email notification to the fictional “support team” when more than 40 instances of the php-cgi process are running and in the event no one responds by the time 80 instances are hit the site is automatically bounced to ensure it’s availability.

The simple method for checking is the use of Task Scheduler and call up the script every 5 minutes, pretty simple yet effective.

, , , ,
May 11, 2013 at 7:30 am Comments (0)

Windows Unicast NLB Performance Tuning

Windows Server 2000 and later offer a clustering option known as Windows Network Load Balancing (or NLB for short). This technology allows for a very cost-effective clustering solution. When working with the lowest common denominator of a switched networking architecture, NLB is limited to Unicast operations only. As this technology has been out for nearly I decade, I pause to bring to being it up but in productions environments I keep running across performance impacts from using this technology. Per Microsoft:

In its default unicast mode of operation, Network Load Balancing reassigns the station address (“MAC” address) of the network adapter for which it is enabled (called the cluster adapter), and all cluster hosts are assigned the same MAC address. Incoming packets are thereby received by all cluster hosts and passed up to the Network Load Balancing driver for filtering.

Network Load Balancing’s unicast mode induces switch flooding in order to simultaneously deliver incoming network traffic to all cluster hosts.

I see of hear of small-to-medium sized organizations introducing multiple vlans on their networks for performance to control broadcast storms, which is a great starting step. Where it stops often is at the data center where there is a single vlan for all server traffic. Microsoft mentions that there is a “port-flooding” condition that may occur, but at what level? For example, lets introduce a pair of IIS NLB clusters into a single vlan with gigabit connectivity along with say less than 100 other servers with traffic in the neighborhood of 200 simultaneous connections or less. Everything still works, performance may seem a little sluggish but nothing to noticeable.

Now lets scale the traffic up by either upping the connections to 1,000+ or having a backup solution point to a DNS entry or one of the IP’s on the network card which is part of the NLB cluster. You will start to see switch ports lit up like a Christmas tree and periodic dropped packets on the given vlan.

With NLB in Unicast mode every packet received is sent as a broadcast packet to every member of the vlan. Separately this can introduce security issues for the environment.

Graphically, what does this look like? Below you will see an RRD graph with the amount of traffic being sent to a monitoring port on the network, the baseline is from 30 to 35 kbps. In this scenario there is on NLB cluster offering up IIS under Server 2003 and a second NLB clustering offering up Microsoft Exchange 2007 CAS/Frontend services. Each cluster introduces approximately 15 kbps of traffic to every node on the vlan. You will also notice that by design the Unicast NLB method introduces this problem on the receive-side only, packet transmission from the cluster does not flood the vlan.

NLB Effect

Correction of this design issue is fairly straight-forward. Each Windows NLB cluster should by design be in vlan isolation to prevent port-flooding. If vlan isolation is not an option for weeks/months for whatever the reason you might be able to reduce the scope of the flooding by adjusting the “Port Rules” option as shown below. For vlan sizing I would take into account whatever your current plans or end-game ideas for the cluster (whichever is larger), then double it, add the number of routing virtual IP’s from the networking side and add one for troubleshooting. For smaller clusters a /28 would be sufficient to meet these requirements which allows for future expansion, cluster node upgrading/replacement, and a spare IP for troubleshooting in case a problem should arise.

Microsoft reference: http://technet.microsoft.com/en-us/library/bb742455.aspx

, , , ,
February 2, 2011 at 1:42 pm Comments (0)

IIS Log File cleaner

All too often I run across web servers with log files being stored until the drive runs out of space. If you are running IIS 7 or later, the following powershell script can automate this purging process.

#==============================================
#       --- Configuration Options ---
#Provide the name of the site
$myweb = "Default Web Site"
#Provide the number of days you wish to keep
$days = 90
#whatif toggle, verify only option
$whatif = $true
#==============================================

Import-Module WebAdministration

$myweb = Get-WebSite -Name "$myweb"
$mylogdir = $myweb.logFile.directory

if($whatif)
{
get-childitem $mylogdir -recurse | 
	where {$_.lastwritetime -lt (get-date).adddays(-$days) -and -not $_.psiscontainer} |% {remove-item $_.fullname -force -WhatIf }
	}
else
{
get-childitem $mylogdir -recurse | 
	where {$_.lastwritetime -lt (get-date).adddays(-$days) -and -not $_.psiscontainer} |% {remove-item $_.fullname -force -ErrorAction SilentlyContinue} | Write-Host $_.fullname
}
,
January 31, 2011 at 1:27 am Comments (0)