LiquidObject

Bulk disabling modules in LibreNMS

So I was in the process of adding in a large quantity of devices into LibreNMS but noticed a few days later the discovery and polling times were getting longer. Not really to be unexpected as there was more devices but some devices took an abnormally amount of time to finish the SNMP scans. Below is an example how to bulk disable attributes under LibreNMS (likely to work on Observium as well).

Outside of globally disabling different modules we will need to talk directly to the MySQL database in order to accomplish what we are looking for.

In the below example I’m looking at disabling the discover_arp-table on all Windows hosts.

select device_id,hostname from devices where OS = 'windows';

Then copy and paste the results into your favorite editor (I generally use excel for data like this).

="insert into devices_attribs (device_id,attrib_type,attrib_value) values ("""&A2&""",""discover_arp-table"",""0"");"

In the above the “A” column contains all the device_id’s I wish to adjust.

This will generate MySQL insert statements like the below

insert into devices_attribs (device_id,attrib_type,attrib_value) values ("4","discover_arp-table","0");

At this point all you need to do is run the generated MySQL statements against the LibreNMS database to disable the given module in bulk.

, ,
October 27, 2014 at 12:06 pm Comments (0)

Mass enabling windows features via PowerShell

Recently I needed to add the SNMP service to a few hundred systems. There are few more involved methods for this via unattended installs with SCCM or batch scripted GPO-linked entries but if you want a quick and easy way how, try the below.

Target OS: Server 2008, 2008R2, 2012 and 2012R2

Source system was a 2012R2 box with the AD and ServerManager PowerShell modules installed

Import-Module ActiveDirectory
$myservers = Get-ADComputer -SearchBase "OU=My Servers,DC=liquidobject,DC=com" -Filter "*"
foreach($i in $myservers)
{
 $mysession = New-PSSession -ComputerName $i.name
 Invoke-Command -Session $mysession {Import-Module Servermanager}
 Invoke-Command -Session $mysession {Add-WindowsFeature SNMP-Service}
}

Any server in the OU offline or running an unsupported os (ie Server 2003….we all have them) will throw an error. The above is pretty basic example on how you can install any Windows feature remotely. The remote PSSession and Invoke-Command methods allow you to perform any supported PowerShell command remotely.

In the event your running a legacy version of Windows, there are still options available. The limitation of the below is that it requires you to login to the given system and run the command with an administrative command prompt.

servermanagercmd.exe -install snmp-service
logout
, , , , ,
October 16, 2014 at 1:59 pm Comments (0)

Backup Exec 2012 database cleanup

When using Backup Exec 2012 by default will install a rather dated at this point in time version of SQL Server 2005 Express Edition. As time goes with and install the database grows and eventually can lead to errors like: “The Backup Exec Database has almost reached the 4-GB limit that is allowed for SQL Server Express Edition. To ensure that Backup Exec continues to function properly, either clean up the Backup Exec Database or upgrade to the full version of SQL Server.”, also known as V-287-13257.

To correct this there have been two trains of thought one to use the built-in BEUtility and the other is to modify SQL directly. I don’t view this as an either-or approach, I view it as a try this first and if not then try that.

First option, use BEUtility. Why, because it’s using the built-in methods provided by Symantec. Run the following options from the utility:

1. Age database

2. compact database

3. repair database

4. rebuild db indices

5. check db consistency.

The second option is to use the SQL Management Studio to manually perform some cleanup. Usually the source of the problem is the BELog table which can grow to gigabytes in size all by itself.

Lets check and see how many records are older than 6 months old:

SELECT count(username) from [BEDB].[dbo].[BELog] where TimeStamp < DateAdd("m", -6, getDate());

On larger environments don’t be surprised if it returns millions.

Now lets check and see if this is anything we still need.

SELECT * from [BEDB].[dbo].[BELog] where TimeStamp < DateAdd("m", -6, getDate());

Ok, now lets purge logs older than 6 months

delete FROM [BEDB].[dbo].[BELog] where TimeStamp < DateAdd("m", -6, getDate());
, , ,
July 9, 2013 at 12:35 pm Comments (0)

Clearing old Print Jobs

When using Windows Server as a print server over time the queues eventually begin to fill up. Print jobs are sent one day when a printer is offline and days later after the printer is turned back on sometimes dozens of print jobs start coming from the printer. The other issue that arises is caused by this backup of print jobs, slowly the print server will use disk space until in space cases it just runs out. Here is a simple PowerShell script to clear up the stale print jobs.

$TooOld = (Get-Date).AddDays(-2)
Get-WmiObject Win32_PrintJob | Where-Object { $_.ConvertToDateTime($_.TimeSubmitted) -lt $TooOld } | Foreach-Object { $_.Delete() }

This can be setup as an easy scheduled task to take of ever needing to worry about this issue in the future.

, , ,
June 25, 2013 at 11:41 am Comments (0)

Runaway process checking

Recently I ran into an issue with PHP exhaustion on a Windows Server running IIS. In this scenario the PHP-CGI.exe process would continue to spawn additional instances as load on the server would increase but over time the application pool would struggle and begin to slow to a crawl. In the past I have seen other applications during various iterations of development run into the same issue where if you run into more than “x” instances of an application it is unhealthy or less than “y” instances it is not running properly.

 

$myprocess = "php-cgi"
$myserver = "WebServer"
$mydomain = "liquidobject.com"
$mail_server = "mail.liquidobject.com"
$mail_recipient = "my_support_team@liquidobject.com"
$toomany = 40
$waytoomany = 80

$mail_sender = "$myserver@$mydomain"
$mailreport_subject = "Script: $myserver $myprocess count"
$body = " "

function SendEmailReport
{
    $body = [string]::join([environment]::NewLine, ($body)) 
    $msg = New-Object System.Net.Mail.MailMessage $mail_sender, $mail_recipient, $mailreport_subject, $body
    $client = New-Object System.Net.Mail.SmtpClient $mail_server
    $client.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
    $client.Send($msg)
}


$mycount =  (Get-Process -Name $myprocess).count

if($toomany -lt $mycount)
{
	$body = "We have $mycount $myprocess processes, something is unusual."
	if($waytoomany -lt $mycount)
        {
             IISRESET /STOP
             IISRESET /START
             $body = "We have $mycount $myprocess processes, IIS has been reset."
        }
        SendEmailReport
}

In this case we are sending an email notification to the fictional “support team” when more than 40 instances of the php-cgi process are running and in the event no one responds by the time 80 instances are hit the site is automatically bounced to ensure it’s availability.

The simple method for checking is the use of Task Scheduler and call up the script every 5 minutes, pretty simple yet effective.

, , , ,
May 11, 2013 at 7:30 am Comments (0)

WSUS repair after backup restoration

Last week during the process of updating a few services one of the WSUS servers had gone belly up. Installing over the top or performing an uninstall then a re-install did not clear up the issue. The next step was to perform a restore on the system (dedicated WSUS/Forefront for Exchange VM) from the last backup taken before the system started going south. The restore worked bringing back a WSUS instance which would talk to the clients without any issues but then I started seeing errors in the application event log. The errors were showing that recently approved updates which were in the database could not be found on the filesystem.

What I ran into was a catch-22 when performing an enterprise deployment of WSUS where the SQL instance is on another VM separate from the WSUS server. Restoring the SQL database is possible but it may or may not be exactly when the backup of the WSUS instance was so I opted the other option which allows for the existing database to remain intact.

1) Stop the “Update Services” and two IIS services
2) Delete everything found in the WsusContent folder, while it can be skipped I wanted to make sure everything was cleanly downloaded
3) Open up a command line
4) cd “\Program Files\Update Services\Tools
5) wsusutil.exe reset

This will issue a full verification of the WSUS installation and pull down all of the approved updates. Which at this point the 30GB or so of updates took a few hours to rebuild the entire local cache. At which time, updates began to flow normally.

, , , , ,
August 24, 2012 at 9:42 am Comments (0)

« Older Posts