Tag Archives: C# - Page 3

Mail Aggregator Service

One problem with creating very useful apps sometimes is that you end up making more work or trouble for yourself 🙂 My QuickMon tool has been created to help me monitor some services and ‘stuff’ which alerts me if things go pear shape… That is all good and wonderful until you (me) realize I’m beginning to ‘spam’ myself with too many messages (since I’m using smtp alerts…). Now they are all ‘valid’ alert messages and I still need to be aware of them but at some point there are just too many individual messages flying around. QuickMon alerts are quite configurable and if you want to some suppression is possible but managing this is almost a job on its own since things change with time, conditions change etc. etc. etc.

So I started thinking (an act that in itself could be dangerous at times) about how to solve the problem by grouping messages before they get sent. QuickMon itself raise alerts and immediately send them off – fire and forget style. Sure, I can build a new notifier that tries to do some batching (not sure how I can do that yet) or modify the QuickMon service itself to ‘hold on’ to alerts for time periods but that won’t be a generic solution any more. A better solution is to build a separate ‘generic’ solution that can also be used from QuickMon plus any other apps I might still create that need smtp messages to be sent out automatically (system level stuff).

This is where the idea of the MailAggregator service started. In itself it is a very simple thing – it runs, periodically checking for all messages from some ‘source’ that need to be send to the same destination, groups them as one message and sends it. The ‘source’ could be a database table or directory with files (or possibly any other place that can store a bunch of messages that can be retrieved at one time. For my own proof of concept I created a simple working example that checks for text files in a directory, use the content as the message content, group/append it, send it and then either deletes or renames the original files.

I decided to make the service a bit more dynamic in that you can add aggregators – implementing a standard interface so adding new aggregator source types in the future would be easy (like Text file and Sql table I already mentioned). Also, the service can run multiple aggregators (of the same or different types) at the same time – each serviced on its own thread (I just love threading but yeh, you have to be careful!). As with QuickMon each instance (or host as I termed it here) can have its own separate config.

So the end result of this little experiment is a working (or should I say workable) example that can already (with some electronic duck tape) be used with QuickMon alerts.

Basic architecture overview

  • MailAggregatorService

The main component of this little project is again a simple ‘Windows Service’. The service on its own is actually very simple – really just a shell that contains the aggregator hosts which calls the aggregator  libraries. A host instance gets created for every config entry (in an array) as specified in the config file.

This service also contains my code for self registration (to install it with the -install command line)

  • MailAggregatorHost

The MailAggregatorHost (class) is the container that holds the ‘aggregator source type library’. Because each source type implements a standard interface the host does not know or care which specific aggregator it is dealing with. The host is the entity that runs in a permanent loop calling the GetMessages() method of the aggregator, sends the smtp messages and then waits until the next iteration.

  • IMailAggregatorSource

Each mail aggregator source library implements the IMailAggregatorSource interface for a particular resource type a.k.a. like Text files, Sql table or whatever. The interface itself is fairly simple and has the following members:

    • event AggregatorError
    • string GetConfigIdentifierType() – will be use later to dynamically load source types
    • bool SetConfig(string config) – configure the source type with specified config
    • List<MailAggregatedMessage> GetMessages() – get the messages dub…
    • string LastError { get;set;} – not used yet but one day….
  • MailAggregatedMessage
This is just a container to hold and pass the aggregated messages from the source type to the host.
  • MailAggregatorFFSource
This is the first example implementation of the IMailAggregatorSource interface that check for plain text files (txt or anything you specify in the config) and groups them for messages. This implementation optionally checks for lines starting with ‘TO:’ and ‘SUBJECT:’. It is possible to specify default values that will be used if these lines are not available. The rest of the file is taken as the ‘body’ of the message.
By default it only groups ‘messages’ by the ‘TO’ address but you can also specify that the subject (plus TO address) will be used for the grouping. Grouped messages simple get appended.
A maximum messages size can be specified so that when a grouped set of ‘messages’ exceed the specified size a new MailAggregatedMessage instance will be created.
Once all the reading and grouping of messages have been done the processed files get either deleted or renamed as per config. When using the renaming option there is another choice – if the renamed file already exists it can either be overwritten or appended. ‘Done’ files are renamed to the original file name plus ‘.done‘.

Example source code

If you like to play with the source code yourself you need VS2010 – it requires .Net 4 (client framework)

MailAggregator

Parallel threading and Control.BeginUpdate or EndUpdate

Yesterday I learned the hard (old) way that using BeginUpdate and EndUpdate in a multithreading environment (Windows Forms) don’t work so ‘lekker’ together. In hindsight, I should have guessed that this would be a problem but 20/20 hindvision really doesn’t help after the fact.

While making enhancements to my QuickMon tool to facilitate multithreading (using the .Net 4 parallel extensions) while calling collectors I started experiencing weird and unpredictable freezes or application hangs – app seems to be hanging but some parts of the UI still updates but the window cannot be moved, closed (in the normal way) etc. I tried various ways using mutexes, Control.Invokes, extra timers and duck tape to try and solve the problems. Only once I removed all my BeginUpdate/EndUpdates did the freezing issue disappear.

As mentioned before it all made sense afterwards. The problem is that when multiple background threads make calls back to the UI thread (yes, even using the proper Control.Invoke() method) that has a BeginUpdate in the beginning and later an EndUpdate there is no guarantee that every BeginUpdate is followed by its EndUpdate. That could mean some EndUpdates might never be reached properly.

The solution (for now) was to just disable all BeginUpdate/EndUpdates. Of course, a more proper design change would be to use some kind of buffer and a separate update UI routine that function outside the callbacks coming from the multiple threads collecting data. This unfortunately is not a small change and cannot be done in just a day or two. In a future iteration I’ll try to implement something like this for the QuickMon Windows client. The Windows service version of the tool wasn’t affected and work as is with the multi-threading change.

DirectorySearcher and SizeLimit

Seems like each time you actually wants to use some MS technology for real you find a serious bug in it…

If you query active directory to get a list of say, all the machines on the network and the list happens to be bigger than a 1000 items there is a problem using DirectorySearcher. There is a property called SizeLimit but even if you set it to anything larger than a 1000 it always only return a 1000 items.

Fortunately (like a lot of other cases) there is a way around it. DirectorySearcher have another property for PageSize which if you set that then the size limit is ignored and you can access more than the 1000 items.

e.g.

string filter = “(objectCategory=Computer)”;
using (DirectorySearcher searcher = new DirectorySearcher(filter))
{
//searcher.SizeLimit = 2000; //ignored
searcher.PageSize = 1000;
SearchResultCollection matches = searcher.FindAll();
foreach (SearchResult match in matches)
{
DirectoryEntry de = match.GetDirectoryEntry();
…
}
}

RegexOptions.Compiled and x64 are not friends

As part of my attempt to use the FastColoredTextBox control I stumble across one of those undocumented .Net bugs that has been laying around for some time without being fixed – The Regex class with RegexOptions.Compiled option compiled under x64 or (Any CPU) that executes extremely slowly. In fact, it performs so slowly that it appears the whole application is frozen for a few minutes while the memory usage slowly climbs to ridiculous levels.

There are only 2 solutions to this issue for now: (1) compile all assemblies with x86 option only (not great if you are running on 64-bit) or (2) do not use the RegexOptions.Compiled option. For me the latter is a better choice since there is hardly a different between compiled and not compiled regular expression processing times – that I can see and certainly better than waiting minutes for a frozen app to start reacting again.

Thanks Microsoft for this one. Great way to waste time trying to look for problems in you own app while it is their crap breaking stuff.

 

Formatting C# code for blogging

I came across a nice example of a tool that can help with formatting of C# source code that you want to publish (on say Word Press that does not support it). The tool’s source code can be extracted from CodeProject (Fast Colored TextBox for syntax highlighting). Incidentally it can also format VB, HTML, SQL and PHP (pitty about JavaScript 😉 )

EventScavenger updates

I’ve added new Installers for the 2 Windows services in EventScavenger – one for x86 and one for x64. These were just added to the latest recommended download on the site. This include the ‘self-installing’ functionality where you can register the services with Service manager using the “-install” or “-uninstall” parameters.

Creating a self installing Windows Service

Being in the business of creating several Windows Services myself (EventScavenger, Event Collector, QuickMon) I started thinking about how to make the whole install and set up process a bit easier and eliminating the manual steps to install a Windows Service.

So what if you can simply run the service ‘exe’ with a “-install” or “-uninstall” parameter and let it do the work for you? Additionally, what if it it can even prompt you for start up type and user account details during the process – all of this without even using installutil…?

Well after some research, testing, trail and error, losing some more hair and eventually finding a way to make it work I now have a way to do all this. Nice thing is it can be used together with an MSI installer as well or even an external app or script.

ProjectInstaller

The trick is to create a custom ProjectInstaller class (not use the wizards that comes with VS to add installers). I’ve added some properties to this class to specify things like Service name, display name, start up type, account type, user name and password.

[RunInstaller(true)]
public class ProjectInstaller : Installer
{

private ServiceProcessInstaller processInstaller;
private System.ServiceProcess.ServiceInstaller serviceInstaller;

#region Added properties
public string ServiceName
{
get { return serviceInstaller.ServiceName; }
set { serviceInstaller.ServiceName = value; }
}
public string DisplayName
{
get { return serviceInstaller.DisplayName; }
set { serviceInstaller.DisplayName = value; }
}
public string Description
{
get { return serviceInstaller.Description; }
set { serviceInstaller.Description = value; }
}
public ServiceStartMode StartType
{
get { return serviceInstaller.StartType; }
set { serviceInstaller.StartType = value; }
}
public ServiceAccount Account
{
get { return processInstaller.Account; }
set { processInstaller.Account = value; }
}
public string ServiceUsername
{
get { return processInstaller.Username; }
set { processInstaller.Username = value; }
}
public string ServicePassword
{
get { return processInstaller.Password; }
set { processInstaller.Password = value; }
}
#endregion

public ProjectInstaller()
{

processInstaller = new ServiceProcessInstaller();
serviceInstaller = new System.ServiceProcess.ServiceInstaller();

Installers.AddRange(new Installer[] {
processInstaller,
serviceInstaller});

}

protected override void OnBeforeInstall(System.Collections.IDictionary savedState)
{

SetContextParameter(“name”, ServiceName);
SetContextParameter(“display”, DisplayName);
SetContextParameter(“desc”, Description);
SetContextParameter(“start”, StartType.ToString());
SetContextParameter(“account”, Account.ToString());

if (Account == ServiceAccount.User)
{

SetContextParameter(“user”, ServiceUsername);
SetContextParameter(“password”, ServicePassword);

}
base.OnBeforeInstall(savedState);

}

public void SetContextParameter(string key, string value)
{

if (!this.Context.Parameters.ContainsKey(key))

this.Context.Parameters.Add(key, value);

else

this.Context.Parameters[key] = value;

}

}

Calling the Installer/uninstaller

In order to get the service executable to handle the -“install” or “-uninstall” parameters I modified the Main method in Program.cs to accept a string parameter array and then call the appropriate method if either of those two parameters are passed.

Example code:

static class Program
{

static void Main(string[] args)
{

if (args.Length > 0)
{

if (args[0].ToUpper() == “-INSTALL”)
{

InstallService();
return;

}
else if (args[0].ToUpper() == “-UNINSTALL”)
{

UnInstallService();
return;

}

}
ServiceBase[] ServicesToRun;
ServicesToRun = new ServiceBase[] { new SelfInstallService() };
ServiceBase.Run(ServicesToRun);

}

private static bool InstallService()
{

bool success = false;
try
{

string exeFullPath = System.Reflection.Assembly.GetExecutingAssembly().Location;
string workingPath = System.IO.Path.GetDirectoryName(exeFullPath);
string logPath = System.IO.Path.Combine(workingPath, “Install.log”);
ServiceStartMode startmode = ServiceStartMode.Automatic;
ServiceAccount account = ServiceAccount.LocalService;
string username = “”;
string password = “”;

InstallerForm installerForm = new InstallerForm();
installerForm.StartType = ServiceStartMode.Automatic;
installerForm.AccountType = ServiceAccount.User;
if (installerForm.ShowDialog() == System.Windows.Forms.DialogResult.OK)
{

startmode = installerForm.StartType;
account = installerForm.AccountType;
if (installerForm.AccountType == ServiceAccount.User)
{

username = installerForm.UserName;
password = installerForm.Password;

}

}

Hashtable savedState = new Hashtable();
ProjectInstaller myProjectInstaller = new ProjectInstaller();
InstallContext myInstallContext = new InstallContext(logPath, new string[] { });
myProjectInstaller.Context = myInstallContext;
myProjectInstaller.ServiceName = “SelfInstallService”;
myProjectInstaller.DisplayName = “Self Install Service”;
myProjectInstaller.Description = “Self Install Service test”;
myProjectInstaller.StartType = startmode;
myProjectInstaller.Account = account;
if (account == ServiceAccount.User)
{

myProjectInstaller.ServiceUsername = username;
myProjectInstaller.ServicePassword = password;

}
myProjectInstaller.Context.Parameters[“AssemblyPath”] = exeFullPath;

myProjectInstaller.Install(savedState);
success = true;

}
catch (Exception ex)
{

System.Windows.Forms.MessageBox.Show(ex.Message, “Install service”, System.Windows.Forms.MessageBoxButtons.OK, System.Windows.Forms.MessageBoxIcon.Error);

}
return success;

}

private static bool UnInstallService()
{

bool success = false;
try
{

string exeFullPath = System.Reflection.Assembly.GetExecutingAssembly().Location;
string workingPath = System.IO.Path.GetDirectoryName(exeFullPath);
string logPath = System.IO.Path.Combine(workingPath, “Install.log”);

ServiceInstaller myServiceInstaller = new ServiceInstaller();
InstallContext Context = new InstallContext(logPath, null);
myServiceInstaller.Context = Context;
myServiceInstaller.ServiceName = “SelfInstallService”;
myServiceInstaller.Uninstall(null);
success = true;

}
catch (Exception ex)
{

System.Windows.Forms.MessageBox.Show(ex.Message, “Uninstall service”, System.Windows.Forms.MessageBoxButtons.OK, System.Windows.Forms.MessageBoxIcon.Error);

}
return success;

}

}

You might notice I use a custom form ‘InstallerForm’ here which I’m not showing the code for. All it is is a simple Windows Form that allows you to enter or choose some settings that this code is using. I’ll upload some code example a bit later.

Now, you can simple go to the command line (or create a batch file) that calls the service exe with either the “-install” or “-uninstall” parameters.

Update: I’ve added some sample code here. There have been some improvements since the article. It turns out the overridden method OnBeforeInstall is not necessary at all.

In a new blog post I’ve enhanced the functionality to include multi instance functionality. See this.

QuickMon Version 2.2

There has been some updates to my QuickMon tool (or set of tools). Since the last update I’ve added a new collector – the SQL Query collector, enhance the Windows Service to allow multiple monitor packs to be use at the same time and tried to simplify the management user interface.

With the latest changes it is becoming even more useful. Try it out!

TreeViewEx – a few enhancements

Sometimes I wonder why some functionality that should be common is simply left out of general controls.. like the WinForms TreeView (and ListView) controls that don’t have specific events for the enter key pressed. I’ve been taught long ago to always provide functionality so keyboard events also can be used to navigate around applications – like when the mouse is unavailable, broken or not your first choice (like for disabled people). Yes, I know the controls have general events for key pressing but it requires some hectic coding each time to do something that should be simple.

Anyway, adding it isn’t too difficult (but I still wished I didn’t have to do it each time I need to use it).

public class TreeViewEx : TreeView
{

public TreeViewEx() : base()
{

DoubleBuffered = true;

}
public event MethodInvoker EnterKeyPressed;
public event MethodInvoker DeleteKeyPressed;
protected override void OnKeyPress(KeyPressEventArgs e)
{

if (e.KeyChar == ‘\r’)

if (EnterKeyPressed != null)
{

EnterKeyPressed();
e.Handled = true;

}

base.OnKeyPress(e);

}

protected override void OnKeyDown(KeyEventArgs e)
{

if (e.KeyCode == Keys.Delete)

if (DeleteKeyPressed != null)
{

DeleteKeyPressed();
e.Handled = true;

}

base.OnKeyDown(e);

}

}

The code for the ListView control is basically (exactly) the same.

QuickMon update

In the past couple of weeks I’ve been busy part time working on this little project to build a simple monitoring system for myself. At the moment the basic ‘bits’ are all there working reasonably well I might add. On the one side I’ve been contemplating putting the project up on CodePlex like many of my other pet projects but this one has some potential to ‘grow’ into something that could generate some thing more (like income or so…) The trouble with that idea is it may be even more powerful if ‘others’ can extend it but then the main source is available to all. Also, this little tool is not in the same league as proper enterprise monitoring tools like Microsoft SCOM and so on.

Anyway, as far as progress goes, the ‘tool’ or set of tools now can do the following (all tested and working)

  • Main shared components for editing, management and configuration of agents plus both a Windows UI and Windows service for hosting the agents (monitor pack).
  • Collector agents for
    (General)
    Disk space – local drive space available
    File count and size – file/directory size
    Performance counters – monitor performance counters
    Ping – ping machines
    Windows service state – are specified services running or not
    WMI query – specify some WMI query and compare the return value
    (Specialized)
    BizTalk ports and orchestrations – check if ports/orchestrations are running
    BizTalk suspended count – check the suspended instance count
    SQL server database size – check database size
    SQL server table size – check row count
    Httpping – like ping but for http
  • Notifier agents for:
    Log file – a simple text log file
    RSS feed – actually really just an xml file that you can host on a web server that is exposed as an RSS feed
    SMTP – Sending out alerts as emails
    SQL database – storing alert details in a sql server
  • A special type of ‘Collector’ that is just a folder to contain others (so you can enable/disable all or set a service window for that branch)
  • Collector agents support the following general properties and methods:
    Enabled
    Check dependent collectors even after warning
    Repeat alert after x minutes
    Only alert once in x minutes
    Service windows (yes, even multiple service windows e.g. 5:00 to 12:00 and 13:00 to 17:00 etc.)
    Launch a detail view window
  • Notifier agents support the following properties:
    Enabled
    Alert level (Info, Warning, Error)
    Detail level (Detail, Summary or both)
    List of collectors to ‘service’ (default = all)
    If implemented – launch a viewer for the notifier

Development

I’m using the .Net 4 client profile framework so download footprint should be fairly small – and I’d like to keep it that way. It is developed using C# and is extendable because of the add-in type of architecture. To add an agent is as simple as implementing a provided interface.

Download example

Soon I’ll add a download link for (the current) test version. It is already fully working as is but I’m continuing to enhance and add things.

Download a first test version here.