Monthly Archives: February 2012

Windows 8 Consumer preview ISO and VS11 Beta downloads

In case you were looking for these:

The VS11 downloads also include the new TFS Express Beta…

Mail Aggregator Service on CodePlex

I’ve created a new project on CodePlex for ‘Mail Aggregator Service‘ so others can share and contribute if they want to.

There is no dedicated installer yet but installing the service is quite easy as the service exe supports my command line installation options like my other services (i.e. -install command line option).

It can be used with or without QuickMon.

I even have a new project icon/image for the project… mail gator šŸ˜‰

QuickMon 2.6 released

Another quick note – QuickMon 2.6 has been released. The biggest change is the introduction of some parallel threading when calling collectors – using the .Net 4.0 parallel extensions. No new collectors or notifiers added yet.

Mail Aggregator Service update

Just a quick update. I’ve now added a ‘Sql aggregator’ as well. This means there are now 2 ways aggregated messages can be sourced – from text files or Sql server data.

The ‘sql aggregator’ can use any existing table with messages by simply adding a bit field that indicates whether or not the message (in the table) has already been sent or not. The aggregator simply calls one stored proc (or if you want to a plain tsql statement can be used but I don’t recommend it) to retrieve any messages that must be send. This list must contains a message Id, ‘to’ address, subject and a body field. How you get by them combining fields, texts and other stuff is up to the creator of the stored proc. Once sent the ‘sql aggregator’ then calls a second stored proc passing the message Id to mark the message as sent. The sql parameter must be named ‘@Id’ and be an ‘int’.

An example sql script to modify the QuickMon sql notifier table plus two stored procedures are included in the sample project.

Eample source: MailAggregator Source code

Mail Aggregator Service

One problem with creating very useful apps sometimes is that you end up making more work or trouble for yourself šŸ™‚ My QuickMon tool has been created to help me monitor some services and ‘stuff’ which alerts me if things go pear shape… That is all good and wonderful until you (me) realize I’m beginning to ‘spam’ myself with too many messages (since I’m using smtp alerts…). Now they are all ‘valid’ alert messages and I still need to be aware of them but at some point there are just too many individual messages flying around. QuickMon alerts are quite configurable and if you want to some suppression is possible but managing this is almost a job on its own since things change with time, conditions change etc. etc. etc.

So I started thinking (an act that in itself could be dangerous at times) about how to solve the problem by grouping messages before they get sent. QuickMon itself raise alerts and immediately send them off – fire and forget style. Sure, I can build a new notifier that tries to do some batching (not sure how I can do that yet) or modify the QuickMon service itself to ‘hold on’ to alerts for time periods but that won’t be a generic solutionĀ any more. A better solution is to build a separate ‘generic’ solution that can also be used from QuickMon plus any other apps I might still create that need smtp messages to be sent out automatically (system level stuff).

This is where the idea of the MailAggregator service started. In itself it is a very simple thing – it runs, periodically checking for all messages from some ‘source’ that need to be send to the same destination, groups them as one message and sends it. The ‘source’ could be a database table or directory with files (or possibly any other place that can store a bunch of messages that can be retrieved at one time. For my own proof of concept I created a simple working example that checks for text files in a directory, use the content as the message content, group/append it, send it and then either deletes or renames the original files.

I decided to make the service a bit more dynamic in that you can add aggregators – implementing a standard interface so adding new aggregator source types in the future would be easy (like Text file and Sql table I already mentioned). Also, the service can run multiple aggregators (of the same or different types) at the same time – each serviced on its own thread (I just love threading but yeh, you have to be careful!). As with QuickMon each instance (or host as I termed it here) can have its own separate config.

So the end result of this little experiment is a working (or should I say workable) example that can already (with some electronic duck tape) be used with QuickMon alerts.

Basic architecture overview

  • MailAggregatorService

The main component of this little project is again a simple ‘Windows Service’. The service on its own is actually very simple – really just a shell that contains the aggregator hosts which calls the aggregator Ā libraries. A host instance gets created for every config entry (in an array) as specified in the config file.

This service also contains my code for self registration (to install it with the -install command line)

  • MailAggregatorHost

The MailAggregatorHost (class) is the container that holds the ‘aggregator source type library’. Because each source type implements a standard interface the host does not know or care which specific aggregator it is dealing with. The host is the entity that runs in a permanent loop calling the GetMessages() method of the aggregator, sends the smtp messages and then waits until the next iteration.

  • IMailAggregatorSource

Each mail aggregator source library implements theĀ IMailAggregatorSource interface for a particular resource type a.k.a. like Text files, Sql table or whatever. The interface itself is fairly simple and has the following members:

    • event AggregatorError
    • string GetConfigIdentifierType() – will be use later to dynamically load source types
    • bool SetConfig(string config) – configure the source type with specified config
    • List<MailAggregatedMessage> GetMessages() – get the messages dub…
    • string LastError { get;set;} – not used yet but one day….
  • MailAggregatedMessage
This is just a container to hold and pass the aggregated messages from the source type to the host.
  • MailAggregatorFFSource
This is the first example implementation of theĀ IMailAggregatorSource interface that check for plain text files (txt or anything you specify in the config) and groups them for messages. This implementation optionally checks for lines starting with ‘TO:’ and ‘SUBJECT:’. It is possible to specify default values that will be used if these lines are not available. The rest of the file is taken as the ‘body’ of the message.
By default it only groups ‘messages’ by the ‘TO’ address but you can also specify that the subject (plus TO address) will be used for the grouping. Grouped messages simple get appended.
A maximum messages size can be specified so that when a grouped set of ‘messages’ exceed the specified size a new MailAggregatedMessage instance will be created.
Once all the reading and grouping of messages have been done the processed files get either deleted or renamed as per config. When using the renaming option there is another choice – if the renamed file already exists it can either be overwritten or appended. ‘Done’ files are renamed to the original file name plus ‘.done‘.

Example source code

If you like to play with the source code yourself you need VS2010 – it requires .Net 4 (client framework)


Parallel threading and Control.BeginUpdate or EndUpdate

Yesterday I learned the hard (old) way that using BeginUpdate and EndUpdate in a multithreading environment (Windows Forms) don’t work so ‘lekker’ together. In hindsight, I should have guessed that this would be a problem but 20/20 hindvision really doesn’t help after the fact.

While making enhancements to my QuickMon tool to facilitate multithreading (using the .Net 4 parallel extensions) while calling collectors I started experiencing weird and unpredictable freezes or application hangs – app seems to be hanging but some parts of the UI still updates but the window cannot be moved, closed (in the normal way) etc. I tried various ways using mutexes, Control.Invokes, extra timers and duck tape to try and solve the problems. Only once I removed all my BeginUpdate/EndUpdates did the freezing issue disappear.

As mentioned before it all made sense afterwards. The problem is that when multiple background threads make calls back to the UI thread (yes, even using the proper Control.Invoke() method) that has a BeginUpdate in the beginning and later an EndUpdate there is no guarantee that every BeginUpdate is followed by its EndUpdate. That could mean some EndUpdates might never be reached properly.

The solution (for now) was to just disable all BeginUpdate/EndUpdates. Of course, a more proper design change would be to use some kind of buffer and a separate update UI routine that function outside the callbacks coming from the multiple threads collecting data. This unfortunately is not a small change and cannot be done in just a day or two. In a future iteration I’ll try to implement something like this for the QuickMon Windows client. The Windows service version of the tool wasn’t affected and work as is with the multi-threading change.

ME3 Demo video

Just thought to mention (in case you somehow missed it) there is a 40 minute of ME3 game play available here

Looking forward to March 9 (lets hope the date is correct)!

Useful Visual Studio 2010 Keyboard Shortcuts

This might be a bit old news for some but for those that forgot or never knew – here is a quick reference link on keyboard shortcuts inside Visual Studio.

Firefox tip

Quick tip: This might only work for version 10 (not tested otherwise) but you can type ‘about:memory’ in the URL bar of the browser to see details of what and how memory is used. Nice!