Fervent Coder

One of my goals for the year was to speak at least three times this year and from the looks of things, it appears I’ll meet that goal in the first half of the year. 

This is what’s going on this year for me (subject to additions/changes):

• Rocky Mountain Tech Tri-Fecta 2.0 - http://rmtechtrifecta.pbworks.com/
• Coders For Charities (C4C) - http://coders4charities.org/
• Chicago Alt.NET May Meeting
• KC .NET UG July Meeting
• Kansas City Developer Conference (KCDC)
• Virtual Alt.NET Meetings
• Other Events

Rocky Mountain Tech Tri-Fecta 2.0

RMTT (hash tag #rmtt) will be the last weekend of February. I’m going to be doing a Birds of a Feather presentation. From what I understand, the BoF presentations will be at 7AM for one hour. I will be facilitating/leading a group discussion on automation tools. Here is the abstract:

Build/Deploy Automation & Developer Automation Tools

This will be an open discussion about build/deploy and developer automation. In the .NET world, there has been a serious need for better tooling in this area and we can discuss the needs/deficits. If the conversation steers toward demos, we can take a look at a build tool called UppercuT, which is the fastest zero to professional build you will find in the .Net market to date (plus it's 100% free!). We can also look at a database change management tool known as RoundhousE. Another tool that may be shown is a project (solution and everything else) templating tool known as Warmup. Other tools are open for discussion and demos. If you have a developer automation tool you love, or you are are a developer automation tool builder, come ready to discuss your tools!

Coders For Charities

Coders For Charities (hash tag #c4c) will be the last weekend of March. This is the 3rd Annual C4C event where “dozens of Kansas City area web developers, designers, and business analysts will engage with local non-profit organizations for a weekend of ‘giving back’ to their communities.” This is something I’ve been wanting to do since the first event three years ago. It looks like an awesome event. If you are in the KC area and you haven’t already signed up for this event, what are you waiting for? This is going to be a great place to meet other developers and work hard on something that benefits a charity or non-profit organization.

Chicago Alt.NET May Meeting

I believe I will be presenting on UppercuT when I go out to Chicago Alt.NET’s monthly meeting in May. I’ve never been to Chicago, and I’ve heard that it’s beautiful in May. There may be some touring in order. I’m pretty excited to get out to see the sites and hang out with a bunch of cool, smart people up in the windy city. If I present on UppercuT, this is the abstract:

Automated Builds: How to UppercuT Your Code!
“Build – it’s not just for F5 anymore.”

How you build your code and verify quality is something that is usually not thought of at the beginning of a project, but is one of the most important things you can add to code! During this session we will go over the conventions in building and verifying code quality. We will see a project that is using automated builds and how all of the conventions are applied. We are going to see UppercuT and how well suited it is for automated builds. UppercuT is a build framework (based in NAnt) that allows rapid and powerful use of NAnt without having to understand the intricacies of NAnt. The last thing we will do is apply UppercuT to a project to show you how fast you can go from F5 to automated builds!

KC .NET User Group July Meeting

At the KC DNUG I’ll be presenting on RoundhousE in July. KC DNUG is a fantastic group and I enjoy every event I can get to with them! Here is the abstract:

Database Change Management with RoundhousE!
"Because kicking your database is a good thing!"

Many would not argue that you need to version your code, and few would argue that you should version your code in a way that can lead you back to a specific point in source control history. However, most people don't really think of doing the same with your database. That's where RoundhousE comes in. RoundhousE versions your database how you want. Not to mention it's one of the most intelligent database migrations tools out there, it also helps you keep your scripts in source control in a way that makes sense. We'll walk through the tool and its features and then open for questions. You'll see how it can make database change management extremely simple for you and how it makes auditors and DBAs smile. RoundhousE - you know you want to learn more...

Kansas City Developers Conference

KCDC (hash tag #kcdc) doesn’t have a firm date yet from what I hear. I don’t know much about this one yet, other than that I am so there. Hopefully talking about something cool as well!

Virtual Alt.NET Meetings

Virtual Alt.NET, lovingly known as the VAN hosts meetings on Wednesday nights. Now I believe that is going to be once a month.  I’m hoping to present on two topics this year. One of those is UppercuT and the other is RoundhousE (same abstracts as above). The dates for these are not set in stone yet, and I probably shouldn’t be trying to jinx it by talking about them yet! I’m really excited to do a presentation online though and how that will work out!

Other Events

I loved Iowa Code Camp last year. It was awesome! I am definitely looking forward to another round with a bunch of awesome guys/gals! Tulsa TechFest was another event I really enjoyed and I would like to get back to this year. One I’ve never been to and think I would like to check out is the Heartland Developers Conference (HDC).

I’m open to other cool events and meeting more people smarter than I am. What conferences are you going to that I might find interesting?

What if there was a tool out there that could let you specify a structure for a project (visual studio solution + everything else) and save you up to 3+ hours of work every time you started a new project?

Warmup was an idea by Dru Sellers to remove all of the setup work required every time you set up a new project. You know, create the solution, add projects, put in your references, etc. Then how about getting the infrastructure for your service/website/console set up as well with things like IoC, etc? What about patterns and other pet items that you put in any project?

Yeah – there’s an app for that. And it’s pretty simple to use. Plus you can change your templates when you have new ideas, so it’s totally rockstar!

The first thing you do is to set up templates somewhere in source control (svn or git). Then you specify where that is to the configuration and what type of source control.

<warmup
  sourceControlWarmupLocation="git://github.com/ferventcoder/warmup-templates.git"
  sourceControlType="git"
  />

Then you run a simple command. If base was one of the folders below the directory above my source control, then that is what I would specify as the first argument.

warmup.exe base nameOfProject

I specify nameofProject. That is what I want my project to be named when I am complete. 

Templating

Let’s start by taking a look at the base template. The basic idea here is simple. Place __NAME__ everywhere you want to be replaced when running Warmup. In the same way UppercuT does token replacement with ConfigBuilder, DocBuilder, SqlBuilder, and DeployBuilder, Warmup does token replacement for an entire solution.

If we were to look at some of the files you would see the absolute depth of how naming really can be replaced. Let’s take a look at the solution though, that will hold quite a bit of meaning for you.

Now that we have our template all set up, we have one thing to do. Open our .sln file in Notepad and delete the first line (not sure what happens here, but this works). Microsoft should be on the first line when we are done.

And now our template is all ready, so we check our changes into source. If we are using git, we need to push back to the repository we are looking at after we finish committing.

Run Warmup

Now that we’ve seen our template, let’s run Warmup.

Let’s call the new project Alpha.

warmup.exe base Alpha

This is the output:

Hardcore git cloning action to: C:\code\warmup\code_drop\warmup\Alpha
Running: cmd  /c git clone git://github.com/ferventcoder/warmup-templates.git C:\code\warmup\code_drop\warmup\Alpha
Initialized empty Git repository in C:/code/warmup/code_drop/warmup/Alpha/.git/

replacing tokens

This project already has UppercuT, RoundhousE, and others already in it. Take a look at the lib folder:

I open a command line and run build.bat.  I get a successful build with 31 passing tests! If I were to go to the code_drop folder, I’m all ready to deploy if I had my deployment framework already here.

I can already run RoundhousE and create my database from here:

Conclusion

I have an entire structure that allows me to just concentrate on the stories at hand. Warmup may not be the best thing since sliced bread, but it’s going to save you oodles of time! If you know someone else that has create a template you want to use and it’s shared publicly, you can just edit the config file to point there. We have been using warmup for close to 3 months now and it saves us a bunch of time. Plus we find more and more things we can put back into the templates to save us time. I believe this aspect of learning and growing your templates over time is the intention of warmup. Plus your template may not be the same as mine and that’s completely cool!

If you are seeing text messages coming in from the year 2016 and thinking this is a total WTF (mate?!), put down the phone. There is no need to break the phone. It’s fine. Nearly all WM phones are experiencing this right now. I repeat, there is no need to break the phone!  There is a fix for this that requires maybe five minutes of your time.

DISCLAIMER: You do this on your own terms. I am in no way liable if you follow these instructions and damage or render useless anything at all including but not limited to your phone. You accept that you are 100% liable for any actions you take based on this or any reference from or to this posting. In other words, you can’t hold me accountable if you screw up your phone which I know you love and cherish. This hack worked for me. YMMV. Are you still reading this part? Move along now…nothing to see here.

I was pointed to this post on twitter today when I complained about it being 2016 all of a sudden (thanks @thines01!): http://www.wmexperts.com/y2016-sms-bug

Seems a new bug has afflicted our phones today: text messages come dated six years from now.

So unless something really weird happened last night, we're seeing a nasty bug in our little gizmos.  Here's what we know as of now:

• Messages received after 1/1/2010 may be dated as 2016
• Effects WM6.1, WM6.5 and even non-Windows Mobile phones
• It is ROM independent, meaning switching won't help
• All Some carriers seem to be experiencing this bug, but T-Mobile seems fine
• HTC and LG are aware, but no official response yet

The fix is actually really simple. You will need a registry editor (or you can just install the cab from the site in the link above – I’ve heard it’s also on PPCGeeks).

1. Find this key: HKLM\Software\OEM\SMS
2. Add a new DWord Value named RecvTimeStamp with a Decimal value of 1.
3. Add a new String Value named Language with a value of 100. NOTE: Unsure if this is required or not. Try it without first.
4. Close the registry editor.
5. Wait.
6. Wait for a new SMS message to come in.
7. Look at that…nice. Now what to do about those other ones that are showing ABOVE the new messages? That’s for you to figure out.

There you go. I actually said the same in less than 140 characters as well. Good luck!

This last year I had a focus to get more involved in Open Source (OSS). The year before I had been involved a little by submitting a patch here and there, but 2009 was a big year for me in OSS. I now manage 4 open source projects that have solved a need for me and others. Coming up on the end of the year it was important for me to get some final touches into at least two projects to finish out the year.

UppercuT

UppercuT (UC), for those of you who have never heard of it, is an automated build tool that uses conventions to build your code. That means that you can have a professional build for most projects out of the box within 3 minutes! You drop it into a project, update a simple configuration and you’re done.  It is also extendable (to the point of steps being completely replaced) so it’s very powerful. The goals we had for UC when we started it are that it’s insanely easy to use, easy to upgrade your builds, and easy to extend. It has met all of those goals and has some features that other build tools do not have (versioning out of the box and templating for configurations, documents, sql, deployment files, etc to name a couple). Check it out at http://projectuppercut.org.

So what milestone did we finish for UC? When the project started, there were plans for support for multiple types of source control for versioning the assemblies. We supported Subversion (SVN) from the beginning, but the priority for other types was lower. I’m pleased to announce that in the last two months we’ve added both TFS and GIT. The GIT one is a little more interesting. It follows what is suggested for versioning by Chris Ortman, GitFu and in the Git documentation. We also go the extra mile to show the SHA1 as well so you could theoretically get to that version.

How easy is it to do versioning with Git? Assuming you have a git repository, you crack open the configuration and change your source control to git:

If you’ve used UppercuT in the past or have been dying to try it out, set a goal to look at the latest verion this year before February.

RoundhousE

RoundhousE (RH), for those of you who have not heard of it, is a database migrations tool that uses SQL Scripts. RH is kind of like Tarantino. We had originally wanted to contribute our ideas to Tarantino, but some of the goals for the two didn’t line up nicely so RoundhousE was born. RoundhousE runs update scripts (the goal is DDL/DML only), but it also looks for other scripts in other folders (like functions, views, and sprocs). The reasoning for the separation is so that versioning and seeing version history is much cleaner in source control. RH also versions databases the way you want. We prefer to version our databases based on source control, but it is extendable to be versioned in any way (or not at all). RH has an MSBuild task, a NAnt task, but most importantly, a command line console (rh.exe). Everything is configurable (including the version and scripts run tables).  Check it out at http://projectroundhouse.org. RH has quite an extensive roadmap for where it is going (we are moving it to be a total migration package, not just a migration runner).

So what milestone did we finish for RH? Making RH environment aware so you can execute environment specific scripts. Why would you want to do that? What if you wanted to insert a bunch of test data into your database but didn’t want that to get into production? What if you want to run the output of templated scripts for granting permissions to different users in different environments?

Here I have a set of permissions scripts. Notice they are environment specific.

Notice also that they have two things. They have “.ENV.” in the name to let RH know they are environment scripts. Then they also have the name of the specific environment (LOCAL, TEST) they should be run in. By the way, If you use UC and templating, you only maintain one file in source control and the file per environment is generated at build time (I digress). So we have some environment files and now I run RH to migrate my database.

We can see from the log notes below that it only runs the LOCAL environment script in the LOCAL environment.

LOCAL.GrantRobDataReaderDataWriterPermissions.ENV.sql is an environment file. We are in the LOCAL environment. This will run based on this check.
Running LOCAL.GrantRobDataReaderDataWriterPermissions.ENV.sql on (local) - TestRoundhousE.
TEST.GrantRobDataReaderDataWriterPermissions.ENV.sql is an environment file. We are in the LOCAL environment. This will NOT run based on this check.

And to be sure, we look in the database:

If you’ve looked at RoundhousE in the past or haven’t yet looked at it, it’s worth downloading and running the sample to see what it can do.

By the way, you’ve probably noticed that have a definite theme with our developer automation tool suite (WarmuP, UppercuT, RoundhousE, DropkicK, HeadlocK, and SidePOP). We are still not solid on the library name, but pretty close. Each of the tools are designed to be enjoyable to use and free. We respond to inquiries and fix problems quickly. Once we have a name for the library, we may offer enterprise support for those that would require that.

Today marks the beginning of a new year. For many of us that means new resolutions like losing weight or quitting smoking. I tend not to focus on resolutions because I am horrible about keeping them. I usually set goals instead of making resolutions. And when I do set goals I intend for them to be SMART (Specific, Measurable, Achievable, Realistic, Timed).  I’ve found that having goals that are measurable and realistic within a given time frame help me focus on how I want to get there from where I am currently. In doing so I tend to set goals like “Lose 5 pounds in 3 weeks” versus the more general “Lose weight.” Measurable goals with a time limit give you a focus on a deadline.

So, in keeping with the spirit of the new year, let’s raise our glasses and toast each other on our SMART goals (or DUMB goals) for the upcoming year. Here, here!

Here are a few of my stated goals:

• Read five books this year (on anything, just read five books).
• Release UppercuT v1 by the end of June.
• Make a for fee product and have that solves a problem for many people by the end of the year. This means the product should be available for purchase and have support.
• Speak at least 3 times this year on topics I and others find interesting.
• Learn 2-3 new things a month, if not at least 1 new thing a day.
• Be awarded MVP by the end of the year for my contributions to the community through OSS and teaching others. I realize this goal is requires outside influence, so I’ll say that if I make every effort to help and contribute to the community, then that is where I measure the achievement.

Tim has already talked about his goals for the year. What are your SMART goals (links to posts are okay) for this year?

The Classic Infinite Email Loop

When building an email parser, one must think about validating an email address that sends you a message. Just because that email address is where the message says it is from does not really mean that is a valid email address.  Some of you can already see where I’m going with this.

I was building something in Bombali that would respond to emails when receiving them. When Bombali received a message from an address, Bombali would send a response. No checking on the address. Classic mistake right? Well new to me, I’ve never been able to do this programmatically until recently. So Bombali received a message with a no reply from address. It sent a response. Then it got a response from the postmaster saying that this was an invalid address. I bet you’re guessing what happens next? Yes, Bombali responded to the postmaster. And then the postmaster responded back. Over a matter of about 30 minutes there was quite a bit of traffic back and forth until I deleted the emails before Bombali could check them. At least I was smart enough to set the email checking to every half minute.

How did I get past this? Validation. An authorized list form of validation. Interested? Read on dear reader.

SidePOP XmlConfigurator

I was exposing my configuration in the last post, so I built an XmlConfigurator that does this for you and gives you back a list of Email Watchers (renamed SidePOPRunner).

private void configure_mail_watcher()
{
    EmailWatcherConfigurator configurator = new SidePopXmlConfigurator();
    foreach (EmailWatcher emailWatcher in configurator.configure())
    {
        emailWatcher.MessagesReceived += runner_messages_received;
        emailWatcher.start();
    }
}

Subscribing to Receiving Email

The code above and the code below show how easy it is for your application to receive email. I map the email to a local object and then I pass that off to a function to parse and send a response.

private void runner_messages_received(object sender, MessageListEventArgs e)
{
    IEnumerable<SidePOPMailMessage> messages = e.Messages;

    foreach (SidePOPMailMessage message in messages)
    {
        Email mail_message = Map.from(message).to<Email>();
        parse_and_send_response(mail_message);
    }
}

So now I am receiving email, but I need to parse it do determine what to do. So let’s show the parser first before the method for sending a response.

Really Simple Email Parser

This is a really simple parser. It’s not by any means what some people would use in production and that’s fine. It works for what Bombali needs.

public MailQueryType parse(Email message, IList<IMonitor> monitors, IDictionary<string, ApprovalType> authorization_dictionary)
{
    MailQueryType query_type = MailQueryType.Authorizing;

    string user = message.from_address.to_lower();
    bool user_is_authorized = false;

    if (authorization_dictionary.ContainsKey(user))
    {
        ApprovalType user_is_approved = authorization_dictionary[user];
        if (user_is_approved == ApprovalType.Approved)
        {
            user_is_authorized = true;
        }
    }

    if (user_is_authorized)
    {
        query_type = MailQueryType.Help;
        string subject_and_body = message.subject + "|" + message.message_body;

        if (message_contains_status(subject_and_body)) query_type = MailQueryType.Status;
        if (message_contains_config(subject_and_body)) query_type = MailQueryType.Configuration;
        if (message_contains_down(subject_and_body)) query_type = MailQueryType.CurrentDownItems;
        if (message_contains_approve(subject_and_body)) query_type = MailQueryType.Authorized;
        if (message_contains_deny(subject_and_body)) query_type = MailQueryType.Denied;
        if (message_contains_version(subject_and_body)) query_type = MailQueryType.Version;
    }

    return query_type;
}

I have a MailQueryType so that I separate what how to respond from actually responding. What you can see is that I’m already checking to see if we have an approved user before I set a different MailQueryType. The mail parser just sends back how Bombali should respond to the message.

Sending a Response

In subscribing to receiving email, I passed the message to the method below. The second line of the method shows me passing the message off to the mail parser to find out how to respond.  The switch could probably be replaced by a Strategy pattern, but for now it works and it’s all I need.

private void parse_and_send_response(Email mail_message)
{
    string respond_to = mail_message.from_address.to_lower();
    MailQueryType query_type = mail_processor.parse(mail_message, monitors, authorization_dictionary);
    Log.bound_to(this).Info("{0} received a message from {1} of type {2}.", ApplicationParameters.name, respond_to, query_type.ToString());

    string response_text = string.Empty;

    if (query_type == MailQueryType.Authorized || query_type == MailQueryType.Denied)
    {
        string[] body_words = mail_message.message_body.Split(' ');
        foreach (string body_word in body_words)
        {
            if (body_word.Contains("@"))
            {
                respond_to = body_word.Replace(Environment.NewLine, "");
                break;
            }
        }
    }

    switch (query_type)
    {
        case MailQueryType.Denied:
            authorization_dictionary.Add(respond_to, ApprovalType.Denied);
            return;
            break;
        case MailQueryType.Authorized:
            authorization_dictionary.Add(respond_to, ApprovalType.Approved);
            response_text = string.Format("Congratulations - you have been approved!{0}Send 'help' for options", Environment.NewLine);
            break;
        case MailQueryType.Help:
            response_text =
                string.Format(
                    "Options - send one:{0} help - this menu{0} status - up time{0} config - all monitors{0} down - current monitors in error{0} version - current version",
                    Environment.NewLine);
            break;
        case MailQueryType.Status:
            TimeSpan up_time_current = up_time.Elapsed;
            response_text = string.Format("{0} has been up and running for {1} days {2} hours {3} minutes and {4} seconds.", ApplicationParameters.name,
                                          up_time_current.Days, up_time_current.Hours, up_time_current.Minutes, up_time_current.Seconds);
            break;
        case MailQueryType.CurrentDownItems:
            response_text = string.Format("Services currently down:{0}", Environment.NewLine);
            foreach (IMonitor monitor in monitors)
            {
                if (monitor.who_to_notify_as_comma_separated_list.to_lower().Contains(respond_to))
                {
                    if (!monitor.status_is_good) response_text += string.Format("{0}{1}", monitor.name, Environment.NewLine);
                }
            }
            break;
        case MailQueryType.Authorizing:
            response_text =
                string.Format("Bombali notified admin to add you to approved list. If you are added, you will receive a response.");
            break;
        case MailQueryType.Version:
            response_text = string.Format("Bombali is currently running version {0}.", Version.get_version());
            break;
        default:
            response_text = string.Format("{0} has not been implemented yet. Please watch for updates.", query_type);
            break;
    }

    SendNotification
        .from(BombaliConfiguration.settings.email_from)
        .to(respond_to)
        .with_subject("Bombali Response")
        .with_message(response_text)
        .and_use_notification_host(BombaliConfiguration.settings.smtp_host);

    if (query_type == MailQueryType.Authorizing)
    {
        SendNotification
        .from(BombaliConfiguration.settings.email_from)
        .to(BombaliConfiguration.settings.administrator_email)
        .with_subject("Bombali Request")
        .with_message(string.Format("{0} reqests approval. Send approve/deny w/email address. Ex. 'deny bob@nowhere.com'", respond_to))
        .and_use_notification_host(BombaliConfiguration.settings.smtp_host);
    }

}

You’ll notice that I’m authorizing any new email addresses that send an email to Bombali before allowing them access to the application. I do that by having an administrator contact I can send a request for approve or deny to before allowing access. I also send an email back to the sender letting them know they are in the process of being approved. At most I would send one email to a bad address. As administrator, the email can be validated by a human. And I only respond to an authorized list. I could also build in something to validate the email by checking for a response from a post master telling me the address doesn’t exist.

So remember readers - always check your sources. Otherwise you might get in an infinite loop. And that would be bad.

From my previous post on SidePOP, I wanted to make it even easier to configure SidePOP.

So instead of the configure_mail_watcher() from the last post, this is what it looks like now: 

private void configure_mail_watcher()
{
    EmailWatcherConfigurator configurator = new SidePopXmlConfigurator();
    foreach (EmailWatcher emailWatcher in configurator.configure())
    {
        emailWatcher.MessagesReceived += runner_messages_received;
        emailWatcher.start();
    }
}

 Pretty.

Notice that Runner became EmailWatcher and instead of run() we start(). In conjunction there is also a stop() method as well.

Background

Sending email has long been easy to do with the .NET Framework. There really hasn’t been a facility for checking email though. I recently had a need to be able to check email with and send a response with Bombali (a monitoring tool). I went searching for examples or a solution I could use. I found a few articles on this from a few years ago including the .NET POP3 MIME Client.  I never found anything that was free and I never found anything that was easy to configure. Those who know me know I have a tinge of the NIH (not invented here), especially when there are no free alternatives. So I created my own based on the articles I had read. And in the spirit of chucknorris, I named it SidePOP!

Configuration

Configuration for receiving email should be easy. Feast your eyes on all that is required for configuration (and for XML haters, it’s not required – you can configure all in code):

<configSections>
  <section name="sidepop" type="sidepop.configuration.SidePOPConfiguration, sidepop"/>
</configSections>

<!-- 110 is normal POP3 SSL uses port 995-->
<sidepop>
  <accounts>
    <add hostName="mail.somewhere" userName="yep" password="" minutesBetweenChecks=".1" />
    <add name="Number1" description="Main account" enabled="true" hostName="mail.somewhere.net" hostPort="110" useSSL="false" userName="" password="" minutesBetweenChecks="1" timeoutInMinutes="1" />
  </accounts>
</sidepop>

You see there are two items here. The first one has the required items in it. The second one is the entire set of items to configure.  Notice that we are asking for a password in plaintext. This is where you might think about encrypting this section somehow.

Code Setup

Configuring SidePOP to run is a little more verbose that I might like right now. Looking below you see that it’s still not that much code. I hope to set up a facility to automatically look at the configuration file and configure SidePOP (for those that like XML). If someone wants to send me a patch as well, that would be awesome.

private void configure_mail_watcher()
{
    foreach (AccountConfigurationElement account in SidePOPConfiguration.settings.accounts)
    {
        if (account.enabled)
        {
            SidePopRunner runner = new SidePopRunner(new DefaultPop3Client(account.hostName, account.hostPort,
                                   account.useSSL, account.userName,
                                   account.password), account.minutes_between_checks);
            runner.MessagesReceived += runner_messages_received;
            runner.run();
            Log.bound_to(this).Info("{0} is configured to watch for messages with user {1} at {2} every {3} minutes.", ApplicationParameters.name,
                                    account.userName, account.hostName, account.minutes_between_checks);
        }
    }
}

private void runner_messages_received(object sender, MessageListEventArgs e)
{
    IEnumerable<SidePOPMailMessage> messages = e.Messages;

    const string subject = "Bombali Response";
    TimeSpan up_time_current = up_time.Elapsed;
    string text_message = string.Format("{0} has been up and running for {1} days {2} hours {3} minutes and {4} seconds.", ApplicationParameters.name, up_time_current.Days, up_time_current.Hours, up_time_current.Minutes, up_time_current.Seconds);

    foreach (SidePOPMailMessage message in messages)
    {
        Log.bound_to(this).Info("{0} received a message from {1}. Responding that service is still running.",ApplicationParameters.name,message.From.Address);
    
        SendNotification.from(BombaliConfiguration.settings.email_from).to(message.From.Address).with_subject(
        subject).with_message(text_message).and_use_notification_host(BombaliConfiguration.settings.smtp_host);
    }
}

When I start the service, I call the method configure_mail_watcher(). Because I can have multiple accounts I can check with SidePOP, I am going to loop through the settings and check each one to ensure they are enabled. If so, then I am going to set up a SidePopRunner and subscribe to runner_messages_received (this may not work with multiple accounts, I haven’t yet tested that).  Then I kick off the runner for that account.

I could instead subscribe to runner.MessageReceived (instead of runner.MessagesReceived) for getting an event notification for every message instead of batching it up to a list if I desired.

When a message is received, I find out how long the service has been up. I didn’t show it here, but up_time is a System.Diagnostics.Stopwatch. In the constructor I start it. Now I just capture how long it has been up. Then for each message, I send a response to the sender of the email address telling them how long the service has been up. To add more sophistication, I could very easily add a rules engine here to process the message based on a list of rules looking for keywords in the email.

That’s all I need to do. Now I’m up and ready to rock.

What It Looks Like

Here is what it looks like running with Bombali. I send the service a text message. This is what it logs.

2009-12-07 23:59:05,875 5 [INFO ] - Bombali received a message from someone @ somewhere.net. Responding that service is still running.
2009-12-07 23:59:05,875 5 [INFO ] - Sending email to someone @ somewhere.net with subject "Bombali Response" and message:
Bombali has been up and running for 0 days 0 hours 31 minutes and 22 seconds.

This is what I get from my phone.

Now I know that my service is still up and running with a simple text. All for the low price of a few lines of code and the sidePOP.dll library.

Conclusion

SidePOP gives me a level of sophistication that I didn’t have before. Now that I have access to get email, I could very easily add rules to do different things based on the content of the email received. How often have you wanted your application to check email and just didn’t want to pay the price for the products out there that can do this? Now it’s completely free with SidePOP.

Download SidePOP

NOTE: This is still early. Any part of the configuration could change. Be sure you take a look at the sample app to always have the newest configuration. Register any bugs you find here: http://code.google.com/p/sidepop/issues/list

WARNING: SidePOP will delete your email when it checks it. That’s how it can be sure it’s only dealing with new messages every time. Do not test on an account you care about. You’ve been warned.

SVN here: http://sidepop.googlecode.com/svn/trunk/

Downloads here: http://code.google.com/p/sidepop/downloads/list

NEXT: XmlConfigurator is done! 

When you have command line arguments to parse (whether on Windows or Linux), the place to look is Mono.Options. To date, I have not found anything better to get the job done. It is a single C# file if you do not want to download the entire Mono Library to use it. I was first introduced in Mono 2.2. The equivalent product is NDesk.Options by Jonathan Pryor. His single Options.cs file has been implemented as Mono.Options.

To demonstrate how easy command line parsing can be, I’m going to show you is part of the source for RoundhousE DB Migrations. RoundhousE has 2 required arguments and 22 total arguments. So I needed something that would make that super easy. And the ramp up time had to be less than 10 minutes. Mono.Options wins on both of these requirements.

Setting Your Command Line Options

OptionSet option_set = new OptionSet()
    .Add("?|help|h", "Prints out the options.", option => help = option != null)
    .Add("d=|db=|database=|databasename=",
       "REQUIRED: DatabaseName - The database you want to create/migrate.",
       option => configuration.DatabaseName = option)
    .Add("f=|files=|sqlfilesdirectory=",
       "REQUIRED: SqlFilesDirectory - The directory where your SQL scripts are.",
       option => configuration.SqlFilesDirectory = option)
    .Add("s=|server=|servername=|instance=|instancename=",
       string.Format("ServerName - The server and instance you would like to run on. (local) and (local)\\SQL2008 are both valid values. Defaults to \"{0}\".",
           ApplicationParameters.default_server_name),
       option => configuration.ServerName = option)
;

I am setting up the arguments here and what they will be assigned to. The first item in .Add is the command line argument (parameter). For more than one, you add a pipe “|”. Jonathan Pryor mentions in his “So you want to parse a command line…” post that parameters are passed following ways:

• Parameters of the form: -flag, --flag, /flag, -flag=value, --flag=value, /flag=value, -flag:value, --flag:value, /flag:value, -flag value, --flag value, /flag value.
• "boolean" parameters of the form: -flag, --flag, and /flag. Boolean parameters can have a `+' or `-' appended to explicitly enable or disable the flag (in the same fashion as mcs -debug+). For boolean callbacks, the provided value is non-null for enabled, and null for disabled.
• "value" parameters with a required value (append `=' to the option name) or an optional value (append `:' to the option name). The option value can either be in the current option (--opt=value) or in the following parameter (--opt value). The actual value is provided as the parameter to the callback delegate, unless it's (1) optional and (2) missing, in which case null is passed.

The second item in .Add is description. When you want to provide information about the argument, put a nice description in here. If you take a look at my last .Add, I have set the description to “ServerName – The server and instance you would like to run on. (local) and (local)\SQL2008 are both valid values. Defaults to ‘(local)’.” It gives enough information to allow someone to understand what is going on. It also lets you know that there is a default setting.

The third item in .Add is what you want the argument set to when found. With the first .Add after help I am setting configuration.DatabaseName to the argument’s (option) value.

Parsing the Command Line

try
{
    option_set.Parse(args);
}
catch (OptionException)
{
    show_help("Error - usage is:", option_set);
}

This is how easy it is to parse the command line. Once your OptionSet is ready, all you need to do is call Parse and pass it the arguments.

Showing Help

if (help)
{
    const string usage_message = 
        "rh.exe /d[atabase] VALUE /[sql]f[ilesdirectory] VALUE " +
        "[ /s[ervername] VALUE ]";
    show_help(usage_message, option_set);
}

If the user asks for help, you set help to true. Then you set up a usage message and pass that to a function that shows both your message and the descriptions of all the arguments.

Checking For Required Items

if (configuration.DatabaseName == null)
{
    show_help("Error: You must specify Database Name (/d).", option_set);
}

When you have required items, you can check to see if the user has sent in an argument properly. Then if not, you send them to the show_help function with a message about a necessary argument not being set.

Show Help Function

public static void show_help(string message, OptionSet option_set)
{
    Console.Error.WriteLine(message);
    option_set.WriteOptionDescriptions(Console.Error);
    Environment.Exit(-1);
}

In show_help you first show the message, then you write out the options and their descriptions.

That’s all great, but what does it look actually like?  Let’s run “rh /?” and see what we get for output.

rh.exe /d[atabase] VALUE /[sql]f[ilesdirectory] VALUE [ /s[ervername] VALUE ]
  -?, --help, -h             Prints out the options.
  -d, --db, --database, --databasename=VALUE
      REQUIRED: DatabaseName - The database you want to create/migrate.
  -f, --files, --sqlfilesdirectory=VALUE
      REQUIRED: SqlFilesDirectory - The directory where your SQL scripts are.
  -s, --server, --servername, --instance, --instancename=VALUE
      ServerName - The server and instance you would like to run on. (local) and (local)\SQL2008 are both valid values. Defaults to "(local)".

Conclusion

Mono.Options definitely wins the Rock Star Tool Award*! Mono.Options makes what would be a nightmare to parse a joy and accomplished without a lot of work. If you have more than one argument to parse with your command line, I would definitely give it a look.

* The rock star tool award means a tool is very powerful but takes less than 10 minutes to learn how to use.

Note: I am always on the lookout for rock star tools that make it easier and more enjoyable to accomplish things in development that need to be done. If you have a rock star that you want me to look at, please feel free to contact me.

public class DevliciousBlog
{
    IBlogger _blogger;

    public Devlicious(IBlogger blogger)
    {
        _blogger = blogger;
    }
}

public class Program 
{
    private static void Main(string[] args)
    {
        IBlogger rob = new FerventCoder();
        DevliciousBlog devlicious = new DevliciousBlog(rob);
        //awesome follows
    }
}

This is not goodbye by any means. Just another opening in my life. I was recently accepted to blog at Devlicio.us. Devlicio.us is a great community and is full of some top notch folks and I look forward to the opportunity to contribute!

GWB has been and continues to be a great community and I plan to continue posting here as well. I will be cross-posting most content here and there. For those of you who subscribe, that will continue to point to here at FerventCoder.com.

Now back to your regularly scheduled programming…

Versioning your database is as important as versioning your code.  How much time do we waste currently before we are even productive when we have to fix or enhance something in a database?

RoundhousE versions your database how you want. The recommended way is to version based on source control, but that may not be your solution. But for a moment let’s pretend it is.

Why would versioning your database based on source control be a big help? What happens if you get a prod fix (or work order) to fix a view or a stored procedure (sproc)?  How do you know where in source control that is? Do you even know what repository it lives in? How can you even be sure right away that the item you have in source control directly maps to the item in production? Oh right. You can’t. At least not without doing a manual comparison (or with a diff tool).  And if you are not even familiar with the item, how do you get back to it in source control? You ask someone. And then someone else. And so on until you find someone that knows where you can find that view in source control. “Oh….Samsonite! I was WAYYYY off.”

How much time are we using up currently before we are even productive? Probably quite a bit more than we realize. Let’s put it in perspective. You are a new hire today. If I was to point you to a database and tell you to fix a view and update the view in source control, how soon do you think you could be productive? What questions are you going to ask me?

Where is the view in source control? When is this needed by? etc etc

If your company instead used RoundhousE (RH) you could empower the new hire or any developer that suffers from can’t remember stuff (CRS)! Just point them to the database. Let’s take a moment to see what happens as we run the migration.

RoundhousE Versions During Migration

Remember our look at the runner? I want to concentrate in just one area.

Attempting to resolve version from C:\code\roundhouse\code_drop\deployment\_BuildInfo.xml using //buildInfo/version.
Found version 0.0.0.67 from C:\code\roundhouse\code_drop\deployment\_BuildInfo.xml.
Migrating TestRoundhousE from version 0 to 0.0.0.67.
Versioning TestRoundhousE database with version 0.0.0.67 based on http://roundhouse.googlecode.com/svn.

What exactly is happening here? We are looking into an xml file that contains a version.  RH also asks for the repository the versioning information comes from.  That all gets recorded in the database.

What does that build file look like?

<?xml version="1.0" ?>

<buildInfo>

  <projectName>RoundhousE</projectName>

  <companyName>FerventCoder Software</companyName>

  <versionMajor>0</versionMajor>

  <versionMinor>0</versionMinor>

  <buildNumber>0</buildNumber>

  <revision>25</revision>

  <version>1.0.22.25</version>

  <repositoryPath>http://someotherplace/svn/repositoryname/</repositoryPath>

  <microsoftNetFramework>net-3.5</microsoftNetFramework>

  <msbuildConfiguration>Release</msbuildConfiguration>

  <msbuildPlatform>Any CPU</msbuildPlatform>

  <builtWith>UppercuT v. 0.9.0.216</builtWith>

</buildInfo>

It is the output of an automated build product called UppercuT. Some of you have heard me talk about it before. It records the version in an xml output file to be used by deployment products etc.

Now I have a way to find the query at an exact point in source control history!

What if I don’t use UppercuT? That’s fine, create something in your automated build to create a simple xml file that contains a version. Then pass the file path and xpath for the version info to RH.

Or another way you can get to version is to point to a DLL you have versioned based on source control.

Attempting to resolve assembly file version from C:\code\roundhouse\code_drop\RoundhousE\NAnt\roundhouse.dll.
Migrating TestRoundhousE from version 0.0.0.67 to 0.0.0.67.
Versioning TestRoundhousE database with version 0.0.0.67 based on http://roundhouse.googlecode.com/svn.

RH will automatically pull the file version from the DLL and use it to version the database.

Back To Your Empowerment

So you query the scripts ran table for the view. You notice version_id is 1.

Now you query the version table.

Boom. You have a path to a repository. You also have an exact point in source to find the item.  You can now look revision 67 versus the current revision in source control.

You now know exactly where to look. Two steps to productivity. You get that production fix done in no time and we are so impressed we give you a raise on the spot.

One Repository – Multiple Databases

That’s great, but I hit multiple databases when I deploy. Easy as pie. Do you remember last time how we said to always include the database name before the scripts (see the picture to the right)? For each database, it is just another call to RH to run telling it the specific information about the next database. Super simple.

Multiple Repositories – One Database

What if I have multiple repositories that address the same database? We got you covered. If you have more than one repository versioning a database, they will version the database independently.

Found version 1.0.22.25 from C:\code\roundhouse\code_drop\deployment\_BuildInfo.xml.
Migrating TestRoundhousE from version 0 to 1.0.22.25.
Versioning TestRoundhousE database with version 1.0.22.25 based on http://someotherplace/svn/repositoryname/.

Notice how RH said the prior version was 0. That’s because versioning is based on repository. That repository had never ran against this database before. Now looking in the version table, we see two different repositories with two different versions.

Conclusion

Versioning your database is as important as versioning your code. RoundhousE is a very powerful product for migrating your database (rivaling even some paid alternatives). And it’s free. RoundhousE is just going to keep getting better. So why not give it a try? DBAs approve. It makes auditors smile. I heard it once helped a lady walk again.

I’m a huge fan of NHibernate. It has excellent documentation and just makes the whole job of getting things into and out of the database much more enjoyable.  There is a whole series of posts on NHibernate from one of the committers, Ayende.  When Fluent NHibernate (FNH) came out, it was like butter on sliced bread.  FNH makes it even easier to use NHibernate.

Ayende had a post a while back on how to use Event Listeners.  That helped me to get to event listeners set up.

Event Listeners

namespace somethingimportant.we.hope.infrastructure.app.auditing

{

    using System;

    using System.Security.Principal;

    using System.Web;

    using domain;

    using NHibernate.Event;

    using NHibernate.Persister.Entity;

    public class AuditEventListener : IPreInsertEventListener, IPreUpdateEventListener

    {

        public string get_identity()

        {

            string identity_of_updater = WindowsIdentity.GetCurrent().Name;

        if (HttpContext.Current != null)

            {

                try

                {

                    identity_of_updater = HttpContext.Current.User.Identity.Name;

                }

                catch

                {

                    //move on

                }

            }

            return identity_of_updater;

        }

        //http://ayende.com/Blog/archive/2009/04/29/nhibernate-ipreupdateeventlistener-amp-ipreinserteventlistener.aspx

        public bool OnPreInsert(PreInsertEvent eventItem)

        {

            Auditable audit = eventItem.Entity as Auditable;

            if (audit == null)

            {

                return false;

            }

            DateTime? entered_date = DateTime.Now;

            DateTime? modified_date = DateTime.Now;

            string identity_of_updater = get_identity();

            store(eventItem.Persister, eventItem.State, "entered_date", entered_date);

            store(eventItem.Persister, eventItem.State, "modified_date", modified_date);

            store(eventItem.Persister, eventItem.State, "updating_user", identity_of_updater);

            audit.entered_date = entered_date;

            audit.modified_date = modified_date;

            audit.updating_user = identity_of_updater;

            return false;

        }

        public bool OnPreUpdate(PreUpdateEvent eventItem)

        {

            Auditable audit = eventItem.Entity as Auditable;

            if (audit == null)

            {

                return false;

            }

            DateTime? modified_date = DateTime.Now;

            string identity_of_updater = get_identity();

            store(eventItem.Persister, eventItem.State, "modified_date", modified_date);

            store(eventItem.Persister, eventItem.State, "updating_user", identity_of_updater);

            audit.modified_date = modified_date;

            audit.updating_user = identity_of_updater;

            //insert auditing object here

            return false;

        }

        public void store(IEntityPersister persister, object[] state, string property_name, object value)

        {

            int index = Array.IndexOf(persister.PropertyNames, property_name);

            if (index == -1)

            {

                return;

            }

            state[index] = value;

        }

    }

}

NHibernate.Cfg.Configuration cfg = new NHibernate.Cfg.Configuration();
cfg.EventListeners.PreUpdateEventListeners = 

    new IPreUpdateEventListener[] {new AuditEventListener()};

cfg.EventListeners.PreInsertEventListeners =

    new IPreInsertEventListener[] { new AuditEventListener() };

_sessionFactory = cfg.BuildSessionFactory();

private void build_factory()

{

    if (nhibernate_session_factory == null)

    {

        nhibernate_session_factory = Fluently.Configure()

            .Database(MsSqlConfiguration.MsSql2005

                .ConnectionString(c => 

                    c.FromConnectionStringWithKey("db")))

            .Mappings(m => 

                m.FluentMappings.AddFromAssemblyOf<VideoMapping>())

            .ExposeConfiguration(cfg => {

                        cfg.EventListeners.PreInsertEventListeners = 

                            new IPreInsertEventListener[] {new AuditEventListener()};

                        cfg.EventListeners.PreUpdateEventListeners = 

                            new IPreUpdateEventListener[] {new AuditEventListener()};

                        })

            .BuildSessionFactory();

    }

}

This same thing can be accomplished with VB.NET by making a call to a method (I’ve done that as well). With it’s discoverability, FNH makes it so easy to figure things out sometimes. :D

This week I introduced RoundhousE and it’s getting quite a stir. Let’s take a look at the configuration. We are going to focus on the MSBuild/NAnt task configuration for now because the console isn’t yet complete. The console will actually have at least this much if not more (we’ve talked about how to use it to create the migration scripts and not just run them).  Because we like to be able to infer things, we have determined that there are only two required items. Thus – the minimal configuration:

Minimal Configuration

<!-- basic functionality, these are the required items-->

<roundhouse

  databaseName="YOURDATABASENAME"

  sqlFilesDirectory="..\..\db"

  />

 This is the minimal configuration that you would need to run RoundhousE (RH for the remainder of this post). This will not version your table unless you have a _BuildInfo.xml file sitting next to your deployment.bat files. You can get that if you use UppercuT to do your automated builds.

When you use the minimal configuration, it is assumed that you are running migration on the default instance of the server/computer you are on at that time.

databaseName is the database you want to create/migrate. Yes, RH will create a database if there is not one already. There is no need to tell it you want it to create or just update. RH infers that you want to do both if there is no database.
sqlFilesDirectory is where your SQL Scripts are. We will get to this in a later post, but basically, in the folder that has your sql scripts, RH is going to look for these folders (in this order) and run scripts in them: up, runFirstAfterUp, functions, views, sprocs, and permissions. In the up folder you put only the files you want to run one time. These are DDL/DML scripts. Once you run them, don’t change them or RH will shut down execution with an error (configurable to a warning).  The other folders are items that should run every time. Right now these items should be idempotent scripts, in other words, written to be run again and again without issue. The sample that comes with with RH shows this behavior.

FUTURE ENHANCEMENT: RH will either drop and recreate these scripts (capturing and restoring permissions) or create the alter script mentality by reading through your script.

Full Configuration

<!-- FULL configuration, for changing conventions -->

<roundhouse

  serverName="(local)"

  databaseName="YOURDATABASENAME"

  sqlFilesDirectory="..\..\db"

  repositoryPath="http://tellmewherethisis.com/svn/"

  versionFile="_BuildInfo.xml"

  versionXPath="//buildInfo/version"

  upFolderName="up"

  downFolderName="down"

  runFirstAfterUpFolderName="runFirstAfterUp"

  functionsFolderName="functions"

  viewsFolderName="views"

  sprocsFolderName="sprocs"

  permissionsFolderName="permissions"

  schemaName="RoundhousE"

  versionTableName="Version"

  scriptsRunTableName="ScriptsRun"

  environmentName="LOCAL"

  restore="false"

  restoreFromPath="\\tell\me\where\YOURDATABASENAME.bak"

  outputPath="C:\RoundhousE_runs"

  warnOnOneTimeScriptChanges="false"

  nonInteractive="false"

  databaseType="roundhouse.databases.sqlserver2008.SqlServerDatabase, roundhouse.databases.sqlserver2008"

  />

This is quite a configuration. It allows for you to do just about everything you would want to do with RH. Let’s get cracking through shall we?

serverName is the server and the instance you would like to go to. (local) and (local)\MSSQL2000 are both valid values.
databaseName we talked about above.
sqlFilesDirectory we talked about above as well. Next.
repositoryPath is literally that. It can be any value because it is only recorded in the version table. Once set it is not recommended you change this value.
versionFile is either an Xml file or a DLL that RH can grab the version from.
versionXPath is what you supply if you supply an Xml File. RH has to know where to go to get the value.
upFolderName is the name of the folder for update scripts. Call it whatever you want, we don’t care. Just tell us here.
downFolderName is for downgrading scripts. This is a future enhancement to RH.

FUTURE ENHANCEMENT: RH will be able to downgrade a database to a particular version.

runFirstAfterUpFolderName is where you put functions, views, sprocs, or permissions that are order dependent. RH has a certain order it runs scripts in based on both folder names and names of scripts. If a view depends on another view and you are not able to get it named appropriately to run before the other view, put it here. If you have a function that depends on a view, you definitely need the view in this folder.
functionsFolderName is for functions. By the way, all of these folders are recursively walked, so if you want to further separate by folders, that’s great, too!
viewsFolderName is for views.
sprocsFolderName is for….ya, you’re figuring it out.
permissionsFolderName. I’m not sure I need to say anything here if you are still with me at this point. On to the cooler settings…
schemaName is for when you want to have the RH tables to be in a different schema. If you put in YOURCOMPANY, you will get a different schema. Once set, do not change this. This is definitely running with scissors and very sharp.
 

 
versionTableName allows you to name the version table something else if you don’t like the name of it or it conflicts with something else. Once set, do not change this.
scriptsRunTableName allows you to change the name of the scripts run table for the same reasons as versionTableName. Once set, do not change this. And when we say once set, it may mean the first time you go to prod and can no longer drop and restore databases on a whim.  These are very sharp to allow you to have flexibility, but because it is a knife you can still get cut if you use it wrong. I’m just saying. You’ve been warned.
environmentName allows you to have certain scripts that only run in certain environments. This is very useful for say the permissions scripts.

FUTURE ENHANCEMENT: Looking at Environments in the script names to determine whether to run or not.

restore tells RH whether to restore a database before migrating. This becomes handy when the DDL/DML scripts are in flux and/or once you’ve gone to production and are doing maintenance. There is an example of how you would do this in the sample.
restoreFromPath tells RH where to get the backed up database. Neat convention: If you use SQLServer and Litespeed, back up your database and put LS at the end of the file name (like TimmyLS.bak) and RH will convert over to do a Litespeed restore.
outputPath is where you want RH to set everything it ran in the migration. For more information see this enhancement: http://code.google.com/p/roundhouse/issues/detail?id=4

FUTURE ENHANCEMENT: If the enhancement request is still open, this is not done yet. Did I mention it will also zip the files up? http://code.google.com/p/roundhouse/issues/detail?id=5

FUTURE ENHANCEMENT: This may be where the backup is located as well: http://code.google.com/p/roundhouse/issues/detail?id=3

warnOnOneTimeScriptChanges is how you can turn off the error that occurs when RH sees a change to a one time script (DDL/DML). It is recommended that you never change the scripts that should only run once, but if you do, you are going to need to set this to true.
nonInteractive tells RH not to ask for user input when it runs. This is how you can set up scheduled jobs to run without you needing to be around.
databaseType is the type of database you are migrating. This is a bit of a misnomer. If you have 2008 installed, you can still deploy to a 2005 database. But it doesn’t work the other way around.  Here are the settings:

- roundhouse.databases.sqlserver2008.SqlServerDatabase, roundhouse.databases.sqlserver2008

- roundhouse.databases.sqlserver2005.SqlServerDatabase, roundhouse.databases.sqlserver2005

- roundhouse.databases.oracle.OracleDatabase, roundhouse.databases.oracle (FUTURE ENHANCEMENT)

- roundhouse.databases.mysql.MySqlDatabase, roundhouse.databases.mysql (FUTURE ENHANCEMENT)

NOTE: databaseType is a plug in model, so if you have a type that is not officially supported yet, you can write your own. Just reference roundhouse.dll and implement roundhouse.sql.Database (it’s an interface).  Then just make sure your DLL sits next to roundhouse.dll, edit databaseType to point to your assembly.

This is a lot to take in, so I’ll stop here. Oh wait - that was all of it. I’ll stop here because we’re done.  If this hasn’t intrigued you to at least download RH and take a look at it, perhaps the versioning aspect will when we talk about it. There is nothing better than knowing what revision in source to look at to find a problem with a stored procedure or a view.

Next up: A closer look at the versioning aspect and how you can version your database based on what you have in source control - RoundhousE : Version Your Database

I just started the appetites with my last post on RoundhousE. RoundhousE itself has a migration sample to help you get familiar with how it works. You can get to that by using a subversion client to download the source. When you download the RoundhousE source from SVN and run build.bat, you get a code_drop folder. Go in there and look at the deployment folder for a sample run.

Prerequisites: Locally installed (and running) SQL Server 2008

Run That Bad Boy

When you get into the code_drop folder, take a look below at the screenshot. Run LOCAL.DBDeployment.bat. It will run the through a NAnt deploy of RoundhousE and create a database called TestRoundhousE.

When you run LOCAL.DBDeployment.bat, this is the output (provided you have SQL Server 2008 installed):

Running RoundhousE on (local) (TestRoundhousE). Looking in C:\code\roundhouse\code_drop\deployment\scripts\..\..\db\TestRoundhousE for sql scripts.
Executing RoundhousE against contents of C:\code\roundhouse\code_drop\deployment\scripts\..\..\db\TestRoundhousE.
Creating TestRoundhousE database on (local) server if it doesn't exist.
Creating RoundhousE schema if it doesn't exist.
Creating [RoundhousE].[Version] table if it doesn't exist.
Creating [RoundhousE].[ScriptsRun] table if it doesn't exist.
Attempting to resolve assembly file version from C:\code\roundhouse\code_drop\RoundhousE\NAnt\RoundhousE.dll.
Migrating TestRoundhousE from version 0 to 0.0.0.59.
Versioning TestRoundhousE database with version 0.0.0.59 based on http://roundhouse.googlecode.com/svn.
Running 0001_CreateTables.sql on (local) - TestRoundhousE.
Running 0002_ChangeTable.sql on (local) - TestRoundhousE.
Running vw_Dude.sql on (local) - TestRoundhousE.

Take a look here. It gives you feedback about where it is running and where it is looking for sql scripts.  Then it attempts to resolve the version based on a Dll’s version (can also do it through an xml file from the build containing a version).  It will create the database if it doesn’t exist (does not require action create, it’s smart about that).  Then RoundhousE will create it’s tables (one for tracking versions, and one for tracking what scripts have run).  Then it’s going to look through folders recursively and find sql files to run.

What happens when I run it a second time?

Running RoundhousE on (local) (TestRoundhousE). Looking in C:\code\roundhouse\code_drop\deployment\scripts\..\..\db\TestRoundhousE for sql scripts.
Executing RoundhousE against contents of C:\code\roundhouse\code_drop\deployment\scripts\..\..\db\TestRoundhousE.
Creating TestRoundhousE database on (local) server if it doesn't exist.
Creating RoundhousE schema if it doesn't exist.
Creating [RoundhousE].[Version] table if it doesn't exist.
Creating [RoundhousE].[ScriptsRun] table if it doesn't exist.
Attempting to resolve assembly file version from C:\code\roundhouse\code_drop\RoundhousE\NAnt\RoundhousE.dll.
Migrating TestRoundhousE from version 0.0.0.59 to 0.0.0.59.
Versioning TestRoundhousE database with version 0.0.0.59 based on http://roundhouse.googlecode.com/svn.
Skipped 0001_CreateTables.sql either due to being a one time script or finding no changes.
Skipped 0002_ChangeTable.sql either due to being a one time script or finding no changes.
Running vw_Dude.sql on (local) - TestRoundhousE.

The second time it runs it is only going to rerun items that are stateless, items that do not contain data, like functions, views and stored procedures.

FUTURE ENHANCEMENT: RoundhousE will only rerun stateless items if they have changed.  That will cut your migration time way down for databases that are heavy one the stored procedures side.

RoundhousE and SQL Server

What does that goodness give you? Let’s crack open SSMS (Sql Server Management Studio).

You get two tables (schemas and tables names are completely configurable): Version and ScriptsRun.

Looking a the version table, I have the idea of both a source repository and a version. What does that mean? It means RoundhousE knows that you may have multiple repositories that hit the database and that’s cool.

The ScriptsRun table here captures some really interesting information. There is script_name, the id of the version it is associated with, and then there is the text_of_script. Whoa…the text of the actual script I ran?! That’s awesome! But ….why?! We do this for two reasons. You now have an audit of what actually ran. This makes both DBAs and auditors happy. RoundhousE also has a goal in mind of versioning back down. This becomes extremely simple if we keep track of those things.

What happens if I start running bigger and bigger scripts? It’s a minimal impact. I tested this with a 15MB production level insert script for running an initial load of data and it ran like a champ. My database also only grew about 4MBish larger than the original database without recording the script. So the impact is minimal.

The next column we’ll focus on is one_time_script. This column is what tells you automatically whether this script is located in a one time run folder (the up folder in this case) or if it is a stateless item that will not cause data loss or error to run again and again.

Okay next column: text_hash – what is that? That is how RoundhousE can very quickly determine if there have been changes to the script. Which is a good transition into the next section.

RoundhousE is Smarter

What happens if someone changes a DDL/DML file that is meant to run only once? Add even one extra space in the file and this is what you get:

RoundhousE encountered an error:
System.Exception: 0001_CreateTables.sql has changed since the last time it was run. By default this is not allowed - scripts that run once should never change. To change this behavior to a warning, please set warnOnOneTimeScriptChanges to true and run again. Stopping execution.

It is configurable to set this to a warning, but the important thing to note is that RoundhousE is going to try to help your developers see that they made changes to something that won’t run. That way they are not making changes to files that should not change.

Go get the source and give it a whirl.

Next up – RoundhousE Configuration. Same bat time. Same bat channel.  And remember, “There is no charge for awesome.”

Right now my post on Sproc Executing Slow is the most popular topic on GWB!

One of the coolest projects I have been working on is how to solve issues with database versioning and migration. A project dubbed RoundhousE.

RoundhousE is an open source automated database deployment (change management) system that allows you to use your current idioms and gain much more. Currently it only supports Microsoft SQL Server, but there are future plans for other databases.

It seeks to solve both maintenance concerns and ease of deployment. We follow some of the same idioms as other database management systems (SQL scripts), but we are different in that we think about future maintenance concerns. We want to always apply certain scripts (anything stateless like functions, views, stored procedures, and permissions), so we don’t have to throw everything into our change scripts. This seeks to solves future source control concerns. How sweet is it when you can version the database according to your current source control version?

RoundhousE (RH) is a very lean implementation for database migration that uses SQL Scripts to apply changes to a database. It is extremely configurable in that you can override the conventions for everything, including the schema, version table, and scripts run table for RH. RoundhousE is implemented as an MSBuild/NAnt Task, a console application (the console is still in development), and in the future possibly a GUI as well. There is an extensive roadmap on the product. Over the next few weeks I will be talking about how to set up and use this product and future enhancements.

Until then, enjoy this image of Chuck showing off his intimate knowledge of open source!

Next in the spin: RoundhousE DB Migration: Action Shots

Iowa Code Camp was a great time! I really enjoyed the conference itself. It was probably one of the best conferences I have ever been to. There are a lot of talented people surrounding Iowa and I had the pleasure of hanging out with some very intelligent people that are doing some really awesome things.

The talk went well as well. Hopefully there were a few more people that got out there and tried UppercuT. Here are the slides from my talk.

So you have stored procedure and it’s having issues. When you run it, it returns really fast. When .NET runs it, its like a dog.

The first thing you learn about SQL Server is that it has something called ARITHABORT and it is turned on in SQL Management Studio and OFF in ADO.NET. Yeah you heard that right. OFF. So when you find this out, your first thought is WTF, mate?! Why do we have to make this hard on ourselves?

Reference post: http://sqladvice.com/blogs/gstark/archive/2008/02/12/Arithabort-Option-Effects-Stored-Procedure-Performance.aspx

So you turn it off in SQL Server Management Studio under {Tools}->{Options} and all is good right?

Well, you might be coming across another problem and you want to optimize the sproc so you pull the text out and start running query analyzer. First things first, compare the text of the sproc versus executing the actual sproc. You might be surprised to find that the actual execution is not as fast as the text.

WTF? Why is the sproc running slower than the actual text? This is due to a bad query plan being cached by SQL. If you use the parameters that are passed in by the sproc, SQL Server tries to guess what those are and it puts in fake values. Then it caches a bad query plan.

So if you’re still with me at this point, you’re probably saying, that’s all fine and dandy Rob, but I just want to fix the freaking thing and you are giving me too much context.

The fix is so easy, it’s crazy. All you have to do is set those parameters to local variables and use the local variables in the sproc.

Create PROC Tim 

    @StartDate DateTime

AS 

 BEGIN

    DECLARE @LocalStartDate DateTime

    SET @LocalStartDate = @StartDate

    /* your code here */

    SELECT @LocalStartDate AS StartDate

    /* instead of 

        SELECT @StartDate AS StartDate

     */

 END

It’s a hack. And only useful when the sproc is being a non-performant nancy boy.

And here’s another good reason not to use Sprocs. I’m just saying. YMMV.

If you are in Des Moines, IA, tomorrow for Iowa Code Camp, I’m speaking on “Automated Builds: How to UppercuT your Code” at 3:45.

If you are going to be there and we’ve only met on Twitter before, I’m looking forward to meeting you!

If you are using UppercuT, you will be pleased to know that it now supports Mono Migration Analyzer (MoMA for short).  All you have to do is upgrade. And with the design of UppercuT, we’ve made it super simple to upgrade.

How hard is it to upgrade UppercuT? Just drop in the files in your build folder, check for changes to the lib folder (especially in the NAnt directory) and new folders, and then check for any changes to the .bat files and UppercuT.config and you’re done.

Mono Migration Analyzer

Here is what the report looks like running against UppercuT.  It looks like UppercuT is okay to run on *nix, but a few of it’s dependencies may have some issues.

If you look closer, you can find whether the method is not implemented, on the TODO list, or the P/Invoke list. The P/Invoke’s will only work if your OS has implemented them.

Yet another reason to use UppercuT. The builds just keep getting better!

Here’s how you can sign a set of assemblies in a project with a key using UppercuT:

1. Open the UppercuT.config file.

2. Change the following to “true”:

3. Done.

Did I mention that creates a private key if it is not there as well?!

This was from a patch sent in by Dru Sellers. Thanks Dru!

With this knowledge you shall build.

The presentation for TopDNUG went pretty well. There were some good questions and back and forth. The room really seemed to come alive when in a matter of less than ten minutes I added UppercuT to two different Open Source projects (Reflexil and Quartz.NET) and had them off of the ground and building. Full builds, too. With versioning, compiling, unit testing, packaging, etc.

I also unveiled the new logo for UppercuT:

Here is the slide deck: UppercuT Presentation (may need to be renamed to .pptx).

Recently I reported that there were going to be some changes to UppercuT. And there have been. These are a summary of some of the most significant changes:

• Uppercut now reports it's version. This is helpful to know where you are versus the current version. It also reports the time when it finishes a build.
• Custom Replacement Tasks are now implemented. This is to add a task to the custom folder that completely replaces the normal workings of the build step it is replacing. A pre or post custom step will still run though.
• Pre, Post and Replacement tasks are implemented for every build step in UppercuT now.
• Support for Gallio Testing has been added.
• EnvironmentBuilder works better than ever now that it is custom code instead of NAnt property expansion which had buffer limitations.
• All .build files are now .step except for the actual builds (open, zip and default). default.build is what used to be __master.build.

http://code.google.com/p/uppercut/downloads/list

Most of this is due to prioritizing the time to get requested features into UppercuT prior to my upcoming presentation for TopDNUG.

There are some more changes on the way to v1. I will be talking about a road map soon.  If you aren’t yet a member of the uppercut users group, you might consider joining. :-)

With this knowledge you shall build.

So Gallio has been out for a little while and I admit that I am a little slow when it comes to looking at new frameworks. I mean there is so much to look at and only so much time in the day allocated to programming. Anyway, there really isn’t much documentation out there yet for using Gallio with NAnt. I am of the thought that is due to all of the people who are really smart with builds are using Rake and/or PSake now. So I set off to create another universal script. This basically follows the conventions from my post with MbUnit’s Universal script and how I added MbUnit2 category filters to it’s NAnt task in UppercuT.

So like I said, Gallio has been out for a little while. This is the first time I have heard that it has underwent some optimizations so now seems a good time to check it out. Until I figure out how to do it another way, this script requires Gallio to be installed on each machine that is going to use it. I am currently trying to figure out how I can get a reduced set of Gallio into source control. The install is 26.4MB right now and that’s bigger than I want my repositories to have to be just for adding a testing framework.

NAnt Script for Gallio

<?xml version="1.0" encoding="utf-8" ?>

<project name="GallioTestRunner" default="go">

  <!-- Project UppercuT - http://projectuppercut.org -->

  <!-- DO NOT EDIT THIS FILE - This follows a convention for testing with Integration tests being separated from Unit tests - find out more at http://uppercut.pbwiki.com -->

  <property name="build.config.settings" value="__NONE__" overwrite="false" />

  <include buildfile="${build.config.settings}" if="${file::exists(build.config.settings)}" />

  <property name="dirs.build" value="${directory::get-parent-directory(project::get-buildfile-path())}\..\..\build_output" />

  <property name="dirs.build_artifacts" value="${dirs.build}\build_artifacts" overwrite="false" />

  <property name="dirs.test_results" value="${dirs.build_artifacts}\gallio" overwrite="false" />

  <property name="file.test_results" value="gallio-results" overwrite="false" />

  <property name="time.limit.in.seconds" value="240" />  <!-- 4 minutes -->

  <target name="go" depends="cleanup, run_tests" description="Tests" />

  <target name="cleanup">

    <echo message="Removing and adding ${dirs.test_results}."/>

    <delete dir="${dirs.test_results}" failonerror="false" />

    <mkdir dir="${dirs.test_results}" />

  </target>

  <target name="load_tasks">

    <echo message="Loading Gallio Nant Tasks from Program Files." />

    <loadtasks assembly="C:\Program Files (x86)\Gallio\bin\Gallio.NAntTasks.dll" if="${file::exists('C:\Program Files (x86)\Gallio\bin\Gallio.NAntTasks.dll')}" />

    <loadtasks assembly="C:\Program Files\Gallio\bin\Gallio.NAntTasks.dll" if="${file::exists('C:\Program Files\Gallio\bin\Gallio.NAntTasks.dll')}" />

  </target>

  <target name="run_tests" depends="cleanup,load_tasks" description="Running Unit Tests">

    <echo message="Running tests using Gallio and putting results in ${dirs.test_results}."/>

    <gallio working-directory="${dirs.build}"

            report-types="Html;Xml;Text"

            report-directory="${dirs.test_results}"

            report-name-format="${file.test_results}"

            show-reports="false"

            failonerror="true"

            verbosity="Normal"

            echo-results="true"

            run-time-limit="${time.limit.in.seconds}"

            filter="exclude Category:Database or Category:Integration or Category:Slow or Category:NotWorking or Categroy:Ignore or Category:database or Category:integration or Category:slow or Category:notworking or Categroy:ignore"

            >

      <files>

        <exclude name="${dirs.build}\*Database*dll" />

        <exclude name="${dirs.build}\*.Integration*dll" />

        <exclude name="${dirs.build}\TestFu.dll" />

        <include name="${dirs.build}\*Test*dll" />

        <include name="${dirs.build}\*.Specs*dll" />

      </files>

    </gallio>

  </target>

  <target name="run_all_tests" depends="cleanup,load_tasks" description="Running All Unit Tests">

    <echo message="Running all tests (including integration tests) using Gallio and putting results in ${dirs.test_results}."/>

    <gallio working-directory="${dirs.build}"

            report-types="Html;Xml;Text"

            report-directory="${dirs.test_results}"

            report-name-format="${file.test_results}"

            show-reports="false"

            failonerror="true"

            verbosity="Normal"

            echo-results="true"

            run-time-limit="${time.limit.in.seconds}"

            >

      <files>

        <exclude name="${dirs.build}\TestFu.dll" />

        <include name="${dirs.build}\*Test*dll" />

        <include name="${dirs.build}\*.Specs*dll" />

      </files>

    </gallio>

  </target>

  <target name="open_results">

    <echo message="Opening results at ${path::get-full-path(dirs.test_results)}\${file.test_results}.html."/>

    <exec

      spawn="true"

      program="${environment::get-folder-path('ProgramFiles')}\Internet Explorer\iexplore.exe"

      commandline="${path::get-full-path(dirs.test_results)}\${file.test_results}.html"

      >

    </exec>

  </target>

</project>

UppercuT and Gallio

UppercuT now has support for Gallio baked in. What that means is that you select gallio as your test framework in the config file. Install Gallio. And you’re done. And when you run test from the command line, you get this in your browser:

What pretty reports you get from Gallio!

With this knowledge, you shall build.

I’m slimming it down and rethinking some of the idioms it is currently using. Stay tuned…