Lessons In Building An Email Parser

The Classic Infinite Email Loop

When building an email parser, one must think about validating an email address that sends you a message. Just because that email address is where the message says it is from does not really mean that is a valid email address.  Some of you can already see where I’m going with this.

I was building something in Bombali that would respond to emails when receiving them. When Bombali received a message from an address, Bombali would send a response. No checking on the address. Classic mistake right? Well new to me, I’ve never been able to do this programmatically until recently. So Bombali received a message with a no reply from address. It sent a response. Then it got a response from the postmaster saying that this was an invalid address. I bet you’re guessing what happens next? Yes, Bombali responded to the postmaster. And then the postmaster responded back. Over a matter of about 30 minutes there was quite a bit of traffic back and forth until I deleted the emails before Bombali could check them. At least I was smart enough to set the email checking to every half minute.

How did I get past this? Validation. An authorized list form of validation. Interested? Read on dear reader.

SidePOP XmlConfigurator

I was exposing my configuration in the last post, so I built an XmlConfigurator that does this for you and gives you back a list of Email Watchers (renamed SidePOPRunner).

private void configure_mail_watcher()
{
    EmailWatcherConfigurator configurator = new SidePopXmlConfigurator();
    foreach (EmailWatcher emailWatcher in configurator.configure())
    {
        emailWatcher.MessagesReceived += runner_messages_received;
        emailWatcher.start();
    }
}

Subscribing to Receiving Email

The code above and the code below show how easy it is for your application to receive email. I map the email to a local object and then I pass that off to a function to parse and send a response.

private void runner_messages_received(object sender, MessageListEventArgs e)
{
    IEnumerable<SidePOPMailMessage> messages = e.Messages;

    foreach (SidePOPMailMessage message in messages)
    {
        Email mail_message = Map.from(message).to<Email>();
        parse_and_send_response(mail_message);
    }
}

So now I am receiving email, but I need to parse it do determine what to do. So let’s show the parser first before the method for sending a response.

Really Simple Email Parser

This is a really simple parser. It’s not by any means what some people would use in production and that’s fine. It works for what Bombali needs.

public MailQueryType parse(Email message, IList<IMonitor> monitors, IDictionary<string, ApprovalType> authorization_dictionary)
{
    MailQueryType query_type = MailQueryType.Authorizing;

    string user = message.from_address.to_lower();
    bool user_is_authorized = false;

    if (authorization_dictionary.ContainsKey(user))
    {
        ApprovalType user_is_approved = authorization_dictionary[user];
        if (user_is_approved == ApprovalType.Approved)
        {
            user_is_authorized = true;
        }
    }

    if (user_is_authorized)
    {
        query_type = MailQueryType.Help;
        string subject_and_body = message.subject + "|" + message.message_body;

        if (message_contains_status(subject_and_body)) query_type = MailQueryType.Status;
        if (message_contains_config(subject_and_body)) query_type = MailQueryType.Configuration;
        if (message_contains_down(subject_and_body)) query_type = MailQueryType.CurrentDownItems;
        if (message_contains_approve(subject_and_body)) query_type = MailQueryType.Authorized;
        if (message_contains_deny(subject_and_body)) query_type = MailQueryType.Denied;
        if (message_contains_version(subject_and_body)) query_type = MailQueryType.Version;
    }

    return query_type;
}

I have a MailQueryType so that I separate what how to respond from actually responding. What you can see is that I’m already checking to see if we have an approved user before I set a different MailQueryType. The mail parser just sends back how Bombali should respond to the message.

Sending a Response

In subscribing to receiving email, I passed the message to the method below. The second line of the method shows me passing the message off to the mail parser to find out how to respond.  The switch could probably be replaced by a Strategy pattern, but for now it works and it’s all I need.

private void parse_and_send_response(Email mail_message)
{
    string respond_to = mail_message.from_address.to_lower();
    MailQueryType query_type = mail_processor.parse(mail_message, monitors, authorization_dictionary);
    Log.bound_to(this).Info("{0} received a message from {1} of type {2}.", ApplicationParameters.name, respond_to, query_type.ToString());

    string response_text = string.Empty;

    if (query_type == MailQueryType.Authorized || query_type == MailQueryType.Denied)
    {
        string[] body_words = mail_message.message_body.Split(' ');
        foreach (string body_word in body_words)
        {
            if (body_word.Contains("@"))
            {
                respond_to = body_word.Replace(Environment.NewLine, "");
                break;
            }
        }
    }

    switch (query_type)
    {
        case MailQueryType.Denied:
            authorization_dictionary.Add(respond_to, ApprovalType.Denied);
            return;
            break;
        case MailQueryType.Authorized:
            authorization_dictionary.Add(respond_to, ApprovalType.Approved);
            response_text = string.Format("Congratulations - you have been approved!{0}Send 'help' for options", Environment.NewLine);
            break;
        case MailQueryType.Help:
            response_text =
                string.Format(
                    "Options - send one:{0} help - this menu{0} status - up time{0} config - all monitors{0} down - current monitors in error{0} version - current version",
                    Environment.NewLine);
            break;
        case MailQueryType.Status:
            TimeSpan up_time_current = up_time.Elapsed;
            response_text = string.Format("{0} has been up and running for {1} days {2} hours {3} minutes and {4} seconds.", ApplicationParameters.name,
                                          up_time_current.Days, up_time_current.Hours, up_time_current.Minutes, up_time_current.Seconds);
            break;
        case MailQueryType.CurrentDownItems:
            response_text = string.Format("Services currently down:{0}", Environment.NewLine);
            foreach (IMonitor monitor in monitors)
            {
                if (monitor.who_to_notify_as_comma_separated_list.to_lower().Contains(respond_to))
                {
                    if (!monitor.status_is_good) response_text += string.Format("{0}{1}", monitor.name, Environment.NewLine);
                }
            }
            break;
        case MailQueryType.Authorizing:
            response_text =
                string.Format("Bombali notified admin to add you to approved list. If you are added, you will receive a response.");
            break;
        case MailQueryType.Version:
            response_text = string.Format("Bombali is currently running version {0}.", Version.get_version());
            break;
        default:
            response_text = string.Format("{0} has not been implemented yet. Please watch for updates.", query_type);
            break;
    }

    SendNotification
        .from(BombaliConfiguration.settings.email_from)
        .to(respond_to)
        .with_subject("Bombali Response")
        .with_message(response_text)
        .and_use_notification_host(BombaliConfiguration.settings.smtp_host);

    if (query_type == MailQueryType.Authorizing)
    {
        SendNotification
        .from(BombaliConfiguration.settings.email_from)
        .to(BombaliConfiguration.settings.administrator_email)
        .with_subject("Bombali Request")
        .with_message(string.Format("{0} reqests approval. Send approve/deny w/email address. Ex. 'deny bob@nowhere.com'", respond_to))
        .and_use_notification_host(BombaliConfiguration.settings.smtp_host);
    }

}

You’ll notice that I’m authorizing any new email addresses that send an email to Bombali before allowing them access to the application. I do that by having an administrator contact I can send a request for approve or deny to before allowing access. I also send an email back to the sender letting them know they are in the process of being approved. At most I would send one email to a bad address. As administrator, the email can be validated by a human. And I only respond to an authorized list. I could also build in something to validate the email by checking for a response from a post master telling me the address doesn’t exist.

So remember readers - always check your sources. Otherwise you might get in an infinite loop. And that would be bad.

kick it on DotNetKicks.com Shout it

Print | posted @ Friday, December 18, 2009 3:15 AM

Comments on this entry:

Comments are closed.

Comments have been closed on this topic.