Fervent Coder

I just recently started using BareTail (which is a free Tail for Windows) and I have to say it is fantastic at what it does.  What is a tail? In Unix it is used to mean the end of the log file.  The follow (-f) option translates into staying with the bottom, or tail end, of the file.

BareTail is a Log File Monitoring tool that you can leave running all the time and it will refresh file changes without ever locking the log files. It has a tab for every log file you would like to monitor and it can be set up for files that don't yet exist; it will start monitoring the file once it is created. 

From the Product Site

· Real-time file viewing

· Follow tail mode

· Tail multiple files

· Configurable highlighting

· International character sets

· Many file formats

· Flexible configuration options and storage

· Single small executable, no installer

Thoughts On BareTail

Did I mention that FREE is one of my favorite 4 letter F words?! There are other tools out there like this that don't perform as well as this tool and they cost much more.  Now being said that it is a free tool, if you find the tool useful, you might think about contributing back to the person that took the time to help you be more productive.  You can register the product for $25.

Three really good ideas are incorporated into this product.

1. No installer required.  Put it where you want and run it. That's awesome.
2. As Dru says, the best tools get out of your way.  You can leave BareTail running all day and it has a very small footprint.
3. You can script out the logs you want it to watch by setting up a batch file with the path to each log file after the filename.

Easily Script Out Paths to Log Files

If you call baretail.exe on a command line, you can give it paths to log files as parameters. Instead of doing that every time, you can script out paths in batch files. Below is a simple example of calling baretail.exe with paths to three log files on the same server.

SET SERVER=TEST

SET DIR=\Logs\Applications\

START baretail.exe \\%SERVER%%DIR%Application1.Console.log \\%SERVER%%DIR%Application2.Console.log \\%SERVER%%DIR%Application3.log

Notice that I put the server variable as the first line.  For each environment, my paths are exactly the same, so I only have to change the server variable.  Right now I am watching four log files for a set of applications we are creating.  I have four environments and we are in three of them currently. I name the batch files like this: baretail_appName_Environment.bat.

Here is a more advanced sample, where you get log files for your web server from today and tomorrow (UTC anyone?). I started with a batch file but found it to be too flaky on date math and turned to VBS, which handled this very easily.

Option Explicit

Function GetLogName(theDate)

    Dim strYear

    Dim strMonth

    Dim strDay

    strYear = Mid(Cstr(Year(theDate)),3)

    strMonth = CStr(Month(theDate))

    strDay = CStr(Day(theDate))

    if Len(strMonth) = 1 then

        strMonth="0" & strMonth

    end if

    if Len(strDay) = 1 then

        strDay="0" & strDay

    end if

    GetLogName="ex" & strYear & strMonth & StrDay & ".log"

End Function

Dim server

Dim directory

Dim application

server ="serverName"

directory ="\\" & server & "\c$\WINNT\system32\LogFiles\W3SVC1\"

application = "baretail.exe"

Dim args

args = ""

Dim today

today = DATE

args = args & directory & GetLogName(today) & " "

Dim tomorrow

tomorrow = DATEADD("d",1,today)

args = args & directory & GetLogName(tomorrow) & " "

Dim WshShell

Set WshShell = WScript.CreateObject("WScript.Shell")

WshShell.Run application & " " & args, 1, False

Set WshShell = Nothing

WScript.Quit

When you run the script you get log files for today and tomorrow. Why would I want tomorrow's date for a log file? Tomorrow because we leave IIS log files to the default UTC time which means it has created a new log file for the next day around 7PM.  If the file doesn't yet exist, there will be a red X instead of a green dot.  That will go away once the file is created and you will be able to watch the logs.

When there are changes to a file, the green arrow show up.

Launchy and BareTail - A Great Combination

Having the batch files available, you can hook it up with Launchy and be even more productive.  I put the BareTail application and batch files in a self-created Launchy applications folder. Then when I want to run baretail, I hit the Hotkey for Launchy (default is Alt+Space), type baretail_ and pick the one from the drop down I want by pressing {Down Arrow} and hitting {Enter}.

BareTail - A Great Product

If you find yourself looking at log files, I would say that you will find this product will save you quite a bit of time (between opening and closing and reopening log files to see the changes). I am finding it to be a very useful tool because I am looking at log files quite a bit.  I started using it for watching log files that I never thought to monitor before because it makes it so easy to do now.  BareTail is a delightfully small product that is free and does one simple thing very well, which is exactly how I like my tools.

Get it here: http://www.baremetalsoft.com/baretail/

Scott C Reynolds (no known relation) wrote an excellent post a few weeks ago that is definitely worth reading on Why Alt.NET.  He talks about the us vs. them and the reasoning that might be behind it.

I will forgo the use of labels that may be seen as derisive and divisive, and state my observation from my career that developers fall into two very broad camps: those who at least lurk but maybe participate in "community" activities (blogs, conferences, mailing lists, etc) and those who, for whatever reason, do not. I will call the former Group A, and the latter Group B. Group A may or may not be doing things right. They may or may not be up on the latest tools or methods, but they are out there trying to learn. Group B, contrary to popular may still be out there trying to learn. Those guys just do it primarily through books, magazines, Google, the MSDN library, and their immediate coworkers. Just because a developer hasn't plugged into a community does not mean he doesn't care.

I bring this up because the Godfather of Alt.NET (Dave Laribee) commented with:

The Group A & B thing. Again, reach is part of it. Education is part of it. But we *must* keep a kernel of innovation going. That kernel should have an open door policy. Meritocracy and first mover advantage over elitism.
A number of ideas (street kit, reach, topical OSTs, enhancements to social networks) have been pitched, but participation remains appallingly low. There's a few people making things happen and we need to expand that if we want to take it to the next level. Hope won't get us there.
Action. Organization. Participation. Engagement. It's very hard to eek out the time, but if people do believe in making progress and innovation happen, time must be made.

Dave was really starting to get to the idea of what I am presenting here.  It takes time. It takes time?!  Why is that?

At lunch today we were talking about how a couple has been married for over 50 years and a family member recorded their marriage ceremony. Fifty years ago! Camcorders were not in wide stream acceptance by the general public at that time, plus they were really expensive.  That means that the member of the family was an early adopter because although camcorders were brought about over 60 years ago, it took a long time to become mainstream and widely accepted.  When VCRs came out and people could see that, they really started to adopt the use of the camcorder.  The idea of the camcorder had to be around for many years before people accepted/adopted it.

It seems today that the acceptance cycle is becoming faster and faster, but this is not central to my point.  The point is that there is an adoption (or acceptance) cycle and we must acknowledge that it applies to the idea of Alt.NET.

I think that all adoption comes down to how a culture adopts technology.  When I was in college, I took some courses on leadership, and one of the things we talked about was the technology adoption lifecycle, also known as Diffusions of Innovation.  Basically diffusion is how long it takes an idea or product to be accepted by the market.

The picture below is from Wikipedia under the GNU Free Document License.  The picture links to the original.

I am going to use the definition for what each group is from the business idea of diffusion:

• Innovators – venturesome, educated, multiple info sources;
• Early adopters – social leaders, popular, educated;
• Early majority – deliberate, many informal social contacts;
• Late majority – skeptical, traditional;
• Laggards – neighbors and friends are main info sources, fear of debt.

What the diagram does not tell you is how long it takes to get to each point of the diffusion cycle.  You want to get to the early and much of the late majority before your idea/technology is widely accepted.  So how much time does that take?  The answer here is that it is hard to tell, but I don't believe that we are yet getting to the point of a 1 year cycle, even a 2 or 5 year cycle.  So this means we have to allow for time, as Dave was talking about.

We are a relatively young culture in .NET so many of the adopters of .NET are just getting used to what Microsoft is ramm-- *ahem* asking us to accept as the best approach.  They are relatively ignoring many of the tools available out there that may be great and many people feel are better.  Many of the innovators and early adopters of Alt.NET came into .NET early on and have tried the Microsoft way for awhile and have since started looking and finding (or creating) new innovations/better ways to do things.  It's kind of the "Been There, Done that, Got the T-shirt, What's Next?" approach.  Many of the innovators and early adopters are the ones who are producing these tools. I feel Alt.NET is really just starting to touch the tip of the Early Majority Iceberg although it could still be in the early adoption phase still.

The one thing we need to remember is patience because this movement will not gain overnight acceptance.  Many will eventually come around.  Given enough time we may even start to get the laggards (stragglers anyone?) because they will make the change out of fear.  This is where Agile is getting to at this point in time. 

Each group comes to these decisions on their own terms.  If we don't understand and respect that each adopts in their own way, we may be quick to turn those would be adopters off.  This is perhaps a little psychology.

There is another story I like to tell that talks about these different groups of people. Sometimes when you are wanting to recruit people, you find that they fall into these groups.  So if you think of basketball, you play the game by giving them information and working the ball for awhile with them.  Then you toss the ball in their court and see what happens. If they toss it back, they are an early adopter/innovator.  They are ready to get started and you can immediately start working with them.  Some people want to examine the ball, understand it, take their time before they toss the ball back.  This takes awhile.  These people are the early majority.  Some people ask a lot of questions and are very slow to make a decision, if in fact they ever do toss the ball back into your court.  They usually come off as very skeptical.  These people are the late majority.  Then there are the laggards.  They only toss the ball back if there is a fear of loss in not accepting.  They take the longest to make a decision, and many times they are not accepting of new ideas.

I would say I fall into the early adopters/early majority area with most everything.  This is how I base where Alt.NET is.

I have two particular posts that I have talked about Alt.NET and what it means to me. To sum up those two articles, Passion is my Alt.NET and I want to see others enjoy what they do.  I believe in some way we are all Alt.NET because we should want to get better at what we do and find better alternatives when tools/ideas are lacking.

I leave you with this thought:

“Don’t worry about people stealing your ideas.  If your ideas are any good, you’ll have to ram them down people’s throats.” - Howard Aiken

I know you see this post and think that I am going to shock you with the best approach to software development and I am!  The great thing is I am not a consultant being paid for this so you don't have to spend countless thousands of dollars for me to come into your organization and tell you what processes are working for you and what you need to change to have successful projects.

Why not?  Shockingly, I don't want to take your money!  I don't want your organization's money either.  What I really want is for you to be successful.  I want you and your project to go as well as it possibly could.

So what's the big secret?  What is the best methodology? What is this utopia I speak of?  It's quite simple really.

The best methodology for successful projects is to:

Find what works for you, and do it. 

I'll let that sink in for a moment.  Take a deep breath.  Breathe out slowly. It's all so simple, yet what does that mean to do what works for me?

• Pair programming works for you? Do it. If it makes sense to only do it say 20% of the time, do it.  If it makes sense to do it more, do it. If it doesn't make sense, don't do it.
• Continuous Integration works for you? Do it. (Gosh I hope it always makes sense to do CI.)  If you don't already do it though, find out when it makes sense to implement and do it then.

Can you see the beauty in this common sense approach to development? If you think about it, it is really about being pragmatic.  Less analyzing about how to get better about the process and more focus on getting things done is the answer. 

Is there a time for analysis? Sure there is. Reserve that for when you are figuring out if something worked for you that you just tried out.  Call it a retrospective (if that term works for you). Does it work for you? Keep doing it. Something needs tweaked? Do it. Not working? Stop doing it.  To be fair, sometimes you need to try things more than once to be sure (you know, give it a fair shake), but don't lock yourself into doing something if it isn't working for you. 

Does it make sense to look out across the software development industry at what others are doing? Sure it does. It makes sense to get new ideas from others, but remember that even though you are fundamentally all doing software development, what others are doing is not what you are doing.  Heck, other people on other projects at the same place you work are not even doing what you are doing. Even more alarming, you are not doing what you did on your last project! If you are, it means you didn't learn from the mistakes you made on the last project. You are not doing what you did on your last project because you have grown. You have gotten better. You have, shall we say, evolved. You are better than you were yesterday, and six months from now you will be even more evolved!  I digress.

What I am trying to say is that what works for one project won't necessarily work for another because there are so many differences.  So many inputs go into software development that there really isn't a set recipe.  It's more of a concoction that is different every time you make it because you can't always control the ingredients.  That is why you have to find what works for you.

Now remember, the most important part of this novel approach is the do. You need to make sure you implement this part of the process. ;-)

And because it bears repeating I'll mention it again.  The best approach you can take to a successful project is to Find what works for you, and do it. 

Those that know me know I am very passionate about my craft.  I have recently been thinking of a great domain name that really addresses how I feel.  There are two things I wanted to address with my domain name. The first is that as a developer we are always seeking perfection and never finding it.  That is a coding utopia. A utopia is defined by Wikipedia as:

An unrealistic ideal that is impossible to achieve.

That is really how I feel about programming.  We can do it our whole lives and never achieve perfection.  But our obsession is to always strive to get there.  We do so in a fervent or ardent way.  That is the second idea I wanted to address.  We also fervently share our ideals with others.  The Free Online dictionary defines fervent as

characterized by intense emotion; Having or showing great emotion or zeal; ardent

Also, fervent explains the passion I have for development as well. That is why I am pleased to announce that the new domain for this blog is http://ferventcoder.com!  I am still hosted by GWB so you can still access the content by going to http://www.geekswithblogs.net/robz if you would like. Or you can use the new address.  All of the links work from either URL.

Those of you subscribed to my feed do not need to do anything new as you have been automatically updated.  However you may notice about 25 postings being counted as new or updated. Otherwise you may not notice anything different although I have renamed my blog to Fervent Coder (instead of The RobLog).

Welcome!  How many times can I say fervent in this post? ;D

One of the questions that I received during my recent presentation was how many different inversion of control containers are there for .NET?  I managed to name about four off of the top of my head.  Scott Hanselman just did a post listing more than ten .NET IOC Containers. Check it out here: http://www.hanselman.com/blog/ListOfNETDependencyInjectionContainersIOC.aspx 

I was recently tagged by Russ, The Caffeinated Coder, to talk about  music I listen to while coding.

First of all, I have to say I really like music. In fact, I love it.  Many things in music have helped carry me through life and some of the hard times I have seen. Like Russ, I have slowed down a little on listening to music as I have moved into a Team Lead position.  My environment is very open, so you can imagine the cubicle noise is always there.  My team is four developers, two BA/QA's, and  a project manager. You can imagine the noise that happens in what we lovingly refer to as "The Pit." :D

When I program I like to listen to just about anything that gets my adrenaline pumping. Sometimes I like to listen to Jazz too, but Russ took his collection with him when he left so I don't really have much of that anymore. 

So what gets my adrenaline pumping? I lovingly refer to that kind of music as "Angry White Man Music."  It is hard rock with peppered with screaming and awesome guitar riffs.  The likes of Linkin Park, Limp Bizkit, Staind, Three Days Grace, Filter, Korn, Tool, Nirvana, Green Day, NiN, etc, really get me pumped up. I like it because it helps me drown everything out and do the one thing that I love most in this world, creating things.  In this case, hacking/coding.  The same music that I code with is the same music I workout to.

A while ago I spoke about giving a presentation at the .NET User Group meeting.  I gave that presentation on Tuesday and it went pretty well.  I am still working on a possible video podcast of the presentation.

Here is the Visual Studio solution (be sure to read the readme file):

Here are all of the references.

Interfaces
http://www.c-sharpcorner.com/UploadFile/rmcochran/csharp_interrfaces03052006095933AM/csharp_interrfaces.aspx?ArticleID=cd6a6952-530a-4250-a6d7-54717ef3b345
http://ifacethoughts.net/2007/10/06/how-interface-helped-me/
http://en.wikipedia.org/wiki/Interface_%28computer_science%29
http://www.two-sdg.demon.co.uk/curbralan/papers/accu/InterfaceBasedDevelopment.pdf
http://ifacethoughts.net/2006/02/06/what-is-interface/

IoC/DI
http://en.wikipedia.org/wiki/Dependency_injection
http://en.wikipedia.org/wiki/Dependency_inversion_principle
http://msdn.microsoft.com/msdnmag/issues/05/09/DesignPatterns/
http://www.theserverside.com/tt/articles/article.tss?l=IOCBeginners
http://en.wikipedia.org/wiki/Inversion_of_Control
http://www.martinfowler.com/articles/injection.html
http://geekswithblogs.net/dsellers/archive/2006/10/16/94211.aspx
http://ayende.com/Blog/archive/2007/06/08/When-IoC-becomes-transparent.aspx

Windsor
http://www.castleproject.org/container/index.html
http://wiki.bittercoder.com/Default.aspx?Page=ContainerTutorials&AspxAutoDetectCookieSupport=1
http://www.castleproject.org
http://www.castleproject.org/container/gettingstarted/index.html
http://www.castleproject.org/container/documentation/trunk/index.html
http://www.castleproject.org/container/gettingstarted/part1/config.html
http://using.castleproject.org/display/IoC/Home
http://ayende.com/Blog/archive/2007/03/07/Building-the-Policy-Injection-in-40-Minutes-with-Windsor.aspx
http://codebetter.com/blogs/david.hayden/archive/2006/10/13/Inversion-of-Control-_2D00_-Dependency-Inversion-Principle-_2D00_-Castle_2700_s-Windsor_2F00_MicroKernel.aspx
http://dotnetslackers.com/articles/designpatterns/InversionOfControlAndDependencyInjectionWithCastleWindsorContainerPart1.aspx
http://msdn2.microsoft.com/en-us/library/aa973811.aspx
http://ayende.com/Blog/archive/2007/07/04/Storming-Castle-Windsor-amp-NHibernate--Code-amp-Presentation.aspx
http://ayende.com/Blog/archive/2007/05/03/Windsor-Patterns.aspx
http://ayende.com/Blog/archive/2007/06/11/Windsor-Decorators-and-Chains-of-Responsabilityes-Oh-YEAH.aspx  

Launchy is the best application launcher/productivity program I have ever used.  Basically when I want to open something, I no longer open the start menu.  I hit {Ctrl}+{Space} and a little window pops up.  I start typing and Launchy starts matching to items in my start menu, my quick launch, my Internet  favorites/bookmarks, and a special folder where I store Launchy specific commands and shortcuts.

Above is a screenshot of the index that I point Launchy to.  These are places I normally tuck away programs or data I may want to get to at some point without going to the folder level and finding those items.  In my D:\Launchy folder I keep items in the format below.  If I have a Folder shortcut, I call it F_FolderName. The Visual Studio Project Solution is just the Project name.

So when I hit {Ctrl}+{Space}, I start typing gmp and Launchy finds the best match.  Hit enter and it opens a highly used VS Project.

The format works for me and requires very little setup when I start working new projects or for other things I find I need to use a lot.

Twitter Integration - Update Status with Launchy

Launchy can be used for more than just application launching. At lifehacker, I learned how I can pull up Launchy and get it to do specific things for me.  Quoting Adam:

Using Launchy, you can append text to files, schedule appointments, add to your to-do list, set reminders, and more—it just takes a little setting up.

To add twitter integration, I have a file called "tw.bat."  It is a batch file that integrates with cURL and sends my updates for Twitter (and Facebook, because I have Twitter Integration) by simply hitting {Ctrl}+{Space}, Type "tw" hit {Tab}, enter my message, and hit {Enter}.

This is what the tw.bat file looks like:

@echo off
cd c:\
cmd /c curl --basic --user username:pass --data status="%*" http://twitter.com/statuses/update.xml
Echo Exiting...

To pick this up and run with it, all you need to do is put your username and password in and save this as a batch file.  Tell Launchy what directory it is in and let it interact with .bat files.

http://lifehacker.com/software/hack-attack/take-launchy-beyond-application-launching-284127.php

This is the link to the files from lifehacker: http://lifehacker.com/assets/resources/2007/07/Launchy%20Batch%20Plug-ins.zip

This is the link to download cURL:  http://curl.haxx.se/latest.cgi?curl=win32-nossl

How Do I Get It?

http://www.launchy.net

How Much Does it Cost?

It's free! If you like it and find it as productive as I have, I would donate to thank the author.  How much? Well how much would you pay someone for something that increases your productivity tenfold? How much would you think the product should cost if it were not free?  I am donating the author around $30 because it seems like about the cost of a product like this if it were not free.  Donate here: http://www.launchy.net/#donate

When you are designing a data warehouse solution that publishes out data marts, always be sure that your warehouse is granularity agnostic.  Granularity means the level of detail you track (defined more in depth below). Being agnostic means that you don't care about or have knowledge of something. To be "granularity agnostic" means that your warehouse doesn't care or have knowledge of how granular your data is. In essence, you design it up front so that it can get as much or as little detail as possible.

For example, your warehouse could start out housing daily items.  At some point, you may have a need to trickle feed your warehouse (per events, possibly multiple times per day).  If your warehouse is granularity agnostic, you do not have to change your database design.  To do that, we borrow the concept of batching, which moves the level of detail away from time or any other element of granularity.  You can batch as much or as little as your want. That means we can pull in as much or as little detail as we need without making changes to our database design.  And it's so simple it doesn't really require big up front design.

This is how Wikipedia defines granularity of data:

The granularity of data refers to the fineness with which data fields are sub-divided. For example, a postal address can be recorded, with low granularity, as one field:

1. address = 200 2nd Ave. South #358, St. Petersburg, FL 33701-4313 USA

or with high granularity, as many fields:

1. street address = 200 2nd Ave. South #358
2. city = St. Petersburg
3. postal code = FL 33701-4313
4. country = USA

or even higher granularity:

1. street number = 200
2. street = 2nd Ave. South #358
3. city = St. Petersburg
4. postal code state = FL
5. postal-code-first-part = 33701
6. postal-code-second-part = 4313
7. country = USA

Higher granularity has overheads for data input and storage, but offers benefits in flexibility of data processing.

To change that a little, we think of granularity within the concept of Data Warehousing to mean how finely you track events/changes.  For example, if you think of investing, a trade could go through several stages during the day.  At the start of a day a trade could be set up. Then it gets approved.  Then it might get settled.  All before the close of the day.  If your grain is on a daily basis, you will only see the trade as it exists at the end of the day.  The day before you don't see anything with this trade and the day of you only see the trade in a settled status.  If instead you are trickle feeding, you see every one of these events in your warehouse. 

Now think in terms of making changes to how often you feed the data to the warehouse.  If you want to change it from daily to hourly, the warehouse design should not have to change at all.  If you want to move from daily to a trickle feed, it should handle that appropriately.  One way it is achieved is by using batching and surrogate keys. That is a data warehouse that is granularity agnostic. 

81 spots have been filled since 4PM CST when registration opened (about 3:30 hours ago).  There is still time if you want a spot!  Hurry though, at this rate registration will close off soon.  There are only 150 spots total.

http://altdotnet.org/events/seattle

Registration looks like it just opened up!  See you there if you get a seat!  This event is April 18-20 at Digipen in Seattle, WA. 

As with the first conference, ALT.NET is free, but only accepts the first 150 people to sign up.

I bet you didn't know that you can reopen a Firefox tab you have closed.  Let's say for instance you closed a Firefox tab, but remembered you needed information from it immediately after closing it, or possibly you closed it by accident. 

1. Click the Red "X" or type Ctrl+W to close this tab.

2. Right click on one of the other tabs and select [Undo Close Tab].

3. And Yes! It's back!

In my last post, I put in the diagram a Type 1 and a Type 2 insert.  I think I need to elaborate on that just a bit.  Type 1 updates don't have any concept of history.  Type 2 updates are what I use to encapsulate all history types (Type 2, Type 3, Type 4, and Type 6). 

These dimensions are where Ralph Kimball's dimensional modeling come in. 

So before anyone points out that I didn't talk about the other types in my diagram, Type 2 updates for me is synonymous with all history in a data mart.

NOTE: This is up for discussion. I am not an expert in Data Warehousing (yet).  I hear from my Data Modeler that we are mostly Kimball in our approach, but I think that only applies to the Data Marts.  When I think of a Data Warehouse, I am thinking of a central place to store historical information.  Then we have a concept of Data Marts, which are separate databases of information in a slightly denormalized fashion (this is where you hear about Facts, Dimensions, and Aggregates).

For Data Warehousing we have chosen a path that is sort of a centralized history for all business domains.  We keep that somewhat normalized (which kind of follows Bill Inmon's approach), with the added 4th dimension of duration.  Then we publish out to Data Marts to be consumed by applications/reporting tools/etc. So we were talking about a few things today with data warehousing and we came up with a few ideas.

The diagram below shows some of the concepts we have been talking about in my workplace.  Transactional tables are 3NF and only hold information as of now. The Data Warehouse holds that information as snapshots at specified moments in time.  Data Marts use the star schema with facts and dimensional tables. The data marts are dimensionally modeled like Ralph Kimball's approach to data warehousing.

When looking at the difference between Ralph Kimball and Bill Inmon, the article on data warehousing in wikipedia describes some of the differences.  Another article sums it up somewhat:

Bill Inmon's paradigm: Data warehouse is one part of the overall business intelligence system. An enterprise has one data warehouse, and data marts source their information from the data warehouse. In the data warehouse, information is stored in 3rd normal form.

Ralph Kimball's paradigm: Data warehouse is the conglomerate of all data marts within the enterprise. Information is always stored in the dimensional model.

Granularity is the idea of to what level you put things into a warehouse.  A good definition can be the "level of detail at which information is viewed or described." If you think about a book, a table of contents gets you to a chapter, an index is more granular because it can get you to the paragraph. Granularity can be events (major milestones, certain events, minor events, all events), time (years, months, weeks, days, hours, minutes, seconds), etc...

When thinking about data marts, we buy heavily into the star schema approach. This is the concept of fact tables with dimensions attached to them like stars.  The facts are units of measure. With the star schema, granularity determines the growth of your fact tables over a period of time.  For instance, if you have a fact table that has an aggregate per day, then the fact table may grow at a rate of one per day.  If you add in a customer dimension, you multiply the granularity of the dimensional tables to determine the growth of your fact table.  For instance, if you have 300 unique customers in your customer dimension and your time granularity is once per day, your fact table may grow by up to 300 rows per day (1 time per day X 300 customers).  You are probably saying, that makes perfect sense, so what?  So if you change your time granularity to hourly, how much growth could you see per day? 7200 (24 X 300) rows per day. 

So what if you add in another level of granularity - say location?  How many locations could there be?  If a customer can only be at one location over the granularity of time, then that would be a granularity factor of one no matter how many customers you have.  If a customer can go to more than one location per day, then we say that we can have up to 600 rows per day. You can see how this gets a little tricky, but lets say our customer is a truck driver and they have two locations for each period.  Where they start from (from the prior reporting point) and where they are as of that point. Now what happens when you go hourly?  300 customers x 2 locations x 24 hours in a day = 14,400 rows per day.

As you are starting to see, granularity is one of the most important factors if not the most important when thinking about data marts and their growth.

Dear Wal-Mart (on Wanamaker in Topeka, KS);

Thank you for employing someone in Hardware department that actually knows about the department he represents.  It has been so long since I have seen an employee in your stores that actually know the department they are in.  Bill was very helpful and had many recommendations when it came to paint.  As we started painting, we found many of his recommendations were accurate and not just "fluff" like we find in other departments (i.e. the electronics department).  We found our experience with Bill to be a pleasant one and were entertained (while learning good lessons) with his actual experiences in the realm of painting. Please employ more people of this caliber in the future.

Thank you Bill in Hardware for knowing your "stuff."

Sincerely,

Rob

http://www.tenthdimension.com/medialinks.php

The tenth dimension is an interesting concept that is being put together by author Rob Bryanton.

This video below is an interesting representation of the concept of dimensions.   We are in the third dimension.  If you follow and believe the concepts presented below, a Data Warehouse would be in the concept of the 4^th dimension (time as points). We would be at the 5^th dimension (a branch of any possible moment) when we look at the Data Warehouse.  You can see how this is difficult to wrap your mind around.  Understanding this leads to understanding how complex data warehousing is.

Today is my 29th birthday.  I would like to say that means I am old, but there are some things in life I have yet to experience:

• Children
• Visiting a country where English is not the first language
• Owning a boat
• Selling a home for a newer, bigger one
• Having a successful business (or the alternative - having a business that fails)
• Many other things I can't think of now

So here I am, a little older, mayhap a little wiser, and definitely ready for everything new life may throw at me over the coming year.  Last year had a lot going on with  two months of military training, going to Vegas for the first time (and then again for DevConnections), seeing Graceland and the St. Louis Arch, being promoted to First Lieutenant, being in a flood, and seeing two winter storms up close. 

Changing out my butter bars for something a little more substantial.

The Bellagio Hotel inside was very exquisite as well.

This is in Independence, KS, on July 1st. Yes, that water is up to the bottom of those railroad track signals.  In this area there were two story homes entirely under water.  We made it out of town with minutes to spare before every exit was flooded.

This is in Topeka, KS, during the first winter storm.  This is the ice storm that happened in Oklahoma, Missouri and Kansas in December.  You can see the toll it is having on some of the trees.

Actually this year was a little slower than last year when I got married, went to the Bahamas, bought a house, attended like six other weddings, adopted a wonderful new pet, and attended military training in three states over a total of nearly five months (away from my new wife).

I am excited to see what will happen over the year of 2008. :D

I just saw this challenge to find real life examples of agile development by Tom Perry. He was hanging Christmas lights and found a way to incorporate test driven development (TDD) and continuous integration (CI) into putting up the lights.  He put up his lights, then turned them on and found he had a problem.

I had broken the first rule of agile development - Test First! Of course, any reasonably competent handyman would have known to try plugging in each strand of lights before beginning the life threatening task of suspending them from the roof - right? As I stood there at the foot of the ladder and contemplated my predicament, It occurred to me that the consequences of failing to test in the real world are a whole lot more painful than in the digital world. It’s really quite amazing that Test Driven Development hadn’t come along much earlier. If you stop and look around, people are doing it everywhere! Doing some woodworking? Measure twice, cut once.

[...]

I think it helps to try and find examples of these principles in your daily life. As we focus more and more on changing our fundamental paradigm for software development from waterfall to agile, we need to keep seeking validation in the “real” world. Look to other disciplines, look to the mundane, the everyday things that we do. Check and see if the premises that we use for Agile Development apply elsewhere.

So my mom recently got a car that has been overheating.  She couldn't drive the car 15 minutes without it overheating.  So my brother-in-law and I worked on her car yesterday and also found a problem with a misfire on spark plug six.  So we tested something, changed one thing, then tested again. We replaced a spark plug, the air filter, and some bolts we stripped taking the exhaust bypass off of the intake manifold.  We replaced the thermostat and bled the air out of the cooling system.  Then we did a build and we tested again by running the car sitting still. After it passed this test, we took the car out for a drive to see if we could get it to overheat.  This was the big test for overheating, or maybe the QA build and test.  In the end we were expecting to get the car to overheat but we couldn't get the car to overheat!  This is the proper way I think to test.  The best way to prove yourself right is to try to prove yourself wrong.  I test with that attitude.

Then we wanted to be sure it was the thermostat (and not just the bleeding of the air out of the cooling system), so we put the old thermostat in a pan of water and brought the water to a boil.  The thermostat never opened.  So I called my brother and gave him some trouble because he said the thermostat was fine and that wasn't the problem.

Today we worked on the misfire issue a little more by replacing the coil, testing, then putting the old coil back, testing.  Then we switched wires and tested again.  We pulled the spark plug to ensure that it was firing by watching the spark during the car start.  We tested the spark plug gap. Each time we erased the errors off of the car's computer and let them pop up again, hoping they wouldn't. :D

So we came to the conclusion that we fixed the overheating problem, which we initially set out to do.  We also troubleshooted the other issue of the misfire.  We got to the point where we believe there is most likely a compression problem due to head gasket issue or bad motor.  We are hoping that when someone takes a look at it, they find it is the gasket because that (although somewhat expensive to replace) is cheaper than a new motor. 

So how many examples of agile development happened here and why didn't we continue working on the misfire problem?

1. We estimated it would take about six hours to fix the overheating issue.  That is six hours total time, not based on resources.
2. The requirements changed on us. We found a more serious problem and alerted the user of it, telling them they needed to correct it and that is was more serious than the problem we fixed.
3. We had a limited timeframe so we could only deliver so many features.  It was butt cold outside and we were working with it snowing both days.
4. The user confirmed that the car has not overheated during a couple of trips out of town, therefore passing User Acceptance Testing.
5. Even though the user would prioritize the misfire issue higher than the overheating issue, it wasn't part of the iteration and we worked concurrently on fixing it as well. 
6. We delivered on what we promised for the one day iteration of working on the car - fixing the overheating issue.

The moral of the story for me is that there are real life examples of agile development out there and that you should make sure you have the proper tools for testing in every environment. We had the proper tools for testing: a code reader.  It allowed us to read off the codes, then erase them from the car's computer if we desired.  Make sure you have the proper tools.

I tag Russell, Troy, Dave, and Dru to find real life examples of agile development.

My friend Raymond just posted a great perspective on development.

I have come to the conclusion that I am a problem solver.  That’s my business.  Whether its process management, software architecture, personal growth for my development team or trying to keep my daughter from stealing my son’s toys, I solve problems.

I totally agree with Raymond about being a problem solver.  I have been saying that for years, so it's nice to see someone else come to this conclusion as well.

One thing I have learned from BDD is that developers spend too much time thinking about technical implementation details rather than the problem at hand itself.  I am continuously working with my current team to get them all to adjust to this way of thinking: think about the problem.  What are we trying to do?  We’re not trying to write software, we are trying to provide value to our customer, and we achieve that through software. 

I also say we are like actors because we need to think like the customers and know as much or more about the process than the customers themselves.  We are like method actors as method actors really become the role they are playing.  To best understand an issue for a banker, we should really understand things like investments and how they work, and understand paperwork that a banker goes through.

If we don't understand their process/pain, we can't correctly solve it (automate it/fix it) for them.  That and customers don't always know what all of their pain points are or don't know exactly what they want. Why would a user not know what all of their pain points are? Have you ever heard the phrase "We've always done it this way"?

If it were up to me, my team would work with (or as) a user of whatever product for which they would be developing a solution.  What length of time though?  A week? Two weeks? A month? Longer? Preferably long enough for them to really understand the problem domain and have ideas about how to make it better. 

So the verdict is that I am both an actor and a problem solver, but mostly a problem solver. Not that there's anything wrong with that. :D

Imagine if you will one future for applications.  You walk up to any computer and plug in your 128GB Flash Drive (USB3 or USB4).  You always have your applications with you and suddenly you have access to them through the flash drive. Your documents and files are on the flash drive as well.  You don't have to have an Internet connection and you don't bog down the Operating System (if you use Windows) because you don't have anything but the base install on it. Your applications are very responsive because the communication channel is capable of 4.8Gb/s. When your done with the applications, you unplug your flash drive and move on. You truly take your data with you everywhere.  Your laptop crashes? No problem, just reinstall your OS and move on.  Your Flash Drive crashes? If you back it up, no problem here either.  You just put in a new flash drive and copy/paste your applications/settings/files/data and move along your way.  A simple copy/paste.  That is a much better experience than reinstalling your applications and setting up your settings again.

So right now you have two options in the portable applications world, Portable Apps and U3.  A friend of mine turned me onto Portable Apps a couple of months ago and it totally rocks.  One of the best applications I use with this is Windows Live Writer Portable.  It means I can take my settings to any laptop. I don't have to install on every computer I use and I don't have to set up my settings again on every computer I get on. I also get to carry my drafts and recent posts with me everywhere I go.  Totally rocks!

U3 is pretty established with a bunch of applications, but installing not as intuitive as with Portable Apps.  With Portable Apps, you just add another folder with what you want to recognize the application under and add the application files.  You click refresh in Portable Apps and, viola, you have the application installed.  With U3, each application has a GUID.  It is a pain to know what you are looking at when you are looking at the folders.  It is much easier to install from the U3 piece itself.  Portable Apps can be used on any Flash Drive and it is OSS and that appeals because it is not device dependent. It looks like Microsoft is partnering with SanDisk for development, which could produce interesting results starting in late 2008/early 2009.  Portable Apps has a path for development as well, but neither really have a mature development toolset yet (keep in mind I am making an assumption with U3).

In late 2008 or 2009 when WCF is mainstream and more people have seen the power of VMWare Fusion and have converted to Mac, three things will be in people's minds, rich web applications (that look a lot like full applicati