It has been five years since I first used NUnit and have used it with great success on almost every project since then. This post briefly reflects the pros and cons of Automated Developer Testing I experienced.
. Zero Defects can be the norm
Given a serious amount of control on projects I insist on ~100% coverage for the most complex pieces. Every time this happened the complex code was delivered with zero (or very close) defects. Admittedly before NUnit I received a lot of praise of stable code and attention to detail, but with with NUnit we can even redesign working code to be more maintainable and still release with almost 100% stable code. Redesigning for maintainability is not practical without a suite of automated tests - it normally requires a complete rewrite.
. 100% Coverage is for Fools
Sometimes creating Automated Tests make prefect sense, and sometime it costs more time than it saves. I aim for 100% coverage on reusable libraries, critical architecture and anytime I think the business rules are far too complex for QA to completely understand let alone regression test on a regular basis. 100% Coverage on simple data manipulation web page will be a frustrating waste of your time.
. TDD - Good luck with that!
Unless your company has several years experience writing Automated Tests you will not get buy in for TDD. Learn to walk before you try to run.
. Continuous Integration is an easier win
I can install a Cruise Control in a day and educate all devs in one 30 minute presentation. Before looking at Automated Developer Tests I highly recommend rolling Continuous Integration our first. It is a massive win for very little effort. Managers and developers alike always see the value, this should build up your credibility enough to start an NUnit pilot.
. Screw Mocks, use a real database, ldap etc
Yes Mocks can work, but in my experience I have seen developers waste time coding Mocks, Object Mothers etc. A far simpler approach is to commit to a reasonably large amount of upfront effort so the Automated Tests setup and tear down a real environment. This framework should be created one expert developer and will result in some complex, hard to maintain code, but it then becomes very simple for even junior developers to test large sections of their code, not just wimpy Unit Tests. This is the foremost reason I have had so much success with NUnit. I have my own framework for SQL Server/ Oracle which took about three re-writes to become as simple and stable as it is today - you will need great database skills and good C# but once written it just works. Via Cruise Control we automatically spot problems in build scripts instantly, if anyone changes sproc parameters without updating the DAL we spot that too! I even have a suite of test on one project that does an end-to-end publish mimicking data coming from several internal feeds, being data scrubbed by our system, transformed into records in our system and push out the other end to table ready for user consumption! Probably 100klocs of C# and PL/SQL are executed by a single test - if anything breaks we know about it very quickly.
. Hundreds of small tests or a few Large Tests?
A classic mistake I see is developers taking the Unit word too literally and set about writing literally hundreds of test for one module. Although sound in theory, it is madness in practice. These developers run out of time and generally I see a stack of pretty useless tests. You should aim for the most coverage you can per test, of course when such tests break it is not immediately obvious what broke but you will be aletred that something broke, and Cruise Control will show you what code changed since the last build - making it an easy fix almost every time. So do you want to spend weeks writing hundreds of tiny tests or a few days writing all-encompassing tests? In reality no team I have worked on has every has been permitted the time to write hundreds of test. Also most developers don’t have that kind of patience which is why TDD has a low adoption rate. To clarify this point in general go black box rather than white box - not I said in general, use your common sense here :)
. NUnit can really stress developers out
This is a serious problem when developers are under the gun to meet deadlines and an NUnit test breaks the Cruise Control build. Of course everyone knows all work should stop until the build is fixed, but 90% of developers will just mark the test with [Ignore] and work on what their manager is shouting for.
. Some Managers think that is what (cheaper) QA staff is for
No explanation is needed I am sure. Some managers get the SDLC and some do not, and I can offer no solution to this problem. Some managers can only focus on very short-terms goals.
Posted by Paul Lockwood as Development (General) at 1:49 PM MST
3 Comments »
This
will be basic trivia to those with a Microsoft development background, but I only learned it on my last project: In a nutshell Release v’s Debug only sets certain compilation switches. Consequently it is no problem to deliver a release build with Debug Symbols. From my research compile time and JIT optimizations are not affected, but I stand to be corrected on that point.
Can anyone tell me why one would NOT always deploy pbd (debug symbols) files with your code other than IP issues? Note to Sys-admins: ‘because I hate all developers and it makes my day to see them struggle debugging blind’ is not a valid answer :)
Coupled with log4net deploying pdbs really helped me stabilize a nightmare of a background process on my last assignment. What is log4net I hear many ask? Attend the first fifteen minutes of my Atlanta Code Camp presentation or read a post or two like this:
http://blogs.acceleration.net/ryan/archive/2004/11/10/380.aspx
Do I hear a few voices at the back are saying they love EIF and the MS Application Block? If you are using them already then probably stick with them as they do work, if not scan these posts and make an educated decision:
http://weblogs.asp.net/cazzu/archive/2004/05/17/133196.aspx
http://www.cauldwell.net/patrick/blog/CommentView,guid,13345.aspx
http://dotnetjunkies.com/WebLog/kenbrubaker/archive/2004/10/04/27581.aspx
There are many more posts debating EIF and log4net:
http://www.google.com/search?hl=en&lr=&q=log4net+eif
Posted by Paul Lockwood as Development (General) at 8:49 PM MST
No Comments »
Atlanta’s Java User Group is excellent. Tonight Justin Gehtland talked about his new book Better, Faster, Lighter Java. It was a thoroughly enjoyable evening, even with the numerous (and almost all inaccurate) .Net jibes.
Prior to the main event Burr Sutter (who I already knew) led a great discussion with the audience on a raft of subjects. Our .Net groups are missing someone with his panache to work the crowd; he is at least a DCC level speaker.
A fundamental subject of the book is Why Do Many Java Project Fail? As a former Java developer I agreed with many of the assertions including:
. Complexity – Java people love buzzwords and apply any they know anywhere!
. Taking things too far – e.g. Layering everything
. EJB misuse – Been there personally!
. Peer Pressure to write ‘smart’ code
It was also interesting to see just how fragmented the Java tools/ platform extensions market has become since I booted the JVM out of my career (March 2001). The number of commercial-quality Open Source tools available is amazing. In fact commercial companies are now viewing projects like Hibernate as serious competition. In my view this fragmentation lowers each tool’s user base and hence their quality. Why is the CLR, .Net Framework and VS.Net so stable? Because a massive number of sales means Microsoft can afford serious development and gargantuan testing efforts. Java EJB Servers cost around $100K and are apparently as stable Scott McNealy’s career. Without the effort of a Microsoft style Product Development Lifecycle you just cannot create Microsoft quality products. I would love to yatter on about the evening but no one reads long blog entries, so watch this space for news from future Java meetings. 
Posted by Paul Lockwood as Development (General) at 8:29 PM MST
No Comments »
GIGO, Garbage In, Garbage Out; Even before my first high school Computer Class back in 1982 this term was well known to me and my few programmer friends. It is up there with Goto considered harmful. This blog tries to explain the why GIGO is no excuse for failure in your own code:
Recently, while on a group cycle ride a fellow rider’s lack of experience/ judgment in jumping a road hazard led to reasonably serious injuries on my part. Was it my fault? Well, he lost control while jumping falling on my bike and taking it down. Still I partially blame myself; I had never seen this guy cycle before, and just assumed he knew how to hop a bike over obstacles. In future when such an obstacle arises, I’ll be very alert for problems and maintain a good distance from unknown cyclists when potential danger arises.
We can do the same in programming. The recent Fail-Fast post highlights one such technique. In Code Complete 2 Steve McConnell collectively groups such techniques as Defensive Programming. This chapter should be read by anyone who writes code for a living. He correctly points out that a fail-fast technique is not always the correct action. Over the last eleven years I have exclusively written serious business applications that often work with financial data. In these cases if an error occurs we would rather fail in a hard manner than have the code work in an inconsistent state:
“A dead program normally does a lot less damage than a crippled one”
But, what if you are writing a video game, or the error happens in simply rendering data to the screen. In such cases a graceful recovery option may be better. Would you rather have Windows crash, or show a desktop icon using the wrong colors?
The chapter is too long to do justice in a blog post. It covers sub-topics including Assertions, Barricades, Graceful Recovery options, Robustness v’s Correctness and Exceptions. At the bare minimum skim-read all the sections (although the Exception section is pretty weak IMHO).
Certainly one project I participated on in 1997 suffered politically because it was the GUI to a lousy messaging system, which feed the GUI with bad data. My manager’s defense was ‘Garbage In, Garbage Out’, and it fell on death ears. With the hindsight of Steve McConnell’s wisdom, if we had Barricaded our code from bad data then fighting politics would have been much easier. The manager could have attended meetings with
the log from the Barricade Layer, proving in black and white where the problems lay. Both teams would then have been able to solve the problems much quicker, without escalating it up the management chain. 
Posted by Paul Lockwood as Development (General) at 3:38 PM MST
No Comments »
Several companies running ASP.Net experience this problem. They have several web applications and need common authentication. Out of the box ASP.Net makes the user login in to each different application. I know of several organizations where this was a frustration to hundreds if not thousands of user.
Luckily the fix is trivial. Open up machine.config (beg, grovel, kiss the feet of your sys-admin if necessary) and make this change:
Original machine.config entry:
<machineKey validationKey=”AutoGenerate,IsolateApps” decryptionKey=”AutoGenerate” validation=”SHA1″/>
To enable sharing of cookie authentication:
<machineKey validationKey=”AutoGenerate” decryptionKey=”AutoGenerate” validation=”SHA1″/>
>>Totally trivial change that took under five minutes of searching documentation but I bet many organizations are struggling with separate logins even today. All the more reason to hire a Microsoft Certified .Net consultant [shameless plug!].
Posted by Paul Lockwood as Development (General) at 8:56 AM MST
No Comments »
It is while since I wrote this code-snippet, but at the time many DataGrid experts said it could not be done. After about twenty minutes of trying to understand some DataGrid events a working solution was found:
if (this.dataGrid.EditItemIndex > -1)
{
// Update the DataGrid
this.dataGrid_UpdateCommand(
this.dataGrid,
new DataGridCommandEventArgs(
this.dataGrid.Items[this.dataGrid.EditItemIndex],
this.dataGrid,
new System.Web.UI.WebControls.CommandEventArgs(“Update”, “”)));
}
Not a pretty event call but it works. On my last ASP.Net project several pages used one Update Button to perform updates on three or more DataGrids simultaneously. If your code forces users to click several update buttons, then use of this code may just make them a little happier.
Posted by Paul Lockwood as Development (General) at 7:20 AM MST
No Comments »
How often have you had to fix a run-time error and taken hours or even days to track down the root cause? Using the Fail-Fast policy can all but eliminate such troubles.
The key concept is to fail early, that is as soon as you can programmatically detect bad data exists throw an exception reporting useful contextual information.
e.g.
using System.Reflection;
…..
DataRow drWorkingData = someoneElseClass.FindTheRow(iPrimaryKey);
if (drWorkingData == null)
throw new ApplicationException(
“Failed to find data for Primary Key >” + iPrimaryKey + “<” +
“[" + MethodInfo.GetCurrentMethod().DeclaringType.FullName +
"." + MethodInfo.GetCurrentMethod().Name + "()]“);
In this example had we not used a Fail-Fast policy the code would have continued until drWorkingData was referenced, at which point the run-time would automatically throw a null object exception. If you have debugged much code in your life, you will realize that this error could be thrown quite a distance from the original source of the error.
Note the use of reflection to obtain the location of the error. This is my most prevalent cut-and-paste reuse. An obvious improvement would be to encapsulate it in a utility class, and walk a step up the call-stack to report the method’s class and name. That is beyond my knowledge of .Net Reflection.
Also note that instead of using ApplicationException I always use a custom Exception called FailFastException which makes it very easy to trap such errors in a central location and report/ log them in a consistent fashion.
Posted by Paul Lockwood as Development (General) at 8:57 AM MST
No Comments »
