Category Archives: Development (General)

Filter log4net to a single thread using Notepad++

Over the years I’ve Googled and Googled for a solution to this. Heck I even tried Bing! Finally time + an urgent need permitted figuring out a simple solution. To filter down to one thread using Notepad++:

  • Copy the angle brackets and thread number to clipboard. E.g. “[12]”
  • Menu -> TextFX Viz -> Hide Lines without (Clipboard) text
  • Press Ctrl-A (Select all text)
  • Menu -> TextFX Viz -> Delete Invisible Selection
  • Press Ctrl-A (Select all text)
  • Menu -> TextFX Edit -> Delete Blank Lines
  • That’s it! You are now viewing one logging from only one Thread

This works very quickly even with 70,000+ line 10MByte log files. IMO it avoids the need for xml log4net logging and Chainsaw (or similar).Simpler is always better.

Demo: Two threads counting to 100:

Copy the angle brackets and thread number to clipboard. E.g. “[12]”

Menu -> TextFX Viz -> Hide Lines without (Clipboard) text

Press Ctrl-A (Select all text)

Menu -> TextFX Viz -> Delete Invisible Selection

Press Ctrl-A (Select all text)

Menu -> TextFX Edit -> Delete Blank Lines

That’s it! You are now viewing one logging from only one Thread

TeamCity Release Build sgen.exe gotcha

TeamCity is great, I had it up and running with Automated Tests in just minutes. But.. attempting to make a Release Build generated the following error. Googling shows many people have the same error and I only found ugly solutions that required manual registry editing, copying of files etc:

c:WINDOWSMicrosoft.NETFrameworkv3.5Microsoft.Common.targets(2015, 9): error MSB3091: Task failed because “sgen.exe” was not found, or the correct Microsoft Windows SDK is not installed. The task is looking for “sgen.exe” in the “bin” subdirectory beneath the location specified in the InstallationFolder value of the registry key HKEY_LOCAL_MACHINESOFTWAREMicrosoftMicrosoft SDKsWindowsv6.0A. You may be able to solve the problem by doing one of the following: 1) Install the Microsoft Windows SDK for Windows Server 2008 and .NET Framework 3.5. 2) Install Visual Studio 2008. 3) Manually set the above registry key to the correct location. 4) Pass the correct location into the “ToolPath” parameter of the task.

The first thing I did was use msbuild from the command line; sure enough Debug Builds were fine and Release Builds had the same error. This ruled out Team City. Obviously installing VS2008 on our build box would solve the issue, but I assumed the build tools must available elsewhere without need all of VS2008. They are:

The solution is: Download the Windows SDK and install .Net Development Tools (it says 2008 Server but I did this on XP SP3):
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=e6e1c3df-a74f-4207-8586-711ebe331cdc

WindowsSdkForServer2008

Reflections on five years NUnit experience

It has been five years since I first used NUnit and have used it with great success on almost every project since then. This post briefly reflects the pros and cons of Automated Developer Testing I experienced.

. Zero Defects can be the norm
Given a serious amount of control on projects I insist on ~100% coverage for the most complex pieces. Every time this happened the complex code was delivered with zero (or very close) defects. Admittedly before NUnit I received a lot of praise of stable code and attention to detail, but with with NUnit we can even redesign working code to be more maintainable and still release with almost 100% stable code. Redesigning for maintainability is not practical without a suite of automated tests – it normally requires a complete rewrite.

. 100% Coverage is for Fools
Sometimes creating Automated Tests make prefect sense, and sometime it costs more time than it saves. I aim for 100% coverage on reusable libraries, critical architecture and anytime I think the business rules are far too complex for QA to completely understand let alone regression test on a regular basis. 100% Coverage on simple data manipulation web page will be a frustrating waste of your time.

. TDD – Good luck with that!
Unless your company has several years experience writing Automated Tests you will not get buy in for TDD. Learn to walk before you try to run.

. Continuous Integration is an easier win
I can install a Cruise Control in a day and educate all devs in one 30 minute presentation. Before looking at Automated Developer Tests I highly recommend rolling Continuous Integration our first. It is a massive win for very little effort. Managers and developers alike always see the value, this should build up your credibility enough to start an NUnit pilot.

. Screw Mocks, use a real database, ldap etc
Yes Mocks can work, but in my experience I have seen developers waste time coding Mocks, Object Mothers etc. A far simpler approach is to commit to a reasonably large amount of upfront effort so the Automated Tests setup and tear down a real environment. This framework should be created one expert developer and will result in some complex, hard to maintain code, but it then becomes very simple for even junior developers to test large sections of their code, not just wimpy Unit Tests. This is the foremost reason I have had so much success with NUnit. I have my own framework for SQL Server/ Oracle which took about three re-writes to become as simple and stable as it is today – you will need great database skills and good C# but once written it just works. Via Cruise Control we automatically spot problems in build scripts instantly, if anyone changes sproc parameters without updating the DAL we spot that too! I even have a suite of test on one project that does an end-to-end publish mimicking data coming from several internal feeds, being data scrubbed by our system, transformed into records in our system and push out the other end to table ready for user consumption! Probably 100klocs of C# and PL/SQL are executed by a single test – if anything breaks we know about it very quickly.

. Hundreds of small tests or a few Large Tests?
A classic mistake I see is developers taking the Unit word too literally and set about writing literally hundreds of test for one module. Although sound in theory, it is madness in practice. These developers run out of time and generally I see a stack of pretty useless tests. You should aim for the most coverage you can per test, of course when such tests break it is not immediately obvious what broke but you will be aletred that something broke, and Cruise Control will show you what code changed since the last build – making it an easy fix almost every time. So do you want to spend weeks writing hundreds of tiny tests or a few days writing all-encompassing tests? In reality no team I have worked on has every has been permitted the time to write hundreds of test. Also most developers don’t have that kind of patience which is why TDD has a low adoption rate. To clarify this point in general go black box rather than white box – not I said in general, use your common sense here :)

. NUnit can really stress developers out
This is a serious problem when developers are under the gun to meet deadlines and an NUnit test breaks the Cruise Control build. Of course everyone knows all work should stop until the build is fixed, but 90% of developers will just mark the test with [Ignore] and work on what their manager is shouting for.

. Some Managers think that is what (cheaper) QA staff is for
No explanation is needed I am sure. Some managers get the SDLC and some do not, and I can offer no solution to this problem. Some managers can only focus on very short-terms goals.

No Brainer laptop upgrade: 7200 rpm HD

Are you frustrated with speed of your its-not-even-one-darn-year-old laptop? You have probably already spent $150+ putting a gig or two of DDR memory in the 32 bit dinosaur.. I had and my 2.8GHz Hyperthreaded P4 was still a dog. Did you know most laptops come with 4200 rpm hard disks? Well they do, and 4200 rpm is SLOW, as in S…..L…….O……….W

For about $120 and using Norton Ghost I transformed my laptop in under an hour and it is now very usable. My hunch is that upgrading to a faster hard disk makes much more difference than upgrading beyond 512MB. With the faster (read hotter) hard disk you do run the risk that the laptop could overheat; this el-cheapo Averatec 6100 has been fine, but do be aware of this risk and backup your data first.

 

 

 

Bizarrely the replacement hard disk looks virtually identical to the old one. It came from the black area in the bottom right of the above picture.

For the real geeks out there my AMD A64-2800 desktop has a 10,000 rpm WD raptor hard disk – the prices on these have finally started to drop and if you are a real developer I say GO FOR THAT TOO. All real developers use desktops right? You know, I have said that for years but if necessary I could develop on this Averatec now.

Free Tools from SysInternals: Watch your File System, Registry and more

There are some great free tools over at http://www.sysinternals.com/.
Get them now before someone like Computer Associates buys the company and starts charging for them.

Personally, one tool in particular called ‘FileMon’ helped solve a tricky file locking problem on a recent project. FileMon showed the massive amount of files opened/closed by our software. A little analysis identified the few files that were not being closed. Stepping through code using debug mode and watching FileMon made it easy to see which chunk of code was opening the file. Five more minutes of work rewrote the complex file opening/closing code and the bug has not been seen again. Before using FileMon I had wasted a full working day try to track down the rogue code.

Thanks to ‘Writing Secure Code’ for mentioning these tools. Michael Howard recommends them for discovering why applications fail to run as non-admin users.

FileMon In Use:

RegMon is very similar, but watches registry access: