Free Code

Lately, I've been really paying attention to how much functionality you can reuse for free from major players. For example, you can check out the various Google APIs (http://code.google.com/)and the Yahoo! User Interface Library (http://developer.yahoo.com/yui/). These libraries are free, released via Open Source licenses, and offer ridiculous amounts of functionality.

As a developer, I love the idea of being able to leverage the work of other people. Most of the time, this is hard to do because so much of the code that everyone else writes is crap. ;-) But seriously, there is even a name for this, we call it Not Invented Here (NIH). The premise being most developers like to know exactly what their code is going to do, so using someone else's code can be hard.

In the case of the big boys like Yahoo! and Google, it becomes a lot easier to trust that the code is going to deliver expected results and not be malicious or buggy. After all, when it is provided (and used) by a company with billions of dollars of online reputation to uphold, you can pretty much bet they've tested it.

Which isn't to say that just because they are all that, you don't need your own bag of chips, they have their issues too. Dependency and version issues, occasional bugs, and warped programming models are abundant. But in the end, you are definitely getting your moneys worth.

The chart at the top of this post is generated using the super cool (and free!) Google Chart API which means I won't be paying for Dundas or ChartFX licenses again. You can find the details at http://code.google.com/apis/chart/.

Why Unit Test?

Recently, I had a down and dirty discussion with a team who is gearing up for a new engagement. This team has been doing Scrum for a while but was struggling with how to fold testing more tightly into their development processes.

The heart of the matter ended up being differences in the purpose and perception of Unit Tests specifically. One camp saw the role of Unit Test as "to find bugs". That's it. They tell you where the bugs are. If they are just about finding bugs, they have more value to new developers than established developers. After all, great developers are much less likely to allow the kind of bugs found by Unit Tests in their code. As a great developer I'm going to catch bad parameters and logic flaws before checking in my code. Integration and functional bugs are probably not going to be found in Unit Tests anyway.

The other camp (read: in my opinion) was resolute that Unit Tests fill a much larger purpose. To my eyes, Units Tests go way beyond just finding bugs. Following are four big ways that Unit Tests can add significant value over and above just finding bugs.

Proving Integration
Today's applications are more distributed and componentized than ever. The number of moving parts has just sky-rocketed with the use of frameworks and new API's, the prevalence of SOA and more component-oriented designs. Personally I see goodness here, but regardless if you like this direction or not, it's happening and is relevant to address. With this fragmentation comes a need to understand when the integration between components is complete. Simple Unit Tests can tell you quickly if the work being delegated downstream of a component is completed and ready. As you are bringing on multiple layers of components the pass rates of the Unit Tests at various layers can quickly show you the level of completeness for the entire integrated system.

Sample Code
One of the difficult parts of delivering a complete software product especially when the product is a framework or an API, if the product is extensible or supports programmability. In these cases specifically, Unit Tests can serve as great examples of how to utilize the API or exercise the programming model. They can often be directly consumed as documentation and sample code for SDKs or in knowledge transfers throughout the life of the product. Building this from outside the team without relying on the author of the component is extremely costly and difficult. However the returns are typically the most valuable.

Proving Design
Designing and building components is easy with the current toolsets. The downside of this is that we can create classes and objects so easily that we don't always think about the bigger picture. Taking time to write code that exercises a new component or interface can act as a sanity check to ensure that the component is easy to use and meets the programmability desires. It's quite often that I see a single API that requires two or three different programming or data interaction models. Some quick Unit Tests would have shown the designer how difficult and inconsistent the usage of the API had become.

Change Management
As a great engineer, it might be totally natural for you to write code that has few bugs. Are the engineers who take up your legacy as experienced? Will the engineer who will open the code in six months to add some features or fix an integration issue be as comfortable knowing how to change things without breaking anything? Would you? Being able to run regressions over time adds a wonderful safety net in the identification of the impact of changes upstream, downstream, or inside your code. It doesn't mean you will (or can) catch everything with Unit Tests but it can sure give you a head start and a level of confidence.

So perhaps this is more about semantics or expectation setting, but I think it helps to keep in mind the variety of purposes Unit Testing can serve. When you expand the contributions made by Unit Tests hopefully you will have an incentive to get even your experienced engineers to spend time writing them.

Running The Gauntlet

This week I had the chance to discuss using Team Foundation Server in a large team setting specifically as concerns Build and Deployment.

As I was organizing my thoughts to answer a question about check-in policies, merging and branching, and so forth and I kept coming back to my guiding principles:

• The Build is the build. There is only one Build.
  
• Build anywhere you like, as much as you like, but it only counts if it is in the Build.
  
• If it doesn't work in the Build, it doesn't work.
  

Now I am certainly familiar with having to manage simultaneous release schedules, juggling branches, and so forth, but that doesn't mean I like it. I absolutely support the mainline concept. Much like the Highlander, when it comes to the build, there can be only one.

We could argue that there are many techniques to allow these fancy behaviors but that doesn't make it okay. Just because you can do something, doesn't mean you should. Experience tells us that if you have a big team, you can't afford to have anyone interrupt the flow of the system. This includes breaking the build, causing failures of critical tests, etc. When more people or organizations are involved, the cost increases.

In the agile communities there is a push towards Continuous Integration (CI) which is really an attempt to implement the Only One Build concept as an ideal. When you can do it, you should. When the amount of code grows or the number of loose connections increases this can be cost-prohibitive, primarily in time. Compiling 1.8 million lines of code is going to take a little while regardless of how big the build machine might be. And when it comes to running Build Verification Tests, these problems are compounded. Especially if you are using disparate services that are loosely coupled.

The idea behind CI is to bring compilation and integration issues to the attention of the team as soon as possible thereby minimizing distribution. Going beyond that are tools that would actually test code entering the system before check-ins occur and reject or accept them based on the results. With TFS policies this is fairly straightforward to set up and an open source effort has begun to bring this to life. You can find out more about this effort at http://www.opengauntlet.org/.

If you prefer to wait until this functionality is built-in, you'll be glad to know that the next product release most likely will include a similar feature with a working title of "Gated Checkin".

The Business of Software Development

There was an interesting exchange of ideas I witnessed today. The crux of the issue was someone trying to understand the benefits of the Model View Controller (MVC) pattern as socialized by Microsoft. Having some interest in the subject, I watched it grow from a comparison with Event Driven Architecture (a complete misnomer) to contrasting with Model View Presenter (a new flavor of old idea), to just plain batting around the technical merits of various implementations of these common concepts. As one person after another jumped on the band-wagon, virtually everyone had something insightful and technical to say. But none of them brought it back to the business problems. This troubled me as it was indicative of the way we forget that we are in the business of software development. Not just doing software development. Here was my contribution:

--

On many systems we see that there are really two types of logic often treated as the same while they are very different. The first type are UI rules which I’ve seen loosely defined as how we paint screens and provide rich user experience. Then there are business rules which I’ve seen loosely defined as how we manage the data and interactions with the services or repository of the system.

Because we tend to not treat these separately, we also tend to see them tied very closely together in designs and implementations. This presents multiple challenges such as making it harder to encapsulate logic and reuse it and the breadth of knowledge grows (i.e. you need to know UI programming and API/data model programming). MVC is one pattern that attempts to address these challenges, and there are many other patterns that address them as well. Get ready for the twist...

On projects with lots of UI rules that are tied closely with lots of business rules, these challenges are compounded. Especially when you consider how we can go about testing all that intertwined code.

There is lots of anecdotal evidence to support that testability remains one of the biggest challenges on projects with hundreds of interfaces. When looking at the cost/benefit of different designs for large applications we have repeatedly seen cost of testability to be one of the single biggest factors in terms of monetary impact. We can argue a design on technical merits all day long and have differing subjective opinions, but we are in the business of software so we have to be able to quantify the impact of a design in terms of cost also. My personal experience has found that the cost to test is one of the most easily overlooked. Having no one else on this thread bring up the subject as a crucial factor sort of reinforces my point.

Being able to write automated tests is imperative when we have hundreds of UIs. Separating the two types of code means it is significantly easier (read: less time, less custom skill, less money) to craft the thousands of test cases required and the controller logic can be run without additional user windows and workstations. This is because the test cases are standard API code, not rich UI code. In these cases the UI acts as the test without providing an actual UI, instead it just executes the controller to test the business rules.

It doesn’t just make the tests of business logic easier to write, maintain and reuse, it can actually increase code coverage allowing us to get to exception areas of the code that default UI logic would prohibit. For example the UI rules for the initial type of application being constructed don’t allow certain behaviors. However, the controller must be able handle them so that other types of UIs can also be crafted. If we only test against the UI directly (read: the actual UI we are delivering to the users in the first release), we won’t test those hard to reach places. If we don’t test those hard to reach places then when we finally do build the second or third kind of interface and it reaches there, we find new bugs in code we thought was tested well. Automating the controllers allows us to minimize this risk.

To sum up: yes the MVC-class of pattern allows us to encapsulate logic, and yes it can provide significant reuse of code, and very definitely yes it can be argued as good design. But in business terms, we often forget to make the leap to testability as a primary motivator when time to quality gets really small.

Too often we swap our opinions on technical subjects from our ivory towers without considering the business factors that should be influencing us.

--

Whatcha think?

Reality Check

Most of the time I don't give much credence to the talking heads in fields like technology. Basically to be competent in this field requires a commitment to be doing that fundamentally restrictions your ability to be talking. So it is generally okay to assume that if you are primarily a talker, you are not really a doer and everything you say needs careful scrutiny. On the other hand, there are those who don't particularly talk a lot and you know it is because they are getting stuff done. When they do open the kimono, it is best to pay attention. Especially when their views align with my own. ;-)

I don't see aspect-oriented programming escaping the "academic ghetto" any day soon, and if it does, it will be less pervasive than OO. When it works, aspect-oriented programming is elegant, but it's not clear how many applications significantly benefit from its use. Also, with AO, it appears difficult to integrate the necessary tools into an industrial-scale programming environment.
--Bjarne Stroustrup

A great example quote, from a Technology Review article, of someone bringing some practical reality to the world of technobabble. Personally, his use of "academic ghetto" about made me fall out of my chair I was laughing so hard. Truth is so much better than fiction.

Thoughts?

New Amazon Controls

If you are reading this on blog.tempusfugate.com then you have probably noticed the cool new additions to the template.  Namely the two new sections on Books & Movies.  These are examples of a new control I've made available that uses Amazon Wishlists to store collections of books, movies, and cds.  Using the Amazon Wishlist service means I can maintain as many lists as I want, and they can hold any Amazon products I desire.  So I have lists for Top Movies, Mentoree Reading Lists, and Favorites Tunes.  Since they come from Amazon, the details are provided their service and can include information like artist, publication or release date, track information, director, cast, and so forth.

The script that draws the controls is embedded using an ASPX wrapper around a special control I've created in C# using the .NET Framework.  You can download the assemblies here or just reference the wrapper directly as I have done.  If you are hosting your own pages, you might want to pull down the assemblies and embed them directly within your own ASPX pages.  By referencing the control directly in your own host environment, it will improve the performance and make it easier for you to customize the look and feel.  Let me know if you'd like access to the assemblies.

If you aren't capable of hosting your own ASPX pages, then you can reference the wrapper page I have provide at scripts.neodiem.com.
To embed the script, add the following code to your template:

<script type="text/javascript" src="http://scripts.neodiem.com/showlist.aspx?
showdetails=false
&showimages=false
&numberofitems=5
&showrandomitems=true
&listid=11C766M4WUWVW">
</script>

You can then change the listid property to point to a list of your own choosing.

The output of the wrapper is JavaScript which will render the items in the WishList as an UnOrdered List.  You can control the look and feel using ClassNames.

Feel free to play with these as much as you like and use them to host your own lists of Amazon items!  You can choose how many items to show, whether they show in random order, and the level of detail associated with each entry.

Any feedback is appreciated.