Eventual Correctness

Have you ever heard the term Directional Correctness or perhaps Directionally Correct? In the software business it is often used for when the concepts, functionality or usefulness of a thing is understood or apparent, even when there are details or bugs that prohibit full acceptance.

If you are building a form for the user to input data, having all the form elements and the submit button without any formatting, style, or validation would be considered Directionally Correct.

The process of building something new is often littered with the broken attempts and failed assertions. This is actually one of the driving motivations between Test Driven Development. Good designers use iterative processes to arrive at a great solution, it doesn't just magically appear. Great designers help those around them iterate also.

Have you ever looked back at the end of the day and realized you worked on the same design element the entire day? Have you ever had a user or consumer completely yank the carpet out from under you and realize that all the progress you made wasn't progress at all? A key reason this occurs is because we get too focused on the details too soon in the game. After all, the greatest engineers are perfectionists. It isn't done until it's Right. And that is what trips us up.

As engineers and designers we should foster or sense of elegance and perfection. Just not right away. We need to evolve our solutions, not magically produce them instantly. Produce something ambiguous and generic but somewhat functional. Or at least demonstrative of functionality. Get the users and consumers hand-waving in the general direction. Once you've achieved some agreement on the direction you are taking, then you can start hammering in nails and confirming details.

Besides being able to verify your work with consumers, it is important to delay details for your own benefit as well. The old Forest vs. Trees problems can impact even the most seasoned woodsman. As you implement your vision the details will reveal themselves. The gaps and holes will become obvious and you can address them with precision and prejudice. If you focus on details only after you have verified your directional correctness, they won't move or shift so you can be certain your specific solutions will serve.

It can be hard to iterate, especially with our innate desire for perfection. Iteration is like delayed gratification. You can make it perfect, it will be amazing and elegant. . .eventually.

Mirror in Motion

In the last few days I've had to spend a much larger than average percentage of time explaining how to design a system.  More specifically how to perform tasks of software architecture in a high-velocity way that is consumable by individual contributors.  Whenever I have a challenge of how to explain something complicated, I generally look for examples of how others have addressed similar opportunities.  Sometimes, what they have written mirrors aspects of my own thoughts which helps give me ideas for how to communicate my own ideas.

One such reading lately is about Microsoft Motion, which is an architecture approach being bandied about at Microsoft. I found the conclusion to be the clearest statement of the overlap with my own thinking:

This paper describes three rules for success in a partitioned-iteration strategy. They are as follows:

1. Start with the low-hanging fruit, focusing on quick time-to-value.
2. Leverage economy of small scale, focusing on agile processes.
3. Centralize interoperability and decentralize implementation, focusing on reduced complexity through partitioning, and faster iteration through agility.

You can find out more about Motion Lite on MSDN. You can also check out some podcasts on Channel 9 here and here.  And lastly the Architecture Journal has an article about it here.

Watch for some upcoming posts on other overlapping subjects from the Architecture Journal coming soon.

Serious Confusion

In my reading, I came across an excellent post about the estimation process for software development.  The author (Serious Confusion) does a solid job of summing up his views with style.  I encourage you to check it out here.

How I Do Design

This past week I spent some time working with some guys doing design work.  As we ping-ponged ideas and styles back and forth, I had opportunity to express my quick list of questions I ask when doing design. It turns out someone took notes so I've provided them here.  Maybe you will find them interesting...

Do I have enough information to design what I am trying to design?

1. What kinds of data?
• Hard types or not? Using classes vs. datasets or xml vs. text
• Comprehensive or not? Is the data normalized, are the options well-known and relatively static.
• Calculated or not? Does it require staging, heavy calculations, is volatile?
• What's the lifecycle? Where does it come from, where does it go? What determines the states it has?
2. What kind of process are you designing?
• Displaying information? How rich is the interface?
• Collecting information? What speed is the collection? How controlled is the interaction?
• Business Rule Processing? Where does context come from? What is the rate of change (ROC) for the rules?
• Transforming Data? Are the schemas well-defined? What is the ROC for the logic? 
• A Workflow? Does it interact with user? How complex is the state machine?
• Others – specialties? Hybrids?
3. What patterns already implement similar behavior?
• Ex: ui - > validator - > access server -> data contract
• Framework provides the patterns.
• Consume others as appropriate.
• If you can’t define it using patterns, define it using prose.
• Patterns are succinct, you only have to call out the exceptions.
4. What is unique about the behavior being implemented?
• Start with prose – write it out, follow the Design Considerations template.
• Move to formalizing your design – use patterns/practices to describe processes.
• Break it out and keep the scope of each component small and isolated.
5. What are the operational considerations?
• Security? How much or little trust? How controlled, and managed?
• Retention? Lifecycle of data and the artifacts from processing.
• Performance?
• These data points go into the Requirements Map.
6. What specific services are to be consumed?
• Using particular APIs? Controls? Schemas?
  
• Whatever specifics you know (typically filled in detailed design)
• If you know it, put it in the Design Considerations document

 These were just my off-the-top list, so keep the flames low, eh?

Pragmatic Project Principles?

My latest project has some interesting challenges.  Firstly, like virtually all interesting projects the deadline is fairly determined.  It happens to be far away, which is good, but fixed nonetheless.  Next, the resources to meet the tasks are unfamiliar with the technologies we are using, they aren't formally trained in software engineering, and have existed in a culture of complacency for the last several years. Okay, so having resources that are barely competent and lazy is different but not insurmountable.  After all, the best solutions I've ever known have been created by lazy people to facilitate their laziness (and yes, I am known for being mayor of lazytown).  Lastly, it is run by formally trained, highly experienced consultants who have been thoroughly brainwashed and bamboozled into plan-driven methodologies (think waterfall approach and the like).

The main challenge with this is that plan-driven approaches revolve, by definition, around a detailed plan of tasks and schedule, hence the name.  Which works well in environments consisting of relatively known behaviors, requirements, and abilities.  For example, a team of consultants all similarly trained on a project with ample time to define and document the requirements in sufficient detail.  That would not be this project.

To embrace the uniqueness of the environment the approach followed must be capable of supporting the differing levels of resources and abilities, the ambiguous nature of the requirements, and a general resistance to behavioral change.  Simply put, when you have people whose performance you can't estimate, working on tasks that are not fully defined, you really won't have much of a plan to manage against.  And any plan you initially create will invariable fall apart under the speed of which changes are encountered. See Sun Tzu for details.

There are various techniques for operating under these types of conditions, after all we're not the first nor will we be the last to find ourselves in such tumultuous waters.  In following posts, I'll discuss some of these approaches which I have found to be especially effective.  They specifically impact the Document and Design phases of the 5-D Approach which I've discussed previously.

InternalsVisibleTo

This particular project I've been on is making judicious use of the InternalsVisibleTo assembly attribute.  It's one of the handiest little capabilities I've fallen for in some time.

Coincidently, part of why I like it so much is that my life is a bit like having InternalsVisibleTo for every single other person out there.  In my quest for transparency being able to let others have a peek behind the curtain is must once in while.

Mind you, I'm nowhere near as capable in this department as my good friend Brian who just seems to have people falling all over themselves to socialize with him wherever he goes.  No kidding, the man spoke to her for 3 minutes and she was making doe-eyes at him!  Next day, the security guard wanted to give him a special Private Screening (if you know what I mean).  And lastly, in the cafeteria, some random person actually just came up to him and asked him about the proper use of dental floss!  I swear this is true, I couldn't make this up if I'd tried.  Dental floss!  At what point do you just walk up to someone and ask about how to use dental floss?  He just brings this out in people.  I should probably be inspired, or better yet take notes and ask for lessons.  As it is, I just ride the roller-coaster of being depressed that he is so much cooler than I am, and then feeling superior because of how cool my friends are.

Good thing I'm straight or all this gushing over a friend could be misconstrued. ;-)

Use Cases +1

In doing some research online, I re-read a post by Paul Hartley (Intentional Software) that brought up Use Cases.

His basic premise is that the practice of law specifically relating to the drafting of contracts is very similar to the practice of software engineering in the implementation of code. This seems like one of those things that should be obvious, seeing as how we use the word contract to describe interface relationships often enough. Once you've finished the article however, it's obvious that the challenges extend beyond just component interactions.

Specifically, his illustrations in how Use Cases are a tool to help in both practices were enlightening. I now have yet another tool when explaining the value and approach for Use Cases.

Playing Catch Up

If you can't tell, I'm kind of playing catch-up today on some topics that I have been sorely neglecting.

Joel posted a new entry a few days ago and I'm just now getting around to going through it.  Turns out he must have been reading my journal.  Now most of the time, I think the Joel comes off as a blow-hard, but then he puts up a post like this one and redeems himself complete.

That should spell out exactly how self-centered I am. It is is easy for me to pooh-pooh his exceptional work until it lines up with my own ramblings. What can I say, I'm a jerk sometimes.  In any case, read the flippin' article. It's good.  Even if he pretty much ripped off what I've been ranting about for past 8 years.  ;-)

Immature Diatribes

Once you have spent long enough in this industry you realize that even the most technically adept can succumb to zeolous bigotry. It's common knowledge I take that stance myself on occasion. Fortunately, this particular post is not about me. An author I have respect for Scott Bellware recently posted a quite considerable flame concerning the recently released Visual Studio toolset. More specifically, his core issue (I believe) was with the prescriptive guidance that was released along with the product.

It's not really relevant to my post that I whole-heartedly agree with Mr. Bellware on this particular subject. It's not relevant that his core issue is dead-on, and shares concerted agreement by most of the accomplished in the industry. What is of concern to me, and the reason for this post, is the lack of finesse and precision with which he made his arguments. To say it less than elegantly: he threw the baby out with the bath-water. Thereby reducing the effectiveness of his argument and his credibility as a balanced voice in the industry.

While I share his sentiments over Microsofts obviously ignorant ploy to co-opt the term Test Driven Development (TDD), his slander over the state of the tools and their general worthiness was distressing. What he fails to realize is that for many, the choice to switch completely towards TDD is at best an impossibility. So any tools and support that increase the visibility and perceived value of testing is in itself a Good Thing.

While a trouncing for the hopefully unintentional ignorance and blatant disregard community evidenced by the Microsoft post was certainly in order, it seems excessive to dismiss and deride their entire efforts. I certainly disagree with heavy-handedness in any form, but I harbor that resentment equally towards every voice. Regardless if the voice is from a behemoth the likes of Microsoft, or tech pundit a la Bellware.

Yet Another Architecture Journal

There is Yet Another Architecture Journal (YAAJ) that has been posting some interesting reads in the last year or so. It has something from pretty much every talking-head, blow-hard, or egomaniacal pundit on the architecture scene today. There are actually a few decent architects and writers represented too. For example the article on Service-Oriented, Distributed, High-Performance Computing actually serves as an excellent primer. While there is nothing new or even translationally creative, it is an excellent starting point which sets the terminology and foundational concept bar at an adequate level. There are several other interesting pieces if you surf around a little.

Oh, and Harry, and I'm still working on getting the reading list published.

Most Ridiculous

Reading around the net today, I came across this new article recently posted on MSDN (http://msdn.microsoft.com/asp.net/default.aspx?pull=/library/en-us/dnaspp/html/CustEntCls.asp). As one of the original members of MSDN I keep my eye on how they are doing from time to time. Let me just say that the schlock these guys are peddling these days is terrible.

#soapbox on

This article on Custom Entity Classes is the most ridiculous offering I've seen yet. Talk about promoting technology for technology sake. Custom Entity Classes have all the disadvantages of O/R mappers, none of the benefits of JIT mapping, spreads DA logic around instead of centralizing it, and is just plain WRONG. If you want a DAL that is purely schema reflective, use a dataset or dataview and save some code! You gain nothing by pushing schema further into the user base. There is already a perfectly good abstraction layer at the stored procedure level, why add a second or third level? This doesn't even make good common sense. Adding special operations to the DAL is completely contrary to Service Oriented Architecture (SOA) and almost every reliable design pattern ever.

In reality this is a good introduction to Custom Entity Classes which are just another form of O/R mapping that is worse that JIT mapping and only paves the way for the lazy to slide into the realm of custom business objects that tie data and operations together. Tying data and operations is the #1 most detrimental flaw in practice today. It inhibits interop, hinders maintainability, and increases complexity. Yet here we are, encouraging people to learn how to practice chaotic, undisciplined coding patterns. I'm thoroughly disgusted at this obvious sell-out.

Maybe if the author had read some of the references and actually built and maintained a few real applications he wouldn't be so quick to sell this drivel.
#soapbox off