CodeBetter.Com
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @CodeBetter

Ian Cooper [MVP]

July 2008 - Posts

  • Alt.Net UK September

    We are pleased to announce that the Alt.Net UK Conference will be returning in September, and that there will be room for more attendees this time!

    The plan is host the event at Conway Hall in London and our thinking is to follow the same sort of schedule as we did in February:
    - Evening planning session on Friday 12th September, following by a trip to a bar to socialise.
    - The Open Spaces sessions all day on Saturday 13th September

    This time we are thinking of starting off Saturday with a Park Bench to get the juices flowing.

    We are very open to listening to feedback from the community if you think that there are ways that we can improve on the conference experience.

    We would especially like to have more testers, technical authors and usability folk attend to foster cross-pollination of ideas.

    User registration will start from Friday 11th July at 07:00 UK time so the early birds will get the worm!

    The following social hubs have also been set up: Upcoming, Facebook, FriendFeed and LinkedIn. Don't forget that you can subscribe to the AltNetUk News River.

    Finally, we are currently looking for sponsorship, so if you know of an organisation that would be interested to be associated with the conference in return for a little lucre, we would love to hear from you / them! (The conference is non-profit)

    Conchango and redgate have generously agreed to be launch sponsors - but more is needed, especially as we have to pay for Conway Hall this time.

    Ian Cooper, Ben Hall and Alan Dean

    PS Thanks to David Laribee for his help in getting us set up on alt.net

  • TDD and Hard to Test Areas, Part1

     

    TDD and Hard-To-Test Areas

    I wanted to talk about the issues that people get when they begin working with TDD, the same issues that tend to make them abandon TDD after an initial experiment. Those are the 'hard-to-test' areas, the things production code needs to do, that those presentations and introductory books just don't seem to explain well. In this post we will start with a quick review of TDD, and then get into why people fail when they start trying to use it. Next time around we will look more closely at solutions.

    Review

    Clean Code Now

    TDD is an approach to development in which we write our tests, before writing production code. The benefit of this are:

    • Tests help us improve quality: Tests give us prompt feedback. We receive immediate confirmation that our code behaves as expected. The cheapest point to fix a defect is at the point you create it.
    • Tests help us spend less time in the debugger. When something breaks our tests are often granular enough to show us what has gone wrong, without requiring us to debug. If they don’t then we probably don’t have granular enough or well-authored tests. Debugging eats time, so anything that helps us stay out of the debugger helps us deliver for a lower cost.
    • Tests help us produce clean code: We don’t add speculative functionality, only code for which we have a test.
    • Tests help us deliver good design: Our test proves not just our code, but our design, because the act of writing a test forces us to make decisions about the design of the SUT.
    • Tests help us keep a good design: Our tests allow us to refactor – changing the implementation to remove code smells, while confirming that our code continues to work. This allows us to do incremental re-architecture, keeping the design lean and fit while we add new features.
    • Tests help to document our system: If you want to know how the SUT should behave examples are an effective means of communicating that information. Tests provide those examples.

    Automated tests lower the cost of performing these tests. We pay a cost once, but because we can then re-run our tests at a marginal cost they help us keep those benefits throughout the system lifetime. Automated tests are ‘the gift that keeps on giving’. Software spends more of its life in maintenance than in development, so reducing the cost of maintenance lowers the cost of software.

    The Steps

    The steps in TDD are often described as Red-Green-Refactor

     Red: Write a failing test (there are no tests-for-tests, so this checks your test for you)

    Green: Make it pass

    Refactor: Clear up any smells in the implementation resulting from the code we just added.

    Where to find out more

    Kent Beck’s book Test-Driven Development, By Example remains the classic text for learning the basics of TDD.

    Quick Definitions

    System Under Test (SUT) – Whatever we are testing, this may differ depending on the level of the test. For a unit test this might be a class or method on that class. For acceptance tests this may be a slice of the application.

    Depended Upon Component (DOC) – Something that the SUT depends on, a class or component.

    What do we mean by hard-to-test?

    The Wall

    When we start using TDD we rapidly hit a wall of hard-to-test areas. Perhaps the simple red-green-refactor cycle gets begins to get bogged down when we start working with infrastructure layer code that talks to the Db or an external web service. Perhaps we don’t know hot to drive our UI through a xUnit framework. Or perhaps we had a legacy codebase, and putting even the smallest part under test quickly became a marathon instead of short sprints.

    TDD newbies often find that it all gets a bit sticky, and faced with schedule pressure, drop TDD. Having dropped it they lose faith in its ability to deliver for them and still meet schedule pressure. We are all the same, under pressure we fall back on what we know; hit a few difficulties in TDD and developers stop writing tests.

    The common thread among hard-to-test areas is that they break the rhythm of development from our rapid test and check-in cycle, and are expensive and time-consuming to write. The tests are often fragile, failing erratically and difficult to maintain.

    The Database

    • Slow Tests: Database tests run slowly, up to 50 times more slowly than normal tests. This breaks the cycle of TDD. Developers tend to skip running all the tests because it takes too long.
    • Shared Fixture Bugs: A database is an example of a shared fixture. A shared fixture shares state across multiple tests. The danger here is that Test A and Test B pass in isolation, but running Test A after test B changes the value of that fixture so that the other test fails unexpectedly. These kinds of bugs are expensive to track down and fix. You end up with a binary search pattern to try and resolve shared fixture issues: trying out combinations of tests to see what combinations fail. Because that is so time consuming developers tend to ignore or delete these tests when they fail.
    • Obscure Tests: To avoid shared fixture issues people sometimes try to start with a clean database. In the setup for their test they populate the Db with any values they need, and in the teardown clean them out. These tests become obscure, because the setup and teardown code adds a lot of noise, distracting from what is really under test. This makes tests hard to read as they are less granular, and thereby harder to find the cause of failure in.  The Db setup and teardown code is another point of failure. Remember that the only test we have for out tests themselves is to write a failing test. Once you get too much complexity in your test itself it can become difficult to know if your test is functioning correctly.  It also makes them harder to write. You spend a lot of time writing setup and tear down code which shifts your focus away from the code you are trying to bring under test, breaking the TDD rhythm.
    • Conditional Logic: Database tests also tend to end up with conditional logic – we are not really sure what we are going to get back, so we have to insert a conditional check to see what we got back. Our tests should not contain conditional logic. We should be able to predict the behavior of our tests. Among other issues, we test our tests by making them fail first. Introducing too many paths creates the risk that the errors are in our test not in the SUT.

    The UI

    • Not xUnit strength: xUnit tools are great at driving an API, but are less good at driving a UI. This tends to be because a UI runs in a framework that the test runner would need to emulate, or interact with. Testing a WinForms app needs the message pump, testing a Web Forms app needs the ASP.NET pipeline. Solutions like NUnitAsp have proved less effective at testing UIs than scripting tools like Watir or Selenium, often lacking support for features like JavaScript on pages.
    • Slow Tests: UI tests tend to be slow tests because they are end-to-end, touching the entire stack down to the Db.
    • Fragile Tests: UI tests tend to be fragile, because they often fall foul of attempts to refactor our UI. So changing the order and position of fields on the UI, or the type of control used will often break our tests. This makes UI tests expensive to maintain.

    The Usual Suspects

    We can identify a list of the usual suspects, who cause issues for successful unit testing.

    • Communicating Across a Network
    • Touching the File System
    • Requires the Environment to be configured
    • An out-of-process call (includes talking to Db)
    • UI

    Where to find out more

    XUnit Patterns: Gerard Meszaros' site and book are essential reading if you want to understand the patterns involved in test-driven development

    Working with Legacy Code: Michael Feathers' book is the definitive guide to test-first development in scenarios where you are working with legacy code that has no tests.

    Next time around we will look at how we solve these issues.

     

    Posted Jul 07 2008, 04:10 PM by Ian Cooper with 21 comment(s)
    Filed under: ,
  • Showing some support for LINQ to SQL

    While I have finished my series on LINQ to SQL I wanted to talk about some of the reaction. In his summary post of 30 June Roger Jennings mentions his concerns that because the SQL Server Data Programmability group, who are bringing us Entity Framework v1, now owns LINQ to SQL we will not see the kind of development I asked for in my last post of my Architecting LINQ to SQL series. Indeed Matt Warren's comment in this post, that the provider model for LINQ to SQL was disabled before release is troubling for what it implies about internal politics over data access strategies in Redmond and might confirm concerns I had a long time ago.  Looking at the the Data Platform team's blogs and site, LINQ to SQL seems almost forgotten.

    I would like to see MS give this product the support it deserves. I would like to see a commitment from the Data Platform team to stop its focus on talking LINQ to SQL down as a RAD tool and tallking up its advantages for use in the OO approaches to software development. For example  I would like to see the Data Platform team talking about LINQ to SQL's support for POCO strategies, lazy loading, etc. and pointing out to customers who request those features. There needs to be more acknowledgement that if you want them you should consider LINQ to SQL. Right now their only response is to repeat that those features will be in Version 2 of the EF. Well, an MS ORM supports those features today, you should point that out to your customers, and give advice on how to achieve it. As of now, in my opinion, LINQ to SQL is their best development tool for OO an approach to development and they need to reflect its strengths in the advice they give, not just focus on its weaknesses when compared to EF. 

    Martin Fowler posted some time ago about different schools of software development. Perhaps one solution for the conflict over the future of these tools is for MS to accept that it has (at least) two audiences and build a product for the OO folks and one for the data-first folks. In that case LINQ to SQL would be a better start for the OO folks because it already contains so much of what they need, it may represent a better starting point for supporting them.

    People feel sorry for the Entity Framework team for the criticism in the open letter. For my part I feel sympathy for the LINQ to SQL team, who fell foul of product strategy decisions with ObjectSpaces and seem to have done so again. Considering how well they understood the OO approaches we wanted, unlike the EF team, the team are unsung for their efforts. The Alt.Net community in particular should give them wider support for having, unlike the Entity Framework team, recognized the needs of those developers taking an OO approach to development. it does not give us everything, but credit where it is due.

    Today, while I would not recommend using the Entity Framework I would recommend looking at LINQ to SQL. Everything needs evaluation for your own needs, but unlike EF, LINQ to SQL is a better contender for OO approaches today.

    Sasha points out when looking at how LINQ to SQL is suprising people who had been misinformed as to what if offered and how we should use it, the noise in the blogsphere from Entity Framework supporters seems to have drowned out the value of LINQ to SQL as an ORM. Indeed I believe the Data Team's own pitching of LINQ to SQL as a RAD tool is an underestimates the product.

    As a community, as people begin to realize the suprising power of LINQ to SQL, I would like to see us dispel many of the myths that seem to have grown up around that product. I would like to see us put pressure on the Data Platform team to provide the support for LINQ to SQL that we want going forward. Community reaction is everything and if the LINQ to SQL community remains silent in the face of the more vocal, but probably less numerous, EF community, we won't get the product we deserve.


    Posted Jul 02 2008, 07:57 AM by Ian Cooper with 27 comment(s)
    Filed under:
  • Architecting Linq to SQL, part 10

     Previously: Architecting Linq to SQL, part 9

    End of the line

    This is intended to be the last part in this series and I wanted to take the opportunity to talk about a number of related if diverse topics. I would like to look at what I would like to see in the next version, and talk about when and where I intend to use Linq to SQL and other ORMs such as NHibernate.

    I will try to get some of the code that goes with this series up onto google code over the coming months, schedules permitting. Roger Jennings requested that I give more than a trivial example of how to do messaging for n-tier scenarios. I'm flattered by Roger's confidence, though I feel that Greg or Udi would be better placed to do a introductory piece on messaging. But if there is demand I will give it a try.

    What would I like to see in the next version?

    The following are, I think, the priorities:

    Support for Value Types. In a fine grained object model we may have classes that are not entities (have a distinct identity independent of their state). A common example would be Money, which has a value and currency. We do not want to map these to rows in a table, but to columns. Right now with Linq to SQL we have to represent money as two fields, amount and currency in an entity. We would like to represent them as one type, which can be mapped independently. As an aside a lot of systems overuse primitive types directly. Often we have something that is not a string or an int even if we can represent it as such. Our systems become clearer if we can wrap these primitives in a name appropriate to the domain such as ShippingReference. This only works if we can map value types easily. This is fairly straightforward if we assume that the column names used by these value types remain the same on any entity that stores them.

    Support for changing loading options. As of today we can only alter the default loading behavior of a DataContext before we use it. This assumes that we can determine what we want to eager load once for a context. The reality is that we may want to set this before we run any query. So it must be possible for us to set eager loading options each time we run a query. An alternative would be to do something more akin to Hibernate's HQL language's ability to add a fetch to the query expression so that we can tell the query to load the relationship eagerly.We also want support for eager loading multiple child associations, not just the one we have now.

    Support for ordered relationship types. Right now an association is treated as a set - an unordered collection. However often are children are ordered, particularly being in a map where we have both a key and a value. The key should be both a primitive type and another entity or value type. While this type of mapping is less common in the relational world, within our domain we often want to use ordered mappings, and support for mapping these to relational tables gives us increased flexibility when mapping domain to Db.

    Support for table per sub-class mapping.  Sometimes we do not want to allow fields on sub-classes to be nullable. Unfortunately this is a requirement of table-per-class-hierarchy mapping strategies. Allowing table per sub-class, using a shared key strategy would allow us to avoid this issue. Table per-subclass with shared key avoids some of the performance issues from moving away from a single table, which might be incurred if we took a union approach to combining data from multiple tables to support subclassing.

    There are also some things I would like to see, though I am less optimistic that they will happen

    Expose the provider model. Allow LINQ to SQL to target multiple Dbs. It exists but was never exposed at RTM. I suspect the resources were not allocated because the Entity Framework became the way to work if you had a non-SQL Server back end. Given EF not being positioned as an ORM let's open it up so we can take LINQ to SQL forward.

    Include an explicit in-memory provider. This will make TDD a breeze.  Once we have an in-memory provider it would be easy to swap out the Db for unit testing purposes.

    Support for second-level caching. MS now has a second level caching technology in Velocity. It would be nice to see support for working with a second level cache within LINQ to SQL (the first level cache is the identity map).

    What I would be cautious about in the next version 

    There are also some things that I would be disappointed about a disproportionate amount of much effort being expended on:

    Support for serialized entities. I hope I have managed to explain why serializing an entity across tiers is a bad architectural style. Instead of corrupting LINQ to SQL with support for this practice, I would like to see an emphasis from the patterns on practices team on dissuading people from approaching n-tier design in this style. We do not want to pollute entities with change tracking or serialize a DataContext.

    More advanced designer options. I appreciate that some folks like designers, but I think that they may be a red herring here. If you work domain-first then you might as well use attributes to mark up your domain model, or hand code your xml mappingfile. If you are going to work in a data first approach, I would push extending SQLMetal with those capabilities instead of a designer.

    In the data first case the design is done in the RDBMS, not in the domain model, so by the time we get to LINQ to SQL we are just generating our entity model from our Db. All the designer gives us is the ability to select a sub-set of tables to generate. A fairly simplistic UI, such as a dropdown list to add tables, could configure the options for a SQLMetal call. Flashy drag and drop layout seems a little bit wasteful. Even better if the property based approach is just a wrapper around a SQLMetal call that makes that command you have configured available. That allows folks to use the command they have created throough the designer in their build scripts to call SQLMetal. This would give more resources for the new functionality people want from their data-first designer such as file per entity, update an existing set of files for changes etc.

    I understand this may not be popular, but command line tools are cheaper to author and can deliver a lot more bang for your buck if your team has limited resources. In addition a designer can blind you to an over-complex approach to mapping. If you cannot easily map by hand, if you require that designer, then I believe that you may have lost your way.

    I understand that these suggestions will be unpopular with some people, but both of them represent dead ends to me, that do not provide us with the ability to write better software. Of course your mileage may vary.

    Linq to SQL over Entity Framework for your ORM

    For my part, and that is of course based of my school of software development, LINQ to SQL is a better ORM than the Entity Framework. That may come as no shock to the EF team who have a bigger vision for their product than ORM. For me, LINQ to SQL get a lot right: support for persistence ignorance, single mapping file that is by-hand authorable, lazy loading as a default strategy. If MS intends to provide an offering in the ORM space, as opposed to whatever space the EF is defining, and thus fulfill the vision that Anders gave us of simplifying the development experience by having data access as part of the language. To me LINQ to SQL is the best MS contender for the crown. Given the resources, LINQ to SQL could become a great tool. I hope that MS continue to allocate a fair share of resources to it.

    LINQ to SQL vs. NHibernate

    To be honest, I have to say that my next project will use NHibernate for its persistence technology instead of LINQ to SQL. Why? It is a large project, with a significant number of entities, and we want to support fine-grained object models and table per sub-class mapping strategies. We also wanted the insurance of being able eager fetch on a query-by-query basis and have a 2nd level cache. It is an old adage but 'there is no silver bullet'. I'm picking one tool out of the kit, it does not mean the others are not valuable.

    At the same time LINQ to SQL still forms part of our strategy, because we believe it to be simpler to approach for many projects.So we also have and will be using LINQ to SQL. If anything LINQ to SQL replaces WORM for us which we used for a number of projects where we had good table to entity affinity. Ironically perhaps WORM was an implementation of the proposed interface for ObjectSpaces. ObjectSpaces was the MS ORM for .NET 2.0, that never saw the light of day. ObjectSpaces became LINQ to SQL, and Matt has full the story here, so it seems a natural inheritor. Let us hope it does not meet the ObjectSpaces fate of being sidelined for a more grandiose vision of data access.

    A valid question might be to ask why I want improve LINQ to SQL, why I do not just tell everyone to use NHibernate. Some of this is a recognition of the market, many people will not use a non-MS ORM and LINQ to SQL is is a solid ORM. Pragmatically we are likely to get more .NET developers who know LINQ to SQL available in the market place than NHibernate developers. But I also believe that with the expressiveness of LINQ MS have a real chance to move the ORM market forward in the .NET space. LINQ to SQL is like ASP.NET MVC, it is a welcome acknowledgement from MS of what developers want, and we should commend them when they do get it right.

    I will be posting a series on NHibernate going forward, so that you can make your own judgements on which to use and when.

     

  • Alt.Net events in London

    Danny over at the Alt.Net yahoo group asked about alt.net events in London. I thought I would publicize more widely for folks that do not know what is on offer.

    I run the London .NET user group. We have covered alt.net topics for the last 5 or so years, and have always had strong alt.net community involvement. For example we ran the XP game for everyone back in 2005. You can find our, much in need of work when I get some time, site here.

    I am also involved in running the altnetconf in London with Ben and Alan with lots of help from Michelle at Conchango. We held an open spaces conference last January and are in the planning stages for a new one in September. We do have a google group that you can find here.  I don't spend as much time there as I should

    Sebastian Lambla is running an alt.net beers get together. I think he is looking to add more technical content to his existing social scene. If you are looking for a more regular and explicit alt.net get together that is worth trying (sorry Seb will try to be in the country for the next one). Seb usually advertises meetings on the above list and you can also pick it up via our newsriver. Thanks to Dave Verwer for that (Dave is a Ruby guys but I believe he is also working on an ASP.NET MVC book with Jeffrey Palermo).

    Skills Matter hosts some 'Open Source .NET Evenings'. London .Net User group will be hosted there a few times this year. You can find their listing on their website.

    So we are pretty well-covered, in fact there are too mauy events to attend all of them.

    In addition there is a wider agile community to attend such as the Extreme Tuesday Club.

    To be honest I am strongly opposed to a user group self-identifying as alt.net as the point is to reach out to the wider audience. I would say that a lot of UK groups tend to run alt.net topics including the NxtGenUG and VBUG guys. I know that because I have spoken at them. If you are interested in alt.net and want to see more events, I would encourage you to offer to speak, for that is the limitation a lot of groups have on alt.net topics - speakers. In feel sure all would welcome you and not turn you away for presenting the alt.net viewpoint. Given how much is already available I would definitely question anyone setting up an explicit alt.net group in the the London area. Presumably they would be focusing on alternatives to things happening at existing groups and be running sessions like 'Waterfall: Return to Righteousness' and 'Procedural Programming: Putting Data Stores Back On Top".



    Posted Jul 01 2008, 12:52 PM by Ian Cooper with 3 comment(s)
    Filed under:
More Posts

Our Sponsors

Free Tech Publications